All of lore.kernel.org
 help / color / mirror / Atom feed
* [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling
@ 2016-10-24  4:59 David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 01/12] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
                   ` (12 more replies)
  0 siblings, 13 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

This series contains a number of cleanups to the libqos code for
accessing PCI devices, and to tests which use it.

The general aim is to improve the consistency of semantics across
functions, and reduce the amount of intimate knowledge of the libqos
PCI layer needed by tests.

This should make it easier to write PCI tests which will be portable
to different guest machines with different PCI host bridge
arrangements.

This series is on top of my ppc-for-2.8 branch, since it contains
patches enabling the virtio tests on ppc, which would otherwise
conflict with the changes here.

Changes since v4:
  * Fixed some remaining abstraction breaks in ahci-test
  * Removed QPCI_BAR_INVALID, turned out not to really be useful

Changes since v3:
  * Fixed another endian bug introduced in ide-test

Changes since v2:
  * Fixed build bugs in virtio-9p-test, which I didn't find earlier
    due to not having the right libraries installed
  * Fixed an endian bug I accidentally introduced in ide-test
  * Better handling of invalid BAR tokens

Changes since v1:
  * Split out updates to tco-test into separate patch
  * Split out updates to ide-test into separate patch
  * Neater and more general handling of legacy PIO addresses
  * Removed now-redundant fields from platform specific bus structures
  * Introduced CONFIG_BASE() macro to virtio-pci to remove many
    similar assignments
  * Fixed handling of two guest testcasesin ivshmem
  * Added 64-bit accessors
  * Rebase on ppc-for-2.8 to avoid conflict with Laurent's series in
    the same area

David Gibson (12):
  libqos: Give qvirtio_config_read*() consistent semantics
  libqos: Handle PCI IO de-multiplexing in common code
  libqos: Move BAR assignment to common code
  libqos: Better handling of PCI legacy IO
  tests: Adjust tco-test to use qpci_legacy_iomap()
  libqos: Add streaming accessors for PCI MMIO
  libqos: Implement mmio accessors in terms of mem{read,write}
  tests: Clean up IO handling in ide-test
  libqos: Add 64-bit PCI IO accessors
  tests: Use qpci_mem{read,write} in ivshmem-test
  tests: Don't assume structure of PCI IO base in ahci-test
  libqos: Change PCI accessors to take opaque BAR handle

 tests/ahci-test.c          |  13 +--
 tests/e1000e-test.c        |   7 +-
 tests/ide-test.c           | 177 +++++++++++++++++++++++++++--------------
 tests/ivshmem-test.c       |  47 +++++++----
 tests/libqos/ahci.c        |   4 +-
 tests/libqos/ahci.h        |   7 +-
 tests/libqos/pci-pc.c      | 187 ++++++++++---------------------------------
 tests/libqos/pci-spapr.c   | 194 ++++++++++++---------------------------------
 tests/libqos/pci.c         | 194 +++++++++++++++++++++++++++++++++++++--------
 tests/libqos/pci.h         |  66 ++++++++++-----
 tests/libqos/usb.c         |   6 +-
 tests/libqos/usb.h         |   2 +-
 tests/libqos/virtio-mmio.c |  16 ++--
 tests/libqos/virtio-pci.c  | 122 ++++++++++++++--------------
 tests/libqos/virtio-pci.h  |   2 +-
 tests/rtl8139-test.c       |  10 +--
 tests/tco-test.c           |  80 +++++++++----------
 tests/usb-hcd-ehci-test.c  |   5 +-
 tests/virtio-9p-test.c     |   8 +-
 tests/virtio-blk-test.c    |  42 +++-------
 tests/virtio-scsi-test.c   |   4 +-
 21 files changed, 598 insertions(+), 595 deletions(-)

-- 
2.7.4

^ permalink raw reply	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 01/12] libqos: Give qvirtio_config_read*() consistent semantics
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 02/12] libqos: Handle PCI IO de-multiplexing in common code David Gibson
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The 'addr' parameter to qvirtio_config_read*() doesn't have a consistent
meaning: when using the virtio-pci versions, it's a full PCI space address,
but for virtio-mmio, it's an offset from the device's base mmio address.

This means that the callers need to do different things to calculate the
addresses in the two cases, which rather defeats the purpose of function
pointer backends.

All the current users of these functions are using them to retrieve
variables from the device specific portion of the virtio config space.
So, this patch alters the semantics to always be an offset into that
device specific config area.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/virtio-mmio.c | 16 ++++++++--------
 tests/libqos/virtio-pci.c  | 25 ++++++++++++++-----------
 tests/virtio-9p-test.c     |  8 ++------
 tests/virtio-blk-test.c    | 42 +++++++++++-------------------------------
 tests/virtio-scsi-test.c   |  4 +---
 5 files changed, 36 insertions(+), 59 deletions(-)

diff --git a/tests/libqos/virtio-mmio.c b/tests/libqos/virtio-mmio.c
index bced680..7aa8383 100644
--- a/tests/libqos/virtio-mmio.c
+++ b/tests/libqos/virtio-mmio.c
@@ -15,28 +15,28 @@
 #include "libqos/malloc-generic.h"
 #include "standard-headers/linux/virtio_ring.h"
 
-static uint8_t qvirtio_mmio_config_readb(QVirtioDevice *d, uint64_t addr)
+static uint8_t qvirtio_mmio_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readb(dev->addr + addr);
+    return readb(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint16_t qvirtio_mmio_config_readw(QVirtioDevice *d, uint64_t addr)
+static uint16_t qvirtio_mmio_config_readw(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readw(dev->addr + addr);
+    return readw(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint32_t qvirtio_mmio_config_readl(QVirtioDevice *d, uint64_t addr)
+static uint32_t qvirtio_mmio_config_readl(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readl(dev->addr + addr);
+    return readl(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
-static uint64_t qvirtio_mmio_config_readq(QVirtioDevice *d, uint64_t addr)
+static uint64_t qvirtio_mmio_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioMMIODevice *dev = (QVirtioMMIODevice *)d;
-    return readq(dev->addr + addr);
+    return readq(dev->addr + QVIRTIO_MMIO_DEVICE_SPECIFIC + off);
 }
 
 static uint32_t qvirtio_mmio_get_features(QVirtioDevice *d)
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index 7e60b3a..fa82132 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -62,10 +62,13 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
     *vpcidev = (QVirtioPCIDevice *)d;
 }
 
-static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t addr)
+#define CONFIG_BASE(dev) \
+    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
+
+static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, (void *)(uintptr_t)addr);
+    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
 }
 
 /* PCI is always read in little-endian order
@@ -76,31 +79,31 @@ static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t addr)
  * case will be managed inside qvirtio_is_big_endian()
  */
 
-static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t addr)
+static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint16_t value;
 
-    value = qpci_io_readw(dev->pdev, (void *)(uintptr_t)addr);
+    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap16(value);
     }
     return value;
 }
 
-static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t addr)
+static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint32_t value;
 
-    value = qpci_io_readl(dev->pdev, (void *)(uintptr_t)addr);
+    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap32(value);
     }
     return value;
 }
 
-static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t addr)
+static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     int i;
@@ -108,13 +111,13 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t addr)
 
     if (qvirtio_is_big_endian(d)) {
         for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev,
-                                (void *)(uintptr_t)addr + i) << (7 - i) * 8;
+            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
+                                           + off + i) << (7 - i) * 8;
         }
     } else {
         for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev,
-                                (void *)(uintptr_t)addr + i) << i * 8;
+            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
+                                           + off + i) << i * 8;
         }
     }
 
diff --git a/tests/virtio-9p-test.c b/tests/virtio-9p-test.c
index 693920a..9c4f6cb 100644
--- a/tests/virtio-9p-test.c
+++ b/tests/virtio-9p-test.c
@@ -95,7 +95,6 @@ static void qvirtio_9p_pci_free(QVirtIO9P *v9p)
 static void pci_basic_config(void)
 {
     QVirtIO9P *v9p;
-    void *addr;
     size_t tag_len;
     char *tag;
     int i;
@@ -104,15 +103,12 @@ static void pci_basic_config(void)
     qs = qvirtio_9p_start();
     v9p = qvirtio_9p_pci_init(qs);
 
-    addr = ((QVirtioPCIDevice *) v9p->dev)->addr + VIRTIO_PCI_CONFIG_OFF(false);
-    tag_len = qvirtio_config_readw(v9p->dev,
-                                   (uint64_t)(uintptr_t)addr);
+    tag_len = qvirtio_config_readw(v9p->dev, 0);
     g_assert_cmpint(tag_len, ==, strlen(mount_tag));
-    addr += sizeof(uint16_t);
 
     tag = g_malloc(tag_len);
     for (i = 0; i < tag_len; i++) {
-        tag[i] = qvirtio_config_readb(v9p->dev, (uint64_t)(uintptr_t)addr + i);
+        tag[i] = qvirtio_config_readb(v9p->dev, i + 2);
     }
     g_assert_cmpmem(tag, tag_len, mount_tag, tag_len);
     g_free(tag);
diff --git a/tests/virtio-blk-test.c b/tests/virtio-blk-test.c
index f737c40..0e32e41 100644
--- a/tests/virtio-blk-test.c
+++ b/tests/virtio-blk-test.c
@@ -155,7 +155,7 @@ static uint64_t virtio_blk_request(QGuestAllocator *alloc, QVirtioDevice *d,
 }
 
 static void test_basic(QVirtioDevice *dev, QGuestAllocator *alloc,
-                       QVirtQueue *vq, uint64_t device_specific)
+                       QVirtQueue *vq)
 {
     QVirtioBlkReq req;
     uint64_t req_addr;
@@ -165,7 +165,7 @@ static void test_basic(QVirtioDevice *dev, QGuestAllocator *alloc,
     uint8_t status;
     char *data;
 
-    capacity = qvirtio_config_readq(dev, device_specific);
+    capacity = qvirtio_config_readq(dev, 0);
 
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
@@ -285,17 +285,13 @@ static void pci_basic(void)
     QVirtioPCIDevice *dev;
     QOSState *qs;
     QVirtQueuePCI *vqpci;
-    void *addr;
 
     qs = pci_test_start();
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
     vqpci = (QVirtQueuePCI *)qvirtqueue_setup(&dev->vdev, qs->alloc, 0);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    test_basic(&dev->vdev, qs->alloc, &vqpci->vq, (uint64_t)(uintptr_t)addr);
+    test_basic(&dev->vdev, qs->alloc, &vqpci->vq);
 
     /* End test */
     qvirtqueue_cleanup(dev->vdev.bus, &vqpci->vq, qs->alloc);
@@ -311,7 +307,6 @@ static void pci_indirect(void)
     QOSState *qs;
     QVirtioBlkReq req;
     QVRingIndirectDesc *indirect;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -323,10 +318,7 @@ static void pci_indirect(void)
 
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -406,17 +398,13 @@ static void pci_config(void)
     QVirtioPCIDevice *dev;
     QOSState *qs;
     int n_size = TEST_IMAGE_SIZE / 2;
-    void *addr;
     uint64_t capacity;
 
     qs = pci_test_start();
 
     dev = virtio_blk_pci_init(qs->pcibus, PCI_SLOT);
 
-    /* MSI-X is not enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     qvirtio_set_driver_ok(&dev->vdev);
@@ -425,7 +413,7 @@ static void pci_config(void)
                                                     " 'size': %d } }", n_size);
     qvirtio_wait_config_isr(&dev->vdev, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     qvirtio_pci_device_disable(dev);
@@ -441,7 +429,6 @@ static void pci_msix(void)
     QVirtQueuePCI *vqpci;
     QVirtioBlkReq req;
     int n_size = TEST_IMAGE_SIZE / 2;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -456,10 +443,7 @@ static void pci_msix(void)
 
     qvirtio_pci_set_msix_configuration_vector(dev, qs->alloc, 0);
 
-    /* MSI-X is enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(true);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -479,7 +463,7 @@ static void pci_msix(void)
 
     qvirtio_wait_config_isr(&dev->vdev, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, (uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     /* Write request */
@@ -550,7 +534,6 @@ static void pci_idx(void)
     QOSState *qs;
     QVirtQueuePCI *vqpci;
     QVirtioBlkReq req;
-    void *addr;
     uint64_t req_addr;
     uint64_t capacity;
     uint32_t features;
@@ -565,10 +548,7 @@ static void pci_idx(void)
 
     qvirtio_pci_set_msix_configuration_vector(dev, qs->alloc, 0);
 
-    /* MSI-X is enabled */
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(true);
-
-    capacity = qvirtio_config_readq(&dev->vdev, (uint64_t)(uintptr_t)addr);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, TEST_IMAGE_SIZE / 512);
 
     features = qvirtio_get_features(&dev->vdev);
@@ -709,14 +689,14 @@ static void mmio_basic(void)
     alloc = generic_alloc_init(MMIO_RAM_ADDR, MMIO_RAM_SIZE, MMIO_PAGE_SIZE);
     vq = qvirtqueue_setup(&dev->vdev, alloc, 0);
 
-    test_basic(&dev->vdev, alloc, vq, QVIRTIO_MMIO_DEVICE_SPECIFIC);
+    test_basic(&dev->vdev, alloc, vq);
 
     qmp("{ 'execute': 'block_resize', 'arguments': { 'device': 'drive0', "
                                                     " 'size': %d } }", n_size);
 
     qvirtio_wait_queue_isr(&dev->vdev, vq, QVIRTIO_BLK_TIMEOUT_US);
 
-    capacity = qvirtio_config_readq(&dev->vdev, QVIRTIO_MMIO_DEVICE_SPECIFIC);
+    capacity = qvirtio_config_readq(&dev->vdev, 0);
     g_assert_cmpint(capacity, ==, n_size / 512);
 
     /* End test */
diff --git a/tests/virtio-scsi-test.c b/tests/virtio-scsi-test.c
index 60dc9ab..69220ef 100644
--- a/tests/virtio-scsi-test.c
+++ b/tests/virtio-scsi-test.c
@@ -143,7 +143,6 @@ static QVirtIOSCSI *qvirtio_scsi_pci_init(int slot)
     QVirtIOSCSI *vs;
     QVirtioPCIDevice *dev;
     struct virtio_scsi_cmd_resp resp;
-    void *addr;
     int i;
 
     vs = g_new0(QVirtIOSCSI, 1);
@@ -161,8 +160,7 @@ static QVirtIOSCSI *qvirtio_scsi_pci_init(int slot)
     qvirtio_set_acknowledge(vs->dev);
     qvirtio_set_driver(vs->dev);
 
-    addr = dev->addr + VIRTIO_PCI_CONFIG_OFF(false);
-    vs->num_queues = qvirtio_config_readl(vs->dev, (uint64_t)(uintptr_t)addr);
+    vs->num_queues = qvirtio_config_readl(vs->dev, 0);
 
     g_assert_cmpint(vs->num_queues, <, MAX_NUM_QUEUES);
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 02/12] libqos: Handle PCI IO de-multiplexing in common code
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 01/12] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 03/12] libqos: Move BAR assignment to " David Gibson
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The PCI IO space (aka PIO, aka legacy IO) and PCI memory space (aka MMIO)
are distinct address spaces by the PCI spec (although parts of one might be
aliased to parts of the other in some cases).

However, qpci_io_read*() and qpci_io_write*() can perform accesses to
either space depending on parameter.  That's convenient for test case
drivers, since there are a fair few devices which can be controlled via
either a PIO or MMIO BAR but with an otherwise identical driver.

This is implemented by having addresses below 64kiB treated as PIO, and
those above treated as MMIO.  This works because low addresses in memory
space are generally reserved for DMA rather than MMIO.

At the moment, this demultiplexing must be handled by each PCI backend
(pc and spapr, so far).  There's no real reason for this - the current
encoding is likely to work for all platforms, and even if it doesn't we
can still use a more complex common encoding since the value returned from
iomap are semi-opaque.

This patch moves the demultiplexing into the common part of the libqos PCI
code, with the backends having simpler, separate accessors for PIO and
MMIO space.  This also means we have a way of explicitly accessing either
space if it's necessary for some special case.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 107 ++++++++++++++++++++----------------------
 tests/libqos/pci-spapr.c | 118 +++++++++++++++++++++++++----------------------
 tests/libqos/pci.c       |  49 +++++++++++++++++---
 tests/libqos/pci.h       |  22 ++++++---
 4 files changed, 170 insertions(+), 126 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 9600ed6..51dff8a 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -36,79 +36,64 @@ typedef struct QPCIBusPC
     uint16_t pci_iohole_alloc;
 } QPCIBusPC;
 
-static uint8_t qpci_pc_io_readb(QPCIBus *bus, void *addr)
+static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint8_t value;
-
-    if (port < 0x10000) {
-        value = inb(port);
-    } else {
-        value = readb(port);
-    }
-
-    return value;
+    return inb(addr);
 }
 
-static uint16_t qpci_pc_io_readw(QPCIBus *bus, void *addr)
+static uint8_t qpci_pc_mmio_readb(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint16_t value;
-
-    if (port < 0x10000) {
-        value = inw(port);
-    } else {
-        value = readw(port);
-    }
+    return readb(addr);
+}
 
-    return value;
+static void qpci_pc_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
+{
+    outb(addr, val);
 }
 
-static uint32_t qpci_pc_io_readl(QPCIBus *bus, void *addr)
+static void qpci_pc_mmio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
-    uint32_t value;
+    writeb(addr, val);
+}
 
-    if (port < 0x10000) {
-        value = inl(port);
-    } else {
-        value = readl(port);
-    }
+static uint16_t qpci_pc_pio_readw(QPCIBus *bus, uint32_t addr)
+{
+    return inw(addr);
+}
 
-    return value;
+static uint16_t qpci_pc_mmio_readw(QPCIBus *bus, uint32_t addr)
+{
+    return readw(addr);
 }
 
-static void qpci_pc_io_writeb(QPCIBus *bus, void *addr, uint8_t value)
+static void qpci_pc_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
+    outw(addr, val);
+}
 
-    if (port < 0x10000) {
-        outb(port, value);
-    } else {
-        writeb(port, value);
-    }
+static void qpci_pc_mmio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    writew(addr, val);
 }
 
-static void qpci_pc_io_writew(QPCIBus *bus, void *addr, uint16_t value)
+static uint32_t qpci_pc_pio_readl(QPCIBus *bus, uint32_t addr)
 {
-    uintptr_t port = (uintptr_t)addr;
+    return inl(addr);
+}
 
-    if (port < 0x10000) {
-        outw(port, value);
-    } else {
-        writew(port, value);
-    }
+static uint32_t qpci_pc_mmio_readl(QPCIBus *bus, uint32_t addr)
+{
+    return readl(addr);
 }
 
-static void qpci_pc_io_writel(QPCIBus *bus, void *addr, uint32_t value)
+static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
-    uintptr_t port = (uintptr_t)addr;
+    outl(addr, val);
+}
 
-    if (port < 0x10000) {
-        outl(port, value);
-    } else {
-        writel(port, value);
-    }
+static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    writel(addr, val);
 }
 
 static uint8_t qpci_pc_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
@@ -218,13 +203,21 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
 
     ret = g_malloc(sizeof(*ret));
 
-    ret->bus.io_readb = qpci_pc_io_readb;
-    ret->bus.io_readw = qpci_pc_io_readw;
-    ret->bus.io_readl = qpci_pc_io_readl;
+    ret->bus.pio_readb = qpci_pc_pio_readb;
+    ret->bus.pio_readw = qpci_pc_pio_readw;
+    ret->bus.pio_readl = qpci_pc_pio_readl;
+
+    ret->bus.pio_writeb = qpci_pc_pio_writeb;
+    ret->bus.pio_writew = qpci_pc_pio_writew;
+    ret->bus.pio_writel = qpci_pc_pio_writel;
+
+    ret->bus.mmio_readb = qpci_pc_mmio_readb;
+    ret->bus.mmio_readw = qpci_pc_mmio_readw;
+    ret->bus.mmio_readl = qpci_pc_mmio_readl;
 
-    ret->bus.io_writeb = qpci_pc_io_writeb;
-    ret->bus.io_writew = qpci_pc_io_writew;
-    ret->bus.io_writel = qpci_pc_io_writel;
+    ret->bus.mmio_writeb = qpci_pc_mmio_writeb;
+    ret->bus.mmio_writew = qpci_pc_mmio_writew;
+    ret->bus.mmio_writel = qpci_pc_mmio_writel;
 
     ret->bus.config_readb = qpci_pc_config_readb;
     ret->bus.config_readw = qpci_pc_config_readw;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 2eaaf91..2d26a94 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -50,78 +50,76 @@ typedef struct QPCIBusSPAPR {
  * so PCI accessors need to swap data endianness
  */
 
-static uint8_t qpci_spapr_io_readb(QPCIBus *bus, void *addr)
+static uint8_t qpci_spapr_pio_readb(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint8_t v;
-    if (port < s->pio.size) {
-        v = readb(s->pio_cpu_base + port);
-    } else {
-        v = readb(s->mmio32_cpu_base + port);
-    }
-    return v;
+    return readb(s->pio_cpu_base + addr);
 }
 
-static uint16_t qpci_spapr_io_readw(QPCIBus *bus, void *addr)
+static uint8_t qpci_spapr_mmio32_readb(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint16_t v;
-    if (port < s->pio.size) {
-        v = readw(s->pio_cpu_base + port);
-    } else {
-        v = readw(s->mmio32_cpu_base + port);
-    }
-    return bswap16(v);
+    return readb(s->mmio32_cpu_base + addr);
 }
 
-static uint32_t qpci_spapr_io_readl(QPCIBus *bus, void *addr)
+static void qpci_spapr_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    uint32_t v;
-    if (port < s->pio.size) {
-        v = readl(s->pio_cpu_base + port);
-    } else {
-        v = readl(s->mmio32_cpu_base + port);
-    }
-    return bswap32(v);
+    writeb(s->pio_cpu_base + addr, val);
 }
 
-static void qpci_spapr_io_writeb(QPCIBus *bus, void *addr, uint8_t value)
+static void qpci_spapr_mmio32_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    if (port < s->pio.size) {
-        writeb(s->pio_cpu_base + port, value);
-    } else {
-        writeb(s->mmio32_cpu_base + port, value);
-    }
+    writeb(s->mmio32_cpu_base + addr, val);
 }
 
-static void qpci_spapr_io_writew(QPCIBus *bus, void *addr, uint16_t value)
+static uint16_t qpci_spapr_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    value = bswap16(value);
-    if (port < s->pio.size) {
-        writew(s->pio_cpu_base + port, value);
-    } else {
-        writew(s->mmio32_cpu_base + port, value);
-    }
+    return bswap16(readw(s->pio_cpu_base + addr));
 }
 
-static void qpci_spapr_io_writel(QPCIBus *bus, void *addr, uint32_t value)
+static uint16_t qpci_spapr_mmio32_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    uint64_t port = (uintptr_t)addr;
-    value = bswap32(value);
-    if (port < s->pio.size) {
-        writel(s->pio_cpu_base + port, value);
-    } else {
-        writel(s->mmio32_cpu_base + port, value);
-    }
+    return bswap16(readw(s->mmio32_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writew(s->pio_cpu_base + addr, bswap16(val));
+}
+
+static void qpci_spapr_mmio32_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writew(s->mmio32_cpu_base + addr, bswap16(val));
+}
+
+static uint32_t qpci_spapr_pio_readl(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap32(readl(s->pio_cpu_base + addr));
+}
+
+static uint32_t qpci_spapr_mmio32_readl(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap32(readl(s->mmio32_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writel(s->pio_cpu_base + addr, bswap32(val));
+}
+
+static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writel(s->mmio32_cpu_base + addr, bswap32(val));
 }
 
 static uint8_t qpci_spapr_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
@@ -248,13 +246,21 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
 
     ret->alloc = alloc;
 
-    ret->bus.io_readb = qpci_spapr_io_readb;
-    ret->bus.io_readw = qpci_spapr_io_readw;
-    ret->bus.io_readl = qpci_spapr_io_readl;
+    ret->bus.pio_readb = qpci_spapr_pio_readb;
+    ret->bus.pio_readw = qpci_spapr_pio_readw;
+    ret->bus.pio_readl = qpci_spapr_pio_readl;
+
+    ret->bus.pio_writeb = qpci_spapr_pio_writeb;
+    ret->bus.pio_writew = qpci_spapr_pio_writew;
+    ret->bus.pio_writel = qpci_spapr_pio_writel;
+
+    ret->bus.mmio_readb = qpci_spapr_mmio32_readb;
+    ret->bus.mmio_readw = qpci_spapr_mmio32_readw;
+    ret->bus.mmio_readl = qpci_spapr_mmio32_readl;
 
-    ret->bus.io_writeb = qpci_spapr_io_writeb;
-    ret->bus.io_writew = qpci_spapr_io_writew;
-    ret->bus.io_writel = qpci_spapr_io_writel;
+    ret->bus.mmio_writeb = qpci_spapr_mmio32_writeb;
+    ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
+    ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
 
     ret->bus.config_readb = qpci_spapr_config_readb;
     ret->bus.config_readw = qpci_spapr_config_readw;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index c3f3382..55b01df 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -224,33 +224,68 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
 
 uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readb(dev->bus, data);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readb(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readb(dev->bus, addr);
+    }
 }
 
 uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readw(dev->bus, data);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readw(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readw(dev->bus, addr);
+    }
 }
 
 uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
 {
-    return dev->bus->io_readl(dev->bus, data);
-}
+    uintptr_t addr = (uintptr_t)data;
 
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readl(dev->bus, addr);
+    } else {
+        return dev->bus->mmio_readl(dev->bus, addr);
+    }
+}
 
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
 {
-    dev->bus->io_writeb(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeb(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writeb(dev->bus, addr, value);
+    }
 }
 
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
 {
-    dev->bus->io_writew(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writew(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writew(dev->bus, addr, value);
+    }
 }
 
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
 {
-    dev->bus->io_writel(dev->bus, data, value);
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writel(dev->bus, addr, value);
+    } else {
+        dev->bus->mmio_writel(dev->bus, addr, value);
+    }
 }
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index c06add8..72a2245 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -15,6 +15,8 @@
 
 #include "libqtest.h"
 
+#define QPCI_PIO_LIMIT    0x10000
+
 #define QPCI_DEVFN(dev, fn) (((dev) << 3) | (fn))
 
 typedef struct QPCIDevice QPCIDevice;
@@ -22,13 +24,21 @@ typedef struct QPCIBus QPCIBus;
 
 struct QPCIBus
 {
-    uint8_t (*io_readb)(QPCIBus *bus, void *addr);
-    uint16_t (*io_readw)(QPCIBus *bus, void *addr);
-    uint32_t (*io_readl)(QPCIBus *bus, void *addr);
+    uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
+    uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
+    uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
+
+    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
+    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
+    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
+
+    void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
+    void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
+    void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
-    void (*io_writeb)(QPCIBus *bus, void *addr, uint8_t value);
-    void (*io_writew)(QPCIBus *bus, void *addr, uint16_t value);
-    void (*io_writel)(QPCIBus *bus, void *addr, uint32_t value);
+    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
+    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
+    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
     uint8_t (*config_readb)(QPCIBus *bus, int devfn, uint8_t offset);
     uint16_t (*config_readw)(QPCIBus *bus, int devfn, uint8_t offset);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 03/12] libqos: Move BAR assignment to common code
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 01/12] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 02/12] libqos: Handle PCI IO de-multiplexing in common code David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO David Gibson
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The PCI backends in libqos each supply an iomap() and iounmap() function
which is used to set up a specified PCI BAR.  But PCI BAR allocation takes
place entirely within PCI space, so doesn't really need per-backend
versions.  For example, Linux includes generic BAR allocation code used on
platforms where that isn't done by firmware.

This patch merges the BAR allocation from the two existing backends into a
single simplified copy.  The back ends just need to set up some parameters
describing the window of PCI IO and PCI memory addresses which are
available for allocation.  Like both the existing versions the new one uses
a simple bump allocator.

Note that (again like the existing versions) this doesn't really handle
64-bit memory BARs properly.  It is actually used for such a BAR by the
ivshmem test, and apparently the 32-bit MMIO BAR logic is close enough to
work, as long as the BAR isn't too big.  Fixing that to properly handle
64-bit BAR allocation is a problem for another time.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 87 ++--------------------------------------------
 tests/libqos/pci-spapr.c | 89 ++----------------------------------------------
 tests/libqos/pci.c       | 56 ++++++++++++++++++++++++++++--
 tests/libqos/pci.h       |  7 ++--
 4 files changed, 63 insertions(+), 176 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 51dff8a..d0eb84a 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -17,7 +17,6 @@
 #include "hw/pci/pci_regs.h"
 
 #include "qemu-common.h"
-#include "qemu/host-utils.h"
 
 
 #define ACPI_PCIHP_ADDR         0xae00
@@ -26,14 +25,6 @@
 typedef struct QPCIBusPC
 {
     QPCIBus bus;
-
-    uint32_t pci_hole_start;
-    uint32_t pci_hole_size;
-    uint32_t pci_hole_alloc;
-
-    uint16_t pci_iohole_start;
-    uint16_t pci_iohole_size;
-    uint16_t pci_iohole_alloc;
 } QPCIBusPC;
 
 static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
@@ -132,71 +123,6 @@ static void qpci_pc_config_writel(QPCIBus *bus, int devfn, uint8_t offset, uint3
     outl(0xcfc, value);
 }
 
-static void *qpci_pc_iomap(QPCIBus *bus, QPCIDevice *dev, int barno, uint64_t *sizeptr)
-{
-    QPCIBusPC *s = container_of(bus, QPCIBusPC, bus);
-    static const int bar_reg_map[] = {
-        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
-        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
-    };
-    int bar_reg;
-    uint32_t addr;
-    uint64_t size;
-    uint32_t io_type;
-
-    g_assert(barno >= 0 && barno <= 5);
-    bar_reg = bar_reg_map[barno];
-
-    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
-    addr = qpci_config_readl(dev, bar_reg);
-
-    io_type = addr & PCI_BASE_ADDRESS_SPACE;
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        addr &= PCI_BASE_ADDRESS_IO_MASK;
-    } else {
-        addr &= PCI_BASE_ADDRESS_MEM_MASK;
-    }
-
-    size = (1ULL << ctzl(addr));
-    if (size == 0) {
-        return NULL;
-    }
-    if (sizeptr) {
-        *sizeptr = size;
-    }
-
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        uint16_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_iohole_alloc, size) + size
-                 <= s->pci_iohole_size);
-        s->pci_iohole_alloc = QEMU_ALIGN_UP(s->pci_iohole_alloc, size);
-        loc = s->pci_iohole_start + s->pci_iohole_alloc;
-        s->pci_iohole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
-
-        return (void *)(intptr_t)loc;
-    } else {
-        uint64_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_hole_alloc, size) + size
-                 <= s->pci_hole_size);
-        s->pci_hole_alloc = QEMU_ALIGN_UP(s->pci_hole_alloc, size);
-        loc = s->pci_hole_start + s->pci_hole_alloc;
-        s->pci_hole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc);
-
-        return (void *)(intptr_t)loc;
-    }
-}
-
-static void qpci_pc_iounmap(QPCIBus *bus, void *data)
-{
-    /* FIXME */
-}
-
 QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
 {
     QPCIBusPC *ret;
@@ -227,16 +153,9 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.config_writew = qpci_pc_config_writew;
     ret->bus.config_writel = qpci_pc_config_writel;
 
-    ret->bus.iomap = qpci_pc_iomap;
-    ret->bus.iounmap = qpci_pc_iounmap;
-
-    ret->pci_hole_start = 0xE0000000;
-    ret->pci_hole_size = 0x20000000;
-    ret->pci_hole_alloc = 0;
-
-    ret->pci_iohole_start = 0xc000;
-    ret->pci_iohole_size = 0x4000;
-    ret->pci_iohole_alloc = 0;
+    ret->bus.pio_alloc_ptr = 0xc000;
+    ret->bus.mmio_alloc_ptr = 0xE0000000;
+    ret->bus.mmio_limit = 0x100000000ULL;
 
     return &ret->bus;
 }
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 2d26a94..70a24b5 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -34,14 +34,6 @@ typedef struct QPCIBusSPAPR {
 
     uint64_t mmio32_cpu_base;
     QPCIWindow mmio32;
-
-    uint64_t pci_hole_start;
-    uint64_t pci_hole_size;
-    uint64_t pci_hole_alloc;
-
-    uint32_t pci_iohole_start;
-    uint32_t pci_iohole_size;
-    uint32_t pci_iohole_alloc;
 } QPCIBusSPAPR;
 
 /*
@@ -167,72 +159,6 @@ static void qpci_spapr_config_writel(QPCIBus *bus, int devfn, uint8_t offset,
     qrtas_ibm_write_pci_config(s->alloc, s->buid, config_addr, 4, value);
 }
 
-static void *qpci_spapr_iomap(QPCIBus *bus, QPCIDevice *dev, int barno,
-                              uint64_t *sizeptr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    static const int bar_reg_map[] = {
-        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
-        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
-    };
-    int bar_reg;
-    uint32_t addr;
-    uint64_t size;
-    uint32_t io_type;
-
-    g_assert(barno >= 0 && barno <= 5);
-    bar_reg = bar_reg_map[barno];
-
-    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
-    addr = qpci_config_readl(dev, bar_reg);
-
-    io_type = addr & PCI_BASE_ADDRESS_SPACE;
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        addr &= PCI_BASE_ADDRESS_IO_MASK;
-    } else {
-        addr &= PCI_BASE_ADDRESS_MEM_MASK;
-    }
-
-    size = (1ULL << ctzl(addr));
-    if (size == 0) {
-        return NULL;
-    }
-    if (sizeptr) {
-        *sizeptr = size;
-    }
-
-    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
-        uint16_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_iohole_alloc, size) + size
-                 <= s->pci_iohole_size);
-        s->pci_iohole_alloc = QEMU_ALIGN_UP(s->pci_iohole_alloc, size);
-        loc = s->pci_iohole_start + s->pci_iohole_alloc;
-        s->pci_iohole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
-
-        return (void *)(unsigned long)loc;
-    } else {
-        uint64_t loc;
-
-        g_assert(QEMU_ALIGN_UP(s->pci_hole_alloc, size) + size
-                 <= s->pci_hole_size);
-        s->pci_hole_alloc = QEMU_ALIGN_UP(s->pci_hole_alloc, size);
-        loc = s->pci_hole_start + s->pci_hole_alloc;
-        s->pci_hole_alloc += size;
-
-        qpci_config_writel(dev, bar_reg, loc);
-
-        return (void *)(unsigned long)loc;
-    }
-}
-
-static void qpci_spapr_iounmap(QPCIBus *bus, void *data)
-{
-    /* FIXME */
-}
-
 #define SPAPR_PCI_BASE               (1ULL << 45)
 
 #define SPAPR_PCI_MMIO32_WIN_SIZE    0x80000000 /* 2 GiB */
@@ -270,9 +196,6 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.config_writew = qpci_spapr_config_writew;
     ret->bus.config_writel = qpci_spapr_config_writel;
 
-    ret->bus.iomap = qpci_spapr_iomap;
-    ret->bus.iounmap = qpci_spapr_iounmap;
-
     /* FIXME: We assume the default location of the PHB for now.
      * Ideally we'd parse the device tree deposited in the guest to
      * get the window locations */
@@ -287,15 +210,9 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->mmio32.pci_base = 0x80000000; /* 2 GiB */
     ret->mmio32.size = SPAPR_PCI_MMIO32_WIN_SIZE;
 
-    ret->pci_hole_start = 0xC0000000;
-    ret->pci_hole_size =
-        ret->mmio32.pci_base + ret->mmio32.size - ret->pci_hole_start;
-    ret->pci_hole_alloc = 0;
-
-    ret->pci_iohole_start = 0xc000;
-    ret->pci_iohole_size =
-        ret->pio.pci_base + ret->pio.size - ret->pci_iohole_start;
-    ret->pci_iohole_alloc = 0;
+    ret->bus.pio_alloc_ptr = 0xc000;
+    ret->bus.mmio_alloc_ptr = ret->mmio32.pci_base;
+    ret->bus.mmio_limit = ret->mmio32.pci_base + ret->mmio32.size;
 
     return &ret->bus;
 }
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 55b01df..bf1c532 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -14,6 +14,7 @@
 #include "libqos/pci.h"
 
 #include "hw/pci/pci_regs.h"
+#include "qemu/host-utils.h"
 
 void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
                          void (*func)(QPCIDevice *dev, int devfn, void *data),
@@ -290,12 +291,63 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
-    return dev->bus->iomap(dev->bus, dev, barno, sizeptr);
+    QPCIBus *bus = dev->bus;
+    static const int bar_reg_map[] = {
+        PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
+        PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
+    };
+    int bar_reg;
+    uint32_t addr, size;
+    uint32_t io_type;
+    uint64_t loc;
+
+    g_assert(barno >= 0 && barno <= 5);
+    bar_reg = bar_reg_map[barno];
+
+    qpci_config_writel(dev, bar_reg, 0xFFFFFFFF);
+    addr = qpci_config_readl(dev, bar_reg);
+
+    io_type = addr & PCI_BASE_ADDRESS_SPACE;
+    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
+        addr &= PCI_BASE_ADDRESS_IO_MASK;
+    } else {
+        addr &= PCI_BASE_ADDRESS_MEM_MASK;
+    }
+
+    g_assert(addr); /* Must have *some* size bits */
+
+    size = 1U << ctz32(addr);
+    if (sizeptr) {
+        *sizeptr = size;
+    }
+
+    if (io_type == PCI_BASE_ADDRESS_SPACE_IO) {
+        loc = QEMU_ALIGN_UP(bus->pio_alloc_ptr, size);
+
+        g_assert(loc >= bus->pio_alloc_ptr);
+        g_assert(loc + size <= QPCI_PIO_LIMIT); /* Keep PIO below 64kiB */
+
+        bus->pio_alloc_ptr = loc + size;
+
+        qpci_config_writel(dev, bar_reg, loc | PCI_BASE_ADDRESS_SPACE_IO);
+    } else {
+        loc = QEMU_ALIGN_UP(bus->mmio_alloc_ptr, size);
+
+        /* Check for space */
+        g_assert(loc >= bus->mmio_alloc_ptr);
+        g_assert(loc + size <= bus->mmio_limit);
+
+        bus->mmio_alloc_ptr = loc + size;
+
+        qpci_config_writel(dev, bar_reg, loc);
+    }
+
+    return (void *)(uintptr_t)loc;
 }
 
 void qpci_iounmap(QPCIDevice *dev, void *data)
 {
-    dev->bus->iounmap(dev->bus, data);
+    /* FIXME */
 }
 
 void qpci_plug_device_test(const char *driver, const char *id,
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 72a2245..f6f916d 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -22,8 +22,7 @@
 typedef struct QPCIDevice QPCIDevice;
 typedef struct QPCIBus QPCIBus;
 
-struct QPCIBus
-{
+struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
@@ -51,8 +50,8 @@ struct QPCIBus
     void (*config_writel)(QPCIBus *bus, int devfn,
                           uint8_t offset, uint32_t value);
 
-    void *(*iomap)(QPCIBus *bus, QPCIDevice *dev, int barno, uint64_t *sizeptr);
-    void (*iounmap)(QPCIBus *bus, void *data);
+    uint16_t pio_alloc_ptr;
+    uint64_t mmio_alloc_ptr, mmio_limit;
 };
 
 struct QPCIDevice
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (2 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 03/12] libqos: Move BAR assignment to " David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-25 12:34   ` Greg Kurz
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
                   ` (8 subsequent siblings)
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The usual model for PCI IO with libqos is to use qpci_iomap() to map a
specific BAR for a PCI device, then perform IOs within that BAR using
qpci_io_{read,write}*().

However, certain devices also have legacy PCI IO.  In this case, instead of
(or as well as) being accessed via PCI BARs, the device can be accessed
via certain well-known, fixed addresses in PCI IO space.

Two existing tests use legacy PCI IO, and take different flawed approaches
to it:
    * tco-test manually constructs a tco_io_base value instead of calling
      qpci_iomap(), which assumes internal knowledge of the structure of
      the value it shouldn't have
    * ide-test uses direct in*() and out*() calls instead of using
      qpci_io_*() accessors, meaning it's not portable to non-x86 machine
      types.

This patch implements a new qpci_iomap_legacy() interface which gets a
handle in the same format as qpci_iomap() but refers to a region in
the legacy PIO space.  For a device which has the same registers
available both in a BAR and in legacy space (quite common), this
allows the same test code to test both options with just a different
iomap() at the beginning.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/libqos/pci.c | 5 +++++
 tests/libqos/pci.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index bf1c532..98a2e56 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -350,6 +350,11 @@ void qpci_iounmap(QPCIDevice *dev, void *data)
     /* FIXME */
 }
 
+void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
+{
+    return (void *)(uintptr_t)addr;
+}
+
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts)
 {
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index f6f916d..b6f855e 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -94,6 +94,7 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
 
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
 void qpci_iounmap(QPCIDevice *dev, void *data);
+void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
 
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap()
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (3 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-25 12:35   ` Greg Kurz
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 06/12] libqos: Add streaming accessors for PCI MMIO David Gibson
                   ` (7 subsequent siblings)
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Avoid tco-test making assumptions about the internal format of the address
tokens passed to PCI IO accessors, by using the new qpci_legacy_iomap()
function.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/tco-test.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/tco-test.c b/tests/tco-test.c
index 0d201b1..129577d 100644
--- a/tests/tco-test.c
+++ b/tests/tco-test.c
@@ -70,7 +70,7 @@ static void test_init(TestData *d)
     /* set Root Complex BAR */
     qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
 
-    d->tco_io_base = (void *)((uintptr_t)PM_IO_BASE_ADDR + 0x60);
+    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
 }
 
 static void stop_tco(const TestData *d)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 06/12] libqos: Add streaming accessors for PCI MMIO
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (4 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Currently PCI memory (aka MMIO) space is accessed via a set of readb/writeb
style accessors.  This is what we want for accessing discrete registers of
a certain size.  However, there are a few cases where we instead need a
"bag of bytes" style streaming interface to PCI MMIO space.  This can be
either for streaming data style registers or when there's actual memory
rather than registers in PCI space, for example frame buffers or ivshmem.

This patch adds backend callbacks, and libqos wrappers for this type of
byte address order preserving accesses.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 14 ++++++++++++++
 tests/libqos/pci-spapr.c | 17 +++++++++++++++++
 tests/libqos/pci.c       | 16 ++++++++++++++++
 tests/libqos/pci.h       |  6 ++++++
 4 files changed, 53 insertions(+)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index d0eb84a..84aee25 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -87,6 +87,17 @@ static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(addr, val);
 }
 
+static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
+{
+    memread(addr, buf, len);
+}
+
+static void qpci_pc_memwrite(QPCIBus *bus, uint32_t addr,
+                             const void *buf, size_t len)
+{
+    memwrite(addr, buf, len);
+}
+
 static uint8_t qpci_pc_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
 {
     outl(0xcf8, (1U << 31) | (devfn << 8) | offset);
@@ -145,6 +156,9 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.mmio_writew = qpci_pc_mmio_writew;
     ret->bus.mmio_writel = qpci_pc_mmio_writel;
 
+    ret->bus.memread = qpci_pc_memread;
+    ret->bus.memwrite = qpci_pc_memwrite;
+
     ret->bus.config_readb = qpci_pc_config_readb;
     ret->bus.config_readw = qpci_pc_config_readw;
     ret->bus.config_readl = qpci_pc_config_readl;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index 70a24b5..ad12c2e 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -114,6 +114,20 @@ static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(s->mmio32_cpu_base + addr, bswap32(val));
 }
 
+static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
+                               void *buf, size_t len)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    memread(s->mmio32_cpu_base + addr, buf, len);
+}
+
+static void qpci_spapr_memwrite(QPCIBus *bus, uint32_t addr,
+                                const void *buf, size_t len)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    memwrite(s->mmio32_cpu_base + addr, buf, len);
+}
+
 static uint8_t qpci_spapr_config_readb(QPCIBus *bus, int devfn, uint8_t offset)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
@@ -188,6 +202,9 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
     ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
 
+    ret->bus.memread = qpci_spapr_memread;
+    ret->bus.memwrite = qpci_spapr_memwrite;
+
     ret->bus.config_readb = qpci_spapr_config_readb;
     ret->bus.config_readw = qpci_spapr_config_readw;
     ret->bus.config_readl = qpci_spapr_config_readl;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 98a2e56..146b063 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -289,6 +289,22 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     }
 }
 
+void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    g_assert(addr >= QPCI_PIO_LIMIT);
+    dev->bus->memread(dev->bus, addr, buf, len);
+}
+
+void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    g_assert(addr >= QPCI_PIO_LIMIT);
+    dev->bus->memwrite(dev->bus, addr, buf, len);
+}
+
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
     QPCIBus *bus = dev->bus;
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index b6f855e..2b08362 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -39,6 +39,9 @@ struct QPCIBus {
     void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
+    void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
+    void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
+
     uint8_t (*config_readb)(QPCIBus *bus, int devfn, uint8_t offset);
     uint16_t (*config_readw)(QPCIBus *bus, int devfn, uint8_t offset);
     uint32_t (*config_readl)(QPCIBus *bus, int devfn, uint8_t offset);
@@ -92,6 +95,9 @@ void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
 
+void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
+void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
+
 void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
 void qpci_iounmap(QPCIDevice *dev, void *data);
 void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (5 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 06/12] libqos: Add streaming accessors for PCI MMIO David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-25  6:47   ` Alexey Kardashevskiy
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test David Gibson
                   ` (5 subsequent siblings)
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

In the libqos PCI code we now have accessors both for registers (byte
significance preserving) and for streaming data (byte address order
preserving).  These exist in both the interface for qtest drivers and in
the machine specific backends.

However, the register-style accessors aren't actually necessary in the
backend.  They can be implemented in terms of the byte address order
preserving accessors by the libqos wrappers.  This works because PCI is
always little endian.

This does assume that the back end byte address order preserving accessors
will perform the equivalent of a single bus transaction for short lengths.
This is the case, and in fact they currently end up using the same
cpu_physical_memory_rw() implementation within the qtest accelerator.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c    | 38 --------------------------------------
 tests/libqos/pci-spapr.c | 44 --------------------------------------------
 tests/libqos/pci.c       | 20 ++++++++++++++------
 tests/libqos/pci.h       |  8 --------
 4 files changed, 14 insertions(+), 96 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 84aee25..849ea56 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -32,61 +32,31 @@ static uint8_t qpci_pc_pio_readb(QPCIBus *bus, uint32_t addr)
     return inb(addr);
 }
 
-static uint8_t qpci_pc_mmio_readb(QPCIBus *bus, uint32_t addr)
-{
-    return readb(addr);
-}
-
 static void qpci_pc_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     outb(addr, val);
 }
 
-static void qpci_pc_mmio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
-{
-    writeb(addr, val);
-}
-
 static uint16_t qpci_pc_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     return inw(addr);
 }
 
-static uint16_t qpci_pc_mmio_readw(QPCIBus *bus, uint32_t addr)
-{
-    return readw(addr);
-}
-
 static void qpci_pc_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
     outw(addr, val);
 }
 
-static void qpci_pc_mmio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
-{
-    writew(addr, val);
-}
-
 static uint32_t qpci_pc_pio_readl(QPCIBus *bus, uint32_t addr)
 {
     return inl(addr);
 }
 
-static uint32_t qpci_pc_mmio_readl(QPCIBus *bus, uint32_t addr)
-{
-    return readl(addr);
-}
-
 static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
     outl(addr, val);
 }
 
-static void qpci_pc_mmio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
-{
-    writel(addr, val);
-}
-
 static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
 {
     memread(addr, buf, len);
@@ -148,14 +118,6 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.pio_writew = qpci_pc_pio_writew;
     ret->bus.pio_writel = qpci_pc_pio_writel;
 
-    ret->bus.mmio_readb = qpci_pc_mmio_readb;
-    ret->bus.mmio_readw = qpci_pc_mmio_readw;
-    ret->bus.mmio_readl = qpci_pc_mmio_readl;
-
-    ret->bus.mmio_writeb = qpci_pc_mmio_writeb;
-    ret->bus.mmio_writew = qpci_pc_mmio_writew;
-    ret->bus.mmio_writel = qpci_pc_mmio_writel;
-
     ret->bus.memread = qpci_pc_memread;
     ret->bus.memwrite = qpci_pc_memwrite;
 
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index ad12c2e..f26488a 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -48,72 +48,36 @@ static uint8_t qpci_spapr_pio_readb(QPCIBus *bus, uint32_t addr)
     return readb(s->pio_cpu_base + addr);
 }
 
-static uint8_t qpci_spapr_mmio32_readb(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return readb(s->mmio32_cpu_base + addr);
-}
-
 static void qpci_spapr_pio_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writeb(s->pio_cpu_base + addr, val);
 }
 
-static void qpci_spapr_mmio32_writeb(QPCIBus *bus, uint32_t addr, uint8_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writeb(s->mmio32_cpu_base + addr, val);
-}
-
 static uint16_t qpci_spapr_pio_readw(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     return bswap16(readw(s->pio_cpu_base + addr));
 }
 
-static uint16_t qpci_spapr_mmio32_readw(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return bswap16(readw(s->mmio32_cpu_base + addr));
-}
-
 static void qpci_spapr_pio_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writew(s->pio_cpu_base + addr, bswap16(val));
 }
 
-static void qpci_spapr_mmio32_writew(QPCIBus *bus, uint32_t addr, uint16_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writew(s->mmio32_cpu_base + addr, bswap16(val));
-}
-
 static uint32_t qpci_spapr_pio_readl(QPCIBus *bus, uint32_t addr)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     return bswap32(readl(s->pio_cpu_base + addr));
 }
 
-static uint32_t qpci_spapr_mmio32_readl(QPCIBus *bus, uint32_t addr)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    return bswap32(readl(s->mmio32_cpu_base + addr));
-}
-
 static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
 {
     QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
     writel(s->pio_cpu_base + addr, bswap32(val));
 }
 
-static void qpci_spapr_mmio32_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
-{
-    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
-    writel(s->mmio32_cpu_base + addr, bswap32(val));
-}
-
 static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
                                void *buf, size_t len)
 {
@@ -194,14 +158,6 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.pio_writew = qpci_spapr_pio_writew;
     ret->bus.pio_writel = qpci_spapr_pio_writel;
 
-    ret->bus.mmio_readb = qpci_spapr_mmio32_readb;
-    ret->bus.mmio_readw = qpci_spapr_mmio32_readw;
-    ret->bus.mmio_readl = qpci_spapr_mmio32_readl;
-
-    ret->bus.mmio_writeb = qpci_spapr_mmio32_writeb;
-    ret->bus.mmio_writew = qpci_spapr_mmio32_writew;
-    ret->bus.mmio_writel = qpci_spapr_mmio32_writel;
-
     ret->bus.memread = qpci_spapr_memread;
     ret->bus.memwrite = qpci_spapr_memwrite;
 
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 146b063..bdffeb3 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -230,7 +230,9 @@ uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readb(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readb(dev->bus, addr);
+        uint8_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return val;
     }
 }
 
@@ -241,7 +243,9 @@ uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readw(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readw(dev->bus, addr);
+        uint16_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le16_to_cpu(val);
     }
 }
 
@@ -252,7 +256,9 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
     if (addr < QPCI_PIO_LIMIT) {
         return dev->bus->pio_readl(dev->bus, addr);
     } else {
-        return dev->bus->mmio_readl(dev->bus, addr);
+        uint32_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le32_to_cpu(val);
     }
 }
 
@@ -263,7 +269,7 @@ void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writeb(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writeb(dev->bus, addr, value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
@@ -274,7 +280,8 @@ void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writew(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writew(dev->bus, addr, value);
+        value = cpu_to_le16(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
@@ -285,7 +292,8 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     if (addr < QPCI_PIO_LIMIT) {
         dev->bus->pio_writel(dev->bus, addr, value);
     } else {
-        dev->bus->mmio_writel(dev->bus, addr, value);
+        value = cpu_to_le32(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
     }
 }
 
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 2b08362..ce6ed08 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -27,18 +27,10 @@ struct QPCIBus {
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
 
-    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
-    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
-    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
-
     void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
     void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
 
-    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
-    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
-    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
-
     void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
     void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (6 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-25  7:01   ` Alexey Kardashevskiy
  2016-10-25 12:36   ` Greg Kurz
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 09/12] libqos: Add 64-bit PCI IO accessors David Gibson
                   ` (4 subsequent siblings)
  12 siblings, 2 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

ide-test uses many explicit inb() / outb() operations for its IO, which
means it's not portable to non-x86 platforms.  This cleans it up to use
the libqos PCI accessors instead.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 118 insertions(+), 61 deletions(-)

diff --git a/tests/ide-test.c b/tests/ide-test.c
index a8a4081..86c4373 100644
--- a/tests/ide-test.c
+++ b/tests/ide-test.c
@@ -137,7 +137,7 @@ static void ide_test_quit(void)
     qtest_end();
 }
 
-static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
+static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
 {
     QPCIDevice *dev;
     uint16_t vendor_id, device_id;
@@ -156,7 +156,9 @@ static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
     g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
 
     /* Map bmdma BAR */
-    *bmdma_base = (uint16_t)(uintptr_t) qpci_iomap(dev, 4, NULL);
+    *bmdma_base = qpci_iomap(dev, 4, NULL);
+
+    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
 
     qpci_device_enable(dev);
 
@@ -179,17 +181,19 @@ typedef struct PrdtEntry {
 
 static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
                             PrdtEntry *prdt, int prdt_entries,
-                            void(*post_exec)(uint64_t sector, int nb_sectors))
+                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
+                                             uint64_t sector, int nb_sectors))
 {
     QPCIDevice *dev;
-    uint16_t bmdma_base;
+    void *bmdma_base;
+    void *ide_base;
     uintptr_t guest_prdt;
     size_t len;
     bool from_dev;
     uint8_t status;
     int flags;
 
-    dev = get_pci_device(&bmdma_base);
+    dev = get_pci_device(&bmdma_base, &ide_base);
 
     flags = cmd & ~0xff;
     cmd &= 0xff;
@@ -214,59 +218,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
     }
 
     /* Select device 0 */
-    outb(IDE_BASE + reg_device, 0 | LBA);
+    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
 
     /* Stop any running transfer, clear any pending interrupt */
-    outb(bmdma_base + bmreg_cmd, 0);
-    outb(bmdma_base + bmreg_status, BM_STS_INTR);
+    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
 
     /* Setup PRDT */
     len = sizeof(*prdt) * prdt_entries;
     guest_prdt = guest_alloc(guest_malloc, len);
     memwrite(guest_prdt, prdt, len);
-    outl(bmdma_base + bmreg_prdt, guest_prdt);
+    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
 
     /* ATA DMA command */
     if (cmd == CMD_PACKET) {
         /* Enables ATAPI DMA; otherwise PIO is attempted */
-        outb(IDE_BASE + reg_feature, 0x01);
+        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
     } else {
-        outb(IDE_BASE + reg_nsectors, nb_sectors);
-        outb(IDE_BASE + reg_lba_low,    sector & 0xff);
-        outb(IDE_BASE + reg_lba_middle, (sector >> 8) & 0xff);
-        outb(IDE_BASE + reg_lba_high,   (sector >> 16) & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
+        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
+        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
     }
 
-    outb(IDE_BASE + reg_command, cmd);
+    qpci_io_writeb(dev, ide_base + reg_command, cmd);
 
     if (post_exec) {
-        post_exec(sector, nb_sectors);
+        post_exec(dev, ide_base, sector, nb_sectors);
     }
 
     /* Start DMA transfer */
-    outb(bmdma_base + bmreg_cmd, BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
+    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
+                   BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
 
     if (flags & CMDF_ABORT) {
-        outb(bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
     }
 
     /* Wait for the DMA transfer to complete */
     do {
-        status = inb(bmdma_base + bmreg_status);
+        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
     } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
 
     g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
 
     /* Check IDE status code */
-    assert_bit_set(inb(IDE_BASE + reg_status), DRDY);
-    assert_bit_clear(inb(IDE_BASE + reg_status), BSY | DRQ);
+    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
 
     /* Reading the status register clears the IRQ */
     g_assert(!get_irq(IDE_PRIMARY_IRQ));
 
     /* Stop DMA transfer if still active */
     if (status & BM_STS_ACTIVE) {
-        outb(bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
     }
 
     free_pci_device(dev);
@@ -276,6 +281,8 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
 
 static void test_bmdma_simple_rw(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
     uint8_t *buf;
     uint8_t *cmpbuf;
@@ -289,6 +296,8 @@ static void test_bmdma_simple_rw(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     buf = g_malloc(len);
     cmpbuf = g_malloc(len);
 
@@ -299,7 +308,7 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Write 0xaa pattern to sector 1 */
     memset(buf, 0xaa, len);
@@ -308,14 +317,14 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Read and verify 0x55 pattern in sector 0 */
     memset(cmpbuf, 0x55, len);
 
     status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -325,7 +334,7 @@ static void test_bmdma_simple_rw(void)
 
     status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -337,6 +346,8 @@ static void test_bmdma_simple_rw(void)
 
 static void test_bmdma_short_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -346,21 +357,25 @@ static void test_bmdma_short_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_one_sector_short_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     /* Read 2 sectors but only give 1 sector in PRDT */
@@ -371,21 +386,25 @@ static void test_bmdma_one_sector_short_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_long_prdt(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -395,23 +414,29 @@ static void test_bmdma_long_prdt(void)
         },
     };
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_no_busmaster(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
      * able to access it anyway because the Bus Master bit in the PCI command
      * register isn't set. This is complete nonsense, but it used to be pretty
@@ -424,7 +449,7 @@ static void test_bmdma_no_busmaster(void)
     /* Not entirely clear what the expected result is, but this is what we get
      * in practice. At least we want to be aware of any changes. */
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 }
 
 static void test_bmdma_setup(void)
@@ -454,6 +479,8 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
 
 static void test_identify(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     uint16_t buf[256];
     int i;
@@ -464,23 +491,25 @@ static void test_identify(void)
         "-global ide-hd.ver=%s",
         tmp_path, "testdisk", "version");
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* IDENTIFY command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_IDENTIFY);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
 
     /* Read in the IDENTIFY buffer and check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     for (i = 0; i < 256; i++) {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
         assert_bit_set(data, DRDY | DRQ);
         assert_bit_clear(data, BSY | DF | ERR);
 
-        ((uint16_t*) buf)[i] = inw(IDE_BASE + reg_data);
+        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
     }
 
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, DRDY);
     assert_bit_clear(data, BSY | DF | ERR | DRQ);
 
@@ -505,11 +534,15 @@ static void test_identify(void)
  */
 static void make_dirty(uint8_t device)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t status;
     size_t len = 512;
     uintptr_t guest_buf;
     void* buf;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     guest_buf = guest_alloc(guest_malloc, len);
     buf = g_malloc(len);
     g_assert(guest_buf);
@@ -527,19 +560,23 @@ static void make_dirty(uint8_t device)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
 
     g_free(buf);
 }
 
 static void test_flush(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
 
     ide_test_start(
         "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
         tmp_path);
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
@@ -549,11 +586,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
 
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -561,11 +598,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
 
     /* Check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -576,6 +613,8 @@ static void test_flush(void)
 
 static void test_retry_flush(const char *machine)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     const char *s;
 
@@ -587,17 +626,19 @@ static void test_retry_flush(const char *machine)
         "rerror=stop,werror=stop",
         debug_path, tmp_path);
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
     make_dirty(0);
 
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = inb(IDE_BASE + reg_status);
+    data = qpci_io_readb(dev, ide_base + reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -608,11 +649,11 @@ static void test_retry_flush(const char *machine)
     qmp_discard_response(s);
 
     /* Check registers */
-    data = inb(IDE_BASE + reg_device);
+    data = qpci_io_readb(dev, ide_base + reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -623,11 +664,16 @@ static void test_retry_flush(const char *machine)
 
 static void test_flush_nodev(void)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
+
     ide_test_start("");
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* FLUSH CACHE command on device 0*/
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
 
     /* Just testing that qemu doesn't crash... */
 
@@ -654,7 +700,8 @@ typedef struct Read10CDB {
     uint16_t padding;
 } __attribute__((__packed__)) Read10CDB;
 
-static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
+static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
+                                 uint64_t lba, int nblocks)
 {
     Read10CDB pkt = { .padding = 0 };
     int i;
@@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
 
     /* Send Packet */
     for (i = 0; i < sizeof(Read10CDB)/2; i++) {
-        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
+        qpci_io_writew(dev, ide_base + reg_data,
+                       le16_to_cpu(((uint16_t *)&pkt)[i]));
     }
 }
 
@@ -683,13 +731,17 @@ static void nsleep(int64_t nsecs)
 
 static uint8_t ide_wait_clear(uint8_t flag)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     uint8_t data;
     time_t st;
 
+    dev = get_pci_device(&bmdma_base, &ide_base);
+
     /* Wait with a 5 second timeout */
     time(&st);
     while (true) {
-        data = inb(IDE_BASE + reg_status);
+        data = qpci_io_readb(dev, ide_base + reg_status);
         if (!(data & flag)) {
             return data;
         }
@@ -723,6 +775,8 @@ static void ide_wait_intr(int irq)
 
 static void cdrom_pio_impl(int nblocks)
 {
+    QPCIDevice *dev;
+    void *bmdma_base, *ide_base;
     FILE *fh;
     int patt_blocks = MAX(16, nblocks);
     size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
@@ -741,13 +795,15 @@ static void cdrom_pio_impl(int nblocks)
 
     ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
                    "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
+    dev = get_pci_device(&bmdma_base, &ide_base);
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* PACKET command on device 0 */
-    outb(IDE_BASE + reg_device, 0);
-    outb(IDE_BASE + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
-    outb(IDE_BASE + reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
-    outb(IDE_BASE + reg_command, CMD_PACKET);
+    qpci_io_writeb(dev, ide_base + reg_device, 0);
+    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
+    qpci_io_writeb(dev, ide_base + reg_lba_high,
+                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
+    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
     /* HP0: Check_Status_A State */
     nsleep(400);
     data = ide_wait_clear(BSY);
@@ -756,7 +812,7 @@ static void cdrom_pio_impl(int nblocks)
     assert_bit_clear(data, ERR | DF | BSY);
 
     /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
-    send_scsi_cdb_read10(0, nblocks);
+    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
 
     /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
      * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
@@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
 
         /* HP4: Transfer_Data */
         for (j = 0; j < MIN((limit / 2), rem); j++) {
-            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
+            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
+                                                       ide_base + reg_data));
         }
     }
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 09/12] libqos: Add 64-bit PCI IO accessors
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (7 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
                   ` (3 subsequent siblings)
  12 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

Currently the libqos PCI layer includes accessor helpers for 8, 16 and 32
bit reads and writes.  It's likely that we'll want 64-bit accesses in the
future (plenty of modern peripherals will have 64-bit reigsters).  This
adds them.

For PIO (not MMIO) accesses on the PC backend, this is implemented as two
32-bit ins or outs.  That's not ideal but AFAICT x86 doesn't have 64-bit
versions of in and out.

This patch also converts the single current user of 64-bit accesses -
virtio-pci.c to use the new mechanism, rather than a sequence of 8 byte
reads.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
---
 tests/libqos/pci-pc.c     | 13 +++++++++++++
 tests/libqos/pci-spapr.c  | 14 ++++++++++++++
 tests/libqos/pci.c        | 25 +++++++++++++++++++++++++
 tests/libqos/pci.h        |  4 ++++
 tests/libqos/virtio-pci.c | 16 ++++------------
 5 files changed, 60 insertions(+), 12 deletions(-)

diff --git a/tests/libqos/pci-pc.c b/tests/libqos/pci-pc.c
index 849ea56..ded1c54 100644
--- a/tests/libqos/pci-pc.c
+++ b/tests/libqos/pci-pc.c
@@ -57,6 +57,17 @@ static void qpci_pc_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     outl(addr, val);
 }
 
+static uint64_t qpci_pc_pio_readq(QPCIBus *bus, uint32_t addr)
+{
+    return (uint64_t)inl(addr) + ((uint64_t)inl(addr + 4) << 32);
+}
+
+static void qpci_pc_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
+{
+    outl(addr, val & 0xffffffff);
+    outl(addr + 4, val >> 32);
+}
+
 static void qpci_pc_memread(QPCIBus *bus, uint32_t addr, void *buf, size_t len)
 {
     memread(addr, buf, len);
@@ -113,10 +124,12 @@ QPCIBus *qpci_init_pc(QGuestAllocator *alloc)
     ret->bus.pio_readb = qpci_pc_pio_readb;
     ret->bus.pio_readw = qpci_pc_pio_readw;
     ret->bus.pio_readl = qpci_pc_pio_readl;
+    ret->bus.pio_readq = qpci_pc_pio_readq;
 
     ret->bus.pio_writeb = qpci_pc_pio_writeb;
     ret->bus.pio_writew = qpci_pc_pio_writew;
     ret->bus.pio_writel = qpci_pc_pio_writel;
+    ret->bus.pio_writeq = qpci_pc_pio_writeq;
 
     ret->bus.memread = qpci_pc_memread;
     ret->bus.memwrite = qpci_pc_memwrite;
diff --git a/tests/libqos/pci-spapr.c b/tests/libqos/pci-spapr.c
index f26488a..1e5d015 100644
--- a/tests/libqos/pci-spapr.c
+++ b/tests/libqos/pci-spapr.c
@@ -78,6 +78,18 @@ static void qpci_spapr_pio_writel(QPCIBus *bus, uint32_t addr, uint32_t val)
     writel(s->pio_cpu_base + addr, bswap32(val));
 }
 
+static uint64_t qpci_spapr_pio_readq(QPCIBus *bus, uint32_t addr)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    return bswap64(readq(s->pio_cpu_base + addr));
+}
+
+static void qpci_spapr_pio_writeq(QPCIBus *bus, uint32_t addr, uint64_t val)
+{
+    QPCIBusSPAPR *s = container_of(bus, QPCIBusSPAPR, bus);
+    writeq(s->pio_cpu_base + addr, bswap64(val));
+}
+
 static void qpci_spapr_memread(QPCIBus *bus, uint32_t addr,
                                void *buf, size_t len)
 {
@@ -153,10 +165,12 @@ QPCIBus *qpci_init_spapr(QGuestAllocator *alloc)
     ret->bus.pio_readb = qpci_spapr_pio_readb;
     ret->bus.pio_readw = qpci_spapr_pio_readw;
     ret->bus.pio_readl = qpci_spapr_pio_readl;
+    ret->bus.pio_readq = qpci_spapr_pio_readq;
 
     ret->bus.pio_writeb = qpci_spapr_pio_writeb;
     ret->bus.pio_writew = qpci_spapr_pio_writew;
     ret->bus.pio_writel = qpci_spapr_pio_writel;
+    ret->bus.pio_writeq = qpci_spapr_pio_writeq;
 
     ret->bus.memread = qpci_spapr_memread;
     ret->bus.memwrite = qpci_spapr_memwrite;
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index bdffeb3..3021651 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -262,6 +262,19 @@ uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
     }
 }
 
+uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readq(dev->bus, addr);
+    } else {
+        uint64_t val;
+        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        return le64_to_cpu(val);
+    }
+}
+
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
 {
     uintptr_t addr = (uintptr_t)data;
@@ -297,6 +310,18 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
     }
 }
 
+void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
+{
+    uintptr_t addr = (uintptr_t)data;
+
+    if (addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeq(dev->bus, addr, value);
+    } else {
+        value = cpu_to_le64(value);
+        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+    }
+}
+
 void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
 {
     uintptr_t addr = (uintptr_t)data;
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index ce6ed08..531e3f7 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -26,10 +26,12 @@ struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
     uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
     uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
+    uint64_t (*pio_readq)(QPCIBus *bus, uint32_t addr);
 
     void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
     void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
     void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
+    void (*pio_writeq)(QPCIBus *bus, uint32_t addr, uint64_t value);
 
     void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
     void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
@@ -82,10 +84,12 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
 uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
 uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
 uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
+uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
 
 void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
 void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
 void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
+void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
 
 void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
 void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index fa82132..c69d09d 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -106,22 +106,14 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
 static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    int i;
-    uint64_t u64 = 0;
+    uint64_t val;
 
+    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
-        for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
-                                           + off + i) << (7 - i) * 8;
-        }
-    } else {
-        for (i = 0; i < 8; ++i) {
-            u64 |= (uint64_t)qpci_io_readb(dev->pdev, CONFIG_BASE(dev)
-                                           + off + i) << i * 8;
-        }
+        val = bswap64(val);
     }
 
-    return u64;
+    return val;
 }
 
 static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (8 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 09/12] libqos: Add 64-bit PCI IO accessors David Gibson
@ 2016-10-24  4:59 ` David Gibson
  2016-10-25  9:23   ` Greg Kurz
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test David Gibson
                   ` (2 subsequent siblings)
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-24  4:59 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

ivshmem implements a block of shared memory in a PCI BAR.  Currently our
test case accesses this using qtest_mem{read,write}.  However, deducing
the correct addresses for these requires making assumptions about the
internel format returned by qpci_iomap(), along with some ugly casts.

This patch changes the test to use the new qpci_mem{read,write} interfaces
which is neater.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
---
 tests/ivshmem-test.c | 35 +++++++++++++++++++++++++++--------
 1 file changed, 27 insertions(+), 8 deletions(-)

diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
index f36bfe7..fbd8258 100644
--- a/tests/ivshmem-test.c
+++ b/tests/ivshmem-test.c
@@ -93,6 +93,25 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
     global_qtest = qtest;
 }
 
+static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
+{
+    QTestState *qtest = global_qtest;
+
+    global_qtest = s->qtest;
+    qpci_memread(s->dev, s->mem_base + off, buf, len);
+    global_qtest = qtest;
+}
+
+static inline void write_mem(IVState *s, uint64_t off,
+                             const void *buf, size_t len)
+{
+    QTestState *qtest = global_qtest;
+
+    global_qtest = s->qtest;
+    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
+    global_qtest = qtest;
+}
+
 static void cleanup_vm(IVState *s)
 {
     g_free(s->dev);
@@ -169,7 +188,7 @@ static void test_ivshmem_single(void)
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
         data[i] = i;
     }
-    qtest_memwrite(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
+    write_mem(s, 0, data, sizeof(data));
 
     /* verify write */
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
@@ -178,7 +197,7 @@ static void test_ivshmem_single(void)
 
     /* read it back and verify read */
     memset(data, 0, sizeof(data));
-    qtest_memread(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
+    read_mem(s, 0, data, sizeof(data));
     for (i = 0; i < G_N_ELEMENTS(data); i++) {
         g_assert_cmpuint(data[i], ==, i);
     }
@@ -201,29 +220,29 @@ static void test_ivshmem_pair(void)
 
     /* host write, guest 1 & 2 read */
     memset(tmpshmem, 0x42, TMPSHMSIZE);
-    qtest_memread(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
+    read_mem(s1, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x42);
     }
-    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s2, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x42);
     }
 
     /* guest 1 write, guest 2 read */
     memset(data, 0x43, TMPSHMSIZE);
-    qtest_memwrite(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
+    write_mem(s1, 0, data, TMPSHMSIZE);
     memset(data, 0, TMPSHMSIZE);
-    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s2, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x43);
     }
 
     /* guest 2 write, guest 1 read */
     memset(data, 0x44, TMPSHMSIZE);
-    qtest_memwrite(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    write_mem(s2, 0, data, TMPSHMSIZE);
     memset(data, 0, TMPSHMSIZE);
-    qtest_memread(s1->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
+    read_mem(s1, 0, data, TMPSHMSIZE);
     for (i = 0; i < TMPSHMSIZE; i++) {
         g_assert_cmpuint(data[i], ==, 0x44);
     }
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (9 preceding siblings ...)
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
@ 2016-10-24  5:00 ` David Gibson
  2016-10-24 16:50   ` John Snow
  2016-10-25  8:58   ` Greg Kurz
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle David Gibson
  2016-10-25  3:35 ` [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
  12 siblings, 2 replies; 31+ messages in thread
From: David Gibson @ 2016-10-24  5:00 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

In a couple of places ahci-test makes assumptions about how the tokens
returned from qpci_iomap() are formatted in ways it probably shouldn't.

First in verify_state() it uses a non-NULL token to indicate that the AHCI
device has been enabled (part of enabling is to iomap()).  This changes it
to use an explicit 'enabled' flag instead.

Second, it uses the fact that the token contains a PCI address, stored when
the BAR is mapped during initialization to check that the BAR has the same
value after a migration.  This changes it to explicitly read the BAR
register before and after the migration and compare.

Together, these changes will  make the test more robust against changes to
the internals of the libqos PCI layer.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/ahci-test.c   | 13 +++++++------
 tests/libqos/ahci.c |  1 +
 tests/libqos/ahci.h |  1 +
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/tests/ahci-test.c b/tests/ahci-test.c
index 9c0adce..70bcafa 100644
--- a/tests/ahci-test.c
+++ b/tests/ahci-test.c
@@ -78,25 +78,23 @@ static void string_bswap16(uint16_t *s, size_t bytes)
 /**
  * Verify that the transfer did not corrupt our state at all.
  */
-static void verify_state(AHCIQState *ahci)
+static void verify_state(AHCIQState *ahci, uint64_t hba_old)
 {
     int i, j;
     uint32_t ahci_fingerprint;
     uint64_t hba_base;
-    uint64_t hba_stored;
     AHCICommandHeader cmd;
 
     ahci_fingerprint = qpci_config_readl(ahci->dev, PCI_VENDOR_ID);
     g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
 
     /* If we haven't initialized, this is as much as can be validated. */
-    if (!ahci->hba_base) {
+    if (!ahci->enabled) {
         return;
     }
 
     hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
-    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
-    g_assert_cmphex(hba_base, ==, hba_stored);
+    g_assert_cmphex(hba_base, ==, hba_old);
 
     g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP), ==, ahci->cap);
     g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP2), ==, ahci->cap2);
@@ -119,12 +117,15 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
     QOSState *tmp = to->parent;
     QPCIDevice *dev = to->dev;
     char *uri_local = NULL;
+    uint64_t hba_old;
 
     if (uri == NULL) {
         uri_local = g_strdup_printf("%s%s", "unix:", mig_socket);
         uri = uri_local;
     }
 
+    hba_old = (uint64_t)qpci_config_readl(from->dev, PCI_BASE_ADDRESS_5);
+
     /* context will be 'to' after completion. */
     migrate(from->parent, to->parent, uri);
 
@@ -141,7 +142,7 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
     from->parent = tmp;
     from->dev = dev;
 
-    verify_state(to);
+    verify_state(to, hba_old);
     g_free(uri_local);
 }
 
diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
index 716ab79..8e789d8 100644
--- a/tests/libqos/ahci.c
+++ b/tests/libqos/ahci.c
@@ -351,6 +351,7 @@ void ahci_hba_enable(AHCIQState *ahci)
     reg = ahci_rreg(ahci, AHCI_GHC);
     ASSERT_BIT_SET(reg, AHCI_GHC_IE);
 
+    ahci->enabled = true;
     /* TODO: The device should now be idling and waiting for commands.
      * In the future, a small test-case to inspect the Register D2H FIS
      * and clear the initial interrupts might be good. */
diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
index c69fb5a..9b0c1d7 100644
--- a/tests/libqos/ahci.h
+++ b/tests/libqos/ahci.h
@@ -327,6 +327,7 @@ typedef struct AHCIQState {
     uint32_t cap;
     uint32_t cap2;
     AHCIPortQState port[32];
+    bool enabled;
 } AHCIQState;
 
 /**
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (10 preceding siblings ...)
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test David Gibson
@ 2016-10-24  5:00 ` David Gibson
  2016-10-25  9:21   ` Greg Kurz
  2016-10-25  3:35 ` [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-24  5:00 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug,
	thuth, David Gibson

The usual use model for the libqos PCI functions is to map a specific PCI
BAR using qpci_iomap() then pass the returned token into IO accessor
functions.  This, and the fact that iomap() returns a (void *) which
actually contains a PCI space address, kind of suggests that the return
value from iomap is supposed to be an opaque token.

..except that the callers expect to be able to add offsets to it.  Which
also assumes the compiler will support pointer arithmetic on a (void *),
and treat it as working with byte offsets.

To clarify this situation change iomap() and the IO accessors to take
a definitely opaque BAR handle (enforced with a wrapper struct) along with
an offset within the BAR.  This changes both the functions and all the
callers.

There were a number of places that checked if iomap() returned non-NULL,
and or initialized it to NULL before hand.  Since iomap() already assert()s
if it fails to map the BAR, these tests were mostly pointless and are
removed.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
---
 tests/e1000e-test.c       |   7 +-
 tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
 tests/ivshmem-test.c      |  16 ++---
 tests/libqos/ahci.c       |   3 +-
 tests/libqos/ahci.h       |   6 +-
 tests/libqos/pci.c        | 149 ++++++++++++++++++---------------------
 tests/libqos/pci.h        |  46 +++++++-----
 tests/libqos/usb.c        |   6 +-
 tests/libqos/usb.h        |   2 +-
 tests/libqos/virtio-pci.c | 101 +++++++++++++-------------
 tests/libqos/virtio-pci.h |   2 +-
 tests/rtl8139-test.c      |  10 ++-
 tests/tco-test.c          |  80 ++++++++++-----------
 tests/usb-hcd-ehci-test.c |   5 +-
 14 files changed, 300 insertions(+), 309 deletions(-)

diff --git a/tests/e1000e-test.c b/tests/e1000e-test.c
index 3979b20..8c42ca9 100644
--- a/tests/e1000e-test.c
+++ b/tests/e1000e-test.c
@@ -87,7 +87,7 @@
 
 typedef struct e1000e_device {
     QPCIDevice *pci_dev;
-    void *mac_regs;
+    QPCIBar mac_regs;
 
     uint64_t tx_ring;
     uint64_t rx_ring;
@@ -119,12 +119,12 @@ static QPCIDevice *e1000e_device_find(QPCIBus *bus)
 
 static void e1000e_macreg_write(e1000e_device *d, uint32_t reg, uint32_t val)
 {
-    qpci_io_writel(d->pci_dev, d->mac_regs + reg, val);
+    qpci_io_writel(d->pci_dev, d->mac_regs, reg, val);
 }
 
 static uint32_t e1000e_macreg_read(e1000e_device *d, uint32_t reg)
 {
-    return qpci_io_readl(d->pci_dev, d->mac_regs + reg);
+    return qpci_io_readl(d->pci_dev, d->mac_regs, reg);
 }
 
 static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
@@ -138,7 +138,6 @@ static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
 
     /* Map BAR0 (mac registers) */
     d->mac_regs = qpci_iomap(d->pci_dev, 0, NULL);
-    g_assert_nonnull(d->mac_regs);
 
     /* Reset the device */
     val = e1000e_macreg_read(d, E1000E_CTRL);
diff --git a/tests/ide-test.c b/tests/ide-test.c
index 86c4373..67c7df0 100644
--- a/tests/ide-test.c
+++ b/tests/ide-test.c
@@ -137,7 +137,7 @@ static void ide_test_quit(void)
     qtest_end();
 }
 
-static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
+static QPCIDevice *get_pci_device(QPCIBar *bmdma_bar, QPCIBar *ide_bar)
 {
     QPCIDevice *dev;
     uint16_t vendor_id, device_id;
@@ -156,9 +156,9 @@ static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
     g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
 
     /* Map bmdma BAR */
-    *bmdma_base = qpci_iomap(dev, 4, NULL);
+    *bmdma_bar = qpci_iomap(dev, 4, NULL);
 
-    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
+    *ide_bar = qpci_legacy_iomap(dev, IDE_BASE);
 
     qpci_device_enable(dev);
 
@@ -181,19 +181,18 @@ typedef struct PrdtEntry {
 
 static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
                             PrdtEntry *prdt, int prdt_entries,
-                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
+                            void(*post_exec)(QPCIDevice *dev, QPCIBar ide_bar,
                                              uint64_t sector, int nb_sectors))
 {
     QPCIDevice *dev;
-    void *bmdma_base;
-    void *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uintptr_t guest_prdt;
     size_t len;
     bool from_dev;
     uint8_t status;
     int flags;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     flags = cmd & ~0xff;
     cmd &= 0xff;
@@ -218,60 +217,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
     }
 
     /* Select device 0 */
-    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0 | LBA);
 
     /* Stop any running transfer, clear any pending interrupt */
-    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
-    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
+    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
+    qpci_io_writeb(dev, bmdma_bar, bmreg_status, BM_STS_INTR);
 
     /* Setup PRDT */
     len = sizeof(*prdt) * prdt_entries;
     guest_prdt = guest_alloc(guest_malloc, len);
     memwrite(guest_prdt, prdt, len);
-    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
+    qpci_io_writel(dev, bmdma_bar, bmreg_prdt, guest_prdt);
 
     /* ATA DMA command */
     if (cmd == CMD_PACKET) {
         /* Enables ATAPI DMA; otherwise PIO is attempted */
-        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
+        qpci_io_writeb(dev, ide_bar, reg_feature, 0x01);
     } else {
-        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
-        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
-        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
-        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_nsectors, nb_sectors);
+        qpci_io_writeb(dev, ide_bar, reg_lba_low,    sector & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_lba_middle, (sector >> 8) & 0xff);
+        qpci_io_writeb(dev, ide_bar, reg_lba_high,   (sector >> 16) & 0xff);
     }
 
-    qpci_io_writeb(dev, ide_base + reg_command, cmd);
+    qpci_io_writeb(dev, ide_bar, reg_command, cmd);
 
     if (post_exec) {
-        post_exec(dev, ide_base, sector, nb_sectors);
+        post_exec(dev, ide_bar, sector, nb_sectors);
     }
 
     /* Start DMA transfer */
-    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
+    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd,
                    BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
 
     if (flags & CMDF_ABORT) {
-        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
     }
 
     /* Wait for the DMA transfer to complete */
     do {
-        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
+        status = qpci_io_readb(dev, bmdma_bar, bmreg_status);
     } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
 
     g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
 
     /* Check IDE status code */
-    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
+    assert_bit_set(qpci_io_readb(dev, ide_bar, reg_status), DRDY);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), BSY | DRQ);
 
     /* Reading the status register clears the IRQ */
     g_assert(!get_irq(IDE_PRIMARY_IRQ));
 
     /* Stop DMA transfer if still active */
     if (status & BM_STS_ACTIVE) {
-        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
+        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
     }
 
     free_pci_device(dev);
@@ -282,7 +281,7 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
 static void test_bmdma_simple_rw(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
     uint8_t *buf;
     uint8_t *cmpbuf;
@@ -296,7 +295,7 @@ static void test_bmdma_simple_rw(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     buf = g_malloc(len);
     cmpbuf = g_malloc(len);
@@ -308,7 +307,7 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Write 0xaa pattern to sector 1 */
     memset(buf, 0xaa, len);
@@ -317,14 +316,14 @@ static void test_bmdma_simple_rw(void)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Read and verify 0x55 pattern in sector 0 */
     memset(cmpbuf, 0x55, len);
 
     status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -334,7 +333,7 @@ static void test_bmdma_simple_rw(void)
 
     status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     memread(guest_buf, buf, len);
     g_assert(memcmp(buf, cmpbuf, len) == 0);
@@ -347,7 +346,7 @@ static void test_bmdma_simple_rw(void)
 static void test_bmdma_short_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -357,25 +356,25 @@ static void test_bmdma_short_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_one_sector_short_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     /* Read 2 sectors but only give 1 sector in PRDT */
@@ -386,25 +385,25 @@ static void test_bmdma_one_sector_short_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, 0);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_long_prdt(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
     PrdtEntry prdt[] = {
@@ -414,28 +413,28 @@ static void test_bmdma_long_prdt(void)
         },
     };
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Normal request */
     status = send_dma_request(CMD_READ_DMA, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     /* Abort the request before it completes */
     status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
                               prdt, ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_no_busmaster(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
      * able to access it anyway because the Bus Master bit in the PCI command
@@ -449,7 +448,7 @@ static void test_bmdma_no_busmaster(void)
     /* Not entirely clear what the expected result is, but this is what we get
      * in practice. At least we want to be aware of any changes. */
     g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 }
 
 static void test_bmdma_setup(void)
@@ -480,7 +479,7 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
 static void test_identify(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     uint16_t buf[256];
     int i;
@@ -491,25 +490,25 @@ static void test_identify(void)
         "-global ide-hd.ver=%s",
         tmp_path, "testdisk", "version");
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* IDENTIFY command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_IDENTIFY);
 
     /* Read in the IDENTIFY buffer and check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     for (i = 0; i < 256; i++) {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
         assert_bit_set(data, DRDY | DRQ);
         assert_bit_clear(data, BSY | DF | ERR);
 
-        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
+        buf[i] = qpci_io_readw(dev, ide_bar, reg_data);
     }
 
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, DRDY);
     assert_bit_clear(data, BSY | DF | ERR | DRQ);
 
@@ -535,13 +534,13 @@ static void test_identify(void)
 static void make_dirty(uint8_t device)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t status;
     size_t len = 512;
     uintptr_t guest_buf;
     void* buf;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     guest_buf = guest_alloc(guest_malloc, len);
     buf = g_malloc(len);
@@ -560,7 +559,7 @@ static void make_dirty(uint8_t device)
     status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
                               ARRAY_SIZE(prdt), NULL);
     g_assert_cmphex(status, ==, BM_STS_INTR);
-    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
+    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
 
     g_free(buf);
 }
@@ -568,14 +567,14 @@ static void make_dirty(uint8_t device)
 static void test_flush(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
 
     ide_test_start(
         "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
         tmp_path);
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
@@ -586,11 +585,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -598,11 +597,11 @@ static void test_flush(void)
     g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
 
     /* Check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -614,7 +613,7 @@ static void test_flush(void)
 static void test_retry_flush(const char *machine)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     const char *s;
 
@@ -626,7 +625,7 @@ static void test_retry_flush(const char *machine)
         "rerror=stop,werror=stop",
         debug_path, tmp_path);
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
@@ -634,11 +633,11 @@ static void test_retry_flush(const char *machine)
     make_dirty(0);
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Check status while request is in flight*/
-    data = qpci_io_readb(dev, ide_base + reg_status);
+    data = qpci_io_readb(dev, ide_bar, reg_status);
     assert_bit_set(data, BSY | DRDY);
     assert_bit_clear(data, DF | ERR | DRQ);
 
@@ -649,11 +648,11 @@ static void test_retry_flush(const char *machine)
     qmp_discard_response(s);
 
     /* Check registers */
-    data = qpci_io_readb(dev, ide_base + reg_device);
+    data = qpci_io_readb(dev, ide_bar, reg_device);
     g_assert_cmpint(data & DEV, ==, 0);
 
     do {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
     } while (data & BSY);
 
     assert_bit_set(data, DRDY);
@@ -665,15 +664,15 @@ static void test_retry_flush(const char *machine)
 static void test_flush_nodev(void)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
 
     ide_test_start("");
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* FLUSH CACHE command on device 0*/
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
 
     /* Just testing that qemu doesn't crash... */
 
@@ -700,7 +699,7 @@ typedef struct Read10CDB {
     uint16_t padding;
 } __attribute__((__packed__)) Read10CDB;
 
-static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
+static void send_scsi_cdb_read10(QPCIDevice *dev, QPCIBar ide_bar,
                                  uint64_t lba, int nblocks)
 {
     Read10CDB pkt = { .padding = 0 };
@@ -717,7 +716,7 @@ static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
 
     /* Send Packet */
     for (i = 0; i < sizeof(Read10CDB)/2; i++) {
-        qpci_io_writew(dev, ide_base + reg_data,
+        qpci_io_writew(dev, ide_bar, reg_data,
                        le16_to_cpu(((uint16_t *)&pkt)[i]));
     }
 }
@@ -732,16 +731,16 @@ static void nsleep(int64_t nsecs)
 static uint8_t ide_wait_clear(uint8_t flag)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     uint8_t data;
     time_t st;
 
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
 
     /* Wait with a 5 second timeout */
     time(&st);
     while (true) {
-        data = qpci_io_readb(dev, ide_base + reg_status);
+        data = qpci_io_readb(dev, ide_bar, reg_status);
         if (!(data & flag)) {
             return data;
         }
@@ -776,7 +775,7 @@ static void ide_wait_intr(int irq)
 static void cdrom_pio_impl(int nblocks)
 {
     QPCIDevice *dev;
-    void *bmdma_base, *ide_base;
+    QPCIBar bmdma_bar, ide_bar;
     FILE *fh;
     int patt_blocks = MAX(16, nblocks);
     size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
@@ -795,15 +794,14 @@ static void cdrom_pio_impl(int nblocks)
 
     ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
                    "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
-    dev = get_pci_device(&bmdma_base, &ide_base);
+    dev = get_pci_device(&bmdma_bar, &ide_bar);
     qtest_irq_intercept_in(global_qtest, "ioapic");
 
     /* PACKET command on device 0 */
-    qpci_io_writeb(dev, ide_base + reg_device, 0);
-    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
-    qpci_io_writeb(dev, ide_base + reg_lba_high,
-                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
-    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
+    qpci_io_writeb(dev, ide_bar, reg_device, 0);
+    qpci_io_writeb(dev, ide_bar, reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
+    qpci_io_writeb(dev, ide_bar, reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
+    qpci_io_writeb(dev, ide_bar, reg_command, CMD_PACKET);
     /* HP0: Check_Status_A State */
     nsleep(400);
     data = ide_wait_clear(BSY);
@@ -812,7 +810,7 @@ static void cdrom_pio_impl(int nblocks)
     assert_bit_clear(data, ERR | DF | BSY);
 
     /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
-    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
+    send_scsi_cdb_read10(dev, ide_bar, 0, nblocks);
 
     /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
      * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
@@ -836,8 +834,8 @@ static void cdrom_pio_impl(int nblocks)
 
         /* HP4: Transfer_Data */
         for (j = 0; j < MIN((limit / 2), rem); j++) {
-            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
-                                                       ide_base + reg_data));
+            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev, ide_bar,
+                                                       reg_data));
         }
     }
 
diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
index fbd8258..04a5c5d 100644
--- a/tests/ivshmem-test.c
+++ b/tests/ivshmem-test.c
@@ -41,7 +41,7 @@ static QPCIDevice *get_device(QPCIBus *pcibus)
 
 typedef struct _IVState {
     QTestState *qtest;
-    void *reg_base, *mem_base;
+    QPCIBar reg_bar, mem_bar;
     QPCIBus *pcibus;
     QPCIDevice *dev;
 } IVState;
@@ -75,7 +75,7 @@ static inline unsigned in_reg(IVState *s, enum Reg reg)
     unsigned res;
 
     global_qtest = s->qtest;
-    res = qpci_io_readl(s->dev, s->reg_base + reg);
+    res = qpci_io_readl(s->dev, s->reg_bar, reg);
     g_test_message("*%s -> %x\n", name, res);
     global_qtest = qtest;
 
@@ -89,7 +89,7 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
 
     global_qtest = s->qtest;
     g_test_message("%x -> *%s\n", v, name);
-    qpci_io_writel(s->dev, s->reg_base + reg, v);
+    qpci_io_writel(s->dev, s->reg_bar, reg, v);
     global_qtest = qtest;
 }
 
@@ -98,7 +98,7 @@ static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
     QTestState *qtest = global_qtest;
 
     global_qtest = s->qtest;
-    qpci_memread(s->dev, s->mem_base + off, buf, len);
+    qpci_memread(s->dev, s->mem_bar, off, buf, len);
     global_qtest = qtest;
 }
 
@@ -108,7 +108,7 @@ static inline void write_mem(IVState *s, uint64_t off,
     QTestState *qtest = global_qtest;
 
     global_qtest = s->qtest;
-    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
+    qpci_memwrite(s->dev, s->mem_bar, off, buf, len);
     global_qtest = qtest;
 }
 
@@ -127,16 +127,14 @@ static void setup_vm_cmd(IVState *s, const char *cmd, bool msix)
     s->pcibus = qpci_init_pc(NULL);
     s->dev = get_device(s->pcibus);
 
-    s->reg_base = qpci_iomap(s->dev, 0, &barsize);
-    g_assert_nonnull(s->reg_base);
+    s->reg_bar = qpci_iomap(s->dev, 0, &barsize);
     g_assert_cmpuint(barsize, ==, 256);
 
     if (msix) {
         qpci_msix_enable(s->dev);
     }
 
-    s->mem_base = qpci_iomap(s->dev, 2, &barsize);
-    g_assert_nonnull(s->mem_base);
+    s->mem_bar = qpci_iomap(s->dev, 2, &barsize);
     g_assert_cmpuint(barsize, ==, TMPSHMSIZE);
 
     qpci_device_enable(s->dev);
diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
index 8e789d8..5180d65 100644
--- a/tests/libqos/ahci.c
+++ b/tests/libqos/ahci.c
@@ -210,8 +210,7 @@ void ahci_pci_enable(AHCIQState *ahci)
 void start_ahci_device(AHCIQState *ahci)
 {
     /* Map AHCI's ABAR (BAR5) */
-    ahci->hba_base = qpci_iomap(ahci->dev, 5, &ahci->barsize);
-    g_assert(ahci->hba_base);
+    ahci->hba_bar = qpci_iomap(ahci->dev, 5, &ahci->barsize);
 
     /* turns on pci.cmd.iose, pci.cmd.mse and pci.cmd.bme */
     qpci_device_enable(ahci->dev);
diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
index 9b0c1d7..caaafe3 100644
--- a/tests/libqos/ahci.h
+++ b/tests/libqos/ahci.h
@@ -321,7 +321,7 @@ typedef struct AHCIPortQState {
 typedef struct AHCIQState {
     QOSState *parent;
     QPCIDevice *dev;
-    void *hba_base;
+    QPCIBar hba_bar;
     uint64_t barsize;
     uint32_t fingerprint;
     uint32_t cap;
@@ -489,12 +489,12 @@ typedef struct AHCIOpts {
 
 static inline uint32_t ahci_mread(AHCIQState *ahci, size_t offset)
 {
-    return qpci_io_readl(ahci->dev, ahci->hba_base + offset);
+    return qpci_io_readl(ahci->dev, ahci->hba_bar, offset);
 }
 
 static inline void ahci_mwrite(AHCIQState *ahci, size_t offset, uint32_t value)
 {
-    qpci_io_writel(ahci->dev, ahci->hba_base + offset, value);
+    qpci_io_writel(ahci->dev, ahci->hba_bar, offset, value);
 }
 
 static inline uint32_t ahci_rreg(AHCIQState *ahci, uint32_t reg_num)
diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
index 3021651..2dcdead 100644
--- a/tests/libqos/pci.c
+++ b/tests/libqos/pci.c
@@ -104,7 +104,6 @@ void qpci_msix_enable(QPCIDevice *dev)
     uint32_t table;
     uint8_t bir_table;
     uint8_t bir_pba;
-    void *offset;
 
     addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
     g_assert_cmphex(addr, !=, 0);
@@ -114,18 +113,16 @@ void qpci_msix_enable(QPCIDevice *dev)
 
     table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE);
     bir_table = table & PCI_MSIX_FLAGS_BIRMASK;
-    offset = qpci_iomap(dev, bir_table, NULL);
-    dev->msix_table = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
+    dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL);
+    dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
 
     table = qpci_config_readl(dev, addr + PCI_MSIX_PBA);
     bir_pba = table & PCI_MSIX_FLAGS_BIRMASK;
     if (bir_pba != bir_table) {
-        offset = qpci_iomap(dev, bir_pba, NULL);
+        dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL);
     }
-    dev->msix_pba = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
+    dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
 
-    g_assert(dev->msix_table != NULL);
-    g_assert(dev->msix_pba != NULL);
     dev->msix_enabled = true;
 }
 
@@ -141,22 +138,23 @@ void qpci_msix_disable(QPCIDevice *dev)
     qpci_config_writew(dev, addr + PCI_MSIX_FLAGS,
                                                 val & ~PCI_MSIX_FLAGS_ENABLE);
 
-    qpci_iounmap(dev, dev->msix_table);
-    qpci_iounmap(dev, dev->msix_pba);
+    qpci_iounmap(dev, dev->msix_table_bar);
+    qpci_iounmap(dev, dev->msix_pba_bar);
     dev->msix_enabled = 0;
-    dev->msix_table = NULL;
-    dev->msix_pba = NULL;
+    dev->msix_table_off = 0;
+    dev->msix_pba_off = 0;
 }
 
 bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry)
 {
     uint32_t pba_entry;
     uint8_t bit_n = entry % 32;
-    void *addr = dev->msix_pba + (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
+    uint64_t  off = (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
 
     g_assert(dev->msix_enabled);
-    pba_entry = qpci_io_readl(dev, addr);
-    qpci_io_writel(dev, addr, pba_entry & ~(1 << bit_n));
+    pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off);
+    qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off,
+                   pba_entry & ~(1 << bit_n));
     return (pba_entry & (1 << bit_n)) != 0;
 }
 
@@ -164,7 +162,7 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
 {
     uint8_t addr;
     uint16_t val;
-    void *vector_addr = dev->msix_table + (entry * PCI_MSIX_ENTRY_SIZE);
+    uint64_t vector_off = dev->msix_table_off + entry * PCI_MSIX_ENTRY_SIZE;
 
     g_assert(dev->msix_enabled);
     addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
@@ -174,8 +172,9 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
     if (val & PCI_MSIX_FLAGS_MASKALL) {
         return true;
     } else {
-        return (qpci_io_readl(dev, vector_addr + PCI_MSIX_ENTRY_VECTOR_CTRL)
-                                            & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
+        return (qpci_io_readl(dev, dev->msix_table_bar,
+                              vector_off + PCI_MSIX_ENTRY_VECTOR_CTRL)
+                & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
     }
 }
 
@@ -222,129 +221,115 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
     dev->bus->config_writel(dev->bus, dev->devfn, offset, value);
 }
 
-
-uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
+uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readb(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readb(dev->bus, token.addr + off);
     } else {
         uint8_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return val;
     }
 }
 
-uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
+uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readw(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readw(dev->bus, token.addr + off);
     } else {
         uint16_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le16_to_cpu(val);
     }
 }
 
-uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
+uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readl(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readl(dev->bus, token.addr + off);
     } else {
         uint32_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le32_to_cpu(val);
     }
 }
 
-uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
+uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        return dev->bus->pio_readq(dev->bus, addr);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        return dev->bus->pio_readq(dev->bus, token.addr + off);
     } else {
         uint64_t val;
-        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
+        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
         return le64_to_cpu(val);
     }
 }
 
-void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
+void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint8_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeb(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeb(dev->bus, token.addr + off, value);
     } else {
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
+void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint16_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writew(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writew(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le16(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
+void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint32_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writel(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writel(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le32(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
+void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint64_t value)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    if (addr < QPCI_PIO_LIMIT) {
-        dev->bus->pio_writeq(dev->bus, addr, value);
+    if (token.addr < QPCI_PIO_LIMIT) {
+        dev->bus->pio_writeq(dev->bus, token.addr + off, value);
     } else {
         value = cpu_to_le64(value);
-        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
+        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
     }
 }
 
-void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
+void qpci_memread(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                  void *buf, size_t len)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    g_assert(addr >= QPCI_PIO_LIMIT);
-    dev->bus->memread(dev->bus, addr, buf, len);
+    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    dev->bus->memread(dev->bus, token.addr + off, buf, len);
 }
 
-void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
+void qpci_memwrite(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                   const void *buf, size_t len)
 {
-    uintptr_t addr = (uintptr_t)data;
-
-    g_assert(addr >= QPCI_PIO_LIMIT);
-    dev->bus->memwrite(dev->bus, addr, buf, len);
+    g_assert(token.addr >= QPCI_PIO_LIMIT);
+    dev->bus->memwrite(dev->bus, token.addr + off, buf, len);
 }
 
-void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
+QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
 {
     QPCIBus *bus = dev->bus;
     static const int bar_reg_map[] = {
         PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
         PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
     };
+    QPCIBar bar;
     int bar_reg;
     uint32_t addr, size;
     uint32_t io_type;
@@ -391,17 +376,19 @@ void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
         qpci_config_writel(dev, bar_reg, loc);
     }
 
-    return (void *)(uintptr_t)loc;
+    bar.addr = loc;
+    return bar;
 }
 
-void qpci_iounmap(QPCIDevice *dev, void *data)
+void qpci_iounmap(QPCIDevice *dev, QPCIBar bar)
 {
     /* FIXME */
 }
 
-void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
+QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
 {
-    return (void *)(uintptr_t)addr;
+    QPCIBar bar = { .addr = addr };
+    return bar;
 }
 
 void qpci_plug_device_test(const char *driver, const char *id,
diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
index 531e3f7..ed48061 100644
--- a/tests/libqos/pci.h
+++ b/tests/libqos/pci.h
@@ -21,6 +21,7 @@
 
 typedef struct QPCIDevice QPCIDevice;
 typedef struct QPCIBus QPCIBus;
+typedef struct QPCIBar QPCIBar;
 
 struct QPCIBus {
     uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
@@ -51,13 +52,17 @@ struct QPCIBus {
     uint64_t mmio_alloc_ptr, mmio_limit;
 };
 
+struct QPCIBar {
+    uint64_t addr;
+};
+
 struct QPCIDevice
 {
     QPCIBus *bus;
     int devfn;
     bool msix_enabled;
-    void *msix_table;
-    void *msix_pba;
+    QPCIBar msix_table_bar, msix_pba_bar;
+    uint64_t msix_table_off, msix_pba_off;
 };
 
 void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
@@ -81,22 +86,27 @@ void qpci_config_writeb(QPCIDevice *dev, uint8_t offset, uint8_t value);
 void qpci_config_writew(QPCIDevice *dev, uint8_t offset, uint16_t value);
 void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
 
-uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
-uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
-uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
-uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
-
-void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
-void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
-void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
-void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
-
-void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
-void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
-
-void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
-void qpci_iounmap(QPCIDevice *dev, void *data);
-void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
+uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off);
+uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off);
+
+void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint8_t value);
+void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint16_t value);
+void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint32_t value);
+void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
+                    uint64_t value);
+
+void qpci_memread(QPCIDevice *bus, QPCIBar token, uint64_t off,
+                  void *buf, size_t len);
+void qpci_memwrite(QPCIDevice *bus, QPCIBar token, uint64_t off,
+                   const void *buf, size_t len);
+QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
+void qpci_iounmap(QPCIDevice *dev, QPCIBar addr);
+QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
 
 void qpci_plug_device_test(const char *driver, const char *id,
                            uint8_t slot, const char *opts);
diff --git a/tests/libqos/usb.c b/tests/libqos/usb.c
index f794d92..72d7a96 100644
--- a/tests/libqos/usb.c
+++ b/tests/libqos/usb.c
@@ -21,14 +21,12 @@ void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc, uint32_t devfn, int bar)
     hc->dev = qpci_device_find(pcibus, devfn);
     g_assert(hc->dev != NULL);
     qpci_device_enable(hc->dev);
-    hc->base = qpci_iomap(hc->dev, bar, NULL);
-    g_assert(hc->base != NULL);
+    hc->bar = qpci_iomap(hc->dev, bar, NULL);
 }
 
 void uhci_port_test(struct qhc *hc, int port, uint16_t expect)
 {
-    void *addr = hc->base + 0x10 + 2 * port;
-    uint16_t value = qpci_io_readw(hc->dev, addr);
+    uint16_t value = qpci_io_readw(hc->dev, hc->bar, 0x10 + 2 * port);
     uint16_t mask = ~(UHCI_PORT_WRITE_CLEAR | UHCI_PORT_RSVD1);
 
     g_assert((value & mask) == (expect & mask));
diff --git a/tests/libqos/usb.h b/tests/libqos/usb.h
index 8fe5687..423dcfd 100644
--- a/tests/libqos/usb.h
+++ b/tests/libqos/usb.h
@@ -5,7 +5,7 @@
 
 struct qhc {
     QPCIDevice *dev;
-    void *base;
+    QPCIBar bar;
 };
 
 void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc,
diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
index c69d09d..d4bf841 100644
--- a/tests/libqos/virtio-pci.c
+++ b/tests/libqos/virtio-pci.c
@@ -62,13 +62,12 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
     *vpcidev = (QVirtioPCIDevice *)d;
 }
 
-#define CONFIG_BASE(dev) \
-    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
+#define CONFIG_BASE(dev) (VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
 
 static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
+    return qpci_io_readb(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
 }
 
 /* PCI is always read in little-endian order
@@ -84,7 +83,7 @@ static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint16_t value;
 
-    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
+    value = qpci_io_readw(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap16(value);
     }
@@ -96,7 +95,7 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint32_t value;
 
-    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
+    value = qpci_io_readl(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         value = bswap32(value);
     }
@@ -108,7 +107,7 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
     uint64_t val;
 
-    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
+    val = qpci_io_readq(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
     if (qvirtio_is_big_endian(d)) {
         val = bswap64(val);
     }
@@ -119,31 +118,31 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
 static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_HOST_FEATURES);
+    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_HOST_FEATURES);
 }
 
 static void qvirtio_pci_set_features(QVirtioDevice *d, uint32_t features)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES, features);
+    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES, features);
 }
 
 static uint32_t qvirtio_pci_get_guest_features(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES);
+    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES);
 }
 
 static uint8_t qvirtio_pci_get_status(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS);
+    return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS);
 }
 
 static void qvirtio_pci_set_status(QVirtioDevice *d, uint8_t status)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS, status);
+    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS, status);
 }
 
 static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
@@ -167,7 +166,7 @@ static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
             }
         }
     } else {
-        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 1;
+        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 1;
     }
 }
 
@@ -191,26 +190,26 @@ static bool qvirtio_pci_get_config_isr_status(QVirtioDevice *d)
             }
         }
     } else {
-        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 2;
+        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 2;
     }
 }
 
 static void qvirtio_pci_queue_select(QVirtioDevice *d, uint16_t index)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_SEL, index);
+    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_SEL, index);
 }
 
 static uint16_t qvirtio_pci_get_queue_size(QVirtioDevice *d)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    return qpci_io_readw(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NUM);
+    return qpci_io_readw(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NUM);
 }
 
 static void qvirtio_pci_set_queue_address(QVirtioDevice *d, uint32_t pfn)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_PFN, pfn);
+    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_PFN, pfn);
 }
 
 static QVirtQueue *qvirtio_pci_virtqueue_setup(QVirtioDevice *d,
@@ -262,7 +261,7 @@ static void qvirtio_pci_virtqueue_cleanup(QVirtQueue *vq,
 static void qvirtio_pci_virtqueue_kick(QVirtioDevice *d, QVirtQueue *vq)
 {
     QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
-    qpci_io_writew(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
+    qpci_io_writew(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
 }
 
 const QVirtioBus qvirtio_pci = {
@@ -309,14 +308,12 @@ QVirtioPCIDevice *qvirtio_pci_device_find(QPCIBus *bus, uint16_t device_type)
 void qvirtio_pci_device_enable(QVirtioPCIDevice *d)
 {
     qpci_device_enable(d->pdev);
-    d->addr = qpci_iomap(d->pdev, 0, NULL);
-    g_assert(d->addr != NULL);
+    d->bar = qpci_iomap(d->pdev, 0, NULL);
 }
 
 void qvirtio_pci_device_disable(QVirtioPCIDevice *d)
 {
-    qpci_iounmap(d->pdev, d->addr);
-    d->addr = NULL;
+    qpci_iounmap(d->pdev, d->bar);
 }
 
 void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
@@ -324,29 +321,33 @@ void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
 {
     uint16_t vector;
     uint32_t control;
-    void *addr;
+    uint64_t off;
 
     g_assert(d->pdev->msix_enabled);
-    addr = d->pdev->msix_table + (entry * 16);
+    off = d->pdev->msix_table_off + (entry * 16);
 
     g_assert_cmpint(entry, >=, 0);
     g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
     vqpci->msix_entry = entry;
 
     vqpci->msix_addr = guest_alloc(alloc, 4);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
-                                                    vqpci->msix_addr & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
-                                            (vqpci->msix_addr >> 32) & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
-
-    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
-                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_LOWER_ADDR, vqpci->msix_addr & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
+                   (vqpci->msix_addr >> 32) & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
+
+    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
+                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
+                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
 
     qvirtio_pci_queue_select(&d->vdev, vqpci->vq.index);
-    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR, entry);
-    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR);
+    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR, entry);
+    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR);
     g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
 }
 
@@ -355,10 +356,10 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
 {
     uint16_t vector;
     uint32_t control;
-    void *addr;
+    uint64_t off;
 
     g_assert(d->pdev->msix_enabled);
-    addr = d->pdev->msix_table + (entry * 16);
+    off = d->pdev->msix_table_off + (entry * 16);
 
     g_assert_cmpint(entry, >=, 0);
     g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
@@ -367,17 +368,21 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
     d->config_msix_data = 0x12345678;
     d->config_msix_addr = guest_alloc(alloc, 4);
 
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
-                                                    d->config_msix_addr & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
-                                            (d->config_msix_addr >> 32) & ~0UL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
-
-    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
-    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
-                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
-
-    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR, entry);
-    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_LOWER_ADDR, d->config_msix_addr & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
+                   (d->config_msix_addr >> 32) & ~0UL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
+
+    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
+                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
+    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
+                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
+                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
+
+    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR, entry);
+    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR);
     g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
 }
diff --git a/tests/libqos/virtio-pci.h b/tests/libqos/virtio-pci.h
index efcac2d..38c54c6 100644
--- a/tests/libqos/virtio-pci.h
+++ b/tests/libqos/virtio-pci.h
@@ -16,7 +16,7 @@
 typedef struct QVirtioPCIDevice {
     QVirtioDevice vdev;
     QPCIDevice *pdev;
-    void *addr;
+    QPCIBar bar;
     uint16_t config_msix_entry;
     uint64_t config_msix_addr;
     uint32_t config_msix_data;
diff --git a/tests/rtl8139-test.c b/tests/rtl8139-test.c
index c2f601a..7de7dc4 100644
--- a/tests/rtl8139-test.c
+++ b/tests/rtl8139-test.c
@@ -22,7 +22,7 @@ static void nop(void)
 
 static QPCIBus *pcibus;
 static QPCIDevice *dev;
-static void *dev_base;
+static QPCIBar dev_bar;
 
 static void save_fn(QPCIDevice *dev, int devfn, void *data)
 {
@@ -45,14 +45,14 @@ static QPCIDevice *get_device(void)
 #define PORT(name, len, val) \
 static unsigned __attribute__((unused)) in_##name(void) \
 { \
-    unsigned res = qpci_io_read##len(dev, dev_base+(val)); \
+    unsigned res = qpci_io_read##len(dev, dev_bar, (val));     \
     g_test_message("*%s -> %x\n", #name, res); \
     return res; \
 } \
 static void out_##name(unsigned v) \
 { \
     g_test_message("%x -> *%s\n", v, #name); \
-    qpci_io_write##len(dev, dev_base+(val), v); \
+    qpci_io_write##len(dev, dev_bar, (val), v);        \
 }
 
 PORT(Timer, l, 0x48)
@@ -186,9 +186,7 @@ static void test_init(void)
 
     dev = get_device();
 
-    dev_base = qpci_iomap(dev, 0, &barsize);
-
-    g_assert(dev_base != NULL);
+    dev_bar = qpci_iomap(dev, 0, &barsize);
 
     qpci_device_enable(dev);
 
diff --git a/tests/tco-test.c b/tests/tco-test.c
index 129577d..ef02ec5 100644
--- a/tests/tco-test.c
+++ b/tests/tco-test.c
@@ -41,7 +41,7 @@ typedef struct {
     const char *args;
     bool noreboot;
     QPCIDevice *dev;
-    void *tco_io_base;
+    QPCIBar tco_io_bar;
 } TestData;
 
 static void test_init(TestData *d)
@@ -70,42 +70,42 @@ static void test_init(TestData *d)
     /* set Root Complex BAR */
     qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
 
-    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
+    d->tco_io_bar = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
 }
 
 static void stop_tco(const TestData *d)
 {
     uint32_t val;
 
-    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
+    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
     val |= TCO_TMR_HLT;
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
 }
 
 static void start_tco(const TestData *d)
 {
     uint32_t val;
 
-    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
+    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
     val &= ~TCO_TMR_HLT;
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
 }
 
 static void load_tco(const TestData *d)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO_RLD, 4);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO_RLD, 4);
 }
 
 static void set_tco_timeout(const TestData *d, uint16_t ticks)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO_TMR, ticks);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO_TMR, ticks);
 }
 
 static void clear_tco_status(const TestData *d)
 {
-    qpci_io_writew(d->dev, d->tco_io_base + TCO1_STS, 0x0008);
-    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0002);
-    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0004);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_STS, 0x0008);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0002);
+    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0004);
 }
 
 static void reset_on_second_timeout(bool enable)
@@ -128,25 +128,25 @@ static void test_tco_defaults(void)
     d.args = NULL;
     d.noreboot = true;
     test_init(&d);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD), ==,
                     TCO_RLD_DEFAULT);
     /* TCO_DAT_IN & TCO_DAT_OUT */
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_DAT_IN), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_DAT_IN), ==,
                     (TCO_DAT_OUT_DEFAULT << 8) | TCO_DAT_IN_DEFAULT);
     /* TCO1_STS & TCO2_STS */
-    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_STS), ==,
+    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_STS), ==,
                     (TCO2_STS_DEFAULT << 16) | TCO1_STS_DEFAULT);
     /* TCO1_CNT & TCO2_CNT */
-    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_CNT), ==,
+    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_CNT), ==,
                     (TCO2_CNT_DEFAULT << 16) | TCO1_CNT_DEFAULT);
     /* TCO_MESSAGE1 & TCO_MESSAGE2 */
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_MESSAGE1), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_MESSAGE1), ==,
                     (TCO_MESSAGE2_DEFAULT << 8) | TCO_MESSAGE1_DEFAULT);
-    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + TCO_WDCNT), ==,
+    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, TCO_WDCNT), ==,
                     TCO_WDCNT_DEFAULT);
-    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + SW_IRQ_GEN), ==,
+    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, SW_IRQ_GEN), ==,
                     SW_IRQ_GEN_DEFAULT);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_TMR), ==,
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_TMR), ==,
                     TCO_TMR_DEFAULT);
     qtest_end();
 }
@@ -171,23 +171,23 @@ static void test_tco_timeout(void)
     clock_step(ticks * TCO_TICK_NSEC);
 
     /* test first timeout */
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
 
     /* test clearing timeout bit */
     val |= TCO_TIMEOUT;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 0);
 
     /* test second timeout */
     clock_step(ticks * TCO_TICK_NSEC);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
     ret = val & TCO_SECOND_TO_STS ? 1 : 0;
     g_assert(ret == 1);
 
@@ -214,13 +214,13 @@ static void test_tco_max_timeout(void)
     start_tco(&d);
     clock_step(((ticks & TCO_TMR_MASK) - 1) * TCO_TICK_NSEC);
 
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD);
     g_assert_cmpint(val & TCO_RLD_MASK, ==, 1);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 0);
     clock_step(TCO_TICK_NSEC);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & TCO_TIMEOUT ? 1 : 0;
     g_assert(ret == 1);
 
@@ -358,11 +358,11 @@ static void test_tco_ticks_counter(void)
     start_tco(&d);
 
     do {
-        rld = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD) & TCO_RLD_MASK;
+        rld = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD) & TCO_RLD_MASK;
         g_assert_cmpint(rld, ==, ticks);
         clock_step(TCO_TICK_NSEC);
         ticks--;
-    } while (!(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS) & TCO_TIMEOUT));
+    } while (!(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS) & TCO_TIMEOUT));
 
     stop_tco(&d);
     qtest_end();
@@ -378,10 +378,10 @@ static void test_tco1_control_bits(void)
     test_init(&d);
 
     val = TCO_LOCK;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
     val &= ~TCO_LOCK;
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_CNT), ==,
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_CNT), ==,
                     TCO_LOCK);
     qtest_end();
 }
@@ -405,13 +405,13 @@ static void test_tco1_status_bits(void)
     start_tco(&d);
     clock_step(ticks * TCO_TICK_NSEC);
 
-    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_IN, 0);
-    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_OUT, 0);
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
+    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_IN, 0);
+    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_OUT, 0);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
     ret = val & (TCO_TIMEOUT | SW_TCO_SMI | TCO_INT_STS) ? 1 : 0;
     g_assert(ret == 1);
-    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS), ==, 0);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS), ==, 0);
     qtest_end();
 }
 
@@ -434,11 +434,11 @@ static void test_tco2_status_bits(void)
     start_tco(&d);
     clock_step(ticks * TCO_TICK_NSEC * 2);
 
-    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
+    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
     ret = val & (TCO_SECOND_TO_STS | TCO_BOOT_STS) ? 1 : 0;
     g_assert(ret == 1);
-    qpci_io_writew(d.dev, d.tco_io_base + TCO2_STS, val);
-    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS), ==, 0);
+    qpci_io_writew(d.dev, d.tco_io_bar, TCO2_STS, val);
+    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS), ==, 0);
     qtest_end();
 }
 
diff --git a/tests/usb-hcd-ehci-test.c b/tests/usb-hcd-ehci-test.c
index a4ceeaa..57af8a0 100644
--- a/tests/usb-hcd-ehci-test.c
+++ b/tests/usb-hcd-ehci-test.c
@@ -38,8 +38,7 @@ static void uhci_port_update(struct qhc *hc, int port,
 
 static void ehci_port_test(struct qhc *hc, int port, uint32_t expect)
 {
-    void *addr = hc->base + 0x64 + 4 * port;
-    uint32_t value = qpci_io_readl(hc->dev, addr);
+    uint32_t value = qpci_io_readl(hc->dev, hc->bar, 0x64 + 4 * port);
     uint16_t mask = ~(PORTSC_CSC | PORTSC_PEDC | PORTSC_OCC);
 
 #if 0
@@ -91,7 +90,7 @@ static void pci_ehci_port_1(void)
 static void pci_ehci_config(void)
 {
     /* hands over all ports from companion uhci to ehci */
-    qpci_io_writew(ehci1.dev, ehci1.base + 0x60, 1);
+    qpci_io_writew(ehci1.dev, ehci1.bar, 0x60, 1);
 }
 
 static void pci_uhci_port_2(void)
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test David Gibson
@ 2016-10-24 16:50   ` John Snow
  2016-10-25  8:58   ` Greg Kurz
  1 sibling, 0 replies; 31+ messages in thread
From: John Snow @ 2016-10-24 16:50 UTC (permalink / raw)
  To: David Gibson, pbonzini, qemu-devel
  Cc: lvivier, thuth, mst, aik, agraf, mdroth, groug, qemu-ppc, stefanha



On 10/24/2016 01:00 AM, David Gibson wrote:
> In a couple of places ahci-test makes assumptions about how the tokens
> returned from qpci_iomap() are formatted in ways it probably shouldn't.
>
> First in verify_state() it uses a non-NULL token to indicate that the AHCI
> device has been enabled (part of enabling is to iomap()).  This changes it
> to use an explicit 'enabled' flag instead.
>
> Second, it uses the fact that the token contains a PCI address, stored when
> the BAR is mapped during initialization to check that the BAR has the same
> value after a migration.  This changes it to explicitly read the BAR
> register before and after the migration and compare.
>
> Together, these changes will  make the test more robust against changes to
> the internals of the libqos PCI layer.
>
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  tests/ahci-test.c   | 13 +++++++------
>  tests/libqos/ahci.c |  1 +
>  tests/libqos/ahci.h |  1 +
>  3 files changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> index 9c0adce..70bcafa 100644
> --- a/tests/ahci-test.c
> +++ b/tests/ahci-test.c
> @@ -78,25 +78,23 @@ static void string_bswap16(uint16_t *s, size_t bytes)
>  /**
>   * Verify that the transfer did not corrupt our state at all.
>   */
> -static void verify_state(AHCIQState *ahci)
> +static void verify_state(AHCIQState *ahci, uint64_t hba_old)
>  {
>      int i, j;
>      uint32_t ahci_fingerprint;
>      uint64_t hba_base;
> -    uint64_t hba_stored;
>      AHCICommandHeader cmd;
>
>      ahci_fingerprint = qpci_config_readl(ahci->dev, PCI_VENDOR_ID);
>      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
>
>      /* If we haven't initialized, this is as much as can be validated. */
> -    if (!ahci->hba_base) {
> +    if (!ahci->enabled) {
>          return;
>      }
>
>      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> -    g_assert_cmphex(hba_base, ==, hba_stored);
> +    g_assert_cmphex(hba_base, ==, hba_old);
>
>      g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP), ==, ahci->cap);
>      g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP2), ==, ahci->cap2);
> @@ -119,12 +117,15 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
>      QOSState *tmp = to->parent;
>      QPCIDevice *dev = to->dev;
>      char *uri_local = NULL;
> +    uint64_t hba_old;
>
>      if (uri == NULL) {
>          uri_local = g_strdup_printf("%s%s", "unix:", mig_socket);
>          uri = uri_local;
>      }
>
> +    hba_old = (uint64_t)qpci_config_readl(from->dev, PCI_BASE_ADDRESS_5);
> +
>      /* context will be 'to' after completion. */
>      migrate(from->parent, to->parent, uri);
>
> @@ -141,7 +142,7 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
>      from->parent = tmp;
>      from->dev = dev;
>
> -    verify_state(to);
> +    verify_state(to, hba_old);
>      g_free(uri_local);
>  }
>
> diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
> index 716ab79..8e789d8 100644
> --- a/tests/libqos/ahci.c
> +++ b/tests/libqos/ahci.c
> @@ -351,6 +351,7 @@ void ahci_hba_enable(AHCIQState *ahci)
>      reg = ahci_rreg(ahci, AHCI_GHC);
>      ASSERT_BIT_SET(reg, AHCI_GHC_IE);
>
> +    ahci->enabled = true;
>      /* TODO: The device should now be idling and waiting for commands.
>       * In the future, a small test-case to inspect the Register D2H FIS
>       * and clear the initial interrupts might be good. */
> diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
> index c69fb5a..9b0c1d7 100644
> --- a/tests/libqos/ahci.h
> +++ b/tests/libqos/ahci.h
> @@ -327,6 +327,7 @@ typedef struct AHCIQState {
>      uint32_t cap;
>      uint32_t cap2;
>      AHCIPortQState port[32];
> +    bool enabled;
>  } AHCIQState;
>
>  /**
>

Reviewed-by: John Snow <jsnow@redhat.com>

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling
  2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
                   ` (11 preceding siblings ...)
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle David Gibson
@ 2016-10-25  3:35 ` David Gibson
  2016-10-25 13:14   ` Greg Kurz
  12 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-25  3:35 UTC (permalink / raw)
  To: pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, aik, mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 3835 bytes --]

On Mon, Oct 24, 2016 at 03:59:49PM +1100, David Gibson wrote:
> This series contains a number of cleanups to the libqos code for
> accessing PCI devices, and to tests which use it.
> 
> The general aim is to improve the consistency of semantics across
> functions, and reduce the amount of intimate knowledge of the libqos
> PCI layer needed by tests.
> 
> This should make it easier to write PCI tests which will be portable
> to different guest machines with different PCI host bridge
> arrangements.
> 
> This series is on top of my ppc-for-2.8 branch, since it contains
> patches enabling the virtio tests on ppc, which would otherwise
> conflict with the changes here.

Greg, Alexey, Michael,

Some reviews from outside RH would be really welcome.

> 
> Changes since v4:
>   * Fixed some remaining abstraction breaks in ahci-test
>   * Removed QPCI_BAR_INVALID, turned out not to really be useful
> 
> Changes since v3:
>   * Fixed another endian bug introduced in ide-test
> 
> Changes since v2:
>   * Fixed build bugs in virtio-9p-test, which I didn't find earlier
>     due to not having the right libraries installed
>   * Fixed an endian bug I accidentally introduced in ide-test
>   * Better handling of invalid BAR tokens
> 
> Changes since v1:
>   * Split out updates to tco-test into separate patch
>   * Split out updates to ide-test into separate patch
>   * Neater and more general handling of legacy PIO addresses
>   * Removed now-redundant fields from platform specific bus structures
>   * Introduced CONFIG_BASE() macro to virtio-pci to remove many
>     similar assignments
>   * Fixed handling of two guest testcasesin ivshmem
>   * Added 64-bit accessors
>   * Rebase on ppc-for-2.8 to avoid conflict with Laurent's series in
>     the same area
> 
> David Gibson (12):
>   libqos: Give qvirtio_config_read*() consistent semantics
>   libqos: Handle PCI IO de-multiplexing in common code
>   libqos: Move BAR assignment to common code
>   libqos: Better handling of PCI legacy IO
>   tests: Adjust tco-test to use qpci_legacy_iomap()
>   libqos: Add streaming accessors for PCI MMIO
>   libqos: Implement mmio accessors in terms of mem{read,write}
>   tests: Clean up IO handling in ide-test
>   libqos: Add 64-bit PCI IO accessors
>   tests: Use qpci_mem{read,write} in ivshmem-test
>   tests: Don't assume structure of PCI IO base in ahci-test
>   libqos: Change PCI accessors to take opaque BAR handle
> 
>  tests/ahci-test.c          |  13 +--
>  tests/e1000e-test.c        |   7 +-
>  tests/ide-test.c           | 177 +++++++++++++++++++++++++++--------------
>  tests/ivshmem-test.c       |  47 +++++++----
>  tests/libqos/ahci.c        |   4 +-
>  tests/libqos/ahci.h        |   7 +-
>  tests/libqos/pci-pc.c      | 187 ++++++++++---------------------------------
>  tests/libqos/pci-spapr.c   | 194 ++++++++++++---------------------------------
>  tests/libqos/pci.c         | 194 +++++++++++++++++++++++++++++++++++++--------
>  tests/libqos/pci.h         |  66 ++++++++++-----
>  tests/libqos/usb.c         |   6 +-
>  tests/libqos/usb.h         |   2 +-
>  tests/libqos/virtio-mmio.c |  16 ++--
>  tests/libqos/virtio-pci.c  | 122 ++++++++++++++--------------
>  tests/libqos/virtio-pci.h  |   2 +-
>  tests/rtl8139-test.c       |  10 +--
>  tests/tco-test.c           |  80 +++++++++----------
>  tests/usb-hcd-ehci-test.c  |   5 +-
>  tests/virtio-9p-test.c     |   8 +-
>  tests/virtio-blk-test.c    |  42 +++-------
>  tests/virtio-scsi-test.c   |   4 +-
>  21 files changed, 598 insertions(+), 595 deletions(-)
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
@ 2016-10-25  6:47   ` Alexey Kardashevskiy
  2016-10-25 12:16     ` David Gibson
  0 siblings, 1 reply; 31+ messages in thread
From: Alexey Kardashevskiy @ 2016-10-25  6:47 UTC (permalink / raw)
  To: David Gibson, pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, mdroth, groug, thuth

On 24/10/16 15:59, David Gibson wrote:
> In the libqos PCI code we now have accessors both for registers (byte
> significance preserving) and for streaming data (byte address order
> preserving).  These exist in both the interface for qtest drivers and in
> the machine specific backends.
> 
> However, the register-style accessors aren't actually necessary in the
> backend.  They can be implemented in terms of the byte address order
> preserving accessors by the libqos wrappers.  This works because PCI is
> always little endian.
> 
> This does assume that the back end byte address order preserving accessors
> will perform the equivalent of a single bus transaction for short lengths.
> This is the case, and in fact they currently end up using the same
> cpu_physical_memory_rw() implementation within the qtest accelerator.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> Reviewed-by: Greg Kurz <groug@kaod.org>
> ---
>  tests/libqos/pci-pc.c    | 38 --------------------------------------
>  tests/libqos/pci-spapr.c | 44 --------------------------------------------
>  tests/libqos/pci.c       | 20 ++++++++++++++------
>  tests/libqos/pci.h       |  8 --------
>  4 files changed, 14 insertions(+), 96 deletions(-)
> 

[...]

> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index 2b08362..ce6ed08 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -27,18 +27,10 @@ struct QPCIBus {
>      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
>      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
>  
> -    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
> -    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
> -    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
> -
>      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
>      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
>      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
>  
> -    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> -    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> -    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> -
>      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
>      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
>  
> 

You added them in "libqos: Handle PCI IO de-multiplexing in common code"
(few patched before) and removing them now - if you moved this patch
earlier, it would reduce the series, or what do I miss?




-- 
Alexey

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test David Gibson
@ 2016-10-25  7:01   ` Alexey Kardashevskiy
  2016-10-25 12:25     ` David Gibson
  2016-10-25 12:36   ` Greg Kurz
  1 sibling, 1 reply; 31+ messages in thread
From: Alexey Kardashevskiy @ 2016-10-25  7:01 UTC (permalink / raw)
  To: David Gibson, pbonzini, qemu-devel
  Cc: qemu-ppc, lvivier, agraf, stefanha, mst, mdroth, groug, thuth

On 24/10/16 15:59, David Gibson wrote:
> ide-test uses many explicit inb() / outb() operations for its IO, which
> means it's not portable to non-x86 platforms.  This cleans it up to use
> the libqos PCI accessors instead.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---
>  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
>  1 file changed, 118 insertions(+), 61 deletions(-)
> 
> diff --git a/tests/ide-test.c b/tests/ide-test.c
> index a8a4081..86c4373 100644
> --- a/tests/ide-test.c
> +++ b/tests/ide-test.c
> @@ -137,7 +137,7 @@ static void ide_test_quit(void)
>      qtest_end();
>  }
>  
> -static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
> +static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
>  {
>      QPCIDevice *dev;
>      uint16_t vendor_id, device_id;
> @@ -156,7 +156,9 @@ static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
>      g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
>  
>      /* Map bmdma BAR */
> -    *bmdma_base = (uint16_t)(uintptr_t) qpci_iomap(dev, 4, NULL);
> +    *bmdma_base = qpci_iomap(dev, 4, NULL);
> +
> +    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
>  
>      qpci_device_enable(dev);
>  
> @@ -179,17 +181,19 @@ typedef struct PrdtEntry {
>  
>  static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>                              PrdtEntry *prdt, int prdt_entries,
> -                            void(*post_exec)(uint64_t sector, int nb_sectors))
> +                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
> +                                             uint64_t sector, int nb_sectors))
>  {
>      QPCIDevice *dev;
> -    uint16_t bmdma_base;
> +    void *bmdma_base;
> +    void *ide_base;
>      uintptr_t guest_prdt;
>      size_t len;
>      bool from_dev;
>      uint8_t status;
>      int flags;
>  
> -    dev = get_pci_device(&bmdma_base);
> +    dev = get_pci_device(&bmdma_base, &ide_base);
>  
>      flags = cmd & ~0xff;
>      cmd &= 0xff;
> @@ -214,59 +218,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>      }
>  
>      /* Select device 0 */
> -    outb(IDE_BASE + reg_device, 0 | LBA);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
>  
>      /* Stop any running transfer, clear any pending interrupt */
> -    outb(bmdma_base + bmreg_cmd, 0);
> -    outb(bmdma_base + bmreg_status, BM_STS_INTR);
> +    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
>  
>      /* Setup PRDT */
>      len = sizeof(*prdt) * prdt_entries;
>      guest_prdt = guest_alloc(guest_malloc, len);
>      memwrite(guest_prdt, prdt, len);
> -    outl(bmdma_base + bmreg_prdt, guest_prdt);
> +    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
>  
>      /* ATA DMA command */
>      if (cmd == CMD_PACKET) {
>          /* Enables ATAPI DMA; otherwise PIO is attempted */
> -        outb(IDE_BASE + reg_feature, 0x01);
> +        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
>      } else {
> -        outb(IDE_BASE + reg_nsectors, nb_sectors);
> -        outb(IDE_BASE + reg_lba_low,    sector & 0xff);
> -        outb(IDE_BASE + reg_lba_middle, (sector >> 8) & 0xff);
> -        outb(IDE_BASE + reg_lba_high,   (sector >> 16) & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
> +        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
>      }
>  
> -    outb(IDE_BASE + reg_command, cmd);
> +    qpci_io_writeb(dev, ide_base + reg_command, cmd);
>  
>      if (post_exec) {
> -        post_exec(sector, nb_sectors);
> +        post_exec(dev, ide_base, sector, nb_sectors);
>      }
>  
>      /* Start DMA transfer */
> -    outb(bmdma_base + bmreg_cmd, BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
> +    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
> +                   BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
>  
>      if (flags & CMDF_ABORT) {
> -        outb(bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
>      }
>  
>      /* Wait for the DMA transfer to complete */
>      do {
> -        status = inb(bmdma_base + bmreg_status);
> +        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
>      } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
>  
>      g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
>  
>      /* Check IDE status code */
> -    assert_bit_set(inb(IDE_BASE + reg_status), DRDY);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), BSY | DRQ);
> +    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
>  
>      /* Reading the status register clears the IRQ */
>      g_assert(!get_irq(IDE_PRIMARY_IRQ));
>  
>      /* Stop DMA transfer if still active */
>      if (status & BM_STS_ACTIVE) {
> -        outb(bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
>      }
>  
>      free_pci_device(dev);
> @@ -276,6 +281,8 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>  
>  static void test_bmdma_simple_rw(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>      uint8_t *buf;
>      uint8_t *cmpbuf;
> @@ -289,6 +296,8 @@ static void test_bmdma_simple_rw(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      buf = g_malloc(len);
>      cmpbuf = g_malloc(len);
>  
> @@ -299,7 +308,7 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Write 0xaa pattern to sector 1 */
>      memset(buf, 0xaa, len);
> @@ -308,14 +317,14 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Read and verify 0x55 pattern in sector 0 */
>      memset(cmpbuf, 0x55, len);
>  
>      status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -325,7 +334,7 @@ static void test_bmdma_simple_rw(void)
>  
>      status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -337,6 +346,8 @@ static void test_bmdma_simple_rw(void)
>  
>  static void test_bmdma_short_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -346,21 +357,25 @@ static void test_bmdma_short_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_one_sector_short_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      /* Read 2 sectors but only give 1 sector in PRDT */
> @@ -371,21 +386,25 @@ static void test_bmdma_one_sector_short_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_long_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -395,23 +414,29 @@ static void test_bmdma_long_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_no_busmaster(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
>       * able to access it anyway because the Bus Master bit in the PCI command
>       * register isn't set. This is complete nonsense, but it used to be pretty
> @@ -424,7 +449,7 @@ static void test_bmdma_no_busmaster(void)
>      /* Not entirely clear what the expected result is, but this is what we get
>       * in practice. At least we want to be aware of any changes. */
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_setup(void)
> @@ -454,6 +479,8 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
>  
>  static void test_identify(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      uint16_t buf[256];
>      int i;
> @@ -464,23 +491,25 @@ static void test_identify(void)
>          "-global ide-hd.ver=%s",
>          tmp_path, "testdisk", "version");
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* IDENTIFY command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_IDENTIFY);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
>  
>      /* Read in the IDENTIFY buffer and check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      for (i = 0; i < 256; i++) {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>          assert_bit_set(data, DRDY | DRQ);
>          assert_bit_clear(data, BSY | DF | ERR);
>  
> -        ((uint16_t*) buf)[i] = inw(IDE_BASE + reg_data);
> +        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
>      }
>  
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, DRDY);
>      assert_bit_clear(data, BSY | DF | ERR | DRQ);
>  
> @@ -505,11 +534,15 @@ static void test_identify(void)
>   */
>  static void make_dirty(uint8_t device)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>      size_t len = 512;
>      uintptr_t guest_buf;
>      void* buf;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      guest_buf = guest_alloc(guest_malloc, len);
>      buf = g_malloc(len);
>      g_assert(guest_buf);
> @@ -527,19 +560,23 @@ static void make_dirty(uint8_t device)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      g_free(buf);
>  }
>  
>  static void test_flush(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>  
>      ide_test_start(
>          "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
>          tmp_path);
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
> @@ -549,11 +586,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
>  
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -561,11 +598,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
>  
>      /* Check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -576,6 +613,8 @@ static void test_flush(void)
>  
>  static void test_retry_flush(const char *machine)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      const char *s;
>  
> @@ -587,17 +626,19 @@ static void test_retry_flush(const char *machine)
>          "rerror=stop,werror=stop",
>          debug_path, tmp_path);
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
>      make_dirty(0);
>  
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -608,11 +649,11 @@ static void test_retry_flush(const char *machine)
>      qmp_discard_response(s);
>  
>      /* Check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -623,11 +664,16 @@ static void test_retry_flush(const char *machine)
>  
>  static void test_flush_nodev(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
> +
>      ide_test_start("");
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Just testing that qemu doesn't crash... */
>  
> @@ -654,7 +700,8 @@ typedef struct Read10CDB {
>      uint16_t padding;
>  } __attribute__((__packed__)) Read10CDB;
>  
> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> +                                 uint64_t lba, int nblocks)
>  {
>      Read10CDB pkt = { .padding = 0 };
>      int i;
> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>  
>      /* Send Packet */
>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> +        qpci_io_writew(dev, ide_base + reg_data,
> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));


cpu_to_le16 -> le16_to_cpu conversion here and below (at the very end) is
not obvious. Right above this chunk the @pkt fields are initialized as BE:

 /* Construct SCSI CDB packet */
 pkt.opcode = 0x28;
 pkt.lba = cpu_to_be32(lba);
 pkt.nblocks = cpu_to_be16(nblocks);

outw() seems to be CPU-endian, and qpci_io_writew() as well, or not?



>      }
>  }
>  
> @@ -683,13 +731,17 @@ static void nsleep(int64_t nsecs)
>  
>  static uint8_t ide_wait_clear(uint8_t flag)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      time_t st;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Wait with a 5 second timeout */
>      time(&st);
>      while (true) {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>          if (!(data & flag)) {
>              return data;
>          }
> @@ -723,6 +775,8 @@ static void ide_wait_intr(int irq)
>  
>  static void cdrom_pio_impl(int nblocks)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      FILE *fh;
>      int patt_blocks = MAX(16, nblocks);
>      size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
> @@ -741,13 +795,15 @@ static void cdrom_pio_impl(int nblocks)
>  
>      ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
>                     "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
> +    dev = get_pci_device(&bmdma_base, &ide_base);
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* PACKET command on device 0 */
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> -    outb(IDE_BASE + reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> -    outb(IDE_BASE + reg_command, CMD_PACKET);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> +    qpci_io_writeb(dev, ide_base + reg_lba_high,
> +                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
>      /* HP0: Check_Status_A State */
>      nsleep(400);
>      data = ide_wait_clear(BSY);
> @@ -756,7 +812,7 @@ static void cdrom_pio_impl(int nblocks)
>      assert_bit_clear(data, ERR | DF | BSY);
>  
>      /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
> -    send_scsi_cdb_read10(0, nblocks);
> +    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
>  
>      /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
>       * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
> @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
>  
>          /* HP4: Transfer_Data */
>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> +                                                       ide_base + reg_data));
>          }
>      }
>  
> 


-- 
Alexey

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test David Gibson
  2016-10-24 16:50   ` John Snow
@ 2016-10-25  8:58   ` Greg Kurz
  1 sibling, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25  8:58 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 16:00:00 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> In a couple of places ahci-test makes assumptions about how the tokens
> returned from qpci_iomap() are formatted in ways it probably shouldn't.
> 
> First in verify_state() it uses a non-NULL token to indicate that the AHCI
> device has been enabled (part of enabling is to iomap()).  This changes it
> to use an explicit 'enabled' flag instead.
> 
> Second, it uses the fact that the token contains a PCI address, stored when
> the BAR is mapped during initialization to check that the BAR has the same
> value after a migration.  This changes it to explicitly read the BAR
> register before and after the migration and compare.
> 
> Together, these changes will  make the test more robust against changes to
> the internals of the libqos PCI layer.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/ahci-test.c   | 13 +++++++------
>  tests/libqos/ahci.c |  1 +
>  tests/libqos/ahci.h |  1 +
>  3 files changed, 9 insertions(+), 6 deletions(-)
> 
> diff --git a/tests/ahci-test.c b/tests/ahci-test.c
> index 9c0adce..70bcafa 100644
> --- a/tests/ahci-test.c
> +++ b/tests/ahci-test.c
> @@ -78,25 +78,23 @@ static void string_bswap16(uint16_t *s, size_t bytes)
>  /**
>   * Verify that the transfer did not corrupt our state at all.
>   */
> -static void verify_state(AHCIQState *ahci)
> +static void verify_state(AHCIQState *ahci, uint64_t hba_old)
>  {
>      int i, j;
>      uint32_t ahci_fingerprint;
>      uint64_t hba_base;
> -    uint64_t hba_stored;
>      AHCICommandHeader cmd;
>  
>      ahci_fingerprint = qpci_config_readl(ahci->dev, PCI_VENDOR_ID);
>      g_assert_cmphex(ahci_fingerprint, ==, ahci->fingerprint);
>  
>      /* If we haven't initialized, this is as much as can be validated. */
> -    if (!ahci->hba_base) {
> +    if (!ahci->enabled) {
>          return;
>      }
>  
>      hba_base = (uint64_t)qpci_config_readl(ahci->dev, PCI_BASE_ADDRESS_5);
> -    hba_stored = (uint64_t)(uintptr_t)ahci->hba_base;
> -    g_assert_cmphex(hba_base, ==, hba_stored);
> +    g_assert_cmphex(hba_base, ==, hba_old);
>  
>      g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP), ==, ahci->cap);
>      g_assert_cmphex(ahci_rreg(ahci, AHCI_CAP2), ==, ahci->cap2);
> @@ -119,12 +117,15 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
>      QOSState *tmp = to->parent;
>      QPCIDevice *dev = to->dev;
>      char *uri_local = NULL;
> +    uint64_t hba_old;
>  
>      if (uri == NULL) {
>          uri_local = g_strdup_printf("%s%s", "unix:", mig_socket);
>          uri = uri_local;
>      }
>  
> +    hba_old = (uint64_t)qpci_config_readl(from->dev, PCI_BASE_ADDRESS_5);
> +
>      /* context will be 'to' after completion. */
>      migrate(from->parent, to->parent, uri);
>  
> @@ -141,7 +142,7 @@ static void ahci_migrate(AHCIQState *from, AHCIQState *to, const char *uri)
>      from->parent = tmp;
>      from->dev = dev;
>  
> -    verify_state(to);
> +    verify_state(to, hba_old);
>      g_free(uri_local);
>  }
>  
> diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
> index 716ab79..8e789d8 100644
> --- a/tests/libqos/ahci.c
> +++ b/tests/libqos/ahci.c
> @@ -351,6 +351,7 @@ void ahci_hba_enable(AHCIQState *ahci)
>      reg = ahci_rreg(ahci, AHCI_GHC);
>      ASSERT_BIT_SET(reg, AHCI_GHC_IE);
>  
> +    ahci->enabled = true;
>      /* TODO: The device should now be idling and waiting for commands.
>       * In the future, a small test-case to inspect the Register D2H FIS
>       * and clear the initial interrupts might be good. */
> diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
> index c69fb5a..9b0c1d7 100644
> --- a/tests/libqos/ahci.h
> +++ b/tests/libqos/ahci.h
> @@ -327,6 +327,7 @@ typedef struct AHCIQState {
>      uint32_t cap;
>      uint32_t cap2;
>      AHCIPortQState port[32];
> +    bool enabled;
>  } AHCIQState;
>  
>  /**

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle
  2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle David Gibson
@ 2016-10-25  9:21   ` Greg Kurz
  0 siblings, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25  9:21 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 16:00:01 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> The usual use model for the libqos PCI functions is to map a specific PCI
> BAR using qpci_iomap() then pass the returned token into IO accessor
> functions.  This, and the fact that iomap() returns a (void *) which
> actually contains a PCI space address, kind of suggests that the return
> value from iomap is supposed to be an opaque token.
> 
> ..except that the callers expect to be able to add offsets to it.  Which
> also assumes the compiler will support pointer arithmetic on a (void *),
> and treat it as working with byte offsets.
> 
> To clarify this situation change iomap() and the IO accessors to take
> a definitely opaque BAR handle (enforced with a wrapper struct) along with
> an offset within the BAR.  This changes both the functions and all the
> callers.
> 
> There were a number of places that checked if iomap() returned non-NULL,
> and or initialized it to NULL before hand.  Since iomap() already assert()s
> if it fails to map the BAR, these tests were mostly pointless and are
> removed.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/e1000e-test.c       |   7 +-
>  tests/ide-test.c          | 176 +++++++++++++++++++++++-----------------------
>  tests/ivshmem-test.c      |  16 ++---
>  tests/libqos/ahci.c       |   3 +-
>  tests/libqos/ahci.h       |   6 +-
>  tests/libqos/pci.c        | 149 ++++++++++++++++++---------------------
>  tests/libqos/pci.h        |  46 +++++++-----
>  tests/libqos/usb.c        |   6 +-
>  tests/libqos/usb.h        |   2 +-
>  tests/libqos/virtio-pci.c | 101 +++++++++++++-------------
>  tests/libqos/virtio-pci.h |   2 +-
>  tests/rtl8139-test.c      |  10 ++-
>  tests/tco-test.c          |  80 ++++++++++-----------
>  tests/usb-hcd-ehci-test.c |   5 +-
>  14 files changed, 300 insertions(+), 309 deletions(-)
> 
> diff --git a/tests/e1000e-test.c b/tests/e1000e-test.c
> index 3979b20..8c42ca9 100644
> --- a/tests/e1000e-test.c
> +++ b/tests/e1000e-test.c
> @@ -87,7 +87,7 @@
>  
>  typedef struct e1000e_device {
>      QPCIDevice *pci_dev;
> -    void *mac_regs;
> +    QPCIBar mac_regs;
>  
>      uint64_t tx_ring;
>      uint64_t rx_ring;
> @@ -119,12 +119,12 @@ static QPCIDevice *e1000e_device_find(QPCIBus *bus)
>  
>  static void e1000e_macreg_write(e1000e_device *d, uint32_t reg, uint32_t val)
>  {
> -    qpci_io_writel(d->pci_dev, d->mac_regs + reg, val);
> +    qpci_io_writel(d->pci_dev, d->mac_regs, reg, val);
>  }
>  
>  static uint32_t e1000e_macreg_read(e1000e_device *d, uint32_t reg)
>  {
> -    return qpci_io_readl(d->pci_dev, d->mac_regs + reg);
> +    return qpci_io_readl(d->pci_dev, d->mac_regs, reg);
>  }
>  
>  static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
> @@ -138,7 +138,6 @@ static void e1000e_device_init(QPCIBus *bus, e1000e_device *d)
>  
>      /* Map BAR0 (mac registers) */
>      d->mac_regs = qpci_iomap(d->pci_dev, 0, NULL);
> -    g_assert_nonnull(d->mac_regs);
>  
>      /* Reset the device */
>      val = e1000e_macreg_read(d, E1000E_CTRL);
> diff --git a/tests/ide-test.c b/tests/ide-test.c
> index 86c4373..67c7df0 100644
> --- a/tests/ide-test.c
> +++ b/tests/ide-test.c
> @@ -137,7 +137,7 @@ static void ide_test_quit(void)
>      qtest_end();
>  }
>  
> -static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
> +static QPCIDevice *get_pci_device(QPCIBar *bmdma_bar, QPCIBar *ide_bar)
>  {
>      QPCIDevice *dev;
>      uint16_t vendor_id, device_id;
> @@ -156,9 +156,9 @@ static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
>      g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
>  
>      /* Map bmdma BAR */
> -    *bmdma_base = qpci_iomap(dev, 4, NULL);
> +    *bmdma_bar = qpci_iomap(dev, 4, NULL);
>  
> -    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
> +    *ide_bar = qpci_legacy_iomap(dev, IDE_BASE);
>  
>      qpci_device_enable(dev);
>  
> @@ -181,19 +181,18 @@ typedef struct PrdtEntry {
>  
>  static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>                              PrdtEntry *prdt, int prdt_entries,
> -                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
> +                            void(*post_exec)(QPCIDevice *dev, QPCIBar ide_bar,
>                                               uint64_t sector, int nb_sectors))
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base;
> -    void *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uintptr_t guest_prdt;
>      size_t len;
>      bool from_dev;
>      uint8_t status;
>      int flags;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      flags = cmd & ~0xff;
>      cmd &= 0xff;
> @@ -218,60 +217,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>      }
>  
>      /* Select device 0 */
> -    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0 | LBA);
>  
>      /* Stop any running transfer, clear any pending interrupt */
> -    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> -    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_status, BM_STS_INTR);
>  
>      /* Setup PRDT */
>      len = sizeof(*prdt) * prdt_entries;
>      guest_prdt = guest_alloc(guest_malloc, len);
>      memwrite(guest_prdt, prdt, len);
> -    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
> +    qpci_io_writel(dev, bmdma_bar, bmreg_prdt, guest_prdt);
>  
>      /* ATA DMA command */
>      if (cmd == CMD_PACKET) {
>          /* Enables ATAPI DMA; otherwise PIO is attempted */
> -        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
> +        qpci_io_writeb(dev, ide_bar, reg_feature, 0x01);
>      } else {
> -        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
> -        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
> -        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
> -        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_nsectors, nb_sectors);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_low,    sector & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_middle, (sector >> 8) & 0xff);
> +        qpci_io_writeb(dev, ide_bar, reg_lba_high,   (sector >> 16) & 0xff);
>      }
>  
> -    qpci_io_writeb(dev, ide_base + reg_command, cmd);
> +    qpci_io_writeb(dev, ide_bar, reg_command, cmd);
>  
>      if (post_exec) {
> -        post_exec(dev, ide_base, sector, nb_sectors);
> +        post_exec(dev, ide_bar, sector, nb_sectors);
>      }
>  
>      /* Start DMA transfer */
> -    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
> +    qpci_io_writeb(dev, bmdma_bar, bmreg_cmd,
>                     BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
>  
>      if (flags & CMDF_ABORT) {
> -        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
>      }
>  
>      /* Wait for the DMA transfer to complete */
>      do {
> -        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
> +        status = qpci_io_readb(dev, bmdma_bar, bmreg_status);
>      } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
>  
>      g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
>  
>      /* Check IDE status code */
> -    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
> +    assert_bit_set(qpci_io_readb(dev, ide_bar, reg_status), DRDY);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), BSY | DRQ);
>  
>      /* Reading the status register clears the IRQ */
>      g_assert(!get_irq(IDE_PRIMARY_IRQ));
>  
>      /* Stop DMA transfer if still active */
>      if (status & BM_STS_ACTIVE) {
> -        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_bar, bmreg_cmd, 0);
>      }
>  
>      free_pci_device(dev);
> @@ -282,7 +281,7 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>  static void test_bmdma_simple_rw(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>      uint8_t *buf;
>      uint8_t *cmpbuf;
> @@ -296,7 +295,7 @@ static void test_bmdma_simple_rw(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      buf = g_malloc(len);
>      cmpbuf = g_malloc(len);
> @@ -308,7 +307,7 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Write 0xaa pattern to sector 1 */
>      memset(buf, 0xaa, len);
> @@ -317,14 +316,14 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Read and verify 0x55 pattern in sector 0 */
>      memset(cmpbuf, 0x55, len);
>  
>      status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -334,7 +333,7 @@ static void test_bmdma_simple_rw(void)
>  
>      status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -347,7 +346,7 @@ static void test_bmdma_simple_rw(void)
>  static void test_bmdma_short_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -357,25 +356,25 @@ static void test_bmdma_short_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_one_sector_short_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      /* Read 2 sectors but only give 1 sector in PRDT */
> @@ -386,25 +385,25 @@ static void test_bmdma_one_sector_short_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_long_prdt(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -414,28 +413,28 @@ static void test_bmdma_long_prdt(void)
>          },
>      };
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_no_busmaster(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
>       * able to access it anyway because the Bus Master bit in the PCI command
> @@ -449,7 +448,7 @@ static void test_bmdma_no_busmaster(void)
>      /* Not entirely clear what the expected result is, but this is what we get
>       * in practice. At least we want to be aware of any changes. */
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_setup(void)
> @@ -480,7 +479,7 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
>  static void test_identify(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      uint16_t buf[256];
>      int i;
> @@ -491,25 +490,25 @@ static void test_identify(void)
>          "-global ide-hd.ver=%s",
>          tmp_path, "testdisk", "version");
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* IDENTIFY command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_IDENTIFY);
>  
>      /* Read in the IDENTIFY buffer and check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      for (i = 0; i < 256; i++) {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>          assert_bit_set(data, DRDY | DRQ);
>          assert_bit_clear(data, BSY | DF | ERR);
>  
> -        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
> +        buf[i] = qpci_io_readw(dev, ide_bar, reg_data);
>      }
>  
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, DRDY);
>      assert_bit_clear(data, BSY | DF | ERR | DRQ);
>  
> @@ -535,13 +534,13 @@ static void test_identify(void)
>  static void make_dirty(uint8_t device)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t status;
>      size_t len = 512;
>      uintptr_t guest_buf;
>      void* buf;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      guest_buf = guest_alloc(guest_malloc, len);
>      buf = g_malloc(len);
> @@ -560,7 +559,7 @@ static void make_dirty(uint8_t device)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_bar, reg_status), DF | ERR);
>  
>      g_free(buf);
>  }
> @@ -568,14 +567,14 @@ static void make_dirty(uint8_t device)
>  static void test_flush(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>  
>      ide_test_start(
>          "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
>          tmp_path);
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
> @@ -586,11 +585,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -598,11 +597,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
>  
>      /* Check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -614,7 +613,7 @@ static void test_flush(void)
>  static void test_retry_flush(const char *machine)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      const char *s;
>  
> @@ -626,7 +625,7 @@ static void test_retry_flush(const char *machine)
>          "rerror=stop,werror=stop",
>          debug_path, tmp_path);
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
> @@ -634,11 +633,11 @@ static void test_retry_flush(const char *machine)
>      make_dirty(0);
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = qpci_io_readb(dev, ide_base + reg_status);
> +    data = qpci_io_readb(dev, ide_bar, reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -649,11 +648,11 @@ static void test_retry_flush(const char *machine)
>      qmp_discard_response(s);
>  
>      /* Check registers */
> -    data = qpci_io_readb(dev, ide_base + reg_device);
> +    data = qpci_io_readb(dev, ide_bar, reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -665,15 +664,15 @@ static void test_retry_flush(const char *machine)
>  static void test_flush_nodev(void)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>  
>      ide_test_start("");
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* FLUSH CACHE command on device 0*/
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_FLUSH_CACHE);
>  
>      /* Just testing that qemu doesn't crash... */
>  
> @@ -700,7 +699,7 @@ typedef struct Read10CDB {
>      uint16_t padding;
>  } __attribute__((__packed__)) Read10CDB;
>  
> -static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> +static void send_scsi_cdb_read10(QPCIDevice *dev, QPCIBar ide_bar,
>                                   uint64_t lba, int nblocks)
>  {
>      Read10CDB pkt = { .padding = 0 };
> @@ -717,7 +716,7 @@ static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
>  
>      /* Send Packet */
>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> -        qpci_io_writew(dev, ide_base + reg_data,
> +        qpci_io_writew(dev, ide_bar, reg_data,
>                         le16_to_cpu(((uint16_t *)&pkt)[i]));
>      }
>  }
> @@ -732,16 +731,16 @@ static void nsleep(int64_t nsecs)
>  static uint8_t ide_wait_clear(uint8_t flag)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      uint8_t data;
>      time_t st;
>  
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>  
>      /* Wait with a 5 second timeout */
>      time(&st);
>      while (true) {
> -        data = qpci_io_readb(dev, ide_base + reg_status);
> +        data = qpci_io_readb(dev, ide_bar, reg_status);
>          if (!(data & flag)) {
>              return data;
>          }
> @@ -776,7 +775,7 @@ static void ide_wait_intr(int irq)
>  static void cdrom_pio_impl(int nblocks)
>  {
>      QPCIDevice *dev;
> -    void *bmdma_base, *ide_base;
> +    QPCIBar bmdma_bar, ide_bar;
>      FILE *fh;
>      int patt_blocks = MAX(16, nblocks);
>      size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
> @@ -795,15 +794,14 @@ static void cdrom_pio_impl(int nblocks)
>  
>      ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
>                     "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
> -    dev = get_pci_device(&bmdma_base, &ide_base);
> +    dev = get_pci_device(&bmdma_bar, &ide_bar);
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* PACKET command on device 0 */
> -    qpci_io_writeb(dev, ide_base + reg_device, 0);
> -    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> -    qpci_io_writeb(dev, ide_base + reg_lba_high,
> -                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> -    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
> +    qpci_io_writeb(dev, ide_bar, reg_device, 0);
> +    qpci_io_writeb(dev, ide_bar, reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> +    qpci_io_writeb(dev, ide_bar, reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> +    qpci_io_writeb(dev, ide_bar, reg_command, CMD_PACKET);
>      /* HP0: Check_Status_A State */
>      nsleep(400);
>      data = ide_wait_clear(BSY);
> @@ -812,7 +810,7 @@ static void cdrom_pio_impl(int nblocks)
>      assert_bit_clear(data, ERR | DF | BSY);
>  
>      /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
> -    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
> +    send_scsi_cdb_read10(dev, ide_bar, 0, nblocks);
>  
>      /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
>       * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
> @@ -836,8 +834,8 @@ static void cdrom_pio_impl(int nblocks)
>  
>          /* HP4: Transfer_Data */
>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> -            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> -                                                       ide_base + reg_data));
> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev, ide_bar,
> +                                                       reg_data));
>          }
>      }
>  
> diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
> index fbd8258..04a5c5d 100644
> --- a/tests/ivshmem-test.c
> +++ b/tests/ivshmem-test.c
> @@ -41,7 +41,7 @@ static QPCIDevice *get_device(QPCIBus *pcibus)
>  
>  typedef struct _IVState {
>      QTestState *qtest;
> -    void *reg_base, *mem_base;
> +    QPCIBar reg_bar, mem_bar;
>      QPCIBus *pcibus;
>      QPCIDevice *dev;
>  } IVState;
> @@ -75,7 +75,7 @@ static inline unsigned in_reg(IVState *s, enum Reg reg)
>      unsigned res;
>  
>      global_qtest = s->qtest;
> -    res = qpci_io_readl(s->dev, s->reg_base + reg);
> +    res = qpci_io_readl(s->dev, s->reg_bar, reg);
>      g_test_message("*%s -> %x\n", name, res);
>      global_qtest = qtest;
>  
> @@ -89,7 +89,7 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
>  
>      global_qtest = s->qtest;
>      g_test_message("%x -> *%s\n", v, name);
> -    qpci_io_writel(s->dev, s->reg_base + reg, v);
> +    qpci_io_writel(s->dev, s->reg_bar, reg, v);
>      global_qtest = qtest;
>  }
>  
> @@ -98,7 +98,7 @@ static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
>      QTestState *qtest = global_qtest;
>  
>      global_qtest = s->qtest;
> -    qpci_memread(s->dev, s->mem_base + off, buf, len);
> +    qpci_memread(s->dev, s->mem_bar, off, buf, len);
>      global_qtest = qtest;
>  }
>  
> @@ -108,7 +108,7 @@ static inline void write_mem(IVState *s, uint64_t off,
>      QTestState *qtest = global_qtest;
>  
>      global_qtest = s->qtest;
> -    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
> +    qpci_memwrite(s->dev, s->mem_bar, off, buf, len);
>      global_qtest = qtest;
>  }
>  
> @@ -127,16 +127,14 @@ static void setup_vm_cmd(IVState *s, const char *cmd, bool msix)
>      s->pcibus = qpci_init_pc(NULL);
>      s->dev = get_device(s->pcibus);
>  
> -    s->reg_base = qpci_iomap(s->dev, 0, &barsize);
> -    g_assert_nonnull(s->reg_base);
> +    s->reg_bar = qpci_iomap(s->dev, 0, &barsize);
>      g_assert_cmpuint(barsize, ==, 256);
>  
>      if (msix) {
>          qpci_msix_enable(s->dev);
>      }
>  
> -    s->mem_base = qpci_iomap(s->dev, 2, &barsize);
> -    g_assert_nonnull(s->mem_base);
> +    s->mem_bar = qpci_iomap(s->dev, 2, &barsize);
>      g_assert_cmpuint(barsize, ==, TMPSHMSIZE);
>  
>      qpci_device_enable(s->dev);
> diff --git a/tests/libqos/ahci.c b/tests/libqos/ahci.c
> index 8e789d8..5180d65 100644
> --- a/tests/libqos/ahci.c
> +++ b/tests/libqos/ahci.c
> @@ -210,8 +210,7 @@ void ahci_pci_enable(AHCIQState *ahci)
>  void start_ahci_device(AHCIQState *ahci)
>  {
>      /* Map AHCI's ABAR (BAR5) */
> -    ahci->hba_base = qpci_iomap(ahci->dev, 5, &ahci->barsize);
> -    g_assert(ahci->hba_base);
> +    ahci->hba_bar = qpci_iomap(ahci->dev, 5, &ahci->barsize);
>  
>      /* turns on pci.cmd.iose, pci.cmd.mse and pci.cmd.bme */
>      qpci_device_enable(ahci->dev);
> diff --git a/tests/libqos/ahci.h b/tests/libqos/ahci.h
> index 9b0c1d7..caaafe3 100644
> --- a/tests/libqos/ahci.h
> +++ b/tests/libqos/ahci.h
> @@ -321,7 +321,7 @@ typedef struct AHCIPortQState {
>  typedef struct AHCIQState {
>      QOSState *parent;
>      QPCIDevice *dev;
> -    void *hba_base;
> +    QPCIBar hba_bar;
>      uint64_t barsize;
>      uint32_t fingerprint;
>      uint32_t cap;
> @@ -489,12 +489,12 @@ typedef struct AHCIOpts {
>  
>  static inline uint32_t ahci_mread(AHCIQState *ahci, size_t offset)
>  {
> -    return qpci_io_readl(ahci->dev, ahci->hba_base + offset);
> +    return qpci_io_readl(ahci->dev, ahci->hba_bar, offset);
>  }
>  
>  static inline void ahci_mwrite(AHCIQState *ahci, size_t offset, uint32_t value)
>  {
> -    qpci_io_writel(ahci->dev, ahci->hba_base + offset, value);
> +    qpci_io_writel(ahci->dev, ahci->hba_bar, offset, value);
>  }
>  
>  static inline uint32_t ahci_rreg(AHCIQState *ahci, uint32_t reg_num)
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index 3021651..2dcdead 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -104,7 +104,6 @@ void qpci_msix_enable(QPCIDevice *dev)
>      uint32_t table;
>      uint8_t bir_table;
>      uint8_t bir_pba;
> -    void *offset;
>  
>      addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
>      g_assert_cmphex(addr, !=, 0);
> @@ -114,18 +113,16 @@ void qpci_msix_enable(QPCIDevice *dev)
>  
>      table = qpci_config_readl(dev, addr + PCI_MSIX_TABLE);
>      bir_table = table & PCI_MSIX_FLAGS_BIRMASK;
> -    offset = qpci_iomap(dev, bir_table, NULL);
> -    dev->msix_table = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
> +    dev->msix_table_bar = qpci_iomap(dev, bir_table, NULL);
> +    dev->msix_table_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
>  
>      table = qpci_config_readl(dev, addr + PCI_MSIX_PBA);
>      bir_pba = table & PCI_MSIX_FLAGS_BIRMASK;
>      if (bir_pba != bir_table) {
> -        offset = qpci_iomap(dev, bir_pba, NULL);
> +        dev->msix_pba_bar = qpci_iomap(dev, bir_pba, NULL);
>      }
> -    dev->msix_pba = offset + (table & ~PCI_MSIX_FLAGS_BIRMASK);
> +    dev->msix_pba_off = table & ~PCI_MSIX_FLAGS_BIRMASK;
>  
> -    g_assert(dev->msix_table != NULL);
> -    g_assert(dev->msix_pba != NULL);
>      dev->msix_enabled = true;
>  }
>  
> @@ -141,22 +138,23 @@ void qpci_msix_disable(QPCIDevice *dev)
>      qpci_config_writew(dev, addr + PCI_MSIX_FLAGS,
>                                                  val & ~PCI_MSIX_FLAGS_ENABLE);
>  
> -    qpci_iounmap(dev, dev->msix_table);
> -    qpci_iounmap(dev, dev->msix_pba);
> +    qpci_iounmap(dev, dev->msix_table_bar);
> +    qpci_iounmap(dev, dev->msix_pba_bar);
>      dev->msix_enabled = 0;
> -    dev->msix_table = NULL;
> -    dev->msix_pba = NULL;
> +    dev->msix_table_off = 0;
> +    dev->msix_pba_off = 0;
>  }
>  
>  bool qpci_msix_pending(QPCIDevice *dev, uint16_t entry)
>  {
>      uint32_t pba_entry;
>      uint8_t bit_n = entry % 32;
> -    void *addr = dev->msix_pba + (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
> +    uint64_t  off = (entry / 32) * PCI_MSIX_ENTRY_SIZE / 4;
>  
>      g_assert(dev->msix_enabled);
> -    pba_entry = qpci_io_readl(dev, addr);
> -    qpci_io_writel(dev, addr, pba_entry & ~(1 << bit_n));
> +    pba_entry = qpci_io_readl(dev, dev->msix_pba_bar, dev->msix_pba_off + off);
> +    qpci_io_writel(dev, dev->msix_pba_bar, dev->msix_pba_off + off,
> +                   pba_entry & ~(1 << bit_n));
>      return (pba_entry & (1 << bit_n)) != 0;
>  }
>  
> @@ -164,7 +162,7 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
>  {
>      uint8_t addr;
>      uint16_t val;
> -    void *vector_addr = dev->msix_table + (entry * PCI_MSIX_ENTRY_SIZE);
> +    uint64_t vector_off = dev->msix_table_off + entry * PCI_MSIX_ENTRY_SIZE;
>  
>      g_assert(dev->msix_enabled);
>      addr = qpci_find_capability(dev, PCI_CAP_ID_MSIX);
> @@ -174,8 +172,9 @@ bool qpci_msix_masked(QPCIDevice *dev, uint16_t entry)
>      if (val & PCI_MSIX_FLAGS_MASKALL) {
>          return true;
>      } else {
> -        return (qpci_io_readl(dev, vector_addr + PCI_MSIX_ENTRY_VECTOR_CTRL)
> -                                            & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
> +        return (qpci_io_readl(dev, dev->msix_table_bar,
> +                              vector_off + PCI_MSIX_ENTRY_VECTOR_CTRL)
> +                & PCI_MSIX_ENTRY_CTRL_MASKBIT) != 0;
>      }
>  }
>  
> @@ -222,129 +221,115 @@ void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value)
>      dev->bus->config_writel(dev->bus, dev->devfn, offset, value);
>  }
>  
> -
> -uint8_t qpci_io_readb(QPCIDevice *dev, void *data)
> +uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readb(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readb(dev->bus, token.addr + off);
>      } else {
>          uint8_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return val;
>      }
>  }
>  
> -uint16_t qpci_io_readw(QPCIDevice *dev, void *data)
> +uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readw(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readw(dev->bus, token.addr + off);
>      } else {
>          uint16_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le16_to_cpu(val);
>      }
>  }
>  
> -uint32_t qpci_io_readl(QPCIDevice *dev, void *data)
> +uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readl(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readl(dev->bus, token.addr + off);
>      } else {
>          uint32_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le32_to_cpu(val);
>      }
>  }
>  
> -uint64_t qpci_io_readq(QPCIDevice *dev, void *data)
> +uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        return dev->bus->pio_readq(dev->bus, addr);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        return dev->bus->pio_readq(dev->bus, token.addr + off);
>      } else {
>          uint64_t val;
> -        dev->bus->memread(dev->bus, addr, &val, sizeof(val));
> +        dev->bus->memread(dev->bus, token.addr + off, &val, sizeof(val));
>          return le64_to_cpu(val);
>      }
>  }
>  
> -void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value)
> +void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint8_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writeb(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writeb(dev->bus, token.addr + off, value);
>      } else {
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value)
> +void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint16_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writew(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writew(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le16(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value)
> +void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint32_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writel(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writel(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le32(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value)
> +void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint64_t value)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    if (addr < QPCI_PIO_LIMIT) {
> -        dev->bus->pio_writeq(dev->bus, addr, value);
> +    if (token.addr < QPCI_PIO_LIMIT) {
> +        dev->bus->pio_writeq(dev->bus, token.addr + off, value);
>      } else {
>          value = cpu_to_le64(value);
> -        dev->bus->memwrite(dev->bus, addr, &value, sizeof(value));
> +        dev->bus->memwrite(dev->bus, token.addr + off, &value, sizeof(value));
>      }
>  }
>  
> -void qpci_memread(QPCIDevice *dev, void *data, void *buf, size_t len)
> +void qpci_memread(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                  void *buf, size_t len)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    g_assert(addr >= QPCI_PIO_LIMIT);
> -    dev->bus->memread(dev->bus, addr, buf, len);
> +    g_assert(token.addr >= QPCI_PIO_LIMIT);
> +    dev->bus->memread(dev->bus, token.addr + off, buf, len);
>  }
>  
> -void qpci_memwrite(QPCIDevice *dev, void *data, const void *buf, size_t len)
> +void qpci_memwrite(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                   const void *buf, size_t len)
>  {
> -    uintptr_t addr = (uintptr_t)data;
> -
> -    g_assert(addr >= QPCI_PIO_LIMIT);
> -    dev->bus->memwrite(dev->bus, addr, buf, len);
> +    g_assert(token.addr >= QPCI_PIO_LIMIT);
> +    dev->bus->memwrite(dev->bus, token.addr + off, buf, len);
>  }
>  
> -void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
> +QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
>  {
>      QPCIBus *bus = dev->bus;
>      static const int bar_reg_map[] = {
>          PCI_BASE_ADDRESS_0, PCI_BASE_ADDRESS_1, PCI_BASE_ADDRESS_2,
>          PCI_BASE_ADDRESS_3, PCI_BASE_ADDRESS_4, PCI_BASE_ADDRESS_5,
>      };
> +    QPCIBar bar;
>      int bar_reg;
>      uint32_t addr, size;
>      uint32_t io_type;
> @@ -391,17 +376,19 @@ void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr)
>          qpci_config_writel(dev, bar_reg, loc);
>      }
>  
> -    return (void *)(uintptr_t)loc;
> +    bar.addr = loc;
> +    return bar;
>  }
>  
> -void qpci_iounmap(QPCIDevice *dev, void *data)
> +void qpci_iounmap(QPCIDevice *dev, QPCIBar bar)
>  {
>      /* FIXME */
>  }
>  
> -void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
> +QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
>  {
> -    return (void *)(uintptr_t)addr;
> +    QPCIBar bar = { .addr = addr };
> +    return bar;
>  }
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index 531e3f7..ed48061 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -21,6 +21,7 @@
>  
>  typedef struct QPCIDevice QPCIDevice;
>  typedef struct QPCIBus QPCIBus;
> +typedef struct QPCIBar QPCIBar;
>  
>  struct QPCIBus {
>      uint8_t (*pio_readb)(QPCIBus *bus, uint32_t addr);
> @@ -51,13 +52,17 @@ struct QPCIBus {
>      uint64_t mmio_alloc_ptr, mmio_limit;
>  };
>  
> +struct QPCIBar {
> +    uint64_t addr;
> +};
> +
>  struct QPCIDevice
>  {
>      QPCIBus *bus;
>      int devfn;
>      bool msix_enabled;
> -    void *msix_table;
> -    void *msix_pba;
> +    QPCIBar msix_table_bar, msix_pba_bar;
> +    uint64_t msix_table_off, msix_pba_off;
>  };
>  
>  void qpci_device_foreach(QPCIBus *bus, int vendor_id, int device_id,
> @@ -81,22 +86,27 @@ void qpci_config_writeb(QPCIDevice *dev, uint8_t offset, uint8_t value);
>  void qpci_config_writew(QPCIDevice *dev, uint8_t offset, uint16_t value);
>  void qpci_config_writel(QPCIDevice *dev, uint8_t offset, uint32_t value);
>  
> -uint8_t qpci_io_readb(QPCIDevice *dev, void *data);
> -uint16_t qpci_io_readw(QPCIDevice *dev, void *data);
> -uint32_t qpci_io_readl(QPCIDevice *dev, void *data);
> -uint64_t qpci_io_readq(QPCIDevice *dev, void *data);
> -
> -void qpci_io_writeb(QPCIDevice *dev, void *data, uint8_t value);
> -void qpci_io_writew(QPCIDevice *dev, void *data, uint16_t value);
> -void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
> -void qpci_io_writeq(QPCIDevice *dev, void *data, uint64_t value);
> -
> -void qpci_memread(QPCIDevice *bus, void *data, void *buf, size_t len);
> -void qpci_memwrite(QPCIDevice *bus, void *data, const void *buf, size_t len);
> -
> -void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
> -void qpci_iounmap(QPCIDevice *dev, void *data);
> -void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
> +uint8_t qpci_io_readb(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint16_t qpci_io_readw(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint32_t qpci_io_readl(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +uint64_t qpci_io_readq(QPCIDevice *dev, QPCIBar token, uint64_t off);
> +
> +void qpci_io_writeb(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint8_t value);
> +void qpci_io_writew(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint16_t value);
> +void qpci_io_writel(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint32_t value);
> +void qpci_io_writeq(QPCIDevice *dev, QPCIBar token, uint64_t off,
> +                    uint64_t value);
> +
> +void qpci_memread(QPCIDevice *bus, QPCIBar token, uint64_t off,
> +                  void *buf, size_t len);
> +void qpci_memwrite(QPCIDevice *bus, QPCIBar token, uint64_t off,
> +                   const void *buf, size_t len);
> +QPCIBar qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
> +void qpci_iounmap(QPCIDevice *dev, QPCIBar addr);
> +QPCIBar qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts);
> diff --git a/tests/libqos/usb.c b/tests/libqos/usb.c
> index f794d92..72d7a96 100644
> --- a/tests/libqos/usb.c
> +++ b/tests/libqos/usb.c
> @@ -21,14 +21,12 @@ void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc, uint32_t devfn, int bar)
>      hc->dev = qpci_device_find(pcibus, devfn);
>      g_assert(hc->dev != NULL);
>      qpci_device_enable(hc->dev);
> -    hc->base = qpci_iomap(hc->dev, bar, NULL);
> -    g_assert(hc->base != NULL);
> +    hc->bar = qpci_iomap(hc->dev, bar, NULL);
>  }
>  
>  void uhci_port_test(struct qhc *hc, int port, uint16_t expect)
>  {
> -    void *addr = hc->base + 0x10 + 2 * port;
> -    uint16_t value = qpci_io_readw(hc->dev, addr);
> +    uint16_t value = qpci_io_readw(hc->dev, hc->bar, 0x10 + 2 * port);
>      uint16_t mask = ~(UHCI_PORT_WRITE_CLEAR | UHCI_PORT_RSVD1);
>  
>      g_assert((value & mask) == (expect & mask));
> diff --git a/tests/libqos/usb.h b/tests/libqos/usb.h
> index 8fe5687..423dcfd 100644
> --- a/tests/libqos/usb.h
> +++ b/tests/libqos/usb.h
> @@ -5,7 +5,7 @@
>  
>  struct qhc {
>      QPCIDevice *dev;
> -    void *base;
> +    QPCIBar bar;
>  };
>  
>  void qusb_pci_init_one(QPCIBus *pcibus, struct qhc *hc,
> diff --git a/tests/libqos/virtio-pci.c b/tests/libqos/virtio-pci.c
> index c69d09d..d4bf841 100644
> --- a/tests/libqos/virtio-pci.c
> +++ b/tests/libqos/virtio-pci.c
> @@ -62,13 +62,12 @@ static void qvirtio_pci_assign_device(QVirtioDevice *d, void *data)
>      *vpcidev = (QVirtioPCIDevice *)d;
>  }
>  
> -#define CONFIG_BASE(dev) \
> -    ((dev)->addr + VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
> +#define CONFIG_BASE(dev) (VIRTIO_PCI_CONFIG_OFF((dev)->pdev->msix_enabled))
>  
>  static uint8_t qvirtio_pci_config_readb(QVirtioDevice *d, uint64_t off)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readb(dev->pdev, CONFIG_BASE(dev) + off);
> +    return qpci_io_readb(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>  }
>  
>  /* PCI is always read in little-endian order
> @@ -84,7 +83,7 @@ static uint16_t qvirtio_pci_config_readw(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint16_t value;
>  
> -    value = qpci_io_readw(dev->pdev, CONFIG_BASE(dev) + off);
> +    value = qpci_io_readw(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          value = bswap16(value);
>      }
> @@ -96,7 +95,7 @@ static uint32_t qvirtio_pci_config_readl(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint32_t value;
>  
> -    value = qpci_io_readl(dev->pdev, CONFIG_BASE(dev) + off);
> +    value = qpci_io_readl(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          value = bswap32(value);
>      }
> @@ -108,7 +107,7 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
>      uint64_t val;
>  
> -    val = qpci_io_readq(dev->pdev, CONFIG_BASE(dev) + off);
> +    val = qpci_io_readq(dev->pdev, dev->bar, CONFIG_BASE(dev) + off);
>      if (qvirtio_is_big_endian(d)) {
>          val = bswap64(val);
>      }
> @@ -119,31 +118,31 @@ static uint64_t qvirtio_pci_config_readq(QVirtioDevice *d, uint64_t off)
>  static uint32_t qvirtio_pci_get_features(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_HOST_FEATURES);
> +    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_HOST_FEATURES);
>  }
>  
>  static void qvirtio_pci_set_features(QVirtioDevice *d, uint32_t features)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES, features);
> +    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES, features);
>  }
>  
>  static uint32_t qvirtio_pci_get_guest_features(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readl(dev->pdev, dev->addr + VIRTIO_PCI_GUEST_FEATURES);
> +    return qpci_io_readl(dev->pdev, dev->bar, VIRTIO_PCI_GUEST_FEATURES);
>  }
>  
>  static uint8_t qvirtio_pci_get_status(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS);
> +    return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS);
>  }
>  
>  static void qvirtio_pci_set_status(QVirtioDevice *d, uint8_t status)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_STATUS, status);
> +    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_STATUS, status);
>  }
>  
>  static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
> @@ -167,7 +166,7 @@ static bool qvirtio_pci_get_queue_isr_status(QVirtioDevice *d, QVirtQueue *vq)
>              }
>          }
>      } else {
> -        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 1;
> +        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 1;
>      }
>  }
>  
> @@ -191,26 +190,26 @@ static bool qvirtio_pci_get_config_isr_status(QVirtioDevice *d)
>              }
>          }
>      } else {
> -        return qpci_io_readb(dev->pdev, dev->addr + VIRTIO_PCI_ISR) & 2;
> +        return qpci_io_readb(dev->pdev, dev->bar, VIRTIO_PCI_ISR) & 2;
>      }
>  }
>  
>  static void qvirtio_pci_queue_select(QVirtioDevice *d, uint16_t index)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writeb(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_SEL, index);
> +    qpci_io_writeb(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_SEL, index);
>  }
>  
>  static uint16_t qvirtio_pci_get_queue_size(QVirtioDevice *d)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    return qpci_io_readw(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NUM);
> +    return qpci_io_readw(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NUM);
>  }
>  
>  static void qvirtio_pci_set_queue_address(QVirtioDevice *d, uint32_t pfn)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writel(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_PFN, pfn);
> +    qpci_io_writel(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_PFN, pfn);
>  }
>  
>  static QVirtQueue *qvirtio_pci_virtqueue_setup(QVirtioDevice *d,
> @@ -262,7 +261,7 @@ static void qvirtio_pci_virtqueue_cleanup(QVirtQueue *vq,
>  static void qvirtio_pci_virtqueue_kick(QVirtioDevice *d, QVirtQueue *vq)
>  {
>      QVirtioPCIDevice *dev = (QVirtioPCIDevice *)d;
> -    qpci_io_writew(dev->pdev, dev->addr + VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
> +    qpci_io_writew(dev->pdev, dev->bar, VIRTIO_PCI_QUEUE_NOTIFY, vq->index);
>  }
>  
>  const QVirtioBus qvirtio_pci = {
> @@ -309,14 +308,12 @@ QVirtioPCIDevice *qvirtio_pci_device_find(QPCIBus *bus, uint16_t device_type)
>  void qvirtio_pci_device_enable(QVirtioPCIDevice *d)
>  {
>      qpci_device_enable(d->pdev);
> -    d->addr = qpci_iomap(d->pdev, 0, NULL);
> -    g_assert(d->addr != NULL);
> +    d->bar = qpci_iomap(d->pdev, 0, NULL);
>  }
>  
>  void qvirtio_pci_device_disable(QVirtioPCIDevice *d)
>  {
> -    qpci_iounmap(d->pdev, d->addr);
> -    d->addr = NULL;
> +    qpci_iounmap(d->pdev, d->bar);
>  }
>  
>  void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
> @@ -324,29 +321,33 @@ void qvirtqueue_pci_msix_setup(QVirtioPCIDevice *d, QVirtQueuePCI *vqpci,
>  {
>      uint16_t vector;
>      uint32_t control;
> -    void *addr;
> +    uint64_t off;
>  
>      g_assert(d->pdev->msix_enabled);
> -    addr = d->pdev->msix_table + (entry * 16);
> +    off = d->pdev->msix_table_off + (entry * 16);
>  
>      g_assert_cmpint(entry, >=, 0);
>      g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
>      vqpci->msix_entry = entry;
>  
>      vqpci->msix_addr = guest_alloc(alloc, 4);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
> -                                                    vqpci->msix_addr & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
> -                                            (vqpci->msix_addr >> 32) & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
> -
> -    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
> -                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_LOWER_ADDR, vqpci->msix_addr & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
> +                   (vqpci->msix_addr >> 32) & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_DATA, vqpci->msix_data);
> +
> +    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
> +                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
> +                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
>  
>      qvirtio_pci_queue_select(&d->vdev, vqpci->vq.index);
> -    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR, entry);
> -    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_QUEUE_VECTOR);
> +    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR, entry);
> +    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_QUEUE_VECTOR);
>      g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
>  }
>  
> @@ -355,10 +356,10 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
>  {
>      uint16_t vector;
>      uint32_t control;
> -    void *addr;
> +    uint64_t off;
>  
>      g_assert(d->pdev->msix_enabled);
> -    addr = d->pdev->msix_table + (entry * 16);
> +    off = d->pdev->msix_table_off + (entry * 16);
>  
>      g_assert_cmpint(entry, >=, 0);
>      g_assert_cmpint(entry, <, qpci_msix_table_size(d->pdev));
> @@ -367,17 +368,21 @@ void qvirtio_pci_set_msix_configuration_vector(QVirtioPCIDevice *d,
>      d->config_msix_data = 0x12345678;
>      d->config_msix_addr = guest_alloc(alloc, 4);
>  
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_LOWER_ADDR,
> -                                                    d->config_msix_addr & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_UPPER_ADDR,
> -                                            (d->config_msix_addr >> 32) & ~0UL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
> -
> -    control = qpci_io_readl(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
> -    qpci_io_writel(d->pdev, addr + PCI_MSIX_ENTRY_VECTOR_CTRL,
> -                                        control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> -
> -    qpci_io_writew(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR, entry);
> -    vector = qpci_io_readw(d->pdev, d->addr + VIRTIO_MSI_CONFIG_VECTOR);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_LOWER_ADDR, d->config_msix_addr & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_UPPER_ADDR,
> +                   (d->config_msix_addr >> 32) & ~0UL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_DATA, d->config_msix_data);
> +
> +    control = qpci_io_readl(d->pdev, d->pdev->msix_table_bar,
> +                            off + PCI_MSIX_ENTRY_VECTOR_CTRL);
> +    qpci_io_writel(d->pdev, d->pdev->msix_table_bar,
> +                   off + PCI_MSIX_ENTRY_VECTOR_CTRL,
> +                   control & ~PCI_MSIX_ENTRY_CTRL_MASKBIT);
> +
> +    qpci_io_writew(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR, entry);
> +    vector = qpci_io_readw(d->pdev, d->bar, VIRTIO_MSI_CONFIG_VECTOR);
>      g_assert_cmphex(vector, !=, VIRTIO_MSI_NO_VECTOR);
>  }
> diff --git a/tests/libqos/virtio-pci.h b/tests/libqos/virtio-pci.h
> index efcac2d..38c54c6 100644
> --- a/tests/libqos/virtio-pci.h
> +++ b/tests/libqos/virtio-pci.h
> @@ -16,7 +16,7 @@
>  typedef struct QVirtioPCIDevice {
>      QVirtioDevice vdev;
>      QPCIDevice *pdev;
> -    void *addr;
> +    QPCIBar bar;
>      uint16_t config_msix_entry;
>      uint64_t config_msix_addr;
>      uint32_t config_msix_data;
> diff --git a/tests/rtl8139-test.c b/tests/rtl8139-test.c
> index c2f601a..7de7dc4 100644
> --- a/tests/rtl8139-test.c
> +++ b/tests/rtl8139-test.c
> @@ -22,7 +22,7 @@ static void nop(void)
>  
>  static QPCIBus *pcibus;
>  static QPCIDevice *dev;
> -static void *dev_base;
> +static QPCIBar dev_bar;
>  
>  static void save_fn(QPCIDevice *dev, int devfn, void *data)
>  {
> @@ -45,14 +45,14 @@ static QPCIDevice *get_device(void)
>  #define PORT(name, len, val) \
>  static unsigned __attribute__((unused)) in_##name(void) \
>  { \
> -    unsigned res = qpci_io_read##len(dev, dev_base+(val)); \
> +    unsigned res = qpci_io_read##len(dev, dev_bar, (val));     \
>      g_test_message("*%s -> %x\n", #name, res); \
>      return res; \
>  } \
>  static void out_##name(unsigned v) \
>  { \
>      g_test_message("%x -> *%s\n", v, #name); \
> -    qpci_io_write##len(dev, dev_base+(val), v); \
> +    qpci_io_write##len(dev, dev_bar, (val), v);        \
>  }
>  
>  PORT(Timer, l, 0x48)
> @@ -186,9 +186,7 @@ static void test_init(void)
>  
>      dev = get_device();
>  
> -    dev_base = qpci_iomap(dev, 0, &barsize);
> -
> -    g_assert(dev_base != NULL);
> +    dev_bar = qpci_iomap(dev, 0, &barsize);
>  
>      qpci_device_enable(dev);
>  
> diff --git a/tests/tco-test.c b/tests/tco-test.c
> index 129577d..ef02ec5 100644
> --- a/tests/tco-test.c
> +++ b/tests/tco-test.c
> @@ -41,7 +41,7 @@ typedef struct {
>      const char *args;
>      bool noreboot;
>      QPCIDevice *dev;
> -    void *tco_io_base;
> +    QPCIBar tco_io_bar;
>  } TestData;
>  
>  static void test_init(TestData *d)
> @@ -70,42 +70,42 @@ static void test_init(TestData *d)
>      /* set Root Complex BAR */
>      qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
>  
> -    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
> +    d->tco_io_bar = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
>  }
>  
>  static void stop_tco(const TestData *d)
>  {
>      uint32_t val;
>  
> -    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
> +    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
>      val |= TCO_TMR_HLT;
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
>  }
>  
>  static void start_tco(const TestData *d)
>  {
>      uint32_t val;
>  
> -    val = qpci_io_readw(d->dev, d->tco_io_base + TCO1_CNT);
> +    val = qpci_io_readw(d->dev, d->tco_io_bar, TCO1_CNT);
>      val &= ~TCO_TMR_HLT;
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_CNT, val);
>  }
>  
>  static void load_tco(const TestData *d)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO_RLD, 4);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO_RLD, 4);
>  }
>  
>  static void set_tco_timeout(const TestData *d, uint16_t ticks)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO_TMR, ticks);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO_TMR, ticks);
>  }
>  
>  static void clear_tco_status(const TestData *d)
>  {
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO1_STS, 0x0008);
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0002);
> -    qpci_io_writew(d->dev, d->tco_io_base + TCO2_STS, 0x0004);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO1_STS, 0x0008);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0002);
> +    qpci_io_writew(d->dev, d->tco_io_bar, TCO2_STS, 0x0004);
>  }
>  
>  static void reset_on_second_timeout(bool enable)
> @@ -128,25 +128,25 @@ static void test_tco_defaults(void)
>      d.args = NULL;
>      d.noreboot = true;
>      test_init(&d);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD), ==,
>                      TCO_RLD_DEFAULT);
>      /* TCO_DAT_IN & TCO_DAT_OUT */
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_DAT_IN), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_DAT_IN), ==,
>                      (TCO_DAT_OUT_DEFAULT << 8) | TCO_DAT_IN_DEFAULT);
>      /* TCO1_STS & TCO2_STS */
> -    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_STS), ==,
> +    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_STS), ==,
>                      (TCO2_STS_DEFAULT << 16) | TCO1_STS_DEFAULT);
>      /* TCO1_CNT & TCO2_CNT */
> -    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_base + TCO1_CNT), ==,
> +    g_assert_cmpint(qpci_io_readl(d.dev, d.tco_io_bar, TCO1_CNT), ==,
>                      (TCO2_CNT_DEFAULT << 16) | TCO1_CNT_DEFAULT);
>      /* TCO_MESSAGE1 & TCO_MESSAGE2 */
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_MESSAGE1), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_MESSAGE1), ==,
>                      (TCO_MESSAGE2_DEFAULT << 8) | TCO_MESSAGE1_DEFAULT);
> -    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + TCO_WDCNT), ==,
> +    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, TCO_WDCNT), ==,
>                      TCO_WDCNT_DEFAULT);
> -    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_base + SW_IRQ_GEN), ==,
> +    g_assert_cmpint(qpci_io_readb(d.dev, d.tco_io_bar, SW_IRQ_GEN), ==,
>                      SW_IRQ_GEN_DEFAULT);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO_TMR), ==,
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO_TMR), ==,
>                      TCO_TMR_DEFAULT);
>      qtest_end();
>  }
> @@ -171,23 +171,23 @@ static void test_tco_timeout(void)
>      clock_step(ticks * TCO_TICK_NSEC);
>  
>      /* test first timeout */
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
>  
>      /* test clearing timeout bit */
>      val |= TCO_TIMEOUT;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 0);
>  
>      /* test second timeout */
>      clock_step(ticks * TCO_TICK_NSEC);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
>      ret = val & TCO_SECOND_TO_STS ? 1 : 0;
>      g_assert(ret == 1);
>  
> @@ -214,13 +214,13 @@ static void test_tco_max_timeout(void)
>      start_tco(&d);
>      clock_step(((ticks & TCO_TMR_MASK) - 1) * TCO_TICK_NSEC);
>  
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD);
>      g_assert_cmpint(val & TCO_RLD_MASK, ==, 1);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 0);
>      clock_step(TCO_TICK_NSEC);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & TCO_TIMEOUT ? 1 : 0;
>      g_assert(ret == 1);
>  
> @@ -358,11 +358,11 @@ static void test_tco_ticks_counter(void)
>      start_tco(&d);
>  
>      do {
> -        rld = qpci_io_readw(d.dev, d.tco_io_base + TCO_RLD) & TCO_RLD_MASK;
> +        rld = qpci_io_readw(d.dev, d.tco_io_bar, TCO_RLD) & TCO_RLD_MASK;
>          g_assert_cmpint(rld, ==, ticks);
>          clock_step(TCO_TICK_NSEC);
>          ticks--;
> -    } while (!(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS) & TCO_TIMEOUT));
> +    } while (!(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS) & TCO_TIMEOUT));
>  
>      stop_tco(&d);
>      qtest_end();
> @@ -378,10 +378,10 @@ static void test_tco1_control_bits(void)
>      test_init(&d);
>  
>      val = TCO_LOCK;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
>      val &= ~TCO_LOCK;
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_CNT, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_CNT), ==,
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_CNT, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_CNT), ==,
>                      TCO_LOCK);
>      qtest_end();
>  }
> @@ -405,13 +405,13 @@ static void test_tco1_status_bits(void)
>      start_tco(&d);
>      clock_step(ticks * TCO_TICK_NSEC);
>  
> -    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_IN, 0);
> -    qpci_io_writeb(d.dev, d.tco_io_base + TCO_DAT_OUT, 0);
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS);
> +    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_IN, 0);
> +    qpci_io_writeb(d.dev, d.tco_io_bar, TCO_DAT_OUT, 0);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS);
>      ret = val & (TCO_TIMEOUT | SW_TCO_SMI | TCO_INT_STS) ? 1 : 0;
>      g_assert(ret == 1);
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO1_STS, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO1_STS), ==, 0);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO1_STS, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO1_STS), ==, 0);
>      qtest_end();
>  }
>  
> @@ -434,11 +434,11 @@ static void test_tco2_status_bits(void)
>      start_tco(&d);
>      clock_step(ticks * TCO_TICK_NSEC * 2);
>  
> -    val = qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS);
> +    val = qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS);
>      ret = val & (TCO_SECOND_TO_STS | TCO_BOOT_STS) ? 1 : 0;
>      g_assert(ret == 1);
> -    qpci_io_writew(d.dev, d.tco_io_base + TCO2_STS, val);
> -    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_base + TCO2_STS), ==, 0);
> +    qpci_io_writew(d.dev, d.tco_io_bar, TCO2_STS, val);
> +    g_assert_cmpint(qpci_io_readw(d.dev, d.tco_io_bar, TCO2_STS), ==, 0);
>      qtest_end();
>  }
>  
> diff --git a/tests/usb-hcd-ehci-test.c b/tests/usb-hcd-ehci-test.c
> index a4ceeaa..57af8a0 100644
> --- a/tests/usb-hcd-ehci-test.c
> +++ b/tests/usb-hcd-ehci-test.c
> @@ -38,8 +38,7 @@ static void uhci_port_update(struct qhc *hc, int port,
>  
>  static void ehci_port_test(struct qhc *hc, int port, uint32_t expect)
>  {
> -    void *addr = hc->base + 0x64 + 4 * port;
> -    uint32_t value = qpci_io_readl(hc->dev, addr);
> +    uint32_t value = qpci_io_readl(hc->dev, hc->bar, 0x64 + 4 * port);
>      uint16_t mask = ~(PORTSC_CSC | PORTSC_PEDC | PORTSC_OCC);
>  
>  #if 0
> @@ -91,7 +90,7 @@ static void pci_ehci_port_1(void)
>  static void pci_ehci_config(void)
>  {
>      /* hands over all ports from companion uhci to ehci */
> -    qpci_io_writew(ehci1.dev, ehci1.base + 0x60, 1);
> +    qpci_io_writew(ehci1.dev, ehci1.bar, 0x60, 1);
>  }
>  
>  static void pci_uhci_port_2(void)

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
@ 2016-10-25  9:23   ` Greg Kurz
  0 siblings, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25  9:23 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 15:59:59 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> ivshmem implements a block of shared memory in a PCI BAR.  Currently our
> test case accesses this using qtest_mem{read,write}.  However, deducing
> the correct addresses for these requires making assumptions about the
> internel format returned by qpci_iomap(), along with some ugly casts.
> 
> This patch changes the test to use the new qpci_mem{read,write} interfaces
> which is neater.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/ivshmem-test.c | 35 +++++++++++++++++++++++++++--------
>  1 file changed, 27 insertions(+), 8 deletions(-)
> 
> diff --git a/tests/ivshmem-test.c b/tests/ivshmem-test.c
> index f36bfe7..fbd8258 100644
> --- a/tests/ivshmem-test.c
> +++ b/tests/ivshmem-test.c
> @@ -93,6 +93,25 @@ static inline void out_reg(IVState *s, enum Reg reg, unsigned v)
>      global_qtest = qtest;
>  }
>  
> +static inline void read_mem(IVState *s, uint64_t off, void *buf, size_t len)
> +{
> +    QTestState *qtest = global_qtest;
> +
> +    global_qtest = s->qtest;
> +    qpci_memread(s->dev, s->mem_base + off, buf, len);
> +    global_qtest = qtest;
> +}
> +
> +static inline void write_mem(IVState *s, uint64_t off,
> +                             const void *buf, size_t len)
> +{
> +    QTestState *qtest = global_qtest;
> +
> +    global_qtest = s->qtest;
> +    qpci_memwrite(s->dev, s->mem_base + off, buf, len);
> +    global_qtest = qtest;
> +}
> +
>  static void cleanup_vm(IVState *s)
>  {
>      g_free(s->dev);
> @@ -169,7 +188,7 @@ static void test_ivshmem_single(void)
>      for (i = 0; i < G_N_ELEMENTS(data); i++) {
>          data[i] = i;
>      }
> -    qtest_memwrite(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
> +    write_mem(s, 0, data, sizeof(data));
>  
>      /* verify write */
>      for (i = 0; i < G_N_ELEMENTS(data); i++) {
> @@ -178,7 +197,7 @@ static void test_ivshmem_single(void)
>  
>      /* read it back and verify read */
>      memset(data, 0, sizeof(data));
> -    qtest_memread(s->qtest, (uintptr_t)s->mem_base, data, sizeof(data));
> +    read_mem(s, 0, data, sizeof(data));
>      for (i = 0; i < G_N_ELEMENTS(data); i++) {
>          g_assert_cmpuint(data[i], ==, i);
>      }
> @@ -201,29 +220,29 @@ static void test_ivshmem_pair(void)
>  
>      /* host write, guest 1 & 2 read */
>      memset(tmpshmem, 0x42, TMPSHMSIZE);
> -    qtest_memread(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
> +    read_mem(s1, 0, data, TMPSHMSIZE);
>      for (i = 0; i < TMPSHMSIZE; i++) {
>          g_assert_cmpuint(data[i], ==, 0x42);
>      }
> -    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
> +    read_mem(s2, 0, data, TMPSHMSIZE);
>      for (i = 0; i < TMPSHMSIZE; i++) {
>          g_assert_cmpuint(data[i], ==, 0x42);
>      }
>  
>      /* guest 1 write, guest 2 read */
>      memset(data, 0x43, TMPSHMSIZE);
> -    qtest_memwrite(s1->qtest, (uintptr_t)s1->mem_base, data, TMPSHMSIZE);
> +    write_mem(s1, 0, data, TMPSHMSIZE);
>      memset(data, 0, TMPSHMSIZE);
> -    qtest_memread(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
> +    read_mem(s2, 0, data, TMPSHMSIZE);
>      for (i = 0; i < TMPSHMSIZE; i++) {
>          g_assert_cmpuint(data[i], ==, 0x43);
>      }
>  
>      /* guest 2 write, guest 1 read */
>      memset(data, 0x44, TMPSHMSIZE);
> -    qtest_memwrite(s2->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
> +    write_mem(s2, 0, data, TMPSHMSIZE);
>      memset(data, 0, TMPSHMSIZE);
> -    qtest_memread(s1->qtest, (uintptr_t)s2->mem_base, data, TMPSHMSIZE);
> +    read_mem(s1, 0, data, TMPSHMSIZE);
>      for (i = 0; i < TMPSHMSIZE; i++) {
>          g_assert_cmpuint(data[i], ==, 0x44);
>      }

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-25  6:47   ` Alexey Kardashevskiy
@ 2016-10-25 12:16     ` David Gibson
  2016-10-26  1:18       ` Alexey Kardashevskiy
  0 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-25 12:16 UTC (permalink / raw)
  To: Alexey Kardashevskiy
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 3359 bytes --]

On Tue, Oct 25, 2016 at 05:47:43PM +1100, Alexey Kardashevskiy wrote:
> On 24/10/16 15:59, David Gibson wrote:
> > In the libqos PCI code we now have accessors both for registers (byte
> > significance preserving) and for streaming data (byte address order
> > preserving).  These exist in both the interface for qtest drivers and in
> > the machine specific backends.
> > 
> > However, the register-style accessors aren't actually necessary in the
> > backend.  They can be implemented in terms of the byte address order
> > preserving accessors by the libqos wrappers.  This works because PCI is
> > always little endian.
> > 
> > This does assume that the back end byte address order preserving accessors
> > will perform the equivalent of a single bus transaction for short lengths.
> > This is the case, and in fact they currently end up using the same
> > cpu_physical_memory_rw() implementation within the qtest accelerator.
> > 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> > Reviewed-by: Greg Kurz <groug@kaod.org>
> > ---
> >  tests/libqos/pci-pc.c    | 38 --------------------------------------
> >  tests/libqos/pci-spapr.c | 44 --------------------------------------------
> >  tests/libqos/pci.c       | 20 ++++++++++++++------
> >  tests/libqos/pci.h       |  8 --------
> >  4 files changed, 14 insertions(+), 96 deletions(-)
> > 
> 
> [...]
> 
> > diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> > index 2b08362..ce6ed08 100644
> > --- a/tests/libqos/pci.h
> > +++ b/tests/libqos/pci.h
> > @@ -27,18 +27,10 @@ struct QPCIBus {
> >      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
> >      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
> >  
> > -    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
> > -    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
> > -    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
> > -
> >      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> >      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> >      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> >  
> > -    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> > -    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> > -    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> > -
> >      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
> >      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
> >  
> > 
> 
> You added them in "libqos: Handle PCI IO de-multiplexing in common code"
> (few patched before) and removing them now - if you moved this patch
> earlier, it would reduce the series, or what do I miss?

Well, it can't go before the PIO / MMIO split, because on x86 the PIO
part is implemented with inw/outw instead of readw/writew, and those
don't have a memread/memwrite equivalent.

The change could go at the same time, but my feeling was that logical
separation of the steps was worth a bit of temporary extra code.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-25  7:01   ` Alexey Kardashevskiy
@ 2016-10-25 12:25     ` David Gibson
  2016-10-26  1:57       ` Alexey Kardashevskiy
  0 siblings, 1 reply; 31+ messages in thread
From: David Gibson @ 2016-10-25 12:25 UTC (permalink / raw)
  To: Alexey Kardashevskiy
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 5655 bytes --]

On Tue, Oct 25, 2016 at 06:01:41PM +1100, Alexey Kardashevskiy wrote:
> On 24/10/16 15:59, David Gibson wrote:
> > ide-test uses many explicit inb() / outb() operations for its IO, which
> > means it's not portable to non-x86 platforms.  This cleans it up to use
> > the libqos PCI accessors instead.
> > 
> > Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[snip]

> > -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> > +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> > +                                 uint64_t lba, int nblocks)
> >  {
> >      Read10CDB pkt = { .padding = 0 };
> >      int i;
> > @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >  
> >      /* Send Packet */
> >      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> > -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> > +        qpci_io_writew(dev, ide_base + reg_data,
> > +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
> 
> 
> cpu_to_le16 -> le16_to_cpu conversion here and below (at the very end) is
> not obvious. Right above this chunk the @pkt fields are initialized as BE:
> 
>  /* Construct SCSI CDB packet */
>  pkt.opcode = 0x28;
>  pkt.lba = cpu_to_be32(lba);
>  pkt.nblocks = cpu_to_be16(nblocks);
> 
> outw() seems to be CPU-endian, and qpci_io_writew() as well, or not?

outw() is guest CPU endian (which is stupid, but that's another
matter).  qpci_io_writew() is different - it is always LE, because PCI
devices are always LE (well, ok, nearly always).

So, yes, this is a bit confusing.  Here's what's going on:
  * the SCSI standard uses BE, so that's what we put into the
    packet structure
  * We need to transfer the packet to the device as a bytestream - so
    no endianness conversions
  * But.. we do so in 16-bit chunks
  * .. and qpci_io_writew() is designed to take CPU values and write
    them out as LE - ie, it contains an implicit cpu_to_le16()
  * So, we use le16_to_cpu() to cancel out that swap, so that we write
    out the bytes in the correct order

If this were a Linux driver here, then we'd use a writew_rep() type
primitive, instead of writew().  The rep / streaming variants don't
byteswap exactly because they're designed for streaming out bytestream
data, not word values.

Really what we want here is a memwrite() type operation, but that
doesn't quite work because this is PIO, not MMIO, so on x86 it turns
into CPU ins and outs which don't have "streaming" variants.


> >      }
> >  }
> >  
> > @@ -683,13 +731,17 @@ static void nsleep(int64_t nsecs)
> >  
> >  static uint8_t ide_wait_clear(uint8_t flag)
> >  {
> > +    QPCIDevice *dev;
> > +    void *bmdma_base, *ide_base;
> >      uint8_t data;
> >      time_t st;
> >  
> > +    dev = get_pci_device(&bmdma_base, &ide_base);
> > +
> >      /* Wait with a 5 second timeout */
> >      time(&st);
> >      while (true) {
> > -        data = inb(IDE_BASE + reg_status);
> > +        data = qpci_io_readb(dev, ide_base + reg_status);
> >          if (!(data & flag)) {
> >              return data;
> >          }
> > @@ -723,6 +775,8 @@ static void ide_wait_intr(int irq)
> >  
> >  static void cdrom_pio_impl(int nblocks)
> >  {
> > +    QPCIDevice *dev;
> > +    void *bmdma_base, *ide_base;
> >      FILE *fh;
> >      int patt_blocks = MAX(16, nblocks);
> >      size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
> > @@ -741,13 +795,15 @@ static void cdrom_pio_impl(int nblocks)
> >  
> >      ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
> >                     "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
> > +    dev = get_pci_device(&bmdma_base, &ide_base);
> >      qtest_irq_intercept_in(global_qtest, "ioapic");
> >  
> >      /* PACKET command on device 0 */
> > -    outb(IDE_BASE + reg_device, 0);
> > -    outb(IDE_BASE + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> > -    outb(IDE_BASE + reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> > -    outb(IDE_BASE + reg_command, CMD_PACKET);
> > +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> > +    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> > +    qpci_io_writeb(dev, ide_base + reg_lba_high,
> > +                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> > +    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
> >      /* HP0: Check_Status_A State */
> >      nsleep(400);
> >      data = ide_wait_clear(BSY);
> > @@ -756,7 +812,7 @@ static void cdrom_pio_impl(int nblocks)
> >      assert_bit_clear(data, ERR | DF | BSY);
> >  
> >      /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
> > -    send_scsi_cdb_read10(0, nblocks);
> > +    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
> >  
> >      /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
> >       * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
> > @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
> >  
> >          /* HP4: Transfer_Data */
> >          for (j = 0; j < MIN((limit / 2), rem); j++) {
> > -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> > +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> > +                                                       ide_base + reg_data));
> >          }
> >      }
> >  
> > 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO David Gibson
@ 2016-10-25 12:34   ` Greg Kurz
  0 siblings, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25 12:34 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 15:59:53 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> The usual model for PCI IO with libqos is to use qpci_iomap() to map a
> specific BAR for a PCI device, then perform IOs within that BAR using
> qpci_io_{read,write}*().
> 
> However, certain devices also have legacy PCI IO.  In this case, instead of
> (or as well as) being accessed via PCI BARs, the device can be accessed
> via certain well-known, fixed addresses in PCI IO space.
> 
> Two existing tests use legacy PCI IO, and take different flawed approaches
> to it:
>     * tco-test manually constructs a tco_io_base value instead of calling
>       qpci_iomap(), which assumes internal knowledge of the structure of
>       the value it shouldn't have
>     * ide-test uses direct in*() and out*() calls instead of using
>       qpci_io_*() accessors, meaning it's not portable to non-x86 machine
>       types.
> 
> This patch implements a new qpci_iomap_legacy() interface which gets a
> handle in the same format as qpci_iomap() but refers to a region in
> the legacy PIO space.  For a device which has the same registers
> available both in a BAR and in legacy space (quite common), this
> allows the same test code to test both options with just a different
> iomap() at the beginning.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---

The explanations provided in 20161022045011.GA19629@umbus.fritz.box are
enough for me.

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/libqos/pci.c | 5 +++++
>  tests/libqos/pci.h | 1 +
>  2 files changed, 6 insertions(+)
> 
> diff --git a/tests/libqos/pci.c b/tests/libqos/pci.c
> index bf1c532..98a2e56 100644
> --- a/tests/libqos/pci.c
> +++ b/tests/libqos/pci.c
> @@ -350,6 +350,11 @@ void qpci_iounmap(QPCIDevice *dev, void *data)
>      /* FIXME */
>  }
>  
> +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr)
> +{
> +    return (void *)(uintptr_t)addr;
> +}
> +
>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts)
>  {
> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> index f6f916d..b6f855e 100644
> --- a/tests/libqos/pci.h
> +++ b/tests/libqos/pci.h
> @@ -94,6 +94,7 @@ void qpci_io_writel(QPCIDevice *dev, void *data, uint32_t value);
>  
>  void *qpci_iomap(QPCIDevice *dev, int barno, uint64_t *sizeptr);
>  void qpci_iounmap(QPCIDevice *dev, void *data);
> +void *qpci_legacy_iomap(QPCIDevice *dev, uint16_t addr);
>  
>  void qpci_plug_device_test(const char *driver, const char *id,
>                             uint8_t slot, const char *opts);

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap()
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
@ 2016-10-25 12:35   ` Greg Kurz
  0 siblings, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25 12:35 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 15:59:54 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> Avoid tco-test making assumptions about the internal format of the address
> tokens passed to PCI IO accessors, by using the new qpci_legacy_iomap()
> function.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/tco-test.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/tco-test.c b/tests/tco-test.c
> index 0d201b1..129577d 100644
> --- a/tests/tco-test.c
> +++ b/tests/tco-test.c
> @@ -70,7 +70,7 @@ static void test_init(TestData *d)
>      /* set Root Complex BAR */
>      qpci_config_writel(d->dev, ICH9_LPC_RCBA, RCBA_BASE_ADDR | 0x1);
>  
> -    d->tco_io_base = (void *)((uintptr_t)PM_IO_BASE_ADDR + 0x60);
> +    d->tco_io_base = qpci_legacy_iomap(d->dev, PM_IO_BASE_ADDR + 0x60);
>  }
>  
>  static void stop_tco(const TestData *d)

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test David Gibson
  2016-10-25  7:01   ` Alexey Kardashevskiy
@ 2016-10-25 12:36   ` Greg Kurz
  1 sibling, 0 replies; 31+ messages in thread
From: Greg Kurz @ 2016-10-25 12:36 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

On Mon, 24 Oct 2016 15:59:57 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> ide-test uses many explicit inb() / outb() operations for its IO, which
> means it's not portable to non-x86 platforms.  This cleans it up to use
> the libqos PCI accessors instead.
> 
> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> ---

Reviewed-by: Greg Kurz <groug@kaod.org>

>  tests/ide-test.c | 179 ++++++++++++++++++++++++++++++++++++-------------------
>  1 file changed, 118 insertions(+), 61 deletions(-)
> 
> diff --git a/tests/ide-test.c b/tests/ide-test.c
> index a8a4081..86c4373 100644
> --- a/tests/ide-test.c
> +++ b/tests/ide-test.c
> @@ -137,7 +137,7 @@ static void ide_test_quit(void)
>      qtest_end();
>  }
>  
> -static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
> +static QPCIDevice *get_pci_device(void **bmdma_base, void **ide_base)
>  {
>      QPCIDevice *dev;
>      uint16_t vendor_id, device_id;
> @@ -156,7 +156,9 @@ static QPCIDevice *get_pci_device(uint16_t *bmdma_base)
>      g_assert(device_id == PCI_DEVICE_ID_INTEL_82371SB_1);
>  
>      /* Map bmdma BAR */
> -    *bmdma_base = (uint16_t)(uintptr_t) qpci_iomap(dev, 4, NULL);
> +    *bmdma_base = qpci_iomap(dev, 4, NULL);
> +
> +    *ide_base = qpci_legacy_iomap(dev, IDE_BASE);
>  
>      qpci_device_enable(dev);
>  
> @@ -179,17 +181,19 @@ typedef struct PrdtEntry {
>  
>  static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>                              PrdtEntry *prdt, int prdt_entries,
> -                            void(*post_exec)(uint64_t sector, int nb_sectors))
> +                            void(*post_exec)(QPCIDevice *dev, void *ide_base,
> +                                             uint64_t sector, int nb_sectors))
>  {
>      QPCIDevice *dev;
> -    uint16_t bmdma_base;
> +    void *bmdma_base;
> +    void *ide_base;
>      uintptr_t guest_prdt;
>      size_t len;
>      bool from_dev;
>      uint8_t status;
>      int flags;
>  
> -    dev = get_pci_device(&bmdma_base);
> +    dev = get_pci_device(&bmdma_base, &ide_base);
>  
>      flags = cmd & ~0xff;
>      cmd &= 0xff;
> @@ -214,59 +218,60 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>      }
>  
>      /* Select device 0 */
> -    outb(IDE_BASE + reg_device, 0 | LBA);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0 | LBA);
>  
>      /* Stop any running transfer, clear any pending interrupt */
> -    outb(bmdma_base + bmreg_cmd, 0);
> -    outb(bmdma_base + bmreg_status, BM_STS_INTR);
> +    qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
> +    qpci_io_writeb(dev, bmdma_base + bmreg_status, BM_STS_INTR);
>  
>      /* Setup PRDT */
>      len = sizeof(*prdt) * prdt_entries;
>      guest_prdt = guest_alloc(guest_malloc, len);
>      memwrite(guest_prdt, prdt, len);
> -    outl(bmdma_base + bmreg_prdt, guest_prdt);
> +    qpci_io_writel(dev, bmdma_base + bmreg_prdt, guest_prdt);
>  
>      /* ATA DMA command */
>      if (cmd == CMD_PACKET) {
>          /* Enables ATAPI DMA; otherwise PIO is attempted */
> -        outb(IDE_BASE + reg_feature, 0x01);
> +        qpci_io_writeb(dev, ide_base + reg_feature, 0x01);
>      } else {
> -        outb(IDE_BASE + reg_nsectors, nb_sectors);
> -        outb(IDE_BASE + reg_lba_low,    sector & 0xff);
> -        outb(IDE_BASE + reg_lba_middle, (sector >> 8) & 0xff);
> -        outb(IDE_BASE + reg_lba_high,   (sector >> 16) & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_nsectors, nb_sectors);
> +        qpci_io_writeb(dev, ide_base + reg_lba_low,    sector & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_lba_middle, (sector >> 8) & 0xff);
> +        qpci_io_writeb(dev, ide_base + reg_lba_high,   (sector >> 16) & 0xff);
>      }
>  
> -    outb(IDE_BASE + reg_command, cmd);
> +    qpci_io_writeb(dev, ide_base + reg_command, cmd);
>  
>      if (post_exec) {
> -        post_exec(sector, nb_sectors);
> +        post_exec(dev, ide_base, sector, nb_sectors);
>      }
>  
>      /* Start DMA transfer */
> -    outb(bmdma_base + bmreg_cmd, BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
> +    qpci_io_writeb(dev, bmdma_base + bmreg_cmd,
> +                   BM_CMD_START | (from_dev ? BM_CMD_WRITE : 0));
>  
>      if (flags & CMDF_ABORT) {
> -        outb(bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
>      }
>  
>      /* Wait for the DMA transfer to complete */
>      do {
> -        status = inb(bmdma_base + bmreg_status);
> +        status = qpci_io_readb(dev, bmdma_base + bmreg_status);
>      } while ((status & (BM_STS_ACTIVE | BM_STS_INTR)) == BM_STS_ACTIVE);
>  
>      g_assert_cmpint(get_irq(IDE_PRIMARY_IRQ), ==, !!(status & BM_STS_INTR));
>  
>      /* Check IDE status code */
> -    assert_bit_set(inb(IDE_BASE + reg_status), DRDY);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), BSY | DRQ);
> +    assert_bit_set(qpci_io_readb(dev, ide_base + reg_status), DRDY);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), BSY | DRQ);
>  
>      /* Reading the status register clears the IRQ */
>      g_assert(!get_irq(IDE_PRIMARY_IRQ));
>  
>      /* Stop DMA transfer if still active */
>      if (status & BM_STS_ACTIVE) {
> -        outb(bmdma_base + bmreg_cmd, 0);
> +        qpci_io_writeb(dev, bmdma_base + bmreg_cmd, 0);
>      }
>  
>      free_pci_device(dev);
> @@ -276,6 +281,8 @@ static int send_dma_request(int cmd, uint64_t sector, int nb_sectors,
>  
>  static void test_bmdma_simple_rw(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>      uint8_t *buf;
>      uint8_t *cmpbuf;
> @@ -289,6 +296,8 @@ static void test_bmdma_simple_rw(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      buf = g_malloc(len);
>      cmpbuf = g_malloc(len);
>  
> @@ -299,7 +308,7 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 0, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Write 0xaa pattern to sector 1 */
>      memset(buf, 0xaa, len);
> @@ -308,14 +317,14 @@ static void test_bmdma_simple_rw(void)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Read and verify 0x55 pattern in sector 0 */
>      memset(cmpbuf, 0x55, len);
>  
>      status = send_dma_request(CMD_READ_DMA, 0, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -325,7 +334,7 @@ static void test_bmdma_simple_rw(void)
>  
>      status = send_dma_request(CMD_READ_DMA, 1, 1, prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      memread(guest_buf, buf, len);
>      g_assert(memcmp(buf, cmpbuf, len) == 0);
> @@ -337,6 +346,8 @@ static void test_bmdma_simple_rw(void)
>  
>  static void test_bmdma_short_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -346,21 +357,25 @@ static void test_bmdma_short_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_one_sector_short_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      /* Read 2 sectors but only give 1 sector in PRDT */
> @@ -371,21 +386,25 @@ static void test_bmdma_one_sector_short_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 2,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, 0);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_long_prdt(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
>      PrdtEntry prdt[] = {
> @@ -395,23 +414,29 @@ static void test_bmdma_long_prdt(void)
>          },
>      };
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Normal request */
>      status = send_dma_request(CMD_READ_DMA, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      /* Abort the request before it completes */
>      status = send_dma_request(CMD_READ_DMA | CMDF_ABORT, 0, 1,
>                                prdt, ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_no_busmaster(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* No PRDT_EOT, each entry addr 0/size 64k, and in theory qemu shouldn't be
>       * able to access it anyway because the Bus Master bit in the PCI command
>       * register isn't set. This is complete nonsense, but it used to be pretty
> @@ -424,7 +449,7 @@ static void test_bmdma_no_busmaster(void)
>      /* Not entirely clear what the expected result is, but this is what we get
>       * in practice. At least we want to be aware of any changes. */
>      g_assert_cmphex(status, ==, BM_STS_ACTIVE | BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  }
>  
>  static void test_bmdma_setup(void)
> @@ -454,6 +479,8 @@ static void string_cpu_to_be16(uint16_t *s, size_t bytes)
>  
>  static void test_identify(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      uint16_t buf[256];
>      int i;
> @@ -464,23 +491,25 @@ static void test_identify(void)
>          "-global ide-hd.ver=%s",
>          tmp_path, "testdisk", "version");
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* IDENTIFY command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_IDENTIFY);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_IDENTIFY);
>  
>      /* Read in the IDENTIFY buffer and check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      for (i = 0; i < 256; i++) {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>          assert_bit_set(data, DRDY | DRQ);
>          assert_bit_clear(data, BSY | DF | ERR);
>  
> -        ((uint16_t*) buf)[i] = inw(IDE_BASE + reg_data);
> +        buf[i] = qpci_io_readw(dev, ide_base + reg_data);
>      }
>  
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, DRDY);
>      assert_bit_clear(data, BSY | DF | ERR | DRQ);
>  
> @@ -505,11 +534,15 @@ static void test_identify(void)
>   */
>  static void make_dirty(uint8_t device)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t status;
>      size_t len = 512;
>      uintptr_t guest_buf;
>      void* buf;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      guest_buf = guest_alloc(guest_malloc, len);
>      buf = g_malloc(len);
>      g_assert(guest_buf);
> @@ -527,19 +560,23 @@ static void make_dirty(uint8_t device)
>      status = send_dma_request(CMD_WRITE_DMA, 1, 1, prdt,
>                                ARRAY_SIZE(prdt), NULL);
>      g_assert_cmphex(status, ==, BM_STS_INTR);
> -    assert_bit_clear(inb(IDE_BASE + reg_status), DF | ERR);
> +    assert_bit_clear(qpci_io_readb(dev, ide_base + reg_status), DF | ERR);
>  
>      g_free(buf);
>  }
>  
>  static void test_flush(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>  
>      ide_test_start(
>          "-drive file=blkdebug::%s,if=ide,cache=writeback,format=raw",
>          tmp_path);
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
> @@ -549,11 +586,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"break flush_to_os A\""));
>  
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -561,11 +598,11 @@ static void test_flush(void)
>      g_free(hmp("qemu-io ide0-hd0 \"resume A\""));
>  
>      /* Check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -576,6 +613,8 @@ static void test_flush(void)
>  
>  static void test_retry_flush(const char *machine)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      const char *s;
>  
> @@ -587,17 +626,19 @@ static void test_retry_flush(const char *machine)
>          "rerror=stop,werror=stop",
>          debug_path, tmp_path);
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* Dirty media so that CMD_FLUSH_CACHE will actually go to disk */
>      make_dirty(0);
>  
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Check status while request is in flight*/
> -    data = inb(IDE_BASE + reg_status);
> +    data = qpci_io_readb(dev, ide_base + reg_status);
>      assert_bit_set(data, BSY | DRDY);
>      assert_bit_clear(data, DF | ERR | DRQ);
>  
> @@ -608,11 +649,11 @@ static void test_retry_flush(const char *machine)
>      qmp_discard_response(s);
>  
>      /* Check registers */
> -    data = inb(IDE_BASE + reg_device);
> +    data = qpci_io_readb(dev, ide_base + reg_device);
>      g_assert_cmpint(data & DEV, ==, 0);
>  
>      do {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>      } while (data & BSY);
>  
>      assert_bit_set(data, DRDY);
> @@ -623,11 +664,16 @@ static void test_retry_flush(const char *machine)
>  
>  static void test_flush_nodev(void)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
> +
>      ide_test_start("");
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* FLUSH CACHE command on device 0*/
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_command, CMD_FLUSH_CACHE);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_FLUSH_CACHE);
>  
>      /* Just testing that qemu doesn't crash... */
>  
> @@ -654,7 +700,8 @@ typedef struct Read10CDB {
>      uint16_t padding;
>  } __attribute__((__packed__)) Read10CDB;
>  
> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> +                                 uint64_t lba, int nblocks)
>  {
>      Read10CDB pkt = { .padding = 0 };
>      int i;
> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>  
>      /* Send Packet */
>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> +        qpci_io_writew(dev, ide_base + reg_data,
> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
>      }
>  }
>  
> @@ -683,13 +731,17 @@ static void nsleep(int64_t nsecs)
>  
>  static uint8_t ide_wait_clear(uint8_t flag)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      uint8_t data;
>      time_t st;
>  
> +    dev = get_pci_device(&bmdma_base, &ide_base);
> +
>      /* Wait with a 5 second timeout */
>      time(&st);
>      while (true) {
> -        data = inb(IDE_BASE + reg_status);
> +        data = qpci_io_readb(dev, ide_base + reg_status);
>          if (!(data & flag)) {
>              return data;
>          }
> @@ -723,6 +775,8 @@ static void ide_wait_intr(int irq)
>  
>  static void cdrom_pio_impl(int nblocks)
>  {
> +    QPCIDevice *dev;
> +    void *bmdma_base, *ide_base;
>      FILE *fh;
>      int patt_blocks = MAX(16, nblocks);
>      size_t patt_len = ATAPI_BLOCK_SIZE * patt_blocks;
> @@ -741,13 +795,15 @@ static void cdrom_pio_impl(int nblocks)
>  
>      ide_test_start("-drive if=none,file=%s,media=cdrom,format=raw,id=sr0,index=0 "
>                     "-device ide-cd,drive=sr0,bus=ide.0", tmp_path);
> +    dev = get_pci_device(&bmdma_base, &ide_base);
>      qtest_irq_intercept_in(global_qtest, "ioapic");
>  
>      /* PACKET command on device 0 */
> -    outb(IDE_BASE + reg_device, 0);
> -    outb(IDE_BASE + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> -    outb(IDE_BASE + reg_lba_high, (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> -    outb(IDE_BASE + reg_command, CMD_PACKET);
> +    qpci_io_writeb(dev, ide_base + reg_device, 0);
> +    qpci_io_writeb(dev, ide_base + reg_lba_middle, BYTE_COUNT_LIMIT & 0xFF);
> +    qpci_io_writeb(dev, ide_base + reg_lba_high,
> +                   (BYTE_COUNT_LIMIT >> 8 & 0xFF));
> +    qpci_io_writeb(dev, ide_base + reg_command, CMD_PACKET);
>      /* HP0: Check_Status_A State */
>      nsleep(400);
>      data = ide_wait_clear(BSY);
> @@ -756,7 +812,7 @@ static void cdrom_pio_impl(int nblocks)
>      assert_bit_clear(data, ERR | DF | BSY);
>  
>      /* SCSI CDB (READ10) -- read n*2048 bytes from block 0 */
> -    send_scsi_cdb_read10(0, nblocks);
> +    send_scsi_cdb_read10(dev, ide_base, 0, nblocks);
>  
>      /* Read data back: occurs in bursts of 'BYTE_COUNT_LIMIT' bytes.
>       * If BYTE_COUNT_LIMIT is odd, we transfer BYTE_COUNT_LIMIT - 1 bytes.
> @@ -780,7 +836,8 @@ static void cdrom_pio_impl(int nblocks)
>  
>          /* HP4: Transfer_Data */
>          for (j = 0; j < MIN((limit / 2), rem); j++) {
> -            rx[offset + j] = le16_to_cpu(inw(IDE_BASE + reg_data));
> +            rx[offset + j] = cpu_to_le16(qpci_io_readw(dev,
> +                                                       ide_base + reg_data));
>          }
>      }
>  

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling
  2016-10-25  3:35 ` [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
@ 2016-10-25 13:14   ` Greg Kurz
  2016-10-26  4:27     ` David Gibson
  0 siblings, 1 reply; 31+ messages in thread
From: Greg Kurz @ 2016-10-25 13:14 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 4241 bytes --]

On Tue, 25 Oct 2016 14:35:52 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Mon, Oct 24, 2016 at 03:59:49PM +1100, David Gibson wrote:
> > This series contains a number of cleanups to the libqos code for
> > accessing PCI devices, and to tests which use it.
> > 
> > The general aim is to improve the consistency of semantics across
> > functions, and reduce the amount of intimate knowledge of the libqos
> > PCI layer needed by tests.
> > 
> > This should make it easier to write PCI tests which will be portable
> > to different guest machines with different PCI host bridge
> > arrangements.
> > 
> > This series is on top of my ppc-for-2.8 branch, since it contains
> > patches enabling the virtio tests on ppc, which would otherwise
> > conflict with the changes here.  
> 
> Greg, Alexey, Michael,
> 
> Some reviews from outside RH would be really welcome.
> 

Done.

I also took time to run 'make check' with all targets on ppc64le, ppc64be,
i686 and ppc32 hosts.

Everything passes for ppc64le, ppc64be and i686.

It fails on ppc32 but this seems to be a TCG issue (QEMU fails with SIGILL
in code_gen_buffer()), not related to this patchset.

Tested-by: Greg Kurz <groug@kaod.org>

> > 
> > Changes since v4:
> >   * Fixed some remaining abstraction breaks in ahci-test
> >   * Removed QPCI_BAR_INVALID, turned out not to really be useful
> > 
> > Changes since v3:
> >   * Fixed another endian bug introduced in ide-test
> > 
> > Changes since v2:
> >   * Fixed build bugs in virtio-9p-test, which I didn't find earlier
> >     due to not having the right libraries installed
> >   * Fixed an endian bug I accidentally introduced in ide-test
> >   * Better handling of invalid BAR tokens
> > 
> > Changes since v1:
> >   * Split out updates to tco-test into separate patch
> >   * Split out updates to ide-test into separate patch
> >   * Neater and more general handling of legacy PIO addresses
> >   * Removed now-redundant fields from platform specific bus structures
> >   * Introduced CONFIG_BASE() macro to virtio-pci to remove many
> >     similar assignments
> >   * Fixed handling of two guest testcasesin ivshmem
> >   * Added 64-bit accessors
> >   * Rebase on ppc-for-2.8 to avoid conflict with Laurent's series in
> >     the same area
> > 
> > David Gibson (12):
> >   libqos: Give qvirtio_config_read*() consistent semantics
> >   libqos: Handle PCI IO de-multiplexing in common code
> >   libqos: Move BAR assignment to common code
> >   libqos: Better handling of PCI legacy IO
> >   tests: Adjust tco-test to use qpci_legacy_iomap()
> >   libqos: Add streaming accessors for PCI MMIO
> >   libqos: Implement mmio accessors in terms of mem{read,write}
> >   tests: Clean up IO handling in ide-test
> >   libqos: Add 64-bit PCI IO accessors
> >   tests: Use qpci_mem{read,write} in ivshmem-test
> >   tests: Don't assume structure of PCI IO base in ahci-test
> >   libqos: Change PCI accessors to take opaque BAR handle
> > 
> >  tests/ahci-test.c          |  13 +--
> >  tests/e1000e-test.c        |   7 +-
> >  tests/ide-test.c           | 177 +++++++++++++++++++++++++++--------------
> >  tests/ivshmem-test.c       |  47 +++++++----
> >  tests/libqos/ahci.c        |   4 +-
> >  tests/libqos/ahci.h        |   7 +-
> >  tests/libqos/pci-pc.c      | 187 ++++++++++---------------------------------
> >  tests/libqos/pci-spapr.c   | 194 ++++++++++++---------------------------------
> >  tests/libqos/pci.c         | 194 +++++++++++++++++++++++++++++++++++++--------
> >  tests/libqos/pci.h         |  66 ++++++++++-----
> >  tests/libqos/usb.c         |   6 +-
> >  tests/libqos/usb.h         |   2 +-
> >  tests/libqos/virtio-mmio.c |  16 ++--
> >  tests/libqos/virtio-pci.c  | 122 ++++++++++++++--------------
> >  tests/libqos/virtio-pci.h  |   2 +-
> >  tests/rtl8139-test.c       |  10 +--
> >  tests/tco-test.c           |  80 +++++++++----------
> >  tests/usb-hcd-ehci-test.c  |   5 +-
> >  tests/virtio-9p-test.c     |   8 +-
> >  tests/virtio-blk-test.c    |  42 +++-------
> >  tests/virtio-scsi-test.c   |   4 +-
> >  21 files changed, 598 insertions(+), 595 deletions(-)
> >   
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-25 12:16     ` David Gibson
@ 2016-10-26  1:18       ` Alexey Kardashevskiy
  2016-10-26  4:22         ` David Gibson
  0 siblings, 1 reply; 31+ messages in thread
From: Alexey Kardashevskiy @ 2016-10-26  1:18 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 3737 bytes --]

On 25/10/16 23:16, David Gibson wrote:
> On Tue, Oct 25, 2016 at 05:47:43PM +1100, Alexey Kardashevskiy wrote:
>> On 24/10/16 15:59, David Gibson wrote:
>>> In the libqos PCI code we now have accessors both for registers (byte
>>> significance preserving) and for streaming data (byte address order
>>> preserving).  These exist in both the interface for qtest drivers and in
>>> the machine specific backends.
>>>
>>> However, the register-style accessors aren't actually necessary in the
>>> backend.  They can be implemented in terms of the byte address order
>>> preserving accessors by the libqos wrappers.  This works because PCI is
>>> always little endian.
>>>
>>> This does assume that the back end byte address order preserving accessors
>>> will perform the equivalent of a single bus transaction for short lengths.
>>> This is the case, and in fact they currently end up using the same
>>> cpu_physical_memory_rw() implementation within the qtest accelerator.
>>>
>>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
>>> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
>>> Reviewed-by: Greg Kurz <groug@kaod.org>
>>> ---
>>>  tests/libqos/pci-pc.c    | 38 --------------------------------------
>>>  tests/libqos/pci-spapr.c | 44 --------------------------------------------
>>>  tests/libqos/pci.c       | 20 ++++++++++++++------
>>>  tests/libqos/pci.h       |  8 --------
>>>  4 files changed, 14 insertions(+), 96 deletions(-)
>>>
>>
>> [...]
>>
>>> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
>>> index 2b08362..ce6ed08 100644
>>> --- a/tests/libqos/pci.h
>>> +++ b/tests/libqos/pci.h
>>> @@ -27,18 +27,10 @@ struct QPCIBus {
>>>      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
>>>      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
>>>  
>>> -    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
>>> -    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
>>> -    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
>>> -
>>>      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
>>>      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
>>>      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
>>>  
>>> -    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
>>> -    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
>>> -    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
>>> -
>>>      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
>>>      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
>>>  
>>>
>>
>> You added them in "libqos: Handle PCI IO de-multiplexing in common code"
>> (few patched before) and removing them now - if you moved this patch
>> earlier, it would reduce the series, or what do I miss?
> 
> Well, it can't go before the PIO / MMIO split, because on x86 the PIO
> part is implemented with inw/outw instead of readw/writew, and those
> don't have a memread/memwrite equivalent.
> 
> The change could go at the same time, but my feeling was that logical
> separation of the steps was worth a bit of temporary extra code.

It is a bit hard to follow the logic of the patchset when you do not know
if the new code is going to stay or not - I automatically assumed it is
staying and when I saw it is being removed - I wondered if you are removing
what you just added, and this - in my opinion - kills the idea of making
smaller patches to make review easier, better just squash them all... But
since Greg is happy and things seems not working worse (make check fails on
my setup but whatever), you can ignore me :)


-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-25 12:25     ` David Gibson
@ 2016-10-26  1:57       ` Alexey Kardashevskiy
  2016-10-26  4:11         ` David Gibson
  0 siblings, 1 reply; 31+ messages in thread
From: Alexey Kardashevskiy @ 2016-10-26  1:57 UTC (permalink / raw)
  To: David Gibson
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 2868 bytes --]

On 25/10/16 23:25, David Gibson wrote:
> On Tue, Oct 25, 2016 at 06:01:41PM +1100, Alexey Kardashevskiy wrote:
>> On 24/10/16 15:59, David Gibson wrote:
>>> ide-test uses many explicit inb() / outb() operations for its IO, which
>>> means it's not portable to non-x86 platforms.  This cleans it up to use
>>> the libqos PCI accessors instead.
>>>
>>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> [snip]
> 
>>> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>>> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
>>> +                                 uint64_t lba, int nblocks)
>>>  {
>>>      Read10CDB pkt = { .padding = 0 };
>>>      int i;
>>> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
>>>  
>>>      /* Send Packet */
>>>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
>>> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
>>> +        qpci_io_writew(dev, ide_base + reg_data,
>>> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
>>
>>
>> cpu_to_le16 -> le16_to_cpu conversion here and below (at the very end) is
>> not obvious. Right above this chunk the @pkt fields are initialized as BE:
>>
>>  /* Construct SCSI CDB packet */
>>  pkt.opcode = 0x28;
>>  pkt.lba = cpu_to_be32(lba);
>>  pkt.nblocks = cpu_to_be16(nblocks);
>>
>> outw() seems to be CPU-endian, and qpci_io_writew() as well, or not?
> 
> outw() is guest CPU endian (which is stupid, but that's another
> matter).  qpci_io_writew() is different - it is always LE, because PCI
> devices are always LE (well, ok, nearly always).
> 
> So, yes, this is a bit confusing.  Here's what's going on:
>   * the SCSI standard uses BE, so that's what we put into the
>     packet structure
>   * We need to transfer the packet to the device as a bytestream - so
>     no endianness conversions
>   * But.. we do so in 16-bit chunks
>   * .. and qpci_io_writew() is designed to take CPU values and write
>     them out as LE - ie, it contains an implicit cpu_to_le16()

dev->bus->pio_writew() calls outw() which calls qtest_outw() and
qtest_sendf() where @value is a text - where does this implicit
cpu_to_le16() happen? Or I am reading the code wrong?

The other branch (for MMIO) in qpci_io_writew() calls cpu_to_le16() explicitly.

I'd expect a function with a generic name as qpci_io_writew() to always
take data in the some known and always the same endianness (LE in this case
as it is PCI).

In the chunk above we convert host-CPU-endian @lba to BE then treat it as
LE when converting to CPU-endian and then expect qpci_io_writew() to do
swapping again (or not - depends on BAR type - IO vs. MMIO - or conversion
always happens?), this confuses me a lot. However, everybody else is happy
so am I :)




-- 
Alexey


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 839 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test
  2016-10-26  1:57       ` Alexey Kardashevskiy
@ 2016-10-26  4:11         ` David Gibson
  0 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-26  4:11 UTC (permalink / raw)
  To: Alexey Kardashevskiy
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 4609 bytes --]

On Wed, Oct 26, 2016 at 12:57:26PM +1100, Alexey Kardashevskiy wrote:
> On 25/10/16 23:25, David Gibson wrote:
> > On Tue, Oct 25, 2016 at 06:01:41PM +1100, Alexey Kardashevskiy wrote:
> >> On 24/10/16 15:59, David Gibson wrote:
> >>> ide-test uses many explicit inb() / outb() operations for its IO, which
> >>> means it's not portable to non-x86 platforms.  This cleans it up to use
> >>> the libqos PCI accessors instead.
> >>>
> >>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> > [snip]
> > 
> >>> -static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >>> +static void send_scsi_cdb_read10(QPCIDevice *dev, void *ide_base,
> >>> +                                 uint64_t lba, int nblocks)
> >>>  {
> >>>      Read10CDB pkt = { .padding = 0 };
> >>>      int i;
> >>> @@ -670,7 +717,8 @@ static void send_scsi_cdb_read10(uint64_t lba, int nblocks)
> >>>  
> >>>      /* Send Packet */
> >>>      for (i = 0; i < sizeof(Read10CDB)/2; i++) {
> >>> -        outw(IDE_BASE + reg_data, cpu_to_le16(((uint16_t *)&pkt)[i]));
> >>> +        qpci_io_writew(dev, ide_base + reg_data,
> >>> +                       le16_to_cpu(((uint16_t *)&pkt)[i]));
> >>
> >>
> >> cpu_to_le16 -> le16_to_cpu conversion here and below (at the very end) is
> >> not obvious. Right above this chunk the @pkt fields are initialized as BE:
> >>
> >>  /* Construct SCSI CDB packet */
> >>  pkt.opcode = 0x28;
> >>  pkt.lba = cpu_to_be32(lba);
> >>  pkt.nblocks = cpu_to_be16(nblocks);
> >>
> >> outw() seems to be CPU-endian, and qpci_io_writew() as well, or not?
> > 
> > outw() is guest CPU endian (which is stupid, but that's another
> > matter).  qpci_io_writew() is different - it is always LE, because PCI
> > devices are always LE (well, ok, nearly always).
> > 
> > So, yes, this is a bit confusing.  Here's what's going on:
> >   * the SCSI standard uses BE, so that's what we put into the
> >     packet structure
> >   * We need to transfer the packet to the device as a bytestream - so
> >     no endianness conversions
> >   * But.. we do so in 16-bit chunks
> >   * .. and qpci_io_writew() is designed to take CPU values and write
> >     them out as LE - ie, it contains an implicit cpu_to_le16()
> 
> dev->bus->pio_writew() calls outw() which calls qtest_outw() and
> qtest_sendf() where @value is a text - where does this implicit
> cpu_to_le16() happen? Or I am reading the code wrong?

You're looking at the PC specific backend, which knows that the target
endianness is LE, and so target_to_le16() is a NOP.  The translation
from hsot to guest endianness happens down inside the outw logic.
qtest.c calls outw, which calls stw_p, which is defined to do the swap
for the target endianness in include/exec/cpu-all.h

If you look at the spapr backend, you'll see that the PIO callbacks
have an unconditional byteswap in them.  The spapr backend is ppc
specific which is notionally BE, so it always needs a swap in order to
get LE writes.

> The other branch (for MMIO) in qpci_io_writew() calls cpu_to_le16() explicitly.
> 
> I'd expect a function with a generic name as qpci_io_writew() to always
> take data in the some known and always the same endianness (LE in this case
> as it is PCI).

It does.  It's just the means to accomplishing that is a bit
convoluted for the PIO case.  That's exactly why I think the base
in/out operations should also be fixed endianness, rather than guest
endian, but that's an argument I'm having elsewhere.

> In the chunk above we convert host-CPU-endian @lba to BE then treat it as
> LE when converting to CPU-endian and then expect qpci_io_writew() to do
> swapping again (or not - depends on BAR type - IO vs. MMIO - or conversion
> always happens?), this confuses me a lot. However, everybody else is happy
> so am I :)

You need to think of this in two different parts.  Building the buffer
as a bytestream, which includes BE components.  Then sending the
buffer to the hardware as a bytestream.  This has balanced le<->cpu
conversions in order to preserve bytestream order.

Remember than endian is a property of a value - something that has a
specific length and location, not of a bytestream or bus of itself.
The fields in the request are BE, hence the BE conversions.  The data
*register* which we write stuff out to is treated as LE, hence the LE
conversions.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write}
  2016-10-26  1:18       ` Alexey Kardashevskiy
@ 2016-10-26  4:22         ` David Gibson
  0 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-26  4:22 UTC (permalink / raw)
  To: Alexey Kardashevskiy
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	mdroth, groug, thuth

[-- Attachment #1: Type: text/plain, Size: 4357 bytes --]

On Wed, Oct 26, 2016 at 12:18:03PM +1100, Alexey Kardashevskiy wrote:
> On 25/10/16 23:16, David Gibson wrote:
> > On Tue, Oct 25, 2016 at 05:47:43PM +1100, Alexey Kardashevskiy wrote:
> >> On 24/10/16 15:59, David Gibson wrote:
> >>> In the libqos PCI code we now have accessors both for registers (byte
> >>> significance preserving) and for streaming data (byte address order
> >>> preserving).  These exist in both the interface for qtest drivers and in
> >>> the machine specific backends.
> >>>
> >>> However, the register-style accessors aren't actually necessary in the
> >>> backend.  They can be implemented in terms of the byte address order
> >>> preserving accessors by the libqos wrappers.  This works because PCI is
> >>> always little endian.
> >>>
> >>> This does assume that the back end byte address order preserving accessors
> >>> will perform the equivalent of a single bus transaction for short lengths.
> >>> This is the case, and in fact they currently end up using the same
> >>> cpu_physical_memory_rw() implementation within the qtest accelerator.
> >>>
> >>> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
> >>> Reviewed-by: Laurent Vivier <lvivier@redhat.com>
> >>> Reviewed-by: Greg Kurz <groug@kaod.org>
> >>> ---
> >>>  tests/libqos/pci-pc.c    | 38 --------------------------------------
> >>>  tests/libqos/pci-spapr.c | 44 --------------------------------------------
> >>>  tests/libqos/pci.c       | 20 ++++++++++++++------
> >>>  tests/libqos/pci.h       |  8 --------
> >>>  4 files changed, 14 insertions(+), 96 deletions(-)
> >>>
> >>
> >> [...]
> >>
> >>> diff --git a/tests/libqos/pci.h b/tests/libqos/pci.h
> >>> index 2b08362..ce6ed08 100644
> >>> --- a/tests/libqos/pci.h
> >>> +++ b/tests/libqos/pci.h
> >>> @@ -27,18 +27,10 @@ struct QPCIBus {
> >>>      uint16_t (*pio_readw)(QPCIBus *bus, uint32_t addr);
> >>>      uint32_t (*pio_readl)(QPCIBus *bus, uint32_t addr);
> >>>  
> >>> -    uint8_t (*mmio_readb)(QPCIBus *bus, uint32_t addr);
> >>> -    uint16_t (*mmio_readw)(QPCIBus *bus, uint32_t addr);
> >>> -    uint32_t (*mmio_readl)(QPCIBus *bus, uint32_t addr);
> >>> -
> >>>      void (*pio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> >>>      void (*pio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> >>>      void (*pio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> >>>  
> >>> -    void (*mmio_writeb)(QPCIBus *bus, uint32_t addr, uint8_t value);
> >>> -    void (*mmio_writew)(QPCIBus *bus, uint32_t addr, uint16_t value);
> >>> -    void (*mmio_writel)(QPCIBus *bus, uint32_t addr, uint32_t value);
> >>> -
> >>>      void (*memread)(QPCIBus *bus, uint32_t addr, void *buf, size_t len);
> >>>      void (*memwrite)(QPCIBus *bus, uint32_t addr, const void *buf, size_t len);
> >>>  
> >>>
> >>
> >> You added them in "libqos: Handle PCI IO de-multiplexing in common code"
> >> (few patched before) and removing them now - if you moved this patch
> >> earlier, it would reduce the series, or what do I miss?
> > 
> > Well, it can't go before the PIO / MMIO split, because on x86 the PIO
> > part is implemented with inw/outw instead of readw/writew, and those
> > don't have a memread/memwrite equivalent.
> > 
> > The change could go at the same time, but my feeling was that logical
> > separation of the steps was worth a bit of temporary extra code.
> 
> It is a bit hard to follow the logic of the patchset when you do not know
> if the new code is going to stay or not - I automatically assumed it is
> staying and when I saw it is being removed - I wondered if you are removing
> what you just added, and this - in my opinion - kills the idea of making
> smaller patches to make review easier, better just squash them all... But
> since Greg is happy and things seems not working worse (make check fails on
> my setup but whatever), you can ignore me :)

Well, I guess it's a trade-off between conceptual simplicity and
minimal code changes.  Putting those callbacks in temporarily means
more code change, but I think it's worth it to make each patch
conceptually simpler.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

* Re: [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling
  2016-10-25 13:14   ` Greg Kurz
@ 2016-10-26  4:27     ` David Gibson
  0 siblings, 0 replies; 31+ messages in thread
From: David Gibson @ 2016-10-26  4:27 UTC (permalink / raw)
  To: Greg Kurz
  Cc: pbonzini, qemu-devel, qemu-ppc, lvivier, agraf, stefanha, mst,
	aik, mdroth, thuth

[-- Attachment #1: Type: text/plain, Size: 4771 bytes --]

On Tue, Oct 25, 2016 at 03:14:00PM +0200, Greg Kurz wrote:
> On Tue, 25 Oct 2016 14:35:52 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Mon, Oct 24, 2016 at 03:59:49PM +1100, David Gibson wrote:
> > > This series contains a number of cleanups to the libqos code for
> > > accessing PCI devices, and to tests which use it.
> > > 
> > > The general aim is to improve the consistency of semantics across
> > > functions, and reduce the amount of intimate knowledge of the libqos
> > > PCI layer needed by tests.
> > > 
> > > This should make it easier to write PCI tests which will be portable
> > > to different guest machines with different PCI host bridge
> > > arrangements.
> > > 
> > > This series is on top of my ppc-for-2.8 branch, since it contains
> > > patches enabling the virtio tests on ppc, which would otherwise
> > > conflict with the changes here.  
> > 
> > Greg, Alexey, Michael,
> > 
> > Some reviews from outside RH would be really welcome.
> > 
> 
> Done.
> 
> I also took time to run 'make check' with all targets on ppc64le, ppc64be,
> i686 and ppc32 hosts.
> 
> Everything passes for ppc64le, ppc64be and i686.
> 
> It fails on ppc32 but this seems to be a TCG issue (QEMU fails with SIGILL
> in code_gen_buffer()), not related to this patchset.
> 
> Tested-by: Greg Kurz <groug@kaod.org>

Thanks for the reviews; I've now merged this into ppc-for-2.8.

> 
> > > 
> > > Changes since v4:
> > >   * Fixed some remaining abstraction breaks in ahci-test
> > >   * Removed QPCI_BAR_INVALID, turned out not to really be useful
> > > 
> > > Changes since v3:
> > >   * Fixed another endian bug introduced in ide-test
> > > 
> > > Changes since v2:
> > >   * Fixed build bugs in virtio-9p-test, which I didn't find earlier
> > >     due to not having the right libraries installed
> > >   * Fixed an endian bug I accidentally introduced in ide-test
> > >   * Better handling of invalid BAR tokens
> > > 
> > > Changes since v1:
> > >   * Split out updates to tco-test into separate patch
> > >   * Split out updates to ide-test into separate patch
> > >   * Neater and more general handling of legacy PIO addresses
> > >   * Removed now-redundant fields from platform specific bus structures
> > >   * Introduced CONFIG_BASE() macro to virtio-pci to remove many
> > >     similar assignments
> > >   * Fixed handling of two guest testcasesin ivshmem
> > >   * Added 64-bit accessors
> > >   * Rebase on ppc-for-2.8 to avoid conflict with Laurent's series in
> > >     the same area
> > > 
> > > David Gibson (12):
> > >   libqos: Give qvirtio_config_read*() consistent semantics
> > >   libqos: Handle PCI IO de-multiplexing in common code
> > >   libqos: Move BAR assignment to common code
> > >   libqos: Better handling of PCI legacy IO
> > >   tests: Adjust tco-test to use qpci_legacy_iomap()
> > >   libqos: Add streaming accessors for PCI MMIO
> > >   libqos: Implement mmio accessors in terms of mem{read,write}
> > >   tests: Clean up IO handling in ide-test
> > >   libqos: Add 64-bit PCI IO accessors
> > >   tests: Use qpci_mem{read,write} in ivshmem-test
> > >   tests: Don't assume structure of PCI IO base in ahci-test
> > >   libqos: Change PCI accessors to take opaque BAR handle
> > > 
> > >  tests/ahci-test.c          |  13 +--
> > >  tests/e1000e-test.c        |   7 +-
> > >  tests/ide-test.c           | 177 +++++++++++++++++++++++++++--------------
> > >  tests/ivshmem-test.c       |  47 +++++++----
> > >  tests/libqos/ahci.c        |   4 +-
> > >  tests/libqos/ahci.h        |   7 +-
> > >  tests/libqos/pci-pc.c      | 187 ++++++++++---------------------------------
> > >  tests/libqos/pci-spapr.c   | 194 ++++++++++++---------------------------------
> > >  tests/libqos/pci.c         | 194 +++++++++++++++++++++++++++++++++++++--------
> > >  tests/libqos/pci.h         |  66 ++++++++++-----
> > >  tests/libqos/usb.c         |   6 +-
> > >  tests/libqos/usb.h         |   2 +-
> > >  tests/libqos/virtio-mmio.c |  16 ++--
> > >  tests/libqos/virtio-pci.c  | 122 ++++++++++++++--------------
> > >  tests/libqos/virtio-pci.h  |   2 +-
> > >  tests/rtl8139-test.c       |  10 +--
> > >  tests/tco-test.c           |  80 +++++++++----------
> > >  tests/usb-hcd-ehci-test.c  |   5 +-
> > >  tests/virtio-9p-test.c     |   8 +-
> > >  tests/virtio-blk-test.c    |  42 +++-------
> > >  tests/virtio-scsi-test.c   |   4 +-
> > >  21 files changed, 598 insertions(+), 595 deletions(-)
> > >   
> > 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 31+ messages in thread

end of thread, other threads:[~2016-10-26  4:27 UTC | newest]

Thread overview: 31+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-24  4:59 [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 01/12] libqos: Give qvirtio_config_read*() consistent semantics David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 02/12] libqos: Handle PCI IO de-multiplexing in common code David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 03/12] libqos: Move BAR assignment to " David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 04/12] libqos: Better handling of PCI legacy IO David Gibson
2016-10-25 12:34   ` Greg Kurz
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 05/12] tests: Adjust tco-test to use qpci_legacy_iomap() David Gibson
2016-10-25 12:35   ` Greg Kurz
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 06/12] libqos: Add streaming accessors for PCI MMIO David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 07/12] libqos: Implement mmio accessors in terms of mem{read, write} David Gibson
2016-10-25  6:47   ` Alexey Kardashevskiy
2016-10-25 12:16     ` David Gibson
2016-10-26  1:18       ` Alexey Kardashevskiy
2016-10-26  4:22         ` David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 08/12] tests: Clean up IO handling in ide-test David Gibson
2016-10-25  7:01   ` Alexey Kardashevskiy
2016-10-25 12:25     ` David Gibson
2016-10-26  1:57       ` Alexey Kardashevskiy
2016-10-26  4:11         ` David Gibson
2016-10-25 12:36   ` Greg Kurz
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 09/12] libqos: Add 64-bit PCI IO accessors David Gibson
2016-10-24  4:59 ` [Qemu-devel] [PATCHv5 10/12] tests: Use qpci_mem{read, write} in ivshmem-test David Gibson
2016-10-25  9:23   ` Greg Kurz
2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 11/12] tests: Don't assume structure of PCI IO base in ahci-test David Gibson
2016-10-24 16:50   ` John Snow
2016-10-25  8:58   ` Greg Kurz
2016-10-24  5:00 ` [Qemu-devel] [PATCHv5 12/12] libqos: Change PCI accessors to take opaque BAR handle David Gibson
2016-10-25  9:21   ` Greg Kurz
2016-10-25  3:35 ` [Qemu-devel] [PATCHv5 00/12] Cleanups to qtest PCI handling David Gibson
2016-10-25 13:14   ` Greg Kurz
2016-10-26  4:27     ` David Gibson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.