* [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support
@ 2016-11-03 9:27 Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 01/11] intel_iommu: fixing source id during IOTLB hash key calculation Jason Wang
` (11 more replies)
0 siblings, 12 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
Hi all:
As the userspace vitio driver became popular, more and more request
were received for secure DMA environemt (DMAR). So this series tries
to makeDMAR works for virtio/vhost. The idea is let virtio/vhost
co-work with userspace iommu implememtation. This is done through:
- for virtio, when platform supports IOMMU (VIRTIO_F_IOMMU_PLATFORM),
virtio will not assume address_space_memory, instead a transport
specific method were introduced for querying the dma address space
and dma helpers were used in device emulation codes.
- for vhost, implement a device IOTLB by using device IOTLB API
supported by recent kernel. With this API, vhost kernel can query
IOTLB entry for a specified iova from qemu, qemu can invalidate an
arbitrary range of iova in vhost kernel.
The device IOTLB API is totaly architecture independent, an example
implementation was done with intel iommu by:
- implement basic ATS (Address Translation Service) for virtio-pci,
this will make device IOTLB visible for iommu driver in guest.
- implement device IOTLB descriptor processing in intel iommu (enabled
through device-iotlb=on), and trigger the device IOTLB invalidation
in vhost through iommu notifier.
It could be easily ported to other IOMMU or architecture even if it
doesn't support device IOTLB. (e.g just invalidate the vhost IOTLB
during IOMMU IOTLB invalidation).
AMD IOMMU suppot for device IOTLB is ready, but it depends on other
fixes to work correctly.
Test was done by:
- intel_iommu=on/strict in guest.
- vfio l2fwd in guest.
This main use case is the programs that use fixed mapping in guest
(e.g dpdk). If 1G hugepage were used in guest, thanks to the SLLPS
support, we can get 100% TLB hit rate for l2fwd in guest.
For the normal kernel driver which uses lots of dynamic mapping and
unmapping, we may see performance penalty, this could be optimized in
the future.
TODO:
- non ATS support (userspace IOTLB snooping)
- more platforms and IOMMU support (AMD IOMMU support is ongoing)
- performance optimizations (e.g merging adjacent mappings)
Changes from V1:
- rebase to HEAD
- avoid calling transport specific dma as fetching method each time by
caching it in vdev
- convert to use new IOMMU notifier API
- silent checkpatch warnings and fix 32bit build
- use "device-iotlb" instead of "device_iotlb"
- rename virtio_memory_map() to vhost_memory_map() and move it to vhost.c
- use memory_region_is_iommu() instead of inventing new one
Changes from RFC:
- rebase to HEAD
- switch to use new vhost device IOTLB API
- use the new feature bit VIRITO_F_IOMMU_PLATFORM
- finalize basic ATS implementation
- add ATSR for Root port ATS transaction
- fix the iommu notifier handling during unregistering
- use snprintf() in patch 3
- correc the loop in address_space_get_iotlb_entry()
- small tweak on the address calculation during device iotlb
descriptor processing.
Jason Wang (11):
intel_iommu: fixing source id during IOTLB hash key calculation
virtio: convert to use DMA api
intel_iommu: name vtd address space with devfn
intel_iommu: allocate new key when creating new address space
exec: introduce address_space_get_iotlb_entry()
intel_iommu: support device iotlb descriptor
virtio-pci: address space translation service (ATS) support
acpi: add ATSR for q35
memory: handle alias for iommu notifier
memory: handle alias in memory_region_is_iommu()
vhost_net: device IOTLB support
exec.c | 33 ++++++
hw/block/virtio-blk.c | 2 +-
hw/char/virtio-serial-bus.c | 3 +-
hw/i386/acpi-build.c | 9 ++
hw/i386/intel_iommu.c | 90 +++++++++++++--
hw/i386/intel_iommu_internal.h | 15 ++-
hw/i386/x86-iommu.c | 17 +++
hw/pci/pcie.c | 16 +++
hw/scsi/virtio-scsi.c | 4 +-
hw/virtio/vhost-backend.c | 104 +++++++++++++++++
hw/virtio/vhost.c | 179 ++++++++++++++++++++++++++----
hw/virtio/virtio-bus.c | 8 ++
hw/virtio/virtio-pci.c | 21 ++++
hw/virtio/virtio-pci.h | 4 +
hw/virtio/virtio.c | 57 ++++++----
include/exec/memory.h | 9 ++
include/hw/acpi/acpi-defs.h | 12 ++
include/hw/i386/x86-iommu.h | 1 +
include/hw/pci/pcie.h | 4 +
include/hw/virtio/vhost-backend.h | 13 +++
include/hw/virtio/vhost.h | 4 +
include/hw/virtio/virtio-access.h | 40 +++++--
include/hw/virtio/virtio-bus.h | 1 +
include/hw/virtio/virtio.h | 9 +-
include/standard-headers/linux/pci_regs.h | 1 +
memory.c | 9 ++
net/tap.c | 1 +
27 files changed, 593 insertions(+), 73 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 01/11] intel_iommu: fixing source id during IOTLB hash key calculation
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api Jason Wang
` (10 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Richard Henderson,
Eduardo Habkost
Using uint8_t for source id will lose bus num and get the
wrong/invalid IOTLB entry. Fixing by using uint16_t instead and
enlarge level shift.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/i386/intel_iommu.c | 2 +-
hw/i386/intel_iommu_internal.h | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 1655a65..5a12ae7 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -218,7 +218,7 @@ static void vtd_reset_iotlb(IntelIOMMUState *s)
g_hash_table_remove_all(s->iotlb);
}
-static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint8_t source_id,
+static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id,
uint32_t level)
{
return gfn | ((uint64_t)(source_id) << VTD_IOTLB_SID_SHIFT) |
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 0829a50..11abfa2 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -115,7 +115,7 @@
/* The shift of source_id in the key of IOTLB hash table */
#define VTD_IOTLB_SID_SHIFT 36
-#define VTD_IOTLB_LVL_SHIFT 44
+#define VTD_IOTLB_LVL_SHIFT 52
#define VTD_IOTLB_MAX_SIZE 1024 /* Max size of the hash table */
/* IOTLB_REG */
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 01/11] intel_iommu: fixing source id during IOTLB hash key calculation Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 19:46 ` Michael S. Tsirkin
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 03/11] intel_iommu: name vtd address space with devfn Jason Wang
` (9 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Stefan Hajnoczi,
Kevin Wolf, Amit Shah, qemu-block
Currently, all virtio devices bypass IOMMU completely. This is because
address_space_memory is assumed and used during DMA emulation. This
patch converts the virtio core API to use DMA API. This idea is
- introducing a new transport specific helper to query the dma address
space. (only pci version is implemented).
- query and use this address space during virtio device guest memory
accessing when iommu platform (VIRTIO_F_IOMMU_PLATFORM) was enabled
for this device.
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Amit Shah <amit.shah@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: qemu-block@nongnu.org
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/block/virtio-blk.c | 2 +-
hw/char/virtio-serial-bus.c | 3 ++-
hw/scsi/virtio-scsi.c | 4 ++-
hw/virtio/virtio-bus.c | 8 ++++++
hw/virtio/virtio-pci.c | 14 ++++++++++
hw/virtio/virtio.c | 57 +++++++++++++++++++++++++--------------
include/hw/virtio/virtio-access.h | 36 ++++++++++++++++++-------
include/hw/virtio/virtio-bus.h | 1 +
include/hw/virtio/virtio.h | 9 ++++---
9 files changed, 98 insertions(+), 36 deletions(-)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 37fe72b..6a2dbaf 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -860,7 +860,7 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
}
}
- req = qemu_get_virtqueue_element(f, sizeof(VirtIOBlockReq));
+ req = qemu_get_virtqueue_element(vdev, f, sizeof(VirtIOBlockReq));
virtio_blk_init_request(s, virtio_get_queue(vdev, vq_idx), req);
req->next = s->rq;
s->rq = req;
diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
index 7975c2c..d544cd9 100644
--- a/hw/char/virtio-serial-bus.c
+++ b/hw/char/virtio-serial-bus.c
@@ -732,6 +732,7 @@ static void virtio_serial_post_load_timer_cb(void *opaque)
static int fetch_active_ports_list(QEMUFile *f,
VirtIOSerial *s, uint32_t nr_active_ports)
{
+ VirtIODevice *vdev = VIRTIO_DEVICE(s);
uint32_t i;
s->post_load = g_malloc0(sizeof(*s->post_load));
@@ -765,7 +766,7 @@ static int fetch_active_ports_list(QEMUFile *f,
qemu_get_be64s(f, &port->iov_offset);
port->elem =
- qemu_get_virtqueue_element(f, sizeof(VirtQueueElement));
+ qemu_get_virtqueue_element(vdev, f, sizeof(VirtQueueElement));
/*
* Port was throttled on source machine. Let's
diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
index 4762f05..1519cee 100644
--- a/hw/scsi/virtio-scsi.c
+++ b/hw/scsi/virtio-scsi.c
@@ -198,12 +198,14 @@ static void *virtio_scsi_load_request(QEMUFile *f, SCSIRequest *sreq)
SCSIBus *bus = sreq->bus;
VirtIOSCSI *s = container_of(bus, VirtIOSCSI, bus);
VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
+ VirtIODevice *vdev = VIRTIO_DEVICE(s);
VirtIOSCSIReq *req;
uint32_t n;
qemu_get_be32s(f, &n);
assert(n < vs->conf.num_queues);
- req = qemu_get_virtqueue_element(f, sizeof(VirtIOSCSIReq) + vs->cdb_size);
+ req = qemu_get_virtqueue_element(vdev, f,
+ sizeof(VirtIOSCSIReq) + vs->cdb_size);
virtio_scsi_init_req(s, vs->cmd_vqs[n], req);
if (virtio_scsi_parse_req(req, sizeof(VirtIOSCSICmdReq) + vs->cdb_size,
diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
index 11f65bd..8597762 100644
--- a/hw/virtio/virtio-bus.c
+++ b/hw/virtio/virtio-bus.c
@@ -28,6 +28,7 @@
#include "hw/qdev.h"
#include "hw/virtio/virtio-bus.h"
#include "hw/virtio/virtio.h"
+#include "exec/address-spaces.h"
/* #define DEBUG_VIRTIO_BUS */
@@ -61,6 +62,13 @@ void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
if (klass->device_plugged != NULL) {
klass->device_plugged(qbus->parent, errp);
}
+
+ if (klass->get_dma_as != NULL &&
+ virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
+ vdev->dma_as = klass->get_dma_as(qbus->parent);
+ } else {
+ vdev->dma_as = &address_space_memory;
+ }
}
/* Reset the virtio_bus */
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 06831de..6ceb43e 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1168,6 +1168,14 @@ static int virtio_pci_query_nvectors(DeviceState *d)
return proxy->nvectors;
}
+static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
+{
+ VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
+ PCIDevice *dev = &proxy->pci_dev;
+
+ return pci_get_address_space(dev);
+}
+
static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
struct virtio_pci_cap *cap)
{
@@ -1624,6 +1632,11 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)
}
if (legacy) {
+ if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
+ error_setg(errp, "VIRTIO_F_IOMMU_PLATFORM was supported by"
+ "neither legacy nor transitional device.");
+ return ;
+ }
/* legacy and transitional */
pci_set_word(config + PCI_SUBSYSTEM_VENDOR_ID,
pci_get_word(config + PCI_VENDOR_ID));
@@ -2544,6 +2557,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
k->ioeventfd_disabled = virtio_pci_ioeventfd_disabled;
k->ioeventfd_set_disabled = virtio_pci_ioeventfd_set_disabled;
k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
+ k->get_dma_as = virtio_pci_get_dma_as;
}
static const TypeInfo virtio_pci_bus_info = {
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index d48d1a9..85ad828 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -23,6 +23,7 @@
#include "hw/virtio/virtio-bus.h"
#include "migration/migration.h"
#include "hw/virtio/virtio-access.h"
+#include "sysemu/dma.h"
/*
* The alignment to use between consumer and producer parts of vring.
@@ -122,7 +123,7 @@ void virtio_queue_update_rings(VirtIODevice *vdev, int n)
static void vring_desc_read(VirtIODevice *vdev, VRingDesc *desc,
hwaddr desc_pa, int i)
{
- address_space_read(&address_space_memory, desc_pa + i * sizeof(VRingDesc),
+ address_space_read(virtio_get_dma_as(vdev), desc_pa + i * sizeof(VRingDesc),
MEMTXATTRS_UNSPECIFIED, (void *)desc, sizeof(VRingDesc));
virtio_tswap64s(vdev, &desc->addr);
virtio_tswap32s(vdev, &desc->len);
@@ -164,7 +165,7 @@ static inline void vring_used_write(VirtQueue *vq, VRingUsedElem *uelem,
virtio_tswap32s(vq->vdev, &uelem->id);
virtio_tswap32s(vq->vdev, &uelem->len);
pa = vq->vring.used + offsetof(VRingUsed, ring[i]);
- address_space_write(&address_space_memory, pa, MEMTXATTRS_UNSPECIFIED,
+ address_space_write(virtio_get_dma_as(vq->vdev), pa, MEMTXATTRS_UNSPECIFIED,
(void *)uelem, sizeof(VRingUsedElem));
}
@@ -244,6 +245,7 @@ int virtio_queue_empty(VirtQueue *vq)
static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
unsigned int len)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vq->vdev);
unsigned int offset;
int i;
@@ -251,17 +253,18 @@ static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
for (i = 0; i < elem->in_num; i++) {
size_t size = MIN(len - offset, elem->in_sg[i].iov_len);
- cpu_physical_memory_unmap(elem->in_sg[i].iov_base,
- elem->in_sg[i].iov_len,
- 1, size);
+ dma_memory_unmap(dma_as, elem->in_sg[i].iov_base,
+ elem->in_sg[i].iov_len,
+ DMA_DIRECTION_FROM_DEVICE, size);
offset += size;
}
for (i = 0; i < elem->out_num; i++)
- cpu_physical_memory_unmap(elem->out_sg[i].iov_base,
- elem->out_sg[i].iov_len,
- 0, elem->out_sg[i].iov_len);
+ dma_memory_unmap(dma_as, elem->out_sg[i].iov_base,
+ elem->out_sg[i].iov_len,
+ DMA_DIRECTION_TO_DEVICE,
+ elem->out_sg[i].iov_len);
}
/* virtqueue_detach_element:
@@ -555,7 +558,10 @@ static bool virtqueue_map_desc(VirtIODevice *vdev, unsigned int *p_num_sg,
goto out;
}
- iov[num_sg].iov_base = cpu_physical_memory_map(pa, &len, is_write);
+ iov[num_sg].iov_base = dma_memory_map(virtio_get_dma_as(vdev), pa, &len,
+ is_write ?
+ DMA_DIRECTION_FROM_DEVICE :
+ DMA_DIRECTION_TO_DEVICE);
if (!iov[num_sg].iov_base) {
virtio_error(vdev, "virtio: bogus descriptor or out of resources");
goto out;
@@ -592,9 +598,9 @@ static void virtqueue_undo_map_desc(unsigned int out_num, unsigned int in_num,
}
}
-static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
- unsigned int *num_sg, unsigned int max_size,
- int is_write)
+static void virtqueue_map_iovec(VirtIODevice *vdev, struct iovec *sg,
+ hwaddr *addr, unsigned int *num_sg,
+ unsigned int max_size, int is_write)
{
unsigned int i;
hwaddr len;
@@ -613,7 +619,10 @@ static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
for (i = 0; i < *num_sg; i++) {
len = sg[i].iov_len;
- sg[i].iov_base = cpu_physical_memory_map(addr[i], &len, is_write);
+ sg[i].iov_base = dma_memory_map(virtio_get_dma_as(vdev),
+ addr[i], &len, is_write ?
+ DMA_DIRECTION_FROM_DEVICE :
+ DMA_DIRECTION_TO_DEVICE);
if (!sg[i].iov_base) {
error_report("virtio: error trying to map MMIO memory");
exit(1);
@@ -625,12 +634,15 @@ static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
}
}
-void virtqueue_map(VirtQueueElement *elem)
+void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem)
{
- virtqueue_map_iovec(elem->in_sg, elem->in_addr, &elem->in_num,
- VIRTQUEUE_MAX_SIZE, 1);
- virtqueue_map_iovec(elem->out_sg, elem->out_addr, &elem->out_num,
- VIRTQUEUE_MAX_SIZE, 0);
+ virtqueue_map_iovec(vdev, elem->in_sg, elem->in_addr, &elem->in_num,
+ MIN(ARRAY_SIZE(elem->in_sg), ARRAY_SIZE(elem->in_addr)),
+ 1);
+ virtqueue_map_iovec(vdev, elem->out_sg, elem->out_addr, &elem->out_num,
+ MIN(ARRAY_SIZE(elem->out_sg),
+ ARRAY_SIZE(elem->out_addr)),
+ 0);
}
void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num)
@@ -783,7 +795,7 @@ typedef struct VirtQueueElementOld {
struct iovec out_sg[VIRTQUEUE_MAX_SIZE];
} VirtQueueElementOld;
-void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz)
+void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz)
{
VirtQueueElement *elem;
VirtQueueElementOld data;
@@ -814,7 +826,7 @@ void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz)
elem->out_sg[i].iov_len = data.out_sg[i].iov_len;
}
- virtqueue_map(elem);
+ virtqueue_map(vdev, elem);
return elem;
}
@@ -873,6 +885,11 @@ static int virtio_validate_features(VirtIODevice *vdev)
{
VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
+ if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM) &&
+ !virtio_vdev_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
+ return -EFAULT;
+ }
+
if (k->validate_features) {
return k->validate_features(vdev);
} else {
diff --git a/include/hw/virtio/virtio-access.h b/include/hw/virtio/virtio-access.h
index 440b455..4643a06 100644
--- a/include/hw/virtio/virtio-access.h
+++ b/include/hw/virtio/virtio-access.h
@@ -17,12 +17,18 @@
#define QEMU_VIRTIO_ACCESS_H
#include "hw/virtio/virtio.h"
+#include "hw/virtio/virtio-bus.h"
#include "exec/address-spaces.h"
#if defined(TARGET_PPC64) || defined(TARGET_ARM)
#define LEGACY_VIRTIO_IS_BIENDIAN 1
#endif
+static inline AddressSpace *virtio_get_dma_as(VirtIODevice *vdev)
+{
+ return vdev->dma_as;
+}
+
static inline bool virtio_access_is_big_endian(VirtIODevice *vdev)
{
#if defined(LEGACY_VIRTIO_IS_BIENDIAN)
@@ -40,45 +46,55 @@ static inline bool virtio_access_is_big_endian(VirtIODevice *vdev)
static inline uint16_t virtio_lduw_phys(VirtIODevice *vdev, hwaddr pa)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
if (virtio_access_is_big_endian(vdev)) {
- return lduw_be_phys(&address_space_memory, pa);
+ return lduw_be_phys(dma_as, pa);
}
- return lduw_le_phys(&address_space_memory, pa);
+ return lduw_le_phys(dma_as, pa);
}
static inline uint32_t virtio_ldl_phys(VirtIODevice *vdev, hwaddr pa)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
if (virtio_access_is_big_endian(vdev)) {
- return ldl_be_phys(&address_space_memory, pa);
+ return ldl_be_phys(dma_as, pa);
}
- return ldl_le_phys(&address_space_memory, pa);
+ return ldl_le_phys(dma_as, pa);
}
static inline uint64_t virtio_ldq_phys(VirtIODevice *vdev, hwaddr pa)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
if (virtio_access_is_big_endian(vdev)) {
- return ldq_be_phys(&address_space_memory, pa);
+ return ldq_be_phys(dma_as, pa);
}
- return ldq_le_phys(&address_space_memory, pa);
+ return ldq_le_phys(dma_as, pa);
}
static inline void virtio_stw_phys(VirtIODevice *vdev, hwaddr pa,
uint16_t value)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
if (virtio_access_is_big_endian(vdev)) {
- stw_be_phys(&address_space_memory, pa, value);
+ stw_be_phys(dma_as, pa, value);
} else {
- stw_le_phys(&address_space_memory, pa, value);
+ stw_le_phys(dma_as, pa, value);
}
}
static inline void virtio_stl_phys(VirtIODevice *vdev, hwaddr pa,
uint32_t value)
{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
if (virtio_access_is_big_endian(vdev)) {
- stl_be_phys(&address_space_memory, pa, value);
+ stl_be_phys(dma_as, pa, value);
} else {
- stl_le_phys(&address_space_memory, pa, value);
+ stl_le_phys(dma_as, pa, value);
}
}
diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
index 2e4b67e..9d5aa79 100644
--- a/include/hw/virtio/virtio-bus.h
+++ b/include/hw/virtio/virtio-bus.h
@@ -98,6 +98,7 @@ typedef struct VirtioBusClass {
* Note that changing this will break migration for this transport.
*/
bool has_variable_vring_alignment;
+ AddressSpace *(*get_dma_as)(DeviceState *d);
} VirtioBusClass;
struct VirtioBusState {
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index b913aac..1245831 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -92,6 +92,7 @@ struct VirtIODevice
char *bus_name;
uint8_t device_endian;
bool use_guest_notifier_mask;
+ AddressSpace *dma_as;
QLIST_HEAD(, VirtQueue) *vector_queues;
};
@@ -163,9 +164,9 @@ bool virtqueue_rewind(VirtQueue *vq, unsigned int num);
void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
unsigned int len, unsigned int idx);
-void virtqueue_map(VirtQueueElement *elem);
+void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem);
void *virtqueue_pop(VirtQueue *vq, size_t sz);
-void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz);
+void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz);
void qemu_put_virtqueue_element(QEMUFile *f, VirtQueueElement *elem);
int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes,
unsigned int out_bytes);
@@ -247,7 +248,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
DEFINE_PROP_BIT64("notify_on_empty", _state, _field, \
VIRTIO_F_NOTIFY_ON_EMPTY, true), \
DEFINE_PROP_BIT64("any_layout", _state, _field, \
- VIRTIO_F_ANY_LAYOUT, true)
+ VIRTIO_F_ANY_LAYOUT, true), \
+ DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
+ VIRTIO_F_IOMMU_PLATFORM, false)
hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
hwaddr virtio_queue_get_avail_addr(VirtIODevice *vdev, int n);
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 03/11] intel_iommu: name vtd address space with devfn
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 01/11] intel_iommu: fixing source id during IOTLB hash key calculation Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 04/11] intel_iommu: allocate new key when creating new address space Jason Wang
` (8 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Richard Henderson,
Eduardo Habkost
To avoid duplicated name and ease debugging.
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/i386/intel_iommu.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 5a12ae7..833c7a6 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2323,6 +2323,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
uintptr_t key = (uintptr_t)bus;
VTDBus *vtd_bus = g_hash_table_lookup(s->vtd_as_by_busptr, &key);
VTDAddressSpace *vtd_dev_as;
+ char name[128];
if (!vtd_bus) {
/* No corresponding free() */
@@ -2336,6 +2337,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
vtd_dev_as = vtd_bus->dev_as[devfn];
if (!vtd_dev_as) {
+ snprintf(name, sizeof(name), "intel_iommu_devfn_%d", devfn);
vtd_bus->dev_as[devfn] = vtd_dev_as = g_malloc0(sizeof(VTDAddressSpace));
vtd_dev_as->bus = bus;
@@ -2350,7 +2352,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
memory_region_add_subregion(&vtd_dev_as->iommu, VTD_INTERRUPT_ADDR_FIRST,
&vtd_dev_as->iommu_ir);
address_space_init(&vtd_dev_as->as,
- &vtd_dev_as->iommu, "intel_iommu");
+ &vtd_dev_as->iommu, name);
}
return vtd_dev_as;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 04/11] intel_iommu: allocate new key when creating new address space
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (2 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 03/11] intel_iommu: name vtd address space with devfn Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry() Jason Wang
` (7 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Richard Henderson,
Eduardo Habkost
We use the pointer to stack for key for new address space, this will break hash
table searching, fixing by g_malloc() a new key instead.
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/i386/intel_iommu.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 833c7a6..eb57cbf 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2326,12 +2326,13 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
char name[128];
if (!vtd_bus) {
+ uintptr_t *new_key = g_malloc(sizeof(*new_key));
+ *new_key = (uintptr_t)bus;
/* No corresponding free() */
vtd_bus = g_malloc0(sizeof(VTDBus) + sizeof(VTDAddressSpace *) * \
X86_IOMMU_PCI_DEVFN_MAX);
vtd_bus->bus = bus;
- key = (uintptr_t)bus;
- g_hash_table_insert(s->vtd_as_by_busptr, &key, vtd_bus);
+ g_hash_table_insert(s->vtd_as_by_busptr, new_key, vtd_bus);
}
vtd_dev_as = vtd_bus->dev_as[devfn];
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry()
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (3 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 04/11] intel_iommu: allocate new key when creating new address space Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 17:03 ` Paolo Bonzini
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 06/11] intel_iommu: support device iotlb descriptor Jason Wang
` (6 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Peter Crosthwaite,
Richard Henderson
This patch introduces a helper to query the iotlb entry for a
possible iova. This will be used by later device IOTLB API to enable
the capability for a dataplane (e.g vhost) to query the IOTLB.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Crosthwaite <crosthwaite.peter@gmail.com>
Cc: Richard Henderson <rth@twiddle.net>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
exec.c | 33 +++++++++++++++++++++++++++++++++
include/exec/memory.h | 6 ++++++
2 files changed, 39 insertions(+)
diff --git a/exec.c b/exec.c
index b1094c0..00c7a2b 100644
--- a/exec.c
+++ b/exec.c
@@ -449,6 +449,39 @@ address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *x
}
/* Called from RCU critical section */
+IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
+ bool is_write)
+{
+ IOMMUTLBEntry iotlb = {0};
+ MemoryRegionSection *section;
+ MemoryRegion *mr;
+ hwaddr plen;
+
+ for (;;) {
+ AddressSpaceDispatch *d = atomic_rcu_read(&as->dispatch);
+ section = address_space_translate_internal(d, addr, &addr, &plen, true);
+ mr = section->mr;
+
+ if (!mr->iommu_ops) {
+ break;
+ }
+
+ iotlb = mr->iommu_ops->translate(mr, addr, is_write);
+ addr = ((iotlb.translated_addr & ~iotlb.addr_mask)
+ | (addr & iotlb.addr_mask));
+ plen = MIN(plen, (addr | iotlb.addr_mask) - addr + 1);
+ if (!(iotlb.perm & (1 << is_write))) {
+ iotlb.target_as = NULL;
+ break;
+ }
+
+ as = iotlb.target_as;
+ }
+
+ return iotlb;
+}
+
+/* Called from RCU critical section */
MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
hwaddr *xlat, hwaddr *plen,
bool is_write)
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 9728a2f..e605de3 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -1404,6 +1404,12 @@ void address_space_stq_le(AddressSpace *as, hwaddr addr, uint64_t val,
void address_space_stq_be(AddressSpace *as, hwaddr addr, uint64_t val,
MemTxAttrs attrs, MemTxResult *result);
+/* address_space_get_iotlb_entry: translate an address into an IOTLB
+ * entry. Should be called from an RCU critical section.
+ */
+IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
+ bool is_write);
+
/* address_space_translate: translate an address range into an address space
* into a MemoryRegion and an address range into that section. Should be
* called from an RCU critical section, to avoid that the last reference
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 06/11] intel_iommu: support device iotlb descriptor
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (4 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry() Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support Jason Wang
` (5 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang, Richard Henderson,
Eduardo Habkost
This patch enables device IOTLB support for intel iommu. The major
work is to implement QI device IOTLB descriptor processing and notify
the device through iommu notifier.
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/i386/intel_iommu.c | 79 ++++++++++++++++++++++++++++++++++++++----
hw/i386/intel_iommu_internal.h | 13 +++++--
hw/i386/x86-iommu.c | 17 +++++++++
include/hw/i386/x86-iommu.h | 1 +
4 files changed, 102 insertions(+), 8 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index eb57cbf..bb41426 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -738,11 +738,18 @@ static int vtd_dev_to_context_entry(IntelIOMMUState *s, uint8_t bus_num,
"context-entry hi 0x%"PRIx64 " lo 0x%"PRIx64,
ce->hi, ce->lo);
return -VTD_FR_CONTEXT_ENTRY_INV;
- } else if (ce->lo & VTD_CONTEXT_ENTRY_TT) {
- VTD_DPRINTF(GENERAL, "error: unsupported Translation Type in "
- "context-entry hi 0x%"PRIx64 " lo 0x%"PRIx64,
- ce->hi, ce->lo);
- return -VTD_FR_CONTEXT_ENTRY_INV;
+ } else {
+ switch (ce->lo & VTD_CONTEXT_ENTRY_TT) {
+ case VTD_CONTEXT_TT_MULTI_LEVEL:
+ /* fall through */
+ case VTD_CONTEXT_TT_DEV_IOTLB:
+ break;
+ default:
+ VTD_DPRINTF(GENERAL, "error: unsupported Translation Type in "
+ "context-entry hi 0x%"PRIx64 " lo 0x%"PRIx64,
+ ce->hi, ce->lo);
+ return -VTD_FR_CONTEXT_ENTRY_INV;
+ }
}
return 0;
}
@@ -1437,7 +1444,59 @@ static bool vtd_process_inv_iec_desc(IntelIOMMUState *s,
vtd_iec_notify_all(s, !inv_desc->iec.granularity,
inv_desc->iec.index,
inv_desc->iec.index_mask);
+ return true;
+}
+
+static bool vtd_process_device_iotlb_desc(IntelIOMMUState *s,
+ VTDInvDesc *inv_desc)
+{
+ VTDAddressSpace *vtd_dev_as;
+ IOMMUTLBEntry entry;
+ struct VTDBus *vtd_bus;
+ hwaddr addr;
+ uint64_t sz;
+ uint16_t sid;
+ uint8_t devfn;
+ bool size;
+ uint8_t bus_num;
+
+ addr = VTD_INV_DESC_DEVICE_IOTLB_ADDR(inv_desc->hi);
+ sid = VTD_INV_DESC_DEVICE_IOTLB_SID(inv_desc->lo);
+ devfn = sid & 0xff;
+ bus_num = sid >> 8;
+ size = VTD_INV_DESC_DEVICE_IOTLB_SIZE(inv_desc->hi);
+
+ if ((inv_desc->lo & VTD_INV_DESC_DEVICE_IOTLB_RSVD_LO) ||
+ (inv_desc->hi & VTD_INV_DESC_DEVICE_IOTLB_RSVD_HI)) {
+ VTD_DPRINTF(GENERAL, "error: non-zero reserved field in Device "
+ "IOTLB Invalidate Descriptor hi 0x%"PRIx64 " lo 0x%"PRIx64,
+ inv_desc->hi, inv_desc->lo);
+ return false;
+ }
+ vtd_bus = vtd_find_as_from_bus_num(s, bus_num);
+ if (!vtd_bus) {
+ goto done;
+ }
+
+ vtd_dev_as = vtd_bus->dev_as[devfn];
+ if (!vtd_dev_as) {
+ goto done;
+ }
+
+ if (size) {
+ sz = 1 << (ctz64(~(addr | (VTD_PAGE_MASK_4K - 1))) + 1);
+ addr &= ~(sz - 1);
+ } else {
+ sz = VTD_PAGE_SIZE;
+ }
+
+ entry.target_as = &vtd_dev_as->as;
+ entry.addr_mask = sz - 1;
+ entry.iova = addr;
+ memory_region_notify_iommu(entry.target_as->root, entry);
+
+done:
return true;
}
@@ -1489,6 +1548,14 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s)
}
break;
+ case VTD_INV_DESC_DEVICE:
+ VTD_DPRINTF(INV, "Device IOTLB Invalidation Descriptor hi 0x%"PRIx64
+ " lo 0x%"PRIx64, inv_desc.hi, inv_desc.lo);
+ if (!vtd_process_device_iotlb_desc(s, &inv_desc)) {
+ return false;
+ }
+ break;
+
default:
VTD_DPRINTF(GENERAL, "error: unkonw Invalidation Descriptor type "
"hi 0x%"PRIx64 " lo 0x%"PRIx64 " type %"PRIu8,
@@ -2384,7 +2451,7 @@ static void vtd_init(IntelIOMMUState *s)
s->next_frcd_reg = 0;
s->cap = VTD_CAP_FRO | VTD_CAP_NFR | VTD_CAP_ND | VTD_CAP_MGAW |
VTD_CAP_SAGAW | VTD_CAP_MAMV | VTD_CAP_PSI | VTD_CAP_SLLPS;
- s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO;
+ s->ecap = VTD_ECAP_QI | VTD_ECAP_DT | VTD_ECAP_IRO;
if (x86_iommu->intr_supported) {
s->ecap |= VTD_ECAP_IR | VTD_ECAP_MHMV;
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 11abfa2..356f188 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -183,6 +183,7 @@
/* (offset >> 4) << 8 */
#define VTD_ECAP_IRO (DMAR_IOTLB_REG_OFFSET << 4)
#define VTD_ECAP_QI (1ULL << 1)
+#define VTD_ECAP_DT (1ULL << 2)
/* Interrupt Remapping support */
#define VTD_ECAP_IR (1ULL << 3)
#define VTD_ECAP_EIM (1ULL << 4)
@@ -326,6 +327,7 @@ typedef union VTDInvDesc VTDInvDesc;
#define VTD_INV_DESC_TYPE 0xf
#define VTD_INV_DESC_CC 0x1 /* Context-cache Invalidate Desc */
#define VTD_INV_DESC_IOTLB 0x2
+#define VTD_INV_DESC_DEVICE 0x3
#define VTD_INV_DESC_IEC 0x4 /* Interrupt Entry Cache
Invalidate Descriptor */
#define VTD_INV_DESC_WAIT 0x5 /* Invalidation Wait Descriptor */
@@ -361,6 +363,13 @@ typedef union VTDInvDesc VTDInvDesc;
#define VTD_INV_DESC_IOTLB_RSVD_LO 0xffffffff0000ff00ULL
#define VTD_INV_DESC_IOTLB_RSVD_HI 0xf80ULL
+/* Mask for Device IOTLB Invalidate Descriptor */
+#define VTD_INV_DESC_DEVICE_IOTLB_ADDR(val) ((val) & 0xfffffffffffff000ULL)
+#define VTD_INV_DESC_DEVICE_IOTLB_SIZE(val) ((val) & 0x1)
+#define VTD_INV_DESC_DEVICE_IOTLB_SID(val) (((val) >> 32) & 0xFFFFULL)
+#define VTD_INV_DESC_DEVICE_IOTLB_RSVD_HI 0xffeULL
+#define VTD_INV_DESC_DEVICE_IOTLB_RSVD_LO 0xffff0000ffe0fff8
+
/* Information about page-selective IOTLB invalidate */
struct VTDIOTLBPageInvInfo {
uint16_t domain_id;
@@ -399,8 +408,8 @@ typedef struct VTDRootEntry VTDRootEntry;
#define VTD_CONTEXT_ENTRY_FPD (1ULL << 1) /* Fault Processing Disable */
#define VTD_CONTEXT_ENTRY_TT (3ULL << 2) /* Translation Type */
#define VTD_CONTEXT_TT_MULTI_LEVEL 0
-#define VTD_CONTEXT_TT_DEV_IOTLB 1
-#define VTD_CONTEXT_TT_PASS_THROUGH 2
+#define VTD_CONTEXT_TT_DEV_IOTLB (1ULL << 2)
+#define VTD_CONTEXT_TT_PASS_THROUGH (2ULL << 2)
/* Second Level Page Translation Pointer*/
#define VTD_CONTEXT_ENTRY_SLPTPTR (~0xfffULL)
#define VTD_CONTEXT_ENTRY_RSVD_LO (0xff0ULL | ~VTD_HAW_MASK)
diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
index 2278af7..23dcd3f 100644
--- a/hw/i386/x86-iommu.c
+++ b/hw/i386/x86-iommu.c
@@ -106,6 +106,18 @@ static void x86_iommu_intremap_prop_set(Object *o, bool value, Error **errp)
s->intr_supported = value;
}
+static bool x86_iommu_device_iotlb_prop_get(Object *o, Error **errp)
+{
+ X86IOMMUState *s = X86_IOMMU_DEVICE(o);
+ return s->dt_supported;
+}
+
+static void x86_iommu_device_iotlb_prop_set(Object *o, bool value, Error **errp)
+{
+ X86IOMMUState *s = X86_IOMMU_DEVICE(o);
+ s->dt_supported = value;
+}
+
static void x86_iommu_instance_init(Object *o)
{
X86IOMMUState *s = X86_IOMMU_DEVICE(o);
@@ -114,6 +126,11 @@ static void x86_iommu_instance_init(Object *o)
s->intr_supported = false;
object_property_add_bool(o, "intremap", x86_iommu_intremap_prop_get,
x86_iommu_intremap_prop_set, NULL);
+ s->dt_supported = false;
+ object_property_add_bool(o, "device-iotlb",
+ x86_iommu_device_iotlb_prop_get,
+ x86_iommu_device_iotlb_prop_set,
+ NULL);
}
static const TypeInfo x86_iommu_info = {
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index 0c89d98..361c07c 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -73,6 +73,7 @@ typedef struct IEC_Notifier IEC_Notifier;
struct X86IOMMUState {
SysBusDevice busdev;
bool intr_supported; /* Whether vIOMMU supports IR */
+ bool dt_supported; /* Whether vIOMMU supports DT */
IommuType type; /* IOMMU type - AMD/Intel */
QLIST_HEAD(, IEC_Notifier) iec_notifiers; /* IEC notify list */
};
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (5 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 06/11] intel_iommu: support device iotlb descriptor Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 19:49 ` Michael S. Tsirkin
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 08/11] acpi: add ATSR for q35 Jason Wang
` (4 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
This patches enable the Address Translation Service support for virtio
pci devices. This is needed for a guest visible Device IOTLB
implementation and will be required by vhost device IOTLB API
implementation for intel IOMMU.
Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/pci/pcie.c | 16 ++++++++++++++++
hw/virtio/virtio-pci.c | 7 +++++++
hw/virtio/virtio-pci.h | 4 ++++
include/hw/pci/pcie.h | 4 ++++
include/standard-headers/linux/pci_regs.h | 1 +
5 files changed, 32 insertions(+)
diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
index 99cfb45..02195d9 100644
--- a/hw/pci/pcie.c
+++ b/hw/pci/pcie.c
@@ -717,3 +717,19 @@ void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num)
PCI_EXT_CAP_DSN_SIZEOF);
pci_set_quad(dev->config + offset + pci_dsn_cap, ser_num);
}
+
+void pcie_ats_init(PCIDevice *dev, uint16_t offset)
+{
+ pcie_add_capability(dev, PCI_EXT_CAP_ID_ATS, 0x1,
+ offset, PCI_EXT_CAP_ATS_SIZEOF);
+
+ dev->exp.ats_cap = offset;
+
+ /* Invalidate Queue Depth 0, Page Aligned Request 0 */
+ pci_set_word(dev->config + offset + PCI_ATS_CAP, 0);
+ /* STU 0, Disabled by default */
+ pci_set_word(dev->config + offset + PCI_ATS_CTRL, 0);
+
+ pci_set_word(dev->wmask + dev->exp.ats_cap + PCI_ATS_CTRL, 0x800f);
+}
+
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index 6ceb43e..e357bdf 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1838,6 +1838,11 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp)
* PCI Power Management Interface Specification.
*/
pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3);
+
+ if (proxy->flags & VIRTIO_PCI_FLAG_ATS) {
+ pcie_ats_init(pci_dev, 256);
+ }
+
} else {
/*
* make future invocations of pci_is_express() return false
@@ -1889,6 +1894,8 @@ static Property virtio_pci_properties[] = {
VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT, false),
DEFINE_PROP_BIT("page-per-vq", VirtIOPCIProxy, flags,
VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT, false),
+ DEFINE_PROP_BIT("ats", VirtIOPCIProxy, flags,
+ VIRTIO_PCI_FLAG_ATS_BIT, false),
DEFINE_PROP_END_OF_LIST(),
};
diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h
index b4edea6..057d49d 100644
--- a/hw/virtio/virtio-pci.h
+++ b/hw/virtio/virtio-pci.h
@@ -69,6 +69,7 @@ enum {
VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY_BIT,
VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT,
VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT,
+ VIRTIO_PCI_FLAG_ATS_BIT,
};
/* Need to activate work-arounds for buggy guests at vmstate load. */
@@ -93,6 +94,9 @@ enum {
#define VIRTIO_PCI_FLAG_PAGE_PER_VQ \
(1 << VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT)
+/* address space translation service */
+#define VIRTIO_PCI_FLAG_ATS (1 << VIRTIO_PCI_FLAG_ATS_BIT)
+
typedef struct {
MSIMessage msg;
int virq;
diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h
index 056d25e..b08451d 100644
--- a/include/hw/pci/pcie.h
+++ b/include/hw/pci/pcie.h
@@ -74,6 +74,9 @@ struct PCIExpressDevice {
/* AER */
uint16_t aer_cap;
PCIEAERLog aer_log;
+
+ /* Offset of ATS capability in config space */
+ uint16_t ats_cap;
};
#define COMPAT_PROP_PCP "power_controller_present"
@@ -120,6 +123,7 @@ void pcie_add_capability(PCIDevice *dev,
void pcie_ari_init(PCIDevice *dev, uint16_t offset, uint16_t nextfn);
void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num);
+void pcie_ats_init(PCIDevice *dev, uint16_t offset);
extern const VMStateDescription vmstate_pcie_device;
diff --git a/include/standard-headers/linux/pci_regs.h b/include/standard-headers/linux/pci_regs.h
index 4040951..ac426a0 100644
--- a/include/standard-headers/linux/pci_regs.h
+++ b/include/standard-headers/linux/pci_regs.h
@@ -674,6 +674,7 @@
#define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_DPC
#define PCI_EXT_CAP_DSN_SIZEOF 12
+#define PCI_EXT_CAP_ATS_SIZEOF 8
#define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
/* Advanced Error Reporting */
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 08/11] acpi: add ATSR for q35
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (6 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier Jason Wang
` (3 subsequent siblings)
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
This patch provides ATSR which was a requirement for software that
wants to enable ATS on endpoint devices behind a Root Port. This is
done simply by setting ALL_PORTS which indicates all PCI-Express Root
Ports support ATS transactions.
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/i386/acpi-build.c | 9 +++++++++
include/hw/acpi/acpi-defs.h | 12 ++++++++++++
2 files changed, 21 insertions(+)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 5cd1da9..2d10b0f 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2568,6 +2568,7 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linker)
AcpiTableDmar *dmar;
AcpiDmarHardwareUnit *drhd;
+ AcpiDmarRootPortATS *atsr;
uint8_t dmar_flags = 0;
X86IOMMUState *iommu = x86_iommu_get_default();
AcpiDmarDeviceScope *scope = NULL;
@@ -2600,6 +2601,14 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linker)
scope->bus = Q35_PSEUDO_BUS_PLATFORM;
scope->path[0] = cpu_to_le16(Q35_PSEUDO_DEVFN_IOAPIC);
+ if (iommu->dt_supported) {
+ atsr = acpi_data_push(table_data, sizeof(*atsr));
+ atsr->type = cpu_to_le16(ACPI_DMAR_TYPE_ATSR);
+ atsr->length = cpu_to_le16(sizeof(*atsr));
+ atsr->flags = ACPI_DMAR_ATSR_ALL_PORTS;
+ atsr->pci_segment = cpu_to_le16(0);
+ }
+
build_header(linker, table_data, (void *)(table_data->data + dmar_start),
"DMAR", table_data->len - dmar_start, 1, NULL, NULL);
}
diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index d1d1d61..420ea36 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -635,8 +635,20 @@ struct AcpiDmarHardwareUnit {
} QEMU_PACKED;
typedef struct AcpiDmarHardwareUnit AcpiDmarHardwareUnit;
+/* Type 2: Root Port ATS Capability Reporting Structure */
+struct AcpiDmarRootPortATS {
+ uint16_t type;
+ uint16_t length;
+ uint8_t flags;
+ uint8_t reserved;
+ uint16_t pci_segment;
+ AcpiDmarDeviceScope scope[0];
+} QEMU_PACKED;
+typedef struct AcpiDmarRootPortATS AcpiDmarRootPortATS;
+
/* Masks for Flags field above */
#define ACPI_DMAR_INCLUDE_PCI_ALL 1
+#define ACPI_DMAR_ATSR_ALL_PORTS 1
/*
* Input Output Remapping Table (IORT)
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (7 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 08/11] acpi: add ATSR for q35 Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 16:55 ` Paolo Bonzini
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu() Jason Wang
` (2 subsequent siblings)
11 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
memory.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/memory.c b/memory.c
index 33110e9..2bfc37f 100644
--- a/memory.c
+++ b/memory.c
@@ -1603,6 +1603,11 @@ static void memory_region_update_iommu_notify_flags(MemoryRegion *mr)
void memory_region_register_iommu_notifier(MemoryRegion *mr,
IOMMUNotifier *n)
{
+ if (mr->alias) {
+ memory_region_register_iommu_notifier(mr->alias, n);
+ return;
+ }
+
/* We need to register for at least one bitfield */
assert(n->notifier_flags != IOMMU_NOTIFIER_NONE);
QLIST_INSERT_HEAD(&mr->iommu_notify, n, node);
@@ -1643,6 +1648,10 @@ void memory_region_iommu_replay(MemoryRegion *mr, IOMMUNotifier *n,
void memory_region_unregister_iommu_notifier(MemoryRegion *mr,
IOMMUNotifier *n)
{
+ if (mr->alias) {
+ memory_region_unregister_iommu_notifier(mr->alias, n);
+ return;
+ }
QLIST_REMOVE(n, node);
memory_region_update_iommu_notify_flags(mr);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu()
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (8 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 16:56 ` Paolo Bonzini
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 11/11] vhost_net: device IOTLB support Jason Wang
2016-11-03 9:48 ` [Qemu-devel] [PATCH V2 00/11] vhost " no-reply
11 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
include/exec/memory.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/exec/memory.h b/include/exec/memory.h
index e605de3..ab37499 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -628,6 +628,9 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
*/
static inline bool memory_region_is_iommu(MemoryRegion *mr)
{
+ if (mr->alias) {
+ return memory_region_is_iommu(mr->alias);
+ }
return mr->iommu_ops;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [PATCH V2 11/11] vhost_net: device IOTLB support
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (9 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu() Jason Wang
@ 2016-11-03 9:27 ` Jason Wang
2016-11-03 9:48 ` [Qemu-devel] [PATCH V2 00/11] vhost " no-reply
11 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-03 9:27 UTC (permalink / raw)
To: mst, peterx, wexu, qemu-devel
Cc: vkaplans, pbonzini, cornelia.huck, Jason Wang
This patches implements Device IOTLB support for vhost kernel. This is
done through:
1) switch to use dma helpers when map/unmap vrings from vhost codes
2) kernel support for Device IOTLB API:
- allow vhost-net to query the IOMMU IOTLB entry through eventfd
- enable the ability for qemu to update a specified mapping of vhost
- through ioctl.
- enable the ability to invalidate a specified range of iova for the
device IOTLB of vhost through ioctl. In x86/intel_iommu case this is
triggered through iommu memory region notifier from device IOTLB
invalidation descriptor processing routine.
With all the above, kernel vhost_net can co-operate with IOMMU.
Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
---
hw/virtio/vhost-backend.c | 104 ++++++++++++++++++++++
hw/virtio/vhost.c | 179 +++++++++++++++++++++++++++++++++-----
include/hw/virtio/vhost-backend.h | 13 +++
include/hw/virtio/vhost.h | 4 +
include/hw/virtio/virtio-access.h | 4 +-
net/tap.c | 1 +
6 files changed, 281 insertions(+), 24 deletions(-)
diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c
index 272a5ec..287829d 100644
--- a/hw/virtio/vhost-backend.c
+++ b/hw/virtio/vhost-backend.c
@@ -185,6 +185,107 @@ static int vhost_kernel_vsock_set_running(struct vhost_dev *dev, int start)
}
#endif /* CONFIG_VHOST_VSOCK */
+static void vhost_kernel_iotlb_read(void *opaque)
+{
+ struct vhost_dev *dev = opaque;
+ struct vhost_msg msg;
+ ssize_t len;
+
+ while ((len = read((uintptr_t)dev->opaque, &msg, sizeof msg)) > 0) {
+ struct vhost_iotlb_msg *imsg = &msg.iotlb;
+ if (len < sizeof msg) {
+ error_report("Wrong vhost message len: %d", (int)len);
+ break;
+ }
+ if (msg.type != VHOST_IOTLB_MSG) {
+ error_report("Unknown vhost iotlb message type");
+ break;
+ }
+ switch (imsg->type) {
+ case VHOST_IOTLB_MISS:
+ vhost_device_iotlb_miss(dev, imsg->iova,
+ imsg->perm != VHOST_ACCESS_RO);
+ break;
+ case VHOST_IOTLB_UPDATE:
+ case VHOST_IOTLB_INVALIDATE:
+ error_report("Unexpected IOTLB message type");
+ break;
+ case VHOST_IOTLB_ACCESS_FAIL:
+ /* FIXME: report device iotlb error */
+ break;
+ default:
+ break;
+ }
+ }
+}
+
+static int vhost_kernel_update_device_iotlb(struct vhost_dev *dev,
+ uint64_t iova, uint64_t uaddr,
+ uint64_t len,
+ IOMMUAccessFlags perm)
+{
+ struct vhost_msg msg = {
+ .type = VHOST_IOTLB_MSG,
+ .iotlb = {
+ .iova = iova,
+ .uaddr = uaddr,
+ .size = len,
+ .type = VHOST_IOTLB_UPDATE,
+ }
+ };
+
+ switch (perm) {
+ case IOMMU_RO:
+ msg.iotlb.perm = VHOST_ACCESS_RO;
+ break;
+ case IOMMU_WO:
+ msg.iotlb.perm = VHOST_ACCESS_WO;
+ break;
+ case IOMMU_RW:
+ msg.iotlb.perm = VHOST_ACCESS_RW;
+ break;
+ default:
+ g_assert_not_reached();
+ }
+
+ if (write((uintptr_t)dev->opaque, &msg, sizeof msg) != sizeof msg) {
+ error_report("Fail to update device iotlb");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
+static int vhost_kernel_invalidate_device_iotlb(struct vhost_dev *dev,
+ uint64_t iova, uint64_t len)
+{
+ struct vhost_msg msg = {
+ .type = VHOST_IOTLB_MSG,
+ .iotlb = {
+ .iova = iova,
+ .size = len,
+ .type = VHOST_IOTLB_INVALIDATE,
+ }
+ };
+
+ if (write((uintptr_t)dev->opaque, &msg, sizeof msg) != sizeof msg) {
+ error_report("Fail to invalidate device iotlb");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
+static void vhost_kernel_set_iotlb_callback(struct vhost_dev *dev,
+ int enabled)
+{
+ if (enabled)
+ qemu_set_fd_handler((uintptr_t)dev->opaque,
+ vhost_kernel_iotlb_read, NULL, dev);
+ else
+ qemu_set_fd_handler((uintptr_t)dev->opaque, NULL, NULL, NULL);
+}
+
static const VhostOps kernel_ops = {
.backend_type = VHOST_BACKEND_TYPE_KERNEL,
.vhost_backend_init = vhost_kernel_init,
@@ -214,6 +315,9 @@ static const VhostOps kernel_ops = {
.vhost_vsock_set_guest_cid = vhost_kernel_vsock_set_guest_cid,
.vhost_vsock_set_running = vhost_kernel_vsock_set_running,
#endif /* CONFIG_VHOST_VSOCK */
+ .vhost_set_iotlb_callback = vhost_kernel_set_iotlb_callback,
+ .vhost_update_device_iotlb = vhost_kernel_update_device_iotlb,
+ .vhost_invalidate_device_iotlb = vhost_kernel_invalidate_device_iotlb,
};
int vhost_set_backend_type(struct vhost_dev *dev, VhostBackendType backend_type)
diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index bd051ab..1fe2df5 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -421,6 +421,33 @@ static inline void vhost_dev_log_resize(struct vhost_dev *dev, uint64_t size)
dev->log_size = size;
}
+static void *vhost_memory_map(VirtIODevice *vdev, hwaddr addr,
+ hwaddr *plen, int is_write)
+{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
+ if (!memory_region_is_iommu(dma_as->root)) {
+ return dma_memory_map(dma_as, addr, plen, is_write ?
+ DMA_DIRECTION_FROM_DEVICE :
+ DMA_DIRECTION_TO_DEVICE);
+ } else {
+ return (void *)(uintptr_t)addr;
+ }
+}
+
+static void vhost_memory_unmap(VirtIODevice *vdev, void *buffer,
+ hwaddr len, int is_write,
+ hwaddr access_len)
+{
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
+ if (!memory_region_is_iommu(dma_as->root)) {
+ dma_memory_unmap(dma_as, buffer, len, is_write ?
+ DMA_DIRECTION_FROM_DEVICE : DMA_DIRECTION_TO_DEVICE,
+ access_len);
+ }
+}
+
static int vhost_verify_ring_mappings(struct vhost_dev *dev,
uint64_t start_addr,
uint64_t size)
@@ -437,7 +464,7 @@ static int vhost_verify_ring_mappings(struct vhost_dev *dev,
continue;
}
l = vq->ring_size;
- p = cpu_physical_memory_map(vq->ring_phys, &l, 1);
+ p = vhost_memory_map(dev->vdev, vq->ring_phys, &l, 1);
if (!p || l != vq->ring_size) {
error_report("Unable to map ring buffer for ring %d", i);
r = -ENOMEM;
@@ -446,7 +473,7 @@ static int vhost_verify_ring_mappings(struct vhost_dev *dev,
error_report("Ring buffer relocated for ring %d", i);
r = -EBUSY;
}
- cpu_physical_memory_unmap(p, l, 0, 0);
+ vhost_memory_unmap(dev->vdev, p, l, 0, 0);
}
return r;
}
@@ -674,13 +701,27 @@ static int vhost_virtqueue_set_addr(struct vhost_dev *dev,
return 0;
}
-static int vhost_dev_set_features(struct vhost_dev *dev, bool enable_log)
+static int vhost_dev_has_iommu(struct vhost_dev *dev)
+{
+ VirtIODevice *vdev = dev->vdev;
+ AddressSpace *dma_as = virtio_get_dma_as(vdev);
+
+ return memory_region_is_iommu(dma_as->root) &&
+ virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM);
+}
+
+static int vhost_dev_set_features(struct vhost_dev *dev,
+ bool enable_log)
{
uint64_t features = dev->acked_features;
+ bool has_iommu = vhost_dev_has_iommu(dev);
int r;
if (enable_log) {
features |= 0x1ULL << VHOST_F_LOG_ALL;
}
+ if (has_iommu) {
+ features |= 0x1ULL << VIRTIO_F_IOMMU_PLATFORM;
+ }
r = dev->vhost_ops->vhost_set_features(dev, features);
if (r < 0) {
VHOST_OPS_DEBUG("vhost_set_features failed");
@@ -817,6 +858,56 @@ static int vhost_virtqueue_set_vring_endian_legacy(struct vhost_dev *dev,
return -errno;
}
+static int vhost_memory_region_lookup(struct vhost_dev *hdev,
+ uint64_t gpa, uint64_t *uaddr,
+ uint64_t *len)
+{
+ int i;
+
+ for (i = 0; i < hdev->mem->nregions; i++) {
+ struct vhost_memory_region *reg = hdev->mem->regions + i;
+
+ if (gpa >= reg->guest_phys_addr &&
+ reg->guest_phys_addr + reg->memory_size > gpa) {
+ *uaddr = reg->userspace_addr + gpa - reg->guest_phys_addr;
+ *len = reg->guest_phys_addr + reg->memory_size - gpa;
+ return 0;
+ }
+ }
+
+ return -EFAULT;
+}
+
+void vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write)
+{
+ IOMMUTLBEntry iotlb;
+ uint64_t uaddr, len;
+
+ rcu_read_lock();
+
+ iotlb = address_space_get_iotlb_entry(virtio_get_dma_as(dev->vdev),
+ iova, write);
+ if (iotlb.target_as != NULL) {
+ if (vhost_memory_region_lookup(dev, iotlb.translated_addr,
+ &uaddr, &len)) {
+ error_report("Fail to lookup the translated address "
+ "%"PRIx64, iotlb.translated_addr);
+ goto out;
+ }
+
+ len = MIN(iotlb.addr_mask + 1, len);
+ iova = iova & ~iotlb.addr_mask;
+
+ if (dev->vhost_ops->vhost_update_device_iotlb(dev, iova, uaddr,
+ len, iotlb.perm)) {
+ error_report("Fail to update device iotlb");
+ goto out;
+ }
+ }
+out:
+ rcu_read_unlock();
+}
+
static int vhost_virtqueue_start(struct vhost_dev *dev,
struct VirtIODevice *vdev,
struct vhost_virtqueue *vq,
@@ -862,21 +953,21 @@ static int vhost_virtqueue_start(struct vhost_dev *dev,
s = l = virtio_queue_get_desc_size(vdev, idx);
a = virtio_queue_get_desc_addr(vdev, idx);
- vq->desc = cpu_physical_memory_map(a, &l, 0);
+ vq->desc = vhost_memory_map(vdev, a, &l, 0);
if (!vq->desc || l != s) {
r = -ENOMEM;
goto fail_alloc_desc;
}
s = l = virtio_queue_get_avail_size(vdev, idx);
a = virtio_queue_get_avail_addr(vdev, idx);
- vq->avail = cpu_physical_memory_map(a, &l, 0);
+ vq->avail = vhost_memory_map(vdev, a, &l, 0);
if (!vq->avail || l != s) {
r = -ENOMEM;
goto fail_alloc_avail;
}
vq->used_size = s = l = virtio_queue_get_used_size(vdev, idx);
vq->used_phys = a = virtio_queue_get_used_addr(vdev, idx);
- vq->used = cpu_physical_memory_map(a, &l, 1);
+ vq->used = vhost_memory_map(vdev, a, &l, 1);
if (!vq->used || l != s) {
r = -ENOMEM;
goto fail_alloc_used;
@@ -884,7 +975,7 @@ static int vhost_virtqueue_start(struct vhost_dev *dev,
vq->ring_size = s = l = virtio_queue_get_ring_size(vdev, idx);
vq->ring_phys = a = virtio_queue_get_ring_addr(vdev, idx);
- vq->ring = cpu_physical_memory_map(a, &l, 1);
+ vq->ring = vhost_memory_map(vdev, a, &l, 1);
if (!vq->ring || l != s) {
r = -ENOMEM;
goto fail_alloc_ring;
@@ -930,17 +1021,17 @@ static int vhost_virtqueue_start(struct vhost_dev *dev,
fail_vector:
fail_kick:
fail_alloc:
- cpu_physical_memory_unmap(vq->ring, virtio_queue_get_ring_size(vdev, idx),
- 0, 0);
+ vhost_memory_unmap(vdev, vq->ring, virtio_queue_get_ring_size(vdev, idx),
+ 0, 0);
fail_alloc_ring:
- cpu_physical_memory_unmap(vq->used, virtio_queue_get_used_size(vdev, idx),
- 0, 0);
+ vhost_memory_unmap(vdev, vq->used, virtio_queue_get_used_size(vdev, idx),
+ 0, 0);
fail_alloc_used:
- cpu_physical_memory_unmap(vq->avail, virtio_queue_get_avail_size(vdev, idx),
- 0, 0);
+ vhost_memory_unmap(vdev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
+ 0, 0);
fail_alloc_avail:
- cpu_physical_memory_unmap(vq->desc, virtio_queue_get_desc_size(vdev, idx),
- 0, 0);
+ vhost_memory_unmap(vdev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
+ 0, 0);
fail_alloc_desc:
return r;
}
@@ -973,14 +1064,14 @@ static void vhost_virtqueue_stop(struct vhost_dev *dev,
vhost_vq_index);
}
- cpu_physical_memory_unmap(vq->ring, virtio_queue_get_ring_size(vdev, idx),
- 0, virtio_queue_get_ring_size(vdev, idx));
- cpu_physical_memory_unmap(vq->used, virtio_queue_get_used_size(vdev, idx),
- 1, virtio_queue_get_used_size(vdev, idx));
- cpu_physical_memory_unmap(vq->avail, virtio_queue_get_avail_size(vdev, idx),
- 0, virtio_queue_get_avail_size(vdev, idx));
- cpu_physical_memory_unmap(vq->desc, virtio_queue_get_desc_size(vdev, idx),
- 0, virtio_queue_get_desc_size(vdev, idx));
+ vhost_memory_unmap(vdev, vq->ring, virtio_queue_get_ring_size(vdev, idx),
+ 0, virtio_queue_get_ring_size(vdev, idx));
+ vhost_memory_unmap(vdev, vq->used, virtio_queue_get_used_size(vdev, idx),
+ 1, virtio_queue_get_used_size(vdev, idx));
+ vhost_memory_unmap(vdev, vq->avail, virtio_queue_get_avail_size(vdev, idx),
+ 0, virtio_queue_get_avail_size(vdev, idx));
+ vhost_memory_unmap(vdev, vq->desc, virtio_queue_get_desc_size(vdev, idx),
+ 0, virtio_queue_get_desc_size(vdev, idx));
}
static void vhost_eventfd_add(MemoryListener *listener,
@@ -1037,6 +1128,9 @@ static int vhost_virtqueue_init(struct vhost_dev *dev,
r = -errno;
goto fail_call;
}
+
+ vq->dev = dev;
+
return 0;
fail_call:
event_notifier_cleanup(&vq->masked_notifier);
@@ -1048,12 +1142,24 @@ static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq)
event_notifier_cleanup(&vq->masked_notifier);
}
+static void vhost_iommu_unmap_notify(IOMMUNotifier *n, IOMMUTLBEntry *iotlb)
+{
+ struct vhost_dev *hdev = container_of(n, struct vhost_dev, n);
+
+ if (hdev->vhost_ops->vhost_invalidate_device_iotlb(hdev,
+ iotlb->iova,
+ iotlb->addr_mask + 1)) {
+ error_report("Fail to invalidate device iotlb");
+ }
+}
+
int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
VhostBackendType backend_type, uint32_t busyloop_timeout)
{
uint64_t features;
int i, r, n_initialized_vqs = 0;
+ hdev->vdev = NULL;
hdev->migration_blocker = NULL;
r = vhost_set_backend_type(hdev, backend_type);
@@ -1118,6 +1224,9 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
.priority = 10
};
+ hdev->n.notify = vhost_iommu_unmap_notify;
+ hdev->n.notifier_flags = IOMMU_NOTIFIER_UNMAP;
+
if (hdev->migration_blocker == NULL) {
if (!(hdev->features & (0x1ULL << VHOST_F_LOG_ALL))) {
error_setg(&hdev->migration_blocker,
@@ -1310,11 +1419,18 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
assert(hdev->vhost_ops);
hdev->started = true;
+ hdev->vdev = vdev;
r = vhost_dev_set_features(hdev, hdev->log_enabled);
if (r < 0) {
goto fail_features;
}
+
+ if (vhost_dev_has_iommu(hdev)) {
+ memory_region_register_iommu_notifier(virtio_get_dma_as(vdev)->root,
+ &hdev->n);
+ }
+
r = hdev->vhost_ops->vhost_set_mem_table(hdev, hdev->mem);
if (r < 0) {
VHOST_OPS_DEBUG("vhost_set_mem_table failed");
@@ -1348,6 +1464,16 @@ int vhost_dev_start(struct vhost_dev *hdev, VirtIODevice *vdev)
}
}
+ hdev->vhost_ops->vhost_set_iotlb_callback(hdev, true);
+
+ if (vhost_dev_has_iommu(hdev)) {
+ /* Update used ring information for IOTLB to work correctly,
+ * vhost-kernel code requires for this.*/
+ for (i = 0; i < hdev->nvqs; ++i) {
+ struct vhost_virtqueue *vq = hdev->vqs + i;
+ vhost_device_iotlb_miss(hdev, vq->used_phys, true);
+ }
+ }
return 0;
fail_log:
vhost_log_put(hdev, false);
@@ -1359,6 +1485,7 @@ fail_vq:
hdev->vq_index + i);
}
i = hdev->nvqs;
+
fail_mem:
fail_features:
@@ -1373,6 +1500,7 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
/* should only be called after backend is connected */
assert(hdev->vhost_ops);
+ hdev->vhost_ops->vhost_set_iotlb_callback(hdev, false);
for (i = 0; i < hdev->nvqs; ++i) {
vhost_virtqueue_stop(hdev,
@@ -1381,8 +1509,13 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev)
hdev->vq_index + i);
}
+ if (vhost_dev_has_iommu(hdev)) {
+ memory_region_unregister_iommu_notifier(virtio_get_dma_as(vdev)->root,
+ &hdev->n);
+ }
vhost_log_put(hdev, true);
hdev->started = false;
+ hdev->vdev = NULL;
}
int vhost_net_set_backend(struct vhost_dev *hdev,
diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-backend.h
index 6e90703..236eb85 100644
--- a/include/hw/virtio/vhost-backend.h
+++ b/include/hw/virtio/vhost-backend.h
@@ -11,6 +11,8 @@
#ifndef VHOST_BACKEND_H
#define VHOST_BACKEND_H
+#include "exec/memory.h"
+
typedef enum VhostBackendType {
VHOST_BACKEND_TYPE_NONE = 0,
VHOST_BACKEND_TYPE_KERNEL = 1,
@@ -76,6 +78,14 @@ typedef bool (*vhost_backend_can_merge_op)(struct vhost_dev *dev,
typedef int (*vhost_vsock_set_guest_cid_op)(struct vhost_dev *dev,
uint64_t guest_cid);
typedef int (*vhost_vsock_set_running_op)(struct vhost_dev *dev, int start);
+typedef void (*vhost_set_iotlb_callback_op)(struct vhost_dev *dev,
+ int enabled);
+typedef int (*vhost_update_device_iotlb_op)(struct vhost_dev *dev,
+ uint64_t iova, uint64_t uaddr,
+ uint64_t len,
+ IOMMUAccessFlags perm);
+typedef int (*vhost_invalidate_device_iotlb_op)(struct vhost_dev *dev,
+ uint64_t iova, uint64_t len);
typedef struct VhostOps {
VhostBackendType backend_type;
@@ -107,6 +117,9 @@ typedef struct VhostOps {
vhost_backend_can_merge_op vhost_backend_can_merge;
vhost_vsock_set_guest_cid_op vhost_vsock_set_guest_cid;
vhost_vsock_set_running_op vhost_vsock_set_running;
+ vhost_set_iotlb_callback_op vhost_set_iotlb_callback;
+ vhost_update_device_iotlb_op vhost_update_device_iotlb;
+ vhost_invalidate_device_iotlb_op vhost_invalidate_device_iotlb;
} VhostOps;
extern const VhostOps user_ops;
diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h
index e433089..4e52a13 100644
--- a/include/hw/virtio/vhost.h
+++ b/include/hw/virtio/vhost.h
@@ -20,6 +20,7 @@ struct vhost_virtqueue {
unsigned long long ring_phys;
unsigned ring_size;
EventNotifier masked_notifier;
+ struct vhost_dev *dev;
};
typedef unsigned long vhost_log_chunk_t;
@@ -37,6 +38,7 @@ struct vhost_log {
struct vhost_memory;
struct vhost_dev {
+ VirtIODevice *vdev;
MemoryListener memory_listener;
struct vhost_memory *mem;
int n_mem_sections;
@@ -61,6 +63,7 @@ struct vhost_dev {
void *opaque;
struct vhost_log *log;
QLIST_ENTRY(vhost_dev) entry;
+ IOMMUNotifier n;
};
int vhost_dev_init(struct vhost_dev *hdev, void *opaque,
@@ -90,4 +93,5 @@ bool vhost_has_free_slot(void);
int vhost_net_set_backend(struct vhost_dev *hdev,
struct vhost_vring_file *file);
+void vhost_device_iotlb_miss(struct vhost_dev *dev, uint64_t iova, int write);
#endif
diff --git a/include/hw/virtio/virtio-access.h b/include/hw/virtio/virtio-access.h
index 4643a06..c122fd8 100644
--- a/include/hw/virtio/virtio-access.h
+++ b/include/hw/virtio/virtio-access.h
@@ -18,6 +18,7 @@
#include "hw/virtio/virtio.h"
#include "hw/virtio/virtio-bus.h"
+#include "sysemu/dma.h"
#include "exec/address-spaces.h"
#if defined(TARGET_PPC64) || defined(TARGET_ARM)
@@ -193,4 +194,5 @@ static inline void virtio_tswap64s(VirtIODevice *vdev, uint64_t *s)
{
*s = virtio_tswap64(vdev, *s);
}
-#endif /* QEMU_VIRTIO_ACCESS_H */
+
+#endif /* _QEMU_VIRTIO_ACCESS_H */
diff --git a/net/tap.c b/net/tap.c
index b6896a7..86071b2 100644
--- a/net/tap.c
+++ b/net/tap.c
@@ -696,6 +696,7 @@ static void net_init_tap_one(const NetdevTapOptions *tap, NetClientState *peer,
"tap: open vhost char device failed");
return;
}
+ fcntl(vhostfd, F_SETFL, O_NONBLOCK);
}
options.opaque = (void *)(uintptr_t)vhostfd;
--
2.7.4
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
` (10 preceding siblings ...)
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 11/11] vhost_net: device IOTLB support Jason Wang
@ 2016-11-03 9:48 ` no-reply
11 siblings, 0 replies; 25+ messages in thread
From: no-reply @ 2016-11-03 9:48 UTC (permalink / raw)
To: jasowang
Cc: famz, mst, peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
Hi,
Your series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.
Type: series
Subject: [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support
Message-id: 1478165243-4767-1-git-send-email-jasowang@redhat.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=16
make docker-test-quick@centos6
make docker-test-mingw@fedora
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
5c2833a vhost_net: device IOTLB support
fa3d27f memory: handle alias in memory_region_is_iommu()
41ae086 memory: handle alias for iommu notifier
58bfd01 acpi: add ATSR for q35
2089d60 virtio-pci: address space translation service (ATS) support
5f5e607 intel_iommu: support device iotlb descriptor
650123f exec: introduce address_space_get_iotlb_entry()
0fbaf62 intel_iommu: allocate new key when creating new address space
e7c13cc intel_iommu: name vtd address space with devfn
964e087 virtio: convert to use DMA api
eb53883 intel_iommu: fixing source id during IOTLB hash key calculation
=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into 'dtc'...
Submodule path 'dtc': checked out '65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf'
BUILD centos6
make[1]: Entering directory `/var/tmp/patchew-tester-tmp-591ndo6_/src'
ARCHIVE qemu.tgz
ARCHIVE dtc.tgz
COPY RUNNER
RUN test-quick in qemu:centos6
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
gcc-4.4.7-17.el6.x86_64
git-1.7.1-4.el6_7.1.x86_64
glib2-devel-2.28.8-5.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64
Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ zlib-devel glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=65efa6d04159
TERM=xterm
MAKEFLAGS= -j16
HISTSIZE=1000
J=16
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env
Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu --prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix /var/tmp/qemu-build/install
BIOS directory /var/tmp/qemu-build/install/share/qemu
binary directory /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory /var/tmp/qemu-build/install/etc
local state directory /var/tmp/qemu-build/install/var
Manual directory /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path /tmp/qemu-test/src
C compiler cc
Host C compiler cc
C++ compiler
Objective-C compiler cc
ARFLAGS rv
CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g
QEMU_CFLAGS -I/usr/include/pixman-1 -pthread -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g
make make
install install
python python -B
smbd /usr/sbin/smbd
module support no
host CPU x86_64
host big endian no
target list x86_64-softmmu aarch64-softmmu
tcg debug enabled no
gprof enabled no
sparse enabled no
strip binaries yes
profiler no
static build no
pixman system
SDL support yes (1.2.14)
GTK support no
GTK GL support no
VTE support no
TLS priority NORMAL
GNUTLS support no
GNUTLS rnd no
libgcrypt no
libgcrypt kdf no
nettle no
nettle kdf no
libtasn1 no
curses support no
virgl support no
curl support no
mingw32 support no
Audio drivers oss
Block whitelist (rw)
Block whitelist (ro)
VirtFS support no
VNC support yes
VNC SASL support no
VNC JPEG support no
VNC PNG support no
xen support no
brlapi support no
bluez support no
Documentation no
PIE yes
vde support no
netmap support no
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support yes
COLO support yes
RDMA support no
TCG interpreter no
fdt support yes
preadv support yes
fdatasync yes
madvise yes
posix_madvise yes
libcap-ng support no
vhost-net support yes
vhost-scsi support yes
vhost-vsock support yes
Trace backends log
spice support no
rbd support no
xfsctl support no
smartcard support no
libusb no
usb net redir no
OpenGL support no
OpenGL dmabufs no
libiscsi support no
libnfs support no
build guest agent yes
QGA VSS support no
QGA w32 disk info no
QGA MSI support no
seccomp support no
coroutine backend ucontext
coroutine pool yes
debug stack usage no
GlusterFS support no
Archipelago support no
gcov gcov
gcov enabled no
TPM support yes
libssh2 support no
TPM passthrough yes
QOM debugging yes
lzo support no
snappy support no
bzip2 support no
NUMA host support no
tcmalloc support no
jemalloc support no
avx2 optimization no
replication support yes
GEN x86_64-softmmu/config-devices.mak.tmp
GEN aarch64-softmmu/config-devices.mak.tmp
GEN config-host.h
GEN qemu-options.def
GEN qapi-types.h
GEN qmp-commands.h
GEN qapi-visit.h
GEN qapi-event.h
GEN qmp-introspect.h
GEN x86_64-softmmu/config-devices.mak
GEN aarch64-softmmu/config-devices.mak
GEN module_block.h
GEN tests/test-qapi-types.h
GEN tests/test-qapi-visit.h
GEN tests/test-qmp-commands.h
GEN tests/test-qapi-event.h
GEN tests/test-qmp-introspect.h
GEN config-all-devices.mak
GEN trace/generated-tracers.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
CC tests/qemu-iotests/socket_scm_helper.o
GEN qga/qapi-generated/qga-qapi-types.h
GEN qga/qapi-generated/qga-qapi-visit.h
GEN qga/qapi-generated/qga-qmp-commands.h
GEN qga/qapi-generated/qga-qapi-types.c
GEN qga/qapi-generated/qga-qapi-visit.c
GEN qga/qapi-generated/qga-qmp-marshal.c
GEN qmp-introspect.c
GEN qapi-types.c
GEN qapi-visit.c
GEN qapi-event.c
CC qapi/qapi-visit-core.o
CC qapi/qapi-dealloc-visitor.o
CC qapi/qobject-input-visitor.o
CC qapi/qobject-output-visitor.o
CC qapi/qmp-registry.o
CC qapi/qmp-dispatch.o
CC qapi/string-input-visitor.o
CC qapi/string-output-visitor.o
CC qapi/opts-visitor.o
CC qapi/qapi-clone-visitor.o
CC qapi/qmp-event.o
CC qapi/qapi-util.o
CC qobject/qnull.o
CC qobject/qint.o
CC qobject/qstring.o
CC qobject/qdict.o
CC qobject/qlist.o
CC qobject/qfloat.o
CC qobject/qbool.o
CC qobject/qjson.o
CC qobject/qobject.o
CC qobject/json-lexer.o
CC qobject/json-streamer.o
CC qobject/json-parser.o
GEN trace/generated-tracers.c
CC trace/control.o
CC trace/qmp.o
CC util/osdep.o
CC util/cutils.o
CC util/unicode.o
CC util/qemu-timer-common.o
CC util/bufferiszero.o
CC util/compatfd.o
CC util/event_notifier-posix.o
CC util/mmap-alloc.o
CC util/oslib-posix.o
CC util/qemu-openpty.o
CC util/qemu-thread-posix.o
CC util/memfd.o
CC util/envlist.o
CC util/path.o
CC util/module.o
CC util/bitmap.o
CC util/bitops.o
CC util/hbitmap.o
CC util/fifo8.o
CC util/acl.o
CC util/error.o
CC util/qemu-error.o
CC util/id.o
CC util/iov.o
CC util/qemu-config.o
CC util/qemu-sockets.o
CC util/uri.o
CC util/notify.o
CC util/qemu-option.o
CC util/qemu-progress.o
CC util/hexdump.o
CC util/crc32c.o
CC util/uuid.o
CC util/throttle.o
CC util/getauxval.o
CC util/readline.o
CC util/rcu.o
CC util/qemu-coroutine.o
CC util/qemu-coroutine-lock.o
CC util/qemu-coroutine-io.o
CC util/qemu-coroutine-sleep.o
CC util/coroutine-ucontext.o
CC util/buffer.o
CC util/timed-average.o
CC util/base64.o
CC util/log.o
CC util/qdist.o
CC util/qht.o
CC util/range.o
CC crypto/pbkdf-stub.o
CC stubs/arch-query-cpu-def.o
CC stubs/arch-query-cpu-model-expansion.o
CC stubs/arch-query-cpu-model-comparison.o
CC stubs/arch-query-cpu-model-baseline.o
CC stubs/bdrv-next-monitor-owned.o
CC stubs/blk-commit-all.o
CC stubs/cpu-get-clock.o
CC stubs/blockdev-close-all-bdrv-states.o
CC stubs/clock-warp.o
CC stubs/cpu-get-icount.o
CC stubs/dump.o
CC stubs/fdset-add-fd.o
CC stubs/fdset-find-fd.o
CC stubs/fdset-get-fd.o
CC stubs/fdset-remove-fd.o
CC stubs/gdbstub.o
CC stubs/get-fd.o
CC stubs/get-next-serial.o
CC stubs/get-vm-name.o
CC stubs/iothread.o
CC stubs/iothread-lock.o
CC stubs/is-daemonized.o
CC stubs/machine-init-done.o
CC stubs/migr-blocker.o
CC stubs/mon-is-qmp.o
CC stubs/mon-printf.o
CC stubs/monitor-init.o
CC stubs/notify-event.o
CC stubs/qtest.o
CC stubs/replay.o
CC stubs/replay-user.o
CC stubs/reset.o
CC stubs/runstate-check.o
CC stubs/set-fd-handler.o
CC stubs/slirp.o
CC stubs/sysbus.o
CC stubs/trace-control.o
CC stubs/uuid.o
CC stubs/vm-stop.o
CC stubs/vmstate.o
CC stubs/kvm.o
CC stubs/cpus.o
CC stubs/qmp_pc_dimm_device_list.o
CC stubs/target-monitor-defs.o
CC stubs/target-get-monitor-def.o
CC stubs/vhost.o
CC stubs/iohandler.o
CC stubs/smbios_type_38.o
CC stubs/ipmi.o
CC stubs/pc_madt_cpu_entry.o
CC stubs/migration-colo.o
CC contrib/ivshmem-client/ivshmem-client.o
CC contrib/ivshmem-client/main.o
CC contrib/ivshmem-server/ivshmem-server.o
CC contrib/ivshmem-server/main.o
CC qemu-nbd.o
CC async.o
CC thread-pool.o
CC block.o
CC blockjob.o
CC main-loop.o
CC iohandler.o
CC qemu-timer.o
CC aio-posix.o
CC qemu-io-cmds.o
CC replication.o
CC block/raw_bsd.o
CC block/qcow.o
CC block/vdi.o
CC block/vmdk.o
CC block/cloop.o
CC block/bochs.o
CC block/vpc.o
CC block/vvfat.o
CC block/dmg.o
CC block/qcow2.o
CC block/qcow2-refcount.o
CC block/qcow2-cluster.o
CC block/qcow2-cache.o
CC block/qcow2-snapshot.o
CC block/qed.o
CC block/qed-gencb.o
CC block/qed-l2-cache.o
CC block/qed-table.o
CC block/qed-cluster.o
CC block/qed-check.o
CC block/vhdx.o
CC block/vhdx-endian.o
CC block/vhdx-log.o
CC block/quorum.o
CC block/parallels.o
CC block/blkdebug.o
CC block/blkverify.o
CC block/blkreplay.o
CC block/snapshot.o
CC block/block-backend.o
CC block/qapi.o
CC block/raw-posix.o
CC block/null.o
CC block/mirror.o
CC block/commit.o
CC block/io.o
CC block/throttle-groups.o
CC block/nbd.o
CC block/nbd-client.o
CC block/sheepdog.o
CC block/accounting.o
CC block/dirty-bitmap.o
CC block/write-threshold.o
CC block/backup.o
CC block/replication.o
CC block/crypto.o
CC nbd/server.o
CC nbd/client.o
CC crypto/init.o
CC nbd/common.o
CC crypto/hash.o
CC crypto/hash-glib.o
CC crypto/aes.o
CC crypto/desrfb.o
CC crypto/cipher.o
CC crypto/tlscreds.o
CC crypto/tlscredsx509.o
CC crypto/tlscredsanon.o
CC crypto/secret.o
CC crypto/tlssession.o
CC crypto/random-platform.o
CC crypto/pbkdf.o
CC crypto/ivgen.o
CC crypto/ivgen-essiv.o
CC crypto/ivgen-plain.o
CC crypto/ivgen-plain64.o
CC crypto/afsplit.o
CC crypto/xts.o
CC crypto/block.o
CC crypto/block-qcow.o
CC crypto/block-luks.o
CC io/channel.o
CC io/channel-buffer.o
CC io/channel-command.o
CC io/channel-file.o
CC io/channel-socket.o
CC io/channel-tls.o
CC io/channel-watch.o
CC io/channel-websock.o
CC io/task.o
CC io/channel-util.o
CC qom/container.o
CC qom/object.o
GEN qemu-img-cmds.h
CC qom/qom-qobject.o
CC qom/object_interfaces.o
CC qemu-io.o
CC qemu-bridge-helper.o
CC blockdev-nbd.o
CC blockdev.o
CC iothread.o
CC qdev-monitor.o
CC device-hotplug.o
CC os-posix.o
CC page_cache.o
CC qemu-char.o
CC accel.o
CC bt-host.o
CC bt-vhci.o
CC dma-helpers.o
CC vl.o
CC tpm.o
CC device_tree.o
GEN qmp-marshal.c
CC qmp.o
CC hmp.o
CC cpus-common.o
CC audio/audio.o
CC audio/noaudio.o
CC audio/wavaudio.o
CC audio/mixeng.o
CC audio/sdlaudio.o
CC audio/ossaudio.o
CC audio/wavcapture.o
CC backends/rng.o
CC backends/rng-egd.o
CC backends/rng-random.o
CC backends/msmouse.o
CC backends/testdev.o
CC backends/tpm.o
CC backends/hostmem.o
CC backends/hostmem-ram.o
CC backends/hostmem-file.o
CC block/stream.o
CC disas/arm.o
CC disas/i386.o
CC fsdev/qemu-fsdev-dummy.o
CC fsdev/qemu-fsdev-opts.o
CC hw/acpi/core.o
CC hw/acpi/piix4.o
CC hw/acpi/pcihp.o
CC hw/acpi/ich9.o
CC hw/acpi/tco.o
CC hw/acpi/cpu_hotplug.o
CC hw/acpi/memory_hotplug.o
CC hw/acpi/memory_hotplug_acpi_table.o
CC hw/acpi/cpu.o
CC hw/acpi/acpi_interface.o
CC hw/acpi/bios-linker-loader.o
CC hw/acpi/aml-build.o
CC hw/acpi/ipmi.o
CC hw/audio/sb16.o
CC hw/audio/es1370.o
CC hw/audio/ac97.o
CC hw/audio/fmopl.o
CC hw/audio/adlib.o
CC hw/audio/gus.o
CC hw/audio/gusemu_hal.o
CC hw/audio/gusemu_mixer.o
CC hw/audio/cs4231a.o
CC hw/audio/intel-hda.o
CC hw/audio/hda-codec.o
CC hw/audio/pcspk.o
CC hw/audio/wm8750.o
CC hw/audio/pl041.o
CC hw/audio/lm4549.o
CC hw/audio/marvell_88w8618.o
CC hw/block/block.o
CC hw/block/cdrom.o
CC hw/block/hd-geometry.o
CC hw/block/fdc.o
CC hw/block/m25p80.o
CC hw/block/pflash_cfi02.o
CC hw/block/nand.o
CC hw/block/pflash_cfi01.o
CC hw/block/ecc.o
CC hw/block/onenand.o
CC hw/block/nvme.o
CC hw/bt/core.o
CC hw/bt/l2cap.o
CC hw/bt/sdp.o
CC hw/bt/hci.o
CC hw/bt/hid.o
CC hw/bt/hci-csr.o
CC hw/char/ipoctal232.o
CC hw/char/parallel.o
CC hw/char/pl011.o
CC hw/char/serial.o
CC hw/char/serial-isa.o
CC hw/char/serial-pci.o
CC hw/char/virtio-console.o
CC hw/char/cadence_uart.o
CC hw/char/debugcon.o
CC hw/char/imx_serial.o
CC hw/core/qdev.o
CC hw/core/qdev-properties.o
CC hw/core/bus.o
CC hw/core/fw-path-provider.o
CC hw/core/irq.o
CC hw/core/sysbus.o
CC hw/core/ptimer.o
CC hw/core/hotplug.o
CC hw/core/machine.o
CC hw/core/null-machine.o
CC hw/core/loader.o
CC hw/core/qdev-properties-system.o
CC hw/core/register.o
CC hw/core/or-irq.o
CC hw/core/platform-bus.o
CC hw/display/ads7846.o
CC hw/display/cirrus_vga.o
CC hw/display/pl110.o
CC hw/display/ssd0303.o
CC hw/display/ssd0323.o
CC hw/display/vga-pci.o
CC hw/display/vga-isa.o
CC hw/display/vmware_vga.o
CC hw/display/blizzard.o
CC hw/display/exynos4210_fimd.o
CC hw/display/framebuffer.o
CC hw/display/tc6393xb.o
CC hw/dma/pl080.o
CC hw/dma/pl330.o
CC hw/dma/i8257.o
CC hw/dma/xlnx-zynq-devcfg.o
CC hw/gpio/max7310.o
CC hw/gpio/pl061.o
CC hw/gpio/gpio_key.o
CC hw/gpio/zaurus.o
CC hw/i2c/core.o
CC hw/i2c/smbus.o
CC hw/i2c/smbus_eeprom.o
CC hw/i2c/i2c-ddc.o
CC hw/i2c/versatile_i2c.o
CC hw/i2c/smbus_ich9.o
CC hw/i2c/pm_smbus.o
CC hw/i2c/bitbang_i2c.o
CC hw/i2c/exynos4210_i2c.o
CC hw/i2c/imx_i2c.o
CC hw/i2c/aspeed_i2c.o
CC hw/ide/core.o
CC hw/ide/atapi.o
CC hw/ide/qdev.o
CC hw/ide/pci.o
CC hw/ide/isa.o
CC hw/ide/piix.o
CC hw/ide/microdrive.o
CC hw/ide/ahci.o
CC hw/ide/ich.o
CC hw/input/hid.o
CC hw/input/lm832x.o
CC hw/input/pckbd.o
CC hw/input/pl050.o
CC hw/input/ps2.o
CC hw/input/stellaris_input.o
CC hw/input/tsc2005.o
CC hw/input/vmmouse.o
CC hw/input/virtio-input.o
CC hw/input/virtio-input-hid.o
CC hw/input/virtio-input-host.o
CC hw/intc/i8259_common.o
CC hw/intc/i8259.o
CC hw/intc/pl190.o
CC hw/intc/imx_avic.o
CC hw/intc/realview_gic.o
CC hw/intc/ioapic_common.o
CC hw/intc/arm_gic_common.o
CC hw/intc/arm_gic.o
CC hw/intc/arm_gicv2m.o
CC hw/intc/arm_gicv3_common.o
CC hw/intc/arm_gicv3.o
CC hw/intc/arm_gicv3_dist.o
CC hw/intc/arm_gicv3_redist.o
CC hw/intc/arm_gicv3_its_common.o
CC hw/intc/intc.o
CC hw/ipack/ipack.o
CC hw/ipack/tpci200.o
CC hw/ipmi/ipmi.o
CC hw/ipmi/ipmi_bmc_sim.o
CC hw/ipmi/ipmi_bmc_extern.o
CC hw/ipmi/isa_ipmi_kcs.o
CC hw/ipmi/isa_ipmi_bt.o
CC hw/isa/isa-bus.o
CC hw/isa/apm.o
CC hw/mem/pc-dimm.o
CC hw/mem/nvdimm.o
CC hw/misc/applesmc.o
CC hw/misc/max111x.o
CC hw/misc/tmp105.o
CC hw/misc/debugexit.o
CC hw/misc/sga.o
CC hw/misc/pc-testdev.o
CC hw/misc/pci-testdev.o
CC hw/misc/arm_l2x0.o
CC hw/misc/arm_integrator_debug.o
CC hw/misc/a9scu.o
CC hw/misc/arm11scu.o
CC hw/net/ne2000.o
CC hw/net/eepro100.o
CC hw/net/pcnet-pci.o
CC hw/net/pcnet.o
CC hw/net/e1000.o
CC hw/net/e1000x_common.o
CC hw/net/net_tx_pkt.o
CC hw/net/net_rx_pkt.o
CC hw/net/e1000e.o
CC hw/net/e1000e_core.o
CC hw/net/rtl8139.o
CC hw/net/vmxnet3.o
CC hw/net/smc91c111.o
CC hw/net/lan9118.o
CC hw/net/ne2000-isa.o
CC hw/net/xgmac.o
CC hw/net/allwinner_emac.o
CC hw/net/imx_fec.o
CC hw/net/cadence_gem.o
CC hw/net/stellaris_enet.o
CC hw/net/rocker/rocker.o
CC hw/net/rocker/rocker_fp.o
CC hw/net/rocker/rocker_desc.o
CC hw/net/rocker/rocker_world.o
CC hw/net/rocker/rocker_of_dpa.o
CC hw/nvram/eeprom93xx.o
CC hw/nvram/fw_cfg.o
CC hw/nvram/chrp_nvram.o
CC hw/pci-bridge/pci_bridge_dev.o
CC hw/pci-bridge/pci_expander_bridge.o
CC hw/pci-bridge/xio3130_upstream.o
CC hw/pci-bridge/ioh3420.o
CC hw/pci-bridge/xio3130_downstream.o
CC hw/pci-bridge/i82801b11.o
CC hw/pci-host/pam.o
CC hw/pci-host/versatile.o
CC hw/pci-host/piix.o
CC hw/pci-host/q35.o
CC hw/pci-host/gpex.o
CC hw/pci/pci.o
CC hw/pci/pci_bridge.o
CC hw/pci/msix.o
CC hw/pci/msi.o
CC hw/pci/shpc.o
CC hw/pci/slotid_cap.o
CC hw/pci/pci_host.o
CC hw/pci/pcie_host.o
CC hw/pci/pcie.o
CC hw/pci/pcie_aer.o
CC hw/pci/pcie_port.o
CC hw/pci/pci-stub.o
CC hw/pcmcia/pcmcia.o
CC hw/scsi/scsi-disk.o
CC hw/scsi/scsi-generic.o
CC hw/scsi/scsi-bus.o
CC hw/scsi/lsi53c895a.o
/tmp/qemu-test/src/hw/nvram/fw_cfg.c: In function ‘fw_cfg_dma_transfer’:
/tmp/qemu-test/src/hw/nvram/fw_cfg.c:329: warning: ‘read’ may be used uninitialized in this function
CC hw/scsi/mptsas.o
CC hw/scsi/mptconfig.o
CC hw/scsi/mptendian.o
CC hw/scsi/megasas.o
CC hw/scsi/vmw_pvscsi.o
CC hw/scsi/esp.o
CC hw/scsi/esp-pci.o
CC hw/sd/pl181.o
CC hw/sd/ssi-sd.o
CC hw/sd/sd.o
CC hw/sd/core.o
CC hw/sd/sdhci.o
CC hw/smbios/smbios.o
CC hw/smbios/smbios_type_38.o
CC hw/ssi/pl022.o
CC hw/ssi/ssi.o
CC hw/ssi/xilinx_spips.o
CC hw/ssi/aspeed_smc.o
CC hw/ssi/stm32f2xx_spi.o
CC hw/timer/arm_timer.o
CC hw/timer/arm_mptimer.o
CC hw/timer/a9gtimer.o
CC hw/timer/cadence_ttc.o
CC hw/timer/ds1338.o
CC hw/timer/hpet.o
CC hw/timer/i8254_common.o
CC hw/timer/i8254.o
CC hw/timer/pl031.o
CC hw/timer/twl92230.o
CC hw/timer/imx_epit.o
CC hw/timer/imx_gpt.o
CC hw/timer/stm32f2xx_timer.o
CC hw/timer/aspeed_timer.o
CC hw/tpm/tpm_tis.o
CC hw/tpm/tpm_passthrough.o
CC hw/tpm/tpm_util.o
CC hw/usb/core.o
CC hw/usb/combined-packet.o
CC hw/usb/bus.o
CC hw/usb/libhw.o
CC hw/usb/desc.o
CC hw/usb/desc-msos.o
CC hw/usb/hcd-uhci.o
CC hw/usb/hcd-ohci.o
CC hw/usb/hcd-ehci.o
CC hw/usb/hcd-ehci-pci.o
CC hw/usb/hcd-ehci-sysbus.o
CC hw/usb/hcd-xhci.o
CC hw/usb/hcd-musb.o
CC hw/usb/dev-hid.o
CC hw/usb/dev-hub.o
CC hw/usb/dev-wacom.o
CC hw/usb/dev-storage.o
CC hw/usb/dev-uas.o
CC hw/usb/dev-audio.o
CC hw/usb/dev-serial.o
CC hw/usb/dev-network.o
CC hw/usb/dev-bluetooth.o
CC hw/usb/dev-smartcard-reader.o
CC hw/usb/dev-mtp.o
CC hw/usb/host-stub.o
CC hw/virtio/virtio-rng.o
CC hw/virtio/virtio-bus.o
CC hw/virtio/virtio-pci.o
CC hw/virtio/virtio-mmio.o
CC hw/watchdog/watchdog.o
CC hw/watchdog/wdt_i6300esb.o
CC hw/watchdog/wdt_ib700.o
CC migration/migration.o
CC migration/socket.o
CC migration/fd.o
CC migration/exec.o
CC migration/tls.o
CC migration/colo-comm.o
CC migration/colo.o
CC migration/colo-failover.o
CC migration/vmstate.o
CC migration/qemu-file.o
CC migration/qemu-file-channel.o
CC migration/xbzrle.o
CC migration/postcopy-ram.o
CC migration/qjson.o
CC migration/block.o
CC net/net.o
CC net/checksum.o
CC net/queue.o
CC net/util.o
CC net/hub.o
CC net/socket.o
CC net/dump.o
CC net/eth.o
CC net/l2tpv3.o
CC net/tap.o
CC net/vhost-user.o
CC net/tap-linux.o
CC net/slirp.o
CC net/filter.o
CC net/filter-buffer.o
CC net/filter-mirror.o
CC net/colo-compare.o
CC net/colo.o
CC net/filter-rewriter.o
CC qom/cpu.o
CC replay/replay.o
CC replay/replay-internal.o
CC replay/replay-events.o
/tmp/qemu-test/src/replay/replay-internal.c: In function ‘replay_put_array’:
/tmp/qemu-test/src/replay/replay-internal.c:65: warning: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result
CC replay/replay-time.o
CC replay/replay-input.o
CC replay/replay-char.o
CC replay/replay-snapshot.o
CC slirp/cksum.o
CC slirp/if.o
CC slirp/ip_icmp.o
CC slirp/ip6_icmp.o
CC slirp/ip6_input.o
CC slirp/ip6_output.o
CC slirp/ip_input.o
CC slirp/dnssearch.o
CC slirp/ip_output.o
CC slirp/dhcpv6.o
CC slirp/mbuf.o
CC slirp/slirp.o
CC slirp/misc.o
CC slirp/sbuf.o
CC slirp/socket.o
CC slirp/tcp_input.o
CC slirp/tcp_output.o
CC slirp/tcp_subr.o
CC slirp/tcp_timer.o
CC slirp/udp.o
/tmp/qemu-test/src/slirp/tcp_input.c: In function ‘tcp_input’:
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_p’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_len’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_tos’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_id’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_off’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_ttl’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_sum’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_src.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:219: warning: ‘save_ip.ip_dst.s_addr’ may be used uninitialized in this function
/tmp/qemu-test/src/slirp/tcp_input.c:220: warning: ‘save_ip6.ip_nh’ may be used uninitialized in this function
CC slirp/udp6.o
CC slirp/bootp.o
CC slirp/tftp.o
CC slirp/arp_table.o
CC ui/keymaps.o
CC slirp/ndp_table.o
CC ui/console.o
CC ui/cursor.o
CC ui/qemu-pixman.o
CC ui/input.o
CC ui/input-keymap.o
CC ui/input-linux.o
CC ui/input-legacy.o
CC ui/sdl.o
CC ui/x_keymap.o
CC ui/sdl_zoom.o
CC ui/vnc-enc-zlib.o
CC ui/vnc.o
CC ui/vnc-enc-hextile.o
CC ui/vnc-enc-tight.o
CC ui/vnc-palette.o
CC ui/vnc-enc-zrle.o
CC ui/vnc-auth-vencrypt.o
CC ui/vnc-ws.o
CC ui/vnc-jobs.o
LINK tests/qemu-iotests/socket_scm_helper
CC qga/commands.o
CC qga/guest-agent-command-state.o
CC qga/main.o
CC qga/commands-posix.o
CC qga/channel-posix.o
AS optionrom/multiboot.o
CC qga/qapi-generated/qga-qapi-types.o
CC qga/qapi-generated/qga-qapi-visit.o
AS optionrom/linuxboot.o
CC qga/qapi-generated/qga-qmp-marshal.o
CC qmp-introspect.o
CC qapi-types.o
CC optionrom/linuxboot_dma.o
cc: unrecognized option '-no-integrated-as'
cc: unrecognized option '-no-integrated-as'
AS optionrom/kvmvapic.o
CC qapi-visit.o
BUILD optionrom/multiboot.img
BUILD optionrom/linuxboot.img
CC qapi-event.o
BUILD optionrom/linuxboot_dma.img
AR libqemustub.a
CC qemu-img.o
BUILD optionrom/linuxboot.raw
BUILD optionrom/multiboot.raw
BUILD optionrom/linuxboot_dma.raw
BUILD optionrom/kvmvapic.img
CC qmp-marshal.o
CC trace/generated-tracers.o
BUILD optionrom/kvmvapic.raw
SIGN optionrom/multiboot.bin
SIGN optionrom/linuxboot.bin
SIGN optionrom/linuxboot_dma.bin
SIGN optionrom/kvmvapic.bin
AR libqemuutil.a
LINK qemu-ga
LINK ivshmem-client
LINK ivshmem-server
LINK qemu-nbd
LINK qemu-img
LINK qemu-io
LINK qemu-bridge-helper
GEN aarch64-softmmu/hmp-commands.h
GEN aarch64-softmmu/hmp-commands-info.h
GEN aarch64-softmmu/config-target.h
CC aarch64-softmmu/exec.o
CC aarch64-softmmu/translate-all.o
CC aarch64-softmmu/cpu-exec.o
CC aarch64-softmmu/cpu-exec-common.o
CC aarch64-softmmu/translate-common.o
CC aarch64-softmmu/tcg/tcg.o
CC aarch64-softmmu/tcg/tcg-op.o
CC aarch64-softmmu/tcg/optimize.o
CC aarch64-softmmu/tcg/tcg-common.o
CC aarch64-softmmu/fpu/softfloat.o
CC aarch64-softmmu/tcg-runtime.o
CC aarch64-softmmu/disas.o
GEN aarch64-softmmu/gdbstub-xml.c
CC aarch64-softmmu/kvm-stub.o
CC aarch64-softmmu/arch_init.o
GEN x86_64-softmmu/hmp-commands.h
GEN x86_64-softmmu/hmp-commands-info.h
CC aarch64-softmmu/cpus.o
GEN x86_64-softmmu/config-target.h
CC aarch64-softmmu/monitor.o
CC aarch64-softmmu/gdbstub.o
CC x86_64-softmmu/exec.o
CC aarch64-softmmu/balloon.o
CC x86_64-softmmu/translate-all.o
CC aarch64-softmmu/ioport.o
CC aarch64-softmmu/numa.o
CC aarch64-softmmu/qtest.o
CC aarch64-softmmu/bootdevice.o
CC aarch64-softmmu/memory.o
CC aarch64-softmmu/cputlb.o
CC x86_64-softmmu/cpu-exec.o
CC aarch64-softmmu/memory_mapping.o
CC x86_64-softmmu/translate-common.o
CC x86_64-softmmu/cpu-exec-common.o
CC x86_64-softmmu/tcg/tcg.o
CC x86_64-softmmu/tcg/tcg-op.o
CC x86_64-softmmu/tcg/optimize.o
CC x86_64-softmmu/tcg/tcg-common.o
CC x86_64-softmmu/fpu/softfloat.o
CC x86_64-softmmu/disas.o
CC x86_64-softmmu/tcg-runtime.o
CC aarch64-softmmu/dump.o
CC x86_64-softmmu/arch_init.o
CC x86_64-softmmu/cpus.o
CC x86_64-softmmu/monitor.o
CC aarch64-softmmu/migration/ram.o
CC x86_64-softmmu/gdbstub.o
CC x86_64-softmmu/balloon.o
CC x86_64-softmmu/ioport.o
CC x86_64-softmmu/numa.o
CC x86_64-softmmu/qtest.o
CC x86_64-softmmu/bootdevice.o
CC x86_64-softmmu/kvm-all.o
CC aarch64-softmmu/migration/savevm.o
CC x86_64-softmmu/memory.o
CC x86_64-softmmu/cputlb.o
CC x86_64-softmmu/memory_mapping.o
CC aarch64-softmmu/xen-common-stub.o
CC x86_64-softmmu/dump.o
CC x86_64-softmmu/migration/ram.o
CC x86_64-softmmu/migration/savevm.o
CC aarch64-softmmu/xen-hvm-stub.o
CC x86_64-softmmu/xen-common-stub.o
CC x86_64-softmmu/xen-hvm-stub.o
CC x86_64-softmmu/hw/acpi/nvdimm.o
CC x86_64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/hw/adc/stm32f2xx_adc.o
CC aarch64-softmmu/hw/block/virtio-blk.o
CC aarch64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/hw/char/exynos4210_uart.o
CC aarch64-softmmu/hw/char/omap_uart.o
CC aarch64-softmmu/hw/char/digic-uart.o
CC aarch64-softmmu/hw/char/stm32f2xx_usart.o
CC aarch64-softmmu/hw/char/bcm2835_aux.o
CC aarch64-softmmu/hw/char/virtio-serial-bus.o
CC x86_64-softmmu/hw/block/dataplane/virtio-blk.o
CC aarch64-softmmu/hw/core/nmi.o
CC x86_64-softmmu/hw/char/virtio-serial-bus.o
CC x86_64-softmmu/hw/core/nmi.o
CC x86_64-softmmu/hw/core/generic-loader.o
CC aarch64-softmmu/hw/core/generic-loader.o
CC x86_64-softmmu/hw/cpu/core.o
CC x86_64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/display/virtio-gpu.o
CC aarch64-softmmu/hw/cpu/arm11mpcore.o
CC aarch64-softmmu/hw/cpu/realview_mpcore.o
CC aarch64-softmmu/hw/cpu/a9mpcore.o
CC aarch64-softmmu/hw/cpu/a15mpcore.o
CC aarch64-softmmu/hw/cpu/core.o
CC x86_64-softmmu/hw/display/virtio-gpu-3d.o
CC x86_64-softmmu/hw/display/virtio-gpu-pci.o
CC x86_64-softmmu/hw/display/virtio-vga.o
CC x86_64-softmmu/hw/intc/apic.o
CC x86_64-softmmu/hw/intc/apic_common.o
CC x86_64-softmmu/hw/intc/ioapic.o
CC aarch64-softmmu/hw/display/omap_dss.o
CC aarch64-softmmu/hw/display/omap_lcdc.o
CC aarch64-softmmu/hw/display/pxa2xx_lcd.o
CC aarch64-softmmu/hw/display/bcm2835_fb.o
CC x86_64-softmmu/hw/isa/lpc_ich9.o
CC aarch64-softmmu/hw/display/vga.o
CC x86_64-softmmu/hw/misc/vmport.o
CC x86_64-softmmu/hw/misc/ivshmem.o
CC aarch64-softmmu/hw/display/virtio-gpu.o
CC x86_64-softmmu/hw/misc/pvpanic.o
CC aarch64-softmmu/hw/display/virtio-gpu-3d.o
CC aarch64-softmmu/hw/display/virtio-gpu-pci.o
CC aarch64-softmmu/hw/display/dpcd.o
CC x86_64-softmmu/hw/misc/edu.o
CC x86_64-softmmu/hw/misc/hyperv_testdev.o
CC aarch64-softmmu/hw/display/xlnx_dp.o
CC aarch64-softmmu/hw/dma/xlnx_dpdma.o
CC aarch64-softmmu/hw/dma/omap_dma.o
CC aarch64-softmmu/hw/dma/soc_dma.o
CC aarch64-softmmu/hw/dma/pxa2xx_dma.o
CC aarch64-softmmu/hw/dma/bcm2835_dma.o
CC aarch64-softmmu/hw/gpio/omap_gpio.o
CC aarch64-softmmu/hw/gpio/imx_gpio.o
CC aarch64-softmmu/hw/i2c/omap_i2c.o
CC aarch64-softmmu/hw/input/pxa2xx_keypad.o
CC aarch64-softmmu/hw/input/tsc210x.o
CC x86_64-softmmu/hw/net/virtio-net.o
CC x86_64-softmmu/hw/net/vhost_net.o
CC x86_64-softmmu/hw/scsi/virtio-scsi.o
CC x86_64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/intc/armv7m_nvic.o
CC aarch64-softmmu/hw/intc/exynos4210_gic.o
CC x86_64-softmmu/hw/scsi/vhost-scsi.o
CC x86_64-softmmu/hw/timer/mc146818rtc.o
CC x86_64-softmmu/hw/vfio/common.o
CC x86_64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/intc/exynos4210_combiner.o
CC aarch64-softmmu/hw/intc/omap_intc.o
CC aarch64-softmmu/hw/intc/bcm2835_ic.o
CC aarch64-softmmu/hw/intc/bcm2836_control.o
CC x86_64-softmmu/hw/vfio/pci-quirks.o
CC x86_64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/intc/allwinner-a10-pic.o
CC aarch64-softmmu/hw/intc/aspeed_vic.o
CC aarch64-softmmu/hw/intc/arm_gicv3_cpuif.o
CC x86_64-softmmu/hw/vfio/calxeda-xgmac.o
CC x86_64-softmmu/hw/vfio/amd-xgbe.o
CC x86_64-softmmu/hw/vfio/spapr.o
CC aarch64-softmmu/hw/misc/ivshmem.o
CC x86_64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/misc/arm_sysctl.o
CC x86_64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/misc/cbus.o
CC aarch64-softmmu/hw/misc/exynos4210_pmu.o
CC aarch64-softmmu/hw/misc/imx_ccm.o
CC x86_64-softmmu/hw/virtio/vhost.o
CC aarch64-softmmu/hw/misc/imx31_ccm.o
CC aarch64-softmmu/hw/misc/imx25_ccm.o
CC x86_64-softmmu/hw/virtio/vhost-backend.o
CC aarch64-softmmu/hw/misc/imx6_ccm.o
CC aarch64-softmmu/hw/misc/imx6_src.o
CC aarch64-softmmu/hw/misc/mst_fpga.o
CC x86_64-softmmu/hw/virtio/vhost-user.o
CC aarch64-softmmu/hw/misc/omap_clk.o
CC aarch64-softmmu/hw/misc/omap_gpmc.o
CC x86_64-softmmu/hw/virtio/vhost-vsock.o
CC aarch64-softmmu/hw/misc/omap_l4.o
CC aarch64-softmmu/hw/misc/omap_sdrc.o
/tmp/qemu-test/src/hw/virtio/vhost-backend.c: In function ‘vhost_kernel_update_device_iotlb’:
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:229: error: unknown field ‘iotlb’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: error: unknown field ‘iova’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: warning: missing braces around initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: warning: (near initialization for ‘msg.<anonymous>.iotlb’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: error: unknown field ‘uaddr’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: error: unknown field ‘size’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: error: unknown field ‘type’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c: In function ‘vhost_kernel_invalidate_device_iotlb’:
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:264: error: unknown field ‘iotlb’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: error: unknown field ‘iova’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: warning: missing braces around initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: warning: (near initialization for ‘msg.<anonymous>.iotlb’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: error: unknown field ‘size’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: error: unknown field ‘type’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: warning: (near initialization for ‘msg.<anonymous>’)
make[1]: *** [hw/virtio/vhost-backend.o] Error 1
make[1]: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/misc/omap_tap.o
CC aarch64-softmmu/hw/misc/bcm2835_mbox.o
CC aarch64-softmmu/hw/misc/bcm2835_property.o
CC aarch64-softmmu/hw/misc/zynq_slcr.o
CC aarch64-softmmu/hw/misc/zynq-xadc.o
CC aarch64-softmmu/hw/misc/stm32f2xx_syscfg.o
CC aarch64-softmmu/hw/misc/edu.o
CC aarch64-softmmu/hw/misc/auxbus.o
CC aarch64-softmmu/hw/misc/aspeed_sdmc.o
CC aarch64-softmmu/hw/misc/aspeed_scu.o
CC aarch64-softmmu/hw/net/virtio-net.o
CC aarch64-softmmu/hw/net/vhost_net.o
CC aarch64-softmmu/hw/pcmcia/pxa2xx.o
CC aarch64-softmmu/hw/scsi/virtio-scsi.o
CC aarch64-softmmu/hw/scsi/virtio-scsi-dataplane.o
CC aarch64-softmmu/hw/scsi/vhost-scsi.o
make: *** [subdir-x86_64-softmmu] Error 2
make: *** Waiting for unfinished jobs....
CC aarch64-softmmu/hw/sd/omap_mmc.o
CC aarch64-softmmu/hw/sd/pxa2xx_mmci.o
CC aarch64-softmmu/hw/ssi/omap_spi.o
CC aarch64-softmmu/hw/ssi/imx_spi.o
CC aarch64-softmmu/hw/timer/exynos4210_mct.o
CC aarch64-softmmu/hw/timer/exynos4210_pwm.o
CC aarch64-softmmu/hw/timer/exynos4210_rtc.o
CC aarch64-softmmu/hw/timer/omap_gptimer.o
CC aarch64-softmmu/hw/timer/omap_synctimer.o
CC aarch64-softmmu/hw/timer/pxa2xx_timer.o
CC aarch64-softmmu/hw/timer/digic-timer.o
CC aarch64-softmmu/hw/timer/allwinner-a10-pit.o
CC aarch64-softmmu/hw/usb/tusb6010.o
CC aarch64-softmmu/hw/vfio/common.o
CC aarch64-softmmu/hw/vfio/pci.o
CC aarch64-softmmu/hw/vfio/pci-quirks.o
CC aarch64-softmmu/hw/vfio/platform.o
CC aarch64-softmmu/hw/vfio/calxeda-xgmac.o
CC aarch64-softmmu/hw/vfio/amd-xgbe.o
CC aarch64-softmmu/hw/vfio/spapr.o
CC aarch64-softmmu/hw/virtio/virtio.o
CC aarch64-softmmu/hw/virtio/virtio-balloon.o
CC aarch64-softmmu/hw/virtio/vhost.o
CC aarch64-softmmu/hw/virtio/vhost-backend.o
CC aarch64-softmmu/hw/virtio/vhost-user.o
CC aarch64-softmmu/hw/virtio/vhost-vsock.o
CC aarch64-softmmu/hw/arm/boot.o
CC aarch64-softmmu/hw/arm/collie.o
CC aarch64-softmmu/hw/arm/exynos4_boards.o
CC aarch64-softmmu/hw/arm/gumstix.o
CC aarch64-softmmu/hw/arm/highbank.o
/tmp/qemu-test/src/hw/virtio/vhost-backend.c: In function ‘vhost_kernel_update_device_iotlb’:
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:229: error: unknown field ‘iotlb’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: error: unknown field ‘iova’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: warning: missing braces around initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:230: warning: (near initialization for ‘msg.<anonymous>.iotlb’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: error: unknown field ‘uaddr’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:231: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: error: unknown field ‘size’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:232: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: error: unknown field ‘type’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:233: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c: In function ‘vhost_kernel_invalidate_device_iotlb’:
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:264: error: unknown field ‘iotlb’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: error: unknown field ‘iova’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: warning: missing braces around initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:265: warning: (near initialization for ‘msg.<anonymous>.iotlb’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: error: unknown field ‘size’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:266: warning: (near initialization for ‘msg.<anonymous>’)
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: error: unknown field ‘type’ specified in initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: warning: excess elements in union initializer
/tmp/qemu-test/src/hw/virtio/vhost-backend.c:267: warning: (near initialization for ‘msg.<anonymous>’)
make[1]: *** [hw/virtio/vhost-backend.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [subdir-aarch64-softmmu] Error 2
make[1]: *** [docker-run] Error 2
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-591ndo6_/src'
make: *** [docker-run-test-quick@centos6] Error 2
=== OUTPUT END ===
Test command exited with code: 2
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier Jason Wang
@ 2016-11-03 16:55 ` Paolo Bonzini
0 siblings, 0 replies; 25+ messages in thread
From: Paolo Bonzini @ 2016-11-03 16:55 UTC (permalink / raw)
To: Jason Wang, mst, peterx, wexu, qemu-devel; +Cc: vkaplans, cornelia.huck
On 03/11/2016 10:27, Jason Wang wrote:
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> memory.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/memory.c b/memory.c
> index 33110e9..2bfc37f 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -1603,6 +1603,11 @@ static void memory_region_update_iommu_notify_flags(MemoryRegion *mr)
> void memory_region_register_iommu_notifier(MemoryRegion *mr,
> IOMMUNotifier *n)
> {
> + if (mr->alias) {
> + memory_region_register_iommu_notifier(mr->alias, n);
> + return;
> + }
> +
> /* We need to register for at least one bitfield */
> assert(n->notifier_flags != IOMMU_NOTIFIER_NONE);
> QLIST_INSERT_HEAD(&mr->iommu_notify, n, node);
> @@ -1643,6 +1648,10 @@ void memory_region_iommu_replay(MemoryRegion *mr, IOMMUNotifier *n,
> void memory_region_unregister_iommu_notifier(MemoryRegion *mr,
> IOMMUNotifier *n)
> {
> + if (mr->alias) {
> + memory_region_unregister_iommu_notifier(mr->alias, n);
> + return;
> + }
> QLIST_REMOVE(n, node);
> memory_region_update_iommu_notify_flags(mr);
> }
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu()
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu() Jason Wang
@ 2016-11-03 16:56 ` Paolo Bonzini
0 siblings, 0 replies; 25+ messages in thread
From: Paolo Bonzini @ 2016-11-03 16:56 UTC (permalink / raw)
To: Jason Wang, mst, peterx, wexu, qemu-devel; +Cc: vkaplans, cornelia.huck
On 03/11/2016 10:27, Jason Wang wrote:
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> include/exec/memory.h | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index e605de3..ab37499 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -628,6 +628,9 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
> */
> static inline bool memory_region_is_iommu(MemoryRegion *mr)
> {
> + if (mr->alias) {
> + return memory_region_is_iommu(mr->alias);
> + }
> return mr->iommu_ops;
> }
>
>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry()
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry() Jason Wang
@ 2016-11-03 17:03 ` Paolo Bonzini
2016-11-04 6:33 ` Jason Wang
0 siblings, 1 reply; 25+ messages in thread
From: Paolo Bonzini @ 2016-11-03 17:03 UTC (permalink / raw)
To: Jason Wang, mst, peterx, wexu, qemu-devel
Cc: vkaplans, cornelia.huck, Peter Crosthwaite, Richard Henderson
On 03/11/2016 10:27, Jason Wang wrote:
> This patch introduces a helper to query the iotlb entry for a
> possible iova. This will be used by later device IOTLB API to enable
> the capability for a dataplane (e.g vhost) to query the IOTLB.
>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Peter Crosthwaite <crosthwaite.peter@gmail.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> exec.c | 33 +++++++++++++++++++++++++++++++++
> include/exec/memory.h | 6 ++++++
> 2 files changed, 39 insertions(+)
>
> diff --git a/exec.c b/exec.c
> index b1094c0..00c7a2b 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -449,6 +449,39 @@ address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *x
> }
>
> /* Called from RCU critical section */
> +IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
> + bool is_write)
> +{
> + IOMMUTLBEntry iotlb = {0};
> + MemoryRegionSection *section;
> + MemoryRegion *mr;
> + hwaddr plen;
plen must be initialized on entry to address_space_translate_internal,
since it's set to
MIN(plen, MIN(section->size,
addr - section->offset_within_address_space))
after address_space_translate_internal calls
address_space_lookup_region. But you don't really need plen, so you
should do this:
section = address_space_lookup_region(d, addr, false);
addr = addr
- section->offset_within_address_space
+ section->offset_within_region;
instead of
section = address_space_translate_internal(d, addr, &addr,
&plen, true);
and then you don't even need a plen anymore.
Also:
> + for (;;) {
> + AddressSpaceDispatch *d = atomic_rcu_read(&as->dispatch);
> + section = address_space_translate_internal(d, addr, &addr, &plen, true);
> + mr = section->mr;
> +
> + if (!mr->iommu_ops) {
> + break;
> + }
> +
> + iotlb = mr->iommu_ops->translate(mr, addr, is_write);
> + addr = ((iotlb.translated_addr & ~iotlb.addr_mask)
> + | (addr & iotlb.addr_mask));
This addr assignment is only needed after the iotlb.perm check, so move
it there.
Thanks,
Paolo
> + plen = MIN(plen, (addr | iotlb.addr_mask) - addr + 1);
> + if (!(iotlb.perm & (1 << is_write))) {
> + iotlb.target_as = NULL;
> + break;
> + }
> +
> + as = iotlb.target_as;
> + }
> +
> + return iotlb;
> +}
> +
> +/* Called from RCU critical section */
> MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
> hwaddr *xlat, hwaddr *plen,
> bool is_write)
> diff --git a/include/exec/memory.h b/include/exec/memory.h
> index 9728a2f..e605de3 100644
> --- a/include/exec/memory.h
> +++ b/include/exec/memory.h
> @@ -1404,6 +1404,12 @@ void address_space_stq_le(AddressSpace *as, hwaddr addr, uint64_t val,
> void address_space_stq_be(AddressSpace *as, hwaddr addr, uint64_t val,
> MemTxAttrs attrs, MemTxResult *result);
>
> +/* address_space_get_iotlb_entry: translate an address into an IOTLB
> + * entry. Should be called from an RCU critical section.
> + */
> +IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
> + bool is_write);
> +
> /* address_space_translate: translate an address range into an address space
> * into a MemoryRegion and an address range into that section. Should be
> * called from an RCU critical section, to avoid that the last reference
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api Jason Wang
@ 2016-11-03 19:46 ` Michael S. Tsirkin
2016-11-04 6:36 ` Jason Wang
0 siblings, 1 reply; 25+ messages in thread
From: Michael S. Tsirkin @ 2016-11-03 19:46 UTC (permalink / raw)
To: Jason Wang
Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck,
Stefan Hajnoczi, Kevin Wolf, Amit Shah, qemu-block
On Thu, Nov 03, 2016 at 05:27:14PM +0800, Jason Wang wrote:
> Currently, all virtio devices bypass IOMMU completely. This is because
> address_space_memory is assumed and used during DMA emulation. This
> patch converts the virtio core API to use DMA API. This idea is
>
> - introducing a new transport specific helper to query the dma address
> space. (only pci version is implemented).
> - query and use this address space during virtio device guest memory
> accessing when iommu platform (VIRTIO_F_IOMMU_PLATFORM) was enabled
> for this device.
>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Amit Shah <amit.shah@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: qemu-block@nongnu.org
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> hw/block/virtio-blk.c | 2 +-
> hw/char/virtio-serial-bus.c | 3 ++-
> hw/scsi/virtio-scsi.c | 4 ++-
> hw/virtio/virtio-bus.c | 8 ++++++
> hw/virtio/virtio-pci.c | 14 ++++++++++
> hw/virtio/virtio.c | 57 +++++++++++++++++++++++++--------------
> include/hw/virtio/virtio-access.h | 36 ++++++++++++++++++-------
> include/hw/virtio/virtio-bus.h | 1 +
> include/hw/virtio/virtio.h | 9 ++++---
> 9 files changed, 98 insertions(+), 36 deletions(-)
>
> diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
> index 37fe72b..6a2dbaf 100644
> --- a/hw/block/virtio-blk.c
> +++ b/hw/block/virtio-blk.c
> @@ -860,7 +860,7 @@ static int virtio_blk_load_device(VirtIODevice *vdev, QEMUFile *f,
> }
> }
>
> - req = qemu_get_virtqueue_element(f, sizeof(VirtIOBlockReq));
> + req = qemu_get_virtqueue_element(vdev, f, sizeof(VirtIOBlockReq));
> virtio_blk_init_request(s, virtio_get_queue(vdev, vq_idx), req);
> req->next = s->rq;
> s->rq = req;
> diff --git a/hw/char/virtio-serial-bus.c b/hw/char/virtio-serial-bus.c
> index 7975c2c..d544cd9 100644
> --- a/hw/char/virtio-serial-bus.c
> +++ b/hw/char/virtio-serial-bus.c
> @@ -732,6 +732,7 @@ static void virtio_serial_post_load_timer_cb(void *opaque)
> static int fetch_active_ports_list(QEMUFile *f,
> VirtIOSerial *s, uint32_t nr_active_ports)
> {
> + VirtIODevice *vdev = VIRTIO_DEVICE(s);
> uint32_t i;
>
> s->post_load = g_malloc0(sizeof(*s->post_load));
> @@ -765,7 +766,7 @@ static int fetch_active_ports_list(QEMUFile *f,
> qemu_get_be64s(f, &port->iov_offset);
>
> port->elem =
> - qemu_get_virtqueue_element(f, sizeof(VirtQueueElement));
> + qemu_get_virtqueue_element(vdev, f, sizeof(VirtQueueElement));
>
> /*
> * Port was throttled on source machine. Let's
> diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c
> index 4762f05..1519cee 100644
> --- a/hw/scsi/virtio-scsi.c
> +++ b/hw/scsi/virtio-scsi.c
> @@ -198,12 +198,14 @@ static void *virtio_scsi_load_request(QEMUFile *f, SCSIRequest *sreq)
> SCSIBus *bus = sreq->bus;
> VirtIOSCSI *s = container_of(bus, VirtIOSCSI, bus);
> VirtIOSCSICommon *vs = VIRTIO_SCSI_COMMON(s);
> + VirtIODevice *vdev = VIRTIO_DEVICE(s);
> VirtIOSCSIReq *req;
> uint32_t n;
>
> qemu_get_be32s(f, &n);
> assert(n < vs->conf.num_queues);
> - req = qemu_get_virtqueue_element(f, sizeof(VirtIOSCSIReq) + vs->cdb_size);
> + req = qemu_get_virtqueue_element(vdev, f,
> + sizeof(VirtIOSCSIReq) + vs->cdb_size);
> virtio_scsi_init_req(s, vs->cmd_vqs[n], req);
>
> if (virtio_scsi_parse_req(req, sizeof(VirtIOSCSICmdReq) + vs->cdb_size,
> diff --git a/hw/virtio/virtio-bus.c b/hw/virtio/virtio-bus.c
> index 11f65bd..8597762 100644
> --- a/hw/virtio/virtio-bus.c
> +++ b/hw/virtio/virtio-bus.c
> @@ -28,6 +28,7 @@
> #include "hw/qdev.h"
> #include "hw/virtio/virtio-bus.h"
> #include "hw/virtio/virtio.h"
> +#include "exec/address-spaces.h"
>
> /* #define DEBUG_VIRTIO_BUS */
>
> @@ -61,6 +62,13 @@ void virtio_bus_device_plugged(VirtIODevice *vdev, Error **errp)
> if (klass->device_plugged != NULL) {
> klass->device_plugged(qbus->parent, errp);
> }
> +
> + if (klass->get_dma_as != NULL &&
> + virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
> + vdev->dma_as = klass->get_dma_as(qbus->parent);
> + } else {
> + vdev->dma_as = &address_space_memory;
> + }
> }
>
> /* Reset the virtio_bus */
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index 06831de..6ceb43e 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1168,6 +1168,14 @@ static int virtio_pci_query_nvectors(DeviceState *d)
> return proxy->nvectors;
> }
>
> +static AddressSpace *virtio_pci_get_dma_as(DeviceState *d)
> +{
> + VirtIOPCIProxy *proxy = VIRTIO_PCI(d);
> + PCIDevice *dev = &proxy->pci_dev;
> +
> + return pci_get_address_space(dev);
> +}
> +
> static int virtio_pci_add_mem_cap(VirtIOPCIProxy *proxy,
> struct virtio_pci_cap *cap)
> {
> @@ -1624,6 +1632,11 @@ static void virtio_pci_device_plugged(DeviceState *d, Error **errp)
> }
>
> if (legacy) {
> + if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
> + error_setg(errp, "VIRTIO_F_IOMMU_PLATFORM was supported by"
> + "neither legacy nor transitional device.");
> + return ;
> + }
> /* legacy and transitional */
> pci_set_word(config + PCI_SUBSYSTEM_VENDOR_ID,
> pci_get_word(config + PCI_VENDOR_ID));
> @@ -2544,6 +2557,7 @@ static void virtio_pci_bus_class_init(ObjectClass *klass, void *data)
> k->ioeventfd_disabled = virtio_pci_ioeventfd_disabled;
> k->ioeventfd_set_disabled = virtio_pci_ioeventfd_set_disabled;
> k->ioeventfd_assign = virtio_pci_ioeventfd_assign;
> + k->get_dma_as = virtio_pci_get_dma_as;
> }
>
> static const TypeInfo virtio_pci_bus_info = {
> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
> index d48d1a9..85ad828 100644
> --- a/hw/virtio/virtio.c
> +++ b/hw/virtio/virtio.c
> @@ -23,6 +23,7 @@
> #include "hw/virtio/virtio-bus.h"
> #include "migration/migration.h"
> #include "hw/virtio/virtio-access.h"
> +#include "sysemu/dma.h"
>
> /*
> * The alignment to use between consumer and producer parts of vring.
> @@ -122,7 +123,7 @@ void virtio_queue_update_rings(VirtIODevice *vdev, int n)
> static void vring_desc_read(VirtIODevice *vdev, VRingDesc *desc,
> hwaddr desc_pa, int i)
> {
> - address_space_read(&address_space_memory, desc_pa + i * sizeof(VRingDesc),
> + address_space_read(virtio_get_dma_as(vdev), desc_pa + i * sizeof(VRingDesc),
> MEMTXATTRS_UNSPECIFIED, (void *)desc, sizeof(VRingDesc));
> virtio_tswap64s(vdev, &desc->addr);
> virtio_tswap32s(vdev, &desc->len);
> @@ -164,7 +165,7 @@ static inline void vring_used_write(VirtQueue *vq, VRingUsedElem *uelem,
> virtio_tswap32s(vq->vdev, &uelem->id);
> virtio_tswap32s(vq->vdev, &uelem->len);
> pa = vq->vring.used + offsetof(VRingUsed, ring[i]);
> - address_space_write(&address_space_memory, pa, MEMTXATTRS_UNSPECIFIED,
> + address_space_write(virtio_get_dma_as(vq->vdev), pa, MEMTXATTRS_UNSPECIFIED,
> (void *)uelem, sizeof(VRingUsedElem));
> }
>
> @@ -244,6 +245,7 @@ int virtio_queue_empty(VirtQueue *vq)
> static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
> unsigned int len)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vq->vdev);
> unsigned int offset;
> int i;
Can't we use
vdev->dma_as
here?
I'd like to avoid query on data path.
Same in most other places below.
>
> @@ -251,17 +253,18 @@ static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
> for (i = 0; i < elem->in_num; i++) {
> size_t size = MIN(len - offset, elem->in_sg[i].iov_len);
>
> - cpu_physical_memory_unmap(elem->in_sg[i].iov_base,
> - elem->in_sg[i].iov_len,
> - 1, size);
> + dma_memory_unmap(dma_as, elem->in_sg[i].iov_base,
> + elem->in_sg[i].iov_len,
> + DMA_DIRECTION_FROM_DEVICE, size);
>
> offset += size;
> }
>
> for (i = 0; i < elem->out_num; i++)
> - cpu_physical_memory_unmap(elem->out_sg[i].iov_base,
> - elem->out_sg[i].iov_len,
> - 0, elem->out_sg[i].iov_len);
> + dma_memory_unmap(dma_as, elem->out_sg[i].iov_base,
> + elem->out_sg[i].iov_len,
> + DMA_DIRECTION_TO_DEVICE,
> + elem->out_sg[i].iov_len);
> }
>
> /* virtqueue_detach_element:
> @@ -555,7 +558,10 @@ static bool virtqueue_map_desc(VirtIODevice *vdev, unsigned int *p_num_sg,
> goto out;
> }
>
> - iov[num_sg].iov_base = cpu_physical_memory_map(pa, &len, is_write);
> + iov[num_sg].iov_base = dma_memory_map(virtio_get_dma_as(vdev), pa, &len,
> + is_write ?
> + DMA_DIRECTION_FROM_DEVICE :
> + DMA_DIRECTION_TO_DEVICE);
> if (!iov[num_sg].iov_base) {
> virtio_error(vdev, "virtio: bogus descriptor or out of resources");
> goto out;
> @@ -592,9 +598,9 @@ static void virtqueue_undo_map_desc(unsigned int out_num, unsigned int in_num,
> }
> }
>
> -static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
> - unsigned int *num_sg, unsigned int max_size,
> - int is_write)
> +static void virtqueue_map_iovec(VirtIODevice *vdev, struct iovec *sg,
> + hwaddr *addr, unsigned int *num_sg,
> + unsigned int max_size, int is_write)
> {
> unsigned int i;
> hwaddr len;
> @@ -613,7 +619,10 @@ static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
>
> for (i = 0; i < *num_sg; i++) {
> len = sg[i].iov_len;
> - sg[i].iov_base = cpu_physical_memory_map(addr[i], &len, is_write);
> + sg[i].iov_base = dma_memory_map(virtio_get_dma_as(vdev),
> + addr[i], &len, is_write ?
> + DMA_DIRECTION_FROM_DEVICE :
> + DMA_DIRECTION_TO_DEVICE);
> if (!sg[i].iov_base) {
> error_report("virtio: error trying to map MMIO memory");
> exit(1);
> @@ -625,12 +634,15 @@ static void virtqueue_map_iovec(struct iovec *sg, hwaddr *addr,
> }
> }
>
> -void virtqueue_map(VirtQueueElement *elem)
> +void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem)
> {
> - virtqueue_map_iovec(elem->in_sg, elem->in_addr, &elem->in_num,
> - VIRTQUEUE_MAX_SIZE, 1);
> - virtqueue_map_iovec(elem->out_sg, elem->out_addr, &elem->out_num,
> - VIRTQUEUE_MAX_SIZE, 0);
> + virtqueue_map_iovec(vdev, elem->in_sg, elem->in_addr, &elem->in_num,
> + MIN(ARRAY_SIZE(elem->in_sg), ARRAY_SIZE(elem->in_addr)),
> + 1);
> + virtqueue_map_iovec(vdev, elem->out_sg, elem->out_addr, &elem->out_num,
> + MIN(ARRAY_SIZE(elem->out_sg),
> + ARRAY_SIZE(elem->out_addr)),
> + 0);
> }
>
> void *virtqueue_alloc_element(size_t sz, unsigned out_num, unsigned in_num)
> @@ -783,7 +795,7 @@ typedef struct VirtQueueElementOld {
> struct iovec out_sg[VIRTQUEUE_MAX_SIZE];
> } VirtQueueElementOld;
>
> -void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz)
> +void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz)
> {
> VirtQueueElement *elem;
> VirtQueueElementOld data;
> @@ -814,7 +826,7 @@ void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz)
> elem->out_sg[i].iov_len = data.out_sg[i].iov_len;
> }
>
> - virtqueue_map(elem);
> + virtqueue_map(vdev, elem);
> return elem;
> }
>
> @@ -873,6 +885,11 @@ static int virtio_validate_features(VirtIODevice *vdev)
> {
> VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
>
> + if (virtio_host_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM) &&
> + !virtio_vdev_has_feature(vdev, VIRTIO_F_IOMMU_PLATFORM)) {
> + return -EFAULT;
> + }
> +
> if (k->validate_features) {
> return k->validate_features(vdev);
> } else {
> diff --git a/include/hw/virtio/virtio-access.h b/include/hw/virtio/virtio-access.h
> index 440b455..4643a06 100644
> --- a/include/hw/virtio/virtio-access.h
> +++ b/include/hw/virtio/virtio-access.h
> @@ -17,12 +17,18 @@
> #define QEMU_VIRTIO_ACCESS_H
>
> #include "hw/virtio/virtio.h"
> +#include "hw/virtio/virtio-bus.h"
> #include "exec/address-spaces.h"
>
> #if defined(TARGET_PPC64) || defined(TARGET_ARM)
> #define LEGACY_VIRTIO_IS_BIENDIAN 1
> #endif
>
> +static inline AddressSpace *virtio_get_dma_as(VirtIODevice *vdev)
> +{
> + return vdev->dma_as;
> +}
> +
> static inline bool virtio_access_is_big_endian(VirtIODevice *vdev)
> {
> #if defined(LEGACY_VIRTIO_IS_BIENDIAN)
> @@ -40,45 +46,55 @@ static inline bool virtio_access_is_big_endian(VirtIODevice *vdev)
>
> static inline uint16_t virtio_lduw_phys(VirtIODevice *vdev, hwaddr pa)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vdev);
> +
> if (virtio_access_is_big_endian(vdev)) {
> - return lduw_be_phys(&address_space_memory, pa);
> + return lduw_be_phys(dma_as, pa);
> }
> - return lduw_le_phys(&address_space_memory, pa);
> + return lduw_le_phys(dma_as, pa);
> }
>
> static inline uint32_t virtio_ldl_phys(VirtIODevice *vdev, hwaddr pa)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vdev);
> +
> if (virtio_access_is_big_endian(vdev)) {
> - return ldl_be_phys(&address_space_memory, pa);
> + return ldl_be_phys(dma_as, pa);
> }
> - return ldl_le_phys(&address_space_memory, pa);
> + return ldl_le_phys(dma_as, pa);
> }
>
> static inline uint64_t virtio_ldq_phys(VirtIODevice *vdev, hwaddr pa)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vdev);
> +
> if (virtio_access_is_big_endian(vdev)) {
> - return ldq_be_phys(&address_space_memory, pa);
> + return ldq_be_phys(dma_as, pa);
> }
> - return ldq_le_phys(&address_space_memory, pa);
> + return ldq_le_phys(dma_as, pa);
> }
>
> static inline void virtio_stw_phys(VirtIODevice *vdev, hwaddr pa,
> uint16_t value)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vdev);
> +
> if (virtio_access_is_big_endian(vdev)) {
> - stw_be_phys(&address_space_memory, pa, value);
> + stw_be_phys(dma_as, pa, value);
> } else {
> - stw_le_phys(&address_space_memory, pa, value);
> + stw_le_phys(dma_as, pa, value);
> }
> }
>
> static inline void virtio_stl_phys(VirtIODevice *vdev, hwaddr pa,
> uint32_t value)
> {
> + AddressSpace *dma_as = virtio_get_dma_as(vdev);
> +
> if (virtio_access_is_big_endian(vdev)) {
> - stl_be_phys(&address_space_memory, pa, value);
> + stl_be_phys(dma_as, pa, value);
> } else {
> - stl_le_phys(&address_space_memory, pa, value);
> + stl_le_phys(dma_as, pa, value);
> }
> }
>
> diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h
> index 2e4b67e..9d5aa79 100644
> --- a/include/hw/virtio/virtio-bus.h
> +++ b/include/hw/virtio/virtio-bus.h
> @@ -98,6 +98,7 @@ typedef struct VirtioBusClass {
> * Note that changing this will break migration for this transport.
> */
> bool has_variable_vring_alignment;
> + AddressSpace *(*get_dma_as)(DeviceState *d);
> } VirtioBusClass;
>
> struct VirtioBusState {
> diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
> index b913aac..1245831 100644
> --- a/include/hw/virtio/virtio.h
> +++ b/include/hw/virtio/virtio.h
> @@ -92,6 +92,7 @@ struct VirtIODevice
> char *bus_name;
> uint8_t device_endian;
> bool use_guest_notifier_mask;
> + AddressSpace *dma_as;
> QLIST_HEAD(, VirtQueue) *vector_queues;
> };
>
> @@ -163,9 +164,9 @@ bool virtqueue_rewind(VirtQueue *vq, unsigned int num);
> void virtqueue_fill(VirtQueue *vq, const VirtQueueElement *elem,
> unsigned int len, unsigned int idx);
>
> -void virtqueue_map(VirtQueueElement *elem);
> +void virtqueue_map(VirtIODevice *vdev, VirtQueueElement *elem);
> void *virtqueue_pop(VirtQueue *vq, size_t sz);
> -void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz);
> +void *qemu_get_virtqueue_element(VirtIODevice *vdev, QEMUFile *f, size_t sz);
> void qemu_put_virtqueue_element(QEMUFile *f, VirtQueueElement *elem);
> int virtqueue_avail_bytes(VirtQueue *vq, unsigned int in_bytes,
> unsigned int out_bytes);
> @@ -247,7 +248,9 @@ typedef struct VirtIORNGConf VirtIORNGConf;
> DEFINE_PROP_BIT64("notify_on_empty", _state, _field, \
> VIRTIO_F_NOTIFY_ON_EMPTY, true), \
> DEFINE_PROP_BIT64("any_layout", _state, _field, \
> - VIRTIO_F_ANY_LAYOUT, true)
> + VIRTIO_F_ANY_LAYOUT, true), \
> + DEFINE_PROP_BIT64("iommu_platform", _state, _field, \
> + VIRTIO_F_IOMMU_PLATFORM, false)
>
> hwaddr virtio_queue_get_desc_addr(VirtIODevice *vdev, int n);
> hwaddr virtio_queue_get_avail_addr(VirtIODevice *vdev, int n);
> --
> 2.7.4
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support Jason Wang
@ 2016-11-03 19:49 ` Michael S. Tsirkin
2016-11-04 6:48 ` Jason Wang
0 siblings, 1 reply; 25+ messages in thread
From: Michael S. Tsirkin @ 2016-11-03 19:49 UTC (permalink / raw)
To: Jason Wang; +Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
> This patches enable the Address Translation Service support for virtio
> pci devices. This is needed for a guest visible Device IOTLB
> implementation and will be required by vhost device IOTLB API
> implementation for intel IOMMU.
>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
I'd like to understand why do you think this is strictly required.
Won't setting CM bit in the IOMMU do the trick.
Also, could you remind me pls - can guests just disable ATS?
What happens then?
> ---
> hw/pci/pcie.c | 16 ++++++++++++++++
> hw/virtio/virtio-pci.c | 7 +++++++
> hw/virtio/virtio-pci.h | 4 ++++
> include/hw/pci/pcie.h | 4 ++++
> include/standard-headers/linux/pci_regs.h | 1 +
> 5 files changed, 32 insertions(+)
>
> diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c
> index 99cfb45..02195d9 100644
> --- a/hw/pci/pcie.c
> +++ b/hw/pci/pcie.c
> @@ -717,3 +717,19 @@ void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num)
> PCI_EXT_CAP_DSN_SIZEOF);
> pci_set_quad(dev->config + offset + pci_dsn_cap, ser_num);
> }
> +
> +void pcie_ats_init(PCIDevice *dev, uint16_t offset)
> +{
> + pcie_add_capability(dev, PCI_EXT_CAP_ID_ATS, 0x1,
> + offset, PCI_EXT_CAP_ATS_SIZEOF);
> +
> + dev->exp.ats_cap = offset;
> +
> + /* Invalidate Queue Depth 0, Page Aligned Request 0 */
> + pci_set_word(dev->config + offset + PCI_ATS_CAP, 0);
> + /* STU 0, Disabled by default */
> + pci_set_word(dev->config + offset + PCI_ATS_CTRL, 0);
> +
> + pci_set_word(dev->wmask + dev->exp.ats_cap + PCI_ATS_CTRL, 0x800f);
> +}
> +
> diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
> index 6ceb43e..e357bdf 100644
> --- a/hw/virtio/virtio-pci.c
> +++ b/hw/virtio/virtio-pci.c
> @@ -1838,6 +1838,11 @@ static void virtio_pci_realize(PCIDevice *pci_dev, Error **errp)
> * PCI Power Management Interface Specification.
> */
> pci_set_word(pci_dev->config + pos + PCI_PM_PMC, 0x3);
> +
> + if (proxy->flags & VIRTIO_PCI_FLAG_ATS) {
> + pcie_ats_init(pci_dev, 256);
> + }
> +
> } else {
> /*
> * make future invocations of pci_is_express() return false
> @@ -1889,6 +1894,8 @@ static Property virtio_pci_properties[] = {
> VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT, false),
> DEFINE_PROP_BIT("page-per-vq", VirtIOPCIProxy, flags,
> VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT, false),
> + DEFINE_PROP_BIT("ats", VirtIOPCIProxy, flags,
> + VIRTIO_PCI_FLAG_ATS_BIT, false),
> DEFINE_PROP_END_OF_LIST(),
> };
>
> diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h
> index b4edea6..057d49d 100644
> --- a/hw/virtio/virtio-pci.h
> +++ b/hw/virtio/virtio-pci.h
> @@ -69,6 +69,7 @@ enum {
> VIRTIO_PCI_FLAG_MODERN_PIO_NOTIFY_BIT,
> VIRTIO_PCI_FLAG_DISABLE_PCIE_BIT,
> VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT,
> + VIRTIO_PCI_FLAG_ATS_BIT,
> };
>
> /* Need to activate work-arounds for buggy guests at vmstate load. */
> @@ -93,6 +94,9 @@ enum {
> #define VIRTIO_PCI_FLAG_PAGE_PER_VQ \
> (1 << VIRTIO_PCI_FLAG_PAGE_PER_VQ_BIT)
>
> +/* address space translation service */
> +#define VIRTIO_PCI_FLAG_ATS (1 << VIRTIO_PCI_FLAG_ATS_BIT)
> +
> typedef struct {
> MSIMessage msg;
> int virq;
> diff --git a/include/hw/pci/pcie.h b/include/hw/pci/pcie.h
> index 056d25e..b08451d 100644
> --- a/include/hw/pci/pcie.h
> +++ b/include/hw/pci/pcie.h
> @@ -74,6 +74,9 @@ struct PCIExpressDevice {
> /* AER */
> uint16_t aer_cap;
> PCIEAERLog aer_log;
> +
> + /* Offset of ATS capability in config space */
> + uint16_t ats_cap;
> };
>
> #define COMPAT_PROP_PCP "power_controller_present"
> @@ -120,6 +123,7 @@ void pcie_add_capability(PCIDevice *dev,
>
> void pcie_ari_init(PCIDevice *dev, uint16_t offset, uint16_t nextfn);
> void pcie_dev_ser_num_init(PCIDevice *dev, uint16_t offset, uint64_t ser_num);
> +void pcie_ats_init(PCIDevice *dev, uint16_t offset);
>
> extern const VMStateDescription vmstate_pcie_device;
>
> diff --git a/include/standard-headers/linux/pci_regs.h b/include/standard-headers/linux/pci_regs.h
> index 4040951..ac426a0 100644
> --- a/include/standard-headers/linux/pci_regs.h
> +++ b/include/standard-headers/linux/pci_regs.h
> @@ -674,6 +674,7 @@
> #define PCI_EXT_CAP_ID_MAX PCI_EXT_CAP_ID_DPC
>
> #define PCI_EXT_CAP_DSN_SIZEOF 12
> +#define PCI_EXT_CAP_ATS_SIZEOF 8
> #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
>
> /* Advanced Error Reporting */
> --
> 2.7.4
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry()
2016-11-03 17:03 ` Paolo Bonzini
@ 2016-11-04 6:33 ` Jason Wang
0 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-04 6:33 UTC (permalink / raw)
To: Paolo Bonzini, mst, peterx, wexu, qemu-devel
Cc: vkaplans, cornelia.huck, Richard Henderson, Peter Crosthwaite
On 2016年11月04日 01:03, Paolo Bonzini wrote:
>
> On 03/11/2016 10:27, Jason Wang wrote:
>> This patch introduces a helper to query the iotlb entry for a
>> possible iova. This will be used by later device IOTLB API to enable
>> the capability for a dataplane (e.g vhost) to query the IOTLB.
>>
>> Cc: Paolo Bonzini <pbonzini@redhat.com>
>> Cc: Peter Crosthwaite <crosthwaite.peter@gmail.com>
>> Cc: Richard Henderson <rth@twiddle.net>
>> Signed-off-by: Jason Wang <jasowang@redhat.com>
>> ---
>> exec.c | 33 +++++++++++++++++++++++++++++++++
>> include/exec/memory.h | 6 ++++++
>> 2 files changed, 39 insertions(+)
>>
>> diff --git a/exec.c b/exec.c
>> index b1094c0..00c7a2b 100644
>> --- a/exec.c
>> +++ b/exec.c
>> @@ -449,6 +449,39 @@ address_space_translate_internal(AddressSpaceDispatch *d, hwaddr addr, hwaddr *x
>> }
>>
>> /* Called from RCU critical section */
>> +IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
>> + bool is_write)
>> +{
>> + IOMMUTLBEntry iotlb = {0};
>> + MemoryRegionSection *section;
>> + MemoryRegion *mr;
>> + hwaddr plen;
> plen must be initialized on entry to address_space_translate_internal,
> since it's set to
>
> MIN(plen, MIN(section->size,
> addr - section->offset_within_address_space))
>
> after address_space_translate_internal calls
> address_space_lookup_region. But you don't really need plen, so you
> should do this:
>
> section = address_space_lookup_region(d, addr, false);
> addr = addr
> - section->offset_within_address_space
> + section->offset_within_region;
>
> instead of
>
> section = address_space_translate_internal(d, addr, &addr,
> &plen, true);
>
> and then you don't even need a plen anymore.
Cool, will do this in next version.
>
> Also:
>
>
>> + for (;;) {
>> + AddressSpaceDispatch *d = atomic_rcu_read(&as->dispatch);
>> + section = address_space_translate_internal(d, addr, &addr, &plen, true);
>> + mr = section->mr;
>> +
>> + if (!mr->iommu_ops) {
>> + break;
>> + }
>> +
>> + iotlb = mr->iommu_ops->translate(mr, addr, is_write);
>> + addr = ((iotlb.translated_addr & ~iotlb.addr_mask)
>> + | (addr & iotlb.addr_mask));
> This addr assignment is only needed after the iotlb.perm check, so move
> it there.
>
> Thanks,
>
> Paolo
Right.
Thanks
>
>> + plen = MIN(plen, (addr | iotlb.addr_mask) - addr + 1);
>> + if (!(iotlb.perm & (1 << is_write))) {
>> + iotlb.target_as = NULL;
>> + break;
>> + }
>> +
>> + as = iotlb.target_as;
>> + }
>> +
>> + return iotlb;
>> +}
>> +
>> +/* Called from RCU critical section */
>> MemoryRegion *address_space_translate(AddressSpace *as, hwaddr addr,
>> hwaddr *xlat, hwaddr *plen,
>> bool is_write)
>> diff --git a/include/exec/memory.h b/include/exec/memory.h
>> index 9728a2f..e605de3 100644
>> --- a/include/exec/memory.h
>> +++ b/include/exec/memory.h
>> @@ -1404,6 +1404,12 @@ void address_space_stq_le(AddressSpace *as, hwaddr addr, uint64_t val,
>> void address_space_stq_be(AddressSpace *as, hwaddr addr, uint64_t val,
>> MemTxAttrs attrs, MemTxResult *result);
>>
>> +/* address_space_get_iotlb_entry: translate an address into an IOTLB
>> + * entry. Should be called from an RCU critical section.
>> + */
>> +IOMMUTLBEntry address_space_get_iotlb_entry(AddressSpace *as, hwaddr addr,
>> + bool is_write);
>> +
>> /* address_space_translate: translate an address range into an address space
>> * into a MemoryRegion and an address range into that section. Should be
>> * called from an RCU critical section, to avoid that the last reference
>>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api
2016-11-03 19:46 ` Michael S. Tsirkin
@ 2016-11-04 6:36 ` Jason Wang
0 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-04 6:36 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck,
Stefan Hajnoczi, Kevin Wolf, Amit Shah, qemu-block
On 2016年11月04日 03:46, Michael S. Tsirkin wrote:
>> @@ -244,6 +245,7 @@ int virtio_queue_empty(VirtQueue *vq)
>> > static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem,
>> > unsigned int len)
>> > {
>> >+ AddressSpace *dma_as = virtio_get_dma_as(vq->vdev);
>> > unsigned int offset;
>> > int i;
> Can't we use
>
> vdev->dma_as
>
> here?
>
> I'd like to avoid query on data path.
>
> Same in most other places below.
>
>
Will use vdev->dma_as in next version.
Thanks
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-03 19:49 ` Michael S. Tsirkin
@ 2016-11-04 6:48 ` Jason Wang
2016-11-10 17:32 ` Michael S. Tsirkin
0 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-04 6:48 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On 2016年11月04日 03:49, Michael S. Tsirkin wrote:
> On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
>> >This patches enable the Address Translation Service support for virtio
>> >pci devices. This is needed for a guest visible Device IOTLB
>> >implementation and will be required by vhost device IOTLB API
>> >implementation for intel IOMMU.
>> >
>> >Cc: Michael S. Tsirkin<mst@redhat.com>
>> >Signed-off-by: Jason Wang<jasowang@redhat.com>
> I'd like to understand why do you think this is strictly required.
> Won't setting CM bit in the IOMMU do the trick.
ATS was chosen for performance. Since there're many problems for CM:
- CM was slow (10%-20% slower on real hardware for things like netperf)
because of each transition between non-present and present mapping needs
an explicit invalidation. It may slow down the whole VM.
- Without ATS/Device IOTLB, IOMMU becomes a bottleneck because of
contending of IOTLB entries. (What we can do in this case is in fact
userspace IOTLB snooping, this could be done even without CM).
It was natural to think of ATS when designing interface between IOMMU
and device/remote IOTLBs. Do you see any drawbacks on ATS here?
Thanks
>
> Also, could you remind me pls - can guests just disable ATS?
>
> What happens then?
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-04 6:48 ` Jason Wang
@ 2016-11-10 17:32 ` Michael S. Tsirkin
2016-11-11 3:26 ` Jason Wang
0 siblings, 1 reply; 25+ messages in thread
From: Michael S. Tsirkin @ 2016-11-10 17:32 UTC (permalink / raw)
To: Jason Wang; +Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On Fri, Nov 04, 2016 at 02:48:20PM +0800, Jason Wang wrote:
>
>
> On 2016年11月04日 03:49, Michael S. Tsirkin wrote:
> > On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
> > > >This patches enable the Address Translation Service support for virtio
> > > >pci devices. This is needed for a guest visible Device IOTLB
> > > >implementation and will be required by vhost device IOTLB API
> > > >implementation for intel IOMMU.
> > > >
> > > >Cc: Michael S. Tsirkin<mst@redhat.com>
> > > >Signed-off-by: Jason Wang<jasowang@redhat.com>
> > I'd like to understand why do you think this is strictly required.
> > Won't setting CM bit in the IOMMU do the trick.
>
> ATS was chosen for performance. Since there're many problems for CM:
>
> - CM was slow (10%-20% slower on real hardware for things like netperf)
> because of each transition between non-present and present mapping needs an
> explicit invalidation. It may slow down the whole VM.
> - Without ATS/Device IOTLB, IOMMU becomes a bottleneck because of contending
> of IOTLB entries. (What we can do in this case is in fact userspace IOTLB
> snooping, this could be done even without CM).
> It was natural to think of ATS when designing interface between IOMMU and
> device/remote IOTLBs. Do you see any drawbacks on ATS here?
>
> Thanks
In fact at this point I'm confused. Any mapping needs to be programmed
in the IOMMU. We need to implement this correctly.
Once we do why do we need ATS?
I think what you need is map/unmap notifiers that Aviv is working on.
No?
> >
> > Also, could you remind me pls - can guests just disable ATS?
> >
> > What happens then?
> >
> >
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-10 17:32 ` Michael S. Tsirkin
@ 2016-11-11 3:26 ` Jason Wang
2016-11-11 3:49 ` Michael S. Tsirkin
0 siblings, 1 reply; 25+ messages in thread
From: Jason Wang @ 2016-11-11 3:26 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On 2016年11月11日 01:32, Michael S. Tsirkin wrote:
> On Fri, Nov 04, 2016 at 02:48:20PM +0800, Jason Wang wrote:
>>
>> On 2016年11月04日 03:49, Michael S. Tsirkin wrote:
>>> On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
>>>>> This patches enable the Address Translation Service support for virtio
>>>>> pci devices. This is needed for a guest visible Device IOTLB
>>>>> implementation and will be required by vhost device IOTLB API
>>>>> implementation for intel IOMMU.
>>>>>
>>>>> Cc: Michael S. Tsirkin<mst@redhat.com>
>>>>> Signed-off-by: Jason Wang<jasowang@redhat.com>
>>> I'd like to understand why do you think this is strictly required.
>>> Won't setting CM bit in the IOMMU do the trick.
>> ATS was chosen for performance. Since there're many problems for CM:
>>
>> - CM was slow (10%-20% slower on real hardware for things like netperf)
>> because of each transition between non-present and present mapping needs an
>> explicit invalidation. It may slow down the whole VM.
>> - Without ATS/Device IOTLB, IOMMU becomes a bottleneck because of contending
>> of IOTLB entries. (What we can do in this case is in fact userspace IOTLB
>> snooping, this could be done even without CM).
>> It was natural to think of ATS when designing interface between IOMMU and
>> device/remote IOTLBs. Do you see any drawbacks on ATS here?
>>
>> Thanks
> In fact at this point I'm confused. Any mapping needs to be programmed
> in the IOMMU. We need to implement this correctly.
> Once we do why do we need ATS?
> I think what you need is map/unmap notifiers that Aviv is working on.
> No?
Let me clarify, device IOTLB API can work without ATS or CM. So there're
three ways to do:
1) without ATS or CM support, the function could be implemented through:
1.1: asking for qemu help if there's an IOTLB miss in vhost
1.2: snooping the userspace IOTLB invalidation (present to non-present
mapping) and update device IOTLB
2) with CM enabled, the only thing we can add is snooping the
non-present to present mapping and update the device IOTLB. This is not
a requirement since we still can get this through asking qemu's(1.2) help.
3) with ATS enabled, guest knows the existence of device IOTLB, and
device IOTLB entires needs to be flushed explicitly by guest. In this
case there's no need to snoop the ordinary IOTLB invalidation in 1.2. We
just need to snoop the device IOTLB specific invalidation request from
guest.
All the above 3 methods work very well, but let's have a look at
performance impact:
- Method 1 (without CM or ATS), the performance is not the best since
guest does not know about the existence of remote IOTLB, this means the
flush of device IOTLB entry could not be done on demand. One example is
some IOMMU driver (e.g intel) tends to optimize the IOTLB invalidations
by issuing a global invalidation periodically. We need to flush the
device IOTLB too in this case. Thus we can notice some jitter (because
of IOTLB miss).
- Method 2 (with CM but without ATS) seems to be the worst case. It has
not only all problems above a but also a new one: each transition needs
to notify the device explicitly. Even if dpdk use static mappings, all
other devices in the VM use dynamic ones which slows down the whole the
system. According to the test, CM is about 10%-20% slower in real hardware.
- Method 3 (ATS) can give the best performance, all the problems have
gone since guest can flush the device IOTLB entry on demand. It was
defined by spec and was designed to solve the issues just like what we
meet here, and was supported by modern IOMMUs.
And what's even better, implementing ATS turns out less than 100 lines
of codes. And it was much more easier to be enabled on other IOMMU (AMD
IOMMU only needs 20 lines of codes). All other ways (I started and have
codes for method 1 for intel IOMMU) need lots of work specific to each
kind of IOMMU.
Consider so much advantages by just adding so small lines of codes. I
don't see why we don't need ATS (for the IOOMUs that supports it).
Thanks
>
>
>>> Also, could you remind me pls - can guests just disable ATS?
>>>
>>> What happens then?
>>>
>>>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-11 3:26 ` Jason Wang
@ 2016-11-11 3:49 ` Michael S. Tsirkin
2016-11-11 4:24 ` Jason Wang
0 siblings, 1 reply; 25+ messages in thread
From: Michael S. Tsirkin @ 2016-11-11 3:49 UTC (permalink / raw)
To: Jason Wang; +Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On Fri, Nov 11, 2016 at 11:26:12AM +0800, Jason Wang wrote:
>
>
> On 2016年11月11日 01:32, Michael S. Tsirkin wrote:
> > On Fri, Nov 04, 2016 at 02:48:20PM +0800, Jason Wang wrote:
> > >
> > > On 2016年11月04日 03:49, Michael S. Tsirkin wrote:
> > > > On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
> > > > > > This patches enable the Address Translation Service support for virtio
> > > > > > pci devices. This is needed for a guest visible Device IOTLB
> > > > > > implementation and will be required by vhost device IOTLB API
> > > > > > implementation for intel IOMMU.
> > > > > >
> > > > > > Cc: Michael S. Tsirkin<mst@redhat.com>
> > > > > > Signed-off-by: Jason Wang<jasowang@redhat.com>
> > > > I'd like to understand why do you think this is strictly required.
> > > > Won't setting CM bit in the IOMMU do the trick.
> > > ATS was chosen for performance. Since there're many problems for CM:
> > >
> > > - CM was slow (10%-20% slower on real hardware for things like netperf)
> > > because of each transition between non-present and present mapping needs an
> > > explicit invalidation. It may slow down the whole VM.
> > > - Without ATS/Device IOTLB, IOMMU becomes a bottleneck because of contending
> > > of IOTLB entries. (What we can do in this case is in fact userspace IOTLB
> > > snooping, this could be done even without CM).
> > > It was natural to think of ATS when designing interface between IOMMU and
> > > device/remote IOTLBs. Do you see any drawbacks on ATS here?
> > >
> > > Thanks
> > In fact at this point I'm confused. Any mapping needs to be programmed
> > in the IOMMU. We need to implement this correctly.
> > Once we do why do we need ATS?
> > I think what you need is map/unmap notifiers that Aviv is working on.
> > No?
>
> Let me clarify, device IOTLB API can work without ATS or CM. So there're
> three ways to do:
>
> 1) without ATS or CM support, the function could be implemented through:
> 1.1: asking for qemu help if there's an IOTLB miss in vhost
> 1.2: snooping the userspace IOTLB invalidation (present to non-present
> mapping) and update device IOTLB
>
> 2) with CM enabled, the only thing we can add is snooping the non-present to
> present mapping and update the device IOTLB. This is not a requirement since
> we still can get this through asking qemu's(1.2) help.
>
> 3) with ATS enabled, guest knows the existence of device IOTLB, and device
> IOTLB entires needs to be flushed explicitly by guest. In this case there's
> no need to snoop the ordinary IOTLB invalidation in 1.2. We just need to
> snoop the device IOTLB specific invalidation request from guest.
>
> All the above 3 methods work very well, but let's have a look at performance
> impact:
>
> - Method 1 (without CM or ATS), the performance is not the best since guest
> does not know about the existence of remote IOTLB, this means the flush of
> device IOTLB entry could not be done on demand. One example is some IOMMU
> driver (e.g intel) tends to optimize the IOTLB invalidations by issuing a
> global invalidation periodically. We need to flush the device IOTLB too in
> this case. Thus we can notice some jitter (because of IOTLB miss).
>
> - Method 2 (with CM but without ATS) seems to be the worst case. It has not
> only all problems above a but also a new one: each transition needs to
> notify the device explicitly. Even if dpdk use static mappings, all other
> devices in the VM use dynamic ones which slows down the whole the system.
> According to the test, CM is about 10%-20% slower in real hardware.
>
> - Method 3 (ATS) can give the best performance, all the problems have gone
> since guest can flush the device IOTLB entry on demand. It was defined by
> spec and was designed to solve the issues just like what we meet here, and
> was supported by modern IOMMUs.
>
> And what's even better, implementing ATS turns out less than 100 lines of
> codes. And it was much more easier to be enabled on other IOMMU (AMD IOMMU
> only needs 20 lines of codes). All other ways (I started and have codes for
> method 1 for intel IOMMU) need lots of work specific to each kind of IOMMU.
method 1 is basically what Aviv implemented except you don't
need map notifiers, only unmap.
>
> Consider so much advantages by just adding so small lines of codes. I don't
> see why we don't need ATS (for the IOOMUs that supports it).
>
> Thanks
I am concerned that not all IOMMUs and guests support ATS.
> >
> >
> > > > Also, could you remind me pls - can guests just disable ATS?
> > > >
> > > > What happens then?
> > > >
> > > >
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support
2016-11-11 3:49 ` Michael S. Tsirkin
@ 2016-11-11 4:24 ` Jason Wang
0 siblings, 0 replies; 25+ messages in thread
From: Jason Wang @ 2016-11-11 4:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: peterx, wexu, qemu-devel, vkaplans, pbonzini, cornelia.huck
On 2016年11月11日 11:49, Michael S. Tsirkin wrote:
> On Fri, Nov 11, 2016 at 11:26:12AM +0800, Jason Wang wrote:
>> >
>> >
>> >On 2016年11月11日 01:32, Michael S. Tsirkin wrote:
>>> > >On Fri, Nov 04, 2016 at 02:48:20PM +0800, Jason Wang wrote:
>>>> > > >
>>>> > > >On 2016年11月04日 03:49, Michael S. Tsirkin wrote:
>>>>> > > > >On Thu, Nov 03, 2016 at 05:27:19PM +0800, Jason Wang wrote:
>>>>>>> > > > > > >This patches enable the Address Translation Service support for virtio
>>>>>>> > > > > > >pci devices. This is needed for a guest visible Device IOTLB
>>>>>>> > > > > > >implementation and will be required by vhost device IOTLB API
>>>>>>> > > > > > >implementation for intel IOMMU.
>>>>>>> > > > > > >
>>>>>>> > > > > > >Cc: Michael S. Tsirkin<mst@redhat.com>
>>>>>>> > > > > > >Signed-off-by: Jason Wang<jasowang@redhat.com>
>>>>> > > > >I'd like to understand why do you think this is strictly required.
>>>>> > > > >Won't setting CM bit in the IOMMU do the trick.
>>>> > > >ATS was chosen for performance. Since there're many problems for CM:
>>>> > > >
>>>> > > >- CM was slow (10%-20% slower on real hardware for things like netperf)
>>>> > > >because of each transition between non-present and present mapping needs an
>>>> > > >explicit invalidation. It may slow down the whole VM.
>>>> > > >- Without ATS/Device IOTLB, IOMMU becomes a bottleneck because of contending
>>>> > > >of IOTLB entries. (What we can do in this case is in fact userspace IOTLB
>>>> > > >snooping, this could be done even without CM).
>>>> > > >It was natural to think of ATS when designing interface between IOMMU and
>>>> > > >device/remote IOTLBs. Do you see any drawbacks on ATS here?
>>>> > > >
>>>> > > >Thanks
>>> > >In fact at this point I'm confused. Any mapping needs to be programmed
>>> > >in the IOMMU. We need to implement this correctly.
>>> > >Once we do why do we need ATS?
>>> > >I think what you need is map/unmap notifiers that Aviv is working on.
>>> > >No?
>> >
>> >Let me clarify, device IOTLB API can work without ATS or CM. So there're
>> >three ways to do:
>> >
>> >1) without ATS or CM support, the function could be implemented through:
>> >1.1: asking for qemu help if there's an IOTLB miss in vhost
>> >1.2: snooping the userspace IOTLB invalidation (present to non-present
>> >mapping) and update device IOTLB
>> >
>> >2) with CM enabled, the only thing we can add is snooping the non-present to
>> >present mapping and update the device IOTLB. This is not a requirement since
>> >we still can get this through asking qemu's(1.2) help.
>> >
>> >3) with ATS enabled, guest knows the existence of device IOTLB, and device
>> >IOTLB entires needs to be flushed explicitly by guest. In this case there's
>> >no need to snoop the ordinary IOTLB invalidation in 1.2. We just need to
>> >snoop the device IOTLB specific invalidation request from guest.
>> >
>> >All the above 3 methods work very well, but let's have a look at performance
>> >impact:
>> >
>> >- Method 1 (without CM or ATS), the performance is not the best since guest
>> >does not know about the existence of remote IOTLB, this means the flush of
>> >device IOTLB entry could not be done on demand. One example is some IOMMU
>> >driver (e.g intel) tends to optimize the IOTLB invalidations by issuing a
>> >global invalidation periodically. We need to flush the device IOTLB too in
>> >this case. Thus we can notice some jitter (because of IOTLB miss).
>> >
>> >- Method 2 (with CM but without ATS) seems to be the worst case. It has not
>> >only all problems above a but also a new one: each transition needs to
>> >notify the device explicitly. Even if dpdk use static mappings, all other
>> >devices in the VM use dynamic ones which slows down the whole the system.
>> >According to the test, CM is about 10%-20% slower in real hardware.
>> >
>> >- Method 3 (ATS) can give the best performance, all the problems have gone
>> >since guest can flush the device IOTLB entry on demand. It was defined by
>> >spec and was designed to solve the issues just like what we meet here, and
>> >was supported by modern IOMMUs.
>> >
>> >And what's even better, implementing ATS turns out less than 100 lines of
>> >codes. And it was much more easier to be enabled on other IOMMU (AMD IOMMU
>> >only needs 20 lines of codes). All other ways (I started and have codes for
>> >method 1 for intel IOMMU) need lots of work specific to each kind of IOMMU.
> method 1 is basically what Aviv implemented except you don't
> need map notifiers, only unmap.
>
>> >
>> >Consider so much advantages by just adding so small lines of codes. I don't
>> >see why we don't need ATS (for the IOOMUs that supports it).
>> >
>> >Thanks
> I am concerned that not all IOMMUs and guests support ATS.
>
For IOMMUs that does not support ATS, we can used method 1.
For legacy guests, it does not even support VIRTIO_F_IOMMU_PLATFORM. So
probably not an issue.
Thanks
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2016-11-11 4:24 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-03 9:27 [Qemu-devel] [PATCH V2 00/11] vhost device IOTLB support Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 01/11] intel_iommu: fixing source id during IOTLB hash key calculation Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 02/11] virtio: convert to use DMA api Jason Wang
2016-11-03 19:46 ` Michael S. Tsirkin
2016-11-04 6:36 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 03/11] intel_iommu: name vtd address space with devfn Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 04/11] intel_iommu: allocate new key when creating new address space Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 05/11] exec: introduce address_space_get_iotlb_entry() Jason Wang
2016-11-03 17:03 ` Paolo Bonzini
2016-11-04 6:33 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 06/11] intel_iommu: support device iotlb descriptor Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 07/11] virtio-pci: address space translation service (ATS) support Jason Wang
2016-11-03 19:49 ` Michael S. Tsirkin
2016-11-04 6:48 ` Jason Wang
2016-11-10 17:32 ` Michael S. Tsirkin
2016-11-11 3:26 ` Jason Wang
2016-11-11 3:49 ` Michael S. Tsirkin
2016-11-11 4:24 ` Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 08/11] acpi: add ATSR for q35 Jason Wang
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 09/11] memory: handle alias for iommu notifier Jason Wang
2016-11-03 16:55 ` Paolo Bonzini
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 10/11] memory: handle alias in memory_region_is_iommu() Jason Wang
2016-11-03 16:56 ` Paolo Bonzini
2016-11-03 9:27 ` [Qemu-devel] [PATCH V2 11/11] vhost_net: device IOTLB support Jason Wang
2016-11-03 9:48 ` [Qemu-devel] [PATCH V2 00/11] vhost " no-reply
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.