* [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device
@ 2018-11-22 17:15 Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 01/17] update-linux-headers: Import virtio_iommu.h Eric Auger
` (17 more replies)
0 siblings, 18 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This series rebases the virtio-iommu device on qemu 3.1.0-rc2
and implements the v0.8(.1) virtio-iommu spec [1]. The pci proxy
for the virtio-iommu device is now available and needs to be
instantiated from the command line using "-device virtio-iommu-pci".
The iommu machvirt option is not used anymore to instantiate the
virtio-iommu.
At the moment the virtio-iommu-device only is functional in the
ARM virt machine. Indeed, besides its instantiation, links between
the PCIe end points and the IOMMU must be described. This is achieved
by DT or ACPI description (IORT). This description currently only is
done in ARM virt.
Best Regards
Eric
This series can be found at:
https://github.com/eauger/qemu/tree/v3.1.0-rc2-virtio-iommu-v0.9
References:
[1] [PATCH v3 0/7] Add virtio-iommu driver
[2] guest branch featuring the virtio-iommu driver v0.8.1 + ACPI
integration not yet officially released by Jean.
https://github.com/eauger/linux/tree/virtio-iommu-v0.8.1
Testing:
- tested with guest using virtio-net-pci
(,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
and virtio-blk-pci
- VFIO/VHOST integration is not part of this series
- When using the virtio-blk-pci, some EDK2 FW versions feature
unmapped transactions and in that case the guest fails to boot.
History:
v8 -> v9:
- virtio-iommu-pci device needs to be instantiated from the command
line (RID is not imposed anymore).
- tail structure properly initialized
v7 -> v8:
- virtio-iommu-pci added
- virt instantiation modified
- DT and ACPI modified to exclude the iommu RID from the mapping
- VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
v6 -> v7:
- rebase on qemu 3.0.0-rc3
- minor update against v0.7
- fix issue with EP not on pci.0 and ACPI probing
- change the instantiation method
v5 -> v6:
- minor update against v0.6 spec
- fix g_hash_table_lookup in virtio_iommu_find_add_as
- replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
v4 -> v5:
- event queue and fault reporting
- we now return the IOAPIC MSI region if the virtio-iommu is instantiated
in a PC machine.
- we bypass transactions on MSI HW region and fault on reserved ones.
- We support ACPI boot with mach-virt (based on IORT proposal)
- We moved to the new driver naming conventions
- simplified mach-virt instantiation
- worked around the disappearing of pci_find_primary_bus
- in virtio_iommu_translate, check the dev->as is not NULL
- initialize as->device_list in virtio_iommu_get_as
- initialize bufstate.error to false in virtio_iommu_probe
v3 -> v4:
- probe request support although no reserved region is returned at
the moment
- unmap semantics less strict, as specified in v0.4
- device registration, attach/detach revisited
- split into smaller patches to ease review
- propose a way to inform the IOMMU mr about the page_size_mask
of underlying HW IOMMU, if any
- remove warning associated with the translation of the MSI doorbell
v2 -> v3:
- rebase on top of 2.10-rc0 and especially
[PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
- add mutex init
- fix as->mappings deletion using g_tree_ref/unref
- when a dev is attached whereas it is already attached to
another address space, first detach it
- fix some error values
- page_sizes = TARGET_PAGE_MASK;
- I haven't changed the unmap() semantics yet, waiting for the
next virtio-iommu spec revision.
v1 -> v2:
- fix redifinition of viommu_as typedef
Eric Auger (17):
update-linux-headers: Import virtio_iommu.h
linux-headers: Partial update for virtio-iommu v0.8
virtio-iommu: Add skeleton
virtio-iommu: Decode the command payload
virtio-iommu: Add the iommu regions
virtio-iommu: Endpoint and domains structs and helpers
virtio-iommu: Implement attach/detach command
virtio-iommu: Implement map/unmap
virtio-iommu: Implement translate
virtio-iommu: Implement probe request
virtio-iommu: Expose the IOAPIC MSI reserved region when relevant
virtio-iommu: Implement fault reporting
virtio_iommu: Handle reserved regions in translation process
virtio-iommu-pci: Add virtio iommu pci support
hw/arm/virt: Add the virtio-iommu device tree mappings
hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
hw/arm/virt-acpi-build.c | 91 +-
hw/arm/virt.c | 57 +-
hw/virtio/Makefile.objs | 1 +
hw/virtio/trace-events | 26 +
hw/virtio/virtio-iommu.c | 1040 +++++++++++++++++
hw/virtio/virtio-pci.c | 51 +
hw/virtio/virtio-pci.h | 14 +
include/hw/acpi/acpi-defs.h | 21 +-
include/hw/arm/virt.h | 2 +
include/hw/pci/pci.h | 1 +
include/hw/virtio/virtio-iommu.h | 65 ++
include/standard-headers/linux/virtio_ids.h | 1 +
include/standard-headers/linux/virtio_iommu.h | 159 +++
linux-headers/linux/virtio_iommu.h | 1 +
qdev-monitor.c | 1 +
scripts/update-linux-headers.sh | 3 +
16 files changed, 1501 insertions(+), 33 deletions(-)
create mode 100644 hw/virtio/virtio-iommu.c
create mode 100644 include/hw/virtio/virtio-iommu.h
create mode 100644 include/standard-headers/linux/virtio_iommu.h
create mode 100644 linux-headers/linux/virtio_iommu.h
--
2.17.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 01/17] update-linux-headers: Import virtio_iommu.h
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 02/17] linux-headers: Partial update for virtio-iommu v0.8 Eric Auger
` (16 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
Update the script to update the virtio_iommu.h header.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
| 3 +++
1 file changed, 3 insertions(+)
--git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh
index 0a964fe240..55fd271a32 100755
--- a/scripts/update-linux-headers.sh
+++ b/scripts/update-linux-headers.sh
@@ -159,6 +159,9 @@ fi
cat <<EOF >$output/linux-headers/linux/virtio_config.h
#include "standard-headers/linux/virtio_config.h"
EOF
+cat <<EOF >$output/linux-headers/linux/virtio_iommu.h
+#include "standard-headers/linux/virtio_iommu.h"
+EOF
cat <<EOF >$output/linux-headers/linux/virtio_ring.h
#include "standard-headers/linux/virtio_ring.h"
EOF
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 02/17] linux-headers: Partial update for virtio-iommu v0.8
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 01/17] update-linux-headers: Import virtio_iommu.h Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 03/17] virtio-iommu: Add skeleton Eric Auger
` (15 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
Partial sync against Jean-Philippe's branch:
git://linux-arm.org/linux-jpb.git virtio-iommu/v0.8
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
include/standard-headers/linux/virtio_ids.h | 1 +
include/standard-headers/linux/virtio_iommu.h | 159 ++++++++++++++++++
| 1 +
3 files changed, 161 insertions(+)
create mode 100644 include/standard-headers/linux/virtio_iommu.h
create mode 100644 linux-headers/linux/virtio_iommu.h
diff --git a/include/standard-headers/linux/virtio_ids.h b/include/standard-headers/linux/virtio_ids.h
index 6d5c3b2d4f..cfe47c5d9a 100644
--- a/include/standard-headers/linux/virtio_ids.h
+++ b/include/standard-headers/linux/virtio_ids.h
@@ -43,5 +43,6 @@
#define VIRTIO_ID_INPUT 18 /* virtio input */
#define VIRTIO_ID_VSOCK 19 /* virtio vsock transport */
#define VIRTIO_ID_CRYPTO 20 /* virtio crypto */
+#define VIRTIO_ID_IOMMU 23 /* virtio IOMMU */
#endif /* _LINUX_VIRTIO_IDS_H */
diff --git a/include/standard-headers/linux/virtio_iommu.h b/include/standard-headers/linux/virtio_iommu.h
new file mode 100644
index 0000000000..0a40b21ea9
--- /dev/null
+++ b/include/standard-headers/linux/virtio_iommu.h
@@ -0,0 +1,159 @@
+/* SPDX-License-Identifier: BSD-3-Clause */
+/*
+ * Virtio-iommu definition v0.8
+ *
+ * Copyright (C) 2018 Arm Ltd.
+ */
+#ifndef _LINUX_VIRTIO_IOMMU_H
+#define _LINUX_VIRTIO_IOMMU_H
+
+#include "standard-headers/linux/types.h"
+
+/* Feature bits */
+#define VIRTIO_IOMMU_F_INPUT_RANGE 0
+#define VIRTIO_IOMMU_F_DOMAIN_BITS 1
+#define VIRTIO_IOMMU_F_MAP_UNMAP 2
+#define VIRTIO_IOMMU_F_BYPASS 3
+#define VIRTIO_IOMMU_F_PROBE 4
+
+struct virtio_iommu_config {
+ /* Supported page sizes */
+ uint64_t page_size_mask;
+ /* Supported IOVA range */
+ struct virtio_iommu_range {
+ uint64_t start;
+ uint64_t end;
+ } input_range;
+ /* Max domain ID size */
+ uint8_t domain_bits;
+ uint8_t padding[3];
+ /* Probe buffer size */
+ uint32_t probe_size;
+};
+
+/* Request types */
+#define VIRTIO_IOMMU_T_ATTACH 0x01
+#define VIRTIO_IOMMU_T_DETACH 0x02
+#define VIRTIO_IOMMU_T_MAP 0x03
+#define VIRTIO_IOMMU_T_UNMAP 0x04
+#define VIRTIO_IOMMU_T_PROBE 0x05
+
+/* Status types */
+#define VIRTIO_IOMMU_S_OK 0x00
+#define VIRTIO_IOMMU_S_IOERR 0x01
+#define VIRTIO_IOMMU_S_UNSUPP 0x02
+#define VIRTIO_IOMMU_S_DEVERR 0x03
+#define VIRTIO_IOMMU_S_INVAL 0x04
+#define VIRTIO_IOMMU_S_RANGE 0x05
+#define VIRTIO_IOMMU_S_NOENT 0x06
+#define VIRTIO_IOMMU_S_FAULT 0x07
+
+struct virtio_iommu_req_head {
+ uint8_t type;
+ uint8_t reserved[3];
+};
+
+struct virtio_iommu_req_tail {
+ uint8_t status;
+ uint8_t reserved[3];
+};
+
+struct virtio_iommu_req_attach {
+ struct virtio_iommu_req_head head;
+ uint32_t domain;
+ uint32_t endpoint;
+ uint8_t reserved[8];
+ struct virtio_iommu_req_tail tail;
+};
+
+struct virtio_iommu_req_detach {
+ struct virtio_iommu_req_head head;
+ uint32_t domain;
+ uint32_t endpoint;
+ uint8_t reserved[8];
+ struct virtio_iommu_req_tail tail;
+};
+
+#define VIRTIO_IOMMU_MAP_F_READ (1 << 0)
+#define VIRTIO_IOMMU_MAP_F_WRITE (1 << 1)
+#define VIRTIO_IOMMU_MAP_F_EXEC (1 << 2)
+#define VIRTIO_IOMMU_MAP_F_MMIO (1 << 3)
+
+#define VIRTIO_IOMMU_MAP_F_MASK (VIRTIO_IOMMU_MAP_F_READ | \
+ VIRTIO_IOMMU_MAP_F_WRITE | \
+ VIRTIO_IOMMU_MAP_F_EXEC | \
+ VIRTIO_IOMMU_MAP_F_MMIO)
+
+struct virtio_iommu_req_map {
+ struct virtio_iommu_req_head head;
+ uint32_t domain;
+ uint64_t virt_start;
+ uint64_t virt_end;
+ uint64_t phys_start;
+ uint32_t flags;
+ struct virtio_iommu_req_tail tail;
+};
+
+struct virtio_iommu_req_unmap {
+ struct virtio_iommu_req_head head;
+ uint32_t domain;
+ uint64_t virt_start;
+ uint64_t virt_end;
+ uint8_t reserved[4];
+ struct virtio_iommu_req_tail tail;
+};
+
+#define VIRTIO_IOMMU_PROBE_T_NONE 0
+#define VIRTIO_IOMMU_PROBE_T_RESV_MEM 1
+
+#define VIRTIO_IOMMU_PROBE_T_MASK 0xfff
+
+struct virtio_iommu_probe_property {
+ uint16_t type;
+ uint16_t length;
+};
+
+#define VIRTIO_IOMMU_RESV_MEM_T_RESERVED 0
+#define VIRTIO_IOMMU_RESV_MEM_T_MSI 1
+
+struct virtio_iommu_probe_resv_mem {
+ struct virtio_iommu_probe_property head;
+ uint8_t subtype;
+ uint8_t reserved[3];
+ uint64_t start;
+ uint64_t end;
+};
+
+struct virtio_iommu_req_probe {
+ struct virtio_iommu_req_head head;
+ uint32_t endpoint;
+ uint8_t reserved[64];
+
+ uint8_t properties[];
+
+ /*
+ * Tail follows the variable-length properties array. No padding,
+ * property lengths are all aligned on 8 bytes.
+ */
+};
+
+/* Fault types */
+#define VIRTIO_IOMMU_FAULT_R_UNKNOWN 0
+#define VIRTIO_IOMMU_FAULT_R_DOMAIN 1
+#define VIRTIO_IOMMU_FAULT_R_MAPPING 2
+
+#define VIRTIO_IOMMU_FAULT_F_READ (1 << 0)
+#define VIRTIO_IOMMU_FAULT_F_WRITE (1 << 1)
+#define VIRTIO_IOMMU_FAULT_F_EXEC (1 << 2)
+#define VIRTIO_IOMMU_FAULT_F_ADDRESS (1 << 8)
+
+struct virtio_iommu_fault {
+ uint8_t reason;
+ uint8_t reserved[3];
+ uint32_t flags;
+ uint32_t endpoint;
+ uint8_t reserved2[4];
+ uint64_t address;
+};
+
+#endif
--git a/linux-headers/linux/virtio_iommu.h b/linux-headers/linux/virtio_iommu.h
new file mode 100644
index 0000000000..2dc4609c16
--- /dev/null
+++ b/linux-headers/linux/virtio_iommu.h
@@ -0,0 +1 @@
+#include "standard-headers/linux/virtio_iommu.h"
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 03/17] virtio-iommu: Add skeleton
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 01/17] update-linux-headers: Import virtio_iommu.h Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 02/17] linux-headers: Partial update for virtio-iommu v0.8 Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 04/17] virtio-iommu: Decode the command payload Eric Auger
` (14 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patchs adds the skeleton for the virtio-iommu device.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- properly initialize tail
v7 -> v8:
- expose VIRTIO_IOMMU_F_BYPASS and VIRTIO_F_VERSION_1
features
- set_config dummy implementation + tracing
- add trace in get_features
- set the features on realize() and store the acked ones
- remove inclusion of linux/virtio_iommu.h
v6 -> v7:
- removed qapi-event.h include
- add primary_bus and associated property
v4 -> v5:
- use the new v0.5 terminology (domain, endpoint)
- add the event virtqueue
v3 -> v4:
- use page_size_mask instead of page_sizes
- added set_features()
- added some traces (reset, set_status, set_features)
- empty virtio_iommu_set_config() as the driver MUST NOT
write to device configuration fields
- add get_config trace
v2 -> v3:
- rebase on 2.10-rc0, ie. use IOMMUMemoryRegion and remove
iommu_ops.
- advertise VIRTIO_IOMMU_F_MAP_UNMAP feature
- page_sizes set to TARGET_PAGE_SIZE
Conflicts:
hw/virtio/trace-events
---
hw/virtio/Makefile.objs | 1 +
hw/virtio/trace-events | 9 +
hw/virtio/virtio-iommu.c | 273 +++++++++++++++++++++++++++++++
include/hw/virtio/virtio-iommu.h | 62 +++++++
4 files changed, 345 insertions(+)
create mode 100644 hw/virtio/virtio-iommu.c
create mode 100644 include/hw/virtio/virtio-iommu.h
diff --git a/hw/virtio/Makefile.objs b/hw/virtio/Makefile.objs
index 1b2799cfd8..49eba67a54 100644
--- a/hw/virtio/Makefile.objs
+++ b/hw/virtio/Makefile.objs
@@ -10,6 +10,7 @@ obj-$(CONFIG_VIRTIO_CRYPTO) += virtio-crypto.o
obj-$(call land,$(CONFIG_VIRTIO_CRYPTO),$(CONFIG_VIRTIO_PCI)) += virtio-crypto-pci.o
obj-$(CONFIG_LINUX) += vhost.o vhost-backend.o vhost-user.o
+obj-$(CONFIG_LINUX) += virtio-iommu.o
obj-$(CONFIG_VHOST_VSOCK) += vhost-vsock.o
endif
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 07bcbe9e85..84da904d8b 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -46,3 +46,12 @@ virtio_balloon_handle_output(const char *name, uint64_t gpa) "section name: %s g
virtio_balloon_get_config(uint32_t num_pages, uint32_t actual) "num_pages: %d actual: %d"
virtio_balloon_set_config(uint32_t actual, uint32_t oldactual) "actual: %d oldactual: %d"
virtio_balloon_to_target(uint64_t target, uint32_t num_pages) "balloon target: 0x%"PRIx64" num_pages: %d"
+
+# hw/virtio/virtio-iommu.c
+#
+virtio_iommu_device_reset(void) "reset!"
+virtio_iommu_get_features(uint64_t features) "device supports features=0x%"PRIx64
+virtio_iommu_set_features(uint64_t features) "features accepted by the driver =0x%"PRIx64
+virtio_iommu_device_status(uint8_t status) "driver status = %d"
+virtio_iommu_get_config(uint64_t page_size_mask, uint64_t start, uint64_t end, uint8_t domain_bits, uint32_t probe_size) "page_size_mask=0x%"PRIx64" start=0x%"PRIx64" end=0x%"PRIx64" domain_bits=%d probe_size=0x%x"
+virtio_iommu_set_config(uint64_t page_size_mask, uint64_t start, uint64_t end, uint8_t domain_bits, uint32_t probe_size) "page_size_mask=0x%"PRIx64" start=0x%"PRIx64" end=0x%"PRIx64" domain_bits=%d probe_size=0x%x"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
new file mode 100644
index 0000000000..d8894f047b
--- /dev/null
+++ b/hw/virtio/virtio-iommu.c
@@ -0,0 +1,273 @@
+/*
+ * virtio-iommu device
+ *
+ * Copyright (c) 2017 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/iov.h"
+#include "qemu-common.h"
+#include "hw/virtio/virtio.h"
+#include "sysemu/kvm.h"
+#include "trace.h"
+
+#include "standard-headers/linux/virtio_ids.h"
+
+#include "hw/virtio/virtio-bus.h"
+#include "hw/virtio/virtio-access.h"
+#include "hw/virtio/virtio-iommu.h"
+
+/* Max size */
+#define VIOMMU_DEFAULT_QUEUE_SIZE 256
+
+static int virtio_iommu_handle_attach(VirtIOIOMMU *s,
+ struct iovec *iov,
+ unsigned int iov_cnt)
+{
+ return -ENOENT;
+}
+static int virtio_iommu_handle_detach(VirtIOIOMMU *s,
+ struct iovec *iov,
+ unsigned int iov_cnt)
+{
+ return -ENOENT;
+}
+static int virtio_iommu_handle_map(VirtIOIOMMU *s,
+ struct iovec *iov,
+ unsigned int iov_cnt)
+{
+ return -ENOENT;
+}
+static int virtio_iommu_handle_unmap(VirtIOIOMMU *s,
+ struct iovec *iov,
+ unsigned int iov_cnt)
+{
+ return -ENOENT;
+}
+
+static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
+{
+ VirtIOIOMMU *s = VIRTIO_IOMMU(vdev);
+ VirtQueueElement *elem;
+ struct virtio_iommu_req_head head;
+ struct virtio_iommu_req_tail tail = {};
+ unsigned int iov_cnt;
+ struct iovec *iov;
+ size_t sz;
+
+ for (;;) {
+ elem = virtqueue_pop(vq, sizeof(VirtQueueElement));
+ if (!elem) {
+ return;
+ }
+
+ if (iov_size(elem->in_sg, elem->in_num) < sizeof(tail) ||
+ iov_size(elem->out_sg, elem->out_num) < sizeof(head)) {
+ virtio_error(vdev, "virtio-iommu erroneous head or tail");
+ virtqueue_detach_element(vq, elem, 0);
+ g_free(elem);
+ break;
+ }
+
+ iov_cnt = elem->out_num;
+ iov = g_memdup(elem->out_sg, sizeof(struct iovec) * elem->out_num);
+ sz = iov_to_buf(iov, iov_cnt, 0, &head, sizeof(head));
+ if (sz != sizeof(head)) {
+ tail.status = VIRTIO_IOMMU_S_UNSUPP;
+ }
+ qemu_mutex_lock(&s->mutex);
+ switch (head.type) {
+ case VIRTIO_IOMMU_T_ATTACH:
+ tail.status = virtio_iommu_handle_attach(s, iov, iov_cnt);
+ break;
+ case VIRTIO_IOMMU_T_DETACH:
+ tail.status = virtio_iommu_handle_detach(s, iov, iov_cnt);
+ break;
+ case VIRTIO_IOMMU_T_MAP:
+ tail.status = virtio_iommu_handle_map(s, iov, iov_cnt);
+ break;
+ case VIRTIO_IOMMU_T_UNMAP:
+ tail.status = virtio_iommu_handle_unmap(s, iov, iov_cnt);
+ break;
+ default:
+ tail.status = VIRTIO_IOMMU_S_UNSUPP;
+ }
+ qemu_mutex_unlock(&s->mutex);
+
+ sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
+ &tail, sizeof(tail));
+ assert(sz == sizeof(tail));
+
+ virtqueue_push(vq, elem, sizeof(tail));
+ virtio_notify(vdev, vq);
+ g_free(elem);
+ }
+}
+
+static void virtio_iommu_get_config(VirtIODevice *vdev, uint8_t *config_data)
+{
+ VirtIOIOMMU *dev = VIRTIO_IOMMU(vdev);
+ struct virtio_iommu_config *config = &dev->config;
+
+ trace_virtio_iommu_get_config(config->page_size_mask,
+ config->input_range.start,
+ config->input_range.end,
+ config->domain_bits,
+ config->probe_size);
+ memcpy(config_data, &dev->config, sizeof(struct virtio_iommu_config));
+}
+
+static void virtio_iommu_set_config(VirtIODevice *vdev,
+ const uint8_t *config_data)
+{
+ struct virtio_iommu_config config;
+
+ memcpy(&config, config_data, sizeof(struct virtio_iommu_config));
+ trace_virtio_iommu_set_config(config.page_size_mask,
+ config.input_range.start,
+ config.input_range.end,
+ config.domain_bits,
+ config.probe_size);
+}
+
+static uint64_t virtio_iommu_get_features(VirtIODevice *vdev, uint64_t f,
+ Error **errp)
+{
+ VirtIOIOMMU *dev = VIRTIO_IOMMU(vdev);
+
+ f |= dev->features;
+ trace_virtio_iommu_get_features(f);
+ return f;
+}
+
+static void virtio_iommu_set_features(VirtIODevice *vdev, uint64_t val)
+{
+ VirtIOIOMMU *dev = VIRTIO_IOMMU(vdev);
+
+ dev->acked_features = val;
+ trace_virtio_iommu_set_features(dev->acked_features);
+}
+
+static int virtio_iommu_post_load_device(void *opaque, int version_id)
+{
+ return 0;
+}
+
+static const VMStateDescription vmstate_virtio_iommu_device = {
+ .name = "virtio-iommu-device",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .post_load = virtio_iommu_post_load_device,
+ .fields = (VMStateField[]) {
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
+{
+ VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+ VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
+
+ virtio_init(vdev, "virtio-iommu", VIRTIO_ID_IOMMU,
+ sizeof(struct virtio_iommu_config));
+
+ s->req_vq = virtio_add_queue(vdev, VIOMMU_DEFAULT_QUEUE_SIZE,
+ virtio_iommu_handle_command);
+ s->event_vq = virtio_add_queue(vdev, VIOMMU_DEFAULT_QUEUE_SIZE, NULL);
+
+ s->config.page_size_mask = TARGET_PAGE_MASK;
+ s->config.input_range.end = -1UL;
+ s->config.domain_bits = 32;
+
+ virtio_add_feature(&s->features, VIRTIO_RING_F_EVENT_IDX);
+ virtio_add_feature(&s->features, VIRTIO_RING_F_INDIRECT_DESC);
+ virtio_add_feature(&s->features, VIRTIO_F_VERSION_1);
+ virtio_add_feature(&s->features, VIRTIO_IOMMU_F_INPUT_RANGE);
+ virtio_add_feature(&s->features, VIRTIO_IOMMU_F_DOMAIN_BITS);
+ virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
+ virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
+}
+
+static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)
+{
+ VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+
+ virtio_cleanup(vdev);
+}
+
+static void virtio_iommu_device_reset(VirtIODevice *vdev)
+{
+ trace_virtio_iommu_device_reset();
+}
+
+static void virtio_iommu_set_status(VirtIODevice *vdev, uint8_t status)
+{
+ trace_virtio_iommu_device_status(status);
+}
+
+static void virtio_iommu_instance_init(Object *obj)
+{
+}
+
+static const VMStateDescription vmstate_virtio_iommu = {
+ .name = "virtio-iommu",
+ .minimum_version_id = 1,
+ .version_id = 1,
+ .fields = (VMStateField[]) {
+ VMSTATE_VIRTIO_DEVICE,
+ VMSTATE_END_OF_LIST()
+ },
+};
+
+static Property virtio_iommu_properties[] = {
+ DEFINE_PROP_LINK("primary-bus", VirtIOIOMMU, primary_bus, "PCI", PCIBus *),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void virtio_iommu_class_init(ObjectClass *klass, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass);
+
+ dc->props = virtio_iommu_properties;
+ dc->vmsd = &vmstate_virtio_iommu;
+
+ set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+ vdc->realize = virtio_iommu_device_realize;
+ vdc->unrealize = virtio_iommu_device_unrealize;
+ vdc->reset = virtio_iommu_device_reset;
+ vdc->get_config = virtio_iommu_get_config;
+ vdc->set_config = virtio_iommu_set_config;
+ vdc->get_features = virtio_iommu_get_features;
+ vdc->set_features = virtio_iommu_set_features;
+ vdc->set_status = virtio_iommu_set_status;
+ vdc->vmsd = &vmstate_virtio_iommu_device;
+}
+
+static const TypeInfo virtio_iommu_info = {
+ .name = TYPE_VIRTIO_IOMMU,
+ .parent = TYPE_VIRTIO_DEVICE,
+ .instance_size = sizeof(VirtIOIOMMU),
+ .instance_init = virtio_iommu_instance_init,
+ .class_init = virtio_iommu_class_init,
+};
+
+static void virtio_register_types(void)
+{
+ type_register_static(&virtio_iommu_info);
+}
+
+type_init(virtio_register_types)
diff --git a/include/hw/virtio/virtio-iommu.h b/include/hw/virtio/virtio-iommu.h
new file mode 100644
index 0000000000..4d47b6abeb
--- /dev/null
+++ b/include/hw/virtio/virtio-iommu.h
@@ -0,0 +1,62 @@
+/*
+ * virtio-iommu device
+ *
+ * Copyright (c) 2017 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2 or later, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef QEMU_VIRTIO_IOMMU_H
+#define QEMU_VIRTIO_IOMMU_H
+
+#include "standard-headers/linux/virtio_iommu.h"
+#include "hw/virtio/virtio.h"
+#include "hw/pci/pci.h"
+
+#define TYPE_VIRTIO_IOMMU "virtio-iommu-device"
+#define VIRTIO_IOMMU(obj) \
+ OBJECT_CHECK(VirtIOIOMMU, (obj), TYPE_VIRTIO_IOMMU)
+
+#define IOMMU_PCI_BUS_MAX 256
+#define IOMMU_PCI_DEVFN_MAX 256
+
+typedef struct IOMMUDevice {
+ void *viommu;
+ PCIBus *bus;
+ int devfn;
+ IOMMUMemoryRegion iommu_mr;
+ AddressSpace as;
+} IOMMUDevice;
+
+typedef struct IOMMUPciBus {
+ PCIBus *bus;
+ IOMMUDevice *pbdev[0]; /* Parent array is sparse, so dynamically alloc */
+} IOMMUPciBus;
+
+typedef struct VirtIOIOMMU {
+ VirtIODevice parent_obj;
+ VirtQueue *req_vq;
+ VirtQueue *event_vq;
+ struct virtio_iommu_config config;
+ uint64_t features;
+ uint64_t acked_features;
+ GHashTable *as_by_busptr;
+ IOMMUPciBus *as_by_bus_num[IOMMU_PCI_BUS_MAX];
+ PCIBus *primary_bus;
+ GTree *domains;
+ QemuMutex mutex;
+ GTree *endpoints;
+} VirtIOIOMMU;
+
+#endif
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 04/17] virtio-iommu: Decode the command payload
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (2 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 03/17] virtio-iommu: Add skeleton Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 05/17] virtio-iommu: Add the iommu regions Eric Auger
` (13 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch adds the command payload decoding and
introduces the functions that will do the actual
command handling. Those functions are not yet implemented.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v7 -> v8:
- handle new domain parameter in detach
- remove reserved checks
v5 -> v6:
- change map/unmap semantics (remove size)
v4 -> v5:
- adopt new v0.5 terminology
v3 -> v4:
- no flags field anymore in struct virtio_iommu_req_unmap
- test reserved on attach/detach, change trace proto
- rebase on v2.10.0.
---
hw/virtio/trace-events | 4 ++
hw/virtio/virtio-iommu.c | 95 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 95 insertions(+), 4 deletions(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 84da904d8b..e6177ca0e4 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -55,3 +55,7 @@ virtio_iommu_set_features(uint64_t features) "features accepted by the driver =0
virtio_iommu_device_status(uint8_t status) "driver status = %d"
virtio_iommu_get_config(uint64_t page_size_mask, uint64_t start, uint64_t end, uint8_t domain_bits, uint32_t probe_size) "page_size_mask=0x%"PRIx64" start=0x%"PRIx64" end=0x%"PRIx64" domain_bits=%d probe_size=0x%x"
virtio_iommu_set_config(uint64_t page_size_mask, uint64_t start, uint64_t end, uint8_t domain_bits, uint32_t probe_size) "page_size_mask=0x%"PRIx64" start=0x%"PRIx64" end=0x%"PRIx64" domain_bits=%d probe_size=0x%x"
+virtio_iommu_attach(uint32_t domain_id, uint32_t ep_id) "domain=%d endpoint=%d"
+virtio_iommu_detach(uint32_t domain_id, uint32_t ep_id) "domain=%d endpoint=%d"
+virtio_iommu_map(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end, uint64_t phys_start, uint32_t flags) "domain=%d virt_start=0x%"PRIx64" virt_end=0x%"PRIx64 " phys_start=0x%"PRIx64" flags=%d"
+virtio_iommu_unmap(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end) "domain=%d virt_start=0x%"PRIx64" virt_end=0x%"PRIx64
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index d8894f047b..fc95751c40 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -33,29 +33,116 @@
/* Max size */
#define VIOMMU_DEFAULT_QUEUE_SIZE 256
+static int virtio_iommu_attach(VirtIOIOMMU *s,
+ struct virtio_iommu_req_attach *req)
+{
+ uint32_t domain_id = le32_to_cpu(req->domain);
+ uint32_t ep_id = le32_to_cpu(req->endpoint);
+
+ trace_virtio_iommu_attach(domain_id, ep_id);
+
+ return VIRTIO_IOMMU_S_UNSUPP;
+}
+
+static int virtio_iommu_detach(VirtIOIOMMU *s,
+ struct virtio_iommu_req_detach *req)
+{
+ uint32_t domain_id = le32_to_cpu(req->domain);
+ uint32_t ep_id = le32_to_cpu(req->endpoint);
+
+ trace_virtio_iommu_detach(domain_id, ep_id);
+
+ return VIRTIO_IOMMU_S_UNSUPP;
+}
+
+static int virtio_iommu_map(VirtIOIOMMU *s,
+ struct virtio_iommu_req_map *req)
+{
+ uint32_t domain_id = le32_to_cpu(req->domain);
+ uint64_t phys_start = le64_to_cpu(req->phys_start);
+ uint64_t virt_start = le64_to_cpu(req->virt_start);
+ uint64_t virt_end = le64_to_cpu(req->virt_end);
+ uint32_t flags = le32_to_cpu(req->flags);
+
+ trace_virtio_iommu_map(domain_id, virt_start, virt_end, phys_start, flags);
+
+ return VIRTIO_IOMMU_S_UNSUPP;
+}
+
+static int virtio_iommu_unmap(VirtIOIOMMU *s,
+ struct virtio_iommu_req_unmap *req)
+{
+ uint32_t domain_id = le32_to_cpu(req->domain);
+ uint64_t virt_start = le64_to_cpu(req->virt_start);
+ uint64_t virt_end = le64_to_cpu(req->virt_end);
+
+ trace_virtio_iommu_unmap(domain_id, virt_start, virt_end);
+
+ return VIRTIO_IOMMU_S_UNSUPP;
+}
+
+#define get_payload_size(req) (\
+sizeof((req)) - sizeof(struct virtio_iommu_req_tail))
+
static int virtio_iommu_handle_attach(VirtIOIOMMU *s,
struct iovec *iov,
unsigned int iov_cnt)
{
- return -ENOENT;
+ struct virtio_iommu_req_attach req;
+ size_t sz, payload_sz;
+
+ payload_sz = get_payload_size(req);
+
+ sz = iov_to_buf(iov, iov_cnt, 0, &req, payload_sz);
+ if (sz != payload_sz) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+ return virtio_iommu_attach(s, &req);
}
static int virtio_iommu_handle_detach(VirtIOIOMMU *s,
struct iovec *iov,
unsigned int iov_cnt)
{
- return -ENOENT;
+ struct virtio_iommu_req_detach req;
+ size_t sz, payload_sz;
+
+ payload_sz = get_payload_size(req);
+
+ sz = iov_to_buf(iov, iov_cnt, 0, &req, payload_sz);
+ if (sz != payload_sz) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+ return virtio_iommu_detach(s, &req);
}
static int virtio_iommu_handle_map(VirtIOIOMMU *s,
struct iovec *iov,
unsigned int iov_cnt)
{
- return -ENOENT;
+ struct virtio_iommu_req_map req;
+ size_t sz, payload_sz;
+
+ payload_sz = get_payload_size(req);
+
+ sz = iov_to_buf(iov, iov_cnt, 0, &req, payload_sz);
+ if (sz != payload_sz) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+ return virtio_iommu_map(s, &req);
}
static int virtio_iommu_handle_unmap(VirtIOIOMMU *s,
struct iovec *iov,
unsigned int iov_cnt)
{
- return -ENOENT;
+ struct virtio_iommu_req_unmap req;
+ size_t sz, payload_sz;
+
+ payload_sz = get_payload_size(req);
+
+ sz = iov_to_buf(iov, iov_cnt, 0, &req, payload_sz);
+ if (sz != payload_sz) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+ return virtio_iommu_unmap(s, &req);
}
static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 05/17] virtio-iommu: Add the iommu regions
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (3 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 04/17] virtio-iommu: Decode the command payload Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers Eric Auger
` (12 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch initializes the iommu memory regions so that
PCIe end point transactions get translated. The translation
function is not yet implemented though.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v6 -> v7:
- use primary_bus
- rebase on new translate proto featuring iommu_idx
v5 -> v6:
- include qapi/error.h
- fix g_hash_table_lookup key in virtio_iommu_find_add_as
v4 -> v5:
- use PCI bus handle as a key
- use get_primary_pci_bus() callback
v3 -> v4:
- add trace_virtio_iommu_init_iommu_mr
v2 -> v3:
- use IOMMUMemoryRegion
- iommu mr name built with BDF
- rename smmu_get_sid into virtio_iommu_get_sid and use PCI_BUILD_BDF
---
hw/virtio/trace-events | 2 +
hw/virtio/virtio-iommu.c | 94 ++++++++++++++++++++++++++++++++
include/hw/virtio/virtio-iommu.h | 2 +
3 files changed, 98 insertions(+)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index e6177ca0e4..9270b0463e 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -59,3 +59,5 @@ virtio_iommu_attach(uint32_t domain_id, uint32_t ep_id) "domain=%d endpoint=%d"
virtio_iommu_detach(uint32_t domain_id, uint32_t ep_id) "domain=%d endpoint=%d"
virtio_iommu_map(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end, uint64_t phys_start, uint32_t flags) "domain=%d virt_start=0x%"PRIx64" virt_end=0x%"PRIx64 " phys_start=0x%"PRIx64" flags=%d"
virtio_iommu_unmap(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end) "domain=%d virt_start=0x%"PRIx64" virt_end=0x%"PRIx64
+virtio_iommu_translate(const char *name, uint32_t rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64" flag=%d"
+virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index fc95751c40..dead062baf 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -22,6 +22,10 @@
#include "qemu-common.h"
#include "hw/virtio/virtio.h"
#include "sysemu/kvm.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/i386/pc.h"
+#include "hw/arm/virt.h"
#include "trace.h"
#include "standard-headers/linux/virtio_ids.h"
@@ -33,6 +37,50 @@
/* Max size */
#define VIOMMU_DEFAULT_QUEUE_SIZE 256
+static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev)
+{
+ return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);
+}
+
+static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
+ int devfn)
+{
+ VirtIOIOMMU *s = opaque;
+ IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
+ IOMMUDevice *sdev;
+
+ if (!sbus) {
+ sbus = g_malloc0(sizeof(IOMMUPciBus) +
+ sizeof(IOMMUDevice *) * IOMMU_PCI_DEVFN_MAX);
+ sbus->bus = bus;
+ g_hash_table_insert(s->as_by_busptr, bus, sbus);
+ }
+
+ sdev = sbus->pbdev[devfn];
+ if (!sdev) {
+ char *name = g_strdup_printf("%s-%d-%d",
+ TYPE_VIRTIO_IOMMU_MEMORY_REGION,
+ pci_bus_num(bus), devfn);
+ sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
+
+ sdev->viommu = s;
+ sdev->bus = bus;
+ sdev->devfn = devfn;
+
+ trace_virtio_iommu_init_iommu_mr(name);
+
+ memory_region_init_iommu(&sdev->iommu_mr, sizeof(sdev->iommu_mr),
+ TYPE_VIRTIO_IOMMU_MEMORY_REGION,
+ OBJECT(s), name,
+ UINT64_MAX);
+ address_space_init(&sdev->as,
+ MEMORY_REGION(&sdev->iommu_mr), TYPE_VIRTIO_IOMMU);
+ }
+
+ return &sdev->as;
+
+}
+
static int virtio_iommu_attach(VirtIOIOMMU *s,
struct virtio_iommu_req_attach *req)
{
@@ -204,6 +252,27 @@ static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
}
}
+static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
+ IOMMUAccessFlags flag,
+ int iommu_idx)
+{
+ IOMMUDevice *sdev = container_of(mr, IOMMUDevice, iommu_mr);
+ uint32_t sid;
+
+ IOMMUTLBEntry entry = {
+ .target_as = &address_space_memory,
+ .iova = addr,
+ .translated_addr = addr,
+ .addr_mask = ~(hwaddr)0,
+ .perm = IOMMU_NONE,
+ };
+
+ sid = virtio_iommu_get_sid(sdev);
+
+ trace_virtio_iommu_translate(mr->parent_obj.name, sid, addr, flag);
+ return entry;
+}
+
static void virtio_iommu_get_config(VirtIODevice *vdev, uint8_t *config_data)
{
VirtIOIOMMU *dev = VIRTIO_IOMMU(vdev);
@@ -286,6 +355,15 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_DOMAIN_BITS);
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
+
+ memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
+ s->as_by_busptr = g_hash_table_new(NULL, NULL);
+
+ if (s->primary_bus) {
+ pci_setup_iommu(s->primary_bus, virtio_iommu_find_add_as, s);
+ } else {
+ error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
+ }
}
static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)
@@ -344,6 +422,14 @@ static void virtio_iommu_class_init(ObjectClass *klass, void *data)
vdc->vmsd = &vmstate_virtio_iommu_device;
}
+static void virtio_iommu_memory_region_class_init(ObjectClass *klass,
+ void *data)
+{
+ IOMMUMemoryRegionClass *imrc = IOMMU_MEMORY_REGION_CLASS(klass);
+
+ imrc->translate = virtio_iommu_translate;
+}
+
static const TypeInfo virtio_iommu_info = {
.name = TYPE_VIRTIO_IOMMU,
.parent = TYPE_VIRTIO_DEVICE,
@@ -352,9 +438,17 @@ static const TypeInfo virtio_iommu_info = {
.class_init = virtio_iommu_class_init,
};
+static const TypeInfo virtio_iommu_memory_region_info = {
+ .parent = TYPE_IOMMU_MEMORY_REGION,
+ .name = TYPE_VIRTIO_IOMMU_MEMORY_REGION,
+ .class_init = virtio_iommu_memory_region_class_init,
+};
+
+
static void virtio_register_types(void)
{
type_register_static(&virtio_iommu_info);
+ type_register_static(&virtio_iommu_memory_region_info);
}
type_init(virtio_register_types)
diff --git a/include/hw/virtio/virtio-iommu.h b/include/hw/virtio/virtio-iommu.h
index 4d47b6abeb..f55f48d304 100644
--- a/include/hw/virtio/virtio-iommu.h
+++ b/include/hw/virtio/virtio-iommu.h
@@ -28,6 +28,8 @@
#define VIRTIO_IOMMU(obj) \
OBJECT_CHECK(VirtIOIOMMU, (obj), TYPE_VIRTIO_IOMMU)
+#define TYPE_VIRTIO_IOMMU_MEMORY_REGION "virtio-iommu-memory-region"
+
#define IOMMU_PCI_BUS_MAX 256
#define IOMMU_PCI_DEVFN_MAX 256
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (4 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 05/17] virtio-iommu: Add the iommu regions Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-23 6:38 ` Bharat Bhushan
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 07/17] virtio-iommu: Implement attach/detach command Eric Auger
` (11 subsequent siblings)
17 siblings, 1 reply; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch introduce domain and endpoint internal
datatypes. Both are stored in RB trees. The domain
owns a list of endpoints attached to it.
Helpers to get/put end points and domains are introduced.
get() helpers will become static in subsequent patches.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v6 -> v7:
- on virtio_iommu_find_add_as the bus number computation may
not be finalized yet so we cannot register the EPs at that time.
Hence, let's remove the get_endpoint and also do not use the
bus number for building the memory region name string (only
used for debug though).
v4 -> v5:
- initialize as->endpoint_list
v3 -> v4:
- new separate patch
---
hw/virtio/trace-events | 4 ++
hw/virtio/virtio-iommu.c | 125 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 128 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 9270b0463e..4b15086872 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -61,3 +61,7 @@ virtio_iommu_map(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end, uin
virtio_iommu_unmap(uint32_t domain_id, uint64_t virt_start, uint64_t virt_end) "domain=%d virt_start=0x%"PRIx64" virt_end=0x%"PRIx64
virtio_iommu_translate(const char *name, uint32_t rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64" flag=%d"
virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
+virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
+virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
+virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
+virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index dead062baf..1b9c3ba416 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -33,20 +33,124 @@
#include "hw/virtio/virtio-bus.h"
#include "hw/virtio/virtio-access.h"
#include "hw/virtio/virtio-iommu.h"
+#include "hw/pci/pci_bus.h"
+#include "hw/pci/pci.h"
/* Max size */
#define VIOMMU_DEFAULT_QUEUE_SIZE 256
+typedef struct viommu_domain {
+ uint32_t id;
+ GTree *mappings;
+ QLIST_HEAD(, viommu_endpoint) endpoint_list;
+} viommu_domain;
+
+typedef struct viommu_endpoint {
+ uint32_t id;
+ viommu_domain *domain;
+ QLIST_ENTRY(viommu_endpoint) next;
+ VirtIOIOMMU *viommu;
+} viommu_endpoint;
+
+typedef struct viommu_interval {
+ uint64_t low;
+ uint64_t high;
+} viommu_interval;
+
static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev)
{
return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);
}
+static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer user_data)
+{
+ viommu_interval *inta = (viommu_interval *)a;
+ viommu_interval *intb = (viommu_interval *)b;
+
+ if (inta->high <= intb->low) {
+ return -1;
+ } else if (intb->high <= inta->low) {
+ return 1;
+ } else {
+ return 0;
+ }
+}
+
+static void virtio_iommu_detach_endpoint_from_domain(viommu_endpoint *ep)
+{
+ QLIST_REMOVE(ep, next);
+ ep->domain = NULL;
+}
+
+viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s, uint32_t ep_id);
+viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s, uint32_t ep_id)
+{
+ viommu_endpoint *ep;
+
+ ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
+ if (ep) {
+ return ep;
+ }
+ ep = g_malloc0(sizeof(*ep));
+ ep->id = ep_id;
+ ep->viommu = s;
+ trace_virtio_iommu_get_endpoint(ep_id);
+ g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
+ return ep;
+}
+
+static void virtio_iommu_put_endpoint(gpointer data)
+{
+ viommu_endpoint *ep = (viommu_endpoint *)data;
+
+ if (ep->domain) {
+ virtio_iommu_detach_endpoint_from_domain(ep);
+ g_tree_unref(ep->domain->mappings);
+ }
+
+ trace_virtio_iommu_put_endpoint(ep->id);
+ g_free(ep);
+}
+
+viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t domain_id);
+viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t domain_id)
+{
+ viommu_domain *domain;
+
+ domain = g_tree_lookup(s->domains, GUINT_TO_POINTER(domain_id));
+ if (domain) {
+ return domain;
+ }
+ domain = g_malloc0(sizeof(*domain));
+ domain->id = domain_id;
+ domain->mappings = g_tree_new_full((GCompareDataFunc)interval_cmp,
+ NULL, (GDestroyNotify)g_free,
+ (GDestroyNotify)g_free);
+ g_tree_insert(s->domains, GUINT_TO_POINTER(domain_id), domain);
+ QLIST_INIT(&domain->endpoint_list);
+ trace_virtio_iommu_get_domain(domain_id);
+ return domain;
+}
+
+static void virtio_iommu_put_domain(gpointer data)
+{
+ viommu_domain *domain = (viommu_domain *)data;
+ viommu_endpoint *iter, *tmp;
+
+ QLIST_FOREACH_SAFE(iter, &domain->endpoint_list, next, tmp) {
+ virtio_iommu_detach_endpoint_from_domain(iter);
+ }
+ g_tree_destroy(domain->mappings);
+ trace_virtio_iommu_put_domain(domain->id);
+ g_free(domain);
+}
+
static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
int devfn)
{
VirtIOIOMMU *s = opaque;
IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
+ static uint32_t mr_index;
IOMMUDevice *sdev;
if (!sbus) {
@@ -60,7 +164,7 @@ static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
if (!sdev) {
char *name = g_strdup_printf("%s-%d-%d",
TYPE_VIRTIO_IOMMU_MEMORY_REGION,
- pci_bus_num(bus), devfn);
+ mr_index++, devfn);
sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
sdev->viommu = s;
@@ -75,6 +179,7 @@ static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
UINT64_MAX);
address_space_init(&sdev->as,
MEMORY_REGION(&sdev->iommu_mr), TYPE_VIRTIO_IOMMU);
+ g_free(name);
}
return &sdev->as;
@@ -332,6 +437,13 @@ static const VMStateDescription vmstate_virtio_iommu_device = {
},
};
+static gint int_cmp(gconstpointer a, gconstpointer b, gpointer user_data)
+{
+ uint ua = GPOINTER_TO_UINT(a);
+ uint ub = GPOINTER_TO_UINT(b);
+ return (ua > ub) - (ua < ub);
+}
+
static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
{
VirtIODevice *vdev = VIRTIO_DEVICE(dev);
@@ -356,6 +468,8 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
+ qemu_mutex_init(&s->mutex);
+
memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
s->as_by_busptr = g_hash_table_new(NULL, NULL);
@@ -364,11 +478,20 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
} else {
error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
}
+
+ s->domains = g_tree_new_full((GCompareDataFunc)int_cmp,
+ NULL, NULL, virtio_iommu_put_domain);
+ s->endpoints = g_tree_new_full((GCompareDataFunc)int_cmp,
+ NULL, NULL, virtio_iommu_put_endpoint);
}
static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)
{
VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+ VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
+
+ g_tree_destroy(s->domains);
+ g_tree_destroy(s->endpoints);
virtio_cleanup(vdev);
}
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 07/17] virtio-iommu: Implement attach/detach command
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (5 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 08/17] virtio-iommu: Implement map/unmap Eric Auger
` (10 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch implements the endpoint attach/detach to/from
a domain.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
---
hw/virtio/virtio-iommu.c | 40 ++++++++++++++++++++++++++++++++++------
1 file changed, 34 insertions(+), 6 deletions(-)
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 1b9c3ba416..5c231f865c 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -82,8 +82,8 @@ static void virtio_iommu_detach_endpoint_from_domain(viommu_endpoint *ep)
ep->domain = NULL;
}
-viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s, uint32_t ep_id);
-viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s, uint32_t ep_id)
+static viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
+ uint32_t ep_id)
{
viommu_endpoint *ep;
@@ -112,8 +112,8 @@ static void virtio_iommu_put_endpoint(gpointer data)
g_free(ep);
}
-viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t domain_id);
-viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t domain_id)
+static viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s,
+ uint32_t domain_id)
{
viommu_domain *domain;
@@ -191,10 +191,27 @@ static int virtio_iommu_attach(VirtIOIOMMU *s,
{
uint32_t domain_id = le32_to_cpu(req->domain);
uint32_t ep_id = le32_to_cpu(req->endpoint);
+ viommu_domain *domain;
+ viommu_endpoint *ep;
trace_virtio_iommu_attach(domain_id, ep_id);
- return VIRTIO_IOMMU_S_UNSUPP;
+ ep = virtio_iommu_get_endpoint(s, ep_id);
+ if (ep->domain) {
+ /*
+ * the device is already attached to a domain,
+ * detach it first
+ */
+ virtio_iommu_detach_endpoint_from_domain(ep);
+ }
+
+ domain = virtio_iommu_get_domain(s, domain_id);
+ QLIST_INSERT_HEAD(&domain->endpoint_list, ep, next);
+
+ ep->domain = domain;
+ g_tree_ref(domain->mappings);
+
+ return VIRTIO_IOMMU_S_OK;
}
static int virtio_iommu_detach(VirtIOIOMMU *s,
@@ -202,10 +219,21 @@ static int virtio_iommu_detach(VirtIOIOMMU *s,
{
uint32_t domain_id = le32_to_cpu(req->domain);
uint32_t ep_id = le32_to_cpu(req->endpoint);
+ viommu_endpoint *ep;
trace_virtio_iommu_detach(domain_id, ep_id);
- return VIRTIO_IOMMU_S_UNSUPP;
+ ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
+ if (!ep) {
+ return VIRTIO_IOMMU_S_NOENT;
+ }
+
+ if (!ep->domain) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+
+ virtio_iommu_detach_endpoint_from_domain(ep);
+ return VIRTIO_IOMMU_S_OK;
}
static int virtio_iommu_map(VirtIOIOMMU *s,
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 08/17] virtio-iommu: Implement map/unmap
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (6 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 07/17] virtio-iommu: Implement attach/detach command Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 09/17] virtio-iommu: Implement translate Eric Auger
` (9 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch implements virtio_iommu_map/unmap.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v5 -> v6:
- use new v0.6 fields
- replace error_report by qemu_log_mask
v3 -> v4:
- implement unmap semantics as specified in v0.4
---
hw/virtio/trace-events | 3 ++
hw/virtio/virtio-iommu.c | 94 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 95 insertions(+), 2 deletions(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 4b15086872..b3c5c2604e 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -65,3 +65,6 @@ virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
+virtio_iommu_unmap_left_interval(uint64_t low, uint64_t high, uint64_t next_low, uint64_t next_high) "Unmap left [0x%"PRIx64",0x%"PRIx64"], new interval=[0x%"PRIx64",0x%"PRIx64"]"
+virtio_iommu_unmap_right_interval(uint64_t low, uint64_t high, uint64_t next_low, uint64_t next_high) "Unmap right [0x%"PRIx64",0x%"PRIx64"], new interval=[0x%"PRIx64",0x%"PRIx64"]"
+virtio_iommu_unmap_inc_interval(uint64_t low, uint64_t high) "Unmap inc [0x%"PRIx64",0x%"PRIx64"]"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 5c231f865c..23f7dc6f7f 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -18,6 +18,7 @@
*/
#include "qemu/osdep.h"
+#include "qemu/log.h"
#include "qemu/iov.h"
#include "qemu-common.h"
#include "hw/virtio/virtio.h"
@@ -57,6 +58,13 @@ typedef struct viommu_interval {
uint64_t high;
} viommu_interval;
+typedef struct viommu_mapping {
+ uint64_t virt_addr;
+ uint64_t phys_addr;
+ uint64_t size;
+ uint32_t flags;
+} viommu_mapping;
+
static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev)
{
return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);
@@ -244,10 +252,37 @@ static int virtio_iommu_map(VirtIOIOMMU *s,
uint64_t virt_start = le64_to_cpu(req->virt_start);
uint64_t virt_end = le64_to_cpu(req->virt_end);
uint32_t flags = le32_to_cpu(req->flags);
+ viommu_domain *domain;
+ viommu_interval *interval;
+ viommu_mapping *mapping;
+
+ interval = g_malloc0(sizeof(*interval));
+
+ interval->low = virt_start;
+ interval->high = virt_end;
+
+ domain = g_tree_lookup(s->domains, GUINT_TO_POINTER(domain_id));
+ if (!domain) {
+ return VIRTIO_IOMMU_S_NOENT;
+ }
+
+ mapping = g_tree_lookup(domain->mappings, (gpointer)interval);
+ if (mapping) {
+ g_free(interval);
+ return VIRTIO_IOMMU_S_INVAL;
+ }
trace_virtio_iommu_map(domain_id, virt_start, virt_end, phys_start, flags);
- return VIRTIO_IOMMU_S_UNSUPP;
+ mapping = g_malloc0(sizeof(*mapping));
+ mapping->virt_addr = virt_start;
+ mapping->phys_addr = phys_start;
+ mapping->size = virt_end - virt_start + 1;
+ mapping->flags = flags;
+
+ g_tree_insert(domain->mappings, interval, mapping);
+
+ return VIRTIO_IOMMU_S_OK;
}
static int virtio_iommu_unmap(VirtIOIOMMU *s,
@@ -256,10 +291,65 @@ static int virtio_iommu_unmap(VirtIOIOMMU *s,
uint32_t domain_id = le32_to_cpu(req->domain);
uint64_t virt_start = le64_to_cpu(req->virt_start);
uint64_t virt_end = le64_to_cpu(req->virt_end);
+ uint64_t size = virt_end - virt_start + 1;
+ viommu_mapping *mapping;
+ viommu_interval interval;
+ viommu_domain *domain;
trace_virtio_iommu_unmap(domain_id, virt_start, virt_end);
- return VIRTIO_IOMMU_S_UNSUPP;
+ domain = g_tree_lookup(s->domains, GUINT_TO_POINTER(domain_id));
+ if (!domain) {
+ qemu_log_mask(LOG_GUEST_ERROR, "%s: no domain\n", __func__);
+ return VIRTIO_IOMMU_S_NOENT;
+ }
+ interval.low = virt_start;
+ interval.high = virt_end;
+
+ mapping = g_tree_lookup(domain->mappings, (gpointer)(&interval));
+
+ while (mapping) {
+ viommu_interval current;
+ uint64_t low = mapping->virt_addr;
+ uint64_t high = mapping->virt_addr + mapping->size - 1;
+
+ current.low = low;
+ current.high = high;
+
+ if (low == interval.low && size >= mapping->size) {
+ g_tree_remove(domain->mappings, (gpointer)(¤t));
+ interval.low = high + 1;
+ trace_virtio_iommu_unmap_left_interval(current.low, current.high,
+ interval.low, interval.high);
+ } else if (high == interval.high && size >= mapping->size) {
+ trace_virtio_iommu_unmap_right_interval(current.low, current.high,
+ interval.low, interval.high);
+ g_tree_remove(domain->mappings, (gpointer)(¤t));
+ interval.high = low - 1;
+ } else if (low > interval.low && high < interval.high) {
+ trace_virtio_iommu_unmap_inc_interval(current.low, current.high);
+ g_tree_remove(domain->mappings, (gpointer)(¤t));
+ } else {
+ break;
+ }
+ if (interval.low >= interval.high) {
+ return VIRTIO_IOMMU_S_OK;
+ } else {
+ mapping = g_tree_lookup(domain->mappings, (gpointer)(&interval));
+ }
+ }
+
+ if (mapping) {
+ qemu_log_mask(LOG_GUEST_ERROR,
+ "****** %s: Unmap 0x%"PRIx64" size=0x%"PRIx64
+ " from 0x%"PRIx64" size=0x%"PRIx64" is not supported\n",
+ __func__, interval.low, size,
+ mapping->virt_addr, mapping->size);
+ } else {
+ return VIRTIO_IOMMU_S_OK;
+ }
+
+ return VIRTIO_IOMMU_S_INVAL;
}
#define get_payload_size(req) (\
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 09/17] virtio-iommu: Implement translate
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (7 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 08/17] virtio-iommu: Implement map/unmap Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 10/17] virtio-iommu: Implement probe request Eric Auger
` (8 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch implements the translate callback
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v6 -> v7:
- implemented bypass-mode
v5 -> v6:
- replace error_report by qemu_log_mask
v4 -> v5:
- check the device domain is not NULL
- s/printf/error_report
- set flags to IOMMU_NONE in case of all translation faults
---
hw/virtio/trace-events | 1 +
hw/virtio/virtio-iommu.c | 57 +++++++++++++++++++++++++++++++++++++++-
2 files changed, 57 insertions(+), 1 deletion(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index b3c5c2604e..1f0e143b55 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -68,3 +68,4 @@ virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
virtio_iommu_unmap_left_interval(uint64_t low, uint64_t high, uint64_t next_low, uint64_t next_high) "Unmap left [0x%"PRIx64",0x%"PRIx64"], new interval=[0x%"PRIx64",0x%"PRIx64"]"
virtio_iommu_unmap_right_interval(uint64_t low, uint64_t high, uint64_t next_low, uint64_t next_high) "Unmap right [0x%"PRIx64",0x%"PRIx64"], new interval=[0x%"PRIx64",0x%"PRIx64"]"
virtio_iommu_unmap_inc_interval(uint64_t low, uint64_t high) "Unmap inc [0x%"PRIx64",0x%"PRIx64"]"
+virtio_iommu_translate_out(uint64_t virt_addr, uint64_t phys_addr, uint32_t sid) "0x%"PRIx64" -> 0x%"PRIx64 " for sid=%d"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 23f7dc6f7f..af90413b37 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -480,19 +480,74 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
int iommu_idx)
{
IOMMUDevice *sdev = container_of(mr, IOMMUDevice, iommu_mr);
+ VirtIOIOMMU *s = sdev->viommu;
uint32_t sid;
+ viommu_endpoint *ep;
+ viommu_mapping *mapping;
+ viommu_interval interval;
+ bool bypass_allowed;
+
+ interval.low = addr;
+ interval.high = addr + 1;
IOMMUTLBEntry entry = {
.target_as = &address_space_memory,
.iova = addr,
.translated_addr = addr,
- .addr_mask = ~(hwaddr)0,
+ .addr_mask = (1 << ctz32(s->config.page_size_mask)) - 1,
.perm = IOMMU_NONE,
};
+ bypass_allowed = virtio_has_feature(s->acked_features,
+ VIRTIO_IOMMU_F_BYPASS);
+
sid = virtio_iommu_get_sid(sdev);
trace_virtio_iommu_translate(mr->parent_obj.name, sid, addr, flag);
+ qemu_mutex_lock(&s->mutex);
+
+ ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(sid));
+ if (!ep) {
+ if (!bypass_allowed) {
+ error_report("%s sid=%d is not known!!", __func__, sid);
+ } else {
+ entry.perm = flag;
+ }
+ goto unlock;
+ }
+
+ if (!ep->domain) {
+ if (!bypass_allowed) {
+ qemu_log_mask(LOG_GUEST_ERROR,
+ "%s %02x:%02x.%01x not attached to any domain\n",
+ __func__, PCI_BUS_NUM(sid), PCI_SLOT(sid), PCI_FUNC(sid));
+ } else {
+ entry.perm = flag;
+ }
+ goto unlock;
+ }
+
+ mapping = g_tree_lookup(ep->domain->mappings, (gpointer)(&interval));
+ if (!mapping) {
+ qemu_log_mask(LOG_GUEST_ERROR,
+ "%s no mapping for 0x%"PRIx64" for sid=%d\n",
+ __func__, addr, sid);
+ goto unlock;
+ }
+
+ if (((flag & IOMMU_RO) && !(mapping->flags & VIRTIO_IOMMU_MAP_F_READ)) ||
+ ((flag & IOMMU_WO) && !(mapping->flags & VIRTIO_IOMMU_MAP_F_WRITE))) {
+ qemu_log_mask(LOG_GUEST_ERROR,
+ "Permission error on 0x%"PRIx64"(%d): allowed=%d\n",
+ addr, flag, mapping->flags);
+ goto unlock;
+ }
+ entry.translated_addr = addr - mapping->virt_addr + mapping->phys_addr;
+ entry.perm = flag;
+ trace_virtio_iommu_translate_out(addr, entry.translated_addr, sid);
+
+unlock:
+ qemu_mutex_unlock(&s->mutex);
return entry;
}
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 10/17] virtio-iommu: Implement probe request
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (8 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 09/17] virtio-iommu: Implement translate Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 11/17] virtio-iommu: Expose the IOAPIC MSI reserved region when relevant Eric Auger
` (7 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch implements the PROBE request. At the moment,
no reserved regions are returned as none are registered
per device. Only a NONE property is returned.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- fix filling of properties (changes induced by v0.7 -> v0.8 spec
evolution)
v7 -> v8:
- adapt to removal of value filed in virtio_iommu_probe_property
v6 -> v7:
- adapt to the change in virtio_iommu_probe_resv_mem fields
- use get_endpoint() instead of directly checking the EP
was registered.
v4 -> v5:
- initialize bufstate.error to false
- add cpu_to_le64(size)
---
hw/virtio/trace-events | 2 +
hw/virtio/virtio-iommu.c | 181 ++++++++++++++++++++++++++++++++++++++-
2 files changed, 181 insertions(+), 2 deletions(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 1f0e143b55..19824c3e91 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -69,3 +69,5 @@ virtio_iommu_unmap_left_interval(uint64_t low, uint64_t high, uint64_t next_low,
virtio_iommu_unmap_right_interval(uint64_t low, uint64_t high, uint64_t next_low, uint64_t next_high) "Unmap right [0x%"PRIx64",0x%"PRIx64"], new interval=[0x%"PRIx64",0x%"PRIx64"]"
virtio_iommu_unmap_inc_interval(uint64_t low, uint64_t high) "Unmap inc [0x%"PRIx64",0x%"PRIx64"]"
virtio_iommu_translate_out(uint64_t virt_addr, uint64_t phys_addr, uint32_t sid) "0x%"PRIx64" -> 0x%"PRIx64 " for sid=%d"
+virtio_iommu_fill_resv_property(uint32_t devid, uint8_t subtype, uint64_t start, uint64_t end, uint32_t flags, size_t filled) "dev= %d, subtype=%d start=0x%"PRIx64" end=0x%"PRIx64" flags=%d filled=0x%lx"
+virtio_iommu_fill_none_property(uint32_t devid) "devid=%d"
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index af90413b37..4fc43494d9 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -39,6 +39,10 @@
/* Max size */
#define VIOMMU_DEFAULT_QUEUE_SIZE 256
+#define VIOMMU_PROBE_SIZE 512
+
+#define SUPPORTED_PROBE_PROPERTIES (\
+ 1 << VIRTIO_IOMMU_PROBE_T_RESV_MEM)
typedef struct viommu_domain {
uint32_t id;
@@ -51,6 +55,7 @@ typedef struct viommu_endpoint {
viommu_domain *domain;
QLIST_ENTRY(viommu_endpoint) next;
VirtIOIOMMU *viommu;
+ GTree *reserved_regions;
} viommu_endpoint;
typedef struct viommu_interval {
@@ -65,6 +70,13 @@ typedef struct viommu_mapping {
uint32_t flags;
} viommu_mapping;
+typedef struct viommu_property_buffer {
+ viommu_endpoint *endpoint;
+ size_t filled;
+ uint8_t *start;
+ bool error;
+} viommu_property_buffer;
+
static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev)
{
return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn);
@@ -104,6 +116,9 @@ static viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
ep->viommu = s;
trace_virtio_iommu_get_endpoint(ep_id);
g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
+ ep->reserved_regions = g_tree_new_full((GCompareDataFunc)interval_cmp,
+ NULL, (GDestroyNotify)g_free,
+ (GDestroyNotify)g_free);
return ep;
}
@@ -117,6 +132,7 @@ static void virtio_iommu_put_endpoint(gpointer data)
}
trace_virtio_iommu_put_endpoint(ep->id);
+ g_tree_destroy(ep->reserved_regions);
g_free(ep);
}
@@ -352,6 +368,131 @@ static int virtio_iommu_unmap(VirtIOIOMMU *s,
return VIRTIO_IOMMU_S_INVAL;
}
+/**
+ * virtio_iommu_fill_resv_mem_prop - Add a RESV_MEM probe
+ * property into the probe request buffer
+ *
+ * @key: interval handle
+ * @value: handle to the reserved memory region
+ * @data: handle to the probe request buffer state
+ */
+static gboolean virtio_iommu_fill_resv_mem_prop(gpointer key,
+ gpointer value,
+ gpointer data)
+{
+ struct virtio_iommu_probe_resv_mem *resv =
+ (struct virtio_iommu_probe_resv_mem *)value;
+ struct virtio_iommu_probe_resv_mem *buf_prop;
+ viommu_property_buffer *bufstate = (viommu_property_buffer *)data;
+ size_t prop_size = sizeof(*resv);
+
+ if (bufstate->filled + prop_size >= VIOMMU_PROBE_SIZE) {
+ bufstate->error = true;
+ /* get the traversal stopped by returning true */
+ return true;
+ }
+ buf_prop = (struct virtio_iommu_probe_resv_mem *)
+ (bufstate->start + bufstate->filled);
+ *buf_prop = *resv;
+
+ bufstate->filled += prop_size;
+ trace_virtio_iommu_fill_resv_property(bufstate->endpoint->id,
+ resv->subtype, resv->start,
+ resv->end, resv->subtype,
+ bufstate->filled);
+ return false;
+}
+
+static int virtio_iommu_fill_none_prop(viommu_property_buffer *bufstate)
+{
+ struct virtio_iommu_probe_property *prop;
+
+ prop = (struct virtio_iommu_probe_property *)
+ (bufstate->start + bufstate->filled);
+ prop->type = cpu_to_le16(VIRTIO_IOMMU_PROBE_T_NONE)
+ & VIRTIO_IOMMU_PROBE_T_MASK;
+ prop->length = 0;
+ bufstate->filled += sizeof(*prop);
+ trace_virtio_iommu_fill_none_property(bufstate->endpoint->id);
+ return 0;
+}
+
+/* Fill the properties[] buffer with properties of type @type */
+static int virtio_iommu_fill_property(int type,
+ viommu_property_buffer *bufstate)
+{
+ int ret = -ENOSPC;
+
+ if (bufstate->filled + sizeof(struct virtio_iommu_probe_property)
+ >= VIOMMU_PROBE_SIZE) {
+ /* no space left for the header */
+ bufstate->error = true;
+ goto out;
+ }
+
+ switch (type) {
+ case VIRTIO_IOMMU_PROBE_T_NONE:
+ ret = virtio_iommu_fill_none_prop(bufstate);
+ break;
+ case VIRTIO_IOMMU_PROBE_T_RESV_MEM:
+ {
+ viommu_endpoint *ep = bufstate->endpoint;
+
+ g_tree_foreach(ep->reserved_regions,
+ virtio_iommu_fill_resv_mem_prop,
+ bufstate);
+ if (!bufstate->error) {
+ ret = 0;
+ }
+ break;
+ }
+ default:
+ ret = -ENOENT;
+ break;
+ }
+out:
+ if (ret) {
+ error_report("%s property of type=%d could not be filled (%d),"
+ " remaining size = 0x%lx",
+ __func__, type, ret, bufstate->filled);
+ }
+ return ret;
+}
+
+/**
+ * virtio_iommu_probe - Fill the probe request buffer with all
+ * the properties the device is able to return and add a NONE
+ * property at the end.
+ */
+static int virtio_iommu_probe(VirtIOIOMMU *s,
+ struct virtio_iommu_req_probe *req,
+ uint8_t *buf)
+{
+ uint32_t ep_id = le32_to_cpu(req->endpoint);
+ int16_t prop_types = SUPPORTED_PROBE_PROPERTIES, type;
+ viommu_property_buffer bufstate;
+ viommu_endpoint *ep;
+ int ret;
+
+ ep = virtio_iommu_get_endpoint(s, ep_id);
+
+ bufstate.start = buf;
+ bufstate.filled = 0;
+ bufstate.error = false;
+ bufstate.endpoint = ep;
+
+ while ((type = ctz32(prop_types)) != 32) {
+ ret = virtio_iommu_fill_property(type, &bufstate);
+ if (ret) {
+ break;
+ }
+ prop_types &= ~(1 << type);
+ }
+ virtio_iommu_fill_property(VIRTIO_IOMMU_PROBE_T_NONE, &bufstate);
+
+ return VIRTIO_IOMMU_S_OK;
+}
+
#define get_payload_size(req) (\
sizeof((req)) - sizeof(struct virtio_iommu_req_tail))
@@ -416,6 +557,24 @@ static int virtio_iommu_handle_unmap(VirtIOIOMMU *s,
return virtio_iommu_unmap(s, &req);
}
+static int virtio_iommu_handle_probe(VirtIOIOMMU *s,
+ struct iovec *iov,
+ unsigned int iov_cnt,
+ uint8_t *buf)
+{
+ struct virtio_iommu_req_probe req;
+ size_t sz, payload_sz;
+
+ payload_sz = sizeof(req);
+
+ sz = iov_to_buf(iov, iov_cnt, 0, &req, payload_sz);
+ if (sz != payload_sz) {
+ return VIRTIO_IOMMU_S_INVAL;
+ }
+
+ return virtio_iommu_probe(s, &req, buf);
+}
+
static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
{
VirtIOIOMMU *s = VIRTIO_IOMMU(vdev);
@@ -460,16 +619,32 @@ static void virtio_iommu_handle_command(VirtIODevice *vdev, VirtQueue *vq)
case VIRTIO_IOMMU_T_UNMAP:
tail.status = virtio_iommu_handle_unmap(s, iov, iov_cnt);
break;
+ case VIRTIO_IOMMU_T_PROBE:
+ {
+ struct virtio_iommu_req_tail *ptail;
+ uint8_t *buf = g_malloc0(s->config.probe_size + sizeof(tail));
+
+ ptail = (struct virtio_iommu_req_tail *)
+ (buf + s->config.probe_size);
+ ptail->status = virtio_iommu_handle_probe(s, iov, iov_cnt, buf);
+
+ sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
+ buf, s->config.probe_size + sizeof(tail));
+ g_free(buf);
+ assert(sz == s->config.probe_size + sizeof(tail));
+ goto push;
+ }
default:
tail.status = VIRTIO_IOMMU_S_UNSUPP;
}
- qemu_mutex_unlock(&s->mutex);
sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
&tail, sizeof(tail));
assert(sz == sizeof(tail));
- virtqueue_push(vq, elem, sizeof(tail));
+push:
+ qemu_mutex_unlock(&s->mutex);
+ virtqueue_push(vq, elem, sz);
virtio_notify(vdev, vq);
g_free(elem);
}
@@ -632,6 +807,7 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
s->config.page_size_mask = TARGET_PAGE_MASK;
s->config.input_range.end = -1UL;
s->config.domain_bits = 32;
+ s->config.probe_size = VIOMMU_PROBE_SIZE;
virtio_add_feature(&s->features, VIRTIO_RING_F_EVENT_IDX);
virtio_add_feature(&s->features, VIRTIO_RING_F_INDIRECT_DESC);
@@ -640,6 +816,7 @@ static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_DOMAIN_BITS);
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
+ virtio_add_feature(&s->features, VIRTIO_IOMMU_F_PROBE);
qemu_mutex_init(&s->mutex);
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 11/17] virtio-iommu: Expose the IOAPIC MSI reserved region when relevant
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (9 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 10/17] virtio-iommu: Implement probe request Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 12/17] virtio-iommu: Implement fault reporting Eric Auger
` (6 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
We introduce a new msi_bypass field which indicates whether
the IOAPIC MSI window [0xFEE00000 - 0xFEEFFFFF] must be exposed
as a reserved region. By default the field is set to true at
instantiation time. Later on we will introduce a property at
virtio pci proxy level to turn it off.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- pass IOAPIC_RANGE_END to virtio_iommu_register_resv_region
- take into account the change in the struct virtio_iommu_probe_resv_mem
definition
- We just introduce the field here. A property will be introduced later on
at pci proxy level.
---
hw/virtio/virtio-iommu.c | 36 ++++++++++++++++++++++++++++++++
include/hw/virtio/virtio-iommu.h | 1 +
2 files changed, 37 insertions(+)
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 4fc43494d9..324518c300 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -41,6 +41,9 @@
#define VIOMMU_DEFAULT_QUEUE_SIZE 256
#define VIOMMU_PROBE_SIZE 512
+#define IOAPIC_RANGE_START (0xfee00000)
+#define IOAPIC_RANGE_END (0xfeefffff)
+
#define SUPPORTED_PROBE_PROPERTIES (\
1 << VIRTIO_IOMMU_PROBE_T_RESV_MEM)
@@ -102,6 +105,30 @@ static void virtio_iommu_detach_endpoint_from_domain(viommu_endpoint *ep)
ep->domain = NULL;
}
+static void virtio_iommu_register_resv_region(viommu_endpoint *ep,
+ uint8_t subtype,
+ uint64_t start, uint64_t end)
+{
+ viommu_interval *interval;
+ struct virtio_iommu_probe_resv_mem *resv_reg_prop;
+ size_t prop_size = sizeof(struct virtio_iommu_probe_resv_mem);
+ size_t value_size = prop_size -
+ sizeof(struct virtio_iommu_probe_property);
+
+ interval = g_malloc0(sizeof(*interval));
+ interval->low = start;
+ interval->high = end;
+
+ resv_reg_prop = g_malloc0(prop_size);
+ resv_reg_prop->head.type = VIRTIO_IOMMU_PROBE_T_RESV_MEM;
+ resv_reg_prop->head.length = cpu_to_le64(value_size);
+ resv_reg_prop->subtype = cpu_to_le64(subtype);
+ resv_reg_prop->start = cpu_to_le64(start);
+ resv_reg_prop->end = cpu_to_le64(end);
+
+ g_tree_insert(ep->reserved_regions, interval, resv_reg_prop);
+}
+
static viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
uint32_t ep_id)
{
@@ -119,6 +146,12 @@ static viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
ep->reserved_regions = g_tree_new_full((GCompareDataFunc)interval_cmp,
NULL, (GDestroyNotify)g_free,
(GDestroyNotify)g_free);
+ if (s->msi_bypass) {
+ virtio_iommu_register_resv_region(ep, VIRTIO_IOMMU_RESV_MEM_T_MSI,
+ IOAPIC_RANGE_START,
+ IOAPIC_RANGE_END);
+ }
+
return ep;
}
@@ -858,6 +891,9 @@ static void virtio_iommu_set_status(VirtIODevice *vdev, uint8_t status)
static void virtio_iommu_instance_init(Object *obj)
{
+ VirtIOIOMMU *s = VIRTIO_IOMMU(obj);
+
+ s->msi_bypass = true;
}
static const VMStateDescription vmstate_virtio_iommu = {
diff --git a/include/hw/virtio/virtio-iommu.h b/include/hw/virtio/virtio-iommu.h
index f55f48d304..56c8b4e57f 100644
--- a/include/hw/virtio/virtio-iommu.h
+++ b/include/hw/virtio/virtio-iommu.h
@@ -59,6 +59,7 @@ typedef struct VirtIOIOMMU {
GTree *domains;
QemuMutex mutex;
GTree *endpoints;
+ bool msi_bypass;
} VirtIOIOMMU;
#endif
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 12/17] virtio-iommu: Implement fault reporting
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (10 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 11/17] virtio-iommu: Expose the IOAPIC MSI reserved region when relevant Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 13/17] virtio_iommu: Handle reserved regions in translation process Eric Auger
` (5 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
The event queue allows to report asynchronous errors.
The translate function now injects faults when relevant.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
hw/virtio/trace-events | 1 +
hw/virtio/virtio-iommu.c | 67 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 65 insertions(+), 3 deletions(-)
diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events
index 19824c3e91..053a07b3fc 100644
--- a/hw/virtio/trace-events
+++ b/hw/virtio/trace-events
@@ -71,3 +71,4 @@ virtio_iommu_unmap_inc_interval(uint64_t low, uint64_t high) "Unmap inc [0x%"PRI
virtio_iommu_translate_out(uint64_t virt_addr, uint64_t phys_addr, uint32_t sid) "0x%"PRIx64" -> 0x%"PRIx64 " for sid=%d"
virtio_iommu_fill_resv_property(uint32_t devid, uint8_t subtype, uint64_t start, uint64_t end, uint32_t flags, size_t filled) "dev= %d, subtype=%d start=0x%"PRIx64" end=0x%"PRIx64" flags=%d filled=0x%lx"
virtio_iommu_fill_none_property(uint32_t devid) "devid=%d"
+virtio_iommu_report_fault(uint8_t reason, uint32_t flags, uint32_t endpoint, uint64_t addr) "FAULT reason=%d flags=%d endpoint=%d address =0x%"PRIx64
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 324518c300..1246dd6bdf 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -683,17 +683,63 @@ push:
}
}
+static void virtio_iommu_report_fault(VirtIOIOMMU *viommu, uint8_t reason,
+ uint32_t flags, uint32_t endpoint,
+ uint64_t address)
+{
+ VirtIODevice *vdev = &viommu->parent_obj;
+ VirtQueue *vq = viommu->event_vq;
+ struct virtio_iommu_fault fault;
+ VirtQueueElement *elem;
+ size_t sz;
+
+ memset(&fault, 0, sizeof(fault));
+ fault.reason = reason;
+ fault.flags = flags;
+ fault.endpoint = endpoint;
+ fault.address = address;
+
+ for (;;) {
+ elem = virtqueue_pop(vq, sizeof(VirtQueueElement));
+
+ if (!elem) {
+ virtio_error(vdev,
+ "no buffer available in event queue to report event");
+ return;
+ }
+
+ if (iov_size(elem->in_sg, elem->in_num) < sizeof(fault)) {
+ virtio_error(vdev, "error buffer of wrong size");
+ virtqueue_detach_element(vq, elem, 0);
+ g_free(elem);
+ continue;
+ }
+ break;
+ }
+ /* we have a buffer to fill in */
+ sz = iov_from_buf(elem->in_sg, elem->in_num, 0,
+ &fault, sizeof(fault));
+ assert(sz == sizeof(fault));
+
+ trace_virtio_iommu_report_fault(reason, flags, endpoint, address);
+ virtqueue_push(vq, elem, sz);
+ virtio_notify(vdev, vq);
+ g_free(elem);
+
+}
+
static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
IOMMUAccessFlags flag,
int iommu_idx)
{
IOMMUDevice *sdev = container_of(mr, IOMMUDevice, iommu_mr);
VirtIOIOMMU *s = sdev->viommu;
- uint32_t sid;
+ uint32_t sid, flags;
viommu_endpoint *ep;
viommu_mapping *mapping;
viommu_interval interval;
bool bypass_allowed;
+ bool read_fault, write_fault;
interval.low = addr;
interval.high = addr + 1;
@@ -718,6 +764,8 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
if (!ep) {
if (!bypass_allowed) {
error_report("%s sid=%d is not known!!", __func__, sid);
+ virtio_iommu_report_fault(s, VIRTIO_IOMMU_FAULT_R_UNKNOWN,
+ 0, sid, 0);
} else {
entry.perm = flag;
}
@@ -729,6 +777,8 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
qemu_log_mask(LOG_GUEST_ERROR,
"%s %02x:%02x.%01x not attached to any domain\n",
__func__, PCI_BUS_NUM(sid), PCI_SLOT(sid), PCI_FUNC(sid));
+ virtio_iommu_report_fault(s, VIRTIO_IOMMU_FAULT_R_DOMAIN,
+ 0, sid, 0);
} else {
entry.perm = flag;
}
@@ -740,14 +790,25 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
qemu_log_mask(LOG_GUEST_ERROR,
"%s no mapping for 0x%"PRIx64" for sid=%d\n",
__func__, addr, sid);
+ virtio_iommu_report_fault(s, VIRTIO_IOMMU_FAULT_R_MAPPING,
+ 0, sid, addr);
goto unlock;
}
- if (((flag & IOMMU_RO) && !(mapping->flags & VIRTIO_IOMMU_MAP_F_READ)) ||
- ((flag & IOMMU_WO) && !(mapping->flags & VIRTIO_IOMMU_MAP_F_WRITE))) {
+ read_fault = (flag & IOMMU_RO) &&
+ !(mapping->flags & VIRTIO_IOMMU_MAP_F_READ);
+ write_fault = (flag & IOMMU_WO) &&
+ !(mapping->flags & VIRTIO_IOMMU_MAP_F_WRITE);
+
+ flags = read_fault ? VIRTIO_IOMMU_FAULT_F_READ : 0;
+ flags |= write_fault ? VIRTIO_IOMMU_FAULT_F_WRITE : 0;
+ if (flags) {
qemu_log_mask(LOG_GUEST_ERROR,
"Permission error on 0x%"PRIx64"(%d): allowed=%d\n",
addr, flag, mapping->flags);
+ flags |= VIRTIO_IOMMU_FAULT_F_ADDRESS;
+ virtio_iommu_report_fault(s, VIRTIO_IOMMU_FAULT_R_MAPPING,
+ flags, sid, addr);
goto unlock;
}
entry.translated_addr = addr - mapping->virt_addr + mapping->phys_addr;
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 13/17] virtio_iommu: Handle reserved regions in translation process
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (11 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 12/17] virtio-iommu: Implement fault reporting Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 14/17] virtio-iommu-pci: Add virtio iommu pci support Eric Auger
` (4 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
When translating an address we need to check if it belongs to
a reserved virtual address range. If it does, there are 2 cases:
- it belongs to a RESERVED region: the guest should neither use
this address in a MAP not instruct the end-point to DMA on
them. We report an error
- It belongs to an MSI region: we bypass the translation.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
hw/virtio/virtio-iommu.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
index 1246dd6bdf..2ec01f3b9e 100644
--- a/hw/virtio/virtio-iommu.c
+++ b/hw/virtio/virtio-iommu.c
@@ -740,6 +740,7 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
viommu_interval interval;
bool bypass_allowed;
bool read_fault, write_fault;
+ struct virtio_iommu_probe_resv_mem *reg;
interval.low = addr;
interval.high = addr + 1;
@@ -772,6 +773,21 @@ static IOMMUTLBEntry virtio_iommu_translate(IOMMUMemoryRegion *mr, hwaddr addr,
goto unlock;
}
+ reg = g_tree_lookup(ep->reserved_regions, (gpointer)(&interval));
+ if (reg) {
+ switch (reg->subtype) {
+ case VIRTIO_IOMMU_RESV_MEM_T_MSI:
+ entry.perm = flag;
+ break;
+ case VIRTIO_IOMMU_RESV_MEM_T_RESERVED:
+ default:
+ virtio_iommu_report_fault(s, VIRTIO_IOMMU_FAULT_R_MAPPING,
+ 0, sid, addr);
+ break;
+ }
+ goto unlock;
+ }
+
if (!ep->domain) {
if (!bypass_allowed) {
qemu_log_mask(LOG_GUEST_ERROR,
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 14/17] virtio-iommu-pci: Add virtio iommu pci support
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (12 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 13/17] virtio_iommu: Handle reserved regions in translation process Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings Eric Auger
` (3 subsequent siblings)
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch adds virtio-iommu-pci, which is the pci proxy for
the virtio-iommu device.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- add the msi-bypass property
---
hw/virtio/virtio-pci.c | 51 ++++++++++++++++++++++++++++++++++++++++++
hw/virtio/virtio-pci.h | 14 ++++++++++++
include/hw/pci/pci.h | 1 +
qdev-monitor.c | 1 +
4 files changed, 67 insertions(+)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index a954799267..cdd18afe9e 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -2342,6 +2342,56 @@ static const TypeInfo virtio_balloon_pci_info = {
.class_init = virtio_balloon_pci_class_init,
};
+/* virtio-iommu-pci */
+
+static Property virtio_iommu_pci_properties[] = {
+ DEFINE_PROP_UINT32("class", VirtIOPCIProxy, class_code, 0),
+ DEFINE_PROP_BOOL("msi-bypass", VirtIOIOMMUPCI, vdev.msi_bypass, true),
+ DEFINE_PROP_END_OF_LIST(),
+};
+
+static void virtio_iommu_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
+{
+ VirtIOIOMMUPCI *dev = VIRTIO_IOMMU_PCI(vpci_dev);
+ DeviceState *vdev = DEVICE(&dev->vdev);
+
+ qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus));
+ object_property_set_link(OBJECT(dev),
+ OBJECT(pci_get_bus(&vpci_dev->pci_dev)),
+ "primary-bus", errp);
+ object_property_set_bool(OBJECT(vdev), true, "realized", errp);
+}
+
+static void virtio_iommu_pci_class_init(ObjectClass *klass, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass);
+ PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass);
+ k->realize = virtio_iommu_pci_realize;
+ set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+ dc->props = virtio_iommu_pci_properties;
+ pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET;
+ pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_IOMMU;
+ pcidev_k->revision = VIRTIO_PCI_ABI_VERSION;
+ pcidev_k->class_id = PCI_CLASS_OTHERS;
+}
+
+static void virtio_iommu_pci_instance_init(Object *obj)
+{
+ VirtIOIOMMUPCI *dev = VIRTIO_IOMMU_PCI(obj);
+
+ virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev),
+ TYPE_VIRTIO_IOMMU);
+}
+
+static const TypeInfo virtio_iommu_pci_info = {
+ .name = TYPE_VIRTIO_IOMMU_PCI,
+ .parent = TYPE_VIRTIO_PCI,
+ .instance_size = sizeof(VirtIOIOMMUPCI),
+ .instance_init = virtio_iommu_pci_instance_init,
+ .class_init = virtio_iommu_pci_class_init,
+};
+
/* virtio-serial-pci */
static void virtio_serial_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp)
@@ -2712,6 +2762,7 @@ static void virtio_pci_register_types(void)
#endif
type_register_static(&virtio_scsi_pci_info);
type_register_static(&virtio_balloon_pci_info);
+ type_register_static(&virtio_iommu_pci_info);
type_register_static(&virtio_serial_pci_info);
type_register_static(&virtio_net_pci_info);
#ifdef CONFIG_VHOST_SCSI
diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h
index 813082b0d7..a17100b4a0 100644
--- a/hw/virtio/virtio-pci.h
+++ b/hw/virtio/virtio-pci.h
@@ -26,6 +26,7 @@
#include "hw/virtio/virtio-input.h"
#include "hw/virtio/virtio-gpu.h"
#include "hw/virtio/virtio-crypto.h"
+#include "hw/virtio/virtio-iommu.h"
#include "hw/virtio/vhost-user-scsi.h"
#if defined(CONFIG_VHOST_USER) && defined(CONFIG_LINUX)
#include "hw/virtio/vhost-user-blk.h"
@@ -57,6 +58,7 @@ typedef struct VirtIOInputHostPCI VirtIOInputHostPCI;
typedef struct VirtIOGPUPCI VirtIOGPUPCI;
typedef struct VHostVSockPCI VHostVSockPCI;
typedef struct VirtIOCryptoPCI VirtIOCryptoPCI;
+typedef struct VirtIOIOMMUPCI VirtIOIOMMUPCI;
/* virtio-pci-bus */
@@ -363,6 +365,18 @@ struct VirtIOInputHIDPCI {
VirtIOInputHID vdev;
};
+/*
+ * * virtio-iommu-pci: This extends VirtioPCIProxy.
+ * */
+#define TYPE_VIRTIO_IOMMU_PCI "virtio-iommu-pci"
+#define VIRTIO_IOMMU_PCI(obj) \
+ OBJECT_CHECK(VirtIOIOMMUPCI, (obj), TYPE_VIRTIO_IOMMU_PCI)
+
+struct VirtIOIOMMUPCI {
+ VirtIOPCIProxy parent_obj;
+ VirtIOIOMMU vdev;
+};
+
#ifdef CONFIG_LINUX
#define TYPE_VIRTIO_INPUT_HOST_PCI "virtio-input-host-pci"
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index e6514bba23..a01fd4d2e1 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -85,6 +85,7 @@ extern bool pci_available;
#define PCI_DEVICE_ID_VIRTIO_RNG 0x1005
#define PCI_DEVICE_ID_VIRTIO_9P 0x1009
#define PCI_DEVICE_ID_VIRTIO_VSOCK 0x1012
+#define PCI_DEVICE_ID_VIRTIO_IOMMU 0x1013
#define PCI_VENDOR_ID_REDHAT 0x1b36
#define PCI_DEVICE_ID_REDHAT_BRIDGE 0x0001
diff --git a/qdev-monitor.c b/qdev-monitor.c
index 07147c63bf..4f1ae056da 100644
--- a/qdev-monitor.c
+++ b/qdev-monitor.c
@@ -62,6 +62,7 @@ static const QDevAlias qdev_alias_table[] = {
{ "virtio-input-host-ccw", "virtio-input-host", QEMU_ARCH_S390X },
{ "virtio-input-host-pci", "virtio-input-host",
QEMU_ARCH_ALL & ~QEMU_ARCH_S390X },
+ { "virtio-iommu-pci", "virtio-iommu", QEMU_ARCH_ALL & ~QEMU_ARCH_S390X },
{ "virtio-keyboard-ccw", "virtio-keyboard", QEMU_ARCH_S390X },
{ "virtio-keyboard-pci", "virtio-keyboard",
QEMU_ARCH_ALL & ~QEMU_ARCH_S390X },
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (13 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 14/17] virtio-iommu-pci: Add virtio iommu pci support Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-27 7:07 ` Bharat Bhushan
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 16/17] hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper Eric Auger
` (2 subsequent siblings)
17 siblings, 1 reply; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
Adds the "virtio,pci-iommu" node in the host bridge node and
the RID mapping, excluding the IOMMU RID.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- disable msi-bypass property
- addition of the subnode is handled is the hotplug handler
and IOMMU RID is notimposed anymore
v6 -> v7:
- align to the smmu instantiation code
v4 -> v5:
- VirtMachineClass no_iommu added in this patch
- Use object_resolve_path_type
---
hw/arm/virt.c | 57 +++++++++++++++++++++++++++++++++++++------
include/hw/arm/virt.h | 2 ++
2 files changed, 52 insertions(+), 7 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index a2b8d8f7c2..b2bbb0ef49 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -29,6 +29,7 @@
*/
#include "qemu/osdep.h"
+#include "monitor/qdev.h"
#include "qapi/error.h"
#include "hw/sysbus.h"
#include "hw/arm/arm.h"
@@ -49,6 +50,7 @@
#include "qemu/bitops.h"
#include "qemu/error-report.h"
#include "hw/pci-host/gpex.h"
+#include "hw/virtio/virtio-pci.h"
#include "hw/arm/sysbus-fdt.h"
#include "hw/platform-bus.h"
#include "hw/arm/fdt.h"
@@ -59,6 +61,7 @@
#include "qapi/visitor.h"
#include "standard-headers/linux/input.h"
#include "hw/arm/smmuv3.h"
+#include "hw/virtio/virtio-iommu.h"
#define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \
static void virt_##major##_##minor##_class_init(ObjectClass *oc, \
@@ -1085,6 +1088,33 @@ static void create_smmu(const VirtMachineState *vms, qemu_irq *pic,
g_free(node);
}
+static void create_virtio_iommu(VirtMachineState *vms, Error **errp)
+{
+ const char compat[] = "virtio,pci-iommu";
+ uint16_t bdf = vms->virtio_iommu_bdf;
+ char *node;
+
+ vms->iommu_phandle = qemu_fdt_alloc_phandle(vms->fdt);
+
+ node = g_strdup_printf("%s/virtio_iommu@%d", vms->pciehb_nodename, bdf);
+ qemu_fdt_add_subnode(vms->fdt, node);
+ qemu_fdt_setprop(vms->fdt, node, "compatible", compat, sizeof(compat));
+ qemu_fdt_setprop_sized_cells(vms->fdt, node, "reg",
+ 1, bdf << 8 /* phys.hi */,
+ 1, 0 /* phys.mid */,
+ 1, 0 /* phys.lo */,
+ 1, 0 /* size.hi */,
+ 1, 0 /* size.low */);
+
+ qemu_fdt_setprop_cell(vms->fdt, node, "#iommu-cells", 1);
+ qemu_fdt_setprop_cell(vms->fdt, node, "phandle", vms->iommu_phandle);
+ g_free(node);
+
+ qemu_fdt_setprop_cells(vms->fdt, vms->pciehb_nodename, "iommu-map",
+ 0x0, vms->iommu_phandle, 0x0, bdf,
+ bdf + 1, vms->iommu_phandle, bdf + 1, 0xffff - bdf);
+}
+
static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
{
hwaddr base_mmio = vms->memmap[VIRT_PCIE_MMIO].base;
@@ -1162,7 +1192,7 @@ static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
}
}
- nodename = g_strdup_printf("/pcie@%" PRIx64, base);
+ nodename = vms->pciehb_nodename = g_strdup_printf("/pcie@%" PRIx64, base);
qemu_fdt_add_subnode(vms->fdt, nodename);
qemu_fdt_setprop_string(vms->fdt, nodename,
"compatible", "pci-host-ecam-generic");
@@ -1205,13 +1235,17 @@ static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
if (vms->iommu) {
vms->iommu_phandle = qemu_fdt_alloc_phandle(vms->fdt);
- create_smmu(vms, pic, pci->bus);
+ switch (vms->iommu) {
+ case VIRT_IOMMU_SMMUV3:
+ create_smmu(vms, pic, pci->bus);
+ qemu_fdt_setprop_cells(vms->fdt, nodename, "iommu-map",
+ 0x0, vms->iommu_phandle, 0x0, 0x10000);
+ break;
+ default:
+ g_assert_not_reached();
+ }
- qemu_fdt_setprop_cells(vms->fdt, nodename, "iommu-map",
- 0x0, vms->iommu_phandle, 0x0, 0x10000);
}
-
- g_free(nodename);
}
static void create_platform_bus(VirtMachineState *vms, qemu_irq *pic)
@@ -1736,12 +1770,21 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev,
SYS_BUS_DEVICE(dev));
}
}
+ if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) {
+ PCIDevice *pdev = PCI_DEVICE(dev);
+
+ vms->iommu = VIRT_IOMMU_VIRTIO;
+ vms->virtio_iommu_bdf = pci_get_bdf(pdev);
+ object_property_set_bool(OBJECT(dev), false, "msi-bypass", errp);
+ create_virtio_iommu(vms, errp);
+ }
}
static HotplugHandler *virt_machine_get_hotplug_handler(MachineState *machine,
DeviceState *dev)
{
- if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE)) {
+ if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE) ||
+ object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) {
return HOTPLUG_HANDLER(machine);
}
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 4cc57a7ef6..c330f6f4b7 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -115,7 +115,9 @@ typedef struct {
bool virt;
int32_t gic_version;
VirtIOMMUType iommu;
+ uint16_t virtio_iommu_bdf;
struct arm_boot_info bootinfo;
+ char *pciehb_nodename;
const MemMapEntry *memmap;
const int *irqmap;
int smp_cpus;
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 16/17] hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (14 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 17/17] hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table Eric Auger
2018-11-27 7:11 ` [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Bharat Bhushan
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
To avoid code duplication, let's introduce an helper that
fills one IORT ID mappings array index.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8: new
---
hw/arm/virt-acpi-build.c | 43 ++++++++++++++++++++++++----------------
1 file changed, 26 insertions(+), 17 deletions(-)
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 5785fb697c..ec7c4835fe 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -396,6 +396,17 @@ build_rsdp(GArray *rsdp_table, BIOSLinker *linker, unsigned xsdt_tbl_offset)
return rsdp_table;
}
+static inline void
+fill_iort_idmap(AcpiIortIdMapping *idmap, int i,
+ uint32_t input_base, uint32_t id_count,
+ uint32_t output_base, uint32_t output_reference)
+{
+ idmap[i].input_base = cpu_to_le32(input_base);
+ idmap[i].id_count = cpu_to_le32(id_count);
+ idmap[i].output_base = cpu_to_le32(output_base);
+ idmap[i].output_reference = cpu_to_le32(output_reference);
+}
+
static void
build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
{
@@ -453,13 +464,12 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
smmu->gerr_gsiv = cpu_to_le32(irq + 2);
smmu->sync_gsiv = cpu_to_le32(irq + 3);
- /* Identity RID mapping covering the whole input RID range */
- idmap = &smmu->id_mapping_array[0];
- idmap->input_base = 0;
- idmap->id_count = cpu_to_le32(0xFFFF);
- idmap->output_base = 0;
- /* output IORT node is the ITS group node (the first node) */
- idmap->output_reference = cpu_to_le32(iort_node_offset);
+ /*
+ * Identity RID mapping covering the whole input RID range.
+ * The output IORT node is the ITS group node (the first node).
+ */
+ fill_iort_idmap(smmu->id_mapping_array, 0, 0, 0xffff, 0,
+ iort_node_offset);
}
/* Root Complex Node */
@@ -477,18 +487,17 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
rc->memory_properties.memory_flags = 0x3; /* CCA = CPM = DCAS = 1 */
rc->pci_segment_number = 0; /* MCFG pci_segment */
- /* Identity RID mapping covering the whole input RID range */
- idmap = &rc->id_mapping_array[0];
- idmap->input_base = 0;
- idmap->id_count = cpu_to_le32(0xFFFF);
- idmap->output_base = 0;
-
if (vms->iommu == VIRT_IOMMU_SMMUV3) {
- /* output IORT node is the smmuv3 node */
- idmap->output_reference = cpu_to_le32(smmu_offset);
+ /* Identity RID mapping and output IORT node is the iommu node */
+ fill_iort_idmap(rc->id_mapping_array, 0, 0, 0xFFFF, 0,
+ smmu_offset);
} else {
- /* output IORT node is the ITS group node (the first node) */
- idmap->output_reference = cpu_to_le32(iort_node_offset);
+ /*
+ * Identity RID mapping and the output IORT node is the ITS group
+ * node (the first node).
+ */
+ fill_iort_idmap(rc->id_mapping_array, 0, 0, 0xFFFF, 0,
+ iort_node_offset);
}
/*
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [Qemu-devel] [RFC v9 17/17] hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (15 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 16/17] hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper Eric Auger
@ 2018-11-22 17:15 ` Eric Auger
2018-11-27 7:11 ` [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Bharat Bhushan
17 siblings, 0 replies; 25+ messages in thread
From: Eric Auger @ 2018-11-22 17:15 UTC (permalink / raw)
To: eric.auger.pro, eric.auger, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, bharat.bhushan, peterx
This patch builds the virtio-iommu node in the ACPI IORT table.
The RID space of the root complex, which spans 0x0-0x10000
maps to streamid space 0x0-0x10000 in the virtio-iommu which in
turn maps to deviceid space 0x0-0x10000 in the ITS group.
The iommu RID is excluded as described in virtio-iommu
specification.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
v8 -> v9:
- iommu RID is not fixed anymore
v7 -> v8:
- exclude the iommu RID (0x8) in the root complex ID mapping
---
hw/arm/virt-acpi-build.c | 50 ++++++++++++++++++++++++++++++-------
include/hw/acpi/acpi-defs.h | 21 +++++++++++++++-
2 files changed, 61 insertions(+), 10 deletions(-)
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index ec7c4835fe..0e621f6551 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -414,14 +414,14 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
AcpiIortIdMapping *idmap;
AcpiIortItsGroup *its;
AcpiIortTable *iort;
- AcpiIortSmmu3 *smmu;
- size_t node_size, iort_node_offset, iort_length, smmu_offset = 0;
+ size_t node_size, iort_node_offset, iort_length, iommu_offset = 0;
AcpiIortRC *rc;
+ int nb_rc_idmappings = 1;
iort = acpi_data_push(table_data, sizeof(*iort));
- if (vms->iommu == VIRT_IOMMU_SMMUV3) {
- nb_nodes = 3; /* RC, ITS, SMMUv3 */
+ if (vms->iommu) {
+ nb_nodes = 3; /* RC, ITS, IOMMU */
} else {
nb_nodes = 2; /* RC, ITS */
}
@@ -446,10 +446,10 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
its->identifiers[0] = 0; /* MADT translation_id */
if (vms->iommu == VIRT_IOMMU_SMMUV3) {
+ AcpiIortSmmu3 *smmu;
int irq = vms->irqmap[VIRT_SMMU];
- /* SMMUv3 node */
- smmu_offset = iort_node_offset + node_size;
+ iommu_offset = iort_node_offset + node_size;
node_size = sizeof(*smmu) + sizeof(*idmap);
iort_length += node_size;
smmu = acpi_data_push(table_data, node_size);
@@ -470,16 +470,38 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
*/
fill_iort_idmap(smmu->id_mapping_array, 0, 0, 0xffff, 0,
iort_node_offset);
+ } else if (vms->iommu == VIRT_IOMMU_VIRTIO) {
+ AcpiIortPVIommuPCI *iommu;
+
+ nb_rc_idmappings = 2;
+ iommu_offset = iort_node_offset + node_size;
+ node_size = sizeof(*iommu) + sizeof(*idmap);
+ iort_length += node_size;
+ iommu = acpi_data_push(table_data, node_size);
+
+ iommu->type = ACPI_IORT_NODE_PARAVIRT;
+ iommu->length = cpu_to_le16(node_size);
+ iommu->mapping_count = cpu_to_le32(2);
+ iommu->mapping_offset = cpu_to_le32(sizeof(*iommu));
+ iommu->devid = cpu_to_le32(vms->virtio_iommu_bdf);
+ iommu->model = cpu_to_le32(ACPI_IORT_NODE_PV_VIRTIO_IOMMU_PCI);
+
+ /*
+ * Identity RID mapping covering the whole input RID range
+ * output IORT node is the ITS group node (the first node)
+ */
+ fill_iort_idmap(iommu->id_mapping_array, 0, 0, 0xffff, 0,
+ iort_node_offset);
}
/* Root Complex Node */
- node_size = sizeof(*rc) + sizeof(*idmap);
+ node_size = sizeof(*rc) + nb_rc_idmappings * sizeof(*idmap);
iort_length += node_size;
rc = acpi_data_push(table_data, node_size);
rc->type = ACPI_IORT_NODE_PCI_ROOT_COMPLEX;
rc->length = cpu_to_le16(node_size);
- rc->mapping_count = cpu_to_le32(1);
+ rc->mapping_count = cpu_to_le32(nb_rc_idmappings);
rc->mapping_offset = cpu_to_le32(sizeof(*rc));
/* fully coherent device */
@@ -490,7 +512,17 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
if (vms->iommu == VIRT_IOMMU_SMMUV3) {
/* Identity RID mapping and output IORT node is the iommu node */
fill_iort_idmap(rc->id_mapping_array, 0, 0, 0xFFFF, 0,
- smmu_offset);
+ iommu_offset);
+ } else if (vms->iommu == VIRT_IOMMU_VIRTIO) {
+ /*
+ * Identity mapping with the IOMMU RID (0x8) excluded. The output
+ * IORT node is the iommu node.
+ */
+ fill_iort_idmap(rc->id_mapping_array, 0, 0, vms->virtio_iommu_bdf, 0,
+ iommu_offset);
+ fill_iort_idmap(rc->id_mapping_array, 1, vms->virtio_iommu_bdf + 1,
+ 0xFFFF - vms->virtio_iommu_bdf,
+ vms->virtio_iommu_bdf + 1, iommu_offset);
} else {
/*
* Identity RID mapping and the output IORT node is the ITS group
diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index af8e023968..b14aa95dc1 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -601,7 +601,8 @@ enum {
ACPI_IORT_NODE_NAMED_COMPONENT = 0x01,
ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
ACPI_IORT_NODE_SMMU = 0x03,
- ACPI_IORT_NODE_SMMU_V3 = 0x04
+ ACPI_IORT_NODE_SMMU_V3 = 0x04,
+ ACPI_IORT_NODE_PARAVIRT = 0x80
};
struct AcpiIortIdMapping {
@@ -628,6 +629,24 @@ struct AcpiIortItsGroup {
} QEMU_PACKED;
typedef struct AcpiIortItsGroup AcpiIortItsGroup;
+struct AcpiIortPVIommuPCI {
+ ACPI_IORT_NODE_HEADER_DEF
+ uint32_t devid;
+ uint8_t reserved2[12];
+ uint32_t model;
+ uint32_t flags;
+ uint8_t reserved3[16];
+ AcpiIortIdMapping id_mapping_array[0];
+} QEMU_PACKED;
+typedef struct AcpiIortPVIommuPCI AcpiIortPVIommuPCI;
+
+enum {
+ ACPI_IORT_NODE_PV_VIRTIO_IOMMU = 0x0,
+ ACPI_IORT_NODE_PV_VIRTIO_IOMMU_PCI = 0x1,
+};
+
+#define ACPI_IORT_NODE_PV_CACHE_COHERENT (1 << 0)
+
struct AcpiIortSmmu3 {
ACPI_IORT_NODE_HEADER_DEF
uint64_t base_address;
--
2.17.2
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers Eric Auger
@ 2018-11-23 6:38 ` Bharat Bhushan
2018-11-23 7:53 ` Auger Eric
0 siblings, 1 reply; 25+ messages in thread
From: Bharat Bhushan @ 2018-11-23 6:38 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, peterx
Hi Eric,
> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: Thursday, November 22, 2018 10:45 PM
> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
> mst@redhat.com; jean-philippe.brucker@arm.com
> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
> <bharat.bhushan@nxp.com>; peterx@redhat.com
> Subject: [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and
> helpers
>
> This patch introduce domain and endpoint internal datatypes. Both are
> stored in RB trees. The domain owns a list of endpoints attached to it.
>
> Helpers to get/put end points and domains are introduced.
> get() helpers will become static in subsequent patches.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v6 -> v7:
> - on virtio_iommu_find_add_as the bus number computation may
> not be finalized yet so we cannot register the EPs at that time.
> Hence, let's remove the get_endpoint and also do not use the
> bus number for building the memory region name string (only
> used for debug though).
Endpoint registration from virtio_iommu_find_add_as to PROBE request.
It is mentioned that " the bus number computation may not be finalized ". Can you please give some more information.
I am asking this because from vfio perspective translate/replay will be called much before the PROBE request and endpoint needed to be registered by that time.
Thanks
-Bharat
>
> v4 -> v5:
> - initialize as->endpoint_list
>
> v3 -> v4:
> - new separate patch
> ---
> hw/virtio/trace-events | 4 ++
> hw/virtio/virtio-iommu.c | 125
> ++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 128 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index
> 9270b0463e..4b15086872 100644
> --- a/hw/virtio/trace-events
> +++ b/hw/virtio/trace-events
> @@ -61,3 +61,7 @@ virtio_iommu_map(uint32_t domain_id, uint64_t
> virt_start, uint64_t virt_end, uin virtio_iommu_unmap(uint32_t domain_id,
> uint64_t virt_start, uint64_t virt_end) "domain=%d virt_start=0x%"PRIx64"
> virt_end=0x%"PRIx64 virtio_iommu_translate(const char *name, uint32_t
> rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64" flag=%d"
> virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
> +virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
> +virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
> +virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
> +virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c index
> dead062baf..1b9c3ba416 100644
> --- a/hw/virtio/virtio-iommu.c
> +++ b/hw/virtio/virtio-iommu.c
> @@ -33,20 +33,124 @@
> #include "hw/virtio/virtio-bus.h"
> #include "hw/virtio/virtio-access.h"
> #include "hw/virtio/virtio-iommu.h"
> +#include "hw/pci/pci_bus.h"
> +#include "hw/pci/pci.h"
>
> /* Max size */
> #define VIOMMU_DEFAULT_QUEUE_SIZE 256
>
> +typedef struct viommu_domain {
> + uint32_t id;
> + GTree *mappings;
> + QLIST_HEAD(, viommu_endpoint) endpoint_list; } viommu_domain;
> +
> +typedef struct viommu_endpoint {
> + uint32_t id;
> + viommu_domain *domain;
> + QLIST_ENTRY(viommu_endpoint) next;
> + VirtIOIOMMU *viommu;
> +} viommu_endpoint;
> +
> +typedef struct viommu_interval {
> + uint64_t low;
> + uint64_t high;
> +} viommu_interval;
> +
> static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev) {
> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn); }
>
> +static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer
> +user_data) {
> + viommu_interval *inta = (viommu_interval *)a;
> + viommu_interval *intb = (viommu_interval *)b;
> +
> + if (inta->high <= intb->low) {
> + return -1;
> + } else if (intb->high <= inta->low) {
> + return 1;
> + } else {
> + return 0;
> + }
> +}
> +
> +static void
> virtio_iommu_detach_endpoint_from_domain(viommu_endpoint
> +*ep) {
> + QLIST_REMOVE(ep, next);
> + ep->domain = NULL;
> +}
> +
> +viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
> uint32_t
> +ep_id); viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
> +uint32_t ep_id) {
> + viommu_endpoint *ep;
> +
> + ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
> + if (ep) {
> + return ep;
> + }
> + ep = g_malloc0(sizeof(*ep));
> + ep->id = ep_id;
> + ep->viommu = s;
> + trace_virtio_iommu_get_endpoint(ep_id);
> + g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
> + return ep;
> +}
> +
> +static void virtio_iommu_put_endpoint(gpointer data) {
> + viommu_endpoint *ep = (viommu_endpoint *)data;
> +
> + if (ep->domain) {
> + virtio_iommu_detach_endpoint_from_domain(ep);
> + g_tree_unref(ep->domain->mappings);
> + }
> +
> + trace_virtio_iommu_put_endpoint(ep->id);
> + g_free(ep);
> +}
> +
> +viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t
> +domain_id); viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU
> *s,
> +uint32_t domain_id) {
> + viommu_domain *domain;
> +
> + domain = g_tree_lookup(s->domains, GUINT_TO_POINTER(domain_id));
> + if (domain) {
> + return domain;
> + }
> + domain = g_malloc0(sizeof(*domain));
> + domain->id = domain_id;
> + domain->mappings =
> g_tree_new_full((GCompareDataFunc)interval_cmp,
> + NULL, (GDestroyNotify)g_free,
> + (GDestroyNotify)g_free);
> + g_tree_insert(s->domains, GUINT_TO_POINTER(domain_id), domain);
> + QLIST_INIT(&domain->endpoint_list);
> + trace_virtio_iommu_get_domain(domain_id);
> + return domain;
> +}
> +
> +static void virtio_iommu_put_domain(gpointer data) {
> + viommu_domain *domain = (viommu_domain *)data;
> + viommu_endpoint *iter, *tmp;
> +
> + QLIST_FOREACH_SAFE(iter, &domain->endpoint_list, next, tmp) {
> + virtio_iommu_detach_endpoint_from_domain(iter);
> + }
> + g_tree_destroy(domain->mappings);
> + trace_virtio_iommu_put_domain(domain->id);
> + g_free(domain);
> +}
> +
> static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void
> *opaque,
> int devfn) {
> VirtIOIOMMU *s = opaque;
> IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
> + static uint32_t mr_index;
> IOMMUDevice *sdev;
>
> if (!sbus) {
> @@ -60,7 +164,7 @@ static AddressSpace
> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
> if (!sdev) {
> char *name = g_strdup_printf("%s-%d-%d",
> TYPE_VIRTIO_IOMMU_MEMORY_REGION,
> - pci_bus_num(bus), devfn);
> + mr_index++, devfn);
> sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
>
> sdev->viommu = s;
> @@ -75,6 +179,7 @@ static AddressSpace
> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
> UINT64_MAX);
> address_space_init(&sdev->as,
> MEMORY_REGION(&sdev->iommu_mr),
> TYPE_VIRTIO_IOMMU);
> + g_free(name);
> }
>
> return &sdev->as;
> @@ -332,6 +437,13 @@ static const VMStateDescription
> vmstate_virtio_iommu_device = {
> },
> };
>
> +static gint int_cmp(gconstpointer a, gconstpointer b, gpointer
> +user_data) {
> + uint ua = GPOINTER_TO_UINT(a);
> + uint ub = GPOINTER_TO_UINT(b);
> + return (ua > ub) - (ua < ub);
> +}
> +
> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp) {
> VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -356,6 +468,8 @@ static
> void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
>
> + qemu_mutex_init(&s->mutex);
> +
> memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
> s->as_by_busptr = g_hash_table_new(NULL, NULL);
>
> @@ -364,11 +478,20 @@ static void
> virtio_iommu_device_realize(DeviceState *dev, Error **errp)
> } else {
> error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
> }
> +
> + s->domains = g_tree_new_full((GCompareDataFunc)int_cmp,
> + NULL, NULL, virtio_iommu_put_domain);
> + s->endpoints = g_tree_new_full((GCompareDataFunc)int_cmp,
> + NULL, NULL,
> + virtio_iommu_put_endpoint);
> }
>
> static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)
> {
> VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> + VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
> +
> + g_tree_destroy(s->domains);
> + g_tree_destroy(s->endpoints);
>
> virtio_cleanup(vdev);
> }
> --
> 2.17.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers
2018-11-23 6:38 ` Bharat Bhushan
@ 2018-11-23 7:53 ` Auger Eric
2018-11-23 9:14 ` Bharat Bhushan
0 siblings, 1 reply; 25+ messages in thread
From: Auger Eric @ 2018-11-23 7:53 UTC (permalink / raw)
To: Bharat Bhushan, eric.auger.pro, qemu-devel, qemu-arm,
peter.maydell, mst, jean-philippe.brucker
Cc: tn, kevin.tian, peterx
Hi Bharat,
On 11/23/18 7:38 AM, Bharat Bhushan wrote:
> Hi Eric,
>
>> -----Original Message-----
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: Thursday, November 22, 2018 10:45 PM
>> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
>> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
>> mst@redhat.com; jean-philippe.brucker@arm.com
>> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
>> <bharat.bhushan@nxp.com>; peterx@redhat.com
>> Subject: [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and
>> helpers
>>
>> This patch introduce domain and endpoint internal datatypes. Both are
>> stored in RB trees. The domain owns a list of endpoints attached to it.
>>
>> Helpers to get/put end points and domains are introduced.
>> get() helpers will become static in subsequent patches.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v6 -> v7:
>> - on virtio_iommu_find_add_as the bus number computation may
>> not be finalized yet so we cannot register the EPs at that time.
>> Hence, let's remove the get_endpoint and also do not use the
>> bus number for building the memory region name string (only
>> used for debug though).
>
> Endpoint registration from virtio_iommu_find_add_as to PROBE request.
> It is mentioned that " the bus number computation may not be finalized ". Can you please give some more information.
> I am asking this because from vfio perspective translate/replay will be called much before the PROBE request and endpoint needed to be registered by that time.
When from virtio_iommu_find_add() gets called, there are cases where the
BDF of the device is not yet computed, typically if the EP is plugged on
a secondary bus. That's why I postponed the registration. Do you have
idea When you would need the registration to happen?
Thanks
Eric
>
>
> Thanks
> -Bharat
>
>>
>> v4 -> v5:
>> - initialize as->endpoint_list
>>
>> v3 -> v4:
>> - new separate patch
>> ---
>> hw/virtio/trace-events | 4 ++
>> hw/virtio/virtio-iommu.c | 125
>> ++++++++++++++++++++++++++++++++++++++-
>> 2 files changed, 128 insertions(+), 1 deletion(-)
>>
>> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index
>> 9270b0463e..4b15086872 100644
>> --- a/hw/virtio/trace-events
>> +++ b/hw/virtio/trace-events
>> @@ -61,3 +61,7 @@ virtio_iommu_map(uint32_t domain_id, uint64_t
>> virt_start, uint64_t virt_end, uin virtio_iommu_unmap(uint32_t domain_id,
>> uint64_t virt_start, uint64_t virt_end) "domain=%d virt_start=0x%"PRIx64"
>> virt_end=0x%"PRIx64 virtio_iommu_translate(const char *name, uint32_t
>> rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64" flag=%d"
>> virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
>> +virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
>> +virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
>> +virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
>> +virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
>> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c index
>> dead062baf..1b9c3ba416 100644
>> --- a/hw/virtio/virtio-iommu.c
>> +++ b/hw/virtio/virtio-iommu.c
>> @@ -33,20 +33,124 @@
>> #include "hw/virtio/virtio-bus.h"
>> #include "hw/virtio/virtio-access.h"
>> #include "hw/virtio/virtio-iommu.h"
>> +#include "hw/pci/pci_bus.h"
>> +#include "hw/pci/pci.h"
>>
>> /* Max size */
>> #define VIOMMU_DEFAULT_QUEUE_SIZE 256
>>
>> +typedef struct viommu_domain {
>> + uint32_t id;
>> + GTree *mappings;
>> + QLIST_HEAD(, viommu_endpoint) endpoint_list; } viommu_domain;
>> +
>> +typedef struct viommu_endpoint {
>> + uint32_t id;
>> + viommu_domain *domain;
>> + QLIST_ENTRY(viommu_endpoint) next;
>> + VirtIOIOMMU *viommu;
>> +} viommu_endpoint;
>> +
>> +typedef struct viommu_interval {
>> + uint64_t low;
>> + uint64_t high;
>> +} viommu_interval;
>> +
>> static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev) {
>> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn); }
>>
>> +static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer
>> +user_data) {
>> + viommu_interval *inta = (viommu_interval *)a;
>> + viommu_interval *intb = (viommu_interval *)b;
>> +
>> + if (inta->high <= intb->low) {
>> + return -1;
>> + } else if (intb->high <= inta->low) {
>> + return 1;
>> + } else {
>> + return 0;
>> + }
>> +}
>> +
>> +static void
>> virtio_iommu_detach_endpoint_from_domain(viommu_endpoint
>> +*ep) {
>> + QLIST_REMOVE(ep, next);
>> + ep->domain = NULL;
>> +}
>> +
>> +viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
>> uint32_t
>> +ep_id); viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
>> +uint32_t ep_id) {
>> + viommu_endpoint *ep;
>> +
>> + ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
>> + if (ep) {
>> + return ep;
>> + }
>> + ep = g_malloc0(sizeof(*ep));
>> + ep->id = ep_id;
>> + ep->viommu = s;
>> + trace_virtio_iommu_get_endpoint(ep_id);
>> + g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
>> + return ep;
>> +}
>> +
>> +static void virtio_iommu_put_endpoint(gpointer data) {
>> + viommu_endpoint *ep = (viommu_endpoint *)data;
>> +
>> + if (ep->domain) {
>> + virtio_iommu_detach_endpoint_from_domain(ep);
>> + g_tree_unref(ep->domain->mappings);
>> + }
>> +
>> + trace_virtio_iommu_put_endpoint(ep->id);
>> + g_free(ep);
>> +}
>> +
>> +viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s, uint32_t
>> +domain_id); viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU
>> *s,
>> +uint32_t domain_id) {
>> + viommu_domain *domain;
>> +
>> + domain = g_tree_lookup(s->domains, GUINT_TO_POINTER(domain_id));
>> + if (domain) {
>> + return domain;
>> + }
>> + domain = g_malloc0(sizeof(*domain));
>> + domain->id = domain_id;
>> + domain->mappings =
>> g_tree_new_full((GCompareDataFunc)interval_cmp,
>> + NULL, (GDestroyNotify)g_free,
>> + (GDestroyNotify)g_free);
>> + g_tree_insert(s->domains, GUINT_TO_POINTER(domain_id), domain);
>> + QLIST_INIT(&domain->endpoint_list);
>> + trace_virtio_iommu_get_domain(domain_id);
>> + return domain;
>> +}
>> +
>> +static void virtio_iommu_put_domain(gpointer data) {
>> + viommu_domain *domain = (viommu_domain *)data;
>> + viommu_endpoint *iter, *tmp;
>> +
>> + QLIST_FOREACH_SAFE(iter, &domain->endpoint_list, next, tmp) {
>> + virtio_iommu_detach_endpoint_from_domain(iter);
>> + }
>> + g_tree_destroy(domain->mappings);
>> + trace_virtio_iommu_put_domain(domain->id);
>> + g_free(domain);
>> +}
>> +
>> static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void
>> *opaque,
>> int devfn) {
>> VirtIOIOMMU *s = opaque;
>> IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
>> + static uint32_t mr_index;
>> IOMMUDevice *sdev;
>>
>> if (!sbus) {
>> @@ -60,7 +164,7 @@ static AddressSpace
>> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
>> if (!sdev) {
>> char *name = g_strdup_printf("%s-%d-%d",
>> TYPE_VIRTIO_IOMMU_MEMORY_REGION,
>> - pci_bus_num(bus), devfn);
>> + mr_index++, devfn);
>> sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
>>
>> sdev->viommu = s;
>> @@ -75,6 +179,7 @@ static AddressSpace
>> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
>> UINT64_MAX);
>> address_space_init(&sdev->as,
>> MEMORY_REGION(&sdev->iommu_mr),
>> TYPE_VIRTIO_IOMMU);
>> + g_free(name);
>> }
>>
>> return &sdev->as;
>> @@ -332,6 +437,13 @@ static const VMStateDescription
>> vmstate_virtio_iommu_device = {
>> },
>> };
>>
>> +static gint int_cmp(gconstpointer a, gconstpointer b, gpointer
>> +user_data) {
>> + uint ua = GPOINTER_TO_UINT(a);
>> + uint ub = GPOINTER_TO_UINT(b);
>> + return (ua > ub) - (ua < ub);
>> +}
>> +
>> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp) {
>> VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -356,6 +468,8 @@ static
>> void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
>>
>> + qemu_mutex_init(&s->mutex);
>> +
>> memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
>> s->as_by_busptr = g_hash_table_new(NULL, NULL);
>>
>> @@ -364,11 +478,20 @@ static void
>> virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>> } else {
>> error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
>> }
>> +
>> + s->domains = g_tree_new_full((GCompareDataFunc)int_cmp,
>> + NULL, NULL, virtio_iommu_put_domain);
>> + s->endpoints = g_tree_new_full((GCompareDataFunc)int_cmp,
>> + NULL, NULL,
>> + virtio_iommu_put_endpoint);
>> }
>>
>> static void virtio_iommu_device_unrealize(DeviceState *dev, Error **errp)
>> {
>> VirtIODevice *vdev = VIRTIO_DEVICE(dev);
>> + VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
>> +
>> + g_tree_destroy(s->domains);
>> + g_tree_destroy(s->endpoints);
>>
>> virtio_cleanup(vdev);
>> }
>> --
>> 2.17.2
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers
2018-11-23 7:53 ` Auger Eric
@ 2018-11-23 9:14 ` Bharat Bhushan
2018-11-23 9:26 ` Auger Eric
0 siblings, 1 reply; 25+ messages in thread
From: Bharat Bhushan @ 2018-11-23 9:14 UTC (permalink / raw)
To: Auger Eric, eric.auger.pro, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: tn, kevin.tian, peterx
Hi Eric,
> -----Original Message-----
> From: Auger Eric <eric.auger@redhat.com>
> Sent: Friday, November 23, 2018 1:23 PM
> To: Bharat Bhushan <bharat.bhushan@nxp.com>;
> eric.auger.pro@gmail.com; qemu-devel@nongnu.org; qemu-
> arm@nongnu.org; peter.maydell@linaro.org; mst@redhat.com; jean-
> philippe.brucker@arm.com
> Cc: tn@semihalf.com; kevin.tian@intel.com; peterx@redhat.com
> Subject: Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and
> domains structs and helpers
>
> Hi Bharat,
>
> On 11/23/18 7:38 AM, Bharat Bhushan wrote:
> > Hi Eric,
> >
> >> -----Original Message-----
> >> From: Eric Auger <eric.auger@redhat.com>
> >> Sent: Thursday, November 22, 2018 10:45 PM
> >> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
> >> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
> >> mst@redhat.com; jean-philippe.brucker@arm.com
> >> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
> >> <bharat.bhushan@nxp.com>; peterx@redhat.com
> >> Subject: [RFC v9 06/17] virtio-iommu: Endpoint and domains structs
> >> and helpers
> >>
> >> This patch introduce domain and endpoint internal datatypes. Both are
> >> stored in RB trees. The domain owns a list of endpoints attached to it.
> >>
> >> Helpers to get/put end points and domains are introduced.
> >> get() helpers will become static in subsequent patches.
> >>
> >> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> >>
> >> ---
> >>
> >> v6 -> v7:
> >> - on virtio_iommu_find_add_as the bus number computation may
> >> not be finalized yet so we cannot register the EPs at that time.
> >> Hence, let's remove the get_endpoint and also do not use the
> >> bus number for building the memory region name string (only
> >> used for debug though).
> >
> > Endpoint registration from virtio_iommu_find_add_as to PROBE request.
> > It is mentioned that " the bus number computation may not be finalized ".
> Can you please give some more information.
> > I am asking this because from vfio perspective translate/replay will be
> called much before the PROBE request and endpoint needed to be
> registered by that time.
> When from virtio_iommu_find_add() gets called, there are cases where the
> BDF of the device is not yet computed, typically if the EP is plugged on a
> secondary bus. That's why I postponed the registration. Do you have idea
> When you would need the registration to happen?
We want the endpoint registeration before replay/translate() is called for both virtio/vfio and I am trying to understand when we should register the endpoint.
I am looking at amd iommu, there pci_setup_iommu() provides the callback function which is called with "devfn" from pci_device_iommu_address_space(). Are you saying that devfn provided by pci_device_iommu_address_space() can be invalid?
Thanks
-Bharat
>
> Thanks
>
> Eric
> >
> >
> > Thanks
> > -Bharat
> >
> >>
> >> v4 -> v5:
> >> - initialize as->endpoint_list
> >>
> >> v3 -> v4:
> >> - new separate patch
> >> ---
> >> hw/virtio/trace-events | 4 ++
> >> hw/virtio/virtio-iommu.c | 125
> >> ++++++++++++++++++++++++++++++++++++++-
> >> 2 files changed, 128 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index
> >> 9270b0463e..4b15086872 100644
> >> --- a/hw/virtio/trace-events
> >> +++ b/hw/virtio/trace-events
> >> @@ -61,3 +61,7 @@ virtio_iommu_map(uint32_t domain_id, uint64_t
> >> virt_start, uint64_t virt_end, uin virtio_iommu_unmap(uint32_t
> >> domain_id, uint64_t virt_start, uint64_t virt_end) "domain=%d
> virt_start=0x%"PRIx64"
> >> virt_end=0x%"PRIx64 virtio_iommu_translate(const char *name,
> >> uint32_t rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64"
> flag=%d"
> >> virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
> >> +virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
> >> +virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
> >> +virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
> >> +virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
> >> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
> >> index
> >> dead062baf..1b9c3ba416 100644
> >> --- a/hw/virtio/virtio-iommu.c
> >> +++ b/hw/virtio/virtio-iommu.c
> >> @@ -33,20 +33,124 @@
> >> #include "hw/virtio/virtio-bus.h"
> >> #include "hw/virtio/virtio-access.h"
> >> #include "hw/virtio/virtio-iommu.h"
> >> +#include "hw/pci/pci_bus.h"
> >> +#include "hw/pci/pci.h"
> >>
> >> /* Max size */
> >> #define VIOMMU_DEFAULT_QUEUE_SIZE 256
> >>
> >> +typedef struct viommu_domain {
> >> + uint32_t id;
> >> + GTree *mappings;
> >> + QLIST_HEAD(, viommu_endpoint) endpoint_list; } viommu_domain;
> >> +
> >> +typedef struct viommu_endpoint {
> >> + uint32_t id;
> >> + viommu_domain *domain;
> >> + QLIST_ENTRY(viommu_endpoint) next;
> >> + VirtIOIOMMU *viommu;
> >> +} viommu_endpoint;
> >> +
> >> +typedef struct viommu_interval {
> >> + uint64_t low;
> >> + uint64_t high;
> >> +} viommu_interval;
> >> +
> >> static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev) {
> >> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn); }
> >>
> >> +static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer
> >> +user_data) {
> >> + viommu_interval *inta = (viommu_interval *)a;
> >> + viommu_interval *intb = (viommu_interval *)b;
> >> +
> >> + if (inta->high <= intb->low) {
> >> + return -1;
> >> + } else if (intb->high <= inta->low) {
> >> + return 1;
> >> + } else {
> >> + return 0;
> >> + }
> >> +}
> >> +
> >> +static void
> >> virtio_iommu_detach_endpoint_from_domain(viommu_endpoint
> >> +*ep) {
> >> + QLIST_REMOVE(ep, next);
> >> + ep->domain = NULL;
> >> +}
> >> +
> >> +viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
> >> uint32_t
> >> +ep_id); viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU
> *s,
> >> +uint32_t ep_id) {
> >> + viommu_endpoint *ep;
> >> +
> >> + ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
> >> + if (ep) {
> >> + return ep;
> >> + }
> >> + ep = g_malloc0(sizeof(*ep));
> >> + ep->id = ep_id;
> >> + ep->viommu = s;
> >> + trace_virtio_iommu_get_endpoint(ep_id);
> >> + g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
> >> + return ep;
> >> +}
> >> +
> >> +static void virtio_iommu_put_endpoint(gpointer data) {
> >> + viommu_endpoint *ep = (viommu_endpoint *)data;
> >> +
> >> + if (ep->domain) {
> >> + virtio_iommu_detach_endpoint_from_domain(ep);
> >> + g_tree_unref(ep->domain->mappings);
> >> + }
> >> +
> >> + trace_virtio_iommu_put_endpoint(ep->id);
> >> + g_free(ep);
> >> +}
> >> +
> >> +viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s,
> uint32_t
> >> +domain_id); viommu_domain
> *virtio_iommu_get_domain(VirtIOIOMMU
> >> *s,
> >> +uint32_t domain_id) {
> >> + viommu_domain *domain;
> >> +
> >> + domain = g_tree_lookup(s->domains,
> GUINT_TO_POINTER(domain_id));
> >> + if (domain) {
> >> + return domain;
> >> + }
> >> + domain = g_malloc0(sizeof(*domain));
> >> + domain->id = domain_id;
> >> + domain->mappings =
> >> g_tree_new_full((GCompareDataFunc)interval_cmp,
> >> + NULL, (GDestroyNotify)g_free,
> >> + (GDestroyNotify)g_free);
> >> + g_tree_insert(s->domains, GUINT_TO_POINTER(domain_id),
> domain);
> >> + QLIST_INIT(&domain->endpoint_list);
> >> + trace_virtio_iommu_get_domain(domain_id);
> >> + return domain;
> >> +}
> >> +
> >> +static void virtio_iommu_put_domain(gpointer data) {
> >> + viommu_domain *domain = (viommu_domain *)data;
> >> + viommu_endpoint *iter, *tmp;
> >> +
> >> + QLIST_FOREACH_SAFE(iter, &domain->endpoint_list, next, tmp) {
> >> + virtio_iommu_detach_endpoint_from_domain(iter);
> >> + }
> >> + g_tree_destroy(domain->mappings);
> >> + trace_virtio_iommu_put_domain(domain->id);
> >> + g_free(domain);
> >> +}
> >> +
> >> static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void
> >> *opaque,
> >> int devfn) {
> >> VirtIOIOMMU *s = opaque;
> >> IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
> >> + static uint32_t mr_index;
> >> IOMMUDevice *sdev;
> >>
> >> if (!sbus) {
> >> @@ -60,7 +164,7 @@ static AddressSpace
> >> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
> >> if (!sdev) {
> >> char *name = g_strdup_printf("%s-%d-%d",
> >> TYPE_VIRTIO_IOMMU_MEMORY_REGION,
> >> - pci_bus_num(bus), devfn);
> >> + mr_index++, devfn);
> >> sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
> >>
> >> sdev->viommu = s;
> >> @@ -75,6 +179,7 @@ static AddressSpace
> >> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
> >> UINT64_MAX);
> >> address_space_init(&sdev->as,
> >> MEMORY_REGION(&sdev->iommu_mr),
> >> TYPE_VIRTIO_IOMMU);
> >> + g_free(name);
> >> }
> >>
> >> return &sdev->as;
> >> @@ -332,6 +437,13 @@ static const VMStateDescription
> >> vmstate_virtio_iommu_device = {
> >> },
> >> };
> >>
> >> +static gint int_cmp(gconstpointer a, gconstpointer b, gpointer
> >> +user_data) {
> >> + uint ua = GPOINTER_TO_UINT(a);
> >> + uint ub = GPOINTER_TO_UINT(b);
> >> + return (ua > ub) - (ua < ub);
> >> +}
> >> +
> >> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
> {
> >> VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -356,6 +468,8 @@
> >> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
> >> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
> >> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
> >>
> >> + qemu_mutex_init(&s->mutex);
> >> +
> >> memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
> >> s->as_by_busptr = g_hash_table_new(NULL, NULL);
> >>
> >> @@ -364,11 +478,20 @@ static void
> >> virtio_iommu_device_realize(DeviceState *dev, Error **errp)
> >> } else {
> >> error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
> >> }
> >> +
> >> + s->domains = g_tree_new_full((GCompareDataFunc)int_cmp,
> >> + NULL, NULL, virtio_iommu_put_domain);
> >> + s->endpoints = g_tree_new_full((GCompareDataFunc)int_cmp,
> >> + NULL, NULL,
> >> + virtio_iommu_put_endpoint);
> >> }
> >>
> >> static void virtio_iommu_device_unrealize(DeviceState *dev, Error
> >> **errp) {
> >> VirtIODevice *vdev = VIRTIO_DEVICE(dev);
> >> + VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
> >> +
> >> + g_tree_destroy(s->domains);
> >> + g_tree_destroy(s->endpoints);
> >>
> >> virtio_cleanup(vdev);
> >> }
> >> --
> >> 2.17.2
> >
> >
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers
2018-11-23 9:14 ` Bharat Bhushan
@ 2018-11-23 9:26 ` Auger Eric
0 siblings, 0 replies; 25+ messages in thread
From: Auger Eric @ 2018-11-23 9:26 UTC (permalink / raw)
To: Bharat Bhushan, eric.auger.pro, qemu-devel, qemu-arm,
peter.maydell, mst, jean-philippe.brucker
Cc: tn, kevin.tian, peterx
Hi Bharat,
On 11/23/18 10:14 AM, Bharat Bhushan wrote:
> Hi Eric,
>
>> -----Original Message-----
>> From: Auger Eric <eric.auger@redhat.com>
>> Sent: Friday, November 23, 2018 1:23 PM
>> To: Bharat Bhushan <bharat.bhushan@nxp.com>;
>> eric.auger.pro@gmail.com; qemu-devel@nongnu.org; qemu-
>> arm@nongnu.org; peter.maydell@linaro.org; mst@redhat.com; jean-
>> philippe.brucker@arm.com
>> Cc: tn@semihalf.com; kevin.tian@intel.com; peterx@redhat.com
>> Subject: Re: [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and
>> domains structs and helpers
>>
>> Hi Bharat,
>>
>> On 11/23/18 7:38 AM, Bharat Bhushan wrote:
>>> Hi Eric,
>>>
>>>> -----Original Message-----
>>>> From: Eric Auger <eric.auger@redhat.com>
>>>> Sent: Thursday, November 22, 2018 10:45 PM
>>>> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
>>>> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
>>>> mst@redhat.com; jean-philippe.brucker@arm.com
>>>> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
>>>> <bharat.bhushan@nxp.com>; peterx@redhat.com
>>>> Subject: [RFC v9 06/17] virtio-iommu: Endpoint and domains structs
>>>> and helpers
>>>>
>>>> This patch introduce domain and endpoint internal datatypes. Both are
>>>> stored in RB trees. The domain owns a list of endpoints attached to it.
>>>>
>>>> Helpers to get/put end points and domains are introduced.
>>>> get() helpers will become static in subsequent patches.
>>>>
>>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>>>
>>>> ---
>>>>
>>>> v6 -> v7:
>>>> - on virtio_iommu_find_add_as the bus number computation may
>>>> not be finalized yet so we cannot register the EPs at that time.
>>>> Hence, let's remove the get_endpoint and also do not use the
>>>> bus number for building the memory region name string (only
>>>> used for debug though).
>>>
>>> Endpoint registration from virtio_iommu_find_add_as to PROBE request.
>>> It is mentioned that " the bus number computation may not be finalized ".
>> Can you please give some more information.
>>> I am asking this because from vfio perspective translate/replay will be
>> called much before the PROBE request and endpoint needed to be
>> registered by that time.
>> When from virtio_iommu_find_add() gets called, there are cases where the
>> BDF of the device is not yet computed, typically if the EP is plugged on a
>> secondary bus. That's why I postponed the registration. Do you have idea
>> When you would need the registration to happen?
>
> We want the endpoint registeration before replay/translate() is called for both virtio/vfio and I am trying to understand when we should register the endpoint.
> I am looking at amd iommu, there pci_setup_iommu() provides the callback function which is called with "devfn" from pci_device_iommu_address_space(). Are you saying that devfn provided by pci_device_iommu_address_space() can be invalid?
pci_bus_num() can return something wrong if called from
virtio_iommu_find_add_as.
That's what on smmuv3 (same on Intel), we use a separate
smmu_find_smmu_pcibus() called later. See comment for smmu_find_smmu_pcibus.
Thanks
Eric
>
> Thanks
> -Bharat
>
>>
>> Thanks
>>
>> Eric
>>>
>>>
>>> Thanks
>>> -Bharat
>>>
>>>>
>>>> v4 -> v5:
>>>> - initialize as->endpoint_list
>>>>
>>>> v3 -> v4:
>>>> - new separate patch
>>>> ---
>>>> hw/virtio/trace-events | 4 ++
>>>> hw/virtio/virtio-iommu.c | 125
>>>> ++++++++++++++++++++++++++++++++++++++-
>>>> 2 files changed, 128 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index
>>>> 9270b0463e..4b15086872 100644
>>>> --- a/hw/virtio/trace-events
>>>> +++ b/hw/virtio/trace-events
>>>> @@ -61,3 +61,7 @@ virtio_iommu_map(uint32_t domain_id, uint64_t
>>>> virt_start, uint64_t virt_end, uin virtio_iommu_unmap(uint32_t
>>>> domain_id, uint64_t virt_start, uint64_t virt_end) "domain=%d
>> virt_start=0x%"PRIx64"
>>>> virt_end=0x%"PRIx64 virtio_iommu_translate(const char *name,
>>>> uint32_t rid, uint64_t iova, int flag) "mr=%s rid=%d addr=0x%"PRIx64"
>> flag=%d"
>>>> virtio_iommu_init_iommu_mr(char *iommu_mr) "init %s"
>>>> +virtio_iommu_get_endpoint(uint32_t ep_id) "Alloc endpoint=%d"
>>>> +virtio_iommu_put_endpoint(uint32_t ep_id) "Free endpoint=%d"
>>>> +virtio_iommu_get_domain(uint32_t domain_id) "Alloc domain=%d"
>>>> +virtio_iommu_put_domain(uint32_t domain_id) "Free domain=%d"
>>>> diff --git a/hw/virtio/virtio-iommu.c b/hw/virtio/virtio-iommu.c
>>>> index
>>>> dead062baf..1b9c3ba416 100644
>>>> --- a/hw/virtio/virtio-iommu.c
>>>> +++ b/hw/virtio/virtio-iommu.c
>>>> @@ -33,20 +33,124 @@
>>>> #include "hw/virtio/virtio-bus.h"
>>>> #include "hw/virtio/virtio-access.h"
>>>> #include "hw/virtio/virtio-iommu.h"
>>>> +#include "hw/pci/pci_bus.h"
>>>> +#include "hw/pci/pci.h"
>>>>
>>>> /* Max size */
>>>> #define VIOMMU_DEFAULT_QUEUE_SIZE 256
>>>>
>>>> +typedef struct viommu_domain {
>>>> + uint32_t id;
>>>> + GTree *mappings;
>>>> + QLIST_HEAD(, viommu_endpoint) endpoint_list; } viommu_domain;
>>>> +
>>>> +typedef struct viommu_endpoint {
>>>> + uint32_t id;
>>>> + viommu_domain *domain;
>>>> + QLIST_ENTRY(viommu_endpoint) next;
>>>> + VirtIOIOMMU *viommu;
>>>> +} viommu_endpoint;
>>>> +
>>>> +typedef struct viommu_interval {
>>>> + uint64_t low;
>>>> + uint64_t high;
>>>> +} viommu_interval;
>>>> +
>>>> static inline uint16_t virtio_iommu_get_sid(IOMMUDevice *dev) {
>>>> return PCI_BUILD_BDF(pci_bus_num(dev->bus), dev->devfn); }
>>>>
>>>> +static gint interval_cmp(gconstpointer a, gconstpointer b, gpointer
>>>> +user_data) {
>>>> + viommu_interval *inta = (viommu_interval *)a;
>>>> + viommu_interval *intb = (viommu_interval *)b;
>>>> +
>>>> + if (inta->high <= intb->low) {
>>>> + return -1;
>>>> + } else if (intb->high <= inta->low) {
>>>> + return 1;
>>>> + } else {
>>>> + return 0;
>>>> + }
>>>> +}
>>>> +
>>>> +static void
>>>> virtio_iommu_detach_endpoint_from_domain(viommu_endpoint
>>>> +*ep) {
>>>> + QLIST_REMOVE(ep, next);
>>>> + ep->domain = NULL;
>>>> +}
>>>> +
>>>> +viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU *s,
>>>> uint32_t
>>>> +ep_id); viommu_endpoint *virtio_iommu_get_endpoint(VirtIOIOMMU
>> *s,
>>>> +uint32_t ep_id) {
>>>> + viommu_endpoint *ep;
>>>> +
>>>> + ep = g_tree_lookup(s->endpoints, GUINT_TO_POINTER(ep_id));
>>>> + if (ep) {
>>>> + return ep;
>>>> + }
>>>> + ep = g_malloc0(sizeof(*ep));
>>>> + ep->id = ep_id;
>>>> + ep->viommu = s;
>>>> + trace_virtio_iommu_get_endpoint(ep_id);
>>>> + g_tree_insert(s->endpoints, GUINT_TO_POINTER(ep_id), ep);
>>>> + return ep;
>>>> +}
>>>> +
>>>> +static void virtio_iommu_put_endpoint(gpointer data) {
>>>> + viommu_endpoint *ep = (viommu_endpoint *)data;
>>>> +
>>>> + if (ep->domain) {
>>>> + virtio_iommu_detach_endpoint_from_domain(ep);
>>>> + g_tree_unref(ep->domain->mappings);
>>>> + }
>>>> +
>>>> + trace_virtio_iommu_put_endpoint(ep->id);
>>>> + g_free(ep);
>>>> +}
>>>> +
>>>> +viommu_domain *virtio_iommu_get_domain(VirtIOIOMMU *s,
>> uint32_t
>>>> +domain_id); viommu_domain
>> *virtio_iommu_get_domain(VirtIOIOMMU
>>>> *s,
>>>> +uint32_t domain_id) {
>>>> + viommu_domain *domain;
>>>> +
>>>> + domain = g_tree_lookup(s->domains,
>> GUINT_TO_POINTER(domain_id));
>>>> + if (domain) {
>>>> + return domain;
>>>> + }
>>>> + domain = g_malloc0(sizeof(*domain));
>>>> + domain->id = domain_id;
>>>> + domain->mappings =
>>>> g_tree_new_full((GCompareDataFunc)interval_cmp,
>>>> + NULL, (GDestroyNotify)g_free,
>>>> + (GDestroyNotify)g_free);
>>>> + g_tree_insert(s->domains, GUINT_TO_POINTER(domain_id),
>> domain);
>>>> + QLIST_INIT(&domain->endpoint_list);
>>>> + trace_virtio_iommu_get_domain(domain_id);
>>>> + return domain;
>>>> +}
>>>> +
>>>> +static void virtio_iommu_put_domain(gpointer data) {
>>>> + viommu_domain *domain = (viommu_domain *)data;
>>>> + viommu_endpoint *iter, *tmp;
>>>> +
>>>> + QLIST_FOREACH_SAFE(iter, &domain->endpoint_list, next, tmp) {
>>>> + virtio_iommu_detach_endpoint_from_domain(iter);
>>>> + }
>>>> + g_tree_destroy(domain->mappings);
>>>> + trace_virtio_iommu_put_domain(domain->id);
>>>> + g_free(domain);
>>>> +}
>>>> +
>>>> static AddressSpace *virtio_iommu_find_add_as(PCIBus *bus, void
>>>> *opaque,
>>>> int devfn) {
>>>> VirtIOIOMMU *s = opaque;
>>>> IOMMUPciBus *sbus = g_hash_table_lookup(s->as_by_busptr, bus);
>>>> + static uint32_t mr_index;
>>>> IOMMUDevice *sdev;
>>>>
>>>> if (!sbus) {
>>>> @@ -60,7 +164,7 @@ static AddressSpace
>>>> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
>>>> if (!sdev) {
>>>> char *name = g_strdup_printf("%s-%d-%d",
>>>> TYPE_VIRTIO_IOMMU_MEMORY_REGION,
>>>> - pci_bus_num(bus), devfn);
>>>> + mr_index++, devfn);
>>>> sdev = sbus->pbdev[devfn] = g_malloc0(sizeof(IOMMUDevice));
>>>>
>>>> sdev->viommu = s;
>>>> @@ -75,6 +179,7 @@ static AddressSpace
>>>> *virtio_iommu_find_add_as(PCIBus *bus, void *opaque,
>>>> UINT64_MAX);
>>>> address_space_init(&sdev->as,
>>>> MEMORY_REGION(&sdev->iommu_mr),
>>>> TYPE_VIRTIO_IOMMU);
>>>> + g_free(name);
>>>> }
>>>>
>>>> return &sdev->as;
>>>> @@ -332,6 +437,13 @@ static const VMStateDescription
>>>> vmstate_virtio_iommu_device = {
>>>> },
>>>> };
>>>>
>>>> +static gint int_cmp(gconstpointer a, gconstpointer b, gpointer
>>>> +user_data) {
>>>> + uint ua = GPOINTER_TO_UINT(a);
>>>> + uint ub = GPOINTER_TO_UINT(b);
>>>> + return (ua > ub) - (ua < ub);
>>>> +}
>>>> +
>>>> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>> {
>>>> VirtIODevice *vdev = VIRTIO_DEVICE(dev); @@ -356,6 +468,8 @@
>>>> static void virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>>>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_MAP_UNMAP);
>>>> virtio_add_feature(&s->features, VIRTIO_IOMMU_F_BYPASS);
>>>>
>>>> + qemu_mutex_init(&s->mutex);
>>>> +
>>>> memset(s->as_by_bus_num, 0, sizeof(s->as_by_bus_num));
>>>> s->as_by_busptr = g_hash_table_new(NULL, NULL);
>>>>
>>>> @@ -364,11 +478,20 @@ static void
>>>> virtio_iommu_device_realize(DeviceState *dev, Error **errp)
>>>> } else {
>>>> error_setg(errp, "VIRTIO-IOMMU is not attached to any PCI bus!");
>>>> }
>>>> +
>>>> + s->domains = g_tree_new_full((GCompareDataFunc)int_cmp,
>>>> + NULL, NULL, virtio_iommu_put_domain);
>>>> + s->endpoints = g_tree_new_full((GCompareDataFunc)int_cmp,
>>>> + NULL, NULL,
>>>> + virtio_iommu_put_endpoint);
>>>> }
>>>>
>>>> static void virtio_iommu_device_unrealize(DeviceState *dev, Error
>>>> **errp) {
>>>> VirtIODevice *vdev = VIRTIO_DEVICE(dev);
>>>> + VirtIOIOMMU *s = VIRTIO_IOMMU(dev);
>>>> +
>>>> + g_tree_destroy(s->domains);
>>>> + g_tree_destroy(s->endpoints);
>>>>
>>>> virtio_cleanup(vdev);
>>>> }
>>>> --
>>>> 2.17.2
>>>
>>>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings Eric Auger
@ 2018-11-27 7:07 ` Bharat Bhushan
0 siblings, 0 replies; 25+ messages in thread
From: Bharat Bhushan @ 2018-11-27 7:07 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, peterx
> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: Thursday, November 22, 2018 10:46 PM
> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
> mst@redhat.com; jean-philippe.brucker@arm.com
> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
> <bharat.bhushan@nxp.com>; peterx@redhat.com
> Subject: [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree
> mappings
>
> Adds the "virtio,pci-iommu" node in the host bridge node and the RID
> mapping, excluding the IOMMU RID.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Bharat Bhushan <bharat.bhushan@nxp.com>
>
> ---
>
> v8 -> v9:
> - disable msi-bypass property
> - addition of the subnode is handled is the hotplug handler
> and IOMMU RID is notimposed anymore
>
> v6 -> v7:
> - align to the smmu instantiation code
>
> v4 -> v5:
> - VirtMachineClass no_iommu added in this patch
> - Use object_resolve_path_type
> ---
> hw/arm/virt.c | 57 +++++++++++++++++++++++++++++++++++++------
> include/hw/arm/virt.h | 2 ++
> 2 files changed, 52 insertions(+), 7 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c index a2b8d8f7c2..b2bbb0ef49
> 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -29,6 +29,7 @@
> */
>
> #include "qemu/osdep.h"
> +#include "monitor/qdev.h"
> #include "qapi/error.h"
> #include "hw/sysbus.h"
> #include "hw/arm/arm.h"
> @@ -49,6 +50,7 @@
> #include "qemu/bitops.h"
> #include "qemu/error-report.h"
> #include "hw/pci-host/gpex.h"
> +#include "hw/virtio/virtio-pci.h"
> #include "hw/arm/sysbus-fdt.h"
> #include "hw/platform-bus.h"
> #include "hw/arm/fdt.h"
> @@ -59,6 +61,7 @@
> #include "qapi/visitor.h"
> #include "standard-headers/linux/input.h"
> #include "hw/arm/smmuv3.h"
> +#include "hw/virtio/virtio-iommu.h"
>
> #define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \
> static void virt_##major##_##minor##_class_init(ObjectClass *oc, \ @@ -
> 1085,6 +1088,33 @@ static void create_smmu(const VirtMachineState *vms,
> qemu_irq *pic,
> g_free(node);
> }
>
> +static void create_virtio_iommu(VirtMachineState *vms, Error **errp) {
> + const char compat[] = "virtio,pci-iommu";
> + uint16_t bdf = vms->virtio_iommu_bdf;
> + char *node;
> +
> + vms->iommu_phandle = qemu_fdt_alloc_phandle(vms->fdt);
> +
> + node = g_strdup_printf("%s/virtio_iommu@%d", vms-
> >pciehb_nodename, bdf);
> + qemu_fdt_add_subnode(vms->fdt, node);
> + qemu_fdt_setprop(vms->fdt, node, "compatible", compat,
> sizeof(compat));
> + qemu_fdt_setprop_sized_cells(vms->fdt, node, "reg",
> + 1, bdf << 8 /* phys.hi */,
> + 1, 0 /* phys.mid */,
> + 1, 0 /* phys.lo */,
> + 1, 0 /* size.hi */,
> + 1, 0 /* size.low */);
> +
> + qemu_fdt_setprop_cell(vms->fdt, node, "#iommu-cells", 1);
> + qemu_fdt_setprop_cell(vms->fdt, node, "phandle", vms-
> >iommu_phandle);
> + g_free(node);
> +
> + qemu_fdt_setprop_cells(vms->fdt, vms->pciehb_nodename, "iommu-
> map",
> + 0x0, vms->iommu_phandle, 0x0, bdf,
> + bdf + 1, vms->iommu_phandle, bdf + 1, 0xffff
> +- bdf); }
> +
> static void create_pcie(VirtMachineState *vms, qemu_irq *pic) {
> hwaddr base_mmio = vms->memmap[VIRT_PCIE_MMIO].base; @@ -
> 1162,7 +1192,7 @@ static void create_pcie(VirtMachineState *vms,
> qemu_irq *pic)
> }
> }
>
> - nodename = g_strdup_printf("/pcie@%" PRIx64, base);
> + nodename = vms->pciehb_nodename = g_strdup_printf("/pcie@%"
> PRIx64,
> + base);
> qemu_fdt_add_subnode(vms->fdt, nodename);
> qemu_fdt_setprop_string(vms->fdt, nodename,
> "compatible", "pci-host-ecam-generic"); @@ -1205,13
> +1235,17 @@ static void create_pcie(VirtMachineState *vms, qemu_irq *pic)
> if (vms->iommu) {
> vms->iommu_phandle = qemu_fdt_alloc_phandle(vms->fdt);
>
> - create_smmu(vms, pic, pci->bus);
> + switch (vms->iommu) {
> + case VIRT_IOMMU_SMMUV3:
> + create_smmu(vms, pic, pci->bus);
> + qemu_fdt_setprop_cells(vms->fdt, nodename, "iommu-map",
> + 0x0, vms->iommu_phandle, 0x0, 0x10000);
> + break;
> + default:
> + g_assert_not_reached();
> + }
>
> - qemu_fdt_setprop_cells(vms->fdt, nodename, "iommu-map",
> - 0x0, vms->iommu_phandle, 0x0, 0x10000);
> }
> -
> - g_free(nodename);
> }
>
> static void create_platform_bus(VirtMachineState *vms, qemu_irq *pic)
> @@ -1736,12 +1770,21 @@ static void
> virt_machine_device_plug_cb(HotplugHandler *hotplug_dev,
> SYS_BUS_DEVICE(dev));
> }
> }
> + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) {
> + PCIDevice *pdev = PCI_DEVICE(dev);
> +
> + vms->iommu = VIRT_IOMMU_VIRTIO;
> + vms->virtio_iommu_bdf = pci_get_bdf(pdev);
> + object_property_set_bool(OBJECT(dev), false, "msi-bypass", errp);
> + create_virtio_iommu(vms, errp);
> + }
> }
>
> static HotplugHandler *virt_machine_get_hotplug_handler(MachineState
> *machine,
> DeviceState *dev) {
> - if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE)) {
> + if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE) ||
> + object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) {
> return HOTPLUG_HANDLER(machine);
> }
>
> diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index
> 4cc57a7ef6..c330f6f4b7 100644
> --- a/include/hw/arm/virt.h
> +++ b/include/hw/arm/virt.h
> @@ -115,7 +115,9 @@ typedef struct {
> bool virt;
> int32_t gic_version;
> VirtIOMMUType iommu;
> + uint16_t virtio_iommu_bdf;
> struct arm_boot_info bootinfo;
> + char *pciehb_nodename;
> const MemMapEntry *memmap;
> const int *irqmap;
> int smp_cpus;
> --
> 2.17.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
` (16 preceding siblings ...)
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 17/17] hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table Eric Auger
@ 2018-11-27 7:11 ` Bharat Bhushan
2019-07-05 15:06 ` Zhangfei Gao
17 siblings, 1 reply; 25+ messages in thread
From: Bharat Bhushan @ 2018-11-27 7:11 UTC (permalink / raw)
To: Eric Auger, eric.auger.pro, qemu-devel, qemu-arm, peter.maydell,
mst, jean-philippe.brucker
Cc: kevin.tian, tn, peterx
Hi Eric,
> -----Original Message-----
> From: Eric Auger <eric.auger@redhat.com>
> Sent: Thursday, November 22, 2018 10:45 PM
> To: eric.auger.pro@gmail.com; eric.auger@redhat.com; qemu-
> devel@nongnu.org; qemu-arm@nongnu.org; peter.maydell@linaro.org;
> mst@redhat.com; jean-philippe.brucker@arm.com
> Cc: kevin.tian@intel.com; tn@semihalf.com; Bharat Bhushan
> <bharat.bhushan@nxp.com>; peterx@redhat.com
> Subject: [RFC v9 00/17] VIRTIO-IOMMU device
>
> This series rebases the virtio-iommu device on qemu 3.1.0-rc2 and
> implements the v0.8(.1) virtio-iommu spec [1]. The pci proxy for the virtio-
> iommu device is now available and needs to be instantiated from the
> command line using "-device virtio-iommu-pci".
> The iommu machvirt option is not used anymore to instantiate the virtio-
> iommu.
>
> At the moment the virtio-iommu-device only is functional in the ARM virt
> machine. Indeed, besides its instantiation, links between the PCIe end points
> and the IOMMU must be described. This is achieved by DT or ACPI
> description (IORT). This description currently only is done in ARM virt.
>
> Best Regards
>
> Eric
>
> This series can be found at:
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
> hub.com%2Feauger%2Fqemu%2Ftree%2Fv3.1.0-rc2-virtio-iommu-
> v0.9&data=02%7C01%7Cbharat.bhushan%40nxp.com%7C031e61e2a2f3
> 4e3b945a08d6509e2918%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%
> 7C636785037552135579&sdata=v99FRL1dQC%2BPTofwwqqoKGNiiqJ%2F
> lGBX3KJB0IUKbQU%3D&reserved=0
>
> References:
> [1] [PATCH v3 0/7] Add virtio-iommu driver
>
> [2] guest branch featuring the virtio-iommu driver v0.8.1 + ACPI
> integration not yet officially released by Jean.
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit
> hub.com%2Feauger%2Flinux%2Ftree%2Fvirtio-iommu-
> v0.8.1&data=02%7C01%7Cbharat.bhushan%40nxp.com%7C031e61e2a2f
> 34e3b945a08d6509e2918%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0
> %7C636785037552135579&sdata=EjHA9BZ6rNX36YFKkNocBQtoz3uf3bdn
> 5T9NSJ6SaRg%3D&reserved=0
>
> Testing:
> - tested with guest using virtio-net-pci
> (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
> and virtio-blk-pci
> - VFIO/VHOST integration is not part of this series
> - When using the virtio-blk-pci, some EDK2 FW versions feature
> unmapped transactions and in that case the guest fails to boot.
I have tested this series with virtio and VFIO both
Tested-by: Bharat Bhushan <bharat.bhushan@nxp.com>
Thanks
-Bharat
>
> History:
>
> v8 -> v9:
> - virtio-iommu-pci device needs to be instantiated from the command
> line (RID is not imposed anymore).
> - tail structure properly initialized
>
> v7 -> v8:
> - virtio-iommu-pci added
> - virt instantiation modified
> - DT and ACPI modified to exclude the iommu RID from the mapping
> - VIRTIO_IOMMU_F_BYPASS, VIRTIO_F_VERSION_1 features exposed
>
> v6 -> v7:
> - rebase on qemu 3.0.0-rc3
> - minor update against v0.7
> - fix issue with EP not on pci.0 and ACPI probing
> - change the instantiation method
>
> v5 -> v6:
> - minor update against v0.6 spec
> - fix g_hash_table_lookup in virtio_iommu_find_add_as
> - replace some error_reports by qemu_log_mask(LOG_GUEST_ERROR, ...)
>
> v4 -> v5:
> - event queue and fault reporting
> - we now return the IOAPIC MSI region if the virtio-iommu is instantiated
> in a PC machine.
> - we bypass transactions on MSI HW region and fault on reserved ones.
> - We support ACPI boot with mach-virt (based on IORT proposal)
> - We moved to the new driver naming conventions
> - simplified mach-virt instantiation
> - worked around the disappearing of pci_find_primary_bus
> - in virtio_iommu_translate, check the dev->as is not NULL
> - initialize as->device_list in virtio_iommu_get_as
> - initialize bufstate.error to false in virtio_iommu_probe
>
> v3 -> v4:
> - probe request support although no reserved region is returned at
> the moment
> - unmap semantics less strict, as specified in v0.4
> - device registration, attach/detach revisited
> - split into smaller patches to ease review
> - propose a way to inform the IOMMU mr about the page_size_mask
> of underlying HW IOMMU, if any
> - remove warning associated with the translation of the MSI doorbell
>
> v2 -> v3:
> - rebase on top of 2.10-rc0 and especially
> [PATCH qemu v9 0/2] memory/iommu: QOM'fy IOMMU MemoryRegion
> - add mutex init
> - fix as->mappings deletion using g_tree_ref/unref
> - when a dev is attached whereas it is already attached to
> another address space, first detach it
> - fix some error values
> - page_sizes = TARGET_PAGE_MASK;
> - I haven't changed the unmap() semantics yet, waiting for the
> next virtio-iommu spec revision.
>
> v1 -> v2:
> - fix redifinition of viommu_as typedef
>
> Eric Auger (17):
> update-linux-headers: Import virtio_iommu.h
> linux-headers: Partial update for virtio-iommu v0.8
> virtio-iommu: Add skeleton
> virtio-iommu: Decode the command payload
> virtio-iommu: Add the iommu regions
> virtio-iommu: Endpoint and domains structs and helpers
> virtio-iommu: Implement attach/detach command
> virtio-iommu: Implement map/unmap
> virtio-iommu: Implement translate
> virtio-iommu: Implement probe request
> virtio-iommu: Expose the IOAPIC MSI reserved region when relevant
> virtio-iommu: Implement fault reporting
> virtio_iommu: Handle reserved regions in translation process
> virtio-iommu-pci: Add virtio iommu pci support
> hw/arm/virt: Add the virtio-iommu device tree mappings
> hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper
> hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table
>
> hw/arm/virt-acpi-build.c | 91 +-
> hw/arm/virt.c | 57 +-
> hw/virtio/Makefile.objs | 1 +
> hw/virtio/trace-events | 26 +
> hw/virtio/virtio-iommu.c | 1040 +++++++++++++++++
> hw/virtio/virtio-pci.c | 51 +
> hw/virtio/virtio-pci.h | 14 +
> include/hw/acpi/acpi-defs.h | 21 +-
> include/hw/arm/virt.h | 2 +
> include/hw/pci/pci.h | 1 +
> include/hw/virtio/virtio-iommu.h | 65 ++
> include/standard-headers/linux/virtio_ids.h | 1 +
> include/standard-headers/linux/virtio_iommu.h | 159 +++
> linux-headers/linux/virtio_iommu.h | 1 +
> qdev-monitor.c | 1 +
> scripts/update-linux-headers.sh | 3 +
> 16 files changed, 1501 insertions(+), 33 deletions(-) create mode 100644
> hw/virtio/virtio-iommu.c create mode 100644 include/hw/virtio/virtio-
> iommu.h create mode 100644 include/standard-
> headers/linux/virtio_iommu.h
> create mode 100644 linux-headers/linux/virtio_iommu.h
>
> --
> 2.17.2
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device
2018-11-27 7:11 ` [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Bharat Bhushan
@ 2019-07-05 15:06 ` Zhangfei Gao
0 siblings, 0 replies; 25+ messages in thread
From: Zhangfei Gao @ 2019-07-05 15:06 UTC (permalink / raw)
To: Bharat Bhushan
Cc: peter.maydell, kevin.tian, mst, jean-philippe.brucker, tn,
qemu-devel, peterx, Eric Auger, qemu-arm, eric.auger.pro
Hi, Bharat
On Tue, Nov 27, 2018 at 3:12 PM Bharat Bhushan <bharat.bhushan@nxp.com> wrote:
> > Testing:
> > - tested with guest using virtio-net-pci
> > (,vhost=off,iommu_platform,disable-modern=off,disable-legacy=on)
> > and virtio-blk-pci
> > - VFIO/VHOST integration is not part of this series
> > - When using the virtio-blk-pci, some EDK2 FW versions feature
> > unmapped transactions and in that case the guest fails to boot.
>
> I have tested this series with virtio and VFIO both
> Tested-by: Bharat Bhushan <bharat.bhushan@nxp.com>
>
Would you mind pasting the qemu test command.
A bit confused about testing vfio, virtio-iommu-pci has no "host" property.
Do we need unbind pf and bind the device to vfio-pci first,
Thanks
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2019-07-05 15:09 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-22 17:15 [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 01/17] update-linux-headers: Import virtio_iommu.h Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 02/17] linux-headers: Partial update for virtio-iommu v0.8 Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 03/17] virtio-iommu: Add skeleton Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 04/17] virtio-iommu: Decode the command payload Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 05/17] virtio-iommu: Add the iommu regions Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 06/17] virtio-iommu: Endpoint and domains structs and helpers Eric Auger
2018-11-23 6:38 ` Bharat Bhushan
2018-11-23 7:53 ` Auger Eric
2018-11-23 9:14 ` Bharat Bhushan
2018-11-23 9:26 ` Auger Eric
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 07/17] virtio-iommu: Implement attach/detach command Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 08/17] virtio-iommu: Implement map/unmap Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 09/17] virtio-iommu: Implement translate Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 10/17] virtio-iommu: Implement probe request Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 11/17] virtio-iommu: Expose the IOAPIC MSI reserved region when relevant Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 12/17] virtio-iommu: Implement fault reporting Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 13/17] virtio_iommu: Handle reserved regions in translation process Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 14/17] virtio-iommu-pci: Add virtio iommu pci support Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 15/17] hw/arm/virt: Add the virtio-iommu device tree mappings Eric Auger
2018-11-27 7:07 ` Bharat Bhushan
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 16/17] hw/arm/virt-acpi-build: Introduce fill_iort_idmap helper Eric Auger
2018-11-22 17:15 ` [Qemu-devel] [RFC v9 17/17] hw/arm/virt-acpi-build: Add virtio-iommu node in IORT table Eric Auger
2018-11-27 7:11 ` [Qemu-devel] [RFC v9 00/17] VIRTIO-IOMMU device Bharat Bhushan
2019-07-05 15:06 ` Zhangfei Gao
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.