* [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU
@ 2016-05-17 7:15 Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 01/25] acpi: enable INTR for DMAR report structure Peter Xu
` (25 more replies)
0 siblings, 26 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
This is v7 patchset for Intel IOMMU IR support. This patchset is
based on the following two patches:
- [PATCH v3] pci: fix pci_requester_id()
https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg02672.html
- [PATCH 0/2] IOAPIC: clear remote IRR for edge interrupts
https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01431.html
Please apply the above two patches as well before testing this
patchset. The first patch fixes issue when IR is used with pci
bridges, like pci-bridge device. The second patch fixes issue when
IR is used with level-triggered interrupt devices, like e1000.
v7 changes (using v6 patch index):
- patch 10: trivial change in debug string (remove one more "\n")
- patch 17-18: ioapic remote irr patches, sent seperately
already. So removed from this series.
- patch 24:
- fix commit message: only irqfd msi routes are maintained, not
all msi routes.
- skip all IOAPIC msi entries (dev == NULL). We only need to
housekeep irqfd users.
- added patches
- pick up Radim's patch on adding MHMV ecap bits [Radim]
- remove all vtd_* patches, instead, use x86-iommu ones at the first
place. This introduced lots of patch order changes and content
changes, which affected from original patch 8 to the end. Sorry!
[Jan]
v6 changes:
- patch 10: use write_with_attrs() rather than write(), preparing
for SID verification [Jan]
- patch 17-18: add r-b line from Radim [Radim]
- new patch 19: put together Jan's EIM patch [Jan]
- new patch 20: add SID validation process
- new patch 21-22: introduce X86IOMMU class, which is the parent of
IntelIOMMU class. Patch 21 only introduce the class and did
nothing, patch 22 cleaned up all the vtd_*() hooks into x86
ones. This is only a start. In the future, we can abstract more
things into X86IOMMU class, like iotlb, address spaces mgmt,
etc. [Jan]
- new patch 23-25: this is to do IEC notify to all irqfd consumers
like vhost/vfio. patch 23 changed interface for
kvm_irqchip_add_msi_route(), provide vector info rather than a raw
MSI message. Patch 24 added new hooks to do arch-specific
notification on addition/deletion of msi routes. Patch 25 is x86
specific, which added one more IEC notifier for msi routes. [Jan]
- new patch 26: this is to partially solve the issue that Jan has
encountered (1 sec delay when invalidating IR cache).
v5 changes:
- patch 10: add vector checking for IOAPIC interrupts (this may help
debug in the future, will only generate warning if specify
IOMMU_DEBUG)
- patch 13: replace error_report() with a trace. [Jan]
- patch 14: rename parameter "intr" to "intremap", to be aligned
with kernel parameter [Jan]
- patch 15: fix comments for vtd_iec_notify_fn
- patch 17 & 18 (added): fix issue when IR enabled with devices
using level-triggered interrupts, like e1000. Adding it to the end
of series, since this issue never happen without IR.
Patch 17 adds read-only check for IOAPIC entries.
Patch 18 clears remote IRR bit when entry configured as
edge-triggered.
v4 changes (all patch number corresponds to v3):
- add one patch at the start of v3 series: I missed to send the
first patch in v3. adding it in. [Jan]
- patch 9: add support for compatible mode (no reason not to support
it, if not, we will get some warnings when using split irqchip)
- patch 11: further simplify ioapic_update_kvm_routes() using the
helper function.
- patch 12: tweak on kvm_arch_fixup_msi_route() rather than
ioapic_update_kvm_routes() only. [Radim]
- add patch 15: introduce IEC (Interrupt Entry Cache) invalidation
notifier list. We can register to this list if we want to be
notified when we got IR invalidation requests [Radim]
- add patch 16: let IOAPIC the first consumer for the above IEC
notifier list. [Radim]
- several other trivial fixes (like moving some defines from .c to
.h, moving several lines of changes from one patch to another to
make it make more sense, etc.)
v3 changes (all patch numbers corresponds to v2):
- patch 1 (-> v3 patch 13)
- move to the end of series [Alex]
- patch 10 (dropped)
- drop this one, since re-worked on IOAPIC support, so we do not
need this any more.
- patch 12 (-> v3 patch 10)
- leverage MSI path for IOAPIC IR [Jan]
- patch 13 (v3 -> patch 9)
- remove vtd_interrupt_remap_msi() declaration by reordering the
functions [mst]
- vtd_generate_msi_message(): init msg using {}, remove FIXME
[mst]
- new patches
- v3 patch 11: introduce ioapic_entry_parse() helper function
- v3 patch 12: add support for kernel-irqchip=split. This needs
more reviews, logically this should enable lots of things:
splitted irqchip, irqfd, vhost, and irqfd support for
passthrough devices (not tested). Please refer to the patch for
more information.
v2 changes:
- patch 1
- rename "int_remap" to "intr" in several places [Marcel]
- remove "Intel" specific words in desc or commit message, prepare
itself with further AMD support [Marcel]
- avoid using object_property_get_bool() [Marcel]
- patch 5
- use PCI bus number 0xff rather than 0xf0 for the IOAPIC scope
definition. (please let me know if anyone knows how I can avoid
user using PCI bus number 0xff... TIA)
- patch 11
- fix comments [Marcel]
- all
- remove intr_supported variable [Marcel]
This patchset provide very basic functionalities for interrupt
remapping (IR) support of the emulated Intel IOMMU device.
By default, IR is disabled to be better compatible with current
QEMU. To enable IR, we can using the following command to boot a
IR-supported VM with virtio-net device with vhost (still do not
support kvm-ioapic, so we need to specify kernel-irqchip={split|off}
here):
$ qemu-system-x86_64 -M q35,iommu=on,intr=on,kernel-irqchip=split \
-enable-kvm -m 1024 \
-netdev tap,id=net0,vhost=on \
-device virtio-net-pci,netdev=user.0 \
-monitor telnet::3333,server,nowait \
/var/lib/libvirt/images/vm1.qcow2
When guest boots, we can verify whether IR enabled by grepping the
dmesg like:
Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: IOAPIC id 0 under DRHD base 0xfed90000 IOMMU 0
Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: Enabled IRQ remapping in x2apic mode
Testing is only covering basic smoke test for the following matrix:
- IR enabled/disable
- kernel irqchip off/split
- network device: tap with/without vhost, e1000
Currently supported:
- Emulated/Splitted irqchip
- Generic PCI Devices
- vhost devices
- pass through device support? Not tested, but suppose it should work.
- IEC (Interrupt Entry Cache) cache invalidation notification
- EIM (from Jan)
- IRTE Source-id validation
TODO List:
- explicit IEC invalidation (currently, we do update without
checking. Also, we can process QI invalidation in bulk, as Jan
suggested)
- IR fault reporting
- migration support (for IOMMU as general?)
- more?
Jan Kiszka (1):
intel_iommu: Add support for Extended Interrupt Mode
Peter Xu (23):
acpi: enable INTR for DMAR report structure
intel_iommu: allow queued invalidation for IR
intel_iommu: set IR bit for ECAP register
acpi: add DMAR scope definition for root IOAPIC
intel_iommu: define interrupt remap table addr register
intel_iommu: handle interrupt remap enable
intel_iommu: define several structs for IOMMU IR
x86-iommu: introduce parent class
x86-iommu: provide x86_iommu_get_default
x86-iommu: q35: generalize find_add_as()
intel_iommu: add IR translation faults defines
intel_iommu: Add support for PCI MSI remap
q35: ioapic: add support for emulated IOAPIC IR
ioapic: introduce ioapic_entry_parse() helper
intel_iommu: add support for split irqchip
q35: add "intremap" parameter to enable IR
x86-iommu: introduce IEC notifiers
ioapic: register IOMMU IEC notifier for ioapic
intel_iommu: add SID validation for IR
kvm-irqchip: simplify kvm_irqchip_add_msi_route
kvm-irqchip: i386: add hook for add/remove virq
kvm-irqchip: x86: add msi route notify fn
kvm-irqchip: do explicit commit when update irq
Radim Krčmář (1):
intel_iommu: support all masks in interrupt entry cache invalidation
hw/core/machine.c | 22 ++
hw/i386/Makefile.objs | 2 +-
hw/i386/acpi-build.c | 38 ++--
hw/i386/intel_iommu.c | 428 +++++++++++++++++++++++++++++++++++++-
hw/i386/intel_iommu_internal.h | 50 ++++-
hw/i386/kvm/pci-assign.c | 10 +-
hw/i386/pc.c | 3 +
hw/i386/x86-iommu.c | 92 ++++++++
hw/intc/ioapic.c | 125 +++++++----
hw/misc/ivshmem.c | 4 +-
hw/pci-host/q35.c | 16 +-
hw/pci/pci.c | 15 ++
hw/vfio/pci.c | 12 +-
hw/virtio/virtio-pci.c | 10 +-
include/hw/acpi/acpi-defs.h | 15 ++
include/hw/boards.h | 1 +
include/hw/i386/apic-msidef.h | 1 +
include/hw/i386/intel_iommu.h | 147 ++++++++++++-
include/hw/i386/ioapic_internal.h | 1 +
include/hw/i386/pc.h | 4 +
include/hw/i386/x86-iommu.h | 102 +++++++++
include/hw/pci-host/q35.h | 9 +
include/hw/pci/pci.h | 2 +
include/sysemu/kvm.h | 21 +-
kvm-all.c | 14 +-
kvm-stub.c | 6 +-
target-arm/kvm.c | 11 +
target-i386/kvm.c | 109 +++++++++-
target-mips/kvm.c | 11 +
target-ppc/kvm.c | 11 +
target-s390x/kvm.c | 11 +
trace-events | 6 +
32 files changed, 1198 insertions(+), 111 deletions(-)
create mode 100644 hw/i386/x86-iommu.c
create mode 100644 include/hw/i386/x86-iommu.h
--
2.4.11
^ permalink raw reply [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 01/25] acpi: enable INTR for DMAR report structure
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 02/25] intel_iommu: allow queued invalidation for IR Peter Xu
` (24 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Introduce iommu_intr in MachineState to show whether IOMMU IR is
enabled. By default, IR is off.
In ACPI DMA remapping report structure, enable INTR flag when specified.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/core/machine.c | 2 ++
hw/i386/acpi-build.c | 12 +++++++++---
include/hw/boards.h | 1 +
include/hw/i386/intel_iommu.h | 2 ++
4 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 6dbbc85..276ad61 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -382,6 +382,8 @@ static void machine_initfn(Object *obj)
ms->kvm_shadow_mem = -1;
ms->dump_guest_core = true;
ms->mem_merge = true;
+ /* Disable interrupt remapping by default. */
+ ms->iommu_intr = false;
object_property_add_str(obj, "accel",
machine_get_accel, machine_set_accel, NULL);
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 279f0d7..ddc6f16 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2554,16 +2554,22 @@ build_mcfg_q35(GArray *table_data, GArray *linker, AcpiMcfgInfo *info)
}
static void
-build_dmar_q35(GArray *table_data, GArray *linker)
+build_dmar_q35(MachineState *ms, GArray *table_data, GArray *linker)
{
int dmar_start = table_data->len;
AcpiTableDmar *dmar;
AcpiDmarHardwareUnit *drhd;
+ uint8_t dmar_flags = 0;
+
+ if (ms->iommu_intr) {
+ /* enable INTR for the IOMMU device */
+ dmar_flags |= DMAR_REPORT_F_INTR;
+ }
dmar = acpi_data_push(table_data, sizeof(*dmar));
dmar->host_address_width = VTD_HOST_ADDRESS_WIDTH - 1;
- dmar->flags = 0; /* No intr_remap for now */
+ dmar->flags = dmar_flags;
/* DMAR Remapping Hardware Unit Definition structure */
drhd = acpi_data_push(table_data, sizeof(*drhd));
@@ -2724,7 +2730,7 @@ void acpi_build(AcpiBuildTables *tables, MachineState *machine)
}
if (acpi_has_iommu()) {
acpi_add_table(table_offsets, tables_blob);
- build_dmar_q35(tables_blob, tables->linker);
+ build_dmar_q35(MACHINE(pcms), tables_blob, tables->linker);
}
if (pcms->acpi_nvdimm_state.is_enabled) {
nvdimm_build_acpi(table_offsets, tables_blob, tables->linker);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 8d4fe56..43f4976 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -152,6 +152,7 @@ struct MachineState {
bool igd_gfx_passthru;
char *firmware;
bool iommu;
+ bool iommu_intr;
bool suppress_vmdesc;
bool enforce_config_section;
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index b024ffa..0d89796 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -44,6 +44,8 @@
#define VTD_HOST_ADDRESS_WIDTH 39
#define VTD_HAW_MASK ((1ULL << VTD_HOST_ADDRESS_WIDTH) - 1)
+#define DMAR_REPORT_F_INTR (1)
+
typedef struct VTDContextEntry VTDContextEntry;
typedef struct VTDContextCacheEntry VTDContextCacheEntry;
typedef struct IntelIOMMUState IntelIOMMUState;
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 02/25] intel_iommu: allow queued invalidation for IR
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 01/25] acpi: enable INTR for DMAR report structure Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 03/25] intel_iommu: set IR bit for ECAP register Peter Xu
` (23 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Queued invalidation is required for IR. This patch add basic support for
interrupt cache invalidate requests. Since we currently have no IR cache
implemented yet, we can just skip all interrupt cache invalidation
requests for now.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 9 +++++++++
hw/i386/intel_iommu_internal.h | 2 ++
2 files changed, 11 insertions(+)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 347718f..4b0558e 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1400,6 +1400,15 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s)
}
break;
+ case VTD_INV_DESC_IEC:
+ VTD_DPRINTF(INV, "Interrupt Entry Cache Invalidation "
+ "not implemented yet");
+ /*
+ * Since currently we do not cache interrupt entries, we can
+ * just mark this descriptor as "good" and move on.
+ */
+ break;
+
default:
VTD_DPRINTF(GENERAL, "error: unkonw Invalidation Descriptor type "
"hi 0x%"PRIx64 " lo 0x%"PRIx64 " type %"PRIu8,
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index e5f514c..b648e69 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -286,6 +286,8 @@ typedef struct VTDInvDesc VTDInvDesc;
#define VTD_INV_DESC_TYPE 0xf
#define VTD_INV_DESC_CC 0x1 /* Context-cache Invalidate Desc */
#define VTD_INV_DESC_IOTLB 0x2
+#define VTD_INV_DESC_IEC 0x4 /* Interrupt Entry Cache
+ Invalidate Descriptor */
#define VTD_INV_DESC_WAIT 0x5 /* Invalidation Wait Descriptor */
#define VTD_INV_DESC_NONE 0 /* Not an Invalidate Descriptor */
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 03/25] intel_iommu: set IR bit for ECAP register
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 01/25] acpi: enable INTR for DMAR report structure Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 02/25] intel_iommu: allow queued invalidation for IR Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 04/25] acpi: add DMAR scope definition for root IOAPIC Peter Xu
` (22 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Enable IR in IOMMU Extended Capability register.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 7 +++++++
hw/i386/intel_iommu_internal.h | 2 ++
2 files changed, 9 insertions(+)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 4b0558e..17668d6 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -24,6 +24,7 @@
#include "exec/address-spaces.h"
#include "intel_iommu_internal.h"
#include "hw/pci/pci.h"
+#include "hw/boards.h"
/*#define DEBUG_INTEL_IOMMU*/
#ifdef DEBUG_INTEL_IOMMU
@@ -1941,6 +1942,8 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
*/
static void vtd_init(IntelIOMMUState *s)
{
+ MachineState *ms = MACHINE(qdev_get_machine());
+
memset(s->csr, 0, DMAR_REG_SIZE);
memset(s->wmask, 0, DMAR_REG_SIZE);
memset(s->w1cmask, 0, DMAR_REG_SIZE);
@@ -1961,6 +1964,10 @@ static void vtd_init(IntelIOMMUState *s)
VTD_CAP_SAGAW | VTD_CAP_MAMV | VTD_CAP_PSI | VTD_CAP_SLLPS;
s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO;
+ if (ms->iommu_intr) {
+ s->ecap |= VTD_ECAP_IR;
+ }
+
vtd_reset_context_cache(s);
vtd_reset_iotlb(s);
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index b648e69..5b98a11 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -176,6 +176,8 @@
/* (offset >> 4) << 8 */
#define VTD_ECAP_IRO (DMAR_IOTLB_REG_OFFSET << 4)
#define VTD_ECAP_QI (1ULL << 1)
+/* Interrupt Remapping support */
+#define VTD_ECAP_IR (1ULL << 3)
/* CAP_REG */
/* (offset >> 4) << 24 */
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 04/25] acpi: add DMAR scope definition for root IOAPIC
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (2 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 03/25] intel_iommu: set IR bit for ECAP register Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 05/25] intel_iommu: define interrupt remap table addr register Peter Xu
` (21 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
To enable interrupt remapping for intel IOMMU device, each IOAPIC device
in the system reported via ACPI MADT must be explicitly enumerated under
one specific remapping hardware unit. This patch adds the root-complex
IOAPIC into the default DMAR device.
Please refer to VT-d spec 8.3.1.1 for more information.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/acpi-build.c | 17 +++++++++++++++--
include/hw/acpi/acpi-defs.h | 15 +++++++++++++++
include/hw/pci-host/q35.h | 9 +++++++++
3 files changed, 39 insertions(+), 2 deletions(-)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index ddc6f16..6c572a3 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -77,6 +77,9 @@
#define ACPI_BUILD_DPRINTF(fmt, ...)
#endif
+/* Default IOAPIC ID */
+#define ACPI_BUILD_IOAPIC_ID 0x0
+
typedef struct AcpiMcfgInfo {
uint64_t mcfg_base;
uint32_t mcfg_size;
@@ -375,7 +378,6 @@ build_madt(GArray *table_data, GArray *linker, PCMachineState *pcms)
io_apic = acpi_data_push(table_data, sizeof *io_apic);
io_apic->type = ACPI_APIC_IO;
io_apic->length = sizeof(*io_apic);
-#define ACPI_BUILD_IOAPIC_ID 0x0
io_apic->io_apic_id = ACPI_BUILD_IOAPIC_ID;
io_apic->address = cpu_to_le32(IO_APIC_DEFAULT_ADDRESS);
io_apic->interrupt = cpu_to_le32(0);
@@ -2561,6 +2563,9 @@ build_dmar_q35(MachineState *ms, GArray *table_data, GArray *linker)
AcpiTableDmar *dmar;
AcpiDmarHardwareUnit *drhd;
uint8_t dmar_flags = 0;
+ AcpiDmarDeviceScope *scope = NULL;
+ /* Root complex IOAPIC use one path[0] only */
+ uint16_t scope_size = sizeof(*scope) + sizeof(uint16_t);
if (ms->iommu_intr) {
/* enable INTR for the IOMMU device */
@@ -2574,11 +2579,19 @@ build_dmar_q35(MachineState *ms, GArray *table_data, GArray *linker)
/* DMAR Remapping Hardware Unit Definition structure */
drhd = acpi_data_push(table_data, sizeof(*drhd));
drhd->type = cpu_to_le16(ACPI_DMAR_TYPE_HARDWARE_UNIT);
- drhd->length = cpu_to_le16(sizeof(*drhd)); /* No device scope now */
+ drhd->length = cpu_to_le16(sizeof(*drhd) + scope_size);
drhd->flags = ACPI_DMAR_INCLUDE_PCI_ALL;
drhd->pci_segment = cpu_to_le16(0);
drhd->address = cpu_to_le64(Q35_HOST_BRIDGE_IOMMU_ADDR);
+ /* Scope definition for the root-complex IOAPIC */
+ scope = acpi_data_push(table_data, scope_size);
+ scope->entry_type = cpu_to_le16(ACPI_DMAR_DEV_SCOPE_TYPE_IOAPIC);
+ scope->length = scope_size;
+ scope->enumeration_id = cpu_to_le16(ACPI_BUILD_IOAPIC_ID);
+ scope->bus = cpu_to_le16(Q35_PSEUDO_BUS_PLATFORM);
+ scope->path[0] = cpu_to_le16(Q35_PSEUDO_DEVFN_IOAPIC);
+
build_header(linker, table_data, (void *)(table_data->data + dmar_start),
"DMAR", table_data->len - dmar_start, 1, NULL, NULL);
}
diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index 850a962..b46e472 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -569,6 +569,20 @@ enum {
/*
* Sub-structures for DMAR
*/
+
+#define ACPI_DMAR_DEV_SCOPE_TYPE_IOAPIC (0x03)
+
+/* Device scope structure for DRHD. */
+struct AcpiDmarDeviceScope {
+ uint8_t entry_type;
+ uint8_t length;
+ uint16_t reserved;
+ uint8_t enumeration_id;
+ uint8_t bus;
+ uint16_t path[0]; /* list of dev:func pairs */
+} QEMU_PACKED;
+typedef struct AcpiDmarDeviceScope AcpiDmarDeviceScope;
+
/* Type 0: Hardware Unit Definition */
struct AcpiDmarHardwareUnit {
uint16_t type;
@@ -577,6 +591,7 @@ struct AcpiDmarHardwareUnit {
uint8_t reserved;
uint16_t pci_segment; /* The PCI Segment associated with this unit */
uint64_t address; /* Base address of remapping hardware register-set */
+ AcpiDmarDeviceScope scope[0];
} QEMU_PACKED;
typedef struct AcpiDmarHardwareUnit AcpiDmarHardwareUnit;
diff --git a/include/hw/pci-host/q35.h b/include/hw/pci-host/q35.h
index c5c073d..9afc221 100644
--- a/include/hw/pci-host/q35.h
+++ b/include/hw/pci-host/q35.h
@@ -175,4 +175,13 @@ typedef struct Q35PCIHost {
uint64_t mch_mcfg_base(void);
+/*
+ * Arbitary but unique BNF number for IOAPIC device. This is only
+ * used when interrupt remapping is enabled.
+ *
+ * TODO: make sure there would have no conflict with real PCI bus
+ */
+#define Q35_PSEUDO_BUS_PLATFORM (0xff)
+#define Q35_PSEUDO_DEVFN_IOAPIC (0x00)
+
#endif /* HW_Q35_H */
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 05/25] intel_iommu: define interrupt remap table addr register
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (3 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 04/25] acpi: add DMAR scope definition for root IOAPIC Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 06/25] intel_iommu: handle interrupt remap enable Peter Xu
` (20 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Defined Interrupt Remap Table Address register to store IR table
pointer. Also, do proper handling on global command register writes to
store table pointer and its size.
One more debug flag "DEBUG_IR" is added for interrupt remapping.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 52 +++++++++++++++++++++++++++++++++++++++++-
hw/i386/intel_iommu_internal.h | 4 ++++
include/hw/i386/intel_iommu.h | 5 ++++
3 files changed, 60 insertions(+), 1 deletion(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 17668d6..00b873c 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -30,7 +30,7 @@
#ifdef DEBUG_INTEL_IOMMU
enum {
DEBUG_GENERAL, DEBUG_CSR, DEBUG_INV, DEBUG_MMU, DEBUG_FLOG,
- DEBUG_CACHE,
+ DEBUG_CACHE, DEBUG_IR,
};
#define VTD_DBGBIT(x) (1 << DEBUG_##x)
static int vtd_dbgflags = VTD_DBGBIT(GENERAL) | VTD_DBGBIT(CSR);
@@ -900,6 +900,19 @@ static void vtd_root_table_setup(IntelIOMMUState *s)
(s->root_extended ? "(extended)" : ""));
}
+static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s)
+{
+ uint64_t value = 0;
+ value = vtd_get_quad_raw(s, DMAR_IRTA_REG);
+ s->intr_size = 1UL << ((value & VTD_IRTA_SIZE_MASK) + 1);
+ s->intr_root = value & VTD_IRTA_ADDR_MASK;
+
+ /* TODO: invalidate interrupt entry cache */
+
+ VTD_DPRINTF(CSR, "int remap table addr 0x%"PRIx64 " size %"PRIu32,
+ s->intr_root, s->intr_size);
+}
+
static void vtd_context_global_invalidate(IntelIOMMUState *s)
{
s->context_cache_gen++;
@@ -1138,6 +1151,16 @@ static void vtd_handle_gcmd_srtp(IntelIOMMUState *s)
vtd_set_clear_mask_long(s, DMAR_GSTS_REG, 0, VTD_GSTS_RTPS);
}
+/* Set Interrupt Remap Table Pointer */
+static void vtd_handle_gcmd_sirtp(IntelIOMMUState *s)
+{
+ VTD_DPRINTF(CSR, "set Interrupt Remap Table Pointer");
+
+ vtd_interrupt_remap_table_setup(s);
+ /* Ok - report back to driver */
+ vtd_set_clear_mask_long(s, DMAR_GSTS_REG, 0, VTD_GSTS_IRTPS);
+}
+
/* Handle Translation Enable/Disable */
static void vtd_handle_gcmd_te(IntelIOMMUState *s, bool en)
{
@@ -1177,6 +1200,10 @@ static void vtd_handle_gcmd_write(IntelIOMMUState *s)
/* Queued Invalidation Enable */
vtd_handle_gcmd_qie(s, val & VTD_GCMD_QIE);
}
+ if (val & VTD_GCMD_SIRTP) {
+ /* Set/update the interrupt remapping root-table pointer */
+ vtd_handle_gcmd_sirtp(s);
+ }
}
/* Handle write to Context Command Register */
@@ -1838,6 +1865,23 @@ static void vtd_mem_write(void *opaque, hwaddr addr,
vtd_update_fsts_ppf(s);
break;
+ case DMAR_IRTA_REG:
+ VTD_DPRINTF(IR, "DMAR_IRTA_REG write addr 0x%"PRIx64
+ ", size %d, val 0x%"PRIx64, addr, size, val);
+ if (size == 4) {
+ vtd_set_long(s, addr, val);
+ } else {
+ vtd_set_quad(s, addr, val);
+ }
+ break;
+
+ case DMAR_IRTA_REG_HI:
+ VTD_DPRINTF(IR, "DMAR_IRTA_REG_HI write addr 0x%"PRIx64
+ ", size %d, val 0x%"PRIx64, addr, size, val);
+ assert(size == 4);
+ vtd_set_long(s, addr, val);
+ break;
+
default:
VTD_DPRINTF(GENERAL, "error: unhandled reg write addr 0x%"PRIx64
", size %d, val 0x%"PRIx64, addr, size, val);
@@ -2017,6 +2061,12 @@ static void vtd_init(IntelIOMMUState *s)
/* Fault Recording Registers, 128-bit */
vtd_define_quad(s, DMAR_FRCD_REG_0_0, 0, 0, 0);
vtd_define_quad(s, DMAR_FRCD_REG_0_2, 0, 0, 0x8000000000000000ULL);
+
+ /*
+ * Interrupt remapping registers, not support extended interrupt
+ * mode for now.
+ */
+ vtd_define_quad(s, DMAR_IRTA_REG, 0, 0xfffffffffffff00fULL, 0);
}
/* Should not reset address_spaces when reset because devices will still use
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 5b98a11..309833f 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -172,6 +172,10 @@
#define VTD_RTADDR_RTT (1ULL << 11)
#define VTD_RTADDR_ADDR_MASK (VTD_HAW_MASK ^ 0xfffULL)
+/* IRTA_REG */
+#define VTD_IRTA_ADDR_MASK (VTD_HAW_MASK ^ 0xfffULL)
+#define VTD_IRTA_SIZE_MASK (0xfULL)
+
/* ECAP_REG */
/* (offset >> 4) << 8 */
#define VTD_ECAP_IRO (DMAR_IOTLB_REG_OFFSET << 4)
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 0d89796..cc49839 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -125,6 +125,11 @@ struct IntelIOMMUState {
MemoryRegionIOMMUOps iommu_ops;
GHashTable *vtd_as_by_busptr; /* VTDBus objects indexed by PCIBus* reference */
VTDBus *vtd_as_by_bus_num[VTD_PCI_BUS_MAX]; /* VTDBus objects indexed by bus number */
+
+ /* interrupt remapping */
+ bool intr_enabled; /* Whether guest enabled IR */
+ dma_addr_t intr_root; /* Interrupt remapping table pointer */
+ uint32_t intr_size; /* Number of IR table entries */
};
/* Find the VTD Address space associated with the given bus pointer,
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 06/25] intel_iommu: handle interrupt remap enable
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (4 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 05/25] intel_iommu: define interrupt remap table addr register Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR Peter Xu
` (19 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Handle writting to IRE bit in global command register.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 00b873c..4d14124 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1180,6 +1180,22 @@ static void vtd_handle_gcmd_te(IntelIOMMUState *s, bool en)
}
}
+/* Handle Interrupt Remap Enable/Disable */
+static void vtd_handle_gcmd_ire(IntelIOMMUState *s, bool en)
+{
+ VTD_DPRINTF(CSR, "Interrupt Remap Enable %s", (en ? "on" : "off"));
+
+ if (en) {
+ s->intr_enabled = true;
+ /* Ok - report back to driver */
+ vtd_set_clear_mask_long(s, DMAR_GSTS_REG, 0, VTD_GSTS_IRES);
+ } else {
+ s->intr_enabled = false;
+ /* Ok - report back to driver */
+ vtd_set_clear_mask_long(s, DMAR_GSTS_REG, VTD_GSTS_IRES, 0);
+ }
+}
+
/* Handle write to Global Command Register */
static void vtd_handle_gcmd_write(IntelIOMMUState *s)
{
@@ -1204,6 +1220,10 @@ static void vtd_handle_gcmd_write(IntelIOMMUState *s)
/* Set/update the interrupt remapping root-table pointer */
vtd_handle_gcmd_sirtp(s);
}
+ if (changed & VTD_GCMD_IRE) {
+ /* Interrupt remap enable/disable */
+ vtd_handle_gcmd_ire(s, val & VTD_GCMD_IRE);
+ }
}
/* Handle write to Context Command Register */
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (5 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 06/25] intel_iommu: handle interrupt remap enable Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-29 8:20 ` David Kiarie
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class Peter Xu
` (18 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Several data structs are defined to better support the rest of the
patches: IRTE to parse remapping table entries, and IOAPIC/MSI related
structure bits to parse interrupt entries to be filled in by guest
kernel.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
include/hw/i386/intel_iommu.h | 60 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index cc49839..4914fe6 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -52,6 +52,9 @@ typedef struct IntelIOMMUState IntelIOMMUState;
typedef struct VTDAddressSpace VTDAddressSpace;
typedef struct VTDIOTLBEntry VTDIOTLBEntry;
typedef struct VTDBus VTDBus;
+typedef union VTD_IRTE VTD_IRTE;
+typedef union VTD_IR_IOAPICEntry VTD_IR_IOAPICEntry;
+typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress;
/* Context-Entry */
struct VTDContextEntry {
@@ -90,6 +93,63 @@ struct VTDIOTLBEntry {
bool write_flags;
};
+/* Interrupt Remapping Table Entry Definition */
+union VTD_IRTE {
+ struct {
+ uint8_t present:1; /* Whether entry present/available */
+ uint8_t fault_disable:1; /* Fault Processing Disable */
+ uint8_t dest_mode:1; /* Destination Mode */
+ uint8_t redir_hint:1; /* Redirection Hint */
+ uint8_t trigger_mode:1; /* Trigger Mode */
+ uint8_t delivery_mode:3; /* Delivery Mode */
+ uint8_t __avail:4; /* Available spaces for software */
+ uint8_t __reserved_0:3; /* Reserved 0 */
+ uint8_t irte_mode:1; /* IRTE Mode */
+ uint8_t vector:8; /* Interrupt Vector */
+ uint8_t __reserved_1:8; /* Reserved 1 */
+ uint32_t dest_id:32; /* Destination ID */
+ uint16_t source_id:16; /* Source-ID */
+ uint8_t sid_q:2; /* Source-ID Qualifier */
+ uint8_t sid_vtype:2; /* Source-ID Validation Type */
+ uint64_t __reserved_2:44; /* Reserved 2 */
+ } QEMU_PACKED;
+ uint64_t data[2];
+};
+
+/* Programming format for IOAPIC table entries */
+union VTD_IR_IOAPICEntry {
+ struct {
+ uint8_t vector:8; /* Vector */
+ uint8_t __zeros:3; /* Reserved (all zero) */
+ uint8_t index_h:1; /* Interrupt Index bit 15 */
+ uint8_t status:1; /* Deliver Status */
+ uint8_t polarity:1; /* Interrupt Polarity */
+ uint8_t remote_irr:1; /* Remote IRR */
+ uint8_t trigger_mode:1; /* Trigger Mode */
+ uint8_t mask:1; /* Mask */
+ uint32_t __reserved:31; /* Reserved (should all zero) */
+ uint8_t int_mode:1; /* Interrupt Format */
+ uint16_t index_l:15; /* Interrupt Index bits 14-0 */
+ } QEMU_PACKED;
+ uint64_t data;
+};
+
+/* Programming format for MSI/MSI-X addresses */
+union VTD_IR_MSIAddress {
+ struct {
+ uint8_t __not_care:2;
+ uint8_t index_h:1; /* Interrupt index bit 15 */
+ uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
+ uint8_t int_mode:1; /* Interrupt format */
+ uint16_t index_l:15; /* Interrupt index bit 14-0 */
+ uint16_t __head:12; /* Should always be: 0x0fee */
+ } QEMU_PACKED;
+ uint32_t data;
+};
+
+/* When IR is enabled, all MSI/MSI-X data bits should be zero */
+#define VTD_IR_MSI_DATA (0)
+
/* The iommu (DMAR) device state struct */
struct IntelIOMMUState {
SysBusDevice busdev;
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (6 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-23 17:06 ` David Kiarie
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 09/25] x86-iommu: provide x86_iommu_get_default Peter Xu
` (17 subsequent siblings)
25 siblings, 1 reply; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Introducing parent class for intel-iommu devices named "x86-iommu". This
is preparation work to abstract shared functionalities out from Intel
and AMD IOMMUs. Currently, only the parent class is introduced. It does
nothing yet.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/Makefile.objs | 2 +-
hw/i386/intel_iommu.c | 5 ++--
hw/i386/x86-iommu.c | 53 +++++++++++++++++++++++++++++++++++++++++++
include/hw/i386/intel_iommu.h | 3 ++-
include/hw/i386/x86-iommu.h | 46 +++++++++++++++++++++++++++++++++++++
5 files changed, 105 insertions(+), 4 deletions(-)
create mode 100644 hw/i386/x86-iommu.c
create mode 100644 include/hw/i386/x86-iommu.h
diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs
index b52d5b8..90e94ff 100644
--- a/hw/i386/Makefile.objs
+++ b/hw/i386/Makefile.objs
@@ -2,7 +2,7 @@ obj-$(CONFIG_KVM) += kvm/
obj-y += multiboot.o
obj-y += pc.o pc_piix.o pc_q35.o
obj-y += pc_sysfw.o
-obj-y += intel_iommu.o
+obj-y += x86-iommu.o intel_iommu.o
obj-$(CONFIG_XEN) += ../xenpv/ xen/
obj-y += kvmvapic.o
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 4d14124..0a70577 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2120,16 +2120,17 @@ static void vtd_realize(DeviceState *dev, Error **errp)
static void vtd_class_init(ObjectClass *klass, void *data)
{
DeviceClass *dc = DEVICE_CLASS(klass);
+ X86IOMMUClass *x86_class = X86_IOMMU_CLASS(klass);
dc->reset = vtd_reset;
- dc->realize = vtd_realize;
dc->vmsd = &vtd_vmstate;
dc->props = vtd_properties;
+ x86_class->realize = vtd_realize;
}
static const TypeInfo vtd_info = {
.name = TYPE_INTEL_IOMMU_DEVICE,
- .parent = TYPE_SYS_BUS_DEVICE,
+ .parent = TYPE_X86_IOMMU_DEVICE,
.instance_size = sizeof(IntelIOMMUState),
.class_init = vtd_class_init,
};
diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
new file mode 100644
index 0000000..d739afb
--- /dev/null
+++ b/hw/i386/x86-iommu.c
@@ -0,0 +1,53 @@
+/*
+ * QEMU emulation of common X86 IOMMU
+ *
+ * Copyright (C) 2016 Peter Xu, Red Hat <peterx@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/sysbus.h"
+#include "hw/boards.h"
+#include "hw/i386/x86-iommu.h"
+
+static void x86_iommu_realize(DeviceState *dev, Error **errp)
+{
+ X86IOMMUClass *x86_class = X86_IOMMU_GET_CLASS(dev);
+ if (x86_class->realize) {
+ x86_class->realize(dev, errp);
+ }
+}
+
+static void x86_iommu_class_init(ObjectClass *klass, void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ dc->realize = x86_iommu_realize;
+}
+
+static const TypeInfo x86_iommu_info = {
+ .name = TYPE_X86_IOMMU_DEVICE,
+ .parent = TYPE_SYS_BUS_DEVICE,
+ .instance_size = sizeof(X86IOMMUState),
+ .class_init = x86_iommu_class_init,
+ .class_size = sizeof(X86IOMMUClass),
+ .abstract = true,
+};
+
+static void x86_iommu_register_types(void)
+{
+ type_register_static(&x86_iommu_info);
+}
+
+type_init(x86_iommu_register_types)
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 4914fe6..c88a931 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -23,6 +23,7 @@
#define INTEL_IOMMU_H
#include "hw/qdev.h"
#include "sysemu/dma.h"
+#include "hw/i386/x86-iommu.h"
#define TYPE_INTEL_IOMMU_DEVICE "intel-iommu"
#define INTEL_IOMMU_DEVICE(obj) \
@@ -152,7 +153,7 @@ union VTD_IR_MSIAddress {
/* The iommu (DMAR) device state struct */
struct IntelIOMMUState {
- SysBusDevice busdev;
+ X86IOMMUState x86_iommu;
MemoryRegion csrmem;
uint8_t csr[DMAR_REG_SIZE]; /* register values */
uint8_t wmask[DMAR_REG_SIZE]; /* R/W bytes */
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
new file mode 100644
index 0000000..924f39a
--- /dev/null
+++ b/include/hw/i386/x86-iommu.h
@@ -0,0 +1,46 @@
+/*
+ * Common IOMMU interface for X86 platform
+ *
+ * Copyright (C) 2016 Peter Xu, Red Hat <peterx@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef IOMMU_COMMON_H
+#define IOMMU_COMMON_H
+
+#include "hw/sysbus.h"
+
+#define TYPE_X86_IOMMU_DEVICE ("x86-iommu")
+#define X86_IOMMU_DEVICE(obj) \
+ OBJECT_CHECK(X86IOMMUState, (obj), TYPE_X86_IOMMU_DEVICE)
+#define X86_IOMMU_CLASS(klass) \
+ OBJECT_CLASS_CHECK(X86IOMMUClass, (klass), TYPE_X86_IOMMU_DEVICE)
+#define X86_IOMMU_GET_CLASS(obj) \
+ OBJECT_GET_CLASS(X86IOMMUClass, obj, TYPE_X86_IOMMU_DEVICE)
+
+typedef struct X86IOMMUState X86IOMMUState;
+typedef struct X86IOMMUClass X86IOMMUClass;
+
+struct X86IOMMUClass {
+ SysBusDeviceClass parent;
+ /* Intel/AMD specific realize() hook */
+ DeviceRealize realize;
+};
+
+struct X86IOMMUState {
+ SysBusDevice busdev;
+};
+
+#endif
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 09/25] x86-iommu: provide x86_iommu_get_default
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (7 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 10/25] x86-iommu: q35: generalize find_add_as() Peter Xu
` (16 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Instead of searching the device tree every time, one static variable is
declared for the default system x86 IOMMU device. Also, some VT-d
macros are replaced by x86 ones.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/acpi-build.c | 9 ++-------
hw/i386/intel_iommu.c | 8 +++++---
hw/i386/x86-iommu.c | 16 ++++++++++++++++
hw/pci-host/q35.c | 2 +-
include/hw/i386/intel_iommu.h | 1 -
include/hw/i386/x86-iommu.h | 9 +++++++++
6 files changed, 33 insertions(+), 12 deletions(-)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 6c572a3..9af1da0 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -51,7 +51,7 @@
#include "hw/i386/ich9.h"
#include "hw/pci/pci_bus.h"
#include "hw/pci-host/q35.h"
-#include "hw/i386/intel_iommu.h"
+#include "hw/i386/x86-iommu.h"
#include "hw/timer/hpet.h"
#include "hw/acpi/aml-build.h"
@@ -2656,12 +2656,7 @@ static bool acpi_get_mcfg(AcpiMcfgInfo *mcfg)
static bool acpi_has_iommu(void)
{
- bool ambiguous;
- Object *intel_iommu;
-
- intel_iommu = object_resolve_path_type("", TYPE_INTEL_IOMMU_DEVICE,
- &ambiguous);
- return intel_iommu && !ambiguous;
+ return !!x86_iommu_get_default();
}
static
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 0a70577..0c7b24d 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -25,6 +25,7 @@
#include "intel_iommu_internal.h"
#include "hw/pci/pci.h"
#include "hw/boards.h"
+#include "hw/i386/x86-iommu.h"
/*#define DEBUG_INTEL_IOMMU*/
#ifdef DEBUG_INTEL_IOMMU
@@ -191,7 +192,7 @@ static void vtd_reset_context_cache(IntelIOMMUState *s)
VTD_DPRINTF(CACHE, "global context_cache_gen=1");
while (g_hash_table_iter_next (&bus_it, NULL, (void**)&vtd_bus)) {
- for (devfn_it = 0; devfn_it < VTD_PCI_DEVFN_MAX; ++devfn_it) {
+ for (devfn_it = 0; devfn_it < X86_IOMMU_PCI_DEVFN_MAX; ++devfn_it) {
vtd_as = vtd_bus->dev_as[devfn_it];
if (!vtd_as) {
continue;
@@ -976,7 +977,7 @@ static void vtd_context_device_invalidate(IntelIOMMUState *s,
vtd_bus = vtd_find_as_from_bus_num(s, VTD_SID_TO_BUS(source_id));
if (vtd_bus) {
devfn = VTD_SID_TO_DEVFN(source_id);
- for (devfn_it = 0; devfn_it < VTD_PCI_DEVFN_MAX; ++devfn_it) {
+ for (devfn_it = 0; devfn_it < X86_IOMMU_PCI_DEVFN_MAX; ++devfn_it) {
vtd_as = vtd_bus->dev_as[devfn_it];
if (vtd_as && ((devfn_it & mask) == (devfn & mask))) {
VTD_DPRINTF(INV, "invalidate context-cahce of devfn 0x%"PRIx16,
@@ -1978,7 +1979,8 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
if (!vtd_bus) {
/* No corresponding free() */
- vtd_bus = g_malloc0(sizeof(VTDBus) + sizeof(VTDAddressSpace *) * VTD_PCI_DEVFN_MAX);
+ vtd_bus = g_malloc0(sizeof(VTDBus) + sizeof(VTDAddressSpace *) * \
+ X86_IOMMU_PCI_DEVFN_MAX);
vtd_bus->bus = bus;
key = (uintptr_t)bus;
g_hash_table_insert(s->vtd_as_by_busptr, &key, vtd_bus);
diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
index d739afb..2d6d221 100644
--- a/hw/i386/x86-iommu.c
+++ b/hw/i386/x86-iommu.c
@@ -22,12 +22,28 @@
#include "hw/boards.h"
#include "hw/i386/x86-iommu.h"
+/* Default X86 IOMMU device */
+static X86IOMMUState *x86_iommu_default = NULL;
+
+static void x86_iommu_set_default(X86IOMMUState *x86_iommu)
+{
+ assert(x86_iommu);
+ assert(x86_iommu_default == NULL);
+ x86_iommu_default = x86_iommu;
+}
+
+X86IOMMUState *x86_iommu_get_default(void)
+{
+ return x86_iommu_default;
+}
+
static void x86_iommu_realize(DeviceState *dev, Error **errp)
{
X86IOMMUClass *x86_class = X86_IOMMU_GET_CLASS(dev);
if (x86_class->realize) {
x86_class->realize(dev, errp);
}
+ x86_iommu_set_default(X86_IOMMU_DEVICE(dev));
}
static void x86_iommu_class_init(ObjectClass *klass, void *data)
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 70f897e..27ee0c8 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -429,7 +429,7 @@ static AddressSpace *q35_host_dma_iommu(PCIBus *bus, void *opaque, int devfn)
IntelIOMMUState *s = opaque;
VTDAddressSpace *vtd_as;
- assert(0 <= devfn && devfn <= VTD_PCI_DEVFN_MAX);
+ assert(0 <= devfn && devfn <= X86_IOMMU_PCI_DEVFN_MAX);
vtd_as = vtd_find_add_as(s, bus, devfn);
return &vtd_as->as;
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index c88a931..04265ca 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -35,7 +35,6 @@
#define VTD_PCI_BUS_MAX 256
#define VTD_PCI_SLOT_MAX 32
#define VTD_PCI_FUNC_MAX 8
-#define VTD_PCI_DEVFN_MAX 256
#define VTD_PCI_SLOT(devfn) (((devfn) >> 3) & 0x1f)
#define VTD_PCI_FUNC(devfn) ((devfn) & 0x07)
#define VTD_SID_TO_BUS(sid) (((sid) >> 8) & 0xff)
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index 924f39a..d6991cb 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -30,6 +30,9 @@
#define X86_IOMMU_GET_CLASS(obj) \
OBJECT_GET_CLASS(X86IOMMUClass, obj, TYPE_X86_IOMMU_DEVICE)
+#define X86_IOMMU_PCI_DEVFN_MAX 256
+#define X86_IOMMU_SID_INVALID (0xffff)
+
typedef struct X86IOMMUState X86IOMMUState;
typedef struct X86IOMMUClass X86IOMMUClass;
@@ -43,4 +46,10 @@ struct X86IOMMUState {
SysBusDevice busdev;
};
+/**
+ * x86_iommu_get_default - get default IOMMU device
+ * @return: pointer to default IOMMU device
+ */
+X86IOMMUState *x86_iommu_get_default(void);
+
#endif
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 10/25] x86-iommu: q35: generalize find_add_as()
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (8 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 09/25] x86-iommu: provide x86_iommu_get_default Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 11/25] intel_iommu: add IR translation faults defines Peter Xu
` (15 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Remove VT-d calls in common q35 codes. Instead, we provide a general
find_add_as() for x86-iommu type.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 7 +++++--
hw/pci-host/q35.c | 10 ++++------
include/hw/i386/intel_iommu.h | 5 -----
include/hw/i386/x86-iommu.h | 3 +++
4 files changed, 12 insertions(+), 13 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 0c7b24d..38cecae 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1971,8 +1971,10 @@ static Property vtd_properties[] = {
};
-VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
+static AddressSpace *vtd_find_add_as(X86IOMMUState *x86_iommu, PCIBus *bus,
+ int devfn)
{
+ IntelIOMMUState *s = (IntelIOMMUState *)x86_iommu;
uintptr_t key = (uintptr_t)bus;
VTDBus *vtd_bus = g_hash_table_lookup(s->vtd_as_by_busptr, &key);
VTDAddressSpace *vtd_dev_as;
@@ -2000,7 +2002,7 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn)
address_space_init(&vtd_dev_as->as,
&vtd_dev_as->iommu, "intel_iommu");
}
- return vtd_dev_as;
+ return &vtd_dev_as->as;
}
/* Do the initialization. It will also be called when reset, so pay
@@ -2128,6 +2130,7 @@ static void vtd_class_init(ObjectClass *klass, void *data)
dc->vmsd = &vtd_vmstate;
dc->props = vtd_properties;
x86_class->realize = vtd_realize;
+ x86_class->find_add_as = vtd_find_add_as;
}
static const TypeInfo vtd_info = {
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 27ee0c8..6835da1 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -426,13 +426,11 @@ static void mch_reset(DeviceState *qdev)
static AddressSpace *q35_host_dma_iommu(PCIBus *bus, void *opaque, int devfn)
{
- IntelIOMMUState *s = opaque;
- VTDAddressSpace *vtd_as;
+ X86IOMMUState *x86_iommu = opaque;
+ X86IOMMUClass *x86_class = X86_IOMMU_GET_CLASS(x86_iommu);
assert(0 <= devfn && devfn <= X86_IOMMU_PCI_DEVFN_MAX);
-
- vtd_as = vtd_find_add_as(s, bus, devfn);
- return &vtd_as->as;
+ return x86_class->find_add_as(x86_iommu, bus, devfn);
}
static void mch_init_dmar(MCHPCIState *mch)
@@ -440,7 +438,7 @@ static void mch_init_dmar(MCHPCIState *mch)
PCIBus *pci_bus = PCI_BUS(qdev_get_parent_bus(DEVICE(mch)));
mch->iommu = INTEL_IOMMU_DEVICE(qdev_create(NULL, TYPE_INTEL_IOMMU_DEVICE));
- object_property_add_child(OBJECT(mch), "intel-iommu",
+ object_property_add_child(OBJECT(mch), TYPE_X86_IOMMU_DEVICE,
OBJECT(mch->iommu), NULL);
qdev_init_nofail(DEVICE(mch->iommu));
sysbus_mmio_map(SYS_BUS_DEVICE(mch->iommu), 0, Q35_HOST_BRIDGE_IOMMU_ADDR);
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 04265ca..ee7620c 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -192,9 +192,4 @@ struct IntelIOMMUState {
uint32_t intr_size; /* Number of IR table entries */
};
-/* Find the VTD Address space associated with the given bus pointer,
- * create a new one if none exists
- */
-VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, int devfn);
-
#endif
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index d6991cb..2070cd1 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -21,6 +21,7 @@
#define IOMMU_COMMON_H
#include "hw/sysbus.h"
+#include "exec/memory.h"
#define TYPE_X86_IOMMU_DEVICE ("x86-iommu")
#define X86_IOMMU_DEVICE(obj) \
@@ -40,6 +41,8 @@ struct X86IOMMUClass {
SysBusDeviceClass parent;
/* Intel/AMD specific realize() hook */
DeviceRealize realize;
+ /* Find/Add IOMMU address space for specific PCI device */
+ AddressSpace *(*find_add_as)(X86IOMMUState *s, PCIBus *bus, int devfn);
};
struct X86IOMMUState {
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 11/25] intel_iommu: add IR translation faults defines
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (9 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 10/25] x86-iommu: q35: generalize find_add_as() Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 12/25] intel_iommu: Add support for PCI MSI remap Peter Xu
` (14 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Adding translation fault definitions for interrupt remapping. Please
refer to VT-d spec section 7.1.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu_internal.h | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 309833f..2a9987f 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -271,6 +271,19 @@ typedef enum VTDFaultReason {
* context-entry.
*/
VTD_FR_CONTEXT_ENTRY_TT,
+
+ /* Interrupt remapping transition faults */
+ VTD_FR_IR_REQ_RSVD = 0x20, /* One or more IR request reserved
+ * fields set */
+ VTD_FR_IR_INDEX_OVER = 0x21, /* Index value greater than max */
+ VTD_FR_IR_ENTRY_P = 0x22, /* Present (P) not set in IRTE */
+ VTD_FR_IR_ROOT_INVAL = 0x23, /* IR Root table invalid */
+ VTD_FR_IR_IRTE_RSVD = 0x24, /* IRTE Rsvd field non-zero with
+ * Present flag set */
+ VTD_FR_IR_REQ_COMPAT = 0x25, /* Encountered compatible IR
+ * request while disabled */
+ VTD_FR_IR_SID_ERR = 0x26, /* Invalid Source-ID */
+
/* This is not a normal fault reason. We use this to indicate some faults
* that are not referenced by the VT-d specification.
* Fault event with such reason should not be recorded.
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 12/25] intel_iommu: Add support for PCI MSI remap
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (10 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 11/25] intel_iommu: add IR translation faults defines Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 13/25] q35: ioapic: add support for emulated IOAPIC IR Peter Xu
` (13 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
This patch enables interrupt remapping for PCI devices.
To play the trick, one memory region "iommu_ir" is added as child region
of the original iommu memory region, covering range 0xfeeXXXXX (which is
the address range for APIC). All the writes to this range will be taken
as MSI, and translation is carried out only when IR is enabled.
Idea suggested by Paolo Bonzini.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 243 +++++++++++++++++++++++++++++++++++++++++
hw/i386/intel_iommu_internal.h | 2 +
include/hw/i386/intel_iommu.h | 52 +++++++++
3 files changed, 297 insertions(+)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 38cecae..b5137ce 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1970,6 +1970,244 @@ static Property vtd_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+/* Read IRTE entry with specific index */
+static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index,
+ VTD_IRTE *entry)
+{
+ dma_addr_t addr = 0x00;
+
+ addr = iommu->intr_root + index * sizeof(*entry);
+ if (dma_memory_read(&address_space_memory, addr, entry,
+ sizeof(*entry))) {
+ VTD_DPRINTF(GENERAL, "error: fail to access IR root at 0x%"PRIx64
+ " + %"PRIu16, iommu->intr_root, index);
+ return -VTD_FR_IR_ROOT_INVAL;
+ }
+
+ if (!entry->present) {
+ VTD_DPRINTF(GENERAL, "error: present flag not set in IRTE"
+ " entry index %u value 0x%"PRIx64 " 0x%"PRIx64,
+ index, le64_to_cpu(entry->data[1]),
+ le64_to_cpu(entry->data[0]));
+ return -VTD_FR_IR_ENTRY_P;
+ }
+
+ if (entry->__reserved_0 || entry->__reserved_1 || \
+ entry->__reserved_2) {
+ VTD_DPRINTF(GENERAL, "error: IRTE entry index %"PRIu16
+ " reserved fields non-zero: 0x%"PRIx64 " 0x%"PRIx64,
+ index, le64_to_cpu(entry->data[1]),
+ le64_to_cpu(entry->data[0]));
+ return -VTD_FR_IR_IRTE_RSVD;
+ }
+
+ /*
+ * TODO: Check Source-ID corresponds to SVT (Source Validation
+ * Type) bits
+ */
+
+ return 0;
+}
+
+/* Fetch IRQ information of specific IR index */
+static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index, VTDIrq *irq)
+{
+ VTD_IRTE irte;
+ int ret = 0;
+
+ bzero(&irte, sizeof(irte));
+
+ ret = vtd_irte_get(iommu, index, &irte);
+ if (ret) {
+ return ret;
+ }
+
+ irq->trigger_mode = irte.trigger_mode;
+ irq->vector = irte.vector;
+ irq->delivery_mode = irte.delivery_mode;
+ /* Not support EIM yet: please refer to vt-d 9.10 DST bits */
+#define VTD_IR_APIC_DEST_MASK (0xff00ULL)
+#define VTD_IR_APIC_DEST_SHIFT (8)
+ irq->dest = (irte.dest_id & VTD_IR_APIC_DEST_MASK) >> \
+ VTD_IR_APIC_DEST_SHIFT;
+ irq->dest_mode = irte.dest_mode;
+ irq->redir_hint = irte.redir_hint;
+
+ VTD_DPRINTF(IR, "remapping interrupt index %d: trig:%u,vec:%u,"
+ "deliver:%u,dest:%u,dest_mode:%u", index,
+ irq->trigger_mode, irq->vector, irq->delivery_mode,
+ irq->dest, irq->dest_mode);
+
+ return 0;
+}
+
+/* Generate one MSI message from VTDIrq info */
+static void vtd_generate_msi_message(VTDIrq *irq, MSIMessage *msg_out)
+{
+ VTD_MSIMessage msg = {};
+
+ /* Generate address bits */
+ msg.dest_mode = irq->dest_mode;
+ msg.redir_hint = irq->redir_hint;
+ msg.dest = irq->dest;
+ msg.__addr_head = 0xfee;
+ /* Keep this from original MSI address bits */
+ msg.__not_used = irq->msi_addr_last_bits;
+
+ /* Generate data bits */
+ msg.vector = irq->vector;
+ msg.delivery_mode = irq->delivery_mode;
+ msg.level = 1;
+ msg.trigger_mode = irq->trigger_mode;
+
+ msg_out->address = msg.msi_addr;
+ msg_out->data = msg.msi_data;
+}
+
+/* Interrupt remapping for MSI/MSI-X entry */
+static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu,
+ MSIMessage *origin,
+ MSIMessage *translated)
+{
+ int ret = 0;
+ VTD_IR_MSIAddress addr;
+ uint16_t index = 0;
+ VTDIrq irq = {0};
+
+ assert(origin && translated);
+
+ if (!iommu || !iommu->intr_enabled) {
+ goto do_not_translate;
+ }
+
+ if (origin->address & VTD_MSI_ADDR_HI_MASK) {
+ VTD_DPRINTF(GENERAL, "error: MSI addr high 32 bits nonzero"
+ " during interrupt remapping: 0x%"PRIx32,
+ (uint32_t)((origin->address & VTD_MSI_ADDR_HI_MASK) >> \
+ VTD_MSI_ADDR_HI_SHIFT));
+ return -VTD_FR_IR_REQ_RSVD;
+ }
+
+ addr.data = origin->address & VTD_MSI_ADDR_LO_MASK;
+ if (addr.__head != 0xfee) {
+ VTD_DPRINTF(GENERAL, "error: MSI addr low 32 bits invalid: "
+ "0x%"PRIx32, addr.data);
+ return -VTD_FR_IR_REQ_RSVD;
+ }
+
+ /* This is compatible mode. */
+ if (addr.int_mode != VTD_IR_INT_FORMAT_REMAP) {
+ goto do_not_translate;
+ }
+
+ index = addr.index_h << 15 | addr.index_l;
+
+ ret = vtd_remap_irq_get(iommu, index, &irq);
+ if (ret) {
+ return ret;
+ }
+
+ if (addr.sub_valid == 1) {
+ VTD_DPRINTF(IR, "received MSI interrupt");
+ if (origin->data) {
+ VTD_DPRINTF(GENERAL, "error: MSI data bits non-zero for "
+ "interrupt remappable entry: 0x%"PRIx32,
+ origin->data);
+ return -VTD_FR_IR_REQ_RSVD;
+ }
+ } else {
+ uint8_t vector = origin->data & 0xff;
+ VTD_DPRINTF(IR, "received IOAPIC interrupt");
+ /* IOAPIC entry vector should be aligned with IRTE vector
+ * (see vt-d spec 5.1.5.1). */
+ if (vector != irq.vector) {
+ VTD_DPRINTF(GENERAL, "IOAPIC vector inconsistent: "
+ "entry: %d, IRTE: %d, index: %d",
+ vector, irq.vector, index);
+ }
+ }
+
+ /*
+ * We'd better keep the last two bits, assuming that guest OS
+ * might modify it. Keep it does not hurt after all.
+ */
+ irq.msi_addr_last_bits = addr.__not_care;
+
+ /* Translate VTDIrq to MSI message */
+ vtd_generate_msi_message(&irq, translated);
+
+ VTD_DPRINTF(IR, "mapping MSI 0x%"PRIx64":0x%"PRIx32 " -> "
+ "0x%"PRIx64":0x%"PRIx32, origin->address, origin->data,
+ translated->address, translated->data);
+ return 0;
+
+do_not_translate:
+ memcpy(translated, origin, sizeof(*origin));
+ return 0;
+}
+
+static MemTxResult vtd_mem_ir_read(void *opaque, hwaddr addr,
+ uint64_t *data, unsigned size,
+ MemTxAttrs attrs)
+{
+ addr += VTD_INTERRUPT_ADDR_FIRST;
+
+ VTD_DPRINTF(IR, "read mem_ir addr 0x%"PRIx64 " size %u",
+ addr, size);
+
+ if (dma_memory_read(&address_space_memory, addr, &data, size)) {
+ VTD_DPRINTF(GENERAL, "error: fail to access 0x%"PRIx64, addr);
+ return MEMTX_ERROR;
+ }
+
+ return MEMTX_OK;
+}
+
+static MemTxResult vtd_mem_ir_write(void *opaque, hwaddr addr,
+ uint64_t value, unsigned size,
+ MemTxAttrs attrs)
+{
+ int ret = 0;
+ MSIMessage from = {0}, to = {0};
+
+ from.address = (uint64_t) addr + VTD_INTERRUPT_ADDR_FIRST;
+ from.data = (uint32_t) value;
+
+ ret = vtd_interrupt_remap_msi(opaque, &from, &to);
+ if (ret) {
+ /* TODO: report error */
+ VTD_DPRINTF(GENERAL, "int remap fail for addr 0x%"PRIx64
+ " data 0x%"PRIx32, from.address, from.data);
+ /* Drop this interrupt */
+ return MEMTX_ERROR;
+ }
+
+ VTD_DPRINTF(IR, "delivering MSI 0x%"PRIx64":0x%"PRIx32
+ " for device sid 0x%04x",
+ to.address, to.data, sid);
+
+ if (dma_memory_write(&address_space_memory, to.address,
+ &to.data, size)) {
+ VTD_DPRINTF(GENERAL, "error: fail to write 0x%"PRIx64
+ " value 0x%"PRIx32, to.address, to.data);
+ }
+
+ return MEMTX_OK;
+}
+
+static const MemoryRegionOps vtd_mem_ir_ops = {
+ .read_with_attrs = vtd_mem_ir_read,
+ .write_with_attrs = vtd_mem_ir_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+ .impl = {
+ .min_access_size = 4,
+ .max_access_size = 4,
+ },
+ .valid = {
+ .min_access_size = 4,
+ .max_access_size = 4,
+ },
+};
static AddressSpace *vtd_find_add_as(X86IOMMUState *x86_iommu, PCIBus *bus,
int devfn)
@@ -1999,6 +2237,11 @@ static AddressSpace *vtd_find_add_as(X86IOMMUState *x86_iommu, PCIBus *bus,
vtd_dev_as->context_cache_entry.context_cache_gen = 0;
memory_region_init_iommu(&vtd_dev_as->iommu, OBJECT(s),
&s->iommu_ops, "intel_iommu", UINT64_MAX);
+ memory_region_init_io(&vtd_dev_as->iommu_ir, OBJECT(s),
+ &vtd_mem_ir_ops, s, "intel_iommu_ir",
+ VTD_INTERRUPT_ADDR_SIZE);
+ memory_region_add_subregion(&vtd_dev_as->iommu, VTD_INTERRUPT_ADDR_FIRST,
+ &vtd_dev_as->iommu_ir);
address_space_init(&vtd_dev_as->as,
&vtd_dev_as->iommu, "intel_iommu");
}
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 2a9987f..e1a08cb 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -110,6 +110,8 @@
/* Interrupt Address Range */
#define VTD_INTERRUPT_ADDR_FIRST 0xfee00000ULL
#define VTD_INTERRUPT_ADDR_LAST 0xfeefffffULL
+#define VTD_INTERRUPT_ADDR_SIZE (VTD_INTERRUPT_ADDR_LAST - \
+ VTD_INTERRUPT_ADDR_FIRST + 1)
/* The shift of source_id in the key of IOTLB hash table */
#define VTD_IOTLB_SID_SHIFT 36
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index ee7620c..2018233 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -24,6 +24,8 @@
#include "hw/qdev.h"
#include "sysemu/dma.h"
#include "hw/i386/x86-iommu.h"
+#include "hw/i386/ioapic.h"
+#include "hw/pci/msi.h"
#define TYPE_INTEL_IOMMU_DEVICE "intel-iommu"
#define INTEL_IOMMU_DEVICE(obj) \
@@ -46,6 +48,10 @@
#define DMAR_REPORT_F_INTR (1)
+#define VTD_MSI_ADDR_HI_MASK (0xffffffff00000000ULL)
+#define VTD_MSI_ADDR_HI_SHIFT (32)
+#define VTD_MSI_ADDR_LO_MASK (0x00000000ffffffffULL)
+
typedef struct VTDContextEntry VTDContextEntry;
typedef struct VTDContextCacheEntry VTDContextCacheEntry;
typedef struct IntelIOMMUState IntelIOMMUState;
@@ -55,6 +61,8 @@ typedef struct VTDBus VTDBus;
typedef union VTD_IRTE VTD_IRTE;
typedef union VTD_IR_IOAPICEntry VTD_IR_IOAPICEntry;
typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress;
+typedef struct VTDIrq VTDIrq;
+typedef struct VTD_MSIMessage VTD_MSIMessage;
/* Context-Entry */
struct VTDContextEntry {
@@ -75,6 +83,7 @@ struct VTDAddressSpace {
uint8_t devfn;
AddressSpace as;
MemoryRegion iommu;
+ MemoryRegion iommu_ir; /* Interrupt region: 0xfeeXXXXX */
IntelIOMMUState *iommu_state;
VTDContextCacheEntry context_cache_entry;
};
@@ -116,6 +125,9 @@ union VTD_IRTE {
uint64_t data[2];
};
+#define VTD_IR_INT_FORMAT_COMPAT (0) /* Compatible Interrupt */
+#define VTD_IR_INT_FORMAT_REMAP (1) /* Remappable Interrupt */
+
/* Programming format for IOAPIC table entries */
union VTD_IR_IOAPICEntry {
struct {
@@ -147,6 +159,46 @@ union VTD_IR_MSIAddress {
uint32_t data;
};
+/* Generic IRQ entry information */
+struct VTDIrq {
+ /* Used by both IOAPIC/MSI interrupt remapping */
+ uint8_t trigger_mode;
+ uint8_t vector;
+ uint8_t delivery_mode;
+ uint32_t dest;
+ uint8_t dest_mode;
+
+ /* only used by MSI interrupt remapping */
+ uint8_t redir_hint;
+ uint8_t msi_addr_last_bits;
+};
+
+struct VTD_MSIMessage {
+ union {
+ struct {
+ uint16_t __not_used:2;
+ uint16_t dest_mode:1;
+ uint16_t redir_hint:1;
+ uint16_t __reserved:8;
+ uint16_t dest:8;
+ uint16_t __addr_head:12; /* 0xfee */
+ uint32_t __addr_hi:32;
+ } QEMU_PACKED;
+ uint64_t msi_addr;
+ };
+ union {
+ struct {
+ uint16_t vector:8;
+ uint16_t delivery_mode:3;
+ uint16_t __resved:3;
+ uint16_t level:1;
+ uint16_t trigger_mode:1;
+ uint16_t __resved1:16;
+ } QEMU_PACKED;
+ uint32_t msi_data;
+ };
+};
+
/* When IR is enabled, all MSI/MSI-X data bits should be zero */
#define VTD_IR_MSI_DATA (0)
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 13/25] q35: ioapic: add support for emulated IOAPIC IR
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (11 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 12/25] intel_iommu: Add support for PCI MSI remap Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 14/25] ioapic: introduce ioapic_entry_parse() helper Peter Xu
` (12 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
This patch translates all IOAPIC interrupts into MSI ones. One pseudo
ioapic address space is added to transfer the MSI message. By default,
it will be system memory address space. When IR is enabled, it will be
IOMMU address space.
Currently, only emulated IOAPIC is supported.
Idea suggested by Jan Kiszka and Rita Sinha in the following patch:
https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01933.html
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/pc.c | 3 +++
hw/intc/ioapic.c | 28 ++++++++++++++++++++++++----
hw/pci-host/q35.c | 4 ++++
include/hw/i386/apic-msidef.h | 1 +
include/hw/i386/ioapic_internal.h | 1 +
include/hw/i386/pc.h | 4 ++++
6 files changed, 37 insertions(+), 4 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 99437e0..365e82f 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1395,6 +1395,9 @@ void pc_memory_init(PCMachineState *pcms,
rom_add_option(option_rom[i].name, option_rom[i].bootindex);
}
pcms->fw_cfg = fw_cfg;
+
+ /* Init default IOAPIC address space */
+ pcms->ioapic_as = &address_space_memory;
}
qemu_irq pc_allocate_cpu_irq(void)
diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
index 51f1c0f..edd50ff 100644
--- a/hw/intc/ioapic.c
+++ b/hw/intc/ioapic.c
@@ -28,6 +28,8 @@
#include "hw/i386/ioapic_internal.h"
#include "include/hw/pci/msi.h"
#include "sysemu/kvm.h"
+#include "target-i386/cpu.h"
+#include "hw/i386/apic-msidef.h"
//#define DEBUG_IOAPIC
@@ -49,13 +51,15 @@ extern int ioapic_no;
static void ioapic_service(IOAPICCommonState *s)
{
+ AddressSpace *ioapic_as = PC_MACHINE(qdev_get_machine())->ioapic_as;
+ uint32_t addr, data;
uint8_t i;
uint8_t trig_mode;
uint8_t vector;
uint8_t delivery_mode;
uint32_t mask;
uint64_t entry;
- uint8_t dest;
+ uint16_t dest_idx;
uint8_t dest_mode;
for (i = 0; i < IOAPIC_NUM_PINS; i++) {
@@ -66,7 +70,14 @@ static void ioapic_service(IOAPICCommonState *s)
entry = s->ioredtbl[i];
if (!(entry & IOAPIC_LVT_MASKED)) {
trig_mode = ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1);
- dest = entry >> IOAPIC_LVT_DEST_SHIFT;
+ /*
+ * By default, this would be dest_id[8] +
+ * reserved[8]. When IR is enabled, this would be
+ * interrupt_index[15] + interrupt_format[1]. This
+ * field never means anything, but only used to
+ * generate corresponding MSI.
+ */
+ dest_idx = entry >> IOAPIC_LVT_DEST_IDX_SHIFT;
dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1;
delivery_mode =
(entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) & IOAPIC_DM_MASK;
@@ -96,8 +107,17 @@ static void ioapic_service(IOAPICCommonState *s)
#else
(void)coalesce;
#endif
- apic_deliver_irq(dest, dest_mode, delivery_mode, vector,
- trig_mode);
+ /* No matter whether IR is enabled, we translate
+ * the IOAPIC message into a MSI one, and its
+ * address space will decide whether we need a
+ * translation. */
+ addr = APIC_DEFAULT_ADDRESS | \
+ (dest_idx << MSI_ADDR_DEST_IDX_SHIFT) |
+ (dest_mode << MSI_ADDR_DEST_MODE_SHIFT);
+ data = (vector << MSI_DATA_VECTOR_SHIFT) |
+ (trig_mode << MSI_DATA_TRIGGER_SHIFT) |
+ (delivery_mode << MSI_DATA_DELIVERY_MODE_SHIFT);
+ stl_le_phys(ioapic_as, addr, data);
}
}
}
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 6835da1..f3d47ad 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -435,6 +435,7 @@ static AddressSpace *q35_host_dma_iommu(PCIBus *bus, void *opaque, int devfn)
static void mch_init_dmar(MCHPCIState *mch)
{
+ PCMachineState *pcms = PC_MACHINE(qdev_get_machine());
PCIBus *pci_bus = PCI_BUS(qdev_get_parent_bus(DEVICE(mch)));
mch->iommu = INTEL_IOMMU_DEVICE(qdev_create(NULL, TYPE_INTEL_IOMMU_DEVICE));
@@ -444,6 +445,9 @@ static void mch_init_dmar(MCHPCIState *mch)
sysbus_mmio_map(SYS_BUS_DEVICE(mch->iommu), 0, Q35_HOST_BRIDGE_IOMMU_ADDR);
pci_setup_iommu(pci_bus, q35_host_dma_iommu, mch->iommu);
+ /* Pseudo address space under root PCI bus. */
+ pcms->ioapic_as = q35_host_dma_iommu(pci_bus, mch->iommu,
+ Q35_PSEUDO_DEVFN_IOAPIC);
}
static void mch_realize(PCIDevice *d, Error **errp)
diff --git a/include/hw/i386/apic-msidef.h b/include/hw/i386/apic-msidef.h
index 6e2eb71..8b4d4cc 100644
--- a/include/hw/i386/apic-msidef.h
+++ b/include/hw/i386/apic-msidef.h
@@ -25,6 +25,7 @@
#define MSI_ADDR_REDIRECTION_SHIFT 3
#define MSI_ADDR_DEST_ID_SHIFT 12
+#define MSI_ADDR_DEST_IDX_SHIFT 4
#define MSI_ADDR_DEST_ID_MASK 0x00ffff0
#endif /* HW_APIC_MSIDEF_H */
diff --git a/include/hw/i386/ioapic_internal.h b/include/hw/i386/ioapic_internal.h
index cab9e67..31dafb3 100644
--- a/include/hw/i386/ioapic_internal.h
+++ b/include/hw/i386/ioapic_internal.h
@@ -31,6 +31,7 @@
#define IOAPIC_VERSION 0x11
#define IOAPIC_LVT_DEST_SHIFT 56
+#define IOAPIC_LVT_DEST_IDX_SHIFT 48
#define IOAPIC_LVT_MASKED_SHIFT 16
#define IOAPIC_LVT_TRIGGER_MODE_SHIFT 15
#define IOAPIC_LVT_REMOTE_IRR_SHIFT 14
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 96f0b66..cde6934 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -72,6 +72,10 @@ struct PCMachineState {
uint64_t numa_nodes;
uint64_t *node_mem;
uint64_t *node_cpu;
+
+ /* Address space used by IOAPIC device. All IOAPIC interrupts
+ * will be translated to MSI messages in the address space. */
+ AddressSpace *ioapic_as;
};
#define PC_MACHINE_ACPI_DEVICE_PROP "acpi-device"
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 14/25] ioapic: introduce ioapic_entry_parse() helper
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (12 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 13/25] q35: ioapic: add support for emulated IOAPIC IR Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 15/25] intel_iommu: add support for split irqchip Peter Xu
` (11 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Abstract IOAPIC entry parsing logic into a helper function.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/intc/ioapic.c | 110 +++++++++++++++++++++++++++----------------------------
1 file changed, 54 insertions(+), 56 deletions(-)
diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
index edd50ff..dafefd0 100644
--- a/hw/intc/ioapic.c
+++ b/hw/intc/ioapic.c
@@ -49,18 +49,56 @@ static IOAPICCommonState *ioapics[MAX_IOAPICS];
/* global variable from ioapic_common.c */
extern int ioapic_no;
+struct ioapic_entry_info {
+ /* fields parsed from IOAPIC entries */
+ uint8_t masked;
+ uint8_t trig_mode;
+ uint16_t dest_idx;
+ uint8_t dest_mode;
+ uint8_t delivery_mode;
+ uint8_t vector;
+
+ /* MSI message generated from above parsed fields */
+ uint32_t addr;
+ uint32_t data;
+};
+
+static void ioapic_entry_parse(uint64_t entry, struct ioapic_entry_info *info)
+{
+ bzero(info, sizeof(*info));
+ info->masked = (entry >> IOAPIC_LVT_MASKED_SHIFT) & 1;
+ info->trig_mode = (entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1;
+ /*
+ * By default, this would be dest_id[8] + reserved[8]. When IR
+ * is enabled, this would be interrupt_index[15] +
+ * interrupt_format[1]. This field never means anything, but
+ * only used to generate corresponding MSI.
+ */
+ info->dest_idx = (entry >> IOAPIC_LVT_DEST_IDX_SHIFT) & 0xffff;
+ info->dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1;
+ info->delivery_mode = (entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) \
+ & IOAPIC_DM_MASK;
+ if (info->delivery_mode == IOAPIC_DM_EXTINT) {
+ info->vector = pic_read_irq(isa_pic);
+ } else {
+ info->vector = entry & IOAPIC_VECTOR_MASK;
+ }
+
+ info->addr = APIC_DEFAULT_ADDRESS | \
+ (info->dest_idx << MSI_ADDR_DEST_IDX_SHIFT) | \
+ (info->dest_mode << MSI_ADDR_DEST_MODE_SHIFT);
+ info->data = (info->vector << MSI_DATA_VECTOR_SHIFT) | \
+ (info->trig_mode << MSI_DATA_TRIGGER_SHIFT) | \
+ (info->delivery_mode << MSI_DATA_DELIVERY_MODE_SHIFT);
+}
+
static void ioapic_service(IOAPICCommonState *s)
{
AddressSpace *ioapic_as = PC_MACHINE(qdev_get_machine())->ioapic_as;
- uint32_t addr, data;
+ struct ioapic_entry_info info;
uint8_t i;
- uint8_t trig_mode;
- uint8_t vector;
- uint8_t delivery_mode;
uint32_t mask;
uint64_t entry;
- uint16_t dest_idx;
- uint8_t dest_mode;
for (i = 0; i < IOAPIC_NUM_PINS; i++) {
mask = 1 << i;
@@ -68,33 +106,18 @@ static void ioapic_service(IOAPICCommonState *s)
int coalesce = 0;
entry = s->ioredtbl[i];
- if (!(entry & IOAPIC_LVT_MASKED)) {
- trig_mode = ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1);
- /*
- * By default, this would be dest_id[8] +
- * reserved[8]. When IR is enabled, this would be
- * interrupt_index[15] + interrupt_format[1]. This
- * field never means anything, but only used to
- * generate corresponding MSI.
- */
- dest_idx = entry >> IOAPIC_LVT_DEST_IDX_SHIFT;
- dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1;
- delivery_mode =
- (entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) & IOAPIC_DM_MASK;
- if (trig_mode == IOAPIC_TRIGGER_EDGE) {
+ ioapic_entry_parse(entry, &info);
+ if (!info.masked) {
+ if (info.trig_mode == IOAPIC_TRIGGER_EDGE) {
s->irr &= ~mask;
} else {
coalesce = s->ioredtbl[i] & IOAPIC_LVT_REMOTE_IRR;
s->ioredtbl[i] |= IOAPIC_LVT_REMOTE_IRR;
}
- if (delivery_mode == IOAPIC_DM_EXTINT) {
- vector = pic_read_irq(isa_pic);
- } else {
- vector = entry & IOAPIC_VECTOR_MASK;
- }
+
#ifdef CONFIG_KVM
if (kvm_irqchip_is_split()) {
- if (trig_mode == IOAPIC_TRIGGER_EDGE) {
+ if (info.trig_mode == IOAPIC_TRIGGER_EDGE) {
kvm_set_irq(kvm_state, i, 1);
kvm_set_irq(kvm_state, i, 0);
} else {
@@ -111,13 +134,7 @@ static void ioapic_service(IOAPICCommonState *s)
* the IOAPIC message into a MSI one, and its
* address space will decide whether we need a
* translation. */
- addr = APIC_DEFAULT_ADDRESS | \
- (dest_idx << MSI_ADDR_DEST_IDX_SHIFT) |
- (dest_mode << MSI_ADDR_DEST_MODE_SHIFT);
- data = (vector << MSI_DATA_VECTOR_SHIFT) |
- (trig_mode << MSI_DATA_TRIGGER_SHIFT) |
- (delivery_mode << MSI_DATA_DELIVERY_MODE_SHIFT);
- stl_le_phys(ioapic_as, addr, data);
+ stl_le_phys(ioapic_as, info.addr, info.data);
}
}
}
@@ -168,30 +185,11 @@ static void ioapic_update_kvm_routes(IOAPICCommonState *s)
if (kvm_irqchip_is_split()) {
for (i = 0; i < IOAPIC_NUM_PINS; i++) {
- uint64_t entry = s->ioredtbl[i];
- uint8_t trig_mode;
- uint8_t delivery_mode;
- uint8_t dest;
- uint8_t dest_mode;
- uint64_t pin_polarity;
MSIMessage msg;
-
- trig_mode = ((entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1);
- dest = entry >> IOAPIC_LVT_DEST_SHIFT;
- dest_mode = (entry >> IOAPIC_LVT_DEST_MODE_SHIFT) & 1;
- pin_polarity = (entry >> IOAPIC_LVT_TRIGGER_MODE_SHIFT) & 1;
- delivery_mode =
- (entry >> IOAPIC_LVT_DELIV_MODE_SHIFT) & IOAPIC_DM_MASK;
-
- msg.address = APIC_DEFAULT_ADDRESS;
- msg.address |= dest_mode << 2;
- msg.address |= dest << 12;
-
- msg.data = entry & IOAPIC_VECTOR_MASK;
- msg.data |= delivery_mode << APIC_DELIVERY_MODE_SHIFT;
- msg.data |= pin_polarity << APIC_POLARITY_SHIFT;
- msg.data |= trig_mode << APIC_TRIG_MODE_SHIFT;
-
+ struct ioapic_entry_info info;
+ ioapic_entry_parse(s->ioredtbl[i], &info);
+ msg.address = info.addr;
+ msg.data = info.data;
kvm_irqchip_update_msi_route(kvm_state, i, msg, NULL);
}
kvm_irqchip_commit_routes(kvm_state);
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 15/25] intel_iommu: add support for split irqchip
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (13 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 14/25] ioapic: introduce ioapic_entry_parse() helper Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 16/25] q35: add "intremap" parameter to enable IR Peter Xu
` (10 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
In split irqchip mode, IOAPIC is working in user space, only update
kernel irq routes when entry changed. When IR is enabled, we directly
update the kernel with translated messages. It works just like a kernel
cache for the remapping entries.
Since KVM irqfd is using kernel gsi routes to deliver interrupts, as
long as we can support split irqchip, we will support irqfd as
well. Also, since kernel gsi routes will cache translated interrupts,
irqfd delivery will not suffer from any performance impact due to IR.
And, since we supported irqfd, vhost devices will be able to work
seamlessly with IR now. Logically this should contain both vhost-net and
vhost-user case.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 7 +++++++
include/hw/i386/intel_iommu.h | 1 +
include/hw/i386/x86-iommu.h | 4 ++++
target-i386/kvm.c | 27 +++++++++++++++++++++++++++
trace-events | 3 +++
5 files changed, 42 insertions(+)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index b5137ce..28e3724 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2146,6 +2146,12 @@ do_not_translate:
return 0;
}
+static int vtd_int_remap(X86IOMMUState *iommu, MSIMessage *src,
+ MSIMessage *dst, uint16_t sid)
+{
+ return vtd_interrupt_remap_msi(INTEL_IOMMU_DEVICE(iommu), src, dst);
+}
+
static MemTxResult vtd_mem_ir_read(void *opaque, hwaddr addr,
uint64_t *data, unsigned size,
MemTxAttrs attrs)
@@ -2374,6 +2380,7 @@ static void vtd_class_init(ObjectClass *klass, void *data)
dc->props = vtd_properties;
x86_class->realize = vtd_realize;
x86_class->find_add_as = vtd_find_add_as;
+ x86_class->int_remap = vtd_int_remap;
}
static const TypeInfo vtd_info = {
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 2018233..7ac63f5 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -26,6 +26,7 @@
#include "hw/i386/x86-iommu.h"
#include "hw/i386/ioapic.h"
#include "hw/pci/msi.h"
+#include "hw/sysbus.h"
#define TYPE_INTEL_IOMMU_DEVICE "intel-iommu"
#define INTEL_IOMMU_DEVICE(obj) \
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index 2070cd1..1eb62cf 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -22,6 +22,7 @@
#include "hw/sysbus.h"
#include "exec/memory.h"
+#include "hw/pci/pci.h"
#define TYPE_X86_IOMMU_DEVICE ("x86-iommu")
#define X86_IOMMU_DEVICE(obj) \
@@ -43,6 +44,9 @@ struct X86IOMMUClass {
DeviceRealize realize;
/* Find/Add IOMMU address space for specific PCI device */
AddressSpace *(*find_add_as)(X86IOMMUState *s, PCIBus *bus, int devfn);
+ /* MSI-based interrupt remapping */
+ int (*int_remap)(X86IOMMUState *iommu, MSIMessage *src,
+ MSIMessage *dst, uint16_t sid);
};
struct X86IOMMUState {
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 799fdfa..693d1c1 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -36,6 +36,7 @@
#include "hw/i386/apic.h"
#include "hw/i386/apic_internal.h"
#include "hw/i386/apic-msidef.h"
+#include "hw/i386/intel_iommu.h"
#include "exec/ioport.h"
#include "standard-headers/asm-x86/hyperv.h"
@@ -43,6 +44,7 @@
#include "hw/pci/msi.h"
#include "migration/migration.h"
#include "exec/memattrs.h"
+#include "trace.h"
//#define DEBUG_KVM
@@ -3327,6 +3329,31 @@ int kvm_device_msix_deassign(KVMState *s, uint32_t dev_id)
int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
uint64_t address, uint32_t data, PCIDevice *dev)
{
+ X86IOMMUState *iommu = x86_iommu_get_default();
+
+ if (iommu) {
+ int ret;
+ MSIMessage src, dst;
+ X86IOMMUClass *class = X86_IOMMU_GET_CLASS(iommu);
+
+ src.address = route->u.msi.address_hi;
+ src.address <<= VTD_MSI_ADDR_HI_SHIFT;
+ src.address |= route->u.msi.address_lo;
+ src.data = route->u.msi.data;
+
+ ret = class->int_remap(iommu, &src, &dst, dev ? \
+ pci_requester_id(dev) : \
+ X86_IOMMU_SID_INVALID);
+ if (ret) {
+ trace_kvm_x86_fixup_msi_error(route->gsi);
+ return 1;
+ }
+
+ route->u.msi.address_hi = dst.address >> VTD_MSI_ADDR_HI_SHIFT;
+ route->u.msi.address_lo = dst.address & VTD_MSI_ADDR_LO_MASK;
+ route->u.msi.data = dst.data;
+ }
+
return 0;
}
diff --git a/trace-events b/trace-events
index 4fce005..6e25884 100644
--- a/trace-events
+++ b/trace-events
@@ -1908,3 +1908,6 @@ aspeed_vic_update_fiq(int flags) "Raising FIQ: %d"
aspeed_vic_update_irq(int flags) "Raising IRQ: %d"
aspeed_vic_read(uint64_t offset, unsigned size, uint32_t value) "From 0x%" PRIx64 " of size %u: 0x%" PRIx32
aspeed_vic_write(uint64_t offset, unsigned size, uint32_t data) "To 0x%" PRIx64 " of size %u: 0x%" PRIx32
+
+# target-i386/kvm.c
+kvm_x86_fixup_msi_error(uint32_t gsi) "VT-d failed to remap interrupt for GSI %" PRIu32
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 16/25] q35: add "intremap" parameter to enable IR
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (14 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 15/25] intel_iommu: add support for split irqchip Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 17/25] x86-iommu: introduce IEC notifiers Peter Xu
` (9 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
One flag is added to specify whether to enable IR for emulated IOMMU. By
default, interrupt remapping is not supportted. To enable it, we should
specify something like:
$ qemu-system-x86_64 -M q35,iommu=on,intremap=on
To be more clear, the following command:
$ qemu-system-x86_64 -M q35,iommu=on
Will enable IOMMU only, without interrupt remapping support.
Currently, Intel IOMMU IR only support kernel-irqchip={off|split}. We
need to specify either of it in -M as well.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/core/machine.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 276ad61..5994b9f 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -300,6 +300,20 @@ static void machine_set_iommu(Object *obj, bool value, Error **errp)
ms->iommu = value;
}
+static bool machine_get_intremap(Object *obj, Error **errp)
+{
+ MachineState *ms = MACHINE(obj);
+
+ return ms->iommu_intr;
+}
+
+static void machine_set_intremap(Object *obj, bool value, Error **errp)
+{
+ MachineState *ms = MACHINE(obj);
+
+ ms->iommu_intr = value;
+}
+
static void machine_set_suppress_vmdesc(Object *obj, bool value, Error **errp)
{
MachineState *ms = MACHINE(obj);
@@ -480,6 +494,12 @@ static void machine_initfn(Object *obj)
object_property_set_description(obj, "iommu",
"Set on/off to enable/disable Intel IOMMU (VT-d)",
NULL);
+ object_property_add_bool(obj, "intremap", machine_get_intremap,
+ machine_set_intremap, NULL);
+ object_property_set_description(obj, "intremap",
+ "Set on/off to enable/disable IOMMU"
+ " interrupt remapping",
+ NULL);
object_property_add_bool(obj, "suppress-vmdesc",
machine_get_suppress_vmdesc,
machine_set_suppress_vmdesc, NULL);
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 17/25] x86-iommu: introduce IEC notifiers
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (15 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 16/25] q35: add "intremap" parameter to enable IR Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 18/25] ioapic: register IOMMU IEC notifier for ioapic Peter Xu
` (8 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
This patch introduces x86 IOMMU IEC (Interrupt Entry Cache)
invalidation notifier list. When vIOMMU receives IEC invalidate
request, all the registered units will be notified with specific
invalidation requests.
Intel IOMMU is the first provider that generates such a event.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 36 +++++++++++++++++++++++++++++-------
hw/i386/intel_iommu_internal.h | 24 ++++++++++++++++++++----
hw/i386/x86-iommu.c | 23 +++++++++++++++++++++++
include/hw/i386/x86-iommu.h | 40 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 112 insertions(+), 11 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 28e3724..ca1e4fb 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -901,6 +901,12 @@ static void vtd_root_table_setup(IntelIOMMUState *s)
(s->root_extended ? "(extended)" : ""));
}
+static void vtd_iec_notify_all(IntelIOMMUState *s, bool global,
+ uint32_t index, uint32_t mask)
+{
+ x86_iommu_iec_notify_all(X86_IOMMU_DEVICE(s), global, index, mask);
+}
+
static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s)
{
uint64_t value = 0;
@@ -908,7 +914,8 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s)
s->intr_size = 1UL << ((value & VTD_IRTA_SIZE_MASK) + 1);
s->intr_root = value & VTD_IRTA_ADDR_MASK;
- /* TODO: invalidate interrupt entry cache */
+ /* Notify global invalidation */
+ vtd_iec_notify_all(s, true, 0, 0);
VTD_DPRINTF(CSR, "int remap table addr 0x%"PRIx64 " size %"PRIu32,
s->intr_root, s->intr_size);
@@ -1410,6 +1417,21 @@ static bool vtd_process_iotlb_desc(IntelIOMMUState *s, VTDInvDesc *inv_desc)
return true;
}
+static bool vtd_process_inv_iec_desc(IntelIOMMUState *s,
+ VTDInvDesc *inv_desc)
+{
+ VTD_DPRINTF(INV, "inv ir glob %d index %d mask %d",
+ inv_desc->iec.granularity,
+ inv_desc->iec.index,
+ inv_desc->iec.index_mask);
+
+ vtd_iec_notify_all(s, inv_desc->iec.granularity,
+ inv_desc->iec.index,
+ inv_desc->iec.index_mask);
+
+ return true;
+}
+
static bool vtd_process_inv_desc(IntelIOMMUState *s)
{
VTDInvDesc inv_desc;
@@ -1450,12 +1472,12 @@ static bool vtd_process_inv_desc(IntelIOMMUState *s)
break;
case VTD_INV_DESC_IEC:
- VTD_DPRINTF(INV, "Interrupt Entry Cache Invalidation "
- "not implemented yet");
- /*
- * Since currently we do not cache interrupt entries, we can
- * just mark this descriptor as "good" and move on.
- */
+ VTD_DPRINTF(INV, "Invalidation Interrupt Entry Cache "
+ "Descriptor hi 0x%"PRIx64 " lo 0x%"PRIx64,
+ inv_desc.hi, inv_desc.lo);
+ if (!vtd_process_inv_iec_desc(s, &inv_desc)) {
+ return false;
+ }
break;
default:
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index e1a08cb..10c20fe 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -296,12 +296,28 @@ typedef enum VTDFaultReason {
#define VTD_CONTEXT_CACHE_GEN_MAX 0xffffffffUL
+/* Interrupt Entry Cache Invalidation Descriptor: VT-d 6.5.2.7. */
+struct VTDInvDescIEC {
+ uint32_t type:4; /* Should always be 0x4 */
+ uint32_t granularity:1; /* If set, it's global IR invalidation */
+ uint32_t resved_1:22;
+ uint32_t index_mask:5; /* 2^N for continuous int invalidation */
+ uint32_t index:16; /* Start index to invalidate */
+ uint32_t reserved_2:16;
+};
+typedef struct VTDInvDescIEC VTDInvDescIEC;
+
/* Queued Invalidation Descriptor */
-struct VTDInvDesc {
- uint64_t lo;
- uint64_t hi;
+union VTDInvDesc {
+ struct {
+ uint64_t lo;
+ uint64_t hi;
+ };
+ union {
+ VTDInvDescIEC iec;
+ };
};
-typedef struct VTDInvDesc VTDInvDesc;
+typedef union VTDInvDesc VTDInvDesc;
/* Masks for struct VTDInvDesc */
#define VTD_INV_DESC_TYPE 0xf
diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
index 2d6d221..0b14b09 100644
--- a/hw/i386/x86-iommu.c
+++ b/hw/i386/x86-iommu.c
@@ -22,6 +22,27 @@
#include "hw/boards.h"
#include "hw/i386/x86-iommu.h"
+void x86_iommu_iec_register_notifier(X86IOMMUState *iommu,
+ iec_notify_fn fn, void *data)
+{
+ IEC_Notifier *notifier = g_new0(IEC_Notifier, 1);
+ notifier->iec_notify = fn;
+ notifier->private = data;
+ QLIST_INSERT_HEAD(&iommu->iec_notifiers, notifier, list);
+}
+
+void x86_iommu_iec_notify_all(X86IOMMUState *iommu, bool global,
+ uint32_t index, uint32_t mask)
+{
+ IEC_Notifier *notifier;
+ QLIST_FOREACH(notifier, &iommu->iec_notifiers, list) {
+ if (notifier->iec_notify) {
+ notifier->iec_notify(notifier->private, global,
+ index, mask);
+ }
+ }
+}
+
/* Default X86 IOMMU device */
static X86IOMMUState *x86_iommu_default = NULL;
@@ -39,7 +60,9 @@ X86IOMMUState *x86_iommu_get_default(void)
static void x86_iommu_realize(DeviceState *dev, Error **errp)
{
+ X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(dev);
X86IOMMUClass *x86_class = X86_IOMMU_GET_CLASS(dev);
+ QLIST_INIT(&x86_iommu->iec_notifiers);
if (x86_class->realize) {
x86_class->realize(dev, errp);
}
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index 1eb62cf..f0aa3a3 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -49,8 +49,27 @@ struct X86IOMMUClass {
MSIMessage *dst, uint16_t sid);
};
+/**
+ * iec_notify_fn - IEC (Interrupt Entry Cache) notifier hook,
+ * triggered when IR invalidation happens.
+ * @private: private data
+ * @global: whether this is a global IEC invalidation
+ * @index: IRTE index to invalidate (start from)
+ * @mask: invalidation mask
+ */
+typedef void (*iec_notify_fn)(void *private, bool global,
+ uint32_t index, uint32_t mask);
+
+struct IEC_Notifier {
+ iec_notify_fn iec_notify;
+ void *private;
+ QLIST_ENTRY(IEC_Notifier) list;
+};
+typedef struct IEC_Notifier IEC_Notifier;
+
struct X86IOMMUState {
SysBusDevice busdev;
+ QLIST_HEAD(, IEC_Notifier) iec_notifiers; /* IEC notify list */
};
/**
@@ -59,4 +78,25 @@ struct X86IOMMUState {
*/
X86IOMMUState *x86_iommu_get_default(void);
+/**
+ * x86_iommu_iec_register_notifier - register IEC (Interrupt Entry
+ * Cache) notifiers
+ * @iommu: IOMMU device to register
+ * @fn: IEC notifier hook function
+ * @data: notifier private data
+ */
+void x86_iommu_iec_register_notifier(X86IOMMUState *iommu,
+ iec_notify_fn fn, void *data);
+
+/**
+ * x86_iommu_iec_notify_all - Notify IEC invalidations
+ * @iommu: IOMMU device that sends the notification
+ * @global: whether this is a global invalidation. If true, @index
+ * and @mask are undefined.
+ * @index: starting index of interrupt entry to invalidate
+ * @mask: index mask for the invalidation
+ */
+void x86_iommu_iec_notify_all(X86IOMMUState *iommu, bool global,
+ uint32_t index, uint32_t mask);
+
#endif
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 18/25] ioapic: register IOMMU IEC notifier for ioapic
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (16 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 17/25] x86-iommu: introduce IEC notifiers Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 19/25] intel_iommu: Add support for Extended Interrupt Mode Peter Xu
` (7 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Let IOAPIC the first consumer of x86 IOMMU IEC invalidation
notifiers. This is only used for split irqchip case, when vIOMMU
receives IR invalidation requests, IOAPIC will be notified to update
kernel irq routes. For simplicity, we just update all IOAPIC routes,
even if the invalidated entries are not IOAPIC ones.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/intc/ioapic.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/hw/intc/ioapic.c b/hw/intc/ioapic.c
index dafefd0..e924100 100644
--- a/hw/intc/ioapic.c
+++ b/hw/intc/ioapic.c
@@ -30,6 +30,7 @@
#include "sysemu/kvm.h"
#include "target-i386/cpu.h"
#include "hw/i386/apic-msidef.h"
+#include "hw/i386/x86-iommu.h"
//#define DEBUG_IOAPIC
@@ -197,6 +198,14 @@ static void ioapic_update_kvm_routes(IOAPICCommonState *s)
#endif
}
+static void ioapic_iec_notifier(void *private, bool global,
+ uint32_t index, uint32_t mask)
+{
+ IOAPICCommonState *s = (IOAPICCommonState *)private;
+ /* For simplicity, we just update all the routes */
+ ioapic_update_kvm_routes(s);
+}
+
void ioapic_eoi_broadcast(int vector)
{
IOAPICCommonState *s;
@@ -363,6 +372,18 @@ static void ioapic_realize(DeviceState *dev, Error **errp)
qdev_init_gpio_in(dev, ioapic_set_irq, IOAPIC_NUM_PINS);
ioapics[ioapic_no] = s;
+
+#ifdef CONFIG_KVM
+ if (kvm_irqchip_is_split()) {
+ X86IOMMUState *iommu = x86_iommu_get_default();
+ if (iommu) {
+ /* Register this IOAPIC with IOMMU IEC notifier, so that
+ * when there are IR invalidates, we can be notified to
+ * update kernel IR cache. */
+ x86_iommu_iec_register_notifier(iommu, ioapic_iec_notifier, s);
+ }
+ }
+#endif
}
static void ioapic_class_init(ObjectClass *klass, void *data)
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 19/25] intel_iommu: Add support for Extended Interrupt Mode
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (17 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 18/25] ioapic: register IOMMU IEC notifier for ioapic Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 20/25] intel_iommu: add SID validation for IR Peter Xu
` (6 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx,
Jan Kiszka
From: Jan Kiszka <jan.kiszka@siemens.com>
As neither QEMU nor KVM support more than 255 CPUs so far, this is
simple: we only need to switch the destination ID translation in
vtd_remap_irq_get if EIME is set.
Once CFI support is there, it will have to take EIM into account as
well. So far, nothing to do for this.
This patch allows to use x2APIC in split irqchip mode of KVM.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
hw/i386/intel_iommu.c | 16 +++++++++-------
hw/i386/intel_iommu_internal.h | 2 ++
include/hw/i386/intel_iommu.h | 1 +
3 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index ca1e4fb..737f43a 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -913,6 +913,7 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s)
value = vtd_get_quad_raw(s, DMAR_IRTA_REG);
s->intr_size = 1UL << ((value & VTD_IRTA_SIZE_MASK) + 1);
s->intr_root = value & VTD_IRTA_ADDR_MASK;
+ s->intr_eime = value & VTD_IRTA_EIME;
/* Notify global invalidation */
vtd_iec_notify_all(s, true, 0, 0);
@@ -2047,11 +2048,13 @@ static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index, VTDIrq *irq
irq->trigger_mode = irte.trigger_mode;
irq->vector = irte.vector;
irq->delivery_mode = irte.delivery_mode;
- /* Not support EIM yet: please refer to vt-d 9.10 DST bits */
+ irq->dest = irte.dest_id;
+ if (!iommu->intr_eime) {
#define VTD_IR_APIC_DEST_MASK (0xff00ULL)
#define VTD_IR_APIC_DEST_SHIFT (8)
- irq->dest = (irte.dest_id & VTD_IR_APIC_DEST_MASK) >> \
- VTD_IR_APIC_DEST_SHIFT;
+ irq->dest = (irq->dest & VTD_IR_APIC_DEST_MASK) >>
+ VTD_IR_APIC_DEST_SHIFT;
+ }
irq->dest_mode = irte.dest_mode;
irq->redir_hint = irte.redir_hint;
@@ -2304,7 +2307,7 @@ static void vtd_init(IntelIOMMUState *s)
s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO;
if (ms->iommu_intr) {
- s->ecap |= VTD_ECAP_IR;
+ s->ecap |= VTD_ECAP_IR | VTD_ECAP_EIM;
}
vtd_reset_context_cache(s);
@@ -2358,10 +2361,9 @@ static void vtd_init(IntelIOMMUState *s)
vtd_define_quad(s, DMAR_FRCD_REG_0_2, 0, 0, 0x8000000000000000ULL);
/*
- * Interrupt remapping registers, not support extended interrupt
- * mode for now.
+ * Interrupt remapping registers.
*/
- vtd_define_quad(s, DMAR_IRTA_REG, 0, 0xfffffffffffff00fULL, 0);
+ vtd_define_quad(s, DMAR_IRTA_REG, 0, 0xfffffffffffff80fULL, 0);
}
/* Should not reset address_spaces when reset because devices will still use
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 10c20fe..72b0114 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -176,6 +176,7 @@
/* IRTA_REG */
#define VTD_IRTA_ADDR_MASK (VTD_HAW_MASK ^ 0xfffULL)
+#define VTD_IRTA_EIME (1ULL << 11)
#define VTD_IRTA_SIZE_MASK (0xfULL)
/* ECAP_REG */
@@ -184,6 +185,7 @@
#define VTD_ECAP_QI (1ULL << 1)
/* Interrupt Remapping support */
#define VTD_ECAP_IR (1ULL << 3)
+#define VTD_ECAP_EIM (1ULL << 4)
/* CAP_REG */
/* (offset >> 4) << 24 */
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 7ac63f5..c3cb391 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -243,6 +243,7 @@ struct IntelIOMMUState {
bool intr_enabled; /* Whether guest enabled IR */
dma_addr_t intr_root; /* Interrupt remapping table pointer */
uint32_t intr_size; /* Number of IR table entries */
+ bool intr_eime; /* Extended interrupt mode enabled */
};
#endif
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 20/25] intel_iommu: add SID validation for IR
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (18 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 19/25] intel_iommu: Add support for Extended Interrupt Mode Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 21/25] kvm-irqchip: simplify kvm_irqchip_add_msi_route Peter Xu
` (5 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
This patch enables SID validation. Invalid interrupts will be dropped.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/intel_iommu.c | 68 ++++++++++++++++++++++++++++++++++++-------
include/hw/i386/intel_iommu.h | 17 +++++++++++
2 files changed, 74 insertions(+), 11 deletions(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 737f43a..0e340b9 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1993,11 +1993,15 @@ static Property vtd_properties[] = {
DEFINE_PROP_END_OF_LIST(),
};
+uint16_t vtd_svt_mask[VTD_SQ_MAX] = {0xffff, 0xfffb, 0xfff9, 0xfff8};
+
/* Read IRTE entry with specific index */
static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index,
- VTD_IRTE *entry)
+ VTD_IRTE *entry, uint16_t sid)
{
dma_addr_t addr = 0x00;
+ uint16_t mask;
+ uint8_t bus, bus_max, bus_min;
addr = iommu->intr_root + index * sizeof(*entry);
if (dma_memory_read(&address_space_memory, addr, entry,
@@ -2024,23 +2028,57 @@ static int vtd_irte_get(IntelIOMMUState *iommu, uint16_t index,
return -VTD_FR_IR_IRTE_RSVD;
}
- /*
- * TODO: Check Source-ID corresponds to SVT (Source Validation
- * Type) bits
- */
+ if (sid != X86_IOMMU_SID_INVALID) {
+ /* Validate IRTE SID */
+ switch (entry->sid_vtype) {
+ case VTD_SVT_NONE:
+ VTD_DPRINTF(IR, "No SID validation for IRTE index %d", index);
+ break;
+
+ case VTD_SVT_ALL:
+ mask = vtd_svt_mask[entry->sid_q];
+ if ((entry->source_id & mask) != (sid & mask)) {
+ VTD_DPRINTF(GENERAL, "SID validation for IRTE index "
+ "%d failed (reqid 0x%04x sid 0x%04x)", index,
+ sid, entry->source_id);
+ return -VTD_FR_IR_SID_ERR;
+ }
+ break;
+
+ case VTD_SVT_BUS:
+ bus_max = entry->source_id >> 8;
+ bus_min = entry->source_id & 0xff;
+ bus = sid >> 8;
+ if (bus > bus_max || bus < bus_min) {
+ VTD_DPRINTF(GENERAL, "SID validation for IRTE index %d "
+ "failed (bus %d outside %d-%d)", index, bus,
+ bus_min, bus_max);
+ return -VTD_FR_IR_SID_ERR;
+ }
+ break;
+
+ default:
+ VTD_DPRINTF(GENERAL, "Invalid SVT bits (0x%x) in IRTE index "
+ "%d", entry->sid_vtype, index);
+ /* Take this as verification failure. */
+ return -VTD_FR_IR_SID_ERR;
+ break;
+ }
+ }
return 0;
}
/* Fetch IRQ information of specific IR index */
-static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index, VTDIrq *irq)
+static int vtd_remap_irq_get(IntelIOMMUState *iommu, uint16_t index,
+ VTDIrq *irq, uint16_t sid)
{
VTD_IRTE irte;
int ret = 0;
bzero(&irte, sizeof(irte));
- ret = vtd_irte_get(iommu, index, &irte);
+ ret = vtd_irte_get(iommu, index, &irte, sid);
if (ret) {
return ret;
}
@@ -2092,7 +2130,8 @@ static void vtd_generate_msi_message(VTDIrq *irq, MSIMessage *msg_out)
/* Interrupt remapping for MSI/MSI-X entry */
static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu,
MSIMessage *origin,
- MSIMessage *translated)
+ MSIMessage *translated,
+ uint16_t sid)
{
int ret = 0;
VTD_IR_MSIAddress addr;
@@ -2127,7 +2166,7 @@ static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu,
index = addr.index_h << 15 | addr.index_l;
- ret = vtd_remap_irq_get(iommu, index, &irq);
+ ret = vtd_remap_irq_get(iommu, index, &irq, sid);
if (ret) {
return ret;
}
@@ -2174,7 +2213,8 @@ do_not_translate:
static int vtd_int_remap(X86IOMMUState *iommu, MSIMessage *src,
MSIMessage *dst, uint16_t sid)
{
- return vtd_interrupt_remap_msi(INTEL_IOMMU_DEVICE(iommu), src, dst);
+ return vtd_interrupt_remap_msi(INTEL_IOMMU_DEVICE(iommu),
+ src, dst, sid);
}
static MemTxResult vtd_mem_ir_read(void *opaque, hwaddr addr,
@@ -2200,11 +2240,17 @@ static MemTxResult vtd_mem_ir_write(void *opaque, hwaddr addr,
{
int ret = 0;
MSIMessage from = {0}, to = {0};
+ uint16_t sid = X86_IOMMU_SID_INVALID;
from.address = (uint64_t) addr + VTD_INTERRUPT_ADDR_FIRST;
from.data = (uint32_t) value;
- ret = vtd_interrupt_remap_msi(opaque, &from, &to);
+ if (!attrs.unspecified) {
+ /* We have explicit Source ID */
+ sid = attrs.requester_id;
+ }
+
+ ret = vtd_interrupt_remap_msi(opaque, &from, &to, sid);
if (ret) {
/* TODO: report error */
VTD_DPRINTF(GENERAL, "int remap fail for addr 0x%"PRIx64
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index c3cb391..70e2925 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -103,6 +103,23 @@ struct VTDIOTLBEntry {
bool write_flags;
};
+/* VT-d Source-ID Qualifier types */
+enum {
+ VTD_SQ_FULL = 0x00, /* Full SID verification */
+ VTD_SQ_IGN_3 = 0x01, /* Ignore bit 3 */
+ VTD_SQ_IGN_2_3 = 0x02, /* Ignore bits 2 & 3 */
+ VTD_SQ_IGN_1_3 = 0x03, /* Ignore bits 1-3 */
+ VTD_SQ_MAX,
+};
+
+/* VT-d Source Validation Types */
+enum {
+ VTD_SVT_NONE = 0x00, /* No validation */
+ VTD_SVT_ALL = 0x01, /* Do full validation */
+ VTD_SVT_BUS = 0x02, /* Validate bus range */
+ VTD_SVT_MAX,
+};
+
/* Interrupt Remapping Table Entry Definition */
union VTD_IRTE {
struct {
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 21/25] kvm-irqchip: simplify kvm_irqchip_add_msi_route
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (19 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 20/25] intel_iommu: add SID validation for IR Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 22/25] kvm-irqchip: i386: add hook for add/remove virq Peter Xu
` (4 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Changing the original MSIMessage parameter in kvm_irqchip_add_msi_route
into the vector number. Vector index provides more information than the
MSIMessage, we can retrieve the MSIMessage using the vector easily. This
will avoid fetching MSIMessage every time before adding MSI routes.
Meanwhile, the vector info will be used in the coming patches to further
enable gsi route update notifications.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/kvm/pci-assign.c | 8 ++------
hw/misc/ivshmem.c | 3 +--
hw/vfio/pci.c | 11 +++++------
hw/virtio/virtio-pci.c | 9 +++------
include/sysemu/kvm.h | 13 ++++++++++++-
kvm-all.c | 18 ++++++++++++++++--
kvm-stub.c | 2 +-
target-i386/kvm.c | 3 +--
8 files changed, 41 insertions(+), 26 deletions(-)
diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index c40ab36..9b43332 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -974,10 +974,9 @@ static void assigned_dev_update_msi(PCIDevice *pci_dev)
}
if (ctrl_byte & PCI_MSI_FLAGS_ENABLE) {
- MSIMessage msg = msi_get_message(pci_dev, 0);
int virq;
- virq = kvm_irqchip_add_msi_route(kvm_state, msg, pci_dev);
+ virq = kvm_irqchip_add_msi_route(kvm_state, 0, pci_dev);
if (virq < 0) {
perror("assigned_dev_update_msi: kvm_irqchip_add_msi_route");
return;
@@ -1042,7 +1041,6 @@ static int assigned_dev_update_msix_mmio(PCIDevice *pci_dev)
uint16_t entries_nr = 0;
int i, r = 0;
MSIXTableEntry *entry = adev->msix_table;
- MSIMessage msg;
/* Get the usable entry number for allocating */
for (i = 0; i < adev->msix_max; i++, entry++) {
@@ -1079,9 +1077,7 @@ static int assigned_dev_update_msix_mmio(PCIDevice *pci_dev)
continue;
}
- msg.address = entry->addr_lo | ((uint64_t)entry->addr_hi << 32);
- msg.data = entry->data;
- r = kvm_irqchip_add_msi_route(kvm_state, msg, pci_dev);
+ r = kvm_irqchip_add_msi_route(kvm_state, i, pci_dev);
if (r < 0) {
return r;
}
diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c
index e40f23b..6909346 100644
--- a/hw/misc/ivshmem.c
+++ b/hw/misc/ivshmem.c
@@ -444,13 +444,12 @@ static void ivshmem_add_kvm_msi_virq(IVShmemState *s, int vector,
Error **errp)
{
PCIDevice *pdev = PCI_DEVICE(s);
- MSIMessage msg = msix_get_message(pdev, vector);
int ret;
IVSHMEM_DPRINTF("ivshmem_add_kvm_msi_virq vector:%d\n", vector);
assert(!s->msi_vectors[vector].pdev);
- ret = kvm_irqchip_add_msi_route(kvm_state, msg, pdev);
+ ret = kvm_irqchip_add_msi_route(kvm_state, vector, pdev);
if (ret < 0) {
error_setg(errp, "kvm_irqchip_add_msi_route failed");
return;
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index d091d8c..2b2f935 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -417,11 +417,11 @@ static int vfio_enable_vectors(VFIOPCIDevice *vdev, bool msix)
}
static void vfio_add_kvm_msi_virq(VFIOPCIDevice *vdev, VFIOMSIVector *vector,
- MSIMessage *msg, bool msix)
+ int vector_n, bool msix)
{
int virq;
- if ((msix && vdev->no_kvm_msix) || (!msix && vdev->no_kvm_msi) || !msg) {
+ if ((msix && vdev->no_kvm_msix) || (!msix && vdev->no_kvm_msi)) {
return;
}
@@ -429,7 +429,7 @@ static void vfio_add_kvm_msi_virq(VFIOPCIDevice *vdev, VFIOMSIVector *vector,
return;
}
- virq = kvm_irqchip_add_msi_route(kvm_state, *msg, &vdev->pdev);
+ virq = kvm_irqchip_add_msi_route(kvm_state, vector_n, &vdev->pdev);
if (virq < 0) {
event_notifier_cleanup(&vector->kvm_interrupt);
return;
@@ -495,7 +495,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
vfio_update_kvm_msi_virq(vector, *msg, pdev);
}
} else {
- vfio_add_kvm_msi_virq(vdev, vector, msg, true);
+ vfio_add_kvm_msi_virq(vdev, vector, nr, true);
}
/*
@@ -639,7 +639,6 @@ retry:
for (i = 0; i < vdev->nr_vectors; i++) {
VFIOMSIVector *vector = &vdev->msi_vectors[i];
- MSIMessage msg = msi_get_message(&vdev->pdev, i);
vector->vdev = vdev;
vector->virq = -1;
@@ -656,7 +655,7 @@ retry:
* Attempt to enable route through KVM irqchip,
* default to userspace handling if unavailable.
*/
- vfio_add_kvm_msi_virq(vdev, vector, &msg, false);
+ vfio_add_kvm_msi_virq(vdev, vector, i, false);
}
/* Set interrupt type prior to possible interrupts */
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index bfedbbf..df85f28 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -727,14 +727,13 @@ static uint32_t virtio_read_config(PCIDevice *pci_dev,
static int kvm_virtio_pci_vq_vector_use(VirtIOPCIProxy *proxy,
unsigned int queue_no,
- unsigned int vector,
- MSIMessage msg)
+ unsigned int vector)
{
VirtIOIRQFD *irqfd = &proxy->vector_irqfd[vector];
int ret;
if (irqfd->users == 0) {
- ret = kvm_irqchip_add_msi_route(kvm_state, msg, &proxy->pci_dev);
+ ret = kvm_irqchip_add_msi_route(kvm_state, vector, &proxy->pci_dev);
if (ret < 0) {
return ret;
}
@@ -787,7 +786,6 @@ static int kvm_virtio_pci_vector_use(VirtIOPCIProxy *proxy, int nvqs)
VirtioDeviceClass *k = VIRTIO_DEVICE_GET_CLASS(vdev);
unsigned int vector;
int ret, queue_no;
- MSIMessage msg;
for (queue_no = 0; queue_no < nvqs; queue_no++) {
if (!virtio_queue_get_num(vdev, queue_no)) {
@@ -797,8 +795,7 @@ static int kvm_virtio_pci_vector_use(VirtIOPCIProxy *proxy, int nvqs)
if (vector >= msix_nr_vectors_allocated(dev)) {
continue;
}
- msg = msix_get_message(dev, vector);
- ret = kvm_virtio_pci_vq_vector_use(proxy, queue_no, vector, msg);
+ ret = kvm_virtio_pci_vq_vector_use(proxy, queue_no, vector);
if (ret < 0) {
goto undo;
}
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 0e18f15..20b52f0 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -470,7 +470,18 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
}
}
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev);
+/**
+ * kvm_irqchip_add_msi_route - Add MSI route for specific vector
+ * @s: KVM state
+ * @vector: which vector to add. This can be either MSI/MSIX
+ * vector. The function will automatically detect whether
+ * MSI/MSIX is enabled, and fetch corresponding MSI
+ * message.
+ * @dev: Owner PCI device to add the route. If @dev is specified
+ * as @NULL, an empty MSI message will be inited.
+ * @return: virq (>=0) when success, errno (<0) when failed.
+ */
+int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev);
int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
PCIDevice *dev);
void kvm_irqchip_release_virq(KVMState *s, int virq);
diff --git a/kvm-all.c b/kvm-all.c
index e7b66df..f0dc769 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -26,6 +26,7 @@
#include "qemu/error-report.h"
#include "hw/hw.h"
#include "hw/pci/msi.h"
+#include "hw/pci/msix.h"
#include "hw/s390x/adapter.h"
#include "exec/gdbstub.h"
#include "sysemu/kvm_int.h"
@@ -1177,10 +1178,23 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg)
return kvm_set_irq(s, route->kroute.gsi, 1);
}
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev)
+int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
{
struct kvm_irq_routing_entry kroute = {};
int virq;
+ MSIMessage msg = {0, 0};
+
+ if (dev) {
+ if (msix_enabled(dev)) {
+ msg = msix_get_message(dev, vector);
+ } else if (msi_enabled(dev)) {
+ msg = msi_get_message(dev, vector);
+ } else {
+ /* Should never happen */
+ error_report("%s: unknown interrupt type", __func__);
+ abort();
+ }
+ }
if (kvm_gsi_direct_mapping()) {
return kvm_arch_msi_data_to_gsi(msg.data);
@@ -1330,7 +1344,7 @@ int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg)
abort();
}
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
+int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
{
return -ENOSYS;
}
diff --git a/kvm-stub.c b/kvm-stub.c
index b962b24..421c9ce 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -112,7 +112,7 @@ int kvm_on_sigbus(int code, void *addr)
}
#ifndef CONFIG_USER_ONLY
-int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg, PCIDevice *dev)
+int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
{
return -ENOSYS;
}
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 693d1c1..3ce568c 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -3158,8 +3158,7 @@ void kvm_arch_init_irq_routing(KVMState *s)
/* If the ioapic is in QEMU and the lapics are in KVM, reserve
MSI routes for signaling interrupts to the local apics. */
for (i = 0; i < IOAPIC_NUM_PINS; i++) {
- struct MSIMessage msg = { 0x0, 0x0 };
- if (kvm_irqchip_add_msi_route(s, msg, NULL) < 0) {
+ if (kvm_irqchip_add_msi_route(s, 0, NULL) < 0) {
error_report("Could not enable split IRQ mode.");
exit(1);
}
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 22/25] kvm-irqchip: i386: add hook for add/remove virq
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (20 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 21/25] kvm-irqchip: simplify kvm_irqchip_add_msi_route Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 23/25] kvm-irqchip: x86: add msi route notify fn Peter Xu
` (3 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
Adding two hooks to be notified when adding/removing msi routes. There
are two kinds of MSI routes:
- in kvm_irqchip_add_irq_route(): before assigning IRQFD. Used by
vhost, vfio, etc.
- in kvm_irqchip_send_msi(): when sending direct MSI message, if
direct MSI not allowed, we will first create one MSI route entry
in the kernel, then trigger it.
This patch only hooks the first one (irqfd case). We do not need to
take care for the 2nd one, since it's only used by QEMU userspace
(kvm-apic) and the messages will always do in-time translation when
triggered. While we need to note them down for the 1st one, so that we
can notify the kernel when cache invalidation happens.
Also, we do not hook IOAPIC msi routes (we have explicit notifier for
IOAPIC to keep its cache updated). We only need to care about irqfd
users.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
include/sysemu/kvm.h | 6 ++++++
kvm-all.c | 2 ++
target-arm/kvm.c | 11 +++++++++++
target-i386/kvm.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
target-mips/kvm.c | 11 +++++++++++
target-ppc/kvm.c | 11 +++++++++++
target-s390x/kvm.c | 11 +++++++++++
trace-events | 2 ++
8 files changed, 102 insertions(+)
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 20b52f0..94a7f63 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -355,6 +355,12 @@ void kvm_arch_init_irq_routing(KVMState *s);
int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
uint64_t address, uint32_t data, PCIDevice *dev);
+/* Notify arch about newly added MSI routes */
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev);
+/* Notify arch about released MSI routes */
+int kvm_arch_release_virq_post(int virq);
+
int kvm_arch_msi_data_to_gsi(uint32_t data);
int kvm_set_irq(KVMState *s, int irq, int level);
diff --git a/kvm-all.c b/kvm-all.c
index f0dc769..a984564 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1073,6 +1073,7 @@ void kvm_irqchip_release_virq(KVMState *s, int virq)
}
}
clear_gsi(s, virq);
+ kvm_arch_release_virq_post(virq);
}
static unsigned int kvm_hash_msi(uint32_t data)
@@ -1221,6 +1222,7 @@ int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
}
kvm_add_routing_entry(s, &kroute);
+ kvm_arch_add_msi_route_post(&kroute, vector, dev);
kvm_irqchip_commit_routes(s);
return virq;
diff --git a/target-arm/kvm.c b/target-arm/kvm.c
index 3671032..90c293e 100644
--- a/target-arm/kvm.c
+++ b/target-arm/kvm.c
@@ -622,6 +622,17 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
return 0;
}
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev)
+{
+ return 0;
+}
+
+int kvm_arch_release_virq_post(int virq)
+{
+ return 0;
+}
+
int kvm_arch_msi_data_to_gsi(uint32_t data)
{
return (data - 32) & 0xffff;
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 3ce568c..3092e37 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -3356,6 +3356,54 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
return 0;
}
+typedef struct MSIRouteEntry MSIRouteEntry;
+
+struct MSIRouteEntry {
+ PCIDevice *dev; /* Device pointer */
+ int vector; /* MSI/MSIX vector index */
+ int virq; /* Virtual IRQ index */
+ QLIST_ENTRY(MSIRouteEntry) list;
+};
+
+/* List of used GSI routes */
+static QLIST_HEAD(, MSIRouteEntry) msi_route_list = \
+ QLIST_HEAD_INITIALIZER(msi_route_list);
+
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev)
+{
+ MSIRouteEntry *entry;
+
+ if (!dev) {
+ /* These are (possibly) IOAPIC routes only used for split
+ * kernel irqchip mode, while what we are housekeeping are
+ * PCI devices only. */
+ return 0;
+ }
+
+ entry = g_new0(MSIRouteEntry, 1);
+ entry->dev = dev;
+ entry->vector = vector;
+ entry->virq = route->gsi;
+ QLIST_INSERT_HEAD(&msi_route_list, entry, list);
+
+ trace_kvm_x86_add_msi_route(route->gsi);
+ return 0;
+}
+
+int kvm_arch_release_virq_post(int virq)
+{
+ MSIRouteEntry *entry, *next;
+ QLIST_FOREACH_SAFE(entry, &msi_route_list, list, next) {
+ if (entry->virq == virq) {
+ trace_kvm_x86_remove_msi_route(virq);
+ QLIST_REMOVE(entry, list);
+ break;
+ }
+ }
+ return 0;
+}
+
int kvm_arch_msi_data_to_gsi(uint32_t data)
{
abort();
diff --git a/target-mips/kvm.c b/target-mips/kvm.c
index 950bc05..1dd7904 100644
--- a/target-mips/kvm.c
+++ b/target-mips/kvm.c
@@ -1044,6 +1044,17 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
return 0;
}
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev)
+{
+ return 0;
+}
+
+int kvm_arch_release_virq_post(int virq)
+{
+ return 0;
+}
+
int kvm_arch_msi_data_to_gsi(uint32_t data)
{
abort();
diff --git a/target-ppc/kvm.c b/target-ppc/kvm.c
index c4c8146..143a2bf 100644
--- a/target-ppc/kvm.c
+++ b/target-ppc/kvm.c
@@ -2566,6 +2566,17 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
return 0;
}
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev)
+{
+ return 0;
+}
+
+int kvm_arch_release_virq_post(int virq)
+{
+ return 0;
+}
+
int kvm_arch_msi_data_to_gsi(uint32_t data)
{
return data & 0xffff;
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index e1859ca..22d2ed4 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -2246,6 +2246,17 @@ int kvm_arch_fixup_msi_route(struct kvm_irq_routing_entry *route,
return 0;
}
+int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
+ int vector, PCIDevice *dev)
+{
+ return 0;
+}
+
+int kvm_arch_release_virq_post(int virq)
+{
+ return 0;
+}
+
int kvm_arch_msi_data_to_gsi(uint32_t data)
{
abort();
diff --git a/trace-events b/trace-events
index 6e25884..e223f57 100644
--- a/trace-events
+++ b/trace-events
@@ -1911,3 +1911,5 @@ aspeed_vic_write(uint64_t offset, unsigned size, uint32_t data) "To 0x%" PRIx64
# target-i386/kvm.c
kvm_x86_fixup_msi_error(uint32_t gsi) "VT-d failed to remap interrupt for GSI %" PRIu32
+kvm_x86_add_msi_route(int virq) "Adding route entry for virq %d"
+kvm_x86_remove_msi_route(int virq) "Removing route entry for virq %d"
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 23/25] kvm-irqchip: x86: add msi route notify fn
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (21 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 22/25] kvm-irqchip: i386: add hook for add/remove virq Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 24/25] kvm-irqchip: do explicit commit when update irq Peter Xu
` (2 subsequent siblings)
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
One more IEC notifier is added to let msi routes know about the IEC
changes. When interrupt invalidation happens, all registered msi routes
will be updated for all PCI devices.
Since both vfio and vhost are possible gsi route consumers, this patch
will go one step further to keep them safe in split irqchip mode and
when irqfd is enabled.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/pci/pci.c | 15 +++++++++++++++
include/hw/pci/pci.h | 2 ++
kvm-all.c | 10 +---------
target-i386/kvm.c | 30 ++++++++++++++++++++++++++++++
trace-events | 1 +
5 files changed, 49 insertions(+), 9 deletions(-)
diff --git a/hw/pci/pci.c b/hw/pci/pci.c
index 7430715..ec1928f 100644
--- a/hw/pci/pci.c
+++ b/hw/pci/pci.c
@@ -2527,6 +2527,21 @@ uint16_t pci_requester_id(PCIDevice *dev)
return result;
}
+MSIMessage pci_get_msi_message(PCIDevice *dev, int vector)
+{
+ MSIMessage msg;
+ if (msix_enabled(dev)) {
+ msg = msix_get_message(dev, vector);
+ } else if (msi_enabled(dev)) {
+ msg = msi_get_message(dev, vector);
+ } else {
+ /* Should never happen */
+ error_report("%s: unknown interrupt type", __func__);
+ abort();
+ }
+ return msg;
+}
+
static const TypeInfo pci_device_type_info = {
.name = TYPE_PCI_DEVICE,
.parent = TYPE_DEVICE,
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h
index 351266c..359c22e 100644
--- a/include/hw/pci/pci.h
+++ b/include/hw/pci/pci.h
@@ -779,4 +779,6 @@ extern const VMStateDescription vmstate_pci_device;
.offset = vmstate_offset_pointer(_state, _field, PCIDevice), \
}
+MSIMessage pci_get_msi_message(PCIDevice *dev, int vector);
+
#endif
diff --git a/kvm-all.c b/kvm-all.c
index a984564..95f1df3 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1186,15 +1186,7 @@ int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
MSIMessage msg = {0, 0};
if (dev) {
- if (msix_enabled(dev)) {
- msg = msix_get_message(dev, vector);
- } else if (msi_enabled(dev)) {
- msg = msi_get_message(dev, vector);
- } else {
- /* Should never happen */
- error_report("%s: unknown interrupt type", __func__);
- abort();
- }
+ msg = pci_get_msi_message(dev, vector);
}
if (kvm_gsi_direct_mapping()) {
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 3092e37..f7a428e 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -37,6 +37,7 @@
#include "hw/i386/apic_internal.h"
#include "hw/i386/apic-msidef.h"
#include "hw/i386/intel_iommu.h"
+#include "hw/i386/x86-iommu.h"
#include "exec/ioport.h"
#include "standard-headers/asm-x86/hyperv.h"
@@ -3369,9 +3370,26 @@ struct MSIRouteEntry {
static QLIST_HEAD(, MSIRouteEntry) msi_route_list = \
QLIST_HEAD_INITIALIZER(msi_route_list);
+static void kvm_update_msi_routes_all(void *private, bool global,
+ uint32_t index, uint32_t mask)
+{
+ int cnt = 0;
+ MSIRouteEntry *entry;
+ MSIMessage msg;
+ /* TODO: explicit route update */
+ QLIST_FOREACH(entry, &msi_route_list, list) {
+ cnt++;
+ msg = pci_get_msi_message(entry->dev, entry->vector);
+ kvm_irqchip_update_msi_route(kvm_state, entry->virq,
+ msg, entry->dev);
+ }
+ trace_kvm_x86_update_msi_routes(cnt);
+}
+
int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
int vector, PCIDevice *dev)
{
+ static bool notify_list_inited = false;
MSIRouteEntry *entry;
if (!dev) {
@@ -3388,6 +3406,18 @@ int kvm_arch_add_msi_route_post(struct kvm_irq_routing_entry *route,
QLIST_INSERT_HEAD(&msi_route_list, entry, list);
trace_kvm_x86_add_msi_route(route->gsi);
+
+ if (!notify_list_inited) {
+ /* For the first time we do add route, add ourselves into
+ * IOMMU's IEC notify list if needed. */
+ X86IOMMUState *iommu = x86_iommu_get_default();
+ if (iommu) {
+ x86_iommu_iec_register_notifier(iommu,
+ kvm_update_msi_routes_all,
+ NULL);
+ }
+ notify_list_inited = true;
+ }
return 0;
}
diff --git a/trace-events b/trace-events
index e223f57..3d30aee 100644
--- a/trace-events
+++ b/trace-events
@@ -1913,3 +1913,4 @@ aspeed_vic_write(uint64_t offset, unsigned size, uint32_t data) "To 0x%" PRIx64
kvm_x86_fixup_msi_error(uint32_t gsi) "VT-d failed to remap interrupt for GSI %" PRIu32
kvm_x86_add_msi_route(int virq) "Adding route entry for virq %d"
kvm_x86_remove_msi_route(int virq) "Removing route entry for virq %d"
+kvm_x86_update_msi_routes(int num) "Updated %d MSI routes"
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 24/25] kvm-irqchip: do explicit commit when update irq
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (22 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 23/25] kvm-irqchip: x86: add msi route notify fn Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 25/25] intel_iommu: support all masks in interrupt entry cache invalidation Peter Xu
2016-05-17 7:22 ` [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
In the past, we are doing gsi route commit for each irqchip route
update. This is not efficient if we are updating lots of routes in the
same time. This patch removes the committing phase in
kvm_irqchip_update_msi_route(). Instead, we do explicit commit after all
routes updated.
Signed-off-by: Peter Xu <peterx@redhat.com>
---
hw/i386/kvm/pci-assign.c | 2 ++
hw/misc/ivshmem.c | 1 +
hw/vfio/pci.c | 1 +
hw/virtio/virtio-pci.c | 1 +
include/sysemu/kvm.h | 2 +-
kvm-all.c | 2 --
kvm-stub.c | 4 ++++
target-i386/kvm.c | 1 +
8 files changed, 11 insertions(+), 3 deletions(-)
diff --git a/hw/i386/kvm/pci-assign.c b/hw/i386/kvm/pci-assign.c
index 9b43332..02ca89e 100644
--- a/hw/i386/kvm/pci-assign.c
+++ b/hw/i386/kvm/pci-assign.c
@@ -1015,6 +1015,7 @@ static void assigned_dev_update_msi_msg(PCIDevice *pci_dev)
kvm_irqchip_update_msi_route(kvm_state, assigned_dev->msi_virq[0],
msi_get_message(pci_dev, 0), pci_dev);
+ kvm_irqchip_commit_routes(kvm_state);
}
static bool assigned_dev_msix_masked(MSIXTableEntry *entry)
@@ -1601,6 +1602,7 @@ static void assigned_dev_msix_mmio_write(void *opaque, hwaddr addr,
if (ret) {
error_report("Error updating irq routing entry (%d)", ret);
}
+ kvm_irqchip_commit_routes(kvm_state);
}
}
}
diff --git a/hw/misc/ivshmem.c b/hw/misc/ivshmem.c
index 6909346..953d7f8 100644
--- a/hw/misc/ivshmem.c
+++ b/hw/misc/ivshmem.c
@@ -325,6 +325,7 @@ static int ivshmem_vector_unmask(PCIDevice *dev, unsigned vector,
if (ret < 0) {
return ret;
}
+ kvm_irqchip_commit_routes(kvm_state);
return kvm_irqchip_add_irqfd_notifier_gsi(kvm_state, n, NULL, v->virq);
}
diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
index 2b2f935..eb09bc6 100644
--- a/hw/vfio/pci.c
+++ b/hw/vfio/pci.c
@@ -458,6 +458,7 @@ static void vfio_update_kvm_msi_virq(VFIOMSIVector *vector, MSIMessage msg,
PCIDevice *pdev)
{
kvm_irqchip_update_msi_route(kvm_state, vector->virq, msg, pdev);
+ kvm_irqchip_commit_routes(kvm_state);
}
static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index df85f28..6342435 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -872,6 +872,7 @@ static int virtio_pci_vq_vector_unmask(VirtIOPCIProxy *proxy,
if (ret < 0) {
return ret;
}
+ kvm_irqchip_commit_routes(kvm_state);
}
}
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 94a7f63..b7a20eb 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -367,7 +367,6 @@ int kvm_set_irq(KVMState *s, int irq, int level);
int kvm_irqchip_send_msi(KVMState *s, MSIMessage msg);
void kvm_irqchip_add_irq_route(KVMState *s, int gsi, int irqchip, int pin);
-void kvm_irqchip_commit_routes(KVMState *s);
void kvm_put_apic_state(DeviceState *d, struct kvm_lapic_state *kapic);
void kvm_get_apic_state(DeviceState *d, struct kvm_lapic_state *kapic);
@@ -490,6 +489,7 @@ static inline void cpu_synchronize_post_init(CPUState *cpu)
int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev);
int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
PCIDevice *dev);
+void kvm_irqchip_commit_routes(KVMState *s);
void kvm_irqchip_release_virq(KVMState *s, int virq);
int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter);
diff --git a/kvm-all.c b/kvm-all.c
index 95f1df3..8106efb 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -1034,8 +1034,6 @@ static int kvm_update_routing_entry(KVMState *s,
*entry = *new_entry;
- kvm_irqchip_commit_routes(s);
-
return 0;
}
diff --git a/kvm-stub.c b/kvm-stub.c
index 421c9ce..d2c1a5b 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -131,6 +131,10 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
return -ENOSYS;
}
+void kvm_irqchip_commit_routes(KVMState *s)
+{
+}
+
int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
{
return -ENOSYS;
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index f7a428e..9b99d21 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -3383,6 +3383,7 @@ static void kvm_update_msi_routes_all(void *private, bool global,
kvm_irqchip_update_msi_route(kvm_state, entry->virq,
msg, entry->dev);
}
+ kvm_irqchip_commit_routes(kvm_state);
trace_kvm_x86_update_msi_routes(cnt);
}
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* [Qemu-devel] [PATCH v7 25/25] intel_iommu: support all masks in interrupt entry cache invalidation
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (23 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 24/25] kvm-irqchip: do explicit commit when update irq Peter Xu
@ 2016-05-17 7:15 ` Peter Xu
2016-05-17 7:22 ` [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:15 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4, peterx
From: Radim Krčmář <rkrcmar@redhat.com>
Linux guests do not gracefully handle cases when the invalidation mask
they wanted is not supported, probably because real hardware always
allowed all.
We can just say that all 16 masks are supported, because both
ioapic_iec_notifier and kvm_update_msi_routes_all invalidate all caches.
Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
---
hw/i386/intel_iommu.c | 2 +-
hw/i386/intel_iommu_internal.h | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 0e340b9..56e2975 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2353,7 +2353,7 @@ static void vtd_init(IntelIOMMUState *s)
s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO;
if (ms->iommu_intr) {
- s->ecap |= VTD_ECAP_IR | VTD_ECAP_EIM;
+ s->ecap |= VTD_ECAP_IR | VTD_ECAP_EIM | VTD_ECAP_MHMV;
}
vtd_reset_context_cache(s);
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index 72b0114..0829a50 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -186,6 +186,7 @@
/* Interrupt Remapping support */
#define VTD_ECAP_IR (1ULL << 3)
#define VTD_ECAP_EIM (1ULL << 4)
+#define VTD_ECAP_MHMV (15ULL << 20)
/* CAP_REG */
/* (offset >> 4) << 24 */
--
2.4.11
^ permalink raw reply related [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
` (24 preceding siblings ...)
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 25/25] intel_iommu: support all masks in interrupt entry cache invalidation Peter Xu
@ 2016-05-17 7:22 ` Peter Xu
25 siblings, 0 replies; 40+ messages in thread
From: Peter Xu @ 2016-05-17 7:22 UTC (permalink / raw)
To: qemu-devel
Cc: imammedo, rth, ehabkost, jasowang, marcel, mst, pbonzini,
jan.kiszka, rkrcmar, alex.williamson, wexu, davidkiarie4
On Tue, May 17, 2016 at 03:15:28PM +0800, Peter Xu wrote:
> This is v7 patchset for Intel IOMMU IR support. This patchset is
> based on the following two patches:
>
> - [PATCH v3] pci: fix pci_requester_id()
>
> https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg02672.html
>
> - [PATCH 0/2] IOAPIC: clear remote IRR for edge interrupts
>
> https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01431.html
>
> Please apply the above two patches as well before testing this
> patchset. The first patch fixes issue when IR is used with pci
> bridges, like pci-bridge device. The second patch fixes issue when
> IR is used with level-triggered interrupt devices, like e1000.
>
> v7 changes (using v6 patch index):
> - patch 10: trivial change in debug string (remove one more "\n")
> - patch 17-18: ioapic remote irr patches, sent seperately
> already. So removed from this series.
> - patch 24:
> - fix commit message: only irqfd msi routes are maintained, not
> all msi routes.
> - skip all IOAPIC msi entries (dev == NULL). We only need to
> housekeep irqfd users.
> - added patches
> - pick up Radim's patch on adding MHMV ecap bits [Radim]
> - remove all vtd_* patches, instead, use x86-iommu ones at the first
> place. This introduced lots of patch order changes and content
> changes, which affected from original patch 8 to the end. Sorry!
> [Jan]
Online branch for convenience:
https://github.com/xzpeter/qemu vtd-intr-v7
Thanks,
-- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class Peter Xu
@ 2016-05-23 17:06 ` David Kiarie
2016-05-23 21:48 ` Marcel Apfelbaum
0 siblings, 1 reply; 40+ messages in thread
From: David Kiarie @ 2016-05-23 17:06 UTC (permalink / raw)
To: Peter Xu
Cc: QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, Jan Kiszka,
rkrcmar, alex.williamson, wexu
On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
> Introducing parent class for intel-iommu devices named "x86-iommu". This
> is preparation work to abstract shared functionalities out from Intel
> and AMD IOMMUs. Currently, only the parent class is introduced. It does
> nothing yet.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> hw/i386/Makefile.objs | 2 +-
> hw/i386/intel_iommu.c | 5 ++--
> hw/i386/x86-iommu.c | 53 +++++++++++++++++++++++++++++++++++++++++++
> include/hw/i386/intel_iommu.h | 3 ++-
> include/hw/i386/x86-iommu.h | 46 +++++++++++++++++++++++++++++++++++++
> 5 files changed, 105 insertions(+), 4 deletions(-)
> create mode 100644 hw/i386/x86-iommu.c
> create mode 100644 include/hw/i386/x86-iommu.h
>
> diff --git a/hw/i386/Makefile.objs b/hw/i386/Makefile.objs
> index b52d5b8..90e94ff 100644
> --- a/hw/i386/Makefile.objs
> +++ b/hw/i386/Makefile.objs
> @@ -2,7 +2,7 @@ obj-$(CONFIG_KVM) += kvm/
> obj-y += multiboot.o
> obj-y += pc.o pc_piix.o pc_q35.o
> obj-y += pc_sysfw.o
> -obj-y += intel_iommu.o
> +obj-y += x86-iommu.o intel_iommu.o
> obj-$(CONFIG_XEN) += ../xenpv/ xen/
>
> obj-y += kvmvapic.o
> diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
> index 4d14124..0a70577 100644
> --- a/hw/i386/intel_iommu.c
> +++ b/hw/i386/intel_iommu.c
> @@ -2120,16 +2120,17 @@ static void vtd_realize(DeviceState *dev, Error **errp)
> static void vtd_class_init(ObjectClass *klass, void *data)
> {
> DeviceClass *dc = DEVICE_CLASS(klass);
> + X86IOMMUClass *x86_class = X86_IOMMU_CLASS(klass);
>
> dc->reset = vtd_reset;
> - dc->realize = vtd_realize;
> dc->vmsd = &vtd_vmstate;
> dc->props = vtd_properties;
> + x86_class->realize = vtd_realize;
> }
>
> static const TypeInfo vtd_info = {
> .name = TYPE_INTEL_IOMMU_DEVICE,
> - .parent = TYPE_SYS_BUS_DEVICE,
> + .parent = TYPE_X86_IOMMU_DEVICE,
> .instance_size = sizeof(IntelIOMMUState),
> .class_init = vtd_class_init,
> };
> diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
> new file mode 100644
> index 0000000..d739afb
> --- /dev/null
> +++ b/hw/i386/x86-iommu.c
> @@ -0,0 +1,53 @@
> +/*
> + * QEMU emulation of common X86 IOMMU
> + *
> + * Copyright (C) 2016 Peter Xu, Red Hat <peterx@redhat.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> +
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> +
> + * You should have received a copy of the GNU General Public License along
> + * with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "hw/sysbus.h"
> +#include "hw/boards.h"
> +#include "hw/i386/x86-iommu.h"
> +
> +static void x86_iommu_realize(DeviceState *dev, Error **errp)
> +{
> + X86IOMMUClass *x86_class = X86_IOMMU_GET_CLASS(dev);
> + if (x86_class->realize) {
> + x86_class->realize(dev, errp);
> + }
> +}
> +
> +static void x86_iommu_class_init(ObjectClass *klass, void *data)
> +{
> + DeviceClass *dc = DEVICE_CLASS(klass);
> + dc->realize = x86_iommu_realize;
> +}
> +
> +static const TypeInfo x86_iommu_info = {
> + .name = TYPE_X86_IOMMU_DEVICE,
> + .parent = TYPE_SYS_BUS_DEVICE,
> + .instance_size = sizeof(X86IOMMUState),
> + .class_init = x86_iommu_class_init,
> + .class_size = sizeof(X86IOMMUClass),
> + .abstract = true,
> +};
As I suspected am having some trouble parenting a PCI device from a
Bus device but I will investigate further to see if I can manage
something.
> +
> +static void x86_iommu_register_types(void)
> +{
> + type_register_static(&x86_iommu_info);
> +}
> +
> +type_init(x86_iommu_register_types)
> diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
> index 4914fe6..c88a931 100644
> --- a/include/hw/i386/intel_iommu.h
> +++ b/include/hw/i386/intel_iommu.h
> @@ -23,6 +23,7 @@
> #define INTEL_IOMMU_H
> #include "hw/qdev.h"
> #include "sysemu/dma.h"
> +#include "hw/i386/x86-iommu.h"
>
> #define TYPE_INTEL_IOMMU_DEVICE "intel-iommu"
> #define INTEL_IOMMU_DEVICE(obj) \
> @@ -152,7 +153,7 @@ union VTD_IR_MSIAddress {
>
> /* The iommu (DMAR) device state struct */
> struct IntelIOMMUState {
> - SysBusDevice busdev;
> + X86IOMMUState x86_iommu;
> MemoryRegion csrmem;
> uint8_t csr[DMAR_REG_SIZE]; /* register values */
> uint8_t wmask[DMAR_REG_SIZE]; /* R/W bytes */
> diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
> new file mode 100644
> index 0000000..924f39a
> --- /dev/null
> +++ b/include/hw/i386/x86-iommu.h
> @@ -0,0 +1,46 @@
> +/*
> + * Common IOMMU interface for X86 platform
> + *
> + * Copyright (C) 2016 Peter Xu, Red Hat <peterx@redhat.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> +
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> +
> + * You should have received a copy of the GNU General Public License along
> + * with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef IOMMU_COMMON_H
> +#define IOMMU_COMMON_H
> +
> +#include "hw/sysbus.h"
> +
> +#define TYPE_X86_IOMMU_DEVICE ("x86-iommu")
> +#define X86_IOMMU_DEVICE(obj) \
> + OBJECT_CHECK(X86IOMMUState, (obj), TYPE_X86_IOMMU_DEVICE)
> +#define X86_IOMMU_CLASS(klass) \
> + OBJECT_CLASS_CHECK(X86IOMMUClass, (klass), TYPE_X86_IOMMU_DEVICE)
> +#define X86_IOMMU_GET_CLASS(obj) \
> + OBJECT_GET_CLASS(X86IOMMUClass, obj, TYPE_X86_IOMMU_DEVICE)
> +
> +typedef struct X86IOMMUState X86IOMMUState;
> +typedef struct X86IOMMUClass X86IOMMUClass;
> +
> +struct X86IOMMUClass {
> + SysBusDeviceClass parent;
> + /* Intel/AMD specific realize() hook */
> + DeviceRealize realize;
> +};
> +
> +struct X86IOMMUState {
> + SysBusDevice busdev;
> +};
> +
> +#endif
> --
> 2.4.11
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-23 17:06 ` David Kiarie
@ 2016-05-23 21:48 ` Marcel Apfelbaum
2016-05-24 10:40 ` Jan Kiszka
0 siblings, 1 reply; 40+ messages in thread
From: Marcel Apfelbaum @ 2016-05-23 21:48 UTC (permalink / raw)
To: David Kiarie, Peter Xu
Cc: ehabkost, Michael S. Tsirkin, jasowang, rkrcmar, QEMU Developers,
alex.williamson, Jan Kiszka, wexu, pbonzini, Marcel Apfelbaum,
imammedo, rth
On 05/23/2016 08:06 PM, David Kiarie wrote:
> On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
>> Introducing parent class for intel-iommu devices named "x86-iommu". This
>> is preparation work to abstract shared functionalities out from Intel
>> and AMD IOMMUs. Currently, only the parent class is introduced. It does
>> nothing yet.
>>
>> Signed-off-by: Peter Xu <peterx@redhat.com>
>> ---
>> hw/i386/Makefile.objs | 2 +-
[...]
>> +
>> +static const TypeInfo x86_iommu_info = {
>> + .name = TYPE_X86_IOMMU_DEVICE,
>> + .parent = TYPE_SYS_BUS_DEVICE,
>> + .instance_size = sizeof(X86IOMMUState),
>> + .class_init = x86_iommu_class_init,
>> + .class_size = sizeof(X86IOMMUClass),
>> + .abstract = true,
>> +};
>
> As I suspected am having some trouble parenting a PCI device from a
> Bus device but I will investigate further to see if I can manage
> something.
>
You cannot derive from both SYS_BUS_DEVICE and PCI_DEVICE.
You would need a composition; your device would be a SYS_BUS_DEVICE
and its state would include a PCI_DEVICE (or the other way around).
Then you can divide the responsibilities between them.
Thanks,
Marcel
>> +
[...]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-23 21:48 ` Marcel Apfelbaum
@ 2016-05-24 10:40 ` Jan Kiszka
2016-05-24 11:02 ` David Kiarie
0 siblings, 1 reply; 40+ messages in thread
From: Jan Kiszka @ 2016-05-24 10:40 UTC (permalink / raw)
To: marcel, David Kiarie, Peter Xu
Cc: ehabkost, Michael S. Tsirkin, jasowang, rkrcmar, QEMU Developers,
alex.williamson, wexu, pbonzini, imammedo, rth
[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]
On 2016-05-23 23:48, Marcel Apfelbaum wrote:
> On 05/23/2016 08:06 PM, David Kiarie wrote:
>> On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
>>> Introducing parent class for intel-iommu devices named "x86-iommu". This
>>> is preparation work to abstract shared functionalities out from Intel
>>> and AMD IOMMUs. Currently, only the parent class is introduced. It does
>>> nothing yet.
>>>
>>> Signed-off-by: Peter Xu <peterx@redhat.com>
>>> ---
>>> hw/i386/Makefile.objs | 2 +-
>
> [...]
>
>>> +
>>> +static const TypeInfo x86_iommu_info = {
>>> + .name = TYPE_X86_IOMMU_DEVICE,
>>> + .parent = TYPE_SYS_BUS_DEVICE,
>>> + .instance_size = sizeof(X86IOMMUState),
>>> + .class_init = x86_iommu_class_init,
>>> + .class_size = sizeof(X86IOMMUClass),
>>> + .abstract = true,
>>> +};
>>
>> As I suspected am having some trouble parenting a PCI device from a
>> Bus device but I will investigate further to see if I can manage
>> something.
>>
>
> You cannot derive from both SYS_BUS_DEVICE and PCI_DEVICE.
> You would need a composition; your device would be a SYS_BUS_DEVICE
> and its state would include a PCI_DEVICE (or the other way around).
> Then you can divide the responsibilities between them.
Given that the AMD IOMMU is more a platform than a PCI device, I would
also go for deriving from SYS_BUS_DEVICE (and later on a common x86
IOMMU class) and embedding a PCI_DEVICE. And the Intel IOMMU has no PCI
device feature at all.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-24 10:40 ` Jan Kiszka
@ 2016-05-24 11:02 ` David Kiarie
2016-05-24 11:29 ` David Kiarie
0 siblings, 1 reply; 40+ messages in thread
From: David Kiarie @ 2016-05-24 11:02 UTC (permalink / raw)
To: Jan Kiszka
Cc: Marcel Apfelbaum, Peter Xu, ehabkost, Michael S. Tsirkin,
jasowang, rkrcmar, QEMU Developers, alex.williamson, wexu,
pbonzini, imammedo, rth
On Tue, May 24, 2016 at 1:40 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
> On 2016-05-23 23:48, Marcel Apfelbaum wrote:
>> On 05/23/2016 08:06 PM, David Kiarie wrote:
>>> On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
>>>> Introducing parent class for intel-iommu devices named "x86-iommu". This
>>>> is preparation work to abstract shared functionalities out from Intel
>>>> and AMD IOMMUs. Currently, only the parent class is introduced. It does
>>>> nothing yet.
>>>>
>>>> Signed-off-by: Peter Xu <peterx@redhat.com>
>>>> ---
>>>> hw/i386/Makefile.objs | 2 +-
>>
>> [...]
>>
>>>> +
>>>> +static const TypeInfo x86_iommu_info = {
>>>> + .name = TYPE_X86_IOMMU_DEVICE,
>>>> + .parent = TYPE_SYS_BUS_DEVICE,
>>>> + .instance_size = sizeof(X86IOMMUState),
>>>> + .class_init = x86_iommu_class_init,
>>>> + .class_size = sizeof(X86IOMMUClass),
>>>> + .abstract = true,
>>>> +};
>>>
>>> As I suspected am having some trouble parenting a PCI device from a
>>> Bus device but I will investigate further to see if I can manage
>>> something.
>>>
>>
>> You cannot derive from both SYS_BUS_DEVICE and PCI_DEVICE.
>> You would need a composition; your device would be a SYS_BUS_DEVICE
>> and its state would include a PCI_DEVICE (or the other way around).
>> Then you can divide the responsibilities between them.
>
> Given that the AMD IOMMU is more a platform than a PCI device, I would
> also go for deriving from SYS_BUS_DEVICE (and later on a common x86
> IOMMU class) and embedding a PCI_DEVICE. And the Intel IOMMU has no PCI
> device feature at all.
Yes, I managed to do that by getting rid of PCI device specific
callbacks(replaced them with DeviceState callbacks) so I get a compile
and no runtime fatality but device(AMD IOMMU) never appears in the
device tree.
>
> Jan
>
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class
2016-05-24 11:02 ` David Kiarie
@ 2016-05-24 11:29 ` David Kiarie
0 siblings, 0 replies; 40+ messages in thread
From: David Kiarie @ 2016-05-24 11:29 UTC (permalink / raw)
To: Jan Kiszka
Cc: Marcel Apfelbaum, Peter Xu, ehabkost, Michael S. Tsirkin,
jasowang, rkrcmar, QEMU Developers, alex.williamson, wexu,
pbonzini, imammedo, rth
On Tue, May 24, 2016 at 2:02 PM, David Kiarie <davidkiarie4@gmail.com> wrote:
> On Tue, May 24, 2016 at 1:40 PM, Jan Kiszka <jan.kiszka@web.de> wrote:
>> On 2016-05-23 23:48, Marcel Apfelbaum wrote:
>>> On 05/23/2016 08:06 PM, David Kiarie wrote:
>>>> On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
>>>>> Introducing parent class for intel-iommu devices named "x86-iommu". This
>>>>> is preparation work to abstract shared functionalities out from Intel
>>>>> and AMD IOMMUs. Currently, only the parent class is introduced. It does
>>>>> nothing yet.
>>>>>
>>>>> Signed-off-by: Peter Xu <peterx@redhat.com>
>>>>> ---
>>>>> hw/i386/Makefile.objs | 2 +-
>>>
>>> [...]
>>>
>>>>> +
>>>>> +static const TypeInfo x86_iommu_info = {
>>>>> + .name = TYPE_X86_IOMMU_DEVICE,
>>>>> + .parent = TYPE_SYS_BUS_DEVICE,
>>>>> + .instance_size = sizeof(X86IOMMUState),
>>>>> + .class_init = x86_iommu_class_init,
>>>>> + .class_size = sizeof(X86IOMMUClass),
>>>>> + .abstract = true,
>>>>> +};
>>>>
>>>> As I suspected am having some trouble parenting a PCI device from a
>>>> Bus device but I will investigate further to see if I can manage
>>>> something.
>>>>
>>>
>>> You cannot derive from both SYS_BUS_DEVICE and PCI_DEVICE.
>>> You would need a composition; your device would be a SYS_BUS_DEVICE
>>> and its state would include a PCI_DEVICE (or the other way around).
>>> Then you can divide the responsibilities between them.
>>
>> Given that the AMD IOMMU is more a platform than a PCI device, I would
>> also go for deriving from SYS_BUS_DEVICE (and later on a common x86
>> IOMMU class) and embedding a PCI_DEVICE. And the Intel IOMMU has no PCI
>> device feature at all.
huh, should it be possible to embed the whole PCI device state
?Haven't tried that yet.
>
> Yes, I managed to do that by getting rid of PCI device specific
> callbacks(replaced them with DeviceState callbacks) so I get a compile
> and no runtime fatality but device(AMD IOMMU) never appears in the
> device tree.
>
>>
>> Jan
>>
>>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR Peter Xu
@ 2016-05-29 8:20 ` David Kiarie
2016-05-29 8:21 ` David Kiarie
0 siblings, 1 reply; 40+ messages in thread
From: David Kiarie @ 2016-05-29 8:20 UTC (permalink / raw)
To: Peter Xu
Cc: QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, Jan Kiszka,
rkrcmar, alex.williamson, wexu
On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
> Several data structs are defined to better support the rest of the
> patches: IRTE to parse remapping table entries, and IOAPIC/MSI related
> structure bits to parse interrupt entries to be filled in by guest
> kernel.
>
> Signed-off-by: Peter Xu <peterx@redhat.com>
> ---
> include/hw/i386/intel_iommu.h | 60 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 60 insertions(+)
>
> diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
> index cc49839..4914fe6 100644
> --- a/include/hw/i386/intel_iommu.h
> +++ b/include/hw/i386/intel_iommu.h
> @@ -52,6 +52,9 @@ typedef struct IntelIOMMUState IntelIOMMUState;
> typedef struct VTDAddressSpace VTDAddressSpace;
> typedef struct VTDIOTLBEntry VTDIOTLBEntry;
> typedef struct VTDBus VTDBus;
> +typedef union VTD_IRTE VTD_IRTE;
> +typedef union VTD_IR_IOAPICEntry VTD_IR_IOAPICEntry;
> +typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress;
>
> /* Context-Entry */
> struct VTDContextEntry {
> @@ -90,6 +93,63 @@ struct VTDIOTLBEntry {
> bool write_flags;
> };
>
> +/* Interrupt Remapping Table Entry Definition */
> +union VTD_IRTE {
> + struct {
> + uint8_t present:1; /* Whether entry present/available */
> + uint8_t fault_disable:1; /* Fault Processing Disable */
> + uint8_t dest_mode:1; /* Destination Mode */
> + uint8_t redir_hint:1; /* Redirection Hint */
> + uint8_t trigger_mode:1; /* Trigger Mode */
> + uint8_t delivery_mode:3; /* Delivery Mode */
> + uint8_t __avail:4; /* Available spaces for software */
> + uint8_t __reserved_0:3; /* Reserved 0 */
> + uint8_t irte_mode:1; /* IRTE Mode */
> + uint8_t vector:8; /* Interrupt Vector */
> + uint8_t __reserved_1:8; /* Reserved 1 */
> + uint32_t dest_id:32; /* Destination ID */
> + uint16_t source_id:16; /* Source-ID */
> + uint8_t sid_q:2; /* Source-ID Qualifier */
> + uint8_t sid_vtype:2; /* Source-ID Validation Type */
> + uint64_t __reserved_2:44; /* Reserved 2 */
> + } QEMU_PACKED;
> + uint64_t data[2];
> +};
> +
> +/* Programming format for IOAPIC table entries */
> +union VTD_IR_IOAPICEntry {
> + struct {
> + uint8_t vector:8; /* Vector */
> + uint8_t __zeros:3; /* Reserved (all zero) */
> + uint8_t index_h:1; /* Interrupt Index bit 15 */
> + uint8_t status:1; /* Deliver Status */
> + uint8_t polarity:1; /* Interrupt Polarity */
> + uint8_t remote_irr:1; /* Remote IRR */
> + uint8_t trigger_mode:1; /* Trigger Mode */
> + uint8_t mask:1; /* Mask */
> + uint32_t __reserved:31; /* Reserved (should all zero) */
> + uint8_t int_mode:1; /* Interrupt Format */
> + uint16_t index_l:15; /* Interrupt Index bits 14-0 */
> + } QEMU_PACKED;
> + uint64_t data;
> +};
> +
> +/* Programming format for MSI/MSI-X addresses */
> +union VTD_IR_MSIAddress {
> + struct {
> + uint8_t __not_care:2;
> + uint8_t index_h:1; /* Interrupt index bit 15 */
> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
> + uint8_t int_mode:1; /* Interrupt format */
> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
> + uint16_t __head:12; /* Should always be: 0x0fee */
> + } QEMU_PACKED;
> + uint32_t data;
> +};
In a recent discussion, it was brought to my attention that you might
have a problem with bitfields when the host cpu is not x86. Have you
considered this ?
> +
> +/* When IR is enabled, all MSI/MSI-X data bits should be zero */
> +#define VTD_IR_MSI_DATA (0)
> +
> /* The iommu (DMAR) device state struct */
> struct IntelIOMMUState {
> SysBusDevice busdev;
> --
> 2.4.11
>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-29 8:20 ` David Kiarie
@ 2016-05-29 8:21 ` David Kiarie
2016-05-30 5:45 ` Peter Xu
0 siblings, 1 reply; 40+ messages in thread
From: David Kiarie @ 2016-05-29 8:21 UTC (permalink / raw)
To: Peter Xu
Cc: QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, Jan Kiszka,
rkrcmar, alex.williamson, wexu
On Sun, May 29, 2016 at 11:20 AM, David Kiarie <davidkiarie4@gmail.com> wrote:
> On Tue, May 17, 2016 at 10:15 AM, Peter Xu <peterx@redhat.com> wrote:
>> Several data structs are defined to better support the rest of the
>> patches: IRTE to parse remapping table entries, and IOAPIC/MSI related
>> structure bits to parse interrupt entries to be filled in by guest
>> kernel.
>>
>> Signed-off-by: Peter Xu <peterx@redhat.com>
>> ---
>> include/hw/i386/intel_iommu.h | 60 +++++++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 60 insertions(+)
>>
>> diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
>> index cc49839..4914fe6 100644
>> --- a/include/hw/i386/intel_iommu.h
>> +++ b/include/hw/i386/intel_iommu.h
>> @@ -52,6 +52,9 @@ typedef struct IntelIOMMUState IntelIOMMUState;
>> typedef struct VTDAddressSpace VTDAddressSpace;
>> typedef struct VTDIOTLBEntry VTDIOTLBEntry;
>> typedef struct VTDBus VTDBus;
>> +typedef union VTD_IRTE VTD_IRTE;
>> +typedef union VTD_IR_IOAPICEntry VTD_IR_IOAPICEntry;
>> +typedef union VTD_IR_MSIAddress VTD_IR_MSIAddress;
>>
>> /* Context-Entry */
>> struct VTDContextEntry {
>> @@ -90,6 +93,63 @@ struct VTDIOTLBEntry {
>> bool write_flags;
>> };
>>
>> +/* Interrupt Remapping Table Entry Definition */
>> +union VTD_IRTE {
>> + struct {
>> + uint8_t present:1; /* Whether entry present/available */
>> + uint8_t fault_disable:1; /* Fault Processing Disable */
>> + uint8_t dest_mode:1; /* Destination Mode */
>> + uint8_t redir_hint:1; /* Redirection Hint */
>> + uint8_t trigger_mode:1; /* Trigger Mode */
>> + uint8_t delivery_mode:3; /* Delivery Mode */
>> + uint8_t __avail:4; /* Available spaces for software */
>> + uint8_t __reserved_0:3; /* Reserved 0 */
>> + uint8_t irte_mode:1; /* IRTE Mode */
>> + uint8_t vector:8; /* Interrupt Vector */
>> + uint8_t __reserved_1:8; /* Reserved 1 */
>> + uint32_t dest_id:32; /* Destination ID */
>> + uint16_t source_id:16; /* Source-ID */
>> + uint8_t sid_q:2; /* Source-ID Qualifier */
>> + uint8_t sid_vtype:2; /* Source-ID Validation Type */
>> + uint64_t __reserved_2:44; /* Reserved 2 */
>> + } QEMU_PACKED;
>> + uint64_t data[2];
>> +};
>> +
>> +/* Programming format for IOAPIC table entries */
>> +union VTD_IR_IOAPICEntry {
>> + struct {
>> + uint8_t vector:8; /* Vector */
>> + uint8_t __zeros:3; /* Reserved (all zero) */
>> + uint8_t index_h:1; /* Interrupt Index bit 15 */
>> + uint8_t status:1; /* Deliver Status */
>> + uint8_t polarity:1; /* Interrupt Polarity */
>> + uint8_t remote_irr:1; /* Remote IRR */
>> + uint8_t trigger_mode:1; /* Trigger Mode */
>> + uint8_t mask:1; /* Mask */
>> + uint32_t __reserved:31; /* Reserved (should all zero) */
>> + uint8_t int_mode:1; /* Interrupt Format */
>> + uint16_t index_l:15; /* Interrupt Index bits 14-0 */
>> + } QEMU_PACKED;
>> + uint64_t data;
>> +};
>> +
>> +/* Programming format for MSI/MSI-X addresses */
>> +union VTD_IR_MSIAddress {
>> + struct {
>> + uint8_t __not_care:2;
>> + uint8_t index_h:1; /* Interrupt index bit 15 */
>> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
>> + uint8_t int_mode:1; /* Interrupt format */
>> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
>> + uint16_t __head:12; /* Should always be: 0x0fee */
>> + } QEMU_PACKED;
>> + uint32_t data;
>> +};
>
> In a recent discussion, it was brought to my attention that you might
> have a problem with bitfields when the host cpu is not x86. Have you
> considered this ?
In a case when say the host cpu is little endian.
>
>> +
>> +/* When IR is enabled, all MSI/MSI-X data bits should be zero */
>> +#define VTD_IR_MSI_DATA (0)
>> +
>> /* The iommu (DMAR) device state struct */
>> struct IntelIOMMUState {
>> SysBusDevice busdev;
>> --
>> 2.4.11
>>
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-29 8:21 ` David Kiarie
@ 2016-05-30 5:45 ` Peter Xu
2016-05-30 5:56 ` Jan Kiszka
0 siblings, 1 reply; 40+ messages in thread
From: Peter Xu @ 2016-05-30 5:45 UTC (permalink / raw)
To: David Kiarie
Cc: QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, Jan Kiszka,
rkrcmar, alex.williamson, wexu
On Sun, May 29, 2016 at 11:21:35AM +0300, David Kiarie wrote:
[...]
> >> +
> >> +/* Programming format for MSI/MSI-X addresses */
> >> +union VTD_IR_MSIAddress {
> >> + struct {
> >> + uint8_t __not_care:2;
> >> + uint8_t index_h:1; /* Interrupt index bit 15 */
> >> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
> >> + uint8_t int_mode:1; /* Interrupt format */
> >> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
> >> + uint16_t __head:12; /* Should always be: 0x0fee */
> >> + } QEMU_PACKED;
> >> + uint32_t data;
> >> +};
> >
> > In a recent discussion, it was brought to my attention that you might
> > have a problem with bitfields when the host cpu is not x86. Have you
> > considered this ?
>
> In a case when say the host cpu is little endian.
I assume you mean when host cpu is big endian. x86 was little endian,
and I was testing on x86.
I think you are right. I should do conditional byte swap for all
uint{16/32/64} cases within the fields. For example, index_l field in
above VTD_IR_MSIAddress. And there are several other cases that need
special treatment in the patchset. Will go over and fix corresponding
issues in next version.
Thanks!
-- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-30 5:45 ` Peter Xu
@ 2016-05-30 5:56 ` Jan Kiszka
2016-05-30 8:14 ` Peter Xu
0 siblings, 1 reply; 40+ messages in thread
From: Jan Kiszka @ 2016-05-30 5:56 UTC (permalink / raw)
To: Peter Xu, David Kiarie
Cc: QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, rkrcmar,
alex.williamson, wexu
[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]
On 2016-05-30 07:45, Peter Xu wrote:
> On Sun, May 29, 2016 at 11:21:35AM +0300, David Kiarie wrote:
> [...]
>>>> +
>>>> +/* Programming format for MSI/MSI-X addresses */
>>>> +union VTD_IR_MSIAddress {
>>>> + struct {
>>>> + uint8_t __not_care:2;
>>>> + uint8_t index_h:1; /* Interrupt index bit 15 */
>>>> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
>>>> + uint8_t int_mode:1; /* Interrupt format */
>>>> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
>>>> + uint16_t __head:12; /* Should always be: 0x0fee */
>>>> + } QEMU_PACKED;
>>>> + uint32_t data;
>>>> +};
>>>
>>> In a recent discussion, it was brought to my attention that you might
>>> have a problem with bitfields when the host cpu is not x86. Have you
>>> considered this ?
>>
>> In a case when say the host cpu is little endian.
>
> I assume you mean when host cpu is big endian. x86 was little endian,
> and I was testing on x86.
>
> I think you are right. I should do conditional byte swap for all
> uint{16/32/64} cases within the fields. For example, index_l field in
> above VTD_IR_MSIAddress. And there are several other cases that need
> special treatment in the patchset. Will go over and fix corresponding
> issues in next version.
You actually need bit-swap with bit fields, see e.g. hw/net/vmxnet3.h.
Jan
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-30 5:56 ` Jan Kiszka
@ 2016-05-30 8:14 ` Peter Xu
2016-05-30 8:54 ` David Kiarie
0 siblings, 1 reply; 40+ messages in thread
From: Peter Xu @ 2016-05-30 8:14 UTC (permalink / raw)
To: Jan Kiszka
Cc: David Kiarie, QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, rkrcmar,
alex.williamson, wexu
On Mon, May 30, 2016 at 07:56:16AM +0200, Jan Kiszka wrote:
> On 2016-05-30 07:45, Peter Xu wrote:
> > On Sun, May 29, 2016 at 11:21:35AM +0300, David Kiarie wrote:
> > [...]
> >>>> +
> >>>> +/* Programming format for MSI/MSI-X addresses */
> >>>> +union VTD_IR_MSIAddress {
> >>>> + struct {
> >>>> + uint8_t __not_care:2;
> >>>> + uint8_t index_h:1; /* Interrupt index bit 15 */
> >>>> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
> >>>> + uint8_t int_mode:1; /* Interrupt format */
> >>>> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
> >>>> + uint16_t __head:12; /* Should always be: 0x0fee */
> >>>> + } QEMU_PACKED;
> >>>> + uint32_t data;
> >>>> +};
> >>>
> >>> In a recent discussion, it was brought to my attention that you might
> >>> have a problem with bitfields when the host cpu is not x86. Have you
> >>> considered this ?
> >>
> >> In a case when say the host cpu is little endian.
> >
> > I assume you mean when host cpu is big endian. x86 was little endian,
> > and I was testing on x86.
> >
> > I think you are right. I should do conditional byte swap for all
> > uint{16/32/64} cases within the fields. For example, index_l field in
> > above VTD_IR_MSIAddress. And there are several other cases that need
> > special treatment in the patchset. Will go over and fix corresponding
> > issues in next version.
>
> You actually need bit-swap with bit fields, see e.g. hw/net/vmxnet3.h.
Not noticed about bit-field ordering before... So maybe I need both?
Thanks,
-- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-30 8:14 ` Peter Xu
@ 2016-05-30 8:54 ` David Kiarie
2016-05-30 9:16 ` Peter Xu
0 siblings, 1 reply; 40+ messages in thread
From: David Kiarie @ 2016-05-30 8:54 UTC (permalink / raw)
To: Peter Xu
Cc: Jan Kiszka, QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, rkrcmar,
alex.williamson, wexu
On Mon, May 30, 2016 at 11:14 AM, Peter Xu <peterx@redhat.com> wrote:
> On Mon, May 30, 2016 at 07:56:16AM +0200, Jan Kiszka wrote:
>> On 2016-05-30 07:45, Peter Xu wrote:
>> > On Sun, May 29, 2016 at 11:21:35AM +0300, David Kiarie wrote:
>> > [...]
>> >>>> +
>> >>>> +/* Programming format for MSI/MSI-X addresses */
>> >>>> +union VTD_IR_MSIAddress {
>> >>>> + struct {
>> >>>> + uint8_t __not_care:2;
>> >>>> + uint8_t index_h:1; /* Interrupt index bit 15 */
>> >>>> + uint8_t sub_valid:1; /* SHV: Sub-Handle Valid bit */
>> >>>> + uint8_t int_mode:1; /* Interrupt format */
>> >>>> + uint16_t index_l:15; /* Interrupt index bit 14-0 */
>> >>>> + uint16_t __head:12; /* Should always be: 0x0fee */
>> >>>> + } QEMU_PACKED;
>> >>>> + uint32_t data;
>> >>>> +};
>> >>>
>> >>> In a recent discussion, it was brought to my attention that you might
>> >>> have a problem with bitfields when the host cpu is not x86. Have you
>> >>> considered this ?
>> >>
>> >> In a case when say the host cpu is little endian.
>> >
>> > I assume you mean when host cpu is big endian. x86 was little endian,
>> > and I was testing on x86.
>> >
>> > I think you are right. I should do conditional byte swap for all
>> > uint{16/32/64} cases within the fields. For example, index_l field in
>> > above VTD_IR_MSIAddress. And there are several other cases that need
>> > special treatment in the patchset. Will go over and fix corresponding
>> > issues in next version.
>>
>> You actually need bit-swap with bit fields, see e.g. hw/net/vmxnet3.h.
>
> Not noticed about bit-field ordering before... So maybe I need both?
Yes, I think we will need both though, I think, byte swapping the
whole struct will break the code but swapping individual fields is
what we need.
Myself, I'm defining bitfields as below:
struct CMDCompletionWait {
#ifdef __BIG_ENDIAN_BITFIELD
uint32_t type:4; /* command type */
uint32_t reserved:8;
uint64_t store_addr:49; /* addr to write */
uint32_t completion_flush:1; /* allow more executions */
uint32_t completion_int:1; /* set MMIOWAITINT */
uint32_t completion_store:1; /* write data to address */
#else
uint32_t completion_store:1;
uint32_t completion_int:1;
uint32_t completion_flush:1;
uint64_t store_addr:49;
uint32_t reserved:8;
uint32_t type:4;
#endif /* __BIG_ENDIAN_BITFIELD */
uint64_t store_data; /* data to write */
if
} QEMU_PACKED;
So, the bitfields are basically aligned to a {1,2,4,8}-byte boundary.
I will have to swap store_addr,type, store_data, e.t.c.
>
> Thanks,
>
> -- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-30 8:54 ` David Kiarie
@ 2016-05-30 9:16 ` Peter Xu
2016-05-30 9:25 ` David Kiarie
0 siblings, 1 reply; 40+ messages in thread
From: Peter Xu @ 2016-05-30 9:16 UTC (permalink / raw)
To: David Kiarie
Cc: Jan Kiszka, QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, rkrcmar,
alex.williamson, wexu
On Mon, May 30, 2016 at 11:54:52AM +0300, David Kiarie wrote:
> On Mon, May 30, 2016 at 11:14 AM, Peter Xu <peterx@redhat.com> wrote:
> > On Mon, May 30, 2016 at 07:56:16AM +0200, Jan Kiszka wrote:
> >> On 2016-05-30 07:45, Peter Xu wrote:
[...]
> >> >
> >> > I assume you mean when host cpu is big endian. x86 was little endian,
> >> > and I was testing on x86.
> >> >
> >> > I think you are right. I should do conditional byte swap for all
> >> > uint{16/32/64} cases within the fields. For example, index_l field in
> >> > above VTD_IR_MSIAddress. And there are several other cases that need
> >> > special treatment in the patchset. Will go over and fix corresponding
> >> > issues in next version.
> >>
> >> You actually need bit-swap with bit fields, see e.g. hw/net/vmxnet3.h.
> >
> > Not noticed about bit-field ordering before... So maybe I need both?
>
> Yes, I think we will need both though, I think, byte swapping the
> whole struct will break the code but swapping individual fields is
> what we need.
>
> Myself, I'm defining bitfields as below:
>
> struct CMDCompletionWait {
>
> #ifdef __BIG_ENDIAN_BITFIELD
> uint32_t type:4; /* command type */
> uint32_t reserved:8;
> uint64_t store_addr:49; /* addr to write */
> uint32_t completion_flush:1; /* allow more executions */
> uint32_t completion_int:1; /* set MMIOWAITINT */
> uint32_t completion_store:1; /* write data to address */
I guess what we need might be this one:
uint64_t type:4; /* command type */
uint64_t reserved:8;
uint64_t store_addr:49; /* addr to write */
uint64_t completion_flush:1; /* allow more executions */
uint64_t completion_int:1; /* set MMIOWAITINT */
uint64_t completion_store:1; /* write data to address */
IIUC, if we define type:4 as uint32_t rather than uint64_t, it should
be bits [29:32] of the struct on big endian machines, not bits
[61:64].
Thanks,
-- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
* Re: [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR
2016-05-30 9:16 ` Peter Xu
@ 2016-05-30 9:25 ` David Kiarie
0 siblings, 0 replies; 40+ messages in thread
From: David Kiarie @ 2016-05-30 9:25 UTC (permalink / raw)
To: Peter Xu
Cc: Jan Kiszka, QEMU Developers, imammedo, rth, ehabkost, jasowang,
Marcel Apfelbaum, Michael S. Tsirkin, pbonzini, rkrcmar,
alex.williamson, wexu
On Mon, May 30, 2016 at 12:16 PM, Peter Xu <peterx@redhat.com> wrote:
> On Mon, May 30, 2016 at 11:54:52AM +0300, David Kiarie wrote:
>> On Mon, May 30, 2016 at 11:14 AM, Peter Xu <peterx@redhat.com> wrote:
>> > On Mon, May 30, 2016 at 07:56:16AM +0200, Jan Kiszka wrote:
>> >> On 2016-05-30 07:45, Peter Xu wrote:
> [...]
>> >> >
>> >> > I assume you mean when host cpu is big endian. x86 was little endian,
>> >> > and I was testing on x86.
>> >> >
>> >> > I think you are right. I should do conditional byte swap for all
>> >> > uint{16/32/64} cases within the fields. For example, index_l field in
>> >> > above VTD_IR_MSIAddress. And there are several other cases that need
>> >> > special treatment in the patchset. Will go over and fix corresponding
>> >> > issues in next version.
>> >>
>> >> You actually need bit-swap with bit fields, see e.g. hw/net/vmxnet3.h.
>> >
>> > Not noticed about bit-field ordering before... So maybe I need both?
>>
>> Yes, I think we will need both though, I think, byte swapping the
>> whole struct will break the code but swapping individual fields is
>> what we need.
>>
>> Myself, I'm defining bitfields as below:
>>
>> struct CMDCompletionWait {
>>
>> #ifdef __BIG_ENDIAN_BITFIELD
>> uint32_t type:4; /* command type */
>> uint32_t reserved:8;
>> uint64_t store_addr:49; /* addr to write */
>> uint32_t completion_flush:1; /* allow more executions */
>> uint32_t completion_int:1; /* set MMIOWAITINT */
>> uint32_t completion_store:1; /* write data to address */
>
> I guess what we need might be this one:
>
> uint64_t type:4; /* command type */
> uint64_t reserved:8;
> uint64_t store_addr:49; /* addr to write */
> uint64_t completion_flush:1; /* allow more executions */
> uint64_t completion_int:1; /* set MMIOWAITINT */
> uint64_t completion_store:1; /* write data to address */
>
> IIUC, if we define type:4 as uint32_t rather than uint64_t, it should
> be bits [29:32] of the struct on big endian machines, not bits
> [61:64].
Yes, you're right.
>
> Thanks,
>
> -- peterx
^ permalink raw reply [flat|nested] 40+ messages in thread
end of thread, other threads:[~2016-05-30 9:25 UTC | newest]
Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-05-17 7:15 [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 01/25] acpi: enable INTR for DMAR report structure Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 02/25] intel_iommu: allow queued invalidation for IR Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 03/25] intel_iommu: set IR bit for ECAP register Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 04/25] acpi: add DMAR scope definition for root IOAPIC Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 05/25] intel_iommu: define interrupt remap table addr register Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 06/25] intel_iommu: handle interrupt remap enable Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 07/25] intel_iommu: define several structs for IOMMU IR Peter Xu
2016-05-29 8:20 ` David Kiarie
2016-05-29 8:21 ` David Kiarie
2016-05-30 5:45 ` Peter Xu
2016-05-30 5:56 ` Jan Kiszka
2016-05-30 8:14 ` Peter Xu
2016-05-30 8:54 ` David Kiarie
2016-05-30 9:16 ` Peter Xu
2016-05-30 9:25 ` David Kiarie
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 08/25] x86-iommu: introduce parent class Peter Xu
2016-05-23 17:06 ` David Kiarie
2016-05-23 21:48 ` Marcel Apfelbaum
2016-05-24 10:40 ` Jan Kiszka
2016-05-24 11:02 ` David Kiarie
2016-05-24 11:29 ` David Kiarie
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 09/25] x86-iommu: provide x86_iommu_get_default Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 10/25] x86-iommu: q35: generalize find_add_as() Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 11/25] intel_iommu: add IR translation faults defines Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 12/25] intel_iommu: Add support for PCI MSI remap Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 13/25] q35: ioapic: add support for emulated IOAPIC IR Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 14/25] ioapic: introduce ioapic_entry_parse() helper Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 15/25] intel_iommu: add support for split irqchip Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 16/25] q35: add "intremap" parameter to enable IR Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 17/25] x86-iommu: introduce IEC notifiers Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 18/25] ioapic: register IOMMU IEC notifier for ioapic Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 19/25] intel_iommu: Add support for Extended Interrupt Mode Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 20/25] intel_iommu: add SID validation for IR Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 21/25] kvm-irqchip: simplify kvm_irqchip_add_msi_route Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 22/25] kvm-irqchip: i386: add hook for add/remove virq Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 23/25] kvm-irqchip: x86: add msi route notify fn Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 24/25] kvm-irqchip: do explicit commit when update irq Peter Xu
2016-05-17 7:15 ` [Qemu-devel] [PATCH v7 25/25] intel_iommu: support all masks in interrupt entry cache invalidation Peter Xu
2016-05-17 7:22 ` [Qemu-devel] [PATCH v7 00/25] IOMMU: Enable interrupt remapping for Intel IOMMU Peter Xu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.