[Qemu-devel] [PATCH v6 0/8] vITS support

[Qemu-devel] [PATCH v6 0/8] vITS support

[Eric Auger]

This series introduces support for in-kernel GICv3 ITS emulation.

On dt guest the functionality is complete and was tested on Cavium ThunderX
with virtio-net-pci and vhost-net.

On ACPI guest the series was tested with virtio-net-pci only. For vhost-net,
using MSIX we currently miss the ACPI IORT table generation linking the
PCIe host controller with the ITS. The work is ongoing and will be submitted
separately. Anyway the kernel ACPI IORT ITS node support is not upstreamed
yet.

The first patch is not really related to virtual ITS but advertises the KVM
GSI routing support which goes along with MSI injection.

For ACPI ITS and PCIe support, use in-flight Tomasz' series:
- [PATCH V10 0/8] Introduce ACPI world to ITS,
  https://lkml.org/lkml/2016/9/6/153
- Support for ARM64 ACPI based PCI host controller,
  https://lwn.net/Articles/690995/

Git Information:
v6: https://github.com/eauger/qemu/tree/v2.7.0-vITS-v6
v5: https://github.com/eauger/qemu/tree/v2.6.0-vITS-v5
v4: https://github.com/eauger/qemu/tree/v2.6.0-vITS-v4
v3: https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg05197.html

History:
v5 => v6:
- moved from RFC to PATCH since all the host kernel dependencies now are
  fixed
- use Shannon's MADT patches I was not aware of
- smoother integration since the kernet ITS init sequence has now been
  updated in 4.8
- kvm_msi_use_devid flag handled in non archutecture specific code
- vmstate_its fields removed and migration_blocker added

v4 => v5:
- fix compilation issue with arm-softmmu target
- rebase on target-arm: Fix unreachable code in gicv3_class_name()
- add ACPI support
- kernel ITS init sequence is not yet similar to GICv2/V3 ones. This will
  be addressed in a subsequent respin.

v3 => v4:
- Took into account Peter's comments (at the exception of sub-class
  operation changes)
- rebase on Andre's kernel ITS emulation series v8
- rework KVM init sequence and KVM device creation/settings overall.
  I do not use kvm_arm_register_device due to the way the kernel
  API is devises (see discussion on the kernel ML)
- change the computation of the GITS_TRANSLATER base address
- fix compilation issues
- new "arget-arm: move gicv3_class_name from machine to kvm_arm.h"

v2 => v3:
- Really added unmigratable flag, was overlooked in v2
- Fixed checkpatch issue with initializing static variable to zero

v1 => v2:
- Added registers and reset method
- Added unmigratable flag
- Rebased on top of current master, use kvm_arch_fixup_msi_route() now

Pavel Fedin (5):
  hw/intc: Implement ITS base class
  kernel: Add vGICv3 ITS definitions
  kvm_arm: Pass requester ID to MSI routing functions
  kvm_arm: Implement support for ITS emulation by KVM
  arm/virt: Add ITS to the virt board


Eric Auger (2):
  hw/intc/arm_gic(v3)_kvm: Initialize gsi routing
  target-arm: move gicv3_class_name from machine to kvm_arm.h

Pavel Fedin (4):
  hw/intc/arm_gicv3_its: Implement ITS base class
  kvm-all: Pass requester ID to MSI routing functions
  hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation
  arm/virt: Add ITS to the virt board

Shannon Zhao (2):
  ACPI: Add GIC Interrupt Translation Service Structure definition
  ARM: Virt: ACPI: Add GIC ITS description in ACPI MADT table

 hw/arm/virt-acpi-build.c               |  12 +++
 hw/arm/virt.c                          |  47 +++++++++--
 hw/intc/Makefile.objs                  |   2 +
 hw/intc/arm_gic_kvm.c                  |  12 +++
 hw/intc/arm_gicv3_its_common.c         | 148 +++++++++++++++++++++++++++++++++
 hw/intc/arm_gicv3_its_kvm.c            | 136 ++++++++++++++++++++++++++++++
 hw/intc/arm_gicv3_kvm.c                |  13 +++
 include/hw/acpi/acpi-defs.h            |  13 ++-
 include/hw/intc/arm_gicv3_its_common.h |  77 +++++++++++++++++
 include/sysemu/kvm.h                   |   1 +
 kvm-all.c                              |   9 ++
 kvm-stub.c                             |   1 +
 target-arm/kvm_arm.h                   |  35 +++++++-
 target-arm/machine.c                   |  15 ----
 14 files changed, 498 insertions(+), 23 deletions(-)
 create mode 100644 hw/intc/arm_gicv3_its_common.c
 create mode 100644 hw/intc/arm_gicv3_its_kvm.c
 create mode 100644 include/hw/intc/arm_gicv3_its_common.h

-- 
2.5.5

[Qemu-devel] [PATCH v6 1/8] hw/intc/arm_gic(v3)_kvm: Initialize gsi routing

[Eric Auger]

From: Eric Auger <eric.auger@linaro.org>

Advertise gsi routing and set up irqchip routing entries for
GIC SPIs.

This is not mandated as long as MSI routing is not used
(because the kernel sets a default irqchip routing table).
However once MSI routing gets used (for VIRTIO-PCI vhost for
example), the first call to KVM_SET_GSI_ROUTING overrides the
kernel default irqchip table.

If no routing entry exists for the GSI, any IRQFD signaling for
this GSI will fail.

Signed-off-by: Eric Auger <eric.auger@linaro.org>

---

RFC -> PATCH:
- KVM gsi/MSI routing for ARM now supported in 4.8
- use kvm_has_gsi_routing()
---
 hw/intc/arm_gic_kvm.c   | 12 ++++++++++++
 hw/intc/arm_gicv3_kvm.c | 13 +++++++++++++
 2 files changed, 25 insertions(+)

diff --git a/hw/intc/arm_gic_kvm.c b/hw/intc/arm_gic_kvm.c
index 5593cdb..ae7ac58 100644
--- a/hw/intc/arm_gic_kvm.c
+++ b/hw/intc/arm_gic_kvm.c
@@ -577,6 +577,18 @@ static void kvm_arm_gic_realize(DeviceState *dev, Error **errp)
                                           "not support vGICv2 migration");
         migrate_add_blocker(s->migration_blocker);
     }
+
+    if (kvm_has_gsi_routing()) {
+        /* set up irq routing */
+        kvm_init_irq_routing(kvm_state);
+        for (i = 0; i < s->num_irq - GIC_INTERNAL; ++i) {
+            kvm_irqchip_add_irq_route(kvm_state, i, 0, i);
+        }
+
+        kvm_gsi_routing_allowed = true;
+
+        kvm_irqchip_commit_routes(kvm_state);
+    }
 }
 
 static void kvm_arm_gic_class_init(ObjectClass *klass, void *data)
diff --git a/hw/intc/arm_gicv3_kvm.c b/hw/intc/arm_gicv3_kvm.c
index 711fde3..199a439 100644
--- a/hw/intc/arm_gicv3_kvm.c
+++ b/hw/intc/arm_gicv3_kvm.c
@@ -85,6 +85,7 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
     GICv3State *s = KVM_ARM_GICV3(dev);
     KVMARMGICv3Class *kgc = KVM_ARM_GICV3_GET_CLASS(s);
     Error *local_err = NULL;
+    int i;
 
     DPRINTF("kvm_arm_gicv3_realize\n");
 
@@ -127,6 +128,18 @@ static void kvm_arm_gicv3_realize(DeviceState *dev, Error **errp)
      */
     error_setg(&s->migration_blocker, "vGICv3 migration is not implemented");
     migrate_add_blocker(s->migration_blocker);
+
+    if (kvm_has_gsi_routing()) {
+        /* set up irq routing */
+        kvm_init_irq_routing(kvm_state);
+        for (i = 0; i < s->num_irq - GIC_INTERNAL; ++i) {
+            kvm_irqchip_add_irq_route(kvm_state, i, 0, i);
+        }
+
+        kvm_gsi_routing_allowed = true;
+
+        kvm_irqchip_commit_routes(kvm_state);
+    }
 }
 
 static void kvm_arm_gicv3_class_init(ObjectClass *klass, void *data)
-- 
2.5.5

[Qemu-devel] [PATCH v6 2/8] hw/intc/arm_gicv3_its: Implement ITS base class

[Eric Auger]

From: Pavel Fedin <p.fedin@samsung.com>

This is the basic skeleton for both KVM and software-emulated ITS.
Since we already prepare status structure, we also introduce complete
VMState description. But, because we currently have no migratable
implementations, we also set unmigratable flag.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v5 -> v6:
- rename iomem_its into iomem_its_translation
- remove vmstate_its fields since the device is not yet
  migratable
- fix gicv3_its_trans_write print format
- add migration_blocker

v4 -> v5:
- send_msi can return 0 meaning the MSI was rejected by guest. Handle this
  value as an error

v3 -> v4:
- fix compilation error
- msi_supported -> msi_nonbroken
- streamid -> requester_id
- use PRIx64
- read ops uses _with_attrs form
- 16/32b write access allowed to GITS_TRANSLATER
- move its_class_name in kvm_arm.h and remove #ifdef TARGET_AARCH64
- add new kvm device dev_fd field
- add new gits_translater_gpa field in GICv3ITSState
---
 hw/intc/Makefile.objs                  |   1 +
 hw/intc/arm_gicv3_its_common.c         | 148 +++++++++++++++++++++++++++++++++
 include/hw/intc/arm_gicv3_its_common.h |  77 +++++++++++++++++
 target-arm/kvm_arm.h                   |  19 +++++
 4 files changed, 245 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_its_common.c
 create mode 100644 include/hw/intc/arm_gicv3_its_common.h

diff --git a/hw/intc/Makefile.objs b/hw/intc/Makefile.objs
index 05ec21b..23a39f7 100644
--- a/hw/intc/Makefile.objs
+++ b/hw/intc/Makefile.objs
@@ -16,6 +16,7 @@ common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_common.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv3.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_dist.o
 common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_redist.o
+common-obj-$(CONFIG_ARM_GIC) += arm_gicv3_its_common.o
 common-obj-$(CONFIG_OPENPIC) += openpic.o
 
 obj-$(CONFIG_APIC) += apic.o apic_common.o
diff --git a/hw/intc/arm_gicv3_its_common.c b/hw/intc/arm_gicv3_its_common.c
new file mode 100644
index 0000000..9d67c5c
--- /dev/null
+++ b/hw/intc/arm_gicv3_its_common.c
@@ -0,0 +1,148 @@
+/*
+ * ITS base class for a GICv3-based system
+ *
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
+ * Written by Pavel Fedin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/pci/msi.h"
+#include "hw/intc/arm_gicv3_its_common.h"
+#include "qemu/log.h"
+
+static void gicv3_its_pre_save(void *opaque)
+{
+    GICv3ITSState *s = (GICv3ITSState *)opaque;
+    GICv3ITSCommonClass *c = ARM_GICV3_ITS_COMMON_GET_CLASS(s);
+
+    if (c->pre_save) {
+        c->pre_save(s);
+    }
+}
+
+static int gicv3_its_post_load(void *opaque, int version_id)
+{
+    GICv3ITSState *s = (GICv3ITSState *)opaque;
+    GICv3ITSCommonClass *c = ARM_GICV3_ITS_COMMON_GET_CLASS(s);
+
+    if (c->post_load) {
+        c->post_load(s);
+    }
+    return 0;
+}
+
+static const VMStateDescription vmstate_its = {
+    .name = "arm_gicv3_its",
+    .pre_save = gicv3_its_pre_save,
+    .post_load = gicv3_its_post_load,
+    .unmigratable = true,
+};
+
+static MemTxResult gicv3_its_trans_read(void *opaque, hwaddr offset,
+                                        uint64_t *data, unsigned size,
+                                        MemTxAttrs attrs)
+{
+    qemu_log_mask(LOG_GUEST_ERROR, "ITS read at offset 0x%"PRIx64"\n", offset);
+    return MEMTX_ERROR;
+}
+
+static MemTxResult gicv3_its_trans_write(void *opaque, hwaddr offset,
+                                         uint64_t value, unsigned size,
+                                         MemTxAttrs attrs)
+{
+    if (offset == 0x0040 && ((size == 2) || (size == 4))) {
+        GICv3ITSState *s = ARM_GICV3_ITS_COMMON(opaque);
+        GICv3ITSCommonClass *c = ARM_GICV3_ITS_COMMON_GET_CLASS(s);
+        int ret = c->send_msi(s, le64_to_cpu(value), attrs.requester_id);
+
+        if (ret <= 0) {
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "ITS: Error sending MSI: %s\n", strerror(-ret));
+            return MEMTX_DECODE_ERROR;
+        }
+
+        return MEMTX_OK;
+    } else {
+        qemu_log_mask(LOG_GUEST_ERROR,
+                      "ITS write at bad offset 0x%"PRIx64"\n", offset);
+        return MEMTX_DECODE_ERROR;
+    }
+}
+
+static const MemoryRegionOps gicv3_its_trans_ops = {
+    .read_with_attrs = gicv3_its_trans_read,
+    .write_with_attrs = gicv3_its_trans_write,
+    .endianness = DEVICE_NATIVE_ENDIAN,
+};
+
+void gicv3_its_init_mmio(GICv3ITSState *s, const MemoryRegionOps *ops)
+{
+    SysBusDevice *sbd = SYS_BUS_DEVICE(s);
+
+    memory_region_init_io(&s->iomem_its_cntrl, OBJECT(s), ops, s,
+                          "control", ITS_CONTROL_SIZE);
+    memory_region_init_io(&s->iomem_its_translation, OBJECT(s),
+                          &gicv3_its_trans_ops, s,
+                          "translation", ITS_TRANS_SIZE);
+
+    /* Our two regions are always adjacent, therefore we now combine them
+     * into a single one in order to make our users' life easier.
+     */
+    memory_region_init(&s->iomem_main, OBJECT(s), "gicv3_its", ITS_SIZE);
+    memory_region_add_subregion(&s->iomem_main, 0, &s->iomem_its_cntrl);
+    memory_region_add_subregion(&s->iomem_main, ITS_CONTROL_SIZE,
+                                &s->iomem_its_translation);
+    sysbus_init_mmio(sbd, &s->iomem_main);
+
+    msi_nonbroken = true;
+}
+
+static void gicv3_its_common_reset(DeviceState *dev)
+{
+    GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
+
+    s->ctlr = 0;
+    s->cbaser = 0;
+    s->cwriter = 0;
+    s->creadr = 0;
+    memset(&s->baser, 0, sizeof(s->baser));
+
+    gicv3_its_post_load(s, 0);
+}
+
+static void gicv3_its_common_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->reset = gicv3_its_common_reset;
+    dc->vmsd = &vmstate_its;
+}
+
+static const TypeInfo gicv3_its_common_info = {
+    .name = TYPE_ARM_GICV3_ITS_COMMON,
+    .parent = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(GICv3ITSState),
+    .class_size = sizeof(GICv3ITSCommonClass),
+    .class_init = gicv3_its_common_class_init,
+    .abstract = true,
+};
+
+static void gicv3_its_common_register_types(void)
+{
+    type_register_static(&gicv3_its_common_info);
+}
+
+type_init(gicv3_its_common_register_types)
diff --git a/include/hw/intc/arm_gicv3_its_common.h b/include/hw/intc/arm_gicv3_its_common.h
new file mode 100644
index 0000000..8f6408f
--- /dev/null
+++ b/include/hw/intc/arm_gicv3_its_common.h
@@ -0,0 +1,77 @@
+/*
+ * ITS support for ARM GICv3
+ *
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
+ * Written by Pavel Fedin
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef QEMU_ARM_GICV3_ITS_COMMON_H
+#define QEMU_ARM_GICV3_ITS_COMMON_H
+
+#include "hw/sysbus.h"
+#include "hw/intc/arm_gicv3_common.h"
+
+#define ITS_CONTROL_SIZE 0x10000
+#define ITS_TRANS_SIZE   0x10000
+#define ITS_SIZE         (ITS_CONTROL_SIZE + ITS_TRANS_SIZE)
+
+struct GICv3ITSState {
+    SysBusDevice parent_obj;
+
+    MemoryRegion iomem_main;
+    MemoryRegion iomem_its_cntrl;
+    MemoryRegion iomem_its_translation;
+
+    GICv3State *gicv3;
+
+    int dev_fd; /* kvm device fd if backed by kvm vgic support */
+    uint64_t gits_translater_gpa;
+
+    /* Registers */
+    uint32_t ctlr;
+    uint64_t cbaser;
+    uint64_t cwriter;
+    uint64_t creadr;
+    uint64_t baser[8];
+
+    Error *migration_blocker;
+};
+
+typedef struct GICv3ITSState GICv3ITSState;
+
+void gicv3_its_init_mmio(GICv3ITSState *s, const MemoryRegionOps *ops);
+
+#define TYPE_ARM_GICV3_ITS_COMMON "arm-gicv3-its-common"
+#define ARM_GICV3_ITS_COMMON(obj) \
+     OBJECT_CHECK(GICv3ITSState, (obj), TYPE_ARM_GICV3_ITS_COMMON)
+#define ARM_GICV3_ITS_COMMON_CLASS(klass) \
+     OBJECT_CLASS_CHECK(GICv3ITSCommonClass, (klass), TYPE_ARM_GICV3_ITS_COMMON)
+#define ARM_GICV3_ITS_COMMON_GET_CLASS(obj) \
+     OBJECT_GET_CLASS(GICv3ITSCommonClass, (obj), TYPE_ARM_GICV3_ITS_COMMON)
+
+struct GICv3ITSCommonClass {
+    /*< private >*/
+    SysBusDeviceClass parent_class;
+    /*< public >*/
+
+    int (*send_msi)(GICv3ITSState *s, uint32_t data, uint16_t devid);
+    void (*pre_save)(GICv3ITSState *s);
+    void (*post_load)(GICv3ITSState *s);
+};
+
+typedef struct GICv3ITSCommonClass GICv3ITSCommonClass;
+
+#endif
diff --git a/target-arm/kvm_arm.h b/target-arm/kvm_arm.h
index a419368..544e404 100644
--- a/target-arm/kvm_arm.h
+++ b/target-arm/kvm_arm.h
@@ -255,4 +255,23 @@ struct kvm_guest_debug_arch;
 
 void kvm_arm_copy_hw_debug_data(struct kvm_guest_debug_arch *ptr);
 
+/**
+ * its_class_name
+ *
+ * Return the ITS class name to use depending on whether KVM acceleration
+ * and KVM CAP_SIGNAL_MSI are supported
+ *
+ * Returns: class name to use or NULL
+ */
+static inline const char *its_class_name(void)
+{
+    if (kvm_irqchip_in_kernel()) {
+        /* KVM implementation requires this capability */
+        return kvm_direct_msi_enabled() ? "arm-its-kvm" : NULL;
+    } else {
+        /* Software emulation is not implemented yet */
+        return NULL;
+    }
+}
+
 #endif
-- 
2.5.5

[Qemu-devel] [PATCH v6 3/8] target-arm: move gicv3_class_name from machine to kvm_arm.h

[Eric Auger]

Machine.c contains code related to migration. Let's move
gicv3_class_name to kvm_arm.h instead.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

---

v5 -> v6:
- add Peter's R-b

v4 -> v5:
- add #include "qemu/error-report.h"
- rebased on target-arm: Fix unreachable code in gicv3_class_name()

v4: creation

Conflicts:
	target-arm/machine.c
---
 target-arm/kvm_arm.h | 16 +++++++++++++++-
 target-arm/machine.c | 15 ---------------
 2 files changed, 15 insertions(+), 16 deletions(-)

diff --git a/target-arm/kvm_arm.h b/target-arm/kvm_arm.h
index 544e404..633d088 100644
--- a/target-arm/kvm_arm.h
+++ b/target-arm/kvm_arm.h
@@ -13,6 +13,7 @@
 
 #include "sysemu/kvm.h"
 #include "exec/memory.h"
+#include "qemu/error-report.h"
 
 /**
  * kvm_arm_vcpu_init:
@@ -223,7 +224,20 @@ static inline const char *gic_class_name(void)
  *
  * Returns: class name to use
  */
-const char *gicv3_class_name(void);
+static inline const char *gicv3_class_name(void)
+{
+    if (kvm_irqchip_in_kernel()) {
+#ifdef TARGET_AARCH64
+        return "kvm-arm-gicv3";
+#else
+        error_report("KVM GICv3 acceleration is not supported on this "
+                     "platform");
+        exit(1);
+#endif
+    } else {
+        return "arm-gicv3";
+    }
+}
 
 /**
  * kvm_arm_handle_debug:
diff --git a/target-arm/machine.c b/target-arm/machine.c
index 7a6ca31..d90943b 100644
--- a/target-arm/machine.c
+++ b/target-arm/machine.c
@@ -331,18 +331,3 @@ const VMStateDescription vmstate_arm_cpu = {
         NULL
     }
 };
-
-const char *gicv3_class_name(void)
-{
-    if (kvm_irqchip_in_kernel()) {
-#ifdef TARGET_AARCH64
-        return "kvm-arm-gicv3";
-#else
-        error_report("KVM GICv3 acceleration is not supported on this "
-                     "platform");
-        exit(1);
-#endif
-    } else {
-        return "arm-gicv3";
-    }
-}
-- 
2.5.5

[Qemu-devel] [PATCH v6 4/8] kvm-all: Pass requester ID to MSI routing functions

[Eric Auger]

From: Pavel Fedin <p.fedin@samsung.com>

Introduce global kvm_msi_use_devid flag and pass the device ID,
if needed, while building the MSI route entry. Device IDs are
required by the ARM GICv3 ITS (IRQ remapping function is based on
this information).

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v5 -> v6:
- move the flag in include/sysemu/kvm.h and populate the devid
  information in kvm-all.c

v3 -> v4:
- OR route->flags with KVM_MSI_VALID_DEVID
---
 include/sysemu/kvm.h | 1 +
 kvm-all.c            | 9 +++++++++
 kvm-stub.c           | 1 +
 3 files changed, 11 insertions(+)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index c9c2436..c31c810 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -53,6 +53,7 @@ extern bool kvm_gsi_direct_mapping;
 extern bool kvm_readonly_mem_allowed;
 extern bool kvm_direct_msi_allowed;
 extern bool kvm_ioeventfd_any_length_allowed;
+extern bool kvm_msi_use_devid;
 
 #if defined CONFIG_KVM || !defined NEED_CPU_H
 #define kvm_enabled()           (kvm_allowed)
diff --git a/kvm-all.c b/kvm-all.c
index ebf35b0..df3bcde 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -119,6 +119,7 @@ bool kvm_readonly_mem_allowed;
 bool kvm_vm_attributes_allowed;
 bool kvm_direct_msi_allowed;
 bool kvm_ioeventfd_any_length_allowed;
+bool kvm_msi_use_devid;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
     KVM_CAP_INFO(USER_MEMORY),
@@ -1275,6 +1276,10 @@ int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
     kroute.u.msi.address_lo = (uint32_t)msg.address;
     kroute.u.msi.address_hi = msg.address >> 32;
     kroute.u.msi.data = le32_to_cpu(msg.data);
+    if (kvm_msi_use_devid) {
+        kroute.flags = KVM_MSI_VALID_DEVID;
+        kroute.u.msi.devid = pci_requester_id(dev);
+    }
     if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
         kvm_irqchip_release_virq(s, virq);
         return -EINVAL;
@@ -1308,6 +1313,10 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
     kroute.u.msi.address_lo = (uint32_t)msg.address;
     kroute.u.msi.address_hi = msg.address >> 32;
     kroute.u.msi.data = le32_to_cpu(msg.data);
+    if (kvm_msi_use_devid) {
+        kroute.flags = KVM_MSI_VALID_DEVID;
+        kroute.u.msi.devid = pci_requester_id(dev);
+    }
     if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
         return -EINVAL;
     }
diff --git a/kvm-stub.c b/kvm-stub.c
index 64e23f6..6f896e9 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -31,6 +31,7 @@ bool kvm_gsi_direct_mapping;
 bool kvm_allowed;
 bool kvm_readonly_mem_allowed;
 bool kvm_ioeventfd_any_length_allowed;
+bool kvm_msi_use_devid;
 
 int kvm_destroy_vcpu(CPUState *cpu)
 {
-- 
2.5.5

[Qemu-devel] [PATCH v6 5/8] hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation

[Eric Auger]

From: Pavel Fedin <p.fedin@samsung.com>

The ITS control frame is in-kernel emulated while accesses to the
GITS_TRANSLATER are mediated through the KVM_SIGNAL_MSI ioctl (MSI
direct MSI injection advertised by the CAP_SIGNAL_MSI capability)

the kvm_gsi_direct_mapping is explicitly set to false to emphasize the
difference with GICv2M. Direct mapping cannot work with ITS since
the content of the MSI data is not the target interrupt ID but an
eventd id.

GSI routing is advertised (kvm_gsi_routing_allowed) as well as
msi/irqfd signaling (kvm_msi_via_irqfd_allowed).

A machine init done notifier still is used to compute the MSI frame
(GITS_TRANSLATER) GPA, which is passed through KVM_SIGNAL_MSI ioctl.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>

---
v5 -> v6:
- now the kernel ITS init sequence has changed let's use the
  kvm_arm_register_device to set the CTRL frame base address
- use extract64()
- add migration blocker
- remove kvm_gsi_routing_allowed setting which is set in GICv3
  KVM device realize
- add migration_blocker

v3 -> v4:
- include "qemu/osdep.h" and  "qapi/error.h"
- rename KVM_VGIC_V3_ADDR_TYPE_ITS into KVM_VGIC_ITS_ADDR_TYPE
- reword commit message
- change kvm_msi_via_irqfd_allowed definition (attached to irqfd dynamic
  availability + MSI controller availability)
- create the ITS KVM device (previously abstracted by the GICv3 KVM device)
- init sequence changed
- absolute GITS_TRANSLATER GPA stored

Signed-off-by: Eric Auger <eric.auger@redhat.com>
---
 hw/intc/Makefile.objs       |   1 +
 hw/intc/arm_gicv3_its_kvm.c | 136 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 137 insertions(+)
 create mode 100644 hw/intc/arm_gicv3_its_kvm.c

diff --git a/hw/intc/Makefile.objs b/hw/intc/Makefile.objs
index 23a39f7..9cca280 100644
--- a/hw/intc/Makefile.objs
+++ b/hw/intc/Makefile.objs
@@ -22,6 +22,7 @@ common-obj-$(CONFIG_OPENPIC) += openpic.o
 obj-$(CONFIG_APIC) += apic.o apic_common.o
 obj-$(CONFIG_ARM_GIC_KVM) += arm_gic_kvm.o
 obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_kvm.o
+obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_its_kvm.o
 obj-$(CONFIG_STELLARIS) += armv7m_nvic.o
 obj-$(CONFIG_EXYNOS4) += exynos4210_gic.o exynos4210_combiner.o
 obj-$(CONFIG_GRLIB) += grlib_irqmp.o
diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
new file mode 100644
index 0000000..b9d10dc
--- /dev/null
+++ b/hw/intc/arm_gicv3_its_kvm.c
@@ -0,0 +1,136 @@
+/*
+ * KVM-based ITS implementation for a GICv3-based system
+ *
+ * Copyright (c) 2015 Samsung Electronics Co., Ltd.
+ * Written by Pavel Fedin <p.fedin@samsung.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/intc/arm_gicv3_its_common.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/kvm.h"
+#include "kvm_arm.h"
+#include "migration/migration.h"
+
+#define TYPE_KVM_ARM_ITS "arm-its-kvm"
+#define KVM_ARM_ITS(obj) OBJECT_CHECK(GICv3ITSState, (obj), TYPE_KVM_ARM_ITS)
+
+static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
+{
+    struct kvm_msi msi;
+
+    msi.address_lo = extract64(s->gits_translater_gpa, 0, 32);
+    msi.address_hi = extract64(s->gits_translater_gpa, 32, 32);
+    msi.data = le32_to_cpu(value);
+    msi.flags = KVM_MSI_VALID_DEVID;
+    msi.devid = devid;
+    memset(msi.pad, 0, sizeof(msi.pad));
+
+    return kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi);
+}
+
+typedef struct ItsInitNotifierParams {
+    Notifier notifier;
+    GICv3ITSState *s;
+} ItsInitNotifierParams;
+
+/* this notifier computes and stores the ITS MSI frame GPA */
+static void its_notify(Notifier *notifier, void *data)
+{
+    ItsInitNotifierParams *p = DO_UPCAST(ItsInitNotifierParams,
+                                         notifier, notifier);
+    GICv3ITSState *s = p->s;
+    MemoryRegion *mr = &s->iomem_its_translation;
+    MemoryRegionSection mrs;
+    uint64_t addr;
+
+    mrs = memory_region_find(mr, 0, 1);
+    addr = mrs.offset_within_address_space;
+    s->gits_translater_gpa = addr + 0x40;
+}
+
+static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
+{
+    GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
+    ItsInitNotifierParams *p = g_new(ItsInitNotifierParams, 1);
+
+    s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
+    if (s->dev_fd < 0) {
+        error_setg_errno(errp, -s->dev_fd, "error creating in-kernel ITS");
+        return;
+    }
+
+    /* explicit init if the ITS */
+    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
+                      KVM_DEV_ARM_VGIC_CTRL_INIT, NULL, true);
+
+    /* register the base address */
+    kvm_arm_register_device(&s->iomem_its_cntrl, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
+                            KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd);
+
+    gicv3_its_init_mmio(s, NULL);
+
+    p->notifier.notify = its_notify;
+    p->s = s;
+    qemu_add_machine_init_done_notifier(&p->notifier);
+
+    /*
+     * Block migration of a KVM GICv3 ITS device: the API for saving and
+     * restoring the state in the kernel is not yet available
+     */
+    error_setg(&s->migration_blocker, "vITS migration is not implemented");
+    migrate_add_blocker(s->migration_blocker);
+
+    kvm_msi_use_devid = true;
+    kvm_gsi_direct_mapping = false;
+    kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
+}
+
+static void kvm_arm_its_init(Object *obj)
+{
+    GICv3ITSState *s = KVM_ARM_ITS(obj);
+
+    object_property_add_link(obj, "parent-gicv3",
+                             "kvm-arm-gicv3", (Object **)&s->gicv3,
+                             object_property_allow_set_link,
+                             OBJ_PROP_LINK_UNREF_ON_RELEASE,
+                             &error_abort);
+}
+
+static void kvm_arm_its_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    GICv3ITSCommonClass *icc = ARM_GICV3_ITS_COMMON_CLASS(klass);
+
+    dc->realize = kvm_arm_its_realize;
+    icc->send_msi = kvm_its_send_msi;
+}
+
+static const TypeInfo kvm_arm_its_info = {
+    .name = TYPE_KVM_ARM_ITS,
+    .parent = TYPE_ARM_GICV3_ITS_COMMON,
+    .instance_size = sizeof(GICv3ITSState),
+    .instance_init = kvm_arm_its_init,
+    .class_init = kvm_arm_its_class_init,
+};
+
+static void kvm_arm_its_register_types(void)
+{
+    type_register_static(&kvm_arm_its_info);
+}
+
+type_init(kvm_arm_its_register_types)
-- 
2.5.5

[Qemu-devel] [PATCH v6 6/8] arm/virt: Add ITS to the virt board

[Eric Auger]

From: Pavel Fedin <p.fedin@samsung.com>

If supported by the configuration, ITS will be added automatically.

This patch also renames v2m_phandle to msi_phandle because it's now used
by both MSI implementations.

Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

--

v3 -> v4:
- added Peter's R-b
---
 hw/arm/virt.c | 47 +++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 41 insertions(+), 6 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index a193b5a..c5f65dd 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -76,7 +76,7 @@ typedef struct VirtBoardInfo {
     int fdt_size;
     uint32_t clock_phandle;
     uint32_t gic_phandle;
-    uint32_t v2m_phandle;
+    uint32_t msi_phandle;
     bool using_psci;
 } VirtBoardInfo;
 
@@ -423,9 +423,22 @@ static void fdt_add_cpu_nodes(const VirtBoardInfo *vbi)
     }
 }
 
+static void fdt_add_its_gic_node(VirtBoardInfo *vbi)
+{
+    vbi->msi_phandle = qemu_fdt_alloc_phandle(vbi->fdt);
+    qemu_fdt_add_subnode(vbi->fdt, "/intc/its");
+    qemu_fdt_setprop_string(vbi->fdt, "/intc/its", "compatible",
+                            "arm,gic-v3-its");
+    qemu_fdt_setprop(vbi->fdt, "/intc/its", "msi-controller", NULL, 0);
+    qemu_fdt_setprop_sized_cells(vbi->fdt, "/intc/its", "reg",
+                                 2, vbi->memmap[VIRT_GIC_ITS].base,
+                                 2, vbi->memmap[VIRT_GIC_ITS].size);
+    qemu_fdt_setprop_cell(vbi->fdt, "/intc/its", "phandle", vbi->msi_phandle);
+}
+
 static void fdt_add_v2m_gic_node(VirtBoardInfo *vbi)
 {
-    vbi->v2m_phandle = qemu_fdt_alloc_phandle(vbi->fdt);
+    vbi->msi_phandle = qemu_fdt_alloc_phandle(vbi->fdt);
     qemu_fdt_add_subnode(vbi->fdt, "/intc/v2m");
     qemu_fdt_setprop_string(vbi->fdt, "/intc/v2m", "compatible",
                             "arm,gic-v2m-frame");
@@ -433,7 +446,7 @@ static void fdt_add_v2m_gic_node(VirtBoardInfo *vbi)
     qemu_fdt_setprop_sized_cells(vbi->fdt, "/intc/v2m", "reg",
                                  2, vbi->memmap[VIRT_GIC_V2M].base,
                                  2, vbi->memmap[VIRT_GIC_V2M].size);
-    qemu_fdt_setprop_cell(vbi->fdt, "/intc/v2m", "phandle", vbi->v2m_phandle);
+    qemu_fdt_setprop_cell(vbi->fdt, "/intc/v2m", "phandle", vbi->msi_phandle);
 }
 
 static void fdt_add_gic_node(VirtBoardInfo *vbi, int type)
@@ -500,6 +513,26 @@ static void fdt_add_pmu_nodes(const VirtBoardInfo *vbi, int gictype)
     }
 }
 
+static void create_its(VirtBoardInfo *vbi, DeviceState *gicdev)
+{
+    const char *itsclass = its_class_name();
+    DeviceState *dev;
+
+    if (!itsclass) {
+        /* Do nothing if not supported */
+        return;
+    }
+
+    dev = qdev_create(NULL, itsclass);
+
+    object_property_set_link(OBJECT(dev), OBJECT(gicdev), "parent-gicv3",
+                             &error_abort);
+    qdev_init_nofail(dev);
+    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, vbi->memmap[VIRT_GIC_ITS].base);
+
+    fdt_add_its_gic_node(vbi);
+}
+
 static void create_v2m(VirtBoardInfo *vbi, qemu_irq *pic)
 {
     int i;
@@ -583,7 +616,9 @@ static void create_gic(VirtBoardInfo *vbi, qemu_irq *pic, int type, bool secure)
 
     fdt_add_gic_node(vbi, type);
 
-    if (type == 2) {
+    if (type == 3) {
+        create_its(vbi, gicdev);
+    } else {
         create_v2m(vbi, pic);
     }
 }
@@ -1025,9 +1060,9 @@ static void create_pcie(const VirtBoardInfo *vbi, qemu_irq *pic,
                            nr_pcie_buses - 1);
     qemu_fdt_setprop(vbi->fdt, nodename, "dma-coherent", NULL, 0);
 
-    if (vbi->v2m_phandle) {
+    if (vbi->msi_phandle) {
         qemu_fdt_setprop_cells(vbi->fdt, nodename, "msi-parent",
-                               vbi->v2m_phandle);
+                               vbi->msi_phandle);
     }
 
     qemu_fdt_setprop_sized_cells(vbi->fdt, nodename, "reg",
-- 
2.5.5

[Qemu-devel] [PATCH v6 7/8] ACPI: Add GIC Interrupt Translation Service Structure definition

[Eric Auger]

From: Shannon Zhao <shannon.zhao@linaro.org>

ACPI Spec 6.0 introduces GIC Interrupt Translation Service Structure.
Here we add the definition of the Structure.

Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
---
 include/hw/acpi/acpi-defs.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index 41c1d95..9c1b7cb 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -294,7 +294,8 @@ typedef struct AcpiMultipleApicTable AcpiMultipleApicTable;
 #define ACPI_APIC_GENERIC_DISTRIBUTOR   12
 #define ACPI_APIC_GENERIC_MSI_FRAME     13
 #define ACPI_APIC_GENERIC_REDISTRIBUTOR 14
-#define ACPI_APIC_RESERVED              15   /* 15 and greater are reserved */
+#define ACPI_APIC_GENERIC_TRANSLATOR    15
+#define ACPI_APIC_RESERVED              16   /* 16 and greater are reserved */
 
 /*
  * MADT sub-structures (Follow MULTIPLE_APIC_DESCRIPTION_TABLE)
@@ -395,6 +396,16 @@ struct AcpiMadtGenericRedistributor {
 
 typedef struct AcpiMadtGenericRedistributor AcpiMadtGenericRedistributor;
 
+struct AcpiMadtGenericTranslator {
+    ACPI_SUB_HEADER_DEF
+    uint16_t reserved;
+    uint32_t translation_id;
+    uint64_t base_address;
+    uint32_t reserved2;
+} QEMU_PACKED;
+
+typedef struct AcpiMadtGenericTranslator AcpiMadtGenericTranslator;
+
 /*
  * Generic Timer Description Table (GTDT)
  */
-- 
2.5.5

[Qemu-devel] [PATCH v6 8/8] ARM: Virt: ACPI: Add GIC ITS description in ACPI MADT table

[Eric Auger]

From: Shannon Zhao <shannon.zhao@linaro.org>

If GIC ITS is supported, add description in ACPI MADT table, then guest
could use ITS when booting with ACPI.

Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org>
---
 hw/arm/virt-acpi-build.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 28fc59c..42608c8 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -44,6 +44,7 @@
 #include "hw/pci/pcie_host.h"
 #include "hw/pci/pci.h"
 #include "sysemu/numa.h"
+#include "kvm_arm.h"
 
 #define ARM_SPI_BASE 32
 #define ACPI_POWER_BUTTON_DEVICE "PWRB"
@@ -546,6 +547,7 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtGuestInfo *guest_info)
     }
 
     if (guest_info->gic_version == 3) {
+        AcpiMadtGenericTranslator *gic_its;
         AcpiMadtGenericRedistributor *gicr = acpi_data_push(table_data,
                                                          sizeof *gicr);
 
@@ -553,6 +555,16 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtGuestInfo *guest_info)
         gicr->length = sizeof(*gicr);
         gicr->base_address = cpu_to_le64(memmap[VIRT_GIC_REDIST].base);
         gicr->range_length = cpu_to_le32(memmap[VIRT_GIC_REDIST].size);
+
+        if (!its_class_name()) {
+            return;
+        }
+
+        gic_its = acpi_data_push(table_data, sizeof *gic_its);
+        gic_its->type = ACPI_APIC_GENERIC_TRANSLATOR;
+        gic_its->length = sizeof(*gic_its);
+        gic_its->translation_id = 0;
+        gic_its->base_address = cpu_to_le64(memmap[VIRT_GIC_ITS].base);
     } else {
         gic_msi = acpi_data_push(table_data, sizeof *gic_msi);
         gic_msi->type = ACPI_APIC_GENERIC_MSI_FRAME;
-- 
2.5.5

Re: [Qemu-devel] [PATCH v6 0/8] vITS support

[Peter Maydell]

On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
> This series introduces support for in-kernel GICv3 ITS emulation.
>
> On dt guest the functionality is complete and was tested on Cavium ThunderX
> with virtio-net-pci and vhost-net.
>
> On ACPI guest the series was tested with virtio-net-pci only. For vhost-net,
> using MSIX we currently miss the ACPI IORT table generation linking the
> PCIe host controller with the ITS. The work is ongoing and will be submitted
> separately. Anyway the kernel ACPI IORT ITS node support is not upstreamed
> yet.
>
> The first patch is not really related to virtual ITS but advertises the KVM
> GSI routing support which goes along with MSI injection.

Patches 7 and 8 are missing your signed-off-by line...

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 1/8] hw/intc/arm_gic(v3)_kvm: Initialize gsi routing

[Peter Maydell]

On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
> From: Eric Auger <eric.auger@linaro.org>
>
> Advertise gsi routing and set up irqchip routing entries for
> GIC SPIs.
>
> This is not mandated as long as MSI routing is not used
> (because the kernel sets a default irqchip routing table).
> However once MSI routing gets used (for VIRTIO-PCI vhost for
> example), the first call to KVM_SET_GSI_ROUTING overrides the
> kernel default irqchip table.
>
> If no routing entry exists for the GSI, any IRQFD signaling for
> this GSI will fail.
>
> Signed-off-by: Eric Auger <eric.auger@linaro.org>

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 2/8] hw/intc/arm_gicv3_its: Implement ITS base class

[Peter Maydell]

On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
> From: Pavel Fedin <p.fedin@samsung.com>
>
> This is the basic skeleton for both KVM and software-emulated ITS.
> Since we already prepare status structure, we also introduce complete
> VMState description. But, because we currently have no migratable
> implementations, we also set unmigratable flag.
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
> v5 -> v6:
> - rename iomem_its into iomem_its_translation
> - remove vmstate_its fields since the device is not yet
>   migratable
> - fix gicv3_its_trans_write print format
> - add migration_blocker
>
> v4 -> v5:
> - send_msi can return 0 meaning the MSI was rejected by guest. Handle this
>   value as an error
>
> v3 -> v4:
> - fix compilation error
> - msi_supported -> msi_nonbroken
> - streamid -> requester_id
> - use PRIx64
> - read ops uses _with_attrs form
> - 16/32b write access allowed to GITS_TRANSLATER
> - move its_class_name in kvm_arm.h and remove #ifdef TARGET_AARCH64
> - add new kvm device dev_fd field
> - add new gits_translater_gpa field in GICv3ITSState
> ---
>  hw/intc/Makefile.objs                  |   1 +
>  hw/intc/arm_gicv3_its_common.c         | 148 +++++++++++++++++++++++++++++++++
>  include/hw/intc/arm_gicv3_its_common.h |  77 +++++++++++++++++
>  target-arm/kvm_arm.h                   |  19 +++++
>  4 files changed, 245 insertions(+)
>  create mode 100644 hw/intc/arm_gicv3_its_common.c
>  create mode 100644 include/hw/intc/arm_gicv3_its_common.h

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 4/8] kvm-all: Pass requester ID to MSI routing functions

[Peter Maydell]

On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
> From: Pavel Fedin <p.fedin@samsung.com>
>
> Introduce global kvm_msi_use_devid flag and pass the device ID,
> if needed, while building the MSI route entry. Device IDs are
> required by the ARM GICv3 ITS (IRQ remapping function is based on
> this information).
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
> v5 -> v6:
> - move the flag in include/sysemu/kvm.h and populate the devid
>   information in kvm-all.c
>
> v3 -> v4:
> - OR route->flags with KVM_MSI_VALID_DEVID
> ---
>  include/sysemu/kvm.h | 1 +
>  kvm-all.c            | 9 +++++++++
>  kvm-stub.c           | 1 +
>  3 files changed, 11 insertions(+)
>
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index c9c2436..c31c810 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -53,6 +53,7 @@ extern bool kvm_gsi_direct_mapping;
>  extern bool kvm_readonly_mem_allowed;
>  extern bool kvm_direct_msi_allowed;
>  extern bool kvm_ioeventfd_any_length_allowed;
> +extern bool kvm_msi_use_devid;
>
>  #if defined CONFIG_KVM || !defined NEED_CPU_H
>  #define kvm_enabled()           (kvm_allowed)

You also need the appropriate #define, like all the other flags
here (with a doc comment, and with the version that's just defined
as '(false)').

The idea is that tests of the condition then don't need to
be guarded by ifdef CONFIG_KVM.

> diff --git a/kvm-all.c b/kvm-all.c
> index ebf35b0..df3bcde 100644
> --- a/kvm-all.c
> +++ b/kvm-all.c
> @@ -119,6 +119,7 @@ bool kvm_readonly_mem_allowed;
>  bool kvm_vm_attributes_allowed;
>  bool kvm_direct_msi_allowed;
>  bool kvm_ioeventfd_any_length_allowed;
> +bool kvm_msi_use_devid;
>
>  static const KVMCapabilityInfo kvm_required_capabilites[] = {
>      KVM_CAP_INFO(USER_MEMORY),
> @@ -1275,6 +1276,10 @@ int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
>      kroute.u.msi.address_lo = (uint32_t)msg.address;
>      kroute.u.msi.address_hi = msg.address >> 32;
>      kroute.u.msi.data = le32_to_cpu(msg.data);
> +    if (kvm_msi_use_devid) {

these should use the macro, not direct access to the flag variable.

> +        kroute.flags = KVM_MSI_VALID_DEVID;
> +        kroute.u.msi.devid = pci_requester_id(dev);
> +    }
>      if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
>          kvm_irqchip_release_virq(s, virq);
>          return -EINVAL;
> @@ -1308,6 +1313,10 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
>      kroute.u.msi.address_lo = (uint32_t)msg.address;
>      kroute.u.msi.address_hi = msg.address >> 32;
>      kroute.u.msi.data = le32_to_cpu(msg.data);
> +    if (kvm_msi_use_devid) {
> +        kroute.flags = KVM_MSI_VALID_DEVID;
> +        kroute.u.msi.devid = pci_requester_id(dev);
> +    }
>      if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
>          return -EINVAL;
>      }
> diff --git a/kvm-stub.c b/kvm-stub.c
> index 64e23f6..6f896e9 100644
> --- a/kvm-stub.c
> +++ b/kvm-stub.c
> @@ -31,6 +31,7 @@ bool kvm_gsi_direct_mapping;
>  bool kvm_allowed;
>  bool kvm_readonly_mem_allowed;
>  bool kvm_ioeventfd_any_length_allowed;
> +bool kvm_msi_use_devid;
>
>  int kvm_destroy_vcpu(CPUState *cpu)
>  {

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 5/8] hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation

[Peter Maydell]

On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
> From: Pavel Fedin <p.fedin@samsung.com>
>
> The ITS control frame is in-kernel emulated while accesses to the
> GITS_TRANSLATER are mediated through the KVM_SIGNAL_MSI ioctl (MSI
> direct MSI injection advertised by the CAP_SIGNAL_MSI capability)
>
> the kvm_gsi_direct_mapping is explicitly set to false to emphasize the
> difference with GICv2M. Direct mapping cannot work with ITS since
> the content of the MSI data is not the target interrupt ID but an
> eventd id.
>
> GSI routing is advertised (kvm_gsi_routing_allowed) as well as
> msi/irqfd signaling (kvm_msi_via_irqfd_allowed).
>
> A machine init done notifier still is used to compute the MSI frame
> (GITS_TRANSLATER) GPA, which is passed through KVM_SIGNAL_MSI ioctl.
>
> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
> v5 -> v6:
> - now the kernel ITS init sequence has changed let's use the
>   kvm_arm_register_device to set the CTRL frame base address
> - use extract64()
> - add migration blocker
> - remove kvm_gsi_routing_allowed setting which is set in GICv3
>   KVM device realize
> - add migration_blocker
>
> v3 -> v4:
> - include "qemu/osdep.h" and  "qapi/error.h"
> - rename KVM_VGIC_V3_ADDR_TYPE_ITS into KVM_VGIC_ITS_ADDR_TYPE
> - reword commit message
> - change kvm_msi_via_irqfd_allowed definition (attached to irqfd dynamic
>   availability + MSI controller availability)
> - create the ITS KVM device (previously abstracted by the GICv3 KVM device)
> - init sequence changed
> - absolute GITS_TRANSLATER GPA stored
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> ---
>  hw/intc/Makefile.objs       |   1 +
>  hw/intc/arm_gicv3_its_kvm.c | 136 ++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 137 insertions(+)
>  create mode 100644 hw/intc/arm_gicv3_its_kvm.c
>
> diff --git a/hw/intc/Makefile.objs b/hw/intc/Makefile.objs
> index 23a39f7..9cca280 100644
> --- a/hw/intc/Makefile.objs
> +++ b/hw/intc/Makefile.objs
> @@ -22,6 +22,7 @@ common-obj-$(CONFIG_OPENPIC) += openpic.o
>  obj-$(CONFIG_APIC) += apic.o apic_common.o
>  obj-$(CONFIG_ARM_GIC_KVM) += arm_gic_kvm.o
>  obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_kvm.o
> +obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_its_kvm.o
>  obj-$(CONFIG_STELLARIS) += armv7m_nvic.o
>  obj-$(CONFIG_EXYNOS4) += exynos4210_gic.o exynos4210_combiner.o
>  obj-$(CONFIG_GRLIB) += grlib_irqmp.o
> diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
> new file mode 100644
> index 0000000..b9d10dc
> --- /dev/null
> +++ b/hw/intc/arm_gicv3_its_kvm.c
> @@ -0,0 +1,136 @@
> +/*
> + * KVM-based ITS implementation for a GICv3-based system
> + *
> + * Copyright (c) 2015 Samsung Electronics Co., Ltd.
> + * Written by Pavel Fedin <p.fedin@samsung.com>
> + *
> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "hw/intc/arm_gicv3_its_common.h"
> +#include "sysemu/sysemu.h"
> +#include "sysemu/kvm.h"
> +#include "kvm_arm.h"
> +#include "migration/migration.h"
> +
> +#define TYPE_KVM_ARM_ITS "arm-its-kvm"
> +#define KVM_ARM_ITS(obj) OBJECT_CHECK(GICv3ITSState, (obj), TYPE_KVM_ARM_ITS)
> +
> +static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
> +{
> +    struct kvm_msi msi;
> +
> +    msi.address_lo = extract64(s->gits_translater_gpa, 0, 32);
> +    msi.address_hi = extract64(s->gits_translater_gpa, 32, 32);
> +    msi.data = le32_to_cpu(value);
> +    msi.flags = KVM_MSI_VALID_DEVID;
> +    msi.devid = devid;
> +    memset(msi.pad, 0, sizeof(msi.pad));
> +
> +    return kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi);
> +}
> +
> +typedef struct ItsInitNotifierParams {
> +    Notifier notifier;
> +    GICv3ITSState *s;
> +} ItsInitNotifierParams;
> +
> +/* this notifier computes and stores the ITS MSI frame GPA */
> +static void its_notify(Notifier *notifier, void *data)
> +{
> +    ItsInitNotifierParams *p = DO_UPCAST(ItsInitNotifierParams,
> +                                         notifier, notifier);
> +    GICv3ITSState *s = p->s;
> +    MemoryRegion *mr = &s->iomem_its_translation;
> +    MemoryRegionSection mrs;
> +    uint64_t addr;
> +
> +    mrs = memory_region_find(mr, 0, 1);
> +    addr = mrs.offset_within_address_space;
> +    s->gits_translater_gpa = addr + 0x40;

This leaks a memory region reference.

Also, I think it would be simpler to just have kvm_its_send_msi() do

   if (unlikely(!s->translater_gpa_known)) {
       s->gits_translater_gpa = ...;
       s->translater_gpa_known = true;
   }

rather than messing around with a machine-init-done notifier.

> +}
> +
> +static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
> +{
> +    GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
> +    ItsInitNotifierParams *p = g_new(ItsInitNotifierParams, 1);
> +
> +    s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
> +    if (s->dev_fd < 0) {
> +        error_setg_errno(errp, -s->dev_fd, "error creating in-kernel ITS");
> +        return;
> +    }
> +
> +    /* explicit init if the ITS */

"of the"

> +    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
> +                      KVM_DEV_ARM_VGIC_CTRL_INIT, NULL, true);
> +
> +    /* register the base address */
> +    kvm_arm_register_device(&s->iomem_its_cntrl, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
> +                            KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd);
> +
> +    gicv3_its_init_mmio(s, NULL);
> +
> +    p->notifier.notify = its_notify;
> +    p->s = s;
> +    qemu_add_machine_init_done_notifier(&p->notifier);
> +
> +    /*
> +     * Block migration of a KVM GICv3 ITS device: the API for saving and
> +     * restoring the state in the kernel is not yet available
> +     */
> +    error_setg(&s->migration_blocker, "vITS migration is not implemented");
> +    migrate_add_blocker(s->migration_blocker);
> +
> +    kvm_msi_use_devid = true;
> +    kvm_gsi_direct_mapping = false;
> +    kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
> +}
> +
> +static void kvm_arm_its_init(Object *obj)
> +{
> +    GICv3ITSState *s = KVM_ARM_ITS(obj);
> +
> +    object_property_add_link(obj, "parent-gicv3",
> +                             "kvm-arm-gicv3", (Object **)&s->gicv3,
> +                             object_property_allow_set_link,
> +                             OBJ_PROP_LINK_UNREF_ON_RELEASE,
> +                             &error_abort);
> +}
> +
> +static void kvm_arm_its_class_init(ObjectClass *klass, void *data)
> +{
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +    GICv3ITSCommonClass *icc = ARM_GICV3_ITS_COMMON_CLASS(klass);
> +
> +    dc->realize = kvm_arm_its_realize;
> +    icc->send_msi = kvm_its_send_msi;
> +}
> +
> +static const TypeInfo kvm_arm_its_info = {
> +    .name = TYPE_KVM_ARM_ITS,
> +    .parent = TYPE_ARM_GICV3_ITS_COMMON,
> +    .instance_size = sizeof(GICv3ITSState),
> +    .instance_init = kvm_arm_its_init,
> +    .class_init = kvm_arm_its_class_init,
> +};
> +
> +static void kvm_arm_its_register_types(void)
> +{
> +    type_register_static(&kvm_arm_its_info);
> +}
> +
> +type_init(kvm_arm_its_register_types)
> --
> 2.5.5

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 0/8] vITS support

[Auger Eric]

Hi Peter,

On 22/09/2016 16:08, Peter Maydell wrote:
> On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
>> This series introduces support for in-kernel GICv3 ITS emulation.
>>
>> On dt guest the functionality is complete and was tested on Cavium ThunderX
>> with virtio-net-pci and vhost-net.
>>
>> On ACPI guest the series was tested with virtio-net-pci only. For vhost-net,
>> using MSIX we currently miss the ACPI IORT table generation linking the
>> PCIe host controller with the ITS. The work is ongoing and will be submitted
>> separately. Anyway the kernel ACPI IORT ITS node support is not upstreamed
>> yet.
>>
>> The first patch is not really related to virtual ITS but advertises the KVM
>> GSI routing support which goes along with MSI injection.
> 
> Patches 7 and 8 are missing your signed-off-by line...

Sure I will add them on top of Shannon's ones. Most credit must be given
to him though.

Thanks

Eric
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] [PATCH v6 5/8] hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation

[Auger Eric]

Hi Peter,

On 22/09/2016 17:03, Peter Maydell wrote:
> On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
>> From: Pavel Fedin <p.fedin@samsung.com>
>>
>> The ITS control frame is in-kernel emulated while accesses to the
>> GITS_TRANSLATER are mediated through the KVM_SIGNAL_MSI ioctl (MSI
>> direct MSI injection advertised by the CAP_SIGNAL_MSI capability)
>>
>> the kvm_gsi_direct_mapping is explicitly set to false to emphasize the
>> difference with GICv2M. Direct mapping cannot work with ITS since
>> the content of the MSI data is not the target interrupt ID but an
>> eventd id.
>>
>> GSI routing is advertised (kvm_gsi_routing_allowed) as well as
>> msi/irqfd signaling (kvm_msi_via_irqfd_allowed).
>>
>> A machine init done notifier still is used to compute the MSI frame
>> (GITS_TRANSLATER) GPA, which is passed through KVM_SIGNAL_MSI ioctl.
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>> v5 -> v6:
>> - now the kernel ITS init sequence has changed let's use the
>>   kvm_arm_register_device to set the CTRL frame base address
>> - use extract64()
>> - add migration blocker
>> - remove kvm_gsi_routing_allowed setting which is set in GICv3
>>   KVM device realize
>> - add migration_blocker
>>
>> v3 -> v4:
>> - include "qemu/osdep.h" and  "qapi/error.h"
>> - rename KVM_VGIC_V3_ADDR_TYPE_ITS into KVM_VGIC_ITS_ADDR_TYPE
>> - reword commit message
>> - change kvm_msi_via_irqfd_allowed definition (attached to irqfd dynamic
>>   availability + MSI controller availability)
>> - create the ITS KVM device (previously abstracted by the GICv3 KVM device)
>> - init sequence changed
>> - absolute GITS_TRANSLATER GPA stored
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> ---
>>  hw/intc/Makefile.objs       |   1 +
>>  hw/intc/arm_gicv3_its_kvm.c | 136 ++++++++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 137 insertions(+)
>>  create mode 100644 hw/intc/arm_gicv3_its_kvm.c
>>
>> diff --git a/hw/intc/Makefile.objs b/hw/intc/Makefile.objs
>> index 23a39f7..9cca280 100644
>> --- a/hw/intc/Makefile.objs
>> +++ b/hw/intc/Makefile.objs
>> @@ -22,6 +22,7 @@ common-obj-$(CONFIG_OPENPIC) += openpic.o
>>  obj-$(CONFIG_APIC) += apic.o apic_common.o
>>  obj-$(CONFIG_ARM_GIC_KVM) += arm_gic_kvm.o
>>  obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_kvm.o
>> +obj-$(call land,$(CONFIG_ARM_GIC_KVM),$(TARGET_AARCH64)) += arm_gicv3_its_kvm.o
>>  obj-$(CONFIG_STELLARIS) += armv7m_nvic.o
>>  obj-$(CONFIG_EXYNOS4) += exynos4210_gic.o exynos4210_combiner.o
>>  obj-$(CONFIG_GRLIB) += grlib_irqmp.o
>> diff --git a/hw/intc/arm_gicv3_its_kvm.c b/hw/intc/arm_gicv3_its_kvm.c
>> new file mode 100644
>> index 0000000..b9d10dc
>> --- /dev/null
>> +++ b/hw/intc/arm_gicv3_its_kvm.c
>> @@ -0,0 +1,136 @@
>> +/*
>> + * KVM-based ITS implementation for a GICv3-based system
>> + *
>> + * Copyright (c) 2015 Samsung Electronics Co., Ltd.
>> + * Written by Pavel Fedin <p.fedin@samsung.com>
>> + *
>> + * This library is free software; you can redistribute it and/or
>> + * modify it under the terms of the GNU Lesser General Public
>> + * License as published by the Free Software Foundation; either
>> + * version 2 of the License, or (at your option) any later version.
>> + *
>> + * This library is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * Lesser General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public
>> + * License along with this library; if not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/error.h"
>> +#include "hw/intc/arm_gicv3_its_common.h"
>> +#include "sysemu/sysemu.h"
>> +#include "sysemu/kvm.h"
>> +#include "kvm_arm.h"
>> +#include "migration/migration.h"
>> +
>> +#define TYPE_KVM_ARM_ITS "arm-its-kvm"
>> +#define KVM_ARM_ITS(obj) OBJECT_CHECK(GICv3ITSState, (obj), TYPE_KVM_ARM_ITS)
>> +
>> +static int kvm_its_send_msi(GICv3ITSState *s, uint32_t value, uint16_t devid)
>> +{
>> +    struct kvm_msi msi;
>> +
>> +    msi.address_lo = extract64(s->gits_translater_gpa, 0, 32);
>> +    msi.address_hi = extract64(s->gits_translater_gpa, 32, 32);
>> +    msi.data = le32_to_cpu(value);
>> +    msi.flags = KVM_MSI_VALID_DEVID;
>> +    msi.devid = devid;
>> +    memset(msi.pad, 0, sizeof(msi.pad));
>> +
>> +    return kvm_vm_ioctl(kvm_state, KVM_SIGNAL_MSI, &msi);
>> +}
>> +
>> +typedef struct ItsInitNotifierParams {
>> +    Notifier notifier;
>> +    GICv3ITSState *s;
>> +} ItsInitNotifierParams;
>> +
>> +/* this notifier computes and stores the ITS MSI frame GPA */
>> +static void its_notify(Notifier *notifier, void *data)
>> +{
>> +    ItsInitNotifierParams *p = DO_UPCAST(ItsInitNotifierParams,
>> +                                         notifier, notifier);
>> +    GICv3ITSState *s = p->s;
>> +    MemoryRegion *mr = &s->iomem_its_translation;
>> +    MemoryRegionSection mrs;
>> +    uint64_t addr;
>> +
>> +    mrs = memory_region_find(mr, 0, 1);
>> +    addr = mrs.offset_within_address_space;
>> +    s->gits_translater_gpa = addr + 0x40;
> 
> This leaks a memory region reference.
Sure I did not notice memory_region_find is adding a reference.
> 
> Also, I think it would be simpler to just have kvm_its_send_msi() do
> 
>    if (unlikely(!s->translater_gpa_known)) {
>        s->gits_translater_gpa = ...;
>        s->translater_gpa_known = true;
>    }
My issue precisely is to get the absolue GPA of the ITS translater.
kvm_its_send_msi is called by the region write callback which works on
relative address wrt start of the translation region while kvm ioctl
uses the absolute GPA. I did not see any other way to retrieve the
absolute GPA than using a machine init done notifier.
> 
> rather than messing around with a machine-init-done notifier.
> 
>> +}
>> +
>> +static void kvm_arm_its_realize(DeviceState *dev, Error **errp)
>> +{
>> +    GICv3ITSState *s = ARM_GICV3_ITS_COMMON(dev);
>> +    ItsInitNotifierParams *p = g_new(ItsInitNotifierParams, 1);
>> +
>> +    s->dev_fd = kvm_create_device(kvm_state, KVM_DEV_TYPE_ARM_VGIC_ITS, false);
>> +    if (s->dev_fd < 0) {
>> +        error_setg_errno(errp, -s->dev_fd, "error creating in-kernel ITS");
>> +        return;
>> +    }
>> +
>> +    /* explicit init if the ITS */
> 
> "of the"
OK
> 
>> +    kvm_device_access(s->dev_fd, KVM_DEV_ARM_VGIC_GRP_CTRL,
>> +                      KVM_DEV_ARM_VGIC_CTRL_INIT, NULL, true);
>> +
>> +    /* register the base address */
>> +    kvm_arm_register_device(&s->iomem_its_cntrl, -1, KVM_DEV_ARM_VGIC_GRP_ADDR,
>> +                            KVM_VGIC_ITS_ADDR_TYPE, s->dev_fd);
>> +
>> +    gicv3_its_init_mmio(s, NULL);
>> +
>> +    p->notifier.notify = its_notify;
>> +    p->s = s;
>> +    qemu_add_machine_init_done_notifier(&p->notifier);
>> +
>> +    /*
>> +     * Block migration of a KVM GICv3 ITS device: the API for saving and
>> +     * restoring the state in the kernel is not yet available
>> +     */
>> +    error_setg(&s->migration_blocker, "vITS migration is not implemented");
>> +    migrate_add_blocker(s->migration_blocker);
>> +
>> +    kvm_msi_use_devid = true;
>> +    kvm_gsi_direct_mapping = false;
>> +    kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
>> +}
>> +
>> +static void kvm_arm_its_init(Object *obj)
>> +{
>> +    GICv3ITSState *s = KVM_ARM_ITS(obj);
>> +
>> +    object_property_add_link(obj, "parent-gicv3",
>> +                             "kvm-arm-gicv3", (Object **)&s->gicv3,
>> +                             object_property_allow_set_link,
>> +                             OBJ_PROP_LINK_UNREF_ON_RELEASE,
>> +                             &error_abort);
>> +}
>> +
>> +static void kvm_arm_its_class_init(ObjectClass *klass, void *data)
>> +{
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +    GICv3ITSCommonClass *icc = ARM_GICV3_ITS_COMMON_CLASS(klass);
>> +
>> +    dc->realize = kvm_arm_its_realize;
>> +    icc->send_msi = kvm_its_send_msi;
>> +}
>> +
>> +static const TypeInfo kvm_arm_its_info = {
>> +    .name = TYPE_KVM_ARM_ITS,
>> +    .parent = TYPE_ARM_GICV3_ITS_COMMON,
>> +    .instance_size = sizeof(GICv3ITSState),
>> +    .instance_init = kvm_arm_its_init,
>> +    .class_init = kvm_arm_its_class_init,
>> +};
>> +
>> +static void kvm_arm_its_register_types(void)
>> +{
>> +    type_register_static(&kvm_arm_its_info);
>> +}
>> +
>> +type_init(kvm_arm_its_register_types)
>> --
>> 2.5.5

Thank you for your time

Best Regards

Eric
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] [PATCH v6 4/8] kvm-all: Pass requester ID to MSI routing functions

[Auger Eric]

Hi Peter,

On 22/09/2016 16:35, Peter Maydell wrote:
> On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
>> From: Pavel Fedin <p.fedin@samsung.com>
>>
>> Introduce global kvm_msi_use_devid flag and pass the device ID,
>> if needed, while building the MSI route entry. Device IDs are
>> required by the ARM GICv3 ITS (IRQ remapping function is based on
>> this information).
>>
>> Signed-off-by: Pavel Fedin <p.fedin@samsung.com>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>> v5 -> v6:
>> - move the flag in include/sysemu/kvm.h and populate the devid
>>   information in kvm-all.c
>>
>> v3 -> v4:
>> - OR route->flags with KVM_MSI_VALID_DEVID
>> ---
>>  include/sysemu/kvm.h | 1 +
>>  kvm-all.c            | 9 +++++++++
>>  kvm-stub.c           | 1 +
>>  3 files changed, 11 insertions(+)
>>
>> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
>> index c9c2436..c31c810 100644
>> --- a/include/sysemu/kvm.h
>> +++ b/include/sysemu/kvm.h
>> @@ -53,6 +53,7 @@ extern bool kvm_gsi_direct_mapping;
>>  extern bool kvm_readonly_mem_allowed;
>>  extern bool kvm_direct_msi_allowed;
>>  extern bool kvm_ioeventfd_any_length_allowed;
>> +extern bool kvm_msi_use_devid;
>>
>>  #if defined CONFIG_KVM || !defined NEED_CPU_H
>>  #define kvm_enabled()           (kvm_allowed)
> 
> You also need the appropriate #define, like all the other flags
> here (with a doc comment, and with the version that's just defined
> as '(false)').
> 
> The idea is that tests of the condition then don't need to
> be guarded by ifdef CONFIG_KVM.

Hum yes I totally forgot this.
> 
>> diff --git a/kvm-all.c b/kvm-all.c
>> index ebf35b0..df3bcde 100644
>> --- a/kvm-all.c
>> +++ b/kvm-all.c
>> @@ -119,6 +119,7 @@ bool kvm_readonly_mem_allowed;
>>  bool kvm_vm_attributes_allowed;
>>  bool kvm_direct_msi_allowed;
>>  bool kvm_ioeventfd_any_length_allowed;
>> +bool kvm_msi_use_devid;
>>
>>  static const KVMCapabilityInfo kvm_required_capabilites[] = {
>>      KVM_CAP_INFO(USER_MEMORY),
>> @@ -1275,6 +1276,10 @@ int kvm_irqchip_add_msi_route(KVMState *s, int vector, PCIDevice *dev)
>>      kroute.u.msi.address_lo = (uint32_t)msg.address;
>>      kroute.u.msi.address_hi = msg.address >> 32;
>>      kroute.u.msi.data = le32_to_cpu(msg.data);
>> +    if (kvm_msi_use_devid) {
> 
> these should use the macro, not direct access to the flag variable.
Sure

Thanks

Eric
> 
>> +        kroute.flags = KVM_MSI_VALID_DEVID;
>> +        kroute.u.msi.devid = pci_requester_id(dev);
>> +    }
>>      if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
>>          kvm_irqchip_release_virq(s, virq);
>>          return -EINVAL;
>> @@ -1308,6 +1313,10 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg,
>>      kroute.u.msi.address_lo = (uint32_t)msg.address;
>>      kroute.u.msi.address_hi = msg.address >> 32;
>>      kroute.u.msi.data = le32_to_cpu(msg.data);
>> +    if (kvm_msi_use_devid) {
>> +        kroute.flags = KVM_MSI_VALID_DEVID;
>> +        kroute.u.msi.devid = pci_requester_id(dev);
>> +    }
>>      if (kvm_arch_fixup_msi_route(&kroute, msg.address, msg.data, dev)) {
>>          return -EINVAL;
>>      }
>> diff --git a/kvm-stub.c b/kvm-stub.c
>> index 64e23f6..6f896e9 100644
>> --- a/kvm-stub.c
>> +++ b/kvm-stub.c
>> @@ -31,6 +31,7 @@ bool kvm_gsi_direct_mapping;
>>  bool kvm_allowed;
>>  bool kvm_readonly_mem_allowed;
>>  bool kvm_ioeventfd_any_length_allowed;
>> +bool kvm_msi_use_devid;
>>
>>  int kvm_destroy_vcpu(CPUState *cpu)
>>  {
> 
> thanks
> -- PMM
> 

Re: [Qemu-devel] [PATCH v6 0/8] vITS support

[Peter Maydell]

On 22 September 2016 at 16:53, Auger Eric <eric.auger@redhat.com> wrote:
> Hi Peter,
>
> On 22/09/2016 16:08, Peter Maydell wrote:
>> On 9 September 2016 at 18:30, Eric Auger <eric.auger@redhat.com> wrote:
>>> This series introduces support for in-kernel GICv3 ITS emulation.
>>>
>>> On dt guest the functionality is complete and was tested on Cavium ThunderX
>>> with virtio-net-pci and vhost-net.
>>>
>>> On ACPI guest the series was tested with virtio-net-pci only. For vhost-net,
>>> using MSIX we currently miss the ACPI IORT table generation linking the
>>> PCIe host controller with the ITS. The work is ongoing and will be submitted
>>> separately. Anyway the kernel ACPI IORT ITS node support is not upstreamed
>>> yet.
>>>
>>> The first patch is not really related to virtual ITS but advertises the KVM
>>> GSI routing support which goes along with MSI injection.
>>
>> Patches 7 and 8 are missing your signed-off-by line...
>
> Sure I will add them on top of Shannon's ones. Most credit must be given
> to him though.

Yep; you just need to add yours since the patches have passed
through you as well.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 5/8] hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation

[Peter Maydell]

On 22 September 2016 at 16:53, Auger Eric <eric.auger@redhat.com> wrote:
> Hi Peter,
>
> On 22/09/2016 17:03, Peter Maydell wrote:
>> Also, I think it would be simpler to just have kvm_its_send_msi() do
>>
>>    if (unlikely(!s->translater_gpa_known)) {
>>        s->gits_translater_gpa = ...;
>>        s->translater_gpa_known = true;
>>    }
> My issue precisely is to get the absolue GPA of the ITS translater.
> kvm_its_send_msi is called by the region write callback which works on
> relative address wrt start of the translation region while kvm ioctl
> uses the absolute GPA. I did not see any other way to retrieve the
> absolute GPA than using a machine init done notifier.

I don't understand. There should be no difference between
(1) calculate gits_translater_gpa in the machine init done notifier
(2) calculate gits_translater_gpa the first time that we need it,
ie on first call to kvm_its_send_msi()

We don't use it anywhere else, do we ?

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6 5/8] hw/intc/arm_gicv3_its: Implement support for in-kernel ITS emulation

[Auger Eric]

Hi Peter,
On 22/09/2016 17:59, Peter Maydell wrote:
> On 22 September 2016 at 16:53, Auger Eric <eric.auger@redhat.com> wrote:
>> Hi Peter,
>>
>> On 22/09/2016 17:03, Peter Maydell wrote:
>>> Also, I think it would be simpler to just have kvm_its_send_msi() do
>>>
>>>    if (unlikely(!s->translater_gpa_known)) {
>>>        s->gits_translater_gpa = ...;
>>>        s->translater_gpa_known = true;
>>>    }
>> My issue precisely is to get the absolue GPA of the ITS translater.
>> kvm_its_send_msi is called by the region write callback which works on
>> relative address wrt start of the translation region while kvm ioctl
>> uses the absolute GPA. I did not see any other way to retrieve the
>> absolute GPA than using a machine init done notifier.
> 
> I don't understand. There should be no difference between
> (1) calculate gits_translater_gpa in the machine init done notifier
> (2) calculate gits_translater_gpa the first time that we need it,
> ie on first call to kvm_its_send_msi()
> 
> We don't use it anywhere else, do we ?
Oh yes I now understand. Indeed I have all the data needed from
kvm_its_send_msi parameters to compute it there. At the moment we do not
use it anywhere else.

I will proceed that way then.

Thanks

Eric
> 
> thanks
> -- PMM
>