[RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Lan Tianyu]

This patchset is to introduce vIOMMU framework and add virtual VTD's
interrupt remapping support according "Xen virtual IOMMU high level
design doc V3"(https://lists.xenproject.org/archives/html/xen-devel/
2016-11/msg01391.html).

- vIOMMU framework
New framework provides viommu_ops and help functions to abstract
vIOMMU operations(E,G create, destroy, handle irq remapping request
and so on). Vendors(Intel, ARM, AMD and son) can implement their
vIOMMU callbacks.

- Xen vIOMMU device model in Qemu 
It's in charge of create/destroy vIOMMU in hypervisor via new vIOMMU
DMOP hypercalls. It will be required to pass virtual devices DMA
request to hypervisor when enable IOVA(DMA request without PASID)
function.

- Virtual VTD
In this patchset, we enable irq remapping function and covers both
MSI and IOAPIC interrupts. Don't support post interrupt mode emulation
and post interrupt mode enabled on host with virtual VTD. Will add
later.   

Chao Gao (19):
  Tools/libxc: Add viommu operations in libxc
  Tools/libacpi: Add DMA remapping reporting (DMAR) ACPI table
    structures
  Tools/libacpi: Add new fields in acpi_config to build DMAR table
  Tools/libacpi: Add a user configurable parameter to control vIOMMU
    attributes
  Tools/libxl: Inform device model to create a guest with a vIOMMU
    device
  x86/hvm: Introduce a emulated VTD for HVM
  X86/vvtd: Add MMIO handler for VVTD
  X86/vvtd: Set Interrupt Remapping Table Pointer through GCMD
  X86/vvtd: Process interrupt remapping request
  X86/vvtd: decode interrupt attribute from IRTE
  X86/vioapic: Hook interrupt delivery of vIOAPIC
  X86/vvtd: Enable Queued Invalidation through GCMD
  X86/vvtd: Enable Interrupt Remapping through GCMD
  x86/vpt: Get interrupt vector through a vioapic interface
  passthrough: move some fields of hvm_gmsi_info to a sub-structure
  Tools/libxc: Add a new interface to bind msi-ir with pirq
  X86/vmsi: Hook guest MSI injection
  X86/vvtd: Handle interrupt translation faults
  X86/vvtd: Add queued invalidation (QI) support

Lan Tianyu (4):
  VIOMMU: Add vIOMMU helper functions to create, destroy and query
    capabilities
  DMOP: Introduce new DMOP commands for vIOMMU support
  VIOMMU: Add irq request callback to deal with irq remapping
  VIOMMU: Add get irq info callback to convert irq remapping request

 tools/libacpi/acpi2_0.h                         |   45 +
 tools/libacpi/build.c                           |   58 ++
 tools/libacpi/libacpi.h                         |   12 +
 tools/libs/devicemodel/core.c                   |   69 ++
 tools/libs/devicemodel/include/xendevicemodel.h |   35 +
 tools/libs/devicemodel/libxendevicemodel.map    |    3 +
 tools/libxc/include/xenctrl.h                   |   17 +
 tools/libxc/include/xenctrl_compat.h            |    5 +
 tools/libxc/xc_devicemodel_compat.c             |   18 +
 tools/libxc/xc_domain.c                         |   55 +
 tools/libxl/libxl_create.c                      |   12 +-
 tools/libxl/libxl_dm.c                          |    9 +
 tools/libxl/libxl_dom.c                         |   85 ++
 tools/libxl/libxl_types.idl                     |    8 +
 tools/xl/xl_parse.c                             |   54 +
 xen/arch/x86/Makefile                           |    1 +
 xen/arch/x86/hvm/Makefile                       |    1 +
 xen/arch/x86/hvm/dm.c                           |   29 +
 xen/arch/x86/hvm/irq.c                          |   10 +
 xen/arch/x86/hvm/vioapic.c                      |   36 +
 xen/arch/x86/hvm/vmsi.c                         |   17 +-
 xen/arch/x86/hvm/vpt.c                          |    2 +-
 xen/arch/x86/hvm/vvtd.c                         | 1229 +++++++++++++++++++++++
 xen/arch/x86/viommu.c                           |   40 +
 xen/common/Makefile                             |    1 +
 xen/common/domain.c                             |    3 +
 xen/common/viommu.c                             |  119 +++
 xen/drivers/passthrough/io.c                    |  183 +++-
 xen/drivers/passthrough/vtd/iommu.h             |  213 +++-
 xen/include/asm-arm/viommu.h                    |   38 +
 xen/include/asm-x86/hvm/vioapic.h               |    1 +
 xen/include/asm-x86/msi.h                       |    3 +
 xen/include/asm-x86/viommu.h                    |   68 ++
 xen/include/public/arch-x86/hvm/save.h          |   19 +
 xen/include/public/domctl.h                     |    7 +
 xen/include/public/hvm/dm_op.h                  |   39 +
 xen/include/public/viommu.h                     |   38 +
 xen/include/xen/hvm/irq.h                       |   20 +-
 xen/include/xen/sched.h                         |    2 +
 xen/include/xen/viommu.h                        |   74 ++
 40 files changed, 2601 insertions(+), 77 deletions(-)
 create mode 100644 xen/arch/x86/hvm/vvtd.c
 create mode 100644 xen/arch/x86/viommu.c
 create mode 100644 xen/common/viommu.c
 create mode 100644 xen/include/asm-arm/viommu.h
 create mode 100644 xen/include/asm-x86/viommu.h
 create mode 100644 xen/include/public/viommu.h
 create mode 100644 xen/include/xen/viommu.h

-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Lan Tianyu]

This patch is to introduct an abstract layer for arch vIOMMU implementation
to deal with requests from dom0. Arch vIOMMU code needs to provide callback
to perform create, destroy and query capabilities operation.

Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/common/Makefile          |  1 +
 xen/common/domain.c          |  3 ++
 xen/common/viommu.c          | 97 ++++++++++++++++++++++++++++++++++++++++++++
 xen/include/asm-arm/viommu.h | 30 ++++++++++++++
 xen/include/asm-x86/viommu.h | 31 ++++++++++++++
 xen/include/public/viommu.h  | 38 +++++++++++++++++
 xen/include/xen/sched.h      |  2 +
 xen/include/xen/viommu.h     | 62 ++++++++++++++++++++++++++++
 8 files changed, 264 insertions(+)
 create mode 100644 xen/common/viommu.c
 create mode 100644 xen/include/asm-arm/viommu.h
 create mode 100644 xen/include/asm-x86/viommu.h
 create mode 100644 xen/include/public/viommu.h
 create mode 100644 xen/include/xen/viommu.h

diff --git a/xen/common/Makefile b/xen/common/Makefile
index 0fed30b..b58de63 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -60,6 +60,7 @@ obj-y += vm_event.o
 obj-y += vmap.o
 obj-y += vsprintf.o
 obj-y += wait.o
+obj-y += viommu.o
 obj-bin-y += warning.init.o
 obj-$(CONFIG_XENOPROF) += xenoprof.o
 obj-y += xmalloc_tlsf.o
diff --git a/xen/common/domain.c b/xen/common/domain.c
index 4492c9c..aafc740 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -398,6 +398,9 @@ struct domain *domain_create(domid_t domid, unsigned int domcr_flags,
         spin_unlock(&domlist_update_lock);
     }
 
+    if ( (err = viommu_init_domain(d)) != 0)
+        goto fail;
+
     return d;
 
  fail:
diff --git a/xen/common/viommu.c b/xen/common/viommu.c
new file mode 100644
index 0000000..4c1c788
--- /dev/null
+++ b/xen/common/viommu.c
@@ -0,0 +1,97 @@
+/*
+ * common/viommu.c
+ * 
+ * Copyright (c) 2017 Intel Corporation
+ * Author: Lan Tianyu <tianyu.lan@intel.com> 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#include <xen/types.h>
+#include <xen/sched.h>
+
+int viommu_init_domain(struct domain *d)
+{
+    d->viommu.nr_viommu = 0;
+    d->viommu.ops = viommu_get_ops();
+
+    return 0;
+}
+
+int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps)
+{
+    struct viommu_info *info = &d->viommu;
+    struct viommu *viommu;
+    int rc;
+
+    if ( !info || !info->ops || !info->ops->create
+	        || info->nr_viommu >= NR_VIOMMU_PER_DOMAIN )
+        return -EINVAL;
+
+    viommu = xzalloc(struct viommu);
+    if ( !viommu )
+        return -EFAULT;
+
+    viommu->base_address = base_address;
+    viommu->length = length;
+    viommu->caps = caps;
+    viommu->viommu_id = info->nr_viommu;
+
+    info->viommu[info->nr_viommu] = viommu;
+    info->nr_viommu++;
+
+    rc = info->ops->create(d, viommu);
+    if ( rc < 0 ) {
+        xfree(viommu);
+        return rc;
+    }
+
+    return viommu->viommu_id;
+}
+
+int viommu_destroy(struct domain *d, u32 viommu_id)
+{
+    struct viommu_info *info = &d->viommu;
+
+    if ( !info || !info->ops || !info->ops->destroy)
+        return -EINVAL;
+
+    if ( viommu_id > info->nr_viommu || !info->viommu[viommu_id] )
+        return -EINVAL;
+
+    if ( info->ops->destroy(info->viommu[viommu_id]) )
+        return -EFAULT;
+
+    info->viommu[viommu_id] = NULL;
+    return 0;
+}
+
+u64 viommu_query_caps(struct domain *d)
+{
+    struct viommu_info *info = &d->viommu;
+
+    if ( !info || !info->ops || !info->ops->query_caps)
+        return -EINVAL;
+
+    return info->ops->query_caps(d);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * End:
+ */
diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
new file mode 100644
index 0000000..ef6a60b
--- /dev/null
+++ b/xen/include/asm-arm/viommu.h
@@ -0,0 +1,30 @@
+/*
+ * include/asm-arm/viommu.h
+ *
+ * Copyright (c) 2017 Intel Corporation
+ * Author: Lan Tianyu <tianyu.lan@intel.com> 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+#ifndef __ARCH_ARM_VIOMMU_H__
+#define __ARCH_ARM_VIOMMU_H__
+
+#include <xen/viommu.h>
+
+static inline const struct viommu_ops *viommu_get_ops(void)
+{
+    return NULL;
+}
+
+#endif /* __ARCH_ARM_VIOMMU_H__ */
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
new file mode 100644
index 0000000..efb435f
--- /dev/null
+++ b/xen/include/asm-x86/viommu.h
@@ -0,0 +1,31 @@
+/*
+ * include/asm-arm/viommu.h
+ *
+ * Copyright (c) 2017 Intel Corporation.
+ * Author: Lan Tianyu <tianyu.lan@intel.com> 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+#ifndef __ARCH_X86_VIOMMU_H__
+#define __ARCH_X86_VIOMMU_H__
+
+#include <xen/viommu.h>
+#include <asm/types.h>
+
+static inline const struct viommu_ops *viommu_get_ops(void)
+{
+    return NULL;
+}
+
+#endif /* __ARCH_X86_VIOMMU_H__ */
diff --git a/xen/include/public/viommu.h b/xen/include/public/viommu.h
new file mode 100644
index 0000000..ca2419b
--- /dev/null
+++ b/xen/include/public/viommu.h
@@ -0,0 +1,38 @@
+/*
+ * include/public/viommu.h
+ *
+ * Copyright (c) 2017 Intel Corporation
+ * Author: Lan Tianyu <tianyu.lan@intel.com> 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef __XEN_PUBLIC_VIOMMU_H__
+#define __XEN_PUBLIC_VIOMMU_H__
+
+/* VIOMMU capabilities*/
+#define VIOMMU_CAP_IRQ_REMAPPING  (1 << 0)
+
+#endif /* __XEN_PUBLIC_VIOMMU_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
+
diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
index 0929c0b..a2e9cdc 100644
--- a/xen/include/xen/sched.h
+++ b/xen/include/xen/sched.h
@@ -21,6 +21,7 @@
 #include <xen/perfc.h>
 #include <asm/atomic.h>
 #include <xen/wait.h>
+#include <xen/viommu.h>
 #include <public/xen.h>
 #include <public/domctl.h>
 #include <public/sysctl.h>
@@ -481,6 +482,7 @@ struct domain
     /* vNUMA topology accesses are protected by rwlock. */
     rwlock_t vnuma_rwlock;
     struct vnuma_info *vnuma;
+    struct viommu_info viommu;
 
     /* Common monitor options */
     struct {
diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
new file mode 100644
index 0000000..a0abbdf
--- /dev/null
+++ b/xen/include/xen/viommu.h
@@ -0,0 +1,62 @@
+/*
+ * include/xen/viommu.h
+ *
+ * Copyright (c) 2017, Intel Corporation
+ * Author: Lan Tianyu <tianyu.lan@intel.com> 
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program; If not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+#ifndef __XEN_VIOMMU_H__
+#define __XEN_VIOMMU_H__
+
+#include <asm/viommu.h>
+
+#define NR_VIOMMU_PER_DOMAIN 1
+
+struct viommu {
+    u64 base_address;
+    u64 length;
+    u64 caps;
+    u32 viommu_id;
+    void *priv;
+};
+
+struct viommu_ops {
+    u64 (*query_caps)(struct domain *d);
+    int (*create)(struct domain *d, struct viommu *viommu);
+    int (*destroy)(struct viommu *viommu);
+};
+
+struct viommu_info {
+    u32 nr_viommu;
+    struct viommu *viommu[NR_VIOMMU_PER_DOMAIN]; /* viommu array*/
+    const struct viommu_ops *ops;     /* viommu_ops */
+};
+
+int viommu_init_domain(struct domain *d);
+int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps);
+int viommu_destroy(struct domain *d, u32 viommu_id);
+u64 viommu_query_caps(struct domain *d);
+
+#endif /* __XEN_VIOMMU_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 2/23] DMOP: Introduce new DMOP commands for vIOMMU support

[Lan Tianyu]

This patch is to introduce create, destroy and query capabilities
command for vIOMMU. vIOMMU layer will deal with requests and call
arch vIOMMU ops.

Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/dm.c          | 29 +++++++++++++++++++++++++++++
 xen/include/public/hvm/dm_op.h | 39 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 68 insertions(+)

diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
index 2122c45..2b28f70 100644
--- a/xen/arch/x86/hvm/dm.c
+++ b/xen/arch/x86/hvm/dm.c
@@ -491,6 +491,35 @@ static int dm_op(domid_t domid,
         break;
     }
 
+    case XEN_DMOP_create_viommu:
+    {
+        struct xen_dm_op_create_viommu *data =
+            &op.u.create_viommu;
+
+        rc = viommu_create(d, data->base_address, data->length, data->capabilities);
+        if (rc >= 0) {
+            data->viommu_id = rc;
+            rc = 0;
+        }
+        break;
+    }
+    case XEN_DMOP_destroy_viommu:
+    {
+        const struct xen_dm_op_destroy_viommu *data =
+            &op.u.destroy_viommu;
+
+        rc = viommu_destroy(d, data->viommu_id);
+        break;
+    }
+    case XEN_DMOP_query_viommu_caps:
+    {
+        struct xen_dm_op_query_viommu_caps *data =
+            &op.u.query_viommu_caps;
+
+        data->caps = viommu_query_caps(d);
+        rc = 0;
+        break;
+    }
     default:
         rc = -EOPNOTSUPP;
         break;
diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
index f54cece..b8c7359 100644
--- a/xen/include/public/hvm/dm_op.h
+++ b/xen/include/public/hvm/dm_op.h
@@ -318,6 +318,42 @@ struct xen_dm_op_inject_msi {
     uint64_aligned_t addr;
 };
 
+/*
+ * XEN_DMOP_create_viommu: Create vIOMMU device.
+ */
+#define XEN_DMOP_create_viommu 15
+
+struct xen_dm_op_create_viommu {
+    /* IN - MMIO base address of vIOMMU */
+    uint64_t base_address;
+    /* IN - Length of MMIO region */
+    uint64_t length;
+    /* IN - Capabilities with which we want to create */
+    uint64_t capabilities;
+    /* OUT - vIOMMU identity */
+    uint32_t viommu_id;
+};
+
+/*
+ * XEN_DMOP_destroy_viommu: Destroy vIOMMU device.
+ */
+#define XEN_DMOP_destroy_viommu 16
+
+struct xen_dm_op_destroy_viommu {
+    /* OUT - vIOMMU identity */
+    uint32_t viommu_id;
+};
+
+/*
+ * XEN_DMOP_q_viommu: Query vIOMMU capabilities.
+ */
+#define XEN_DMOP_query_viommu_caps 17
+
+struct xen_dm_op_query_viommu_caps {
+    /* OUT - vIOMMU Capabilities*/
+    uint64_t caps;
+};
+
 struct xen_dm_op {
     uint32_t op;
     uint32_t pad;
@@ -336,6 +372,9 @@ struct xen_dm_op {
         struct xen_dm_op_set_mem_type set_mem_type;
         struct xen_dm_op_inject_event inject_event;
         struct xen_dm_op_inject_msi inject_msi;
+        struct xen_dm_op_create_viommu create_viommu;
+        struct xen_dm_op_destroy_viommu destroy_viommu;
+        struct xen_dm_op_query_viommu_caps query_viommu_caps;
     } u;
 };
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 3/23] VIOMMU: Add irq request callback to deal with irq remapping

[Lan Tianyu]

This patch is to add irq request callback for platform implementation
to deal with irq remapping request.

Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/common/viommu.c          | 11 +++++++++++
 xen/include/asm-arm/viommu.h |  4 ++++
 xen/include/asm-x86/viommu.h | 15 +++++++++++++++
 xen/include/xen/viommu.h     |  8 ++++++++
 4 files changed, 38 insertions(+)

diff --git a/xen/common/viommu.c b/xen/common/viommu.c
index 4c1c788..62c66db 100644
--- a/xen/common/viommu.c
+++ b/xen/common/viommu.c
@@ -87,6 +87,17 @@ u64 viommu_query_caps(struct domain *d)
     return info->ops->query_caps(d);
 }
 
+int viommu_handle_irq_request(struct domain *d,
+        struct irq_remapping_request *request)
+{
+    struct viommu_info *info = &d->viommu;
+
+    if ( !info || !info->ops || !info->ops->handle_irq_request)
+        return -EINVAL;
+
+    return info->ops->handle_irq_request(d, request);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
index ef6a60b..6a81ecb 100644
--- a/xen/include/asm-arm/viommu.h
+++ b/xen/include/asm-arm/viommu.h
@@ -22,6 +22,10 @@
 
 #include <xen/viommu.h>
 
+struct irq_remapping_request
+{
+};
+
 static inline const struct viommu_ops *viommu_get_ops(void)
 {
     return NULL;
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
index efb435f..b6e01a5 100644
--- a/xen/include/asm-x86/viommu.h
+++ b/xen/include/asm-x86/viommu.h
@@ -23,6 +23,21 @@
 #include <xen/viommu.h>
 #include <asm/types.h>
 
+struct irq_remapping_request
+{
+    u8 type;
+    u16 source_id;
+    union {
+        /* MSI */
+        struct {
+            u64 addr;
+            u32 data;
+        } msi;
+        /* Redirection Entry in IOAPIC */
+        u64 rte;
+    } msg;
+};
+
 static inline const struct viommu_ops *viommu_get_ops(void)
 {
     return NULL;
diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
index a0abbdf..246b29d 100644
--- a/xen/include/xen/viommu.h
+++ b/xen/include/xen/viommu.h
@@ -24,6 +24,10 @@
 
 #define NR_VIOMMU_PER_DOMAIN 1
 
+/* IRQ request type */
+#define VIOMMU_REQUEST_IRQ_MSI          0
+#define VIOMMU_REQUEST_IRQ_APIC         1
+
 struct viommu {
     u64 base_address;
     u64 length;
@@ -36,6 +40,8 @@ struct viommu_ops {
     u64 (*query_caps)(struct domain *d);
     int (*create)(struct domain *d, struct viommu *viommu);
     int (*destroy)(struct viommu *viommu);
+    int (*handle_irq_request)(struct domain *d,
+                              struct irq_remapping_request *request);
 };
 
 struct viommu_info {
@@ -48,6 +54,8 @@ int viommu_init_domain(struct domain *d);
 int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps);
 int viommu_destroy(struct domain *d, u32 viommu_id);
 u64 viommu_query_caps(struct domain *d);
+int viommu_handle_irq_request(struct domain *d,
+                              struct irq_remapping_request *request);
 
 #endif /* __XEN_VIOMMU_H__ */
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 4/23] VIOMMU: Add get irq info callback to convert irq remapping request

[Lan Tianyu]

This patch is to add get_irq_info callback for platform implementation
to convert irq remapping request to irq info (E,G vector, dest, dest_mode
and so on).

Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/common/viommu.c          | 11 +++++++++++
 xen/include/asm-arm/viommu.h |  4 ++++
 xen/include/asm-x86/viommu.h |  8 ++++++++
 xen/include/xen/viommu.h     |  4 ++++
 4 files changed, 27 insertions(+)

diff --git a/xen/common/viommu.c b/xen/common/viommu.c
index 62c66db..dbec692 100644
--- a/xen/common/viommu.c
+++ b/xen/common/viommu.c
@@ -98,6 +98,17 @@ int viommu_handle_irq_request(struct domain *d,
     return info->ops->handle_irq_request(d, request);
 }
 
+int viommu_get_irq_info(struct domain *d, struct irq_remapping_request *request,
+                        struct irq_remapping_info *irq_info)
+{
+    struct viommu_info *info = &d->viommu;
+
+    if ( !info || !info->ops || !info->ops->get_irq_info)
+        return -EINVAL;
+
+    return info->ops->get_irq_info(d, request, irq_info);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
index 6a81ecb..6ce4e0a 100644
--- a/xen/include/asm-arm/viommu.h
+++ b/xen/include/asm-arm/viommu.h
@@ -22,6 +22,10 @@
 
 #include <xen/viommu.h>
 
+struct irq_remapping_info
+{
+};
+
 struct irq_remapping_request
 {
 };
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
index b6e01a5..43e446e 100644
--- a/xen/include/asm-x86/viommu.h
+++ b/xen/include/asm-x86/viommu.h
@@ -23,6 +23,14 @@
 #include <xen/viommu.h>
 #include <asm/types.h>
 
+struct irq_remapping_info
+{
+    u8  vector;
+    u32 dest;
+    u32 dest_mode:1;
+    u32 delivery_mode:3;
+};
+
 struct irq_remapping_request
 {
     u8 type;
diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
index 246b29d..d733012 100644
--- a/xen/include/xen/viommu.h
+++ b/xen/include/xen/viommu.h
@@ -42,6 +42,8 @@ struct viommu_ops {
     int (*destroy)(struct viommu *viommu);
     int (*handle_irq_request)(struct domain *d,
                               struct irq_remapping_request *request);
+    int (*get_irq_info)(struct domain *d, struct irq_remapping_request *request,
+                        struct irq_remapping_info *info);
 };
 
 struct viommu_info {
@@ -56,6 +58,8 @@ int viommu_destroy(struct domain *d, u32 viommu_id);
 u64 viommu_query_caps(struct domain *d);
 int viommu_handle_irq_request(struct domain *d,
                               struct irq_remapping_request *request);
+int viommu_get_irq_info(struct domain *d, struct irq_remapping_request *request,
+                        struct irq_remapping_info *irq_info);
 
 #endif /* __XEN_VIOMMU_H__ */
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

In previous patch, we introduce a common vIOMMU layer. In our design,
we create/destroy vIOMMU through DMOP interface instead of creating it
according to a config flag of domain. It makes it is possible
to create vIOMMU in device model or in tool stack.

The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:
 - query capabilities of vIOMMU emulated by Xen
 - create vIOMMU in Xen hypervisor with base address, capability
 - destroy vIOMMU specified by viommu_id

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libs/devicemodel/core.c                   | 69 +++++++++++++++++++++++++
 tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
 tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
 tools/libxc/include/xenctrl_compat.h            |  5 ++
 tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
 5 files changed, 130 insertions(+)

diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
index a85cb49..aee1150 100644
--- a/tools/libs/devicemodel/core.c
+++ b/tools/libs/devicemodel/core.c
@@ -498,6 +498,75 @@ int xendevicemodel_restrict(xendevicemodel_handle *dmod, domid_t domid)
     return osdep_xendevicemodel_restrict(dmod, domid);
 }
 
+int xendevicemodel_viommu_query_cap(
+    xendevicemodel_handle *dmod, domid_t dom, uint64_t *cap)
+{
+    struct xen_dm_op op;
+    struct xen_dm_op_query_viommu_caps *data;
+    int rc;
+
+    if ( !cap )
+        return -EINVAL;
+
+    memset(&op, 0, sizeof(op));
+
+    op.op = XEN_DMOP_query_viommu_caps;
+    data = &op.u.query_viommu_caps;
+
+    rc = xendevicemodel_op(dmod, dom, 1, &op, sizeof(op));
+    if ( rc )
+        return rc;
+
+    *cap = data->caps;
+    return 0;
+}
+
+int xendevicemodel_viommu_create(
+    xendevicemodel_handle *dmod, domid_t dom, uint64_t base_addr,
+    uint64_t cap, uint32_t *viommu_id)
+{
+    struct xen_dm_op op;
+    struct xen_dm_op_create_viommu *data;
+    int rc;
+
+    if ( !viommu_id )
+        return -EINVAL;
+
+    memset(&op, 0, sizeof(op));
+
+    op.op = XEN_DMOP_create_viommu;
+    data = &op.u.create_viommu;
+
+    data->base_address = base_addr;
+    data->capabilities = cap;
+
+    rc = xendevicemodel_op(dmod, dom, 1, &op, sizeof(op));
+    if ( rc )
+        return rc;
+
+    *viommu_id = data->viommu_id;
+    return 0;
+}
+
+int xendevicemodel_viommu_destroy(
+    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id)
+{
+    struct xen_dm_op op;
+    struct xen_dm_op_destroy_viommu *data;
+
+    if ( !viommu_id )
+        return -EINVAL;
+
+    memset(&op, 0, sizeof(op));
+
+    op.op = XEN_DMOP_destroy_viommu;
+    data = &op.u.destroy_viommu;
+
+    data->viommu_id = viommu_id;
+
+    return xendevicemodel_op(dmod, dom, 1, &op, sizeof(op));
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libs/devicemodel/include/xendevicemodel.h b/tools/libs/devicemodel/include/xendevicemodel.h
index b3f600e..e133dd5 100644
--- a/tools/libs/devicemodel/include/xendevicemodel.h
+++ b/tools/libs/devicemodel/include/xendevicemodel.h
@@ -293,6 +293,41 @@ int xendevicemodel_inject_event(
  */
 int xendevicemodel_restrict(xendevicemodel_handle *dmod, domid_t domid);
 
+/**
+ * This function queries the capabilitites of vIOMMU emulated by Xen.
+ *
+ * @parm dmod a handle to an open devicemodel interface.
+ * @parm dom the domain id to be serviced.
+ * @parm cap points to memory to store the capability. 
+ * @return 0 on success, -1 on failure.
+ */
+int xendevicemodel_viommu_query_cap(
+    xendevicemodel_handle *dmod, domid_t dom, uint64_t *cap);
+
+/**
+ * This function creates vIOMMU in Xen hypervisor with base_addr, capability.
+ *
+ * @parm dmod a handle to an open devicemodel interface.
+ * @parm dom the domain id to be serviced.
+ * @parm base_addr base address of register set of the vIOMMU. 
+ * @parm cap the capability owned by the vIOMMU to be created. 
+ * @parm viommu_id points to memory to store the vIOMMU id.
+ * @return 0 on success, -1 on failure.
+ */
+int xendevicemodel_viommu_create(
+    xendevicemodel_handle *dmod, domid_t dom, uint64_t base_addr,
+    uint64_t cap, uint32_t *viommu_id);
+
+/**
+ * This function destroies vIOMMU specified by viommu_id.
+ *
+ * @parm dmod a handle to an open devicemodel interface.
+ * @parm dom the domain id to be serviced.
+ * @parm viommu_id spcifies the id of the vIOMMU to be destroied. 
+ * @return 0 on success, -1 on failure.
+ */
+int xendevicemodel_viommu_destroy(
+    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
 #endif /* __XEN_TOOLS__ */
 
 #endif /* XENDEVICEMODEL_H */
diff --git a/tools/libs/devicemodel/libxendevicemodel.map b/tools/libs/devicemodel/libxendevicemodel.map
index 45c773e..c2e0968 100644
--- a/tools/libs/devicemodel/libxendevicemodel.map
+++ b/tools/libs/devicemodel/libxendevicemodel.map
@@ -17,6 +17,9 @@ VERS_1.0 {
 		xendevicemodel_modified_memory;
 		xendevicemodel_set_mem_type;
 		xendevicemodel_inject_event;
+		xendevicemodel_viommu_query_cap;
+		xendevicemodel_viommu_create;
+		xendevicemodel_viommu_destroy;
 		xendevicemodel_restrict;
 		xendevicemodel_close;
 	local: *; /* Do not expose anything by default */
diff --git a/tools/libxc/include/xenctrl_compat.h b/tools/libxc/include/xenctrl_compat.h
index 040e7b2..315c45d 100644
--- a/tools/libxc/include/xenctrl_compat.h
+++ b/tools/libxc/include/xenctrl_compat.h
@@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
 int xc_hvm_inject_trap(
     xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
     uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
+int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t *cap);
+int xc_viommu_create(
+    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
+    uint32_t *viommu_id);
+int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t viommu_id);
 
 #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
 
diff --git a/tools/libxc/xc_devicemodel_compat.c b/tools/libxc/xc_devicemodel_compat.c
index e4edeea..62f703a 100644
--- a/tools/libxc/xc_devicemodel_compat.c
+++ b/tools/libxc/xc_devicemodel_compat.c
@@ -128,6 +128,24 @@ int xc_hvm_inject_trap(
                                        type, error_code, insn_len, cr2);
 }
 
+int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t *cap)
+{
+    return xendevicemodel_viommu_query_cap(xch->dmod, dom, cap);
+}
+
+int xc_viommu_create(
+    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
+    uint32_t *viommu_id)
+{
+    return xendevicemodel_viommu_create(xch->dmod, dom, base_addr, cap,
+                                        viommu_id);
+}
+
+int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t viommu_id)
+{
+    return xendevicemodel_viommu_destroy(xch->dmod, dom, viommu_id);
+}
+
 /*
  * Local variables:
  * mode: C
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 6/23] Tools/libacpi: Add DMA remapping reporting (DMAR) ACPI table structures

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Add dmar table structure according Chapter 8 "BIOS Considerations" of
VTd spec Rev. 2.4.

VTd spec:http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libacpi/acpi2_0.h | 45 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 45 insertions(+)

diff --git a/tools/libacpi/acpi2_0.h b/tools/libacpi/acpi2_0.h
index 2619ba3..8f942b5 100644
--- a/tools/libacpi/acpi2_0.h
+++ b/tools/libacpi/acpi2_0.h
@@ -421,6 +421,49 @@ struct acpi_20_slit {
     uint8_t entry[0];
 };
 
+/* DMA Remapping Table in VTd spec Rev. 2.4. */
+struct acpi_dmar {
+    struct acpi_header header;
+    uint8_t host_address_width;
+    uint8_t flags;
+    uint8_t reserved[10]; /* reserved(0) */
+};
+
+/* Remapping Structure Types */
+enum {
+    ACPI_DMAR_TYPE_HARDWARE_UNIT = 0,       /* DRHD */
+    ACPI_DMAR_TYPE_RESERVED_MEMORY = 1,     /* RMRR */
+    ACPI_DMAR_TYPE_ATSR = 2,                /* ATSR */
+    ACPI_DMAR_TYPE_HARDWARE_AFFINITY = 3,   /* RHSR */
+    ACPI_DMAR_TYPE_ANDD = 4,                /* ANDD */
+    ACPI_DMAR_TYPE_RESERVED = 5             /* Reserved for furture use */
+};
+
+struct dmar_device_scope {
+    uint8_t type;
+    uint8_t length;
+    uint8_t reserved[2]; /* reserved(0) */
+    uint8_t enumeration_id;
+    uint8_t bus;
+    uint16_t path[0];
+};
+
+struct acpi_dmar_hardware_unit {
+    uint16_t type;
+    uint16_t length;
+    uint8_t flags;
+    uint8_t reserved; /* reserved(0) */
+    uint16_t pci_segment; /* The PCI segment associated with this unit */
+    uint64_t address; /* Base address of remapping hardware register-set */
+    struct dmar_device_scope scope[0];
+};
+
+/* Device scope type */
+#define ACPI_DMAR_DEVICE_SCOPE_IOAPIC   0x03
+
+/* Masks for flags field of struct acpi_dmar_hardware_unit */
+#define ACPI_DMAR_INCLUDE_PCI_ALL   1
+
 /*
  * Table Signatures.
  */
@@ -435,6 +478,7 @@ struct acpi_20_slit {
 #define ACPI_2_0_WAET_SIGNATURE ASCII32('W','A','E','T')
 #define ACPI_2_0_SRAT_SIGNATURE ASCII32('S','R','A','T')
 #define ACPI_2_0_SLIT_SIGNATURE ASCII32('S','L','I','T')
+#define ACPI_2_0_DMAR_SIGNATURE ASCII32('D','M','A','R')
 
 /*
  * Table revision numbers.
@@ -449,6 +493,7 @@ struct acpi_20_slit {
 #define ACPI_1_0_FADT_REVISION 0x01
 #define ACPI_2_0_SRAT_REVISION 0x01
 #define ACPI_2_0_SLIT_REVISION 0x01
+#define ACPI_2_0_DMAR_REVISION 0x01
 
 #pragma pack ()
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 7/23] Tools/libacpi: Add new fields in acpi_config to build DMAR table

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

The BIOS reports the remapping hardware units in a platform to system software
through the DMA Remapping Reporting (DMAR) ACPI table.

To build DMAR table during domain construction, two fields are added to struct
acpi_config. One is dmar_flag which indicates whether interrupt remapping is
supported and whether enabling X2APIC mode is premitted. The other is the base
address of remapping hardware register-set for a remapping unit. Also, a
function construct_dmar() is added to build DMAR table according the two
fields. Note that we don't add a ACPI_HAS_DMAR table flag there for DMAR table
will be only built for HVM guest in this version. But several fields of
DMAR table is better to be determined during domain creation rather than
compiling tool stack.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libacpi/build.c   | 53 +++++++++++++++++++++++++++++++++++++++++++++++++
 tools/libacpi/libacpi.h | 11 ++++++++++
 2 files changed, 64 insertions(+)

diff --git a/tools/libacpi/build.c b/tools/libacpi/build.c
index a02ffbf..89a3c6c 100644
--- a/tools/libacpi/build.c
+++ b/tools/libacpi/build.c
@@ -27,6 +27,10 @@
 
 #define ACPI_MAX_SECONDARY_TABLES 16
 
+#define VTD_HOST_ADDRESS_WIDTH 39
+#define I440_PSEUDO_BUS_PLATFORM 0xff
+#define I440_PSEUDO_DEVFN_IOAPIC 0x0
+
 #define align16(sz)        (((sz) + 15) & ~15)
 #define fixed_strcpy(d, s) strncpy((d), (s), sizeof(d))
 
@@ -302,6 +306,55 @@ static struct acpi_20_slit *construct_slit(struct acpi_ctxt *ctxt,
     return slit;
 }
 
+struct acpi_dmar *construct_dmar(struct acpi_ctxt *ctxt,
+                                 const struct acpi_config *config)
+{
+    struct acpi_dmar *dmar;
+    struct acpi_dmar_hardware_unit *drhd;
+    struct dmar_device_scope *scope;
+    unsigned int size;
+    unsigned int ioapic_scope_size = sizeof(*scope) + sizeof(scope->path[0]);
+
+    size = sizeof(*dmar) + sizeof(*drhd) + ioapic_scope_size;
+
+    dmar = ctxt->mem_ops.alloc(ctxt, size, 16);
+    if ( !dmar )
+        return NULL;
+
+    memset(dmar, 0, size);
+    dmar->header.signature = ACPI_2_0_DMAR_SIGNATURE;
+    dmar->header.revision = ACPI_2_0_DMAR_REVISION;
+    dmar->header.length = size;
+    fixed_strcpy(dmar->header.oem_id, ACPI_OEM_ID);
+    fixed_strcpy(dmar->header.oem_table_id, ACPI_OEM_TABLE_ID);
+    dmar->header.oem_revision = ACPI_OEM_REVISION;
+    dmar->header.creator_id   = ACPI_CREATOR_ID;
+    dmar->header.creator_revision = ACPI_CREATOR_REVISION;
+    dmar->host_address_width = VTD_HOST_ADDRESS_WIDTH - 1;
+    dmar->flags = config->dmar_flag & (DMAR_INTR_REMAP|DMAR_X2APIC_OPT_OUT);
+
+    drhd = (struct acpi_dmar_hardware_unit *)((void*)dmar + sizeof(*dmar));
+    drhd->type = ACPI_DMAR_TYPE_HARDWARE_UNIT;
+    drhd->length = sizeof(*drhd) + ioapic_scope_size;
+    drhd->flags = ACPI_DMAR_INCLUDE_PCI_ALL;
+    drhd->pci_segment = 0;
+    drhd->address = config->viommu_base_addr;
+
+    scope = &drhd->scope[0];
+    scope->type = ACPI_DMAR_DEVICE_SCOPE_IOAPIC;
+    scope->length = ioapic_scope_size;
+    /*
+     * This field provides the I/O APICID as provided in the I/O APIC structure
+     * in the ACPI MADT (Multiple APIC Descriptor Table).
+     */
+    scope->enumeration_id = 1;
+    scope->bus = I440_PSEUDO_BUS_PLATFORM;
+    scope->path[0] = I440_PSEUDO_DEVFN_IOAPIC;
+
+    set_checksum(dmar, offsetof(struct acpi_header, checksum), size);
+    return dmar;
+}
+
 static int construct_passthrough_tables(struct acpi_ctxt *ctxt,
                                         unsigned long *table_ptrs,
                                         int nr_tables,
diff --git a/tools/libacpi/libacpi.h b/tools/libacpi/libacpi.h
index 67bd67f..ee08c45 100644
--- a/tools/libacpi/libacpi.h
+++ b/tools/libacpi/libacpi.h
@@ -20,6 +20,8 @@
 #ifndef __LIBACPI_H__
 #define __LIBACPI_H__
 
+#include "acpi2_0.h"
+
 #define ACPI_HAS_COM1        (1<<0)
 #define ACPI_HAS_COM2        (1<<1)
 #define ACPI_HAS_LPT1        (1<<2)
@@ -35,6 +37,7 @@
 #define ACPI_HAS_VGA         (1<<12)
 #define ACPI_HAS_8042        (1<<13)
 #define ACPI_HAS_CMOS_RTC    (1<<14)
+#define ACPI_HAS_DMAR        (1<<15)
 
 struct xen_vmemrange;
 struct acpi_numa {
@@ -95,8 +98,16 @@ struct acpi_config {
     uint32_t ioapic_base_address;
     uint16_t pci_isa_irq_mask;
     uint8_t ioapic_id;
+
+    /* dmar info */
+    uint8_t dmar_flag;
+    uint64_t viommu_base_addr;
 };
 
+#define DMAR_INTR_REMAP 0x1
+#define DMAR_X2APIC_OPT_OUT 0x2
+struct acpi_dmar *construct_dmar(struct acpi_ctxt *ctxt,
+                                 const struct acpi_config *config);
 int acpi_build_tables(struct acpi_ctxt *ctxt, struct acpi_config *config);
 
 #endif /* __LIBACPI_H__ */
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 8/23] Tools/libacpi: Add a user configurable parameter to control vIOMMU attributes

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

a field, viommu_info, is added to struct libxl_domain_build_info. Several
attributes can be specified by guest configuration file for the DMAR table
building and dummy vIOMMU creation.

In domain creation process, a new logic is added to build ACPI DMAR table in
tool stack according VM configuration and to pass though it to hvmloader via
xenstore ACPI PT channel. If there are ACPI tables needed to pass through, we
joint the tables.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libacpi/build.c       |  5 +++
 tools/libacpi/libacpi.h     |  1 +
 tools/libxl/libxl_dom.c     | 85 +++++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl |  8 +++++
 tools/xl/xl_parse.c         | 54 ++++++++++++++++++++++++++++
 5 files changed, 153 insertions(+)

diff --git a/tools/libacpi/build.c b/tools/libacpi/build.c
index 89a3c6c..080413e 100644
--- a/tools/libacpi/build.c
+++ b/tools/libacpi/build.c
@@ -550,6 +550,11 @@ static int new_vm_gid(struct acpi_ctxt *ctxt,
     return 1;
 }
 
+uint32_t acpi_get_table_size(struct acpi_header * header)
+{
+    return header ? header->length : 0;
+}
+
 int acpi_build_tables(struct acpi_ctxt *ctxt, struct acpi_config *config)
 {
     struct acpi_info *acpi_info;
diff --git a/tools/libacpi/libacpi.h b/tools/libacpi/libacpi.h
index ee08c45..0882729 100644
--- a/tools/libacpi/libacpi.h
+++ b/tools/libacpi/libacpi.h
@@ -108,6 +108,7 @@ struct acpi_config {
 #define DMAR_X2APIC_OPT_OUT 0x2
 struct acpi_dmar *construct_dmar(struct acpi_ctxt *ctxt,
                                  const struct acpi_config *config);
+uint32_t acpi_get_table_size(struct acpi_header * header);
 int acpi_build_tables(struct acpi_ctxt *ctxt, struct acpi_config *config);
 
 #endif /* __LIBACPI_H__ */
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index d519c8d..99132d0 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -19,6 +19,7 @@
 
 #include "libxl_internal.h"
 #include "libxl_arch.h"
+#include "libacpi/libacpi.h"
 
 #include <xc_dom.h>
 #include <xen/hvm/hvm_info_table.h>
@@ -908,6 +909,43 @@ out:
     return rc;
 }
 
+static unsigned long acpi_v2p(struct acpi_ctxt *ctxt, void *v)
+{
+    return (unsigned long)v;
+}
+
+static void *acpi_mem_alloc(struct acpi_ctxt *ctxt,
+                            uint32_t size, uint32_t align)
+{
+    return aligned_alloc(align, size);
+}
+
+static void acpi_mem_free(struct acpi_ctxt *ctxt,
+                          void *v, uint32_t size)
+{
+    /* ACPI builder currently doesn't free memory so this is just a stub */
+}
+
+static int libxl__acpi_build_dmar(libxl__gc *gc,
+                                  struct acpi_config *config,
+                                  void **data_r, int *datalen_r)
+{
+    struct acpi_ctxt ctxt;
+    void *table;
+
+    ctxt.mem_ops.alloc = acpi_mem_alloc;
+    ctxt.mem_ops.free = acpi_mem_free;
+    ctxt.mem_ops.v2p = acpi_v2p;
+
+    table = construct_dmar(&ctxt, config);
+    if ( !table )
+        return ERROR_FAIL;
+
+    *data_r = table;
+    *datalen_r = acpi_get_table_size((struct acpi_header *)table);
+    return 0;
+}
+
 static int libxl__domain_firmware(libxl__gc *gc,
                                   libxl_domain_build_info *info,
                                   struct xc_dom_image *dom)
@@ -1028,6 +1066,53 @@ static int libxl__domain_firmware(libxl__gc *gc,
         }
     }
 
+    /* build DMAR table according guest configuration and joint it with other
+     * apci tables specified by acpi_modules */
+    if (!libxl_defbool_is_default(info->u.hvm.viommu.intremap) &&
+        info->device_model_version == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {
+        struct acpi_config config;
+
+        memset(&config, 0, sizeof(config));
+        if (libxl_defbool_val(info->u.hvm.viommu.intremap)) {
+            config.table_flags |= ACPI_HAS_DMAR;
+            config.dmar_flag = DMAR_INTR_REMAP;
+            if (!libxl_defbool_is_default(info->u.hvm.viommu.x2apic_opt_out)
+                && libxl_defbool_val(info->u.hvm.viommu.x2apic_opt_out))
+                config.dmar_flag |= DMAR_X2APIC_OPT_OUT;
+
+            config.viommu_base_addr = info->u.hvm.viommu.base_addr;
+            data = NULL;
+            e = libxl__acpi_build_dmar(gc, &config, &data, &datalen);
+            if (e) {
+                LOGE(ERROR, "failed to build DMAR table");
+                rc = ERROR_FAIL;
+                goto out;
+            }
+
+            libxl__ptr_add(gc, data);
+            if (datalen) {
+                if (!dom->acpi_modules[0].data) {
+                    dom->acpi_modules[0].data = data;
+                    dom->acpi_modules[0].length = (uint32_t)datalen;
+                } else {
+                    /* joint tables */
+                    void *newdata;
+                    newdata = malloc(datalen + dom->acpi_modules[0].length);
+                    if (!newdata) {
+                        LOGE(ERROR, "failed to joint DMAR table to acpi modules");
+                        rc = ERROR_FAIL;
+                        goto out;
+                    }
+                    memcpy(newdata, dom->acpi_modules[0].data,
+                           dom->acpi_modules[0].length);
+                    memcpy(newdata + dom->acpi_modules[0].length, data, datalen);
+                    dom->acpi_modules[0].data = newdata;
+                    dom->acpi_modules[0].length += (uint32_t)datalen;
+                }
+            }
+        }
+    }
+
     return 0;
 out:
     assert(rc != 0);
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index a612d1f..912582a 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -440,6 +440,13 @@ libxl_rdm_reserve = Struct("rdm_reserve", [
     ("policy",      libxl_rdm_reserve_policy),
     ])
 
+libxl_viommu_info = Struct("viommu_info", [
+    ("intremap",        libxl_defbool),
+    ("x2apic_opt_out",  libxl_defbool),
+    ("cap",             uint64),
+    ("base_addr",       uint64),
+    ])
+
 libxl_domain_build_info = Struct("domain_build_info",[
     ("max_vcpus",       integer),
     ("avail_vcpus",     libxl_bitmap),
@@ -550,6 +557,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
                                        ("serial_list",      libxl_string_list),
                                        ("rdm", libxl_rdm_reserve),
                                        ("rdm_mem_boundary_memkb", MemKB),
+                                       ("viommu",           libxl_viommu_info),
                                        ])),
                  ("pv", Struct(None, [("kernel", string),
                                       ("slack_memkb", MemKB),
diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c
index 1ef0c27..9349367 100644
--- a/tools/xl/xl_parse.c
+++ b/tools/xl/xl_parse.c
@@ -18,6 +18,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <xen/hvm/e820.h>
+#include <xen/viommu.h>
 
 #include <libxl.h>
 #include <libxl_utils.h>
@@ -29,6 +30,8 @@
 
 extern void set_default_nic_values(libxl_device_nic *nic);
 
+#define VIOMMU_BASE_ADDR 0xfed90000
+
 #define ARRAY_EXTEND_INIT__CORE(array,count,initfn,more)                \
     ({                                                                  \
         typeof((count)) array_extend_old_count = (count);               \
@@ -707,6 +710,25 @@ int parse_usbdev_config(libxl_device_usbdev *usbdev, char *token)
     return 0;
 }
 
+/* Parses viommu data and adds info into viommu
+ * Returns 1 if the input token does not match one of the keys
+ * or parsed values are not correct. Successful parse returns 0 */
+static int parse_viommu_config(libxl_viommu_info *viommu, char *token)
+{
+    char *oparg;
+
+    if (MATCH_OPTION("intremap", token, oparg)) {
+        libxl_defbool_set(&viommu->intremap, !!strtoul(oparg, NULL, 0));
+    } else if (MATCH_OPTION("x2apic_opt_out", token, oparg)) {
+        libxl_defbool_set(&viommu->x2apic_opt_out, !!strtoul(oparg, NULL, 0));
+    } else {
+        fprintf(stderr, "Unknown string `%s' in viommu spec\n", token);
+        return 1;
+    }
+
+    return 0;
+}
+
 void parse_config_data(const char *config_source,
                        const char *config_data,
                        int config_len,
@@ -1084,6 +1106,38 @@ void parse_config_data(const char *config_source,
 
         if (!xlu_cfg_get_long (config, "rdm_mem_boundary", &l, 0))
             b_info->u.hvm.rdm_mem_boundary_memkb = l * 1024;
+
+        if (!xlu_cfg_get_string(config, "viommu_info", &buf, 0)) {
+            libxl_viommu_info viommu;
+            char *p, *str2;
+
+            str2 = strdup(buf);
+            if (!str2) {
+                fprintf(stderr, "ERROR: strdup failed\n");
+                exit (1);
+            }
+            p = strtok(str2, ",");
+            if (!p) {
+                fprintf(stderr, "ERROR: invalid viommu_info format\n");
+                exit (1);
+            }
+            do {
+                if (*p == ' ')
+                    p++;
+                if (parse_viommu_config(&viommu, p)) {
+                    fprintf(stderr, "ERROR: invalid viommu settting\n");
+                    exit (1);
+                }
+            } while ((p=strtok(NULL, ",")) != NULL);
+            free(str2);
+            b_info->u.hvm.viommu.intremap = viommu.intremap;
+            b_info->u.hvm.viommu.x2apic_opt_out = viommu.x2apic_opt_out;
+            if ( libxl_defbool_val(b_info->u.hvm.viommu.intremap) )
+            {
+                b_info->u.hvm.viommu.cap = VIOMMU_CAP_IRQ_REMAPPING;
+                b_info->u.hvm.viommu.base_addr = VIOMMU_BASE_ADDR;
+            }
+        }
         break;
     case LIBXL_DOMAIN_TYPE_PV:
     {
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 9/23] Tools/libxl: Inform device model to create a guest with a vIOMMU device

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

A new device, xen_viommu, is added to qemu. In xen side, we can inform device
model to create a guest with a vIOMMU device through "-device xen_viommu" when
a viommu has been configurated in guest configuration file. Some specific
parameters are passed through xenstore. Qemu will create a dummy device to
preserve the address range and the dummy device will launch a viommu creation
and destruction throuth libxc interface.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libxl/libxl_create.c | 12 +++++++++++-
 tools/libxl/libxl_dm.c     |  9 +++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index e741b9a..7954186 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -453,7 +453,7 @@ int libxl__domain_build(libxl__gc *gc,
         vments[4] = "start_time";
         vments[5] = GCSPRINTF("%lu.%02d", start_time.tv_sec,(int)start_time.tv_usec/10000);
 
-        localents = libxl__calloc(gc, 9, sizeof(char *));
+        localents = libxl__calloc(gc, 11, sizeof(char *));
         i = 0;
         localents[i++] = "platform/acpi";
         localents[i++] = libxl__acpi_defbool_val(info) ? "1" : "0";
@@ -472,6 +472,11 @@ int libxl__domain_build(libxl__gc *gc,
                                    info->u.hvm.mmio_hole_memkb << 10);
             }
         }
+        if (info->u.hvm.viommu.base_addr) {
+            localents[i++] = "viommu/base_addr";
+            localents[i++] =
+                GCSPRINTF("%"PRIu64, info->u.hvm.viommu.base_addr);
+        }
 
         break;
     case LIBXL_DOMAIN_TYPE_PV:
@@ -680,6 +685,11 @@ retry_transaction:
                     GCSPRINTF("%s/attr", dom_path),
                     rwperm, ARRAY_SIZE(rwperm));
 
+    if (info->type == LIBXL_DOMAIN_TYPE_HVM)
+        libxl__xs_mknod(gc, t,
+                        GCSPRINTF("%s/viommu", dom_path),
+                        noperm, ARRAY_SIZE(noperm));
+
     if (libxl_defbool_val(info->driver_domain)) {
         /*
          * Create a local "libxl" directory for each guest, since we might want
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 281058d..f7fb81e 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -1252,6 +1252,15 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
             flexarray_append(dm_args, "-net");
             flexarray_append(dm_args, "none");
         }
+
+        if ( !libxl_defbool_is_default(b_info->u.hvm.viommu.intremap)
+             && libxl_defbool_val(b_info->u.hvm.viommu.intremap) )
+        {
+            flexarray_append(dm_args, "-device");
+            flexarray_append(dm_args,
+                             GCSPRINTF("xen_viommu,cap=%ld",
+                                       b_info->u.hvm.viommu.cap));
+        }
     } else {
         if (!sdl && !vnc) {
             flexarray_append(dm_args, "-nographic");
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 10/23] x86/hvm: Introduce a emulated VTD for HVM

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Only add create/destroy function for the emulated VTD in this patch and adapt
it to common VIOMMU layer. we will add more sub-feature of this emulated VTD.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/Makefile           |   1 +
 xen/arch/x86/hvm/vvtd.c             | 173 ++++++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h | 102 ++++++++++++++++-----
 xen/include/asm-x86/viommu.h        |   9 ++
 4 files changed, 262 insertions(+), 23 deletions(-)
 create mode 100644 xen/arch/x86/hvm/vvtd.c

diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index ec0daae..3a6ce50 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -21,6 +21,7 @@ obj-y += rtc.o
 obj-y += save.o
 obj-y += stdvga.o
 obj-y += vioapic.o
+obj-y += vvtd.o
 obj-y += viridian.o
 obj-y += vlapic.o
 obj-y += vmsi.o
diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
new file mode 100644
index 0000000..13842b9
--- /dev/null
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -0,0 +1,173 @@
+/*
+ * vvtd.c
+ *
+ * virtualize VTD for HVM.
+ *
+ * Copyright (C) 2017 Chao Gao, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms and conditions of the GNU General Public
+ * License, version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/domain_page.h>
+#include <xen/sched.h>
+#include <xen/types.h>
+#include <xen/viommu.h>
+#include <xen/xmalloc.h>
+#include <asm/current.h>
+#include <asm/hvm/domain.h>
+#include <asm/page.h>
+#include <public/viommu.h>
+
+#include "../../../drivers/passthrough/vtd/iommu.h"
+
+struct hvm_hw_vvtd_regs {
+    uint8_t data[1024];
+};
+
+/* Status field of struct vvtd */
+#define VIOMMU_STATUS_IRQ_REMAPPING_ENABLED     (1 << 0)
+#define VIOMMU_STATUS_DMA_REMAPPING_ENABLED     (1 << 1)
+
+struct vvtd {
+    /* VIOMMU_STATUS_XXX_REMAPPING_ENABLED */
+    int status;
+    /* Base address of remapping hardware register-set */
+    uint64_t base_addr;
+    /* Point back to the owner domain */
+    struct domain *domain;
+    struct hvm_hw_vvtd_regs *regs;
+    struct page_info *regs_page;
+};
+
+static inline void vvtd_set_reg(struct vvtd *vtd, uint32_t reg,
+                                uint32_t value)
+{
+    *((uint32_t *)(&vtd->regs->data[reg])) = value;
+}
+
+static inline uint32_t vvtd_get_reg(struct vvtd *vtd, uint32_t reg)
+{
+    return *((uint32_t *)(&vtd->regs->data[reg]));
+}
+
+static inline uint8_t vvtd_get_reg_byte(struct vvtd *vtd, uint32_t reg)
+{
+    return *((uint8_t *)(&vtd->regs->data[reg]));
+}
+
+#define vvtd_get_reg_quad(vvtd, reg, val) do { \
+    (val) = vvtd_get_reg(vvtd, (reg) + 4 ); \
+    (val) = (val) << 32; \
+    (val) += vvtd_get_reg(vvtd, reg); \
+} while(0)
+#define vvtd_set_reg_quad(vvtd, reg, val) do { \
+    vvtd_set_reg(vvtd, reg, (uint32_t)((val) & 0xffffffff)); \
+    vvtd_set_reg(vvtd, (reg) + 4, (uint32_t)((val) >> 32)); \
+} while(0)
+
+static void vvtd_reset(struct vvtd *vvtd, uint64_t capability)
+{
+    uint64_t cap, ecap;
+
+    cap = DMA_CAP_NFR | DMA_CAP_SLLPS | DMA_CAP_FRO | \
+          DMA_CAP_MGAW | DMA_CAP_SAGAW | DMA_CAP_ND;
+    ecap = DMA_ECAP_IR | DMA_ECAP_EIM | DMA_ECAP_QI;
+    vvtd_set_reg(vvtd, DMAR_VER_REG, 0x10UL);
+    vvtd_set_reg_quad(vvtd, DMAR_CAP_REG, cap);
+    vvtd_set_reg_quad(vvtd, DMAR_ECAP_REG, ecap);
+    vvtd_set_reg(vvtd, DMAR_GCMD_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_GSTS_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_RTADDR_REG, 0);
+    vvtd_set_reg_quad(vvtd, DMAR_CCMD_REG, 0x0ULL);
+    vvtd_set_reg(vvtd, DMAR_FSTS_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_FECTL_REG, 0x80000000UL);
+    vvtd_set_reg(vvtd, DMAR_FEDATA_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_FEADDR_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_FEUADDR_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_PMEN_REG, 0);
+    vvtd_set_reg_quad(vvtd, DMAR_IQH_REG, 0x0ULL);
+    vvtd_set_reg_quad(vvtd, DMAR_IQT_REG, 0x0ULL);
+    vvtd_set_reg_quad(vvtd, DMAR_IQA_REG, 0x0ULL);
+    vvtd_set_reg(vvtd, DMAR_ICS_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_IECTL_REG, 0x80000000UL);
+    vvtd_set_reg(vvtd, DMAR_IEDATA_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_IEADDR_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_IEUADDR_REG, 0);
+    vvtd_set_reg(vvtd, DMAR_IRTA_REG, 0);
+}
+
+static struct vvtd *__vvtd_create(struct domain *d,
+                                  uint64_t base_addr,
+                                  uint64_t cap)
+{
+    struct vvtd *vvtd;
+
+    if ( !is_hvm_domain(d) )
+        return 0;
+
+    vvtd = xmalloc_bytes(sizeof(struct vvtd));
+    if ( vvtd == NULL )
+        return NULL;
+
+    vvtd->regs_page = alloc_domheap_page(d, MEMF_no_owner);
+    if ( vvtd->regs_page == NULL )
+        goto out1;
+
+    vvtd->regs = __map_domain_page_global(vvtd->regs_page);
+    if ( vvtd->regs == NULL )
+        goto out2;
+    clear_page(vvtd->regs);
+
+    vvtd_reset(vvtd, cap);
+    vvtd->base_addr = base_addr;
+    vvtd->domain = d;
+    vvtd->status = 0;
+    return vvtd;
+
+out2:
+    free_domheap_page(vvtd->regs_page);
+out1:
+    xfree(vvtd);
+    return NULL;
+}
+
+static void __vvtd_destroy(struct vvtd *vvtd)
+{
+    unmap_domain_page_global(vvtd->regs);
+    free_domheap_page(vvtd->regs_page);
+    xfree(vvtd);
+}
+
+static u64 vvtd_query_caps(struct domain *d)
+{
+    return VIOMMU_CAP_IRQ_REMAPPING;
+}
+
+static int vvtd_create(struct domain *d, struct viommu *viommu)
+{
+    viommu->priv = (void *)__vvtd_create(d, viommu->base_address, viommu->caps);
+    return viommu->priv ? 0 : -ENOMEM;
+}
+
+static int vvtd_destroy(struct viommu *viommu)
+{
+    if ( viommu->priv )
+        __vvtd_destroy(viommu->priv);
+    return 0;
+}
+
+struct viommu_ops vvtd_hvm_vmx_ops = {
+    .query_caps = vvtd_query_caps,
+    .create = vvtd_create,
+    .destroy = vvtd_destroy
+};
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index 72c1a2e..2e9dcaa 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -23,31 +23,54 @@
 #include <asm/msi.h>
 
 /*
- * Intel IOMMU register specification per version 1.0 public spec.
+ * Intel IOMMU register specification per version 2.4 public spec.
  */
 
-#define    DMAR_VER_REG    0x0    /* Arch version supported by this IOMMU */
-#define    DMAR_CAP_REG    0x8    /* Hardware supported capabilities */
-#define    DMAR_ECAP_REG    0x10    /* Extended capabilities supported */
-#define    DMAR_GCMD_REG    0x18    /* Global command register */
-#define    DMAR_GSTS_REG    0x1c    /* Global status register */
-#define    DMAR_RTADDR_REG    0x20    /* Root entry table */
-#define    DMAR_CCMD_REG    0x28    /* Context command reg */
-#define    DMAR_FSTS_REG    0x34    /* Fault Status register */
-#define    DMAR_FECTL_REG    0x38    /* Fault control register */
-#define    DMAR_FEDATA_REG    0x3c    /* Fault event interrupt data register */
-#define    DMAR_FEADDR_REG    0x40    /* Fault event interrupt addr register */
-#define    DMAR_FEUADDR_REG 0x44    /* Upper address register */
-#define    DMAR_AFLOG_REG    0x58    /* Advanced Fault control */
-#define    DMAR_PMEN_REG    0x64    /* Enable Protected Memory Region */
-#define    DMAR_PLMBASE_REG 0x68    /* PMRR Low addr */
-#define    DMAR_PLMLIMIT_REG 0x6c    /* PMRR low limit */
-#define    DMAR_PHMBASE_REG 0x70    /* pmrr high base addr */
-#define    DMAR_PHMLIMIT_REG 0x78    /* pmrr high limit */
-#define    DMAR_IQH_REG    0x80    /* invalidation queue head */
-#define    DMAR_IQT_REG    0x88    /* invalidation queue tail */
-#define    DMAR_IQA_REG    0x90    /* invalidation queue addr */
-#define    DMAR_IRTA_REG   0xB8    /* intr remap */
+#define DMAR_VER_REG            0x0  /* Arch version supported by this IOMMU */
+#define DMAR_CAP_REG            0x8  /* Hardware supported capabilities */
+#define DMAR_ECAP_REG           0x10 /* Extended capabilities supported */
+#define DMAR_GCMD_REG           0x18 /* Global command register */
+#define DMAR_GSTS_REG           0x1c /* Global status register */
+#define DMAR_RTADDR_REG         0x20 /* Root entry table */
+#define DMAR_CCMD_REG           0x28 /* Context command reg */
+#define DMAR_FSTS_REG           0x34 /* Fault Status register */
+#define DMAR_FECTL_REG          0x38 /* Fault control register */
+#define DMAR_FEDATA_REG         0x3c /* Fault event interrupt data register */
+#define DMAR_FEADDR_REG         0x40 /* Fault event interrupt addr register */
+#define DMAR_FEUADDR_REG        0x44 /* Upper address register */
+#define DMAR_AFLOG_REG          0x58 /* Advanced Fault control */
+#define DMAR_PMEN_REG           0x64 /* Enable Protected Memory Region */
+#define DMAR_PLMBASE_REG        0x68 /* PMRR Low addr */
+#define DMAR_PLMLIMIT_REG       0x6c /* PMRR low limit */
+#define DMAR_PHMBASE_REG        0x70 /* pmrr high base addr */
+#define DMAR_PHMLIMIT_REG       0x78 /* pmrr high limit */
+#define DMAR_IQH_REG            0x80 /* invalidation queue head */
+#define DMAR_IQT_REG            0x88 /* invalidation queue tail */
+#define DMAR_IQT_REG_HI         0x8c
+#define DMAR_IQA_REG            0x90 /* invalidation queue addr */
+#define DMAR_IQA_REG_HI         0x94
+#define DMAR_ICS_REG            0x9c /* Invalidation complete status */
+#define DMAR_IECTL_REG          0xa0 /* Invalidation event control */
+#define DMAR_IEDATA_REG         0xa4 /* Invalidation event data */
+#define DMAR_IEADDR_REG         0xa8 /* Invalidation event address */
+#define DMAR_IEUADDR_REG        0xac /* Invalidation event address */
+#define DMAR_IRTA_REG           0xb8 /* Interrupt remapping table addr */
+#define DMAR_IRTA_REG_HI        0xbc
+#define DMAR_PQH_REG            0xc0 /* Page request queue head */
+#define DMAR_PQH_REG_HI         0xc4
+#define DMAR_PQT_REG            0xc8 /* Page request queue tail*/
+#define DMAR_PQT_REG_HI         0xcc
+#define DMAR_PQA_REG            0xd0 /* Page request queue address */
+#define DMAR_PQA_REG_HI         0xd4
+#define DMAR_PRS_REG            0xdc /* Page request status */
+#define DMAR_PECTL_REG          0xe0 /* Page request event control */
+#define DMAR_PEDATA_REG         0xe4 /* Page request event data */
+#define DMAR_PEADDR_REG         0xe8 /* Page request event address */
+#define DMAR_PEUADDR_REG        0xec /* Page event upper address */
+#define DMAR_MTRRCAP_REG        0x100 /* MTRR capability */
+#define DMAR_MTRRCAP_REG_HI     0x104
+#define DMAR_MTRRDEF_REG        0x108 /* MTRR default type */
+#define DMAR_MTRRDEF_REG_HI     0x10c
 
 #define OFFSET_STRIDE        (9)
 #define dmar_readl(dmar, reg) readl((dmar) + (reg))
@@ -58,6 +81,31 @@
 #define VER_MAJOR(v)        (((v) & 0xf0) >> 4)
 #define VER_MINOR(v)        ((v) & 0x0f)
 
+/* CAP_REG */
+/* (offset >> 4) << 24 */
+#define DMA_DOMAIN_ID_SHIFT         16  /* 16-bit domain id for 64K domains */
+#define DMA_DOMAIN_ID_MASK          ((1UL << DMA_DOMAIN_ID_SHIFT) - 1)
+#define DMA_CAP_ND                  (((DMA_DOMAIN_ID_SHIFT - 4) / 2) & 7ULL)
+#define DMA_MGAW                    39  /* Maximum Guest Address Width */
+#define DMA_CAP_MGAW                (((DMA_MGAW - 1) & 0x3fULL) << 16)
+#define DMA_MAMV                    18ULL
+#define DMA_CAP_MAMV                (DMA_MAMV << 48)
+#define DMA_CAP_PSI                 (1ULL << 39)
+#define DMA_CAP_SLLPS               ((1ULL << 34) | (1ULL << 35))
+#define DMAR_FRCD_REG_NR            1ULL
+#define DMA_CAP_FRO_OFFSET          0x220ULL
+#define DMA_CAP_FRO                 (DMA_CAP_FRO_OFFSET << 20)
+#define DMA_CAP_NFR                 ((DMAR_FRCD_REG_NR - 1) << 40)
+
+/* Supported Adjusted Guest Address Widths */
+#define DMA_CAP_SAGAW_SHIFT         8
+#define DMA_CAP_SAGAW_MASK          (0x1fULL << DMA_CAP_SAGAW_SHIFT)
+ /* 39-bit AGAW, 3-level page-table */
+#define DMA_CAP_SAGAW_39bit         (0x2ULL << DMA_CAP_SAGAW_SHIFT)
+ /* 48-bit AGAW, 4-level page-table */
+#define DMA_CAP_SAGAW_48bit         (0x4ULL << DMA_CAP_SAGAW_SHIFT)
+#define DMA_CAP_SAGAW               DMA_CAP_SAGAW_39bit
+
 /*
  * Decoding Capability Register
  */
@@ -89,6 +137,14 @@
 #define cap_afl(c)        (((c) >> 3) & 1)
 #define cap_ndoms(c)        (1 << (4 + 2 * ((c) & 0x7)))
 
+/* ECAP_REG */
+/* (offset >> 4) << 8 */
+#define DMA_ECAP_QI                 (1ULL << 1)
+/* Interrupt Remapping support */
+#define DMA_ECAP_IR                 (1ULL << 3)
+#define DMA_ECAP_EIM                (1ULL << 4)
+#define DMA_ECAP_MHMV               (15ULL << 20)
+
 /*
  * Extended Capability Register
  */
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
index 43e446e..0b25f34 100644
--- a/xen/include/asm-x86/viommu.h
+++ b/xen/include/asm-x86/viommu.h
@@ -22,6 +22,9 @@
 
 #include <xen/viommu.h>
 #include <asm/types.h>
+#include <asm/processor.h>
+
+extern struct viommu_ops vvtd_hvm_vmx_ops;
 
 struct irq_remapping_info
 {
@@ -48,6 +51,12 @@ struct irq_remapping_request
 
 static inline const struct viommu_ops *viommu_get_ops(void)
 {
+    /*
+     * Don't mean that viommu relies on hardward cpu vendor, but just
+     * limit the usage to minimize the impact to existing code.
+     */
+    if ( boot_cpu_data.x86_vendor == X86_VENDOR_INTEL )
+        return &vvtd_hvm_vmx_ops;
     return NULL;
 }
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 11/23] X86/vvtd: Add MMIO handler for VVTD

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

This patch is to add VVTD MMIO handler to deal with MMIO access.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c | 135 ++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 135 insertions(+)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index 13842b9..9473fe0 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -49,6 +49,38 @@ struct vvtd {
     struct page_info *regs_page;
 };
 
+#define __DEBUG_VVTD__
+#ifdef __DEBUG_VVTD__
+extern unsigned int vvtd_debug_level;
+#define VVTD_DBG_INFO     1
+#define VVTD_DBG_TRANS    (1<<1)
+#define VVTD_DBG_RW       (1<<2)
+#define VVTD_DBG_FAULT    (1<<3)
+#define VVTD_DBG_EOI      (1<<4)
+#define VVTD_DEBUG(lvl, _f, _a...) do { \
+    if ( vvtd_debug_level & lvl ) \
+    printk("VVTD %s:" _f "\n", __func__, ## _a);    \
+} while(0)
+#else
+#define VVTD_DEBUG(fmt...) do {} while(0)
+#endif
+
+unsigned int vvtd_debug_level __read_mostly;
+integer_param("vvtd_debug", vvtd_debug_level);
+
+struct vvtd *domain_vvtd(struct domain *d)
+{
+    struct viommu_info *info = &d->viommu;
+
+    BUILD_BUG_ON(NR_VIOMMU_PER_DOMAIN != 1);
+    return (info && info->viommu[0]) ? info->viommu[0]->priv : NULL;
+}
+
+static inline struct vvtd *vcpu_vvtd(struct vcpu *v)
+{
+    return domain_vvtd(v->domain);
+}
+
 static inline void vvtd_set_reg(struct vvtd *vtd, uint32_t reg,
                                 uint32_t value)
 {
@@ -75,6 +107,108 @@ static inline uint8_t vvtd_get_reg_byte(struct vvtd *vtd, uint32_t reg)
     vvtd_set_reg(vvtd, (reg) + 4, (uint32_t)((val) >> 32)); \
 } while(0)
 
+static int vvtd_range(struct vcpu *v, unsigned long addr)
+{
+    struct vvtd *vvtd = vcpu_vvtd(v);
+
+    if ( vvtd )
+        return (addr >= vvtd->base_addr) &&
+               (addr < vvtd->base_addr + PAGE_SIZE);
+    return 0;
+}
+
+static int vvtd_read(struct vcpu *v, unsigned long addr,
+                     unsigned int len, unsigned long *pval)
+{
+    struct vvtd *vvtd = vcpu_vvtd(v);
+    unsigned int offset = addr - vvtd->base_addr;
+    unsigned int offset_aligned = offset & ~3;
+
+    if ( !pval )
+        return X86EMUL_OKAY;
+
+    VVTD_DEBUG(VVTD_DBG_RW, "READ INFO: offset %x len %d.", offset, len);
+
+    if ( len != 1 && (offset & 3) != 0 )
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "Alignment is not canonical.");
+        return X86EMUL_OKAY;
+    }
+
+    switch( len )
+    {
+    case 1:
+        *pval = vvtd_get_reg_byte(vvtd, offset);
+        break;
+
+    case 4:
+        *pval = vvtd_get_reg(vvtd, offset_aligned);
+        break;
+
+    case 8:
+        vvtd_get_reg_quad(vvtd, offset_aligned, *pval);
+        break;
+
+    default:
+        break;
+    }
+
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write(struct vcpu *v, unsigned long addr,
+                      unsigned int len, unsigned long val)
+{
+    struct vvtd *vvtd = vcpu_vvtd(v);
+    unsigned int offset = addr - vvtd->base_addr;
+    unsigned int offset_aligned = offset & ~0x3;
+    unsigned long val_lo = (val & ((1ULL << 32) - 1));
+    int ret;
+
+    VVTD_DEBUG(VVTD_DBG_RW, "WRITE INFO: offset %x len %d val %lx.",
+               offset, len, val);
+
+    if ( offset & 3 )
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "Alignment is not canonical");
+        goto error;
+    }
+
+    if ( len != 4 && len != 8)
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "Len is not canonical");
+        goto error;
+    }
+
+    ret = X86EMUL_UNHANDLEABLE;
+    switch ( offset_aligned  )
+    {
+    case DMAR_IEDATA_REG:
+    case DMAR_IEADDR_REG:
+    case DMAR_IEUADDR_REG:
+    case DMAR_FEDATA_REG:
+    case DMAR_FEADDR_REG:
+    case DMAR_FEUADDR_REG:
+        if ( len == 8 )
+            goto error;
+        vvtd_set_reg(vvtd, offset_aligned, val_lo);
+        ret = X86EMUL_OKAY;
+        break;
+
+    default:
+        break;
+    }
+
+error:
+    return X86EMUL_OKAY;
+}
+
+static const struct hvm_mmio_ops vvtd_mmio_ops = {
+    .check = vvtd_range,
+    .read = vvtd_read,
+    .write = vvtd_write
+};
+
 static void vvtd_reset(struct vvtd *vvtd, uint64_t capability)
 {
     uint64_t cap, ecap;
@@ -132,6 +266,7 @@ static struct vvtd *__vvtd_create(struct domain *d,
     vvtd->base_addr = base_addr;
     vvtd->domain = d;
     vvtd->status = 0;
+    register_mmio_handler(d, &vvtd_mmio_ops);
     return vvtd;
 
 out2:
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 12/23] X86/vvtd: Set Interrupt Remapping Table Pointer through GCMD

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Software sets this field to set/update the interrupt remapping table pointer
used by hardware. The interrupt remapping table pointer is specified through
the Interrupt Remapping Table Address (IRTA_REG) register.

This patch emulates this operation and uses some fields of VVTD to track
info about interrupt remapping table.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c             | 70 +++++++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h |  9 ++++-
 2 files changed, 78 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index 9473fe0..a12b4d1 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -45,6 +45,13 @@ struct vvtd {
     uint64_t base_addr;
     /* Point back to the owner domain */
     struct domain *domain;
+    /* Is in Extended Interrupt Mode */
+    bool eim;
+    /* Interrupt remapping table base gfn */
+    uint64_t irt;
+    /* Max remapping entries in IRT */
+    int irt_max_entry;
+
     struct hvm_hw_vvtd_regs *regs;
     struct page_info *regs_page;
 };
@@ -81,6 +88,11 @@ static inline struct vvtd *vcpu_vvtd(struct vcpu *v)
     return domain_vvtd(v->domain);
 }
 
+static inline void __vvtd_set_bit(struct vvtd *vvtd, uint32_t reg, int nr)
+{
+    return __set_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
+}
+
 static inline void vvtd_set_reg(struct vvtd *vtd, uint32_t reg,
                                 uint32_t value)
 {
@@ -107,6 +119,41 @@ static inline uint8_t vvtd_get_reg_byte(struct vvtd *vtd, uint32_t reg)
     vvtd_set_reg(vvtd, (reg) + 4, (uint32_t)((val) >> 32)); \
 } while(0)
 
+static int vvtd_handle_gcmd_sirtp(struct vvtd *vvtd, unsigned long val)
+{
+    uint64_t irta;
+
+    if ( !(val & DMA_GCMD_SIRTP) )
+        return X86EMUL_OKAY;
+
+    vvtd_get_reg_quad(vvtd, DMAR_IRTA_REG, irta);
+    vvtd->irt = DMA_IRTA_ADDR(irta) >> PAGE_SHIFT;
+    vvtd->irt_max_entry = DMA_IRTA_SIZE(irta);
+    vvtd->eim = !!(irta & IRTA_EIME);
+    VVTD_DEBUG(VVTD_DBG_RW, "Update IR info (addr=%lx eim=%d size=%d).",
+               vvtd->irt, vvtd->eim, vvtd->irt_max_entry);
+    __vvtd_set_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_SIRTPS_BIT);
+
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_gcmd(struct vvtd *vvtd, unsigned long val)
+{
+    uint32_t orig = vvtd_get_reg(vvtd, DMAR_GSTS_REG);
+    uint32_t changed = orig ^ val;
+
+    if ( !changed )
+        return X86EMUL_OKAY;
+    if ( (changed & (changed - 1)) )
+        VVTD_DEBUG(VVTD_DBG_RW, "Guest attempts to update multiple fields "
+                     "of GCMD_REG in one write transation.");
+
+    if ( changed & DMA_GCMD_SIRTP )
+        vvtd_handle_gcmd_sirtp(vvtd, val);
+
+    return X86EMUL_OKAY;
+}
+
 static int vvtd_range(struct vcpu *v, unsigned long addr)
 {
     struct vvtd *vvtd = vcpu_vvtd(v);
@@ -183,6 +230,26 @@ static int vvtd_write(struct vcpu *v, unsigned long addr,
     ret = X86EMUL_UNHANDLEABLE;
     switch ( offset_aligned  )
     {
+    case DMAR_GCMD_REG:
+        if ( len == 8 )
+            goto error;
+        ret = vvtd_write_gcmd(vvtd, val_lo);
+        break;
+
+    case DMAR_IRTA_REG:
+        if ( len == 8 )
+            vvtd_set_reg_quad(vvtd, DMAR_IRTA_REG, val);
+        else
+            vvtd_set_reg(vvtd, DMAR_IRTA_REG, val_lo);
+        break;
+
+    case DMAR_IRTA_REG_HI:
+        if ( len == 8 )
+            goto error;
+        vvtd_set_reg(vvtd, DMAR_IRTA_REG_HI, val_lo);
+        ret = X86EMUL_OKAY;
+        break;
+
     case DMAR_IEDATA_REG:
     case DMAR_IEADDR_REG:
     case DMAR_IEUADDR_REG:
@@ -266,6 +333,9 @@ static struct vvtd *__vvtd_create(struct domain *d,
     vvtd->base_addr = base_addr;
     vvtd->domain = d;
     vvtd->status = 0;
+    vvtd->eim = 0;
+    vvtd->irt = 0;
+    vvtd->irt_max_entry = 0;
     register_mmio_handler(d, &vvtd_mmio_ops);
     return vvtd;
 
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index 2e9dcaa..fd040d0 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -195,9 +195,16 @@
 #define DMA_GSTS_WBFS   (((u64)1) << 27)
 #define DMA_GSTS_QIES   (((u64)1) <<26)
 #define DMA_GSTS_IRES   (((u64)1) <<25)
-#define DMA_GSTS_SIRTPS (((u64)1) << 24)
+#define DMA_GSTS_SIRTPS_BIT     24
+#define DMA_GSTS_SIRTPS (((u64)1) << DMA_GSTS_SIRTPS_BIT)
 #define DMA_GSTS_CFIS   (((u64)1) <<23)
 
+/* IRTA_REG */
+#define DMA_IRTA_ADDR(val)      (val & ~0xfffULL)
+#define DMA_IRTA_EIME(val)      (!!(val & (1 << 11)))
+#define DMA_IRTA_S(val)         (val & 0xf)
+#define DMA_IRTA_SIZE(val)      (1UL << (DMA_IRTA_S(val) + 1))
+
 /* PMEN_REG */
 #define DMA_PMEN_EPM    (((u32)1) << 31)
 #define DMA_PMEN_PRS    (((u32)1) << 0)
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 13/23] X86/vvtd: Process interrupt remapping request

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

When a remapping interrupt request arrives, remapping hardware computes the
interrupt_index per the algorithm described in VTD spec
"Interrupt Remapping Table", interprets the IRTE and generates a remapped
interrupte request.

In this patch, a new function viommu_handle_irq_request() is introduced to
emulate the process how remapping hardware handles a remapping interrupt
request.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c                | 295 ++++++++++++++++++++++++++++++++-
 xen/drivers/passthrough/vtd/iommu.h    |  15 ++
 xen/include/public/arch-x86/hvm/save.h |  18 ++
 xen/include/xen/hvm/irq.h              |   2 +
 4 files changed, 329 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index a12b4d1..c5df77d 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -23,9 +23,13 @@
 #include <xen/types.h>
 #include <xen/viommu.h>
 #include <xen/xmalloc.h>
+#include <asm/apic.h>
 #include <asm/current.h>
+#include <asm/event.h>
 #include <asm/hvm/domain.h>
+#include <asm/io_apic.h>
 #include <asm/page.h>
+#include <asm/p2m.h>
 #include <public/viommu.h>
 
 #include "../../../drivers/passthrough/vtd/iommu.h"
@@ -38,6 +42,9 @@ struct hvm_hw_vvtd_regs {
 #define VIOMMU_STATUS_IRQ_REMAPPING_ENABLED     (1 << 0)
 #define VIOMMU_STATUS_DMA_REMAPPING_ENABLED     (1 << 1)
 
+#define vvtd_irq_remapping_enabled(vvtd) \
+            (vvtd->status & VIOMMU_STATUS_IRQ_REMAPPING_ENABLED)
+
 struct vvtd {
     /* VIOMMU_STATUS_XXX_REMAPPING_ENABLED */
     int status;
@@ -119,6 +126,138 @@ static inline uint8_t vvtd_get_reg_byte(struct vvtd *vtd, uint32_t reg)
     vvtd_set_reg(vvtd, (reg) + 4, (uint32_t)((val) >> 32)); \
 } while(0)
 
+static int map_guest_page(struct domain *d, uint64_t gfn, void **virt)
+{
+    struct page_info *p;
+
+    p = get_page_from_gfn(d, gfn, NULL, P2M_ALLOC);
+    if ( !p )
+        return -EINVAL;
+
+    if ( !get_page_type(p, PGT_writable_page) )
+    {
+        put_page(p);
+        return -EINVAL;
+    }
+
+    *virt = __map_domain_page_global(p);
+    if ( !*virt )
+    {
+        put_page_and_type(p);
+        return -ENOMEM;
+    }
+    return 0;
+}
+
+static void unmap_guest_page(void *virt)
+{
+    struct page_info *page;
+
+    if ( !virt )
+        return;
+
+    virt = (void *)((unsigned long)virt & PAGE_MASK);
+    page = mfn_to_page(domain_page_map_to_mfn(virt));
+
+    unmap_domain_page_global(virt);
+    put_page_and_type(page);
+}
+
+static void vvtd_inj_irq(
+    struct vlapic *target,
+    uint8_t vector,
+    uint8_t trig_mode,
+    uint8_t delivery_mode)
+{
+    VVTD_DEBUG(VVTD_DBG_INFO, "dest=v%d, delivery_mode=%x vector=%d "
+               "trig_mode=%d.",
+               vlapic_vcpu(target)->vcpu_id, delivery_mode,
+               vector, trig_mode);
+
+    ASSERT((delivery_mode == dest_Fixed) ||
+           (delivery_mode == dest_LowestPrio));
+
+    vlapic_set_irq(target, vector, trig_mode);
+}
+
+static int vvtd_delivery(
+    struct domain *d, int vector,
+    uint32_t dest, uint8_t dest_mode,
+    uint8_t delivery_mode, uint8_t trig_mode)
+{
+    struct vlapic *target;
+    struct vcpu *v;
+
+    switch ( delivery_mode )
+    {
+    case dest_LowestPrio:
+        target = vlapic_lowest_prio(d, NULL, 0, dest, dest_mode);
+        if ( target != NULL )
+        {
+            vvtd_inj_irq(target, vector, trig_mode, delivery_mode);
+            break;
+        }
+        VVTD_DEBUG(VVTD_DBG_INFO, "null round robin: vector=%02x\n", vector);
+        break;
+
+    case dest_Fixed:
+        for_each_vcpu ( d, v )
+            if ( vlapic_match_dest(vcpu_vlapic(v), NULL, 0, dest,
+                                   dest_mode) )
+                vvtd_inj_irq(vcpu_vlapic(v), vector,
+                             trig_mode, delivery_mode);
+        break;
+
+    case dest_NMI:
+        for_each_vcpu ( d, v )
+            if ( vlapic_match_dest(vcpu_vlapic(v), NULL, 0, dest, dest_mode)
+                 && !test_and_set_bool(v->nmi_pending) )
+                vcpu_kick(v);
+        break;
+
+    default:
+        printk(XENLOG_G_WARNING
+               "%pv: Unsupported VTD delivery mode %d for Dom%d\n",
+               current, delivery_mode, d->domain_id);
+        return -EINVAL;
+    }
+
+    return 0;
+}
+
+static uint32_t irq_remapping_request_index(struct irq_remapping_request *irq)
+{
+    switch ( irq->type )
+    {
+    case VIOMMU_REQUEST_IRQ_MSI:
+        return IR_MSI_INDEX(irq->msg.msi.data, irq->msg.msi.addr);
+
+    case VIOMMU_REQUEST_IRQ_APIC:
+        return IR_IOAPIC_RTE_INDEX((struct ir_ioapic_rte *)&irq->msg.rte);
+
+    default:
+        break;
+    }
+    BUG();
+    return 0;
+}
+
+static inline uint32_t irte_dest(struct vvtd *vvtd,
+                                 struct iremap_entry *irte)
+{
+    uint64_t irta;
+
+    /* In xAPIC mode, only 8-bits([15:8]) are valid*/
+    vvtd_get_reg_quad(vvtd, DMAR_IRTA_REG, irta);
+    return (irta & IRTA_EIME) ? irte->remap.dst : (irte->remap.dst) >> 8 & 0xff;
+}
+
+static int vvtd_log_fault(struct vvtd *vvtd, struct irq_remapping_request *irq,
+                          int reason)
+{
+    return 0;
+}
+
 static int vvtd_handle_gcmd_sirtp(struct vvtd *vvtd, unsigned long val)
 {
     uint64_t irta;
@@ -276,6 +415,159 @@ static const struct hvm_mmio_ops vvtd_mmio_ops = {
     .write = vvtd_write
 };
 
+static bool ir_sid_valid(struct iremap_entry *irte, uint32_t source_id)
+{
+    return TRUE;
+}
+
+/* @log_fault: a flag to indicate whether we need log a fault when checking
+ * vIRTE. (1 means logging it, 0 means ignoring the fault). log_fault = 0 is
+ * used in parse process in which only the encoded attributes is cared.
+ */
+static int vvtd_get_entry(struct vvtd *vvtd,
+                          struct irq_remapping_request *irq,
+                          struct iremap_entry *dest,
+                          bool log_fault)
+{
+    int ret;
+    uint32_t entry = irq_remapping_request_index(irq);
+    struct iremap_entry  *irte, *irt_page;
+
+    ASSERT(entry < IREMAP_ENTRY_NR);
+
+    if ( entry > vvtd->irt_max_entry )
+    {
+        ret = VTD_FR_IR_INDEX_OVER;
+        goto handle_fault;
+    }
+
+    ret = map_guest_page(vvtd->domain, vvtd->irt + (entry >> IREMAP_ENTRY_ORDER),
+                         (void**)&irt_page);
+    if ( ret )
+    {
+        ret = VTD_FR_IR_ROOT_INVAL;
+        goto handle_fault;
+    }
+
+    irte = irt_page + (entry % (1 << IREMAP_ENTRY_ORDER));
+    dest->val = irte->val;
+    if ( !qinval_present(*irte) )
+    {
+        ret = VTD_FR_IR_ENTRY_P;
+        goto unmap_handle_fault;
+    }
+
+    /* Check reserved bits */
+    if ( (irte->remap.res_1 || irte->remap.res_2 || irte->remap.res_3 ||
+          irte->remap.res_4) )
+    {
+        ret = VTD_FR_IR_IRTE_RSVD;
+        goto unmap_handle_fault;
+    }
+
+    /* TODO:Intel64 platforms block compatibility format interrupt request */
+
+    if (!ir_sid_valid(irte, irq->source_id))
+    {
+        ret = VTD_FR_IR_SID_ERR;
+        goto unmap_handle_fault;
+    }
+    unmap_guest_page(irt_page);
+    return 0;
+
+unmap_handle_fault:
+    unmap_guest_page(irt_page);
+handle_fault:
+    if ( !log_fault )
+        goto out;
+
+    switch ( ret )
+    {
+    case VTD_FR_IR_SID_ERR:
+    case VTD_FR_IR_IRTE_RSVD:
+    case VTD_FR_IR_ENTRY_P:
+        if ( qinval_fault_disable(*irte) )
+            break;
+    /* fall through */
+    case VTD_FR_IR_INDEX_OVER:
+    case VTD_FR_IR_ROOT_INVAL:
+        vvtd_log_fault(vvtd, irq, ret);
+        break;
+
+    default:
+        gdprintk(XENLOG_G_INFO, "Can't handle VT-d fault %x\n", ret);
+        goto out;
+    }
+out:
+    return ret;
+}
+
+static int vvtd_ioapic_check(struct vvtd *vvtd,
+                             struct ir_ioapic_rte rte)
+{
+    VVTD_DEBUG(VVTD_DBG_INFO, "IOAPIC (%lx)", *(uint64_t *)&(rte));
+    ASSERT(rte.format);
+
+    if ( rte.reserved || rte.reserved1 || rte.reserved2[0] ||
+         rte.reserved2[1] || rte.reserved2[2] )
+        return VTD_FR_IR_REQ_RSVD;
+    return 0;
+}
+
+static int vvtd_msi_check(struct vvtd *vvtd,
+                          uint32_t data,
+                          uint64_t addr)
+{
+    ASSERT((addr >> 20) == 0xfee);
+    return 0;
+}
+
+static int vvtd_irq_request_sanity_check(struct vvtd *vvtd,
+                                         struct irq_remapping_request *irq)
+{
+    switch ( irq->type )
+    {
+    case VIOMMU_REQUEST_IRQ_MSI:
+        return vvtd_msi_check(vvtd, irq->msg.msi.data, irq->msg.msi.addr);
+
+    case VIOMMU_REQUEST_IRQ_APIC:
+        return vvtd_ioapic_check(vvtd, *(struct ir_ioapic_rte *)&irq->msg.rte);
+
+    default:
+        break;
+    }
+
+    BUG();
+    return 0;
+}
+
+static int vvtd_handle_irq_request(struct domain *d,
+                                   struct irq_remapping_request *irq)
+{
+    struct iremap_entry irte;
+    int ret;
+    struct vvtd *vvtd = domain_vvtd(d);
+
+    if ( !vvtd || !vvtd_irq_remapping_enabled(vvtd) )
+        return -EINVAL;
+
+    ret = vvtd_irq_request_sanity_check(vvtd, irq);
+    if ( ret )
+    {
+        vvtd_log_fault(vvtd, irq, ret);
+        return ret;
+    }
+
+    if ( !vvtd_get_entry(vvtd, irq, &irte, 1) )
+    {
+        vvtd_delivery(vvtd->domain, irte.remap.vector,
+                      irte_dest(vvtd, &irte), irte.remap.dm,
+                      irte.remap.dlm, irte.remap.tm);
+        return 0;
+    }
+    return -EFAULT;
+}
+
 static void vvtd_reset(struct vvtd *vvtd, uint64_t capability)
 {
     uint64_t cap, ecap;
@@ -374,5 +666,6 @@ static int vvtd_destroy(struct viommu *viommu)
 struct viommu_ops vvtd_hvm_vmx_ops = {
     .query_caps = vvtd_query_caps,
     .create = vvtd_create,
-    .destroy = vvtd_destroy
+    .destroy = vvtd_destroy,
+    .handle_irq_request = vvtd_handle_irq_request
 };
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index fd040d0..aa1f1a6 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -247,6 +247,21 @@
 #define dma_frcd_source_id(c) (c & 0xffff)
 #define dma_frcd_page_addr(d) (d & (((u64)-1) << 12)) /* low 64 bit */
 
+enum VTD_FAULT_TYPE
+{
+    /* Interrupt remapping transition faults */
+    VTD_FR_IR_REQ_RSVD = 0x20,   /* One or more IR request reserved
+                                  * fields set */
+    VTD_FR_IR_INDEX_OVER = 0x21, /* Index value greater than max */
+    VTD_FR_IR_ENTRY_P = 0x22,    /* Present (P) not set in IRTE */
+    VTD_FR_IR_ROOT_INVAL = 0x23, /* IR Root table invalid */
+    VTD_FR_IR_IRTE_RSVD = 0x24,  /* IRTE Rsvd field non-zero with
+                                  * Present flag set */
+    VTD_FR_IR_REQ_COMPAT = 0x25, /* Encountered compatible IR
+                                  * request while disabled */
+    VTD_FR_IR_SID_ERR = 0x26,    /* Invalid Source-ID */
+};
+
 /*
  * 0: Present
  * 1-11: Reserved
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 419a3b2..6127f89 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -363,6 +363,24 @@ DECLARE_HVM_SAVE_TYPE(PIC, 3, struct hvm_hw_vpic);
 
 #define VIOAPIC_NUM_PINS  48 /* 16 ISA IRQs, 32 non-legacy PCI IRQS. */
 
+struct ir_ioapic_rte
+{
+    uint8_t vector;
+    uint8_t reserved:3; /* Reserved(0) */
+    uint8_t index2:1;   /* Interrupt Index [15] */
+    uint8_t delivery_status:1;
+    uint8_t polarity:1;
+    uint8_t remote_irr:1;
+    uint8_t trig_mode:1;
+    uint8_t mask:1;
+    uint8_t reserved1:7; /* Reserved(0) */
+    uint8_t reserved2[3]; /* Reserved(0) */
+    uint16_t format:1; /* Should always be 1 */
+    uint16_t index1:15; /* Interrupt Index [14:0] */
+};
+
+#define IR_IOAPIC_RTE_INDEX(rte)    (((rte)->index2 << 15) + (rte)->index1)
+
 struct hvm_hw_vioapic {
     uint64_t base_address;
     uint32_t ioregsel;
diff --git a/xen/include/xen/hvm/irq.h b/xen/include/xen/hvm/irq.h
index d3f8623..ea332bb 100644
--- a/xen/include/xen/hvm/irq.h
+++ b/xen/include/xen/hvm/irq.h
@@ -65,6 +65,8 @@ struct hvm_gmsi_info {
     int dest_vcpu_id; /* -1 :multi-dest, non-negative: dest_vcpu_id */
 };
 
+#define IR_MSI_INDEX(data, addr) (((((addr) & 0x4) << 13) + (((addr) & 0xfffff) >> 5)) + (!!((addr) & 0x8)) * ((data) & 0xffff))
+
 struct hvm_girq_dpci_mapping {
     struct list_head list;
     uint8_t bus;
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 14/23] X86/vvtd: decode interrupt attribute from IRTE

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index c5df77d..0c49294 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -568,6 +568,25 @@ static int vvtd_handle_irq_request(struct domain *d,
     return -EFAULT;
 }
 
+static int vvtd_get_irq_info(struct domain *d,
+                             struct irq_remapping_request *irq,
+                             struct irq_remapping_info *info)
+{
+    int ret;
+    struct iremap_entry irte;
+    struct vvtd *vvtd = domain_vvtd(d);
+
+    ret = vvtd_get_entry(vvtd, irq, &irte, 0);
+    if ( ret )
+        return -ret;
+
+    info->vector = irte.remap.vector;
+    info->dest = irte_dest(vvtd, &irte);
+    info->dest_mode = irte.remap.dm;
+    info->delivery_mode = irte.remap.dlm;
+    return 0;
+}
+
 static void vvtd_reset(struct vvtd *vvtd, uint64_t capability)
 {
     uint64_t cap, ecap;
@@ -667,5 +686,6 @@ struct viommu_ops vvtd_hvm_vmx_ops = {
     .query_caps = vvtd_query_caps,
     .create = vvtd_create,
     .destroy = vvtd_destroy,
-    .handle_irq_request = vvtd_handle_irq_request
+    .handle_irq_request = vvtd_handle_irq_request,
+    .get_irq_info = vvtd_get_irq_info
 };
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 15/23] X86/vioapic: Hook interrupt delivery of vIOAPIC

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

When irq remapping enabled, IOAPIC Redirection Entry maybe is in remapping
format. If that, generate a irq_remapping_request and send it to domain.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/Makefile                  |  1 +
 xen/arch/x86/hvm/vioapic.c             | 10 ++++++++++
 xen/arch/x86/viommu.c                  | 30 ++++++++++++++++++++++++++++++
 xen/include/asm-x86/viommu.h           |  3 +++
 xen/include/public/arch-x86/hvm/save.h |  1 +
 5 files changed, 45 insertions(+)
 create mode 100644 xen/arch/x86/viommu.c

diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
index f75eca0..d49f8c8 100644
--- a/xen/arch/x86/Makefile
+++ b/xen/arch/x86/Makefile
@@ -66,6 +66,7 @@ obj-y += usercopy.o
 obj-y += x86_emulate.o
 obj-$(CONFIG_TBOOT) += tboot.o
 obj-y += hpet.o
+obj-y += viommu.o
 obj-y += vm_event.o
 obj-y += xstate.o
 
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index fdbb21f..6a00644 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -30,6 +30,7 @@
 #include <xen/lib.h>
 #include <xen/errno.h>
 #include <xen/sched.h>
+#include <xen/viommu.h>
 #include <public/hvm/ioreq.h>
 #include <asm/hvm/io.h>
 #include <asm/hvm/vpic.h>
@@ -285,9 +286,18 @@ static void vioapic_deliver(struct hvm_hw_vioapic *vioapic, int irq)
     struct domain *d = vioapic_domain(vioapic);
     struct vlapic *target;
     struct vcpu *v;
+    struct irq_remapping_request request;
 
     ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
 
+    if ( vioapic->redirtbl[irq].ir.format )
+    {
+        irq_request_ioapic_fill(&request, vioapic->id,
+                                vioapic->redirtbl[irq].bits);
+        viommu_handle_irq_request(d, &request);
+        return;
+    }
+
     HVM_DBG_LOG(DBG_LEVEL_IOAPIC,
                 "dest=%x dest_mode=%x delivery_mode=%x "
                 "vector=%x trig_mode=%x",
diff --git a/xen/arch/x86/viommu.c b/xen/arch/x86/viommu.c
new file mode 100644
index 0000000..ef78d3b
--- /dev/null
+++ b/xen/arch/x86/viommu.c
@@ -0,0 +1,30 @@
+/*
+ * viommu.c
+ *
+ * virtualize IOMMU.
+ *
+ * Copyright (C) 2017 Chao Gao, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms and conditions of the GNU General Public
+ * License, version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/viommu.h>
+
+void irq_request_ioapic_fill(struct irq_remapping_request *req,
+                             uint32_t ioapic_id, uint64_t rte)
+{
+    ASSERT(req);
+    req->type = VIOMMU_REQUEST_IRQ_APIC;
+    req->source_id = ioapic_id;
+    req->msg.rte = rte;
+}
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
index 0b25f34..fcf3c24 100644
--- a/xen/include/asm-x86/viommu.h
+++ b/xen/include/asm-x86/viommu.h
@@ -49,6 +49,9 @@ struct irq_remapping_request
     } msg;
 };
 
+void irq_request_ioapic_fill(struct irq_remapping_request *req,
+                             uint32_t ioapic_id, uint64_t rte);
+
 static inline const struct viommu_ops *viommu_get_ops(void)
 {
     /*
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 6127f89..06be4a5 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -401,6 +401,7 @@ struct hvm_hw_vioapic {
             uint8_t reserved[4];
             uint8_t dest_id;
         } fields;
+        struct ir_ioapic_rte ir;
     } redirtbl[VIOAPIC_NUM_PINS];
 };
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 16/23] X86/vvtd: Enable Queued Invalidation through GCMD

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Software writes to QIE fields of GCMD to enable or disable queued
invalidations. This patch emulates QIE fields of GCMD.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c             | 18 ++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h |  3 ++-
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index 0c49294..ba4d585 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -100,6 +100,11 @@ static inline void __vvtd_set_bit(struct vvtd *vvtd, uint32_t reg, int nr)
     return __set_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
 }
 
+static inline void __vvtd_clear_bit(struct vvtd *vvtd, uint32_t reg, int nr)
+{
+    return __clear_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
+}
+
 static inline void vvtd_set_reg(struct vvtd *vtd, uint32_t reg,
                                 uint32_t value)
 {
@@ -258,6 +263,17 @@ static int vvtd_log_fault(struct vvtd *vvtd, struct irq_remapping_request *irq,
     return 0;
 }
 
+static int vvtd_handle_gcmd_qie(struct vvtd *vvtd, unsigned long val)
+{
+    VVTD_DEBUG(VVTD_DBG_RW, "Enable Queue Invalidation.");
+
+    if ( val & DMA_GCMD_QIE )
+        __vvtd_set_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_QIES_BIT);
+    else
+        __vvtd_clear_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_QIES_BIT);
+    return X86EMUL_OKAY;
+}
+
 static int vvtd_handle_gcmd_sirtp(struct vvtd *vvtd, unsigned long val)
 {
     uint64_t irta;
@@ -289,6 +305,8 @@ static int vvtd_write_gcmd(struct vvtd *vvtd, unsigned long val)
 
     if ( changed & DMA_GCMD_SIRTP )
         vvtd_handle_gcmd_sirtp(vvtd, val);
+    if ( changed & DMA_GCMD_QIE )
+        vvtd_handle_gcmd_qie(vvtd, val);
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index aa1f1a6..dd38c39 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -193,7 +193,8 @@
 #define DMA_GSTS_FLS    (((u64)1) << 29)
 #define DMA_GSTS_AFLS   (((u64)1) << 28)
 #define DMA_GSTS_WBFS   (((u64)1) << 27)
-#define DMA_GSTS_QIES   (((u64)1) <<26)
+#define DMA_GSTS_QIES_BIT       26
+#define DMA_GSTS_QIES           (((u64)1) << DMA_GSTS_QIES_BIT)
 #define DMA_GSTS_IRES   (((u64)1) <<25)
 #define DMA_GSTS_SIRTPS_BIT     24
 #define DMA_GSTS_SIRTPS (((u64)1) << DMA_GSTS_SIRTPS_BIT)
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 17/23] X86/vvtd: Enable Interrupt Remapping through GCMD

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Software uses this field to enable/disable interrupt reampping. This patch
emulate IRES field of GCMD.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c             | 26 ++++++++++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h |  3 ++-
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index ba4d585..a7bba51 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -274,6 +274,24 @@ static int vvtd_handle_gcmd_qie(struct vvtd *vvtd, unsigned long val)
     return X86EMUL_OKAY;
 }
 
+static int vvtd_handle_gcmd_ire(struct vvtd *vvtd, unsigned long val)
+{
+    VVTD_DEBUG(VVTD_DBG_RW, "Enable Interrupt Remapping.");
+
+    if ( val & DMA_GCMD_IRE )
+    {
+        vvtd->status |= VIOMMU_STATUS_IRQ_REMAPPING_ENABLED;
+        __vvtd_set_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_IRES_BIT);
+    }
+    else
+    {
+        vvtd->status |= ~VIOMMU_STATUS_IRQ_REMAPPING_ENABLED;
+        __vvtd_clear_bit(vvtd, DMAR_GSTS_REG, DMA_GSTS_IRES_BIT);
+    }
+
+    return X86EMUL_OKAY;
+}
+
 static int vvtd_handle_gcmd_sirtp(struct vvtd *vvtd, unsigned long val)
 {
     uint64_t irta;
@@ -281,6 +299,10 @@ static int vvtd_handle_gcmd_sirtp(struct vvtd *vvtd, unsigned long val)
     if ( !(val & DMA_GCMD_SIRTP) )
         return X86EMUL_OKAY;
 
+    if ( vvtd_irq_remapping_enabled(vvtd) )
+        VVTD_DEBUG(VVTD_DBG_RW, "Update Interrupt Remapping Table when "
+                   "active." );
+
     vvtd_get_reg_quad(vvtd, DMAR_IRTA_REG, irta);
     vvtd->irt = DMA_IRTA_ADDR(irta) >> PAGE_SHIFT;
     vvtd->irt_max_entry = DMA_IRTA_SIZE(irta);
@@ -307,6 +329,10 @@ static int vvtd_write_gcmd(struct vvtd *vvtd, unsigned long val)
         vvtd_handle_gcmd_sirtp(vvtd, val);
     if ( changed & DMA_GCMD_QIE )
         vvtd_handle_gcmd_qie(vvtd, val);
+    if ( changed & DMA_GCMD_IRE )
+        vvtd_handle_gcmd_ire(vvtd, val);
+    if ( changed & ~(DMA_GCMD_QIE | DMA_GCMD_SIRTP | DMA_GCMD_IRE) )
+        gdprintk(XENLOG_INFO, "Only QIE,SIRTP,IRE in GCMD_REG are handled.\n");
 
     return X86EMUL_OKAY;
 }
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index dd38c39..14fba1e 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -195,7 +195,8 @@
 #define DMA_GSTS_WBFS   (((u64)1) << 27)
 #define DMA_GSTS_QIES_BIT       26
 #define DMA_GSTS_QIES           (((u64)1) << DMA_GSTS_QIES_BIT)
-#define DMA_GSTS_IRES   (((u64)1) <<25)
+#define DMA_GSTS_IRES_BIT       25
+#define DMA_GSTS_IRES   (((u64)1) << DMA_GSTS_IRES_BIT)
 #define DMA_GSTS_SIRTPS_BIT     24
 #define DMA_GSTS_SIRTPS (((u64)1) << DMA_GSTS_SIRTPS_BIT)
 #define DMA_GSTS_CFIS   (((u64)1) <<23)
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 18/23] x86/vpt: Get interrupt vector through a vioapic interface

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

When irq remapping enabled, IRE may not contain the vector of interrupt. So, a
new function, vioapic_gsi_vector() is added to translate gsi to vector.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vioapic.c        | 26 ++++++++++++++++++++++++++
 xen/arch/x86/hvm/vpt.c            |  2 +-
 xen/include/asm-x86/hvm/vioapic.h |  1 +
 3 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 6a00644..95a2e15 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -276,6 +276,32 @@ static inline int pit_channel0_enabled(void)
     return pt_active(&current->domain->arch.vpit.pt0);
 }
 
+int vioapic_gsi_vector(struct domain *d, unsigned int gsi)
+{
+    int err;
+    struct irq_remapping_request request;
+    struct hvm_hw_vioapic *vioapic = domain_vioapic(d);
+    struct irq_remapping_info info;
+
+    if ( vioapic->redirtbl[gsi].ir.format )
+    {
+        irq_request_ioapic_fill(&request, vioapic->id,
+                                vioapic->redirtbl[gsi].bits);
+        err = viommu_get_irq_info(d, &request, &info);
+        if ( err < 0 )
+        {
+            gdprintk(XENLOG_ERR, "Bad gsi or bad interrupt remapping table "
+                     "entry.\n");
+            domain_crash(d);
+        }
+        return info.vector;
+    }
+    else
+    {
+        return vioapic->redirtbl[gsi].fields.vector;
+    }
+}
+
 static void vioapic_deliver(struct hvm_hw_vioapic *vioapic, int irq)
 {
     uint16_t dest = vioapic->redirtbl[irq].fields.dest_id;
diff --git a/xen/arch/x86/hvm/vpt.c b/xen/arch/x86/hvm/vpt.c
index 5c48fdb..a1fafe2 100644
--- a/xen/arch/x86/hvm/vpt.c
+++ b/xen/arch/x86/hvm/vpt.c
@@ -91,7 +91,7 @@ static int pt_irq_vector(struct periodic_time *pt, enum hvm_intsrc src)
                 + (isa_irq & 7));
 
     ASSERT(src == hvm_intsrc_lapic);
-    return domain_vioapic(v->domain)->redirtbl[gsi].fields.vector;
+    return vioapic_gsi_vector(v->domain, gsi);
 }
 
 static int pt_irq_masked(struct periodic_time *pt)
diff --git a/xen/include/asm-x86/hvm/vioapic.h b/xen/include/asm-x86/hvm/vioapic.h
index 745c09a..fc3dc22 100644
--- a/xen/include/asm-x86/hvm/vioapic.h
+++ b/xen/include/asm-x86/hvm/vioapic.h
@@ -56,6 +56,7 @@ struct hvm_vioapic {
 #define vioapic_domain(v) (container_of((v), struct hvm_vioapic, \
                                         hvm_hw_vioapic)->domain)
 
+int vioapic_gsi_vector(struct domain *d, unsigned int gsi);
 int vioapic_init(struct domain *d);
 void vioapic_deinit(struct domain *d);
 void vioapic_reset(struct domain *d);
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 19/23] passthrough: move some fields of hvm_gmsi_info to a sub-structure

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

No functional change.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vmsi.c      |  4 ++--
 xen/drivers/passthrough/io.c | 34 +++++++++++++++++-----------------
 xen/include/xen/hvm/irq.h    |  8 ++++++--
 3 files changed, 25 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index 25f5756..d126498 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -101,8 +101,8 @@ int vmsi_deliver(
 
 void vmsi_deliver_pirq(struct domain *d, const struct hvm_pirq_dpci *pirq_dpci)
 {
-    uint32_t flags = pirq_dpci->gmsi.gflags;
-    int vector = pirq_dpci->gmsi.gvec;
+    uint32_t flags = pirq_dpci->gmsi.legacy.gflags;
+    int vector = pirq_dpci->gmsi.legacy.gvec;
     uint8_t dest = (uint8_t)flags;
     uint8_t dest_mode = !!(flags & VMSI_DM_MASK);
     uint8_t delivery_mode = (flags & VMSI_DELIV_MASK)
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index 080183e..90e48f3 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -370,8 +370,8 @@ int pt_irq_create_bind(
         {
             pirq_dpci->flags = HVM_IRQ_DPCI_MAPPED | HVM_IRQ_DPCI_MACH_MSI |
                                HVM_IRQ_DPCI_GUEST_MSI;
-            pirq_dpci->gmsi.gvec = pt_irq_bind->u.msi.gvec;
-            pirq_dpci->gmsi.gflags = pt_irq_bind->u.msi.gflags;
+            pirq_dpci->gmsi.legacy.gvec = pt_irq_bind->u.msi.gvec;
+            pirq_dpci->gmsi.legacy.gflags = pt_irq_bind->u.msi.gflags;
             /*
              * 'pt_irq_create_bind' can be called after 'pt_irq_destroy_bind'.
              * The 'pirq_cleanup_check' which would free the structure is only
@@ -403,8 +403,8 @@ int pt_irq_create_bind(
             }
             if ( unlikely(rc) )
             {
-                pirq_dpci->gmsi.gflags = 0;
-                pirq_dpci->gmsi.gvec = 0;
+                pirq_dpci->gmsi.legacy.gflags = 0;
+                pirq_dpci->gmsi.legacy.gvec = 0;
                 pirq_dpci->dom = NULL;
                 pirq_dpci->flags = 0;
                 pirq_cleanup_check(info, d);
@@ -423,20 +423,20 @@ int pt_irq_create_bind(
             }
 
             /* If pirq is already mapped as vmsi, update guest data/addr. */
-            if ( pirq_dpci->gmsi.gvec != pt_irq_bind->u.msi.gvec ||
-                 pirq_dpci->gmsi.gflags != pt_irq_bind->u.msi.gflags )
+            if ( pirq_dpci->gmsi.legacy.gvec != pt_irq_bind->u.msi.gvec ||
+                 pirq_dpci->gmsi.legacy.gflags != pt_irq_bind->u.msi.gflags )
             {
                 /* Directly clear pending EOIs before enabling new MSI info. */
                 pirq_guest_eoi(info);
 
-                pirq_dpci->gmsi.gvec = pt_irq_bind->u.msi.gvec;
-                pirq_dpci->gmsi.gflags = pt_irq_bind->u.msi.gflags;
+                pirq_dpci->gmsi.legacy.gvec = pt_irq_bind->u.msi.gvec;
+                pirq_dpci->gmsi.legacy.gflags = pt_irq_bind->u.msi.gflags;
             }
         }
         /* Calculate dest_vcpu_id for MSI-type pirq migration. */
-        dest = pirq_dpci->gmsi.gflags & VMSI_DEST_ID_MASK;
-        dest_mode = !!(pirq_dpci->gmsi.gflags & VMSI_DM_MASK);
-        delivery_mode = (pirq_dpci->gmsi.gflags & VMSI_DELIV_MASK) >>
+        dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
+        dest_mode = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
+        delivery_mode = (pirq_dpci->gmsi.legacy.gflags & VMSI_DELIV_MASK) >>
                          GFLAGS_SHIFT_DELIV_MODE;
 
         dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
@@ -449,14 +449,14 @@ int pt_irq_create_bind(
         if ( iommu_intpost )
         {
             const struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode,
-                                          delivery_mode, pirq_dpci->gmsi.gvec);
+                                   delivery_mode, pirq_dpci->gmsi.legacy.gvec);
 
             if ( vcpu )
-                pi_update_irte( vcpu, info, pirq_dpci->gmsi.gvec );
+                pi_update_irte(vcpu, info, pirq_dpci->gmsi.legacy.gvec);
             else
                 dprintk(XENLOG_G_INFO,
                         "%pv: deliver interrupt in remapping mode,gvec:%02x\n",
-                        vcpu, pirq_dpci->gmsi.gvec);
+                        vcpu, pirq_dpci->gmsi.legacy.gvec);
         }
 
         break;
@@ -770,10 +770,10 @@ static int _hvm_dpci_msi_eoi(struct domain *d,
     int vector = (long)arg;
 
     if ( (pirq_dpci->flags & HVM_IRQ_DPCI_MACH_MSI) &&
-         (pirq_dpci->gmsi.gvec == vector) )
+         (pirq_dpci->gmsi.legacy.gvec == vector) )
     {
-        int dest = pirq_dpci->gmsi.gflags & VMSI_DEST_ID_MASK;
-        int dest_mode = !!(pirq_dpci->gmsi.gflags & VMSI_DM_MASK);
+        int dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
+        int dest_mode = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
 
         if ( vlapic_match_dest(vcpu_vlapic(current), NULL, 0, dest,
                                dest_mode) )
diff --git a/xen/include/xen/hvm/irq.h b/xen/include/xen/hvm/irq.h
index ea332bb..5fd809f 100644
--- a/xen/include/xen/hvm/irq.h
+++ b/xen/include/xen/hvm/irq.h
@@ -60,8 +60,12 @@ struct dev_intx_gsi_link {
 #define GFLAGS_SHIFT_TRG_MODE       15
 
 struct hvm_gmsi_info {
-    uint32_t gvec;
-    uint32_t gflags;
+    union {
+        struct {
+            uint32_t gvec;
+            uint32_t gflags;
+        } legacy;
+    };
     int dest_vcpu_id; /* -1 :multi-dest, non-negative: dest_vcpu_id */
 };
 
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 20/23] Tools/libxc: Add a new interface to bind msi-ir with pirq

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Introduce a new guest interrupt type and provide a new interface to Qemu to
setup/update/unbind the new interrupt type.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 tools/libxc/include/xenctrl.h |  17 ++++++
 tools/libxc/xc_domain.c       |  55 ++++++++++++++++++
 xen/arch/x86/viommu.c         |  10 ++++
 xen/drivers/passthrough/io.c  | 127 +++++++++++++++++++++++++++++++++++-------
 xen/include/asm-x86/viommu.h  |   2 +
 xen/include/public/domctl.h   |   7 +++
 xen/include/xen/hvm/irq.h     |  10 +++-
 7 files changed, 207 insertions(+), 21 deletions(-)

diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index a48981a..8d15059 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1683,6 +1683,15 @@ int xc_domain_ioport_mapping(xc_interface *xch,
                              uint32_t nr_ports,
                              uint32_t add_mapping);
 
+int xc_domain_update_msi_irq_remapping(
+    xc_interface *xch,
+    uint32_t domid,
+    uint32_t pirq,
+    uint32_t source_id,
+    uint32_t data,
+    uint64_t addr,
+    uint64_t gtable);
+
 int xc_domain_update_msi_irq(
     xc_interface *xch,
     uint32_t domid,
@@ -1697,6 +1706,14 @@ int xc_domain_unbind_msi_irq(xc_interface *xch,
                              uint32_t pirq,
                              uint32_t gflags);
 
+int xc_domain_unbind_msi_irq_remapping(
+    xc_interface *xch,
+    uint32_t domid,
+    uint32_t pirq,
+    uint32_t source_id,
+    uint32_t data,
+    uint64_t addr);
+
 int xc_domain_bind_pt_irq(xc_interface *xch,
                           uint32_t domid,
                           uint8_t machine_irq,
diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index d862e53..f0c75ab 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -1592,8 +1592,35 @@ int xc_deassign_dt_device(
     return rc;
 }
 
+int xc_domain_update_msi_irq_remapping(
+    xc_interface *xch,
+    uint32_t domid,
+    uint32_t pirq,
+    uint32_t source_id,
+    uint32_t data,
+    uint64_t addr,
+    uint64_t gtable)
+{
+    int rc;
+    xen_domctl_bind_pt_irq_t *bind;
 
+    DECLARE_DOMCTL;
 
+    domctl.cmd = XEN_DOMCTL_bind_pt_irq;
+    domctl.domain = (domid_t)domid;
+
+    bind = &(domctl.u.bind_pt_irq);
+    bind->hvm_domid = domid;
+    bind->irq_type = PT_IRQ_TYPE_MSI_IR;
+    bind->machine_irq = pirq;
+    bind->u.msi_ir.source_id = source_id;
+    bind->u.msi_ir.data = data;
+    bind->u.msi_ir.addr = addr;
+    bind->u.msi_ir.gtable = gtable;
+
+    rc = do_domctl(xch, &domctl);
+    return rc;
+}
 
 int xc_domain_update_msi_irq(
     xc_interface *xch,
@@ -1623,6 +1650,34 @@ int xc_domain_update_msi_irq(
     return rc;
 }
 
+int xc_domain_unbind_msi_irq_remapping(
+    xc_interface *xch,
+    uint32_t domid,
+    uint32_t pirq,
+    uint32_t source_id,
+    uint32_t data,
+    uint64_t addr)
+{
+    int rc;
+    xen_domctl_bind_pt_irq_t *bind;
+
+    DECLARE_DOMCTL;
+
+    domctl.cmd = XEN_DOMCTL_unbind_pt_irq;
+    domctl.domain = (domid_t)domid;
+
+    bind = &(domctl.u.bind_pt_irq);
+    bind->hvm_domid = domid;
+    bind->irq_type = PT_IRQ_TYPE_MSI_IR;
+    bind->machine_irq = pirq;
+    bind->u.msi_ir.source_id = source_id;
+    bind->u.msi_ir.data = data;
+    bind->u.msi_ir.addr = addr;
+
+    rc = do_domctl(xch, &domctl);
+    return rc;
+}
+
 int xc_domain_unbind_msi_irq(
     xc_interface *xch,
     uint32_t domid,
diff --git a/xen/arch/x86/viommu.c b/xen/arch/x86/viommu.c
index ef78d3b..89c9e87 100644
--- a/xen/arch/x86/viommu.c
+++ b/xen/arch/x86/viommu.c
@@ -28,3 +28,13 @@ void irq_request_ioapic_fill(struct irq_remapping_request *req,
     req->source_id = ioapic_id;
     req->msg.rte = rte;
 }
+
+void irq_request_msi_fill(struct irq_remapping_request *req,
+                          uint32_t source_id, uint64_t addr, uint32_t data)
+{
+    ASSERT(req);
+    req->type = VIOMMU_REQUEST_IRQ_MSI;
+    req->source_id = source_id;
+    req->msg.msi.addr = addr;
+    req->msg.msi.data = data;
+}
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index 90e48f3..4557670 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -305,6 +305,88 @@ static struct vcpu *pi_find_dest_vcpu(const struct domain *d, uint32_t dest_id,
     return NULL;
 }
 
+static inline void set_hvm_gmsi_info(struct hvm_gmsi_info *msi,
+                                     xen_domctl_bind_pt_irq_t *pt_irq_bind,
+                                     int irq_type)
+{
+    if ( irq_type == PT_IRQ_TYPE_MSI )
+    {
+        msi->legacy.gvec = pt_irq_bind->u.msi.gvec;
+        msi->legacy.gflags = pt_irq_bind->u.msi.gflags;
+    }
+    else if ( irq_type == PT_IRQ_TYPE_MSI_IR )
+    {
+        msi->intremap.source_id = pt_irq_bind->u.msi_ir.source_id;
+        msi->intremap.data = pt_irq_bind->u.msi_ir.data;
+        msi->intremap.addr = pt_irq_bind->u.msi_ir.addr;
+    }
+    else
+        BUG();
+}
+
+static inline void clear_hvm_gmsi_info(struct hvm_gmsi_info *msi, int irq_type)
+{
+    if ( irq_type == PT_IRQ_TYPE_MSI )
+    {
+        msi->legacy.gvec = 0;
+        msi->legacy.gflags = 0;
+    }
+    else if ( irq_type == PT_IRQ_TYPE_MSI_IR )
+    {
+        msi->intremap.source_id = 0;
+        msi->intremap.data = 0;
+        msi->intremap.addr = 0;
+    }
+    BUG();
+}
+
+static inline bool hvm_gmsi_info_need_update(struct hvm_gmsi_info *msi,
+                                         xen_domctl_bind_pt_irq_t *pt_irq_bind,
+                                         int irq_type)
+{
+    if ( irq_type == PT_IRQ_TYPE_MSI )
+        return ((msi->legacy.gvec != pt_irq_bind->u.msi.gvec) ||
+                (msi->legacy.gflags != pt_irq_bind->u.msi.gflags));
+    else if ( irq_type == PT_IRQ_TYPE_MSI_IR )
+        return ((msi->intremap.source_id != pt_irq_bind->u.msi_ir.source_id) ||
+                (msi->intremap.data != pt_irq_bind->u.msi_ir.data) ||
+                (msi->intremap.addr != pt_irq_bind->u.msi_ir.addr));
+    BUG();
+    return 0;
+}
+
+static int pirq_dpci_2_msi_attr(struct domain *d,
+                                struct hvm_pirq_dpci *pirq_dpci, uint8_t *gvec,
+                                uint8_t *dest, uint8_t *dm, uint8_t *dlm)
+{
+    int rc = 0;
+    if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
+    {
+        *gvec = pirq_dpci->gmsi.legacy.gvec;
+        *dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
+        *dm = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
+        *dlm = (pirq_dpci->gmsi.legacy.gflags & VMSI_DELIV_MASK) >>
+                GFLAGS_SHIFT_DELIV_MODE;
+    }
+    else if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI_IR )
+    {
+        struct irq_remapping_request request;
+        struct irq_remapping_info irq_info;
+        irq_request_msi_fill(&request, pirq_dpci->gmsi.intremap.source_id,
+                             pirq_dpci->gmsi.intremap.addr,
+                             pirq_dpci->gmsi.intremap.data);
+        rc = viommu_get_irq_info(d, &request, &irq_info);
+        if ( !rc )
+        {
+            *gvec = irq_info.vector;
+            *dest = irq_info.dest;
+            *dm = irq_info.dest_mode;
+            *dlm = irq_info.delivery_mode;
+        }
+    }
+    return rc;
+}
+
 int pt_irq_create_bind(
     struct domain *d, xen_domctl_bind_pt_irq_t *pt_irq_bind)
 {
@@ -362,16 +444,21 @@ int pt_irq_create_bind(
     switch ( pt_irq_bind->irq_type )
     {
     case PT_IRQ_TYPE_MSI:
+    case PT_IRQ_TYPE_MSI_IR:
     {
-        uint8_t dest, dest_mode, delivery_mode;
+        uint8_t dest = 0, dest_mode = 0, delivery_mode = 0, gvec;
         int dest_vcpu_id;
+        int irq_type = pt_irq_bind->irq_type;
+        bool ir = (pt_irq_bind->irq_type == PT_IRQ_TYPE_MSI_IR);
+        uint64_t gtable = ir ? pt_irq_bind->u.msi_ir.gtable :
+                          pt_irq_bind->u.msi.gtable;
 
         if ( !(pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) )
         {
             pirq_dpci->flags = HVM_IRQ_DPCI_MAPPED | HVM_IRQ_DPCI_MACH_MSI |
-                               HVM_IRQ_DPCI_GUEST_MSI;
-            pirq_dpci->gmsi.legacy.gvec = pt_irq_bind->u.msi.gvec;
-            pirq_dpci->gmsi.legacy.gflags = pt_irq_bind->u.msi.gflags;
+                               (ir ? HVM_IRQ_DPCI_GUEST_MSI_IR :
+                                HVM_IRQ_DPCI_GUEST_MSI);
+            set_hvm_gmsi_info(&pirq_dpci->gmsi, pt_irq_bind, irq_type);
             /*
              * 'pt_irq_create_bind' can be called after 'pt_irq_destroy_bind'.
              * The 'pirq_cleanup_check' which would free the structure is only
@@ -386,9 +473,9 @@ int pt_irq_create_bind(
             pirq_dpci->dom = d;
             /* bind after hvm_irq_dpci is setup to avoid race with irq handler*/
             rc = pirq_guest_bind(d->vcpu[0], info, 0);
-            if ( rc == 0 && pt_irq_bind->u.msi.gtable )
+            if ( rc == 0 && gtable )
             {
-                rc = msixtbl_pt_register(d, info, pt_irq_bind->u.msi.gtable);
+                rc = msixtbl_pt_register(d, info, gtable);
                 if ( unlikely(rc) )
                 {
                     pirq_guest_unbind(d, info);
@@ -403,8 +490,7 @@ int pt_irq_create_bind(
             }
             if ( unlikely(rc) )
             {
-                pirq_dpci->gmsi.legacy.gflags = 0;
-                pirq_dpci->gmsi.legacy.gvec = 0;
+                clear_hvm_gmsi_info(&pirq_dpci->gmsi, irq_type);
                 pirq_dpci->dom = NULL;
                 pirq_dpci->flags = 0;
                 pirq_cleanup_check(info, d);
@@ -414,7 +500,8 @@ int pt_irq_create_bind(
         }
         else
         {
-            uint32_t mask = HVM_IRQ_DPCI_MACH_MSI | HVM_IRQ_DPCI_GUEST_MSI;
+            uint32_t mask = HVM_IRQ_DPCI_MACH_MSI |
+                     (ir ? HVM_IRQ_DPCI_GUEST_MSI_IR : HVM_IRQ_DPCI_GUEST_MSI);
 
             if ( (pirq_dpci->flags & mask) != mask )
             {
@@ -423,30 +510,29 @@ int pt_irq_create_bind(
             }
 
             /* If pirq is already mapped as vmsi, update guest data/addr. */
-            if ( pirq_dpci->gmsi.legacy.gvec != pt_irq_bind->u.msi.gvec ||
-                 pirq_dpci->gmsi.legacy.gflags != pt_irq_bind->u.msi.gflags )
+            if ( hvm_gmsi_info_need_update(&pirq_dpci->gmsi, pt_irq_bind,
+                                           irq_type) )
             {
                 /* Directly clear pending EOIs before enabling new MSI info. */
                 pirq_guest_eoi(info);
 
-                pirq_dpci->gmsi.legacy.gvec = pt_irq_bind->u.msi.gvec;
-                pirq_dpci->gmsi.legacy.gflags = pt_irq_bind->u.msi.gflags;
+                set_hvm_gmsi_info(&pirq_dpci->gmsi, pt_irq_bind, irq_type);
             }
         }
         /* Calculate dest_vcpu_id for MSI-type pirq migration. */
-        dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
-        dest_mode = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
-        delivery_mode = (pirq_dpci->gmsi.legacy.gflags & VMSI_DELIV_MASK) >>
-                         GFLAGS_SHIFT_DELIV_MODE;
-
-        dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
+        rc = pirq_dpci_2_msi_attr(d, pirq_dpci, &gvec, &dest, &dest_mode,
+                                  &delivery_mode);
+        if ( rc )
+            dest_vcpu_id = -2; /* -2: Internal Error */
+        else
+            dest_vcpu_id = hvm_girq_dest_2_vcpu_id(d, dest, dest_mode);
         pirq_dpci->gmsi.dest_vcpu_id = dest_vcpu_id;
         spin_unlock(&d->event_lock);
         if ( dest_vcpu_id >= 0 )
             hvm_migrate_pirqs(d->vcpu[dest_vcpu_id]);
 
         /* Use interrupt posting if it is supported. */
-        if ( iommu_intpost )
+        if ( iommu_intpost && !ir )
         {
             const struct vcpu *vcpu = pi_find_dest_vcpu(d, dest, dest_mode,
                                    delivery_mode, pirq_dpci->gmsi.legacy.gvec);
@@ -588,6 +674,7 @@ int pt_irq_destroy_bind(
         }
         break;
     case PT_IRQ_TYPE_MSI:
+    case PT_IRQ_TYPE_MSI_IR:
         break;
     default:
         return -EOPNOTSUPP;
diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
index fcf3c24..e677287 100644
--- a/xen/include/asm-x86/viommu.h
+++ b/xen/include/asm-x86/viommu.h
@@ -51,6 +51,8 @@ struct irq_remapping_request
 
 void irq_request_ioapic_fill(struct irq_remapping_request *req,
                              uint32_t ioapic_id, uint64_t rte);
+void irq_request_msi_fill(struct irq_remapping_request *req,
+                          uint32_t source_id, uint64_t addr, uint32_t data);
 
 static inline const struct viommu_ops *viommu_get_ops(void)
 {
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 85cbb7c..00033c4 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -560,6 +560,7 @@ typedef enum pt_irq_type_e {
     PT_IRQ_TYPE_MSI,
     PT_IRQ_TYPE_MSI_TRANSLATE,
     PT_IRQ_TYPE_SPI,    /* ARM: valid range 32-1019 */
+    PT_IRQ_TYPE_MSI_IR,
 } pt_irq_type_t;
 struct xen_domctl_bind_pt_irq {
     uint32_t machine_irq;
@@ -581,6 +582,12 @@ struct xen_domctl_bind_pt_irq {
             uint64_aligned_t gtable;
         } msi;
         struct {
+            uint32_t source_id;
+            uint32_t data;
+            uint64_t addr;
+            uint64_aligned_t gtable;
+        } msi_ir;
+        struct {
             uint16_t spi;
         } spi;
     } u;
diff --git a/xen/include/xen/hvm/irq.h b/xen/include/xen/hvm/irq.h
index 5fd809f..8dc1196 100644
--- a/xen/include/xen/hvm/irq.h
+++ b/xen/include/xen/hvm/irq.h
@@ -40,6 +40,7 @@ struct dev_intx_gsi_link {
 #define _HVM_IRQ_DPCI_EOI_LATCH_SHIFT           3
 #define _HVM_IRQ_DPCI_GUEST_PCI_SHIFT           4
 #define _HVM_IRQ_DPCI_GUEST_MSI_SHIFT           5
+#define _HVM_IRQ_DPCI_GUEST_MSI_IR_SHIFT        6
 #define _HVM_IRQ_DPCI_TRANSLATE_SHIFT          15
 #define HVM_IRQ_DPCI_MACH_PCI        (1 << _HVM_IRQ_DPCI_MACH_PCI_SHIFT)
 #define HVM_IRQ_DPCI_MACH_MSI        (1 << _HVM_IRQ_DPCI_MACH_MSI_SHIFT)
@@ -47,6 +48,7 @@ struct dev_intx_gsi_link {
 #define HVM_IRQ_DPCI_EOI_LATCH       (1 << _HVM_IRQ_DPCI_EOI_LATCH_SHIFT)
 #define HVM_IRQ_DPCI_GUEST_PCI       (1 << _HVM_IRQ_DPCI_GUEST_PCI_SHIFT)
 #define HVM_IRQ_DPCI_GUEST_MSI       (1 << _HVM_IRQ_DPCI_GUEST_MSI_SHIFT)
+#define HVM_IRQ_DPCI_GUEST_MSI_IR    (1 << _HVM_IRQ_DPCI_GUEST_MSI_IR_SHIFT)
 #define HVM_IRQ_DPCI_TRANSLATE       (1 << _HVM_IRQ_DPCI_TRANSLATE_SHIFT)
 
 #define VMSI_DEST_ID_MASK 0xff
@@ -65,8 +67,14 @@ struct hvm_gmsi_info {
             uint32_t gvec;
             uint32_t gflags;
         } legacy;
+        struct {
+            uint32_t source_id;
+            uint32_t data;
+            uint64_t addr;
+        } intremap;
     };
-    int dest_vcpu_id; /* -1 :multi-dest, non-negative: dest_vcpu_id */
+    /* -2 :internal error, -1 :multi-dest, non-negative: dest_vcpu_id */
+    int dest_vcpu_id;
 };
 
 #define IR_MSI_INDEX(data, addr) (((((addr) & 0x4) << 13) + (((addr) & 0xfffff) >> 5)) + (!!((addr) & 0x8)) * ((data) & 0xffff))
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 21/23] X86/vmsi: Hook guest MSI injection

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

In two situations, hypervisor injects a msi to a hvm guest. One is when qemu
sends a request to hypervisor through XEN_DMOP_inject_msi. The other is when a
physical interrupt arrives and it has been binded to a msi interrupt of guest.
The binding relationship is setup and updated by qemu.

When interrupt remapping enabled, the binding implementation needs to be
extended because some useful fields which are useless if interrupt remapping
is disabled are discarded in the process. So, we add a new type guest
interrupt struct hvm_gmsi_ir_info and a new type flag
HVM_IRQ_DPCI_GUEST_MSI_IR which means the binding relationship is between
physical interrupts and guest remappable msi.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/irq.c       | 10 +++++++++
 xen/arch/x86/hvm/vmsi.c      | 13 ++++++++++--
 xen/drivers/passthrough/io.c | 50 ++++++++++++++++++++++++++++++++------------
 xen/include/asm-x86/msi.h    |  3 +++
 4 files changed, 61 insertions(+), 15 deletions(-)

diff --git a/xen/arch/x86/hvm/irq.c b/xen/arch/x86/hvm/irq.c
index ff7d288..c3d2fba 100644
--- a/xen/arch/x86/hvm/irq.c
+++ b/xen/arch/x86/hvm/irq.c
@@ -26,6 +26,7 @@
 #include <asm/hvm/domain.h>
 #include <asm/hvm/support.h>
 #include <asm/msi.h>
+#include <asm/viommu.h>
 
 /* Must be called with hvm_domain->irq_lock hold */
 static void assert_gsi(struct domain *d, unsigned ioapic_gsi)
@@ -288,6 +289,15 @@ int hvm_inject_msi(struct domain *d, uint64_t addr, uint32_t data)
         >> MSI_DATA_TRIGGER_SHIFT;
     uint8_t vector = data & MSI_DATA_VECTOR_MASK;
 
+    if ( addr & MSI_ADDR_INTEFORMAT_MASK )
+    {
+        struct irq_remapping_request request;
+
+        irq_request_msi_fill(&request, 0, addr, data);
+        viommu_handle_irq_request(d, &request);
+        return 0;
+    }
+
     if ( !vector )
     {
         int pirq = ((addr >> 32) & 0xffffff00) | dest;
diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index d126498..53af07f 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -114,9 +114,18 @@ void vmsi_deliver_pirq(struct domain *d, const struct hvm_pirq_dpci *pirq_dpci)
                 "vector=%x trig_mode=%x\n",
                 dest, dest_mode, delivery_mode, vector, trig_mode);
 
-    ASSERT(pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI);
+    ASSERT(pirq_dpci->flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_GUEST_MSI_IR));
+    if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI_IR )
+    {
+        struct irq_remapping_request request;
 
-    vmsi_deliver(d, vector, dest, dest_mode, delivery_mode, trig_mode);
+        irq_request_msi_fill(&request, pirq_dpci->gmsi.intremap.source_id,
+                             pirq_dpci->gmsi.intremap.addr,
+                             pirq_dpci->gmsi.intremap.data);
+        viommu_handle_irq_request(d, &request);
+    }
+    else
+        vmsi_deliver(d, vector, dest, dest_mode, delivery_mode, trig_mode);
 }
 
 /* Return value, -1 : multi-dests, non-negative value: dest_vcpu_id */
diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c
index 4557670..426f56d 100644
--- a/xen/drivers/passthrough/io.c
+++ b/xen/drivers/passthrough/io.c
@@ -139,7 +139,9 @@ static void pt_pirq_softirq_reset(struct hvm_pirq_dpci *pirq_dpci)
 
 bool_t pt_irq_need_timer(uint32_t flags)
 {
-    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_TRANSLATE));
+    return !(flags & (HVM_IRQ_DPCI_GUEST_MSI_IR |
+                      HVM_IRQ_DPCI_GUEST_MSI |
+                      HVM_IRQ_DPCI_TRANSLATE));
 }
 
 static int pt_irq_guest_eoi(struct domain *d, struct hvm_pirq_dpci *pirq_dpci,
@@ -693,7 +695,8 @@ int pt_irq_destroy_bind(
     pirq = pirq_info(d, machine_gsi);
     pirq_dpci = pirq_dpci(pirq);
 
-    if ( pt_irq_bind->irq_type != PT_IRQ_TYPE_MSI )
+    if ( (pt_irq_bind->irq_type != PT_IRQ_TYPE_MSI_IR) &&
+         (pt_irq_bind->irq_type != PT_IRQ_TYPE_MSI) )
     {
         unsigned int bus = pt_irq_bind->u.pci.bus;
         unsigned int device = pt_irq_bind->u.pci.device;
@@ -856,20 +859,39 @@ static int _hvm_dpci_msi_eoi(struct domain *d,
 {
     int vector = (long)arg;
 
-    if ( (pirq_dpci->flags & HVM_IRQ_DPCI_MACH_MSI) &&
-         (pirq_dpci->gmsi.legacy.gvec == vector) )
+    if ( pirq_dpci->flags & HVM_IRQ_DPCI_MACH_MSI )
     {
-        int dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
-        int dest_mode = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
+        if ( (pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI) &&
+             (pirq_dpci->gmsi.legacy.gvec == vector) )
+        {
+            int dest = pirq_dpci->gmsi.legacy.gflags & VMSI_DEST_ID_MASK;
+            int dest_mode = !!(pirq_dpci->gmsi.legacy.gflags & VMSI_DM_MASK);
 
-        if ( vlapic_match_dest(vcpu_vlapic(current), NULL, 0, dest,
-                               dest_mode) )
+            if ( vlapic_match_dest(vcpu_vlapic(current), NULL, 0, dest,
+                                   dest_mode) )
+            {
+                __msi_pirq_eoi(pirq_dpci);
+                return 1;
+            }
+        }
+        else if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI_IR )
         {
-            __msi_pirq_eoi(pirq_dpci);
-            return 1;
+            int ret;
+            struct irq_remapping_request request;
+            struct irq_remapping_info irq_info;
+
+            irq_request_msi_fill(&request, pirq_dpci->gmsi.intremap.source_id,
+                                 pirq_dpci->gmsi.intremap.addr,
+                                 pirq_dpci->gmsi.intremap.data);
+            ret = viommu_get_irq_info(d, &request, &irq_info);
+            if ( (!ret) && (irq_info.vector == vector) && vlapic_match_dest(
+                vcpu_vlapic(current),NULL, 0, irq_info.dest, irq_info.dest_mode) )
+            {
+                __msi_pirq_eoi(pirq_dpci);
+                return 1;
+            }
         }
     }
-
     return 0;
 }
 
@@ -901,14 +923,16 @@ static void hvm_dirq_assist(struct domain *d, struct hvm_pirq_dpci *pirq_dpci)
         {
             send_guest_pirq(d, pirq);
 
-            if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
+            if ( pirq_dpci->flags
+                 & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_GUEST_MSI_IR) )
             {
                 spin_unlock(&d->event_lock);
                 return;
             }
         }
 
-        if ( pirq_dpci->flags & HVM_IRQ_DPCI_GUEST_MSI )
+        if ( pirq_dpci->flags
+             & (HVM_IRQ_DPCI_GUEST_MSI | HVM_IRQ_DPCI_GUEST_MSI_IR) )
         {
             vmsi_deliver_pirq(d, pirq_dpci);
             spin_unlock(&d->event_lock);
diff --git a/xen/include/asm-x86/msi.h b/xen/include/asm-x86/msi.h
index 9c02945..2e76f6a 100644
--- a/xen/include/asm-x86/msi.h
+++ b/xen/include/asm-x86/msi.h
@@ -48,6 +48,9 @@
 #define MSI_ADDR_REDIRECTION_CPU    (0 << MSI_ADDR_REDIRECTION_SHIFT)
 #define MSI_ADDR_REDIRECTION_LOWPRI (1 << MSI_ADDR_REDIRECTION_SHIFT)
 
+#define MSI_ADDR_INTEFORMAT_SHIFT   4
+#define MSI_ADDR_INTEFORMAT_MASK    (1 << MSI_ADDR_INTEFORMAT_SHIFT)
+
 #define MSI_ADDR_DEST_ID_SHIFT		12
 #define	 MSI_ADDR_DEST_ID_MASK		0x00ff000
 #define  MSI_ADDR_DEST_ID(dest)		(((dest) << MSI_ADDR_DEST_ID_SHIFT) & MSI_ADDR_DEST_ID_MASK)
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 22/23] X86/vvtd: Handle interrupt translation faults

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Interrupt translation faults are non-recoverable. When fault is triggered, it
needs to populate fault info to Fault Recording Registers and inject vIOMMU
msi interrupt to notify guest IOMMU driver to deal with faults.

In this patch, the process how hardware handles interrupt translation
faults (more information about the process can be found in VT-d spec, chipter
"Translation Faults", section "Non-Recoverable Fault Reporting" and
section "Non-Recoverable Logging") is emulated, specifically
viommu_log_fault() to log the fault information and
viommu_report_non_recoverable_fault() to report faults to software. Currently,
only Primary Fault Logging is supported and the Number of Fault-recording
Registers is 1.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c             | 250 +++++++++++++++++++++++++++++++++++-
 xen/drivers/passthrough/vtd/iommu.h |  52 +++++++-
 2 files changed, 295 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index a7bba51..28d8f36 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -19,6 +19,7 @@
  */
 
 #include <xen/domain_page.h>
+#include <xen/lib.h>
 #include <xen/sched.h>
 #include <xen/types.h>
 #include <xen/viommu.h>
@@ -30,6 +31,7 @@
 #include <asm/io_apic.h>
 #include <asm/page.h>
 #include <asm/p2m.h>
+#include <asm/system.h>
 #include <public/viommu.h>
 
 #include "../../../drivers/passthrough/vtd/iommu.h"
@@ -52,6 +54,7 @@ struct vvtd {
     uint64_t base_addr;
     /* Point back to the owner domain */
     struct domain *domain;
+    int frcd_idx;
     /* Is in Extended Interrupt Mode */
     bool eim;
     /* Interrupt remapping table base gfn */
@@ -95,6 +98,23 @@ static inline struct vvtd *vcpu_vvtd(struct vcpu *v)
     return domain_vvtd(v->domain);
 }
 
+static inline int vvtd_test_and_set_bit(struct vvtd *vvtd, uint32_t reg,
+                                        int nr)
+{
+    return test_and_set_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
+}
+
+static inline int vvtd_test_and_clear_bit(struct vvtd *vvtd, uint32_t reg,
+                                          int nr)
+{
+    return test_and_clear_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
+}
+
+static inline int vvtd_test_bit(struct vvtd *vvtd, uint32_t reg, int nr)
+{
+    return test_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
+}
+
 static inline void __vvtd_set_bit(struct vvtd *vvtd, uint32_t reg, int nr)
 {
     return __set_bit(nr, (uint32_t *)&vvtd->regs->data[reg]);
@@ -230,6 +250,24 @@ static int vvtd_delivery(
     return 0;
 }
 
+void vvtd_generate_interrupt(struct vvtd *vvtd,
+                             uint32_t addr,
+                             uint32_t data)
+{
+    uint8_t dest, dm, dlm, tm, vector;
+
+    VVTD_DEBUG(VVTD_DBG_FAULT, "Sending interrupt %x %x to d%d",
+               addr, data, vvtd->domain->domain_id);
+
+    dest = (addr & MSI_ADDR_DEST_ID_MASK) >> MSI_ADDR_DEST_ID_SHIFT;
+    dm = !!(addr & MSI_ADDR_DESTMODE_MASK);
+    dlm = (data & MSI_DATA_DELIVERY_MODE_MASK) >> MSI_DATA_DELIVERY_MODE_SHIFT;
+    tm = (data & MSI_DATA_TRIGGER_MASK) >> MSI_DATA_TRIGGER_SHIFT;
+    vector = data & MSI_DATA_VECTOR_MASK;
+
+    vvtd_delivery(vvtd->domain, vector, dest, dm, dlm, tm);
+}
+
 static uint32_t irq_remapping_request_index(struct irq_remapping_request *irq)
 {
     switch ( irq->type )
@@ -257,10 +295,188 @@ static inline uint32_t irte_dest(struct vvtd *vvtd,
     return (irta & IRTA_EIME) ? irte->remap.dst : (irte->remap.dst) >> 8 & 0xff;
 }
 
-static int vvtd_log_fault(struct vvtd *vvtd, struct irq_remapping_request *irq,
+static void vvtd_report_non_recoverable_fault(struct vvtd *vvtd, int reason)
+{
+    uint32_t fsts;
+
+    ASSERT(reason & DMA_FSTS_FAULTS);
+    fsts = vvtd_get_reg(vvtd, DMAR_FSTS_REG);
+    __vvtd_set_bit(vvtd, DMAR_FSTS_REG, reason);
+
+    /*
+     * Accoroding to VT-d spec "Non-Recoverable Fault Event" chapter, if
+     * there are any previously reported interrupt conditions that are yet to
+     * be sevices by software, the Fault Event interrrupt is not generated.
+     */
+    if ( fsts & DMA_FSTS_FAULTS )
+        return;
+
+    __vvtd_set_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT);
+    if ( !vvtd_test_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IM_BIT) )
+    {
+        uint32_t fe_data, fe_addr;
+        fe_data = vvtd_get_reg(vvtd, DMAR_FEDATA_REG);
+        fe_addr = vvtd_get_reg(vvtd, DMAR_FEADDR_REG);
+        vvtd_generate_interrupt(vvtd, fe_addr, fe_data);
+        __vvtd_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT);
+    }
+}
+
+static void vvtd_recomputing_ppf(struct vvtd *vvtd)
+{
+    int i;
+
+    for ( i = 0; i < DMAR_FRCD_REG_NR; i++ )
+    {
+        if ( vvtd_test_bit(vvtd, DMA_FRCD(i, DMA_FRCD3_OFFSET),
+                           DMA_FRCD_F_BIT) )
+        {
+            vvtd_report_non_recoverable_fault(vvtd, DMA_FSTS_PPF_BIT);
+            return;
+        }
+    }
+    /*
+     * No Primary Fault is in Fault Record Registers, thus clear PPF bit in
+     * FSTS.
+     */
+    __vvtd_clear_bit(vvtd, DMAR_FSTS_REG, DMA_FSTS_PPF_BIT);
+
+    /* If no fault is in FSTS, clear pending bit in FECTL. */
+    if ( !(vvtd_get_reg(vvtd, DMAR_FSTS_REG) & DMA_FSTS_FAULTS) )
+        __vvtd_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT);
+}
+
+/*
+ * Commit a frcd to emulated Fault Record Registers.
+ */
+static void vvtd_commit_frcd(struct vvtd *vvtd, int idx,
+                             struct vtd_fault_record_register *frcd)
+{
+    vvtd_set_reg_quad(vvtd, DMA_FRCD(idx, DMA_FRCD0_OFFSET), frcd->bits.lo);
+    vvtd_set_reg_quad(vvtd, DMA_FRCD(idx, DMA_FRCD2_OFFSET), frcd->bits.hi);
+    vvtd_recomputing_ppf(vvtd);
+}
+
+/*
+ * Allocate a FRCD for the caller. If success, return the FRI. Or, return -1
+ * when failure.
+ */
+static int vvtd_alloc_frcd(struct vvtd *vvtd)
+{
+    int prev;
+    /* Set the F bit to indicate the FRCD is in use. */
+    if ( vvtd_test_and_set_bit(vvtd, DMA_FRCD(vvtd->frcd_idx, DMA_FRCD3_OFFSET),
+                               DMA_FRCD_F_BIT) )
+    {
+        prev = vvtd->frcd_idx;
+        vvtd->frcd_idx = (prev + 1) % DMAR_FRCD_REG_NR;
+        return vvtd->frcd_idx;
+    }
+    return -1;
+}
+
+static void vvtd_free_frcd(struct vvtd *vvtd, int i)
+{
+    __vvtd_clear_bit(vvtd, DMA_FRCD(i, DMA_FRCD3_OFFSET), DMA_FRCD_F_BIT);
+}
+
+static int vvtd_log_fault(struct vvtd *vvtd,
+                          struct irq_remapping_request *request,
                           int reason)
 {
-    return 0;
+    struct vtd_fault_record_register frcd;
+    int frcd_idx;
+
+    switch(reason)
+    {
+    case VTD_FR_IR_REQ_RSVD:
+    case VTD_FR_IR_INDEX_OVER:
+    case VTD_FR_IR_ENTRY_P:
+    case VTD_FR_IR_ROOT_INVAL:
+    case VTD_FR_IR_IRTE_RSVD:
+    case VTD_FR_IR_REQ_COMPAT:
+    case VTD_FR_IR_SID_ERR:
+        if ( vvtd_test_bit(vvtd, DMAR_FSTS_REG, DMA_FSTS_PFO_BIT) )
+            return X86EMUL_OKAY;
+
+        /* No available Fault Record means Fault overflowed */
+        frcd_idx = vvtd_alloc_frcd(vvtd);
+        if ( frcd_idx == -1 )
+        {
+            vvtd_report_non_recoverable_fault(vvtd, DMA_FSTS_PFO_BIT);
+            return X86EMUL_OKAY;
+        }
+        memset(&frcd, 0, sizeof(frcd));
+        frcd.fields.FR = (u8)reason;
+        frcd.fields.FI = ((u64)irq_remapping_request_index(request)) << 36;
+        frcd.fields.SID = (u16)request->source_id;
+        frcd.fields.F = 1;
+        vvtd_commit_frcd(vvtd, frcd_idx, &frcd);
+        return X86EMUL_OKAY;
+
+    default:
+        break;
+    }
+
+    gdprintk(XENLOG_ERR, "Can't handle vVTD Fault (reason 0x%x).", reason);
+    domain_crash(vvtd->domain);
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_frcd3(struct vvtd *vvtd, unsigned long val)
+{
+    /* Writing a 1 means clear fault */
+    if ( val & DMA_FRCD_F )
+    {
+        vvtd_free_frcd(vvtd, 0);
+        vvtd_recomputing_ppf(vvtd);
+    }
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_fectl(struct vvtd *vvtd, unsigned long val)
+{
+    /*
+     * Only DMA_FECTL_IM bit is writable. Generate pending event when unmask.
+     */
+    if ( !(val & DMA_FECTL_IM) )
+    {
+        /* Clear IM */
+        __vvtd_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IM_BIT);
+        if ( vvtd_test_and_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT) )
+        {
+            uint32_t fe_data, fe_addr;
+            fe_data = vvtd_get_reg(vvtd, DMAR_FEDATA_REG);
+            fe_addr = vvtd_get_reg(vvtd, DMAR_FEADDR_REG);
+            vvtd_generate_interrupt(vvtd, fe_addr, fe_data);
+        }
+    }
+    else
+        __vvtd_set_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IM_BIT);
+
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_fsts(struct vvtd *vvtd, unsigned long val)
+{
+    int i, max_fault_index = DMA_FSTS_PRO_BIT + 1;
+
+    val = val & DMA_FSTS_RW1CS;
+    i = find_first_bit(&val, max_fault_index);
+    while ( i < max_fault_index )
+    {
+        __vvtd_clear_bit(vvtd, DMAR_FSTS_REG, i);
+        i = find_next_bit(&val, max_fault_index, i + 1);
+    }
+
+    /*
+     * Clear IP field when all status fields in the Fault Status Register
+     * being clear.
+     */
+    if ( !((vvtd_get_reg(vvtd, DMAR_FSTS_REG) & DMA_FSTS_FAULTS)) )
+        __vvtd_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT);
+
+    return X86EMUL_OKAY;
 }
 
 static int vvtd_handle_gcmd_qie(struct vvtd *vvtd, unsigned long val)
@@ -393,6 +609,7 @@ static int vvtd_write(struct vcpu *v, unsigned long addr,
     unsigned int offset = addr - vvtd->base_addr;
     unsigned int offset_aligned = offset & ~0x3;
     unsigned long val_lo = (val & ((1ULL << 32) - 1));
+    unsigned long val_hi = (val >> 32);
     int ret;
 
     VVTD_DEBUG(VVTD_DBG_RW, "WRITE INFO: offset %x len %d val %lx.",
@@ -443,6 +660,34 @@ static int vvtd_write(struct vcpu *v, unsigned long addr,
             goto error;
         vvtd_set_reg(vvtd, offset_aligned, val_lo);
         ret = X86EMUL_OKAY;
+
+    case DMAR_FSTS_REG:
+        if ( len == 8 )
+            goto error;
+        vvtd_write_fsts(vvtd, val_lo);
+        break;
+
+    case DMAR_FECTL_REG:
+        if ( len == 8 )
+            goto error;
+        ret = vvtd_write_fectl(vvtd, val_lo);
+        break;
+
+    case DMA_CAP_FRO_OFFSET + DMA_FRCD0_OFFSET:
+    case DMA_CAP_FRO_OFFSET + DMA_FRCD1_OFFSET:
+        break;
+
+    case DMA_CAP_FRO_OFFSET + DMA_FRCD2_OFFSET:
+        if ( len == 4 ) {
+            break;
+        }
+        vvtd_write_frcd3(vvtd, val_hi);
+        break;
+
+    case DMA_CAP_FRO_OFFSET + DMA_FRCD3_OFFSET:
+        if ( len == 8 )
+            goto error;
+        vvtd_write_frcd3(vvtd, val_lo);
         break;
 
     default:
@@ -691,6 +936,7 @@ static struct vvtd *__vvtd_create(struct domain *d,
     vvtd->eim = 0;
     vvtd->irt = 0;
     vvtd->irt_max_entry = 0;
+    vvtd->frcd_idx = 0;
     register_mmio_handler(d, &vvtd_mmio_ops);
     return vvtd;
 
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index 14fba1e..20e6172 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -229,26 +229,68 @@
 #define DMA_CCMD_CAIG_MASK(x) (((u64)x) & ((u64) 0x3 << 59))
 
 /* FECTL_REG */
-#define DMA_FECTL_IM (((u64)1) << 31)
+#define DMA_FECTL_IM_BIT 31
+#define DMA_FECTL_IM (((u64)1) << DMA_FECTL_IM_BIT)
+#define DMA_FECTL_IP_BIT 30
+#define DMA_FECTL_IP (((u64)1) << DMA_FECTL_IP_BIT)
 
 /* FSTS_REG */
-#define DMA_FSTS_PFO ((u64)1 << 0)
-#define DMA_FSTS_PPF ((u64)1 << 1)
+#define DMA_FSTS_PFO_BIT 0
+#define DMA_FSTS_PFO ((u64)1 << DMA_FSTS_PFO_BIT)
+#define DMA_FSTS_PPF_BIT 1
+#define DMA_FSTS_PPF ((u64)1 << DMA_FSTS_PPF_BIT)
 #define DMA_FSTS_AFO ((u64)1 << 2)
 #define DMA_FSTS_APF ((u64)1 << 3)
 #define DMA_FSTS_IQE ((u64)1 << 4)
 #define DMA_FSTS_ICE ((u64)1 << 5)
 #define DMA_FSTS_ITE ((u64)1 << 6)
-#define DMA_FSTS_FAULTS    DMA_FSTS_PFO | DMA_FSTS_PPF | DMA_FSTS_AFO | DMA_FSTS_APF | DMA_FSTS_IQE | DMA_FSTS_ICE | DMA_FSTS_ITE
+#define DMA_FSTS_PRO_BIT 7
+#define DMA_FSTS_PRO ((u64)1 << DMA_FSTS_PRO_BIT)
+#define DMA_FSTS_FAULTS    (DMA_FSTS_PFO | DMA_FSTS_PPF | DMA_FSTS_AFO | DMA_FSTS_APF | DMA_FSTS_IQE | DMA_FSTS_ICE | DMA_FSTS_ITE | DMA_FSTS_PRO)
+#define DMA_FSTS_RW1CS     (DMA_FSTS_PFO | DMA_FSTS_AFO | DMA_FSTS_APF | DMA_FSTS_IQE | DMA_FSTS_ICE | DMA_FSTS_ITE | DMA_FSTS_PRO)
 #define dma_fsts_fault_record_index(s) (((s) >> 8) & 0xff)
 
 /* FRCD_REG, 32 bits access */
-#define DMA_FRCD_F (((u64)1) << 31)
+#define DMA_FRCD_LEN            0x10
+#define DMA_FRCD0_OFFSET        0x0
+#define DMA_FRCD1_OFFSET        0x4
+#define DMA_FRCD2_OFFSET        0x8
+#define DMA_FRCD3_OFFSET        0xc
+#define DMA_FRCD3_FR_MASK       0xffUL
+#define DMA_FRCD_F_BIT 31
+#define DMA_FRCD_F ((u64)1 << DMA_FRCD_F_BIT)
+#define DMA_FRCD(idx, offset) (DMA_CAP_FRO_OFFSET + DMA_FRCD_LEN * idx + offset)
 #define dma_frcd_type(d) ((d >> 30) & 1)
 #define dma_frcd_fault_reason(c) (c & 0xff)
 #define dma_frcd_source_id(c) (c & 0xffff)
 #define dma_frcd_page_addr(d) (d & (((u64)-1) << 12)) /* low 64 bit */
 
+struct vtd_fault_record_register
+{
+    union {
+        struct {
+            u64 lo;
+            u64 hi;
+        } bits;
+        struct {
+            u64 rsvd0   :12,
+                FI      :52; /* Fault Info */
+            u64 SID     :16, /* Source Identifier */
+                rsvd1   :9,
+                PRIV    :1,  /* Privilege Mode Requested */
+                EXE     :1,  /* Execute Permission Requested */
+                PP      :1,  /* PASID Present */
+                FR      :8,  /* Fault Reason */
+                PV      :20, /* PASID Value */
+                AT      :2,  /* Address Type */
+                T       :1,  /* Type. (0) Write Request (1) Read Request or
+                              * AtomicOp request
+                              */
+                F       :1;  /* Fault */
+        } fields;
+    };
+};
+
 enum VTD_FAULT_TYPE
 {
     /* Interrupt remapping transition faults */
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[RFC PATCH 23/23] X86/vvtd: Add queued invalidation (QI) support

[Lan Tianyu]

From: Chao Gao <chao.gao@intel.com>

Queued Invalidation Interface is an expanded invalidation interface with
extended capabilities. Hardware implementations report support for queued
invalidation interface through the Extended Capability Register. The queued
invalidation interface uses an Invalidation Queue (IQ), which is a circular
buffer in system memory. Software submits commands by writing Invalidation
Descriptors to the IQ.

In this patch, a new function viommu_process_iq() is used for emulating how
hardware handles invalidation requests through QI.

Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
---
 xen/arch/x86/hvm/vvtd.c             | 248 ++++++++++++++++++++++++++++++++++++
 xen/drivers/passthrough/vtd/iommu.h |  29 ++++-
 2 files changed, 276 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vvtd.c b/xen/arch/x86/hvm/vvtd.c
index 28d8f36..6d11075 100644
--- a/xen/arch/x86/hvm/vvtd.c
+++ b/xen/arch/x86/hvm/vvtd.c
@@ -423,6 +423,185 @@ static int vvtd_log_fault(struct vvtd *vvtd,
     return X86EMUL_OKAY;
 }
 
+/*
+ * Process a invalidation descriptor. Currently, only Two types descriptors,
+ * Interrupt Entry Cache invalidation descritor and Invalidation Wait
+ * Descriptor are handled.
+ * @vvtd: the virtual vtd instance
+ * @i: the index of the invalidation descriptor to be processed
+ *
+ * If success return 0, or return -1 when failure.
+ */
+static int process_iqe(struct vvtd *vvtd, int i)
+{
+    uint64_t iqa, addr;
+    struct qinval_entry *qinval_page;
+    void *pg;
+    int ret;
+
+    vvtd_get_reg_quad(vvtd, DMAR_IQA_REG, iqa);
+    ret = map_guest_page(vvtd->domain, DMA_IQA_ADDR(iqa)>>PAGE_SHIFT,
+                         (void**)&qinval_page);
+    if ( ret )
+    {
+        gdprintk(XENLOG_ERR, "Can't map guest IRT (rc %d)", ret);
+        return -1;
+    }
+
+    switch ( qinval_page[i].q.inv_wait_dsc.lo.type )
+    {
+    case TYPE_INVAL_WAIT:
+        if ( qinval_page[i].q.inv_wait_dsc.lo.sw )
+        {
+            addr = (qinval_page[i].q.inv_wait_dsc.hi.saddr << 2);
+            ret = map_guest_page(vvtd->domain, addr >> PAGE_SHIFT, &pg);
+            if ( ret )
+            {
+                gdprintk(XENLOG_ERR, "Can't map guest memory to inform guest "
+                         "IWC completion (rc %d)", ret);
+                goto error;
+            }
+            *(uint32_t *)((uint64_t)pg + (addr & ~PAGE_MASK)) =
+                qinval_page[i].q.inv_wait_dsc.lo.sdata;
+            unmap_guest_page(pg);
+        }
+
+        /*
+         * The following code generates an invalidation completion event
+         * indicating the invalidation wait descriptor completion. Note that
+         * the following code fragment is not tested properly.
+         */
+        if ( qinval_page[i].q.inv_wait_dsc.lo.iflag )
+        {
+            uint32_t ie_data, ie_addr;
+            if ( !vvtd_test_and_set_bit(vvtd, DMAR_ICS_REG, DMA_ICS_IWC_BIT) )
+            {
+                __vvtd_set_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IP_BIT);
+                if ( !vvtd_test_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IM_BIT) )
+                {
+                    ie_data = vvtd_get_reg(vvtd, DMAR_IEDATA_REG);
+                    ie_addr = vvtd_get_reg(vvtd, DMAR_IEADDR_REG);
+                    vvtd_generate_interrupt(vvtd, ie_addr, ie_data);
+                    __vvtd_clear_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IP_BIT);
+                }
+            }
+        }
+        break;
+
+    case TYPE_INVAL_IEC:
+        /*
+         * Currently, no cache is preserved in hypervisor. Only need to update
+         * pIRTEs which are modified in binding process.
+         */
+        break;
+
+    default:
+        goto error;
+    }
+
+    unmap_guest_page((void*)qinval_page);
+    return 0;
+
+error:
+    unmap_guest_page((void*)qinval_page);
+    gdprintk(XENLOG_ERR, "Internal error in Queue Invalidation.\n");
+    domain_crash(vvtd->domain);
+    return -1;
+}
+
+/*
+ * Invalidate all the descriptors in Invalidation Queue.
+ */
+static void vvtd_process_iq(struct vvtd *vvtd)
+{
+    uint64_t iqh, iqt, iqa, max_entry, i;
+    int ret = 0;
+
+    /*
+     * No new descriptor is fetched from the Invalidation Queue until
+     * software clears the IQE field in the Fault Status Register
+     */
+    if ( vvtd_test_bit(vvtd, DMAR_FSTS_REG, DMA_FSTS_IQE_BIT) )
+        return;
+
+    vvtd_get_reg_quad(vvtd, DMAR_IQH_REG, iqh);
+    vvtd_get_reg_quad(vvtd, DMAR_IQT_REG, iqt);
+    vvtd_get_reg_quad(vvtd, DMAR_IQA_REG, iqa);
+
+    max_entry = DMA_IQA_ENTRY_PER_PAGE << DMA_IQA_QS(iqa);
+    iqh = DMA_IQH_QH(iqh);
+    iqt = DMA_IQT_QT(iqt);
+
+    ASSERT(iqt < max_entry);
+    if ( iqh == iqt )
+        return;
+
+    i = iqh;
+    while ( i != iqt )
+    {
+        ret = process_iqe(vvtd, i);
+        if ( ret )
+            break;
+        else
+            i = (i + 1) % max_entry;
+        vvtd_set_reg_quad(vvtd, DMAR_IQH_REG, i << DMA_IQH_QH_SHIFT);
+    }
+
+    /*
+     * When IQE set, IQH references the desriptor associated with the error.
+     */
+    if ( ret )
+        vvtd_report_non_recoverable_fault(vvtd, DMA_FSTS_IQE_BIT);
+}
+
+static int vvtd_write_iqt(struct vvtd *vvtd, unsigned long val)
+{
+    uint64_t iqa;
+
+    if ( val & DMA_IQT_RSVD )
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "Attempt to set reserved bits in "
+                   "Invalidation Queue Tail.");
+        return X86EMUL_OKAY;
+    }
+
+    vvtd_get_reg_quad(vvtd, DMAR_IQA_REG, iqa);
+    if ( DMA_IQT_QT(val) >= DMA_IQA_ENTRY_PER_PAGE << DMA_IQA_QS(iqa) )
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "IQT: Value %lx exceeded supported max "
+                   "index.", val);
+        return X86EMUL_OKAY;
+    }
+
+    vvtd_set_reg_quad(vvtd, DMAR_IQT_REG, val);
+    vvtd_process_iq(vvtd);
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_iqa(struct vvtd *vvtd, unsigned long val)
+{
+    if ( val & DMA_IQA_RSVD )
+    {
+        VVTD_DEBUG(VVTD_DBG_RW, "Attempt to set reserved bits in "
+                   "Invalidation Queue Address.");
+        return X86EMUL_OKAY;
+    }
+
+    vvtd_set_reg_quad(vvtd, DMAR_IQA_REG, val);
+    return X86EMUL_OKAY;
+}
+
+static int vvtd_write_ics(struct vvtd *vvtd, unsigned long val)
+{
+    if ( val & DMA_ICS_IWC )
+    {
+        __vvtd_clear_bit(vvtd, DMAR_ICS_REG, DMA_ICS_IWC_BIT);
+        /*When IWC field is cleared, the IP field needs to be cleared */
+        __vvtd_clear_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IP_BIT);
+    }
+    return X86EMUL_OKAY;
+}
+
 static int vvtd_write_frcd3(struct vvtd *vvtd, unsigned long val)
 {
     /* Writing a 1 means clear fault */
@@ -434,6 +613,29 @@ static int vvtd_write_frcd3(struct vvtd *vvtd, unsigned long val)
     return X86EMUL_OKAY;
 }
 
+static int vvtd_write_iectl(struct vvtd *vvtd, unsigned long val)
+{
+    /*
+     * Only DMA_IECTL_IM bit is writable. Generate pending event when unmask.
+     */
+    if ( !(val & DMA_IECTL_IM) )
+    {
+        /* Clear IM and clear IP */
+        __vvtd_clear_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IM_BIT);
+        if ( vvtd_test_and_clear_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IP_BIT) )
+        {
+            uint32_t ie_data, ie_addr;
+            ie_data = vvtd_get_reg(vvtd, DMAR_IEDATA_REG);
+            ie_addr = vvtd_get_reg(vvtd, DMAR_IEADDR_REG);
+            vvtd_generate_interrupt(vvtd, ie_addr, ie_data);
+        }
+    }
+    else
+        __vvtd_set_bit(vvtd, DMAR_IECTL_REG, DMA_IECTL_IM_BIT);
+
+    return X86EMUL_OKAY;
+}
+
 static int vvtd_write_fectl(struct vvtd *vvtd, unsigned long val)
 {
     /*
@@ -476,6 +678,10 @@ static int vvtd_write_fsts(struct vvtd *vvtd, unsigned long val)
     if ( !((vvtd_get_reg(vvtd, DMAR_FSTS_REG) & DMA_FSTS_FAULTS)) )
         __vvtd_clear_bit(vvtd, DMAR_FECTL_REG, DMA_FECTL_IP_BIT);
 
+    /* Continue to deal invalidation when IQE is clear */
+    if ( !vvtd_test_bit(vvtd, DMAR_FSTS_REG, DMA_FSTS_IQE_BIT) )
+        vvtd_process_iq(vvtd);
+
     return X86EMUL_OKAY;
 }
 
@@ -636,6 +842,48 @@ static int vvtd_write(struct vcpu *v, unsigned long addr,
         ret = vvtd_write_gcmd(vvtd, val_lo);
         break;
 
+    case DMAR_IQT_REG:
+        if ( len == 8 )
+            ret = vvtd_write_iqt(vvtd, val);
+        else
+            ret = vvtd_write_iqt(vvtd, val_lo);
+        break;
+
+    case DMAR_IQA_REG:
+        if ( len == 8 )
+            ret = vvtd_write_iqa(vvtd, val);
+        else
+        {
+            unsigned long iqa_hi;
+
+            iqa_hi = vvtd_get_reg(vvtd, DMAR_IQA_REG_HI);
+            ret = vvtd_write_iqa(vvtd, val_lo | (iqa_hi << 32));
+        }
+        break;
+
+    case DMAR_IQA_REG_HI:
+    {
+        unsigned long iqa_lo;
+
+        if ( len == 8 )
+            goto error;
+        iqa_lo = vvtd_get_reg(vvtd, DMAR_IQA_REG);
+        ret = vvtd_write_iqa(vvtd, (val_lo << 32) | iqa_lo);
+        break;
+    }
+
+    case DMAR_ICS_REG:
+        if ( len == 8 )
+            goto error;
+        ret = vvtd_write_ics(vvtd, val_lo);
+        break;
+
+    case DMAR_IECTL_REG:
+        if ( len == 8 )
+            goto error;
+        ret = vvtd_write_iectl(vvtd, val_lo);
+        break;
+
     case DMAR_IRTA_REG:
         if ( len == 8 )
             vvtd_set_reg_quad(vvtd, DMAR_IRTA_REG, val);
diff --git a/xen/drivers/passthrough/vtd/iommu.h b/xen/drivers/passthrough/vtd/iommu.h
index 20e6172..2ee7c5c 100644
--- a/xen/drivers/passthrough/vtd/iommu.h
+++ b/xen/drivers/passthrough/vtd/iommu.h
@@ -207,6 +207,32 @@
 #define DMA_IRTA_S(val)         (val & 0xf)
 #define DMA_IRTA_SIZE(val)      (1UL << (DMA_IRTA_S(val) + 1))
 
+/* IQH_REG */
+#define DMA_IQH_QH_SHIFT        4
+#define DMA_IQH_QH(val)         ((val >> 4) & 0x7fffULL)
+
+/* IQT_REG */
+#define DMA_IQT_QT_SHIFT        4
+#define DMA_IQT_QT(val)         ((val >> 4) & 0x7fffULL)
+#define DMA_IQT_RSVD            0xfffffffffff80007ULL
+
+/* IQA_REG */
+#define DMA_MGAW                39  /* Maximum Guest Address Width */
+#define DMA_IQA_ADDR(val)       (val & ~0xfffULL)
+#define DMA_IQA_QS(val)         (val & 0x7)
+#define DMA_IQA_ENTRY_PER_PAGE  (1 << 8)
+#define DMA_IQA_RSVD            (~((1ULL << DMA_MGAW) -1 ) | 0xff8ULL)
+
+/* IECTL_REG */
+#define DMA_IECTL_IM_BIT 31
+#define DMA_IECTL_IM            (1 << DMA_IECTL_IM_BIT)
+#define DMA_IECTL_IP_BIT 30
+#define DMA_IECTL_IP (((u64)1) << DMA_IECTL_IP_BIT)
+
+/* ICS_REG */
+#define DMA_ICS_IWC_BIT         0
+#define DMA_ICS_IWC             (1 << DMA_ICS_IWC_BIT)
+
 /* PMEN_REG */
 #define DMA_PMEN_EPM    (((u32)1) << 31)
 #define DMA_PMEN_PRS    (((u32)1) << 0)
@@ -241,7 +267,8 @@
 #define DMA_FSTS_PPF ((u64)1 << DMA_FSTS_PPF_BIT)
 #define DMA_FSTS_AFO ((u64)1 << 2)
 #define DMA_FSTS_APF ((u64)1 << 3)
-#define DMA_FSTS_IQE ((u64)1 << 4)
+#define DMA_FSTS_IQE_BIT 4
+#define DMA_FSTS_IQE ((u64)1 << DMA_FSTS_IQE_BIT)
 #define DMA_FSTS_ICE ((u64)1 << 5)
 #define DMA_FSTS_ITE ((u64)1 << 6)
 #define DMA_FSTS_PRO_BIT 7
-- 
1.8.3.1


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Roger Pau Monné]

On Fri, Mar 17, 2017 at 07:27:00PM +0800, Lan Tianyu wrote:
> This patchset is to introduce vIOMMU framework and add virtual VTD's
> interrupt remapping support according "Xen virtual IOMMU high level
> design doc V3"(https://lists.xenproject.org/archives/html/xen-devel/
> 2016-11/msg01391.html).
> 
> - vIOMMU framework
> New framework provides viommu_ops and help functions to abstract
> vIOMMU operations(E,G create, destroy, handle irq remapping request
> and so on). Vendors(Intel, ARM, AMD and son) can implement their
> vIOMMU callbacks.
> 
> - Xen vIOMMU device model in Qemu 
> It's in charge of create/destroy vIOMMU in hypervisor via new vIOMMU
> DMOP hypercalls. It will be required to pass virtual devices DMA
> request to hypervisor when enable IOVA(DMA request without PASID)
> function.
> 
> - Virtual VTD
> In this patchset, we enable irq remapping function and covers both
> MSI and IOAPIC interrupts. Don't support post interrupt mode emulation
> and post interrupt mode enabled on host with virtual VTD. Will add
> later.   
> 
> Chao Gao (19):
>   Tools/libxc: Add viommu operations in libxc
>   Tools/libacpi: Add DMA remapping reporting (DMAR) ACPI table
>     structures
>   Tools/libacpi: Add new fields in acpi_config to build DMAR table
>   Tools/libacpi: Add a user configurable parameter to control vIOMMU
>     attributes
>   Tools/libxl: Inform device model to create a guest with a vIOMMU
>     device
>   x86/hvm: Introduce a emulated VTD for HVM
>   X86/vvtd: Add MMIO handler for VVTD
>   X86/vvtd: Set Interrupt Remapping Table Pointer through GCMD
>   X86/vvtd: Process interrupt remapping request
>   X86/vvtd: decode interrupt attribute from IRTE
>   X86/vioapic: Hook interrupt delivery of vIOAPIC
>   X86/vvtd: Enable Queued Invalidation through GCMD
>   X86/vvtd: Enable Interrupt Remapping through GCMD
>   x86/vpt: Get interrupt vector through a vioapic interface
>   passthrough: move some fields of hvm_gmsi_info to a sub-structure
>   Tools/libxc: Add a new interface to bind msi-ir with pirq
>   X86/vmsi: Hook guest MSI injection
>   X86/vvtd: Handle interrupt translation faults
>   X86/vvtd: Add queued invalidation (QI) support
> 
> Lan Tianyu (4):
>   VIOMMU: Add vIOMMU helper functions to create, destroy and query
>     capabilities
>   DMOP: Introduce new DMOP commands for vIOMMU support
>   VIOMMU: Add irq request callback to deal with irq remapping
>   VIOMMU: Add get irq info callback to convert irq remapping request
> 
>  tools/libacpi/acpi2_0.h                         |   45 +
>  tools/libacpi/build.c                           |   58 ++
>  tools/libacpi/libacpi.h                         |   12 +
>  tools/libs/devicemodel/core.c                   |   69 ++
>  tools/libs/devicemodel/include/xendevicemodel.h |   35 +
>  tools/libs/devicemodel/libxendevicemodel.map    |    3 +
>  tools/libxc/include/xenctrl.h                   |   17 +
>  tools/libxc/include/xenctrl_compat.h            |    5 +
>  tools/libxc/xc_devicemodel_compat.c             |   18 +
>  tools/libxc/xc_domain.c                         |   55 +
>  tools/libxl/libxl_create.c                      |   12 +-
>  tools/libxl/libxl_dm.c                          |    9 +
>  tools/libxl/libxl_dom.c                         |   85 ++
>  tools/libxl/libxl_types.idl                     |    8 +
>  tools/xl/xl_parse.c                             |   54 +
>  xen/arch/x86/Makefile                           |    1 +
>  xen/arch/x86/hvm/Makefile                       |    1 +
>  xen/arch/x86/hvm/dm.c                           |   29 +
>  xen/arch/x86/hvm/irq.c                          |   10 +
>  xen/arch/x86/hvm/vioapic.c                      |   36 +
>  xen/arch/x86/hvm/vmsi.c                         |   17 +-
>  xen/arch/x86/hvm/vpt.c                          |    2 +-
>  xen/arch/x86/hvm/vvtd.c                         | 1229 +++++++++++++++++++++++
>  xen/arch/x86/viommu.c                           |   40 +
>  xen/common/Makefile                             |    1 +
>  xen/common/domain.c                             |    3 +
>  xen/common/viommu.c                             |  119 +++
>  xen/drivers/passthrough/io.c                    |  183 +++-
>  xen/drivers/passthrough/vtd/iommu.h             |  213 +++-
>  xen/include/asm-arm/viommu.h                    |   38 +
>  xen/include/asm-x86/hvm/vioapic.h               |    1 +
>  xen/include/asm-x86/msi.h                       |    3 +
>  xen/include/asm-x86/viommu.h                    |   68 ++
>  xen/include/public/arch-x86/hvm/save.h          |   19 +
>  xen/include/public/domctl.h                     |    7 +
>  xen/include/public/hvm/dm_op.h                  |   39 +
>  xen/include/public/viommu.h                     |   38 +
>  xen/include/xen/hvm/irq.h                       |   20 +-
>  xen/include/xen/sched.h                         |    2 +
>  xen/include/xen/viommu.h                        |   74 ++
>  40 files changed, 2601 insertions(+), 77 deletions(-)
>  create mode 100644 xen/arch/x86/hvm/vvtd.c
>  create mode 100644 xen/arch/x86/viommu.c
>  create mode 100644 xen/common/viommu.c
>  create mode 100644 xen/include/asm-arm/viommu.h
>  create mode 100644 xen/include/asm-x86/viommu.h
>  create mode 100644 xen/include/public/viommu.h
>  create mode 100644 xen/include/xen/viommu.h

Thanks! So you add all this vIOMMU code, but the maximum number of allowed
vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
there any missing pieces in order to bump this?

Also, have you tested if this series works with PVH guests? Boris added PVH
support to Linux not long ago, so you should be able to test it just by picking
the latest Linux kernel.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Lan Tianyu]

On 2017年03月20日 22:23, Roger Pau Monné wrote:
> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
> there any missing pieces in order to bump this?

To increase vcpu number, we need to change APIC ID rule and now it's
APICID = VCPUID * 2. Andrew's CPUID improvement will change it and so
our following patches of increasing vcpu number will base on Andrew's job.


> 
> Also, have you tested if this series works with PVH guests? Boris added PVH
> support to Linux not long ago, so you should be able to test it just by picking
> the latest Linux kernel.

Our patchset just targets hvm guest and it will not work for PV guest.

-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Lan Tianyu]

On 2017年03月21日 10:28, Lan Tianyu wrote:
> On 2017年03月20日 22:23, Roger Pau Monné wrote:
>> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
>> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
>> there any missing pieces in order to bump this?
> 
> To increase vcpu number, we need to change APIC ID rule and now it's
> APICID = VCPUID * 2. Andrew's CPUID improvement will change it and so
> our following patches of increasing vcpu number will base on Andrew's job.
> 
> 
>>
>> Also, have you tested if this series works with PVH guests? Boris added PVH
>> support to Linux not long ago, so you should be able to test it just by picking
>> the latest Linux kernel.
> 
> Our patchset just targets hvm guest and it will not work for PV guest. 

New hypercalls introduced by this patchset also can reuse for PVH to
enable vIOMMU. This patchset relies on Qemu Xen-vIOMMU device model to
create/destroy vIOMMU. If we want to enable DMA translation for hvm
guest later, virtual device's DMA request would be passed from Qemu to
Xen hypervisor and the device model in Qemu is necessary.

-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Julien Grall]

Hello Tinayu Lan,

On 03/17/2017 11:27 AM, Lan Tianyu wrote:
> This patch is to introduct an abstract layer for arch vIOMMU implementation

s/introduct/introduce/

> to deal with requests from dom0. Arch vIOMMU code needs to provide callback
> to perform create, destroy and query capabilities operation.
>
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  xen/common/Makefile          |  1 +
>  xen/common/domain.c          |  3 ++
>  xen/common/viommu.c          | 97 ++++++++++++++++++++++++++++++++++++++++++++
>  xen/include/asm-arm/viommu.h | 30 ++++++++++++++
>  xen/include/asm-x86/viommu.h | 31 ++++++++++++++
>  xen/include/public/viommu.h  | 38 +++++++++++++++++
>  xen/include/xen/sched.h      |  2 +
>  xen/include/xen/viommu.h     | 62 ++++++++++++++++++++++++++++
>  8 files changed, 264 insertions(+)
>  create mode 100644 xen/common/viommu.c
>  create mode 100644 xen/include/asm-arm/viommu.h
>  create mode 100644 xen/include/asm-x86/viommu.h
>  create mode 100644 xen/include/public/viommu.h
>  create mode 100644 xen/include/xen/viommu.h
>
> diff --git a/xen/common/Makefile b/xen/common/Makefile
> index 0fed30b..b58de63 100644
> --- a/xen/common/Makefile
> +++ b/xen/common/Makefile
> @@ -60,6 +60,7 @@ obj-y += vm_event.o
>  obj-y += vmap.o
>  obj-y += vsprintf.o
>  obj-y += wait.o
> +obj-y += viommu.o

I see very little point to enable viommu by default on all architecture. 
This is x86 specific and I am yet sure how we would be able to use it on 
ARM as the current series rely on QEMU. Also this is waste space in 
struct domain.

I would prefer if you introduce a Kconfig that would be select by x86 only.

Regards,

>  obj-bin-y += warning.init.o
>  obj-$(CONFIG_XENOPROF) += xenoprof.o
>  obj-y += xmalloc_tlsf.o
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 4492c9c..aafc740 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -398,6 +398,9 @@ struct domain *domain_create(domid_t domid, unsigned int domcr_flags,
>          spin_unlock(&domlist_update_lock);
>      }
>
> +    if ( (err = viommu_init_domain(d)) != 0)
> +        goto fail;
> +
>      return d;
>
>   fail:
> diff --git a/xen/common/viommu.c b/xen/common/viommu.c
> new file mode 100644
> index 0000000..4c1c788
> --- /dev/null
> +++ b/xen/common/viommu.c
> @@ -0,0 +1,97 @@

[...]

> +int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps)
> +{
> +    struct viommu_info *info = &d->viommu;
> +    struct viommu *viommu;
> +    int rc;
> +
> +    if ( !info || !info->ops || !info->ops->create
> +	        || info->nr_viommu >= NR_VIOMMU_PER_DOMAIN )
> +        return -EINVAL;
> +
> +    viommu = xzalloc(struct viommu);
> +    if ( !viommu )
> +        return -EFAULT;
> +
> +    viommu->base_address = base_address;
> +    viommu->length = length;
> +    viommu->caps = caps;
> +    viommu->viommu_id = info->nr_viommu;
> +
> +    info->viommu[info->nr_viommu] = viommu;
> +    info->nr_viommu++;
> +
> +    rc = info->ops->create(d, viommu);
> +    if ( rc < 0 ) {

Coding style:

if ( ... )
{

[...]

> diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
> new file mode 100644
> index 0000000..ef6a60b
> --- /dev/null
> +++ b/xen/include/asm-arm/viommu.h
> @@ -0,0 +1,30 @@
> +/*
> + * include/asm-arm/viommu.h
> + *
> + * Copyright (c) 2017 Intel Corporation
> + * Author: Lan Tianyu <tianyu.lan@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +#ifndef __ARCH_ARM_VIOMMU_H__
> +#define __ARCH_ARM_VIOMMU_H__
> +
> +#include <xen/viommu.h>
> +
> +static inline const struct viommu_ops *viommu_get_ops(void)
> +{
> +    return NULL;
> +}
> +
> +#endif /* __ARCH_ARM_VIOMMU_H__ */

Missing emacs magic.

> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
> new file mode 100644
> index 0000000..efb435f
> --- /dev/null
> +++ b/xen/include/asm-x86/viommu.h
> @@ -0,0 +1,31 @@
> +/*
> + * include/asm-arm/viommu.h
> + *
> + * Copyright (c) 2017 Intel Corporation.
> + * Author: Lan Tianyu <tianyu.lan@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License along with
> + * this program; If not, see <http://www.gnu.org/licenses/>.
> + *
> + */
> +#ifndef __ARCH_X86_VIOMMU_H__
> +#define __ARCH_X86_VIOMMU_H__
> +
> +#include <xen/viommu.h>
> +#include <asm/types.h>
> +
> +static inline const struct viommu_ops *viommu_get_ops(void)
> +{
> +    return NULL;
> +}
> +
> +#endif /* __ARCH_X86_VIOMMU_H__ */

Ditto

> diff --git a/xen/include/public/viommu.h b/xen/include/public/viommu.h
> new file mode 100644
> index 0000000..ca2419b
> --- /dev/null
> +++ b/xen/include/public/viommu.h
> @@ -0,0 +1,38 @@
> +/*
> + * include/public/viommu.h
> + *
> + * Copyright (c) 2017 Intel Corporation
> + * Author: Lan Tianyu <tianyu.lan@intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2, as published by the Free Software Foundation.

Public headers sould not be GPLv2 otherwise it will cause some trouble 
for non-GPLv2 OS. See the license in xen/include/public/COPYING.

Regards.

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Tian, Kevin]

> From: Julien Grall [mailto:julien.grall@arm.com]
> Sent: Wednesday, March 22, 2017 3:57 AM
> 
> >
> > diff --git a/xen/common/Makefile b/xen/common/Makefile index
> > 0fed30b..b58de63 100644
> > --- a/xen/common/Makefile
> > +++ b/xen/common/Makefile
> > @@ -60,6 +60,7 @@ obj-y += vm_event.o
> >  obj-y += vmap.o
> >  obj-y += vsprintf.o
> >  obj-y += wait.o
> > +obj-y += viommu.o
> 
> I see very little point to enable viommu by default on all architecture.
> This is x86 specific and I am yet sure how we would be able to use it on ARM
> as the current series rely on QEMU. Also this is waste space in struct domain.
> 
> I would prefer if you introduce a Kconfig that would be select by x86 only.
> 
> Regards,
> 

Also viommu.c is too generic. Each vendor should has his own
implementation. better change to vvtd.c (and make more sense
move to hvm)

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Lan Tianyu]

Hi  Julien:
	Thanks for review.

On 2017年03月22日 03:56, Julien Grall wrote:
> =======================================
> 
> diff --git a/xen/include/public/viommu.h b/xen/include/public/viommu.h
> new file mode 100644
> index 0000000..ca2419b
> 
> --- /dev/null
> 
> +++ b/xen/include/public/viommu.h
> 
> @@ -0,0 +1,9 @@
> 
> +/*
> +·*·include/public/viommu.h
> +·*
> +·*·Copyright·(c)·2017·Intel·Corporation
> +·*·Author:·Lan·Tianyu·<tianyu.lan@intel.com>
> +·*
> +·*·This·program·is·free·software;·you·can·redistribute·it·and/or·modify·it
> +·*·under·the·terms·and·conditions·of·the·GNU·General·Public·License,
> +·*·version·2,·as·published·by·the·Free·Software·Foundation.
> 
>  obj-y += vmap.o
>  obj-y += vsprintf.o
>  obj-y += wait.o
> +obj-y += viommu.o
> I see very little point to enable viommu by default on all architecture. 
> This is x86 specific and I am yet sure how we would be able to use it on 
> ARM as the current series rely on QEMU. Also this is waste space in 
> struct domain.

XEN_DMOP_create/destroy_viommu hypercalls we introduced are generic for
all platforms and can use in toolstack to create/destroy vIOMMU rather
than just in Qemu. This takes PVH case into account which also don't use
Qemu.


> I would prefer if you introduce a Kconfig that would be select by x86 only.
> Regards,


> Public headers sould not be GPLv2 otherwise it will cause some trouble 
> for non-GPLv2 OS. See the license in xen/include/public/COPYING.

Yes, it should be MIT license.
> 
> Regards.
> 
> -- Julien Grall


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Julien Grall]

Hello,

On 22/03/17 08:45, Lan Tianyu wrote:
> Hi  Julien:
> 	Thanks for review.
>
> On 2017年03月22日 03:56, Julien Grall wrote:
>> =======================================
>>
>> diff --git a/xen/include/public/viommu.h b/xen/include/public/viommu.h
>> new file mode 100644
>> index 0000000..ca2419b
>>
>> --- /dev/null
>>
>> +++ b/xen/include/public/viommu.h
>>
>> @@ -0,0 +1,9 @@
>>
>> +/*
>> +·*·include/public/viommu.h
>> +·*
>> +·*·Copyright·(c)·2017·Intel·Corporation
>> +·*·Author:·Lan·Tianyu·<tianyu.lan@intel.com>
>> +·*
>> +·*·This·program·is·free·software;·you·can·redistribute·it·and/or·modify·it
>> +·*·under·the·terms·and·conditions·of·the·GNU·General·Public·License,
>> +·*·version·2,·as·published·by·the·Free·Software·Foundation.
>>
>>  obj-y += vmap.o
>>  obj-y += vsprintf.o
>>  obj-y += wait.o
>> +obj-y += viommu.o
>> I see very little point to enable viommu by default on all architecture.
>> This is x86 specific and I am yet sure how we would be able to use it on
>> ARM as the current series rely on QEMU. Also this is waste space in
>> struct domain.
>
> XEN_DMOP_create/destroy_viommu hypercalls we introduced are generic for
> all platforms and can use in toolstack to create/destroy vIOMMU rather
> than just in Qemu. This takes PVH case into account which also don't use
> Qemu.

I am afraid that none of the DMOP you suggested in this series will fit 
for ARM.

For instance it is not possible to select via DMOP_CREATE the kind of 
vIOMMU (e.g SMMUv2, SMMUv3, IPMMU-VMSA...).

To be clear, I am not asking to get this code ready for ARM, but at 
least we need to make sure the API could be easily extended. During the 
discussion on the design documented it was suggested to add a 
iommu_version field to make it "future proof".

Also, I was not asking to move this code in arch/x86 but not compiling 
the code on ARM by default as it is currently unusable.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Lan, Tianyu]

On 3/22/2017 4:36 PM, Tian, Kevin wrote:
>> From: Julien Grall [mailto:julien.grall@arm.com]
>> Sent: Wednesday, March 22, 2017 3:57 AM
>>
>>>
>>> diff --git a/xen/common/Makefile b/xen/common/Makefile index
>>> 0fed30b..b58de63 100644
>>> --- a/xen/common/Makefile
>>> +++ b/xen/common/Makefile
>>> @@ -60,6 +60,7 @@ obj-y += vm_event.o
>>>  obj-y += vmap.o
>>>  obj-y += vsprintf.o
>>>  obj-y += wait.o
>>> +obj-y += viommu.o
>>
>> I see very little point to enable viommu by default on all architecture.
>> This is x86 specific and I am yet sure how we would be able to use it on ARM
>> as the current series rely on QEMU. Also this is waste space in struct domain.
>>
>> I would prefer if you introduce a Kconfig that would be select by x86 only.
>>
>> Regards,
>>
>
> Also viommu.c is too generic. Each vendor should has his own
> implementation. better change to vvtd.c (and make more sense
> move to hvm)

Hi Kevin:
     vIommu is an abstract layer and we have added vvtd.c under hvm 
directory in the following patch. vvtd will register its callbacks to 
vIOMMU layer. This works just like IOMMU core and VTD driver.

>
> Thanks
> Kevin
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 1/23] VIOMMU: Add vIOMMU helper functions to create, destroy and query capabilities

[Lan, Tianyu]

On 3/22/2017 7:40 PM, Julien Grall wrote:
> Hello,
>
> On 22/03/17 08:45, Lan Tianyu wrote:
>> Hi  Julien:
>>     Thanks for review.
>>
>> On 2017年03月22日 03:56, Julien Grall wrote:
>>> =======================================
>>>
>>> diff --git a/xen/include/public/viommu.h b/xen/include/public/viommu.h
>>> new file mode 100644
>>> index 0000000..ca2419b
>>>
>>> --- /dev/null
>>>
>>> +++ b/xen/include/public/viommu.h
>>>
>>> @@ -0,0 +1,9 @@
>>>
>>> +/*
>>> +·*·include/public/viommu.h
>>> +·*
>>> +·*·Copyright·(c)·2017·Intel·Corporation
>>> +·*·Author:·Lan·Tianyu·<tianyu.lan@intel.com>
>>> +·*
>>> +·*·This·program·is·free·software;·you·can·redistribute·it·and/or·modify·it
>>>
>>> +·*·under·the·terms·and·conditions·of·the·GNU·General·Public·License,
>>> +·*·version·2,·as·published·by·the·Free·Software·Foundation.
>>>
>>>  obj-y += vmap.o
>>>  obj-y += vsprintf.o
>>>  obj-y += wait.o
>>> +obj-y += viommu.o
>>> I see very little point to enable viommu by default on all architecture.
>>> This is x86 specific and I am yet sure how we would be able to use it on
>>> ARM as the current series rely on QEMU. Also this is waste space in
>>> struct domain.
>>
>> XEN_DMOP_create/destroy_viommu hypercalls we introduced are generic for
>> all platforms and can use in toolstack to create/destroy vIOMMU rather
>> than just in Qemu. This takes PVH case into account which also don't use
>> Qemu.
>
> I am afraid that none of the DMOP you suggested in this series will fit
> for ARM.
>
> For instance it is not possible to select via DMOP_CREATE the kind of
> vIOMMU (e.g SMMUv2, SMMUv3, IPMMU-VMSA...).

Thanks for your information. I am not sure whether we can introduce arch 
specific hypercalls for different vIOMMU implementations and So try to 
make it more general. To support more type vIOMMUs or more vIOMMU 
subfeature, we may extend input parameter structure.

>
> To be clear, I am not asking to get this code ready for ARM, but at
> least we need to make sure the API could be easily extended. During the
> discussion on the design documented it was suggested to add a
> iommu_version field to make it "future proof".

Sure. That's very good suggestion. Sorry, I missed that in this series. 
and thought "capability" field in struct xen_dm_op_create_viommu is 
enough for other vendors to extend more sub features. Will change it.

>
> Also, I was not asking to move this code in arch/x86 but not compiling
> the code on ARM by default as it is currently unusable.

Sure. Will change it.

>
> Regards,
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Wei Liu]

On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
> From: Chao Gao <chao.gao@intel.com>
> 
> In previous patch, we introduce a common vIOMMU layer. In our design,
> we create/destroy vIOMMU through DMOP interface instead of creating it
> according to a config flag of domain. It makes it is possible
> to create vIOMMU in device model or in tool stack.
> 
> The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:

Hypercalls, not syscalls.

>  - query capabilities of vIOMMU emulated by Xen
>  - create vIOMMU in Xen hypervisor with base address, capability
>  - destroy vIOMMU specified by viommu_id
> 
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  tools/libs/devicemodel/core.c                   | 69 +++++++++++++++++++++++++
>  tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
>  tools/libxc/include/xenctrl_compat.h            |  5 ++
>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
>  5 files changed, 130 insertions(+)
> 
> diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
> index a85cb49..aee1150 100644
> --- a/tools/libs/devicemodel/core.c
> +++ b/tools/libs/devicemodel/core.c

Bear in mind that this library is stable, so whatever ends up here can
change in the future.

This is not saying the following code is problematic. It is just a
general FYI.

Obviously the toolstack side is going to follow the hypervisor
interface, so I will do a detailed review later.

> +int xendevicemodel_viommu_destroy(
> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
>  #endif /* __XEN_TOOLS__ */
>  
>  #endif /* XENDEVICEMODEL_H */
> diff --git a/tools/libs/devicemodel/libxendevicemodel.map b/tools/libs/devicemodel/libxendevicemodel.map
> index 45c773e..c2e0968 100644
> --- a/tools/libs/devicemodel/libxendevicemodel.map
> +++ b/tools/libs/devicemodel/libxendevicemodel.map
> @@ -17,6 +17,9 @@ VERS_1.0 {
>  		xendevicemodel_modified_memory;
>  		xendevicemodel_set_mem_type;
>  		xendevicemodel_inject_event;
> +		xendevicemodel_viommu_query_cap;
> +		xendevicemodel_viommu_create;
> +		xendevicemodel_viommu_destroy;
>  		xendevicemodel_restrict;
>  		xendevicemodel_close;

I suppose this series is going to miss 4.9.

Please add these functions to VERS_1.1.

>  	local: *; /* Do not expose anything by default */
> diff --git a/tools/libxc/include/xenctrl_compat.h b/tools/libxc/include/xenctrl_compat.h
> index 040e7b2..315c45d 100644
> --- a/tools/libxc/include/xenctrl_compat.h
> +++ b/tools/libxc/include/xenctrl_compat.h
> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
>  int xc_hvm_inject_trap(
>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t *cap);
> +int xc_viommu_create(
> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
> +    uint32_t *viommu_id);
> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t viommu_id);
>  
>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
>  
> diff --git a/tools/libxc/xc_devicemodel_compat.c b/tools/libxc/xc_devicemodel_compat.c
> index e4edeea..62f703a 100644
> --- a/tools/libxc/xc_devicemodel_compat.c
> +++ b/tools/libxc/xc_devicemodel_compat.c

I don't think you need to provide compat wrappers for them. They are new
APIs.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 9/23] Tools/libxl: Inform device model to create a guest with a vIOMMU device

[Wei Liu]

On Fri, Mar 17, 2017 at 07:27:09PM +0800, Lan Tianyu wrote:
> From: Chao Gao <chao.gao@intel.com>
> 
> A new device, xen_viommu, is added to qemu. In xen side, we can inform device
> model to create a guest with a vIOMMU device through "-device xen_viommu" when
> a viommu has been configurated in guest configuration file. Some specific
> parameters are passed through xenstore. Qemu will create a dummy device to
> preserve the address range and the dummy device will launch a viommu creation
> and destruction throuth libxc interface.
> 
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>

The code looks fine, but the design needs to be agreed upon first.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Chao Gao]

Tianyu is on vacation this two weeks, so I will try to address
some comments on this series.

On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>> From: Chao Gao <chao.gao@intel.com>
>> 
>> In previous patch, we introduce a common vIOMMU layer. In our design,
>> we create/destroy vIOMMU through DMOP interface instead of creating it
>> according to a config flag of domain. It makes it is possible
>> to create vIOMMU in device model or in tool stack.
>> 
>> The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:
>
>Hypercalls, not syscalls.
>
>>  - query capabilities of vIOMMU emulated by Xen
>>  - create vIOMMU in Xen hypervisor with base address, capability
>>  - destroy vIOMMU specified by viommu_id
>> 
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>> ---
>>  tools/libs/devicemodel/core.c                   | 69 +++++++++++++++++++++++++
>>  tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
>>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
>>  tools/libxc/include/xenctrl_compat.h            |  5 ++
>>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
>>  5 files changed, 130 insertions(+)
>> 
>> diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
>> index a85cb49..aee1150 100644
>> --- a/tools/libs/devicemodel/core.c
>> +++ b/tools/libs/devicemodel/core.c
>
>Bear in mind that this library is stable, so whatever ends up here can
>change in the future.
>
>This is not saying the following code is problematic. It is just a
>general FYI.
>
>Obviously the toolstack side is going to follow the hypervisor
>interface, so I will do a detailed review later.

Sure. If the hypervisor interface settles down, we can inform you.

>
>> +int xendevicemodel_viommu_destroy(
>> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
>>  #endif /* __XEN_TOOLS__ */
>>  
>>  #endif /* XENDEVICEMODEL_H */
>> diff --git a/tools/libs/devicemodel/libxendevicemodel.map b/tools/libs/devicemodel/libxendevicemodel.map
>> index 45c773e..c2e0968 100644
>> --- a/tools/libs/devicemodel/libxendevicemodel.map
>> +++ b/tools/libs/devicemodel/libxendevicemodel.map
>> @@ -17,6 +17,9 @@ VERS_1.0 {
>>  		xendevicemodel_modified_memory;
>>  		xendevicemodel_set_mem_type;
>>  		xendevicemodel_inject_event;
>> +		xendevicemodel_viommu_query_cap;
>> +		xendevicemodel_viommu_create;
>> +		xendevicemodel_viommu_destroy;
>>  		xendevicemodel_restrict;
>>  		xendevicemodel_close;
>
>I suppose this series is going to miss 4.9.
>
>Please add these functions to VERS_1.1.

Yes. We will fix this.

>
>>  	local: *; /* Do not expose anything by default */
>> diff --git a/tools/libxc/include/xenctrl_compat.h b/tools/libxc/include/xenctrl_compat.h
>> index 040e7b2..315c45d 100644
>> --- a/tools/libxc/include/xenctrl_compat.h
>> +++ b/tools/libxc/include/xenctrl_compat.h
>> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
>>  int xc_hvm_inject_trap(
>>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
>>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
>> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t *cap);
>> +int xc_viommu_create(
>> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
>> +    uint32_t *viommu_id);
>> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t viommu_id);
>>  
>>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
>>  
>> diff --git a/tools/libxc/xc_devicemodel_compat.c b/tools/libxc/xc_devicemodel_compat.c
>> index e4edeea..62f703a 100644
>> --- a/tools/libxc/xc_devicemodel_compat.c
>> +++ b/tools/libxc/xc_devicemodel_compat.c
>
>I don't think you need to provide compat wrappers for them. They are new
>APIs.

OK. Got it.

Thanks,
Chao
>
>Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Chao Gao]

On Wed, Mar 29, 2017 at 09:00:58AM +0100, Roger Pau Monné wrote:
>On Tue, Mar 21, 2017 at 01:29:26PM +0800, Lan Tianyu wrote:
>> On 2017年03月21日 10:28, Lan Tianyu wrote:
>> > On 2017年03月20日 22:23, Roger Pau Monné wrote:
>> >> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
>> >> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
>> >> there any missing pieces in order to bump this?
>> > 
>> > To increase vcpu number, we need to change APIC ID rule and now it's
>> > APICID = VCPUID * 2. Andrew's CPUID improvement will change it and so
>> > our following patches of increasing vcpu number will base on Andrew's job.
>> > 
>> > 
>> >>
>> >> Also, have you tested if this series works with PVH guests? Boris added PVH
>> >> support to Linux not long ago, so you should be able to test it just by picking
>> >> the latest Linux kernel.
>> > 
>> > Our patchset just targets hvm guest and it will not work for PV guest. 
>> 
>> New hypercalls introduced by this patchset also can reuse for PVH to
>> enable vIOMMU. This patchset relies on Qemu Xen-vIOMMU device model to
>> create/destroy vIOMMU. If we want to enable DMA translation for hvm
>> guest later, virtual device's DMA request would be passed from Qemu to
>> Xen hypervisor and the device model in Qemu is necessary.
>
>The DMA remapping probably doesn't make sense for PVH because PVH doesn't have
>emulated devices, but in any case it would be good to disentangle all this from
>the QEMU (device model) code. AFAICT the DMA remapping part would still be
>useful when doing PCI-passthrough to a PVH guest, and that doesn't involve
>QEMU.

I agree that we need a DMA remapping function in Hypervisor to handle DMA
remapping of pass through device for both HVM and PVH guest and this is also
in Tianyu's v3 design (https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg01391.html).
But this doesn't mean that we don't need the dummy vIOMMU in qemu to handle DMA
remapping of virtual device.

>
>Also, the vIOMMU hypercalls are added as DMOPs, which seems to imply that they
>should be used with/from a device model (and this won't be the case for PVH).

No. When we decided to add vIOMMU hypercalls, we were worry about whether it can
be used by tool stack. We found this:

Privileged toolstack software is permitted to use DMOPs as well as
other hypercalls, of course.  So there is no need to duplicate
functionality between DMOPs and non-stable privileged toolstack
hypercalls.
in https://lists.xenproject.org/archives/html/xen-devel/2016-08/msg00048.html 

I think it just means these interfaces are safe to be used by dm. We needn't
worry a dm will compromise the domain that isn't associated with it. I think
only the interfaces that qemu won't use can be exposed as another format other
than DMOP.

Thank,
Chao

>
>Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Roger Pau Monné]

On Tue, Mar 21, 2017 at 01:29:26PM +0800, Lan Tianyu wrote:
> On 2017年03月21日 10:28, Lan Tianyu wrote:
> > On 2017年03月20日 22:23, Roger Pau Monné wrote:
> >> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
> >> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
> >> there any missing pieces in order to bump this?
> > 
> > To increase vcpu number, we need to change APIC ID rule and now it's
> > APICID = VCPUID * 2. Andrew's CPUID improvement will change it and so
> > our following patches of increasing vcpu number will base on Andrew's job.
> > 
> > 
> >>
> >> Also, have you tested if this series works with PVH guests? Boris added PVH
> >> support to Linux not long ago, so you should be able to test it just by picking
> >> the latest Linux kernel.
> > 
> > Our patchset just targets hvm guest and it will not work for PV guest. 
> 
> New hypercalls introduced by this patchset also can reuse for PVH to
> enable vIOMMU. This patchset relies on Qemu Xen-vIOMMU device model to
> create/destroy vIOMMU. If we want to enable DMA translation for hvm
> guest later, virtual device's DMA request would be passed from Qemu to
> Xen hypervisor and the device model in Qemu is necessary.

The DMA remapping probably doesn't make sense for PVH because PVH doesn't have
emulated devices, but in any case it would be good to disentangle all this from
the QEMU (device model) code. AFAICT the DMA remapping part would still be
useful when doing PCI-passthrough to a PVH guest, and that doesn't involve
QEMU.

Also, the vIOMMU hypercalls are added as DMOPs, which seems to imply that they
should be used with/from a device model (and this won't be the case for PVH).

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Paul Durrant]

> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
> Chao Gao
> Sent: 29 March 2017 01:40
> To: Wei Liu <wei.liu2@citrix.com>
> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
> operations in libxc
> 
> Tianyu is on vacation this two weeks, so I will try to address
> some comments on this series.
> 
> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
> >On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
> >> From: Chao Gao <chao.gao@intel.com>
> >>
> >> In previous patch, we introduce a common vIOMMU layer. In our design,
> >> we create/destroy vIOMMU through DMOP interface instead of creating
> it
> >> according to a config flag of domain. It makes it is possible
> >> to create vIOMMU in device model or in tool stack.
> >>

I've not been following this closely so apologies if this has already been asked...

Why would you need to create a vIOMMU instance in an external device model. Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?

  Paul

> >> The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:
> >
> >Hypercalls, not syscalls.
> >
> >>  - query capabilities of vIOMMU emulated by Xen
> >>  - create vIOMMU in Xen hypervisor with base address, capability
> >>  - destroy vIOMMU specified by viommu_id
> >>
> >> Signed-off-by: Chao Gao <chao.gao@intel.com>
> >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >> ---
> >>  tools/libs/devicemodel/core.c                   | 69
> +++++++++++++++++++++++++
> >>  tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
> >>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
> >>  tools/libxc/include/xenctrl_compat.h            |  5 ++
> >>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
> >>  5 files changed, 130 insertions(+)
> >>
> >> diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
> >> index a85cb49..aee1150 100644
> >> --- a/tools/libs/devicemodel/core.c
> >> +++ b/tools/libs/devicemodel/core.c
> >
> >Bear in mind that this library is stable, so whatever ends up here can
> >change in the future.
> >
> >This is not saying the following code is problematic. It is just a
> >general FYI.
> >
> >Obviously the toolstack side is going to follow the hypervisor
> >interface, so I will do a detailed review later.
> 
> Sure. If the hypervisor interface settles down, we can inform you.
> 
> >
> >> +int xendevicemodel_viommu_destroy(
> >> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
> >>  #endif /* __XEN_TOOLS__ */
> >>
> >>  #endif /* XENDEVICEMODEL_H */
> >> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
> b/tools/libs/devicemodel/libxendevicemodel.map
> >> index 45c773e..c2e0968 100644
> >> --- a/tools/libs/devicemodel/libxendevicemodel.map
> >> +++ b/tools/libs/devicemodel/libxendevicemodel.map
> >> @@ -17,6 +17,9 @@ VERS_1.0 {
> >>  		xendevicemodel_modified_memory;
> >>  		xendevicemodel_set_mem_type;
> >>  		xendevicemodel_inject_event;
> >> +		xendevicemodel_viommu_query_cap;
> >> +		xendevicemodel_viommu_create;
> >> +		xendevicemodel_viommu_destroy;
> >>  		xendevicemodel_restrict;
> >>  		xendevicemodel_close;
> >
> >I suppose this series is going to miss 4.9.
> >
> >Please add these functions to VERS_1.1.
> 
> Yes. We will fix this.
> 
> >
> >>  	local: *; /* Do not expose anything by default */
> >> diff --git a/tools/libxc/include/xenctrl_compat.h
> b/tools/libxc/include/xenctrl_compat.h
> >> index 040e7b2..315c45d 100644
> >> --- a/tools/libxc/include/xenctrl_compat.h
> >> +++ b/tools/libxc/include/xenctrl_compat.h
> >> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
> >>  int xc_hvm_inject_trap(
> >>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
> >>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
> >> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t
> *cap);
> >> +int xc_viommu_create(
> >> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
> >> +    uint32_t *viommu_id);
> >> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t
> viommu_id);
> >>
> >>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
> >>
> >> diff --git a/tools/libxc/xc_devicemodel_compat.c
> b/tools/libxc/xc_devicemodel_compat.c
> >> index e4edeea..62f703a 100644
> >> --- a/tools/libxc/xc_devicemodel_compat.c
> >> +++ b/tools/libxc/xc_devicemodel_compat.c
> >
> >I don't think you need to provide compat wrappers for them. They are new
> >APIs.
> 
> OK. Got it.
> 
> Thanks,
> Chao
> >
> >Wei.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Chao Gao]

On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
>> -----Original Message-----
>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
>> Chao Gao
>> Sent: 29 March 2017 01:40
>> To: Wei Liu <wei.liu2@citrix.com>
>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>> operations in libxc
>> 
>> Tianyu is on vacation this two weeks, so I will try to address
>> some comments on this series.
>> 
>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>> >On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>> >> From: Chao Gao <chao.gao@intel.com>
>> >>
>> >> In previous patch, we introduce a common vIOMMU layer. In our design,
>> >> we create/destroy vIOMMU through DMOP interface instead of creating
>> it
>> >> according to a config flag of domain. It makes it is possible
>> >> to create vIOMMU in device model or in tool stack.
>> >>
>
>I've not been following this closely so apologies if this has already been asked...
>
>Why would you need to create a vIOMMU instance in an external device model.
> Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?

I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
I think creating in toolstack is also ok and is easier to be reused by pvh.

If Tianyu has no concern about this, will move this part to toolstack.

Thanks,
Chao

>
>  Paul
>
>> >> The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:
>> >
>> >Hypercalls, not syscalls.
>> >
>> >>  - query capabilities of vIOMMU emulated by Xen
>> >>  - create vIOMMU in Xen hypervisor with base address, capability
>> >>  - destroy vIOMMU specified by viommu_id
>> >>
>> >> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>> >> ---
>> >>  tools/libs/devicemodel/core.c                   | 69
>> +++++++++++++++++++++++++
>> >>  tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
>> >>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
>> >>  tools/libxc/include/xenctrl_compat.h            |  5 ++
>> >>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
>> >>  5 files changed, 130 insertions(+)
>> >>
>> >> diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
>> >> index a85cb49..aee1150 100644
>> >> --- a/tools/libs/devicemodel/core.c
>> >> +++ b/tools/libs/devicemodel/core.c
>> >
>> >Bear in mind that this library is stable, so whatever ends up here can
>> >change in the future.
>> >
>> >This is not saying the following code is problematic. It is just a
>> >general FYI.
>> >
>> >Obviously the toolstack side is going to follow the hypervisor
>> >interface, so I will do a detailed review later.
>> 
>> Sure. If the hypervisor interface settles down, we can inform you.
>> 
>> >
>> >> +int xendevicemodel_viommu_destroy(
>> >> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
>> >>  #endif /* __XEN_TOOLS__ */
>> >>
>> >>  #endif /* XENDEVICEMODEL_H */
>> >> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
>> b/tools/libs/devicemodel/libxendevicemodel.map
>> >> index 45c773e..c2e0968 100644
>> >> --- a/tools/libs/devicemodel/libxendevicemodel.map
>> >> +++ b/tools/libs/devicemodel/libxendevicemodel.map
>> >> @@ -17,6 +17,9 @@ VERS_1.0 {
>> >>  		xendevicemodel_modified_memory;
>> >>  		xendevicemodel_set_mem_type;
>> >>  		xendevicemodel_inject_event;
>> >> +		xendevicemodel_viommu_query_cap;
>> >> +		xendevicemodel_viommu_create;
>> >> +		xendevicemodel_viommu_destroy;
>> >>  		xendevicemodel_restrict;
>> >>  		xendevicemodel_close;
>> >
>> >I suppose this series is going to miss 4.9.
>> >
>> >Please add these functions to VERS_1.1.
>> 
>> Yes. We will fix this.
>> 
>> >
>> >>  	local: *; /* Do not expose anything by default */
>> >> diff --git a/tools/libxc/include/xenctrl_compat.h
>> b/tools/libxc/include/xenctrl_compat.h
>> >> index 040e7b2..315c45d 100644
>> >> --- a/tools/libxc/include/xenctrl_compat.h
>> >> +++ b/tools/libxc/include/xenctrl_compat.h
>> >> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
>> >>  int xc_hvm_inject_trap(
>> >>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
>> >>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
>> >> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t
>> *cap);
>> >> +int xc_viommu_create(
>> >> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
>> >> +    uint32_t *viommu_id);
>> >> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t
>> viommu_id);
>> >>
>> >>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
>> >>
>> >> diff --git a/tools/libxc/xc_devicemodel_compat.c
>> b/tools/libxc/xc_devicemodel_compat.c
>> >> index e4edeea..62f703a 100644
>> >> --- a/tools/libxc/xc_devicemodel_compat.c
>> >> +++ b/tools/libxc/xc_devicemodel_compat.c
>> >
>> >I don't think you need to provide compat wrappers for them. They are new
>> >APIs.
>> 
>> OK. Got it.
>> 
>> Thanks,
>> Chao
>> >
>> >Wei.
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan, Tianyu]

Hi Paul:
	Sorry for later response.

On 3/31/2017 3:57 AM, Chao Gao wrote:
> On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
>>> -----Original Message-----
>>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
>>> Chao Gao
>>> Sent: 29 March 2017 01:40
>>> To: Wei Liu <wei.liu2@citrix.com>
>>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
>>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
>>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>>> operations in libxc
>>>
>>> Tianyu is on vacation this two weeks, so I will try to address
>>> some comments on this series.
>>>
>>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>>>>> From: Chao Gao <chao.gao@intel.com>
>>>>>
>>>>> In previous patch, we introduce a common vIOMMU layer. In our design,
>>>>> we create/destroy vIOMMU through DMOP interface instead of creating
>>> it
>>>>> according to a config flag of domain. It makes it is possible
>>>>> to create vIOMMU in device model or in tool stack.
>>>>>
>>
>> I've not been following this closely so apologies if this has already been asked...
>>
>> Why would you need to create a vIOMMU instance in an external device model.
>> Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?
>
> I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
> I think creating in toolstack is also ok and is easier to be reused by pvh.
>
> If Tianyu has no concern about this, will move this part to toolstack.

We can move create/destroy vIOMMU in the tool stack but we still need to 
add such dummy vIOMMU device model in Qemu to pass virtual device's DMA 
request into Xen hypervisor. Qemu is required to use DMOP hypercall and 
tool stack may use domctl hyercall. vIOMMU hypercalls will be divided 
into two part.

Domctl:
	create, destroy and query.
DMOP:
	vDev's DMA related operations.

Is this OK?

>
> Thanks,
> Chao
>
>>
>>  Paul
>>
>>>>> The following toolstack code is to add XEN_DMOP_viommu_XXX syscalls:
>>>>
>>>> Hypercalls, not syscalls.
>>>>
>>>>>  - query capabilities of vIOMMU emulated by Xen
>>>>>  - create vIOMMU in Xen hypervisor with base address, capability
>>>>>  - destroy vIOMMU specified by viommu_id
>>>>>
>>>>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>>>>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>>>>> ---
>>>>>  tools/libs/devicemodel/core.c                   | 69
>>> +++++++++++++++++++++++++
>>>>>  tools/libs/devicemodel/include/xendevicemodel.h | 35 +++++++++++++
>>>>>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
>>>>>  tools/libxc/include/xenctrl_compat.h            |  5 ++
>>>>>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
>>>>>  5 files changed, 130 insertions(+)
>>>>>
>>>>> diff --git a/tools/libs/devicemodel/core.c b/tools/libs/devicemodel/core.c
>>>>> index a85cb49..aee1150 100644
>>>>> --- a/tools/libs/devicemodel/core.c
>>>>> +++ b/tools/libs/devicemodel/core.c
>>>>
>>>> Bear in mind that this library is stable, so whatever ends up here can
>>>> change in the future.
>>>>
>>>> This is not saying the following code is problematic. It is just a
>>>> general FYI.
>>>>
>>>> Obviously the toolstack side is going to follow the hypervisor
>>>> interface, so I will do a detailed review later.
>>>
>>> Sure. If the hypervisor interface settles down, we can inform you.
>>>
>>>>
>>>>> +int xendevicemodel_viommu_destroy(
>>>>> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t viommu_id);
>>>>>  #endif /* __XEN_TOOLS__ */
>>>>>
>>>>>  #endif /* XENDEVICEMODEL_H */
>>>>> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
>>> b/tools/libs/devicemodel/libxendevicemodel.map
>>>>> index 45c773e..c2e0968 100644
>>>>> --- a/tools/libs/devicemodel/libxendevicemodel.map
>>>>> +++ b/tools/libs/devicemodel/libxendevicemodel.map
>>>>> @@ -17,6 +17,9 @@ VERS_1.0 {
>>>>>  		xendevicemodel_modified_memory;
>>>>>  		xendevicemodel_set_mem_type;
>>>>>  		xendevicemodel_inject_event;
>>>>> +		xendevicemodel_viommu_query_cap;
>>>>> +		xendevicemodel_viommu_create;
>>>>> +		xendevicemodel_viommu_destroy;
>>>>>  		xendevicemodel_restrict;
>>>>>  		xendevicemodel_close;
>>>>
>>>> I suppose this series is going to miss 4.9.
>>>>
>>>> Please add these functions to VERS_1.1.
>>>
>>> Yes. We will fix this.
>>>
>>>>
>>>>>  	local: *; /* Do not expose anything by default */
>>>>> diff --git a/tools/libxc/include/xenctrl_compat.h
>>> b/tools/libxc/include/xenctrl_compat.h
>>>>> index 040e7b2..315c45d 100644
>>>>> --- a/tools/libxc/include/xenctrl_compat.h
>>>>> +++ b/tools/libxc/include/xenctrl_compat.h
>>>>> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
>>>>>  int xc_hvm_inject_trap(
>>>>>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
>>>>>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
>>>>> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom, uint64_t
>>> *cap);
>>>>> +int xc_viommu_create(
>>>>> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t cap,
>>>>> +    uint32_t *viommu_id);
>>>>> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t
>>> viommu_id);
>>>>>
>>>>>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
>>>>>
>>>>> diff --git a/tools/libxc/xc_devicemodel_compat.c
>>> b/tools/libxc/xc_devicemodel_compat.c
>>>>> index e4edeea..62f703a 100644
>>>>> --- a/tools/libxc/xc_devicemodel_compat.c
>>>>> +++ b/tools/libxc/xc_devicemodel_compat.c
>>>>
>>>> I don't think you need to provide compat wrappers for them. They are new
>>>> APIs.
>>>
>>> OK. Got it.
>>>
>>> Thanks,
>>> Chao
>>>>
>>>> Wei.
>>>
>>> _______________________________________________
>>> Xen-devel mailing list
>>> Xen-devel@lists.xen.org
>>> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Wei Liu]

On Fri, Apr 14, 2017 at 11:38:15PM +0800, Lan, Tianyu wrote:
> Hi Paul:
> 	Sorry for later response.
> 
> On 3/31/2017 3:57 AM, Chao Gao wrote:
> > On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
> > > > -----Original Message-----
> > > > From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
> > > > Chao Gao
> > > > Sent: 29 March 2017 01:40
> > > > To: Wei Liu <wei.liu2@citrix.com>
> > > > Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
> > > > Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
> > > > Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
> > > > operations in libxc
> > > > 
> > > > Tianyu is on vacation this two weeks, so I will try to address
> > > > some comments on this series.
> > > > 
> > > > On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
> > > > > On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
> > > > > > From: Chao Gao <chao.gao@intel.com>
> > > > > > 
> > > > > > In previous patch, we introduce a common vIOMMU layer. In our design,
> > > > > > we create/destroy vIOMMU through DMOP interface instead of creating
> > > > it
> > > > > > according to a config flag of domain. It makes it is possible
> > > > > > to create vIOMMU in device model or in tool stack.
> > > > > > 
> > > 
> > > I've not been following this closely so apologies if this has already been asked...
> > > 
> > > Why would you need to create a vIOMMU instance in an external device model.
> > > Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?
> > 
> > I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
> > I think creating in toolstack is also ok and is easier to be reused by pvh.
> > 
> > If Tianyu has no concern about this, will move this part to toolstack.
> 
> We can move create/destroy vIOMMU in the tool stack but we still need to add
> such dummy vIOMMU device model in Qemu to pass virtual device's DMA request
> into Xen hypervisor. Qemu is required to use DMOP hypercall and tool stack
> may use domctl hyercall. vIOMMU hypercalls will be divided into two part.
> 
> Domctl:
> 	create, destroy and query.
> DMOP:
> 	vDev's DMA related operations.
> 
> Is this OK?
> 

Why are they divided into two libraries? Can't they be in DMOP at the
same time?

Just asking questions, not suggesting it should be done one way or the
other.  Sorry if there are some obvious reasons that I missed.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan Tianyu]

On 2017年04月17日 19:08, Wei Liu wrote:
> On Fri, Apr 14, 2017 at 11:38:15PM +0800, Lan, Tianyu wrote:
>> Hi Paul:
>> 	Sorry for later response.
>>
>> On 3/31/2017 3:57 AM, Chao Gao wrote:
>>> On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
>>>>> -----Original Message-----
>>>>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
>>>>> Chao Gao
>>>>> Sent: 29 March 2017 01:40
>>>>> To: Wei Liu <wei.liu2@citrix.com>
>>>>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
>>>>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
>>>>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>>>>> operations in libxc
>>>>>
>>>>> Tianyu is on vacation this two weeks, so I will try to address
>>>>> some comments on this series.
>>>>>
>>>>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>>>>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>>>>>>> From: Chao Gao <chao.gao@intel.com>
>>>>>>>
>>>>>>> In previous patch, we introduce a common vIOMMU layer. In our design,
>>>>>>> we create/destroy vIOMMU through DMOP interface instead of creating
>>>>> it
>>>>>>> according to a config flag of domain. It makes it is possible
>>>>>>> to create vIOMMU in device model or in tool stack.
>>>>>>>
>>>>
>>>> I've not been following this closely so apologies if this has already been asked...
>>>>
>>>> Why would you need to create a vIOMMU instance in an external device model.
>>>> Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?
>>>
>>> I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
>>> I think creating in toolstack is also ok and is easier to be reused by pvh.
>>>
>>> If Tianyu has no concern about this, will move this part to toolstack.
>>
>> We can move create/destroy vIOMMU in the tool stack but we still need to add
>> such dummy vIOMMU device model in Qemu to pass virtual device's DMA request
>> into Xen hypervisor. Qemu is required to use DMOP hypercall and tool stack
>> may use domctl hyercall. vIOMMU hypercalls will be divided into two part.
>>
>> Domctl:
>> 	create, destroy and query.
>> DMOP:
>> 	vDev's DMA related operations.
>>
>> Is this OK?
>>
> 
> Why are they divided into two libraries? Can't they be in DMOP at the
> same time?

Yes, we can use DMOP for all vIOMMU hyercalls if it's necessary to keep
unified vIOMMU hyercall type. In theory, DMOP dedicates to be used by
Qemu but we also can use it in tool stack. If we move create, destroy
and query operation to tool stack, it isn't necessary to use DMOP for
them since only tool stack will call them. This is why I said we could
use domctl for these operations. Both two ways will not affect function
implementation. Which one it's better from your view? :)


> 
> Just asking questions, not suggesting it should be done one way or the
> other.  Sorry if there are some obvious reasons that I missed.
> 
> Wei.
> 


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 2/23] DMOP: Introduce new DMOP commands for vIOMMU support

[Konrad Rzeszutek Wilk]

On Fri, Mar 17, 2017 at 07:27:02PM +0800, Lan Tianyu wrote:
> This patch is to introduce create, destroy and query capabilities
> command for vIOMMU. vIOMMU layer will deal with requests and call
> arch vIOMMU ops.
> 
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  xen/arch/x86/hvm/dm.c          | 29 +++++++++++++++++++++++++++++
>  xen/include/public/hvm/dm_op.h | 39 +++++++++++++++++++++++++++++++++++++++
>  2 files changed, 68 insertions(+)
> 
> diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
> index 2122c45..2b28f70 100644
> --- a/xen/arch/x86/hvm/dm.c
> +++ b/xen/arch/x86/hvm/dm.c
> @@ -491,6 +491,35 @@ static int dm_op(domid_t domid,
>          break;
>      }
>  
> +    case XEN_DMOP_create_viommu:
> +    {
> +        struct xen_dm_op_create_viommu *data =
> +            &op.u.create_viommu;
> +
> +        rc = viommu_create(d, data->base_address, data->length, data->capabilities);
> +        if (rc >= 0) {

The style guide is is to have a space here and { on a newline.

> +            data->viommu_id = rc;
> +            rc = 0;
> +        }
> +        break;
> +    }

Newline here..


> +    case XEN_DMOP_destroy_viommu:
> +    {
> +        const struct xen_dm_op_destroy_viommu *data =
> +            &op.u.destroy_viommu;
> +
> +        rc = viommu_destroy(d, data->viommu_id);
> +        break;
> +    }

Ahem?
> +    case XEN_DMOP_query_viommu_caps:
> +    {
> +        struct xen_dm_op_query_viommu_caps *data =
> +            &op.u.query_viommu_caps;
> +
> +        data->caps = viommu_query_caps(d);
> +        rc = 0;
> +        break;
> +    }

And here.
>      default:
>          rc = -EOPNOTSUPP;
>          break;
> diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
> index f54cece..b8c7359 100644
> --- a/xen/include/public/hvm/dm_op.h
> +++ b/xen/include/public/hvm/dm_op.h
> @@ -318,6 +318,42 @@ struct xen_dm_op_inject_msi {
>      uint64_aligned_t addr;
>  };
>  
> +/*
> + * XEN_DMOP_create_viommu: Create vIOMMU device.
> + */
> +#define XEN_DMOP_create_viommu 15
> +
> +struct xen_dm_op_create_viommu {
> +    /* IN - MMIO base address of vIOMMU */

Any limit? Can it be zero?

> +    uint64_t base_address;
> +    /* IN - Length of MMIO region */

Any restrictions? Can it be say 2 bytes? Or is this in page-size granularity?

> +    uint64_t length;
> +    /* IN - Capabilities with which we want to create */
> +    uint64_t capabilities;

That sounds like some form of flags?

> +    /* OUT - vIOMMU identity */
> +    uint32_t viommu_id;
> +};
> +
> +/*
> + * XEN_DMOP_destroy_viommu: Destroy vIOMMU device.
> + */
> +#define XEN_DMOP_destroy_viommu 16
> +
> +struct xen_dm_op_destroy_viommu {
> +    /* OUT - vIOMMU identity */

Out? Not in?

> +    uint32_t viommu_id;
> +};
> +
> +/*
> + * XEN_DMOP_q_viommu: Query vIOMMU capabilities.
> + */
> +#define XEN_DMOP_query_viommu_caps 17
> +
> +struct xen_dm_op_query_viommu_caps {
> +    /* OUT - vIOMMU Capabilities*/

Don't you need to also mention which vIOMMU? As you
could have potentially many of them?

> +    uint64_t caps;
> +};
> +
>  struct xen_dm_op {
>      uint32_t op;
>      uint32_t pad;
> @@ -336,6 +372,9 @@ struct xen_dm_op {
>          struct xen_dm_op_set_mem_type set_mem_type;
>          struct xen_dm_op_inject_event inject_event;
>          struct xen_dm_op_inject_msi inject_msi;
> +        struct xen_dm_op_create_viommu create_viommu;
> +        struct xen_dm_op_destroy_viommu destroy_viommu;
> +        struct xen_dm_op_query_viommu_caps query_viommu_caps;
>      } u;
>  };
>  
> -- 
> 1.8.3.1
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 3/23] VIOMMU: Add irq request callback to deal with irq remapping

[Konrad Rzeszutek Wilk]

On Fri, Mar 17, 2017 at 07:27:03PM +0800, Lan Tianyu wrote:
> This patch is to add irq request callback for platform implementation
> to deal with irq remapping request.
> 
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  xen/common/viommu.c          | 11 +++++++++++
>  xen/include/asm-arm/viommu.h |  4 ++++
>  xen/include/asm-x86/viommu.h | 15 +++++++++++++++
>  xen/include/xen/viommu.h     |  8 ++++++++
>  4 files changed, 38 insertions(+)
> 
> diff --git a/xen/common/viommu.c b/xen/common/viommu.c
> index 4c1c788..62c66db 100644
> --- a/xen/common/viommu.c
> +++ b/xen/common/viommu.c
> @@ -87,6 +87,17 @@ u64 viommu_query_caps(struct domain *d)
>      return info->ops->query_caps(d);
>  }
>  
> +int viommu_handle_irq_request(struct domain *d,
> +        struct irq_remapping_request *request)
> +{
> +    struct viommu_info *info = &d->viommu;
> +
> +    if ( !info || !info->ops || !info->ops->handle_irq_request)

You are missing an space at the end.
> +        return -EINVAL;
> +
> +    return info->ops->handle_irq_request(d, request);
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
> index ef6a60b..6a81ecb 100644
> --- a/xen/include/asm-arm/viommu.h
> +++ b/xen/include/asm-arm/viommu.h
> @@ -22,6 +22,10 @@
>  
>  #include <xen/viommu.h>
>  
> +struct irq_remapping_request
> +{
> +};
> +
>  static inline const struct viommu_ops *viommu_get_ops(void)
>  {
>      return NULL;
> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
> index efb435f..b6e01a5 100644
> --- a/xen/include/asm-x86/viommu.h
> +++ b/xen/include/asm-x86/viommu.h
> @@ -23,6 +23,21 @@
>  #include <xen/viommu.h>
>  #include <asm/types.h>
>  
> +struct irq_remapping_request
> +{
> +    u8 type;
> +    u16 source_id;
> +    union {
> +        /* MSI */
> +        struct {
> +            u64 addr;
> +            u32 data;
> +        } msi;
> +        /* Redirection Entry in IOAPIC */
> +        u64 rte;
> +    } msg;
> +};

Will this work right? As in with the default padding and such?
> +
>  static inline const struct viommu_ops *viommu_get_ops(void)
>  {
>      return NULL;
> diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
> index a0abbdf..246b29d 100644
> --- a/xen/include/xen/viommu.h
> +++ b/xen/include/xen/viommu.h
> @@ -24,6 +24,10 @@
>  
>  #define NR_VIOMMU_PER_DOMAIN 1
>  
> +/* IRQ request type */
> +#define VIOMMU_REQUEST_IRQ_MSI          0
> +#define VIOMMU_REQUEST_IRQ_APIC         1

What is this used for?
> +
>  struct viommu {
>      u64 base_address;
>      u64 length;
> @@ -36,6 +40,8 @@ struct viommu_ops {
>      u64 (*query_caps)(struct domain *d);
>      int (*create)(struct domain *d, struct viommu *viommu);
>      int (*destroy)(struct viommu *viommu);
> +    int (*handle_irq_request)(struct domain *d,
> +                              struct irq_remapping_request *request);
>  };
>  
>  struct viommu_info {
> @@ -48,6 +54,8 @@ int viommu_init_domain(struct domain *d);
>  int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps);
>  int viommu_destroy(struct domain *d, u32 viommu_id);
>  u64 viommu_query_caps(struct domain *d);
> +int viommu_handle_irq_request(struct domain *d,
> +                              struct irq_remapping_request *request);
>  
>  #endif /* __XEN_VIOMMU_H__ */
>  
> -- 
> 1.8.3.1
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 4/23] VIOMMU: Add get irq info callback to convert irq remapping request

[Konrad Rzeszutek Wilk]

On Fri, Mar 17, 2017 at 07:27:04PM +0800, Lan Tianyu wrote:
> This patch is to add get_irq_info callback for platform implementation
> to convert irq remapping request to irq info (E,G vector, dest, dest_mode
> and so on).
> 
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  xen/common/viommu.c          | 11 +++++++++++
>  xen/include/asm-arm/viommu.h |  4 ++++
>  xen/include/asm-x86/viommu.h |  8 ++++++++
>  xen/include/xen/viommu.h     |  4 ++++
>  4 files changed, 27 insertions(+)
> 
> diff --git a/xen/common/viommu.c b/xen/common/viommu.c
> index 62c66db..dbec692 100644
> --- a/xen/common/viommu.c
> +++ b/xen/common/viommu.c
> @@ -98,6 +98,17 @@ int viommu_handle_irq_request(struct domain *d,
>      return info->ops->handle_irq_request(d, request);
>  }
>  
> +int viommu_get_irq_info(struct domain *d, struct irq_remapping_request *request,
> +                        struct irq_remapping_info *irq_info)
> +{
> +    struct viommu_info *info = &d->viommu;
> +
> +    if ( !info || !info->ops || !info->ops->get_irq_info)

Ahem.
> +        return -EINVAL;
> +
> +    return info->ops->get_irq_info(d, request, irq_info);
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
> index 6a81ecb..6ce4e0a 100644
> --- a/xen/include/asm-arm/viommu.h
> +++ b/xen/include/asm-arm/viommu.h
> @@ -22,6 +22,10 @@
>  
>  #include <xen/viommu.h>
>  
> +struct irq_remapping_info
> +{
> +};
> +
>  struct irq_remapping_request
>  {
>  };
> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
> index b6e01a5..43e446e 100644
> --- a/xen/include/asm-x86/viommu.h
> +++ b/xen/include/asm-x86/viommu.h
> @@ -23,6 +23,14 @@
>  #include <xen/viommu.h>
>  #include <asm/types.h>
>  
> +struct irq_remapping_info
> +{
> +    u8  vector;
> +    u32 dest;
> +    u32 dest_mode:1;
> +    u32 delivery_mode:3;
> +};
> +
>  struct irq_remapping_request
>  {
>      u8 type;
> diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
> index 246b29d..d733012 100644
> --- a/xen/include/xen/viommu.h
> +++ b/xen/include/xen/viommu.h
> @@ -42,6 +42,8 @@ struct viommu_ops {
>      int (*destroy)(struct viommu *viommu);
>      int (*handle_irq_request)(struct domain *d,
>                                struct irq_remapping_request *request);
> +    int (*get_irq_info)(struct domain *d, struct irq_remapping_request *request,
> +                        struct irq_remapping_info *info);
>  };
>  
>  struct viommu_info {
> @@ -56,6 +58,8 @@ int viommu_destroy(struct domain *d, u32 viommu_id);
>  u64 viommu_query_caps(struct domain *d);
>  int viommu_handle_irq_request(struct domain *d,
>                                struct irq_remapping_request *request);
> +int viommu_get_irq_info(struct domain *d, struct irq_remapping_request *request,
> +                        struct irq_remapping_info *irq_info);
>  
>  #endif /* __XEN_VIOMMU_H__ */
>  
> -- 
> 1.8.3.1
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Konrad Rzeszutek Wilk]

On Mon, Mar 20, 2017 at 02:23:02PM +0000, Roger Pau Monné wrote:
> On Fri, Mar 17, 2017 at 07:27:00PM +0800, Lan Tianyu wrote:
> > This patchset is to introduce vIOMMU framework and add virtual VTD's
> > interrupt remapping support according "Xen virtual IOMMU high level
> > design doc V3"(https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.xenproject.org_archives_html_xen-2Ddevel_&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=wAkdPB9j1dAH7AI494B5wFV3Jws7EfB2Q3Sw-K-88Rk&m=7dZfaODS8zbwpYC0vm7gKQXyM8pBPxfGpz8QMDQzU2k&s=3hxzmHH4X0gz9Oz5_PYoOmWFTkyETYTFPCqJ9iXD910&e= 
> > 2016-11/msg01391.html).

It would be awesome if that was as a patch in docs/misc/

Thanks.

> > 
> > - vIOMMU framework
> > New framework provides viommu_ops and help functions to abstract
> > vIOMMU operations(E,G create, destroy, handle irq remapping request
> > and so on). Vendors(Intel, ARM, AMD and son) can implement their
> > vIOMMU callbacks.
> > 
> > - Xen vIOMMU device model in Qemu 
> > It's in charge of create/destroy vIOMMU in hypervisor via new vIOMMU
> > DMOP hypercalls. It will be required to pass virtual devices DMA
> > request to hypervisor when enable IOVA(DMA request without PASID)
> > function.
> > 
> > - Virtual VTD
> > In this patchset, we enable irq remapping function and covers both
> > MSI and IOAPIC interrupts. Don't support post interrupt mode emulation
> > and post interrupt mode enabled on host with virtual VTD. Will add
> > later.   
> > 
> > Chao Gao (19):
> >   Tools/libxc: Add viommu operations in libxc
> >   Tools/libacpi: Add DMA remapping reporting (DMAR) ACPI table
> >     structures
> >   Tools/libacpi: Add new fields in acpi_config to build DMAR table
> >   Tools/libacpi: Add a user configurable parameter to control vIOMMU
> >     attributes
> >   Tools/libxl: Inform device model to create a guest with a vIOMMU
> >     device
> >   x86/hvm: Introduce a emulated VTD for HVM
> >   X86/vvtd: Add MMIO handler for VVTD
> >   X86/vvtd: Set Interrupt Remapping Table Pointer through GCMD
> >   X86/vvtd: Process interrupt remapping request
> >   X86/vvtd: decode interrupt attribute from IRTE
> >   X86/vioapic: Hook interrupt delivery of vIOAPIC
> >   X86/vvtd: Enable Queued Invalidation through GCMD
> >   X86/vvtd: Enable Interrupt Remapping through GCMD
> >   x86/vpt: Get interrupt vector through a vioapic interface
> >   passthrough: move some fields of hvm_gmsi_info to a sub-structure
> >   Tools/libxc: Add a new interface to bind msi-ir with pirq
> >   X86/vmsi: Hook guest MSI injection
> >   X86/vvtd: Handle interrupt translation faults
> >   X86/vvtd: Add queued invalidation (QI) support
> > 
> > Lan Tianyu (4):
> >   VIOMMU: Add vIOMMU helper functions to create, destroy and query
> >     capabilities
> >   DMOP: Introduce new DMOP commands for vIOMMU support
> >   VIOMMU: Add irq request callback to deal with irq remapping
> >   VIOMMU: Add get irq info callback to convert irq remapping request
> > 
> >  tools/libacpi/acpi2_0.h                         |   45 +
> >  tools/libacpi/build.c                           |   58 ++
> >  tools/libacpi/libacpi.h                         |   12 +
> >  tools/libs/devicemodel/core.c                   |   69 ++
> >  tools/libs/devicemodel/include/xendevicemodel.h |   35 +
> >  tools/libs/devicemodel/libxendevicemodel.map    |    3 +
> >  tools/libxc/include/xenctrl.h                   |   17 +
> >  tools/libxc/include/xenctrl_compat.h            |    5 +
> >  tools/libxc/xc_devicemodel_compat.c             |   18 +
> >  tools/libxc/xc_domain.c                         |   55 +
> >  tools/libxl/libxl_create.c                      |   12 +-
> >  tools/libxl/libxl_dm.c                          |    9 +
> >  tools/libxl/libxl_dom.c                         |   85 ++
> >  tools/libxl/libxl_types.idl                     |    8 +
> >  tools/xl/xl_parse.c                             |   54 +
> >  xen/arch/x86/Makefile                           |    1 +
> >  xen/arch/x86/hvm/Makefile                       |    1 +
> >  xen/arch/x86/hvm/dm.c                           |   29 +
> >  xen/arch/x86/hvm/irq.c                          |   10 +
> >  xen/arch/x86/hvm/vioapic.c                      |   36 +
> >  xen/arch/x86/hvm/vmsi.c                         |   17 +-
> >  xen/arch/x86/hvm/vpt.c                          |    2 +-
> >  xen/arch/x86/hvm/vvtd.c                         | 1229 +++++++++++++++++++++++
> >  xen/arch/x86/viommu.c                           |   40 +
> >  xen/common/Makefile                             |    1 +
> >  xen/common/domain.c                             |    3 +
> >  xen/common/viommu.c                             |  119 +++
> >  xen/drivers/passthrough/io.c                    |  183 +++-
> >  xen/drivers/passthrough/vtd/iommu.h             |  213 +++-
> >  xen/include/asm-arm/viommu.h                    |   38 +
> >  xen/include/asm-x86/hvm/vioapic.h               |    1 +
> >  xen/include/asm-x86/msi.h                       |    3 +
> >  xen/include/asm-x86/viommu.h                    |   68 ++
> >  xen/include/public/arch-x86/hvm/save.h          |   19 +
> >  xen/include/public/domctl.h                     |    7 +
> >  xen/include/public/hvm/dm_op.h                  |   39 +
> >  xen/include/public/viommu.h                     |   38 +
> >  xen/include/xen/hvm/irq.h                       |   20 +-
> >  xen/include/xen/sched.h                         |    2 +
> >  xen/include/xen/viommu.h                        |   74 ++
> >  40 files changed, 2601 insertions(+), 77 deletions(-)
> >  create mode 100644 xen/arch/x86/hvm/vvtd.c
> >  create mode 100644 xen/arch/x86/viommu.c
> >  create mode 100644 xen/common/viommu.c
> >  create mode 100644 xen/include/asm-arm/viommu.h
> >  create mode 100644 xen/include/asm-x86/viommu.h
> >  create mode 100644 xen/include/public/viommu.h
> >  create mode 100644 xen/include/xen/viommu.h
> 
> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
> there any missing pieces in order to bump this?
> 
> Also, have you tested if this series works with PVH guests? Boris added PVH
> support to Linux not long ago, so you should be able to test it just by picking
> the latest Linux kernel.
> 
> Roger.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.xen.org_xen-2Ddevel&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=wAkdPB9j1dAH7AI494B5wFV3Jws7EfB2Q3Sw-K-88Rk&m=7dZfaODS8zbwpYC0vm7gKQXyM8pBPxfGpz8QMDQzU2k&s=RinrlfGer5iCrg48EvpWXwDjHMkLhdtKjdhnsNhzUIo&e= 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 15/23] X86/vioapic: Hook interrupt delivery of vIOAPIC

[Konrad Rzeszutek Wilk]

On Fri, Mar 17, 2017 at 07:27:15PM +0800, Lan Tianyu wrote:
> From: Chao Gao <chao.gao@intel.com>
> 
> When irq remapping enabled, IOAPIC Redirection Entry maybe is in remapping
> format. If that, generate a irq_remapping_request and send it to domain.
> 
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> ---
>  xen/arch/x86/Makefile                  |  1 +
>  xen/arch/x86/hvm/vioapic.c             | 10 ++++++++++
>  xen/arch/x86/viommu.c                  | 30 ++++++++++++++++++++++++++++++
>  xen/include/asm-x86/viommu.h           |  3 +++
>  xen/include/public/arch-x86/hvm/save.h |  1 +
>  5 files changed, 45 insertions(+)
>  create mode 100644 xen/arch/x86/viommu.c
> 
> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
> index f75eca0..d49f8c8 100644
> --- a/xen/arch/x86/Makefile
> +++ b/xen/arch/x86/Makefile
> @@ -66,6 +66,7 @@ obj-y += usercopy.o
>  obj-y += x86_emulate.o
>  obj-$(CONFIG_TBOOT) += tboot.o
>  obj-y += hpet.o
> +obj-y += viommu.o
>  obj-y += vm_event.o
>  obj-y += xstate.o
>  
> diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
> index fdbb21f..6a00644 100644
> --- a/xen/arch/x86/hvm/vioapic.c
> +++ b/xen/arch/x86/hvm/vioapic.c
> @@ -30,6 +30,7 @@
>  #include <xen/lib.h>
>  #include <xen/errno.h>
>  #include <xen/sched.h>
> +#include <xen/viommu.h>
>  #include <public/hvm/ioreq.h>
>  #include <asm/hvm/io.h>
>  #include <asm/hvm/vpic.h>
> @@ -285,9 +286,18 @@ static void vioapic_deliver(struct hvm_hw_vioapic *vioapic, int irq)
>      struct domain *d = vioapic_domain(vioapic);
>      struct vlapic *target;
>      struct vcpu *v;
> +    struct irq_remapping_request request;
>  
>      ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
>  
> +    if ( vioapic->redirtbl[irq].ir.format )
> +    {
> +        irq_request_ioapic_fill(&request, vioapic->id,
> +                                vioapic->redirtbl[irq].bits);
> +        viommu_handle_irq_request(d, &request);
> +        return;
> +    }
> +
>      HVM_DBG_LOG(DBG_LEVEL_IOAPIC,
>                  "dest=%x dest_mode=%x delivery_mode=%x "
>                  "vector=%x trig_mode=%x",
> diff --git a/xen/arch/x86/viommu.c b/xen/arch/x86/viommu.c
> new file mode 100644
> index 0000000..ef78d3b
> --- /dev/null
> +++ b/xen/arch/x86/viommu.c
> @@ -0,0 +1,30 @@
> +/*
> + * viommu.c
> + *
> + * virtualize IOMMU.
> + *
> + * Copyright (C) 2017 Chao Gao, Intel Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms and conditions of the GNU General Public
> + * License, version 2, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public
> + * License along with this program; If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/viommu.h>
> +
> +void irq_request_ioapic_fill(struct irq_remapping_request *req,
> +                             uint32_t ioapic_id, uint64_t rte)
> +{
> +    ASSERT(req);
> +    req->type = VIOMMU_REQUEST_IRQ_APIC;
> +    req->source_id = ioapic_id;
> +    req->msg.rte = rte;

Considering we get 'req' from the stack and it may have garbage, would
it be good to fill out the rest of the entries with sensible values? Or
is there no need for that?
> +}

This being a new file, you should probably include the nice
editor configuration block.

> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
> index 0b25f34..fcf3c24 100644
> --- a/xen/include/asm-x86/viommu.h
> +++ b/xen/include/asm-x86/viommu.h
> @@ -49,6 +49,9 @@ struct irq_remapping_request
>      } msg;
>  };
>  
> +void irq_request_ioapic_fill(struct irq_remapping_request *req,
> +                             uint32_t ioapic_id, uint64_t rte);
> +
>  static inline const struct viommu_ops *viommu_get_ops(void)
>  {
>      /*
> diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
> index 6127f89..06be4a5 100644
> --- a/xen/include/public/arch-x86/hvm/save.h
> +++ b/xen/include/public/arch-x86/hvm/save.h
> @@ -401,6 +401,7 @@ struct hvm_hw_vioapic {
>              uint8_t reserved[4];
>              uint8_t dest_id;
>          } fields;
> +        struct ir_ioapic_rte ir;
>      } redirtbl[VIOAPIC_NUM_PINS];
>  };
>  
> -- 
> 1.8.3.1
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 2/23] DMOP: Introduce new DMOP commands for vIOMMU support

[Lan Tianyu]

Hi Konrad:
	Thanks for your review.

On 2017年04月17日 22:36, Konrad Rzeszutek Wilk wrote:
> On Fri, Mar 17, 2017 at 07:27:02PM +0800, Lan Tianyu wrote:
>> This patch is to introduce create, destroy and query capabilities
>> command for vIOMMU. vIOMMU layer will deal with requests and call
>> arch vIOMMU ops.
>>
>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>> ---
>>  xen/arch/x86/hvm/dm.c          | 29 +++++++++++++++++++++++++++++
>>  xen/include/public/hvm/dm_op.h | 39 +++++++++++++++++++++++++++++++++++++++
>>  2 files changed, 68 insertions(+)
>>
>> diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
>> index 2122c45..2b28f70 100644
>> --- a/xen/arch/x86/hvm/dm.c
>> +++ b/xen/arch/x86/hvm/dm.c
>> @@ -491,6 +491,35 @@ static int dm_op(domid_t domid,
>>          break;
>>      }
>>  
>> +    case XEN_DMOP_create_viommu:
>> +    {
>> +        struct xen_dm_op_create_viommu *data =
>> +            &op.u.create_viommu;
>> +
>> +        rc = viommu_create(d, data->base_address, data->length, data->capabilities);
>> +        if (rc >= 0) {
> 
> The style guide is is to have a space here and { on a newline.

Yes, will fix.

> 
>> +            data->viommu_id = rc;
>> +            rc = 0;
>> +        }
>> +        break;
>> +    }
> 
> Newline here..
> 
> 
>> +    case XEN_DMOP_destroy_viommu:
>> +    {
>> +        const struct xen_dm_op_destroy_viommu *data =
>> +            &op.u.destroy_viommu;
>> +
>> +        rc = viommu_destroy(d, data->viommu_id);
>> +        break;
>> +    }
> 
> Ahem?
>> +    case XEN_DMOP_query_viommu_caps:
>> +    {
>> +        struct xen_dm_op_query_viommu_caps *data =
>> +            &op.u.query_viommu_caps;
>> +
>> +        data->caps = viommu_query_caps(d);
>> +        rc = 0;
>> +        break;
>> +    }
> 
> And here.
>>      default:
>>          rc = -EOPNOTSUPP;
>>          break;
>> diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
>> index f54cece..b8c7359 100644
>> --- a/xen/include/public/hvm/dm_op.h
>> +++ b/xen/include/public/hvm/dm_op.h
>> @@ -318,6 +318,42 @@ struct xen_dm_op_inject_msi {
>>      uint64_aligned_t addr;
>>  };
>>  
>> +/*
>> + * XEN_DMOP_create_viommu: Create vIOMMU device.
>> + */
>> +#define XEN_DMOP_create_viommu 15
>> +
>> +struct xen_dm_op_create_viommu {
>> +    /* IN - MMIO base address of vIOMMU */
> 
> Any limit? Can it be zero?

In current patchset, base address is allocated by toolstack and passed
to Qemu to create vIOMMU in hyervisor. Toolstack should make sure the
range won't be conflicted with other resource.

> 
>> +    uint64_t base_address;
>> +    /* IN - Length of MMIO region */
> 
> Any restrictions? Can it be say 2 bytes? Or is this in page-size granularity?

From the VTD spec, register size must be an integer multiple of 4KB and
I think the vIOMMU device model(E,G vvtd) in hypervisor should check the
lengh. Different vendor may have different restriction.

> 
>> +    uint64_t length;
>> +    /* IN - Capabilities with which we want to create */
>> +    uint64_t capabilities;
> 
> That sounds like some form of flags?

Yes, this patchset just introduces interrupt remapping flag and other
vendor also can use it to add new features.

> 
>> +    /* OUT - vIOMMU identity */
>> +    uint32_t viommu_id;
>> +};
>> +
>> +/*
>> + * XEN_DMOP_destroy_viommu: Destroy vIOMMU device.
>> + */
>> +#define XEN_DMOP_destroy_viommu 16
>> +
>> +struct xen_dm_op_destroy_viommu {
>> +    /* OUT - vIOMMU identity */
> 
> Out? Not in?

Sorry, it should be OUT parameter.

> 
>> +    uint32_t viommu_id;
>> +};
>> +
>> +/*
>> + * XEN_DMOP_q_viommu: Query vIOMMU capabilities.
>> + */
>> +#define XEN_DMOP_query_viommu_caps 17
>> +
>> +struct xen_dm_op_query_viommu_caps {
>> +    /* OUT - vIOMMU Capabilities*/
> 
> Don't you need to also mention which vIOMMU? As you
> could have potentially many of them?

If we want to support different vendors' vIOMMU, it's necessary to do
that and we need to introduce a new field "vIOMMU type" (E,G Intel, AMD
and ARM IOMMU).


> 
>> +    uint64_t caps;
>> +};
>> +
>>  struct xen_dm_op {
>>      uint32_t op;
>>      uint32_t pad;
>> @@ -336,6 +372,9 @@ struct xen_dm_op {
>>          struct xen_dm_op_set_mem_type set_mem_type;
>>          struct xen_dm_op_inject_event inject_event;
>>          struct xen_dm_op_inject_msi inject_msi;
>> +        struct xen_dm_op_create_viommu create_viommu;
>> +        struct xen_dm_op_destroy_viommu destroy_viommu;
>> +        struct xen_dm_op_query_viommu_caps query_viommu_caps;
>>      } u;
>>  };
>>  
>> -- 
>> 1.8.3.1
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 3/23] VIOMMU: Add irq request callback to deal with irq remapping

[Lan Tianyu]

On 2017年04月17日 22:39, Konrad Rzeszutek Wilk wrote:
> On Fri, Mar 17, 2017 at 07:27:03PM +0800, Lan Tianyu wrote:
>> This patch is to add irq request callback for platform implementation
>> to deal with irq remapping request.
>>
>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>> ---
>>  xen/common/viommu.c          | 11 +++++++++++
>>  xen/include/asm-arm/viommu.h |  4 ++++
>>  xen/include/asm-x86/viommu.h | 15 +++++++++++++++
>>  xen/include/xen/viommu.h     |  8 ++++++++
>>  4 files changed, 38 insertions(+)
>>
>> diff --git a/xen/common/viommu.c b/xen/common/viommu.c
>> index 4c1c788..62c66db 100644
>> --- a/xen/common/viommu.c
>> +++ b/xen/common/viommu.c
>> @@ -87,6 +87,17 @@ u64 viommu_query_caps(struct domain *d)
>>      return info->ops->query_caps(d);
>>  }
>>  
>> +int viommu_handle_irq_request(struct domain *d,
>> +        struct irq_remapping_request *request)
>> +{
>> +    struct viommu_info *info = &d->viommu;
>> +
>> +    if ( !info || !info->ops || !info->ops->handle_irq_request)
> 
> You are missing an space at the end.

Yes, will fix.

>> +        return -EINVAL;
>> +
>> +    return info->ops->handle_irq_request(d, request);
>> +}
>> +
>>  /*
>>   * Local variables:
>>   * mode: C
>> diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
>> index ef6a60b..6a81ecb 100644
>> --- a/xen/include/asm-arm/viommu.h
>> +++ b/xen/include/asm-arm/viommu.h
>> @@ -22,6 +22,10 @@
>>  
>>  #include <xen/viommu.h>
>>  
>> +struct irq_remapping_request
>> +{
>> +};
>> +
>>  static inline const struct viommu_ops *viommu_get_ops(void)
>>  {
>>      return NULL;
>> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
>> index efb435f..b6e01a5 100644
>> --- a/xen/include/asm-x86/viommu.h
>> +++ b/xen/include/asm-x86/viommu.h
>> @@ -23,6 +23,21 @@
>>  #include <xen/viommu.h>
>>  #include <asm/types.h>
>>  
>> +struct irq_remapping_request
>> +{
>> +    u8 type;
>> +    u16 source_id;
>> +    union {
>> +        /* MSI */
>> +        struct {
>> +            u64 addr;
>> +            u32 data;
>> +        } msi;
>> +        /* Redirection Entry in IOAPIC */
>> +        u64 rte;
>> +    } msg;
>> +};
> 
> Will this work right? As in with the default padding and such?

Sorry. Could you elaborate this?

>> +
>>  static inline const struct viommu_ops *viommu_get_ops(void)
>>  {
>>      return NULL;
>> diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
>> index a0abbdf..246b29d 100644
>> --- a/xen/include/xen/viommu.h
>> +++ b/xen/include/xen/viommu.h
>> @@ -24,6 +24,10 @@
>>  
>>  #define NR_VIOMMU_PER_DOMAIN 1
>>  
>> +/* IRQ request type */
>> +#define VIOMMU_REQUEST_IRQ_MSI          0
>> +#define VIOMMU_REQUEST_IRQ_APIC         1
> 
> What is this used for?

This is to designate interrupt type of irq remapping request which
contains in the structure irq_remapping_request. The vIOMMU device model
uses it to parse request data.

>> +
>>  struct viommu {
>>      u64 base_address;
>>      u64 length;
>> @@ -36,6 +40,8 @@ struct viommu_ops {
>>      u64 (*query_caps)(struct domain *d);
>>      int (*create)(struct domain *d, struct viommu *viommu);
>>      int (*destroy)(struct viommu *viommu);
>> +    int (*handle_irq_request)(struct domain *d,
>> +                              struct irq_remapping_request *request);
>>  };
>>  
>>  struct viommu_info {
>> @@ -48,6 +54,8 @@ int viommu_init_domain(struct domain *d);
>>  int viommu_create(struct domain *d, u64 base_address, u64 length, u64 caps);
>>  int viommu_destroy(struct domain *d, u32 viommu_id);
>>  u64 viommu_query_caps(struct domain *d);
>> +int viommu_handle_irq_request(struct domain *d,
>> +                              struct irq_remapping_request *request);
>>  
>>  #endif /* __XEN_VIOMMU_H__ */
>>  
>> -- 
>> 1.8.3.1
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 00/23] xen/vIOMMU: Add vIOMMU support with irq remapping fucntion on Intel platform

[Lan Tianyu]

On 2017年04月17日 22:41, Konrad Rzeszutek Wilk wrote:
> On Mon, Mar 20, 2017 at 02:23:02PM +0000, Roger Pau Monné wrote:
>> On Fri, Mar 17, 2017 at 07:27:00PM +0800, Lan Tianyu wrote:
>>> This patchset is to introduce vIOMMU framework and add virtual VTD's
>>> interrupt remapping support according "Xen virtual IOMMU high level
>>> design doc V3"(https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.xenproject.org_archives_html_xen-2Ddevel_&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=wAkdPB9j1dAH7AI494B5wFV3Jws7EfB2Q3Sw-K-88Rk&m=7dZfaODS8zbwpYC0vm7gKQXyM8pBPxfGpz8QMDQzU2k&s=3hxzmHH4X0gz9Oz5_PYoOmWFTkyETYTFPCqJ9iXD910&e= 
>>> 2016-11/msg01391.html).
> 
> It would be awesome if that was as a patch in docs/misc/

Will do that.

> 
> Thanks.
> 
>>>
>>> - vIOMMU framework
>>> New framework provides viommu_ops and help functions to abstract
>>> vIOMMU operations(E,G create, destroy, handle irq remapping request
>>> and so on). Vendors(Intel, ARM, AMD and son) can implement their
>>> vIOMMU callbacks.
>>>
>>> - Xen vIOMMU device model in Qemu 
>>> It's in charge of create/destroy vIOMMU in hypervisor via new vIOMMU
>>> DMOP hypercalls. It will be required to pass virtual devices DMA
>>> request to hypervisor when enable IOVA(DMA request without PASID)
>>> function.
>>>
>>> - Virtual VTD
>>> In this patchset, we enable irq remapping function and covers both
>>> MSI and IOAPIC interrupts. Don't support post interrupt mode emulation
>>> and post interrupt mode enabled on host with virtual VTD. Will add
>>> later.   
>>>
>>> Chao Gao (19):
>>>   Tools/libxc: Add viommu operations in libxc
>>>   Tools/libacpi: Add DMA remapping reporting (DMAR) ACPI table
>>>     structures
>>>   Tools/libacpi: Add new fields in acpi_config to build DMAR table
>>>   Tools/libacpi: Add a user configurable parameter to control vIOMMU
>>>     attributes
>>>   Tools/libxl: Inform device model to create a guest with a vIOMMU
>>>     device
>>>   x86/hvm: Introduce a emulated VTD for HVM
>>>   X86/vvtd: Add MMIO handler for VVTD
>>>   X86/vvtd: Set Interrupt Remapping Table Pointer through GCMD
>>>   X86/vvtd: Process interrupt remapping request
>>>   X86/vvtd: decode interrupt attribute from IRTE
>>>   X86/vioapic: Hook interrupt delivery of vIOAPIC
>>>   X86/vvtd: Enable Queued Invalidation through GCMD
>>>   X86/vvtd: Enable Interrupt Remapping through GCMD
>>>   x86/vpt: Get interrupt vector through a vioapic interface
>>>   passthrough: move some fields of hvm_gmsi_info to a sub-structure
>>>   Tools/libxc: Add a new interface to bind msi-ir with pirq
>>>   X86/vmsi: Hook guest MSI injection
>>>   X86/vvtd: Handle interrupt translation faults
>>>   X86/vvtd: Add queued invalidation (QI) support
>>>
>>> Lan Tianyu (4):
>>>   VIOMMU: Add vIOMMU helper functions to create, destroy and query
>>>     capabilities
>>>   DMOP: Introduce new DMOP commands for vIOMMU support
>>>   VIOMMU: Add irq request callback to deal with irq remapping
>>>   VIOMMU: Add get irq info callback to convert irq remapping request
>>>
>>>  tools/libacpi/acpi2_0.h                         |   45 +
>>>  tools/libacpi/build.c                           |   58 ++
>>>  tools/libacpi/libacpi.h                         |   12 +
>>>  tools/libs/devicemodel/core.c                   |   69 ++
>>>  tools/libs/devicemodel/include/xendevicemodel.h |   35 +
>>>  tools/libs/devicemodel/libxendevicemodel.map    |    3 +
>>>  tools/libxc/include/xenctrl.h                   |   17 +
>>>  tools/libxc/include/xenctrl_compat.h            |    5 +
>>>  tools/libxc/xc_devicemodel_compat.c             |   18 +
>>>  tools/libxc/xc_domain.c                         |   55 +
>>>  tools/libxl/libxl_create.c                      |   12 +-
>>>  tools/libxl/libxl_dm.c                          |    9 +
>>>  tools/libxl/libxl_dom.c                         |   85 ++
>>>  tools/libxl/libxl_types.idl                     |    8 +
>>>  tools/xl/xl_parse.c                             |   54 +
>>>  xen/arch/x86/Makefile                           |    1 +
>>>  xen/arch/x86/hvm/Makefile                       |    1 +
>>>  xen/arch/x86/hvm/dm.c                           |   29 +
>>>  xen/arch/x86/hvm/irq.c                          |   10 +
>>>  xen/arch/x86/hvm/vioapic.c                      |   36 +
>>>  xen/arch/x86/hvm/vmsi.c                         |   17 +-
>>>  xen/arch/x86/hvm/vpt.c                          |    2 +-
>>>  xen/arch/x86/hvm/vvtd.c                         | 1229 +++++++++++++++++++++++
>>>  xen/arch/x86/viommu.c                           |   40 +
>>>  xen/common/Makefile                             |    1 +
>>>  xen/common/domain.c                             |    3 +
>>>  xen/common/viommu.c                             |  119 +++
>>>  xen/drivers/passthrough/io.c                    |  183 +++-
>>>  xen/drivers/passthrough/vtd/iommu.h             |  213 +++-
>>>  xen/include/asm-arm/viommu.h                    |   38 +
>>>  xen/include/asm-x86/hvm/vioapic.h               |    1 +
>>>  xen/include/asm-x86/msi.h                       |    3 +
>>>  xen/include/asm-x86/viommu.h                    |   68 ++
>>>  xen/include/public/arch-x86/hvm/save.h          |   19 +
>>>  xen/include/public/domctl.h                     |    7 +
>>>  xen/include/public/hvm/dm_op.h                  |   39 +
>>>  xen/include/public/viommu.h                     |   38 +
>>>  xen/include/xen/hvm/irq.h                       |   20 +-
>>>  xen/include/xen/sched.h                         |    2 +
>>>  xen/include/xen/viommu.h                        |   74 ++
>>>  40 files changed, 2601 insertions(+), 77 deletions(-)
>>>  create mode 100644 xen/arch/x86/hvm/vvtd.c
>>>  create mode 100644 xen/arch/x86/viommu.c
>>>  create mode 100644 xen/common/viommu.c
>>>  create mode 100644 xen/include/asm-arm/viommu.h
>>>  create mode 100644 xen/include/asm-x86/viommu.h
>>>  create mode 100644 xen/include/public/viommu.h
>>>  create mode 100644 xen/include/xen/viommu.h
>>
>> Thanks! So you add all this vIOMMU code, but the maximum number of allowed
>> vCPUs for HVM guests is still limited to 128 (HVM_MAX_VCPUS is not touched). Is
>> there any missing pieces in order to bump this?
>>
>> Also, have you tested if this series works with PVH guests? Boris added PVH
>> support to Linux not long ago, so you should be able to test it just by picking
>> the latest Linux kernel.
>>
>> Roger.
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.xen.org_xen-2Ddevel&d=DwIGaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=wAkdPB9j1dAH7AI494B5wFV3Jws7EfB2Q3Sw-K-88Rk&m=7dZfaODS8zbwpYC0vm7gKQXyM8pBPxfGpz8QMDQzU2k&s=RinrlfGer5iCrg48EvpWXwDjHMkLhdtKjdhnsNhzUIo&e= 


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 15/23] X86/vioapic: Hook interrupt delivery of vIOAPIC

[Lan Tianyu]

On 2017年04月17日 22:43, Konrad Rzeszutek Wilk wrote:
> On Fri, Mar 17, 2017 at 07:27:15PM +0800, Lan Tianyu wrote:
>> From: Chao Gao <chao.gao@intel.com>
>>
>> When irq remapping enabled, IOAPIC Redirection Entry maybe is in remapping
>> format. If that, generate a irq_remapping_request and send it to domain.
>>
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>> ---
>>  xen/arch/x86/Makefile                  |  1 +
>>  xen/arch/x86/hvm/vioapic.c             | 10 ++++++++++
>>  xen/arch/x86/viommu.c                  | 30 ++++++++++++++++++++++++++++++
>>  xen/include/asm-x86/viommu.h           |  3 +++
>>  xen/include/public/arch-x86/hvm/save.h |  1 +
>>  5 files changed, 45 insertions(+)
>>  create mode 100644 xen/arch/x86/viommu.c
>>
>> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
>> index f75eca0..d49f8c8 100644
>> --- a/xen/arch/x86/Makefile
>> +++ b/xen/arch/x86/Makefile
>> @@ -66,6 +66,7 @@ obj-y += usercopy.o
>>  obj-y += x86_emulate.o
>>  obj-$(CONFIG_TBOOT) += tboot.o
>>  obj-y += hpet.o
>> +obj-y += viommu.o
>>  obj-y += vm_event.o
>>  obj-y += xstate.o
>>  
>> diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
>> index fdbb21f..6a00644 100644
>> --- a/xen/arch/x86/hvm/vioapic.c
>> +++ b/xen/arch/x86/hvm/vioapic.c
>> @@ -30,6 +30,7 @@
>>  #include <xen/lib.h>
>>  #include <xen/errno.h>
>>  #include <xen/sched.h>
>> +#include <xen/viommu.h>
>>  #include <public/hvm/ioreq.h>
>>  #include <asm/hvm/io.h>
>>  #include <asm/hvm/vpic.h>
>> @@ -285,9 +286,18 @@ static void vioapic_deliver(struct hvm_hw_vioapic *vioapic, int irq)
>>      struct domain *d = vioapic_domain(vioapic);
>>      struct vlapic *target;
>>      struct vcpu *v;
>> +    struct irq_remapping_request request;
>>  
>>      ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
>>  
>> +    if ( vioapic->redirtbl[irq].ir.format )
>> +    {
>> +        irq_request_ioapic_fill(&request, vioapic->id,
>> +                                vioapic->redirtbl[irq].bits);
>> +        viommu_handle_irq_request(d, &request);
>> +        return;
>> +    }
>> +
>>      HVM_DBG_LOG(DBG_LEVEL_IOAPIC,
>>                  "dest=%x dest_mode=%x delivery_mode=%x "
>>                  "vector=%x trig_mode=%x",
>> diff --git a/xen/arch/x86/viommu.c b/xen/arch/x86/viommu.c
>> new file mode 100644
>> index 0000000..ef78d3b
>> --- /dev/null
>> +++ b/xen/arch/x86/viommu.c
>> @@ -0,0 +1,30 @@
>> +/*
>> + * viommu.c
>> + *
>> + * virtualize IOMMU.
>> + *
>> + * Copyright (C) 2017 Chao Gao, Intel Corporation.
>> + *
>> + * This program is free software; you can redistribute it and/or
>> + * modify it under the terms and conditions of the GNU General Public
>> + * License, version 2, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public
>> + * License along with this program; If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <xen/viommu.h>
>> +
>> +void irq_request_ioapic_fill(struct irq_remapping_request *req,
>> +                             uint32_t ioapic_id, uint64_t rte)
>> +{
>> +    ASSERT(req);
>> +    req->type = VIOMMU_REQUEST_IRQ_APIC;
>> +    req->source_id = ioapic_id;
>> +    req->msg.rte = rte;
> 
> Considering we get 'req' from the stack and it may have garbage, would
> it be good to fill out the rest of the entries with sensible values? Or
> is there no need for that?

Both AMD and Intel will use the function to pass interrupt remapping
request. I am afraid different vendors may have different IOAPIC
remapping format. How about to parse and check remapping request data in
the vendor vIOMMU device module(E,G vvtd)? :)

>> +}
> 
> This being a new file, you should probably include the nice
> editor configuration block.

OK. Will add it.

> 
>> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
>> index 0b25f34..fcf3c24 100644
>> --- a/xen/include/asm-x86/viommu.h
>> +++ b/xen/include/asm-x86/viommu.h
>> @@ -49,6 +49,9 @@ struct irq_remapping_request
>>      } msg;
>>  };
>>  
>> +void irq_request_ioapic_fill(struct irq_remapping_request *req,
>> +                             uint32_t ioapic_id, uint64_t rte);
>> +
>>  static inline const struct viommu_ops *viommu_get_ops(void)
>>  {
>>      /*
>> diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
>> index 6127f89..06be4a5 100644
>> --- a/xen/include/public/arch-x86/hvm/save.h
>> +++ b/xen/include/public/arch-x86/hvm/save.h
>> @@ -401,6 +401,7 @@ struct hvm_hw_vioapic {
>>              uint8_t reserved[4];
>>              uint8_t dest_id;
>>          } fields;
>> +        struct ir_ioapic_rte ir;
>>      } redirtbl[VIOAPIC_NUM_PINS];
>>  };
>>  
>> -- 
>> 1.8.3.1
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> https://lists.xen.org/xen-devel


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Paul Durrant]

> -----Original Message-----
> From: Lan, Tianyu [mailto:tianyu.lan@intel.com]
> Sent: 14 April 2017 16:38
> To: Paul Durrant <Paul.Durrant@citrix.com>; Wei Liu <wei.liu2@citrix.com>;
> Kevin Tian <kevin.tian@intel.com>; Ian Jackson <Ian.Jackson@citrix.com>;
> xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
> operations in libxc
> 
> Hi Paul:
> 	Sorry for later response.
> 
> On 3/31/2017 3:57 AM, Chao Gao wrote:
> > On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
> >>> -----Original Message-----
> >>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf
> Of
> >>> Chao Gao
> >>> Sent: 29 March 2017 01:40
> >>> To: Wei Liu <wei.liu2@citrix.com>
> >>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian
> <kevin.tian@intel.com>;
> >>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
> >>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
> >>> operations in libxc
> >>>
> >>> Tianyu is on vacation this two weeks, so I will try to address
> >>> some comments on this series.
> >>>
> >>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
> >>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
> >>>>> From: Chao Gao <chao.gao@intel.com>
> >>>>>
> >>>>> In previous patch, we introduce a common vIOMMU layer. In our
> design,
> >>>>> we create/destroy vIOMMU through DMOP interface instead of
> creating
> >>> it
> >>>>> according to a config flag of domain. It makes it is possible
> >>>>> to create vIOMMU in device model or in tool stack.
> >>>>>
> >>
> >> I've not been following this closely so apologies if this has already been
> asked...
> >>
> >> Why would you need to create a vIOMMU instance in an external device
> model.
> >> Since the toolstack should be in control of the device model configuration
> why would it not know in advance that one was required?
> >
> > I assume your question is why we don't create a vIOMMU instance via
> hypercall in toolstack.
> > I think creating in toolstack is also ok and is easier to be reused by pvh.
> >
> > If Tianyu has no concern about this, will move this part to toolstack.
> 
> We can move create/destroy vIOMMU in the tool stack but we still need to
> add such dummy vIOMMU device model in Qemu to pass virtual device's
> DMA
> request into Xen hypervisor.

Not quite sure I understand this. The QEMu device model does not 'pass DMA requests' as such, it maps guest RAM and reads or writes to emulate DMA, right? So, what's needed is a mechanism to map guest RAM by 'bus address'... i.e. an address that will need to be translated through the vIOMMU mappings. This is just an evolution of the current 'priv mapping' operations that allow guest RAM to be mapped by guest physical address. So you don't need a vIOMMU 'device model' as such, do you?

> Qemu is required to use DMOP hypercall and
> tool stack may use domctl hyercall. vIOMMU hypercalls will be divided
> into two part.
> 
> Domctl:
> 	create, destroy and query.
> DMOP:
> 	vDev's DMA related operations.

Yes, the mapping/unmapping operations should be DMOPs and IMO should be designed such that they can be unified with replacements for current 'priv map' ops such that QEMU can use the same function call, but with different address space identifiers (i.e. bus address, guest physical address, etc.). BTW, I say 'etc.' because we should also consider mapping the ioreq pages from Xen using the same call - with a dedicated address space identifier - as well.

  Cheers,

    Paul

> 
> Is this OK?
> 
> >
> > Thanks,
> > Chao
> >
> >>
> >>  Paul
> >>
> >>>>> The following toolstack code is to add XEN_DMOP_viommu_XXX
> syscalls:
> >>>>
> >>>> Hypercalls, not syscalls.
> >>>>
> >>>>>  - query capabilities of vIOMMU emulated by Xen
> >>>>>  - create vIOMMU in Xen hypervisor with base address, capability
> >>>>>  - destroy vIOMMU specified by viommu_id
> >>>>>
> >>>>> Signed-off-by: Chao Gao <chao.gao@intel.com>
> >>>>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >>>>> ---
> >>>>>  tools/libs/devicemodel/core.c                   | 69
> >>> +++++++++++++++++++++++++
> >>>>>  tools/libs/devicemodel/include/xendevicemodel.h | 35
> +++++++++++++
> >>>>>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
> >>>>>  tools/libxc/include/xenctrl_compat.h            |  5 ++
> >>>>>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
> >>>>>  5 files changed, 130 insertions(+)
> >>>>>
> >>>>> diff --git a/tools/libs/devicemodel/core.c
> b/tools/libs/devicemodel/core.c
> >>>>> index a85cb49..aee1150 100644
> >>>>> --- a/tools/libs/devicemodel/core.c
> >>>>> +++ b/tools/libs/devicemodel/core.c
> >>>>
> >>>> Bear in mind that this library is stable, so whatever ends up here can
> >>>> change in the future.
> >>>>
> >>>> This is not saying the following code is problematic. It is just a
> >>>> general FYI.
> >>>>
> >>>> Obviously the toolstack side is going to follow the hypervisor
> >>>> interface, so I will do a detailed review later.
> >>>
> >>> Sure. If the hypervisor interface settles down, we can inform you.
> >>>
> >>>>
> >>>>> +int xendevicemodel_viommu_destroy(
> >>>>> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t
> viommu_id);
> >>>>>  #endif /* __XEN_TOOLS__ */
> >>>>>
> >>>>>  #endif /* XENDEVICEMODEL_H */
> >>>>> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
> >>> b/tools/libs/devicemodel/libxendevicemodel.map
> >>>>> index 45c773e..c2e0968 100644
> >>>>> --- a/tools/libs/devicemodel/libxendevicemodel.map
> >>>>> +++ b/tools/libs/devicemodel/libxendevicemodel.map
> >>>>> @@ -17,6 +17,9 @@ VERS_1.0 {
> >>>>>  		xendevicemodel_modified_memory;
> >>>>>  		xendevicemodel_set_mem_type;
> >>>>>  		xendevicemodel_inject_event;
> >>>>> +		xendevicemodel_viommu_query_cap;
> >>>>> +		xendevicemodel_viommu_create;
> >>>>> +		xendevicemodel_viommu_destroy;
> >>>>>  		xendevicemodel_restrict;
> >>>>>  		xendevicemodel_close;
> >>>>
> >>>> I suppose this series is going to miss 4.9.
> >>>>
> >>>> Please add these functions to VERS_1.1.
> >>>
> >>> Yes. We will fix this.
> >>>
> >>>>
> >>>>>  	local: *; /* Do not expose anything by default */
> >>>>> diff --git a/tools/libxc/include/xenctrl_compat.h
> >>> b/tools/libxc/include/xenctrl_compat.h
> >>>>> index 040e7b2..315c45d 100644
> >>>>> --- a/tools/libxc/include/xenctrl_compat.h
> >>>>> +++ b/tools/libxc/include/xenctrl_compat.h
> >>>>> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
> >>>>>  int xc_hvm_inject_trap(
> >>>>>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
> >>>>>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
> >>>>> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom,
> uint64_t
> >>> *cap);
> >>>>> +int xc_viommu_create(
> >>>>> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t
> cap,
> >>>>> +    uint32_t *viommu_id);
> >>>>> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t
> >>> viommu_id);
> >>>>>
> >>>>>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
> >>>>>
> >>>>> diff --git a/tools/libxc/xc_devicemodel_compat.c
> >>> b/tools/libxc/xc_devicemodel_compat.c
> >>>>> index e4edeea..62f703a 100644
> >>>>> --- a/tools/libxc/xc_devicemodel_compat.c
> >>>>> +++ b/tools/libxc/xc_devicemodel_compat.c
> >>>>
> >>>> I don't think you need to provide compat wrappers for them. They are
> new
> >>>> APIs.
> >>>
> >>> OK. Got it.
> >>>
> >>> Thanks,
> >>> Chao
> >>>>
> >>>> Wei.
> >>>
> >>> _______________________________________________
> >>> Xen-devel mailing list
> >>> Xen-devel@lists.xen.org
> >>> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan Tianyu]

On 2017年04月18日 17:08, Paul Durrant wrote:
>> -----Original Message-----
>> From: Lan, Tianyu [mailto:tianyu.lan@intel.com]
>> Sent: 14 April 2017 16:38
>> To: Paul Durrant <Paul.Durrant@citrix.com>; Wei Liu <wei.liu2@citrix.com>;
>> Kevin Tian <kevin.tian@intel.com>; Ian Jackson <Ian.Jackson@citrix.com>;
>> xen-devel@lists.xen.org
>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>> operations in libxc
>>
>> Hi Paul:
>> 	Sorry for later response.
>>
>> On 3/31/2017 3:57 AM, Chao Gao wrote:
>>> On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
>>>>> -----Original Message-----
>>>>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf
>> Of
>>>>> Chao Gao
>>>>> Sent: 29 March 2017 01:40
>>>>> To: Wei Liu <wei.liu2@citrix.com>
>>>>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian
>> <kevin.tian@intel.com>;
>>>>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
>>>>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>>>>> operations in libxc
>>>>>
>>>>> Tianyu is on vacation this two weeks, so I will try to address
>>>>> some comments on this series.
>>>>>
>>>>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>>>>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>>>>>>> From: Chao Gao <chao.gao@intel.com>
>>>>>>>
>>>>>>> In previous patch, we introduce a common vIOMMU layer. In our
>> design,
>>>>>>> we create/destroy vIOMMU through DMOP interface instead of
>> creating
>>>>> it
>>>>>>> according to a config flag of domain. It makes it is possible
>>>>>>> to create vIOMMU in device model or in tool stack.
>>>>>>>
>>>>
>>>> I've not been following this closely so apologies if this has already been
>> asked...
>>>>
>>>> Why would you need to create a vIOMMU instance in an external device
>> model.
>>>> Since the toolstack should be in control of the device model configuration
>> why would it not know in advance that one was required?
>>>
>>> I assume your question is why we don't create a vIOMMU instance via
>> hypercall in toolstack.
>>> I think creating in toolstack is also ok and is easier to be reused by pvh.
>>>
>>> If Tianyu has no concern about this, will move this part to toolstack.
>>
>> We can move create/destroy vIOMMU in the tool stack but we still need to
>> add such dummy vIOMMU device model in Qemu to pass virtual device's
>> DMA
>> request into Xen hypervisor.
> 
> Not quite sure I understand this. The QEMu device model does not 'pass DMA requests' as such, it maps guest RAM and reads or writes to emulate DMA, right? So, what's needed is a mechanism to map guest RAM by 'bus address'... i.e. an address that will need to be translated through the vIOMMU mappings. This is just an evolution of the current 'priv mapping' operations that allow guest RAM to be mapped by guest physical address. So you don't need a vIOMMU 'device model' as such, do you?


Guest also may enable DMA protection mechanism in linux kernel which
limits address space of emulated device and this depends on the vIOMMU's
DMA translation function. In vIOMMU's MMIO emulation part is in the Xen
hypersior and the guest shadow IO page table will be only in the
hypervisor. To translate emulated device's DMA request. It's necessary
to pass the DMA request to hypervisor.

So far we don't support DMA translation and so doesn't pass DMA request.

Map/umap guest memory already support in Qemu and just like emulated
device model access guest memory. Qemu also provides vIOMMU hook to
receive DMA request and return target guest address. vIOMMU framework
will read/write target address. What we need to do is to translate DMA
request to target address according shadow IO page table in the hypervisor.



> 
>> Qemu is required to use DMOP hypercall and
>> tool stack may use domctl hyercall. vIOMMU hypercalls will be divided
>> into two part.
>>
>> Domctl:
>> 	create, destroy and query.
>> DMOP:
>> 	vDev's DMA related operations.
> 
> Yes, the mapping/unmapping operations should be DMOPs and IMO should be designed such that they can be unified with replacements for current 'priv map' ops such that QEMU can use the same function call, but with different address space identifiers (i.e. bus address, guest physical address, etc.). BTW, I say 'etc.' because we should also consider mapping the ioreq pages from Xen using the same call - with a dedicated address space identifier - as well.
> 

So you agree to divide vIOMMU's hypercalls into two parts(DMOP and
Domctl), right?


>   Cheers,
> 
>     Paul
> 
>>
>> Is this OK?
>>
>>>
>>> Thanks,
>>> Chao
>>>
>>>>
>>>>  Paul
>>>>
>>>>>>> The following toolstack code is to add XEN_DMOP_viommu_XXX
>> syscalls:
>>>>>>
>>>>>> Hypercalls, not syscalls.
>>>>>>
>>>>>>>  - query capabilities of vIOMMU emulated by Xen
>>>>>>>  - create vIOMMU in Xen hypervisor with base address, capability
>>>>>>>  - destroy vIOMMU specified by viommu_id
>>>>>>>
>>>>>>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>>>>>>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
>>>>>>> ---
>>>>>>>  tools/libs/devicemodel/core.c                   | 69
>>>>> +++++++++++++++++++++++++
>>>>>>>  tools/libs/devicemodel/include/xendevicemodel.h | 35
>> +++++++++++++
>>>>>>>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
>>>>>>>  tools/libxc/include/xenctrl_compat.h            |  5 ++
>>>>>>>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
>>>>>>>  5 files changed, 130 insertions(+)
>>>>>>>
>>>>>>> diff --git a/tools/libs/devicemodel/core.c
>> b/tools/libs/devicemodel/core.c
>>>>>>> index a85cb49..aee1150 100644
>>>>>>> --- a/tools/libs/devicemodel/core.c
>>>>>>> +++ b/tools/libs/devicemodel/core.c
>>>>>>
>>>>>> Bear in mind that this library is stable, so whatever ends up here can
>>>>>> change in the future.
>>>>>>
>>>>>> This is not saying the following code is problematic. It is just a
>>>>>> general FYI.
>>>>>>
>>>>>> Obviously the toolstack side is going to follow the hypervisor
>>>>>> interface, so I will do a detailed review later.
>>>>>
>>>>> Sure. If the hypervisor interface settles down, we can inform you.
>>>>>
>>>>>>
>>>>>>> +int xendevicemodel_viommu_destroy(
>>>>>>> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t
>> viommu_id);
>>>>>>>  #endif /* __XEN_TOOLS__ */
>>>>>>>
>>>>>>>  #endif /* XENDEVICEMODEL_H */
>>>>>>> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
>>>>> b/tools/libs/devicemodel/libxendevicemodel.map
>>>>>>> index 45c773e..c2e0968 100644
>>>>>>> --- a/tools/libs/devicemodel/libxendevicemodel.map
>>>>>>> +++ b/tools/libs/devicemodel/libxendevicemodel.map
>>>>>>> @@ -17,6 +17,9 @@ VERS_1.0 {
>>>>>>>  		xendevicemodel_modified_memory;
>>>>>>>  		xendevicemodel_set_mem_type;
>>>>>>>  		xendevicemodel_inject_event;
>>>>>>> +		xendevicemodel_viommu_query_cap;
>>>>>>> +		xendevicemodel_viommu_create;
>>>>>>> +		xendevicemodel_viommu_destroy;
>>>>>>>  		xendevicemodel_restrict;
>>>>>>>  		xendevicemodel_close;
>>>>>>
>>>>>> I suppose this series is going to miss 4.9.
>>>>>>
>>>>>> Please add these functions to VERS_1.1.
>>>>>
>>>>> Yes. We will fix this.
>>>>>
>>>>>>
>>>>>>>  	local: *; /* Do not expose anything by default */
>>>>>>> diff --git a/tools/libxc/include/xenctrl_compat.h
>>>>> b/tools/libxc/include/xenctrl_compat.h
>>>>>>> index 040e7b2..315c45d 100644
>>>>>>> --- a/tools/libxc/include/xenctrl_compat.h
>>>>>>> +++ b/tools/libxc/include/xenctrl_compat.h
>>>>>>> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
>>>>>>>  int xc_hvm_inject_trap(
>>>>>>>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
>>>>>>>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t cr2);
>>>>>>> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom,
>> uint64_t
>>>>> *cap);
>>>>>>> +int xc_viommu_create(
>>>>>>> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t
>> cap,
>>>>>>> +    uint32_t *viommu_id);
>>>>>>> +int xc_viommu_destroy(xc_interface *xch, domid_t dom, uint32_t
>>>>> viommu_id);
>>>>>>>
>>>>>>>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
>>>>>>>
>>>>>>> diff --git a/tools/libxc/xc_devicemodel_compat.c
>>>>> b/tools/libxc/xc_devicemodel_compat.c
>>>>>>> index e4edeea..62f703a 100644
>>>>>>> --- a/tools/libxc/xc_devicemodel_compat.c
>>>>>>> +++ b/tools/libxc/xc_devicemodel_compat.c
>>>>>>
>>>>>> I don't think you need to provide compat wrappers for them. They are
>> new
>>>>>> APIs.
>>>>>
>>>>> OK. Got it.
>>>>>
>>>>> Thanks,
>>>>> Chao
>>>>>>
>>>>>> Wei.
>>>>>
>>>>> _______________________________________________
>>>>> Xen-devel mailing list
>>>>> Xen-devel@lists.xen.org
>>>>> https://lists.xen.org/xen-devel


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 2/23] DMOP: Introduce new DMOP commands for vIOMMU support

[Konrad Rzeszutek Wilk]

On Tue, Apr 18, 2017 at 03:24:35PM +0800, Lan Tianyu wrote:
> Hi Konrad:
> 	Thanks for your review.
> 
> On 2017年04月17日 22:36, Konrad Rzeszutek Wilk wrote:
> > On Fri, Mar 17, 2017 at 07:27:02PM +0800, Lan Tianyu wrote:
> >> This patch is to introduce create, destroy and query capabilities
> >> command for vIOMMU. vIOMMU layer will deal with requests and call
> >> arch vIOMMU ops.
> >>
> >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >> ---
> >>  xen/arch/x86/hvm/dm.c          | 29 +++++++++++++++++++++++++++++
> >>  xen/include/public/hvm/dm_op.h | 39 +++++++++++++++++++++++++++++++++++++++
> >>  2 files changed, 68 insertions(+)
> >>
> >> diff --git a/xen/arch/x86/hvm/dm.c b/xen/arch/x86/hvm/dm.c
> >> index 2122c45..2b28f70 100644
> >> --- a/xen/arch/x86/hvm/dm.c
> >> +++ b/xen/arch/x86/hvm/dm.c
> >> @@ -491,6 +491,35 @@ static int dm_op(domid_t domid,
> >>          break;
> >>      }
> >>  
> >> +    case XEN_DMOP_create_viommu:
> >> +    {
> >> +        struct xen_dm_op_create_viommu *data =
> >> +            &op.u.create_viommu;
> >> +
> >> +        rc = viommu_create(d, data->base_address, data->length, data->capabilities);
> >> +        if (rc >= 0) {
> > 
> > The style guide is is to have a space here and { on a newline.
> 
> Yes, will fix.
> 
> > 
> >> +            data->viommu_id = rc;
> >> +            rc = 0;
> >> +        }
> >> +        break;
> >> +    }
> > 
> > Newline here..
> > 
> > 
> >> +    case XEN_DMOP_destroy_viommu:
> >> +    {
> >> +        const struct xen_dm_op_destroy_viommu *data =
> >> +            &op.u.destroy_viommu;
> >> +
> >> +        rc = viommu_destroy(d, data->viommu_id);
> >> +        break;
> >> +    }
> > 
> > Ahem?
> >> +    case XEN_DMOP_query_viommu_caps:
> >> +    {
> >> +        struct xen_dm_op_query_viommu_caps *data =
> >> +            &op.u.query_viommu_caps;
> >> +
> >> +        data->caps = viommu_query_caps(d);
> >> +        rc = 0;
> >> +        break;
> >> +    }
> > 
> > And here.
> >>      default:
> >>          rc = -EOPNOTSUPP;
> >>          break;
> >> diff --git a/xen/include/public/hvm/dm_op.h b/xen/include/public/hvm/dm_op.h
> >> index f54cece..b8c7359 100644
> >> --- a/xen/include/public/hvm/dm_op.h
> >> +++ b/xen/include/public/hvm/dm_op.h
> >> @@ -318,6 +318,42 @@ struct xen_dm_op_inject_msi {
> >>      uint64_aligned_t addr;
> >>  };
> >>  
> >> +/*
> >> + * XEN_DMOP_create_viommu: Create vIOMMU device.
> >> + */
> >> +#define XEN_DMOP_create_viommu 15
> >> +
> >> +struct xen_dm_op_create_viommu {
> >> +    /* IN - MMIO base address of vIOMMU */
> > 
> > Any limit? Can it be zero?
> 
> In current patchset, base address is allocated by toolstack and passed
> to Qemu to create vIOMMU in hyervisor. Toolstack should make sure the
> range won't be conflicted with other resource.

Sure, but the hypervisor should also do some sanity checking. Having some idea
of limits/sizes would be quite helpfull. Either in the code or in this
comment.
> 
> > 
> >> +    uint64_t base_address;
> >> +    /* IN - Length of MMIO region */
> > 
> > Any restrictions? Can it be say 2 bytes? Or is this in page-size granularity?
> 
> >From the VTD spec, register size must be an integer multiple of 4KB and
> I think the vIOMMU device model(E,G vvtd) in hypervisor should check the
> lengh. Different vendor may have different restriction.

Right, but I thinking you should document this, or at least make it clear
what the expectations are. 
> 
> > 
> >> +    uint64_t length;
> >> +    /* IN - Capabilities with which we want to create */
> >> +    uint64_t capabilities;
> > 
> > That sounds like some form of flags?
> 
> Yes, this patchset just introduces interrupt remapping flag and other
> vendor also can use it to add new features.
> 
> > 
> >> +    /* OUT - vIOMMU identity */
> >> +    uint32_t viommu_id;
> >> +};
> >> +
> >> +/*
> >> + * XEN_DMOP_destroy_viommu: Destroy vIOMMU device.
> >> + */
> >> +#define XEN_DMOP_destroy_viommu 16
> >> +
> >> +struct xen_dm_op_destroy_viommu {
> >> +    /* OUT - vIOMMU identity */
> > 
> > Out? Not in?
> 
> Sorry, it should be OUT parameter.
> 
> > 
> >> +    uint32_t viommu_id;
> >> +};
> >> +
> >> +/*
> >> + * XEN_DMOP_q_viommu: Query vIOMMU capabilities.
> >> + */
> >> +#define XEN_DMOP_query_viommu_caps 17
> >> +
> >> +struct xen_dm_op_query_viommu_caps {
> >> +    /* OUT - vIOMMU Capabilities*/
> > 
> > Don't you need to also mention which vIOMMU? As you
> > could have potentially many of them?
> 
> If we want to support different vendors' vIOMMU, it's necessary to do
> that and we need to introduce a new field "vIOMMU type" (E,G Intel, AMD
> and ARM IOMMU).

Right, but the 'xen_dm_op_create_viommu' has the capabilities
and retuirns the vIOMMU identity. It looks like it could be called
multiple times which means you could have multiple vIOMMU and
each could have a different capability.

As such this call should also have as IN the vIOMMU identity
to at least be symmetrical with the other hypercalls.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 3/23] VIOMMU: Add irq request callback to deal with irq remapping

[Konrad Rzeszutek Wilk]

On Tue, Apr 18, 2017 at 04:18:52PM +0800, Lan Tianyu wrote:
> On 2017年04月17日 22:39, Konrad Rzeszutek Wilk wrote:
> > On Fri, Mar 17, 2017 at 07:27:03PM +0800, Lan Tianyu wrote:
> >> This patch is to add irq request callback for platform implementation
> >> to deal with irq remapping request.
> >>
> >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >> ---
> >>  xen/common/viommu.c          | 11 +++++++++++
> >>  xen/include/asm-arm/viommu.h |  4 ++++
> >>  xen/include/asm-x86/viommu.h | 15 +++++++++++++++
> >>  xen/include/xen/viommu.h     |  8 ++++++++
> >>  4 files changed, 38 insertions(+)
> >>
> >> diff --git a/xen/common/viommu.c b/xen/common/viommu.c
> >> index 4c1c788..62c66db 100644
> >> --- a/xen/common/viommu.c
> >> +++ b/xen/common/viommu.c
> >> @@ -87,6 +87,17 @@ u64 viommu_query_caps(struct domain *d)
> >>      return info->ops->query_caps(d);
> >>  }
> >>  
> >> +int viommu_handle_irq_request(struct domain *d,
> >> +        struct irq_remapping_request *request)
> >> +{
> >> +    struct viommu_info *info = &d->viommu;
> >> +
> >> +    if ( !info || !info->ops || !info->ops->handle_irq_request)
> > 
> > You are missing an space at the end.
> 
> Yes, will fix.
> 
> >> +        return -EINVAL;
> >> +
> >> +    return info->ops->handle_irq_request(d, request);
> >> +}
> >> +
> >>  /*
> >>   * Local variables:
> >>   * mode: C
> >> diff --git a/xen/include/asm-arm/viommu.h b/xen/include/asm-arm/viommu.h
> >> index ef6a60b..6a81ecb 100644
> >> --- a/xen/include/asm-arm/viommu.h
> >> +++ b/xen/include/asm-arm/viommu.h
> >> @@ -22,6 +22,10 @@
> >>  
> >>  #include <xen/viommu.h>
> >>  
> >> +struct irq_remapping_request
> >> +{
> >> +};
> >> +
> >>  static inline const struct viommu_ops *viommu_get_ops(void)
> >>  {
> >>      return NULL;
> >> diff --git a/xen/include/asm-x86/viommu.h b/xen/include/asm-x86/viommu.h
> >> index efb435f..b6e01a5 100644
> >> --- a/xen/include/asm-x86/viommu.h
> >> +++ b/xen/include/asm-x86/viommu.h
> >> @@ -23,6 +23,21 @@
> >>  #include <xen/viommu.h>
> >>  #include <asm/types.h>
> >>  
> >> +struct irq_remapping_request
> >> +{
> >> +    u8 type;
> >> +    u16 source_id;
> >> +    union {
> >> +        /* MSI */
> >> +        struct {
> >> +            u64 addr;
> >> +            u32 data;
> >> +        } msi;
> >> +        /* Redirection Entry in IOAPIC */
> >> +        u64 rte;
> >> +    } msg;
> >> +};
> > 
> > Will this work right? As in with the default padding and such?
> 
> Sorry. Could you elaborate this?

If you run 'pahole' on the xen-syms and then look at this
structure - do the offsets match with what the hardware expects?

As in, I see this:

struct irq_remapping_request {
        u8                         type;                 /*     0     1 */

        /* XXX 1 byte hole, try to pack */

        u16                        source_id;            /*     2     2 */

        /* XXX 4 bytes hole, try to pack */

        union {
                struct {
                        u64        addr;                 /*     8     8 */
                        u32        data;                 /*    16     4 */
                } msi;                                   /*          16 */
                u64                rte;                  /*           8 */
        } msg;                                           /*     8    16 */

        /* size: 24, cachelines: 1, members: 3 */
        /* sum members: 19, holes: 2, sum holes: 5 */
        /* last cacheline: 24 bytes */
};

Is this OK? Or should the offsets be closer together?

> 
> >> +
> >>  static inline const struct viommu_ops *viommu_get_ops(void)
> >>  {
> >>      return NULL;
> >> diff --git a/xen/include/xen/viommu.h b/xen/include/xen/viommu.h
> >> index a0abbdf..246b29d 100644
> >> --- a/xen/include/xen/viommu.h
> >> +++ b/xen/include/xen/viommu.h
> >> @@ -24,6 +24,10 @@
> >>  
> >>  #define NR_VIOMMU_PER_DOMAIN 1
> >>  
> >> +/* IRQ request type */
> >> +#define VIOMMU_REQUEST_IRQ_MSI          0
> >> +#define VIOMMU_REQUEST_IRQ_APIC         1
> > 
> > What is this used for?
> 
> This is to designate interrupt type of irq remapping request which
> contains in the structure irq_remapping_request. The vIOMMU device model
> uses it to parse request data.

Does it make sense to have it as part of this patch? Even thought
it is not used here?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 15/23] X86/vioapic: Hook interrupt delivery of vIOAPIC

[Konrad Rzeszutek Wilk]

On Tue, Apr 18, 2017 at 04:34:52PM +0800, Lan Tianyu wrote:
> On 2017年04月17日 22:43, Konrad Rzeszutek Wilk wrote:
> > On Fri, Mar 17, 2017 at 07:27:15PM +0800, Lan Tianyu wrote:
> >> From: Chao Gao <chao.gao@intel.com>
> >>
> >> When irq remapping enabled, IOAPIC Redirection Entry maybe is in remapping
> >> format. If that, generate a irq_remapping_request and send it to domain.
> >>
> >> Signed-off-by: Chao Gao <chao.gao@intel.com>
> >> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >> ---
> >>  xen/arch/x86/Makefile                  |  1 +
> >>  xen/arch/x86/hvm/vioapic.c             | 10 ++++++++++
> >>  xen/arch/x86/viommu.c                  | 30 ++++++++++++++++++++++++++++++
> >>  xen/include/asm-x86/viommu.h           |  3 +++
> >>  xen/include/public/arch-x86/hvm/save.h |  1 +
> >>  5 files changed, 45 insertions(+)
> >>  create mode 100644 xen/arch/x86/viommu.c
> >>
> >> diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile
> >> index f75eca0..d49f8c8 100644
> >> --- a/xen/arch/x86/Makefile
> >> +++ b/xen/arch/x86/Makefile
> >> @@ -66,6 +66,7 @@ obj-y += usercopy.o
> >>  obj-y += x86_emulate.o
> >>  obj-$(CONFIG_TBOOT) += tboot.o
> >>  obj-y += hpet.o
> >> +obj-y += viommu.o
> >>  obj-y += vm_event.o
> >>  obj-y += xstate.o
> >>  
> >> diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
> >> index fdbb21f..6a00644 100644
> >> --- a/xen/arch/x86/hvm/vioapic.c
> >> +++ b/xen/arch/x86/hvm/vioapic.c
> >> @@ -30,6 +30,7 @@
> >>  #include <xen/lib.h>
> >>  #include <xen/errno.h>
> >>  #include <xen/sched.h>
> >> +#include <xen/viommu.h>
> >>  #include <public/hvm/ioreq.h>
> >>  #include <asm/hvm/io.h>
> >>  #include <asm/hvm/vpic.h>
> >> @@ -285,9 +286,18 @@ static void vioapic_deliver(struct hvm_hw_vioapic *vioapic, int irq)
> >>      struct domain *d = vioapic_domain(vioapic);
> >>      struct vlapic *target;
> >>      struct vcpu *v;
> >> +    struct irq_remapping_request request;
> >>  
> >>      ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
> >>  
> >> +    if ( vioapic->redirtbl[irq].ir.format )
> >> +    {
> >> +        irq_request_ioapic_fill(&request, vioapic->id,
> >> +                                vioapic->redirtbl[irq].bits);
> >> +        viommu_handle_irq_request(d, &request);
> >> +        return;
> >> +    }
> >> +
> >>      HVM_DBG_LOG(DBG_LEVEL_IOAPIC,
> >>                  "dest=%x dest_mode=%x delivery_mode=%x "
> >>                  "vector=%x trig_mode=%x",
> >> diff --git a/xen/arch/x86/viommu.c b/xen/arch/x86/viommu.c
> >> new file mode 100644
> >> index 0000000..ef78d3b
> >> --- /dev/null
> >> +++ b/xen/arch/x86/viommu.c
> >> @@ -0,0 +1,30 @@
> >> +/*
> >> + * viommu.c
> >> + *
> >> + * virtualize IOMMU.
> >> + *
> >> + * Copyright (C) 2017 Chao Gao, Intel Corporation.
> >> + *
> >> + * This program is free software; you can redistribute it and/or
> >> + * modify it under the terms and conditions of the GNU General Public
> >> + * License, version 2, as published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> >> + * General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public
> >> + * License along with this program; If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#include <xen/viommu.h>
> >> +
> >> +void irq_request_ioapic_fill(struct irq_remapping_request *req,
> >> +                             uint32_t ioapic_id, uint64_t rte)
> >> +{
> >> +    ASSERT(req);
> >> +    req->type = VIOMMU_REQUEST_IRQ_APIC;
> >> +    req->source_id = ioapic_id;
> >> +    req->msg.rte = rte;
> > 
> > Considering we get 'req' from the stack and it may have garbage, would
> > it be good to fill out the rest of the entries with sensible values? Or
> > is there no need for that?
> 
> Both AMD and Intel will use the function to pass interrupt remapping
> request. I am afraid different vendors may have different IOAPIC
> remapping format. How about to parse and check remapping request data in
> the vendor vIOMMU device module(E,G vvtd)? :)

Yeah that is fine. Just please put the comment in the code saying
that is the justification - or alternatively put it in the
commit description.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Paul Durrant]

> -----Original Message-----
[snip]
> >
> > Not quite sure I understand this. The QEMu device model does not 'pass
> DMA requests' as such, it maps guest RAM and reads or writes to emulate
> DMA, right? So, what's needed is a mechanism to map guest RAM by 'bus
> address'... i.e. an address that will need to be translated through the
> vIOMMU mappings. This is just an evolution of the current 'priv mapping'
> operations that allow guest RAM to be mapped by guest physical address. So
> you don't need a vIOMMU 'device model' as such, do you?
> 
> 
> Guest also may enable DMA protection mechanism in linux kernel which
> limits address space of emulated device and this depends on the vIOMMU's
> DMA translation function. In vIOMMU's MMIO emulation part is in the Xen
> hypersior and the guest shadow IO page table will be only in the
> hypervisor. To translate emulated device's DMA request. It's necessary
> to pass the DMA request to hypervisor.
> 

What do you mean by DMA request though? Are you intending to make some form of hypercall to read or write guest memory? If so then why not introduce a call to map the guest memory (via bus address) and read or write directly.

> So far we don't support DMA translation and so doesn't pass DMA request.
> 

Indeed. We map guest memory using guest physical address because, without an emulated IOMMU, guest physical address === bus address. This is why I suggest a new mapping operation rather than 'passing a DMA request' to the hypervisor.

> Map/umap guest memory already support in Qemu and just like emulated
> device model access guest memory. Qemu also provides vIOMMU hook to
> receive DMA request and return target guest address. vIOMMU framework
> will read/write target address.

That's the part I don't get... why have the vIOMMU code do the reads and writes? Why not have it provide a mapping function and then have the device model in QEMU read and write directly as it does now?

> What we need to do is to translate DMA
> request to target address according shadow IO page table in the hypervisor.
> 

Yes, so the mapping has to be done by the hypervisor (as is the case for priv mapping or grant mapping) but the memory accesses themselves can be done directly by the device model in QEMU.

> 
> 
> >
> >> Qemu is required to use DMOP hypercall and
> >> tool stack may use domctl hyercall. vIOMMU hypercalls will be divided
> >> into two part.
> >>
> >> Domctl:
> >> 	create, destroy and query.
> >> DMOP:
> >> 	vDev's DMA related operations.
> >
> > Yes, the mapping/unmapping operations should be DMOPs and IMO
> should be designed such that they can be unified with replacements for
> current 'priv map' ops such that QEMU can use the same function call, but
> with different address space identifiers (i.e. bus address, guest physical
> address, etc.). BTW, I say 'etc.' because we should also consider mapping the
> ioreq pages from Xen using the same call - with a dedicated address space
> identifier - as well.
> >
> 
> So you agree to divide vIOMMU's hypercalls into two parts(DMOP and
> Domctl), right?
> 

Yes, I agree with the logic of the split.

  Cheers,

   Paul

> 
> >   Cheers,
> >
> >     Paul
> >
> >>
> >> Is this OK?
> >>
> >>>
> >>> Thanks,
> >>> Chao
> >>>
> >>>>
> >>>>  Paul
> >>>>
> >>>>>>> The following toolstack code is to add XEN_DMOP_viommu_XXX
> >> syscalls:
> >>>>>>
> >>>>>> Hypercalls, not syscalls.
> >>>>>>
> >>>>>>>  - query capabilities of vIOMMU emulated by Xen
> >>>>>>>  - create vIOMMU in Xen hypervisor with base address, capability
> >>>>>>>  - destroy vIOMMU specified by viommu_id
> >>>>>>>
> >>>>>>> Signed-off-by: Chao Gao <chao.gao@intel.com>
> >>>>>>> Signed-off-by: Lan Tianyu <tianyu.lan@intel.com>
> >>>>>>> ---
> >>>>>>>  tools/libs/devicemodel/core.c                   | 69
> >>>>> +++++++++++++++++++++++++
> >>>>>>>  tools/libs/devicemodel/include/xendevicemodel.h | 35
> >> +++++++++++++
> >>>>>>>  tools/libs/devicemodel/libxendevicemodel.map    |  3 ++
> >>>>>>>  tools/libxc/include/xenctrl_compat.h            |  5 ++
> >>>>>>>  tools/libxc/xc_devicemodel_compat.c             | 18 +++++++
> >>>>>>>  5 files changed, 130 insertions(+)
> >>>>>>>
> >>>>>>> diff --git a/tools/libs/devicemodel/core.c
> >> b/tools/libs/devicemodel/core.c
> >>>>>>> index a85cb49..aee1150 100644
> >>>>>>> --- a/tools/libs/devicemodel/core.c
> >>>>>>> +++ b/tools/libs/devicemodel/core.c
> >>>>>>
> >>>>>> Bear in mind that this library is stable, so whatever ends up here can
> >>>>>> change in the future.
> >>>>>>
> >>>>>> This is not saying the following code is problematic. It is just a
> >>>>>> general FYI.
> >>>>>>
> >>>>>> Obviously the toolstack side is going to follow the hypervisor
> >>>>>> interface, so I will do a detailed review later.
> >>>>>
> >>>>> Sure. If the hypervisor interface settles down, we can inform you.
> >>>>>
> >>>>>>
> >>>>>>> +int xendevicemodel_viommu_destroy(
> >>>>>>> +    xendevicemodel_handle *dmod, domid_t dom, uint32_t
> >> viommu_id);
> >>>>>>>  #endif /* __XEN_TOOLS__ */
> >>>>>>>
> >>>>>>>  #endif /* XENDEVICEMODEL_H */
> >>>>>>> diff --git a/tools/libs/devicemodel/libxendevicemodel.map
> >>>>> b/tools/libs/devicemodel/libxendevicemodel.map
> >>>>>>> index 45c773e..c2e0968 100644
> >>>>>>> --- a/tools/libs/devicemodel/libxendevicemodel.map
> >>>>>>> +++ b/tools/libs/devicemodel/libxendevicemodel.map
> >>>>>>> @@ -17,6 +17,9 @@ VERS_1.0 {
> >>>>>>>  		xendevicemodel_modified_memory;
> >>>>>>>  		xendevicemodel_set_mem_type;
> >>>>>>>  		xendevicemodel_inject_event;
> >>>>>>> +		xendevicemodel_viommu_query_cap;
> >>>>>>> +		xendevicemodel_viommu_create;
> >>>>>>> +		xendevicemodel_viommu_destroy;
> >>>>>>>  		xendevicemodel_restrict;
> >>>>>>>  		xendevicemodel_close;
> >>>>>>
> >>>>>> I suppose this series is going to miss 4.9.
> >>>>>>
> >>>>>> Please add these functions to VERS_1.1.
> >>>>>
> >>>>> Yes. We will fix this.
> >>>>>
> >>>>>>
> >>>>>>>  	local: *; /* Do not expose anything by default */
> >>>>>>> diff --git a/tools/libxc/include/xenctrl_compat.h
> >>>>> b/tools/libxc/include/xenctrl_compat.h
> >>>>>>> index 040e7b2..315c45d 100644
> >>>>>>> --- a/tools/libxc/include/xenctrl_compat.h
> >>>>>>> +++ b/tools/libxc/include/xenctrl_compat.h
> >>>>>>> @@ -164,6 +164,11 @@ int xc_hvm_set_mem_type(
> >>>>>>>  int xc_hvm_inject_trap(
> >>>>>>>      xc_interface *xch, domid_t domid, int vcpu, uint8_t vector,
> >>>>>>>      uint8_t type, uint32_t error_code, uint8_t insn_len, uint64_t
> cr2);
> >>>>>>> +int xc_viommu_query_cap(xc_interface *xch, domid_t dom,
> >> uint64_t
> >>>>> *cap);
> >>>>>>> +int xc_viommu_create(
> >>>>>>> +    xc_interface *xch, domid_t dom, uint64_t base_addr, uint64_t
> >> cap,
> >>>>>>> +    uint32_t *viommu_id);
> >>>>>>> +int xc_viommu_destroy(xc_interface *xch, domid_t dom,
> uint32_t
> >>>>> viommu_id);
> >>>>>>>
> >>>>>>>  #endif /* XC_WANT_COMPAT_DEVICEMODEL_API */
> >>>>>>>
> >>>>>>> diff --git a/tools/libxc/xc_devicemodel_compat.c
> >>>>> b/tools/libxc/xc_devicemodel_compat.c
> >>>>>>> index e4edeea..62f703a 100644
> >>>>>>> --- a/tools/libxc/xc_devicemodel_compat.c
> >>>>>>> +++ b/tools/libxc/xc_devicemodel_compat.c
> >>>>>>
> >>>>>> I don't think you need to provide compat wrappers for them. They
> are
> >> new
> >>>>>> APIs.
> >>>>>
> >>>>> OK. Got it.
> >>>>>
> >>>>> Thanks,
> >>>>> Chao
> >>>>>>
> >>>>>> Wei.
> >>>>>
> >>>>> _______________________________________________
> >>>>> Xen-devel mailing list
> >>>>> Xen-devel@lists.xen.org
> >>>>> https://lists.xen.org/xen-devel
> 
> 
> --
> Best regards
> Tianyu Lan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan Tianyu]

On 2017年04月18日 22:15, Paul Durrant wrote:
>> -----Original Message-----
> [snip]
>>> > >
>>> > > Not quite sure I understand this. The QEMu device model does not 'pass
>> > DMA requests' as such, it maps guest RAM and reads or writes to emulate
>> > DMA, right? So, what's needed is a mechanism to map guest RAM by 'bus
>> > address'... i.e. an address that will need to be translated through the
>> > vIOMMU mappings. This is just an evolution of the current 'priv mapping'
>> > operations that allow guest RAM to be mapped by guest physical address. So
>> > you don't need a vIOMMU 'device model' as such, do you?
>> > 
>> > 
>> > Guest also may enable DMA protection mechanism in linux kernel which
>> > limits address space of emulated device and this depends on the vIOMMU's
>> > DMA translation function. In vIOMMU's MMIO emulation part is in the Xen
>> > hypersior and the guest shadow IO page table will be only in the
>> > hypervisor. To translate emulated device's DMA request. It's necessary
>> > to pass the DMA request to hypervisor.
>> > 
> What do you mean by DMA request though? Are you intending to make some form of hypercall to read or write guest memory? If so then why not introduce a call to map the guest memory (via bus address) and read or write directly.

Such "DMA request" in Qemu vIOMMU framework just contains IOVA(IO
virtual address) and write/read flag. vIOMMU device model just
translates IOVA to GPA and then return back to vIOMMU core which will be
in charge of memory access. So hyercall we want to introduce is to
translate IOVA to GPA.

The data to write and target address to store read data aren't passed to
vIOMMU device model and we can't perform read/write directly there.

>> > So far we don't support DMA translation and so doesn't pass DMA request.
>> > 
> Indeed. We map guest memory using guest physical address because, without an emulated IOMMU, guest physical address === bus address. This is why I suggest a new mapping operation rather than 'passing a DMA request' to the hypervisor.
>
>> > Map/umap guest memory already support in Qemu and just like emulated
>> > device model access guest memory. Qemu also provides vIOMMU hook to
>> > receive DMA request and return target guest address. vIOMMU framework
>> > will read/write target address.
> That's the part I don't get... why have the vIOMMU code do the reads and writes? Why not have it provide a mapping function and then have the device model in QEMU read and write directly as it does now?
> 

Actually it's common interface in Qemu to read/write guest memory. The
code will check whether there is a vIOMMU translation callback or not
before performing read/write. If yes, call the callback and vIOMMU
device model translate IOVA to GPA and then do read/write operation.


>> > What we need to do is to translate DMA
>> > request to target address according shadow IO page table in the hypervisor.
>> > 
> Yes, so the mapping has to be done by the hypervisor (as is the case for priv mapping or grant mapping) but the memory accesses themselves can be done directly by the device model in QEMU.

Yes.

> 
>> > 
>> > 
>>> > >
>>>> > >> Qemu is required to use DMOP hypercall and
>>>> > >> tool stack may use domctl hyercall. vIOMMU hypercalls will be divided
>>>> > >> into two part.
>>>> > >>
>>>> > >> Domctl:
>>>> > >> 	create, destroy and query.
>>>> > >> DMOP:
>>>> > >> 	vDev's DMA related operations.
>>> > >
>>> > > Yes, the mapping/unmapping operations should be DMOPs and IMO
>> > should be designed such that they can be unified with replacements for
>> > current 'priv map' ops such that QEMU can use the same function call, but
>> > with different address space identifiers (i.e. bus address, guest physical
>> > address, etc.). BTW, I say 'etc.' because we should also consider mapping the
>> > ioreq pages from Xen using the same call - with a dedicated address space
>> > identifier - as well.
>>> > >
>> > 
>> > So you agree to divide vIOMMU's hypercalls into two parts(DMOP and
>> > Domctl), right?
>> > 
> Yes, I agree with the logic of the split.
> 
>   Cheers,
> 
>    Paul
> 


-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Lan Tianyu]

On 2017年05月11日 20:35, Wei Liu wrote:
> On Mon, Apr 17, 2017 at 08:01:56PM +0800, Lan Tianyu wrote:
>> On 2017年04月17日 19:08, Wei Liu wrote:
>>> On Fri, Apr 14, 2017 at 11:38:15PM +0800, Lan, Tianyu wrote:
>>>> Hi Paul:
>>>> 	Sorry for later response.
>>>>
>>>> On 3/31/2017 3:57 AM, Chao Gao wrote:
>>>>> On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
>>>>>>> -----Original Message-----
>>>>>>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
>>>>>>> Chao Gao
>>>>>>> Sent: 29 March 2017 01:40
>>>>>>> To: Wei Liu <wei.liu2@citrix.com>
>>>>>>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
>>>>>>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
>>>>>>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
>>>>>>> operations in libxc
>>>>>>>
>>>>>>> Tianyu is on vacation this two weeks, so I will try to address
>>>>>>> some comments on this series.
>>>>>>>
>>>>>>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
>>>>>>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
>>>>>>>>> From: Chao Gao <chao.gao@intel.com>
>>>>>>>>>
>>>>>>>>> In previous patch, we introduce a common vIOMMU layer. In our design,
>>>>>>>>> we create/destroy vIOMMU through DMOP interface instead of creating
>>>>>>> it
>>>>>>>>> according to a config flag of domain. It makes it is possible
>>>>>>>>> to create vIOMMU in device model or in tool stack.
>>>>>>>>>
>>>>>>
>>>>>> I've not been following this closely so apologies if this has already been asked...
>>>>>>
>>>>>> Why would you need to create a vIOMMU instance in an external device model.
>>>>>> Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?
>>>>>
>>>>> I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
>>>>> I think creating in toolstack is also ok and is easier to be reused by pvh.
>>>>>
>>>>> If Tianyu has no concern about this, will move this part to toolstack.
>>>>
>>>> We can move create/destroy vIOMMU in the tool stack but we still need to add
>>>> such dummy vIOMMU device model in Qemu to pass virtual device's DMA request
>>>> into Xen hypervisor. Qemu is required to use DMOP hypercall and tool stack
>>>> may use domctl hyercall. vIOMMU hypercalls will be divided into two part.
>>>>
>>>> Domctl:
>>>> 	create, destroy and query.
>>>> DMOP:
>>>> 	vDev's DMA related operations.
>>>>
>>>> Is this OK?
>>>>
>>>
>>> Why are they divided into two libraries? Can't they be in DMOP at the
>>> same time?
>>
>> Yes, we can use DMOP for all vIOMMU hyercalls if it's necessary to keep
>> unified vIOMMU hyercall type. In theory, DMOP dedicates to be used by
>> Qemu but we also can use it in tool stack. If we move create, destroy
>> and query operation to tool stack, it isn't necessary to use DMOP for
>> them since only tool stack will call them. This is why I said we could
>> use domctl for these operations. Both two ways will not affect function
>> implementation. Which one it's better from your view? :)
>>
> 
> 
> After reading the subthread I think I agree with Paul. I.e. please
> separate them.
> 

Sure. Will update.

-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [RFC PATCH 5/23] Tools/libxc: Add viommu operations in libxc

[Wei Liu]

On Mon, Apr 17, 2017 at 08:01:56PM +0800, Lan Tianyu wrote:
> On 2017年04月17日 19:08, Wei Liu wrote:
> > On Fri, Apr 14, 2017 at 11:38:15PM +0800, Lan, Tianyu wrote:
> >> Hi Paul:
> >> 	Sorry for later response.
> >>
> >> On 3/31/2017 3:57 AM, Chao Gao wrote:
> >>> On Wed, Mar 29, 2017 at 09:08:06AM +0000, Paul Durrant wrote:
> >>>>> -----Original Message-----
> >>>>> From: Xen-devel [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of
> >>>>> Chao Gao
> >>>>> Sent: 29 March 2017 01:40
> >>>>> To: Wei Liu <wei.liu2@citrix.com>
> >>>>> Cc: Lan Tianyu <tianyu.lan@intel.com>; Kevin Tian <kevin.tian@intel.com>;
> >>>>> Ian Jackson <Ian.Jackson@citrix.com>; xen-devel@lists.xen.org
> >>>>> Subject: Re: [Xen-devel] [RFC PATCH 5/23] Tools/libxc: Add viommu
> >>>>> operations in libxc
> >>>>>
> >>>>> Tianyu is on vacation this two weeks, so I will try to address
> >>>>> some comments on this series.
> >>>>>
> >>>>> On Tue, Mar 28, 2017 at 05:24:03PM +0100, Wei Liu wrote:
> >>>>>> On Fri, Mar 17, 2017 at 07:27:05PM +0800, Lan Tianyu wrote:
> >>>>>>> From: Chao Gao <chao.gao@intel.com>
> >>>>>>>
> >>>>>>> In previous patch, we introduce a common vIOMMU layer. In our design,
> >>>>>>> we create/destroy vIOMMU through DMOP interface instead of creating
> >>>>> it
> >>>>>>> according to a config flag of domain. It makes it is possible
> >>>>>>> to create vIOMMU in device model or in tool stack.
> >>>>>>>
> >>>>
> >>>> I've not been following this closely so apologies if this has already been asked...
> >>>>
> >>>> Why would you need to create a vIOMMU instance in an external device model.
> >>>> Since the toolstack should be in control of the device model configuration why would it not know in advance that one was required?
> >>>
> >>> I assume your question is why we don't create a vIOMMU instance via hypercall in toolstack.
> >>> I think creating in toolstack is also ok and is easier to be reused by pvh.
> >>>
> >>> If Tianyu has no concern about this, will move this part to toolstack.
> >>
> >> We can move create/destroy vIOMMU in the tool stack but we still need to add
> >> such dummy vIOMMU device model in Qemu to pass virtual device's DMA request
> >> into Xen hypervisor. Qemu is required to use DMOP hypercall and tool stack
> >> may use domctl hyercall. vIOMMU hypercalls will be divided into two part.
> >>
> >> Domctl:
> >> 	create, destroy and query.
> >> DMOP:
> >> 	vDev's DMA related operations.
> >>
> >> Is this OK?
> >>
> > 
> > Why are they divided into two libraries? Can't they be in DMOP at the
> > same time?
> 
> Yes, we can use DMOP for all vIOMMU hyercalls if it's necessary to keep
> unified vIOMMU hyercall type. In theory, DMOP dedicates to be used by
> Qemu but we also can use it in tool stack. If we move create, destroy
> and query operation to tool stack, it isn't necessary to use DMOP for
> them since only tool stack will call them. This is why I said we could
> use domctl for these operations. Both two ways will not affect function
> implementation. Which one it's better from your view? :)
> 


After reading the subthread I think I agree with Paul. I.e. please
separate them.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel