* [PATCH v5 01/19] iommu: Add a timeout parameter for PRQ response
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 02/19] iommu: handle page response timeout Jacob Pan
` (18 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When an I/O page request is processed outside the IOMMU subsystem,
response can be delayed or lost. Add a tunable setup parameter such that
user can choose the timeout for IOMMU to track pending page requests.
This timeout mechanism is a basic safety net which can be implemented in
conjunction with credit based or device level page response exception
handling.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
Documentation/admin-guide/kernel-parameters.txt | 8 ++++++
drivers/iommu/iommu.c | 33 +++++++++++++++++++++++++
2 files changed, 41 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 47d981a..7da5a83 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1817,6 +1817,14 @@
1 - Bypass the IOMMU for DMA.
unset - Use value of CONFIG_IOMMU_DEFAULT_PASSTHROUGH.
+ iommu.prq_timeout=
+ Timeout in seconds to wait for page response
+ of a pending page request.
+ Format: <integer>
+ Default: 10
+ 0 - no timeout tracking
+ 1 to 100 - allowed range
+
io7= [HW] IO7 for Marvel based alpha systems
See comment before marvel_specify_io7 in
arch/alpha/kernel/core_marvel.c.
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 0c674d8..5b26499 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -33,6 +33,19 @@ static unsigned int iommu_def_domain_type = IOMMU_DOMAIN_DMA;
#endif
static bool iommu_dma_strict __read_mostly = true;
+/*
+ * Timeout to wait for page response of a pending page request. This is
+ * intended as a basic safty net in case a pending page request is not
+ * responded for an exceptionally long time. Device may also implement
+ * its own protection mechanism against this exception.
+ * Units are in jiffies with a range between 1 - 100 seconds equivalent.
+ * Default to 10 seconds.
+ * Setting 0 means no timeout tracking.
+ */
+#define IOMMU_PAGE_RESPONSE_MAX_TIMEOUT (HZ * 100)
+#define IOMMU_PAGE_RESPONSE_DEF_TIMEOUT (HZ * 10)
+static unsigned long prq_timeout = IOMMU_PAGE_RESPONSE_DEF_TIMEOUT;
+
struct iommu_group {
struct kobject kobj;
struct kobject *devices_kobj;
@@ -176,6 +189,26 @@ static int __init iommu_dma_setup(char *str)
}
early_param("iommu.strict", iommu_dma_setup);
+static int __init iommu_set_prq_timeout(char *str)
+{
+ int ret;
+ unsigned long timeout;
+
+ if (!str)
+ return -EINVAL;
+
+ ret = kstrtoul(str, 10, &timeout);
+ if (ret)
+ return ret;
+ timeout = timeout * HZ;
+ if (timeout > IOMMU_PAGE_RESPONSE_MAX_TIMEOUT)
+ return -EINVAL;
+ prq_timeout = timeout;
+
+ return 0;
+}
+early_param("iommu.prq_timeout", iommu_set_prq_timeout);
+
static ssize_t iommu_group_attr_show(struct kobject *kobj,
struct attribute *__attr, char *buf)
{
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 02/19] iommu: handle page response timeout
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
2019-08-15 20:13 ` [PATCH v5 01/19] iommu: Add a timeout parameter for PRQ response Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 03/19] trace/iommu: Add sva trace events Jacob Pan
` (17 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When IO page faults are reported outside IOMMU subsystem, the page
request handler may fail for various reasons. E.g. a guest received
page requests but did not have a chance to run for a long time. The
irresponsive behavior could hold off limited resources on the pending
device.
There can be hardware or credit based software solutions as suggested
in the PCI ATS Ch-4. To provide a basic safty net this patch
introduces a per device deferrable timer which monitors the longest
pending page fault that requires a response. Proper action such as
sending failure response code could be taken when timer expires but not
included in this patch. We need to consider the life cycle of page
groupd ID to prevent confusion with reused group ID by a device.
For now, a warning message provides clue of such failure.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
---
drivers/iommu/iommu.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 4 ++++
2 files changed, 59 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 5b26499..8f2c7d5 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -909,6 +909,39 @@ int iommu_group_unregister_notifier(struct iommu_group *group,
}
EXPORT_SYMBOL_GPL(iommu_group_unregister_notifier);
+static void iommu_dev_fault_timer_fn(struct timer_list *t)
+{
+ struct iommu_fault_param *fparam = from_timer(fparam, t, timer);
+ struct iommu_fault_event *evt;
+ struct iommu_fault_page_request *prm;
+
+ u64 now;
+
+ now = get_jiffies_64();
+
+ /* The goal is to ensure driver or guest page fault handler(via vfio)
+ * send page response on time. Otherwise, limited queue resources
+ * may be occupied by some irresponsive guests or drivers.
+ * When per device pending fault list is not empty, we periodically checks
+ * if any anticipated page response time has expired.
+ *
+ * TODO:
+ * We could do the following if response time expires:
+ * 1. send page response code FAILURE to all pending PRQ
+ * 2. inform device driver or vfio
+ * 3. drain in-flight page requests and responses for this device
+ * 4. clear pending fault list such that driver can unregister fault
+ * handler(otherwise blocked when pending faults are present).
+ */
+ list_for_each_entry(evt, &fparam->faults, list) {
+ prm = &evt->fault.prm;
+ if (time_after64(now, evt->expire))
+ pr_err("Page response time expired!, pasid %d gid %d exp %llu now %llu\n",
+ prm->pasid, prm->grpid, evt->expire, now);
+ }
+ mod_timer(t, now + prq_timeout);
+}
+
/**
* iommu_register_device_fault_handler() - Register a device fault handler
* @dev: the device
@@ -956,6 +989,9 @@ int iommu_register_device_fault_handler(struct device *dev,
mutex_init(¶m->fault_param->lock);
INIT_LIST_HEAD(¶m->fault_param->faults);
+ if (prq_timeout)
+ timer_setup(¶m->fault_param->timer, iommu_dev_fault_timer_fn,
+ TIMER_DEFERRABLE);
done_unlock:
mutex_unlock(¶m->lock);
@@ -1017,7 +1053,9 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
struct iommu_param *param = dev->iommu_param;
struct iommu_fault_event *evt_pending = NULL;
struct iommu_fault_param *fparam;
+ struct timer_list *tmr;
int ret = 0;
+ u64 exp;
if (!param || !evt)
return -EINVAL;
@@ -1038,7 +1076,17 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
ret = -ENOMEM;
goto done_unlock;
}
+ /* Keep track of response expiration time */
+ exp = get_jiffies_64() + prq_timeout;
+ evt_pending->expire = exp;
mutex_lock(&fparam->lock);
+ if (list_empty(&fparam->faults)) {
+ /* First pending event, start timer */
+ tmr = &dev->iommu_param->fault_param->timer;
+ WARN_ON(timer_pending(tmr));
+ mod_timer(tmr, exp);
+ }
+
list_add_tail(&evt_pending->list, &fparam->faults);
mutex_unlock(&fparam->lock);
}
@@ -1103,6 +1151,13 @@ int iommu_page_response(struct device *dev,
break;
}
+ /* stop response timer if no more pending request */
+ if (list_empty(¶m->fault_param->faults) &&
+ timer_pending(¶m->fault_param->timer)) {
+ pr_debug("no pending PRQ, stop timer\n");
+ del_timer(¶m->fault_param->timer);
+ }
+
done_unlock:
mutex_unlock(¶m->fault_param->lock);
return ret;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index fdc355c..39d371b 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -317,10 +317,12 @@ struct iommu_device {
*
* @fault: fault descriptor
* @list: pending fault event list, used for tracking responses
+ * @expire: time limit in jiffies will wait for page response
*/
struct iommu_fault_event {
struct iommu_fault fault;
struct list_head list;
+ u64 expire;
};
/**
@@ -329,11 +331,13 @@ struct iommu_fault_event {
* @data: handler private data
* @faults: holds the pending faults which needs response
* @lock: protect pending faults list
+ * @timer: track page request pending time limit
*/
struct iommu_fault_param {
iommu_dev_fault_handler_t handler;
void *data;
struct list_head faults;
+ struct timer_list timer;
struct mutex lock;
};
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 03/19] trace/iommu: Add sva trace events
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
2019-08-15 20:13 ` [PATCH v5 01/19] iommu: Add a timeout parameter for PRQ response Jacob Pan
2019-08-15 20:13 ` [PATCH v5 02/19] iommu: handle page response timeout Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 04/19] iommu: Use device fault trace event Jacob Pan
` (16 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
For development only, trace I/O page faults and responses.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
[JPB: removed the invalidate trace event, that will be added later]
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
include/trace/events/iommu.h | 84 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 84 insertions(+)
diff --git a/include/trace/events/iommu.h b/include/trace/events/iommu.h
index 72b4582..767b92c 100644
--- a/include/trace/events/iommu.h
+++ b/include/trace/events/iommu.h
@@ -12,6 +12,8 @@
#define _TRACE_IOMMU_H
#include <linux/tracepoint.h>
+#include <linux/iommu.h>
+#include <uapi/linux/iommu.h>
struct device;
@@ -161,6 +163,88 @@ DEFINE_EVENT(iommu_error, io_page_fault,
TP_ARGS(dev, iova, flags)
);
+
+TRACE_EVENT(dev_fault,
+
+ TP_PROTO(struct device *dev, struct iommu_fault *evt),
+
+ TP_ARGS(dev, evt),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(dev))
+ __field(int, type)
+ __field(int, reason)
+ __field(u64, addr)
+ __field(u64, fetch_addr)
+ __field(u32, pasid)
+ __field(u32, grpid)
+ __field(u32, flags)
+ __field(u32, prot)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(dev));
+ __entry->type = evt->type;
+ if (evt->type == IOMMU_FAULT_DMA_UNRECOV) {
+ __entry->reason = evt->event.reason;
+ __entry->flags = evt->event.flags;
+ __entry->pasid = evt->event.pasid;
+ __entry->grpid = 0;
+ __entry->prot = evt->event.perm;
+ __entry->addr = evt->event.addr;
+ __entry->fetch_addr = evt->event.fetch_addr;
+ } else {
+ __entry->reason = 0;
+ __entry->flags = evt->prm.flags;
+ __entry->pasid = evt->prm.pasid;
+ __entry->grpid = evt->prm.grpid;
+ __entry->prot = evt->prm.perm;
+ __entry->addr = evt->prm.addr;
+ __entry->fetch_addr = 0;
+ }
+ ),
+
+ TP_printk("IOMMU:%s type=%d reason=%d addr=0x%016llx fetch=0x%016llx pasid=%d group=%d flags=%x prot=%d",
+ __get_str(device),
+ __entry->type,
+ __entry->reason,
+ __entry->addr,
+ __entry->fetch_addr,
+ __entry->pasid,
+ __entry->grpid,
+ __entry->flags,
+ __entry->prot
+ )
+);
+
+TRACE_EVENT(dev_page_response,
+
+ TP_PROTO(struct device *dev, struct iommu_fault_page_response *msg),
+
+ TP_ARGS(dev, msg),
+
+ TP_STRUCT__entry(
+ __string(device, dev_name(dev))
+ __field(int, code)
+ __field(u32, pasid)
+ __field(u32, grpid)
+ ),
+
+ TP_fast_assign(
+ __assign_str(device, dev_name(dev));
+ __entry->code = msg->code;
+ __entry->pasid = msg->pasid;
+ __entry->grpid = msg->grpid;
+ ),
+
+ TP_printk("IOMMU:%s code=%d pasid=%d group=%d",
+ __get_str(device),
+ __entry->code,
+ __entry->pasid,
+ __entry->grpid
+ )
+);
+
#endif /* _TRACE_IOMMU_H */
/* This part must be outside protection */
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 04/19] iommu: Use device fault trace event
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (2 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 03/19] trace/iommu: Add sva trace events Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 05/19] iommu: Introduce attach/detach_pasid_table API Jacob Pan
` (15 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
For performance and debugging purposes, these trace events help
analyzing device faults that interact with IOMMU subsystem.
E.g.
IOMMU:0000:00:0a.0 type=2 reason=0 addr=0x00000000007ff000 pasid=1
group=1 last=0 prot=1
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
[JPB: removed invalidate event, that will be added later]
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/iommu.c | 2 ++
include/trace/events/iommu.h | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 8f2c7d5..feada31 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1098,6 +1098,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
mutex_unlock(&fparam->lock);
kfree(evt_pending);
}
+ trace_dev_fault(dev, &evt->fault);
done_unlock:
mutex_unlock(¶m->lock);
return ret;
@@ -1146,6 +1147,7 @@ int iommu_page_response(struct device *dev,
msg->flags = pasid_valid ? IOMMU_PAGE_RESP_PASID_VALID : 0;
ret = domain->ops->page_response(dev, evt, msg);
+ trace_dev_page_response(dev, msg);
list_del(&evt->list);
kfree(evt);
break;
diff --git a/include/trace/events/iommu.h b/include/trace/events/iommu.h
index 767b92c..7a7801b 100644
--- a/include/trace/events/iommu.h
+++ b/include/trace/events/iommu.h
@@ -219,7 +219,7 @@ TRACE_EVENT(dev_fault,
TRACE_EVENT(dev_page_response,
- TP_PROTO(struct device *dev, struct iommu_fault_page_response *msg),
+ TP_PROTO(struct device *dev, struct iommu_page_response *msg),
TP_ARGS(dev, msg),
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 05/19] iommu: Introduce attach/detach_pasid_table API
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (3 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 04/19] iommu: Use device fault trace event Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 06/19] iommu: Introduce cache_invalidate API Jacob Pan
` (14 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
In virtualization use case, when a guest is assigned
a PCI host device, protected by a virtual IOMMU on the guest,
the physical IOMMU must be programmed to be consistent with
the guest mappings. If the physical IOMMU supports two
translation stages it makes sense to program guest mappings
onto the first stage/level (ARM/Intel terminology) while the host
owns the stage/level 2.
In that case, it is mandated to trap on guest configuration
settings and pass those to the physical iommu driver.
This patch adds a new API to the iommu subsystem that allows
to set/unset the pasid table information.
A generic iommu_pasid_table_config struct is introduced in
a new iommu.h uapi header. This is going to be used by the VFIO
user API.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
---
drivers/iommu/iommu.c | 19 +++++++++++++++++
include/linux/iommu.h | 18 ++++++++++++++++
include/uapi/linux/iommu.h | 51 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 88 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index feada31..155ebef 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1700,6 +1700,25 @@ int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
}
EXPORT_SYMBOL_GPL(iommu_attach_device);
+int iommu_attach_pasid_table(struct iommu_domain *domain,
+ struct iommu_pasid_table_config *cfg)
+{
+ if (unlikely(!domain->ops->attach_pasid_table))
+ return -ENODEV;
+
+ return domain->ops->attach_pasid_table(domain, cfg);
+}
+EXPORT_SYMBOL_GPL(iommu_attach_pasid_table);
+
+void iommu_detach_pasid_table(struct iommu_domain *domain)
+{
+ if (unlikely(!domain->ops->detach_pasid_table))
+ return;
+
+ domain->ops->detach_pasid_table(domain);
+}
+EXPORT_SYMBOL_GPL(iommu_detach_pasid_table);
+
static void __iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 39d371b..8c64065 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -228,6 +228,8 @@ struct iommu_sva_ops {
* @sva_unbind: Unbind process address space from device
* @sva_get_pasid: Get PASID associated to a SVA handle
* @page_response: handle page request response
+ * @attach_pasid_table: attach a pasid table
+ * @detach_pasid_table: detach the pasid table
* @pgsize_bitmap: bitmap of all possible supported page sizes
*/
struct iommu_ops {
@@ -287,6 +289,9 @@ struct iommu_ops {
void *drvdata);
void (*sva_unbind)(struct iommu_sva *handle);
int (*sva_get_pasid)(struct iommu_sva *handle);
+ int (*attach_pasid_table)(struct iommu_domain *domain,
+ struct iommu_pasid_table_config *cfg);
+ void (*detach_pasid_table)(struct iommu_domain *domain);
int (*page_response)(struct device *dev,
struct iommu_fault_event *evt,
@@ -399,6 +404,9 @@ extern int iommu_attach_device(struct iommu_domain *domain,
struct device *dev);
extern void iommu_detach_device(struct iommu_domain *domain,
struct device *dev);
+extern int iommu_attach_pasid_table(struct iommu_domain *domain,
+ struct iommu_pasid_table_config *cfg);
+extern void iommu_detach_pasid_table(struct iommu_domain *domain);
extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev);
extern struct iommu_domain *iommu_get_dma_domain(struct device *dev);
extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
@@ -941,6 +949,16 @@ static inline int iommu_sva_get_pasid(struct iommu_sva *handle)
return IOMMU_PASID_INVALID;
}
+static inline
+int iommu_attach_pasid_table(struct iommu_domain *domain,
+ struct iommu_pasid_table_config *cfg)
+{
+ return -ENODEV;
+}
+
+static inline
+void iommu_detach_pasid_table(struct iommu_domain *domain) {}
+
#endif /* CONFIG_IOMMU_API */
#ifdef CONFIG_IOMMU_DEBUGFS
diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h
index fc00c5d..0f9d249 100644
--- a/include/uapi/linux/iommu.h
+++ b/include/uapi/linux/iommu.h
@@ -152,4 +152,55 @@ struct iommu_page_response {
__u32 code;
};
+/**
+ * struct iommu_pasid_smmuv3 - ARM SMMUv3 Stream Table Entry stage 1 related
+ * information
+ * @version: API version of this structure
+ * @s1fmt: STE s1fmt (format of the CD table: single CD, linear table
+ * or 2-level table)
+ * @s1dss: STE s1dss (specifies the behavior when @pasid_bits != 0
+ * and no PASID is passed along with the incoming transaction)
+ * @padding: reserved for future use (should be zero)
+ *
+ * The PASID table is referred to as the Context Descriptor (CD) table on ARM
+ * SMMUv3. Please refer to the ARM SMMU 3.x spec (ARM IHI 0070A) for full
+ * details.
+ */
+struct iommu_pasid_smmuv3 {
+#define PASID_TABLE_SMMUV3_CFG_VERSION_1 1
+ __u32 version;
+ __u8 s1fmt;
+ __u8 s1dss;
+ __u8 padding[2];
+};
+
+/**
+ * struct iommu_pasid_table_config - PASID table data used to bind guest PASID
+ * table to the host IOMMU
+ * @version: API version to prepare for future extensions
+ * @format: format of the PASID table
+ * @base_ptr: guest physical address of the PASID table
+ * @pasid_bits: number of PASID bits used in the PASID table
+ * @config: indicates whether the guest translation stage must
+ * be translated, bypassed or aborted.
+ * @padding: reserved for future use (should be zero)
+ * @smmuv3: table information when @format is %IOMMU_PASID_FORMAT_SMMUV3
+ */
+struct iommu_pasid_table_config {
+#define PASID_TABLE_CFG_VERSION_1 1
+ __u32 version;
+#define IOMMU_PASID_FORMAT_SMMUV3 1
+ __u32 format;
+ __u64 base_ptr;
+ __u8 pasid_bits;
+#define IOMMU_PASID_CONFIG_TRANSLATE 1
+#define IOMMU_PASID_CONFIG_BYPASS 2
+#define IOMMU_PASID_CONFIG_ABORT 3
+ __u8 config;
+ __u8 padding[6];
+ union {
+ struct iommu_pasid_smmuv3 smmuv3;
+ };
+};
+
#endif /* _UAPI_IOMMU_H */
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 06/19] iommu: Introduce cache_invalidate API
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (4 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 05/19] iommu: Introduce attach/detach_pasid_table API Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 07/19] iommu: Add I/O ASID allocator Jacob Pan
` (13 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
From: Yi L Liu <yi.l.liu@linux.intel.com>
In any virtualization use case, when the first translation stage
is "owned" by the guest OS, the host IOMMU driver has no knowledge
of caching structure updates unless the guest invalidation activities
are trapped by the virtualizer and passed down to the host.
Since the invalidation data are obtained from user space and will be
written into physical IOMMU, we must allow security check at various
layers. Therefore, generic invalidation data format are proposed here,
model specific IOMMU drivers need to convert them into their own format.
Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
---
drivers/iommu/iommu.c | 10 +++++
include/linux/iommu.h | 14 ++++++
include/uapi/linux/iommu.h | 110 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 134 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 155ebef..6228d5d 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1719,6 +1719,16 @@ void iommu_detach_pasid_table(struct iommu_domain *domain)
}
EXPORT_SYMBOL_GPL(iommu_detach_pasid_table);
+int iommu_cache_invalidate(struct iommu_domain *domain, struct device *dev,
+ struct iommu_cache_invalidate_info *inv_info)
+{
+ if (unlikely(!domain->ops->cache_invalidate))
+ return -ENODEV;
+
+ return domain->ops->cache_invalidate(domain, dev, inv_info);
+}
+EXPORT_SYMBOL_GPL(iommu_cache_invalidate);
+
static void __iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 8c64065..28f1a8c 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -230,6 +230,7 @@ struct iommu_sva_ops {
* @page_response: handle page request response
* @attach_pasid_table: attach a pasid table
* @detach_pasid_table: detach the pasid table
+ * @cache_invalidate: invalidate translation caches
* @pgsize_bitmap: bitmap of all possible supported page sizes
*/
struct iommu_ops {
@@ -296,6 +297,8 @@ struct iommu_ops {
int (*page_response)(struct device *dev,
struct iommu_fault_event *evt,
struct iommu_page_response *msg);
+ int (*cache_invalidate)(struct iommu_domain *domain, struct device *dev,
+ struct iommu_cache_invalidate_info *inv_info);
unsigned long pgsize_bitmap;
};
@@ -407,6 +410,9 @@ extern void iommu_detach_device(struct iommu_domain *domain,
extern int iommu_attach_pasid_table(struct iommu_domain *domain,
struct iommu_pasid_table_config *cfg);
extern void iommu_detach_pasid_table(struct iommu_domain *domain);
+extern int iommu_cache_invalidate(struct iommu_domain *domain,
+ struct device *dev,
+ struct iommu_cache_invalidate_info *inv_info);
extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev);
extern struct iommu_domain *iommu_get_dma_domain(struct device *dev);
extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
@@ -959,6 +965,14 @@ int iommu_attach_pasid_table(struct iommu_domain *domain,
static inline
void iommu_detach_pasid_table(struct iommu_domain *domain) {}
+static inline int
+iommu_cache_invalidate(struct iommu_domain *domain,
+ struct device *dev,
+ struct iommu_cache_invalidate_info *inv_info)
+{
+ return -ENODEV;
+}
+
#endif /* CONFIG_IOMMU_API */
#ifdef CONFIG_IOMMU_DEBUGFS
diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h
index 0f9d249..919ea02 100644
--- a/include/uapi/linux/iommu.h
+++ b/include/uapi/linux/iommu.h
@@ -203,4 +203,114 @@ struct iommu_pasid_table_config {
};
};
+/* defines the granularity of the invalidation */
+enum iommu_inv_granularity {
+ IOMMU_INV_GRANU_DOMAIN, /* domain-selective invalidation */
+ IOMMU_INV_GRANU_PASID, /* PASID-selective invalidation */
+ IOMMU_INV_GRANU_ADDR, /* page-selective invalidation */
+ IOMMU_INV_GRANU_NR, /* number of invalidation granularities */
+};
+
+/**
+ * struct iommu_inv_addr_info - Address Selective Invalidation Structure
+ *
+ * @flags: indicates the granularity of the address-selective invalidation
+ * - If the PASID bit is set, the @pasid field is populated and the invalidation
+ * relates to cache entries tagged with this PASID and matching the address
+ * range.
+ * - If ARCHID bit is set, @archid is populated and the invalidation relates
+ * to cache entries tagged with this architecture specific ID and matching
+ * the address range.
+ * - Both PASID and ARCHID can be set as they may tag different caches.
+ * - If neither PASID or ARCHID is set, global addr invalidation applies.
+ * - The LEAF flag indicates whether only the leaf PTE caching needs to be
+ * invalidated and other paging structure caches can be preserved.
+ * @archid: architecture-specific ID
+ * @pasid: process address space ID
+ * @addr: first stage/level input address
+ * @granule_size: page/block size of the mapping in bytes
+ * @nb_granules: number of contiguous granules to be invalidated
+ */
+struct iommu_inv_addr_info {
+#define IOMMU_INV_ADDR_FLAGS_PASID (1 << 0)
+#define IOMMU_INV_ADDR_FLAGS_ARCHID (1 << 1)
+#define IOMMU_INV_ADDR_FLAGS_LEAF (1 << 2)
+ __u32 flags;
+ __u32 archid;
+ __u64 pasid;
+ __u64 addr;
+ __u64 granule_size;
+ __u64 nb_granules;
+};
+
+/**
+ * struct iommu_inv_pasid_info - PASID Selective Invalidation Structure
+ *
+ * @flags: indicates the granularity of the PASID-selective invalidation
+ * - If the PASID bit is set, the @pasid field is populated and the invalidation
+ * relates to cache entries tagged with this PASID and matching the address
+ * range.
+ * - If the ARCHID bit is set, the @archid is populated and the invalidation
+ * relates to cache entries tagged with this architecture specific ID and
+ * matching the address range.
+ * - Both PASID and ARCHID can be set as they may tag different caches.
+ * - At least one of PASID or ARCHID must be set.
+ * @archid: architecture-specific ID
+ * @pasid: process address space ID
+ */
+struct iommu_inv_pasid_info {
+#define IOMMU_INV_PASID_FLAGS_PASID (1 << 0)
+#define IOMMU_INV_PASID_FLAGS_ARCHID (1 << 1)
+ __u32 flags;
+ __u32 archid;
+ __u64 pasid;
+};
+
+/**
+ * struct iommu_cache_invalidate_info - First level/stage invalidation
+ * information
+ * @version: API version of this structure
+ * @cache: bitfield that allows to select which caches to invalidate
+ * @granularity: defines the lowest granularity used for the invalidation:
+ * domain > PASID > addr
+ * @padding: reserved for future use (should be zero)
+ * @pasid_info: invalidation data when @granularity is %IOMMU_INV_GRANU_PASID
+ * @addr_info: invalidation data when @granularity is %IOMMU_INV_GRANU_ADDR
+ *
+ * Not all the combinations of cache/granularity are valid:
+ *
+ * +--------------+---------------+---------------+---------------+
+ * | type / | DEV_IOTLB | IOTLB | PASID |
+ * | granularity | | | cache |
+ * +==============+===============+===============+===============+
+ * | DOMAIN | N/A | Y | Y |
+ * +--------------+---------------+---------------+---------------+
+ * | PASID | Y | Y | Y |
+ * +--------------+---------------+---------------+---------------+
+ * | ADDR | Y | Y | N/A |
+ * +--------------+---------------+---------------+---------------+
+ *
+ * Invalidations by %IOMMU_INV_GRANU_DOMAIN don't take any argument other than
+ * @version and @cache.
+ *
+ * If multiple cache types are invalidated simultaneously, they all
+ * must support the used granularity.
+ */
+struct iommu_cache_invalidate_info {
+#define IOMMU_CACHE_INVALIDATE_INFO_VERSION_1 1
+ __u32 version;
+/* IOMMU paging structure cache */
+#define IOMMU_CACHE_INV_TYPE_IOTLB (1 << 0) /* IOMMU IOTLB */
+#define IOMMU_CACHE_INV_TYPE_DEV_IOTLB (1 << 1) /* Device IOTLB */
+#define IOMMU_CACHE_INV_TYPE_PASID (1 << 2) /* PASID cache */
+#define IOMMU_CACHE_INV_TYPE_NR (3)
+ __u8 cache;
+ __u8 granularity;
+ __u8 padding[2];
+ union {
+ struct iommu_inv_pasid_info pasid_info;
+ struct iommu_inv_addr_info addr_info;
+ };
+};
+
#endif /* _UAPI_IOMMU_H */
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 07/19] iommu: Add I/O ASID allocator
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (5 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 06/19] iommu: Introduce cache_invalidate API Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 08/19] iommu/ioasid: Add custom allocators Jacob Pan
` (12 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Some devices might support multiple DMA address spaces, in particular
those that have the PCI PASID feature. PASID (Process Address Space ID)
allows to share process address spaces with devices (SVA), partition a
device into VM-assignable entities (VFIO mdev) or simply provide
multiple DMA address space to kernel drivers. Add a global PASID
allocator usable by different drivers at the same time. Name it I/O ASID
to avoid confusion with ASIDs allocated by arch code, which are usually
a separate ID space.
The IOASID space is global. Each device can have its own PASID space,
but by convention the IOMMU ended up having a global PASID space, so
that with SVA, each mm_struct is associated to a single PASID.
The allocator is primarily used by IOMMU subsystem but in rare occasions
drivers would like to allocate PASIDs for devices that aren't managed by
an IOMMU, using the same ID space as IOMMU.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/Kconfig | 4 ++
drivers/iommu/Makefile | 1 +
drivers/iommu/ioasid.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++
include/linux/ioasid.h | 47 +++++++++++++++
4 files changed, 203 insertions(+)
create mode 100644 drivers/iommu/ioasid.c
create mode 100644 include/linux/ioasid.h
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index e15cdcd..0ade8a0 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -3,6 +3,10 @@
config IOMMU_IOVA
tristate
+# The IOASID library may also be used by non-IOMMU_API users
+config IOASID
+ tristate
+
# IOMMU_API always gets selected by whoever wants it.
config IOMMU_API
bool
diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile
index f13f36a..011429e 100644
--- a/drivers/iommu/Makefile
+++ b/drivers/iommu/Makefile
@@ -7,6 +7,7 @@ obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o
obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o
obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o
obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o
+obj-$(CONFIG_IOASID) += ioasid.o
obj-$(CONFIG_IOMMU_IOVA) += iova.o
obj-$(CONFIG_OF_IOMMU) += of_iommu.o
obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o
diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c
new file mode 100644
index 0000000..6fbea76
--- /dev/null
+++ b/drivers/iommu/ioasid.c
@@ -0,0 +1,151 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * I/O Address Space ID allocator. There is one global IOASID space, split into
+ * subsets. Users create a subset with DECLARE_IOASID_SET, then allocate and
+ * free IOASIDs with ioasid_alloc and ioasid_free.
+ */
+#include <linux/ioasid.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/xarray.h>
+
+struct ioasid_data {
+ ioasid_t id;
+ struct ioasid_set *set;
+ void *private;
+ struct rcu_head rcu;
+};
+
+static DEFINE_XARRAY_ALLOC(ioasid_xa);
+
+/**
+ * ioasid_set_data - Set private data for an allocated ioasid
+ * @ioasid: the ID to set data
+ * @data: the private data
+ *
+ * For IOASID that is already allocated, private data can be set
+ * via this API. Future lookup can be done via ioasid_find.
+ */
+int ioasid_set_data(ioasid_t ioasid, void *data)
+{
+ struct ioasid_data *ioasid_data;
+ int ret = 0;
+
+ xa_lock(&ioasid_xa);
+ ioasid_data = xa_load(&ioasid_xa, ioasid);
+ if (ioasid_data)
+ rcu_assign_pointer(ioasid_data->private, data);
+ else
+ ret = -ENOENT;
+ xa_unlock(&ioasid_xa);
+
+ /*
+ * Wait for readers to stop accessing the old private data, so the
+ * caller can free it.
+ */
+ if (!ret)
+ synchronize_rcu();
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(ioasid_set_data);
+
+/**
+ * ioasid_alloc - Allocate an IOASID
+ * @set: the IOASID set
+ * @min: the minimum ID (inclusive)
+ * @max: the maximum ID (inclusive)
+ * @private: data private to the caller
+ *
+ * Allocate an ID between @min and @max. The @private pointer is stored
+ * internally and can be retrieved with ioasid_find().
+ *
+ * Return: the allocated ID on success, or %INVALID_IOASID on failure.
+ */
+ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
+ void *private)
+{
+ ioasid_t id;
+ struct ioasid_data *data;
+
+ data = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!data)
+ return INVALID_IOASID;
+
+ data->set = set;
+ data->private = private;
+
+ if (xa_alloc(&ioasid_xa, &id, data, XA_LIMIT(min, max), GFP_KERNEL)) {
+ pr_err("Failed to alloc ioasid from %d to %d\n", min, max);
+ goto exit_free;
+ }
+ data->id = id;
+
+ return id;
+exit_free:
+ kfree(data);
+ return INVALID_IOASID;
+}
+EXPORT_SYMBOL_GPL(ioasid_alloc);
+
+/**
+ * ioasid_free - Free an IOASID
+ * @ioasid: the ID to remove
+ */
+void ioasid_free(ioasid_t ioasid)
+{
+ struct ioasid_data *ioasid_data;
+
+ ioasid_data = xa_erase(&ioasid_xa, ioasid);
+
+ kfree_rcu(ioasid_data, rcu);
+}
+EXPORT_SYMBOL_GPL(ioasid_free);
+
+/**
+ * ioasid_find - Find IOASID data
+ * @set: the IOASID set
+ * @ioasid: the IOASID to find
+ * @getter: function to call on the found object
+ *
+ * The optional getter function allows to take a reference to the found object
+ * under the rcu lock. The function can also check if the object is still valid:
+ * if @getter returns false, then the object is invalid and NULL is returned.
+ *
+ * If the IOASID exists, return the private pointer passed to ioasid_alloc.
+ * Private data can be NULL if not set. Return an error if the IOASID is not
+ * found, or if @set is not NULL and the IOASID does not belong to the set.
+ */
+void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
+ bool (*getter)(void *))
+{
+ void *priv;
+ struct ioasid_data *ioasid_data;
+
+ rcu_read_lock();
+ ioasid_data = xa_load(&ioasid_xa, ioasid);
+ if (!ioasid_data) {
+ priv = ERR_PTR(-ENOENT);
+ goto unlock;
+ }
+ if (set && ioasid_data->set != set) {
+ /* data found but does not belong to the set */
+ priv = ERR_PTR(-EACCES);
+ goto unlock;
+ }
+ /* Now IOASID and its set is verified, we can return the private data */
+ priv = rcu_dereference(ioasid_data->private);
+ if (getter && !getter(priv))
+ priv = NULL;
+unlock:
+ rcu_read_unlock();
+
+ return priv;
+}
+EXPORT_SYMBOL_GPL(ioasid_find);
+
+MODULE_AUTHOR("Jean-Philippe Brucker <jean-philippe.brucker@arm.com>");
+MODULE_AUTHOR("Jacob Pan <jacob.jun.pan@linux.intel.com>");
+MODULE_DESCRIPTION("IO Address Space ID (IOASID) allocator");
+MODULE_LICENSE("GPL");
diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h
new file mode 100644
index 0000000..0c272d9
--- /dev/null
+++ b/include/linux/ioasid.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __LINUX_IOASID_H
+#define __LINUX_IOASID_H
+
+#include <linux/types.h>
+
+#define INVALID_IOASID ((ioasid_t)-1)
+typedef unsigned int ioasid_t;
+
+struct ioasid_set {
+ int dummy;
+};
+
+#define DECLARE_IOASID_SET(name) struct ioasid_set name = { 0 }
+
+#if IS_ENABLED(CONFIG_IOASID)
+ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
+ void *private);
+void ioasid_free(ioasid_t ioasid);
+void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
+ bool (*getter)(void *));
+int ioasid_set_data(ioasid_t ioasid, void *data);
+
+#else /* !CONFIG_IOASID */
+static inline ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min,
+ ioasid_t max, void *private)
+{
+ return INVALID_IOASID;
+}
+
+static inline void ioasid_free(ioasid_t ioasid)
+{
+}
+
+static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
+ bool (*getter)(void *))
+{
+ return NULL;
+}
+
+static inline int ioasid_set_data(ioasid_t ioasid, void *data)
+{
+ return -ENODEV;
+}
+
+#endif /* CONFIG_IOASID */
+#endif /* __LINUX_IOASID_H */
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 08/19] iommu/ioasid: Add custom allocators
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (6 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 07/19] iommu: Add I/O ASID allocator Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 09/19] iommu: Introduce guest PASID bind function Jacob Pan
` (11 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Custom IOASID allocators can be registered at runtime and take precedence
over the default XArray allocator. They have these attributes:
- provides platform specific alloc()/free() functions with private data.
- allocation results lookup are not provided by the allocator, lookup
request must be done by the IOASID framework by its own XArray.
- allocators can be unregistered at runtime, either fallback to the next
custom allocator or to the default allocator.
- custom allocators can share the same set of alloc()/free() helpers, in
this case they also share the same IOASID space, thus the same XArray.
- switching between allocators requires all outstanding IOASIDs to be
freed unless the two allocators share the same alloc()/free() helpers.
Signed-off-by: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Link: https://lkml.org/lkml/2019/4/26/462
---
drivers/iommu/ioasid.c | 302 +++++++++++++++++++++++++++++++++++++++++++++++--
include/linux/ioasid.h | 28 +++++
2 files changed, 320 insertions(+), 10 deletions(-)
diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c
index 6fbea76..b2d9ea5 100644
--- a/drivers/iommu/ioasid.c
+++ b/drivers/iommu/ioasid.c
@@ -17,7 +17,254 @@ struct ioasid_data {
struct rcu_head rcu;
};
-static DEFINE_XARRAY_ALLOC(ioasid_xa);
+/*
+ * struct ioasid_allocator_data - Internal data structure to hold information
+ * about an allocator. There are two types of allocators:
+ *
+ * - Default allocator always has its own XArray to track the IOASIDs allocated.
+ * - Custom allocators may share allocation helpers with different private data.
+ * Custom allocators share the same helper functions also share the same
+ * XArray.
+ * Rules:
+ * 1. Default allocator is always available, not dynamically registered. This is
+ * to prevent race conditions with early boot code that want to register
+ * custom allocators or allocate IOASIDs.
+ * 2. Custom allocators take precedence over the default allocator.
+ * 3. When all custom allocators sharing the same helper functions are
+ * unregistered (e.g. due to hotplug), all outstanding IOASIDs must be
+ * freed.
+ * 4. When switching between custom allocators sharing the same helper
+ * functions, outstanding IOASIDs are preserved.
+ * 5. When switching between custom allocator and default allocator, all IOASIDs
+ * must be freed to ensure unadulterated space for the new allocator.
+ *
+ * @ops: allocator helper functions and its data
+ * @list: registered custom allocators
+ * @slist: allocators share the same ops but different data
+ * @flags: attributes of the allocator
+ * @xa xarray holds the IOASID space
+ * @users number of allocators sharing the same ops and XArray
+ */
+struct ioasid_allocator_data {
+ struct ioasid_allocator_ops *ops;
+ struct list_head list;
+ struct list_head slist;
+#define IOASID_ALLOCATOR_CUSTOM BIT(0) /* Needs framework to track results */
+ unsigned long flags;
+ struct xarray xa;
+ refcount_t users;
+};
+
+static DEFINE_MUTEX(ioasid_allocator_lock);
+static LIST_HEAD(allocators_list);
+
+static ioasid_t default_alloc(ioasid_t min, ioasid_t max, void *opaque);
+static void default_free(ioasid_t ioasid, void *opaque);
+
+static struct ioasid_allocator_ops default_ops = {
+ .alloc = default_alloc,
+ .free = default_free,
+};
+
+static struct ioasid_allocator_data default_allocator = {
+ .ops = &default_ops,
+ .flags = 0,
+ .xa = XARRAY_INIT(ioasid_xa, XA_FLAGS_ALLOC),
+};
+
+static struct ioasid_allocator_data *active_allocator = &default_allocator;
+
+static ioasid_t default_alloc(ioasid_t min, ioasid_t max, void *opaque)
+{
+ ioasid_t id;
+
+ if (xa_alloc(&default_allocator.xa, &id, opaque, XA_LIMIT(min, max), GFP_KERNEL)) {
+ pr_err("Failed to alloc ioasid from %d to %d\n", min, max);
+ return INVALID_IOASID;
+ }
+
+ return id;
+}
+
+static void default_free(ioasid_t ioasid, void *opaque)
+{
+ struct ioasid_data *ioasid_data;
+
+ ioasid_data = xa_erase(&default_allocator.xa, ioasid);
+ kfree_rcu(ioasid_data, rcu);
+}
+
+/* Allocate and initialize a new custom allocator with its helper functions */
+static struct ioasid_allocator_data *ioasid_alloc_allocator(struct ioasid_allocator_ops *ops)
+{
+ struct ioasid_allocator_data *ia_data;
+
+ ia_data = kzalloc(sizeof(*ia_data), GFP_KERNEL);
+ if (!ia_data)
+ return NULL;
+
+ xa_init_flags(&ia_data->xa, XA_FLAGS_ALLOC);
+ INIT_LIST_HEAD(&ia_data->slist);
+ ia_data->flags |= IOASID_ALLOCATOR_CUSTOM;
+ ia_data->ops = ops;
+
+ /* For tracking custom allocators that share the same ops */
+ list_add_tail(&ops->list, &ia_data->slist);
+ refcount_set(&ia_data->users, 1);
+
+ return ia_data;
+}
+
+static bool use_same_ops(struct ioasid_allocator_ops *a, struct ioasid_allocator_ops *b)
+{
+ return (a->free == b->free) && (a->alloc == b->alloc);
+}
+
+/**
+ * ioasid_register_allocator - register a custom allocator
+ * @ops: the custom allocator ops to be registered
+ *
+ * Custom allocators take precedence over the default xarray based allocator.
+ * Private data associated with the IOASID allocated by the custom allocators
+ * are managed by IOASID framework similar to data stored in xa by default
+ * allocator.
+ *
+ * There can be multiple allocators registered but only one is active. In case
+ * of runtime removal of a custom allocator, the next one is activated based
+ * on the registration ordering.
+ *
+ * Multiple allocators can share the same alloc() function, in this case the
+ * IOASID space is shared.
+ */
+int ioasid_register_allocator(struct ioasid_allocator_ops *ops)
+{
+ struct ioasid_allocator_data *ia_data;
+ struct ioasid_allocator_data *pallocator;
+ int ret = 0;
+
+ mutex_lock(&ioasid_allocator_lock);
+
+ ia_data = ioasid_alloc_allocator(ops);
+ if (!ia_data) {
+ ret = -ENOMEM;
+ goto out_unlock;
+ }
+
+ /*
+ * No particular preference, we activate the first one and keep
+ * the later registered allocators in a list in case the first one gets
+ * removed due to hotplug.
+ */
+ if (list_empty(&allocators_list)) {
+ WARN_ON(active_allocator != &default_allocator);
+ /* Use this new allocator if default is not active */
+ if (xa_empty(&active_allocator->xa)) {
+ active_allocator = ia_data;
+ list_add_tail(&ia_data->list, &allocators_list);
+ goto out_unlock;
+ }
+ pr_warn("Default allocator active with outstanding IOASID\n");
+ ret = -EAGAIN;
+ goto out_free;
+ }
+
+ /* Check if the allocator is already registered */
+ list_for_each_entry(pallocator, &allocators_list, list) {
+ if (pallocator->ops == ops) {
+ pr_err("IOASID allocator already registered\n");
+ ret = -EEXIST;
+ goto out_free;
+ } else if (use_same_ops(pallocator->ops, ops)) {
+ /*
+ * If the new allocator shares the same ops,
+ * then they will share the same IOASID space.
+ * We should put them under the same xarray.
+ */
+ list_add_tail(&ops->list, &pallocator->slist);
+ refcount_inc(&pallocator->users);
+ pr_info("New IOASID allocator ops shared %u times\n",
+ refcount_read(&pallocator->users));
+ goto out_free;
+ }
+ }
+ list_add_tail(&ia_data->list, &allocators_list);
+
+ mutex_unlock(&ioasid_allocator_lock);
+ return 0;
+out_free:
+ kfree(ia_data);
+out_unlock:
+ mutex_unlock(&ioasid_allocator_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(ioasid_register_allocator);
+
+static inline bool has_shared_ops(struct ioasid_allocator_data *allocator)
+{
+ return refcount_read(&allocator->users) > 1;
+}
+
+/**
+ * ioasid_unregister_allocator - Remove a custom IOASID allocator ops
+ * @ops: the custom allocator to be removed
+ *
+ * Remove an allocator from the list, activate the next allocator in
+ * the order it was registered. Or revert to default allocator if all
+ * custom allocators are unregistered without outstanding IOASIDs.
+ */
+void ioasid_unregister_allocator(struct ioasid_allocator_ops *ops)
+{
+ struct ioasid_allocator_data *pallocator;
+ struct ioasid_allocator_ops *sops;
+
+ mutex_lock(&ioasid_allocator_lock);
+ if (list_empty(&allocators_list)) {
+ pr_warn("No custom IOASID allocators active!\n");
+ goto exit_unlock;
+ }
+
+ list_for_each_entry(pallocator, &allocators_list, list) {
+ if (!use_same_ops(pallocator->ops, ops))
+ continue;
+
+ if (refcount_read(&pallocator->users) == 1) {
+ /* No shared helper functions */
+ list_del(&pallocator->list);
+ /*
+ * All IOASIDs should have been freed before
+ * the last allocator that shares the same ops
+ * is unregistered.
+ */
+ WARN_ON(!xa_empty(&pallocator->xa));
+ kfree(pallocator);
+ if (list_empty(&allocators_list)) {
+ pr_info("No custom IOASID allocators, switch to default.\n");
+ active_allocator = &default_allocator;
+ } else if (pallocator == active_allocator) {
+ active_allocator = list_first_entry(&allocators_list, struct ioasid_allocator_data, list);
+ pr_info("IOASID allocator changed");
+ }
+ break;
+ }
+ /*
+ * Find the matching shared ops to delete,
+ * but keep outstanding IOASIDs
+ */
+ list_for_each_entry(sops, &pallocator->slist, list) {
+ if (sops == ops) {
+ list_del(&ops->list);
+ if (refcount_dec_and_test(&pallocator->users))
+ pr_err("no shared ops\n");
+ break;
+ }
+ }
+ break;
+ }
+
+exit_unlock:
+ mutex_unlock(&ioasid_allocator_lock);
+}
+EXPORT_SYMBOL_GPL(ioasid_unregister_allocator);
/**
* ioasid_set_data - Set private data for an allocated ioasid
@@ -32,13 +279,13 @@ int ioasid_set_data(ioasid_t ioasid, void *data)
struct ioasid_data *ioasid_data;
int ret = 0;
- xa_lock(&ioasid_xa);
- ioasid_data = xa_load(&ioasid_xa, ioasid);
+ mutex_lock(&ioasid_allocator_lock);
+ ioasid_data = xa_load(&active_allocator->xa, ioasid);
if (ioasid_data)
rcu_assign_pointer(ioasid_data->private, data);
else
ret = -ENOENT;
- xa_unlock(&ioasid_xa);
+ mutex_unlock(&ioasid_allocator_lock);
/*
* Wait for readers to stop accessing the old private data, so the
@@ -66,8 +313,9 @@ EXPORT_SYMBOL_GPL(ioasid_set_data);
ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
void *private)
{
- ioasid_t id;
struct ioasid_data *data;
+ void *adata;
+ ioasid_t id;
data = kzalloc(sizeof(*data), GFP_KERNEL);
if (!data)
@@ -76,14 +324,34 @@ ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
data->set = set;
data->private = private;
- if (xa_alloc(&ioasid_xa, &id, data, XA_LIMIT(min, max), GFP_KERNEL)) {
- pr_err("Failed to alloc ioasid from %d to %d\n", min, max);
+ /*
+ * Custom allocator needs allocator data to perform platform specific
+ * operations.
+ */
+ adata = active_allocator->flags & IOASID_ALLOCATOR_CUSTOM ? active_allocator->ops->pdata : data;
+ mutex_lock(&ioasid_allocator_lock);
+ id = active_allocator->ops->alloc(min, max, adata);
+ if (id == INVALID_IOASID) {
+ pr_err("Failed ASID allocation %lu\n", active_allocator->flags);
goto exit_free;
}
+
+ if (active_allocator->flags & IOASID_ALLOCATOR_CUSTOM) {
+ /* Custom allocator needs framework to store and track allocation results */
+ min = max = id;
+
+ if (xa_alloc(&active_allocator->xa, &id, data, XA_LIMIT(min, max), GFP_KERNEL)) {
+ pr_err("Failed to alloc ioasid from %d to %d\n", min, max);
+ active_allocator->ops->free(id, NULL);
+ goto exit_free;
+ }
+ }
data->id = id;
+ mutex_unlock(&ioasid_allocator_lock);
return id;
exit_free:
+ mutex_unlock(&ioasid_allocator_lock);
kfree(data);
return INVALID_IOASID;
}
@@ -97,9 +365,23 @@ void ioasid_free(ioasid_t ioasid)
{
struct ioasid_data *ioasid_data;
- ioasid_data = xa_erase(&ioasid_xa, ioasid);
+ mutex_lock(&ioasid_allocator_lock);
- kfree_rcu(ioasid_data, rcu);
+ ioasid_data = xa_load(&active_allocator->xa, ioasid);
+ if (!ioasid_data) {
+ pr_err("Trying to free unknown IOASID %u\n", ioasid);
+ goto exit_unlock;
+ }
+
+ active_allocator->ops->free(ioasid, active_allocator->ops->pdata);
+ /* Custom allocator needs additional steps to free the xa element */
+ if (active_allocator->flags & IOASID_ALLOCATOR_CUSTOM) {
+ ioasid_data = xa_erase(&active_allocator->xa, ioasid);
+ kfree_rcu(ioasid_data, rcu);
+ }
+
+exit_unlock:
+ mutex_unlock(&ioasid_allocator_lock);
}
EXPORT_SYMBOL_GPL(ioasid_free);
@@ -124,7 +406,7 @@ void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
struct ioasid_data *ioasid_data;
rcu_read_lock();
- ioasid_data = xa_load(&ioasid_xa, ioasid);
+ ioasid_data = xa_load(&active_allocator->xa, ioasid);
if (!ioasid_data) {
priv = ERR_PTR(-ENOENT);
goto unlock;
diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h
index 0c272d9..33ef3ea 100644
--- a/include/linux/ioasid.h
+++ b/include/linux/ioasid.h
@@ -6,11 +6,28 @@
#define INVALID_IOASID ((ioasid_t)-1)
typedef unsigned int ioasid_t;
+typedef ioasid_t (*ioasid_alloc_fn_t)(ioasid_t min, ioasid_t max, void *data);
+typedef void (*ioasid_free_fn_t)(ioasid_t ioasid, void *data);
struct ioasid_set {
int dummy;
};
+/**
+ * struct ioasid_allocator_ops - IOASID allocator helper functions and data
+ *
+ * @alloc: helper function to allocate IOASID
+ * @free: helper function to free IOASID
+ * @list: for tracking ops that share helper functions but not data
+ * @pdata: data belong to the allocator, provided when calling alloc()
+ */
+struct ioasid_allocator_ops {
+ ioasid_alloc_fn_t alloc;
+ ioasid_free_fn_t free;
+ struct list_head list;
+ void *pdata;
+};
+
#define DECLARE_IOASID_SET(name) struct ioasid_set name = { 0 }
#if IS_ENABLED(CONFIG_IOASID)
@@ -19,6 +36,8 @@ ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max,
void ioasid_free(ioasid_t ioasid);
void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
bool (*getter)(void *));
+int ioasid_register_allocator(struct ioasid_allocator_ops *allocator);
+void ioasid_unregister_allocator(struct ioasid_allocator_ops *allocator);
int ioasid_set_data(ioasid_t ioasid, void *data);
#else /* !CONFIG_IOASID */
@@ -38,6 +57,15 @@ static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid,
return NULL;
}
+static inline int ioasid_register_allocator(struct ioasid_allocator_ops *allocator)
+{
+ return -ENODEV;
+}
+
+static inline void ioasid_unregister_allocator(struct ioasid_allocator_ops *allocator)
+{
+}
+
static inline int ioasid_set_data(ioasid_t ioasid, void *data)
{
return -ENODEV;
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 09/19] iommu: Introduce guest PASID bind function
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (7 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 08/19] iommu/ioasid: Add custom allocators Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 10/19] iommu/vt-d: Enlightened PASID allocation Jacob Pan
` (10 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Guest shared virtual address (SVA) may require host to shadow guest
PASID tables. Guest PASID can also be allocated from the host via
enlightened interfaces. In this case, guest needs to bind the guest
mm, i.e. cr3 in guest physical address to the actual PASID table in
the host IOMMU. Nesting will be turned on such that guest virtual
address can go through a two level translation:
- 1st level translates GVA to GPA
- 2nd level translates GPA to HPA
This patch introduces APIs to bind guest PASID data to the assigned
device entry in the physical IOMMU. See the diagram below for usage
explanation.
.-------------. .---------------------------.
| vIOMMU | | Guest process mm, FL only |
| | '---------------------------'
.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | GP
'-------------'
Guest
------| Shadow |----------------------- GP->HP* ---------
v v |
Host v
.-------------. .----------------------.
| pIOMMU | | Bind FL for GVA-GPA |
| | '----------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.---------------------.
| | |Set SL to GPA-HPA |
| | '---------------------'
'-------------'
Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables
- GP = Guest PASID
- HP = Host PASID
* Conversion needed if non-identity GP-HP mapping option is chosen.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
---
drivers/iommu/iommu.c | 20 ++++++++++++++++
include/linux/iommu.h | 22 ++++++++++++++++++
include/uapi/linux/iommu.h | 58 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 100 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 6228d5d..c19ea1f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1729,6 +1729,26 @@ int iommu_cache_invalidate(struct iommu_domain *domain, struct device *dev,
}
EXPORT_SYMBOL_GPL(iommu_cache_invalidate);
+int iommu_sva_bind_gpasid(struct iommu_domain *domain,
+ struct device *dev, struct iommu_gpasid_bind_data *data)
+{
+ if (unlikely(!domain->ops->sva_bind_gpasid))
+ return -ENODEV;
+
+ return domain->ops->sva_bind_gpasid(domain, dev, data);
+}
+EXPORT_SYMBOL_GPL(iommu_sva_bind_gpasid);
+
+int iommu_sva_unbind_gpasid(struct iommu_domain *domain, struct device *dev,
+ ioasid_t pasid)
+{
+ if (unlikely(!domain->ops->sva_unbind_gpasid))
+ return -ENODEV;
+
+ return domain->ops->sva_unbind_gpasid(dev, pasid);
+}
+EXPORT_SYMBOL_GPL(iommu_sva_unbind_gpasid);
+
static void __iommu_detach_device(struct iommu_domain *domain,
struct device *dev)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 28f1a8c..91370e7 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -13,6 +13,7 @@
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/of.h>
+#include <linux/ioasid.h>
#include <uapi/linux/iommu.h>
#define IOMMU_READ (1 << 0)
@@ -232,6 +233,8 @@ struct iommu_sva_ops {
* @detach_pasid_table: detach the pasid table
* @cache_invalidate: invalidate translation caches
* @pgsize_bitmap: bitmap of all possible supported page sizes
+ * @sva_bind_gpasid: bind guest pasid and mm
+ * @sva_unbind_gpasid: unbind guest pasid and mm
*/
struct iommu_ops {
bool (*capable)(enum iommu_cap);
@@ -299,6 +302,10 @@ struct iommu_ops {
struct iommu_page_response *msg);
int (*cache_invalidate)(struct iommu_domain *domain, struct device *dev,
struct iommu_cache_invalidate_info *inv_info);
+ int (*sva_bind_gpasid)(struct iommu_domain *domain,
+ struct device *dev, struct iommu_gpasid_bind_data *data);
+
+ int (*sva_unbind_gpasid)(struct device *dev, int pasid);
unsigned long pgsize_bitmap;
};
@@ -413,6 +420,10 @@ extern void iommu_detach_pasid_table(struct iommu_domain *domain);
extern int iommu_cache_invalidate(struct iommu_domain *domain,
struct device *dev,
struct iommu_cache_invalidate_info *inv_info);
+extern int iommu_sva_bind_gpasid(struct iommu_domain *domain,
+ struct device *dev, struct iommu_gpasid_bind_data *data);
+extern int iommu_sva_unbind_gpasid(struct iommu_domain *domain,
+ struct device *dev, ioasid_t pasid);
extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev);
extern struct iommu_domain *iommu_get_dma_domain(struct device *dev);
extern int iommu_map(struct iommu_domain *domain, unsigned long iova,
@@ -972,6 +983,17 @@ iommu_cache_invalidate(struct iommu_domain *domain,
{
return -ENODEV;
}
+static inline int iommu_sva_bind_gpasid(struct iommu_domain *domain,
+ struct device *dev, struct iommu_gpasid_bind_data *data)
+{
+ return -ENODEV;
+}
+
+static inline int iommu_sva_unbind_gpasid(struct iommu_domain *domain,
+ struct device *dev, int pasid)
+{
+ return -ENODEV;
+}
#endif /* CONFIG_IOMMU_API */
diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h
index 919ea02..780dc1f 100644
--- a/include/uapi/linux/iommu.h
+++ b/include/uapi/linux/iommu.h
@@ -313,4 +313,62 @@ struct iommu_cache_invalidate_info {
};
};
+/**
+ * struct iommu_gpasid_bind_data_vtd - Intel VT-d specific data on device and guest
+ * SVA binding.
+ *
+ * @flags: VT-d PASID table entry attributes
+ * @pat: Page attribute table data to compute effective memory type
+ * @emt: Extended memory type
+ *
+ * Only guest vIOMMU selectable and effective options are passed down to
+ * the host IOMMU.
+ */
+struct iommu_gpasid_bind_data_vtd {
+#define IOMMU_SVA_VTD_GPASID_SRE (1 << 0) /* supervisor request */
+#define IOMMU_SVA_VTD_GPASID_EAFE (1 << 1) /* extended access enable */
+#define IOMMU_SVA_VTD_GPASID_PCD (1 << 2) /* page-level cache disable */
+#define IOMMU_SVA_VTD_GPASID_PWT (1 << 3) /* page-level write through */
+#define IOMMU_SVA_VTD_GPASID_EMTE (1 << 4) /* extended mem type enable */
+#define IOMMU_SVA_VTD_GPASID_CD (1 << 5) /* PASID-level cache disable */
+ __u64 flags;
+ __u32 pat;
+ __u32 emt;
+};
+
+/**
+ * struct iommu_gpasid_bind_data - Information about device and guest PASID binding
+ * @version: Version of this data structure
+ * @format: PASID table entry format
+ * @flags: Additional information on guest bind request
+ * @gpgd: Guest page directory base of the guest mm to bind
+ * @hpasid: Process address space ID used for the guest mm in host IOMMU
+ * @gpasid: Process address space ID used for the guest mm in guest IOMMU
+ * @addr_width: Guest virtual address width
+ * @vtd: Intel VT-d specific data
+ *
+ * Guest to host PASID mapping can be an identity or non-identity, where guest
+ * has its own PASID space. For non-identify mapping, guest to host PASID lookup
+ * is needed when VM programs guest PASID into an assigned device. VMM may
+ * trap such PASID programming then request host IOMMU driver to convert guest
+ * PASID to host PASID based on this bind data.
+ */
+struct iommu_gpasid_bind_data {
+#define IOMMU_GPASID_BIND_VERSION_1 1
+ __u32 version;
+#define IOMMU_PASID_FORMAT_INTEL_VTD 1
+ __u32 format;
+#define IOMMU_SVA_GPASID_VAL (1 << 0) /* guest PASID valid */
+ __u64 flags;
+ __u64 gpgd;
+ __u64 hpasid;
+ __u64 gpasid;
+ __u32 addr_width;
+ __u8 padding[4];
+ /* Vendor specific data */
+ union {
+ struct iommu_gpasid_bind_data_vtd vtd;
+ };
+};
+
#endif /* _UAPI_IOMMU_H */
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 10/19] iommu/vt-d: Enlightened PASID allocation
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (8 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 09/19] iommu: Introduce guest PASID bind function Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 11/19] iommu/vt-d: Add custom allocator for IOASID Jacob Pan
` (9 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
From: Lu Baolu <baolu.lu@linux.intel.com>
If Intel IOMMU runs in caching mode, a.k.a. virtual IOMMU, the
IOMMU driver should rely on the emulation software to allocate
and free PASID IDs. The Intel vt-d spec revision 3.0 defines a
register set to support this. This includes a capability register,
a virtual command register and a virtual response register. Refer
to section 10.4.42, 10.4.43, 10.4.44 for more information.
This patch adds the enlightened PASID allocation/free interfaces
via the virtual command register.
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
---
drivers/iommu/intel-pasid.c | 83 +++++++++++++++++++++++++++++++++++++++++++++
drivers/iommu/intel-pasid.h | 13 ++++++-
include/linux/intel-iommu.h | 2 ++
3 files changed, 97 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-pasid.c b/drivers/iommu/intel-pasid.c
index 040a445..76bcbb2 100644
--- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c
@@ -63,6 +63,89 @@ void *intel_pasid_lookup_id(int pasid)
return p;
}
+static int check_vcmd_pasid(struct intel_iommu *iommu)
+{
+ u64 cap;
+
+ if (!ecap_vcs(iommu->ecap)) {
+ pr_warn("IOMMU: %s: Hardware doesn't support virtual command\n",
+ iommu->name);
+ return -ENODEV;
+ }
+
+ cap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
+ if (!(cap & DMA_VCS_PAS)) {
+ pr_warn("IOMMU: %s: Emulation software doesn't support PASID allocation\n",
+ iommu->name);
+ return -ENODEV;
+ }
+
+ return 0;
+}
+
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid)
+{
+ u64 res;
+ u8 status_code;
+ unsigned long flags;
+ int ret = 0;
+
+ ret = check_vcmd_pasid(iommu);
+ if (ret)
+ return ret;
+
+ raw_spin_lock_irqsave(&iommu->register_lock, flags);
+ dmar_writeq(iommu->reg + DMAR_VCMD_REG, VCMD_CMD_ALLOC);
+ IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+ !(res & VCMD_VRSP_IP), res);
+ raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+ status_code = VCMD_VRSP_SC(res);
+ switch (status_code) {
+ case VCMD_VRSP_SC_SUCCESS:
+ *pasid = VCMD_VRSP_RESULT(res);
+ break;
+ case VCMD_VRSP_SC_NO_PASID_AVAIL:
+ pr_info("IOMMU: %s: No PASID available\n", iommu->name);
+ ret = -ENOMEM;
+ break;
+ default:
+ ret = -ENODEV;
+ pr_warn("IOMMU: %s: Unexpected error code %d\n",
+ iommu->name, status_code);
+ }
+
+ return ret;
+}
+
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid)
+{
+ u64 res;
+ u8 status_code;
+ unsigned long flags;
+
+ if (check_vcmd_pasid(iommu))
+ return;
+
+ raw_spin_lock_irqsave(&iommu->register_lock, flags);
+ dmar_writeq(iommu->reg + DMAR_VCMD_REG, (pasid << 8) | VCMD_CMD_FREE);
+ IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+ !(res & VCMD_VRSP_IP), res);
+ raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+ status_code = VCMD_VRSP_SC(res);
+ switch (status_code) {
+ case VCMD_VRSP_SC_SUCCESS:
+ break;
+ case VCMD_VRSP_SC_INVALID_PASID:
+ pr_info("IOMMU: %s: Invalid PASID\n", iommu->name);
+ break;
+ default:
+ pr_warn("IOMMU: %s: Unexpected error code %d\n",
+ iommu->name, status_code);
+ }
+}
+
/*
* Per device pasid table management:
*/
diff --git a/drivers/iommu/intel-pasid.h b/drivers/iommu/intel-pasid.h
index fc8cd8f..e413e88 100644
--- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h
@@ -23,6 +23,16 @@
#define is_pasid_enabled(entry) (((entry)->lo >> 3) & 0x1)
#define get_pasid_dir_size(entry) (1 << ((((entry)->lo >> 9) & 0x7) + 7))
+/* Virtual command interface for enlightened pasid management. */
+#define VCMD_CMD_ALLOC 0x1
+#define VCMD_CMD_FREE 0x2
+#define VCMD_VRSP_IP 0x1
+#define VCMD_VRSP_SC(e) (((e) >> 1) & 0x3)
+#define VCMD_VRSP_SC_SUCCESS 0
+#define VCMD_VRSP_SC_NO_PASID_AVAIL 1
+#define VCMD_VRSP_SC_INVALID_PASID 1
+#define VCMD_VRSP_RESULT(e) (((e) >> 8) & 0xfffff)
+
/*
* Domain ID reserved for pasid entries programmed for first-level
* only and pass-through transfer modes.
@@ -95,5 +105,6 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
struct device *dev, int pasid);
void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
struct device *dev, int pasid);
-
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid);
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid);
#endif /* __INTEL_PASID_H */
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index f2ae8a0..37fb0c9 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -161,6 +161,7 @@
#define ecap_smpwc(e) (((e) >> 48) & 0x1)
#define ecap_flts(e) (((e) >> 47) & 0x1)
#define ecap_slts(e) (((e) >> 46) & 0x1)
+#define ecap_vcs(e) (((e) >> 44) & 0x1)
#define ecap_smts(e) (((e) >> 43) & 0x1)
#define ecap_dit(e) ((e >> 41) & 0x1)
#define ecap_pasid(e) ((e >> 40) & 0x1)
@@ -277,6 +278,7 @@
/* PRS_REG */
#define DMA_PRS_PPR ((u32)1)
+#define DMA_VCS_PAS ((u64)1)
#define IOMMU_WAIT_OP(iommu, offset, op, cond, sts) \
do { \
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 11/19] iommu/vt-d: Add custom allocator for IOASID
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (9 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 10/19] iommu/vt-d: Enlightened PASID allocation Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 12/19] iommu/vt-d: Replace Intel specific PASID allocator with IOASID Jacob Pan
` (8 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When VT-d driver runs in the guest, PASID allocation must be
performed via virtual command interface. This patch registers a
custom IOASID allocator which takes precedence over the default
XArray based allocator. The resulting IOASID allocation will always
come from the host. This ensures that PASID namespace is system-
wide.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Liu, Yi L <yi.l.liu@intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/Kconfig | 1 +
drivers/iommu/intel-iommu.c | 67 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/intel-iommu.h | 2 ++
3 files changed, 70 insertions(+)
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 0ade8a0..d5ca821 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -210,6 +210,7 @@ config INTEL_IOMMU_SVM
bool "Support for Shared Virtual Memory with Intel IOMMU"
depends on INTEL_IOMMU && X86
select PCI_PASID
+ select IOASID
select MMU_NOTIFIER
help
Shared Virtual Memory (SVM) provides a facility for devices
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index bdaed2d..b15ec58 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1693,6 +1693,8 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
if (ecap_prs(iommu->ecap))
intel_svm_finish_prq(iommu);
}
+ ioasid_unregister_allocator(&iommu->pasid_allocator);
+
#endif
}
@@ -4619,6 +4621,46 @@ static int __init probe_acpi_namespace_devices(void)
return 0;
}
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max, void *data)
+{
+ struct intel_iommu *iommu = data;
+ ioasid_t ioasid;
+
+ /*
+ * VT-d virtual command interface always uses the full 20 bit
+ * PASID range. Host can partition guest PASID range based on
+ * policies but it is out of guest's control.
+ */
+ if (min < PASID_MIN || max > PASID_MAX)
+ return INVALID_IOASID;
+
+ if (vcmd_alloc_pasid(iommu, &ioasid))
+ return INVALID_IOASID;
+
+ return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+ struct iommu_pasid_alloc_info *svm;
+ struct intel_iommu *iommu = data;
+
+ if (!iommu)
+ return;
+ /*
+ * Sanity check the ioasid owner is done at upper layer, e.g. VFIO
+ * We can only free the PASID when all the devices are unbond.
+ */
+ svm = ioasid_find(NULL, ioasid, NULL);
+ if (!svm) {
+ pr_warn("Freeing unbond IOASID %d\n", ioasid);
+ return;
+ }
+ vcmd_free_pasid(iommu, ioasid);
+}
+#endif
+
int __init intel_iommu_init(void)
{
int ret = -ENODEV;
@@ -4722,6 +4764,31 @@ int __init intel_iommu_init(void)
"%s", iommu->name);
iommu_device_set_ops(&iommu->iommu, &intel_iommu_ops);
iommu_device_register(&iommu->iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+ if (cap_caching_mode(iommu->cap) && sm_supported(iommu)) {
+ /*
+ * Register a custom ASID allocator if we are running
+ * in a guest, the purpose is to have a system wide PASID
+ * namespace among all PASID users.
+ * There can be multiple vIOMMUs in each guest but only
+ * one allocator is active. All vIOMMU allocators will
+ * eventually be calling the same host allocator.
+ */
+ iommu->pasid_allocator.alloc = intel_ioasid_alloc;
+ iommu->pasid_allocator.free = intel_ioasid_free;
+ iommu->pasid_allocator.pdata = (void *)iommu;
+ ret = ioasid_register_allocator(&iommu->pasid_allocator);
+ if (ret) {
+ pr_warn("Custom PASID allocator registeration failed\n");
+ /*
+ * Disable scalable mode on this IOMMU if there
+ * is no custom allocator. Mixing SM capable vIOMMU
+ * and non-SM vIOMMU are not supported.
+ */
+ intel_iommu_sm = 0;
+ }
+ }
+#endif
}
bus_set_iommu(&pci_bus_type, &intel_iommu_ops);
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 37fb0c9..80318c5 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -19,6 +19,7 @@
#include <linux/iommu.h>
#include <linux/io-64-nonatomic-lo-hi.h>
#include <linux/dmar.h>
+#include <linux/ioasid.h>
#include <asm/cacheflush.h>
#include <asm/iommu.h>
@@ -543,6 +544,7 @@ struct intel_iommu {
#ifdef CONFIG_INTEL_IOMMU_SVM
struct page_req_dsc *prq;
unsigned char prq_name[16]; /* Name for PRQ interrupt */
+ struct ioasid_allocator_ops pasid_allocator; /* Custom allocator for PASIDs */
#endif
struct q_inval *qi; /* Queued invalidation info */
u32 *iommu_state; /* Store iommu states between suspend and resume.*/
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 12/19] iommu/vt-d: Replace Intel specific PASID allocator with IOASID
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (10 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 11/19] iommu/vt-d: Add custom allocator for IOASID Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 13/19] iommu/vt-d: Move domain helper to header Jacob Pan
` (7 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Make use of generic IOASID code to manage PASID allocation,
free, and lookup. Replace Intel specific code.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 12 ++++++------
drivers/iommu/intel-pasid.c | 36 ------------------------------------
drivers/iommu/intel-svm.c | 37 +++++++++++++++++++++----------------
3 files changed, 27 insertions(+), 58 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index b15ec58..96defc3 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4989,7 +4989,7 @@ static void auxiliary_unlink_device(struct dmar_domain *domain,
domain->auxd_refcnt--;
if (!domain->auxd_refcnt && domain->default_pasid > 0)
- intel_pasid_free_id(domain->default_pasid);
+ ioasid_free(domain->default_pasid);
}
static int aux_domain_add_dev(struct dmar_domain *domain,
@@ -5007,10 +5007,10 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
if (domain->default_pasid <= 0) {
int pasid;
- pasid = intel_pasid_alloc_id(domain, PASID_MIN,
- pci_max_pasids(to_pci_dev(dev)),
- GFP_KERNEL);
- if (pasid <= 0) {
+ /* No private data needed for the default pasid */
+ pasid = ioasid_alloc(NULL, PASID_MIN, pci_max_pasids(to_pci_dev(dev)) - 1,
+ NULL);
+ if (pasid == INVALID_IOASID) {
pr_err("Can't allocate default pasid\n");
return -ENODEV;
}
@@ -5046,7 +5046,7 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
spin_unlock(&iommu->lock);
spin_unlock_irqrestore(&device_domain_lock, flags);
if (!domain->auxd_refcnt && domain->default_pasid > 0)
- intel_pasid_free_id(domain->default_pasid);
+ ioasid_free(domain->default_pasid);
return ret;
}
diff --git a/drivers/iommu/intel-pasid.c b/drivers/iommu/intel-pasid.c
index 76bcbb2..c0d1f28 100644
--- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c
@@ -26,42 +26,6 @@
*/
static DEFINE_SPINLOCK(pasid_lock);
u32 intel_pasid_max_id = PASID_MAX;
-static DEFINE_IDR(pasid_idr);
-
-int intel_pasid_alloc_id(void *ptr, int start, int end, gfp_t gfp)
-{
- int ret, min, max;
-
- min = max_t(int, start, PASID_MIN);
- max = min_t(int, end, intel_pasid_max_id);
-
- WARN_ON(in_interrupt());
- idr_preload(gfp);
- spin_lock(&pasid_lock);
- ret = idr_alloc(&pasid_idr, ptr, min, max, GFP_ATOMIC);
- spin_unlock(&pasid_lock);
- idr_preload_end();
-
- return ret;
-}
-
-void intel_pasid_free_id(int pasid)
-{
- spin_lock(&pasid_lock);
- idr_remove(&pasid_idr, pasid);
- spin_unlock(&pasid_lock);
-}
-
-void *intel_pasid_lookup_id(int pasid)
-{
- void *p;
-
- spin_lock(&pasid_lock);
- p = idr_find(&pasid_idr, pasid);
- spin_unlock(&pasid_lock);
-
- return p;
-}
static int check_vcmd_pasid(struct intel_iommu *iommu)
{
diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index 780de0c..5a688a5 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -17,6 +17,7 @@
#include <linux/dmar.h>
#include <linux/interrupt.h>
#include <linux/mm_types.h>
+#include <linux/ioasid.h>
#include <asm/page.h>
#include "intel-pasid.h"
@@ -324,16 +325,15 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
if (pasid_max > intel_pasid_max_id)
pasid_max = intel_pasid_max_id;
- /* Do not use PASID 0 in caching mode (virtualised IOMMU) */
- ret = intel_pasid_alloc_id(svm,
- !!cap_caching_mode(iommu->cap),
- pasid_max - 1, GFP_KERNEL);
- if (ret < 0) {
+ /* Do not use PASID 0, reserved for RID to PASID */
+ svm->pasid = ioasid_alloc(NULL, PASID_MIN,
+ pasid_max - 1, svm);
+ if (svm->pasid == INVALID_IOASID) {
kfree(svm);
kfree(sdev);
+ ret = ENOSPC;
goto out;
}
- svm->pasid = ret;
svm->notifier.ops = &intel_mmuops;
svm->mm = mm;
svm->flags = flags;
@@ -343,7 +343,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
if (mm) {
ret = mmu_notifier_register(&svm->notifier, mm);
if (ret) {
- intel_pasid_free_id(svm->pasid);
+ ioasid_free(svm->pasid);
kfree(svm);
kfree(sdev);
goto out;
@@ -359,7 +359,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
if (ret) {
if (mm)
mmu_notifier_unregister(&svm->notifier, mm);
- intel_pasid_free_id(svm->pasid);
+ ioasid_free(svm->pasid);
kfree(svm);
kfree(sdev);
goto out;
@@ -407,7 +407,12 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
if (!iommu)
goto out;
- svm = intel_pasid_lookup_id(pasid);
+ svm = ioasid_find(NULL, pasid, NULL);
+ if (IS_ERR(svm)) {
+ ret = PTR_ERR(svm);
+ goto out;
+ }
+
if (!svm)
goto out;
@@ -429,7 +434,7 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
kfree_rcu(sdev, rcu);
if (list_empty(&svm->devs)) {
- intel_pasid_free_id(svm->pasid);
+ ioasid_free(svm->pasid);
if (svm->mm)
mmu_notifier_unregister(&svm->notifier, svm->mm);
@@ -464,10 +469,11 @@ int intel_svm_is_pasid_valid(struct device *dev, int pasid)
if (!iommu)
goto out;
- svm = intel_pasid_lookup_id(pasid);
- if (!svm)
+ svm = ioasid_find(NULL, pasid, NULL);
+ if (IS_ERR(svm)) {
+ ret = PTR_ERR(svm);
goto out;
-
+ }
/* init_mm is used in this case */
if (!svm->mm)
ret = 1;
@@ -574,13 +580,12 @@ static irqreturn_t prq_event_thread(int irq, void *d)
if (!svm || svm->pasid != req->pasid) {
rcu_read_lock();
- svm = intel_pasid_lookup_id(req->pasid);
+ svm = ioasid_find(NULL, req->pasid, NULL);
/* It *can't* go away, because the driver is not permitted
* to unbind the mm while any page faults are outstanding.
* So we only need RCU to protect the internal idr code. */
rcu_read_unlock();
-
- if (!svm) {
+ if (IS_ERR(svm) || !svm) {
pr_err("%s: Page request for invalid PASID %d: %08llx %08llx\n",
iommu->name, req->pasid, ((unsigned long long *)req)[0],
((unsigned long long *)req)[1]);
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 13/19] iommu/vt-d: Move domain helper to header
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (11 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 12/19] iommu/vt-d: Replace Intel specific PASID allocator with IOASID Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 14/19] iommu/vt-d: Avoid duplicated code for PASID setup Jacob Pan
` (6 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Move domain helper to header to be used by SVA code.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
---
drivers/iommu/intel-iommu.c | 6 ------
include/linux/intel-iommu.h | 6 ++++++
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 96defc3..d2cc355 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -418,12 +418,6 @@ static void init_translation_status(struct intel_iommu *iommu)
iommu->flags |= VTD_FLAG_TRANS_PRE_ENABLED;
}
-/* Convert generic 'struct iommu_domain to private struct dmar_domain */
-static struct dmar_domain *to_dmar_domain(struct iommu_domain *dom)
-{
- return container_of(dom, struct dmar_domain, domain);
-}
-
static int __init intel_iommu_setup(char *str)
{
if (!str)
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 80318c5..e1865f1 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -591,6 +591,12 @@ static inline void __iommu_flush_cache(
clflush_cache_range(addr, size);
}
+/* Convert generic struct iommu_domain to private struct dmar_domain */
+static inline struct dmar_domain *to_dmar_domain(struct iommu_domain *dom)
+{
+ return container_of(dom, struct dmar_domain, domain);
+}
+
/*
* 0: readable
* 1: writable
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 14/19] iommu/vt-d: Avoid duplicated code for PASID setup
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (12 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 13/19] iommu/vt-d: Move domain helper to header Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 15/19] iommu/vt-d: Add nested translation helper function Jacob Pan
` (5 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
After each setup for PASID entry, related translation caches must be flushed.
We can combine duplicated code into one function which is less error prone.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/intel-pasid.c | 48 +++++++++++++++++----------------------------
1 file changed, 18 insertions(+), 30 deletions(-)
diff --git a/drivers/iommu/intel-pasid.c b/drivers/iommu/intel-pasid.c
index c0d1f28..9c5affc 100644
--- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c
@@ -512,6 +512,21 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
devtlb_invalidation_with_pasid(iommu, dev, pasid);
}
+static void pasid_flush_caches(struct intel_iommu *iommu,
+ struct pasid_entry *pte,
+ int pasid, u16 did)
+{
+ if (!ecap_coherent(iommu->ecap))
+ clflush_cache_range(pte, sizeof(*pte));
+
+ if (cap_caching_mode(iommu->cap)) {
+ pasid_cache_invalidation_with_pasid(iommu, did, pasid);
+ iotlb_invalidation_with_pasid(iommu, did, pasid);
+ } else {
+ iommu_flush_write_buffer(iommu);
+ }
+}
+
/*
* Set up the scalable mode pasid table entry for first only
* translation type.
@@ -557,16 +572,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
/* Setup Present and PASID Granular Transfer Type: */
pasid_set_translation_type(pte, 1);
pasid_set_present(pte);
-
- if (!ecap_coherent(iommu->ecap))
- clflush_cache_range(pte, sizeof(*pte));
-
- if (cap_caching_mode(iommu->cap)) {
- pasid_cache_invalidation_with_pasid(iommu, did, pasid);
- iotlb_invalidation_with_pasid(iommu, did, pasid);
- } else {
- iommu_flush_write_buffer(iommu);
- }
+ pasid_flush_caches(iommu, pte, pasid, did);
return 0;
}
@@ -630,16 +636,7 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
*/
pasid_set_sre(pte);
pasid_set_present(pte);
-
- if (!ecap_coherent(iommu->ecap))
- clflush_cache_range(pte, sizeof(*pte));
-
- if (cap_caching_mode(iommu->cap)) {
- pasid_cache_invalidation_with_pasid(iommu, did, pasid);
- iotlb_invalidation_with_pasid(iommu, did, pasid);
- } else {
- iommu_flush_write_buffer(iommu);
- }
+ pasid_flush_caches(iommu, pte, pasid, did);
return 0;
}
@@ -673,16 +670,7 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
*/
pasid_set_sre(pte);
pasid_set_present(pte);
-
- if (!ecap_coherent(iommu->ecap))
- clflush_cache_range(pte, sizeof(*pte));
-
- if (cap_caching_mode(iommu->cap)) {
- pasid_cache_invalidation_with_pasid(iommu, did, pasid);
- iotlb_invalidation_with_pasid(iommu, did, pasid);
- } else {
- iommu_flush_write_buffer(iommu);
- }
+ pasid_flush_caches(iommu, pte, pasid, did);
return 0;
}
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 15/19] iommu/vt-d: Add nested translation helper function
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (13 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 14/19] iommu/vt-d: Avoid duplicated code for PASID setup Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM Jacob Pan
` (4 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.
With PASID granular translation type set to 0x11b, translation
result from the first level(FL) also subject to a second level(SL)
page table translation. This mode is used for SVA virtualization,
where FL performs guest virtual to guest physical translation and
SL performs guest physical to host physical translation.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com>
---
drivers/iommu/intel-pasid.c | 207 ++++++++++++++++++++++++++++++++++++++++++++
drivers/iommu/intel-pasid.h | 12 +++
2 files changed, 219 insertions(+)
diff --git a/drivers/iommu/intel-pasid.c b/drivers/iommu/intel-pasid.c
index 9c5affc..fd2c82f 100644
--- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c
@@ -442,6 +442,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 value)
pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
}
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+ pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << 27);
+}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+ pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << 27);
+}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+ pasid_set_bits(&pe->val[1], 1 << 25, 1);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+ pasid_set_bits(&pe->val[1], 1 << 26, 1);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+ pasid_set_bits(&pe->val[2], 1 << 7, 1);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+ pasid_set_bits(&pe->val[1], 1 << 31, 1);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+ pasid_set_bits(&pe->val[1], 1 << 30, 1);
+}
+
static void
pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
u16 did, int pasid)
@@ -674,3 +744,140 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
return 0;
}
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+ struct pasid_entry *pte,
+ struct iommu_gpasid_bind_data_vtd *pasid_data)
+{
+ /*
+ * Not all guest PASID table entry fields are passed down during bind,
+ * here we only set up the ones that are dependent on guest settings.
+ * Execution related bits such as NXE, SMEP are not meaningful to IOMMU,
+ * therefore not set. Other fields, such as snoop related, are set based
+ * on host needs regardless of guest settings.
+ */
+ if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+ if (!ecap_srs(iommu->ecap)) {
+ pr_err("No supervisor request support on %s\n",
+ iommu->name);
+ return -EINVAL;
+ }
+ pasid_set_sre(pte);
+ }
+
+ if ((pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) && ecap_eafs(iommu->ecap))
+ pasid_set_eafe(pte);
+
+ if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
+ pasid_set_emte(pte);
+ pasid_set_emt(pte, pasid_data->emt);
+ }
+
+ /*
+ * Memory type is only applicable to devices inside processor coherent
+ * domain. PCIe devices are not included. We can skip the rest of the
+ * flags if IOMMU does not support MTS.
+ */
+ if (!ecap_mts(iommu->ecap)) {
+ pr_info("%s does not support memory type bind guest PASID\n",
+ iommu->name);
+ return 0;
+ }
+
+ if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+ pasid_set_pcd(pte);
+ if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+ pasid_set_pwt(pte);
+ if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+ pasid_set_cd(pte);
+ pasid_set_pat(pte, pasid_data->pat);
+
+ return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested translation
+ * which is used for vSVA. The first level page tables are used for
+ * GVA-GPA translation in the guest, second level page tables are used
+ * for GPA to HPA translation.
+ *
+ * @iommu: Iommu which the device belong to
+ * @dev: Device to be set up for translation
+ * @gpgd: FLPTPTR: First Level Page translation pointer in GPA
+ * @pasid: PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+ struct device *dev, pgd_t *gpgd,
+ int pasid, struct iommu_gpasid_bind_data_vtd *pasid_data,
+ struct dmar_domain *domain,
+ int addr_width)
+{
+ struct pasid_entry *pte;
+ struct dma_pte *pgd;
+ u64 pgd_val;
+ int agaw;
+ u16 did;
+
+ if (!ecap_nest(iommu->ecap)) {
+ pr_err("IOMMU: %s: No nested translation support\n",
+ iommu->name);
+ return -EINVAL;
+ }
+
+ pte = intel_pasid_get_entry(dev, pasid);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ pasid_clear_entry(pte);
+
+ /* Sanity checking performed by caller to make sure address
+ * width matching in two dimensions:
+ * 1. CPU vs. IOMMU
+ * 2. Guest vs. Host.
+ */
+ switch (addr_width) {
+ case 57:
+ pasid_set_flpm(pte, 1);
+ break;
+ case 48:
+ pasid_set_flpm(pte, 0);
+ break;
+ default:
+ dev_err(dev, "Invalid paging mode %d\n", addr_width);
+ return -EINVAL;
+ }
+
+ pasid_set_flptr(pte, (u64)gpgd);
+
+ intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+
+ /* Setup the second level based on the given domain */
+ pgd = domain->pgd;
+
+ for (agaw = domain->agaw; agaw != iommu->agaw; agaw--) {
+ pgd = phys_to_virt(dma_pte_addr(pgd));
+ if (!dma_pte_present(pgd)) {
+ dev_err(dev, "Invalid domain page table\n");
+ return -EINVAL;
+ }
+ }
+ pgd_val = virt_to_phys(pgd);
+ pasid_set_slptr(pte, pgd_val);
+ pasid_set_fault_enable(pte);
+
+ did = domain->iommu_did[iommu->seq_id];
+ pasid_set_domain_id(pte, did);
+
+ pasid_set_address_width(pte, agaw);
+ pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+ pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+ pasid_set_present(pte);
+ pasid_flush_caches(iommu, pte, pasid, did);
+
+ return 0;
+}
diff --git a/drivers/iommu/intel-pasid.h b/drivers/iommu/intel-pasid.h
index e413e88..09c85db 100644
--- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h
@@ -46,6 +46,7 @@
* to vmalloc or even module mappings.
*/
#define PASID_FLAG_SUPERVISOR_MODE BIT(0)
+#define PASID_FLAG_NESTED BIT(1)
struct pasid_dir_entry {
u64 val;
@@ -55,6 +56,11 @@ struct pasid_entry {
u64 val[8];
};
+#define PASID_ENTRY_PGTT_FL_ONLY (1)
+#define PASID_ENTRY_PGTT_SL_ONLY (2)
+#define PASID_ENTRY_PGTT_NESTED (3)
+#define PASID_ENTRY_PGTT_PT (4)
+
/* The representative of a PASID table */
struct pasid_table {
void *table; /* pasid table pointer */
@@ -103,6 +109,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
struct dmar_domain *domain,
struct device *dev, int pasid);
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+ struct device *dev, pgd_t *pgd,
+ int pasid,
+ struct iommu_gpasid_bind_data_vtd *pasid_data,
+ struct dmar_domain *domain,
+ int addr_width);
void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
struct device *dev, int pasid);
int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid);
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (14 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 15/19] iommu/vt-d: Add nested translation helper function Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 21:17 ` Andy Shevchenko
2019-08-15 20:13 ` [PATCH v5 17/19] iommu/vt-d: Add bind guest PASID support Jacob Pan
` (3 subsequent siblings)
19 siblings, 1 reply; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
Use combined macros for_each_svm_dev() to simplify SVM device iteration
and error checking.
Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
---
drivers/iommu/intel-svm.c | 85 +++++++++++++++++++++++------------------------
1 file changed, 41 insertions(+), 44 deletions(-)
diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index 5a688a5..ea6f2e2 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -218,6 +218,10 @@ static const struct mmu_notifier_ops intel_mmuops = {
static DEFINE_MUTEX(pasid_mutex);
static LIST_HEAD(global_svm_list);
+#define for_each_svm_dev(svm, dev) \
+ list_for_each_entry(sdev, &svm->devs, list) \
+ if (dev == sdev->dev) \
+
int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
{
struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
@@ -263,15 +267,13 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
goto out;
}
- list_for_each_entry(sdev, &svm->devs, list) {
- if (dev == sdev->dev) {
- if (sdev->ops != ops) {
- ret = -EBUSY;
- goto out;
- }
- sdev->users++;
- goto success;
+ for_each_svm_dev(svm, dev) {
+ if (sdev->ops != ops) {
+ ret = -EBUSY;
+ goto out;
}
+ sdev->users++;
+ goto success;
}
break;
@@ -408,48 +410,43 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
goto out;
svm = ioasid_find(NULL, pasid, NULL);
- if (IS_ERR(svm)) {
+ if (IS_ERR_OR_NULL(svm)) {
ret = PTR_ERR(svm);
goto out;
}
- if (!svm)
- goto out;
+ for_each_svm_dev(svm, dev) {
+ ret = 0;
+ sdev->users--;
+ if (!sdev->users) {
+ list_del_rcu(&sdev->list);
+ /* Flush the PASID cache and IOTLB for this device.
+ * Note that we do depend on the hardware *not* using
+ * the PASID any more. Just as we depend on other
+ * devices never using PASIDs that they have no right
+ * to use. We have a *shared* PASID table, because it's
+ * large and has to be physically contiguous. So it's
+ * hard to be as defensive as we might like. */
+ intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
+ intel_flush_svm_range_dev(svm, sdev, 0, -1, 0, !svm->mm);
+ kfree_rcu(sdev, rcu);
+
+ if (list_empty(&svm->devs)) {
+ ioasid_free(svm->pasid);
+ if (svm->mm)
+ mmu_notifier_unregister(&svm->notifier, svm->mm);
- list_for_each_entry(sdev, &svm->devs, list) {
- if (dev == sdev->dev) {
- ret = 0;
- sdev->users--;
- if (!sdev->users) {
- list_del_rcu(&sdev->list);
- /* Flush the PASID cache and IOTLB for this device.
- * Note that we do depend on the hardware *not* using
- * the PASID any more. Just as we depend on other
- * devices never using PASIDs that they have no right
- * to use. We have a *shared* PASID table, because it's
- * large and has to be physically contiguous. So it's
- * hard to be as defensive as we might like. */
- intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
- intel_flush_svm_range_dev(svm, sdev, 0, -1, 0, !svm->mm);
- kfree_rcu(sdev, rcu);
-
- if (list_empty(&svm->devs)) {
- ioasid_free(svm->pasid);
- if (svm->mm)
- mmu_notifier_unregister(&svm->notifier, svm->mm);
-
- list_del(&svm->list);
-
- /* We mandate that no page faults may be outstanding
- * for the PASID when intel_svm_unbind_mm() is called.
- * If that is not obeyed, subtle errors will happen.
- * Let's make them less subtle... */
- memset(svm, 0x6b, sizeof(*svm));
- kfree(svm);
- }
+ list_del(&svm->list);
+
+ /* We mandate that no page faults may be outstanding
+ * for the PASID when intel_svm_unbind_mm() is called.
+ * If that is not obeyed, subtle errors will happen.
+ * Let's make them less subtle... */
+ memset(svm, 0x6b, sizeof(*svm));
+ kfree(svm);
}
- break;
}
+ break;
}
out:
mutex_unlock(&pasid_mutex);
@@ -585,7 +582,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)
* to unbind the mm while any page faults are outstanding.
* So we only need RCU to protect the internal idr code. */
rcu_read_unlock();
- if (IS_ERR(svm) || !svm) {
+ if (IS_ERR_OR_NULL(svm)) {
pr_err("%s: Page request for invalid PASID %d: %08llx %08llx\n",
iommu->name, req->pasid, ((unsigned long long *)req)[0],
((unsigned long long *)req)[1]);
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* Re: [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM
2019-08-15 20:13 ` [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM Jacob Pan
@ 2019-08-15 21:17 ` Andy Shevchenko
2019-08-15 22:50 ` Jacob Pan
0 siblings, 1 reply; 23+ messages in thread
From: Andy Shevchenko @ 2019-08-15 21:17 UTC (permalink / raw)
To: Jacob Pan
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, iommu, LKML,
Alex Williamson, David Woodhouse, Jonathan Cameron
On Thu, Aug 15, 2019 at 11:52 PM Jacob Pan
<jacob.jun.pan@linux.intel.com> wrote:
>
> Use combined macros for_each_svm_dev() to simplify SVM device iteration
> and error checking.
>
> Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Reviewed-by: Eric Auger <eric.auger@redhat.com>
> ---
> drivers/iommu/intel-svm.c | 85 +++++++++++++++++++++++------------------------
> 1 file changed, 41 insertions(+), 44 deletions(-)
>
> diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
> index 5a688a5..ea6f2e2 100644
> --- a/drivers/iommu/intel-svm.c
> +++ b/drivers/iommu/intel-svm.c
> @@ -218,6 +218,10 @@ static const struct mmu_notifier_ops intel_mmuops = {
> static DEFINE_MUTEX(pasid_mutex);
> static LIST_HEAD(global_svm_list);
>
> +#define for_each_svm_dev(svm, dev) \
> + list_for_each_entry(sdev, &svm->devs, list) \
> + if (dev == sdev->dev) \
This should be
if (dev != sdev->dev) {} else
and no trailing \ is neeeded.
The rationale of above form to avoid
for_each_foo() {
} else {
...WTF?!..
}
> +
> int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
> {
> struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
> @@ -263,15 +267,13 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
> goto out;
> }
>
> - list_for_each_entry(sdev, &svm->devs, list) {
> - if (dev == sdev->dev) {
> - if (sdev->ops != ops) {
> - ret = -EBUSY;
> - goto out;
> - }
> - sdev->users++;
> - goto success;
> + for_each_svm_dev(svm, dev) {
> + if (sdev->ops != ops) {
> + ret = -EBUSY;
> + goto out;
> }
> + sdev->users++;
> + goto success;
> }
>
> break;
> @@ -408,48 +410,43 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
> goto out;
>
> svm = ioasid_find(NULL, pasid, NULL);
> - if (IS_ERR(svm)) {
> + if (IS_ERR_OR_NULL(svm)) {
> ret = PTR_ERR(svm);
> goto out;
> }
>
> - if (!svm)
> - goto out;
> + for_each_svm_dev(svm, dev) {
> + ret = 0;
> + sdev->users--;
> + if (!sdev->users) {
> + list_del_rcu(&sdev->list);
> + /* Flush the PASID cache and IOTLB for this device.
> + * Note that we do depend on the hardware *not* using
> + * the PASID any more. Just as we depend on other
> + * devices never using PASIDs that they have no right
> + * to use. We have a *shared* PASID table, because it's
> + * large and has to be physically contiguous. So it's
> + * hard to be as defensive as we might like. */
> + intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
> + intel_flush_svm_range_dev(svm, sdev, 0, -1, 0, !svm->mm);
> + kfree_rcu(sdev, rcu);
> +
> + if (list_empty(&svm->devs)) {
> + ioasid_free(svm->pasid);
> + if (svm->mm)
> + mmu_notifier_unregister(&svm->notifier, svm->mm);
>
> - list_for_each_entry(sdev, &svm->devs, list) {
> - if (dev == sdev->dev) {
> - ret = 0;
> - sdev->users--;
> - if (!sdev->users) {
> - list_del_rcu(&sdev->list);
> - /* Flush the PASID cache and IOTLB for this device.
> - * Note that we do depend on the hardware *not* using
> - * the PASID any more. Just as we depend on other
> - * devices never using PASIDs that they have no right
> - * to use. We have a *shared* PASID table, because it's
> - * large and has to be physically contiguous. So it's
> - * hard to be as defensive as we might like. */
> - intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
> - intel_flush_svm_range_dev(svm, sdev, 0, -1, 0, !svm->mm);
> - kfree_rcu(sdev, rcu);
> -
> - if (list_empty(&svm->devs)) {
> - ioasid_free(svm->pasid);
> - if (svm->mm)
> - mmu_notifier_unregister(&svm->notifier, svm->mm);
> -
> - list_del(&svm->list);
> -
> - /* We mandate that no page faults may be outstanding
> - * for the PASID when intel_svm_unbind_mm() is called.
> - * If that is not obeyed, subtle errors will happen.
> - * Let's make them less subtle... */
> - memset(svm, 0x6b, sizeof(*svm));
> - kfree(svm);
> - }
> + list_del(&svm->list);
> +
> + /* We mandate that no page faults may be outstanding
> + * for the PASID when intel_svm_unbind_mm() is called.
> + * If that is not obeyed, subtle errors will happen.
> + * Let's make them less subtle... */
> + memset(svm, 0x6b, sizeof(*svm));
> + kfree(svm);
> }
> - break;
> }
> + break;
> }
> out:
> mutex_unlock(&pasid_mutex);
> @@ -585,7 +582,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)
> * to unbind the mm while any page faults are outstanding.
> * So we only need RCU to protect the internal idr code. */
> rcu_read_unlock();
> - if (IS_ERR(svm) || !svm) {
> + if (IS_ERR_OR_NULL(svm)) {
> pr_err("%s: Page request for invalid PASID %d: %08llx %08llx\n",
> iommu->name, req->pasid, ((unsigned long long *)req)[0],
> ((unsigned long long *)req)[1]);
> --
> 2.7.4
>
--
With Best Regards,
Andy Shevchenko
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM
2019-08-15 21:17 ` Andy Shevchenko
@ 2019-08-15 22:50 ` Jacob Pan
0 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 22:50 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, iommu, LKML,
Alex Williamson, David Woodhouse, Jonathan Cameron
On Fri, 16 Aug 2019 00:17:44 +0300
Andy Shevchenko <andy.shevchenko@gmail.com> wrote:
> On Thu, Aug 15, 2019 at 11:52 PM Jacob Pan
> <jacob.jun.pan@linux.intel.com> wrote:
> >
> > Use combined macros for_each_svm_dev() to simplify SVM device
> > iteration and error checking.
> >
> > Suggested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> > Reviewed-by: Eric Auger <eric.auger@redhat.com>
> > ---
> > drivers/iommu/intel-svm.c | 85
> > +++++++++++++++++++++++------------------------ 1 file changed, 41
> > insertions(+), 44 deletions(-)
> >
> > diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
> > index 5a688a5..ea6f2e2 100644
> > --- a/drivers/iommu/intel-svm.c
> > +++ b/drivers/iommu/intel-svm.c
> > @@ -218,6 +218,10 @@ static const struct mmu_notifier_ops
> > intel_mmuops = { static DEFINE_MUTEX(pasid_mutex);
> > static LIST_HEAD(global_svm_list);
> >
> > +#define for_each_svm_dev(svm, dev) \
> > + list_for_each_entry(sdev, &svm->devs, list) \
>
> > + if (dev == sdev->dev) \
>
> This should be
> if (dev != sdev->dev) {} else
> and no trailing \ is neeeded.
>
> The rationale of above form to avoid
> for_each_foo() {
> } else {
> ...WTF?!..
> }
>
I understand, but until we have the else {} case we don't have anything
to avoid. The current code only has a simple positive logic.
> > +
> > int intel_svm_bind_mm(struct device *dev, int *pasid, int flags,
> > struct svm_dev_ops *ops) {
> > struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
> > @@ -263,15 +267,13 @@ int intel_svm_bind_mm(struct device *dev, int
> > *pasid, int flags, struct svm_dev_ goto out;
> > }
> >
> > - list_for_each_entry(sdev, &svm->devs, list)
> > {
> > - if (dev == sdev->dev) {
> > - if (sdev->ops != ops) {
> > - ret = -EBUSY;
> > - goto out;
> > - }
> > - sdev->users++;
> > - goto success;
> > + for_each_svm_dev(svm, dev) {
> > + if (sdev->ops != ops) {
> > + ret = -EBUSY;
> > + goto out;
> > }
> > + sdev->users++;
> > + goto success;
> > }
> >
> > break;
> > @@ -408,48 +410,43 @@ int intel_svm_unbind_mm(struct device *dev,
> > int pasid) goto out;
> >
> > svm = ioasid_find(NULL, pasid, NULL);
> > - if (IS_ERR(svm)) {
> > + if (IS_ERR_OR_NULL(svm)) {
> > ret = PTR_ERR(svm);
> > goto out;
> > }
> >
> > - if (!svm)
> > - goto out;
> > + for_each_svm_dev(svm, dev) {
> > + ret = 0;
> > + sdev->users--;
> > + if (!sdev->users) {
> > + list_del_rcu(&sdev->list);
> > + /* Flush the PASID cache and IOTLB for this
> > device.
> > + * Note that we do depend on the hardware
> > *not* using
> > + * the PASID any more. Just as we depend on
> > other
> > + * devices never using PASIDs that they
> > have no right
> > + * to use. We have a *shared* PASID table,
> > because it's
> > + * large and has to be physically
> > contiguous. So it's
> > + * hard to be as defensive as we might
> > like. */
> > + intel_pasid_tear_down_entry(iommu, dev,
> > svm->pasid);
> > + intel_flush_svm_range_dev(svm, sdev, 0, -1,
> > 0, !svm->mm);
> > + kfree_rcu(sdev, rcu);
> > +
> > + if (list_empty(&svm->devs)) {
> > + ioasid_free(svm->pasid);
> > + if (svm->mm)
> > +
> > mmu_notifier_unregister(&svm->notifier, svm->mm);
> >
> > - list_for_each_entry(sdev, &svm->devs, list) {
> > - if (dev == sdev->dev) {
> > - ret = 0;
> > - sdev->users--;
> > - if (!sdev->users) {
> > - list_del_rcu(&sdev->list);
> > - /* Flush the PASID cache and IOTLB
> > for this device.
> > - * Note that we do depend on the
> > hardware *not* using
> > - * the PASID any more. Just as we
> > depend on other
> > - * devices never using PASIDs that
> > they have no right
> > - * to use. We have a *shared* PASID
> > table, because it's
> > - * large and has to be physically
> > contiguous. So it's
> > - * hard to be as defensive as we
> > might like. */
> > - intel_pasid_tear_down_entry(iommu,
> > dev, svm->pasid);
> > - intel_flush_svm_range_dev(svm,
> > sdev, 0, -1, 0, !svm->mm);
> > - kfree_rcu(sdev, rcu);
> > -
> > - if (list_empty(&svm->devs)) {
> > - ioasid_free(svm->pasid);
> > - if (svm->mm)
> > -
> > mmu_notifier_unregister(&svm->notifier, svm->mm); -
> > - list_del(&svm->list);
> > -
> > - /* We mandate that no page
> > faults may be outstanding
> > - * for the PASID when
> > intel_svm_unbind_mm() is called.
> > - * If that is not obeyed,
> > subtle errors will happen.
> > - * Let's make them less
> > subtle... */
> > - memset(svm, 0x6b,
> > sizeof(*svm));
> > - kfree(svm);
> > - }
> > + list_del(&svm->list);
> > +
> > + /* We mandate that no page faults
> > may be outstanding
> > + * for the PASID when
> > intel_svm_unbind_mm() is called.
> > + * If that is not obeyed, subtle
> > errors will happen.
> > + * Let's make them less subtle... */
> > + memset(svm, 0x6b, sizeof(*svm));
> > + kfree(svm);
> > }
> > - break;
> > }
> > + break;
> > }
> > out:
> > mutex_unlock(&pasid_mutex);
> > @@ -585,7 +582,7 @@ static irqreturn_t prq_event_thread(int irq,
> > void *d)
> > * to unbind the mm while any page faults
> > are outstanding.
> > * So we only need RCU to protect the
> > internal idr code. */ rcu_read_unlock();
> > - if (IS_ERR(svm) || !svm) {
> > + if (IS_ERR_OR_NULL(svm)) {
> > pr_err("%s: Page request for
> > invalid PASID %d: %08llx %08llx\n", iommu->name, req->pasid,
> > ((unsigned long long *)req)[0], ((unsigned long long *)req)[1]);
> > --
> > 2.7.4
> >
>
>
[Jacob Pan]
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH v5 17/19] iommu/vt-d: Add bind guest PASID support
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (15 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 16/19] iommu/vt-d: Misc macro clean up for SVM Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 18/19] iommu/vt-d: Support flushing more translation cache types Jacob Pan
` (2 subsequent siblings)
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When supporting guest SVA with emulated IOMMU, the guest PASID
table is shadowed in VMM. Updates to guest vIOMMU PASID table
will result in PASID cache flush which will be passed down to
the host as bind guest PASID calls.
For the SL page tables, it will be harvested from device's
default domain (request w/o PASID), or aux domain in case of
mediated device.
.-------------. .---------------------------.
| vIOMMU | | Guest process CR3, FL only|
| | '---------------------------'
.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest
------| Shadow |--------------------------|--------
v v v
Host
.-------------. .----------------------.
| pIOMMU | | Bind FL for GVA-GPA |
| | '----------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.------------------------------.
| | |SL for GPA-HPA, default domain|
| | '------------------------------'
'-------------'
Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 4 +
drivers/iommu/intel-svm.c | 184 ++++++++++++++++++++++++++++++++++++++++++++
include/linux/intel-iommu.h | 8 +-
include/linux/intel-svm.h | 17 ++++
4 files changed, 212 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d2cc355..dcac964 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -5691,6 +5691,10 @@ const struct iommu_ops intel_iommu_ops = {
.dev_disable_feat = intel_iommu_dev_disable_feat,
.is_attach_deferred = intel_iommu_is_attach_deferred,
.pgsize_bitmap = INTEL_IOMMU_PGSIZES,
+#ifdef CONFIG_INTEL_IOMMU_SVM
+ .sva_bind_gpasid = intel_svm_bind_gpasid,
+ .sva_unbind_gpasid = intel_svm_unbind_gpasid,
+#endif
};
static void quirk_iommu_g4x_gfx(struct pci_dev *dev)
diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index ea6f2e2..c6edef2 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -222,6 +222,190 @@ static LIST_HEAD(global_svm_list);
list_for_each_entry(sdev, &svm->devs, list) \
if (dev == sdev->dev) \
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+ struct device *dev,
+ struct iommu_gpasid_bind_data *data)
+{
+ struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+ struct dmar_domain *ddomain;
+ struct intel_svm_dev *sdev;
+ struct intel_svm *svm;
+ int ret = 0;
+
+ if (WARN_ON(!iommu) || !data)
+ return -EINVAL;
+
+ if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+ data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+ return -EINVAL;
+
+ if (dev_is_pci(dev)) {
+ /* VT-d supports devices with full 20 bit PASIDs only */
+ if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+ return -EINVAL;
+ }
+
+ /*
+ * We only check host PASID range, we have no knowledge to check
+ * guest PASID range nor do we use the guest PASID.
+ */
+ if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+ return -EINVAL;
+
+ ddomain = to_dmar_domain(domain);
+ /* REVISIT:
+ * Sanity check adddress width and paging mode support
+ * width matching in two dimensions:
+ * 1. paging mode CPU <= IOMMU
+ * 2. address width Guest <= Host.
+ */
+ mutex_lock(&pasid_mutex);
+ svm = ioasid_find(NULL, data->hpasid, NULL);
+ if (IS_ERR(svm)) {
+ ret = PTR_ERR(svm);
+ goto out;
+ }
+ if (svm) {
+ /*
+ * If we found svm for the PASID, there must be at
+ * least one device bond, otherwise svm should be freed.
+ */
+ BUG_ON(list_empty(&svm->devs));
+
+ for_each_svm_dev(svm, dev) {
+ /* In case of multiple sub-devices of the same pdev assigned, we should
+ * allow multiple bind calls with the same PASID and pdev.
+ */
+ sdev->users++;
+ goto out;
+ }
+ } else {
+ /* We come here when PASID has never been bond to a device. */
+ svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+ if (!svm) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ /* REVISIT: upper layer/VFIO can track host process that bind the PASID.
+ * ioasid_set = mm might be sufficient for vfio to check pasid VMM
+ * ownership.
+ */
+ svm->mm = get_task_mm(current);
+ svm->pasid = data->hpasid;
+ if (data->flags & IOMMU_SVA_GPASID_VAL) {
+ svm->gpasid = data->gpasid;
+ svm->flags |= SVM_FLAG_GUEST_PASID;
+ }
+ ioasid_set_data(data->hpasid, svm);
+ INIT_LIST_HEAD_RCU(&svm->devs);
+ INIT_LIST_HEAD(&svm->list);
+
+ mmput(svm->mm);
+ }
+ sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+ if (!sdev) {
+ if (list_empty(&svm->devs))
+ kfree(svm);
+ ret = -ENOMEM;
+ goto out;
+ }
+ sdev->dev = dev;
+ sdev->users = 1;
+
+ /* Set up device context entry for PASID if not enabled already */
+ ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+ if (ret) {
+ dev_err(dev, "Failed to enable PASID capability\n");
+ kfree(sdev);
+ goto out;
+ }
+
+ /*
+ * For guest bind, we need to set up PASID table entry as follows:
+ * - FLPM matches guest paging mode
+ * - turn on nested mode
+ * - SL guest address width matching
+ */
+ ret = intel_pasid_setup_nested(iommu,
+ dev,
+ (pgd_t *)data->gpgd,
+ data->hpasid,
+ &data->vtd,
+ ddomain,
+ data->addr_width);
+ if (ret) {
+ dev_err(dev, "Failed to set up PASID %llu in nested mode, Err %d\n",
+ data->hpasid, ret);
+ kfree(sdev);
+ goto out;
+ }
+ svm->flags |= SVM_FLAG_GUEST_MODE;
+
+ init_rcu_head(&sdev->rcu);
+ list_add_rcu(&sdev->list, &svm->devs);
+ out:
+ mutex_unlock(&pasid_mutex);
+ return ret;
+}
+
+int intel_svm_unbind_gpasid(struct device *dev, int pasid)
+{
+ struct intel_svm_dev *sdev;
+ struct intel_iommu *iommu;
+ struct intel_svm *svm;
+ int ret = -EINVAL;
+
+ mutex_lock(&pasid_mutex);
+ iommu = intel_svm_device_to_iommu(dev);
+ if (!iommu)
+ goto out;
+
+ svm = ioasid_find(NULL, pasid, NULL);
+ if (IS_ERR_OR_NULL(svm)) {
+ ret = PTR_ERR(svm);
+ goto out;
+ }
+
+ for_each_svm_dev(svm, dev) {
+ ret = 0;
+ sdev->users--;
+ if (!sdev->users) {
+ list_del_rcu(&sdev->list);
+ intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
+ /* TODO: Drain in flight PRQ for the PASID since it
+ * may get reused soon, we don't want to
+ * confuse with its previous life.
+ * intel_svm_drain_prq(dev, pasid);
+ */
+ kfree_rcu(sdev, rcu);
+
+ if (list_empty(&svm->devs)) {
+ list_del(&svm->list);
+ kfree(svm);
+ /*
+ * We do not free PASID here until explicit call
+ * from VFIO to free. The PASID life cycle
+ * management is largely tied to VFIO management
+ * of assigned device life cycles. In case of
+ * guest exit without a explicit free PASID call,
+ * the responsibility lies in VFIO layer to free
+ * the PASIDs allocated for the guest.
+ * For security reasons, VFIO has to track the
+ * PASID ownership per guest anyway to ensure
+ * that PASID allocated by one guest cannot be
+ * used by another.
+ */
+ ioasid_set_data(pasid, NULL);
+ }
+ }
+ break;
+ }
+ out:
+ mutex_unlock(&pasid_mutex);
+
+ return ret;
+}
+
int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
{
struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index e1865f1..d673b39 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -670,7 +670,9 @@ int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev);
int intel_svm_init(struct intel_iommu *iommu);
extern int intel_svm_enable_prq(struct intel_iommu *iommu);
extern int intel_svm_finish_prq(struct intel_iommu *iommu);
-
+extern int intel_svm_bind_gpasid(struct iommu_domain *domain,
+ struct device *dev, struct iommu_gpasid_bind_data *data);
+extern int intel_svm_unbind_gpasid(struct device *dev, int pasid);
struct svm_dev_ops;
struct intel_svm_dev {
@@ -687,9 +689,13 @@ struct intel_svm_dev {
struct intel_svm {
struct mmu_notifier notifier;
struct mm_struct *mm;
+
struct intel_iommu *iommu;
int flags;
int pasid;
+ int gpasid; /* Guest PASID in case of vSVA bind with non-identity host
+ * to guest PASID mapping.
+ */
struct list_head devs;
struct list_head list;
};
diff --git a/include/linux/intel-svm.h b/include/linux/intel-svm.h
index 94f047a..a2c189a 100644
--- a/include/linux/intel-svm.h
+++ b/include/linux/intel-svm.h
@@ -44,6 +44,23 @@ struct svm_dev_ops {
* do such IOTLB flushes automatically.
*/
#define SVM_FLAG_SUPERVISOR_MODE (1<<1)
+/*
+ * The SVM_FLAG_GUEST_MODE flag is used when a guest process bind to a device.
+ * In this case the mm_struct is in the guest kernel or userspace, its life
+ * cycle is managed by VMM and VFIO layer. For IOMMU driver, this API provides
+ * means to bind/unbind guest CR3 with PASIDs allocated for a device.
+ */
+#define SVM_FLAG_GUEST_MODE (1<<2)
+/*
+ * The SVM_FLAG_GUEST_PASID flag is used when a guest has its own PASID space,
+ * which requires guest and host PASID translation at both directions. We keep
+ * track of guest PASID in order to provide lookup service to device drivers.
+ * One such example is a physical function (PF) driver that supports mediated
+ * device (mdev) assignment. Guest programming of mdev configuration space can
+ * only be done with guest PASID, therefore PF driver needs to find the matching
+ * host PASID to program the real hardware.
+ */
+#define SVM_FLAG_GUEST_PASID (1<<3)
#ifdef CONFIG_INTEL_IOMMU_SVM
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 18/19] iommu/vt-d: Support flushing more translation cache types
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (16 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 17/19] iommu/vt-d: Add bind guest PASID support Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-15 20:13 ` [PATCH v5 19/19] iommu/vt-d: Add svm/sva invalidate function Jacob Pan
2019-08-26 17:12 ` [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable
IOTLB invalidation may be passed down from outside IOMMU subsystems.
This patch adds invalidation functions that can be used for additional
translation cache types.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
---
drivers/iommu/dmar.c | 46 +++++++++++++++++++++++++++++++++++++++++++++
drivers/iommu/intel-pasid.c | 3 ++-
include/linux/intel-iommu.h | 21 +++++++++++++++++----
3 files changed, 65 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 5d0754e..1da4c68 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -1345,6 +1345,21 @@ void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
qi_submit_sync(&desc, iommu);
}
+/* PASID-based IOTLB Invalidate */
+void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u64 addr, u32 pasid,
+ unsigned int size_order, u64 granu, int ih)
+{
+ struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+ desc.qw0 = QI_EIOTLB_PASID(pasid) | QI_EIOTLB_DID(did) |
+ QI_EIOTLB_GRAN(granu) | QI_EIOTLB_TYPE;
+ desc.qw1 = QI_EIOTLB_ADDR(addr) | QI_EIOTLB_IH(ih) |
+ QI_EIOTLB_AM(size_order);
+ desc.qw2 = 0;
+ desc.qw3 = 0;
+ qi_submit_sync(&desc, iommu);
+}
+
void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
u16 qdep, u64 addr, unsigned mask)
{
@@ -1368,6 +1383,37 @@ void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
qi_submit_sync(&desc, iommu);
}
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_piotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+ u32 pasid, u16 qdep, u64 addr, unsigned size_order, u64 granu)
+{
+ struct qi_desc desc;
+ unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
+
+ desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
+ QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+ QI_DEV_IOTLB_PFSID(pfsid);
+ desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+ desc.qw1 |= addr & ~mask;
+ /* If S bit is 0, we only flush a single page. If S bit is set,
+ * The least significant zero bit indicates the invalidation address
+ * range. VT-d spec 6.5.2.6.
+ * e.g. address bit 12[0] indicates 8KB, 13[0] indicates 16KB.
+ */
+ if (size_order)
+ desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+ qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid)
+{
+ struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+ desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE;
+ qi_submit_sync(&desc, iommu);
+}
/*
* Disable Queued Invalidation interface.
*/
diff --git a/drivers/iommu/intel-pasid.c b/drivers/iommu/intel-pasid.c
index fd2c82f..ff7e877 100644
--- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c
@@ -518,7 +518,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
{
struct qi_desc desc;
- desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL | QI_PC_PASID(pasid);
+ desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+ QI_PC_PASID(pasid) | QI_PC_TYPE;
desc.qw1 = 0;
desc.qw2 = 0;
desc.qw3 = 0;
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index d673b39..682eafa1 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -327,7 +327,7 @@ enum {
#define QI_IOTLB_GRAN(gran) (((u64)gran) >> (DMA_TLB_FLUSH_GRANU_OFFSET-4))
#define QI_IOTLB_ADDR(addr) (((u64)addr) & VTD_PAGE_MASK)
#define QI_IOTLB_IH(ih) (((u64)ih) << 6)
-#define QI_IOTLB_AM(am) (((u8)am))
+#define QI_IOTLB_AM(am) (((u8)am) & 0x3f)
#define QI_CC_FM(fm) (((u64)fm) << 48)
#define QI_CC_SID(sid) (((u64)sid) << 32)
@@ -345,17 +345,22 @@ enum {
#define QI_PC_DID(did) (((u64)did) << 16)
#define QI_PC_GRAN(gran) (((u64)gran) << 4)
-#define QI_PC_ALL_PASIDS (QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL (QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS 0
+#define QI_PC_PASID_SEL 1
#define QI_EIOTLB_ADDR(addr) ((u64)(addr) & VTD_PAGE_MASK)
#define QI_EIOTLB_GL(gl) (((u64)gl) << 7)
#define QI_EIOTLB_IH(ih) (((u64)ih) << 6)
-#define QI_EIOTLB_AM(am) (((u64)am))
+#define QI_EIOTLB_AM(am) (((u64)am) & 0x3f)
#define QI_EIOTLB_PASID(pasid) (((u64)pasid) << 32)
#define QI_EIOTLB_DID(did) (((u64)did) << 16)
#define QI_EIOTLB_GRAN(gran) (((u64)gran) << 4)
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL 1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL 0
+
#define QI_DEV_EIOTLB_ADDR(a) ((u64)(a) & VTD_PAGE_MASK)
#define QI_DEV_EIOTLB_SIZE (((u64)1) << 11)
#define QI_DEV_EIOTLB_GLOB(g) ((u64)g)
@@ -652,8 +657,16 @@ extern void qi_flush_context(struct intel_iommu *iommu, u16 did, u16 sid,
u8 fm, u64 type);
extern void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
unsigned int size_order, u64 type);
+extern void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u64 addr,
+ u32 pasid, unsigned int size_order, u64 type, int ih);
extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
u16 qdep, u64 addr, unsigned mask);
+
+extern void qi_flush_dev_piotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+ u32 pasid, u16 qdep, u64 addr, unsigned size_order, u64 granu);
+
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid);
+
extern int qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
extern int dmar_ir_support(void);
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PATCH v5 19/19] iommu/vt-d: Add svm/sva invalidate function
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (17 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 18/19] iommu/vt-d: Support flushing more translation cache types Jacob Pan
@ 2019-08-15 20:13 ` Jacob Pan
2019-08-26 17:12 ` [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-15 20:13 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Tian, Kevin, Raj Ashok, Jean-Philippe Brucker, Jonathan Cameron
When Shared Virtual Address (SVA) is enabled for a guest OS via
vIOMMU, we need to provide invalidation support at IOMMU API and driver
level. This patch adds Intel VT-d specific function to implement
iommu passdown invalidate API for shared virtual address.
The use case is for supporting caching structure invalidation
of assigned SVM capable devices. Emulated IOMMU exposes queue
invalidation capability and passes down all descriptors from the guest
to the physical IOMMU.
The assumption is that guest to host device ID mapping should be
resolved prior to calling IOMMU driver. Based on the device handle,
host IOMMU driver can replace certain fields before submit to the
invalidation queue.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 170 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 170 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index dcac964..b7ca33a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -5169,6 +5169,175 @@ static void intel_iommu_aux_detach_device(struct iommu_domain *domain,
aux_domain_remove_dev(to_dmar_domain(domain), dev);
}
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned device where guest
+ * could own the first level page tables without being shadowed by QEMU. In
+ * this case there is no pass down unmap to the host IOMMU as a result of unmap
+ * in the guest. Only invalidations are trapped and passed down.
+ * In all cases, only first level TLB invalidation (request with PASID) can be
+ * passed down, therefore we do not include IOTLB granularity for request
+ * without PASID (second level).
+ *
+ * For an example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+ /* PASID based IOTLB, support PASID selective and page selective */
+ {0, 1, 1},
+ /* PASID based dev TLBs, only support all PASIDs or single PASID */
+ {1, 1, 0},
+ /* PASID cache */
+ {1, 1, 0}
+};
+
+const static u64 inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+ /* PASID based IOTLB */
+ {0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+ /* PASID based dev TLBs */
+ {QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+ /* PASID cache */
+ {QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, u64 *vtd_granu)
+{
+ if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >= IOMMU_INV_GRANU_NR ||
+ !inv_type_granu_map[type][granu])
+ return -EINVAL;
+
+ *vtd_granu = inv_type_granu_table[type][granu];
+
+ return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+ u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+ /* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+ * IOMMU cache invalidate API passes granu_size in bytes, and number of
+ * granu size in contiguous memory.
+ */
+ return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+ struct device *dev, struct iommu_cache_invalidate_info *inv_info)
+{
+ struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+ struct device_domain_info *info;
+ struct intel_iommu *iommu;
+ unsigned long flags;
+ int cache_type;
+ u8 bus, devfn;
+ u16 did, sid;
+ int ret = 0;
+ u64 size;
+
+ if (!inv_info || !dmar_domain ||
+ inv_info->version != IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)
+ return -EINVAL;
+
+ if (!dev || !dev_is_pci(dev))
+ return -ENODEV;
+
+ iommu = device_to_iommu(dev, &bus, &devfn);
+ if (!iommu)
+ return -ENODEV;
+
+ spin_lock_irqsave(&device_domain_lock, flags);
+ spin_lock(&iommu->lock);
+ info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+ if (!info) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ did = dmar_domain->iommu_did[iommu->seq_id];
+ sid = PCI_DEVID(bus, devfn);
+ size = to_vtd_size(inv_info->addr_info.granule_size, inv_info->addr_info.nb_granules);
+
+ for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {
+ u64 granu = 0;
+ u64 pasid = 0;
+
+ ret = to_vtd_granularity(cache_type, inv_info->granularity, &granu);
+ if (ret) {
+ pr_err("Invalid cache type and granu combination %d/%d\n", cache_type,
+ inv_info->granularity);
+ break;
+ }
+
+ /* PASID is stored in different locations based on granularity */
+ if (inv_info->granularity == IOMMU_INV_GRANU_PASID)
+ pasid = inv_info->pasid_info.pasid;
+ else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR)
+ pasid = inv_info->addr_info.pasid;
+ else {
+ pr_err("Cannot find PASID for given cache type and granularity\n");
+ break;
+ }
+
+ switch (BIT(cache_type)) {
+ case IOMMU_CACHE_INV_TYPE_IOTLB:
+ if (size && (inv_info->addr_info.addr & ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {
+ pr_err("Address out of range, 0x%llx, size order %llu\n",
+ inv_info->addr_info.addr, size);
+ ret = -ERANGE;
+ goto out_unlock;
+ }
+
+ qi_flush_piotlb(iommu, did, mm_to_dma_pfn(inv_info->addr_info.addr),
+ pasid, size, granu, inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_LEAF);
+
+ /*
+ * Always flush device IOTLB if ATS is enabled since guest
+ * vIOMMU exposes CM = 1, no device IOTLB flush will be passed
+ * down.
+ */
+ if (info->ats_enabled) {
+ qi_flush_dev_piotlb(iommu, sid, info->pfsid,
+ pasid, info->ats_qdep,
+ inv_info->addr_info.addr, size,
+ granu);
+ }
+ break;
+ case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+ if (info->ats_enabled) {
+ qi_flush_dev_piotlb(iommu, sid, info->pfsid,
+ inv_info->addr_info.pasid, info->ats_qdep,
+ inv_info->addr_info.addr, size,
+ granu);
+ } else
+ pr_warn("Passdown device IOTLB flush w/o ATS!\n");
+
+ break;
+ case IOMMU_CACHE_INV_TYPE_PASID:
+ qi_flush_pasid_cache(iommu, did, granu, inv_info->pasid_info.pasid);
+
+ break;
+ default:
+ dev_err(dev, "Unsupported IOMMU invalidation type %d\n",
+ cache_type);
+ ret = -EINVAL;
+ }
+ }
+out_unlock:
+ spin_unlock(&iommu->lock);
+ spin_unlock_irqrestore(&device_domain_lock, flags);
+
+ return ret;
+}
+#endif
+
static int intel_iommu_map(struct iommu_domain *domain,
unsigned long iova, phys_addr_t hpa,
size_t size, int iommu_prot)
@@ -5692,6 +5861,7 @@ const struct iommu_ops intel_iommu_ops = {
.is_attach_deferred = intel_iommu_is_attach_deferred,
.pgsize_bitmap = INTEL_IOMMU_PGSIZES,
#ifdef CONFIG_INTEL_IOMMU_SVM
+ .cache_invalidate = intel_iommu_sva_invalidate,
.sva_bind_gpasid = intel_svm_bind_gpasid,
.sva_unbind_gpasid = intel_svm_unbind_gpasid,
#endif
--
2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 23+ messages in thread* Re: [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support
2019-08-15 20:13 [PATCH v5 00/19] Shared virtual address IOMMU and VT-d support Jacob Pan
` (18 preceding siblings ...)
2019-08-15 20:13 ` [PATCH v5 19/19] iommu/vt-d: Add svm/sva invalidate function Jacob Pan
@ 2019-08-26 17:12 ` Jacob Pan
19 siblings, 0 replies; 23+ messages in thread
From: Jacob Pan @ 2019-08-26 17:12 UTC (permalink / raw)
To: iommu, LKML, Joerg Roedel, David Woodhouse, Alex Williamson
Cc: Jean-Philippe Brucker, Tian, Kevin, Raj Ashok, Jonathan Cameron
Hi Joerg and all,
Just wondering if you have more comments for this version. Since
the complete vSVA patchset is quite large, my hope is that we can make
incremental progress with smaller sets.
My plans are:
1. vSVA IOMMU APIs and VT-d support of PCI devices (this patchset)
2. Page request service
3. mdev support
4. code merge with Jean's generic sva_bind() APIs
For this set, we don't need PRS related patches (1-4). I included
them because they are in Jean's common API repo. Could you review 5-19
and consider for v5.4? This will make further submissions much easier.
Thanks,
Jacob
On Thu, 15 Aug 2019 13:13:06 -0700
Jacob Pan <jacob.jun.pan@linux.intel.com> wrote:
> Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on
> Intel platforms allow address space sharing between device DMA and
> applications. SVA can reduce programming complexity and enhance
> security. This series is intended to enable SVA virtualization, i.e.
> shared guest application address space and physical device DMA
> address. Only IOMMU portion of the changes are included in this
> series. Additional support is needed in VFIO and QEMU (will be
> submitted separately) to complete this functionality.
>
> To make incremental changes and reduce the size of each patchset.
> This series does not inlcude support for page request services.
>
> In VT-d implementation, PASID table is per device and maintained in
> the host. Guest PASID table is shadowed in VMM where virtual IOMMU is
> emulated.
>
> .-------------. .---------------------------.
> | vIOMMU | | Guest process CR3, FL only|
> | | '---------------------------'
> .----------------/
> | PASID Entry |--- PASID cache flush -
> '-------------' |
> | | V
> | | CR3 in GPA
> '-------------'
> Guest
> ------| Shadow |--------------------------|--------
> v v v
> Host
> .-------------. .----------------------.
> | pIOMMU | | Bind FL for GVA-GPA |
> | | '----------------------'
> .----------------/ |
> | PASID Entry | V (Nested xlate)
> '----------------\.------------------------------.
> | | |SL for GPA-HPA, default domain|
> | | '------------------------------'
> '-------------'
> Where:
> - FL = First level/stage one page tables
> - SL = Second level/stage two page tables
>
>
> This work is based on collaboration with other developers on the IOMMU
> mailing list. Notably,
>
> [1] Common APIs git://linux-arm.org/linux-jpb.git sva/api
>
> [2] [RFC PATCH 2/6] drivers core: Add I/O ASID allocator by
> Jean-Philippe Brucker
> https://www.spinics.net/lists/iommu/msg30639.html
>
> [3] [RFC PATCH 0/5] iommu: APIs for paravirtual PASID allocation by
> Lu Baolu https://lkml.org/lkml/2018/11/12/1921
>
> [4] [PATCH v5 00/23] IOMMU and VT-d driver support for Shared Virtual
> Address (SVA)
> https://lwn.net/Articles/754331/
>
> There are roughly three parts:
> 1. Generic PASID allocator [1] with extension to support custom
> allocator 2. IOMMU cache invalidation passdown from guest to host
> 3. Guest PASID bind for nested translation
>
> All generic IOMMU APIs are reused from [1] with minor tweaks. With
> this patchset, guest SVA without page request works on VT-d. PRS
> patches will come next as we try to avoid large patchset that is hard
> to review. The patches for basic SVA support (w/o PRS) starts:
> [PATCH v5 05/19] iommu: Introduce attach/detach_pasid_table API
>
> It is worth noting that unlike sMMU nested stage setup, where PASID
> table is owned by the guest, VT-d PASID table is owned by the host,
> individual PASIDs are bound instead of the PASID table.
>
> This series is based on the new VT-d 3.0 Specification
> (https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf).
> This is different than the older series in [4] which was based on the
> older specification that does not have scalable mode.
>
>
> ChangeLog:
> - V5
> Rebased on v5.3-rc4 which has some of the IOMMU fault APIs
> merged. Addressed v4 review comments from Eric Auger, Baolu Lu, and
> Jonathan Cameron. Specific changes are as follows:
> - Refined custom IOASID allocator to support multiple
> vIOMMU, hotplug cases.
> - Extracted vendor data from IOMMU guest PASID bind data,
> for VT-d will support all necessary guest PASID entry fields for PASID
> bind.
> - Support non-identity host-guest PASID mapping
> - Exception handling in various cases
>
> - V4
> - Redesigned IOASID allocator such that it can support
> custom allocators with shared helper functions. Use separate XArray
> to store IOASIDs per allocator. Took advice from Eric Auger
> to have default allocator use the generic allocator structure.
> Combined into one patch in that the default allocator is
> just "another" allocator now. Can be built as a module in case of
> driver use without IOMMU.
> - Extended bind guest PASID data to support SMMU and
> non-identity guest to host PASID mapping
> https://lkml.org/lkml/2019/5/21/802
> - Rebased on Jean's sva/api common tree, new patches starts
> with [PATCH v4 10/22]
>
> - V3
> - Addressed thorough review comments from Eric Auger (Thank
> you!)
> - Moved IOASID allocator from driver core to IOMMU code per
> suggestion by Christoph Hellwig
> (https://lkml.org/lkml/2019/4/26/462)
> - Rebased on top of Jean's SVA API branch and Eric's v7[1]
> (git://linux-arm.org/linux-jpb.git sva/api)
> - All IOMMU APIs are unmodified (except the new bind guest
> PASID call in patch 9/16)
>
> - V2
> - Rebased on Joerg's IOMMU x86/vt-d branch v5.1-rc4
> - Integrated with Eric Auger's new v7 series for common APIs
> (https://github.com/eauger/linux/tree/v5.1-rc3-2stage-v7)
> - Addressed review comments from Andy Shevchenko and Alex
> Williamson on IOASID custom allocator.
> - Support multiple custom IOASID allocators (vIOMMUs) and
> dynamic registration.
>
>
> Jacob Pan (14):
> iommu: Add a timeout parameter for PRQ response
> iommu: handle page response timeout
> iommu: Introduce attach/detach_pasid_table API
> iommu/ioasid: Add custom allocators
> iommu: Introduce guest PASID bind function
> iommu/vt-d: Add custom allocator for IOASID
> iommu/vt-d: Replace Intel specific PASID allocator with IOASID
> iommu/vt-d: Move domain helper to header
> iommu/vt-d: Avoid duplicated code for PASID setup
> iommu/vt-d: Add nested translation helper function
> iommu/vt-d: Misc macro clean up for SVM
> iommu/vt-d: Add bind guest PASID support
> iommu/vt-d: Support flushing more translation cache types
> iommu/vt-d: Add svm/sva invalidate function
>
> Jean-Philippe Brucker (3):
> trace/iommu: Add sva trace events
> iommu: Use device fault trace event
> iommu: Add I/O ASID allocator
>
> Lu Baolu (1):
> iommu/vt-d: Enlightened PASID allocation
>
> Yi L Liu (1):
> iommu: Introduce cache_invalidate API
>
> Documentation/admin-guide/kernel-parameters.txt | 8 +
> drivers/iommu/Kconfig | 5 +
> drivers/iommu/Makefile | 1 +
> drivers/iommu/dmar.c | 46 +++
> drivers/iommu/intel-iommu.c | 259 +++++++++++++-
> drivers/iommu/intel-pasid.c | 343
> ++++++++++++++++--- drivers/iommu/intel-pasid.h
> | 25 +- drivers/iommu/intel-svm.c | 298
> +++++++++++++--- drivers/iommu/ioasid.c |
> 433 ++++++++++++++++++++++++
> drivers/iommu/iommu.c | 139 ++++++++
> include/linux/intel-iommu.h | 39 ++-
> include/linux/intel-svm.h | 17 +
> include/linux/ioasid.h | 75 ++++
> include/linux/iommu.h | 58 ++++
> include/trace/events/iommu.h | 84 +++++
> include/uapi/linux/iommu.h | 219 ++++++++++++ 16
> files changed, 1925 insertions(+), 124 deletions(-) create mode
> 100644 drivers/iommu/ioasid.c create mode 100644
> include/linux/ioasid.h
>
[Jacob Pan]
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 23+ messages in thread