[PATCH RFC 0/4] iommu: Add support to change default domain of a group

[PATCH RFC 0/4] iommu: Add support to change default domain of a group

[Sai Praneeth Prakhya]

Presently, the default domain of a group is allocated during boot time and it
cannot be changed later. So, the device would typically be either in
identity/pass through mode or the device would be in DMA mode as long as the
system is up and running. There is no way to change the default domain type
dynamically i.e. after booting, a device cannot switch between identity mode and
DMA mode.

Assume a use case where-in the priviliged user would want to use the device in
pass-through mode when the device is used for host but would want to switch to
dma protected mode when switching for VFIO in user space. Presently, this is not
supported and hence add support to change default domain of a group dynamically.

Support this through a sysfs file, namely "/sys/kernel/iommu_groups/<grp_id>/type".

Testing:
--------
Tested by dynamically changing USB device from identity mode to dma and from dma
to identity. Only for x86_64 (i.e. intel_iommu/vt-d). Haven't tested for other
architectures.

Sai Praneeth Prakhya (4):
  iommu/vt-d: Modify device_def_domain_type() to use at runtime
  iommu: Add device_def_domain_type() call back function to iommu_ops
  iommu: Add support to change default domain of a group
  iommu: Document usage of "/sys/kernel/iommu_groups/<grp_id>/type" file

 .../ABI/testing/sysfs-kernel-iommu_groups     |  34 ++++
 drivers/iommu/intel-iommu.c                   |  32 +++-
 drivers/iommu/iommu.c                         | 178 +++++++++++++++++-
 include/linux/iommu.h                         |   3 +
 4 files changed, 237 insertions(+), 10 deletions(-)

Cc: Christoph Hellwig <hch@lst.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>

-- 
2.19.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH RFC 1/4] iommu/vt-d: Modify device_def_domain_type() to use at runtime

[Sai Praneeth Prakhya]

device_def_domain_type() determines the domain type a device could have and
it's called only during boot. But, to change the domain of a group through
sysfs, kernel has to call this function during runtime. Hence, add an
argument to the function which lets the function know if it's being called
at boot time or runtime.

Normally, device_def_domain_type() considers the value of command line
arguments like "intel_iommu=igfx_off" or "iommu=pt" to determine the domain
type of a device, but at runtime, user request should take higher priority
over command line arguments. Hence, use 'startup' argument to modify
device_def_domain_type() such that command line argument settings apply only at
boot time.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>
---
 drivers/iommu/intel-iommu.c | 29 ++++++++++++++++++++---------
 1 file changed, 20 insertions(+), 9 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index ac4172c02244..54e82415e396 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2862,15 +2862,19 @@ static bool device_is_rmrr_locked(struct device *dev)
 /*
  * Return the required default domain type for a specific device.
  *
- * @dev: the device in query
- * @startup: true if this is during early boot
+ * @dev:	The device in query
+ * @startup:	Should command line arguments (E.g: igfx_off or iommu=pt) be
+ *		considered to decide the domain type of a device? Which is
+ *		'true' while booting but 'false' if the user requests kernel to
+ *		change the domain type by writing to
+ *		/sys/kernel/iommu_groups/<group_id>/type.
  *
  * Returns:
  *  - IOMMU_DOMAIN_DMA: device requires a dynamic mapping domain
  *  - IOMMU_DOMAIN_IDENTITY: device requires an identical mapping domain
  *  - 0: both identity and dynamic domains work for this device
  */
-static int device_def_domain_type(struct device *dev)
+static int device_def_domain_type(struct device *dev, bool startup)
 {
 	if (dev_is_pci(dev)) {
 		struct pci_dev *pdev = to_pci_dev(dev);
@@ -2888,8 +2892,11 @@ static int device_def_domain_type(struct device *dev)
 		if ((iommu_identity_mapping & IDENTMAP_AZALIA) && IS_AZALIA(pdev))
 			return IOMMU_DOMAIN_IDENTITY;
 
-		if ((iommu_identity_mapping & IDENTMAP_GFX) && IS_GFX_DEVICE(pdev))
-			return IOMMU_DOMAIN_IDENTITY;
+		if (startup) {
+			if ((iommu_identity_mapping & IDENTMAP_GFX) &&
+			    IS_GFX_DEVICE(pdev))
+				return IOMMU_DOMAIN_IDENTITY;
+		}
 
 		/*
 		 * We want to start off with all devices in the 1:1 domain, and
@@ -2920,8 +2927,12 @@ static int device_def_domain_type(struct device *dev)
 			return IOMMU_DOMAIN_DMA;
 	}
 
-	return (iommu_identity_mapping & IDENTMAP_ALL) ?
-			IOMMU_DOMAIN_IDENTITY : 0;
+	if (startup) {
+		if (iommu_identity_mapping & IDENTMAP_ALL)
+			return IOMMU_DOMAIN_IDENTITY;
+	}
+
+	return 0;
 }
 
 static void intel_iommu_init_qi(struct intel_iommu *iommu)
@@ -5275,7 +5286,7 @@ static int intel_iommu_add_device(struct device *dev)
 	domain = iommu_get_domain_for_dev(dev);
 	dmar_domain = to_dmar_domain(domain);
 	if (domain->type == IOMMU_DOMAIN_DMA) {
-		if (device_def_domain_type(dev) == IOMMU_DOMAIN_IDENTITY) {
+		if (device_def_domain_type(dev, true) == IOMMU_DOMAIN_IDENTITY) {
 			ret = iommu_request_dm_for_dev(dev);
 			if (ret) {
 				dmar_domain->flags |= DOMAIN_FLAG_LOSE_CHILDREN;
@@ -5285,7 +5296,7 @@ static int intel_iommu_add_device(struct device *dev)
 			}
 		}
 	} else {
-		if (device_def_domain_type(dev) == IOMMU_DOMAIN_DMA) {
+		if (device_def_domain_type(dev, true) == IOMMU_DOMAIN_DMA) {
 			ret = iommu_request_dma_domain_for_dev(dev);
 			if (ret) {
 				dmar_domain->flags |= DOMAIN_FLAG_LOSE_CHILDREN;
-- 
2.19.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH RFC 1/4] iommu/vt-d: Modify device_def_domain_type() to use at runtime

[Joerg Roedel]

On Tue, Jul 02, 2019 at 06:53:59PM -0700, Sai Praneeth Prakhya wrote:
> device_def_domain_type() determines the domain type a device could have and
> it's called only during boot. But, to change the domain of a group through
> sysfs, kernel has to call this function during runtime. Hence, add an
> argument to the function which lets the function know if it's being called
> at boot time or runtime.

I don't think it should make a difference when the function is actually
called. The sysfs input is just another variable to take into account
when the default domain type is determined.

What I'd like to see for example is that I can write 'auto' to the file
and get back the systems decision for the default domain type. I'd also
like to be able to forbid changing the type for e.g.  Thunderbolt
connected devices.


Regards,

	Joerg
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH RFC 1/4] iommu/vt-d: Modify device_def_domain_type() to use at runtime

[Prakhya, Sai Praneeth]

> On Tue, Jul 02, 2019 at 06:53:59PM -0700, Sai Praneeth Prakhya wrote:
> > device_def_domain_type() determines the domain type a device could
> > have and it's called only during boot. But, to change the domain of a
> > group through sysfs, kernel has to call this function during runtime.
> > Hence, add an argument to the function which lets the function know if
> > it's being called at boot time or runtime.
> 
> I don't think it should make a difference when the function is actually called. The
> sysfs input is just another variable to take into account when the default domain
> type is determined.

Sure! Makes sense. I will modify the code accordingly.

> What I'd like to see for example is that I can write 'auto' to the file and get back
> the systems decision for the default domain type.

Sure! Sounds good to me. Will add this functionality.

> I'd also like to be able to forbid changing the type for e.g.  Thunderbolt connected devices.

This is presently supported in the patch set. But, I got your point, will make sure that 
untrusted devices are not allowed to change the group.

Regards,
Sai
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH RFC 2/4] iommu: Add device_def_domain_type() call back function to iommu_ops

[Sai Praneeth Prakhya]

When user requests kernel to change the default domain type of a group
through sysfs, kernel has to make sure if it's ok to change the domain type
of every device in the group to the requested domain (every device may not
support both the domain types i.e. DMA and identity). Hence, add a call
back function that could be implemented per architecture that performs the
above check.

For intel_iommu, this is already done by device_def_domain_type(), but
every call back function in intel_iommu_ops starts with intel_iommu prefix,
hence modify device_def_domain_type() so that it follows the same semantics.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>
---
 drivers/iommu/intel-iommu.c | 9 ++++++---
 include/linux/iommu.h       | 3 +++
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 54e82415e396..0c991eb5034a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2874,7 +2874,7 @@ static bool device_is_rmrr_locked(struct device *dev)
  *  - IOMMU_DOMAIN_IDENTITY: device requires an identical mapping domain
  *  - 0: both identity and dynamic domains work for this device
  */
-static int device_def_domain_type(struct device *dev, bool startup)
+static int intel_iommu_device_def_domain_type(struct device *dev, bool startup)
 {
 	if (dev_is_pci(dev)) {
 		struct pci_dev *pdev = to_pci_dev(dev);
@@ -5286,7 +5286,8 @@ static int intel_iommu_add_device(struct device *dev)
 	domain = iommu_get_domain_for_dev(dev);
 	dmar_domain = to_dmar_domain(domain);
 	if (domain->type == IOMMU_DOMAIN_DMA) {
-		if (device_def_domain_type(dev, true) == IOMMU_DOMAIN_IDENTITY) {
+		if (intel_iommu_device_def_domain_type(dev, true) ==
+		    IOMMU_DOMAIN_IDENTITY) {
 			ret = iommu_request_dm_for_dev(dev);
 			if (ret) {
 				dmar_domain->flags |= DOMAIN_FLAG_LOSE_CHILDREN;
@@ -5296,7 +5297,8 @@ static int intel_iommu_add_device(struct device *dev)
 			}
 		}
 	} else {
-		if (device_def_domain_type(dev, true) == IOMMU_DOMAIN_DMA) {
+		if (intel_iommu_device_def_domain_type(dev, true) ==
+		    IOMMU_DOMAIN_DMA) {
 			ret = iommu_request_dma_domain_for_dev(dev);
 			if (ret) {
 				dmar_domain->flags |= DOMAIN_FLAG_LOSE_CHILDREN;
@@ -5637,6 +5639,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_enable_feat	= intel_iommu_dev_enable_feat,
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
+	.device_def_domain_type	= intel_iommu_device_def_domain_type,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 };
 
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index fdc355ccc570..2d172c02be05 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -228,6 +228,7 @@ struct iommu_sva_ops {
  * @sva_unbind: Unbind process address space from device
  * @sva_get_pasid: Get PASID associated to a SVA handle
  * @page_response: handle page request response
+ * @device_def_domain_type: Return the required default domain type for a device
  * @pgsize_bitmap: bitmap of all possible supported page sizes
  */
 struct iommu_ops {
@@ -292,6 +293,8 @@ struct iommu_ops {
 			     struct iommu_fault_event *evt,
 			     struct iommu_page_response *msg);
 
+	int (*device_def_domain_type)(struct device *dev, bool startup);
+
 	unsigned long pgsize_bitmap;
 };
 
-- 
2.19.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH RFC 3/4] iommu: Add support to change default domain of a group

[Sai Praneeth Prakhya]

Presently, the default domain of a group is allocated during boot time
(i.e. when a device is being added to a group) and it cannot be changed
later. So, the device would typically be either in identity/pass_through
mode (controlled by "iommu=pt" kernel command line argument) or the device
would be in DMA mode (as long as the machine is up and running). There is
no way to change the default domain type dynamically i.e. after booting, a
device cannot switch between identity mode and DMA mode.

But, assume a use case where in there is an SR-IOV device and if the
privileged user decides to use some VF's natively (i.e. they are available
only to host) and he wants them to be high performing and also believes
that his OS is secure enough. In this scenario, some VF's should bypass
IOMMU. Presently, this is not supported and hence it will be helpful if
there is some way to change the default domain of a B:D.F dynamically.
Since, linux iommu subsystem prefers to deal at group level instead of
B:D.F level, it might be helpful if there is some way to change the default
domain of a *group* dynamically. Hence, add such support.

A privileged user could request the kernel to change the default domain
type of a group by writing to "/sys/kernel/iommu_groups/<grp_id>/type"
file. Presently, only two values are supported "identity" and "dma".
"identity" means that all the DMA transactions from the devices in this
group are *not* translated by the iommu and where as "dma" means that all
the DMA transactions from the devices in this group are translated by the
iommu. Also please note that a group type could be modified only when *all*
the devices in the group are not binded to any device driver.

Please see "Documentation/ABI/testing/sysfs-kernel-iommu_groups" for more
information.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>
---
 drivers/iommu/iommu.c | 178 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 177 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 0c674d80c37f..92fadbea36b1 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -155,6 +155,8 @@ static int __iommu_attach_group(struct iommu_domain *domain,
 				struct iommu_group *group);
 static void __iommu_detach_group(struct iommu_domain *domain,
 				 struct iommu_group *group);
+static ssize_t iommu_group_store_type(struct iommu_group *group,
+				      const char *buf, size_t count);
 
 static int __init iommu_set_def_domain_type(char *str)
 {
@@ -376,7 +378,8 @@ static IOMMU_GROUP_ATTR(name, S_IRUGO, iommu_group_show_name, NULL);
 static IOMMU_GROUP_ATTR(reserved_regions, 0444,
 			iommu_group_show_resv_regions, NULL);
 
-static IOMMU_GROUP_ATTR(type, 0444, iommu_group_show_type, NULL);
+static IOMMU_GROUP_ATTR(type, 0644, iommu_group_show_type,
+			iommu_group_store_type);
 
 static void iommu_group_release(struct kobject *kobj)
 {
@@ -2468,3 +2471,176 @@ int iommu_sva_get_pasid(struct iommu_sva *handle)
 	return ops->sva_get_pasid(handle);
 }
 EXPORT_SYMBOL_GPL(iommu_sva_get_pasid);
+
+/*
+ * Changes the default domain of a group. This function is heavily inspired from
+ * request_default_domain_for_dev() and couldn't re-use the same because:
+ * 1. The domain should be changed even if there are devices under this group
+ *    because the driver is already unbinded and it's safe to do so. Also, note
+ *    that *only* default_domain is being changed and hence the devices list in
+ *    the group need not be changed.
+ * 2. Unlike request_default_domain_for_dev(), a domain is allocated only once
+ *    for the whole group, where as the former allocates a domain per device.
+ *
+ * @group: The group for which the default domain should be changed
+ * @prev_domain: The previous domain that is being switched from
+ * @type: The type of the new default domain that gets associated with the group
+ *
+ * Returns 0 on success and error code on failure
+ *
+ * Note:
+ * 1. Presently, this function is called only when user requests to change the
+ *    group's default domain type through /sys/kernel/iommu_groups/<grp_id>/type
+ *    Be aware to take a closer look if intended to use for other cases.
+ * 2. Assumes group->mutex is already taken
+ */
+static int iommu_group_change_def_domain(struct iommu_group *group,
+					 struct iommu_domain *prev_domain,
+					 int type)
+{
+	struct group_device *grp_dev;
+	struct iommu_domain *new_domain;
+	int ret = 0;
+
+	/*
+	 * iommu_domain_alloc() takes "struct bus_type" as an argument which is
+	 * a member in "struct device". Changing a group's default domain type
+	 * deals at iommu_group level rather than device level and hence there
+	 * is no straight forward way to get "bus_type" of an iommu_group that
+	 * could be passed to iommu_domain_alloc(). So, instead of directly
+	 * calling iommu_domain_alloc(), use iommu_ops from previous default
+	 * domain.
+	 */
+	if (!prev_domain || !prev_domain->ops ||
+	    !prev_domain->ops->domain_alloc)
+		return -EINVAL;
+
+	/* Allocate a new domain of requested type */
+	new_domain = prev_domain->ops->domain_alloc(type);
+	if (!new_domain) {
+		pr_err("Unable to allocate memory for the new domain\n");
+		return -ENOMEM;
+	}
+
+	new_domain->type = type;
+	new_domain->ops = prev_domain->ops;
+	new_domain->pgsize_bitmap = prev_domain->pgsize_bitmap;
+
+	/* Attach all the devices in the group to the newly created domain */
+	ret = __iommu_attach_group(new_domain, group);
+	if (ret) {
+		pr_err("Unable to attach all the devices in the group to the new domain\n");
+		goto free_new_domain;
+	}
+
+	/*
+	 * Map reserved regions if the group's default domain is being changed
+	 * from identity domain to dma domain
+	 */
+	if (type == IOMMU_DOMAIN_IDENTITY)
+		goto free_prev_domain;
+
+	list_for_each_entry(grp_dev, &group->devices, list) {
+		ret = iommu_group_create_direct_mappings(group, grp_dev->dev);
+		if (ret) {
+			dev_err(grp_dev->dev, "Failed to create direct mappings for reserved regions\n");
+			goto free_new_domain;
+		}
+	}
+
+free_prev_domain:
+	/*
+	 * Free the existing default domain and replace it with the newly
+	 * created default domain. No need to set group->domain because
+	 * __iommu_attach_group() already does it on success.
+	 */
+	iommu_domain_free(prev_domain);
+	group->default_domain = new_domain;
+	return 0;
+
+free_new_domain:
+	iommu_domain_free(new_domain);
+	return ret;
+}
+
+static int is_driver_binded(struct device *dev, void *not_used)
+{
+	int ret = 0;
+
+	device_lock(dev);
+	if (device_is_bound(dev))
+		ret = 1;
+	device_unlock(dev);
+	return ret;
+}
+
+static ssize_t iommu_group_store_type(struct iommu_group *group,
+				      const char *buf, size_t count)
+{
+	int ret, req_type, allowed_types;
+	struct iommu_domain *prev_domain;
+	struct group_device *grp_dev;
+
+	if (!capable(CAP_SYS_ADMIN) || !capable(CAP_SYS_RAWIO))
+		return -EACCES;
+
+	if (WARN_ON(!group))
+		return -EINVAL;
+
+	if (sysfs_streq(buf, "identity"))
+		req_type = IOMMU_DOMAIN_IDENTITY;
+	else if (sysfs_streq(buf, "dma"))
+		req_type = IOMMU_DOMAIN_DMA;
+	else
+		return -EINVAL;
+
+	/* Check if any device in the group still has a driver binded to it */
+	if (iommu_group_for_each_dev(group, NULL, is_driver_binded)) {
+		pr_err("Active drivers exist for devices in the group\n");
+		return -EBUSY;
+	}
+
+	mutex_lock(&group->mutex);
+	prev_domain = group->default_domain;
+	if (!prev_domain) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	/*
+	 * Switch to a new domain only if the requested domain type is different
+	 * from the existing default domain type
+	 */
+	if (prev_domain->type == req_type) {
+		ret = count;
+		goto out;
+	}
+
+	/*
+	 * Every device may not support both the domain types (namely DMA and
+	 * identity), so check if it's ok to change domain type of every device
+	 * in the group to the requested domain
+	 */
+	if (prev_domain->ops && prev_domain->ops->device_def_domain_type) {
+		list_for_each_entry(grp_dev, &group->devices, list) {
+			const struct iommu_ops *ops;
+			struct device *dev;
+
+			dev = grp_dev->dev;
+			ops = prev_domain->ops;
+			allowed_types = ops->device_def_domain_type(dev, false);
+			if (allowed_types && allowed_types != req_type) {
+				dev_err(dev, "Cannot be in %s domain\n", buf);
+				ret = -EINVAL;
+				goto out;
+			}
+		}
+	}
+
+	ret = iommu_group_change_def_domain(group, prev_domain, req_type);
+	if (!ret)
+		ret = count;
+out:
+	mutex_unlock(&group->mutex);
+	return ret;
+}
-- 
2.19.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH RFC 4/4] iommu: Document usage of "/sys/kernel/iommu_groups/<grp_id>/type" file

[Sai Praneeth Prakhya]

The default domain type of an iommu group can be changed using file
"/sys/kernel/iommu_groups/<grp_id>/type". Hence, document it's usage and
more importantly spell out it's limitations and an example.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: Sohil Mehta <sohil.mehta@intel.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>
---
 .../ABI/testing/sysfs-kernel-iommu_groups     | 34 +++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/Documentation/ABI/testing/sysfs-kernel-iommu_groups b/Documentation/ABI/testing/sysfs-kernel-iommu_groups
index 017f5bc3920c..7eb1b784c5e3 100644
--- a/Documentation/ABI/testing/sysfs-kernel-iommu_groups
+++ b/Documentation/ABI/testing/sysfs-kernel-iommu_groups
@@ -33,3 +33,37 @@ Description:    In case an RMRR is used only by graphics or USB devices
 		it is now exposed as "direct-relaxable" instead of "direct".
 		In device assignment use case, for instance, those RMRR
 		are considered to be relaxable and safe.
+
+What:		/sys/kernel/iommu_groups/<grp_id>/type
+Date:		July 2019
+KernelVersion:	v5.4
+Contact:	Sai Praneeth Prakhya <sai.praneeth.prakhya@intel.com>
+Description:	/sys/kernel/iommu_groups/<grp_id>/type lets the user know the
+		type of default domain in use by iommu for this group. A
+		privileged user could request kernel to change the group type by
+		writing to this file. Presently, only two types are supported
+		1. dma: All the DMA transactions from the devices in this group
+			are translated by the iommu.
+		2. identity: All the DMA transactions from the devices in this
+			     group are *not* translated by the iommu.
+		Note:
+		-----
+		A group type could be modified only when *all* the devices in
+		the group are not binded to any device driver. So, the user has
+		to first unbind the appropriate drivers and then change the
+		default domain type.
+		Caution:
+		--------
+		Unbinding a device driver will take away the drivers control
+		over the device and if done on devices that host root file
+		system could lead to catastrophic effects (the user might
+		need to reboot the machine to get it to normal state). So, it's
+		expected that the user understands what he is doing.
+		Example:
+		--------
+		# Unbind USB device driver
+		1. echo "0000:00:14.0" > /sys/bus/pci/drivers/xhci_hcd/unbind
+		# Put the USB device in identity mode (a.k.a pass through)
+		2. echo "identity" > /sys/kernel/iommu_groups/<grp_id>/type
+		# Re-bind the driver
+		3. echo "0000:00:14.0" > /sys/bus/pci/drivers/xhci_hcd/bind
-- 
2.19.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH RFC 0/4] iommu: Add support to change default domain of a group

[Joerg Roedel]

On Tue, Jul 02, 2019 at 06:53:58PM -0700, Sai Praneeth Prakhya wrote:
> Assume a use case where-in the priviliged user would want to use the device in
> pass-through mode when the device is used for host but would want to switch to
> dma protected mode when switching for VFIO in user space. Presently, this is not
> supported and hence add support to change default domain of a group dynamically.

VFIO does it's own iommu magic with the device and moves the group out
of the default domain, so that doesn't sound like a valid use-case. More
valid would be something like putting a device into passthrough mode to
improve performance, or do you have other valid use-cases in mind?


Regards,

	Joerg
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [PATCH RFC 0/4] iommu: Add support to change default domain of a group

[Prakhya, Sai Praneeth]

> On Tue, Jul 02, 2019 at 06:53:58PM -0700, Sai Praneeth Prakhya wrote:
> > Assume a use case where-in the priviliged user would want to use the
> > device in pass-through mode when the device is used for host but would
> > want to switch to dma protected mode when switching for VFIO in user
> > space. Presently, this is not supported and hence add support to change
> default domain of a group dynamically.
> 
> VFIO does it's own iommu magic with the device and moves the group out of the
> default domain, so that doesn't sound like a valid use-case.

Thanks a lot! for the reply Joerg.
I wasn't aware about this as I have very limited exposure to VFIO.
I will take a look into this.

> More valid would be something like putting a device into passthrough 
>  mode to improve performance, or do you have other valid use-cases in mind?

Presently, we don't have anything else except that putting a device in pass through 
mode will improve its performance.

Regards,
Sai
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu