Re: [PATCH v4 1/5] iommu: Add optional subdev in aux_at(de)tach ops

[Lu Baolu <baolu.lu@linux.intel.com>]

Hi Alex,

On 9/11/20 6:05 AM, Alex Williamson wrote:
> On Tue,  1 Sep 2020 11:34:18 +0800
> Lu Baolu <baolu.lu@linux.intel.com> wrote:
> 
>> In the vfio/mdev use case of aux-domain, the subdevices are created from
>> the physical devices with IOMMU_DEV_FEAT_AUX enabled and the aux-domains
>> are attached to the subdevices through the iommu_ops.aux_attach_dev()
>> interface.
>>
>> Current iommu_ops.aux_at(de)tach_dev() design only takes the aux-domain
>> and the physical device as the parameters, this is insufficient if we
>> want the vendor iommu drivers to learn the knowledge about relationships
>> between the aux-domains and the subdevices. Add a @subdev parameter to
>> iommu_ops.aux_at(de)tach_dev() interfaces so that a subdevice could be
>> opt-in.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/iommu/intel/iommu.c | 10 ++++++----
>>   drivers/iommu/iommu.c       |  4 ++--
>>   include/linux/iommu.h       |  6 ++++--
>>   3 files changed, 12 insertions(+), 8 deletions(-)
>>
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index bce158468abf..3c12fd06856c 100644
>> --- a/drivers/iommu/intel/iommu.c
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -5338,8 +5338,9 @@ static int intel_iommu_attach_device(struct iommu_domain *domain,
>>   	return domain_add_dev_info(to_dmar_domain(domain), dev);
>>   }
>>   
>> -static int intel_iommu_aux_attach_device(struct iommu_domain *domain,
>> -					 struct device *dev)
>> +static int
>> +intel_iommu_aux_attach_device(struct iommu_domain *domain,
>> +			      struct device *dev, struct device *subdev)
>>   {
>>   	int ret;
>>   
>> @@ -5359,8 +5360,9 @@ static void intel_iommu_detach_device(struct iommu_domain *domain,
>>   	dmar_remove_one_dev_info(dev);
>>   }
>>   
>> -static void intel_iommu_aux_detach_device(struct iommu_domain *domain,
>> -					  struct device *dev)
>> +static void
>> +intel_iommu_aux_detach_device(struct iommu_domain *domain, struct device *dev,
>> +			      struct device *subdev)
>>   {
>>   	aux_domain_remove_dev(to_dmar_domain(domain), dev);
>>   }
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 609bd25bf154..38cdfeb887e1 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -2728,7 +2728,7 @@ int iommu_aux_attach_device(struct iommu_domain *domain, struct device *dev)
>>   	int ret = -ENODEV;
>>   
>>   	if (domain->ops->aux_attach_dev)
>> -		ret = domain->ops->aux_attach_dev(domain, dev);
>> +		ret = domain->ops->aux_attach_dev(domain, dev, NULL);
>>   
>>   	if (!ret)
>>   		trace_attach_device_to_domain(dev);
>> @@ -2740,7 +2740,7 @@ EXPORT_SYMBOL_GPL(iommu_aux_attach_device);
>>   void iommu_aux_detach_device(struct iommu_domain *domain, struct device *dev)
>>   {
>>   	if (domain->ops->aux_detach_dev) {
>> -		domain->ops->aux_detach_dev(domain, dev);
>> +		domain->ops->aux_detach_dev(domain, dev, NULL);
>>   		trace_detach_device_from_domain(dev);
>>   	}
>>   }
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index fee209efb756..871267104915 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -279,8 +279,10 @@ struct iommu_ops {
>>   	int (*dev_disable_feat)(struct device *dev, enum iommu_dev_features f);
>>   
>>   	/* Aux-domain specific attach/detach entries */
>> -	int (*aux_attach_dev)(struct iommu_domain *domain, struct device *dev);
>> -	void (*aux_detach_dev)(struct iommu_domain *domain, struct device *dev);
>> +	int (*aux_attach_dev)(struct iommu_domain *domain, struct device *dev,
>> +			      struct device *subdev);
>> +	void (*aux_detach_dev)(struct iommu_domain *domain, struct device *dev,
>> +			       struct device *subdev);
>>   	int (*aux_get_pasid)(struct iommu_domain *domain, struct device *dev);
>>   
>>   	struct iommu_sva *(*sva_bind)(struct device *dev, struct mm_struct *mm,
> 
> Would this be a good spot in the code to provide comments more formally
> defining this subdevice concept?  For example, what exactly is the
> relationship between the device and the subdevice and which device do
> we use for the remaining aux domain functions, ex. aux_get_pasid().

Yes. Agreed. I will add below comments and adjust the function
parameters according to the naming rule.

/*
- * Aux-domain specific attach/detach.
+ * Aux-domain specific interfaces.
   *
   * Only works if iommu_dev_feature_enabled(dev, IOMMU_DEV_FEAT_AUX) 
returns
   * true. Also, as long as domains are attached to a device through this
@@ -2722,6 +2722,14 @@ EXPORT_SYMBOL_GPL(iommu_dev_feature_enabled);
   * (iommu_detach_device() can't fail, so we fail when trying to 
re-attach).
   * This should make us safe against a device being attached to a guest 
as a
   * whole while there are still pasid users on it (aux and sva).
+ *
+ * Some physical devices can be configured to generate several subdevices.
+ * The modern IOMMUs support the identification and isolation of these
+ * subdevices. Hence they could be passed through to users space. VFIO/mdev
+ * provides a generic framework for subdevice passthrough. Below interfaces
+ * are extended to support such use case. Generally, among the 
parameters of
+ * the following aux-domain specific functions, @physdev represents a 
physical
+ * device and @subdev represents a subdevice.
   */

Please help to review.

Best regards,
baolu

> Thanks,
> 
> Alex
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu