linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 1/1] iommu/vt-d: Use device numa domain if RHSA is missing
@ 2020-09-04  1:03 Lu Baolu
  2020-09-04  2:16 ` Tian, Kevin
  0 siblings, 1 reply; 3+ messages in thread
From: Lu Baolu @ 2020-09-04  1:03 UTC (permalink / raw)
  To: Joerg Roedel
  Cc: iommu, linux-kernel, Lu Baolu, Jacob Pan, Kevin Tian, Ashok Raj

If there are multiple NUMA domains but the RHSA is missing in ACPI/DMAR
table, we could default to the device NUMA domain as fall back. This could
also benefit a vIOMMU use case where only single vIOMMU is exposed, hence
no RHSA will be present but device numa domain can be correct.

Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 37 +++++++++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

Change log:
v1->v2:
  - Add a comment as suggested by Kevin.
    https://lore.kernel.org/linux-iommu/MWHPR11MB1645E6D6BD1EFDFA139AA37C8C520@MWHPR11MB1645.namprd11.prod.outlook.com/

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 7f844d1c8cd9..69d5a87188f4 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -698,12 +698,47 @@ static int domain_update_iommu_superpage(struct dmar_domain *domain,
 	return fls(mask);
 }
 
+static int domain_update_device_node(struct dmar_domain *domain)
+{
+	struct device_domain_info *info;
+	int nid = NUMA_NO_NODE;
+
+	assert_spin_locked(&device_domain_lock);
+
+	if (list_empty(&domain->devices))
+		return NUMA_NO_NODE;
+
+	list_for_each_entry(info, &domain->devices, link) {
+		if (!info->dev)
+			continue;
+
+		/*
+		 * There could possibly be multiple device numa nodes as devices
+		 * within the same domain may sit behind different IOMMUs. There
+		 * isn't perfect answer in such situation, so we select first
+		 * come first served policy.
+		 */
+		nid = dev_to_node(info->dev);
+		if (nid != NUMA_NO_NODE)
+			break;
+	}
+
+	return nid;
+}
+
 /* Some capabilities may be different across iommus */
 static void domain_update_iommu_cap(struct dmar_domain *domain)
 {
 	domain_update_iommu_coherency(domain);
 	domain->iommu_snooping = domain_update_iommu_snooping(NULL);
 	domain->iommu_superpage = domain_update_iommu_superpage(domain, NULL);
+
+	/*
+	 * If RHSA is missing, we should default to the device numa domain
+	 * as fall back.
+	 */
+	if (domain->nid == NUMA_NO_NODE)
+		domain->nid = domain_update_device_node(domain);
 }
 
 struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus,
@@ -5096,8 +5131,6 @@ static struct iommu_domain *intel_iommu_domain_alloc(unsigned type)
 		if (type == IOMMU_DOMAIN_DMA)
 			intel_init_iova_domain(dmar_domain);
 
-		domain_update_iommu_cap(dmar_domain);
-
 		domain = &dmar_domain->domain;
 		domain->geometry.aperture_start = 0;
 		domain->geometry.aperture_end   =
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 3+ messages in thread
* Re: [PATCH v2 1/1] iommu/vt-d: Use device numa domain if RHSA is missing
  2020-09-04  2:16 ` Tian, Kevin
@ 2020-09-04  2:16   ` Lu Baolu
  0 siblings, 0 replies; 3+ messages in thread
From: Lu Baolu @ 2020-09-04  2:16 UTC (permalink / raw)
  To: Tian, Kevin, Joerg Roedel; +Cc: baolu.lu, Raj, Ashok, linux-kernel, iommu

Hi Kevin,

On 9/4/20 10:16 AM, Tian, Kevin wrote:
>> From: Lu Baolu
>> Sent: Friday, September 4, 2020 9:03 AM
>>
>> If there are multiple NUMA domains but the RHSA is missing in ACPI/DMAR
>> table, we could default to the device NUMA domain as fall back. This could
>> also benefit a vIOMMU use case where only single vIOMMU is exposed,
>> hence
>> no RHSA will be present but device numa domain can be correct.
> 
> My comment on this is not fixed. It is not restricted to single-vIOMMU situation.
> and actually this may also happen on physical platform if some FW doesn't
> provide RHSA information.

Ah, yes. I will remove this sentence since it's same for both bare metal
and virtualization.

> 
> with that being fixed:
> 
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Thank you!

Best regards,
baolu

> 
>>
>> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
>> Cc: Kevin Tian <kevin.tian@intel.com>
>> Cc: Ashok Raj <ashok.raj@intel.com>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/iommu/intel/iommu.c | 37
>> +++++++++++++++++++++++++++++++++++--
>>   1 file changed, 35 insertions(+), 2 deletions(-)
>>
>> Change log:
>> v1->v2:
>>    - Add a comment as suggested by Kevin.
>>      https://lore.kernel.org/linux-
>> iommu/MWHPR11MB1645E6D6BD1EFDFA139AA37C8C520@MWHPR11MB1
>> 645.namprd11.prod.outlook.com/
>>
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 7f844d1c8cd9..69d5a87188f4 100644
>> --- a/drivers/iommu/intel/iommu.c
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -698,12 +698,47 @@ static int
>> domain_update_iommu_superpage(struct dmar_domain *domain,
>>   	return fls(mask);
>>   }
>>
>> +static int domain_update_device_node(struct dmar_domain *domain)
>> +{
>> +	struct device_domain_info *info;
>> +	int nid = NUMA_NO_NODE;
>> +
>> +	assert_spin_locked(&device_domain_lock);
>> +
>> +	if (list_empty(&domain->devices))
>> +		return NUMA_NO_NODE;
>> +
>> +	list_for_each_entry(info, &domain->devices, link) {
>> +		if (!info->dev)
>> +			continue;
>> +
>> +		/*
>> +		 * There could possibly be multiple device numa nodes as
>> devices
>> +		 * within the same domain may sit behind different IOMMUs.
>> There
>> +		 * isn't perfect answer in such situation, so we select first
>> +		 * come first served policy.
>> +		 */
>> +		nid = dev_to_node(info->dev);
>> +		if (nid != NUMA_NO_NODE)
>> +			break;
>> +	}
>> +
>> +	return nid;
>> +}
>> +
>>   /* Some capabilities may be different across iommus */
>>   static void domain_update_iommu_cap(struct dmar_domain *domain)
>>   {
>>   	domain_update_iommu_coherency(domain);
>>   	domain->iommu_snooping =
>> domain_update_iommu_snooping(NULL);
>>   	domain->iommu_superpage =
>> domain_update_iommu_superpage(domain, NULL);
>> +
>> +	/*
>> +	 * If RHSA is missing, we should default to the device numa domain
>> +	 * as fall back.
>> +	 */
>> +	if (domain->nid == NUMA_NO_NODE)
>> +		domain->nid = domain_update_device_node(domain);
>>   }
>>
>>   struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8
>> bus,
>> @@ -5096,8 +5131,6 @@ static struct iommu_domain
>> *intel_iommu_domain_alloc(unsigned type)
>>   		if (type == IOMMU_DOMAIN_DMA)
>>   			intel_init_iova_domain(dmar_domain);
>>
>> -		domain_update_iommu_cap(dmar_domain);
>> -
>>   		domain = &dmar_domain->domain;
>>   		domain->geometry.aperture_start = 0;
>>   		domain->geometry.aperture_end   =
>> --
>> 2.17.1
>>
>> _______________________________________________
>> iommu mailing list
>> iommu@lists.linux-foundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 3+ messages in thread
* RE: [PATCH v2 1/1] iommu/vt-d: Use device numa domain if RHSA is missing
  2020-09-04  1:03 [PATCH v2 1/1] iommu/vt-d: Use device numa domain if RHSA is missing Lu Baolu
@ 2020-09-04  2:16 ` Tian, Kevin
  2020-09-04  2:16   ` Lu Baolu
  0 siblings, 1 reply; 3+ messages in thread
From: Tian, Kevin @ 2020-09-04  2:16 UTC (permalink / raw)
  To: Lu Baolu, Joerg Roedel; +Cc: Raj, Ashok, linux-kernel, iommu

> From: Lu Baolu
> Sent: Friday, September 4, 2020 9:03 AM
> 
> If there are multiple NUMA domains but the RHSA is missing in ACPI/DMAR
> table, we could default to the device NUMA domain as fall back. This could
> also benefit a vIOMMU use case where only single vIOMMU is exposed,
> hence
> no RHSA will be present but device numa domain can be correct.

My comment on this is not fixed. It is not restricted to single-vIOMMU situation.
and actually this may also happen on physical platform if some FW doesn't
provide RHSA information.

with that being fixed:

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

> 
> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Ashok Raj <ashok.raj@intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>  drivers/iommu/intel/iommu.c | 37
> +++++++++++++++++++++++++++++++++++--
>  1 file changed, 35 insertions(+), 2 deletions(-)
> 
> Change log:
> v1->v2:
>   - Add a comment as suggested by Kevin.
>     https://lore.kernel.org/linux-
> iommu/MWHPR11MB1645E6D6BD1EFDFA139AA37C8C520@MWHPR11MB1
> 645.namprd11.prod.outlook.com/
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 7f844d1c8cd9..69d5a87188f4 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -698,12 +698,47 @@ static int
> domain_update_iommu_superpage(struct dmar_domain *domain,
>  	return fls(mask);
>  }
> 
> +static int domain_update_device_node(struct dmar_domain *domain)
> +{
> +	struct device_domain_info *info;
> +	int nid = NUMA_NO_NODE;
> +
> +	assert_spin_locked(&device_domain_lock);
> +
> +	if (list_empty(&domain->devices))
> +		return NUMA_NO_NODE;
> +
> +	list_for_each_entry(info, &domain->devices, link) {
> +		if (!info->dev)
> +			continue;
> +
> +		/*
> +		 * There could possibly be multiple device numa nodes as
> devices
> +		 * within the same domain may sit behind different IOMMUs.
> There
> +		 * isn't perfect answer in such situation, so we select first
> +		 * come first served policy.
> +		 */
> +		nid = dev_to_node(info->dev);
> +		if (nid != NUMA_NO_NODE)
> +			break;
> +	}
> +
> +	return nid;
> +}
> +
>  /* Some capabilities may be different across iommus */
>  static void domain_update_iommu_cap(struct dmar_domain *domain)
>  {
>  	domain_update_iommu_coherency(domain);
>  	domain->iommu_snooping =
> domain_update_iommu_snooping(NULL);
>  	domain->iommu_superpage =
> domain_update_iommu_superpage(domain, NULL);
> +
> +	/*
> +	 * If RHSA is missing, we should default to the device numa domain
> +	 * as fall back.
> +	 */
> +	if (domain->nid == NUMA_NO_NODE)
> +		domain->nid = domain_update_device_node(domain);
>  }
> 
>  struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8
> bus,
> @@ -5096,8 +5131,6 @@ static struct iommu_domain
> *intel_iommu_domain_alloc(unsigned type)
>  		if (type == IOMMU_DOMAIN_DMA)
>  			intel_init_iova_domain(dmar_domain);
> 
> -		domain_update_iommu_cap(dmar_domain);
> -
>  		domain = &dmar_domain->domain;
>  		domain->geometry.aperture_start = 0;
>  		domain->geometry.aperture_end   =
> --
> 2.17.1
> 
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu

^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-09-04  2:21 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-04  1:03 [PATCH v2 1/1] iommu/vt-d: Use device numa domain if RHSA is missing Lu Baolu
2020-09-04  2:16 ` Tian, Kevin
2020-09-04  2:16   ` Lu Baolu

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).