* [PATCH 0/2] iommu/vt-d: Add ESRTPS & ESIRTPS check
@ 2022-09-19 6:25 Lu Baolu
2022-09-19 6:25 ` [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation Lu Baolu
2022-09-19 6:25 ` [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA " Lu Baolu
0 siblings, 2 replies; 8+ messages in thread
From: Lu Baolu @ 2022-09-19 6:25 UTC (permalink / raw)
To: iommu
Cc: Joerg Roedel, Will Deacon, Robin Murphy, Kevin Tian, Jacob Pan,
linux-kernel, Lu Baolu
Hi folks,
Some VT-d hardware implementations invalidates all DMA remapping
or interrupt remapping hardware translation caches as part of SRTP or
SIRTPS flow. The VT-d spec introduced two capability bits for these
hardware behaviors. This series fine tuned the VT-d driver to avoid
unnecessary global cache validations.
Best regards,
baolu
Lu Baolu (2):
iommu/vt-d: Avoid unnecessary global IRTE cache invalidation
iommu/vt-d: Avoid unnecessary global DMA cache invalidation
drivers/iommu/intel/iommu.h | 2 ++
drivers/iommu/intel/iommu.c | 7 +++++++
drivers/iommu/intel/irq_remapping.c | 3 ++-
3 files changed, 11 insertions(+), 1 deletion(-)
--
2.34.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation
2022-09-19 6:25 [PATCH 0/2] iommu/vt-d: Add ESRTPS & ESIRTPS check Lu Baolu
@ 2022-09-19 6:25 ` Lu Baolu
2022-09-19 8:29 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
2022-09-19 6:25 ` [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA " Lu Baolu
1 sibling, 2 replies; 8+ messages in thread
From: Lu Baolu @ 2022-09-19 6:25 UTC (permalink / raw)
To: iommu
Cc: Joerg Roedel, Will Deacon, Robin Murphy, Kevin Tian, Jacob Pan,
linux-kernel, Lu Baolu
Some VT-d hardware implementations invalidate all interrupt remapping
hardware translation caches as part of SIRTP flow. The VT-d spec adds
a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
software has no need to issue the global invalidation request.
Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
drivers/iommu/intel/iommu.h | 1 +
drivers/iommu/intel/irq_remapping.c | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index eef1a166b855..5407d82df4d1 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -146,6 +146,7 @@
/*
* Decoding Capability Register
*/
+#define cap_esirtps(c) (((c) >> 62) & 1)
#define cap_fl5lp_support(c) (((c) >> 60) & 1)
#define cap_pi_support(c) (((c) >> 59) & 1)
#define cap_fl1gp_support(c) (((c) >> 56) & 1)
diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c
index 2e9683e970f8..b4a91fbd1c2f 100644
--- a/drivers/iommu/intel/irq_remapping.c
+++ b/drivers/iommu/intel/irq_remapping.c
@@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct intel_iommu *iommu, int mode)
* Global invalidation of interrupt entry cache to make sure the
* hardware uses the new irq remapping table.
*/
- qi_global_iec(iommu);
+ if (!cap_esirtps(iommu->cap))
+ qi_global_iec(iommu);
}
static void iommu_enable_irq_remapping(struct intel_iommu *iommu)
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation
2022-09-19 6:25 ` [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation Lu Baolu
@ 2022-09-19 8:29 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
1 sibling, 0 replies; 8+ messages in thread
From: Jerry Snitselaar @ 2022-09-19 8:29 UTC (permalink / raw)
To: Lu Baolu
Cc: iommu, Joerg Roedel, Will Deacon, Robin Murphy, Kevin Tian,
Jacob Pan, linux-kernel
On Mon, Sep 19, 2022 at 02:25:22PM +0800, Lu Baolu wrote:
> Some VT-d hardware implementations invalidate all interrupt remapping
> hardware translation caches as part of SIRTP flow. The VT-d spec adds
> a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
> 11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
> software has no need to issue the global invalidation request.
>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com
> ---
> drivers/iommu/intel/iommu.h | 1 +
> drivers/iommu/intel/irq_remapping.c | 3 ++-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index eef1a166b855..5407d82df4d1 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -146,6 +146,7 @@
> /*
> * Decoding Capability Register
> */
> +#define cap_esirtps(c) (((c) >> 62) & 1)
> #define cap_fl5lp_support(c) (((c) >> 60) & 1)
> #define cap_pi_support(c) (((c) >> 59) & 1)
> #define cap_fl1gp_support(c) (((c) >> 56) & 1)
> diff --git a/drivers/iommu/intel/irq_remapping.c b/drivers/iommu/intel/irq_remapping.c
> index 2e9683e970f8..b4a91fbd1c2f 100644
> --- a/drivers/iommu/intel/irq_remapping.c
> +++ b/drivers/iommu/intel/irq_remapping.c
> @@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct intel_iommu *iommu, int mode)
> * Global invalidation of interrupt entry cache to make sure the
> * hardware uses the new irq remapping table.
> */
> - qi_global_iec(iommu);
> + if (!cap_esirtps(iommu->cap))
> + qi_global_iec(iommu);
> }
>
> static void iommu_enable_irq_remapping(struct intel_iommu *iommu)
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation
2022-09-19 6:25 ` [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation Lu Baolu
2022-09-19 8:29 ` Jerry Snitselaar
@ 2022-09-20 7:51 ` Tian, Kevin
2022-09-21 6:49 ` Baolu Lu
1 sibling, 1 reply; 8+ messages in thread
From: Tian, Kevin @ 2022-09-20 7:51 UTC (permalink / raw)
To: Lu Baolu, iommu
Cc: Joerg Roedel, Will Deacon, Robin Murphy, Jacob Pan, linux-kernel
> From: Lu Baolu <baolu.lu@linux.intel.com>
> Sent: Monday, September 19, 2022 2:25 PM
>
> Some VT-d hardware implementations invalidate all interrupt remapping
> hardware translation caches as part of SIRTP flow. The VT-d spec adds
> a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
> 11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
> software has no need to issue the global invalidation request.
>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
> drivers/iommu/intel/iommu.h | 1 +
> drivers/iommu/intel/irq_remapping.c | 3 ++-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index eef1a166b855..5407d82df4d1 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -146,6 +146,7 @@
> /*
> * Decoding Capability Register
> */
> +#define cap_esirtps(c) (((c) >> 62) & 1)
> #define cap_fl5lp_support(c) (((c) >> 60) & 1)
> #define cap_pi_support(c) (((c) >> 59) & 1)
> #define cap_fl1gp_support(c) (((c) >> 56) & 1)
> diff --git a/drivers/iommu/intel/irq_remapping.c
> b/drivers/iommu/intel/irq_remapping.c
> index 2e9683e970f8..b4a91fbd1c2f 100644
> --- a/drivers/iommu/intel/irq_remapping.c
> +++ b/drivers/iommu/intel/irq_remapping.c
> @@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct
> intel_iommu *iommu, int mode)
> * Global invalidation of interrupt entry cache to make sure the
> * hardware uses the new irq remapping table.
> */
> - qi_global_iec(iommu);
> + if (!cap_esirtps(iommu->cap))
> + qi_global_iec(iommu);
> }
>
Same check is also required in iommu_disable_irq_remapping().
The spec says that:
--
For implementations reporting the Enhanced Set Interrupt Remap Table
Pointer Support (ESIRTPS) field as Set, hardware performs global
invalidation on all Interrupt remapping caches as part of Interrupt
Remapping Disable operation.
--
While looking at that code I think it's problematic to invalidate interrupt
cache before disabling interrupt remapping. Presumably invalidations
should be done only after a configuration is changed, no matter it's 'enable'
or 'disable'.
Thanks
Kevin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation
2022-09-20 7:51 ` Tian, Kevin
@ 2022-09-21 6:49 ` Baolu Lu
0 siblings, 0 replies; 8+ messages in thread
From: Baolu Lu @ 2022-09-21 6:49 UTC (permalink / raw)
To: Tian, Kevin, iommu
Cc: baolu.lu, Joerg Roedel, Will Deacon, Robin Murphy, Jacob Pan,
linux-kernel
On 2022/9/20 15:51, Tian, Kevin wrote:
>> From: Lu Baolu <baolu.lu@linux.intel.com>
>> Sent: Monday, September 19, 2022 2:25 PM
>>
>> Some VT-d hardware implementations invalidate all interrupt remapping
>> hardware translation caches as part of SIRTP flow. The VT-d spec adds
>> a ESIRTPS (Enhanced Set Interrupt Remap Table Pointer Support, section
>> 11.4.2 in VT-d spec) capability bit to indicate this. With this bit set,
>> software has no need to issue the global invalidation request.
>>
>> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>> drivers/iommu/intel/iommu.h | 1 +
>> drivers/iommu/intel/irq_remapping.c | 3 ++-
>> 2 files changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
>> index eef1a166b855..5407d82df4d1 100644
>> --- a/drivers/iommu/intel/iommu.h
>> +++ b/drivers/iommu/intel/iommu.h
>> @@ -146,6 +146,7 @@
>> /*
>> * Decoding Capability Register
>> */
>> +#define cap_esirtps(c) (((c) >> 62) & 1)
>> #define cap_fl5lp_support(c) (((c) >> 60) & 1)
>> #define cap_pi_support(c) (((c) >> 59) & 1)
>> #define cap_fl1gp_support(c) (((c) >> 56) & 1)
>> diff --git a/drivers/iommu/intel/irq_remapping.c
>> b/drivers/iommu/intel/irq_remapping.c
>> index 2e9683e970f8..b4a91fbd1c2f 100644
>> --- a/drivers/iommu/intel/irq_remapping.c
>> +++ b/drivers/iommu/intel/irq_remapping.c
>> @@ -494,7 +494,8 @@ static void iommu_set_irq_remapping(struct
>> intel_iommu *iommu, int mode)
>> * Global invalidation of interrupt entry cache to make sure the
>> * hardware uses the new irq remapping table.
>> */
>> - qi_global_iec(iommu);
>> + if (!cap_esirtps(iommu->cap))
>> + qi_global_iec(iommu);
>> }
>>
>
> Same check is also required in iommu_disable_irq_remapping().
>
> The spec says that:
>
> --
> For implementations reporting the Enhanced Set Interrupt Remap Table
> Pointer Support (ESIRTPS) field as Set, hardware performs global
> invalidation on all Interrupt remapping caches as part of Interrupt
> Remapping Disable operation.
> --
Yes. I will add the same check in iommu_disable_irq_remapping(). Thanks
for the reminding.
> While looking at that code I think it's problematic to invalidate interrupt
> cache before disabling interrupt remapping. Presumably invalidations
> should be done only after a configuration is changed, no matter it's 'enable'
> or 'disable'.
I understand your concern. The spec doesn't state the sequence in the
interrupt remapping disabling flow. Need to check it with the spec
developers before any change here.
Best regards,
baolu
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA cache invalidation
2022-09-19 6:25 [PATCH 0/2] iommu/vt-d: Add ESRTPS & ESIRTPS check Lu Baolu
2022-09-19 6:25 ` [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation Lu Baolu
@ 2022-09-19 6:25 ` Lu Baolu
2022-09-19 8:30 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
1 sibling, 2 replies; 8+ messages in thread
From: Lu Baolu @ 2022-09-19 6:25 UTC (permalink / raw)
To: iommu
Cc: Joerg Roedel, Will Deacon, Robin Murphy, Kevin Tian, Jacob Pan,
linux-kernel, Lu Baolu
Some VT-d hardware implementations invalidate all DMA remapping hardware
translation caches as part of SRTP flow. The VT-d spec adds a ESRTPS
(Enhanced Set Root Table Pointer Support, section 11.4.2 in VT-d spec)
capability bit to indicate this. With this bit set, software has no need
to issue the global invalidation request.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
drivers/iommu/intel/iommu.h | 1 +
drivers/iommu/intel/iommu.c | 7 +++++++
2 files changed, 8 insertions(+)
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 5407d82df4d1..251a609fdce3 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -146,6 +146,7 @@
/*
* Decoding Capability Register
*/
+#define cap_esrtps(c) (((c) >> 63) & 1)
#define cap_esirtps(c) (((c) >> 62) & 1)
#define cap_fl5lp_support(c) (((c) >> 60) & 1)
#define cap_pi_support(c) (((c) >> 59) & 1)
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 8e5d99197d39..c20f3e060bac 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1239,6 +1239,13 @@ static void iommu_set_root_entry(struct intel_iommu *iommu)
raw_spin_unlock_irqrestore(&iommu->register_lock, flag);
+ /*
+ * Hardware invalidates all DMA remapping hardware translation
+ * caches as part of SRTP flow.
+ */
+ if (cap_esrtps(iommu->cap))
+ return;
+
iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
if (sm_supported(iommu))
qi_flush_pasid_cache(iommu, 0, QI_PC_GLOBAL, 0);
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA cache invalidation
2022-09-19 6:25 ` [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA " Lu Baolu
@ 2022-09-19 8:30 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
1 sibling, 0 replies; 8+ messages in thread
From: Jerry Snitselaar @ 2022-09-19 8:30 UTC (permalink / raw)
To: Lu Baolu
Cc: iommu, Joerg Roedel, Will Deacon, Robin Murphy, Kevin Tian,
Jacob Pan, linux-kernel
On Mon, Sep 19, 2022 at 02:25:23PM +0800, Lu Baolu wrote:
> Some VT-d hardware implementations invalidate all DMA remapping hardware
> translation caches as part of SRTP flow. The VT-d spec adds a ESRTPS
> (Enhanced Set Root Table Pointer Support, section 11.4.2 in VT-d spec)
> capability bit to indicate this. With this bit set, software has no need
> to issue the global invalidation request.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com
> ---
> drivers/iommu/intel/iommu.h | 1 +
> drivers/iommu/intel/iommu.c | 7 +++++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index 5407d82df4d1..251a609fdce3 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -146,6 +146,7 @@
> /*
> * Decoding Capability Register
> */
> +#define cap_esrtps(c) (((c) >> 63) & 1)
> #define cap_esirtps(c) (((c) >> 62) & 1)
> #define cap_fl5lp_support(c) (((c) >> 60) & 1)
> #define cap_pi_support(c) (((c) >> 59) & 1)
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 8e5d99197d39..c20f3e060bac 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -1239,6 +1239,13 @@ static void iommu_set_root_entry(struct intel_iommu *iommu)
>
> raw_spin_unlock_irqrestore(&iommu->register_lock, flag);
>
> + /*
> + * Hardware invalidates all DMA remapping hardware translation
> + * caches as part of SRTP flow.
> + */
> + if (cap_esrtps(iommu->cap))
> + return;
> +
> iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
> if (sm_supported(iommu))
> qi_flush_pasid_cache(iommu, 0, QI_PC_GLOBAL, 0);
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA cache invalidation
2022-09-19 6:25 ` [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA " Lu Baolu
2022-09-19 8:30 ` Jerry Snitselaar
@ 2022-09-20 7:51 ` Tian, Kevin
1 sibling, 0 replies; 8+ messages in thread
From: Tian, Kevin @ 2022-09-20 7:51 UTC (permalink / raw)
To: Lu Baolu, iommu
Cc: Joerg Roedel, Will Deacon, Robin Murphy, Jacob Pan, linux-kernel
> From: Lu Baolu <baolu.lu@linux.intel.com>
> Sent: Monday, September 19, 2022 2:25 PM
>
> Some VT-d hardware implementations invalidate all DMA remapping
> hardware
> translation caches as part of SRTP flow. The VT-d spec adds a ESRTPS
> (Enhanced Set Root Table Pointer Support, section 11.4.2 in VT-d spec)
> capability bit to indicate this. With this bit set, software has no need
> to issue the global invalidation request.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-09-21 6:49 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-19 6:25 [PATCH 0/2] iommu/vt-d: Add ESRTPS & ESIRTPS check Lu Baolu
2022-09-19 6:25 ` [PATCH 1/2] iommu/vt-d: Avoid unnecessary global IRTE cache invalidation Lu Baolu
2022-09-19 8:29 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
2022-09-21 6:49 ` Baolu Lu
2022-09-19 6:25 ` [PATCH 2/2] iommu/vt-d: Avoid unnecessary global DMA " Lu Baolu
2022-09-19 8:30 ` Jerry Snitselaar
2022-09-20 7:51 ` Tian, Kevin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).