[PATCH 0/4] Add support for Intel IOMMU 5-level paging

[PATCH 0/4] Add support for Intel IOMMU 5-level paging

[Sohil Mehta]

Hi All,

Upcoming hardware plans to introduce support for 5-level paging[1]. The support
for CPU 5-level paging has already been merged in kernel v4.14. The following
patches add support for Intel IOMMU 5-level paging. The patches are based on
the Intel Virtualization Technology for Directed I/O spec revision - 2.5 [2]

Intel IOMMU Extended-context entries support two levels of translation,
referred to as first-level translation and second-level translation. The first
level page table points to CPU page table when IOMMU is using Shared virtual
memory(SVM). The second level page tables are built by IOMMU driver for IOVA
translations when functions like dma_map/iommu_map are called.

Use of 5-level paging for first-level translation is controlled through
programming of the new paging mode field in the PASID entry for the IOMMU PASID
(Process Address space ID) table. Use of 5-level paging for second-level
translation is controlled through the programming of already existing Address
Width (AW) field in the context/extended-context entry.

The first patch adds 5-level paging support for second level translations by
increasing the supported domain address width. The following patches add the
necessary checks before enabling SVM. The last patch programs the 5-level
paging mode in the PASID entry.

Regards,
Sohil

[1] https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf
[2] https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf


Sohil Mehta (4):
  iommu/vt-d: Enable upto 57 bits of domain address width
  iommu/vt-d: Add a check for 1GB page support
  iommu/vt-d: Add a check for 5-level paging support
  iommu/vt-d: Enable 5-level paging mode in the PASID entry

 drivers/iommu/intel-iommu.c |  2 +-
 drivers/iommu/intel-svm.c   | 23 +++++++++++++++++++++--
 include/linux/intel-iommu.h |  2 ++
 3 files changed, 24 insertions(+), 3 deletions(-)

-- 
2.7.4

[PATCH 0/4] Add support for Intel IOMMU 5-level paging

[Sohil Mehta]

Hi All,

Upcoming hardware plans to introduce support for 5-level paging[1]. The support
for CPU 5-level paging has already been merged in kernel v4.14. The following
patches add support for Intel IOMMU 5-level paging. The patches are based on
the Intel Virtualization Technology for Directed I/O spec revision - 2.5 [2]

Intel IOMMU Extended-context entries support two levels of translation,
referred to as first-level translation and second-level translation. The first
level page table points to CPU page table when IOMMU is using Shared virtual
memory(SVM). The second level page tables are built by IOMMU driver for IOVA
translations when functions like dma_map/iommu_map are called.

Use of 5-level paging for first-level translation is controlled through
programming of the new paging mode field in the PASID entry for the IOMMU PASID
(Process Address space ID) table. Use of 5-level paging for second-level
translation is controlled through the programming of already existing Address
Width (AW) field in the context/extended-context entry.

The first patch adds 5-level paging support for second level translations by
increasing the supported domain address width. The following patches add the
necessary checks before enabling SVM. The last patch programs the 5-level
paging mode in the PASID entry.

Regards,
Sohil

[1] https://software.intel.com/sites/default/files/managed/2b/80/5-level_paging_white_paper.pdf
[2] https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf


Sohil Mehta (4):
  iommu/vt-d: Enable upto 57 bits of domain address width
  iommu/vt-d: Add a check for 1GB page support
  iommu/vt-d: Add a check for 5-level paging support
  iommu/vt-d: Enable 5-level paging mode in the PASID entry

 drivers/iommu/intel-iommu.c |  2 +-
 drivers/iommu/intel-svm.c   | 23 +++++++++++++++++++++--
 include/linux/intel-iommu.h |  2 ++
 3 files changed, 24 insertions(+), 3 deletions(-)

-- 
2.7.4

[PATCH 1/4] iommu/vt-d: Enable upto 57 bits of domain address width

[Sohil Mehta]

Update the IOMMU default domain address width to 57 bits. This would
enable the IOMMU to do upto 5-levels of paging for second level
translations - IOVA translation requests without PASID.

Even though the maximum supported address width is being increased to
57, __iommu_calculate_agaw() would set the actual supported address
width to the maximum support available in IOMMU hardware.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
---
 drivers/iommu/intel-iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 4a2de34..457795b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -64,7 +64,7 @@
 #define IOAPIC_RANGE_END	(0xfeefffff)
 #define IOVA_START_ADDR		(0x1000)
 
-#define DEFAULT_DOMAIN_ADDRESS_WIDTH 48
+#define DEFAULT_DOMAIN_ADDRESS_WIDTH 57
 
 #define MAX_AGAW_WIDTH 64
 #define MAX_AGAW_PFN_WIDTH	(MAX_AGAW_WIDTH - VTD_PAGE_SHIFT)
-- 
2.7.4

[PATCH 1/4] iommu/vt-d: Enable upto 57 bits of domain address width

[Sohil Mehta]

Update the IOMMU default domain address width to 57 bits. This would
enable the IOMMU to do upto 5-levels of paging for second level
translations - IOVA translation requests without PASID.

Even though the maximum supported address width is being increased to
57, __iommu_calculate_agaw() would set the actual supported address
width to the maximum support available in IOMMU hardware.

Signed-off-by: Sohil Mehta <sohil.mehta-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 4a2de34..457795b 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -64,7 +64,7 @@
 #define IOAPIC_RANGE_END	(0xfeefffff)
 #define IOVA_START_ADDR		(0x1000)
 
-#define DEFAULT_DOMAIN_ADDRESS_WIDTH 48
+#define DEFAULT_DOMAIN_ADDRESS_WIDTH 57
 
 #define MAX_AGAW_WIDTH 64
 #define MAX_AGAW_PFN_WIDTH	(MAX_AGAW_WIDTH - VTD_PAGE_SHIFT)
-- 
2.7.4

[PATCH 2/4] iommu/vt-d: Add a check for 1GB page support

[Sohil Mehta]

Add a check to verify IOMMU 1GB page support. If the CPU supports 1GB
pages but the IOMMU does not support it then disable SVM by not
allocating PASID tables.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
---
 drivers/iommu/intel-svm.c   | 4 ++++
 include/linux/intel-iommu.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index ed1cf7c..fcab440 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -41,6 +41,10 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)
 	struct page *pages;
 	int order;
 
+	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
+			!cap_fl1gp_support(iommu->cap))
+		return -EINVAL;
+
 	/* Start at 2 because it's defined as 2^(1+PSS) */
 	iommu->pasid_max = 2 << ecap_pss(iommu->ecap);
 
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index f3274d9..a56bab1 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -84,6 +84,7 @@
  * Decoding Capability Register
  */
 #define cap_pi_support(c)	(((c) >> 59) & 1)
+#define cap_fl1gp_support(c)	(((c) >> 56) & 1)
 #define cap_read_drain(c)	(((c) >> 55) & 1)
 #define cap_write_drain(c)	(((c) >> 54) & 1)
 #define cap_max_amask_val(c)	(((c) >> 48) & 0x3f)
-- 
2.7.4

[PATCH 2/4] iommu/vt-d: Add a check for 1GB page support

[Sohil Mehta]

Add a check to verify IOMMU 1GB page support. If the CPU supports 1GB
pages but the IOMMU does not support it then disable SVM by not
allocating PASID tables.

Signed-off-by: Sohil Mehta <sohil.mehta-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/iommu/intel-svm.c   | 4 ++++
 include/linux/intel-iommu.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index ed1cf7c..fcab440 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -41,6 +41,10 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)
 	struct page *pages;
 	int order;
 
+	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
+			!cap_fl1gp_support(iommu->cap))
+		return -EINVAL;
+
 	/* Start at 2 because it's defined as 2^(1+PSS) */
 	iommu->pasid_max = 2 << ecap_pss(iommu->ecap);
 
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index f3274d9..a56bab1 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -84,6 +84,7 @@
  * Decoding Capability Register
  */
 #define cap_pi_support(c)	(((c) >> 59) & 1)
+#define cap_fl1gp_support(c)	(((c) >> 56) & 1)
 #define cap_read_drain(c)	(((c) >> 55) & 1)
 #define cap_write_drain(c)	(((c) >> 54) & 1)
 #define cap_max_amask_val(c)	(((c) >> 48) & 0x3f)
-- 
2.7.4

[PATCH 3/4] iommu/vt-d: Add a check for 5-level paging support

[Sohil Mehta]

Add a check to verify IOMMU 5-level paging support. If the CPU supports
supports 5-level paging but the IOMMU does not support it then disable
SVM by not allocating PASID tables.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
---
 drivers/iommu/intel-svm.c   | 4 ++++
 include/linux/intel-iommu.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index fcab440..4072a18 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -45,6 +45,10 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)
 			!cap_fl1gp_support(iommu->cap))
 		return -EINVAL;
 
+	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			!cap_5lp_support(iommu->cap))
+		return -EINVAL;
+
 	/* Start at 2 because it's defined as 2^(1+PSS) */
 	iommu->pasid_max = 2 << ecap_pss(iommu->ecap);
 
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a56bab1..8dad3dd 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -83,6 +83,7 @@
 /*
  * Decoding Capability Register
  */
+#define cap_5lp_support(c)	(((c) >> 60) & 1)
 #define cap_pi_support(c)	(((c) >> 59) & 1)
 #define cap_fl1gp_support(c)	(((c) >> 56) & 1)
 #define cap_read_drain(c)	(((c) >> 55) & 1)
-- 
2.7.4

[PATCH 3/4] iommu/vt-d: Add a check for 5-level paging support

[Sohil Mehta]

Add a check to verify IOMMU 5-level paging support. If the CPU supports
supports 5-level paging but the IOMMU does not support it then disable
SVM by not allocating PASID tables.

Signed-off-by: Sohil Mehta <sohil.mehta-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/iommu/intel-svm.c   | 4 ++++
 include/linux/intel-iommu.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index fcab440..4072a18 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -45,6 +45,10 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)
 			!cap_fl1gp_support(iommu->cap))
 		return -EINVAL;
 
+	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			!cap_5lp_support(iommu->cap))
+		return -EINVAL;
+
 	/* Start at 2 because it's defined as 2^(1+PSS) */
 	iommu->pasid_max = 2 << ecap_pss(iommu->ecap);
 
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a56bab1..8dad3dd 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -83,6 +83,7 @@
 /*
  * Decoding Capability Register
  */
+#define cap_5lp_support(c)	(((c) >> 60) & 1)
 #define cap_pi_support(c)	(((c) >> 59) & 1)
 #define cap_fl1gp_support(c)	(((c) >> 56) & 1)
 #define cap_read_drain(c)	(((c) >> 55) & 1)
-- 
2.7.4

[PATCH 4/4] iommu/vt-d: Enable 5-level paging mode in the PASID entry

[Sohil Mehta]

If the CPU has support for 5-level paging enabled and the IOMMU also
supports 5-level paging then enable the 5-level paging mode for first-
level translations - used when SVM is enabled.

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
---
 drivers/iommu/intel-svm.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index 4072a18..d7095dd 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -26,6 +26,10 @@
 #include <linux/interrupt.h>
 #include <asm/page.h>
 
+#define PASID_ENTRY_P		BIT_ULL(0)
+#define PASID_ENTRY_FLPM_5LP	BIT_ULL(9)
+#define PASID_ENTRY_SRE		BIT_ULL(11)
+
 static irqreturn_t prq_event_thread(int irq, void *d);
 
 struct pasid_entry {
@@ -297,6 +301,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 	struct intel_svm_dev *sdev;
 	struct intel_svm *svm = NULL;
 	struct mm_struct *mm = NULL;
+	u64 pasid_entry_val;
 	int pasid_max;
 	int ret;
 
@@ -403,9 +408,15 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 				kfree(sdev);
 				goto out;
 			}
-			iommu->pasid_table[svm->pasid].val = (u64)__pa(mm->pgd) | 1;
+			pasid_entry_val = (u64)__pa(mm->pgd) | PASID_ENTRY_P;
 		} else
-			iommu->pasid_table[svm->pasid].val = (u64)__pa(init_mm.pgd) | 1 | (1ULL << 11);
+			pasid_entry_val = (u64)__pa(init_mm.pgd) |
+					  PASID_ENTRY_P | PASID_ENTRY_SRE;
+		if (cpu_feature_enabled(X86_FEATURE_LA57))
+			pasid_entry_val |= PASID_ENTRY_FLPM_5LP;
+
+		iommu->pasid_table[svm->pasid].val = pasid_entry_val;
+
 		wmb();
 		/* In caching mode, we still have to flush with PASID 0 when
 		 * a PASID table entry becomes present. Not entirely clear
-- 
2.7.4

[PATCH 4/4] iommu/vt-d: Enable 5-level paging mode in the PASID entry

[Sohil Mehta]

If the CPU has support for 5-level paging enabled and the IOMMU also
supports 5-level paging then enable the 5-level paging mode for first-
level translations - used when SVM is enabled.

Signed-off-by: Sohil Mehta <sohil.mehta-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/iommu/intel-svm.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/intel-svm.c b/drivers/iommu/intel-svm.c
index 4072a18..d7095dd 100644
--- a/drivers/iommu/intel-svm.c
+++ b/drivers/iommu/intel-svm.c
@@ -26,6 +26,10 @@
 #include <linux/interrupt.h>
 #include <asm/page.h>
 
+#define PASID_ENTRY_P		BIT_ULL(0)
+#define PASID_ENTRY_FLPM_5LP	BIT_ULL(9)
+#define PASID_ENTRY_SRE		BIT_ULL(11)
+
 static irqreturn_t prq_event_thread(int irq, void *d);
 
 struct pasid_entry {
@@ -297,6 +301,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 	struct intel_svm_dev *sdev;
 	struct intel_svm *svm = NULL;
 	struct mm_struct *mm = NULL;
+	u64 pasid_entry_val;
 	int pasid_max;
 	int ret;
 
@@ -403,9 +408,15 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 				kfree(sdev);
 				goto out;
 			}
-			iommu->pasid_table[svm->pasid].val = (u64)__pa(mm->pgd) | 1;
+			pasid_entry_val = (u64)__pa(mm->pgd) | PASID_ENTRY_P;
 		} else
-			iommu->pasid_table[svm->pasid].val = (u64)__pa(init_mm.pgd) | 1 | (1ULL << 11);
+			pasid_entry_val = (u64)__pa(init_mm.pgd) |
+					  PASID_ENTRY_P | PASID_ENTRY_SRE;
+		if (cpu_feature_enabled(X86_FEATURE_LA57))
+			pasid_entry_val |= PASID_ENTRY_FLPM_5LP;
+
+		iommu->pasid_table[svm->pasid].val = pasid_entry_val;
+
 		wmb();
 		/* In caching mode, we still have to flush with PASID 0 when
 		 * a PASID table entry becomes present. Not entirely clear
-- 
2.7.4

Re: [PATCH 0/4] Add support for Intel IOMMU 5-level paging

[Joerg Roedel]

On Wed, Dec 20, 2017 at 11:59:23AM -0800, Sohil Mehta wrote:
> Sohil Mehta (4):
>   iommu/vt-d: Enable upto 57 bits of domain address width
>   iommu/vt-d: Add a check for 1GB page support
>   iommu/vt-d: Add a check for 5-level paging support
>   iommu/vt-d: Enable 5-level paging mode in the PASID entry
> 
>  drivers/iommu/intel-iommu.c |  2 +-
>  drivers/iommu/intel-svm.c   | 23 +++++++++++++++++++++--
>  include/linux/intel-iommu.h |  2 ++
>  3 files changed, 24 insertions(+), 3 deletions(-)

Applied, thanks.

Re: [PATCH 0/4] Add support for Intel IOMMU 5-level paging

[Joerg Roedel]

On Wed, Dec 20, 2017 at 11:59:23AM -0800, Sohil Mehta wrote:
> Sohil Mehta (4):
>   iommu/vt-d: Enable upto 57 bits of domain address width
>   iommu/vt-d: Add a check for 1GB page support
>   iommu/vt-d: Add a check for 5-level paging support
>   iommu/vt-d: Enable 5-level paging mode in the PASID entry
> 
>  drivers/iommu/intel-iommu.c |  2 +-
>  drivers/iommu/intel-svm.c   | 23 +++++++++++++++++++++--
>  include/linux/intel-iommu.h |  2 ++
>  3 files changed, 24 insertions(+), 3 deletions(-)

Applied, thanks.