[PATCH v2 0/2] Add page alignment check in Intel IOMMU.

[PATCH v2 0/2] Add page alignment check in Intel IOMMU.

[sathyanarayanan.kuppuswamy]

From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

As per Intel vt-d specification, Rev 3.0 (section 7.5.1.1, title
"Page Request Descriptor"), Intel IOMMU page request descriptor
only uses bits[63:12] of the Page Address. Hence its required to
enforce that the device will only send page request with
page-aligned address. So, this patch set adds support to verify
whether the device uses page aligned address before enabling the
ATS service in Intel IOMMU driver.

Changes since v1:
 * Fixed issue with PCI_ATS_CAP_PAGE_ALIGNED macro.
 * Fixed comments.

Kuppuswamy Sathyanarayanan (2):
  PCI/ATS: Add pci_ats_page_aligned() interface
  iommu/vt-d: Enable ATS only if the device uses page aligned address.

 drivers/iommu/intel-iommu.c   |  1 +
 drivers/pci/ats.c             | 27 +++++++++++++++++++++++++++
 include/linux/pci.h           |  2 ++
 include/uapi/linux/pci_regs.h |  1 +
 4 files changed, 31 insertions(+)

-- 
2.20.1

[PATCH v2 1/2] PCI/ATS: Add pci_ats_page_aligned() interface

[sathyanarayanan.kuppuswamy]

From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

Return the Page Aligned Request bit in the ATS Capability Register.

As per PCIe spec r4.0, sec 10.5.1.2, If Page Aligned Request bit is
set, then it indicates the Untranslated Addresses generated by the
device are alwayis always aligned to a 4096 byte boundary.

This interface will be used by drivers like IOMMU, if it is required
to check whether the Untranslated Address generated by the device will
be aligned before enabling the ATS service.

Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
---
 drivers/pci/ats.c             | 27 +++++++++++++++++++++++++++
 include/linux/pci.h           |  2 ++
 include/uapi/linux/pci_regs.h |  1 +
 3 files changed, 30 insertions(+)

diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
index 5b78f3b1b918..b3c7f1496081 100644
--- a/drivers/pci/ats.c
+++ b/drivers/pci/ats.c
@@ -142,6 +142,33 @@ int pci_ats_queue_depth(struct pci_dev *dev)
 }
 EXPORT_SYMBOL_GPL(pci_ats_queue_depth);
 
+/**
+ * pci_ats_page_aligned - Return Page Aligned Request bit status.
+ * @pdev: the PCI device
+ *
+ * Returns 1, if Untranslated Addresses generated by the device are
+ * always aligned or 0 otherwise.
+ *
+ * Per PCIe spec r4.0, sec 10.5.1.2, If Page Aligned Request bit is
+ * set, it indicates the Untranslated Addresses generated by the
+ * device are always aligned to a 4096 byte boundary.
+ */
+int pci_ats_page_aligned(struct pci_dev *pdev)
+{
+	u16 cap;
+
+	if (!pdev->ats_cap)
+		return 0;
+
+	pci_read_config_word(pdev, pdev->ats_cap + PCI_ATS_CAP, &cap);
+
+	if (cap & PCI_ATS_CAP_PAGE_ALIGNED)
+		return 1;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(pci_ats_page_aligned);
+
 #ifdef CONFIG_PCI_PRI
 /**
  * pci_enable_pri - Enable PRI capability
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 65f1d8c2f082..9724a8c0496b 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1524,11 +1524,13 @@ void pci_ats_init(struct pci_dev *dev);
 int pci_enable_ats(struct pci_dev *dev, int ps);
 void pci_disable_ats(struct pci_dev *dev);
 int pci_ats_queue_depth(struct pci_dev *dev);
+int pci_ats_page_aligned(struct pci_dev *dev);
 #else
 static inline void pci_ats_init(struct pci_dev *d) { }
 static inline int pci_enable_ats(struct pci_dev *d, int ps) { return -ENODEV; }
 static inline void pci_disable_ats(struct pci_dev *d) { }
 static inline int pci_ats_queue_depth(struct pci_dev *d) { return -ENODEV; }
+static inline int pci_ats_page_aligned(struct pci_dev *dev) { return 0; }
 #endif
 
 #ifdef CONFIG_PCIE_PTM
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index e1e9888c85e6..7973bb02ed4b 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -866,6 +866,7 @@
 #define PCI_ATS_CAP		0x04	/* ATS Capability Register */
 #define  PCI_ATS_CAP_QDEP(x)	((x) & 0x1f)	/* Invalidate Queue Depth */
 #define  PCI_ATS_MAX_QDEP	32	/* Max Invalidate Queue Depth */
+#define  PCI_ATS_CAP_PAGE_ALIGNED	0x0020 /* Page Aligned Request */
 #define PCI_ATS_CTRL		0x06	/* ATS Control Register */
 #define  PCI_ATS_CTRL_ENABLE	0x8000	/* ATS Enable */
 #define  PCI_ATS_CTRL_STU(x)	((x) & 0x1f)	/* Smallest Translation Unit */
-- 
2.20.1

Re: [PATCH v2 1/2] PCI/ATS: Add pci_ats_page_aligned() interface

[Bjorn Helgaas]

On Mon, Feb 11, 2019 at 01:44:34PM -0800, sathyanarayanan.kuppuswamy@linux.intel.com wrote:
> From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
> 
> Return the Page Aligned Request bit in the ATS Capability Register.
> 
> As per PCIe spec r4.0, sec 10.5.1.2, If Page Aligned Request bit is
> set, then it indicates the Untranslated Addresses generated by the
> device are alwayis always aligned to a 4096 byte boundary.

s/, If/, if the/
s/then it/it/
s/alwayis//

> This interface will be used by drivers like IOMMU, if it is required
> to check whether the Untranslated Address generated by the device will
> be aligned before enabling the ATS service.

Maybe something like this?

  An IOMMU that can only translate page-aligned addresses can only be used
  with devices that always produce aligned Untranslated Addresses.  This
  interface will be used by drivers for such IOMMUs to determine whether
  devices can use the ATS service.

> Cc: Ashok Raj <ashok.raj@intel.com>
> Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Cc: Keith Busch <keith.busch@intel.com>
> Suggested-by: Ashok Raj <ashok.raj@intel.com>
> Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

With typos addressed (more below),

Acked-by: Bjorn Helgaas <bhelgaas@google.com>

> ---
>  drivers/pci/ats.c             | 27 +++++++++++++++++++++++++++
>  include/linux/pci.h           |  2 ++
>  include/uapi/linux/pci_regs.h |  1 +
>  3 files changed, 30 insertions(+)
> 
> diff --git a/drivers/pci/ats.c b/drivers/pci/ats.c
> index 5b78f3b1b918..b3c7f1496081 100644
> --- a/drivers/pci/ats.c
> +++ b/drivers/pci/ats.c
> @@ -142,6 +142,33 @@ int pci_ats_queue_depth(struct pci_dev *dev)
>  }
>  EXPORT_SYMBOL_GPL(pci_ats_queue_depth);
>  
> +/**
> + * pci_ats_page_aligned - Return Page Aligned Request bit status.
> + * @pdev: the PCI device
> + *
> + * Returns 1, if Untranslated Addresses generated by the device are
> + * always aligned or 0 otherwise.
> + *
> + * Per PCIe spec r4.0, sec 10.5.1.2, If Page Aligned Request bit is
> + * set, it indicates the Untranslated Addresses generated by the
> + * device are always aligned to a 4096 byte boundary.

s/, If/, if the/

> + */
> +int pci_ats_page_aligned(struct pci_dev *pdev)
> +{
> +	u16 cap;
> +
> +	if (!pdev->ats_cap)
> +		return 0;
> +
> +	pci_read_config_word(pdev, pdev->ats_cap + PCI_ATS_CAP, &cap);
> +
> +	if (cap & PCI_ATS_CAP_PAGE_ALIGNED)
> +		return 1;
> +
> +	return 0;
> +}
> +EXPORT_SYMBOL_GPL(pci_ats_page_aligned);
> +
>  #ifdef CONFIG_PCI_PRI
>  /**
>   * pci_enable_pri - Enable PRI capability
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 65f1d8c2f082..9724a8c0496b 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1524,11 +1524,13 @@ void pci_ats_init(struct pci_dev *dev);
>  int pci_enable_ats(struct pci_dev *dev, int ps);
>  void pci_disable_ats(struct pci_dev *dev);
>  int pci_ats_queue_depth(struct pci_dev *dev);
> +int pci_ats_page_aligned(struct pci_dev *dev);
>  #else
>  static inline void pci_ats_init(struct pci_dev *d) { }
>  static inline int pci_enable_ats(struct pci_dev *d, int ps) { return -ENODEV; }
>  static inline void pci_disable_ats(struct pci_dev *d) { }
>  static inline int pci_ats_queue_depth(struct pci_dev *d) { return -ENODEV; }
> +static inline int pci_ats_page_aligned(struct pci_dev *dev) { return 0; }
>  #endif
>  
>  #ifdef CONFIG_PCIE_PTM
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index e1e9888c85e6..7973bb02ed4b 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -866,6 +866,7 @@
>  #define PCI_ATS_CAP		0x04	/* ATS Capability Register */
>  #define  PCI_ATS_CAP_QDEP(x)	((x) & 0x1f)	/* Invalidate Queue Depth */
>  #define  PCI_ATS_MAX_QDEP	32	/* Max Invalidate Queue Depth */
> +#define  PCI_ATS_CAP_PAGE_ALIGNED	0x0020 /* Page Aligned Request */
>  #define PCI_ATS_CTRL		0x06	/* ATS Control Register */
>  #define  PCI_ATS_CTRL_ENABLE	0x8000	/* ATS Enable */
>  #define  PCI_ATS_CTRL_STU(x)	((x) & 0x1f)	/* Smallest Translation Unit */
> -- 
> 2.20.1
> 

[PATCH v2 2/2] iommu/vt-d: Enable ATS only if the device uses page aligned address.

[sathyanarayanan.kuppuswamy]

From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

As per Intel vt-d specification, Rev 3.0 (section 7.5.1.1, title "Page
Request Descriptor"), Intel IOMMU page request descriptor only uses
bits[63:12] of the page address. Hence Intel IOMMU driver would only
permit devices that advertise they would only send page aligned requests
to participate in ATS service.

Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Suggested-by: Ashok Raj <ashok.raj@intel.com>
Signed-off-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
---
 drivers/iommu/intel-iommu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 1457f931218e..6a0b5270cd2e 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1406,6 +1406,7 @@ static void iommu_enable_dev_iotlb(struct device_domain_info *info)
 		info->pri_enabled = 1;
 #endif
 	if (!pdev->untrusted && info->ats_supported &&
+	    pci_ats_page_aligned(pdev) &&
 	    !pci_enable_ats(pdev, VTD_PAGE_SHIFT)) {
 		info->ats_enabled = 1;
 		domain_update_iotlb(info->domain);
-- 
2.20.1

[PATCH v2 0/2] Add page alignment check in Intel IOMMU.

[sathyanarayanan.kuppuswamy]

From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>

As per Intel vt-d specification, Rev 3.0 (section 7.5.1.1, title "Page Request Descriptor"), Intel IOMMU Page Request Descriptor only provides bits[63:12] of the page address. Hence its required to enforce that the device will only send page request with page-aligned address. So, this patch set adds support to verify whether the device uses page aligned address before enabling the ATS service in Intel IOMMU driver.

Kuppuswamy Sathyanarayanan (2):
  PCI: ATS: Add function to check ATS page alignment status.
  iommu/vt-d: Enable ATS only if the device uses page aligned address.

 drivers/iommu/intel-iommu.c   |  1 +
 drivers/pci/ats.c             | 24 ++++++++++++++++++++++++
 include/linux/pci.h           |  2 ++
 include/uapi/linux/pci_regs.h |  1 +
 4 files changed, 28 insertions(+)

-- 
2.20.1

Re: [PATCH v2 0/2] Add page alignment check in Intel IOMMU.

[sathyanarayanan kuppuswamy]

On 2/11/19 1:43 PM, sathyanarayanan.kuppuswamy@linux.intel.com wrote:
> From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
>
> As per Intel vt-d specification, Rev 3.0 (section 7.5.1.1, title "Page Request Descriptor"), Intel IOMMU Page Request Descriptor only provides bits[63:12] of the page address. Hence its required to enforce that the device will only send page request with page-aligned address. So, this patch set adds support to verify whether the device uses page aligned address before enabling the ATS service in Intel IOMMU driver.
Please ignore this mail. This is my older copy.
>
> Kuppuswamy Sathyanarayanan (2):
>    PCI: ATS: Add function to check ATS page alignment status.
>    iommu/vt-d: Enable ATS only if the device uses page aligned address.
>
>   drivers/iommu/intel-iommu.c   |  1 +
>   drivers/pci/ats.c             | 24 ++++++++++++++++++++++++
>   include/linux/pci.h           |  2 ++
>   include/uapi/linux/pci_regs.h |  1 +
>   4 files changed, 28 insertions(+)
>
-- 
Sathyanarayanan Kuppuswamy
Linux kernel developer