[PATCH 0/3] Add pm_runtime and DG2 support

[PATCH 0/3] Add pm_runtime and DG2 support

[David E. Box]

This series adds runtime pm support to the Intel Platform Monitoring
Technology (PMT) driver. This is needed by devices whose slots cannot go to
D3 without power gating the PMT endpoint. To do this, this series also
removes the current binary mmap support since the implementation doesn't
allow for a proper intercept point to perform runtime pm.

David E. Box (3):
  platform/x86/intel: pmt: Remove bin_attribute mmap support to runtime
    pm
  platform/x86/intel: vsec: Enable runtime D3
  platform/x86/intel: vsec: Add DG2 support

 .../ABI/testing/sysfs-class-intel_pmt         | 24 ++++++------
 drivers/platform/x86/intel/pmt/class.c        | 38 ++++---------------
 drivers/platform/x86/intel/pmt/class.h        |  3 ++
 drivers/platform/x86/intel/vsec.c             | 32 ++++++++++++++++
 4 files changed, 54 insertions(+), 43 deletions(-)

-- 
2.25.1

[PATCH 1/3] platform/x86/intel: pmt: Remove bin_attribute mmap support to runtime pm

[David E. Box]

PMT devices need to support runtime D3. However, binary attributes don't
provide access to open/release methods that could be used to control
runtime pm. Therefore, remove the mmap operation. The data may still be
accessed with read() calls.

Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---
V0 comments:

I expect that this is an undesirable solution because of the ABI change.
I don't know if anyone is using this ABI outside of our Intel tools which
are willing to make this change. I'd rather find a solution to keep the
mmap support. I initially wrote a patch to simply add the missing open and
release callbacks to binary attributes but this was thought to be too heavy
handed in our internal review. I'm open to suggestions. Thanks.

David

 .../ABI/testing/sysfs-class-intel_pmt         | 24 +++++++-------
 drivers/platform/x86/intel/pmt/class.c        | 31 -------------------
 2 files changed, 12 insertions(+), 43 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-class-intel_pmt b/Documentation/ABI/testing/sysfs-class-intel_pmt
index ed4c886a21b1..4182f67dcef8 100644
--- a/Documentation/ABI/testing/sysfs-class-intel_pmt
+++ b/Documentation/ABI/testing/sysfs-class-intel_pmt
@@ -15,10 +15,10 @@ Description:
 		The telem<x> directory contains files describing an instance of
 		a PMT telemetry device that exposes hardware telemetry. Each
 		telem<x> directory has an associated telem file. This file
-		may be opened and mapped or read to access the telemetry space
-		of the device. The register layout of the telemetry space is
-		determined from an XML file that matches the PCI device id and
-		GUID for the device.
+		may be opened and read to access the telemetry space of the
+		device. The register layout of the telemetry space is determined
+		from an XML file that matches the PCI device id and GUID for the
+		device.
 
 What:		/sys/class/intel_pmt/telem<x>/telem
 Date:		October 2020
@@ -26,7 +26,7 @@ KernelVersion:	5.10
 Contact:	David Box <david.e.box@linux.intel.com>
 Description:
 		(RO) The telemetry data for this telemetry device. This file
-		may be mapped or read to obtain the data.
+		may be read to obtain the data.
 
 What:		/sys/class/intel_pmt/telem<x>/guid
 Date:		October 2020
@@ -43,7 +43,7 @@ KernelVersion:	5.10
 Contact:	David Box <david.e.box@linux.intel.com>
 Description:
 		(RO) The size of telemetry region in bytes that corresponds to
-		the mapping size for the telem file.
+		the size of the telem file.
 
 What:		/sys/class/intel_pmt/telem<x>/offset
 Date:		October 2020
@@ -51,7 +51,7 @@ KernelVersion:	5.10
 Contact:	David Box <david.e.box@linux.intel.com>
 Description:
 		(RO) The offset of telemetry region in bytes that corresponds to
-		the mapping for the telem file.
+		the size of the telem file.
 
 What:		/sys/class/intel_pmt/crashlog<x>
 Date:		October 2020
@@ -61,10 +61,10 @@ Description:
 		The crashlog<x> directory contains files for configuring an
 		instance of a PMT crashlog device that can perform crash data
 		recording. Each crashlog<x> device has an associated crashlog
-		file. This file can be opened and mapped or read to access the
-		resulting crashlog buffer. The register layout for the buffer
-		can be determined from an XML file of specified GUID for the
-		parent device.
+		file. This file can be opened and read to access the resulting
+		crashlog buffer. The register layout for the buffer can be
+		determined from an XML file of specified GUID for the parent
+		device.
 
 What:		/sys/class/intel_pmt/crashlog<x>/crashlog
 Date:		October 2020
@@ -72,7 +72,7 @@ KernelVersion:	5.10
 Contact:	David Box <david.e.box@linux.intel.com>
 Description:
 		(RO) The crashlog buffer for this crashlog device. This file
-		may be mapped or read to obtain the data.
+		may be read to obtain the data.
 
 What:		/sys/class/intel_pmt/crashlog<x>/guid
 Date:		October 2020
diff --git a/drivers/platform/x86/intel/pmt/class.c b/drivers/platform/x86/intel/pmt/class.c
index 1c9e3f3ea41c..85fc159961c1 100644
--- a/drivers/platform/x86/intel/pmt/class.c
+++ b/drivers/platform/x86/intel/pmt/class.c
@@ -68,36 +68,6 @@ intel_pmt_read(struct file *filp, struct kobject *kobj,
 	return count;
 }
 
-static int
-intel_pmt_mmap(struct file *filp, struct kobject *kobj,
-		struct bin_attribute *attr, struct vm_area_struct *vma)
-{
-	struct intel_pmt_entry *entry = container_of(attr,
-						     struct intel_pmt_entry,
-						     pmt_bin_attr);
-	unsigned long vsize = vma->vm_end - vma->vm_start;
-	struct device *dev = kobj_to_dev(kobj);
-	unsigned long phys = entry->base_addr;
-	unsigned long pfn = PFN_DOWN(phys);
-	unsigned long psize;
-
-	if (vma->vm_flags & (VM_WRITE | VM_MAYWRITE))
-		return -EROFS;
-
-	psize = (PFN_UP(entry->base_addr + entry->size) - pfn) * PAGE_SIZE;
-	if (vsize > psize) {
-		dev_err(dev, "Requested mmap size is too large\n");
-		return -EINVAL;
-	}
-
-	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
-	if (io_remap_pfn_range(vma, vma->vm_start, pfn,
-		vsize, vma->vm_page_prot))
-		return -EAGAIN;
-
-	return 0;
-}
-
 static ssize_t
 guid_show(struct device *dev, struct device_attribute *attr, char *buf)
 {
@@ -263,7 +233,6 @@ static int intel_pmt_dev_register(struct intel_pmt_entry *entry,
 	sysfs_bin_attr_init(&entry->pmt_bin_attr);
 	entry->pmt_bin_attr.attr.name = ns->name;
 	entry->pmt_bin_attr.attr.mode = 0440;
-	entry->pmt_bin_attr.mmap = intel_pmt_mmap;
 	entry->pmt_bin_attr.read = intel_pmt_read;
 	entry->pmt_bin_attr.size = entry->size;
 
-- 
2.25.1

[PATCH 2/3] platform/x86/intel: vsec: Enable runtime D3

[David E. Box]

Add pm_runtime helpers to allow the PCI device to go to D3 when
not in use.

Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---
 drivers/platform/x86/intel/pmt/class.c |  5 +++++
 drivers/platform/x86/intel/pmt/class.h |  3 +++
 drivers/platform/x86/intel/vsec.c      | 23 +++++++++++++++++++++++
 3 files changed, 31 insertions(+)

diff --git a/drivers/platform/x86/intel/pmt/class.c b/drivers/platform/x86/intel/pmt/class.c
index 85fc159961c1..a3ec09fe2f38 100644
--- a/drivers/platform/x86/intel/pmt/class.c
+++ b/drivers/platform/x86/intel/pmt/class.c
@@ -12,6 +12,7 @@
 #include <linux/module.h>
 #include <linux/mm.h>
 #include <linux/pci.h>
+#include <linux/pm_runtime.h>
 
 #include "../vsec.h"
 #include "class.h"
@@ -63,7 +64,10 @@ intel_pmt_read(struct file *filp, struct kobject *kobj,
 	if (count > entry->size - off)
 		count = entry->size - off;
 
+	pm_runtime_get_sync(&entry->pdev->dev);
 	memcpy_fromio(buf, entry->base + off, count);
+	pm_runtime_mark_last_busy(&entry->pdev->dev);
+	pm_runtime_put_autosuspend(&entry->pdev->dev);
 
 	return count;
 }
@@ -209,6 +213,7 @@ static int intel_pmt_dev_register(struct intel_pmt_entry *entry,
 	}
 
 	entry->kobj = &dev->kobj;
+	entry->pdev = to_pci_dev(parent->parent);
 
 	if (ns->attr_grp) {
 		ret = sysfs_create_group(entry->kobj, ns->attr_grp);
diff --git a/drivers/platform/x86/intel/pmt/class.h b/drivers/platform/x86/intel/pmt/class.h
index db11d58867ce..b4df7ee91c51 100644
--- a/drivers/platform/x86/intel/pmt/class.h
+++ b/drivers/platform/x86/intel/pmt/class.h
@@ -18,9 +18,12 @@
 #define GET_BIR(v)		((v) & GENMASK(2, 0))
 #define GET_ADDRESS(v)		((v) & GENMASK(31, 3))
 
+struct pci_dev;
+
 struct intel_pmt_entry {
 	struct bin_attribute	pmt_bin_attr;
 	struct kobject		*kobj;
+	struct pci_dev		*pdev;
 	void __iomem		*disc_table;
 	void __iomem		*base;
 	unsigned long		base_addr;
diff --git a/drivers/platform/x86/intel/vsec.c b/drivers/platform/x86/intel/vsec.c
index c3bdd75ed690..d182122c261d 100644
--- a/drivers/platform/x86/intel/vsec.c
+++ b/drivers/platform/x86/intel/vsec.c
@@ -19,6 +19,7 @@
 #include <linux/idr.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <linux/pm_runtime.h>
 #include <linux/types.h>
 
 #include "vsec.h"
@@ -355,9 +356,21 @@ static int intel_vsec_pci_probe(struct pci_dev *pdev, const struct pci_device_id
 	if (!have_devices)
 		return -ENODEV;
 
+	pm_runtime_put(&pdev->dev);
+	pm_runtime_use_autosuspend(&pdev->dev);
+	pm_runtime_set_autosuspend_delay(&pdev->dev, 1000);
+	pm_runtime_allow(&pdev->dev);
+
 	return 0;
 }
 
+static void intel_vsec_pci_remove(struct pci_dev *pdev)
+{
+	pm_runtime_forbid(&pdev->dev);
+	pm_runtime_dont_use_autosuspend(&pdev->dev);
+	pm_runtime_get(&pdev->dev);
+}
+
 /* TGL info */
 static const struct intel_vsec_platform_info tgl_info = {
 	.quirks = VSEC_QUIRK_NO_WATCHER | VSEC_QUIRK_NO_CRASHLOG | VSEC_QUIRK_TABLE_SHIFT,
@@ -383,6 +396,10 @@ static const struct intel_vsec_platform_info dg1_info = {
 	.quirks = VSEC_QUIRK_NO_DVSEC,
 };
 
+#ifdef CONFIG_PM_SLEEP
+static const struct dev_pm_ops intel_vsec_pm_ops = {};
+#endif
+
 #define PCI_DEVICE_ID_INTEL_VSEC_ADL		0x467d
 #define PCI_DEVICE_ID_INTEL_VSEC_DG1		0x490e
 #define PCI_DEVICE_ID_INTEL_VSEC_OOBMSM		0x09a7
@@ -400,6 +417,12 @@ static struct pci_driver intel_vsec_pci_driver = {
 	.name = "intel_vsec",
 	.id_table = intel_vsec_pci_ids,
 	.probe = intel_vsec_pci_probe,
+	.remove = intel_vsec_pci_remove,
+	.driver = {
+#ifdef CONFIG_PM_SLEEP
+		.pm = &intel_vsec_pm_ops,
+#endif
+	},
 };
 module_pci_driver(intel_vsec_pci_driver);
 
-- 
2.25.1

[PATCH 3/3] platform/x86/intel: vsec: Add DG2 support

[David E. Box]

Add Platform Monitoring Technology support for DG2 platforms.

Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---
 drivers/platform/x86/intel/pmt/class.c | 2 ++
 drivers/platform/x86/intel/vsec.c      | 9 +++++++++
 2 files changed, 11 insertions(+)

diff --git a/drivers/platform/x86/intel/pmt/class.c b/drivers/platform/x86/intel/pmt/class.c
index a3ec09fe2f38..d25fffb2103d 100644
--- a/drivers/platform/x86/intel/pmt/class.c
+++ b/drivers/platform/x86/intel/pmt/class.c
@@ -31,6 +31,8 @@
 static const struct pci_device_id pmt_telem_early_client_pci_ids[] = {
 	{ PCI_VDEVICE(INTEL, 0x467d) }, /* ADL */
 	{ PCI_VDEVICE(INTEL, 0x490e) }, /* DG1 */
+	{ PCI_VDEVICE(INTEL, 0x4f93) }, /* DG2_G10 */
+	{ PCI_VDEVICE(INTEL, 0x4f95) }, /* DG2_G11 */
 	{ PCI_VDEVICE(INTEL, 0x9a0d) }, /* TGL */
 	{ }
 };
diff --git a/drivers/platform/x86/intel/vsec.c b/drivers/platform/x86/intel/vsec.c
index d182122c261d..f256c7ca5452 100644
--- a/drivers/platform/x86/intel/vsec.c
+++ b/drivers/platform/x86/intel/vsec.c
@@ -396,17 +396,26 @@ static const struct intel_vsec_platform_info dg1_info = {
 	.quirks = VSEC_QUIRK_NO_DVSEC,
 };
 
+/* DG2 info */
+static const struct intel_vsec_platform_info dg2_info = {
+	.quirks = VSEC_QUIRK_TABLE_SHIFT
+};
+
 #ifdef CONFIG_PM_SLEEP
 static const struct dev_pm_ops intel_vsec_pm_ops = {};
 #endif
 
 #define PCI_DEVICE_ID_INTEL_VSEC_ADL		0x467d
 #define PCI_DEVICE_ID_INTEL_VSEC_DG1		0x490e
+#define PCI_DEVICE_ID_INTEL_VSEC_DG2_G10	0x4f93
+#define PCI_DEVICE_ID_INTEL_VSEC_DG2_G11	0x4f95
 #define PCI_DEVICE_ID_INTEL_VSEC_OOBMSM		0x09a7
 #define PCI_DEVICE_ID_INTEL_VSEC_TGL		0x9a0d
 static const struct pci_device_id intel_vsec_pci_ids[] = {
 	{ PCI_DEVICE_DATA(INTEL, VSEC_ADL, &tgl_info) },
 	{ PCI_DEVICE_DATA(INTEL, VSEC_DG1, &dg1_info) },
+	{ PCI_DEVICE_DATA(INTEL, VSEC_DG2_G10, &dg2_info) },
+	{ PCI_DEVICE_DATA(INTEL, VSEC_DG2_G11, &dg2_info) },
 	{ PCI_DEVICE_DATA(INTEL, VSEC_OOBMSM, NULL) },
 	{ PCI_DEVICE_DATA(INTEL, VSEC_TGL, &tgl_info) },
 	{ }
-- 
2.25.1

Re: [PATCH 1/3] platform/x86/intel: pmt: Remove bin_attribute mmap support to runtime pm

[Hans de Goede]

Hi,

On 2/14/22 22:32, David E. Box wrote:
> PMT devices need to support runtime D3. However, binary attributes don't
> provide access to open/release methods that could be used to control
> runtime pm. Therefore, remove the mmap operation. The data may still be
> accessed with read() calls.
> 
> Signed-off-by: David E. Box <david.e.box@linux.intel.com>
> ---
> V0 comments:
> 
> I expect that this is an undesirable solution because of the ABI change.
> I don't know if anyone is using this ABI outside of our Intel tools which
> are willing to make this change. I'd rather find a solution to keep the
> mmap support. I initially wrote a patch to simply add the missing open and
> release callbacks to binary attributes but this was thought to be too heavy
> handed in our internal review. I'm open to suggestions. Thanks.

We really cannot go and break userspace API like this. Even if you are
dropping mmap support from the Intel tools; and we assume that the Intel
tools are the only consumer, then we still cannot drop mmap support
because users may install a new kernel without updating the tools.

The never break userspace rule applies here and that is a very clear
and hard rule.

So please respin the series using the approach with open and release
callbacks.

If you want to get rid of mmap in the future you need to follow the
official deprecation process:

1. Announce that mmap is going away
2. Move the ABI doc with the mmap support to
   Documentation/ABI/obsolete with a note explaining what is being
   removed and why it is being removed. Since you are only removing mmap you
   will want to keep the testing version with mmap removed and have the 2
   point to each other
3. Wait at least 1 year until after a kernel with the docs in the obsolete dir
   has been released
4. Remove the mmap API and move the Documentation/ABI/obsolete version
   of the ABI doc to Documentation/ABI/removed

Regards,

Hans






>  .../ABI/testing/sysfs-class-intel_pmt         | 24 +++++++-------
>  drivers/platform/x86/intel/pmt/class.c        | 31 -------------------
>  2 files changed, 12 insertions(+), 43 deletions(-)
> 
> diff --git a/Documentation/ABI/testing/sysfs-class-intel_pmt b/Documentation/ABI/testing/sysfs-class-intel_pmt
> index ed4c886a21b1..4182f67dcef8 100644
> --- a/Documentation/ABI/testing/sysfs-class-intel_pmt
> +++ b/Documentation/ABI/testing/sysfs-class-intel_pmt
> @@ -15,10 +15,10 @@ Description:
>  		The telem<x> directory contains files describing an instance of
>  		a PMT telemetry device that exposes hardware telemetry. Each
>  		telem<x> directory has an associated telem file. This file
> -		may be opened and mapped or read to access the telemetry space
> -		of the device. The register layout of the telemetry space is
> -		determined from an XML file that matches the PCI device id and
> -		GUID for the device.
> +		may be opened and read to access the telemetry space of the
> +		device. The register layout of the telemetry space is determined
> +		from an XML file that matches the PCI device id and GUID for the
> +		device.
>  
>  What:		/sys/class/intel_pmt/telem<x>/telem
>  Date:		October 2020
> @@ -26,7 +26,7 @@ KernelVersion:	5.10
>  Contact:	David Box <david.e.box@linux.intel.com>
>  Description:
>  		(RO) The telemetry data for this telemetry device. This file
> -		may be mapped or read to obtain the data.
> +		may be read to obtain the data.
>  
>  What:		/sys/class/intel_pmt/telem<x>/guid
>  Date:		October 2020
> @@ -43,7 +43,7 @@ KernelVersion:	5.10
>  Contact:	David Box <david.e.box@linux.intel.com>
>  Description:
>  		(RO) The size of telemetry region in bytes that corresponds to
> -		the mapping size for the telem file.
> +		the size of the telem file.
>  
>  What:		/sys/class/intel_pmt/telem<x>/offset
>  Date:		October 2020
> @@ -51,7 +51,7 @@ KernelVersion:	5.10
>  Contact:	David Box <david.e.box@linux.intel.com>
>  Description:
>  		(RO) The offset of telemetry region in bytes that corresponds to
> -		the mapping for the telem file.
> +		the size of the telem file.
>  
>  What:		/sys/class/intel_pmt/crashlog<x>
>  Date:		October 2020
> @@ -61,10 +61,10 @@ Description:
>  		The crashlog<x> directory contains files for configuring an
>  		instance of a PMT crashlog device that can perform crash data
>  		recording. Each crashlog<x> device has an associated crashlog
> -		file. This file can be opened and mapped or read to access the
> -		resulting crashlog buffer. The register layout for the buffer
> -		can be determined from an XML file of specified GUID for the
> -		parent device.
> +		file. This file can be opened and read to access the resulting
> +		crashlog buffer. The register layout for the buffer can be
> +		determined from an XML file of specified GUID for the parent
> +		device.
>  
>  What:		/sys/class/intel_pmt/crashlog<x>/crashlog
>  Date:		October 2020
> @@ -72,7 +72,7 @@ KernelVersion:	5.10
>  Contact:	David Box <david.e.box@linux.intel.com>
>  Description:
>  		(RO) The crashlog buffer for this crashlog device. This file
> -		may be mapped or read to obtain the data.
> +		may be read to obtain the data.
>  
>  What:		/sys/class/intel_pmt/crashlog<x>/guid
>  Date:		October 2020
> diff --git a/drivers/platform/x86/intel/pmt/class.c b/drivers/platform/x86/intel/pmt/class.c
> index 1c9e3f3ea41c..85fc159961c1 100644
> --- a/drivers/platform/x86/intel/pmt/class.c
> +++ b/drivers/platform/x86/intel/pmt/class.c
> @@ -68,36 +68,6 @@ intel_pmt_read(struct file *filp, struct kobject *kobj,
>  	return count;
>  }
>  
> -static int
> -intel_pmt_mmap(struct file *filp, struct kobject *kobj,
> -		struct bin_attribute *attr, struct vm_area_struct *vma)
> -{
> -	struct intel_pmt_entry *entry = container_of(attr,
> -						     struct intel_pmt_entry,
> -						     pmt_bin_attr);
> -	unsigned long vsize = vma->vm_end - vma->vm_start;
> -	struct device *dev = kobj_to_dev(kobj);
> -	unsigned long phys = entry->base_addr;
> -	unsigned long pfn = PFN_DOWN(phys);
> -	unsigned long psize;
> -
> -	if (vma->vm_flags & (VM_WRITE | VM_MAYWRITE))
> -		return -EROFS;
> -
> -	psize = (PFN_UP(entry->base_addr + entry->size) - pfn) * PAGE_SIZE;
> -	if (vsize > psize) {
> -		dev_err(dev, "Requested mmap size is too large\n");
> -		return -EINVAL;
> -	}
> -
> -	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> -	if (io_remap_pfn_range(vma, vma->vm_start, pfn,
> -		vsize, vma->vm_page_prot))
> -		return -EAGAIN;
> -
> -	return 0;
> -}
> -
>  static ssize_t
>  guid_show(struct device *dev, struct device_attribute *attr, char *buf)
>  {
> @@ -263,7 +233,6 @@ static int intel_pmt_dev_register(struct intel_pmt_entry *entry,
>  	sysfs_bin_attr_init(&entry->pmt_bin_attr);
>  	entry->pmt_bin_attr.attr.name = ns->name;
>  	entry->pmt_bin_attr.attr.mode = 0440;
> -	entry->pmt_bin_attr.mmap = intel_pmt_mmap;
>  	entry->pmt_bin_attr.read = intel_pmt_read;
>  	entry->pmt_bin_attr.size = entry->size;
>  

Re: [PATCH 1/3] platform/x86/intel: pmt: Remove bin_attribute mmap support to runtime pm

[David E. Box]

On Mon, 2022-02-21 at 10:05 +0100, Hans de Goede wrote:
> Hi,
> 
> On 2/14/22 22:32, David E. Box wrote:
> > PMT devices need to support runtime D3. However, binary attributes
> > don't
> > provide access to open/release methods that could be used to
> > control
> > runtime pm. Therefore, remove the mmap operation. The data may
> > still be
> > accessed with read() calls.
> > 
> > Signed-off-by: David E. Box <david.e.box@linux.intel.com>
> > ---
> > V0 comments:
> > 
> > I expect that this is an undesirable solution because of the ABI
> > change.
> > I don't know if anyone is using this ABI outside of our Intel tools
> > which
> > are willing to make this change. I'd rather find a solution to keep
> > the
> > mmap support. I initially wrote a patch to simply add the missing
> > open and
> > release callbacks to binary attributes but this was thought to be
> > too heavy
> > handed in our internal review. I'm open to suggestions. Thanks.
> 
> We really cannot go and break userspace API like this. Even if you
> are
> dropping mmap support from the Intel tools; and we assume that the
> Intel
> tools are the only consumer, then we still cannot drop mmap support
> because users may install a new kernel without updating the tools.
> 
> The never break userspace rule applies here and that is a very clear
> and hard rule.
> 
> So please respin the series using the approach with open and release
> callbacks.

Thanks Hans. I'll send out that series for comment.

David