Linux-ACPI Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 0/2] Add support for StorageD3Enable _DSD property
@ 2020-04-28  0:32 David E. Box
  2020-04-28  0:32 ` [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support David E. Box
                   ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: David E. Box @ 2020-04-28  0:32 UTC (permalink / raw)
  To: rjw, lenb, bhelgaas, kbusch, axboe, hch, sagi
  Cc: David E. Box, linux-acpi, linux-pci, linux-nvme, linux-kernel

NVMe storage power management during suspend-to-idle, particularly on
laptops, has been inconsistent with some devices working with D3 while
others must rely on NVMe APST in order for power savings to be realized.
Currently the default is to use APST unless quirked to do otherwise.
However newer platforms, like Intel Comet Lake systems, may require NVMe
drives to use D3 in order for the PCIe ports to be properly power managed.
To make it easier for drivers to choose, these platforms may supply a
special "StorageD3Enable" _DSD property under the root port that the device
is attached to. If supplied, the driver must use D3 in order for the
platform to realize the deepest power savings in suspend-to-idle.
    
The first patch adds the new _DSD GUID and fowards the property through the
pci/acpi layer to the pci device.

The second patch adds support for the property to the nvme driver.

David E. Box (2):
  pci: Add ACPI StorageD3Enable _DSD support
  drivers/nvme: Add support for ACPI StorageD3Enable property

 drivers/acpi/property.c |  3 +++
 drivers/nvme/host/pci.c |  7 ++++++
 drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 6 files changed, 68 insertions(+)

-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support
  2020-04-28  0:32 [PATCH 0/2] Add support for StorageD3Enable _DSD property David E. Box
@ 2020-04-28  0:32 ` David E. Box
  2020-05-18 12:34   ` Rafael J. Wysocki
  2020-04-28  0:32 ` [PATCH 2/2] drivers/nvme: Add support for ACPI StorageD3Enable property David E. Box
  2020-04-28  5:13 ` [PATCH 0/2] Add support for StorageD3Enable _DSD property Christoph Hellwig
  2 siblings, 1 reply; 17+ messages in thread
From: David E. Box @ 2020-04-28  0:32 UTC (permalink / raw)
  To: rjw, lenb, bhelgaas, kbusch, axboe, hch, sagi
  Cc: David E. Box, linux-acpi, linux-pci, linux-nvme, linux-kernel,
	Dan Williams

NVMe storage power management during suspend-to-idle, particularly on
laptops, has been inconsistent with some devices working with D3 while
others must rely on NVMe APST in order for power savings to be realized.
Currently the default is to use APST unless quirked to do otherwise.
However newer platforms, like Intel Comet Lake systems, may require NVMe
drives to use D3 in order for the PCIe ports to be properly power managed.
To make it easier for drivers to choose, these platforms may supply a
special "StorageD3Enable" _DSD property under the root port that the device
is attached to. If supplied, the driver must use D3 in order for the
platform to realize the deepest power savings in suspend-to-idle.

Adds support for the _DSD to the pci/acpi layer.

Acked-by: Dan Williams <dan.j.williams@intel.com>
Link: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---
 drivers/acpi/property.c |  3 +++
 drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
 drivers/pci/pci.c       |  6 ++++++
 drivers/pci/pci.h       |  4 ++++
 include/linux/pci.h     |  1 +
 5 files changed, 61 insertions(+)

diff --git a/drivers/acpi/property.c b/drivers/acpi/property.c
index e601c4511a8b..f09375ab40e4 100644
--- a/drivers/acpi/property.c
+++ b/drivers/acpi/property.c
@@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
 	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
 	GUID_INIT(0x6c501103, 0xc189, 0x4296,
 		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
+	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-99a5189762d0 */
+	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
+		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
 };
 
 /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
index d21969fba6ab..5df249ebf022 100644
--- a/drivers/pci/pci-acpi.c
+++ b/drivers/pci/pci-acpi.c
@@ -972,6 +972,52 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
 	return val == 1;
 }
 
+static bool acpi_pci_storage_d3(struct pci_dev *dev)
+{
+	const struct fwnode_handle *fwnode;
+	struct acpi_device *adev;
+	struct pci_dev *root;
+	acpi_handle handle;
+	acpi_status status;
+	u8 val;
+
+	/*
+	 * Look for _DSD property specifying that the storage device on
+	 * the port must use D3 to support deep platform power savings during
+	 * suspend-to-idle
+	 */
+	root = pci_find_pcie_root_port(dev);
+	if (!root)
+		return false;
+
+	adev = ACPI_COMPANION(&root->dev);
+	if (root == dev) {
+		/*
+		 * It is possible that the ACPI companion is not yet bound
+		 * for the root port so look it up manually here.
+		 */
+		if (!adev && !pci_dev_is_added(root))
+			adev = acpi_pci_find_companion(&root->dev);
+	}
+
+	if (!adev)
+		return false;
+
+	status = acpi_get_handle(adev->handle, "PXSX", &handle);
+	if (ACPI_FAILURE(status))
+		return false;
+
+	adev = acpi_bus_get_acpi_device(handle);
+	if (!adev)
+		return false;
+
+	fwnode = acpi_fwnode_handle(adev);
+	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
+		return val == 1;
+
+	return false;
+}
+
 static bool acpi_pci_power_manageable(struct pci_dev *dev)
 {
 	struct acpi_device *adev = ACPI_COMPANION(&dev->dev);
@@ -1100,6 +1146,7 @@ static bool acpi_pci_need_resume(struct pci_dev *dev)
 
 static const struct pci_platform_pm_ops acpi_pci_platform_pm = {
 	.bridge_d3 = acpi_pci_bridge_d3,
+	.storage_d3 = acpi_pci_storage_d3,
 	.is_manageable = acpi_pci_power_manageable,
 	.set_state = acpi_pci_set_power_state,
 	.get_state = acpi_pci_get_power_state,
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 595fcf59843f..217d89f31e8b 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -871,6 +871,11 @@ static inline bool platform_pci_bridge_d3(struct pci_dev *dev)
 	return pci_platform_pm ? pci_platform_pm->bridge_d3(dev) : false;
 }
 
+static inline bool platform_pci_storage_d3(struct pci_dev *dev)
+{
+	return pci_platform_pm->storage_d3(dev);
+}
+
 /**
  * pci_raw_set_power_state - Use PCI PM registers to set the power state of
  *			     given PCI device
@@ -2903,6 +2908,7 @@ void pci_pm_init(struct pci_dev *dev)
 	dev->d3_delay = PCI_PM_D3_WAIT;
 	dev->d3cold_delay = PCI_PM_D3COLD_WAIT;
 	dev->bridge_d3 = pci_bridge_d3_possible(dev);
+	dev->storage_d3 = platform_pci_storage_d3(dev);
 	dev->d3cold_allowed = true;
 
 	dev->d1_support = false;
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 6d3f75867106..4dc4d05da810 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -53,6 +53,9 @@ int pci_bus_error_reset(struct pci_dev *dev);
  *
  * @bridge_d3: Does the bridge allow entering into D3
  *
+ * @storage_d3: 'true' if the bridge contains a storage device that must use D3
+ *		to support platform s2idle
+ *
  * @is_manageable: returns 'true' if given device is power manageable by the
  *		   platform firmware
  *
@@ -77,6 +80,7 @@ int pci_bus_error_reset(struct pci_dev *dev);
  */
 struct pci_platform_pm_ops {
 	bool (*bridge_d3)(struct pci_dev *dev);
+	bool (*storage_d3)(struct pci_dev *dev);
 	bool (*is_manageable)(struct pci_dev *dev);
 	int (*set_state)(struct pci_dev *dev, pci_power_t state);
 	pci_power_t (*get_state)(struct pci_dev *dev);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 83ce1cdf5676..cbba1555020b 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -346,6 +346,7 @@ struct pci_dev {
 	unsigned int	no_d1d2:1;	/* D1 and D2 are forbidden */
 	unsigned int	no_d3cold:1;	/* D3cold is forbidden */
 	unsigned int	bridge_d3:1;	/* Allow D3 for bridge */
+	unsigned int	storage_d3:1;	/* Storage dev must use D3 for s2idle */
 	unsigned int	d3cold_allowed:1;	/* D3cold is allowed by user */
 	unsigned int	mmio_always_on:1;	/* Disallow turning off io/mem
 						   decoding during BAR sizing */
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH 2/2] drivers/nvme: Add support for ACPI StorageD3Enable property
  2020-04-28  0:32 [PATCH 0/2] Add support for StorageD3Enable _DSD property David E. Box
  2020-04-28  0:32 ` [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support David E. Box
@ 2020-04-28  0:32 ` David E. Box
  2020-04-28  5:13 ` [PATCH 0/2] Add support for StorageD3Enable _DSD property Christoph Hellwig
  2 siblings, 0 replies; 17+ messages in thread
From: David E. Box @ 2020-04-28  0:32 UTC (permalink / raw)
  To: rjw, lenb, bhelgaas, kbusch, axboe, hch, sagi
  Cc: David E. Box, linux-acpi, linux-pci, linux-nvme, linux-kernel,
	Dan Williams

NVMe storage power management during suspend-to-idle, particularly on
laptops, has been inconsistent with some devices working with D3 while
others must rely on NVMe APST in order for power savings to be realized.
Currently the default is to use APST unless quirked to do otherwise.
However newer platforms, like Intel Comet Lake systems, may require NVMe
drives to use D3 in order for the PCIe ports to be properly power managed.
To make it easier for drivers to choose, these platforms may supply a
special "StorageD3Enable" _DSD property under the root port that the device
is attached to. If supplied, the driver must use D3 in order for the
platform to realize the deepest power savings in suspend-to-idle.

Add check of StorageD3Enable property during probe to use D3 as specified
by platform firmware.

Acked-by: Dan Williams <dan.j.williams@intel.com>
Link: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
---
 drivers/nvme/host/pci.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 4e79e412b276..4d67735975f6 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2777,6 +2777,13 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 
 	quirks |= check_vendor_combination_bug(pdev);
 
+	/*
+	 * Platform requires storage device to use D3 for kernel managed
+	 * suspend.
+	 */
+	if (pdev->storage_d3)
+		quirks |= NVME_QUIRK_SIMPLE_SUSPEND;
+
 	/*
 	 * Double check that our mempool alloc size will cover the biggest
 	 * command we support.
-- 
2.20.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-28  0:32 [PATCH 0/2] Add support for StorageD3Enable _DSD property David E. Box
  2020-04-28  0:32 ` [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support David E. Box
  2020-04-28  0:32 ` [PATCH 2/2] drivers/nvme: Add support for ACPI StorageD3Enable property David E. Box
@ 2020-04-28  5:13 ` Christoph Hellwig
  2020-04-28 14:09   ` David E. Box
  2 siblings, 1 reply; 17+ messages in thread
From: Christoph Hellwig @ 2020-04-28  5:13 UTC (permalink / raw)
  To: David E. Box
  Cc: rjw, lenb, bhelgaas, kbusch, axboe, hch, sagi, linux-acpi,
	linux-pci, linux-nvme, linux-kernel

On Mon, Apr 27, 2020 at 05:32:12PM -0700, David E. Box wrote:
> NVMe storage power management during suspend-to-idle, particularly on
> laptops, has been inconsistent with some devices working with D3 while
> others must rely on NVMe APST in order for power savings to be realized.
> Currently the default is to use APST unless quirked to do otherwise.
> However newer platforms, like Intel Comet Lake systems, may require NVMe
> drives to use D3 in order for the PCIe ports to be properly power managed.
> To make it easier for drivers to choose, these platforms may supply a
> special "StorageD3Enable" _DSD property under the root port that the device
> is attached to. If supplied, the driver must use D3 in order for the
> platform to realize the deepest power savings in suspend-to-idle.
>     
> The first patch adds the new _DSD GUID and fowards the property through the
> pci/acpi layer to the pci device.
> 
> The second patch adds support for the property to the nvme driver.

I'm not sure who came up with the idea to put this into ACPI, but it
belongs into NVMe.  Please talk to the NVMe technical working group
instead of trying to overrules them in an unrelated group that doesn't
apply to all of PCIe.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-28  5:13 ` [PATCH 0/2] Add support for StorageD3Enable _DSD property Christoph Hellwig
@ 2020-04-28 14:09   ` David E. Box
  2020-04-28 14:22     ` Christoph Hellwig
  0 siblings, 1 reply; 17+ messages in thread
From: David E. Box @ 2020-04-28 14:09 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: rjw, lenb, bhelgaas, kbusch, axboe, sagi, linux-acpi, linux-pci,
	linux-nvme, linux-kernel

On Tue, 2020-04-28 at 07:13 +0200, Christoph Hellwig wrote:
> On Mon, Apr 27, 2020 at 05:32:12PM -0700, David E. Box wrote:
> > NVMe storage power management during suspend-to-idle, particularly
> > on
> > laptops, has been inconsistent with some devices working with D3
> > while
> > others must rely on NVMe APST in order for power savings to be
> > realized.
> > Currently the default is to use APST unless quirked to do
> > otherwise.
> > However newer platforms, like Intel Comet Lake systems, may require
> > NVMe
> > drives to use D3 in order for the PCIe ports to be properly power
> > managed.
> > To make it easier for drivers to choose, these platforms may supply
> > a
> > special "StorageD3Enable" _DSD property under the root port that
> > the device
> > is attached to. If supplied, the driver must use D3 in order for
> > the
> > platform to realize the deepest power savings in suspend-to-idle.
> >     
> > The first patch adds the new _DSD GUID and fowards the property
> > through the
> > pci/acpi layer to the pci device.
> > 
> > The second patch adds support for the property to the nvme driver.
> 
> I'm not sure who came up with the idea to put this into ACPI, but it
> belongs into NVMe.  Please talk to the NVMe technical working group
> instead of trying to overrules them in an unrelated group that
> doesn't
> apply to all of PCIe.

Agreed that this is not ideal since it does not apply to all of PCIe.
But as the property already exists on shipping systems, we need to be
able to read it in the NVMe driver and the patch is consitent with the
way properties under PCI ports are read.

David


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-28 14:09   ` David E. Box
@ 2020-04-28 14:22     ` Christoph Hellwig
  2020-04-28 15:27       ` David E. Box
  0 siblings, 1 reply; 17+ messages in thread
From: Christoph Hellwig @ 2020-04-28 14:22 UTC (permalink / raw)
  To: David E. Box
  Cc: Christoph Hellwig, rjw, lenb, bhelgaas, kbusch, axboe, sagi,
	linux-acpi, linux-pci, linux-nvme, linux-kernel

On Tue, Apr 28, 2020 at 07:09:59AM -0700, David E. Box wrote:
> > I'm not sure who came up with the idea to put this into ACPI, but it
> > belongs into NVMe.  Please talk to the NVMe technical working group
> > instead of trying to overrules them in an unrelated group that
> > doesn't
> > apply to all of PCIe.
> 
> Agreed that this is not ideal since it does not apply to all of PCIe.
> But as the property already exists on shipping systems, we need to be
> able to read it in the NVMe driver and the patch is consitent with the
> way properties under PCI ports are read.

The point is that it is not the BIOSes job do decide how Linux does
power management.  For example D3 has really horrible entry and exit
latencies in many cases, and will lead to higher power usage.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-28 14:22     ` Christoph Hellwig
@ 2020-04-28 15:27       ` David E. Box
  2020-04-29  5:20         ` Williams, Dan J
  0 siblings, 1 reply; 17+ messages in thread
From: David E. Box @ 2020-04-28 15:27 UTC (permalink / raw)
  To: Christoph Hellwig
  Cc: rjw, lenb, bhelgaas, kbusch, axboe, sagi, linux-acpi, linux-pci,
	linux-nvme, linux-kernel

On Tue, 2020-04-28 at 16:22 +0200, Christoph Hellwig wrote:
> On Tue, Apr 28, 2020 at 07:09:59AM -0700, David E. Box wrote:
> > > I'm not sure who came up with the idea to put this into ACPI, but
> > > it
> > > belongs into NVMe.  Please talk to the NVMe technical working
> > > group
> > > instead of trying to overrules them in an unrelated group that
> > > doesn't
> > > apply to all of PCIe.
> > 
> > Agreed that this is not ideal since it does not apply to all of
> > PCIe.
> > But as the property already exists on shipping systems, we need to
> > be
> > able to read it in the NVMe driver and the patch is consitent with
> > the
> > way properties under PCI ports are read.
> 
> The point is that it is not the BIOSes job do decide how Linux does
> power management.  For example D3 has really horrible entry and exit
> latencies in many cases, and will lead to higher power usage.

The platform can know which pm policies will save the most power. But
since the solution doesn't apply to all PCIe devices (despite BIOS
specifying it that way) I'll withdraw this patch. Thanks.

David


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-28 15:27       ` David E. Box
@ 2020-04-29  5:20         ` Williams, Dan J
  2020-04-29 15:10           ` Keith Busch
                             ` (3 more replies)
  0 siblings, 4 replies; 17+ messages in thread
From: Williams, Dan J @ 2020-04-29  5:20 UTC (permalink / raw)
  To: hch, david.e.box
  Cc: linux-nvme, sagi, lenb, rjw, linux-kernel, linux-acpi, linux-pci,
	axboe, kbusch, bhelgaas

On Tue, 2020-04-28 at 08:27 -0700, David E. Box wrote:
> On Tue, 2020-04-28 at 16:22 +0200, Christoph Hellwig wrote:
> > On Tue, Apr 28, 2020 at 07:09:59AM -0700, David E. Box wrote:
> > > > I'm not sure who came up with the idea to put this into ACPI,
> > > > but
> > > > it
> > > > belongs into NVMe.  Please talk to the NVMe technical working
> > > > group
> > > > instead of trying to overrules them in an unrelated group that
> > > > doesn't
> > > > apply to all of PCIe.
> > > 
> > > Agreed that this is not ideal since it does not apply to all of
> > > PCIe.
> > > But as the property already exists on shipping systems, we need
> > > to
> > > be
> > > able to read it in the NVMe driver and the patch is consitent
> > > with
> > > the
> > > way properties under PCI ports are read.
> > 
> > The point is that it is not the BIOSes job do decide how Linux does
> > power management.  For example D3 has really horrible entry and
> > exit
> > latencies in many cases, and will lead to higher power usage.
> 
> The platform can know which pm policies will save the most power. But
> since the solution doesn't apply to all PCIe devices (despite BIOS
> specifying it that way) I'll withdraw this patch. Thanks.

Wait, why withdraw? In this case the platform is unfortunately
preventing the standard driver from making a proper determination. So
while I agree that it's not the BIOSes job, when the platform actively
prevents proper operation due to some ill conceived non-standard
platform property what is Linux left to do on these systems?

The *patch* is not trying to overrule NVME, and the best I can say is
that the Intel Linux team was not in the loop when this was being
decided between the platform BIOS implemenation and  whomever  thought
they could just publish random ACPI properties that impacted NVME
operation [1].

So now David is trying to get these platform unbroken because they are
already shipping with this b0rkage.

[1]: 
https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-29  5:20         ` Williams, Dan J
@ 2020-04-29 15:10           ` Keith Busch
  2020-04-29 16:11           ` David E. Box
                             ` (2 subsequent siblings)
  3 siblings, 0 replies; 17+ messages in thread
From: Keith Busch @ 2020-04-29 15:10 UTC (permalink / raw)
  To: Williams, Dan J
  Cc: hch, david.e.box, linux-nvme, sagi, lenb, rjw, linux-kernel,
	linux-acpi, linux-pci, axboe, bhelgaas

On Wed, Apr 29, 2020 at 05:20:09AM +0000, Williams, Dan J wrote:
> On Tue, 2020-04-28 at 08:27 -0700, David E. Box wrote:
> > On Tue, 2020-04-28 at 16:22 +0200, Christoph Hellwig wrote:
> > > On Tue, Apr 28, 2020 at 07:09:59AM -0700, David E. Box wrote:
> > > > > I'm not sure who came up with the idea to put this into ACPI,
> > > > > but
> > > > > it
> > > > > belongs into NVMe.  Please talk to the NVMe technical working
> > > > > group
> > > > > instead of trying to overrules them in an unrelated group that
> > > > > doesn't
> > > > > apply to all of PCIe.
> > > > 
> > > > Agreed that this is not ideal since it does not apply to all of
> > > > PCIe.
> > > > But as the property already exists on shipping systems, we need
> > > > to
> > > > be
> > > > able to read it in the NVMe driver and the patch is consitent
> > > > with
> > > > the
> > > > way properties under PCI ports are read.
> > > 
> > > The point is that it is not the BIOSes job do decide how Linux does
> > > power management.  For example D3 has really horrible entry and
> > > exit
> > > latencies in many cases, and will lead to higher power usage.
> > 
> > The platform can know which pm policies will save the most power. But
> > since the solution doesn't apply to all PCIe devices (despite BIOS
> > specifying it that way) I'll withdraw this patch. Thanks.
> 
> Wait, why withdraw? In this case the platform is unfortunately
> preventing the standard driver from making a proper determination. So
> while I agree that it's not the BIOSes job, when the platform actively
> prevents proper operation due to some ill conceived non-standard
> platform property what is Linux left to do on these systems?
> 
> The *patch* is not trying to overrule NVME, and the best I can say is
> that the Intel Linux team was not in the loop when this was being
> decided between the platform BIOS implemenation and  whomever  thought
> they could just publish random ACPI properties that impacted NVME
> operation [1].
> 
> So now David is trying to get these platform unbroken because they are
> already shipping with this b0rkage.

Rather than quirking all these cases, which I get the feeling there
are many more than we've currently got in our quirk list, perhaps it'd
be simpler to default to the simple suspend. AFAIK, the simple suspend
works for all platforms, though it may not realize the best power savings
and/or exit latency.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-29  5:20         ` Williams, Dan J
  2020-04-29 15:10           ` Keith Busch
@ 2020-04-29 16:11           ` David E. Box
  2020-05-01 13:12             ` hch
  2020-05-01 13:10           ` hch
  2020-05-18 13:51           ` David Woodhouse
  3 siblings, 1 reply; 17+ messages in thread
From: David E. Box @ 2020-04-29 16:11 UTC (permalink / raw)
  To: Williams, Dan J, hch
  Cc: linux-nvme, sagi, lenb, rjw, linux-kernel, linux-acpi, linux-pci,
	axboe, kbusch, bhelgaas

On Wed, 2020-04-29 at 05:20 +0000, Williams, Dan J wrote:
> On Tue, 2020-04-28 at 08:27 -0700, David E. Box wrote:
> > On Tue, 2020-04-28 at 16:22 +0200, Christoph Hellwig wrote:
> > > On Tue, Apr 28, 2020 at 07:09:59AM -0700, David E. Box wrote:
> > > > > I'm not sure who came up with the idea to put this into ACPI,
> > > > > but
> > > > > it
> > > > > belongs into NVMe.  Please talk to the NVMe technical working
> > > > > group
> > > > > instead of trying to overrules them in an unrelated group
> > > > > that
> > > > > doesn't
> > > > > apply to all of PCIe.
> > > > 
> > > > Agreed that this is not ideal since it does not apply to all of
> > > > PCIe.
> > > > But as the property already exists on shipping systems, we need
> > > > to
> > > > be
> > > > able to read it in the NVMe driver and the patch is consitent
> > > > with
> > > > the
> > > > way properties under PCI ports are read.
> > > 
> > > The point is that it is not the BIOSes job do decide how Linux
> > > does
> > > power management.  For example D3 has really horrible entry and
> > > exit
> > > latencies in many cases, and will lead to higher power usage.
> > 
> > The platform can know which pm policies will save the most power.
> > But
> > since the solution doesn't apply to all PCIe devices (despite BIOS
> > specifying it that way) I'll withdraw this patch. Thanks.
> 
> Wait, why withdraw? In this case the platform is unfortunately
> preventing the standard driver from making a proper determination. So
> while I agree that it's not the BIOSes job, when the platform
> actively
> prevents proper operation due to some ill conceived non-standard
> platform property what is Linux left to do on these systems?
> 
> The *patch* is not trying to overrule NVME, and the best I can say is
> that the Intel Linux team was not in the loop when this was being
> decided between the platform BIOS implemenation
> and  whomever  thought
> they could just publish random ACPI properties that impacted NVME
> operation [1].
> 
> So now David is trying to get these platform unbroken because they
> are
> already shipping with this b0rkage.

Not drop completely. This patch copied the code used to read _DSD
properties under PCI root ports. But I agree that such properties
should apply to all devices on those ports and unfortuntely that's not
the case here. BIOS got it wrong. My thought in dropping this patch is
to rewrite it to read the property directly from the nvme driver. Not
the way it's typically done either but it would avoid a global change
in the pci core while allowing us to deal with the firmware we have.

David


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-29  5:20         ` Williams, Dan J
  2020-04-29 15:10           ` Keith Busch
  2020-04-29 16:11           ` David E. Box
@ 2020-05-01 13:10           ` hch
  2020-05-18 13:51           ` David Woodhouse
  3 siblings, 0 replies; 17+ messages in thread
From: hch @ 2020-05-01 13:10 UTC (permalink / raw)
  To: Williams, Dan J
  Cc: hch, david.e.box, linux-nvme, sagi, lenb, rjw, linux-kernel,
	linux-acpi, linux-pci, axboe, kbusch, bhelgaas

On Wed, Apr 29, 2020 at 05:20:09AM +0000, Williams, Dan J wrote:
> > The platform can know which pm policies will save the most power. But
> > since the solution doesn't apply to all PCIe devices (despite BIOS
> > specifying it that way) I'll withdraw this patch. Thanks.
> 
> Wait, why withdraw? In this case the platform is unfortunately
> preventing the standard driver from making a proper determination. So
> while I agree that it's not the BIOSes job, when the platform actively
> prevents proper operation due to some ill conceived non-standard
> platform property what is Linux left to do on these systems?
> 
> The *patch* is not trying to overrule NVME, and the best I can say is
> that the Intel Linux team was not in the loop when this was being
> decided between the platform BIOS implemenation and  whomever  thought
> they could just publish random ACPI properties that impacted NVME
> operation [1].
> 
> So now David is trying to get these platform unbroken because they are
> already shipping with this b0rkage.

So can we please clearly mark this as a quirk and warn in the kernel
log about a buggy BIOS?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-29 16:11           ` David E. Box
@ 2020-05-01 13:12             ` hch
  2020-05-01 15:54               ` David E. Box
  0 siblings, 1 reply; 17+ messages in thread
From: hch @ 2020-05-01 13:12 UTC (permalink / raw)
  To: David E. Box
  Cc: Williams, Dan J, hch, linux-nvme, sagi, lenb, rjw, linux-kernel,
	linux-acpi, linux-pci, axboe, kbusch, bhelgaas

On Wed, Apr 29, 2020 at 09:11:13AM -0700, David E. Box wrote:
> Not drop completely. This patch copied the code used to read _DSD
> properties under PCI root ports. But I agree that such properties
> should apply to all devices on those ports and unfortuntely that's not
> the case here. BIOS got it wrong. My thought in dropping this patch is
> to rewrite it to read the property directly from the nvme driver. Not
> the way it's typically done either but it would avoid a global change
> in the pci core while allowing us to deal with the firmware we have.

I'd be happy to heave less of this crap in nvme actually.  But I'm really
pissed this shit got out in the wild.  It wasn't clear from the mail
that this is something already out there because the idiots coming up
with it just went ahead with it.  Please just update the commit logs
and implementation to clearly mark it as a workaround for buggys
systems, which just happen to at least be nice enough to tell us that
they are buggy as f^$k.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-05-01 13:12             ` hch
@ 2020-05-01 15:54               ` David E. Box
  0 siblings, 0 replies; 17+ messages in thread
From: David E. Box @ 2020-05-01 15:54 UTC (permalink / raw)
  To: hch
  Cc: Williams, Dan J, linux-nvme, sagi, lenb, rjw, linux-kernel,
	linux-acpi, linux-pci, axboe, kbusch, bhelgaas

On Fri, 2020-05-01 at 15:12 +0200, hch@lst.de wrote:
> On Wed, Apr 29, 2020 at 09:11:13AM -0700, David E. Box wrote:
> > Not drop completely. This patch copied the code used to read _DSD
> > properties under PCI root ports. But I agree that such properties
> > should apply to all devices on those ports and unfortuntely that's
> > not
> > the case here. BIOS got it wrong. My thought in dropping this patch
> > is
> > to rewrite it to read the property directly from the nvme driver.
> > Not
> > the way it's typically done either but it would avoid a global
> > change
> > in the pci core while allowing us to deal with the firmware we
> > have.
> 
> I'd be happy to heave less of this crap in nvme actually.  But I'm
> really
> pissed this shit got out in the wild.  It wasn't clear from the mail
> that this is something already out there because the idiots coming up
> with it just went ahead with it.  Please just update the commit logs
> and implementation to clearly mark it as a workaround for buggys
> systems, which just happen to at least be nice enough to tell us that
> they are buggy as f^$k.

Sure.


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support
  2020-04-28  0:32 ` [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support David E. Box
@ 2020-05-18 12:34   ` Rafael J. Wysocki
  2020-05-19 17:10     ` David E. Box
  0 siblings, 1 reply; 17+ messages in thread
From: Rafael J. Wysocki @ 2020-05-18 12:34 UTC (permalink / raw)
  To: David E. Box
  Cc: lenb, bhelgaas, kbusch, axboe, hch, sagi, linux-acpi, linux-pci,
	linux-nvme, linux-kernel, Dan Williams

On Tuesday, April 28, 2020 2:32:13 AM CEST David E. Box wrote:
> NVMe storage power management during suspend-to-idle, particularly on
> laptops, has been inconsistent with some devices working with D3 while
> others must rely on NVMe APST in order for power savings to be realized.
> Currently the default is to use APST unless quirked to do otherwise.
> However newer platforms, like Intel Comet Lake systems, may require NVMe
> drives to use D3 in order for the PCIe ports to be properly power managed.
> To make it easier for drivers to choose, these platforms may supply a
> special "StorageD3Enable" _DSD property under the root port that the device
> is attached to. If supplied, the driver must use D3 in order for the
> platform to realize the deepest power savings in suspend-to-idle.
> 
> Adds support for the _DSD to the pci/acpi layer.
> 
> Acked-by: Dan Williams <dan.j.williams@intel.com>
> Link: https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro
> Signed-off-by: David E. Box <david.e.box@linux.intel.com>
> ---
>  drivers/acpi/property.c |  3 +++
>  drivers/pci/pci-acpi.c  | 47 +++++++++++++++++++++++++++++++++++++++++
>  drivers/pci/pci.c       |  6 ++++++
>  drivers/pci/pci.h       |  4 ++++
>  include/linux/pci.h     |  1 +
>  5 files changed, 61 insertions(+)
> 
> diff --git a/drivers/acpi/property.c b/drivers/acpi/property.c
> index e601c4511a8b..f09375ab40e4 100644
> --- a/drivers/acpi/property.c
> +++ b/drivers/acpi/property.c
> @@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
>  	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-ba72-9bf5a26ebe5d */
>  	GUID_INIT(0x6c501103, 0xc189, 0x4296,
>  		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
> +	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-99a5189762d0 */
> +	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
> +		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
>  };
>  
>  /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-1319f52a966b */
> diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> index d21969fba6ab..5df249ebf022 100644
> --- a/drivers/pci/pci-acpi.c
> +++ b/drivers/pci/pci-acpi.c
> @@ -972,6 +972,52 @@ static bool acpi_pci_bridge_d3(struct pci_dev *dev)
>  	return val == 1;
>  }
>  
> +static bool acpi_pci_storage_d3(struct pci_dev *dev)
> +{
> +	const struct fwnode_handle *fwnode;
> +	struct acpi_device *adev;
> +	struct pci_dev *root;
> +	acpi_handle handle;
> +	acpi_status status;
> +	u8 val;
> +
> +	/*
> +	 * Look for _DSD property specifying that the storage device on
> +	 * the port must use D3 to support deep platform power savings during
> +	 * suspend-to-idle
> +	 */
> +	root = pci_find_pcie_root_port(dev);
> +	if (!root)
> +		return false;
> +
> +	adev = ACPI_COMPANION(&root->dev);
> +	if (root == dev) {
> +		/*
> +		 * It is possible that the ACPI companion is not yet bound
> +		 * for the root port so look it up manually here.
> +		 */
> +		if (!adev && !pci_dev_is_added(root))
> +			adev = acpi_pci_find_companion(&root->dev);
> +	}
> +
> +	if (!adev)
> +		return false;
> +
> +	status = acpi_get_handle(adev->handle, "PXSX", &handle);
> +	if (ACPI_FAILURE(status))
> +		return false;
> +
> +	adev = acpi_bus_get_acpi_device(handle);
> +	if (!adev)
> +		return false;
> +
> +	fwnode = acpi_fwnode_handle(adev);
> +	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
> +		return val == 1;
> +
> +	return false;
> +}

Kind of orthogonal to what happens to the second patch in this series, I don't
think that the PCI changes below are all needed.

IMO it would be sufficient to export the function above, maybe as
pci_acpi_storage_d3(), to drivers, so that they can call it directly as
desired.

Since _DSD return data are not allowed by the spec to change between
subsequent invocations of it, the interested driver may call this function
once at the device init time and quirk it accordingly if needed.

Cheers!




^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-04-29  5:20         ` Williams, Dan J
                             ` (2 preceding siblings ...)
  2020-05-01 13:10           ` hch
@ 2020-05-18 13:51           ` David Woodhouse
  2020-05-18 17:20             ` Dan Williams
  3 siblings, 1 reply; 17+ messages in thread
From: David Woodhouse @ 2020-05-18 13:51 UTC (permalink / raw)
  To: Williams, Dan J, hch, david.e.box
  Cc: linux-nvme, sagi, lenb, rjw, linux-kernel, linux-acpi, linux-pci,
	axboe, kbusch, bhelgaas


[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

On Wed, 2020-04-29 at 05:20 +0000, Williams, Dan J wrote:
> The *patch* is not trying to overrule NVME, and the best I can say is
> that the Intel Linux team was not in the loop when this was being
> decided between the platform BIOS implemenation and  whomever  thought
> they could just publish random ACPI properties that impacted NVME
> operation [1].
> 
> So now David is trying to get these platform unbroken because they are
> already shipping with this b0rkage.

This is what we have WARN_TAINT() for though, right? It can suitably
warn users when such breakage is detected in the platform.



[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5174 bytes --]

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 0/2] Add support for StorageD3Enable _DSD property
  2020-05-18 13:51           ` David Woodhouse
@ 2020-05-18 17:20             ` Dan Williams
  0 siblings, 0 replies; 17+ messages in thread
From: Dan Williams @ 2020-05-18 17:20 UTC (permalink / raw)
  To: David Woodhouse
  Cc: hch, david.e.box, linux-nvme, sagi, lenb, rjw, linux-kernel,
	linux-acpi, linux-pci, axboe, kbusch, bhelgaas

On Mon, May 18, 2020 at 6:52 AM David Woodhouse <dwmw2@infradead.org> wrote:
>
> On Wed, 2020-04-29 at 05:20 +0000, Williams, Dan J wrote:
> > The *patch* is not trying to overrule NVME, and the best I can say is
> > that the Intel Linux team was not in the loop when this was being
> > decided between the platform BIOS implemenation and  whomever  thought
> > they could just publish random ACPI properties that impacted NVME
> > operation [1].
> >
> > So now David is trying to get these platform unbroken because they are
> > already shipping with this b0rkage.
>
> This is what we have WARN_TAINT() for though, right? It can suitably
> warn users when such breakage is detected in the platform.
>

I see WARN_TAINT() as "BIOS implemented its specification wrong". This
case is BIOS "implemented a mechanism in the wrong specification".

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support
  2020-05-18 12:34   ` Rafael J. Wysocki
@ 2020-05-19 17:10     ` David E. Box
  0 siblings, 0 replies; 17+ messages in thread
From: David E. Box @ 2020-05-19 17:10 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: lenb, bhelgaas, kbusch, axboe, hch, sagi, linux-acpi, linux-pci,
	linux-nvme, linux-kernel, Dan Williams

On Mon, 2020-05-18 at 14:34 +0200, Rafael J. Wysocki wrote:
> On Tuesday, April 28, 2020 2:32:13 AM CEST David E. Box wrote:
> > NVMe storage power management during suspend-to-idle, particularly
> > on
> > laptops, has been inconsistent with some devices working with D3
> > while
> > others must rely on NVMe APST in order for power savings to be
> > realized.
> > Currently the default is to use APST unless quirked to do
> > otherwise.
> > However newer platforms, like Intel Comet Lake systems, may require
> > NVMe
> > drives to use D3 in order for the PCIe ports to be properly power
> > managed.
> > To make it easier for drivers to choose, these platforms may supply
> > a
> > special "StorageD3Enable" _DSD property under the root port that
> > the device
> > is attached to. If supplied, the driver must use D3 in order for
> > the
> > platform to realize the deepest power savings in suspend-to-idle.
> > 
> > Adds support for the _DSD to the pci/acpi layer.
> > 
> > Acked-by: Dan Williams <dan.j.williams@intel.com>
> > Link: 
> > https://docs.microsoft.com/en-us/windows-hardware/design/component-guidelines/power-management-for-storage-hardware-devices-intro
> > Signed-off-by: David E. Box <david.e.box@linux.intel.com>
> > ---
> >  drivers/acpi/property.c |  3 +++
> >  drivers/pci/pci-acpi.c  | 47
> > +++++++++++++++++++++++++++++++++++++++++
> >  drivers/pci/pci.c       |  6 ++++++
> >  drivers/pci/pci.h       |  4 ++++
> >  include/linux/pci.h     |  1 +
> >  5 files changed, 61 insertions(+)
> > 
> > diff --git a/drivers/acpi/property.c b/drivers/acpi/property.c
> > index e601c4511a8b..f09375ab40e4 100644
> > --- a/drivers/acpi/property.c
> > +++ b/drivers/acpi/property.c
> > @@ -45,6 +45,9 @@ static const guid_t prp_guids[] = {
> >  	/* Thunderbolt GUID for WAKE_SUPPORTED: 6c501103-c189-4296-
> > ba72-9bf5a26ebe5d */
> >  	GUID_INIT(0x6c501103, 0xc189, 0x4296,
> >  		  0xba, 0x72, 0x9b, 0xf5, 0xa2, 0x6e, 0xbe, 0x5d),
> > +	/* D3 Support for storage devivce: 5025030f-842f-4ab4-a561-
> > 99a5189762d0 */
> > +	GUID_INIT(0x5025030f, 0x842f, 0x4ab4,
> > +		  0xa5, 0x61, 0x99, 0xa5, 0x18, 0x97, 0x62, 0xd0),
> >  };
> >  
> >  /* ACPI _DSD data subnodes GUID: dbb8e3e6-5886-4ba6-8795-
> > 1319f52a966b */
> > diff --git a/drivers/pci/pci-acpi.c b/drivers/pci/pci-acpi.c
> > index d21969fba6ab..5df249ebf022 100644
> > --- a/drivers/pci/pci-acpi.c
> > +++ b/drivers/pci/pci-acpi.c
> > @@ -972,6 +972,52 @@ static bool acpi_pci_bridge_d3(struct pci_dev
> > *dev)
> >  	return val == 1;
> >  }
> >  
> > +static bool acpi_pci_storage_d3(struct pci_dev *dev)
> > +{
> > +	const struct fwnode_handle *fwnode;
> > +	struct acpi_device *adev;
> > +	struct pci_dev *root;
> > +	acpi_handle handle;
> > +	acpi_status status;
> > +	u8 val;
> > +
> > +	/*
> > +	 * Look for _DSD property specifying that the storage device on
> > +	 * the port must use D3 to support deep platform power savings
> > during
> > +	 * suspend-to-idle
> > +	 */
> > +	root = pci_find_pcie_root_port(dev);
> > +	if (!root)
> > +		return false;
> > +
> > +	adev = ACPI_COMPANION(&root->dev);
> > +	if (root == dev) {
> > +		/*
> > +		 * It is possible that the ACPI companion is not yet
> > bound
> > +		 * for the root port so look it up manually here.
> > +		 */
> > +		if (!adev && !pci_dev_is_added(root))
> > +			adev = acpi_pci_find_companion(&root->dev);
> > +	}
> > +
> > +	if (!adev)
> > +		return false;
> > +
> > +	status = acpi_get_handle(adev->handle, "PXSX", &handle);
> > +	if (ACPI_FAILURE(status))
> > +		return false;
> > +
> > +	adev = acpi_bus_get_acpi_device(handle);
> > +	if (!adev)
> > +		return false;
> > +
> > +	fwnode = acpi_fwnode_handle(adev);
> > +	if (!fwnode_property_read_u8(fwnode, "StorageD3Enable", &val))
> > +		return val == 1;
> > +
> > +	return false;
> > +}
> 
> Kind of orthogonal to what happens to the second patch in this
> series, I don't
> think that the PCI changes below are all needed.
> 
> IMO it would be sufficient to export the function above, maybe as
> pci_acpi_storage_d3(), to drivers, so that they can call it directly
> as
> desired.
> 
> Since _DSD return data are not allowed by the spec to change between
> subsequent invocations of it, the interested driver may call this
> function
> once at the device init time and quirk it accordingly if needed.

Yeah this is better. Will do. Thanks.


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, back to index

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-28  0:32 [PATCH 0/2] Add support for StorageD3Enable _DSD property David E. Box
2020-04-28  0:32 ` [PATCH 1/2] pci: Add ACPI StorageD3Enable _DSD support David E. Box
2020-05-18 12:34   ` Rafael J. Wysocki
2020-05-19 17:10     ` David E. Box
2020-04-28  0:32 ` [PATCH 2/2] drivers/nvme: Add support for ACPI StorageD3Enable property David E. Box
2020-04-28  5:13 ` [PATCH 0/2] Add support for StorageD3Enable _DSD property Christoph Hellwig
2020-04-28 14:09   ` David E. Box
2020-04-28 14:22     ` Christoph Hellwig
2020-04-28 15:27       ` David E. Box
2020-04-29  5:20         ` Williams, Dan J
2020-04-29 15:10           ` Keith Busch
2020-04-29 16:11           ` David E. Box
2020-05-01 13:12             ` hch
2020-05-01 15:54               ` David E. Box
2020-05-01 13:10           ` hch
2020-05-18 13:51           ` David Woodhouse
2020-05-18 17:20             ` Dan Williams

Linux-ACPI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-acpi/0 linux-acpi/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-acpi linux-acpi/ https://lore.kernel.org/linux-acpi \
		linux-acpi@vger.kernel.org
	public-inbox-index linux-acpi

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-acpi


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git