* [PATCH V2 2/2] PCI: vmd: Override ASPM on TGL/ADL VMD devices
2021-12-08 0:20 [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function David E. Box
@ 2021-12-08 0:20 ` David E. Box
2021-12-08 10:58 ` [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function kernel test robot
2021-12-08 10:58 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: David E. Box @ 2021-12-08 0:20 UTC (permalink / raw)
To: nirmal.patel, jonathan.derrick, lorenzo.pieralisi, kw, bhelgaas,
david.e.box, michael.a.bottini, rafael
Cc: linux-pci, linux-kernel, Adhitya Mohan
From: Michael Bottini <michael.a.bottini@linux.intel.com>
On Tiger Lake and Alder Lake platforms, VMD controllers do not have ASPM
enabled nor LTR values set by BIOS. This leads high power consumption on
these platforms when VMD is enabled as reported in bugzilla [1]. Enable
these features in the VMD driver using pcie_aspm_policy_override() to set
the ASPM policy for the root ports.
To do this, add an additional flag in VMD features to specify devices that
must have their respective policies overridden.
[1] https://bugzilla.kernel.org/show_bug.cgi?id=213717
Signed-off-by: Michael Bottini <michael.a.bottini@linux.intel.com>
Signed-off-by: David E. Box <david.e.box@linux.intel.com>
Tested-by: Adhitya Mohan <me@adhityamohan.in>
---
V2
- Use return status to print pci_info message if ASPM cannot be enabled.
- Add missing static declaration, caught by lkp@intel.com
drivers/pci/controller/vmd.c | 42 +++++++++++++++++++++++++++++++++---
1 file changed, 39 insertions(+), 3 deletions(-)
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index a45e8e59d3d4..e555dcae73cc 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -20,6 +20,8 @@
#include <asm/irqdomain.h>
+#include "../pci.h"
+
#define VMD_CFGBAR 0
#define VMD_MEMBAR1 2
#define VMD_MEMBAR2 4
@@ -67,6 +69,12 @@ enum vmd_features {
* interrupt handling.
*/
VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4),
+
+ /*
+ * Device must have ASPM policy overridden, as its default policy is
+ * incorrect.
+ */
+ VMD_FEAT_QUIRK_OVERRIDE_ASPM = (1 << 5),
};
static DEFINE_IDA(vmd_instance_ida);
@@ -661,6 +669,29 @@ static int vmd_alloc_irqs(struct vmd_dev *vmd)
return 0;
}
+/*
+ * Override the BIOS ASPM policy and set the LTR value for PCI storage
+ * devices on the VMD bride.
+ */
+static int vmd_enable_aspm(struct pci_dev *pdev, void *userdata)
+{
+ int features = *(int *)userdata;
+
+ if (features & VMD_FEAT_QUIRK_OVERRIDE_ASPM &&
+ pdev->class == PCI_CLASS_STORAGE_EXPRESS) {
+ int pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR);
+
+ if (pos) {
+ pci_write_config_word(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, 0x1003);
+ pci_write_config_word(pdev, pos + PCI_LTR_MAX_NOSNOOP_LAT, 0x1003);
+ if (pcie_aspm_policy_override(pdev))
+ pci_info(pdev, "Unable of override ASPM policy\n");
+ }
+ }
+
+ return 0;
+}
+
static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
{
struct pci_sysdata *sd = &vmd->sysdata;
@@ -807,6 +838,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
pci_scan_child_bus(vmd->bus);
pci_assign_unassigned_bus_resources(vmd->bus);
+ pci_walk_bus(vmd->bus, vmd_enable_aspm, &features);
+
/*
* VMD root buses are virtual and don't return true on pci_is_pcie()
* and will fail pcie_bus_configure_settings() early. It can instead be
@@ -948,15 +981,18 @@ static const struct pci_device_id vmd_ids[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x467f),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
VMD_FEAT_HAS_BUS_RESTRICTIONS |
- VMD_FEAT_OFFSET_FIRST_VECTOR,},
+ VMD_FEAT_OFFSET_FIRST_VECTOR |
+ VMD_FEAT_QUIRK_OVERRIDE_ASPM,},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x4c3d),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
VMD_FEAT_HAS_BUS_RESTRICTIONS |
- VMD_FEAT_OFFSET_FIRST_VECTOR,},
+ VMD_FEAT_OFFSET_FIRST_VECTOR |
+ VMD_FEAT_QUIRK_OVERRIDE_ASPM,},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP |
VMD_FEAT_HAS_BUS_RESTRICTIONS |
- VMD_FEAT_OFFSET_FIRST_VECTOR,},
+ VMD_FEAT_OFFSET_FIRST_VECTOR |
+ VMD_FEAT_QUIRK_OVERRIDE_ASPM,},
{0,}
};
MODULE_DEVICE_TABLE(pci, vmd_ids);
--
2.25.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function
2021-12-08 0:20 [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function David E. Box
2021-12-08 0:20 ` [PATCH V2 2/2] PCI: vmd: Override ASPM on TGL/ADL VMD devices David E. Box
@ 2021-12-08 10:58 ` kernel test robot
2021-12-08 10:58 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2021-12-08 10:58 UTC (permalink / raw)
To: David E. Box, nirmal.patel, jonathan.derrick, lorenzo.pieralisi,
kw, bhelgaas, michael.a.bottini, rafael
Cc: llvm, kbuild-all, linux-pci, linux-kernel
Hi "David,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on v5.16-rc4]
[also build test ERROR on next-20211207]
[cannot apply to helgaas-pci/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/David-E-Box/PCI-ASPM-Add-ASPM-BIOS-override-function/20211208-082303
base: 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: arm64-randconfig-r022-20211207 (https://download.01.org/0day-ci/archive/20211208/202112081821.vVNb7kbL-lkp@intel.com/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 097a1cb1d5ebb3a0ec4bcaed8ba3ff6a8e33c00a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build
# apt-get install binutils-aarch64-linux-gnu
# https://github.com/0day-ci/linux/commit/78c85417651fe465aafee7ef1841ab75619b165b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review David-E-Box/PCI-ASPM-Add-ASPM-BIOS-override-function/20211208-082303
git checkout 78c85417651fe465aafee7ef1841ab75619b165b
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash ./ drivers/
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from drivers/pci/controller/pci-aardvark.c:28:
>> drivers/pci/controller/../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
--
In file included from drivers/pci/controller/dwc/pci-dra7xx.c:32:
>> drivers/pci/controller/dwc/../../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
--
In file included from drivers/pci/pcie/portdrv_core.c:19:
>> drivers/pci/pcie/../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
--
In file included from drivers/pci/hotplug/pci_hotplug_core.c:32:
>> drivers/pci/hotplug/../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
--
In file included from drivers/xen/pci.c:18:
>> drivers/xen/../pci/pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
--
In file included from drivers/pci/controller/mobiveil/pcie-mobiveil.c:18:
In file included from drivers/pci/controller/mobiveil/pcie-mobiveil.h:18:
>> drivers/pci/controller/mobiveil/../../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
vim +571 drivers/pci/controller/../pci.h
553
554 /* PCI error reporting and recovery */
555 pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
556 pci_channel_state_t state,
557 pci_ers_result_t (*reset_subordinates)(struct pci_dev *pdev));
558
559 bool pcie_wait_for_link(struct pci_dev *pdev, bool active);
560 #ifdef CONFIG_PCIEASPM
561 void pcie_aspm_init_link_state(struct pci_dev *pdev);
562 void pcie_aspm_exit_link_state(struct pci_dev *pdev);
563 void pcie_aspm_pm_state_change(struct pci_dev *pdev);
564 void pcie_aspm_powersave_config_link(struct pci_dev *pdev);
565 int pcie_aspm_policy_override(struct pci_dev *dev);
566 #else
567 static inline void pcie_aspm_init_link_state(struct pci_dev *pdev) { }
568 static inline void pcie_aspm_exit_link_state(struct pci_dev *pdev) { }
569 static inline void pcie_aspm_pm_state_change(struct pci_dev *pdev) { }
570 static inline void pcie_aspm_powersave_config_link(struct pci_dev *pdev) { }
> 571 static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
572 #endif
573
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function
2021-12-08 0:20 [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function David E. Box
2021-12-08 0:20 ` [PATCH V2 2/2] PCI: vmd: Override ASPM on TGL/ADL VMD devices David E. Box
2021-12-08 10:58 ` [PATCH V2 1/2] PCI/ASPM: Add ASPM BIOS override function kernel test robot
@ 2021-12-08 10:58 ` kernel test robot
2 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2021-12-08 10:58 UTC (permalink / raw)
To: David E. Box, nirmal.patel, jonathan.derrick, lorenzo.pieralisi,
kw, bhelgaas, michael.a.bottini, rafael
Cc: llvm, kbuild-all, linux-pci, linux-kernel
Hi "David,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on v5.16-rc4]
[also build test ERROR on next-20211208]
[cannot apply to helgaas-pci/next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/David-E-Box/PCI-ASPM-Add-ASPM-BIOS-override-function/20211208-082303
base: 0fcfb00b28c0b7884635dacf38e46d60bf3d4eb1
config: i386-randconfig-r024-20211207 (https://download.01.org/0day-ci/archive/20211208/202112081833.kaiKplAi-lkp@intel.com/config)
compiler: clang version 14.0.0 (https://github.com/llvm/llvm-project 097a1cb1d5ebb3a0ec4bcaed8ba3ff6a8e33c00a)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/78c85417651fe465aafee7ef1841ab75619b165b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review David-E-Box/PCI-ASPM-Add-ASPM-BIOS-override-function/20211208-082303
git checkout 78c85417651fe465aafee7ef1841ab75619b165b
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from drivers/pci/controller/cadence/pci-j721e.c:23:
>> drivers/pci/controller/cadence/../../pci.h:571:82: error: expected ';' after return statement
static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
^
;
1 error generated.
vim +571 drivers/pci/controller/cadence/../../pci.h
553
554 /* PCI error reporting and recovery */
555 pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
556 pci_channel_state_t state,
557 pci_ers_result_t (*reset_subordinates)(struct pci_dev *pdev));
558
559 bool pcie_wait_for_link(struct pci_dev *pdev, bool active);
560 #ifdef CONFIG_PCIEASPM
561 void pcie_aspm_init_link_state(struct pci_dev *pdev);
562 void pcie_aspm_exit_link_state(struct pci_dev *pdev);
563 void pcie_aspm_pm_state_change(struct pci_dev *pdev);
564 void pcie_aspm_powersave_config_link(struct pci_dev *pdev);
565 int pcie_aspm_policy_override(struct pci_dev *dev);
566 #else
567 static inline void pcie_aspm_init_link_state(struct pci_dev *pdev) { }
568 static inline void pcie_aspm_exit_link_state(struct pci_dev *pdev) { }
569 static inline void pcie_aspm_pm_state_change(struct pci_dev *pdev) { }
570 static inline void pcie_aspm_powersave_config_link(struct pci_dev *pdev) { }
> 571 static inline int pcie_aspm_policy_override(struct pci_dev *dev) { return -EINVAL }
572 #endif
573
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 4+ messages in thread