RE: [PATCH V8 4/5] PCI/ASPM: save power on values during bridge init

["Patel, Mayurkumar" <mayurkumar.patel@intel.com>]

Hi Bjorn/Kaya,


>
>On 4/17/2017 12:38 PM, Bjorn Helgaas wrote:
>>> Like you said, what do we do by default is the question. Should we opt
>>> for safe like we are doing, or try to save some power.
>> I think safety is paramount.  Every user should be able to boot safely
>> without any kernel parameters.  We don't want users to have a problem
>> booting and then have to search for a workaround like booting with
>> "pcie_aspm=off".  Most users will never do that.
>>
>
>OK, no problem with leaving the behavior as it is.
>
>My initial approach was #2. We knew this way that user had full control
>over the ASPM policy by changing the BIOS option. Then, Mayurkumar
>complained that ASPM is not enabled following a hotplug insertion to an
>empty slot. That's when I switched to #3 as it sounded like a good thing
>to have for us.
>
>> Here's a long-term strawman proposal, see what you think:
>>
>>   - Deprecate CONFIG_PCIEASPM_DEFAULT, CONFIG_PCIEASPM_POWERSAVE, etc.
>>   - Default aspm_policy is POLICY_DEFAULT always.
>>   - POLICY_DEFAULT means Linux doesn't touch anything: if BIOS enabled
>> ASPM, we leave it that way; we leave ASPM disabled on hot-added
>> devices.
>
I am also ok with leaving the same behavior as now.
But still following is something open I feel besides, Which may be there in your comments redundantly.
The current problem is, pcie_aspm_exit_link_state() disables the ASPM configuration even
if POLICY_DEFAULT was set.
I am seeing already following problem(or may be influence) with it. The Endpoint I have does not have
does not have "Presence detect change" mechanism. Hot plug is working with Link status events.
When link is in L1 or L1SS and if EP is powered off, no Link status change event are triggered (It might be
the expected behavior in L1 or L1SS).  When next time EP is powered on there are link down and
link up events coming one after other. BIOS enables ASPM on Root port and Endpoint, but while
processing link status down, pcie_aspm_exit_link_state() clears the ASPM already which were enabled by BIOS. 
If we want to follow above approach then shall we consider having something similar as following?

diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 1dfa10c..bf5be6d 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -940,7 +940,8 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)
        parent_link = link->parent;

        /* All functions are removed, so just disable ASPM for the link */
-       pcie_config_aspm_link(link, 0);
+       if (aspm_policy != POLICY_DEFAULT)
+               pcie_config_aspm_link(link, 0);
        list_del(&link->sibling);
        list_del(&link->link);
        /* Clock PM is for endpoint device */
 

>I can easily see people complaining the other way around. There
>could be some boot FW that doesn't know what ASPM is and this particular
>system could rely on the compile time option for enabling power options.
>Maybe, a command line option will be required for them to keep the existing
>behavior.
>
>>   - Deprecate kernel boot parameters (possibly keep pcie_aspm=off for
>> debugging use).
>>   - Use /sys/module/pcie_aspm/parameters/policy for run-time
>> system-wide  control, including for future hot-added devices.
>>   - Remove CONFIG_PCIEASPM_DEBUG and enable that code always, so we
>> have fine-grained run-time control.
>>
>
>Runtime control sounds like a better plan. We need hooks into the system
>power management policy.
>
>>> Maybe, we are missing a HPP option from the PCI spec.
>> That's an interesting idea.  _HPX does have provision for manipulating
>> Link Control bits (see ACPI r5.0, sec 6.2.8.3), but I don't think very
>> many systems implement it.  And there's currently no connection
>> between program_hpp_type2() and aspm.c, so I'm a little worried that
>> we might have issues if a system did implement an _HPX that sets any
>> of the ASPM bits.
>
>I looked at the spec some more. These are there to restore the register
>settings following hotplug insertion. I agree it won't play nice with ASPM
>as the control bits need to be enabled in coordination with the upstream
>device.
>
>I think the right approach is to let the userspace feed the required
>policy to the kernel like you suggested. Then, it needs to be per port
>via link_state to have the most flexibility.
>
>
>--
>Sinan Kaya
>Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
>Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.
Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928