Linux-ACPI Archive on lore.kernel.org
 help / color / Atom feed
* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
       [not found] ` <1505028180.591737.1589564161284@mail.yahoo.com>
@ 2020-05-15 21:41   ` Andy Shevchenko
  2020-05-18 10:26     ` Rafael J. Wysocki
  0 siblings, 1 reply; 7+ messages in thread
From: Andy Shevchenko @ 2020-05-15 21:41 UTC (permalink / raw)
  To: larsh, Rafael J. Wysocki
  Cc: ibm-acpi-devel, platform-driver-x86, ACPI Devel Maling List

+Cc: ACPI ML and Rafael

On Fri, May 15, 2020 at 8:36 PM larsh@apache.org <larsh@apache.org> wrote:
>
> Hi. I hope this is the right forum to raise this...
>
> For a while I have noticed that my CPU (i9-9880H in a Lenovo X1 Extreme Gen2) never enters any sleep mode below pc2.
> (Confirmed with powertop and /sys/kernel/debug/pmc_core/package_cstate_show)
>
> Interestingly the CPU *can* reachers deeper C states *after* a resume from sleep (either S0ix or S3, i.e. freeze or mem).
>
> This article finally pointed me in the right direction: https://01.org/blogs/qwang59/2020/linux-s0ix-troubleshooting
>
> Somehow SOUTHPORT_A is requesting a max latency of 1 us.
> There are no external devices attached.
>
> This is before a resume:
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show
> SOUTHPORT_A                             LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024             Snoop(ns): 32768           <-------
> SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SATA                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> GIGABIT_ETHERNET                        LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> XHCI                                    LTR: RAW: 0x13ff                Non-Snoop(ns): 0                Snoop(ns): 0
> Reserved                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ME                                      LTR: RAW: 0x8000800             Non-Snoop(ns): 0                Snoop(ns): 0
> EVA                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0
> HD_AUDIO                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CNV                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> LPSS                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CAMERA                                  LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ESPI                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SCC                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ISH                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> UFSX2                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> EMMC                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> WIGIG                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CURRENT_PLATFORM                        LTR: RAW: 0x40201               Non-Snoop(ns): 0                Snoop(ns): 0
> AGGREGATED_SYSTEM                       LTR: RAW: 0x7fbfdfe             Non-Snoop(ns): 0                Snoop(ns): 0
>
> Notice the 1000ns max latency requirement for SOUTHPORT_A.
>
> Ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore subsequently allows the CPU to reach deep sleep states.
>
> After a resume it looks like suddenly SOUTHPORT_C is active and with a less tight latency requirement:
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show
> SOUTHPORT_A                             LTR: RAW: 0x8010c01             Non-Snoop(ns): 0                Snoop(ns): 0               <--------
> SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SATA                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> GIGABIT_ETHERNET                        LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> XHCI                                    LTR: RAW: 0x13ff                Non-Snoop(ns): 0                Snoop(ns): 0
> Reserved                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ME                                      LTR: RAW: 0x8000800             Non-Snoop(ns): 0                Snoop(ns): 0
> EVA                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                             LTR: RAW: 0x88468846            Non-Snoop(ns): 71680            Snoop(ns): 71680           <---------
> HD_AUDIO                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CNV                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> LPSS                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CAMERA                                  LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ESPI                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SCC                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> ISH                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> UFSX2                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> EMMC                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> WIGIG                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> CURRENT_PLATFORM                        LTR: RAW: 0x40201               Non-Snoop(ns): 0                Snoop(ns): 0
> AGGREGATED_SYSTEM                       LTR: RAW: 0x904824              Non-Snoop(ns): 0                Snoop(ns): 0
>
> Does anybody know what's going on or how to debug this further?
>
> As stated above, I was able to work around this problem by ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> There has to be a better way, and I'm sure I'm not the only one running into this.
>
> Thanks.
>
> -- Lars



-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-15 21:41   ` Low Latency Tolerance preventing Intel Package from entering deep sleep states Andy Shevchenko
@ 2020-05-18 10:26     ` Rafael J. Wysocki
  2020-05-19 16:03       ` David E. Box
  0 siblings, 1 reply; 7+ messages in thread
From: Rafael J. Wysocki @ 2020-05-18 10:26 UTC (permalink / raw)
  To: Andy Shevchenko
  Cc: larsh, ibm-acpi-devel, platform-driver-x86,
	ACPI Devel Maling List, David Box

On Friday, May 15, 2020 11:41:16 PM CEST Andy Shevchenko wrote:
> +Cc: ACPI ML and Rafael

+Cc: David Box

> On Fri, May 15, 2020 at 8:36 PM larsh@apache.org <larsh@apache.org> wrote:
> >
> > Hi. I hope this is the right forum to raise this...
> >
> > For a while I have noticed that my CPU (i9-9880H in a Lenovo X1 Extreme Gen2) never enters any sleep mode below pc2.
> > (Confirmed with powertop and /sys/kernel/debug/pmc_core/package_cstate_show)
> >
> > Interestingly the CPU *can* reachers deeper C states *after* a resume from sleep (either S0ix or S3, i.e. freeze or mem).
> >
> > This article finally pointed me in the right direction: https://01.org/blogs/qwang59/2020/linux-s0ix-troubleshooting
> >
> > Somehow SOUTHPORT_A is requesting a max latency of 1 us.
> > There are no external devices attached.
> >
> > This is before a resume:
> >
> > $ cat /sys/kernel/debug/pmc_core/ltr_show
> > SOUTHPORT_A                             LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024             Snoop(ns): 32768           <-------
> > SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SATA                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > GIGABIT_ETHERNET                        LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > XHCI                                    LTR: RAW: 0x13ff                Non-Snoop(ns): 0                Snoop(ns): 0
> > Reserved                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ME                                      LTR: RAW: 0x8000800             Non-Snoop(ns): 0                Snoop(ns): 0
> > EVA                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0
> > HD_AUDIO                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CNV                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > LPSS                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> > SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CAMERA                                  LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ESPI                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SCC                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ISH                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > UFSX2                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > EMMC                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > WIGIG                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CURRENT_PLATFORM                        LTR: RAW: 0x40201               Non-Snoop(ns): 0                Snoop(ns): 0
> > AGGREGATED_SYSTEM                       LTR: RAW: 0x7fbfdfe             Non-Snoop(ns): 0                Snoop(ns): 0
> >
> > Notice the 1000ns max latency requirement for SOUTHPORT_A.
> >
> > Ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore subsequently allows the CPU to reach deep sleep states.
> >
> > After a resume it looks like suddenly SOUTHPORT_C is active and with a less tight latency requirement:
> >
> > $ cat /sys/kernel/debug/pmc_core/ltr_show
> > SOUTHPORT_A                             LTR: RAW: 0x8010c01             Non-Snoop(ns): 0                Snoop(ns): 0               <--------
> > SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SATA                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > GIGABIT_ETHERNET                        LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > XHCI                                    LTR: RAW: 0x13ff                Non-Snoop(ns): 0                Snoop(ns): 0
> > Reserved                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ME                                      LTR: RAW: 0x8000800             Non-Snoop(ns): 0                Snoop(ns): 0
> > EVA                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SOUTHPORT_C                             LTR: RAW: 0x88468846            Non-Snoop(ns): 71680            Snoop(ns): 71680           <---------
> > HD_AUDIO                                LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CNV                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > LPSS                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> > SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CAMERA                                  LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ESPI                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > SCC                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > ISH                                     LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > UFSX2                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > EMMC                                    LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > WIGIG                                   LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> > CURRENT_PLATFORM                        LTR: RAW: 0x40201               Non-Snoop(ns): 0                Snoop(ns): 0
> > AGGREGATED_SYSTEM                       LTR: RAW: 0x904824              Non-Snoop(ns): 0                Snoop(ns): 0
> >
> > Does anybody know what's going on or how to debug this further?
> >
> > As stated above, I was able to work around this problem by ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> > There has to be a better way, and I'm sure I'm not the only one running into this.
> >
> > Thanks.
> >
> > -- Lars
> 
> 
> 
> 





^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-18 10:26     ` Rafael J. Wysocki
@ 2020-05-19 16:03       ` David E. Box
  2020-05-22  6:14         ` larsh
  0 siblings, 1 reply; 7+ messages in thread
From: David E. Box @ 2020-05-19 16:03 UTC (permalink / raw)
  To: Rafael J. Wysocki, Andy Shevchenko
  Cc: larsh, ibm-acpi-devel, platform-driver-x86, ACPI Devel Maling List

> > > Does anybody know what's going on or how to debug this further?
> > > As stated above, I was able to work around this problem by
> > > ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> > > There has to be a better way, and I'm sure I'm not the only one
> > > running into this.

ltr_show shows the PMC's (Power Management Controller) view of SoC
devices and busses. The SOUTHPORTs are the PCIe root ports on your
system. When you run lspci they are the PCI bridges. Generally, the
bridges are enumerated in the same order as the SOUTHPORTs, so
SOUTHPORT_A is your first bridge and the device attached to it (shown
in lspci -t) is the device that was blocking deeper PC states according
to your debug.

Determine what this device is on your system. If the ltr was low it's
because that is what the device requested. You should first check that
runtime pm is enabled for the device. To do this, check the control
file in /sys/bus/pci/devices/<SSSS:BB:DD.F>/power, where SSSS:BB:DD.F
is the enumeration of your device as shown in lspci. If it is 'on' then
runtime pm is disabled. To enable it echo 'auto' into the file with
root privileges. Enabling runtime pm should allow the driver to reduce
functionality of the device when idle. This should lead to a larger
latency request on the PCI bus which should be reflected in ltr_show.
You can see if the device is actually runtime suspended and how much
time it's been suspended (or active) by reading the associated files in
the power folder.

If this doesn't work, then it's possible that your device doesn't
support runtime pm. This may be purposely for reliability reasons or
the driver may just lack support. Check forums discussing issues with
the device and look for possible options in the driver to force pm
support (generally this will be centered around enabling ASPM).

You can also download powertop to see the package c-state residencies
more clearly as percentages of time. powertop also has a tunables tab
that will show the status of runtime pm on all devices on the system
and allow you to enable them individually.

David


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-19 16:03       ` David E. Box
@ 2020-05-22  6:14         ` larsh
  2020-05-22  8:58           ` Andy Shevchenko
  0 siblings, 1 reply; 7+ messages in thread
From: larsh @ 2020-05-22  6:14 UTC (permalink / raw)
  To: Rafael J. Wysocki, Andy Shevchenko, David E. Box
  Cc: ibm-acpi-devel, platform-driver-x86, ACPI Devel Maling List

Thanks David!

With this I tracked down the SD Card Reader (Genesys Logic, Inc Device 9755) as the culprit.
These are standard in many ThinkPads.
The curious part is that resume from suspend (S3 or S0iX) also fixes the problem.
Looks like the driver is not initializing correctly at boot time.

Transcript:

$ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
SOUTHPORT_A                             LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024             Snoop(ns): 32768           
SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0               
SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0               
SOUTHPORT_D                             LTR: RAW: 0x88aa88aa            Non-Snoop(ns): 174080           Snoop(ns): 174080          
SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0               

$ lspci -t
-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-02.0
           +-04.0
           +-08.0
           +-12.0
           +-14.0
           +-14.2
           +-15.0
           +-16.0
           +-1c.0-[53]----00.0
           +-1d.0-[02]----00.0
           +-1d.6-[52]----00.0
           +-1e.0
           +-1f.0
           +-1f.3
           +-1f.4
           +-1f.5
           \-1f.6

$ lspci | grep 53
53:00.0 SD Host controller: Genesys Logic, Inc Device 9755

$ cat /sys/bus/pci/devices/0000\:53\:00.0/power/control
auto

$ echo 1 > /sys/bus/pci/devices/0000\:53\:00.0/remove
1

$ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
SOUTHPORT_A                             LTR: RAW: 0x8010c01             Non-Snoop(ns): 0                Snoop(ns): 0               
SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0               
SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0               
SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512         
SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0               

Cheers.

-- Lars








On Tuesday, May 19, 2020, 9:03:53 AM PDT, David E. Box <david.e.box@linux.intel.com> wrote: 





> > > Does anybody know what's going on or how to debug this further?
> > > As stated above, I was able to work around this problem by
> > > ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> > > There has to be a better way, and I'm sure I'm not the only one
> > > running into this.

ltr_show shows the PMC's (Power Management Controller) view of SoC
devices and busses. The SOUTHPORTs are the PCIe root ports on your
system. When you run lspci they are the PCI bridges. Generally, the
bridges are enumerated in the same order as the SOUTHPORTs, so
SOUTHPORT_A is your first bridge and the device attached to it (shown
in lspci -t) is the device that was blocking deeper PC states according
to your debug.

Determine what this device is on your system. If the ltr was low it's
because that is what the device requested. You should first check that
runtime pm is enabled for the device. To do this, check the control
file in /sys/bus/pci/devices/<SSSS:BB:DD.F>/power, where SSSS:BB:DD.F
is the enumeration of your device as shown in lspci. If it is 'on' then
runtime pm is disabled. To enable it echo 'auto' into the file with
root privileges. Enabling runtime pm should allow the driver to reduce
functionality of the device when idle. This should lead to a larger
latency request on the PCI bus which should be reflected in ltr_show.
You can see if the device is actually runtime suspended and how much
time it's been suspended (or active) by reading the associated files in
the power folder.

If this doesn't work, then it's possible that your device doesn't
support runtime pm. This may be purposely for reliability reasons or
the driver may just lack support. Check forums discussing issues with
the device and look for possible options in the driver to force pm
support (generally this will be centered around enabling ASPM).

You can also download powertop to see the package c-state residencies
more clearly as percentages of time. powertop also has a tunables tab
that will show the status of runtime pm on all devices on the system
and allow you to enable them individually.


David


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-22  6:14         ` larsh
@ 2020-05-22  8:58           ` Andy Shevchenko
  2020-05-22 16:16             ` larsh
  0 siblings, 1 reply; 7+ messages in thread
From: Andy Shevchenko @ 2020-05-22  8:58 UTC (permalink / raw)
  To: larsh, Adrian Hunter
  Cc: Rafael J. Wysocki, David E. Box, ibm-acpi-devel,
	platform-driver-x86, ACPI Devel Maling List

+Cc: Adrian

On Fri, May 22, 2020 at 9:15 AM larsh@apache.org <larsh@apache.org> wrote:
>
> Thanks David!
>
> With this I tracked down the SD Card Reader (Genesys Logic, Inc Device 9755) as the culprit.
> These are standard in many ThinkPads.
> The curious part is that resume from suspend (S3 or S0iX) also fixes the problem.
> Looks like the driver is not initializing correctly at boot time.
>
> Transcript:
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
> SOUTHPORT_A                             LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024             Snoop(ns): 32768
> SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                             LTR: RAW: 0x88aa88aa            Non-Snoop(ns): 174080           Snoop(ns): 174080
> SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
>
> $ lspci -t
> -[0000:00]-+-00.0
>            +-01.0-[01]--+-00.0
>            |            \-00.1
>            +-02.0
>            +-04.0
>            +-08.0
>            +-12.0
>            +-14.0
>            +-14.2
>            +-15.0
>            +-16.0
>            +-1c.0-[53]----00.0
>            +-1d.0-[02]----00.0
>            +-1d.6-[52]----00.0
>            +-1e.0
>            +-1f.0
>            +-1f.3
>            +-1f.4
>            +-1f.5
>            \-1f.6
>
> $ lspci | grep 53
> 53:00.0 SD Host controller: Genesys Logic, Inc Device 9755
>
> $ cat /sys/bus/pci/devices/0000\:53\:00.0/power/control
> auto
>
> $ echo 1 > /sys/bus/pci/devices/0000\:53\:00.0/remove
> 1
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
> SOUTHPORT_A                             LTR: RAW: 0x8010c01             Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_B                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                             LTR: RAW: 0x9f409f4             Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                             LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> SOUTHPORT_E                             LTR: RAW: 0x0                   Non-Snoop(ns): 0                Snoop(ns): 0
>
> Cheers.
>
> -- Lars
>
>
>
>
>
>
>
>
> On Tuesday, May 19, 2020, 9:03:53 AM PDT, David E. Box <david.e.box@linux.intel.com> wrote:
>
>
>
>
>
> > > > Does anybody know what's going on or how to debug this further?
> > > > As stated above, I was able to work around this problem by
> > > > ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> > > > There has to be a better way, and I'm sure I'm not the only one
> > > > running into this.
>
> ltr_show shows the PMC's (Power Management Controller) view of SoC
> devices and busses. The SOUTHPORTs are the PCIe root ports on your
> system. When you run lspci they are the PCI bridges. Generally, the
> bridges are enumerated in the same order as the SOUTHPORTs, so
> SOUTHPORT_A is your first bridge and the device attached to it (shown
> in lspci -t) is the device that was blocking deeper PC states according
> to your debug.
>
> Determine what this device is on your system. If the ltr was low it's
> because that is what the device requested. You should first check that
> runtime pm is enabled for the device. To do this, check the control
> file in /sys/bus/pci/devices/<SSSS:BB:DD.F>/power, where SSSS:BB:DD.F
> is the enumeration of your device as shown in lspci. If it is 'on' then
> runtime pm is disabled. To enable it echo 'auto' into the file with
> root privileges. Enabling runtime pm should allow the driver to reduce
> functionality of the device when idle. This should lead to a larger
> latency request on the PCI bus which should be reflected in ltr_show.
> You can see if the device is actually runtime suspended and how much
> time it's been suspended (or active) by reading the associated files in
> the power folder.
>
> If this doesn't work, then it's possible that your device doesn't
> support runtime pm. This may be purposely for reliability reasons or
> the driver may just lack support. Check forums discussing issues with
> the device and look for possible options in the driver to force pm
> support (generally this will be centered around enabling ASPM).
>
> You can also download powertop to see the package c-state residencies
> more clearly as percentages of time. powertop also has a tunables tab
> that will show the status of runtime pm on all devices on the system
> and allow you to enable them individually.
>
>
> David
>


-- 
With Best Regards,
Andy Shevchenko

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-22  8:58           ` Andy Shevchenko
@ 2020-05-22 16:16             ` larsh
  2020-05-25  5:37               ` Adrian Hunter
  0 siblings, 1 reply; 7+ messages in thread
From: larsh @ 2020-05-22 16:16 UTC (permalink / raw)
  To: Adrian Hunter, Andy Shevchenko
  Cc: Rafael J. Wysocki, David E. Box, ibm-acpi-devel,
	platform-driver-x86, ACPI Devel Maling List

Relevant logs:

May 21 23:05:31 host kernel: sdhci-pci 0000:53:00.0: SDHCI controller found [17a0:9755] (rev 0)
May 21 23:05:31 host kernel: sdhci-pci 0000:53:00.0: enabling device (0000 -> 0002)
May 21 23:05:31 host kernel: mmc0: SDHCI controller on PCI [0000:53:00.0] using ADMA 64-bit



On Friday, May 22, 2020, 1:59:08 AM PDT, Andy Shevchenko <andy.shevchenko@gmail.com> wrote: 





+Cc: Adrian

On Fri, May 22, 2020 at 9:15 AM larsh@apache.org <larsh@apache.org> wrote:
>
> Thanks David!
>
> With this I tracked down the SD Card Reader (Genesys Logic, Inc Device 9755) as the culprit.
> These are standard in many ThinkPads.
> The curious part is that resume from suspend (S3 or S0iX) also fixes the problem.
> Looks like the driver is not initializing correctly at boot time.
>
> Transcript:
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
> SOUTHPORT_A                            LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024            Snoop(ns): 32768
> SOUTHPORT_B                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                            LTR: RAW: 0x9f409f4            Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                            LTR: RAW: 0x88aa88aa            Non-Snoop(ns): 174080          Snoop(ns): 174080
> SOUTHPORT_E                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>
> $ lspci -t
> -[0000:00]-+-00.0
>            +-01.0-[01]--+-00.0
>            |            \-00.1
>            +-02.0
>            +-04.0
>            +-08.0
>            +-12.0
>            +-14.0
>            +-14.2
>            +-15.0
>            +-16.0
>            +-1c.0-[53]----00.0
>            +-1d.0-[02]----00.0
>            +-1d.6-[52]----00.0
>            +-1e.0
>            +-1f.0
>            +-1f.3
>            +-1f.4
>            +-1f.5
>            \-1f.6
>
> $ lspci | grep 53
> 53:00.0 SD Host controller: Genesys Logic, Inc Device 9755
>
> $ cat /sys/bus/pci/devices/0000\:53\:00.0/power/control
> auto
>
> $ echo 1 > /sys/bus/pci/devices/0000\:53\:00.0/remove
> 1
>
> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
> SOUTHPORT_A                            LTR: RAW: 0x8010c01            Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_B                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_C                            LTR: RAW: 0x9f409f4            Non-Snoop(ns): 0                Snoop(ns): 0
> SOUTHPORT_D                            LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
> SOUTHPORT_E                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>
> Cheers.
>
> -- Lars
>
>
>
>
>
>
>
>
> On Tuesday, May 19, 2020, 9:03:53 AM PDT, David E. Box <david.e.box@linux.intel.com> wrote:
>
>
>
>
>
> > > > Does anybody know what's going on or how to debug this further?
> > > > As stated above, I was able to work around this problem by
> > > > ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
> > > > There has to be a better way, and I'm sure I'm not the only one
> > > > running into this.
>
> ltr_show shows the PMC's (Power Management Controller) view of SoC
> devices and busses. The SOUTHPORTs are the PCIe root ports on your
> system. When you run lspci they are the PCI bridges. Generally, the
> bridges are enumerated in the same order as the SOUTHPORTs, so
> SOUTHPORT_A is your first bridge and the device attached to it (shown
> in lspci -t) is the device that was blocking deeper PC states according
> to your debug.
>
> Determine what this device is on your system. If the ltr was low it's
> because that is what the device requested. You should first check that
> runtime pm is enabled for the device. To do this, check the control
> file in /sys/bus/pci/devices/<SSSS:BB:DD.F>/power, where SSSS:BB:DD.F
> is the enumeration of your device as shown in lspci. If it is 'on' then
> runtime pm is disabled. To enable it echo 'auto' into the file with
> root privileges. Enabling runtime pm should allow the driver to reduce
> functionality of the device when idle. This should lead to a larger
> latency request on the PCI bus which should be reflected in ltr_show.
> You can see if the device is actually runtime suspended and how much
> time it's been suspended (or active) by reading the associated files in
> the power folder.
>
> If this doesn't work, then it's possible that your device doesn't
> support runtime pm. This may be purposely for reliability reasons or
> the driver may just lack support. Check forums discussing issues with
> the device and look for possible options in the driver to force pm
> support (generally this will be centered around enabling ASPM).
>
> You can also download powertop to see the package c-state residencies
> more clearly as percentages of time. powertop also has a tunables tab
> that will show the status of runtime pm on all devices on the system
> and allow you to enable them individually.
>
>
> David

>


-- 
With Best Regards,
Andy Shevchenko


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Low Latency Tolerance preventing Intel Package from entering deep sleep states
  2020-05-22 16:16             ` larsh
@ 2020-05-25  5:37               ` Adrian Hunter
  0 siblings, 0 replies; 7+ messages in thread
From: Adrian Hunter @ 2020-05-25  5:37 UTC (permalink / raw)
  To: larsh, Andy Shevchenko
  Cc: Rafael J. Wysocki, David E. Box, ibm-acpi-devel,
	platform-driver-x86, ACPI Devel Maling List, Ben Chuang

+ Ben Chuang <ben.chuang@genesyslogic.com.tw>

It does not look like that driver allows runtime pm.


On 22/05/20 7:16 pm, larsh@apache.org wrote:
> Relevant logs:
> 
> May 21 23:05:31 host kernel: sdhci-pci 0000:53:00.0: SDHCI controller found [17a0:9755] (rev 0)
> May 21 23:05:31 host kernel: sdhci-pci 0000:53:00.0: enabling device (0000 -> 0002)
> May 21 23:05:31 host kernel: mmc0: SDHCI controller on PCI [0000:53:00.0] using ADMA 64-bit
> 
> 
> 
> On Friday, May 22, 2020, 1:59:08 AM PDT, Andy Shevchenko <andy.shevchenko@gmail.com> wrote: 
> 
> 
> 
> 
> 
> +Cc: Adrian
> 
> On Fri, May 22, 2020 at 9:15 AM larsh@apache.org <larsh@apache.org> wrote:
>>
>> Thanks David!
>>
>> With this I tracked down the SD Card Reader (Genesys Logic, Inc Device 9755) as the culprit.
>> These are standard in many ThinkPads.
>> The curious part is that resume from suspend (S3 or S0iX) also fixes the problem.
>> Looks like the driver is not initializing correctly at boot time.
>>
>> Transcript:
>>
>> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
>> SOUTHPORT_A                            LTR: RAW: 0x88018c01            Non-Snoop(ns): 1024            Snoop(ns): 32768
>> SOUTHPORT_B                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>> SOUTHPORT_C                            LTR: RAW: 0x9f409f4            Non-Snoop(ns): 0                Snoop(ns): 0
>> SOUTHPORT_D                            LTR: RAW: 0x88aa88aa            Non-Snoop(ns): 174080          Snoop(ns): 174080
>> SOUTHPORT_E                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>>
>> $ lspci -t
>> -[0000:00]-+-00.0
>>             +-01.0-[01]--+-00.0
>>             |            \-00.1
>>             +-02.0
>>             +-04.0
>>             +-08.0
>>             +-12.0
>>             +-14.0
>>             +-14.2
>>             +-15.0
>>             +-16.0
>>             +-1c.0-[53]----00.0
>>             +-1d.0-[02]----00.0
>>             +-1d.6-[52]----00.0
>>             +-1e.0
>>             +-1f.0
>>             +-1f.3
>>             +-1f.4
>>             +-1f.5
>>             \-1f.6
>>
>> $ lspci | grep 53
>> 53:00.0 SD Host controller: Genesys Logic, Inc Device 9755
>>
>> $ cat /sys/bus/pci/devices/0000\:53\:00.0/power/control
>> auto
>>
>> $ echo 1 > /sys/bus/pci/devices/0000\:53\:00.0/remove
>> 1
>>
>> $ cat /sys/kernel/debug/pmc_core/ltr_show | grep SOUTHPORT
>> SOUTHPORT_A                            LTR: RAW: 0x8010c01            Non-Snoop(ns): 0                Snoop(ns): 0
>> SOUTHPORT_B                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>> SOUTHPORT_C                            LTR: RAW: 0x9f409f4            Non-Snoop(ns): 0                Snoop(ns): 0
>> SOUTHPORT_D                            LTR: RAW: 0x8c548c54            Non-Snoop(ns): 2752512          Snoop(ns): 2752512
>> SOUTHPORT_E                            LTR: RAW: 0x0                  Non-Snoop(ns): 0                Snoop(ns): 0
>>
>> Cheers.
>>
>> -- Lars
>>
>>
>>
>>
>>
>>
>>
>>
>> On Tuesday, May 19, 2020, 9:03:53 AM PDT, David E. Box <david.e.box@linux.intel.com> wrote:
>>
>>
>>
>>
>>
>>>>> Does anybody know what's going on or how to debug this further?
>>>>> As stated above, I was able to work around this problem by
>>>>> ignoring SOUTHPORT_A via /sys/kernel/debug/pmc_core/ltr_ignore.
>>>>> There has to be a better way, and I'm sure I'm not the only one
>>>>> running into this.
>>
>> ltr_show shows the PMC's (Power Management Controller) view of SoC
>> devices and busses. The SOUTHPORTs are the PCIe root ports on your
>> system. When you run lspci they are the PCI bridges. Generally, the
>> bridges are enumerated in the same order as the SOUTHPORTs, so
>> SOUTHPORT_A is your first bridge and the device attached to it (shown
>> in lspci -t) is the device that was blocking deeper PC states according
>> to your debug.
>>
>> Determine what this device is on your system. If the ltr was low it's
>> because that is what the device requested. You should first check that
>> runtime pm is enabled for the device. To do this, check the control
>> file in /sys/bus/pci/devices/<SSSS:BB:DD.F>/power, where SSSS:BB:DD.F
>> is the enumeration of your device as shown in lspci. If it is 'on' then
>> runtime pm is disabled. To enable it echo 'auto' into the file with
>> root privileges. Enabling runtime pm should allow the driver to reduce
>> functionality of the device when idle. This should lead to a larger
>> latency request on the PCI bus which should be reflected in ltr_show.
>> You can see if the device is actually runtime suspended and how much
>> time it's been suspended (or active) by reading the associated files in
>> the power folder.
>>
>> If this doesn't work, then it's possible that your device doesn't
>> support runtime pm. This may be purposely for reliability reasons or
>> the driver may just lack support. Check forums discussing issues with
>> the device and look for possible options in the driver to force pm
>> support (generally this will be centered around enabling ASPM).
>>
>> You can also download powertop to see the package c-state residencies
>> more clearly as percentages of time. powertop also has a tunables tab
>> that will show the status of runtime pm on all devices on the system
>> and allow you to enable them individually.
>>
>>
>> David
> 
>>
> 
> 


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, back to index

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <1505028180.591737.1589564161284.ref@mail.yahoo.com>
     [not found] ` <1505028180.591737.1589564161284@mail.yahoo.com>
2020-05-15 21:41   ` Low Latency Tolerance preventing Intel Package from entering deep sleep states Andy Shevchenko
2020-05-18 10:26     ` Rafael J. Wysocki
2020-05-19 16:03       ` David E. Box
2020-05-22  6:14         ` larsh
2020-05-22  8:58           ` Andy Shevchenko
2020-05-22 16:16             ` larsh
2020-05-25  5:37               ` Adrian Hunter

Linux-ACPI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-acpi/0 linux-acpi/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-acpi linux-acpi/ https://lore.kernel.org/linux-acpi \
		linux-acpi@vger.kernel.org
	public-inbox-index linux-acpi

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-acpi


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git