[Bug 215938] New: amd-pstate ignoring scaling_max_freq after waking from suspend

[Bug 215938] New: amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

            Bug ID: 215938
           Summary: amd-pstate ignoring scaling_max_freq after waking from
                    suspend
           Product: Power Management
           Version: 2.5
    Kernel Version: 5.17.5
          Hardware: All
                OS: Linux
              Tree: Mainline
            Status: NEW
          Severity: normal
          Priority: P1
         Component: cpufreq
          Assignee: linux-pm@vger.kernel.org
          Reporter: alexmaras@gmail.com
        Regression: No

Using the amd-pstate driver for cpu scaling, I'm only able to enforce a maximum
clock speed when my laptop has not slept. I do the following:

echo 'ondemand' | sudo tee
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

echo '1400000' | sudo tee
/sys/devices/system/cpu/cpufreq/policy*/scaling_max_freq

cat /sys/devices/system/cpu/cpufreq/policy*/scaling_cur_freq

At this point, all reported frequencies for all cores are below 1.4ghz. If I
then suspend the laptop, then wake it up, then checking the core frequencies
again will show that it is behaving as if the ondemand governor was not set.
This behavior will persist until a reboot, even if the governor is switched to
another mode and then back, or if the scaling_max_freq value is changed. All
settings will be ignored. 

I have replicated this behavior on another identical model of laptop (Lenovo
IdeaPad 5 14ARE05), 

cpupower frequency-info also reports that the current CPU frequency is set to
1.4GHZ and that the governor is set to ondemand.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

Artem S. Tashkinov (aros@gmx.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |aros@gmx.com

--- Comment #1 from Artem S. Tashkinov (aros@gmx.com) ---
Could you please check if bug 215800 affects you?

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #2 from Alex Maras (alexmaras@gmail.com) ---
(In reply to Artem S. Tashkinov from comment #1)
> Could you please check if bug 215800 affects you?

No, I can set my frequency to 3GHZ and run stress and my scaling_cur_freq value
reports ~3GHZ. I'm also on a AMD Ryzen 7 4800U, which is a Zen2 model.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #3 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #2)
> (In reply to Artem S. Tashkinov from comment #1)
> > Could you please check if bug 215800 affects you?
> 
> No, I can set my frequency to 3GHZ and run stress and my scaling_cur_freq
> value reports ~3GHZ. I'm also on a AMD Ryzen 7 4800U, which is a Zen2 model.

Hi Alex.
Could you help to attach the dmesg log and all CPU model info ?
Meanwhile I will check if I can reproduce the issue on my side if I have the
any similar CPU model.

Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #4 from Alex Maras (dev@alexmaras.com) ---
Created attachment 301003
  --> https://bugzilla.kernel.org/attachment.cgi?id=301003&action=edit
dmesg output after sleeping

1. Started up laptop fresh
2. Set governor to ondemand and frequency to 1.4GHZ
3. Checked CPU frequency, settings respected
4. Shut laptop lid, waited 10 seconds, opened laptop lid
5. Checked CPU frequency, settings disregarded
6. Grabbing dmesg output

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #5 from Alex Maras (dev@alexmaras.com) ---
Created attachment 301004
  --> https://bugzilla.kernel.org/attachment.cgi?id=301004&action=edit
cpuinfo

CPUInfo output

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #6 from Alex Maras (dev@alexmaras.com) ---
Hey Perry,

I've attached the /proc/cpuinfo output and dmesg output after the listed
process. Do you want anything more for the moment? Let me know if you need
anything, happy to help.

Alex

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #7 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #6)
> Hey Perry,
> 
> I've attached the /proc/cpuinfo output and dmesg output after the listed
> process. Do you want anything more for the moment? Let me know if you need
> anything, happy to help.
> 
> Alex

Thanks for your info.
Could you try this kernel ?

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/log/drivers/cpufreq/amd-pstate.c?h=v5.18

add this to grub.

amd-pstate.dyndbg=+p  acpi-cpufreq.dyndbg=+p freq_table.dyndbg=+p
cpufreq.dyndbg=+p cpufreq_governor.dyndbg=+p

I didn't find amd pstate driver related log in your last dmesg.
more debug logs will be helpful.

Thanks.
Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

Alex Maras (dev@alexmaras.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
 Attachment #301003|0                           |1
        is obsolete|                            |

--- Comment #8 from Alex Maras (dev@alexmaras.com) ---
Created attachment 301080
  --> https://bugzilla.kernel.org/attachment.cgi?id=301080&action=edit
dmesg output after sleeping - with debug params

I've attached the dmesg output here after applying those kernel params. My
kernel version has the latest commit. I'll compile from git to ensure the
results are the same in the meantime. 

The dmesg results are interesting - cat-ing scaling_cur_freq still shows the
same behavior, but dmesg doesn't show any values above 1.4gHz. I ran `stress`
either side of sleeping and watched the scaling_cur_freq, and can confirm that
it was maxing out at 1.4gHz before suspend, and not limited after suspend.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #9 from Alex Maras (dev@alexmaras.com) ---
Created attachment 301137
  --> https://bugzilla.kernel.org/attachment.cgi?id=301137&action=edit
dmesg output - 5.18

same process with dmesg output with kernel built from 5.18 release tag

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #10 from Alex Maras (dev@alexmaras.com) ---
Note - in the most recent test, I was limiting to 1gHz instead of 1.4 - results
are the same, successfully stating at 1gHz prior to sleeping, unbound after
waking from sleep.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

Perry_Yuan (perry_yuan@dell.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |perry_yuan@dell.com

--- Comment #11 from Perry_Yuan (perry_yuan@dell.com) ---
(In reply to Alex Maras from comment #10)
> Note - in the most recent test, I was limiting to 1gHz instead of 1.4 -
> results are the same, successfully stating at 1gHz prior to sleeping,
> unbound after waking from sleep.

Hi Alex.
Thanks for your detail and info share, I will check the logs soon. then will
give you some feedback.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #12 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #8)
> Created attachment 301080 [details]
> dmesg output after sleeping - with debug params
> 
> I've attached the dmesg output here after applying those kernel params. My
> kernel version has the latest commit. I'll compile from git to ensure the
> results are the same in the meantime. 
> 
> The dmesg results are interesting - cat-ing scaling_cur_freq still shows the
> same behavior, but dmesg doesn't show any values above 1.4gHz. I ran
> `stress` either side of sleeping and watched the scaling_cur_freq, and can
> confirm that it was maxing out at 1.4gHz before suspend, and not limited
> after suspend.

Hi Alex.
Could you help to try if you can change the cores frequency when you switch to
userspace governor after suspend/resume test?


 # sudo cpupower frequency-set -g userspace
 # systemctl suspend 
 Resume from Suspend pressing power button or keyboards.
 # cd /sys/devices/system/cpu/cpufreq/policy0
 # echo 1752589 > scaling_setspeed  //change this core freq to see if it can be
changed 
 # cat scaling_setspeed
 # cat scaling_governor
 # cat scaling_cur_freq


I would like to see if the cpu cores still can change freq or not after
resuming.

If the cpu freq cannot be changed, even you change the cpuinfo_max_freq, core
freq(scaling_cur_freq) will not be limited  successfully as well. 


Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #13 from Alex Maras (dev@alexmaras.com) ---
Hey Perry, results below:

[I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
[sudo] password for alex:
Setting cpu: 0
Setting cpu: 1
Setting cpu: 2
Setting cpu: 3
Setting cpu: 4
Setting cpu: 5
Setting cpu: 6
Setting cpu: 7
Setting cpu: 8
Setting cpu: 9
Setting cpu: 10
Setting cpu: 11
Setting cpu: 12
Setting cpu: 13
Setting cpu: 14
Setting cpu: 15
[I] alex@alex-laptop ~> systemctl suspend
[I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
[I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee
scaling_setspeed
1752589
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
1752589
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
userspace
[I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
2796520
[I] alex@alex-laptop /s/d/s/c/c/policy0>


Looks like the same issue continuing. Do you want the same dmesg with pstate
logging turned on?

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #14 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #13)
> Hey Perry, results below:
> 
> [I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
> [sudo] password for alex:
> Setting cpu: 0
> Setting cpu: 1
> Setting cpu: 2
> Setting cpu: 3
> Setting cpu: 4
> Setting cpu: 5
> Setting cpu: 6
> Setting cpu: 7
> Setting cpu: 8
> Setting cpu: 9
> Setting cpu: 10
> Setting cpu: 11
> Setting cpu: 12
> Setting cpu: 13
> Setting cpu: 14
> Setting cpu: 15
> [I] alex@alex-laptop ~> systemctl suspend
> [I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
> [I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee
> scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
> userspace
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
> 2796520
> [I] alex@alex-laptop /s/d/s/c/c/policy0>
> 
> 
> Looks like the same issue continuing. Do you want the same dmesg with pstate
> logging turned on?

Seeing from you result, issue can be reproduced by my side, the core frequency
is not updated by low level firmware.
I suppose that the issue was caused by suspend/resume , firmware has problem to
resume the CPU frequency control, so any new frequency set will be ignored.
I will further do some debugging and try to resolve the problem.
Thanks for your confirm.

Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

Zhang Rui (rui.zhang@intel.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |rui.zhang@intel.com
           Hardware|All                         |AMD

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #15 from perry_yuan@outlook.com ---
Hi Alex.

I have been preparing some fix patches to resolve the prolem.
Will update here after patches sent.

Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

a@annel.in changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |a@annel.in

--- Comment #16 from a@annel.in ---
*** Bug 216189 has been marked as a duplicate of this bug. ***

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #17 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #13)
> Hey Perry, results below:
> 
> [I] alex@alex-laptop ~> sudo cpupower frequency-set -g userspace
> [sudo] password for alex:
> Setting cpu: 0
> Setting cpu: 1
> Setting cpu: 2
> Setting cpu: 3
> Setting cpu: 4
> Setting cpu: 5
> Setting cpu: 6
> Setting cpu: 7
> Setting cpu: 8
> Setting cpu: 9
> Setting cpu: 10
> Setting cpu: 11
> Setting cpu: 12
> Setting cpu: 13
> Setting cpu: 14
> Setting cpu: 15
> [I] alex@alex-laptop ~> systemctl suspend
> [I] alex@alex-laptop ~> cd /sys/devices/system/cpu/cpufreq/policy0
> [I] alex@alex-laptop /s/d/s/c/c/policy0> echo 1752589 | sudo tee
> scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_setspeed
> 1752589
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_governor
> userspace
> [I] alex@alex-laptop /s/d/s/c/c/policy0> cat scaling_cur_freq
> 2796520
> [I] alex@alex-laptop /s/d/s/c/c/policy0>
> 
> 
> Looks like the same issue continuing. Do you want the same dmesg with pstate
> logging turned on?

The suspend issue has been fixed by below patch.
Please try the below patch for the suspend/resume issue.

https://patchwork.kernel.org/project/linux-pm/patch/20220623031509.555269-1-Jinzhou.Su@amd.com/

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

Alex Maras (dev@alexmaras.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |CODE_FIX

--- Comment #18 from Alex Maras (dev@alexmaras.com) ---
Thanks Perry - tested and working as expected. I'm now able to set the CPU freq
with amd-pstate, suspend, resume and see that the cpu frequency limits are now
respected correctly, and changing both the governor and the frequency limits
works as expected. 

I've set this to resolved with a code fix.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #19 from perry_yuan@outlook.com ---
(In reply to Alex Maras from comment #18)
> Thanks Perry - tested and working as expected. I'm now able to set the CPU
> freq with amd-pstate, suspend, resume and see that the cpu frequency limits
> are now respected correctly, and changing both the governor and the
> frequency limits works as expected. 
> 
> I've set this to resolved with a code fix.

HI Alex.

Good to hear that. 
Thanks for your confirm. 

Perry.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

--- Comment #20 from a@annel.in ---
Guys, have you get amd_pstate working with schedutil governor?  
On my system, it completely ignores frequency limits and boost state. 

```
~  sudo cpupower frequency-set -g schedutil -d 1200MHz -u 1600MHz               
~  sudo cpupower frequency-info                                                 
analyzing CPU 0:
  driver: amd-pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency: 131 us
  hardware limits: 400 MHz - 1.90 GHz
  available cpufreq governors: ondemand performance schedutil
  current policy: frequency should be within 1.20 GHz and 1.60 GHz.
                  The governor "schedutil" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 2.60 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: no
    AMD PSTATE Highest Performance: 166. Maximum Frequency: 4.51 GHz.
    AMD PSTATE Nominal Performance: 70. Nominal Frequency: 1.90 GHz.
    AMD PSTATE Lowest Non-linear Performance: 41. Lowest Non-linear Frequency:
1.11 GHz.
    AMD PSTATE Lowest Performance: 15. Lowest Frequency: 400 MHz.
```

Other governors (ondemand/performance) works as intended.  
Is it a bug or something else?

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 215938] amd-pstate ignoring scaling_max_freq after waking from suspend

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=215938

regulator (tristang@posteo.net) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tristang@posteo.net

--- Comment #21 from regulator (tristang@posteo.net) ---
(In reply to a from comment #20)
> Guys, have you get amd_pstate working with schedutil governor?  
> On my system, it completely ignores frequency limits and boost state. 

On my 5800HS using amd-pstate, the lowest clock speed schedutil goes to is
2500mhz compared to other governors which go down to 400mhz, we should probably
log a separate bug report? This is any time not just when waking from suspend

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.