Power-saving/performance toggles for amdgpu

Power-saving/performance toggles for amdgpu

[Bastien Nocera]

Hey,

I'm currently working on a daemon that allows a UI that will eventually
look like the power mode section in this mockup:
https://gitlab.gnome.org/Teams/Design/settings-mockups/-/blob/master/power/power.png

The daemon:
https://gitlab.freedesktop.org/hadess/power-profiles-daemon
The UI:
https://gitlab.gnome.org/GNOME/gnome-control-center/-/merge_requests/816

It might be interesting, on some systems, to nudge the GPU drivers
in the direction wanted by the user, either enabling aggressive power-
saving or allowing more power to be used for better temporary
performance.

I was wondering whether the tweaks
in /sys/class/drm/card*/device/power_profile
and /sys/class/drm/card*/device/power_method were things that this
daemon could and should tweak, or whether it was present solely for
debug purposes and shouldn't be needed even when high performance or
power-saving is wanted:
https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/3

Thanks in advance!

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: Power-saving/performance toggles for amdgpu

[Alex Deucher]

On Mon, Sep 7, 2020 at 10:30 AM Bastien Nocera <hadess@hadess.net> wrote:
>
> Hey,
>
> I'm currently working on a daemon that allows a UI that will eventually
> look like the power mode section in this mockup:
> https://gitlab.gnome.org/Teams/Design/settings-mockups/-/blob/master/power/power.png
>
> The daemon:
> https://gitlab.freedesktop.org/hadess/power-profiles-daemon
> The UI:
> https://gitlab.gnome.org/GNOME/gnome-control-center/-/merge_requests/816
>
> It might be interesting, on some systems, to nudge the GPU drivers
> in the direction wanted by the user, either enabling aggressive power-
> saving or allowing more power to be used for better temporary
> performance.
>
> I was wondering whether the tweaks
> in /sys/class/drm/card*/device/power_profile
> and /sys/class/drm/card*/device/power_method were things that this
> daemon could and should tweak, or whether it was present solely for
> debug purposes and shouldn't be needed even when high performance or
> power-saving is wanted:
> https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/3
>
> Thanks in advance!
>

On older radeons (e.g., pre-GCN hardware), there were separate power
states for battery and AC, but these asics are supported by the radeon
kernel driver.  None of the hardware supported by amdgpu exposes
anything like that anymore.  The rest is mainly for profiling and
debugging.  For more information see the relevant kernel
documentation:
https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
I don't think there is anything you'd want to tweak there.

Alex
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: Power-saving/performance toggles for amdgpu

[Bastien Nocera]

On Mon, 2020-09-14 at 01:46 -0400, Alex Deucher wrote:
> 
<snip>
> On older radeons (e.g., pre-GCN hardware), there were separate power
> states for battery and AC, but these asics are supported by the
> radeon
> kernel driver.  None of the hardware supported by amdgpu exposes
> anything like that anymore.  The rest is mainly for profiling and
> debugging.  For more information see the relevant kernel
> documentation:
> https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
> I don't think there is anything you'd want to tweak there.

That was very helpful, thanks very much!

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: Power-saving/performance toggles for amdgpu

[Bastien Nocera]

Nearly a year later, hello again, :)

On Mon, 2020-09-14 at 01:46 -0400, Alex Deucher wrote:
> On older radeons (e.g., pre-GCN hardware), there were separate power
> states for battery and AC, but these asics are supported by the radeon
> kernel driver.  None of the hardware supported by amdgpu exposes
> anything like that anymore.  The rest is mainly for profiling and
> debugging.  For more information see the relevant kernel
> documentation:
> https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
> I don't think there is anything you'd want to tweak there.

Is the power_dpm_force_performance_level sysfs property for the amdgpu
driver not something that one could tweak?
https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#power-dpm-force-performance-level

System76's own power management daemon changes it:
https://github.com/pop-os/system76-power/blob/master/src/radeon.rs

So I'm wondering whether it would have any effect, for example, I would
expect setting "high" when a performance mode is requested so that
there's little latency in terms of frequency switching, "low" to force
minimal power draw in a power saving mode, and "auto" the rest of the
time.

Cheers

Re: Power-saving/performance toggles for amdgpu

[Alex Deucher]

On Fri, Aug 6, 2021 at 10:29 AM Bastien Nocera <hadess@hadess.net> wrote:
>
> Nearly a year later, hello again, :)
>
> On Mon, 2020-09-14 at 01:46 -0400, Alex Deucher wrote:
> > On older radeons (e.g., pre-GCN hardware), there were separate power
> > states for battery and AC, but these asics are supported by the radeon
> > kernel driver.  None of the hardware supported by amdgpu exposes
> > anything like that anymore.  The rest is mainly for profiling and
> > debugging.  For more information see the relevant kernel
> > documentation:
> > https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
> > I don't think there is anything you'd want to tweak there.
>
> Is the power_dpm_force_performance_level sysfs property for the amdgpu
> driver not something that one could tweak?
> https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#power-dpm-force-performance-level
>
> System76's own power management daemon changes it:
> https://github.com/pop-os/system76-power/blob/master/src/radeon.rs
>
> So I'm wondering whether it would have any effect, for example, I would
> expect setting "high" when a performance mode is requested so that
> there's little latency in terms of frequency switching, "low" to force
> minimal power draw in a power saving mode, and "auto" the rest of the
> time.

One could, but it's mainly there for debugging or profiling.  There is
a microcontroller on the GPU that dynamically adjusts the clocks and
voltages at runtime based on GPU load.  It's tuned to do a good job in
as many cases as possible by default.  If you really want to tweak
things, it would probably be better to adjust pp_power_profile_mode.
That has a bunch of preset profiles (or you can specify a custom one)
that adjust the heuristics (how quickly the clocks ramp up/down, etc.)
for the dynamic reclocking done by the GPU.  That said, for most
workloads, I doubt you'll see much of a change.  When the GPU is idle,
clock and power gating kick in for most blocks and we also support
runtime power management so the dGPU will effectively get turned off
if it's idle.  The only tricky part with runtime power management, is
that we can't enable it if the framebuffer console is enabled because
the kernel keeps a persistent kmap of the framebuffer, so we can't
power down the GPU because we never know when an access might come in.
We probably need some sort of deferred IO or shadow framebuffer design
to handle that, but we haven't had time to delve into fixing that.

Alex

Re: Power-saving/performance toggles for amdgpu

[Bastien Nocera]

On Fri, 2021-08-06 at 11:08 -0400, Alex Deucher wrote:
> On Fri, Aug 6, 2021 at 10:29 AM Bastien Nocera <hadess@hadess.net>
> wrote:
> > 
> > Nearly a year later, hello again, :)
> > 
> > On Mon, 2020-09-14 at 01:46 -0400, Alex Deucher wrote:
> > > On older radeons (e.g., pre-GCN hardware), there were separate
> > > power
> > > states for battery and AC, but these asics are supported by the
> > > radeon
> > > kernel driver.  None of the hardware supported by amdgpu exposes
> > > anything like that anymore.  The rest is mainly for profiling and
> > > debugging.  For more information see the relevant kernel
> > > documentation:
> > > https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
> > > I don't think there is anything you'd want to tweak there.
> > 
> > Is the power_dpm_force_performance_level sysfs property for the
> > amdgpu
> > driver not something that one could tweak?
> > https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#power-dpm-force-performance-level
> > 
> > System76's own power management daemon changes it:
> > https://github.com/pop-os/system76-power/blob/master/src/radeon.rs
> > 
> > So I'm wondering whether it would have any effect, for example, I
> > would
> > expect setting "high" when a performance mode is requested so that
> > there's little latency in terms of frequency switching, "low" to
> > force
> > minimal power draw in a power saving mode, and "auto" the rest of
> > the
> > time.
> 
> One could, but it's mainly there for debugging or profiling.  There
> is
> a microcontroller on the GPU that dynamically adjusts the clocks and
> voltages at runtime based on GPU load.  It's tuned to do a good job
> in
> as many cases as possible by default.  If you really want to tweak
> things, it would probably be better to adjust pp_power_profile_mode.
> That has a bunch of preset profiles (or you can specify a custom one)
> that adjust the heuristics (how quickly the clocks ramp up/down,
> etc.)
> for the dynamic reclocking done by the GPU.

But the microcontroller doesn't really know user-intent, and can't
really predict the future, which is the reason why a lot of OSes still
have power profile selection knobs.

So I'm mostly wondering whether:
- those clock ramping transitions could be a problem on heavy workloads
with varying intensity, say stuttering in a game that needs to be able
to go from simple to really complicated in short order
- setting the minimum clock would avoid short bursts of activity
clocking up then down (in a GPU-based desktop environment for example),
thus reducing the battery life

>   That said, for most
> workloads, I doubt you'll see much of a change.

I would indeed expect "automatic" to work as expected, but not always
be what the user intends for the GPU to be doing.

>   When the GPU is idle,
> clock and power gating kick in for most blocks and we also support
> runtime power management so the dGPU will effectively get turned off
> if it's idle.  The only tricky part with runtime power management, is
> that we can't enable it if the framebuffer console is enabled because
> the kernel keeps a persistent kmap of the framebuffer, so we can't
> power down the GPU because we never know when an access might come
> in.
> We probably need some sort of deferred IO or shadow framebuffer
> design
> to handle that, but we haven't had time to delve into fixing that.
> 
> Alex

Re: Power-saving/performance toggles for amdgpu

[Alex Deucher]

On Fri, Aug 6, 2021 at 11:37 AM Bastien Nocera <hadess@hadess.net> wrote:
>
> On Fri, 2021-08-06 at 11:08 -0400, Alex Deucher wrote:
> > On Fri, Aug 6, 2021 at 10:29 AM Bastien Nocera <hadess@hadess.net>
> > wrote:
> > >
> > > Nearly a year later, hello again, :)
> > >
> > > On Mon, 2020-09-14 at 01:46 -0400, Alex Deucher wrote:
> > > > On older radeons (e.g., pre-GCN hardware), there were separate
> > > > power
> > > > states for battery and AC, but these asics are supported by the
> > > > radeon
> > > > kernel driver.  None of the hardware supported by amdgpu exposes
> > > > anything like that anymore.  The rest is mainly for profiling and
> > > > debugging.  For more information see the relevant kernel
> > > > documentation:
> > > > https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#gpu-power-thermal-controls-and-monitoring
> > > > I don't think there is anything you'd want to tweak there.
> > >
> > > Is the power_dpm_force_performance_level sysfs property for the
> > > amdgpu
> > > driver not something that one could tweak?
> > > https://www.kernel.org/doc/html/latest/gpu/amdgpu.html#power-dpm-force-performance-level
> > >
> > > System76's own power management daemon changes it:
> > > https://github.com/pop-os/system76-power/blob/master/src/radeon.rs
> > >
> > > So I'm wondering whether it would have any effect, for example, I
> > > would
> > > expect setting "high" when a performance mode is requested so that
> > > there's little latency in terms of frequency switching, "low" to
> > > force
> > > minimal power draw in a power saving mode, and "auto" the rest of
> > > the
> > > time.
> >
> > One could, but it's mainly there for debugging or profiling.  There
> > is
> > a microcontroller on the GPU that dynamically adjusts the clocks and
> > voltages at runtime based on GPU load.  It's tuned to do a good job
> > in
> > as many cases as possible by default.  If you really want to tweak
> > things, it would probably be better to adjust pp_power_profile_mode.
> > That has a bunch of preset profiles (or you can specify a custom one)
> > that adjust the heuristics (how quickly the clocks ramp up/down,
> > etc.)
> > for the dynamic reclocking done by the GPU.
>
> But the microcontroller doesn't really know user-intent, and can't
> really predict the future, which is the reason why a lot of OSes still
> have power profile selection knobs.
>
> So I'm mostly wondering whether:
> - those clock ramping transitions could be a problem on heavy workloads
> with varying intensity, say stuttering in a game that needs to be able
> to go from simple to really complicated in short order
> - setting the minimum clock would avoid short bursts of activity
> clocking up then down (in a GPU-based desktop environment for example),
> thus reducing the battery life

You could set one of the profiles which sets more or less aggressive
clocking, but you still get the advantages of the SMU being able to
dynamically adjust the clocks.  If you manually force the clock to low
or high, you end up forcing all clocks, even if a particular engine is
not in use.  E.g., if you are not using video decode, there is no need
to force the decoder clocks high as well.  Also, if the userspace tool
dies for some reason, that will leave the clocks in the forced state.

Alex

>
> >   That said, for most
> > workloads, I doubt you'll see much of a change.
>
> I would indeed expect "automatic" to work as expected, but not always
> be what the user intends for the GPU to be doing.
>
> >   When the GPU is idle,
> > clock and power gating kick in for most blocks and we also support
> > runtime power management so the dGPU will effectively get turned off
> > if it's idle.  The only tricky part with runtime power management, is
> > that we can't enable it if the framebuffer console is enabled because
> > the kernel keeps a persistent kmap of the framebuffer, so we can't
> > power down the GPU because we never know when an access might come
> > in.
> > We probably need some sort of deferred IO or shadow framebuffer
> > design
> > to handle that, but we haven't had time to delve into fixing that.
> >
> > Alex
>

Re: Power-saving/performance toggles for amdgpu

[Bastien Nocera]

On Fri, 2021-08-06 at 11:45 -0400, Alex Deucher wrote:
> You could set one of the profiles which sets more or less aggressive
> clocking, but you still get the advantages of the SMU being able to
> dynamically adjust the clocks.  If you manually force the clock to low
> or high, you end up forcing all clocks, even if a particular engine is
> not in use.  E.g., if you are not using video decode, there is no need
> to force the decoder clocks high as well.  Also, if the userspace tool
> dies for some reason, that will leave the clocks in the forced state.

This looks like the best option for that use case. I've documented it
here:
https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/3#note_1021578

Thanks very much for your help!

Re: Power-saving/performance toggles for amdgpu

[Alex Deucher]

On Fri, Aug 6, 2021 at 12:20 PM Bastien Nocera <hadess@hadess.net> wrote:
>
> On Fri, 2021-08-06 at 11:45 -0400, Alex Deucher wrote:
> > You could set one of the profiles which sets more or less aggressive
> > clocking, but you still get the advantages of the SMU being able to
> > dynamically adjust the clocks.  If you manually force the clock to low
> > or high, you end up forcing all clocks, even if a particular engine is
> > not in use.  E.g., if you are not using video decode, there is no need
> > to force the decoder clocks high as well.  Also, if the userspace tool
> > dies for some reason, that will leave the clocks in the forced state.
>
> This looks like the best option for that use case. I've documented it
> here:
> https://gitlab.freedesktop.org/hadess/power-profiles-daemon/-/issues/3#note_1021578
>
> Thanks very much for your help!

No problem.

Alex