* [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
@ 2023-02-17 16:13 Wyes Karny
2023-02-19 11:52 ` Oleksandr Natalenko
2023-02-20 11:45 ` Peter Zijlstra
0 siblings, 2 replies; 10+ messages in thread
From: Wyes Karny @ 2023-02-17 16:13 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
Thomas Gleixner, Borislav Petkov, Dave Hansen, hpa
Cc: x86, linux-perf-users, linux-kernel, gautham.shenoy,
ananth.narayan, Wyes Karny
AMD processors support per-package and per-core energy monitoring
through RAPL counters which can be accessed by users running in
supervisor mode.
Core RAPL counters gives power consumption information per core. For
AMD processors the package level RAPL counter are already exposed to
perf. Expose the core level RAPL counters also.
sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
Output:
S0-D0-C0 2 8.73 Joules power/energy-cores/
S0-D0-C1 2 8.73 Joules power/energy-cores/
S0-D0-C2 2 8.73 Joules power/energy-cores/
S0-D0-C3 2 8.73 Joules power/energy-cores/
S0-D0-C4 2 8.73 Joules power/energy-cores/
Signed-off-by: Wyes Karny <wyes.karny@amd.com>
---
arch/x86/events/rapl.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
index 52e6e7ed4f78..d301bbbc3b93 100644
--- a/arch/x86/events/rapl.c
+++ b/arch/x86/events/rapl.c
@@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
* - want to use same event codes across both architectures
*/
static struct perf_msr amd_rapl_msrs[] = {
- [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
+ [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
[PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
[PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
[PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
@@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
};
static struct rapl_model model_amd_hygon = {
- .events = BIT(PERF_RAPL_PKG),
+ .events = BIT(PERF_RAPL_PP0) |
+ BIT(PERF_RAPL_PKG),
.msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
.rapl_msrs = amd_rapl_msrs,
};
--
2.34.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-17 16:13 [PATCH] perf/x86/rapl: Enable Core RAPL for AMD Wyes Karny
@ 2023-02-19 11:52 ` Oleksandr Natalenko
2023-03-01 18:32 ` Wyes Karny
2023-02-20 11:45 ` Peter Zijlstra
1 sibling, 1 reply; 10+ messages in thread
From: Oleksandr Natalenko @ 2023-02-19 11:52 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
Thomas Gleixner, Borislav Petkov, Dave Hansen, hpa, Wyes Karny
Cc: x86, linux-perf-users, linux-kernel, gautham.shenoy,
ananth.narayan, Wyes Karny
Hello.
On pátek 17. února 2023 17:13:54 CET Wyes Karny wrote:
> AMD processors support per-package and per-core energy monitoring
> through RAPL counters which can be accessed by users running in
> supervisor mode.
>
> Core RAPL counters gives power consumption information per core. For
> AMD processors the package level RAPL counter are already exposed to
> perf. Expose the core level RAPL counters also.
>
> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>
> Output:
> S0-D0-C0 2 8.73 Joules power/energy-cores/
> S0-D0-C1 2 8.73 Joules power/energy-cores/
> S0-D0-C2 2 8.73 Joules power/energy-cores/
> S0-D0-C3 2 8.73 Joules power/energy-cores/
> S0-D0-C4 2 8.73 Joules power/energy-cores/
>
> Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> ---
> arch/x86/events/rapl.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
> index 52e6e7ed4f78..d301bbbc3b93 100644
> --- a/arch/x86/events/rapl.c
> +++ b/arch/x86/events/rapl.c
> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
> * - want to use same event codes across both architectures
> */
> static struct perf_msr amd_rapl_msrs[] = {
> - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
> + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
> [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
> [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
> [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
> };
>
> static struct rapl_model model_amd_hygon = {
> - .events = BIT(PERF_RAPL_PKG),
> + .events = BIT(PERF_RAPL_PP0) |
> + BIT(PERF_RAPL_PKG),
> .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
> .rapl_msrs = amd_rapl_msrs,
> };
>
With this patch:
```
$ lscpu | grep 'Model name'
Model name: AMD Ryzen 9 5950X 16-Core Processor
$ sudo perf stat -a --per-core -C 0-15 -e power/energy-cores/ -- dd if=/dev/zero of=/dev/null bs=1M count=100000
100000+0 records in
100000+0 records out
104857600000 bytes (105 GB, 98 GiB) copied, 1,59252 s, 65,8 GB/s
Performance counter stats for 'system wide':
S0-D0-C0 1 1,56 Joules power/energy-cores/
S0-D0-C1 1 1,56 Joules power/energy-cores/
S0-D0-C2 1 1,56 Joules power/energy-cores/
S0-D0-C3 1 1,56 Joules power/energy-cores/
S0-D0-C4 1 1,56 Joules power/energy-cores/
S0-D0-C5 1 1,56 Joules power/energy-cores/
S0-D0-C6 1 1,56 Joules power/energy-cores/
S0-D0-C7 1 1,56 Joules power/energy-cores/
S0-D0-C8 1 1,56 Joules power/energy-cores/
S0-D0-C9 1 1,56 Joules power/energy-cores/
S0-D0-C10 1 1,56 Joules power/energy-cores/
S0-D0-C11 1 1,56 Joules power/energy-cores/
S0-D0-C12 1 1,56 Joules power/energy-cores/
S0-D0-C13 1 1,56 Joules power/energy-cores/
S0-D0-C14 1 1,56 Joules power/energy-cores/
S0-D0-C15 1 1,56 Joules power/energy-cores/
1,593982452 seconds time elapsed
```
Hence,
Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
Thank you.
--
Oleksandr Natalenko (post-factum)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-17 16:13 [PATCH] perf/x86/rapl: Enable Core RAPL for AMD Wyes Karny
2023-02-19 11:52 ` Oleksandr Natalenko
@ 2023-02-20 11:45 ` Peter Zijlstra
2023-02-20 21:29 ` Stephane Eranian
1 sibling, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2023-02-20 11:45 UTC (permalink / raw)
To: Wyes Karny, Stephane Eranian
Cc: Ingo Molnar, Arnaldo Carvalho de Melo, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Namhyung Kim, Thomas Gleixner,
Borislav Petkov, Dave Hansen, hpa, x86, linux-perf-users,
linux-kernel, gautham.shenoy, ananth.narayan
On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
> AMD processors support per-package and per-core energy monitoring
> through RAPL counters which can be accessed by users running in
> supervisor mode.
>
> Core RAPL counters gives power consumption information per core. For
> AMD processors the package level RAPL counter are already exposed to
> perf. Expose the core level RAPL counters also.
>
> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>
> Output:
> S0-D0-C0 2 8.73 Joules power/energy-cores/
> S0-D0-C1 2 8.73 Joules power/energy-cores/
> S0-D0-C2 2 8.73 Joules power/energy-cores/
> S0-D0-C3 2 8.73 Joules power/energy-cores/
> S0-D0-C4 2 8.73 Joules power/energy-cores/
>
> Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> ---
> arch/x86/events/rapl.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
> index 52e6e7ed4f78..d301bbbc3b93 100644
> --- a/arch/x86/events/rapl.c
> +++ b/arch/x86/events/rapl.c
> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
> * - want to use same event codes across both architectures
> */
> static struct perf_msr amd_rapl_msrs[] = {
> - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
> + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
Stephane, this was an oversight?
> [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
> [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
> [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
> };
>
> static struct rapl_model model_amd_hygon = {
> - .events = BIT(PERF_RAPL_PKG),
> + .events = BIT(PERF_RAPL_PP0) |
> + BIT(PERF_RAPL_PKG),
> .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
> .rapl_msrs = amd_rapl_msrs,
> };
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-20 11:45 ` Peter Zijlstra
@ 2023-02-20 21:29 ` Stephane Eranian
2023-02-21 4:53 ` Wyes Karny
0 siblings, 1 reply; 10+ messages in thread
From: Stephane Eranian @ 2023-02-20 21:29 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Wyes Karny, Ingo Molnar, Arnaldo Carvalho de Melo, Mark Rutland,
Alexander Shishkin, Jiri Olsa, Namhyung Kim, Thomas Gleixner,
Borislav Petkov, Dave Hansen, hpa, x86, linux-perf-users,
linux-kernel, gautham.shenoy, ananth.narayan
On Mon, Feb 20, 2023 at 3:45 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
> > AMD processors support per-package and per-core energy monitoring
> > through RAPL counters which can be accessed by users running in
> > supervisor mode.
> >
> > Core RAPL counters gives power consumption information per core. For
> > AMD processors the package level RAPL counter are already exposed to
> > perf. Expose the core level RAPL counters also.
> >
> > sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
> >
> > Output:
> > S0-D0-C0 2 8.73 Joules power/energy-cores/
> > S0-D0-C1 2 8.73 Joules power/energy-cores/
> > S0-D0-C2 2 8.73 Joules power/energy-cores/
> > S0-D0-C3 2 8.73 Joules power/energy-cores/
> > S0-D0-C4 2 8.73 Joules power/energy-cores/
> >
> > Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> > ---
> > arch/x86/events/rapl.c | 5 +++--
> > 1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
> > index 52e6e7ed4f78..d301bbbc3b93 100644
> > --- a/arch/x86/events/rapl.c
> > +++ b/arch/x86/events/rapl.c
> > @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
> > * - want to use same event codes across both architectures
> > */
> > static struct perf_msr amd_rapl_msrs[] = {
> > - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
> > + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
>
> Stephane, this was an oversight?
>
I think it may depend on the CPU model. I remember it returning either
0 or bogus values on my systems. They may have improved that.
The commit msg does not show which CPU model this is run on.
>
> > [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
> > [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
> > [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
> > @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
> > };
> >
> > static struct rapl_model model_amd_hygon = {
> > - .events = BIT(PERF_RAPL_PKG),
> > + .events = BIT(PERF_RAPL_PP0) |
> > + BIT(PERF_RAPL_PKG),
> > .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
> > .rapl_msrs = amd_rapl_msrs,
> > };
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-20 21:29 ` Stephane Eranian
@ 2023-02-21 4:53 ` Wyes Karny
2023-02-21 8:50 ` Stephane Eranian
0 siblings, 1 reply; 10+ messages in thread
From: Wyes Karny @ 2023-02-21 4:53 UTC (permalink / raw)
To: Stephane Eranian
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
Thomas Gleixner, Borislav Petkov, Dave Hansen, hpa, x86,
linux-perf-users, linux-kernel, gautham.shenoy, ananth.narayan
On 20 Feb 13:29, Stephane Eranian wrote:
> On Mon, Feb 20, 2023 at 3:45 AM Peter Zijlstra <peterz@infradead.org> wrote:
> >
> > On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
> > > AMD processors support per-package and per-core energy monitoring
> > > through RAPL counters which can be accessed by users running in
> > > supervisor mode.
> > >
> > > Core RAPL counters gives power consumption information per core. For
> > > AMD processors the package level RAPL counter are already exposed to
> > > perf. Expose the core level RAPL counters also.
> > >
> > > sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
> > >
> > > Output:
> > > S0-D0-C0 2 8.73 Joules power/energy-cores/
> > > S0-D0-C1 2 8.73 Joules power/energy-cores/
> > > S0-D0-C2 2 8.73 Joules power/energy-cores/
> > > S0-D0-C3 2 8.73 Joules power/energy-cores/
> > > S0-D0-C4 2 8.73 Joules power/energy-cores/
> > >
> > > Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> > > ---
> > > arch/x86/events/rapl.c | 5 +++--
> > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
> > > index 52e6e7ed4f78..d301bbbc3b93 100644
> > > --- a/arch/x86/events/rapl.c
> > > +++ b/arch/x86/events/rapl.c
> > > @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
> > > * - want to use same event codes across both architectures
> > > */
> > > static struct perf_msr amd_rapl_msrs[] = {
> > > - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
> > > + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
> >
> > Stephane, this was an oversight?
> >
> I think it may depend on the CPU model. I remember it returning either
> 0 or bogus values on my systems. They may have improved that.
> The commit msg does not show which CPU model this is run on.
I've tested this on Zen 2, 3 and 4 server systems.
Thanks,
Wyes
>
> >
> > > [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
> > > [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
> > > [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
> > > @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
> > > };
> > >
> > > static struct rapl_model model_amd_hygon = {
> > > - .events = BIT(PERF_RAPL_PKG),
> > > + .events = BIT(PERF_RAPL_PP0) |
> > > + BIT(PERF_RAPL_PKG),
> > > .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
> > > .rapl_msrs = amd_rapl_msrs,
> > > };
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-21 4:53 ` Wyes Karny
@ 2023-02-21 8:50 ` Stephane Eranian
2023-02-28 14:20 ` Muhammad Usama Anjum
2023-03-01 18:27 ` Wyes Karny
0 siblings, 2 replies; 10+ messages in thread
From: Stephane Eranian @ 2023-02-21 8:50 UTC (permalink / raw)
To: Wyes Karny
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
Thomas Gleixner, Borislav Petkov, Dave Hansen, hpa, x86,
linux-perf-users, linux-kernel, gautham.shenoy, ananth.narayan
Hi,
Testing Wyes' patch for energy-cores on Zen3 server loaded with triad
bench on socket0:
$ perf stat --per-core -a -C0-63 -I 1000 -e
power/energy-cores/,power/energy-pkg/
# time core cpus counts unit events
1.001019203 S0-D0-C0 1 1.28 Joules
power/energy-cores/
1.001019203 S0-D0-C0 1 231.38 Joules
power/energy-pkg/
1.001019203 S0-D0-C1 1 4,294,967,130.96 Joules
power/energy-cores/
1.001019203 S0-D0-C1 1 231.38 Joules
power/energy-pkg/
1.001019203 S0-D0-C2 1 4,294,967,126.23 Joules
power/energy-cores/
1.001019203 S0-D0-C2 1 231.38 Joules
power/energy-pkg/
1.001019203 S0-D0-C3 1 4,294,967,122.50 Joules
power/energy-cores/
1.001019203 S0-D0-C3 1 231.38 Joules
power/energy-pkg/
1.001019203 S0-D0-C4 1 4,294,967,129.92 Joules
power/energy-cores/
1.001019203 S0-D0-C4 1 231.38 Joules
power/energy-pkg/
1.001019203 S0-D0-C5 1 4,294,967,121.49 Joules
power/energy-cores/
1.001019203 S0-D0-C5 1 231.39 Joules
power/energy-pkg/
I think the result of energy-cores is not reliable and I think that is
why I did not
include it in the patch.
Could also be a problem with the kernel code, but I don't know why it would only
impact energy-cores given energy-pkg looks reasonable here.
On Mon, Feb 20, 2023 at 8:53 PM Wyes Karny <wyes.karny@amd.com> wrote:
>
> On 20 Feb 13:29, Stephane Eranian wrote:
> > On Mon, Feb 20, 2023 at 3:45 AM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
> > > > AMD processors support per-package and per-core energy monitoring
> > > > through RAPL counters which can be accessed by users running in
> > > > supervisor mode.
> > > >
> > > > Core RAPL counters gives power consumption information per core. For
> > > > AMD processors the package level RAPL counter are already exposed to
> > > > perf. Expose the core level RAPL counters also.
> > > >
> > > > sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
> > > >
> > > > Output:
> > > > S0-D0-C0 2 8.73 Joules power/energy-cores/
> > > > S0-D0-C1 2 8.73 Joules power/energy-cores/
> > > > S0-D0-C2 2 8.73 Joules power/energy-cores/
> > > > S0-D0-C3 2 8.73 Joules power/energy-cores/
> > > > S0-D0-C4 2 8.73 Joules power/energy-cores/
> > > >
> > > > Signed-off-by: Wyes Karny <wyes.karny@amd.com>
> > > > ---
> > > > arch/x86/events/rapl.c | 5 +++--
> > > > 1 file changed, 3 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
> > > > index 52e6e7ed4f78..d301bbbc3b93 100644
> > > > --- a/arch/x86/events/rapl.c
> > > > +++ b/arch/x86/events/rapl.c
> > > > @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
> > > > * - want to use same event codes across both architectures
> > > > */
> > > > static struct perf_msr amd_rapl_msrs[] = {
> > > > - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
> > > > + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
> > >
> > > Stephane, this was an oversight?
> > >
> > I think it may depend on the CPU model. I remember it returning either
> > 0 or bogus values on my systems. They may have improved that.
> > The commit msg does not show which CPU model this is run on.
>
> I've tested this on Zen 2, 3 and 4 server systems.
>
> Thanks,
> Wyes
> >
> > >
> > > > [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
> > > > [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
> > > > [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
> > > > @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
> > > > };
> > > >
> > > > static struct rapl_model model_amd_hygon = {
> > > > - .events = BIT(PERF_RAPL_PKG),
> > > > + .events = BIT(PERF_RAPL_PP0) |
> > > > + BIT(PERF_RAPL_PKG),
> > > > .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
> > > > .rapl_msrs = amd_rapl_msrs,
> > > > };
> > > > --
> > > > 2.34.1
> > > >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-21 8:50 ` Stephane Eranian
@ 2023-02-28 14:20 ` Muhammad Usama Anjum
2023-02-28 15:55 ` Wyes Karny
2023-03-01 18:27 ` Wyes Karny
1 sibling, 1 reply; 10+ messages in thread
From: Muhammad Usama Anjum @ 2023-02-28 14:20 UTC (permalink / raw)
To: Stephane Eranian, Wyes Karny
Cc: Muhammad Usama Anjum, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
Jiri Olsa, Namhyung Kim, Thomas Gleixner, Borislav Petkov,
Dave Hansen, hpa, x86, linux-perf-users, linux-kernel,
gautham.shenoy, ananth.narayan, kernel
On 2/21/23 1:50 PM, Stephane Eranian wrote:
> Hi,
>
> Testing Wyes' patch for energy-cores on Zen3 server loaded with triad
> bench on socket0:
>
> $ perf stat --per-core -a -C0-63 -I 1000 -e
> power/energy-cores/,power/energy-pkg/
> # time core cpus counts unit events
> 1.001019203 S0-D0-C0 1 1.28 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C0 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C1 1 4,294,967,130.96 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C1 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C2 1 4,294,967,126.23 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C2 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C3 1 4,294,967,122.50 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C3 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C4 1 4,294,967,129.92 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C4 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C5 1 4,294,967,121.49 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C5 1 231.39 Joules
> power/energy-pkg/
>
> I think the result of energy-cores is not reliable and I think that is
> why I did not
> include it in the patch.
>
> Could also be a problem with the kernel code, but I don't know why it would only
> impact energy-cores given energy-pkg looks reasonable here.
>
I'm getting consistent per core energy measurements on AMD Custom APU 405.
Probably consistency depends on the chip.
sudo perf stat -a --per-core -C 0-7 -e power/energy-cores/
Performance counter stats for 'system wide':
S0-D0-C0 2 0.51 Joules power/energy-cores/
S0-D0-C1 2 0.51 Joules power/energy-cores/
S0-D0-C2 2 0.51 Joules power/energy-cores/
S0-D0-C3 2 0.51 Joules power/energy-cores/
What is the fate of this patch now?
[..]
>>>> Stephane, this was an oversight?
>>>>
>>> I think it may depend on the CPU model. I remember it returning either
>>> 0 or bogus values on my systems. They may have improved that.
>>> The commit msg does not show which CPU model this is run on.
>>
>> I've tested this on Zen 2, 3 and 4 server systems.
>>
>> Thanks,
>> Wyes
--
BR,
Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-28 14:20 ` Muhammad Usama Anjum
@ 2023-02-28 15:55 ` Wyes Karny
0 siblings, 0 replies; 10+ messages in thread
From: Wyes Karny @ 2023-02-28 15:55 UTC (permalink / raw)
To: Muhammad Usama Anjum
Cc: Stephane Eranian, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
Jiri Olsa, Namhyung Kim, Thomas Gleixner, Borislav Petkov,
Dave Hansen, hpa, x86, linux-perf-users, linux-kernel,
gautham.shenoy, ananth.narayan, kernel
Hi Muhammad,
On 28 Feb 19:20, Muhammad Usama Anjum wrote:
> On 2/21/23 1:50 PM, Stephane Eranian wrote:
> > Hi,
> >
> > Testing Wyes' patch for energy-cores on Zen3 server loaded with triad
> > bench on socket0:
> >
> > $ perf stat --per-core -a -C0-63 -I 1000 -e
> > power/energy-cores/,power/energy-pkg/
> > # time core cpus counts unit events
> > 1.001019203 S0-D0-C0 1 1.28 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C0 1 231.38 Joules
> > power/energy-pkg/
> > 1.001019203 S0-D0-C1 1 4,294,967,130.96 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C1 1 231.38 Joules
> > power/energy-pkg/
> > 1.001019203 S0-D0-C2 1 4,294,967,126.23 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C2 1 231.38 Joules
> > power/energy-pkg/
> > 1.001019203 S0-D0-C3 1 4,294,967,122.50 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C3 1 231.38 Joules
> > power/energy-pkg/
> > 1.001019203 S0-D0-C4 1 4,294,967,129.92 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C4 1 231.38 Joules
> > power/energy-pkg/
> > 1.001019203 S0-D0-C5 1 4,294,967,121.49 Joules
> > power/energy-cores/
> > 1.001019203 S0-D0-C5 1 231.39 Joules
> > power/energy-pkg/
> >
> > I think the result of energy-cores is not reliable and I think that is
> > why I did not
> > include it in the patch.
> >
> > Could also be a problem with the kernel code, but I don't know why it would only
> > impact energy-cores given energy-pkg looks reasonable here.
> >
>
> I'm getting consistent per core energy measurements on AMD Custom APU 405.
> Probably consistency depends on the chip.
>
> sudo perf stat -a --per-core -C 0-7 -e power/energy-cores/
>
> Performance counter stats for 'system wide':
>
> S0-D0-C0 2 0.51 Joules power/energy-cores/
> S0-D0-C1 2 0.51 Joules power/energy-cores/
> S0-D0-C2 2 0.51 Joules power/energy-cores/
> S0-D0-C3 2 0.51 Joules power/energy-cores/
>
> What is the fate of this patch now?
I'm working on the issue reported by Stephane. Will send v2 very soon.
Thanks,
Wyes
>
> [..]
>
> >>>> Stephane, this was an oversight?
> >>>>
> >>> I think it may depend on the CPU model. I remember it returning either
> >>> 0 or bogus values on my systems. They may have improved that.
> >>> The commit msg does not show which CPU model this is run on.
> >>
> >> I've tested this on Zen 2, 3 and 4 server systems.
> >>
> >> Thanks,
> >> Wyes
>
> --
> BR,
> Muhammad Usama Anjum
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-21 8:50 ` Stephane Eranian
2023-02-28 14:20 ` Muhammad Usama Anjum
@ 2023-03-01 18:27 ` Wyes Karny
1 sibling, 0 replies; 10+ messages in thread
From: Wyes Karny @ 2023-03-01 18:27 UTC (permalink / raw)
To: Stephane Eranian
Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
Thomas Gleixner, Borislav Petkov, Dave Hansen, hpa, x86,
linux-perf-users, linux-kernel, gautham.shenoy, ananth.narayan
Hi Stephane,
On 2/21/2023 2:20 PM, Stephane Eranian wrote:
> Hi,
>
> Testing Wyes' patch for energy-cores on Zen3 server loaded with triad
> bench on socket0:
>
> $ perf stat --per-core -a -C0-63 -I 1000 -e
> power/energy-cores/,power/energy-pkg/
> # time core cpus counts unit events
> 1.001019203 S0-D0-C0 1 1.28 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C0 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C1 1 4,294,967,130.96 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C1 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C2 1 4,294,967,126.23 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C2 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C3 1 4,294,967,122.50 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C3 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C4 1 4,294,967,129.92 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C4 1 231.38 Joules
> power/energy-pkg/
> 1.001019203 S0-D0-C5 1 4,294,967,121.49 Joules
> power/energy-cores/
> 1.001019203 S0-D0-C5 1 231.39 Joules
> power/energy-pkg/
>
> I think the result of energy-cores is not reliable and I think that is
> why I did not
> include it in the patch.
>
> Could also be a problem with the kernel code, but I don't know why it would only
> impact energy-cores given energy-pkg looks reasonable here.
This is a kernel issue. I've addressed this in v2.
Thanks,
Wyes
>
>
> On Mon, Feb 20, 2023 at 8:53 PM Wyes Karny <wyes.karny@amd.com> wrote:
>>
>> On 20 Feb 13:29, Stephane Eranian wrote:
>>> On Mon, Feb 20, 2023 at 3:45 AM Peter Zijlstra <peterz@infradead.org> wrote:
>>>>
>>>> On Fri, Feb 17, 2023 at 04:13:54PM +0000, Wyes Karny wrote:
>>>>> AMD processors support per-package and per-core energy monitoring
>>>>> through RAPL counters which can be accessed by users running in
>>>>> supervisor mode.
>>>>>
>>>>> Core RAPL counters gives power consumption information per core. For
>>>>> AMD processors the package level RAPL counter are already exposed to
>>>>> perf. Expose the core level RAPL counters also.
>>>>>
>>>>> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>>>>>
>>>>> Output:
>>>>> S0-D0-C0 2 8.73 Joules power/energy-cores/
>>>>> S0-D0-C1 2 8.73 Joules power/energy-cores/
>>>>> S0-D0-C2 2 8.73 Joules power/energy-cores/
>>>>> S0-D0-C3 2 8.73 Joules power/energy-cores/
>>>>> S0-D0-C4 2 8.73 Joules power/energy-cores/
>>>>>
>>>>> Signed-off-by: Wyes Karny <wyes.karny@amd.com>
>>>>> ---
>>>>> arch/x86/events/rapl.c | 5 +++--
>>>>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>>>>
>>>>> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
>>>>> index 52e6e7ed4f78..d301bbbc3b93 100644
>>>>> --- a/arch/x86/events/rapl.c
>>>>> +++ b/arch/x86/events/rapl.c
>>>>> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
>>>>> * - want to use same event codes across both architectures
>>>>> */
>>>>> static struct perf_msr amd_rapl_msrs[] = {
>>>>> - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
>>>>> + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
>>>>
>>>> Stephane, this was an oversight?
>>>>
>>> I think it may depend on the CPU model. I remember it returning either
>>> 0 or bogus values on my systems. They may have improved that.
>>> The commit msg does not show which CPU model this is run on.
>>
>> I've tested this on Zen 2, 3 and 4 server systems.
>>
>> Thanks,
>> Wyes
>>>
>>>>
>>>>> [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
>>>>> [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
>>>>> [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
>>>>> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
>>>>> };
>>>>>
>>>>> static struct rapl_model model_amd_hygon = {
>>>>> - .events = BIT(PERF_RAPL_PKG),
>>>>> + .events = BIT(PERF_RAPL_PP0) |
>>>>> + BIT(PERF_RAPL_PKG),
>>>>> .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
>>>>> .rapl_msrs = amd_rapl_msrs,
>>>>> };
>>>>> --
>>>>> 2.34.1
>>>>>
--
Thanks & Regards,
Wyes
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
2023-02-19 11:52 ` Oleksandr Natalenko
@ 2023-03-01 18:32 ` Wyes Karny
0 siblings, 0 replies; 10+ messages in thread
From: Wyes Karny @ 2023-03-01 18:32 UTC (permalink / raw)
To: Oleksandr Natalenko, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
Jiri Olsa, Namhyung Kim, Thomas Gleixner, Borislav Petkov,
Dave Hansen, hpa
Cc: x86, linux-perf-users, linux-kernel, gautham.shenoy, ananth.narayan
Hi Oleksandr,
On 2/19/2023 5:22 PM, Oleksandr Natalenko wrote:
> Hello.
>
> On pátek 17. února 2023 17:13:54 CET Wyes Karny wrote:
>> AMD processors support per-package and per-core energy monitoring
>> through RAPL counters which can be accessed by users running in
>> supervisor mode.
>>
>> Core RAPL counters gives power consumption information per core. For
>> AMD processors the package level RAPL counter are already exposed to
>> perf. Expose the core level RAPL counters also.
>>
>> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>>
>> Output:
>> S0-D0-C0 2 8.73 Joules power/energy-cores/
>> S0-D0-C1 2 8.73 Joules power/energy-cores/
>> S0-D0-C2 2 8.73 Joules power/energy-cores/
>> S0-D0-C3 2 8.73 Joules power/energy-cores/
>> S0-D0-C4 2 8.73 Joules power/energy-cores/
>>
>> Signed-off-by: Wyes Karny <wyes.karny@amd.com>
>> ---
>> arch/x86/events/rapl.c | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
>> index 52e6e7ed4f78..d301bbbc3b93 100644
>> --- a/arch/x86/events/rapl.c
>> +++ b/arch/x86/events/rapl.c
>> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
>> * - want to use same event codes across both architectures
>> */
>> static struct perf_msr amd_rapl_msrs[] = {
>> - [PERF_RAPL_PP0] = { 0, &rapl_events_cores_group, 0, false, 0 },
>> + [PERF_RAPL_PP0] = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
>> [PERF_RAPL_PKG] = { MSR_AMD_PKG_ENERGY_STATUS, &rapl_events_pkg_group, test_msr, false, RAPL_MSR_MASK },
>> [PERF_RAPL_RAM] = { 0, &rapl_events_ram_group, 0, false, 0 },
>> [PERF_RAPL_PP1] = { 0, &rapl_events_gpu_group, 0, false, 0 },
>> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
>> };
>>
>> static struct rapl_model model_amd_hygon = {
>> - .events = BIT(PERF_RAPL_PKG),
>> + .events = BIT(PERF_RAPL_PP0) |
>> + BIT(PERF_RAPL_PKG),
>> .msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
>> .rapl_msrs = amd_rapl_msrs,
>> };
>>
>
> With this patch:
>
> ```
> $ lscpu | grep 'Model name'
> Model name: AMD Ryzen 9 5950X 16-Core Processor
>
> $ sudo perf stat -a --per-core -C 0-15 -e power/energy-cores/ -- dd if=/dev/zero of=/dev/null bs=1M count=100000
> 100000+0 records in
> 100000+0 records out
> 104857600000 bytes (105 GB, 98 GiB) copied, 1,59252 s, 65,8 GB/s
>
> Performance counter stats for 'system wide':
>
> S0-D0-C0 1 1,56 Joules power/energy-cores/
> S0-D0-C1 1 1,56 Joules power/energy-cores/
> S0-D0-C2 1 1,56 Joules power/energy-cores/
> S0-D0-C3 1 1,56 Joules power/energy-cores/
> S0-D0-C4 1 1,56 Joules power/energy-cores/
> S0-D0-C5 1 1,56 Joules power/energy-cores/
> S0-D0-C6 1 1,56 Joules power/energy-cores/
> S0-D0-C7 1 1,56 Joules power/energy-cores/
> S0-D0-C8 1 1,56 Joules power/energy-cores/
> S0-D0-C9 1 1,56 Joules power/energy-cores/
> S0-D0-C10 1 1,56 Joules power/energy-cores/
> S0-D0-C11 1 1,56 Joules power/energy-cores/
> S0-D0-C12 1 1,56 Joules power/energy-cores/
> S0-D0-C13 1 1,56 Joules power/energy-cores/
> S0-D0-C14 1 1,56 Joules power/energy-cores/
> S0-D0-C15 1 1,56 Joules power/energy-cores/
>
> 1,593982452 seconds time elapsed
> ```
>
> Hence,
>
> Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>
I haven't put your tested-by tag in v2 as I've added a new patch there.
Please let me know if v2 works well for you and if I can add tested-by on v2.
>
> Thank you.
>
--
Thanks & Regards,
Wyes
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2023-03-01 18:33 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-17 16:13 [PATCH] perf/x86/rapl: Enable Core RAPL for AMD Wyes Karny
2023-02-19 11:52 ` Oleksandr Natalenko
2023-03-01 18:32 ` Wyes Karny
2023-02-20 11:45 ` Peter Zijlstra
2023-02-20 21:29 ` Stephane Eranian
2023-02-21 4:53 ` Wyes Karny
2023-02-21 8:50 ` Stephane Eranian
2023-02-28 14:20 ` Muhammad Usama Anjum
2023-02-28 15:55 ` Wyes Karny
2023-03-01 18:27 ` Wyes Karny
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).