Re: [PATCH 1/3 v7] hwmon: Add amd_energy driver to report energy counters

[Naveen Krishna Ch <naveenkrishna.ch@gmail.com>]

Hi Alexander,

Thanks for using the driver.

On Wed, 27 May 2020 at 04:07, Alexander Monakov <amonakov@ispras.ru> wrote:
>
> On Fri, 22 May 2020, Naveen Krishna Ch wrote:
>
> > > This version looks ok, and it looks like there are no more
> > > unexpected counter wraps either. Series applied to hwmon-next.
> >
> > Thank you for your guidance and support.
>
> Thank you, looking forward to using this driver. Meanwhile, I have
> a couple of questions about AMD RAPL counters.
>
> The documentation says the PKG_ENERGY_STAT MSR is instantiated per CCD
> and L3 complex, implying that energy is accumulated for 3 or 4 cores
> in an L3 complex.
>
> However your patch reads that MSR per socket, and from testing on my
> Ryzen 4500U CPU I can confirm that the MSR gives the same value when
> accessed from each core. Therefore I think your code is correct and
> the documentation is wrong, can you confirm?
I confirmed this with the hardware folks, the definition of package has
changed with the chiplet design. May be the documentation needs an update.

>
> Next, I'm seeing paradoxical results when attempting to test accuracy
> of the counters. When running an AVX workload on 6 cores, I see
> CORE_ENERGY_STAT MSR reporting values just above 9 Watts per core, with
> 55 Watts in total, and yet PKG_ENERGY_STAT reporting only 30 Watts.
> This is clearly a contradiction since package energy must be at least
> the sum of core energy readings. Furthermore, this is happening on a
> 18 W CPU in a laptop, which definitely cannot sustain 55 Watts, and
> even 30 Watts seems too high.
>
> Can you clarify how the counters work and under what conditions they
> give accurate readings?

These registers are 32bit counters, they might wrap-around quite faster at
high work loads. So, we used a kernel thread to accumulate the values of
each core and socket to 64bit values.

Depending on when the module is inserted in the system, the initial values
of the counters could be different and we do not have a way to know, how
many time the registers are wrapped around in the past.

Difference of Energy over Time is the best way to use this information.
For example:
at t1 = 0, Read the register before starting the job, say E1.
at t2= 300secs, Read the register after the completion of the job, say E2.
power consumed = (E2-E1)/(t2 - t1) in Watts.

In our evaluation, the sum of the energy consumption of cores of a socket was
always less (actually far lesser) than the socket energy consumption.

One other thing could be the enumeration of the cores of a package in the
Desktop/Laptop systems might need different handling in the read. I will check
this.

>
> Thanks.
> Alexander

-- 
Shine bright,
(: Nav :)