Enable "PMU" counters for Khadas VIM3 in the Google AOSP kernel

Enable "PMU" counters for Khadas VIM3 in the Google AOSP kernel

[Ehsan Aghapour]

Hello All,

I am working on Google AOSP kernel and require to enable PMU. PMU is
working well in Khadas kernel for both A53 and A73 cores but it is not
enabled in Google AOSP kernel 5.4. I try adding arm_pmu definition in
device tree at /arch/arm64/boot/dts/amlogic/meson-g12b.dtsi as follow:
arm_pmu {
                compatible = "arm,armv8-pmuv3";
                clusterb-enabled;
                interrupts = <GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>,
                        <GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>;
                reg = <0x0 0xff634680 0x0 0x4>,
                        <0x0 0xff6347c0 0x0 0x04>;
                cpumasks = <0x3 0x3C>;
                /* default 10ms */
                relax-timer-ns = <10000000>;
                /* default 10000us */
                max-wait-cnt = <10000>;
        };

However in this case I only see A53 performance counters in DS5
Streamline and performance counters of A73 cores are zero yet.

Would you please help me solve the problem? (If device tree need
change or kernel config to enable pmu counters for both CPUs).


This is related kernel log:
"[    1.965309] hw perfevents: no interrupt-affinity property for
/arm_pmu, guessing.
[    1.970821] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7
counters available
...
[    7.131341] ueventd: LoadWithAliases was unable to load
of:Narm_pmuT(null)Carm,armv8-pmuv3"


Best regards,
Ehsan

Re: Enable "PMU" counters for Khadas VIM3 in the Google AOSP kernel

[Robin Murphy]

Hi Ehsan,

On 2021-10-08 00:11, Ehsan Aghapour wrote:
> Hello All,
> 
> I am working on Google AOSP kernel and require to enable PMU. PMU is
> working well in Khadas kernel for both A53 and A73 cores but it is not
> enabled in Google AOSP kernel 5.4. I try adding arm_pmu definition in
> device tree at /arch/arm64/boot/dts/amlogic/meson-g12b.dtsi as follow:
> arm_pmu {
>                  compatible = "arm,armv8-pmuv3";
>                  clusterb-enabled;
>                  interrupts = <GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>,
>                          <GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>;
>                  reg = <0x0 0xff634680 0x0 0x4>,
>                          <0x0 0xff6347c0 0x0 0x04>;
>                  cpumasks = <0x3 0x3C>;
>                  /* default 10ms */
>                  relax-timer-ns = <10000000>;
>                  /* default 10000us */
>                  max-wait-cnt = <10000>;
>          };

Note that pretty much none of that is valid per the PMU binding[1].

> However in this case I only see A53 performance counters in DS5
> Streamline and performance counters of A73 cores are zero yet.
> 
> Would you please help me solve the problem? (If device tree need
> change or kernel config to enable pmu counters for both CPUs).

You should have two nodes describing the A53 and A73 PMUs distinctly. If 
the interrupts are SPIs you need the appropriate property describing 
which interrupt belongs to which CPU - see [2] for an example. Note that 
if you really do only have 2 interrupts for 6 CPUs (implying maybe 
they're combined together per cluster?) then unfortunately it's never 
going to work - the upstream PMU driver does not support shared 
interrupts (it used to attempt to, but it's impractically complicated 
and gives poor-quality results at best).

Robin.

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/arm/pmu.yaml
[2] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/arm/juno.dts#n209

> This is related kernel log:
> "[    1.965309] hw perfevents: no interrupt-affinity property for
> /arm_pmu, guessing.
> [    1.970821] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7
> counters available
> ...
> [    7.131341] ueventd: LoadWithAliases was unable to load
> of:Narm_pmuT(null)Carm,armv8-pmuv3"
> 
> 
> Best regards,
> Ehsan
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
> 

Re: Enable "PMU" counters for Khadas VIM3 in the Google AOSP kernel

[Marc Zyngier]

On Thu, 07 Oct 2021 23:13:30 +0100,
Ehsan Aghapour <aghapour.ehsan17@gmail.com> wrote:
> 
> I am working on Google AOSP kernel and require to enable PMU. PMU is
> working well in Khadas kernel for both A53 and A73 cores but it is not
> enabled in Google AOSP kernel 5.4. I try adding arm_pmu definition in
> device tree at /arch/arm64/boot/dts/amlogic/meson-g12b.dtsi as follow:
> arm_pmu {
>                 compatible = "arm,armv8-pmuv3";
>                 clusterb-enabled;
>                 interrupts = <GIC_SPI 137 IRQ_TYPE_LEVEL_HIGH>,
>                         <GIC_SPI 171 IRQ_TYPE_LEVEL_HIGH>;
>                 reg = <0x0 0xff634680 0x0 0x4>,
>                         <0x0 0xff6347c0 0x0 0x04>;
>                 cpumasks = <0x3 0x3C>;
>                 /* default 10ms */
>                 relax-timer-ns = <10000000>;
>                 /* default 10000us */
>                 max-wait-cnt = <10000>;

Most of these properties don't exist in the binding, and are thus
ignored by the driver.

>         };
> 
> However in this case I only see A53 performance counters in DS5 Streamline
> and performance counters of A73 cores are zero yet.
> 
> Would you please help me solve the problem? (If device tree need change or
> kernel config to enable pmu counters for both CPUs).

The problem is that all the Amlogic SoCs have a totally broken PMU
integration. They OR'd all the PMU interrupts from the CPUs inside a
cluster, which is why you end-up with only two interrupts in a system
that should have 6.

There is no good workaround for this. The downstream kernel may have
all sort of hacks to cope with the brokenness, but upstream will
simply not work. I have my own set of hacks to deal with the PMU on
the A55-based version of that SoC[1], but there is no way this is
going upstream.

Thanks,

	M.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/commit/?h=hack/vim3l-crap&id=6bde69695241344ddf7f74880314a0c6cbdaf963

-- 
Without deviation from the norm, progress is not possible.