Re: [PATCH v13 3/5] arm64: perf: Add userspace counter access disable switch

Re: [PATCH v13 3/5] arm64: perf: Add userspace counter access disable switch

[Geert Uytterhoeven]

Hi Rob,

On Wed, Dec 8, 2021 at 9:19 PM Rob Herring <robh@kernel.org> wrote:
> Like x86, some users may want to disable userspace PMU counter
> altogether. Add a sysctl 'perf_user_access' file to control userspace
> counter access. The default is '0' which is disabled. Writing '1'
> enables access.
>
> Note that x86 supports globally enabling user access by writing '2' to
> /sys/bus/event_source/devices/cpu/rdpmc. As there's not existing
> userspace support to worry about, this shouldn't be necessary for Arm.
> It could be added later if the need arises.

Thanks for your patch, which is now commit e2012600810c9ded ("arm64:
perf: Add userspace counter access disable switch") in arm64/for-next/core.

This is causing two issues on Renesas Salvator-XS with R-Car H3.
One during kernel boot:

     hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7
counters available
    +sysctl duplicate entry: /kernel//perf_user_access
    +CPU: 0 PID: 1 Comm: swapper/0 Not tainted
5.16.0-rc3-arm64-renesas-00003-ge2012600810c #1420
    +Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT)
    +Call trace:
    + dump_backtrace+0x0/0x190
    + show_stack+0x14/0x20
    + dump_stack_lvl+0x88/0xb0
    + dump_stack+0x14/0x2c
    + __register_sysctl_table+0x384/0x818
    + register_sysctl+0x20/0x28
    + armv8_pmu_init.constprop.0+0x118/0x150
    + armv8_a57_pmu_init+0x1c/0x28
    + arm_pmu_device_probe+0x1b4/0x558
    + armv8_pmu_device_probe+0x18/0x20
    + platform_probe+0x64/0xd0
    + really_probe+0xb4/0x2f8
    + __driver_probe_device+0x74/0xd8
    + driver_probe_device+0x3c/0xe0
    + __driver_attach+0x80/0x110
    + bus_for_each_dev+0x6c/0xc0
    + driver_attach+0x20/0x28
    + bus_add_driver+0x138/0x1e0
    + driver_register+0x60/0x110
    + __platform_driver_register+0x24/0x30
    + armv8_pmu_driver_init+0x18/0x20
    + do_one_initcall+0x15c/0x31c
    + kernel_init_freeable+0x2f0/0x354
    + kernel_init+0x20/0x120
    + ret_from_fork+0x10/0x20
     hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7
counters available

Presumably the same entry is added twice, once for the A53 PMU,
and a second time for the A57 PMU?

A second during systemd startup:

    systemd-journald[326]: Failed to open runtime journal: No such
file or directory
    systemd-journald[345]: File
/run/log/journal/09223238c0464b38ad4fc1d505d98e17/system.journal
corrupted or uncleanly shut down, renaming and replacing.

followed by lots of

    systemd[<n>]: <foo>.service: Failed to connect stdout to the
journal socket, ignoring: Connection refused

failures.

> --- a/arch/arm64/kernel/perf_event.c
> +++ b/arch/arm64/kernel/perf_event.c
> @@ -286,6 +286,8 @@ static const struct attribute_group armv8_pmuv3_events_attr_group = {
>  PMU_FORMAT_ATTR(event, "config:0-15");
>  PMU_FORMAT_ATTR(long, "config1:0");
>
> +static int sysctl_perf_user_access __read_mostly;
> +
>  static inline bool armv8pmu_event_is_64bit(struct perf_event *event)
>  {
>         return event->attr.config1 & 0x1;
> @@ -1104,6 +1106,19 @@ static int armv8pmu_probe_pmu(struct arm_pmu *cpu_pmu)
>         return probe.present ? 0 : -ENODEV;
>  }
>
> +static struct ctl_table armv8_pmu_sysctl_table[] = {
> +       {
> +               .procname       = "perf_user_access",
> +               .data           = &sysctl_perf_user_access,
> +               .maxlen         = sizeof(unsigned int),
> +               .mode           = 0644,
> +               .proc_handler   = proc_dointvec_minmax,
> +               .extra1         = SYSCTL_ZERO,
> +               .extra2         = SYSCTL_ONE,
> +       },
> +       { }
> +};
> +
>  static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
>                           int (*map_event)(struct perf_event *event),
>                           const struct attribute_group *events,
> @@ -1136,6 +1151,8 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
>         cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_CAPS] = caps ?
>                         caps : &armv8_pmuv3_caps_attr_group;
>
> +       register_sysctl("kernel", armv8_pmu_sysctl_table);
> +
>         return 0;
>  }

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH v13 3/5] arm64: perf: Add userspace counter access disable switch

[Will Deacon]

Hi Geert,

On Tue, Dec 28, 2021 at 12:07:02PM +0100, Geert Uytterhoeven wrote:
> On Wed, Dec 8, 2021 at 9:19 PM Rob Herring <robh@kernel.org> wrote:
> > Like x86, some users may want to disable userspace PMU counter
> > altogether. Add a sysctl 'perf_user_access' file to control userspace
> > counter access. The default is '0' which is disabled. Writing '1'
> > enables access.
> >
> > Note that x86 supports globally enabling user access by writing '2' to
> > /sys/bus/event_source/devices/cpu/rdpmc. As there's not existing
> > userspace support to worry about, this shouldn't be necessary for Arm.
> > It could be added later if the need arises.
> 
> Thanks for your patch, which is now commit e2012600810c9ded ("arm64:
> perf: Add userspace counter access disable switch") in arm64/for-next/core.
> 
> This is causing two issues on Renesas Salvator-XS with R-Car H3.
> One during kernel boot:
> 
>      hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7
> counters available
>     +sysctl duplicate entry: /kernel//perf_user_access
>     +CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 5.16.0-rc3-arm64-renesas-00003-ge2012600810c #1420
>     +Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT)
>     +Call trace:
>     + dump_backtrace+0x0/0x190
>     + show_stack+0x14/0x20
>     + dump_stack_lvl+0x88/0xb0
>     + dump_stack+0x14/0x2c
>     + __register_sysctl_table+0x384/0x818
>     + register_sysctl+0x20/0x28
>     + armv8_pmu_init.constprop.0+0x118/0x150
>     + armv8_a57_pmu_init+0x1c/0x28
>     + arm_pmu_device_probe+0x1b4/0x558
>     + armv8_pmu_device_probe+0x18/0x20
>     + platform_probe+0x64/0xd0
>     + really_probe+0xb4/0x2f8
>     + __driver_probe_device+0x74/0xd8
>     + driver_probe_device+0x3c/0xe0
>     + __driver_attach+0x80/0x110
>     + bus_for_each_dev+0x6c/0xc0
>     + driver_attach+0x20/0x28
>     + bus_add_driver+0x138/0x1e0
>     + driver_register+0x60/0x110
>     + __platform_driver_register+0x24/0x30
>     + armv8_pmu_driver_init+0x18/0x20
>     + do_one_initcall+0x15c/0x31c
>     + kernel_init_freeable+0x2f0/0x354
>     + kernel_init+0x20/0x120
>     + ret_from_fork+0x10/0x20
>      hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7
> counters available
> 
> Presumably the same entry is added twice, once for the A53 PMU,
> and a second time for the A57 PMU?

Looks like it, and perhaps that's also what is confusing systemd?
Rob -- how come you didn't see this during your testing?

Anywho, please can you try the untested diff below?

Thanks,

Will

--->8

diff --git a/arch/arm64/kernel/perf_event.c b/arch/arm64/kernel/perf_event.c
index 81cc9f0e718a..639f632aaa66 100644
--- a/arch/arm64/kernel/perf_event.c
+++ b/arch/arm64/kernel/perf_event.c
@@ -1214,6 +1214,14 @@ static struct ctl_table armv8_pmu_sysctl_table[] = {
        { }
 };
 
+static void armv8_pmu_register_sysctl_table(void)
+{
+       static u32 tbl_registered = 0;
+
+       if (!cmpxchg_relaxed(&tbl_registered, 0, 1))
+               register_sysctl("kernel", armv8_pmu_sysctl_table);
+}
+
 static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
                          int (*map_event)(struct perf_event *event),
                          const struct attribute_group *events,
@@ -1248,8 +1256,7 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
        cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_CAPS] = caps ?
                        caps : &armv8_pmuv3_caps_attr_group;
 
-       register_sysctl("kernel", armv8_pmu_sysctl_table);
-
+       armv8_pmu_register_sysctl_table();
        return 0;
 }
 

Re: [PATCH v13 3/5] arm64: perf: Add userspace counter access disable switch

[Will Deacon]

On Tue, Jan 04, 2022 at 01:56:59PM +0000, Will Deacon wrote:
> On Tue, Dec 28, 2021 at 12:07:02PM +0100, Geert Uytterhoeven wrote:
> > On Wed, Dec 8, 2021 at 9:19 PM Rob Herring <robh@kernel.org> wrote:
> > > Like x86, some users may want to disable userspace PMU counter
> > > altogether. Add a sysctl 'perf_user_access' file to control userspace
> > > counter access. The default is '0' which is disabled. Writing '1'
> > > enables access.
> > >
> > > Note that x86 supports globally enabling user access by writing '2' to
> > > /sys/bus/event_source/devices/cpu/rdpmc. As there's not existing
> > > userspace support to worry about, this shouldn't be necessary for Arm.
> > > It could be added later if the need arises.
> > 
> > Thanks for your patch, which is now commit e2012600810c9ded ("arm64:
> > perf: Add userspace counter access disable switch") in arm64/for-next/core.
> > 
> > This is causing two issues on Renesas Salvator-XS with R-Car H3.
> > One during kernel boot:
> > 
> >      hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7
> > counters available
> >     +sysctl duplicate entry: /kernel//perf_user_access
> >     +CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> > 5.16.0-rc3-arm64-renesas-00003-ge2012600810c #1420
> >     +Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT)
> >     +Call trace:
> >     + dump_backtrace+0x0/0x190
> >     + show_stack+0x14/0x20
> >     + dump_stack_lvl+0x88/0xb0
> >     + dump_stack+0x14/0x2c
> >     + __register_sysctl_table+0x384/0x818
> >     + register_sysctl+0x20/0x28
> >     + armv8_pmu_init.constprop.0+0x118/0x150
> >     + armv8_a57_pmu_init+0x1c/0x28
> >     + arm_pmu_device_probe+0x1b4/0x558
> >     + armv8_pmu_device_probe+0x18/0x20
> >     + platform_probe+0x64/0xd0
> >     + really_probe+0xb4/0x2f8
> >     + __driver_probe_device+0x74/0xd8
> >     + driver_probe_device+0x3c/0xe0
> >     + __driver_attach+0x80/0x110
> >     + bus_for_each_dev+0x6c/0xc0
> >     + driver_attach+0x20/0x28
> >     + bus_add_driver+0x138/0x1e0
> >     + driver_register+0x60/0x110
> >     + __platform_driver_register+0x24/0x30
> >     + armv8_pmu_driver_init+0x18/0x20
> >     + do_one_initcall+0x15c/0x31c
> >     + kernel_init_freeable+0x2f0/0x354
> >     + kernel_init+0x20/0x120
> >     + ret_from_fork+0x10/0x20
> >      hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7
> > counters available
> > 
> > Presumably the same entry is added twice, once for the A53 PMU,
> > and a second time for the A57 PMU?
> 
> Looks like it, and perhaps that's also what is confusing systemd?
> Rob -- how come you didn't see this during your testing?
> 
> Anywho, please can you try the untested diff below?

I just remembered I have a big/little SoC on my desk after borrowing a
NanoPi (RK3399) from Marc Z, so I took this diff for a spin there and
both the kernel and systemd seem happy.

Will

Re: [PATCH v13 3/5] arm64: perf: Add userspace counter access disable switch

[Geert Uytterhoeven]

Hi Will,

On Tue, Jan 4, 2022 at 2:57 PM Will Deacon <will@kernel.org> wrote:
> On Tue, Dec 28, 2021 at 12:07:02PM +0100, Geert Uytterhoeven wrote:
> > On Wed, Dec 8, 2021 at 9:19 PM Rob Herring <robh@kernel.org> wrote:
> > > Like x86, some users may want to disable userspace PMU counter
> > > altogether. Add a sysctl 'perf_user_access' file to control userspace
> > > counter access. The default is '0' which is disabled. Writing '1'
> > > enables access.
> > >
> > > Note that x86 supports globally enabling user access by writing '2' to
> > > /sys/bus/event_source/devices/cpu/rdpmc. As there's not existing
> > > userspace support to worry about, this shouldn't be necessary for Arm.
> > > It could be added later if the need arises.
> >
> > Thanks for your patch, which is now commit e2012600810c9ded ("arm64:
> > perf: Add userspace counter access disable switch") in arm64/for-next/core.
> >
> > This is causing two issues on Renesas Salvator-XS with R-Car H3.
> > One during kernel boot:
> >
> >      hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7
> > counters available
> >     +sysctl duplicate entry: /kernel//perf_user_access
> >     +CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> > 5.16.0-rc3-arm64-renesas-00003-ge2012600810c #1420
> >     +Hardware name: Renesas Salvator-X 2nd version board based on r8a77951 (DT)
> >     +Call trace:
> >     + dump_backtrace+0x0/0x190
> >     + show_stack+0x14/0x20
> >     + dump_stack_lvl+0x88/0xb0
> >     + dump_stack+0x14/0x2c
> >     + __register_sysctl_table+0x384/0x818
> >     + register_sysctl+0x20/0x28
> >     + armv8_pmu_init.constprop.0+0x118/0x150
> >     + armv8_a57_pmu_init+0x1c/0x28
> >     + arm_pmu_device_probe+0x1b4/0x558
> >     + armv8_pmu_device_probe+0x18/0x20
> >     + platform_probe+0x64/0xd0
> >     + really_probe+0xb4/0x2f8
> >     + __driver_probe_device+0x74/0xd8
> >     + driver_probe_device+0x3c/0xe0
> >     + __driver_attach+0x80/0x110
> >     + bus_for_each_dev+0x6c/0xc0
> >     + driver_attach+0x20/0x28
> >     + bus_add_driver+0x138/0x1e0
> >     + driver_register+0x60/0x110
> >     + __platform_driver_register+0x24/0x30
> >     + armv8_pmu_driver_init+0x18/0x20
> >     + do_one_initcall+0x15c/0x31c
> >     + kernel_init_freeable+0x2f0/0x354
> >     + kernel_init+0x20/0x120
> >     + ret_from_fork+0x10/0x20
> >      hw perfevents: enabled with armv8_cortex_a57 PMU driver, 7
> > counters available
> >
> > Presumably the same entry is added twice, once for the A53 PMU,
> > and a second time for the A57 PMU?
>
> Looks like it, and perhaps that's also what is confusing systemd?
> Rob -- how come you didn't see this during your testing?
>
> Anywho, please can you try the untested diff below?

Thank you, this fixes the issue.

Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>

> --- a/arch/arm64/kernel/perf_event.c
> +++ b/arch/arm64/kernel/perf_event.c
> @@ -1214,6 +1214,14 @@ static struct ctl_table armv8_pmu_sysctl_table[] = {
>         { }
>  };
>
> +static void armv8_pmu_register_sysctl_table(void)
> +{
> +       static u32 tbl_registered = 0;
> +
> +       if (!cmpxchg_relaxed(&tbl_registered, 0, 1))
> +               register_sysctl("kernel", armv8_pmu_sysctl_table);
> +}
> +
>  static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
>                           int (*map_event)(struct perf_event *event),
>                           const struct attribute_group *events,
> @@ -1248,8 +1256,7 @@ static int armv8_pmu_init(struct arm_pmu *cpu_pmu, char *name,
>         cpu_pmu->attr_groups[ARMPMU_ATTR_GROUP_CAPS] = caps ?
>                         caps : &armv8_pmuv3_caps_attr_group;
>
> -       register_sysctl("kernel", armv8_pmu_sysctl_table);
> -
> +       armv8_pmu_register_sysctl_table();
>         return 0;
>  }

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds