From: Mark Rutland <mark.rutland@arm.com>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Hector Martin <marcan@marcan.st>, Ian Rogers <irogers@google.com>,
Marc Zyngier <maz@kernel.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
James Clark <james.clark@arm.com>,
linux-perf-users@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>,
Asahi Linux <asahi@lists.linux.dev>
Subject: Re: [REGRESSION] Perf (userspace) broken on big.LITTLE systems since v6.5
Date: Wed, 22 Nov 2023 15:49:18 +0000 [thread overview]
Message-ID: <ZV4i_lrhbOVdEpwH@FVFF77S0Q05N> (raw)
In-Reply-To: <ZV38z3+p2S2ETtzG@kernel.org>
On Wed, Nov 22, 2023 at 10:06:23AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Nov 22, 2023 at 12:23:27PM +0900, Hector Martin escreveu:
> > On 2023/11/22 1:38, Ian Rogers wrote:
> > > On Tue, Nov 21, 2023 at 8:15 AM Mark Rutland <mark.rutland@arm.com> wrote:
> > >> On Tue, Nov 21, 2023 at 08:09:37AM -0800, Ian Rogers wrote:
> > >>> On Tue, Nov 21, 2023 at 8:03 AM Mark Rutland <mark.rutland@arm.com> wrote:
> > >>>> On Tue, Nov 21, 2023 at 07:46:57AM -0800, Ian Rogers wrote:
> > >>>>> On Tue, Nov 21, 2023 at 7:40 AM Mark Rutland <mark.rutland@arm.com> wrote:
> > >>>>>> On Tue, Nov 21, 2023 at 03:24:25PM +0000, Marc Zyngier wrote:
> > >>>>>>> On Tue, 21 Nov 2023 13:40:31 +0000,
> > >>>>>>> Marc Zyngier <maz@kernel.org> wrote:
> > >>>>>>>>
> > >>>>>>>> [Adding key people on Cc]
> > >>>>>>>>
> > >>>>>>>> On Tue, 21 Nov 2023 12:08:48 +0000,
> > >>>>>>>> Hector Martin <marcan@marcan.st> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Perf broke on all Apple ARM64 systems (tested almost everything), and
> > >>>>>>>>> according to maz also on Juno (so, probably all big.LITTLE) since v6.5.
> > >>>>>>>>
> > >>>>>>>> I can confirm that at least on 6.7-rc2, perf is pretty busted on any
> > >>>>>>>> asymmetric ARM platform. It isn't clear what criteria is used to pick
> > >>>>>>>> the PMU, but nothing works anymore.
> > >>>>>>>>
> > >>>>>>>> The saving grace in my case is that Debian still ships a 6.1 perftool
> > >>>>>>>> package, but that's obviously not going to last.
> > >>>>>>>>
> > >>>>>>>> I'm happy to test potential fixes.
> > >>>>>>>
> > >>>>>>> At Mark's request, I've dumped a couple of perf (as of -rc2) runs with
> > >>>>>>> -vvv. And it is quite entertaining (this is taskset to an 'icestorm'
> > >>>>>>> CPU):
> > >>>>>>
> > >>>>>> IIUC the tool is doing the wrong thing here and overriding explicit
> > >>>>>> ${pmu}/${event}/ events with PERF_TYPE_HARDWARE events rather than events using
> > >>>>>> that ${pmu}'s type and event namespace.
> > >>>>>>
> > >>>>>> Regardless of the *new* ABI that allows PERF_TYPE_HARDWARE events to be
> > >>>>>> targetted to a specific PMU, it's semantically wrong to rewrite events like
> > >>>>>> this since ${pmu}/${event}/ is not necessarily equivalent to a similarly-named
> > >>>>>> PERF_COUNT_HW_${EVENT}.
> > >>>>>
> > >>>>> If you name a PMU and an event then the event should only be opened on
> > >>>>> that PMU, 100% agree. There's a bunch of output, but when the legacy
> > >>>>> cycles event is opened it appears to be because it was explicitly
> > >>>>> requested.
> > >>>>
> > >>>> I think you've missed that the named PMU events are being erreously transformed
> > >>>> into PERF_TYPE_HARDWARE events. Look at the -vvv output, e.g.
> > >>>>
> > >>>> Opening: apple_firestorm_pmu/cycles/
> > >>>> ------------------------------------------------------------
> > >>>> perf_event_attr:
> > >>>> type 0 (PERF_TYPE_HARDWARE)
> > >>>> size 136
> > >>>> config 0 (PERF_COUNT_HW_CPU_CYCLES)
> > >>>> sample_type IDENTIFIER
> > >>>> read_format TOTAL_TIME_ENABLED|TOTAL_TIME_RUNNING
> > >>>> disabled 1
> > >>>> inherit 1
> > >>>> enable_on_exec 1
> > >>>> exclude_guest 1
> > >>>> ------------------------------------------------------------
> > >>>> sys_perf_event_open: pid 1045843 cpu -1 group_fd -1 flags 0x8 = 4
> > >>>>
> > >>>> ... which should not be PERF_TYPE_HARDWARE && PERF_COUNT_HW_CPU_CYCLES.
> > >>>>
> > >>>> Marc said that he bisected the issue down to commit:
> > >>>>
> > >>>> 5ea8f2ccffb23983 ("perf parse-events: Support hardware events as terms")
> > >>>>
> > >>>> ... so it looks like something is going wrong when the events are being parsed,
> > >>>> e.g. losing the HW PMU information?
> > >>>
> > >>> Ok, I think I'm getting confused by other things. This looks like the issue.
> > >>>
> > >>> I think it may be working as intended, but not how you intended :-) If
> > >>> a core PMU is listed and then a legacy event, the legacy event should
>
> The point is that "cycles" when prefixed with "pmu/" shouldn't be
> considered "cycles" as HW/0, in that setting it is "cycles" for that
> PMU.
Exactly.
> (but we only have "cpu_cycles" for at least the a53 and a72 PMUs I
> have access in a Libre Computer rockchip 3399-pc hybrid board, if we use
> it, then we get what we want/had before, see below):
Both Cortex-A53 and Cortex-A72 have the common PMUv3 events, so they have
"cpu_cycles" and "bus_cycles".
The Apple PMUs that Hector and Marc anre using don't follow the PMUv3
architecture, and just have a "cycles" event.
[...]
> So what we need here seems to be to translate the generic term "cycles"
> to "cpu_cycles" when a PMU is explicitely passed in the event name and
> it doesn't have "cycles" and then just retry.
I'm not sure we need to map that.
My thinking is:
* If the user asks for "cycles" without a PMU name, that should use the
PERF_TYPE_HARDWARE cycles event. The ARM PMUs handle that correctly when the
event is directed to them.
* If the user asks for "${pmu}/cycles/", that should only use the "cycles"
event in that PMU's namespace, not PERF_TYPE_HARDWARE.
* If we need a way so say "use the PERF_TYPE_HARDWARE cycles event on ${pmu}",
then we should have a new syntax for that (e.g. as we have for raw events),
e.g. it would be possible to have "pmu/hw:cycles/" or something like that.
That way there's no ambiguity.
Mark.
next prev parent reply other threads:[~2023-11-22 15:50 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-11-21 12:08 [REGRESSION] Perf (userspace) broken on big.LITTLE systems since v6.5 Hector Martin
2023-11-21 13:40 ` Marc Zyngier
2023-11-21 15:24 ` Marc Zyngier
2023-11-21 15:40 ` Mark Rutland
2023-11-21 15:46 ` Ian Rogers
2023-11-21 16:02 ` Mark Rutland
2023-11-21 16:09 ` Ian Rogers
2023-11-21 16:15 ` Mark Rutland
2023-11-21 16:38 ` Ian Rogers
2023-11-22 3:23 ` Hector Martin
2023-11-22 13:06 ` Arnaldo Carvalho de Melo
2023-11-22 15:33 ` Ian Rogers
2023-11-22 15:49 ` Mark Rutland [this message]
2023-11-22 16:04 ` Ian Rogers
2023-11-22 16:26 ` Arnaldo Carvalho de Melo
2023-11-22 16:33 ` Ian Rogers
2023-11-22 16:19 ` Arnaldo Carvalho de Melo
2023-11-22 13:03 ` Mark Rutland
2023-11-22 15:29 ` Ian Rogers
2023-11-22 16:08 ` Mark Rutland
2023-11-22 16:29 ` Ian Rogers
2023-11-22 16:55 ` Arnaldo Carvalho de Melo
2023-11-22 16:59 ` Ian Rogers
2023-11-23 4:33 ` Ian Rogers
2023-11-21 15:41 ` Ian Rogers
2023-11-21 15:56 ` Mark Rutland
2023-11-21 16:03 ` Ian Rogers
2023-11-21 16:08 ` Mark Rutland
2023-11-23 14:23 ` Mark Rutland
2023-11-23 14:45 ` Marc Zyngier
2023-11-23 15:14 ` Ian Rogers
2023-11-23 16:48 ` Mark Rutland
2023-11-23 17:08 ` James Clark
2023-11-23 17:15 ` Mark Rutland
2023-11-21 23:43 ` Bagas Sanjaya
2023-12-06 12:09 ` Linux regression tracking #update (Thorsten Leemhuis)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZV4i_lrhbOVdEpwH@FVFF77S0Q05N \
--to=mark.rutland@arm.com \
--cc=acme@kernel.org \
--cc=acme@redhat.com \
--cc=asahi@lists.linux.dev \
--cc=irogers@google.com \
--cc=james.clark@arm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=maz@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).