[PATCH v4 00/11] Better fixes for grouping of events

[PATCH v4 00/11] Better fixes for grouping of events

[Ian Rogers]

The rules for grouping events have grown more complex. Topdown events
must be grouped, but flags like --metric-no-group and flags on metrics
don't respect this. Uncore events may be expanded using wild cards for
PMU names, but then the events need reordering so the group members
are adjacent. Rather than fixing metrics, this change fixes the main
event parsing code to first sort and then regroup evsels.

As this is shared functionality changes to it should cause
concern. The change is done with the intent of simplifying and making
more robust the grouping logic, examples are given. If additional
changes are necessary, they are most likely necessary to the
evsel__group_pmu_name logic as the code avoids breaking groups that
are on the same "group" PMU. The group_pmu_name is a variant of the
pmu_name tweaked in the case of software and aux events, that use
groups in a slightly different manner to conventional events.

The code was manually tested as well as passing perf test on a Intel
tigerlake CPU with intel-pt.

v4. Move the Intel pmu->auxtrace initialization to the existing
    perf_pmu__get_default_config as suggested by Adrian Hunter.
v3. Rename pmu_name to group_pmu_name and add patch to warn when
    events are regrouped as requested by Namhyung.
v2. Fix up the commit message on 4/10 (thanks Arnaldo). Drop
    unnecessary v1 5/10 (thanks Kan). evlist->core.nr_groups wasn't
    being correctly maintained after the sort/regrouping and so the
    new patch 10/10 removes that variable and computes it from the
    evlist when necessary, generally just tests.

Ian Rogers (11):
  libperf evlist: Avoid a use of evsel idx
  perf stat: Don't remove all grouped events when CPU maps disagree
  perf pmu: Earlier PMU auxtrace initialization
  perf stat: Modify the group test
  perf evsel: Allow const evsel for certain accesses
  perf evsel: Add function to compute group PMU name
  perf parse-events: Pass ownership of the group name
  perf parse-events: Sort and group parsed events
  perf evsel: Remove use_uncore_alias
  perf evlist: Remove nr_groups
  perf parse-events: Warn when events are regrouped

 tools/lib/perf/evlist.c                  |  31 ++-
 tools/lib/perf/include/internal/evlist.h |   1 -
 tools/lib/perf/include/perf/evlist.h     |   1 +
 tools/perf/arch/x86/util/auxtrace.c      |   4 -
 tools/perf/arch/x86/util/evlist.c        |  39 ++--
 tools/perf/arch/x86/util/pmu.c           |   8 +-
 tools/perf/builtin-record.c              |   2 +-
 tools/perf/builtin-report.c              |   2 +-
 tools/perf/builtin-stat.c                |  24 +-
 tools/perf/tests/bpf.c                   |   1 -
 tools/perf/tests/parse-events.c          |  24 +-
 tools/perf/tests/pfm.c                   |  12 +-
 tools/perf/tests/pmu-events.c            |   2 +-
 tools/perf/util/evlist.c                 |   2 +-
 tools/perf/util/evlist.h                 |   8 +-
 tools/perf/util/evsel.c                  |  27 ++-
 tools/perf/util/evsel.h                  |   8 +-
 tools/perf/util/header.c                 |   3 +-
 tools/perf/util/metricgroup.c            |   3 +-
 tools/perf/util/parse-events.c           | 268 +++++++++++------------
 tools/perf/util/parse-events.h           |  14 +-
 tools/perf/util/parse-events.y           |  28 +--
 tools/perf/util/pfm.c                    |   1 -
 tools/perf/util/pmu.c                    |   6 +-
 tools/perf/util/python.c                 |   2 +-
 tools/perf/util/stat-shadow.c            |   2 +-
 26 files changed, 277 insertions(+), 246 deletions(-)

-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 01/11] libperf evlist: Avoid a use of evsel idx

[Ian Rogers]

Setting the leader iterates the list, so rather than use idx (which
may be changed through list reordering) just count the elements and
set afterwards.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/perf/evlist.c | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/tools/lib/perf/evlist.c b/tools/lib/perf/evlist.c
index 61b637f29b82..2d6121e89ccb 100644
--- a/tools/lib/perf/evlist.c
+++ b/tools/lib/perf/evlist.c
@@ -687,15 +687,14 @@ perf_evlist__next_mmap(struct perf_evlist *evlist, struct perf_mmap *map,
 
 void __perf_evlist__set_leader(struct list_head *list, struct perf_evsel *leader)
 {
-	struct perf_evsel *first, *last, *evsel;
-
-	first = list_first_entry(list, struct perf_evsel, node);
-	last = list_last_entry(list, struct perf_evsel, node);
-
-	leader->nr_members = last->idx - first->idx + 1;
+	struct perf_evsel *evsel;
+	int n = 0;
 
-	__perf_evlist__for_each_entry(list, evsel)
+	__perf_evlist__for_each_entry(list, evsel) {
 		evsel->leader = leader;
+		n++;
+	}
+	leader->nr_members = n;
 }
 
 void perf_evlist__set_leader(struct perf_evlist *evlist)
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 02/11] perf stat: Don't remove all grouped events when CPU maps disagree

[Ian Rogers]

If the events in an evlist's CPU map differ then the entire group is
removed. For example:

```
$ perf stat -e '{imc_free_running/data_read/,imc_free_running/data_write/,cs}' -a sleep 1
WARNING: grouped events cpus do not match, disabling group:
  anon group { imc_free_running/data_read/, imc_free_running/data_write/, cs }
```

Change the behavior so that just the events not matching the leader
are removed. So in the example above, just 'cs' will be removed.

Modify the warning so that it is produced once for each group, rather
than once for the entire evlist. Shrink the scope and size of the
warning text buffer.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/builtin-stat.c | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index d70b1ec88594..5c12ae5efce5 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -181,14 +181,13 @@ static bool cpus_map_matched(struct evsel *a, struct evsel *b)
 
 static void evlist__check_cpu_maps(struct evlist *evlist)
 {
-	struct evsel *evsel, *pos, *leader;
-	char buf[1024];
+	struct evsel *evsel, *warned_leader = NULL;
 
 	if (evlist__has_hybrid(evlist))
 		evlist__warn_hybrid_group(evlist);
 
 	evlist__for_each_entry(evlist, evsel) {
-		leader = evsel__leader(evsel);
+		struct evsel *leader = evsel__leader(evsel);
 
 		/* Check that leader matches cpus with each member. */
 		if (leader == evsel)
@@ -197,19 +196,26 @@ static void evlist__check_cpu_maps(struct evlist *evlist)
 			continue;
 
 		/* If there's mismatch disable the group and warn user. */
-		WARN_ONCE(1, "WARNING: grouped events cpus do not match, disabling group:\n");
-		evsel__group_desc(leader, buf, sizeof(buf));
-		pr_warning("  %s\n", buf);
-
+		if (warned_leader != leader) {
+			char buf[200];
+
+			pr_warning("WARNING: grouped events cpus do not match.\n"
+				"Events with CPUs not matching the leader will "
+				"be removed from the group.\n");
+			evsel__group_desc(leader, buf, sizeof(buf));
+			pr_warning("  %s\n", buf);
+			warned_leader = leader;
+		}
 		if (verbose > 0) {
+			char buf[200];
+
 			cpu_map__snprint(leader->core.cpus, buf, sizeof(buf));
 			pr_warning("     %s: %s\n", leader->name, buf);
 			cpu_map__snprint(evsel->core.cpus, buf, sizeof(buf));
 			pr_warning("     %s: %s\n", evsel->name, buf);
 		}
 
-		for_each_group_evsel(pos, leader)
-			evsel__remove_from_group(pos, leader);
+		evsel__remove_from_group(evsel, leader);
 	}
 }
 
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 03/11] perf pmu: Earlier PMU auxtrace initialization

[Ian Rogers]

This allows event parsing to use the evsel__is_aux_event function,
which is important when determining event grouping.

Suggested-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/arch/x86/util/auxtrace.c | 4 ----
 tools/perf/arch/x86/util/pmu.c      | 8 ++++++--
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/tools/perf/arch/x86/util/auxtrace.c b/tools/perf/arch/x86/util/auxtrace.c
index 3da506e13f49..330d03216b0e 100644
--- a/tools/perf/arch/x86/util/auxtrace.c
+++ b/tools/perf/arch/x86/util/auxtrace.c
@@ -26,11 +26,7 @@ struct auxtrace_record *auxtrace_record__init_intel(struct evlist *evlist,
 	bool found_bts = false;
 
 	intel_pt_pmu = perf_pmu__find(INTEL_PT_PMU_NAME);
-	if (intel_pt_pmu)
-		intel_pt_pmu->auxtrace = true;
 	intel_bts_pmu = perf_pmu__find(INTEL_BTS_PMU_NAME);
-	if (intel_bts_pmu)
-		intel_bts_pmu->auxtrace = true;
 
 	evlist__for_each_entry(evlist, evsel) {
 		if (intel_pt_pmu && evsel->core.attr.type == intel_pt_pmu->type)
diff --git a/tools/perf/arch/x86/util/pmu.c b/tools/perf/arch/x86/util/pmu.c
index 358340b34243..f73b80dcd8bd 100644
--- a/tools/perf/arch/x86/util/pmu.c
+++ b/tools/perf/arch/x86/util/pmu.c
@@ -27,10 +27,14 @@ static bool cached_list;
 struct perf_event_attr *perf_pmu__get_default_config(struct perf_pmu *pmu __maybe_unused)
 {
 #ifdef HAVE_AUXTRACE_SUPPORT
-	if (!strcmp(pmu->name, INTEL_PT_PMU_NAME))
+	if (!strcmp(pmu->name, INTEL_PT_PMU_NAME)) {
+		pmu->auxtrace = true;
 		return intel_pt_pmu_default_config(pmu);
-	if (!strcmp(pmu->name, INTEL_BTS_PMU_NAME))
+	}
+	if (!strcmp(pmu->name, INTEL_BTS_PMU_NAME)) {
+		pmu->auxtrace = true;
 		pmu->selectable = true;
+	}
 #endif
 	return NULL;
 }
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 04/11] perf stat: Modify the group test

[Ian Rogers]

Currently nr_members is 0 for an event with no group, however, they
are always a leader of their own group. A later change will make that
count 1 because the event is its own leader. Make the find_stat logic
consistent with this, an improvement suggested by Namhyung Kim.

Suggested-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/stat-shadow.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index ef85f1ae1ab2..eeccab6751d7 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -163,7 +163,7 @@ static double find_stat(const struct evsel *evsel, int aggr_idx, enum stat_type
 			continue;
 
 		/* Ignore evsels that are part of different groups. */
-		if (evsel->core.leader->nr_members &&
+		if (evsel->core.leader->nr_members > 1 &&
 		    evsel->core.leader != cur->core.leader)
 			continue;
 		/* Ignore evsels with mismatched modifiers. */
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 05/11] perf evsel: Allow const evsel for certain accesses

[Ian Rogers]

List sorting, added later to evlist, passes const elements requiring
helper functions to also be const. Make the argument to
evsel__find_pmu, evsel__is_aux_event and evsel__leader const.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/evsel.c  | 2 +-
 tools/perf/util/evsel.h  | 6 +++---
 tools/perf/util/pmu.c    | 6 +++---
 tools/perf/util/python.c | 2 +-
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 51e8ce6edddc..2dc2c24252bb 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -3139,7 +3139,7 @@ bool evsel__is_hybrid(const struct evsel *evsel)
 	return evsel->pmu_name && perf_pmu__is_hybrid(evsel->pmu_name);
 }
 
-struct evsel *evsel__leader(struct evsel *evsel)
+struct evsel *evsel__leader(const struct evsel *evsel)
 {
 	return container_of(evsel->core.leader, struct evsel, core);
 }
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 814a49ebb7e3..676c499323e9 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -212,8 +212,8 @@ int evsel__object_config(size_t object_size,
 			 int (*init)(struct evsel *evsel),
 			 void (*fini)(struct evsel *evsel));
 
-struct perf_pmu *evsel__find_pmu(struct evsel *evsel);
-bool evsel__is_aux_event(struct evsel *evsel);
+struct perf_pmu *evsel__find_pmu(const struct evsel *evsel);
+bool evsel__is_aux_event(const struct evsel *evsel);
 
 struct evsel *evsel__new_idx(struct perf_event_attr *attr, int idx);
 
@@ -505,7 +505,7 @@ int evsel__store_ids(struct evsel *evsel, struct evlist *evlist);
 
 void evsel__zero_per_pkg(struct evsel *evsel);
 bool evsel__is_hybrid(const struct evsel *evsel);
-struct evsel *evsel__leader(struct evsel *evsel);
+struct evsel *evsel__leader(const struct evsel *evsel);
 bool evsel__has_leader(struct evsel *evsel, struct evsel *leader);
 bool evsel__is_leader(struct evsel *evsel);
 void evsel__set_leader(struct evsel *evsel, struct evsel *leader);
diff --git a/tools/perf/util/pmu.c b/tools/perf/util/pmu.c
index 43b6182d96b7..45d9b8e28e16 100644
--- a/tools/perf/util/pmu.c
+++ b/tools/perf/util/pmu.c
@@ -988,7 +988,7 @@ struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu)
 	return NULL;
 }
 
-struct perf_pmu *evsel__find_pmu(struct evsel *evsel)
+struct perf_pmu *evsel__find_pmu(const struct evsel *evsel)
 {
 	struct perf_pmu *pmu = NULL;
 
@@ -1000,11 +1000,11 @@ struct perf_pmu *evsel__find_pmu(struct evsel *evsel)
 			break;
 	}
 
-	evsel->pmu = pmu;
+	((struct evsel *)evsel)->pmu = pmu;
 	return pmu;
 }
 
-bool evsel__is_aux_event(struct evsel *evsel)
+bool evsel__is_aux_event(const struct evsel *evsel)
 {
 	struct perf_pmu *pmu = evsel__find_pmu(evsel);
 
diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index 42e8b813d010..ab48ffbb6448 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -83,7 +83,7 @@ const char *perf_env__arch(struct perf_env *env __maybe_unused)
  * far, for the perf python binding known usecases, revisit if this become
  * necessary.
  */
-struct perf_pmu *evsel__find_pmu(struct evsel *evsel __maybe_unused)
+struct perf_pmu *evsel__find_pmu(const struct evsel *evsel __maybe_unused)
 {
 	return NULL;
 }
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 06/11] perf evsel: Add function to compute group PMU name

[Ian Rogers]

The computed name respects software events and aux event groups, such
that the pmu_name is changed to be that of the aux event leader or
group leader for software events. This is done as a later change will
split events that are in different PMUs into different groups.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/evsel.c | 24 ++++++++++++++++++++++++
 tools/perf/util/evsel.h |  1 +
 2 files changed, 25 insertions(+)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 2dc2c24252bb..51d9650267d0 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -821,6 +821,30 @@ const char *evsel__name(struct evsel *evsel)
 	return "unknown";
 }
 
+const char *evsel__group_pmu_name(const struct evsel *evsel)
+{
+	const struct evsel *leader;
+
+	/* If the pmu_name is set use it. pmu_name isn't set for CPU and software events. */
+	if (evsel->pmu_name)
+		return evsel->pmu_name;
+	/*
+	 * Software events may be in a group with other uncore PMU events. Use
+	 * the pmu_name of the group leader to avoid breaking the software event
+	 * out of the group.
+	 *
+	 * Aux event leaders, like intel_pt, expect a group with events from
+	 * other PMUs, so substitute the AUX event's PMU in this case.
+	 */
+	leader  = evsel__leader(evsel);
+	if ((evsel->core.attr.type == PERF_TYPE_SOFTWARE || evsel__is_aux_event(leader)) &&
+	    leader->pmu_name) {
+		return leader->pmu_name;
+	}
+
+	return "cpu";
+}
+
 const char *evsel__metric_id(const struct evsel *evsel)
 {
 	if (evsel->metric_id)
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 676c499323e9..d26745ca6147 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -280,6 +280,7 @@ int arch_evsel__hw_name(struct evsel *evsel, char *bf, size_t size);
 
 int __evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result, char *bf, size_t size);
 const char *evsel__name(struct evsel *evsel);
+const char *evsel__group_pmu_name(const struct evsel *evsel);
 const char *evsel__metric_id(const struct evsel *evsel);
 
 static inline bool evsel__is_tool(const struct evsel *evsel)
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 07/11] perf parse-events: Pass ownership of the group name

[Ian Rogers]

Pass ownership of the group name rather than copying and freeing the
original. This saves a memory allocation and copy.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/parse-events.c | 3 ++-
 tools/perf/util/parse-events.y | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 0336ff27c15f..1be454697d57 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1761,6 +1761,7 @@ parse_events__set_leader_for_uncore_aliase(char *name, struct list_head *list,
 
 handled:
 	ret = 1;
+	free(name);
 out:
 	free(leaders);
 	return ret;
@@ -1786,7 +1787,7 @@ void parse_events__set_leader(char *name, struct list_head *list,
 
 	leader = arch_evlist__leader(list);
 	__perf_evlist__set_leader(list, &leader->core);
-	leader->group_name = name ? strdup(name) : NULL;
+	leader->group_name = name;
 	list_move(&leader->core.node, list);
 }
 
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index be8c51770051..541b8dde2063 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -202,8 +202,8 @@ PE_NAME '{' events '}'
 	struct list_head *list = $3;
 
 	inc_group_count(list, _parse_state);
+	/* Takes ownership of $1. */
 	parse_events__set_leader($1, list, _parse_state);
-	free($1);
 	$$ = list;
 }
 |
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 08/11] perf parse-events: Sort and group parsed events

[Ian Rogers]

This change is intended to be a no-op for most current cases, the
default sort order is the order the events were parsed. Where it
varies is in how groups are handled. Previously an uncore and core
event that are grouped would most often cause the group to be removed:

```
$ perf stat -e '{instructions,uncore_imc_free_running_0/data_total/}' -a sleep 1
WARNING: grouped events cpus do not match, disabling group:
  anon group { instructions, uncore_imc_free_running_0/data_total/ }
...
```

However, when wildcards are used the events should be re-sorted and
re-grouped in parse_events__set_leader, but this currently fails for
simple examples:

```
$ perf stat -e '{uncore_imc_free_running/data_read/,uncore_imc_free_running/data_write/}' -a sleep 1

 Performance counter stats for 'system wide':

     <not counted> MiB  uncore_imc_free_running/data_read/
     <not counted> MiB  uncore_imc_free_running/data_write/

       1.000996992 seconds time elapsed
```

A futher failure mode, fixed in this patch, is to force topdown events
into a group.

This change moves sorting the evsels in the evlist after parsing. It
requires parsing to set up groups. First the evsels are sorted
respecting the existing groupings and parse order, but also reordering
to ensure evsels of the same PMU and group appear together. So that
software and aux events respect groups, their pmu_name is taken from
the group leader. The sorting is done with list_sort removing a memory
allocation.

After sorting a pass is done to correct the group leaders and for
topdown events ensuring they have a group leader.

This fixes the problems seen before:

```
$ perf stat -e '{uncore_imc_free_running/data_read/,uncore_imc_free_running/data_write/}' -a sleep 1

 Performance counter stats for 'system wide':

            727.42 MiB  uncore_imc_free_running/data_read/
             81.84 MiB  uncore_imc_free_running/data_write/

       1.000948615 seconds time elapsed
```

As well as making groups not fail for cases like:

```
$ perf stat -e '{imc_free_running_0/data_total/,imc_free_running_1/data_total/}' -a sleep 1

 Performance counter stats for 'system wide':

            256.47 MiB  imc_free_running_0/data_total/
            256.48 MiB  imc_free_running_1/data_total/

       1.001165442 seconds time elapsed
```

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/arch/x86/util/evlist.c |  39 ++---
 tools/perf/util/evlist.h          |   2 +-
 tools/perf/util/parse-events.c    | 240 +++++++++++++++---------------
 tools/perf/util/parse-events.h    |   3 +-
 tools/perf/util/parse-events.y    |   4 +-
 5 files changed, 136 insertions(+), 152 deletions(-)

diff --git a/tools/perf/arch/x86/util/evlist.c b/tools/perf/arch/x86/util/evlist.c
index 8a7ae4162563..d4193479a364 100644
--- a/tools/perf/arch/x86/util/evlist.c
+++ b/tools/perf/arch/x86/util/evlist.c
@@ -65,29 +65,22 @@ int arch_evlist__add_default_attrs(struct evlist *evlist,
 	return ___evlist__add_default_attrs(evlist, attrs, nr_attrs);
 }
 
-struct evsel *arch_evlist__leader(struct list_head *list)
+int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs)
 {
-	struct evsel *evsel, *first, *slots = NULL;
-	bool has_topdown = false;
-
-	first = list_first_entry(list, struct evsel, core.node);
-
-	if (!topdown_sys_has_perf_metrics())
-		return first;
-
-	/* If there is a slots event and a topdown event then the slots event comes first. */
-	__evlist__for_each_entry(list, evsel) {
-		if (evsel->pmu_name && !strncmp(evsel->pmu_name, "cpu", 3) && evsel->name) {
-			if (strcasestr(evsel->name, "slots")) {
-				slots = evsel;
-				if (slots == first)
-					return first;
-			}
-			if (strcasestr(evsel->name, "topdown"))
-				has_topdown = true;
-			if (slots && has_topdown)
-				return slots;
-		}
+	if (topdown_sys_has_perf_metrics() &&
+	    (!lhs->pmu_name || !strncmp(lhs->pmu_name, "cpu", 3))) {
+		/* Ensure the topdown slots comes first. */
+		if (strcasestr(lhs->name, "slots"))
+			return -1;
+		if (strcasestr(rhs->name, "slots"))
+			return 1;
+		/* Followed by topdown events. */
+		if (strcasestr(lhs->name, "topdown") && !strcasestr(rhs->name, "topdown"))
+			return -1;
+		if (!strcasestr(lhs->name, "topdown") && strcasestr(rhs->name, "topdown"))
+			return 1;
 	}
-	return first;
+
+	/* Default ordering by insertion index. */
+	return lhs->core.idx - rhs->core.idx;
 }
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 01fa9d592c5a..d89d8f92802b 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -119,7 +119,7 @@ int arch_evlist__add_default_attrs(struct evlist *evlist,
 #define evlist__add_default_attrs(evlist, array) \
 	arch_evlist__add_default_attrs(evlist, array, ARRAY_SIZE(array))
 
-struct evsel *arch_evlist__leader(struct list_head *list);
+int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs);
 
 int evlist__add_dummy(struct evlist *evlist);
 struct evsel *evlist__add_aux_dummy(struct evlist *evlist, bool system_wide);
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 1be454697d57..394ab23089d0 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <linux/hw_breakpoint.h>
 #include <linux/err.h>
+#include <linux/list_sort.h>
 #include <linux/zalloc.h>
 #include <dirent.h>
 #include <errno.h>
@@ -1655,125 +1656,7 @@ int parse_events__modifier_group(struct list_head *list,
 	return parse_events__modifier_event(list, event_mod, true);
 }
 
-/*
- * Check if the two uncore PMUs are from the same uncore block
- * The format of the uncore PMU name is uncore_#blockname_#pmuidx
- */
-static bool is_same_uncore_block(const char *pmu_name_a, const char *pmu_name_b)
-{
-	char *end_a, *end_b;
-
-	end_a = strrchr(pmu_name_a, '_');
-	end_b = strrchr(pmu_name_b, '_');
-
-	if (!end_a || !end_b)
-		return false;
-
-	if ((end_a - pmu_name_a) != (end_b - pmu_name_b))
-		return false;
-
-	return (strncmp(pmu_name_a, pmu_name_b, end_a - pmu_name_a) == 0);
-}
-
-static int
-parse_events__set_leader_for_uncore_aliase(char *name, struct list_head *list,
-					   struct parse_events_state *parse_state)
-{
-	struct evsel *evsel, *leader;
-	uintptr_t *leaders;
-	bool is_leader = true;
-	int i, nr_pmu = 0, total_members, ret = 0;
-
-	leader = list_first_entry(list, struct evsel, core.node);
-	evsel = list_last_entry(list, struct evsel, core.node);
-	total_members = evsel->core.idx - leader->core.idx + 1;
-
-	leaders = calloc(total_members, sizeof(uintptr_t));
-	if (WARN_ON(!leaders))
-		return 0;
-
-	/*
-	 * Going through the whole group and doing sanity check.
-	 * All members must use alias, and be from the same uncore block.
-	 * Also, storing the leader events in an array.
-	 */
-	__evlist__for_each_entry(list, evsel) {
-
-		/* Only split the uncore group which members use alias */
-		if (!evsel->use_uncore_alias)
-			goto out;
-
-		/* The events must be from the same uncore block */
-		if (!is_same_uncore_block(leader->pmu_name, evsel->pmu_name))
-			goto out;
-
-		if (!is_leader)
-			continue;
-		/*
-		 * If the event's PMU name starts to repeat, it must be a new
-		 * event. That can be used to distinguish the leader from
-		 * other members, even they have the same event name.
-		 */
-		if ((leader != evsel) &&
-		    !strcmp(leader->pmu_name, evsel->pmu_name)) {
-			is_leader = false;
-			continue;
-		}
-
-		/* Store the leader event for each PMU */
-		leaders[nr_pmu++] = (uintptr_t) evsel;
-	}
-
-	/* only one event alias */
-	if (nr_pmu == total_members) {
-		parse_state->nr_groups--;
-		goto handled;
-	}
-
-	/*
-	 * An uncore event alias is a joint name which means the same event
-	 * runs on all PMUs of a block.
-	 * Perf doesn't support mixed events from different PMUs in the same
-	 * group. The big group has to be split into multiple small groups
-	 * which only include the events from the same PMU.
-	 *
-	 * Here the uncore event aliases must be from the same uncore block.
-	 * The number of PMUs must be same for each alias. The number of new
-	 * small groups equals to the number of PMUs.
-	 * Setting the leader event for corresponding members in each group.
-	 */
-	i = 0;
-	__evlist__for_each_entry(list, evsel) {
-		if (i >= nr_pmu)
-			i = 0;
-		evsel__set_leader(evsel, (struct evsel *) leaders[i++]);
-	}
-
-	/* The number of members and group name are same for each group */
-	for (i = 0; i < nr_pmu; i++) {
-		evsel = (struct evsel *) leaders[i];
-		evsel->core.nr_members = total_members / nr_pmu;
-		evsel->group_name = name ? strdup(name) : NULL;
-	}
-
-	/* Take the new small groups into account */
-	parse_state->nr_groups += nr_pmu - 1;
-
-handled:
-	ret = 1;
-	free(name);
-out:
-	free(leaders);
-	return ret;
-}
-
-__weak struct evsel *arch_evlist__leader(struct list_head *list)
-{
-	return list_first_entry(list, struct evsel, core.node);
-}
-
-void parse_events__set_leader(char *name, struct list_head *list,
-			      struct parse_events_state *parse_state)
+void parse_events__set_leader(char *name, struct list_head *list)
 {
 	struct evsel *leader;
 
@@ -1782,13 +1665,9 @@ void parse_events__set_leader(char *name, struct list_head *list,
 		return;
 	}
 
-	if (parse_events__set_leader_for_uncore_aliase(name, list, parse_state))
-		return;
-
-	leader = arch_evlist__leader(list);
+	leader = list_first_entry(list, struct evsel, core.node);
 	__perf_evlist__set_leader(list, &leader->core);
 	leader->group_name = name;
-	list_move(&leader->core.node, list);
 }
 
 /* list_event is assumed to point to malloc'ed memory */
@@ -2245,6 +2124,117 @@ static int parse_events__with_hybrid_pmu(struct parse_events_state *parse_state,
 	return ret;
 }
 
+__weak int arch_evlist__cmp(const struct evsel *lhs, const struct evsel *rhs)
+{
+	/* Order by insertion index. */
+	return lhs->core.idx - rhs->core.idx;
+}
+
+static int evlist__cmp(void *state, const struct list_head *l, const struct list_head *r)
+{
+	const struct perf_evsel *lhs_core = container_of(l, struct perf_evsel, node);
+	const struct evsel *lhs = container_of(lhs_core, struct evsel, core);
+	const struct perf_evsel *rhs_core = container_of(r, struct perf_evsel, node);
+	const struct evsel *rhs = container_of(rhs_core, struct evsel, core);
+	int *leader_idx = state;
+	int lhs_leader_idx = *leader_idx, rhs_leader_idx = *leader_idx, ret;
+	const char *lhs_pmu_name, *rhs_pmu_name;
+
+	/*
+	 * First sort by grouping/leader. Read the leader idx only if the evsel
+	 * is part of a group, as -1 indicates no group.
+	 */
+	if (lhs_core->leader != lhs_core || lhs_core->nr_members > 1)
+		lhs_leader_idx = lhs_core->leader->idx;
+	if (rhs_core->leader != rhs_core || rhs_core->nr_members > 1)
+		rhs_leader_idx = rhs_core->leader->idx;
+
+	if (lhs_leader_idx != rhs_leader_idx)
+		return lhs_leader_idx - rhs_leader_idx;
+
+	/* Group by PMU. Groups can't span PMUs. */
+	lhs_pmu_name = evsel__group_pmu_name(lhs);
+	rhs_pmu_name = evsel__group_pmu_name(rhs);
+	ret = strcmp(lhs_pmu_name, rhs_pmu_name);
+	if (ret)
+		return ret;
+
+	/* Architecture specific sorting. */
+	return arch_evlist__cmp(lhs, rhs);
+}
+
+static void parse_events__sort_events_and_fix_groups(struct list_head *list)
+{
+	int idx = -1;
+	struct evsel *pos, *cur_leader = NULL;
+	struct perf_evsel *cur_leaders_grp = NULL;
+
+	/*
+	 * Compute index to insert ungrouped events at. Place them where the
+	 * first ungrouped event appears.
+	 */
+	list_for_each_entry(pos, list, core.node) {
+		const struct evsel *pos_leader = evsel__leader(pos);
+
+		if (pos != pos_leader || pos->core.nr_members > 1)
+			continue;
+
+		idx = pos->core.idx;
+		break;
+	}
+
+	/* Sort events. */
+	list_sort(&idx, list, evlist__cmp);
+
+	/*
+	 * Recompute groups, splitting for PMUs and adding groups for events
+	 * that require them.
+	 */
+	idx = 0;
+	list_for_each_entry(pos, list, core.node) {
+		const struct evsel *pos_leader = evsel__leader(pos);
+		const char *pos_pmu_name = evsel__group_pmu_name(pos);
+		const char *cur_leader_pmu_name, *pos_leader_pmu_name;
+		bool force_grouped = arch_evsel__must_be_in_group(pos);
+
+		/* Reset index and nr_members. */
+		pos->core.idx = idx++;
+		pos->core.nr_members = 0;
+
+		/*
+		 * Set the group leader respecting the given groupings and that
+		 * groups can't span PMUs.
+		 */
+		if (!cur_leader)
+			cur_leader = pos;
+
+		cur_leader_pmu_name = evsel__group_pmu_name(cur_leader);
+		if ((cur_leaders_grp != pos->core.leader && !force_grouped) ||
+		    strcmp(cur_leader_pmu_name, pos_pmu_name)) {
+			/* Event is for a different group/PMU than last. */
+			cur_leader = pos;
+			/*
+			 * Remember the leader's group before it is overwritten,
+			 * so that later events match as being in the same
+			 * group.
+			 */
+			cur_leaders_grp = pos->core.leader;
+		}
+		pos_leader_pmu_name = evsel__group_pmu_name(pos_leader);
+		if (strcmp(pos_leader_pmu_name, pos_pmu_name) || force_grouped) {
+			/*
+			 * Event's PMU differs from its leader's. Groups can't
+			 * span PMUs, so update leader from the group/PMU
+			 * tracker.
+			 */
+			evsel__set_leader(pos, cur_leader);
+		}
+	}
+	list_for_each_entry(pos, list, core.node) {
+		pos->core.leader->nr_members++;
+	}
+}
+
 int __parse_events(struct evlist *evlist, const char *str,
 		   struct parse_events_error *err, struct perf_pmu *fake_pmu)
 {
@@ -2266,6 +2256,8 @@ int __parse_events(struct evlist *evlist, const char *str,
 		return -1;
 	}
 
+	parse_events__sort_events_and_fix_groups(&parse_state.list);
+
 	/*
 	 * Add list to the evlist even with errors to allow callers to clean up.
 	 */
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 428e72eaafcc..22fc11b0bd59 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -200,8 +200,7 @@ int parse_events_copy_term_list(struct list_head *old,
 
 enum perf_pmu_event_symbol_type
 perf_pmu__parse_check(const char *name);
-void parse_events__set_leader(char *name, struct list_head *list,
-			      struct parse_events_state *parse_state);
+void parse_events__set_leader(char *name, struct list_head *list);
 void parse_events_update_lists(struct list_head *list_event,
 			       struct list_head *list_all);
 void parse_events_evlist_error(struct parse_events_state *parse_state,
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 541b8dde2063..90d12f2bc8be 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -203,7 +203,7 @@ PE_NAME '{' events '}'
 
 	inc_group_count(list, _parse_state);
 	/* Takes ownership of $1. */
-	parse_events__set_leader($1, list, _parse_state);
+	parse_events__set_leader($1, list);
 	$$ = list;
 }
 |
@@ -212,7 +212,7 @@ PE_NAME '{' events '}'
 	struct list_head *list = $2;
 
 	inc_group_count(list, _parse_state);
-	parse_events__set_leader(NULL, list, _parse_state);
+	parse_events__set_leader(NULL, list);
 	$$ = list;
 }
 
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 09/11] perf evsel: Remove use_uncore_alias

[Ian Rogers]

This flag used to be used when regrouping uncore events in particular
due to wildcard matches. This is now handled by sorting evlist and so
the flag is redundant.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/evsel.c        |  1 -
 tools/perf/util/evsel.h        |  1 -
 tools/perf/util/parse-events.c | 12 +++---------
 tools/perf/util/parse-events.h |  3 +--
 tools/perf/util/parse-events.y | 11 +++++++----
 5 files changed, 11 insertions(+), 17 deletions(-)

diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 51d9650267d0..b9b05091bc8f 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -458,7 +458,6 @@ struct evsel *evsel__clone(struct evsel *orig)
 	evsel->per_pkg = orig->per_pkg;
 	evsel->percore = orig->percore;
 	evsel->precise_max = orig->precise_max;
-	evsel->use_uncore_alias = orig->use_uncore_alias;
 	evsel->is_libpfm_event = orig->is_libpfm_event;
 
 	evsel->exclude_GH = orig->exclude_GH;
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index d26745ca6147..c272c06565c0 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -89,7 +89,6 @@ struct evsel {
 		bool			per_pkg;
 		bool			percore;
 		bool			precise_max;
-		bool			use_uncore_alias;
 		bool			is_libpfm_event;
 		bool			auto_merge_stats;
 		bool			collect_stat;
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 394ab23089d0..93a90651266f 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -1445,15 +1445,13 @@ static int parse_events__inside_hybrid_pmu(struct parse_events_state *parse_stat
 int parse_events_add_pmu(struct parse_events_state *parse_state,
 			 struct list_head *list, char *name,
 			 struct list_head *head_config,
-			 bool auto_merge_stats,
-			 bool use_alias)
+			 bool auto_merge_stats)
 {
 	struct perf_event_attr attr;
 	struct perf_pmu_info info;
 	struct perf_pmu *pmu;
 	struct evsel *evsel;
 	struct parse_events_error *err = parse_state->error;
-	bool use_uncore_alias;
 	LIST_HEAD(config_terms);
 
 	pmu = parse_state->fake_pmu ?: perf_pmu__find(name);
@@ -1488,8 +1486,6 @@ int parse_events_add_pmu(struct parse_events_state *parse_state,
 		memset(&attr, 0, sizeof(attr));
 	}
 
-	use_uncore_alias = (pmu->is_uncore && use_alias);
-
 	if (!head_config) {
 		attr.type = pmu->type;
 		evsel = __add_event(list, &parse_state->idx, &attr,
@@ -1499,7 +1495,6 @@ int parse_events_add_pmu(struct parse_events_state *parse_state,
 				    /*cpu_list=*/NULL);
 		if (evsel) {
 			evsel->pmu_name = name ? strdup(name) : NULL;
-			evsel->use_uncore_alias = use_uncore_alias;
 			return 0;
 		} else {
 			return -ENOMEM;
@@ -1560,7 +1555,6 @@ int parse_events_add_pmu(struct parse_events_state *parse_state,
 		evsel->use_config_name = true;
 
 	evsel->pmu_name = name ? strdup(name) : NULL;
-	evsel->use_uncore_alias = use_uncore_alias;
 	evsel->percore = config_term_percore(&evsel->config_terms);
 
 	if (parse_state->fake_pmu)
@@ -1622,7 +1616,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
 				parse_events_copy_term_list(head, &orig_head);
 				if (!parse_events_add_pmu(parse_state, list,
 							  pmu->name, orig_head,
-							  true, true)) {
+							  /*auto_merge_stats=*/true)) {
 					pr_debug("%s -> %s/%s/\n", str,
 						 pmu->name, alias->str);
 					ok++;
@@ -1634,7 +1628,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,
 
 	if (parse_state->fake_pmu) {
 		if (!parse_events_add_pmu(parse_state, list, str, head,
-					  true, true)) {
+					  /*auto_merge_stats=*/true)) {
 			pr_debug("%s -> %s/%s/\n", str, "fake_pmu", str);
 			ok++;
 		}
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 22fc11b0bd59..fdac44dc696b 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -183,8 +183,7 @@ int parse_events_add_breakpoint(struct list_head *list, int *idx,
 int parse_events_add_pmu(struct parse_events_state *parse_state,
 			 struct list_head *list, char *name,
 			 struct list_head *head_config,
-			 bool auto_merge_stats,
-			 bool use_alias);
+			 bool auto_merge_stats);
 
 struct evsel *parse_events__add_event(int idx, struct perf_event_attr *attr,
 				      const char *name, const char *metric_id,
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 90d12f2bc8be..f1b153c72d67 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -313,7 +313,7 @@ event_pmu_name opt_pmu_config
 	list = alloc_list();
 	if (!list)
 		CLEANUP_YYABORT;
-	if (parse_events_add_pmu(_parse_state, list, $1, $2, false, false)) {
+	if (parse_events_add_pmu(_parse_state, list, $1, $2, /*auto_merge_stats=*/false)) {
 		struct perf_pmu *pmu = NULL;
 		int ok = 0;
 
@@ -330,8 +330,10 @@ event_pmu_name opt_pmu_config
 			    !perf_pmu__match(pattern, pmu->alias_name, $1)) {
 				if (parse_events_copy_term_list(orig_terms, &terms))
 					CLEANUP_YYABORT;
-				if (!parse_events_add_pmu(_parse_state, list, pmu->name, terms, true, false))
+				if (!parse_events_add_pmu(_parse_state, list, pmu->name, terms,
+							  /*auto_merge_stats=*/true)) {
 					ok++;
+				}
 				parse_events_terms__delete(terms);
 			}
 		}
@@ -407,7 +409,8 @@ PE_PMU_EVENT_FAKE sep_dc
 	if (!list)
 		YYABORT;
 
-	err = parse_events_add_pmu(_parse_state, list, $1, NULL, false, false);
+	err = parse_events_add_pmu(_parse_state, list, $1, /*head_config=*/NULL,
+				   /*auto_merge_stats=*/false);
 	free($1);
 	if (err < 0) {
 		free(list);
@@ -425,7 +428,7 @@ PE_PMU_EVENT_FAKE opt_pmu_config
 	if (!list)
 		YYABORT;
 
-	err = parse_events_add_pmu(_parse_state, list, $1, $2, false, false);
+	err = parse_events_add_pmu(_parse_state, list, $1, $2, /*auto_merge_stats=*/false);
 	free($1);
 	parse_events_terms__delete($2);
 	if (err < 0) {
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 10/11] perf evlist: Remove nr_groups

[Ian Rogers]

Maintaining the number of groups during event parsing is problematic
and since changing to sort/regroup events can only be computed by a
linear pass over the evlist. As the value is generally only used in
tests, rather than hold it in a variable compute it by passing over
the evlist when necessary.

This change highlights that libpfm's counting of groups with a single
entry disagreed with regular event parsing. The libpfm tests are
updated accordingly.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/lib/perf/evlist.c                  | 18 +++++++++++++++++-
 tools/lib/perf/include/internal/evlist.h |  1 -
 tools/lib/perf/include/perf/evlist.h     |  1 +
 tools/perf/builtin-record.c              |  2 +-
 tools/perf/builtin-report.c              |  2 +-
 tools/perf/tests/bpf.c                   |  1 -
 tools/perf/tests/parse-events.c          | 22 +++++++++++-----------
 tools/perf/tests/pfm.c                   | 12 ++++++------
 tools/perf/util/evlist.c                 |  2 +-
 tools/perf/util/evlist.h                 |  6 ++++++
 tools/perf/util/header.c                 |  3 +--
 tools/perf/util/parse-events.c           |  1 -
 tools/perf/util/parse-events.h           |  1 -
 tools/perf/util/parse-events.y           | 10 ----------
 tools/perf/util/pfm.c                    |  1 -
 15 files changed, 45 insertions(+), 38 deletions(-)

diff --git a/tools/lib/perf/evlist.c b/tools/lib/perf/evlist.c
index 2d6121e89ccb..81e8b5fcd8ba 100644
--- a/tools/lib/perf/evlist.c
+++ b/tools/lib/perf/evlist.c
@@ -703,7 +703,23 @@ void perf_evlist__set_leader(struct perf_evlist *evlist)
 		struct perf_evsel *first = list_entry(evlist->entries.next,
 						struct perf_evsel, node);
 
-		evlist->nr_groups = evlist->nr_entries > 1 ? 1 : 0;
 		__perf_evlist__set_leader(&evlist->entries, first);
 	}
 }
+
+int perf_evlist__nr_groups(struct perf_evlist *evlist)
+{
+	struct perf_evsel *evsel;
+	int nr_groups = 0;
+
+	perf_evlist__for_each_evsel(evlist, evsel) {
+		/*
+		 * evsels by default have a nr_members of 1, and they are their
+		 * own leader. If the nr_members is >1 then this is an
+		 * indication of a group.
+		 */
+		if (evsel->leader == evsel && evsel->nr_members > 1)
+			nr_groups++;
+	}
+	return nr_groups;
+}
diff --git a/tools/lib/perf/include/internal/evlist.h b/tools/lib/perf/include/internal/evlist.h
index 850f07070036..3339bc2f1765 100644
--- a/tools/lib/perf/include/internal/evlist.h
+++ b/tools/lib/perf/include/internal/evlist.h
@@ -17,7 +17,6 @@ struct perf_mmap_param;
 struct perf_evlist {
 	struct list_head	 entries;
 	int			 nr_entries;
-	int			 nr_groups;
 	bool			 has_user_cpus;
 	bool			 needs_map_propagation;
 	/**
diff --git a/tools/lib/perf/include/perf/evlist.h b/tools/lib/perf/include/perf/evlist.h
index 9ca399d49bb4..e894b770779e 100644
--- a/tools/lib/perf/include/perf/evlist.h
+++ b/tools/lib/perf/include/perf/evlist.h
@@ -47,4 +47,5 @@ LIBPERF_API struct perf_mmap *perf_evlist__next_mmap(struct perf_evlist *evlist,
 	     (pos) = perf_evlist__next_mmap((evlist), (pos), overwrite))
 
 LIBPERF_API void perf_evlist__set_leader(struct perf_evlist *evlist);
+LIBPERF_API int perf_evlist__nr_groups(struct perf_evlist *evlist);
 #endif /* __LIBPERF_EVLIST_H */
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 8374117e66f6..e71c110a0cd4 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -2474,7 +2474,7 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 		rec->tool.ordered_events = false;
 	}
 
-	if (!rec->evlist->core.nr_groups)
+	if (evlist__nr_groups(rec->evlist) == 0)
 		perf_header__clear_feat(&session->header, HEADER_GROUP_DESC);
 
 	if (data->is_pipe) {
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 2ee2ecca208e..6400615b5e98 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -1481,7 +1481,7 @@ int cmd_report(int argc, const char **argv)
 
 	setup_forced_leader(&report, session->evlist);
 
-	if (symbol_conf.group_sort_idx && !session->evlist->core.nr_groups) {
+	if (symbol_conf.group_sort_idx && evlist__nr_groups(session->evlist) == 0) {
 		parse_options_usage(NULL, options, "group-sort-idx", 0);
 		ret = -EINVAL;
 		goto error;
diff --git a/tools/perf/tests/bpf.c b/tools/perf/tests/bpf.c
index ae9223f27cfb..8beb46066034 100644
--- a/tools/perf/tests/bpf.c
+++ b/tools/perf/tests/bpf.c
@@ -153,7 +153,6 @@ static int do_test(struct bpf_object *obj, int (*func)(void),
 	}
 
 	evlist__splice_list_tail(evlist, &parse_state.list);
-	evlist->core.nr_groups = parse_state.nr_groups;
 
 	evlist__config(evlist, &opts, NULL);
 
diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
index 71a5cb343311..ffa6f0a90741 100644
--- a/tools/perf/tests/parse-events.c
+++ b/tools/perf/tests/parse-events.c
@@ -53,7 +53,7 @@ static int test__checkevent_tracepoint(struct evlist *evlist)
 	struct evsel *evsel = evlist__first(evlist);
 
 	TEST_ASSERT_VAL("wrong number of entries", 1 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 0 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 0 == evlist__nr_groups(evlist));
 	TEST_ASSERT_VAL("wrong type", PERF_TYPE_TRACEPOINT == evsel->core.attr.type);
 	TEST_ASSERT_VAL("wrong sample_type",
 		PERF_TP_SAMPLE_TYPE == evsel->core.attr.sample_type);
@@ -66,7 +66,7 @@ static int test__checkevent_tracepoint_multi(struct evlist *evlist)
 	struct evsel *evsel;
 
 	TEST_ASSERT_VAL("wrong number of entries", evlist->core.nr_entries > 1);
-	TEST_ASSERT_VAL("wrong number of groups", 0 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 0 == evlist__nr_groups(evlist));
 
 	evlist__for_each_entry(evlist, evsel) {
 		TEST_ASSERT_VAL("wrong type",
@@ -677,7 +677,7 @@ static int test__group1(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* instructions:k */
 	evsel = leader = evlist__first(evlist);
@@ -719,7 +719,7 @@ static int test__group2(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 3 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* faults + :ku modifier */
 	evsel = leader = evlist__first(evlist);
@@ -775,7 +775,7 @@ static int test__group3(struct evlist *evlist __maybe_unused)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 5 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 2 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 2 == evlist__nr_groups(evlist));
 
 	/* group1 syscalls:sys_enter_openat:H */
 	evsel = leader = evlist__first(evlist);
@@ -868,7 +868,7 @@ static int test__group4(struct evlist *evlist __maybe_unused)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* cycles:u + p */
 	evsel = leader = evlist__first(evlist);
@@ -912,7 +912,7 @@ static int test__group5(struct evlist *evlist __maybe_unused)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 5 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 2 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 2 == evlist__nr_groups(evlist));
 
 	/* cycles + G */
 	evsel = leader = evlist__first(evlist);
@@ -998,7 +998,7 @@ static int test__group_gh1(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* cycles + :H group modifier */
 	evsel = leader = evlist__first(evlist);
@@ -1038,7 +1038,7 @@ static int test__group_gh2(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* cycles + :G group modifier */
 	evsel = leader = evlist__first(evlist);
@@ -1078,7 +1078,7 @@ static int test__group_gh3(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* cycles:G + :u group modifier */
 	evsel = leader = evlist__first(evlist);
@@ -1118,7 +1118,7 @@ static int test__group_gh4(struct evlist *evlist)
 	struct evsel *evsel, *leader;
 
 	TEST_ASSERT_VAL("wrong number of entries", 2 == evlist->core.nr_entries);
-	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist->core.nr_groups);
+	TEST_ASSERT_VAL("wrong number of groups", 1 == evlist__nr_groups(evlist));
 
 	/* cycles:G + :uG group modifier */
 	evsel = leader = evlist__first(evlist);
diff --git a/tools/perf/tests/pfm.c b/tools/perf/tests/pfm.c
index 71b76deb1f92..2e38dfa34b6c 100644
--- a/tools/perf/tests/pfm.c
+++ b/tools/perf/tests/pfm.c
@@ -76,7 +76,7 @@ static int test__pfm_events(struct test_suite *test __maybe_unused,
 				count_pfm_events(&evlist->core),
 				table[i].nr_events);
 		TEST_ASSERT_EQUAL(table[i].events,
-				evlist->core.nr_groups,
+				evlist__nr_groups(evlist),
 				0);
 
 		evlist__delete(evlist);
@@ -103,22 +103,22 @@ static int test__pfm_group(struct test_suite *test __maybe_unused,
 		{
 			.events = "{instructions}",
 			.nr_events = 1,
-			.nr_groups = 1,
+			.nr_groups = 0,
 		},
 		{
 			.events = "{instructions},{}",
 			.nr_events = 1,
-			.nr_groups = 1,
+			.nr_groups = 0,
 		},
 		{
 			.events = "{},{instructions}",
 			.nr_events = 1,
-			.nr_groups = 1,
+			.nr_groups = 0,
 		},
 		{
 			.events = "{instructions},{instructions}",
 			.nr_events = 2,
-			.nr_groups = 2,
+			.nr_groups = 0,
 		},
 		{
 			.events = "{instructions,cycles},{instructions,cycles}",
@@ -161,7 +161,7 @@ static int test__pfm_group(struct test_suite *test __maybe_unused,
 				count_pfm_events(&evlist->core),
 				table[i].nr_events);
 		TEST_ASSERT_EQUAL(table[i].events,
-				evlist->core.nr_groups,
+				evlist__nr_groups(evlist),
 				table[i].nr_groups);
 
 		evlist__delete(evlist);
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index 817df2504a1e..a5f406c468f8 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -1777,7 +1777,7 @@ bool evlist__exclude_kernel(struct evlist *evlist)
  */
 void evlist__force_leader(struct evlist *evlist)
 {
-	if (!evlist->core.nr_groups) {
+	if (evlist__nr_groups(evlist) == 0) {
 		struct evsel *leader = evlist__first(evlist);
 
 		evlist__set_leader(evlist);
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index d89d8f92802b..46cf402add93 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -9,6 +9,7 @@
 #include <api/fd/array.h>
 #include <internal/evlist.h>
 #include <internal/evsel.h>
+#include <perf/evlist.h>
 #include "events_stats.h"
 #include "evsel.h"
 #include <pthread.h>
@@ -255,6 +256,11 @@ static inline struct evsel *evlist__last(struct evlist *evlist)
 	return container_of(evsel, struct evsel, core);
 }
 
+static inline int evlist__nr_groups(struct evlist *evlist)
+{
+	return perf_evlist__nr_groups(&evlist->core);
+}
+
 int evlist__strerror_open(struct evlist *evlist, int err, char *buf, size_t size);
 int evlist__strerror_mmap(struct evlist *evlist, int err, char *buf, size_t size);
 
diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index 404d816ca124..276870221ce0 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -786,7 +786,7 @@ static int write_pmu_mappings(struct feat_fd *ff,
 static int write_group_desc(struct feat_fd *ff,
 			    struct evlist *evlist)
 {
-	u32 nr_groups = evlist->core.nr_groups;
+	u32 nr_groups = evlist__nr_groups(evlist);
 	struct evsel *evsel;
 	int ret;
 
@@ -2807,7 +2807,6 @@ static int process_group_desc(struct feat_fd *ff, void *data __maybe_unused)
 	 * Rebuild group relationship based on the group_desc
 	 */
 	session = container_of(ff->ph, struct perf_session, header);
-	session->evlist->core.nr_groups = nr_groups;
 
 	i = nr = 0;
 	evlist__for_each_entry(session->evlist, evsel) {
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 93a90651266f..9ec3c1dc81e0 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -2260,7 +2260,6 @@ int __parse_events(struct evlist *evlist, const char *str,
 	if (!ret) {
 		struct evsel *last;
 
-		evlist->core.nr_groups += parse_state.nr_groups;
 		last = evlist__last(evlist);
 		last->cmdline_group_boundary = true;
 
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index fdac44dc696b..767ad1729228 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -122,7 +122,6 @@ struct parse_events_error {
 struct parse_events_state {
 	struct list_head	   list;
 	int			   idx;
-	int			   nr_groups;
 	struct parse_events_error *error;
 	struct evlist		  *evlist;
 	struct list_head	  *terms;
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index f1b153c72d67..3a04602d2982 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -49,14 +49,6 @@ static void free_list_evsel(struct list_head* list_evsel)
 	free(list_evsel);
 }
 
-static void inc_group_count(struct list_head *list,
-		       struct parse_events_state *parse_state)
-{
-	/* Count groups only have more than 1 members */
-	if (!list_is_last(list->next, list))
-		parse_state->nr_groups++;
-}
-
 %}
 
 %token PE_START_EVENTS PE_START_TERMS
@@ -201,7 +193,6 @@ PE_NAME '{' events '}'
 {
 	struct list_head *list = $3;
 
-	inc_group_count(list, _parse_state);
 	/* Takes ownership of $1. */
 	parse_events__set_leader($1, list);
 	$$ = list;
@@ -211,7 +202,6 @@ PE_NAME '{' events '}'
 {
 	struct list_head *list = $2;
 
-	inc_group_count(list, _parse_state);
 	parse_events__set_leader(NULL, list);
 	$$ = list;
 }
diff --git a/tools/perf/util/pfm.c b/tools/perf/util/pfm.c
index b59ba825ddc9..6c11914c179f 100644
--- a/tools/perf/util/pfm.c
+++ b/tools/perf/util/pfm.c
@@ -112,7 +112,6 @@ int parse_libpfm_events_option(const struct option *opt, const char *str,
 				   "cannot close a non-existing event group\n");
 				goto error;
 			}
-			evlist->core.nr_groups++;
 			grp_leader = NULL;
 			grp_evt = -1;
 		}
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

[PATCH v4 11/11] perf parse-events: Warn when events are regrouped

[Ian Rogers]

Use if an event is reordered or the number of groups increases to
signal that regrouping has happened and warn about it. Disable the
warning in the case wild card PMU names are used and for metrics.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/parse-events.c |  2 +-
 tools/perf/tests/pmu-events.c   |  2 +-
 tools/perf/util/metricgroup.c   |  3 ++-
 tools/perf/util/parse-events.c  | 28 ++++++++++++++++++++--------
 tools/perf/util/parse-events.h  |  7 ++++---
 tools/perf/util/parse-events.y  |  1 +
 6 files changed, 29 insertions(+), 14 deletions(-)

diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
index ffa6f0a90741..b1c2f0a20306 100644
--- a/tools/perf/tests/parse-events.c
+++ b/tools/perf/tests/parse-events.c
@@ -2103,7 +2103,7 @@ static int test_event_fake_pmu(const char *str)
 
 	parse_events_error__init(&err);
 	perf_pmu__test_parse_init();
-	ret = __parse_events(evlist, str, &err, &perf_pmu__fake);
+	ret = __parse_events(evlist, str, &err, &perf_pmu__fake, /*warn_if_reordered=*/true);
 	if (ret) {
 		pr_debug("failed to parse event '%s', err %d, str '%s'\n",
 			 str, ret, err.str);
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index 6ccd413b5983..7f8e86452527 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -785,7 +785,7 @@ static int check_parse_id(const char *id, struct parse_events_error *error,
 		 */
 		perf_pmu__test_parse_init();
 	}
-	ret = __parse_events(evlist, dup, error, fake_pmu);
+	ret = __parse_events(evlist, dup, error, fake_pmu, /*warn_if_reordered=*/true);
 	free(dup);
 
 	evlist__delete(evlist);
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index de6dd527a2ba..5783f4c2d1ef 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -1441,7 +1441,8 @@ static int parse_ids(bool metric_no_merge, struct perf_pmu *fake_pmu,
 	}
 	pr_debug("Parsing metric events '%s'\n", events.buf);
 	parse_events_error__init(&parse_error);
-	ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu);
+	ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu,
+			     /*warn_if_reordered=*/false);
 	if (ret) {
 		parse_events_error__print(&parse_error, events.buf);
 		goto err_out;
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index 9ec3c1dc81e0..4bb28c32b511 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -2157,11 +2157,13 @@ static int evlist__cmp(void *state, const struct list_head *l, const struct list
 	return arch_evlist__cmp(lhs, rhs);
 }
 
-static void parse_events__sort_events_and_fix_groups(struct list_head *list)
+static bool parse_events__sort_events_and_fix_groups(struct list_head *list)
 {
 	int idx = -1;
 	struct evsel *pos, *cur_leader = NULL;
 	struct perf_evsel *cur_leaders_grp = NULL;
+	bool idx_changed = false;
+	int orig_num_leaders = 0, num_leaders = 0;
 
 	/*
 	 * Compute index to insert ungrouped events at. Place them where the
@@ -2170,11 +2172,11 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
 	list_for_each_entry(pos, list, core.node) {
 		const struct evsel *pos_leader = evsel__leader(pos);
 
-		if (pos != pos_leader || pos->core.nr_members > 1)
-			continue;
+		if (pos == pos_leader)
+			orig_num_leaders++;
 
-		idx = pos->core.idx;
-		break;
+		if (idx == -1 && pos == pos_leader && pos->core.nr_members < 2)
+			idx = pos->core.idx;
 	}
 
 	/* Sort events. */
@@ -2192,6 +2194,8 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
 		bool force_grouped = arch_evsel__must_be_in_group(pos);
 
 		/* Reset index and nr_members. */
+		if (pos->core.idx != idx)
+			idx_changed = true;
 		pos->core.idx = idx++;
 		pos->core.nr_members = 0;
 
@@ -2225,12 +2229,18 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
 		}
 	}
 	list_for_each_entry(pos, list, core.node) {
-		pos->core.leader->nr_members++;
+		struct evsel *pos_leader = evsel__leader(pos);
+
+		if (pos == pos_leader)
+			num_leaders++;
+		pos_leader->core.nr_members++;
 	}
+	return idx_changed || num_leaders != orig_num_leaders;
 }
 
 int __parse_events(struct evlist *evlist, const char *str,
-		   struct parse_events_error *err, struct perf_pmu *fake_pmu)
+		   struct parse_events_error *err, struct perf_pmu *fake_pmu,
+		   bool warn_if_reordered)
 {
 	struct parse_events_state parse_state = {
 		.list	  = LIST_HEAD_INIT(parse_state.list),
@@ -2250,7 +2260,9 @@ int __parse_events(struct evlist *evlist, const char *str,
 		return -1;
 	}
 
-	parse_events__sort_events_and_fix_groups(&parse_state.list);
+	if (parse_events__sort_events_and_fix_groups(&parse_state.list) &&
+	    warn_if_reordered && !parse_state.wild_card_pmus)
+		pr_warning("WARNING: events were regrouped to match PMUs\n");
 
 	/*
 	 * Add list to the evlist even with errors to allow callers to clean up.
diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
index 767ad1729228..46204c1a7916 100644
--- a/tools/perf/util/parse-events.h
+++ b/tools/perf/util/parse-events.h
@@ -26,13 +26,13 @@ int parse_events_option(const struct option *opt, const char *str, int unset);
 int parse_events_option_new_evlist(const struct option *opt, const char *str, int unset);
 __attribute__((nonnull(1, 2, 3)))
 int __parse_events(struct evlist *evlist, const char *str, struct parse_events_error *error,
-		   struct perf_pmu *fake_pmu);
+		   struct perf_pmu *fake_pmu, bool warn_if_reordered);
 
-__attribute__((nonnull))
+__attribute__((nonnull(1, 2, 3)))
 static inline int parse_events(struct evlist *evlist, const char *str,
 			       struct parse_events_error *err)
 {
-	return __parse_events(evlist, str, err, NULL);
+	return __parse_events(evlist, str, err, /*fake_pmu=*/NULL, /*warn_if_reordered=*/true);
 }
 
 int parse_event(struct evlist *evlist, const char *str);
@@ -128,6 +128,7 @@ struct parse_events_state {
 	int			   stoken;
 	struct perf_pmu		  *fake_pmu;
 	char			  *hybrid_pmu_name;
+	bool			   wild_card_pmus;
 };
 
 void parse_events__shrink_config_terms(void);
diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
index 3a04602d2982..4488443e506e 100644
--- a/tools/perf/util/parse-events.y
+++ b/tools/perf/util/parse-events.y
@@ -323,6 +323,7 @@ event_pmu_name opt_pmu_config
 				if (!parse_events_add_pmu(_parse_state, list, pmu->name, terms,
 							  /*auto_merge_stats=*/true)) {
 					ok++;
+					parse_state->wild_card_pmus = true;
 				}
 				parse_events_terms__delete(terms);
 			}
-- 
2.40.0.rc0.216.gc4246ad0f0-goog

Re: [PATCH v4 00/11] Better fixes for grouping of events

[Liang, Kan]

On 2023-03-08 5:59 p.m., Ian Rogers wrote:
> The rules for grouping events have grown more complex. Topdown events
> must be grouped, but flags like --metric-no-group and flags on metrics
> don't respect this. Uncore events may be expanded using wild cards for
> PMU names, but then the events need reordering so the group members
> are adjacent. Rather than fixing metrics, this change fixes the main
> event parsing code to first sort and then regroup evsels.
> 
> As this is shared functionality changes to it should cause
> concern. The change is done with the intent of simplifying and making
> more robust the grouping logic, examples are given. If additional
> changes are necessary, they are most likely necessary to the
> evsel__group_pmu_name logic as the code avoids breaking groups that
> are on the same "group" PMU. The group_pmu_name is a variant of the
> pmu_name tweaked in the case of software and aux events, that use
> groups in a slightly different manner to conventional events.
> 
> The code was manually tested as well as passing perf test on a Intel
> tigerlake CPU with intel-pt.
> 
> v4. Move the Intel pmu->auxtrace initialization to the existing
>     perf_pmu__get_default_config as suggested by Adrian Hunter.
> v3. Rename pmu_name to group_pmu_name and add patch to warn when
>     events are regrouped as requested by Namhyung.
> v2. Fix up the commit message on 4/10 (thanks Arnaldo). Drop
>     unnecessary v1 5/10 (thanks Kan). evlist->core.nr_groups wasn't
>     being correctly maintained after the sort/regrouping and so the
>     new patch 10/10 removes that variable and computes it from the
>     evlist when necessary, generally just tests.
> 
> Ian Rogers (11):
>   libperf evlist: Avoid a use of evsel idx
>   perf stat: Don't remove all grouped events when CPU maps disagree
>   perf pmu: Earlier PMU auxtrace initialization
>   perf stat: Modify the group test
>   perf evsel: Allow const evsel for certain accesses
>   perf evsel: Add function to compute group PMU name
>   perf parse-events: Pass ownership of the group name
>   perf parse-events: Sort and group parsed events
>   perf evsel: Remove use_uncore_alias
>   perf evlist: Remove nr_groups
>   perf parse-events: Warn when events are regrouped
> 

Thanks Ian. The patch series looks good to me.

Reviewed-by: Kan Liang <kan.liang@linux.intel.com>

Thanks,
Kan

>  tools/lib/perf/evlist.c                  |  31 ++-
>  tools/lib/perf/include/internal/evlist.h |   1 -
>  tools/lib/perf/include/perf/evlist.h     |   1 +
>  tools/perf/arch/x86/util/auxtrace.c      |   4 -
>  tools/perf/arch/x86/util/evlist.c        |  39 ++--
>  tools/perf/arch/x86/util/pmu.c           |   8 +-
>  tools/perf/builtin-record.c              |   2 +-
>  tools/perf/builtin-report.c              |   2 +-
>  tools/perf/builtin-stat.c                |  24 +-
>  tools/perf/tests/bpf.c                   |   1 -
>  tools/perf/tests/parse-events.c          |  24 +-
>  tools/perf/tests/pfm.c                   |  12 +-
>  tools/perf/tests/pmu-events.c            |   2 +-
>  tools/perf/util/evlist.c                 |   2 +-
>  tools/perf/util/evlist.h                 |   8 +-
>  tools/perf/util/evsel.c                  |  27 ++-
>  tools/perf/util/evsel.h                  |   8 +-
>  tools/perf/util/header.c                 |   3 +-
>  tools/perf/util/metricgroup.c            |   3 +-
>  tools/perf/util/parse-events.c           | 268 +++++++++++------------
>  tools/perf/util/parse-events.h           |  14 +-
>  tools/perf/util/parse-events.y           |  28 +--
>  tools/perf/util/pfm.c                    |   1 -
>  tools/perf/util/pmu.c                    |   6 +-
>  tools/perf/util/python.c                 |   2 +-
>  tools/perf/util/stat-shadow.c            |   2 +-
>  26 files changed, 277 insertions(+), 246 deletions(-)
> 

Re: [PATCH v4 11/11] perf parse-events: Warn when events are regrouped

[Adrian Hunter]

On 9/03/23 00:59, Ian Rogers wrote:
> Use if an event is reordered or the number of groups increases to
> signal that regrouping has happened and warn about it. Disable the
> warning in the case wild card PMU names are used and for metrics.

The new warning seems to happen no matter what the order
in some cases e.g.

$ perf record -e branch-misses//u -e cycles:u  -- uname
WARNING: events were regrouped to match PMUs
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.004 MB perf.data (11 samples) ]

$ perf record -e cycles:u -e branch-misses//u -- uname
WARNING: events were regrouped to match PMUs
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.005 MB perf.data (16 samples) ]

> 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/tests/parse-events.c |  2 +-
>  tools/perf/tests/pmu-events.c   |  2 +-
>  tools/perf/util/metricgroup.c   |  3 ++-
>  tools/perf/util/parse-events.c  | 28 ++++++++++++++++++++--------
>  tools/perf/util/parse-events.h  |  7 ++++---
>  tools/perf/util/parse-events.y  |  1 +
>  6 files changed, 29 insertions(+), 14 deletions(-)
> 
> diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
> index ffa6f0a90741..b1c2f0a20306 100644
> --- a/tools/perf/tests/parse-events.c
> +++ b/tools/perf/tests/parse-events.c
> @@ -2103,7 +2103,7 @@ static int test_event_fake_pmu(const char *str)
>  
>  	parse_events_error__init(&err);
>  	perf_pmu__test_parse_init();
> -	ret = __parse_events(evlist, str, &err, &perf_pmu__fake);
> +	ret = __parse_events(evlist, str, &err, &perf_pmu__fake, /*warn_if_reordered=*/true);
>  	if (ret) {
>  		pr_debug("failed to parse event '%s', err %d, str '%s'\n",
>  			 str, ret, err.str);
> diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
> index 6ccd413b5983..7f8e86452527 100644
> --- a/tools/perf/tests/pmu-events.c
> +++ b/tools/perf/tests/pmu-events.c
> @@ -785,7 +785,7 @@ static int check_parse_id(const char *id, struct parse_events_error *error,
>  		 */
>  		perf_pmu__test_parse_init();
>  	}
> -	ret = __parse_events(evlist, dup, error, fake_pmu);
> +	ret = __parse_events(evlist, dup, error, fake_pmu, /*warn_if_reordered=*/true);
>  	free(dup);
>  
>  	evlist__delete(evlist);
> diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
> index de6dd527a2ba..5783f4c2d1ef 100644
> --- a/tools/perf/util/metricgroup.c
> +++ b/tools/perf/util/metricgroup.c
> @@ -1441,7 +1441,8 @@ static int parse_ids(bool metric_no_merge, struct perf_pmu *fake_pmu,
>  	}
>  	pr_debug("Parsing metric events '%s'\n", events.buf);
>  	parse_events_error__init(&parse_error);
> -	ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu);
> +	ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu,
> +			     /*warn_if_reordered=*/false);
>  	if (ret) {
>  		parse_events_error__print(&parse_error, events.buf);
>  		goto err_out;
> diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> index 9ec3c1dc81e0..4bb28c32b511 100644
> --- a/tools/perf/util/parse-events.c
> +++ b/tools/perf/util/parse-events.c
> @@ -2157,11 +2157,13 @@ static int evlist__cmp(void *state, const struct list_head *l, const struct list
>  	return arch_evlist__cmp(lhs, rhs);
>  }
>  
> -static void parse_events__sort_events_and_fix_groups(struct list_head *list)
> +static bool parse_events__sort_events_and_fix_groups(struct list_head *list)
>  {
>  	int idx = -1;
>  	struct evsel *pos, *cur_leader = NULL;
>  	struct perf_evsel *cur_leaders_grp = NULL;
> +	bool idx_changed = false;
> +	int orig_num_leaders = 0, num_leaders = 0;
>  
>  	/*
>  	 * Compute index to insert ungrouped events at. Place them where the
> @@ -2170,11 +2172,11 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
>  	list_for_each_entry(pos, list, core.node) {
>  		const struct evsel *pos_leader = evsel__leader(pos);
>  
> -		if (pos != pos_leader || pos->core.nr_members > 1)
> -			continue;
> +		if (pos == pos_leader)
> +			orig_num_leaders++;
>  
> -		idx = pos->core.idx;
> -		break;
> +		if (idx == -1 && pos == pos_leader && pos->core.nr_members < 2)
> +			idx = pos->core.idx;
>  	}
>  
>  	/* Sort events. */
> @@ -2192,6 +2194,8 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
>  		bool force_grouped = arch_evsel__must_be_in_group(pos);
>  
>  		/* Reset index and nr_members. */
> +		if (pos->core.idx != idx)
> +			idx_changed = true;
>  		pos->core.idx = idx++;
>  		pos->core.nr_members = 0;
>  
> @@ -2225,12 +2229,18 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
>  		}
>  	}
>  	list_for_each_entry(pos, list, core.node) {
> -		pos->core.leader->nr_members++;
> +		struct evsel *pos_leader = evsel__leader(pos);
> +
> +		if (pos == pos_leader)
> +			num_leaders++;
> +		pos_leader->core.nr_members++;
>  	}
> +	return idx_changed || num_leaders != orig_num_leaders;
>  }
>  
>  int __parse_events(struct evlist *evlist, const char *str,
> -		   struct parse_events_error *err, struct perf_pmu *fake_pmu)
> +		   struct parse_events_error *err, struct perf_pmu *fake_pmu,
> +		   bool warn_if_reordered)
>  {
>  	struct parse_events_state parse_state = {
>  		.list	  = LIST_HEAD_INIT(parse_state.list),
> @@ -2250,7 +2260,9 @@ int __parse_events(struct evlist *evlist, const char *str,
>  		return -1;
>  	}
>  
> -	parse_events__sort_events_and_fix_groups(&parse_state.list);
> +	if (parse_events__sort_events_and_fix_groups(&parse_state.list) &&
> +	    warn_if_reordered && !parse_state.wild_card_pmus)
> +		pr_warning("WARNING: events were regrouped to match PMUs\n");
>  
>  	/*
>  	 * Add list to the evlist even with errors to allow callers to clean up.
> diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> index 767ad1729228..46204c1a7916 100644
> --- a/tools/perf/util/parse-events.h
> +++ b/tools/perf/util/parse-events.h
> @@ -26,13 +26,13 @@ int parse_events_option(const struct option *opt, const char *str, int unset);
>  int parse_events_option_new_evlist(const struct option *opt, const char *str, int unset);
>  __attribute__((nonnull(1, 2, 3)))
>  int __parse_events(struct evlist *evlist, const char *str, struct parse_events_error *error,
> -		   struct perf_pmu *fake_pmu);
> +		   struct perf_pmu *fake_pmu, bool warn_if_reordered);
>  
> -__attribute__((nonnull))
> +__attribute__((nonnull(1, 2, 3)))
>  static inline int parse_events(struct evlist *evlist, const char *str,
>  			       struct parse_events_error *err)
>  {
> -	return __parse_events(evlist, str, err, NULL);
> +	return __parse_events(evlist, str, err, /*fake_pmu=*/NULL, /*warn_if_reordered=*/true);
>  }
>  
>  int parse_event(struct evlist *evlist, const char *str);
> @@ -128,6 +128,7 @@ struct parse_events_state {
>  	int			   stoken;
>  	struct perf_pmu		  *fake_pmu;
>  	char			  *hybrid_pmu_name;
> +	bool			   wild_card_pmus;
>  };
>  
>  void parse_events__shrink_config_terms(void);
> diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
> index 3a04602d2982..4488443e506e 100644
> --- a/tools/perf/util/parse-events.y
> +++ b/tools/perf/util/parse-events.y
> @@ -323,6 +323,7 @@ event_pmu_name opt_pmu_config
>  				if (!parse_events_add_pmu(_parse_state, list, pmu->name, terms,
>  							  /*auto_merge_stats=*/true)) {
>  					ok++;
> +					parse_state->wild_card_pmus = true;
>  				}
>  				parse_events_terms__delete(terms);
>  			}

Re: [PATCH v4 03/11] perf pmu: Earlier PMU auxtrace initialization

[Adrian Hunter]

On 9/03/23 00:59, Ian Rogers wrote:
> This allows event parsing to use the evsel__is_aux_event function,
> which is important when determining event grouping.
> 
> Suggested-by: Adrian Hunter <adrian.hunter@intel.com>
> Signed-off-by: Ian Rogers <irogers@google.com>

Thank you!

Acked-by: Adrian Hunter <adrian.hunter@intel.com>

> ---
>  tools/perf/arch/x86/util/auxtrace.c | 4 ----
>  tools/perf/arch/x86/util/pmu.c      | 8 ++++++--
>  2 files changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/tools/perf/arch/x86/util/auxtrace.c b/tools/perf/arch/x86/util/auxtrace.c
> index 3da506e13f49..330d03216b0e 100644
> --- a/tools/perf/arch/x86/util/auxtrace.c
> +++ b/tools/perf/arch/x86/util/auxtrace.c
> @@ -26,11 +26,7 @@ struct auxtrace_record *auxtrace_record__init_intel(struct evlist *evlist,
>  	bool found_bts = false;
>  
>  	intel_pt_pmu = perf_pmu__find(INTEL_PT_PMU_NAME);
> -	if (intel_pt_pmu)
> -		intel_pt_pmu->auxtrace = true;
>  	intel_bts_pmu = perf_pmu__find(INTEL_BTS_PMU_NAME);
> -	if (intel_bts_pmu)
> -		intel_bts_pmu->auxtrace = true;
>  
>  	evlist__for_each_entry(evlist, evsel) {
>  		if (intel_pt_pmu && evsel->core.attr.type == intel_pt_pmu->type)
> diff --git a/tools/perf/arch/x86/util/pmu.c b/tools/perf/arch/x86/util/pmu.c
> index 358340b34243..f73b80dcd8bd 100644
> --- a/tools/perf/arch/x86/util/pmu.c
> +++ b/tools/perf/arch/x86/util/pmu.c
> @@ -27,10 +27,14 @@ static bool cached_list;
>  struct perf_event_attr *perf_pmu__get_default_config(struct perf_pmu *pmu __maybe_unused)
>  {
>  #ifdef HAVE_AUXTRACE_SUPPORT
> -	if (!strcmp(pmu->name, INTEL_PT_PMU_NAME))
> +	if (!strcmp(pmu->name, INTEL_PT_PMU_NAME)) {
> +		pmu->auxtrace = true;
>  		return intel_pt_pmu_default_config(pmu);
> -	if (!strcmp(pmu->name, INTEL_BTS_PMU_NAME))
> +	}
> +	if (!strcmp(pmu->name, INTEL_BTS_PMU_NAME)) {
> +		pmu->auxtrace = true;
>  		pmu->selectable = true;
> +	}
>  #endif
>  	return NULL;
>  }

Re: [PATCH v4 11/11] perf parse-events: Warn when events are regrouped

[Ian Rogers]

On Thu, Mar 9, 2023 at 11:29 PM Adrian Hunter <adrian.hunter@intel.com> wrote:
>
> On 9/03/23 00:59, Ian Rogers wrote:
> > Use if an event is reordered or the number of groups increases to
> > signal that regrouping has happened and warn about it. Disable the
> > warning in the case wild card PMU names are used and for metrics.
>
> The new warning seems to happen no matter what the order
> in some cases e.g.
>
> $ perf record -e branch-misses//u -e cycles:u  -- uname
> WARNING: events were regrouped to match PMUs
> Linux
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.004 MB perf.data (11 samples) ]
>
> $ perf record -e cycles:u -e branch-misses//u -- uname
> WARNING: events were regrouped to match PMUs
> Linux
> [ perf record: Woken up 1 times to write data ]
> [ perf record: Captured and wrote 0.005 MB perf.data (16 samples) ]

Thanks Adrian! It is caused by lists of evsels being merged because of
the multiple -e options. There was an assumption idx values would be
sequential and when we merge them this is no longer the case. I added
a fix for v5.

Thanks,
Ian

> >
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/tests/parse-events.c |  2 +-
> >  tools/perf/tests/pmu-events.c   |  2 +-
> >  tools/perf/util/metricgroup.c   |  3 ++-
> >  tools/perf/util/parse-events.c  | 28 ++++++++++++++++++++--------
> >  tools/perf/util/parse-events.h  |  7 ++++---
> >  tools/perf/util/parse-events.y  |  1 +
> >  6 files changed, 29 insertions(+), 14 deletions(-)
> >
> > diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
> > index ffa6f0a90741..b1c2f0a20306 100644
> > --- a/tools/perf/tests/parse-events.c
> > +++ b/tools/perf/tests/parse-events.c
> > @@ -2103,7 +2103,7 @@ static int test_event_fake_pmu(const char *str)
> >
> >       parse_events_error__init(&err);
> >       perf_pmu__test_parse_init();
> > -     ret = __parse_events(evlist, str, &err, &perf_pmu__fake);
> > +     ret = __parse_events(evlist, str, &err, &perf_pmu__fake, /*warn_if_reordered=*/true);
> >       if (ret) {
> >               pr_debug("failed to parse event '%s', err %d, str '%s'\n",
> >                        str, ret, err.str);
> > diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
> > index 6ccd413b5983..7f8e86452527 100644
> > --- a/tools/perf/tests/pmu-events.c
> > +++ b/tools/perf/tests/pmu-events.c
> > @@ -785,7 +785,7 @@ static int check_parse_id(const char *id, struct parse_events_error *error,
> >                */
> >               perf_pmu__test_parse_init();
> >       }
> > -     ret = __parse_events(evlist, dup, error, fake_pmu);
> > +     ret = __parse_events(evlist, dup, error, fake_pmu, /*warn_if_reordered=*/true);
> >       free(dup);
> >
> >       evlist__delete(evlist);
> > diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
> > index de6dd527a2ba..5783f4c2d1ef 100644
> > --- a/tools/perf/util/metricgroup.c
> > +++ b/tools/perf/util/metricgroup.c
> > @@ -1441,7 +1441,8 @@ static int parse_ids(bool metric_no_merge, struct perf_pmu *fake_pmu,
> >       }
> >       pr_debug("Parsing metric events '%s'\n", events.buf);
> >       parse_events_error__init(&parse_error);
> > -     ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu);
> > +     ret = __parse_events(parsed_evlist, events.buf, &parse_error, fake_pmu,
> > +                          /*warn_if_reordered=*/false);
> >       if (ret) {
> >               parse_events_error__print(&parse_error, events.buf);
> >               goto err_out;
> > diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
> > index 9ec3c1dc81e0..4bb28c32b511 100644
> > --- a/tools/perf/util/parse-events.c
> > +++ b/tools/perf/util/parse-events.c
> > @@ -2157,11 +2157,13 @@ static int evlist__cmp(void *state, const struct list_head *l, const struct list
> >       return arch_evlist__cmp(lhs, rhs);
> >  }
> >
> > -static void parse_events__sort_events_and_fix_groups(struct list_head *list)
> > +static bool parse_events__sort_events_and_fix_groups(struct list_head *list)
> >  {
> >       int idx = -1;
> >       struct evsel *pos, *cur_leader = NULL;
> >       struct perf_evsel *cur_leaders_grp = NULL;
> > +     bool idx_changed = false;
> > +     int orig_num_leaders = 0, num_leaders = 0;
> >
> >       /*
> >        * Compute index to insert ungrouped events at. Place them where the
> > @@ -2170,11 +2172,11 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
> >       list_for_each_entry(pos, list, core.node) {
> >               const struct evsel *pos_leader = evsel__leader(pos);
> >
> > -             if (pos != pos_leader || pos->core.nr_members > 1)
> > -                     continue;
> > +             if (pos == pos_leader)
> > +                     orig_num_leaders++;
> >
> > -             idx = pos->core.idx;
> > -             break;
> > +             if (idx == -1 && pos == pos_leader && pos->core.nr_members < 2)
> > +                     idx = pos->core.idx;
> >       }
> >
> >       /* Sort events. */
> > @@ -2192,6 +2194,8 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
> >               bool force_grouped = arch_evsel__must_be_in_group(pos);
> >
> >               /* Reset index and nr_members. */
> > +             if (pos->core.idx != idx)
> > +                     idx_changed = true;
> >               pos->core.idx = idx++;
> >               pos->core.nr_members = 0;
> >
> > @@ -2225,12 +2229,18 @@ static void parse_events__sort_events_and_fix_groups(struct list_head *list)
> >               }
> >       }
> >       list_for_each_entry(pos, list, core.node) {
> > -             pos->core.leader->nr_members++;
> > +             struct evsel *pos_leader = evsel__leader(pos);
> > +
> > +             if (pos == pos_leader)
> > +                     num_leaders++;
> > +             pos_leader->core.nr_members++;
> >       }
> > +     return idx_changed || num_leaders != orig_num_leaders;
> >  }
> >
> >  int __parse_events(struct evlist *evlist, const char *str,
> > -                struct parse_events_error *err, struct perf_pmu *fake_pmu)
> > +                struct parse_events_error *err, struct perf_pmu *fake_pmu,
> > +                bool warn_if_reordered)
> >  {
> >       struct parse_events_state parse_state = {
> >               .list     = LIST_HEAD_INIT(parse_state.list),
> > @@ -2250,7 +2260,9 @@ int __parse_events(struct evlist *evlist, const char *str,
> >               return -1;
> >       }
> >
> > -     parse_events__sort_events_and_fix_groups(&parse_state.list);
> > +     if (parse_events__sort_events_and_fix_groups(&parse_state.list) &&
> > +         warn_if_reordered && !parse_state.wild_card_pmus)
> > +             pr_warning("WARNING: events were regrouped to match PMUs\n");
> >
> >       /*
> >        * Add list to the evlist even with errors to allow callers to clean up.
> > diff --git a/tools/perf/util/parse-events.h b/tools/perf/util/parse-events.h
> > index 767ad1729228..46204c1a7916 100644
> > --- a/tools/perf/util/parse-events.h
> > +++ b/tools/perf/util/parse-events.h
> > @@ -26,13 +26,13 @@ int parse_events_option(const struct option *opt, const char *str, int unset);
> >  int parse_events_option_new_evlist(const struct option *opt, const char *str, int unset);
> >  __attribute__((nonnull(1, 2, 3)))
> >  int __parse_events(struct evlist *evlist, const char *str, struct parse_events_error *error,
> > -                struct perf_pmu *fake_pmu);
> > +                struct perf_pmu *fake_pmu, bool warn_if_reordered);
> >
> > -__attribute__((nonnull))
> > +__attribute__((nonnull(1, 2, 3)))
> >  static inline int parse_events(struct evlist *evlist, const char *str,
> >                              struct parse_events_error *err)
> >  {
> > -     return __parse_events(evlist, str, err, NULL);
> > +     return __parse_events(evlist, str, err, /*fake_pmu=*/NULL, /*warn_if_reordered=*/true);
> >  }
> >
> >  int parse_event(struct evlist *evlist, const char *str);
> > @@ -128,6 +128,7 @@ struct parse_events_state {
> >       int                        stoken;
> >       struct perf_pmu           *fake_pmu;
> >       char                      *hybrid_pmu_name;
> > +     bool                       wild_card_pmus;
> >  };
> >
> >  void parse_events__shrink_config_terms(void);
> > diff --git a/tools/perf/util/parse-events.y b/tools/perf/util/parse-events.y
> > index 3a04602d2982..4488443e506e 100644
> > --- a/tools/perf/util/parse-events.y
> > +++ b/tools/perf/util/parse-events.y
> > @@ -323,6 +323,7 @@ event_pmu_name opt_pmu_config
> >                               if (!parse_events_add_pmu(_parse_state, list, pmu->name, terms,
> >                                                         /*auto_merge_stats=*/true)) {
> >                                       ok++;
> > +                                     parse_state->wild_card_pmus = true;
> >                               }
> >                               parse_events_terms__delete(terms);
> >                       }
>

Re: [PATCH v4 06/11] perf evsel: Add function to compute group PMU name

[Arnaldo Carvalho de Melo]

Em Wed, Mar 08, 2023 at 02:59:07PM -0800, Ian Rogers escreveu:
> The computed name respects software events and aux event groups, such
> that the pmu_name is changed to be that of the aux event leader or
> group leader for software events. This is done as a later change will
> split events that are in different PMUs into different groups.

This makes 'perf test python' to fail:

⬢[acme@toolbox perf-tools-next]$ perf test -v python
Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
 19: 'import perf' in python                                         :
--- start ---
test child forked, pid 720242
python usage test: "echo "import sys ; sys.path.append('/tmp/build/perf-tools-next/python'); import perf" | '/usr/bin/python3' "
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /tmp/build/perf-tools-next/python/perf.cpython-311-x86_64-linux-gnu.so: undefined symbol: evsel__is_aux_event
test child finished with -1
---- end ----
'import perf' in python: FAILED!
⬢[acme@toolbox perf-tools-next]$

So I added the following patch, please run 'perf test' and 'make -C
tools/perf build-test' before submitting patch series.

- Arnaldo

diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
index ab48ffbb644805df..be336f1b2b689602 100644
--- a/tools/perf/util/python.c
+++ b/tools/perf/util/python.c
@@ -93,6 +93,11 @@ int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,
 	return EOF;
 }
 
+bool evsel__is_aux_event(const struct evsel *evsel __maybe_unused)
+{
+	return false;
+}
+
 /*
  * Add this one here not to drag util/metricgroup.c
  */
 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/evsel.c | 24 ++++++++++++++++++++++++
>  tools/perf/util/evsel.h |  1 +
>  2 files changed, 25 insertions(+)
> 
> diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> index 2dc2c24252bb..51d9650267d0 100644
> --- a/tools/perf/util/evsel.c
> +++ b/tools/perf/util/evsel.c
> @@ -821,6 +821,30 @@ const char *evsel__name(struct evsel *evsel)
>  	return "unknown";
>  }
>  
> +const char *evsel__group_pmu_name(const struct evsel *evsel)
> +{
> +	const struct evsel *leader;
> +
> +	/* If the pmu_name is set use it. pmu_name isn't set for CPU and software events. */
> +	if (evsel->pmu_name)
> +		return evsel->pmu_name;
> +	/*
> +	 * Software events may be in a group with other uncore PMU events. Use
> +	 * the pmu_name of the group leader to avoid breaking the software event
> +	 * out of the group.
> +	 *
> +	 * Aux event leaders, like intel_pt, expect a group with events from
> +	 * other PMUs, so substitute the AUX event's PMU in this case.
> +	 */
> +	leader  = evsel__leader(evsel);
> +	if ((evsel->core.attr.type == PERF_TYPE_SOFTWARE || evsel__is_aux_event(leader)) &&
> +	    leader->pmu_name) {
> +		return leader->pmu_name;
> +	}
> +
> +	return "cpu";
> +}
> +
>  const char *evsel__metric_id(const struct evsel *evsel)
>  {
>  	if (evsel->metric_id)
> diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> index 676c499323e9..d26745ca6147 100644
> --- a/tools/perf/util/evsel.h
> +++ b/tools/perf/util/evsel.h
> @@ -280,6 +280,7 @@ int arch_evsel__hw_name(struct evsel *evsel, char *bf, size_t size);
>  
>  int __evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result, char *bf, size_t size);
>  const char *evsel__name(struct evsel *evsel);
> +const char *evsel__group_pmu_name(const struct evsel *evsel);
>  const char *evsel__metric_id(const struct evsel *evsel);
>  
>  static inline bool evsel__is_tool(const struct evsel *evsel)
> -- 
> 2.40.0.rc0.216.gc4246ad0f0-goog
> 

-- 

- Arnaldo

Re: [PATCH v4 06/11] perf evsel: Add function to compute group PMU name

[Ian Rogers]

On Mon, Mar 13, 2023 at 1:41 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Wed, Mar 08, 2023 at 02:59:07PM -0800, Ian Rogers escreveu:
> > The computed name respects software events and aux event groups, such
> > that the pmu_name is changed to be that of the aux event leader or
> > group leader for software events. This is done as a later change will
> > split events that are in different PMUs into different groups.
>
> This makes 'perf test python' to fail:
>
> ⬢[acme@toolbox perf-tools-next]$ perf test -v python
> Couldn't bump rlimit(MEMLOCK), failures may take place when creating BPF maps, etc
>  19: 'import perf' in python                                         :
> --- start ---
> test child forked, pid 720242
> python usage test: "echo "import sys ; sys.path.append('/tmp/build/perf-tools-next/python'); import perf" | '/usr/bin/python3' "
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> ImportError: /tmp/build/perf-tools-next/python/perf.cpython-311-x86_64-linux-gnu.so: undefined symbol: evsel__is_aux_event
> test child finished with -1
> ---- end ----
> 'import perf' in python: FAILED!
> ⬢[acme@toolbox perf-tools-next]$
>
> So I added the following patch, please run 'perf test' and 'make -C
> tools/perf build-test' before submitting patch series.
>
> - Arnaldo

Hmm.. was passing for me:

```
 19: 'import perf' in python                                         :
--- start ---
test child forked, pid 3161744
python usage test: "echo "import sys ;
sys.path.append('/tmp/perf/python'); import perf" | '/usr/bin/python3'
"
test child finished with 0
---- end ----
'import perf' in python: Ok
```

albeit:
```
$ nm /tmp/perf/python/perf.cpython-310-x86_64-linux-gnu.so |grep is_aux_event
                U evsel__is_aux_event
```

Perhaps there's some more aggressive linking flag I need to enable.

Thanks,
Ian

> diff --git a/tools/perf/util/python.c b/tools/perf/util/python.c
> index ab48ffbb644805df..be336f1b2b689602 100644
> --- a/tools/perf/util/python.c
> +++ b/tools/perf/util/python.c
> @@ -93,6 +93,11 @@ int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,
>         return EOF;
>  }
>
> +bool evsel__is_aux_event(const struct evsel *evsel __maybe_unused)
> +{
> +       return false;
> +}
> +
>  /*
>   * Add this one here not to drag util/metricgroup.c
>   */
>
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/util/evsel.c | 24 ++++++++++++++++++++++++
> >  tools/perf/util/evsel.h |  1 +
> >  2 files changed, 25 insertions(+)
> >
> > diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
> > index 2dc2c24252bb..51d9650267d0 100644
> > --- a/tools/perf/util/evsel.c
> > +++ b/tools/perf/util/evsel.c
> > @@ -821,6 +821,30 @@ const char *evsel__name(struct evsel *evsel)
> >       return "unknown";
> >  }
> >
> > +const char *evsel__group_pmu_name(const struct evsel *evsel)
> > +{
> > +     const struct evsel *leader;
> > +
> > +     /* If the pmu_name is set use it. pmu_name isn't set for CPU and software events. */
> > +     if (evsel->pmu_name)
> > +             return evsel->pmu_name;
> > +     /*
> > +      * Software events may be in a group with other uncore PMU events. Use
> > +      * the pmu_name of the group leader to avoid breaking the software event
> > +      * out of the group.
> > +      *
> > +      * Aux event leaders, like intel_pt, expect a group with events from
> > +      * other PMUs, so substitute the AUX event's PMU in this case.
> > +      */
> > +     leader  = evsel__leader(evsel);
> > +     if ((evsel->core.attr.type == PERF_TYPE_SOFTWARE || evsel__is_aux_event(leader)) &&
> > +         leader->pmu_name) {
> > +             return leader->pmu_name;
> > +     }
> > +
> > +     return "cpu";
> > +}
> > +
> >  const char *evsel__metric_id(const struct evsel *evsel)
> >  {
> >       if (evsel->metric_id)
> > diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
> > index 676c499323e9..d26745ca6147 100644
> > --- a/tools/perf/util/evsel.h
> > +++ b/tools/perf/util/evsel.h
> > @@ -280,6 +280,7 @@ int arch_evsel__hw_name(struct evsel *evsel, char *bf, size_t size);
> >
> >  int __evsel__hw_cache_type_op_res_name(u8 type, u8 op, u8 result, char *bf, size_t size);
> >  const char *evsel__name(struct evsel *evsel);
> > +const char *evsel__group_pmu_name(const struct evsel *evsel);
> >  const char *evsel__metric_id(const struct evsel *evsel);
> >
> >  static inline bool evsel__is_tool(const struct evsel *evsel)
> > --
> > 2.40.0.rc0.216.gc4246ad0f0-goog
> >
>
> --
>
> - Arnaldo