linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v2 0/7] Rewrite jevents program in python
@ 2022-05-11 21:15 Ian Rogers
  2022-05-11 21:15 ` [PATCH v2 1/7] perf jevents: Append PMU description later Ian Rogers
                   ` (6 more replies)
  0 siblings, 7 replies; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

New architectures bring new complexity, such as Intel's hybrid
models. jevents provides an alternative to specifying events in the
kernel and exposing them through sysfs, however, it is difficult to
work with. For example, an error in the json input would yield an
error message but no json file or location. It is also a challenge to
update jsmn.c given its forked nature.

The changes here switch from jevents.c to a rewrite in python called
jevents.py. This means there is a build time dependency on python, but
such a dependency already exists for asciidoc (used to generate perf's
man pages). If the build detects that python isn't present or is older
than version 3.5 (released Sept. 2015) then an empty file is
substituted for the generated one.

A challenge with this code is in avoiding regressions. For this reason
the jevents.py produces identical output to jevents.c, validated with a
test script and build target.

A difference in the python to the C approach is that the python loads
an entire json file in to memory, while the C code works from token to
token. In some cases the C approach was sensitive to the order of
dictionary items in the json file. To ensure matching output there are
two changes made to jevents.c to cause it to read all values before
creating output.

The changes also found a bug in Ivytown's UNC_M_ACT_COUNT.RD event
encoding, as well as unnecessary whitespace introduced in Alderlake's
metrics. In these cases the json input is fixed.

v2. Fixes the build for architectures that don't have pmu-events json
    (Suggested-by: John Garry <john.garry@huawei.com>) and fixes the
    build for python not being present or too old (Suggested-by: Peter
    Zijlstra <peterz@infradead.org>/John Garry <john.garry@huawei.com>).

Ian Rogers (7):
  perf jevents: Append PMU description later
  perf vendor events: Fix Alderlake metric groups
  perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask
  perf jevents: Modify match field
  perf jevents: Add python converter script
  perf jevents: Switch build to use jevents.py
  perf jevents: Remove jevents.c

 tools/perf/Makefile.config                    |   19 +
 tools/perf/Makefile.perf                      |   16 +-
 tools/perf/pmu-events/Build                   |   15 +-
 .../arch/x86/alderlake/adl-metrics.json       |   32 -
 .../arch/x86/ivytown/uncore-memory.json       |    3 +-
 tools/perf/pmu-events/empty-pmu-events.c      |   21 +
 tools/perf/pmu-events/jevents.c               | 1322 -----------------
 tools/perf/pmu-events/jevents.py              |  392 +++++
 tools/perf/pmu-events/jsmn.h                  |   68 -
 tools/perf/pmu-events/json.c                  |  162 --
 tools/perf/pmu-events/json.h                  |   39 -
 tools/perf/tests/pmu-events.c                 |   30 +-
 12 files changed, 460 insertions(+), 1659 deletions(-)
 create mode 100644 tools/perf/pmu-events/empty-pmu-events.c
 delete mode 100644 tools/perf/pmu-events/jevents.c
 create mode 100755 tools/perf/pmu-events/jevents.py
 delete mode 100644 tools/perf/pmu-events/jsmn.h
 delete mode 100644 tools/perf/pmu-events/json.c
 delete mode 100644 tools/perf/pmu-events/json.h

-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply	[flat|nested] 19+ messages in thread

* [PATCH v2 1/7] perf jevents: Append PMU description later
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-17 13:28   ` John Garry
  2022-05-11 21:15 ` [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups Ian Rogers
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

Append the PMU information from "Unit" to the description later. This
avoids a problem when "Unit" appears early in a json event and the
information prepends the description rather than being the expected
suffix.

Update the pmu-events test so that expectations now match the improved
output.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/pmu-events/jevents.c | 8 +++++---
 tools/perf/tests/pmu-events.c   | 6 +++---
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index 159d9eab6e79..e1f7c7afd435 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -652,9 +652,6 @@ static int json_events(const char *fn,
 					for (s = je.pmu; *s; s++)
 						*s = tolower(*s);
 				}
-				addfield(map, &je.desc, ". ", "Unit: ", NULL);
-				addfield(map, &je.desc, "", je.pmu, NULL);
-				addfield(map, &je.desc, "", " ", NULL);
 			} else if (json_streq(map, field, "Filter")) {
 				addfield(map, &filter, "", "", val);
 			} else if (json_streq(map, field, "ScaleUnit")) {
@@ -697,6 +694,11 @@ static int json_events(const char *fn,
 			addfield(map, &je.desc, " ", extra_desc, NULL);
 		if (je.long_desc && extra_desc)
 			addfield(map, &je.long_desc, " ", extra_desc, NULL);
+		if (je.pmu) {
+			addfield(map, &je.desc, ". ", "Unit: ", NULL);
+			addfield(map, &je.desc, "", je.pmu, NULL);
+			addfield(map, &je.desc, "", " ", NULL);
+		}
 		if (filter)
 			addfield(map, &event, ",", filter, NULL);
 		if (msr != NULL)
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index 299a215eb54c..b74c6ef59e51 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -133,7 +133,7 @@ static const struct perf_pmu_test_event unc_cbo_xsnp_response_miss_eviction = {
 	.event = {
 		.name = "unc_cbo_xsnp_response.miss_eviction",
 		.event = "umask=0x81,event=0x22",
-		.desc = "Unit: uncore_cbox A cross-core snoop resulted from L3 Eviction which misses in some processor core",
+		.desc = "A cross-core snoop resulted from L3 Eviction which misses in some processor core. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "A cross-core snoop resulted from L3 Eviction which misses in some processor core",
 		.pmu = "uncore_cbox",
@@ -147,7 +147,7 @@ static const struct perf_pmu_test_event uncore_hyphen = {
 	.event = {
 		.name = "event-hyphen",
 		.event = "umask=0x00,event=0xe0",
-		.desc = "Unit: uncore_cbox UNC_CBO_HYPHEN",
+		.desc = "UNC_CBO_HYPHEN. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "UNC_CBO_HYPHEN",
 		.pmu = "uncore_cbox",
@@ -161,7 +161,7 @@ static const struct perf_pmu_test_event uncore_two_hyph = {
 	.event = {
 		.name = "event-two-hyph",
 		.event = "umask=0x00,event=0xc0",
-		.desc = "Unit: uncore_cbox UNC_CBO_TWO_HYPH",
+		.desc = "UNC_CBO_TWO_HYPH. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "UNC_CBO_TWO_HYPH",
 		.pmu = "uncore_cbox",
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
  2022-05-11 21:15 ` [PATCH v2 1/7] perf jevents: Append PMU description later Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-17 13:30   ` John Garry
  2022-05-11 21:15 ` [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask Ian Rogers
                   ` (4 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

Remove unnecessary empty groups.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 .../arch/x86/alderlake/adl-metrics.json       | 32 -------------------
 1 file changed, 32 deletions(-)

diff --git a/tools/perf/pmu-events/arch/x86/alderlake/adl-metrics.json b/tools/perf/pmu-events/arch/x86/alderlake/adl-metrics.json
index 4d172687f936..6b24958737b5 100644
--- a/tools/perf/pmu-events/arch/x86/alderlake/adl-metrics.json
+++ b/tools/perf/pmu-events/arch/x86/alderlake/adl-metrics.json
@@ -477,224 +477,192 @@
     {
         "BriefDescription": "",
         "MetricExpr": "CPU_CLK_UNHALTED.CORE",
-        "MetricGroup": " ",
         "MetricName": "CLKS",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "",
         "MetricExpr": "CPU_CLK_UNHALTED.CORE_P",
-        "MetricGroup": " ",
         "MetricName": "CLKS_P",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "",
         "MetricExpr": "5 * CPU_CLK_UNHALTED.CORE",
-        "MetricGroup": " ",
         "MetricName": "SLOTS",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instructions Per Cycle",
         "MetricExpr": "INST_RETIRED.ANY / CPU_CLK_UNHALTED.CORE",
-        "MetricGroup": " ",
         "MetricName": "IPC",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Cycles Per Instruction",
         "MetricExpr": "CPU_CLK_UNHALTED.CORE / INST_RETIRED.ANY",
-        "MetricGroup": " ",
         "MetricName": "CPI",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Uops Per Instruction",
         "MetricExpr": "UOPS_RETIRED.ALL / INST_RETIRED.ANY",
-        "MetricGroup": " ",
         "MetricName": "UPI",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of total non-speculative loads with a store forward or unknown store address block",
         "MetricExpr": "100 * LD_BLOCKS.DATA_UNKNOWN / MEM_UOPS_RETIRED.ALL_LOADS",
-        "MetricGroup": "",
         "MetricName": "Store_Fwd_Blocks",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of total non-speculative loads with a address aliasing block",
         "MetricExpr": "100 * LD_BLOCKS.4K_ALIAS / MEM_UOPS_RETIRED.ALL_LOADS",
-        "MetricGroup": "",
         "MetricName": "Address_Alias_Blocks",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of total non-speculative loads that are splits",
         "MetricExpr": "100 * MEM_UOPS_RETIRED.SPLIT_LOADS / MEM_UOPS_RETIRED.ALL_LOADS",
-        "MetricGroup": "",
         "MetricName": "Load_Splits",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instructions per Branch (lower number means higher occurrence rate)",
         "MetricExpr": "INST_RETIRED.ANY / BR_INST_RETIRED.ALL_BRANCHES",
-        "MetricGroup": " ",
         "MetricName": "IpBranch",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instruction per (near) call (lower number means higher occurrence rate)",
         "MetricExpr": "INST_RETIRED.ANY / BR_INST_RETIRED.CALL",
-        "MetricGroup": " ",
         "MetricName": "IpCall",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instructions per Load",
         "MetricExpr": "INST_RETIRED.ANY / MEM_UOPS_RETIRED.ALL_LOADS",
-        "MetricGroup": " ",
         "MetricName": "IpLoad",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instructions per Store",
         "MetricExpr": "INST_RETIRED.ANY / MEM_UOPS_RETIRED.ALL_STORES",
-        "MetricGroup": " ",
         "MetricName": "IpStore",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Number of Instructions per non-speculative Branch Misprediction",
         "MetricExpr": "INST_RETIRED.ANY / BR_MISP_RETIRED.ALL_BRANCHES",
-        "MetricGroup": " ",
         "MetricName": "IpMispredict",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Instructions per Far Branch",
         "MetricExpr": "INST_RETIRED.ANY / ( BR_INST_RETIRED.FAR_BRANCH / 2 )",
-        "MetricGroup": " ",
         "MetricName": "IpFarBranch",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Ratio of all branches which mispredict",
         "MetricExpr": "BR_MISP_RETIRED.ALL_BRANCHES / BR_INST_RETIRED.ALL_BRANCHES",
-        "MetricGroup": " ",
         "MetricName": "Branch_Mispredict_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Ratio between Mispredicted branches and unknown branches",
         "MetricExpr": "BR_MISP_RETIRED.ALL_BRANCHES / BACLEARS.ANY",
-        "MetricGroup": " ",
         "MetricName": "Branch_Mispredict_to_Unknown_Branch_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of all uops which are ucode ops",
         "MetricExpr": "100 * UOPS_RETIRED.MS / UOPS_RETIRED.ALL",
-        "MetricGroup": " ",
         "MetricName": "Microcode_Uop_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of all uops which are FPDiv uops",
         "MetricExpr": "100 * UOPS_RETIRED.FPDIV / UOPS_RETIRED.ALL",
-        "MetricGroup": " ",
         "MetricName": "FPDiv_Uop_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of all uops which are IDiv uops",
         "MetricExpr": "100 * UOPS_RETIRED.IDIV / UOPS_RETIRED.ALL",
-        "MetricGroup": " ",
         "MetricName": "IDiv_Uop_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percentage of all uops which are x87 uops",
         "MetricExpr": "100 * UOPS_RETIRED.X87 / UOPS_RETIRED.ALL",
-        "MetricGroup": " ",
         "MetricName": "X87_Uop_Ratio",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Average Frequency Utilization relative nominal frequency",
         "MetricExpr": "CPU_CLK_UNHALTED.CORE / CPU_CLK_UNHALTED.REF_TSC",
-        "MetricGroup": " ",
         "MetricName": "Turbo_Utilization",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Fraction of cycles spent in Kernel mode",
         "MetricExpr": "CPU_CLK_UNHALTED.CORE:k / CPU_CLK_UNHALTED.CORE",
-        "MetricGroup": " ",
         "MetricName": "Kernel_Utilization",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Average CPU Utilization",
         "MetricExpr": "CPU_CLK_UNHALTED.REF_TSC / msr@tsc@",
-        "MetricGroup": " ",
         "MetricName": "CPU_Utilization",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Estimated Pause cost. In percent",
         "MetricExpr": "100 * SERIALIZATION.NON_C01_MS_SCB / ( 5 * CPU_CLK_UNHALTED.CORE )",
-        "MetricGroup": " ",
         "MetricName": "Estimated_Pause_Cost",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Cycle cost per L2 hit",
         "MetricExpr": "MEM_BOUND_STALLS.LOAD_L2_HIT / MEM_LOAD_UOPS_RETIRED.L2_HIT",
-        "MetricGroup": " ",
         "MetricName": "Cycles_per_Demand_Load_L2_Hit",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Cycle cost per LLC hit",
         "MetricExpr": "MEM_BOUND_STALLS.LOAD_LLC_HIT / MEM_LOAD_UOPS_RETIRED.L3_HIT",
-        "MetricGroup": " ",
         "MetricName": "Cycles_per_Demand_Load_L3_Hit",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Cycle cost per DRAM hit",
         "MetricExpr": "MEM_BOUND_STALLS.LOAD_DRAM_HIT / MEM_LOAD_UOPS_RETIRED.DRAM_HIT",
-        "MetricGroup": " ",
         "MetricName": "Cycles_per_Demand_Load_DRAM_Hit",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percent of instruction miss cost that hit in the L2",
         "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_L2_HIT / ( MEM_BOUND_STALLS.IFETCH )",
-        "MetricGroup": " ",
         "MetricName": "Inst_Miss_Cost_L2Hit_Percent",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percent of instruction miss cost that hit in the L3",
         "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_LLC_HIT / ( MEM_BOUND_STALLS.IFETCH )",
-        "MetricGroup": " ",
         "MetricName": "Inst_Miss_Cost_L3Hit_Percent",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "Percent of instruction miss cost that hit in DRAM",
         "MetricExpr": "100 * MEM_BOUND_STALLS.IFETCH_DRAM_HIT / ( MEM_BOUND_STALLS.IFETCH )",
-        "MetricGroup": " ",
         "MetricName": "Inst_Miss_Cost_DRAMHit_Percent",
         "Unit": "cpu_atom"
     },
     {
         "BriefDescription": "load ops retired per 1000 instruction",
         "MetricExpr": "1000 * MEM_UOPS_RETIRED.ALL_LOADS / INST_RETIRED.ANY",
-        "MetricGroup": " ",
         "MetricName": "MemLoadPKI",
         "Unit": "cpu_atom"
     },
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
  2022-05-11 21:15 ` [PATCH v2 1/7] perf jevents: Append PMU description later Ian Rogers
  2022-05-11 21:15 ` [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-17 13:31   ` John Garry
  2022-05-11 21:15 ` [PATCH v2 4/7] perf jevents: Modify match field Ian Rogers
                   ` (3 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

The event had two umasks with the umask of 3 being correct.
Note: this change wasn't automatically generated as there is no CSV for
Ivytown uncore events at:
https://github.com/intel/event-converter-for-linux-perf

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/pmu-events/arch/x86/ivytown/uncore-memory.json | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/tools/perf/pmu-events/arch/x86/ivytown/uncore-memory.json b/tools/perf/pmu-events/arch/x86/ivytown/uncore-memory.json
index df4b43294fa0..e8917cb59566 100644
--- a/tools/perf/pmu-events/arch/x86/ivytown/uncore-memory.json
+++ b/tools/perf/pmu-events/arch/x86/ivytown/uncore-memory.json
@@ -5,8 +5,7 @@
         "EventCode": "0x1",
         "EventName": "UNC_M_ACT_COUNT.RD",
         "PerPkg": "1",
-        "UMask": "0x1",
-        "Umask": "0x3",
+        "UMask": "0x3",
         "Unit": "iMC"
     },
     {
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 4/7] perf jevents: Modify match field
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
                   ` (2 preceding siblings ...)
  2022-05-11 21:15 ` [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-17 13:35   ` John Garry
  2022-05-11 21:15 ` [PATCH v2 5/7] perf jevents: Add python converter script Ian Rogers
                   ` (2 subsequent siblings)
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

The match_field function looks for json values to append to the event
string. As the C code processes these in order the output order matches
that in the json dictionary. Python json readers read the entire
dictionary and lose the ordering. To make the python and C output
comparable make the C code first read the extra fields then append them
to the event in an order not determined by their order in the file.

Modify the pmu-events test so that test expectations match the new
order.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/pmu-events/jevents.c | 82 ++++++++++++++++++++-------------
 tools/perf/tests/pmu-events.c   | 24 +++++-----
 2 files changed, 62 insertions(+), 44 deletions(-)

diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
index e1f7c7afd435..cee61c4ed59e 100644
--- a/tools/perf/pmu-events/jevents.c
+++ b/tools/perf/pmu-events/jevents.c
@@ -207,21 +207,6 @@ static struct msrmap {
 	{ NULL, NULL }
 };
 
-static struct field {
-	const char *field;
-	const char *kernel;
-} fields[] = {
-	{ "UMask",	"umask=" },
-	{ "CounterMask", "cmask=" },
-	{ "Invert",	"inv=" },
-	{ "AnyThread",	"any=" },
-	{ "EdgeDetect",	"edge=" },
-	{ "SampleAfterValue", "period=" },
-	{ "FCMask",	"fc_mask=" },
-	{ "PortMask",	"ch_mask=" },
-	{ NULL, NULL }
-};
-
 static void cut_comma(char *map, jsmntok_t *newval)
 {
 	int i;
@@ -233,21 +218,6 @@ static void cut_comma(char *map, jsmntok_t *newval)
 	}
 }
 
-static int match_field(char *map, jsmntok_t *field, int nz,
-		       char **event, jsmntok_t *val)
-{
-	struct field *f;
-	jsmntok_t newval = *val;
-
-	for (f = fields; f->field; f++)
-		if (json_streq(map, field, f->field) && nz) {
-			cut_comma(map, &newval);
-			addfield(map, event, ",", f->kernel, &newval);
-			return 1;
-		}
-	return 0;
-}
-
 static struct msrmap *lookup_msr(char *map, jsmntok_t *val)
 {
 	jsmntok_t newval = *val;
@@ -581,6 +551,14 @@ static int json_events(const char *fn,
 		jsmntok_t *precise = NULL;
 		jsmntok_t *obj = tok++;
 		bool configcode_present = false;
+		char *umask = NULL;
+		char *cmask = NULL;
+		char *inv = NULL;
+		char *any = NULL;
+		char *edge = NULL;
+		char *period = NULL;
+		char *fc_mask = NULL;
+		char *ch_mask = NULL;
 
 		EXPECT(obj->type == JSMN_OBJECT, obj, "expected object");
 		for (j = 0; j < obj->size; j += 2) {
@@ -596,8 +574,23 @@ static int json_events(const char *fn,
 			       "Expected string value");
 
 			nz = !json_streq(map, val, "0");
-			if (match_field(map, field, nz, &event, val)) {
-				/* ok */
+			/* match_field */
+			if (json_streq(map, field, "UMask") && nz) {
+				addfield(map, &umask, "", "umask=", val);
+			} else if (json_streq(map, field, "CounterMask") && nz) {
+				addfield(map, &cmask, "", "cmask=", val);
+			} else if (json_streq(map, field, "Invert") && nz) {
+				addfield(map, &inv, "", "inv=", val);
+			} else if (json_streq(map, field, "AnyThread") && nz) {
+				addfield(map, &any, "", "any=", val);
+			} else if (json_streq(map, field, "EdgeDetect") && nz) {
+				addfield(map, &edge, "", "edge=", val);
+			} else if (json_streq(map, field, "SampleAfterValue") && nz) {
+				addfield(map, &period, "", "period=", val);
+			} else if (json_streq(map, field, "FCMask") && nz) {
+				addfield(map, &fc_mask, "", "fc_mask=", val);
+			} else if (json_streq(map, field, "PortMask") && nz) {
+				addfield(map, &ch_mask, "", "ch_mask=", val);
 			} else if (json_streq(map, field, "EventCode")) {
 				char *code = NULL;
 				addfield(map, &code, "", "", val);
@@ -690,6 +683,23 @@ static int json_events(const char *fn,
 		else
 			snprintf(buf, sizeof buf, "event=%#llx", eventcode);
 		addfield(map, &event, ",", buf, NULL);
+		if (any)
+			addfield(map, &event, ",", any, NULL);
+		if (ch_mask)
+			addfield(map, &event, ",", ch_mask, NULL);
+		if (cmask)
+			addfield(map, &event, ",", cmask, NULL);
+		if (edge)
+			addfield(map, &event, ",", edge, NULL);
+		if (fc_mask)
+			addfield(map, &event, ",", fc_mask, NULL);
+		if (inv)
+			addfield(map, &event, ",", inv, NULL);
+		if (period)
+			addfield(map, &event, ",", period, NULL);
+		if (umask)
+			addfield(map, &event, ",", umask, NULL);
+
 		if (je.desc && extra_desc)
 			addfield(map, &je.desc, " ", extra_desc, NULL);
 		if (je.long_desc && extra_desc)
@@ -718,6 +728,14 @@ static int json_events(const char *fn,
 		je.event = real_event(je.name, event);
 		err = func(data, &je);
 free_strings:
+		free(umask);
+		free(cmask);
+		free(inv);
+		free(any);
+		free(edge);
+		free(period);
+		free(fc_mask);
+		free(ch_mask);
 		free(event);
 		free(je.desc);
 		free(je.name);
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index b74c6ef59e51..f13368569d8b 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -63,33 +63,33 @@ static const struct perf_pmu_test_event bp_l2_btb_correct = {
 static const struct perf_pmu_test_event segment_reg_loads_any = {
 	.event = {
 		.name = "segment_reg_loads.any",
-		.event = "umask=0x80,period=200000,event=0x6",
+		.event = "event=0x6,period=200000,umask=0x80",
 		.desc = "Number of segment register loads",
 		.topic = "other",
 	},
-	.alias_str = "umask=0x80,period=0x30d40,event=0x6",
+	.alias_str = "event=0x6,period=0x30d40,umask=0x80",
 	.alias_long_desc = "Number of segment register loads",
 };
 
 static const struct perf_pmu_test_event dispatch_blocked_any = {
 	.event = {
 		.name = "dispatch_blocked.any",
-		.event = "umask=0x20,period=200000,event=0x9",
+		.event = "event=0x9,period=200000,umask=0x20",
 		.desc = "Memory cluster signals to block micro-op dispatch for any reason",
 		.topic = "other",
 	},
-	.alias_str = "umask=0x20,period=0x30d40,event=0x9",
+	.alias_str = "event=0x9,period=0x30d40,umask=0x20",
 	.alias_long_desc = "Memory cluster signals to block micro-op dispatch for any reason",
 };
 
 static const struct perf_pmu_test_event eist_trans = {
 	.event = {
 		.name = "eist_trans",
-		.event = "umask=0x0,period=200000,event=0x3a",
+		.event = "event=0x3a,period=200000,umask=0x0",
 		.desc = "Number of Enhanced Intel SpeedStep(R) Technology (EIST) transitions",
 		.topic = "other",
 	},
-	.alias_str = "umask=0,period=0x30d40,event=0x3a",
+	.alias_str = "event=0x3a,period=0x30d40,umask=0",
 	.alias_long_desc = "Number of Enhanced Intel SpeedStep(R) Technology (EIST) transitions",
 };
 
@@ -132,13 +132,13 @@ static const struct perf_pmu_test_event uncore_hisi_ddrc_flux_wcmd = {
 static const struct perf_pmu_test_event unc_cbo_xsnp_response_miss_eviction = {
 	.event = {
 		.name = "unc_cbo_xsnp_response.miss_eviction",
-		.event = "umask=0x81,event=0x22",
+		.event = "event=0x22,umask=0x81",
 		.desc = "A cross-core snoop resulted from L3 Eviction which misses in some processor core. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "A cross-core snoop resulted from L3 Eviction which misses in some processor core",
 		.pmu = "uncore_cbox",
 	},
-	.alias_str = "umask=0x81,event=0x22",
+	.alias_str = "event=0x22,umask=0x81",
 	.alias_long_desc = "A cross-core snoop resulted from L3 Eviction which misses in some processor core",
 	.matching_pmu = "uncore_cbox_0",
 };
@@ -146,13 +146,13 @@ static const struct perf_pmu_test_event unc_cbo_xsnp_response_miss_eviction = {
 static const struct perf_pmu_test_event uncore_hyphen = {
 	.event = {
 		.name = "event-hyphen",
-		.event = "umask=0x00,event=0xe0",
+		.event = "event=0xe0,umask=0x00",
 		.desc = "UNC_CBO_HYPHEN. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "UNC_CBO_HYPHEN",
 		.pmu = "uncore_cbox",
 	},
-	.alias_str = "umask=0,event=0xe0",
+	.alias_str = "event=0xe0,umask=0",
 	.alias_long_desc = "UNC_CBO_HYPHEN",
 	.matching_pmu = "uncore_cbox_0",
 };
@@ -160,13 +160,13 @@ static const struct perf_pmu_test_event uncore_hyphen = {
 static const struct perf_pmu_test_event uncore_two_hyph = {
 	.event = {
 		.name = "event-two-hyph",
-		.event = "umask=0x00,event=0xc0",
+		.event = "event=0xc0,umask=0x00",
 		.desc = "UNC_CBO_TWO_HYPH. Unit: uncore_cbox ",
 		.topic = "uncore",
 		.long_desc = "UNC_CBO_TWO_HYPH",
 		.pmu = "uncore_cbox",
 	},
-	.alias_str = "umask=0,event=0xc0",
+	.alias_str = "event=0xc0,umask=0",
 	.alias_long_desc = "UNC_CBO_TWO_HYPH",
 	.matching_pmu = "uncore_cbox_0",
 };
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 5/7] perf jevents: Add python converter script
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
                   ` (3 preceding siblings ...)
  2022-05-11 21:15 ` [PATCH v2 4/7] perf jevents: Modify match field Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-11 21:47   ` Andi Kleen
  2022-05-11 21:15 ` [PATCH v2 6/7] perf jevents: Switch build to use jevents.py Ian Rogers
  2022-05-11 21:15 ` [PATCH v2 7/7] perf jevents: Remove jevents.c Ian Rogers
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

jevents.c is large, has a dependency on an old forked version of jsmn,
and is challenging to work upon. A lot of jevents.c's complexity comes
from needing to write json and csv parsing from first principles. In
contrast python has this functionality in standard libraries and is
already a build pre-requisite for tools like asciidoc (that builds all
of the perf man pages).

Introduce jevents.py that produces identical output to jevents.c. Add a
test that runs both converter tools and validates there are no output
differences. The test can be invoked with a phony build target like:

make -C tools/perf jevents-py-test

The python code deliberately tries to replicate the behavior of
jevents.c so that the output matches and transitioning tools shouldn't
introduce regressions. In some cases the code isn't as elegant as hoped,
but fixing this can be done as follow up.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.perf              |   6 +
 tools/perf/pmu-events/jevents-test.sh |  33 +++
 tools/perf/pmu-events/jevents.py      | 392 ++++++++++++++++++++++++++
 3 files changed, 431 insertions(+)
 create mode 100755 tools/perf/pmu-events/jevents-test.sh
 create mode 100755 tools/perf/pmu-events/jevents.py

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 6e5aded855cc..078f0b06f08a 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -669,6 +669,12 @@ $(JEVENTS_IN): FORCE
 $(JEVENTS): $(JEVENTS_IN)
 	$(QUIET_LINK)$(HOSTCC) $(JEVENTS_IN) -o $@
 
+JEVENTS_PY	:=  pmu-events/jevents.py
+JEVENTS_PY_TEST	:=  pmu-events/jevents-test.sh
+.PHONY: jevents-py-test
+jevents-py-test: $(JEVENTS)
+	$(Q)$(call echo-cmd,gen)$(JEVENTS_PY_TEST) $(JEVENTS) $(JEVENTS_PY) pmu-events/arch
+
 $(PMU_EVENTS_IN): $(JEVENTS) FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
 
diff --git a/tools/perf/pmu-events/jevents-test.sh b/tools/perf/pmu-events/jevents-test.sh
new file mode 100755
index 000000000000..195ec83630ad
--- /dev/null
+++ b/tools/perf/pmu-events/jevents-test.sh
@@ -0,0 +1,33 @@
+#!/bin/sh
+# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+# Validate that the legacy jevents and jevents.py produce identical output.
+set -e
+
+JEVENTS="$1"
+JEVENTS_PY="$2"
+ARCH_PATH="$3"
+JEVENTS_C_GENERATED=$(mktemp /tmp/jevents_c.XXXXX.c)
+JEVENTS_PY_GENERATED=$(mktemp /tmp/jevents_py.XXXXX.c)
+
+cleanup() {
+  rm "$JEVENTS_C_GENERATED" "$JEVENTS_PY_GENERATED"
+  trap - exit term int
+}
+trap cleanup exit term int
+
+for path in "$ARCH_PATH"/*
+do
+  arch=$(basename $path)
+  if [[ "$arch" == "test" ]]
+  then
+    continue
+  fi
+  echo "Checking architecture: $arch"
+  echo "Generating using jevents.c"
+  "$JEVENTS" "$arch" "$ARCH_PATH" "$JEVENTS_C_GENERATED"
+  echo "Generating using jevents.py"
+  "$JEVENTS_PY" "$arch" "$ARCH_PATH" "$JEVENTS_PY_GENERATED"
+  echo "Diffing"
+  diff -u "$JEVENTS_C_GENERATED" "$JEVENTS_PY_GENERATED"
+done
+cleanup
diff --git a/tools/perf/pmu-events/jevents.py b/tools/perf/pmu-events/jevents.py
new file mode 100755
index 000000000000..3a3d7471eb71
--- /dev/null
+++ b/tools/perf/pmu-events/jevents.py
@@ -0,0 +1,392 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+"""Convert directories of JSON events to C code."""
+import argparse
+import csv
+import json
+import os
+import sys
+from typing import Callable
+
+# Global command line arguments.
+args = None
+# List of event tables generated from "/sys" directories.
+sys_event_tables = []
+# Map from an event name to an architecture standard
+# JsonEvent. Architecture standard events are in json files in the top
+# f'{args.starting_dir}/{args.arch}' directory.
+arch_std_events = dict()
+# Track whether an events table is currently being defined and needs closing.
+close_table = False
+
+
+def file_name_to_table_name(parents: list[str], dirname: str) -> str:
+  """Generate a C table name from directory names."""
+  tblname = 'pme'
+  for p in parents:
+    tblname += '_' + p
+  tblname += '_' + dirname
+  return tblname.replace('-', '_')
+
+
+class JsonEvent:
+  """Representation of an event loaded from a json file dictionary."""
+
+  def __init__(self, dict):
+    """Constructor passed the dictionary of parsed json values."""
+
+    def llx(x: int) -> str:
+      """Convert an int to a string similar to a printf modifier of %#llx."""
+      return '0' if x == 0 else hex(x)
+
+    def fixdesc(s: str) -> str:
+      """Fix formatting issue for the desc string."""
+      return s.removesuffix('.  ').removesuffix('. ').removesuffix('.').replace(
+          '\n', '\\n').replace('\"', '\\"').replace('\r', '\\r') if s else None
+
+    def convert_aggr_mode(aggr_mode: str) -> str:
+      """Returns the aggr_mode_class enum value associated with the JSON string."""
+      if not aggr_mode:
+        return None
+      aggr_mode_to_enum = {
+          'PerChip': '1',
+          'PerCore': '2',
+      }
+      return aggr_mode_to_enum[aggr_mode]
+
+    def lookup_msr(num: str) -> str:
+      """Converts the msr number, or first in a list to the appropriate event field."""
+      if not num:
+        return None
+      msrmap = {
+          0x3F6: 'ldlat=',
+          0x1A6: 'offcore_rsp=',
+          0x1A7: 'offcore_rsp=',
+          0x3F7: 'frontend=',
+      }
+      return msrmap[int(num.split(',', 1)[0], 0)]
+
+    def real_event(name: str, event: str) -> str:
+      """Convert well known event names to an event string otherwise use the event argument."""
+      fixed = {
+          'inst_retired.any': 'event=0xc0,period=2000003',
+          'inst_retired.any_p': 'event=0xc0,period=2000003',
+          'cpu_clk_unhalted.ref': 'event=0x0,umask=0x03,period=2000003',
+          'cpu_clk_unhalted.thread': 'event=0x3c,period=2000003',
+          'cpu_clk_unhalted.core': 'event=0x3c,period=2000003',
+          'cpu_clk_unhalted.thread_any': 'event=0x3c,any=1,period=2000003',
+      }
+      if not name:
+        return None
+      if name.lower() in fixed:
+        return fixed[name.lower()]
+      return event
+
+    def unit_to_pmu(unit: str) -> str:
+      """Convert a JSON Unit to Linux PMU name."""
+      if not unit:
+        return None
+      # Comment brought over from jevents.c:
+      # it's not realistic to keep adding these, we need something more scalable ...
+      table = {
+          'CBO': 'uncore_cbox',
+          'QPI LL': 'uncore_qpi',
+          'SBO': 'uncore_sbox',
+          'iMPH-U': 'uncore_arb',
+          'CPU-M-CF': 'cpum_cf',
+          'CPU-M-SF': 'cpum_sf',
+          'UPI LL': 'uncore_upi',
+          'hisi_sicl,cpa': 'hisi_sicl,cpa',
+          'hisi_sccl,ddrc': 'hisi_sccl,ddrc',
+          'hisi_sccl,hha': 'hisi_sccl,hha',
+          'hisi_sccl,l3c': 'hisi_sccl,l3c',
+          'imx8_ddr': 'imx8_ddr',
+          'L3PMC': 'amd_l3',
+          'DFPMC': 'amd_df',
+          'cpu_core': 'cpu_core',
+          'cpu_atom': 'cpu_atom',
+      }
+      return table[unit] if unit in table else f'uncore_{unit.lower()}'
+
+    eventcode = 0
+    if 'EventCode' in dict:
+      eventcode = int(dict['EventCode'].split(',', 1)[0], 0)
+    if 'ExtSel' in dict:
+      eventcode |= int(dict['ExtSel']) << 21
+    configcode = int(dict['ConfigCode'], 0) if 'ConfigCode' in dict else None
+    self.name = dict['EventName'].lower() if 'EventName' in dict else None
+    self.compat = dict.get('Compat')
+    self.desc = fixdesc(dict.get('BriefDescription'))
+    self.long_desc = fixdesc(dict.get('PublicDescription'))
+    precise = dict.get('PEBS')
+    msr = lookup_msr(dict.get('MSRIndex'))
+    msrval = dict.get('MSRValue')
+    extra_desc = ''
+    if 'Data_LA' in dict:
+      extra_desc += '  Supports address when precise'
+      if 'Errata' in dict:
+        extra_desc += '.'
+    if 'Errata' in dict:
+      extra_desc += '  Spec update: ' + dict['Errata']
+    self.pmu = unit_to_pmu(dict.get('Unit'))
+    filter = dict.get('Filter')
+    self.unit = dict.get('ScaleUnit')
+    self.perpkg = dict.get('PerPkg')
+    self.aggr_mode = convert_aggr_mode(dict.get('AggregationMode'))
+    self.deprecated = dict.get('Deprecated')
+    self.metric_name = dict.get('MetricName')
+    self.metric_group = dict.get('MetricGroup')
+    self.metric_constraint = dict.get('MetricConstraint')
+    self.metric_expr = dict.get('MetricExpr')
+    if self.metric_expr:
+      self.metric_expr = self.metric_expr.replace('\\', '\\\\')
+    arch_std = dict.get('ArchStdEvent')
+    if precise and self.desc and not '(Precise Event)' in self.desc:
+      extra_desc += ' (Must be precise)' if precise == '2' else (' (Precise '
+                                                                 'event)')
+    event = f'config={llx(configcode)}' if configcode is not None else f'event={llx(eventcode)}'
+    event_fields = [
+        ('AnyThread', 'any='),
+        ('PortMask', 'ch_mask='),
+        ('CounterMask', 'cmask='),
+        ('EdgeDetect', 'edge='),
+        ('FCMask', 'fc_mask='),
+        ('Invert', 'inv='),
+        ('SampleAfterValue', 'period='),
+        ('UMask', 'umask='),
+    ]
+    for key, value in event_fields:
+      if key in dict and dict[key] != '0':
+        event += ',' + value + dict[key]
+    if filter:
+      event += f',{filter}'
+    if msr:
+      event += f',{msr}{msrval}'
+    if self.desc and extra_desc:
+      self.desc += extra_desc
+    if self.long_desc and extra_desc:
+      self.long_desc += extra_desc
+    if self.pmu:
+      if self.desc and not self.desc.endswith('. '):
+        self.desc += '. '
+      self.desc = (self.desc if self.desc else '') + ('Unit: ' + self.pmu + ' ')
+    if arch_std and arch_std.lower() in arch_std_events:
+      event = arch_std_events[arch_std.lower()].event
+      # Copy from the architecture standard event to self for undefined fields.
+      for attr, value in arch_std_events[arch_std.lower()].__dict__.items():
+        if hasattr(self, attr) and not getattr(self, attr):
+          setattr(self, attr, value)
+
+    self.event = real_event(self.name, event)
+
+  def __repr__(self) -> str:
+    """String representation primarily for debugging."""
+    s = '{\n'
+    for attr, value in self.__dict__.items():
+      if value:
+        s += f'\t{attr} = {value},\n'
+    return s + '}'
+
+  def ToCString(self, topic_local: str) -> str:
+    """Representation of the event as a C struct initializer."""
+
+    def AttrString(attr: str, value: str) -> str:
+      return '\t.%s = \"%s\",\n' % (attr, value)
+
+    def StrIfPresent(self, attr: str) -> str:
+      if not getattr(self, attr):
+        return ''
+      return AttrString(attr, getattr(self, attr))
+
+    s = '{\n'
+    for attr in ['name', 'event']:
+      s += StrIfPresent(self, attr)
+    if self.desc:
+      s += AttrString('desc', self.desc)
+    else:
+      s += AttrString('desc', '(null)')
+    s += StrIfPresent(self, 'compat')
+    s += f'\t.topic = "{topic_local}",\n'
+    for attr in [
+        'long_desc', 'pmu', 'unit', 'perpkg', 'aggr_mode', 'metric_expr',
+        'metric_name', 'metric_group', 'deprecated', 'metric_constraint'
+    ]:
+      s += StrIfPresent(self, attr)
+    s += '},\n'
+    return s
+
+
+def read_json_events(path: str) -> list[JsonEvent]:
+  """Read json events from the specified file."""
+  return json.load(open(path), object_hook=lambda d: JsonEvent(d))
+
+def preprocess_arch_std_files(archpath: str) -> None:
+  """Read in all architecture standard events."""
+  global arch_std_events
+  for item in os.scandir(archpath):
+    if item.is_file() and item.name.endswith('.json'):
+      for event in read_json_events(item.path):
+        if event.name:
+          arch_std_events[event.name.lower()] = event
+
+
+def print_events_table_prefix(tblname: str) -> None:
+  """Called when a new events table is started."""
+  global close_table
+  if close_table:
+    raise IOError('Printing table prefix but last table has no suffix')
+  args.output_file.write('static const struct pmu_event %s[] = {\n' % tblname)
+  close_table = True
+
+
+def print_events_table_entries(item: os.DirEntry, topic: str) -> None:
+  """Create contents of an events table."""
+  if not close_table:
+    raise IOError('Table entries missing prefix')
+  for event in read_json_events(item.path):
+    args.output_file.write(event.ToCString(topic))
+
+def print_events_table_suffix() -> None:
+  """Optionally close events table."""
+  global close_table
+  if close_table:
+    args.output_file.write("""{
+\t.name = 0,
+\t.event = 0,
+\t.desc = 0,
+},
+};
+""")
+  close_table = False
+
+def process_one_file(parents: list[str], item: os.DirEntry) -> None:
+  """Process a JSON file during the main walk."""
+  global sys_event_tables
+
+  def get_topic(topic: str) -> str:
+    return topic.removesuffix('.json').replace('-', ' ')
+
+  def is_leaf_dir(path: str) -> bool:
+    for item in os.scandir(path):
+      if item.is_dir():
+        return False
+    return True
+
+  # model directory, reset topic
+  if item.is_dir() and is_leaf_dir(item.path):
+    print_events_table_suffix()
+
+    tblname = file_name_to_table_name(parents, item.name)
+    if item.name == 'sys':
+      sys_event_tables.append(tblname)
+    print_events_table_prefix(tblname)
+    return
+
+  # base dir or too deep
+  level = len(parents)
+  if level == 0 or level > 4:
+    return
+
+  # Ignore other directories. If the file name does not have a .json
+  # extension, ignore it. It could be a readme.txt for instance.
+  if not item.is_file() or not item.name.endswith('.json'):
+    return
+
+  print_events_table_entries(item, get_topic(item.name))
+
+
+def print_mapping_table() -> None:
+  """Read the mapfile and generate the struct from cpuid string to event table."""
+  table = csv.reader(open(f'{args.starting_dir}/{args.arch}/mapfile.csv'))
+  args.output_file.write('const struct pmu_events_map pmu_events_map[] = {\n')
+  first = True
+  for row in table:
+    # Skip the first row or any row beginning with #.
+    if not first and len(row) > 0 and not row[0].startswith('#'):
+      tblname = file_name_to_table_name([], row[2].replace('/', '_'))
+      args.output_file.write("""{
+\t.cpuid = \"%s\",
+\t.version = \"%s\",
+\t.type = \"%s\",
+\t.table = %s
+},
+""" % (row[0].replace('\\', '\\\\'), row[1], row[3], tblname))
+    first = False
+
+  args.output_file.write("""{
+\t.cpuid = "testcpu",
+\t.version = "v1",
+\t.type = "core",
+\t.table = pme_test_soc_cpu,
+},
+{
+\t.cpuid = 0,
+\t.version = 0,
+\t.type = 0,
+\t.table = 0,
+},
+};
+""")
+
+
+def print_system_mapping_table() -> None:
+  """C struct mapping table array for tables from /sys directories."""
+  args.output_file.write(
+      '\nconst struct pmu_sys_events pmu_sys_event_tables[] = {\n')
+  for tblname in sys_event_tables:
+    args.output_file.write("""\t{
+\t\t.table = %s,
+\t\t.name = \"%s\",
+\t},
+""" % (tblname, tblname))
+  args.output_file.write("""\t{
+\t\t.table = 0
+\t},
+};
+""")
+
+
+def main() -> None:
+  global args
+
+  def dir_path(path: str) -> str:
+    """Validate path is a directory for argparse."""
+    if os.path.isdir(path):
+      return path
+    else:
+      raise argparse.ArgumentTypeError(f'\'{path}\' is not a valid directory')
+
+  def ftw(path: str, parents: list[str],
+          action: Callable[[list[str], os.DirEntry], None]) -> None:
+    """Replicate the directory/file walking behavior of C's file tree walk."""
+    for item in os.scandir(path):
+      action(parents, item)
+      if item.is_dir():
+        ftw(item.path, parents + [item.name], action)
+
+  ap = argparse.ArgumentParser()
+  ap.add_argument('arch', help='Architecture name like x86')
+  ap.add_argument(
+      'starting_dir',
+      type=dir_path,
+      help='Root of tree containing architecture directories containing json files'
+  )
+  ap.add_argument(
+      'output_file', type=argparse.FileType('w'), nargs='?', default=sys.stdout)
+  args = ap.parse_args()
+
+  args.output_file.write("#include \"pmu-events/pmu-events.h\"\n")
+  for path in [args.arch, 'test']:
+    arch_path = f'{args.starting_dir}/{path}'
+    if not os.path.isdir(arch_path):
+      raise IOError(f'Missing architecture directory in \'{arch_path}\'')
+    preprocess_arch_std_files(arch_path)
+    ftw(arch_path, [], process_one_file)
+    print_events_table_suffix()
+
+  print_mapping_table()
+  print_system_mapping_table()
+
+
+if __name__ == '__main__':
+  main()
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 6/7] perf jevents: Switch build to use jevents.py
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
                   ` (4 preceding siblings ...)
  2022-05-11 21:15 ` [PATCH v2 5/7] perf jevents: Add python converter script Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  2022-05-13 15:37   ` John Garry
  2022-05-11 21:15 ` [PATCH v2 7/7] perf jevents: Remove jevents.c Ian Rogers
  6 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

Generate pmu-events.c using jevents.py rather than the binary built from
jevents.c. Add a new config variable NO_JEVENTS that is set when there
is no architecture json or an appropriate python interpreter isn't present.
When NO_JEVENTS is defined the file pmu-events/empty-pmu-events.c is copied
and used as the pmu-events.c file.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.config               | 19 +++++++++++++++++++
 tools/perf/Makefile.perf                 |  1 +
 tools/perf/pmu-events/Build              | 11 +++++++++--
 tools/perf/pmu-events/empty-pmu-events.c | 21 +++++++++++++++++++++
 4 files changed, 50 insertions(+), 2 deletions(-)
 create mode 100644 tools/perf/pmu-events/empty-pmu-events.c

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index d9b699ad402c..5a19469f0048 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -854,6 +854,25 @@ else
   endif
 endif
 
+ifneq ($(NO_JEVENTS),1)
+  ifeq ($(wildcard pmu-events/arch/$(SRCARCH)/mapfile.csv),)
+    NO_JEVENTS := 1
+  endif
+endif
+ifneq ($(NO_JEVENTS),1)
+  NO_JEVENTS := 0
+  ifndef PYTHON
+    $(warning No python interpreter disabling jevent generation)
+    NO_JEVENTS := 1
+  else
+    # jevents.py uses os.scandir and type hints present in Python 3.5 released in Sept. 2015.
+    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 5) else "0")')
+    ifneq ($(JEVENTS_PYTHON_GOOD), 1)
+      $(warning Python interpreter too old (older than 3.5) disabling jevent generation)
+      NO_JEVENTS := 1
+    endif
+  endif
+endif
 
 ifndef NO_LIBBFD
   ifeq ($(feature-libbfd), 1)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 078f0b06f08a..db416a13dca7 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -655,6 +655,7 @@ JEVENTS       := $(OUTPUT)pmu-events/jevents
 JEVENTS_IN    := $(OUTPUT)pmu-events/jevents-in.o
 
 PMU_EVENTS_IN := $(OUTPUT)pmu-events/pmu-events-in.o
+export NO_JEVENTS
 
 export JEVENTS
 
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index a055dee6a46a..5e459e1c7a85 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -9,10 +9,17 @@ JSON		=  $(shell [ -d $(JDIR) ] &&				\
 JDIR_TEST	=  pmu-events/arch/test
 JSON_TEST	=  $(shell [ -d $(JDIR_TEST) ] &&			\
 			find $(JDIR_TEST) -name '*.json')
+JEVENTS_PY	=  pmu-events/jevents.py
 
 #
 # Locate/process JSON files in pmu-events/arch/
 # directory and create tables in pmu-events.c.
 #
-$(OUTPUT)pmu-events/pmu-events.c: $(JSON) $(JSON_TEST) $(JEVENTS)
-	$(Q)$(call echo-cmd,gen)$(JEVENTS) $(SRCARCH) pmu-events/arch $(OUTPUT)pmu-events/pmu-events.c $(V)
+
+ifeq ($(NO_JEVENTS),1)
+$(OUTPUT)pmu-events/pmu-events.c: pmu-events/empty-pmu-events.c
+	$(Q)cp $< $@
+else
+$(OUTPUT)pmu-events/pmu-events.c: $(JSON) $(JSON_TEST) $(JEVENTS_PY)
+	$(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(SRCARCH) pmu-events/arch $@
+endif
diff --git a/tools/perf/pmu-events/empty-pmu-events.c b/tools/perf/pmu-events/empty-pmu-events.c
new file mode 100644
index 000000000000..08fee615e015
--- /dev/null
+++ b/tools/perf/pmu-events/empty-pmu-events.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * An empty pmu-events.c file used when there is no architecture json files in
+ * arch or when the jevents.py script cannot be run.
+ */
+#include "pmu-events/pmu-events.h"
+
+const struct pmu_events_map pmu_events_map[] = {
+	{
+		.cpuid = 0,
+		.version = 0,
+		.type = 0,
+		.table = 0,
+	},
+};
+
+const struct pmu_sys_events pmu_sys_event_tables[] = {
+	{
+		.table = 0
+	},
+};
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [PATCH v2 7/7] perf jevents: Remove jevents.c
  2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
                   ` (5 preceding siblings ...)
  2022-05-11 21:15 ` [PATCH v2 6/7] perf jevents: Switch build to use jevents.py Ian Rogers
@ 2022-05-11 21:15 ` Ian Rogers
  6 siblings, 0 replies; 19+ messages in thread
From: Ian Rogers @ 2022-05-11 21:15 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau,
	Qi Liu, Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar
  Cc: Stephane Eranian, Ian Rogers

Remove files and build rules.
Remove test for comparing with jevents.py as there is no longer a binary
to compare with.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/Makefile.perf              |   21 +-
 tools/perf/pmu-events/Build           |    4 -
 tools/perf/pmu-events/jevents-test.sh |   33 -
 tools/perf/pmu-events/jevents.c       | 1342 -------------------------
 tools/perf/pmu-events/jsmn.h          |   68 --
 tools/perf/pmu-events/json.c          |  162 ---
 tools/perf/pmu-events/json.h          |   39 -
 7 files changed, 2 insertions(+), 1667 deletions(-)
 delete mode 100755 tools/perf/pmu-events/jevents-test.sh
 delete mode 100644 tools/perf/pmu-events/jevents.c
 delete mode 100644 tools/perf/pmu-events/jsmn.h
 delete mode 100644 tools/perf/pmu-events/json.c
 delete mode 100644 tools/perf/pmu-events/json.h

diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index db416a13dca7..b26da1eeb27c 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -651,32 +651,15 @@ strip: $(PROGRAMS) $(OUTPUT)perf
 
 PERF_IN := $(OUTPUT)perf-in.o
 
-JEVENTS       := $(OUTPUT)pmu-events/jevents
-JEVENTS_IN    := $(OUTPUT)pmu-events/jevents-in.o
-
 PMU_EVENTS_IN := $(OUTPUT)pmu-events/pmu-events-in.o
 export NO_JEVENTS
 
-export JEVENTS
-
 build := -f $(srctree)/tools/build/Makefile.build dir=. obj
 
 $(PERF_IN): prepare FORCE
 	$(Q)$(MAKE) $(build)=perf
 
-$(JEVENTS_IN): FORCE
-	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=jevents
-
-$(JEVENTS): $(JEVENTS_IN)
-	$(QUIET_LINK)$(HOSTCC) $(JEVENTS_IN) -o $@
-
-JEVENTS_PY	:=  pmu-events/jevents.py
-JEVENTS_PY_TEST	:=  pmu-events/jevents-test.sh
-.PHONY: jevents-py-test
-jevents-py-test: $(JEVENTS)
-	$(Q)$(call echo-cmd,gen)$(JEVENTS_PY_TEST) $(JEVENTS) $(JEVENTS_PY) pmu-events/arch
-
-$(PMU_EVENTS_IN): $(JEVENTS) FORCE
+$(PMU_EVENTS_IN): FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events
 
 $(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) $(LIBTRACEEVENT_DYNAMIC_LIST)
@@ -1095,7 +1078,7 @@ clean:: $(LIBTRACEEVENT)-clean $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clea
 	$(call QUIET_CLEAN, core-objs)  $(RM) $(LIBPERF_A) $(OUTPUT)perf-archive $(OUTPUT)perf-iostat $(LANG_BINDINGS)
 	$(Q)find $(or $(OUTPUT),.) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete
 	$(Q)$(RM) $(OUTPUT).config-detected
-	$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 perf-read-vdsox32 $(OUTPUT)pmu-events/jevents $(OUTPUT)$(LIBJVMTI).so
+	$(call QUIET_CLEAN, core-progs) $(RM) $(ALL_PROGRAMS) perf perf-read-vdso32 perf-read-vdsox32 $(OUTPUT)$(LIBJVMTI).so
 	$(call QUIET_CLEAN, core-gen)   $(RM)  *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)FEATURE-DUMP $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* \
 		$(OUTPUT)util/intel-pt-decoder/inat-tables.c \
 		$(OUTPUT)tests/llvm-src-{base,kbuild,prologue,relocation}.c \
diff --git a/tools/perf/pmu-events/Build b/tools/perf/pmu-events/Build
index 5e459e1c7a85..82e07be354fb 100644
--- a/tools/perf/pmu-events/Build
+++ b/tools/perf/pmu-events/Build
@@ -1,7 +1,3 @@
-hostprogs := jevents
-
-jevents-y	+= json.o jsmn.o jevents.o
-HOSTCFLAGS_jevents.o	= -I$(srctree)/tools/include
 pmu-events-y	+= pmu-events.o
 JDIR		=  pmu-events/arch/$(SRCARCH)
 JSON		=  $(shell [ -d $(JDIR) ] &&				\
diff --git a/tools/perf/pmu-events/jevents-test.sh b/tools/perf/pmu-events/jevents-test.sh
deleted file mode 100755
index 195ec83630ad..000000000000
--- a/tools/perf/pmu-events/jevents-test.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/bin/sh
-# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
-# Validate that the legacy jevents and jevents.py produce identical output.
-set -e
-
-JEVENTS="$1"
-JEVENTS_PY="$2"
-ARCH_PATH="$3"
-JEVENTS_C_GENERATED=$(mktemp /tmp/jevents_c.XXXXX.c)
-JEVENTS_PY_GENERATED=$(mktemp /tmp/jevents_py.XXXXX.c)
-
-cleanup() {
-  rm "$JEVENTS_C_GENERATED" "$JEVENTS_PY_GENERATED"
-  trap - exit term int
-}
-trap cleanup exit term int
-
-for path in "$ARCH_PATH"/*
-do
-  arch=$(basename $path)
-  if [[ "$arch" == "test" ]]
-  then
-    continue
-  fi
-  echo "Checking architecture: $arch"
-  echo "Generating using jevents.c"
-  "$JEVENTS" "$arch" "$ARCH_PATH" "$JEVENTS_C_GENERATED"
-  echo "Generating using jevents.py"
-  "$JEVENTS_PY" "$arch" "$ARCH_PATH" "$JEVENTS_PY_GENERATED"
-  echo "Diffing"
-  diff -u "$JEVENTS_C_GENERATED" "$JEVENTS_PY_GENERATED"
-done
-cleanup
diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
deleted file mode 100644
index cee61c4ed59e..000000000000
--- a/tools/perf/pmu-events/jevents.c
+++ /dev/null
@@ -1,1342 +0,0 @@
-#define  _XOPEN_SOURCE 500	/* needed for nftw() */
-#define  _GNU_SOURCE		/* needed for asprintf() */
-
-/* Parse event JSON files */
-
-/*
- * Copyright (c) 2014, Intel Corporation
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimer.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
- * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
- * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
- * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
- * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
- * OF THE POSSIBILITY OF SUCH DAMAGE.
-*/
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <errno.h>
-#include <string.h>
-#include <ctype.h>
-#include <unistd.h>
-#include <stdarg.h>
-#include <libgen.h>
-#include <limits.h>
-#include <dirent.h>
-#include <sys/time.h>			/* getrlimit */
-#include <sys/resource.h>		/* getrlimit */
-#include <ftw.h>
-#include <sys/stat.h>
-#include <linux/compiler.h>
-#include <linux/list.h>
-#include "jsmn.h"
-#include "json.h"
-#include "pmu-events.h"
-
-int verbose;
-char *prog;
-
-struct json_event {
-	char *name;
-	char *compat;
-	char *event;
-	char *desc;
-	char *long_desc;
-	char *pmu;
-	char *unit;
-	char *perpkg;
-	char *aggr_mode;
-	char *metric_expr;
-	char *metric_name;
-	char *metric_group;
-	char *deprecated;
-	char *metric_constraint;
-};
-
-static enum aggr_mode_class convert(const char *aggr_mode)
-{
-	if (!strcmp(aggr_mode, "PerCore"))
-		return PerCore;
-	else if (!strcmp(aggr_mode, "PerChip"))
-		return PerChip;
-
-	pr_err("%s: Wrong AggregationMode value '%s'\n", prog, aggr_mode);
-	return -1;
-}
-
-static LIST_HEAD(sys_event_tables);
-
-struct sys_event_table {
-	struct list_head list;
-	char *soc_id;
-};
-
-static void free_sys_event_tables(void)
-{
-	struct sys_event_table *et, *next;
-
-	list_for_each_entry_safe(et, next, &sys_event_tables, list) {
-		free(et->soc_id);
-		free(et);
-	}
-}
-
-int eprintf(int level, int var, const char *fmt, ...)
-{
-
-	int ret;
-	va_list args;
-
-	if (var < level)
-		return 0;
-
-	va_start(args, fmt);
-
-	ret = vfprintf(stderr, fmt, args);
-
-	va_end(args);
-
-	return ret;
-}
-
-static void addfield(char *map, char **dst, const char *sep,
-		     const char *a, jsmntok_t *bt)
-{
-	unsigned int len = strlen(a) + 1 + strlen(sep);
-	int olen = *dst ? strlen(*dst) : 0;
-	int blen = bt ? json_len(bt) : 0;
-	char *out;
-
-	out = realloc(*dst, len + olen + blen);
-	if (!out) {
-		/* Don't add field in this case */
-		return;
-	}
-	*dst = out;
-
-	if (!olen)
-		*(*dst) = 0;
-	else
-		strcat(*dst, sep);
-	strcat(*dst, a);
-	if (bt)
-		strncat(*dst, map + bt->start, blen);
-}
-
-static void fixname(char *s)
-{
-	for (; *s; s++)
-		*s = tolower(*s);
-}
-
-static void fixdesc(char *s)
-{
-	char *e = s + strlen(s);
-
-	/* Remove trailing dots that look ugly in perf list */
-	--e;
-	while (e >= s && isspace(*e))
-		--e;
-	if (*e == '.')
-		*e = 0;
-}
-
-/* Add escapes for '\' so they are proper C strings. */
-static char *fixregex(char *s)
-{
-	int len = 0;
-	int esc_count = 0;
-	char *fixed = NULL;
-	char *p, *q;
-
-	/* Count the number of '\' in string */
-	for (p = s; *p; p++) {
-		++len;
-		if (*p == '\\')
-			++esc_count;
-	}
-
-	if (esc_count == 0)
-		return s;
-
-	/* allocate space for a new string */
-	fixed = (char *) malloc(len + esc_count + 1);
-	if (!fixed)
-		return NULL;
-
-	/* copy over the characters */
-	q = fixed;
-	for (p = s; *p; p++) {
-		if (*p == '\\') {
-			*q = '\\';
-			++q;
-		}
-		*q = *p;
-		++q;
-	}
-	*q = '\0';
-	return fixed;
-}
-
-static struct msrmap {
-	const char *num;
-	const char *pname;
-} msrmap[] = {
-	{ "0x3F6", "ldlat=" },
-	{ "0x1A6", "offcore_rsp=" },
-	{ "0x1A7", "offcore_rsp=" },
-	{ "0x3F7", "frontend=" },
-	{ NULL, NULL }
-};
-
-static void cut_comma(char *map, jsmntok_t *newval)
-{
-	int i;
-
-	/* Cut off everything after comma */
-	for (i = newval->start; i < newval->end; i++) {
-		if (map[i] == ',')
-			newval->end = i;
-	}
-}
-
-static struct msrmap *lookup_msr(char *map, jsmntok_t *val)
-{
-	jsmntok_t newval = *val;
-	static bool warned;
-	int i;
-
-	cut_comma(map, &newval);
-	for (i = 0; msrmap[i].num; i++)
-		if (json_streq(map, &newval, msrmap[i].num))
-			return &msrmap[i];
-	if (!warned) {
-		warned = true;
-		pr_err("%s: Unknown MSR in event file %.*s\n", prog,
-			json_len(val), map + val->start);
-	}
-	return NULL;
-}
-
-static struct map {
-	const char *json;
-	const char *perf;
-} unit_to_pmu[] = {
-	{ "CBO", "uncore_cbox" },
-	{ "QPI LL", "uncore_qpi" },
-	{ "SBO", "uncore_sbox" },
-	{ "iMPH-U", "uncore_arb" },
-	{ "CPU-M-CF", "cpum_cf" },
-	{ "CPU-M-SF", "cpum_sf" },
-	{ "UPI LL", "uncore_upi" },
-	{ "hisi_sicl,cpa", "hisi_sicl,cpa"},
-	{ "hisi_sccl,ddrc", "hisi_sccl,ddrc" },
-	{ "hisi_sccl,hha", "hisi_sccl,hha" },
-	{ "hisi_sccl,l3c", "hisi_sccl,l3c" },
-	/* it's not realistic to keep adding these, we need something more scalable ... */
-	{ "imx8_ddr", "imx8_ddr" },
-	{ "L3PMC", "amd_l3" },
-	{ "DFPMC", "amd_df" },
-	{ "cpu_core", "cpu_core" },
-	{ "cpu_atom", "cpu_atom" },
-	{}
-};
-
-static const char *field_to_perf(struct map *table, char *map, jsmntok_t *val)
-{
-	int i;
-
-	for (i = 0; table[i].json; i++) {
-		if (json_streq(map, val, table[i].json))
-			return table[i].perf;
-	}
-	return NULL;
-}
-
-#define EXPECT(e, t, m) do { if (!(e)) {			\
-	jsmntok_t *loc = (t);					\
-	if (!(t)->start && (t) > tokens)			\
-		loc = (t) - 1;					\
-	pr_err("%s:%d: " m ", got %s\n", fn,			\
-	       json_line(map, loc),				\
-	       json_name(t));					\
-	err = -EIO;						\
-	goto out_free;						\
-} } while (0)
-
-static char *topic;
-
-static char *get_topic(void)
-{
-	char *tp;
-	int i;
-
-	/* tp is free'd in process_one_file() */
-	i = asprintf(&tp, "%s", topic);
-	if (i < 0) {
-		pr_info("%s: asprintf() error %s\n", prog);
-		return NULL;
-	}
-
-	for (i = 0; i < (int) strlen(tp); i++) {
-		char c = tp[i];
-
-		if (c == '-')
-			tp[i] = ' ';
-		else if (c == '.') {
-			tp[i] = '\0';
-			break;
-		}
-	}
-
-	return tp;
-}
-
-static int add_topic(char *bname)
-{
-	free(topic);
-	topic = strdup(bname);
-	if (!topic) {
-		pr_info("%s: strdup() error %s for file %s\n", prog,
-				strerror(errno), bname);
-		return -ENOMEM;
-	}
-	return 0;
-}
-
-struct perf_entry_data {
-	FILE *outfp;
-	char *topic;
-};
-
-static int close_table;
-
-static void print_events_table_prefix(FILE *fp, const char *tblname)
-{
-	fprintf(fp, "static const struct pmu_event %s[] = {\n", tblname);
-	close_table = 1;
-}
-
-static int print_events_table_entry(void *data, struct json_event *je)
-{
-	struct perf_entry_data *pd = data;
-	FILE *outfp = pd->outfp;
-	char *topic_local = pd->topic;
-
-	/*
-	 * TODO: Remove formatting chars after debugging to reduce
-	 *	 string lengths.
-	 */
-	fprintf(outfp, "{\n");
-
-	if (je->name)
-		fprintf(outfp, "\t.name = \"%s\",\n", je->name);
-	if (je->event)
-		fprintf(outfp, "\t.event = \"%s\",\n", je->event);
-	fprintf(outfp, "\t.desc = \"%s\",\n", je->desc);
-	if (je->compat)
-		fprintf(outfp, "\t.compat = \"%s\",\n", je->compat);
-	fprintf(outfp, "\t.topic = \"%s\",\n", topic_local);
-	if (je->long_desc && je->long_desc[0])
-		fprintf(outfp, "\t.long_desc = \"%s\",\n", je->long_desc);
-	if (je->pmu)
-		fprintf(outfp, "\t.pmu = \"%s\",\n", je->pmu);
-	if (je->unit)
-		fprintf(outfp, "\t.unit = \"%s\",\n", je->unit);
-	if (je->perpkg)
-		fprintf(outfp, "\t.perpkg = \"%s\",\n", je->perpkg);
-	if (je->aggr_mode)
-		fprintf(outfp, "\t.aggr_mode = \"%d\",\n", convert(je->aggr_mode));
-	if (je->metric_expr)
-		fprintf(outfp, "\t.metric_expr = \"%s\",\n", je->metric_expr);
-	if (je->metric_name)
-		fprintf(outfp, "\t.metric_name = \"%s\",\n", je->metric_name);
-	if (je->metric_group)
-		fprintf(outfp, "\t.metric_group = \"%s\",\n", je->metric_group);
-	if (je->deprecated)
-		fprintf(outfp, "\t.deprecated = \"%s\",\n", je->deprecated);
-	if (je->metric_constraint)
-		fprintf(outfp, "\t.metric_constraint = \"%s\",\n", je->metric_constraint);
-	fprintf(outfp, "},\n");
-
-	return 0;
-}
-
-struct event_struct {
-	struct list_head list;
-	char *name;
-	char *event;
-	char *compat;
-	char *desc;
-	char *long_desc;
-	char *pmu;
-	char *unit;
-	char *perpkg;
-	char *aggr_mode;
-	char *metric_expr;
-	char *metric_name;
-	char *metric_group;
-	char *deprecated;
-	char *metric_constraint;
-};
-
-#define ADD_EVENT_FIELD(field) do { if (je->field) {		\
-	es->field = strdup(je->field);				\
-	if (!es->field)						\
-		goto out_free;					\
-} } while (0)
-
-#define FREE_EVENT_FIELD(field) free(es->field)
-
-#define TRY_FIXUP_FIELD(field) do { if (es->field && !je->field) {\
-	je->field = strdup(es->field);				\
-	if (!je->field)						\
-		return -ENOMEM;					\
-} } while (0)
-
-#define FOR_ALL_EVENT_STRUCT_FIELDS(op) do {			\
-	op(name);						\
-	op(event);						\
-	op(desc);						\
-	op(long_desc);						\
-	op(pmu);						\
-	op(unit);						\
-	op(perpkg);						\
-	op(aggr_mode);						\
-	op(metric_expr);					\
-	op(metric_name);					\
-	op(metric_group);					\
-	op(deprecated);						\
-} while (0)
-
-static LIST_HEAD(arch_std_events);
-
-static void free_arch_std_events(void)
-{
-	struct event_struct *es, *next;
-
-	list_for_each_entry_safe(es, next, &arch_std_events, list) {
-		FOR_ALL_EVENT_STRUCT_FIELDS(FREE_EVENT_FIELD);
-		list_del_init(&es->list);
-		free(es);
-	}
-}
-
-static int save_arch_std_events(void *data __maybe_unused, struct json_event *je)
-{
-	struct event_struct *es;
-
-	es = malloc(sizeof(*es));
-	if (!es)
-		return -ENOMEM;
-	memset(es, 0, sizeof(*es));
-	FOR_ALL_EVENT_STRUCT_FIELDS(ADD_EVENT_FIELD);
-	list_add_tail(&es->list, &arch_std_events);
-	return 0;
-out_free:
-	FOR_ALL_EVENT_STRUCT_FIELDS(FREE_EVENT_FIELD);
-	free(es);
-	return -ENOMEM;
-}
-
-static void print_events_table_suffix(FILE *outfp)
-{
-	fprintf(outfp, "{\n");
-
-	fprintf(outfp, "\t.name = 0,\n");
-	fprintf(outfp, "\t.event = 0,\n");
-	fprintf(outfp, "\t.desc = 0,\n");
-
-	fprintf(outfp, "},\n");
-	fprintf(outfp, "};\n");
-	close_table = 0;
-}
-
-static struct fixed {
-	const char *name;
-	const char *event;
-} fixed[] = {
-	{ "inst_retired.any", "event=0xc0,period=2000003" },
-	{ "inst_retired.any_p", "event=0xc0,period=2000003" },
-	{ "cpu_clk_unhalted.ref", "event=0x0,umask=0x03,period=2000003" },
-	{ "cpu_clk_unhalted.thread", "event=0x3c,period=2000003" },
-	{ "cpu_clk_unhalted.core", "event=0x3c,period=2000003" },
-	{ "cpu_clk_unhalted.thread_any", "event=0x3c,any=1,period=2000003" },
-	{ NULL, NULL},
-};
-
-/*
- * Handle different fixed counter encodings between JSON and perf.
- */
-static char *real_event(const char *name, char *event)
-{
-	int i;
-
-	if (!name)
-		return NULL;
-
-	for (i = 0; fixed[i].name; i++)
-		if (!strcasecmp(name, fixed[i].name))
-			return (char *)fixed[i].event;
-	return event;
-}
-
-static int
-try_fixup(const char *fn, char *arch_std, struct json_event *je, char **event)
-{
-	/* try to find matching event from arch standard values */
-	struct event_struct *es;
-
-	list_for_each_entry(es, &arch_std_events, list) {
-		if (!strcmp(arch_std, es->name)) {
-			FOR_ALL_EVENT_STRUCT_FIELDS(TRY_FIXUP_FIELD);
-			*event = je->event;
-			return 0;
-		}
-	}
-
-	pr_err("%s: could not find matching %s for %s\n",
-					prog, arch_std, fn);
-	return -1;
-}
-
-/* Call func with each event in the json file */
-static int json_events(const char *fn,
-		int (*func)(void *data, struct json_event *je),
-			void *data)
-{
-	int err;
-	size_t size;
-	jsmntok_t *tokens, *tok;
-	int i, j, len;
-	char *map;
-	char buf[128];
-
-	if (!fn)
-		return -ENOENT;
-
-	tokens = parse_json(fn, &map, &size, &len);
-	if (!tokens)
-		return -EIO;
-	EXPECT(tokens->type == JSMN_ARRAY, tokens, "expected top level array");
-	tok = tokens + 1;
-	for (i = 0; i < tokens->size; i++) {
-		char *event = NULL;
-		char *extra_desc = NULL;
-		char *filter = NULL;
-		struct json_event je = {};
-		char *arch_std = NULL;
-		unsigned long long eventcode = 0;
-		unsigned long long configcode = 0;
-		struct msrmap *msr = NULL;
-		jsmntok_t *msrval = NULL;
-		jsmntok_t *precise = NULL;
-		jsmntok_t *obj = tok++;
-		bool configcode_present = false;
-		char *umask = NULL;
-		char *cmask = NULL;
-		char *inv = NULL;
-		char *any = NULL;
-		char *edge = NULL;
-		char *period = NULL;
-		char *fc_mask = NULL;
-		char *ch_mask = NULL;
-
-		EXPECT(obj->type == JSMN_OBJECT, obj, "expected object");
-		for (j = 0; j < obj->size; j += 2) {
-			jsmntok_t *field, *val;
-			int nz;
-			char *s;
-
-			field = tok + j;
-			EXPECT(field->type == JSMN_STRING, tok + j,
-			       "Expected field name");
-			val = tok + j + 1;
-			EXPECT(val->type == JSMN_STRING, tok + j + 1,
-			       "Expected string value");
-
-			nz = !json_streq(map, val, "0");
-			/* match_field */
-			if (json_streq(map, field, "UMask") && nz) {
-				addfield(map, &umask, "", "umask=", val);
-			} else if (json_streq(map, field, "CounterMask") && nz) {
-				addfield(map, &cmask, "", "cmask=", val);
-			} else if (json_streq(map, field, "Invert") && nz) {
-				addfield(map, &inv, "", "inv=", val);
-			} else if (json_streq(map, field, "AnyThread") && nz) {
-				addfield(map, &any, "", "any=", val);
-			} else if (json_streq(map, field, "EdgeDetect") && nz) {
-				addfield(map, &edge, "", "edge=", val);
-			} else if (json_streq(map, field, "SampleAfterValue") && nz) {
-				addfield(map, &period, "", "period=", val);
-			} else if (json_streq(map, field, "FCMask") && nz) {
-				addfield(map, &fc_mask, "", "fc_mask=", val);
-			} else if (json_streq(map, field, "PortMask") && nz) {
-				addfield(map, &ch_mask, "", "ch_mask=", val);
-			} else if (json_streq(map, field, "EventCode")) {
-				char *code = NULL;
-				addfield(map, &code, "", "", val);
-				eventcode |= strtoul(code, NULL, 0);
-				free(code);
-			} else if (json_streq(map, field, "ConfigCode")) {
-				char *code = NULL;
-				addfield(map, &code, "", "", val);
-				configcode |= strtoul(code, NULL, 0);
-				free(code);
-				configcode_present = true;
-			} else if (json_streq(map, field, "ExtSel")) {
-				char *code = NULL;
-				addfield(map, &code, "", "", val);
-				eventcode |= strtoul(code, NULL, 0) << 21;
-				free(code);
-			} else if (json_streq(map, field, "EventName")) {
-				addfield(map, &je.name, "", "", val);
-			} else if (json_streq(map, field, "Compat")) {
-				addfield(map, &je.compat, "", "", val);
-			} else if (json_streq(map, field, "BriefDescription")) {
-				addfield(map, &je.desc, "", "", val);
-				fixdesc(je.desc);
-			} else if (json_streq(map, field,
-					     "PublicDescription")) {
-				addfield(map, &je.long_desc, "", "", val);
-				fixdesc(je.long_desc);
-			} else if (json_streq(map, field, "PEBS") && nz) {
-				precise = val;
-			} else if (json_streq(map, field, "MSRIndex") && nz) {
-				msr = lookup_msr(map, val);
-			} else if (json_streq(map, field, "MSRValue")) {
-				msrval = val;
-			} else if (json_streq(map, field, "Errata") &&
-				   !json_streq(map, val, "null")) {
-				addfield(map, &extra_desc, ". ",
-					" Spec update: ", val);
-			} else if (json_streq(map, field, "Data_LA") && nz) {
-				addfield(map, &extra_desc, ". ",
-					" Supports address when precise",
-					NULL);
-			} else if (json_streq(map, field, "Unit")) {
-				const char *ppmu;
-
-				ppmu = field_to_perf(unit_to_pmu, map, val);
-				if (ppmu) {
-					je.pmu = strdup(ppmu);
-				} else {
-					if (!je.pmu)
-						je.pmu = strdup("uncore_");
-					addfield(map, &je.pmu, "", "", val);
-					for (s = je.pmu; *s; s++)
-						*s = tolower(*s);
-				}
-			} else if (json_streq(map, field, "Filter")) {
-				addfield(map, &filter, "", "", val);
-			} else if (json_streq(map, field, "ScaleUnit")) {
-				addfield(map, &je.unit, "", "", val);
-			} else if (json_streq(map, field, "PerPkg")) {
-				addfield(map, &je.perpkg, "", "", val);
-			} else if (json_streq(map, field, "AggregationMode")) {
-				addfield(map, &je.aggr_mode, "", "", val);
-			} else if (json_streq(map, field, "Deprecated")) {
-				addfield(map, &je.deprecated, "", "", val);
-			} else if (json_streq(map, field, "MetricName")) {
-				addfield(map, &je.metric_name, "", "", val);
-			} else if (json_streq(map, field, "MetricGroup")) {
-				addfield(map, &je.metric_group, "", "", val);
-			} else if (json_streq(map, field, "MetricConstraint")) {
-				addfield(map, &je.metric_constraint, "", "", val);
-			} else if (json_streq(map, field, "MetricExpr")) {
-				addfield(map, &je.metric_expr, "", "", val);
-			} else if (json_streq(map, field, "ArchStdEvent")) {
-				addfield(map, &arch_std, "", "", val);
-				for (s = arch_std; *s; s++)
-					*s = tolower(*s);
-			}
-			/* ignore unknown fields */
-		}
-		if (precise && je.desc && !strstr(je.desc, "(Precise Event)")) {
-			if (json_streq(map, precise, "2"))
-				addfield(map, &extra_desc, " ",
-						"(Must be precise)", NULL);
-			else
-				addfield(map, &extra_desc, " ",
-						"(Precise event)", NULL);
-		}
-		if (configcode_present)
-			snprintf(buf, sizeof buf, "config=%#llx", configcode);
-		else
-			snprintf(buf, sizeof buf, "event=%#llx", eventcode);
-		addfield(map, &event, ",", buf, NULL);
-		if (any)
-			addfield(map, &event, ",", any, NULL);
-		if (ch_mask)
-			addfield(map, &event, ",", ch_mask, NULL);
-		if (cmask)
-			addfield(map, &event, ",", cmask, NULL);
-		if (edge)
-			addfield(map, &event, ",", edge, NULL);
-		if (fc_mask)
-			addfield(map, &event, ",", fc_mask, NULL);
-		if (inv)
-			addfield(map, &event, ",", inv, NULL);
-		if (period)
-			addfield(map, &event, ",", period, NULL);
-		if (umask)
-			addfield(map, &event, ",", umask, NULL);
-
-		if (je.desc && extra_desc)
-			addfield(map, &je.desc, " ", extra_desc, NULL);
-		if (je.long_desc && extra_desc)
-			addfield(map, &je.long_desc, " ", extra_desc, NULL);
-		if (je.pmu) {
-			addfield(map, &je.desc, ". ", "Unit: ", NULL);
-			addfield(map, &je.desc, "", je.pmu, NULL);
-			addfield(map, &je.desc, "", " ", NULL);
-		}
-		if (filter)
-			addfield(map, &event, ",", filter, NULL);
-		if (msr != NULL)
-			addfield(map, &event, ",", msr->pname, msrval);
-		if (je.name)
-			fixname(je.name);
-
-		if (arch_std) {
-			/*
-			 * An arch standard event is referenced, so try to
-			 * fixup any unassigned values.
-			 */
-			err = try_fixup(fn, arch_std, &je, &event);
-			if (err)
-				goto free_strings;
-		}
-		je.event = real_event(je.name, event);
-		err = func(data, &je);
-free_strings:
-		free(umask);
-		free(cmask);
-		free(inv);
-		free(any);
-		free(edge);
-		free(period);
-		free(fc_mask);
-		free(ch_mask);
-		free(event);
-		free(je.desc);
-		free(je.name);
-		free(je.compat);
-		free(je.long_desc);
-		free(extra_desc);
-		free(je.pmu);
-		free(filter);
-		free(je.perpkg);
-		free(je.aggr_mode);
-		free(je.deprecated);
-		free(je.unit);
-		free(je.metric_expr);
-		free(je.metric_name);
-		free(je.metric_group);
-		free(je.metric_constraint);
-		free(arch_std);
-
-		if (err)
-			break;
-		tok += j;
-	}
-	EXPECT(tok - tokens == len, tok, "unexpected objects at end");
-	err = 0;
-out_free:
-	free_json(map, size, tokens);
-	return err;
-}
-
-static char *file_name_to_table_name(char *fname)
-{
-	unsigned int i;
-	int n;
-	int c;
-	char *tblname;
-
-	/*
-	 * Ensure tablename starts with alphabetic character.
-	 * Derive rest of table name from basename of the JSON file,
-	 * replacing hyphens and stripping out .json suffix.
-	 */
-	n = asprintf(&tblname, "pme_%s", fname);
-	if (n < 0) {
-		pr_info("%s: asprintf() error %s for file %s\n", prog,
-				strerror(errno), fname);
-		return NULL;
-	}
-
-	for (i = 0; i < strlen(tblname); i++) {
-		c = tblname[i];
-
-		if (c == '-' || c == '/')
-			tblname[i] = '_';
-		else if (c == '.') {
-			tblname[i] = '\0';
-			break;
-		} else if (!isalnum(c) && c != '_') {
-			pr_err("%s: Invalid character '%c' in file name %s\n",
-					prog, c, basename(fname));
-			free(tblname);
-			tblname = NULL;
-			break;
-		}
-	}
-
-	return tblname;
-}
-
-static bool is_sys_dir(char *fname)
-{
-	size_t len = strlen(fname), len2 = strlen("/sys");
-
-	if (len2 > len)
-		return false;
-	return !strcmp(fname+len-len2, "/sys");
-}
-
-static void print_mapping_table_prefix(FILE *outfp)
-{
-	fprintf(outfp, "const struct pmu_events_map pmu_events_map[] = {\n");
-}
-
-static void print_mapping_table_suffix(FILE *outfp)
-{
-	/*
-	 * Print the terminating, NULL entry.
-	 */
-	fprintf(outfp, "{\n");
-	fprintf(outfp, "\t.cpuid = 0,\n");
-	fprintf(outfp, "\t.version = 0,\n");
-	fprintf(outfp, "\t.type = 0,\n");
-	fprintf(outfp, "\t.table = 0,\n");
-	fprintf(outfp, "},\n");
-
-	/* and finally, the closing curly bracket for the struct */
-	fprintf(outfp, "};\n");
-}
-
-static void print_mapping_test_table(FILE *outfp)
-{
-	/*
-	 * Print the terminating, NULL entry.
-	 */
-	fprintf(outfp, "{\n");
-	fprintf(outfp, "\t.cpuid = \"testcpu\",\n");
-	fprintf(outfp, "\t.version = \"v1\",\n");
-	fprintf(outfp, "\t.type = \"core\",\n");
-	fprintf(outfp, "\t.table = pme_test_soc_cpu,\n");
-	fprintf(outfp, "},\n");
-}
-
-static void print_system_event_mapping_table_prefix(FILE *outfp)
-{
-	fprintf(outfp, "\nconst struct pmu_sys_events pmu_sys_event_tables[] = {");
-}
-
-static void print_system_event_mapping_table_suffix(FILE *outfp)
-{
-	fprintf(outfp, "\n\t{\n\t\t.table = 0\n\t},");
-	fprintf(outfp, "\n};\n");
-}
-
-static int process_system_event_tables(FILE *outfp)
-{
-	struct sys_event_table *sys_event_table;
-
-	print_system_event_mapping_table_prefix(outfp);
-
-	list_for_each_entry(sys_event_table, &sys_event_tables, list) {
-		fprintf(outfp, "\n\t{\n\t\t.table = %s,\n\t\t.name = \"%s\",\n\t},",
-			sys_event_table->soc_id,
-			sys_event_table->soc_id);
-	}
-
-	print_system_event_mapping_table_suffix(outfp);
-
-	return 0;
-}
-
-static int process_mapfile(FILE *outfp, char *fpath)
-{
-	int n = 16384;
-	FILE *mapfp;
-	char *save = NULL;
-	char *line, *p;
-	int line_num;
-	char *tblname;
-	int ret = 0;
-
-	pr_info("%s: Processing mapfile %s\n", prog, fpath);
-
-	line = malloc(n);
-	if (!line)
-		return -1;
-
-	mapfp = fopen(fpath, "r");
-	if (!mapfp) {
-		pr_info("%s: Error %s opening %s\n", prog, strerror(errno),
-				fpath);
-		free(line);
-		return -1;
-	}
-
-	print_mapping_table_prefix(outfp);
-
-	/* Skip first line (header) */
-	p = fgets(line, n, mapfp);
-	if (!p)
-		goto out;
-
-	line_num = 1;
-	while (1) {
-		char *cpuid, *version, *type, *fname;
-
-		line_num++;
-		p = fgets(line, n, mapfp);
-		if (!p)
-			break;
-
-		if (line[0] == '#' || line[0] == '\n')
-			continue;
-
-		if (line[strlen(line)-1] != '\n') {
-			/* TODO Deal with lines longer than 16K */
-			pr_info("%s: Mapfile %s: line %d too long, aborting\n",
-					prog, fpath, line_num);
-			ret = -1;
-			goto out;
-		}
-		line[strlen(line)-1] = '\0';
-
-		cpuid = fixregex(strtok_r(p, ",", &save));
-		version = strtok_r(NULL, ",", &save);
-		fname = strtok_r(NULL, ",", &save);
-		type = strtok_r(NULL, ",", &save);
-
-		tblname = file_name_to_table_name(fname);
-		fprintf(outfp, "{\n");
-		fprintf(outfp, "\t.cpuid = \"%s\",\n", cpuid);
-		fprintf(outfp, "\t.version = \"%s\",\n", version);
-		fprintf(outfp, "\t.type = \"%s\",\n", type);
-
-		/*
-		 * CHECK: We can't use the type (eg "core") field in the
-		 * table name. For us to do that, we need to somehow tweak
-		 * the other caller of file_name_to_table(), process_json()
-		 * to determine the type. process_json() file has no way
-		 * of knowing these are "core" events unless file name has
-		 * core in it. If filename has core in it, we can safely
-		 * ignore the type field here also.
-		 */
-		fprintf(outfp, "\t.table = %s\n", tblname);
-		fprintf(outfp, "},\n");
-	}
-
-out:
-	print_mapping_test_table(outfp);
-	print_mapping_table_suffix(outfp);
-	fclose(mapfp);
-	free(line);
-	return ret;
-}
-
-/*
- * If we fail to locate/process JSON and map files, create a NULL mapping
- * table. This would at least allow perf to build even if we can't find/use
- * the aliases.
- */
-static void create_empty_mapping(const char *output_file)
-{
-	FILE *outfp;
-
-	pr_info("%s: Creating empty pmu_events_map[] table\n", prog);
-
-	/* Truncate file to clear any partial writes to it */
-	outfp = fopen(output_file, "w");
-	if (!outfp) {
-		perror("fopen()");
-		_Exit(1);
-	}
-
-	fprintf(outfp, "#include \"pmu-events/pmu-events.h\"\n");
-	print_mapping_table_prefix(outfp);
-	print_mapping_table_suffix(outfp);
-	print_system_event_mapping_table_prefix(outfp);
-	print_system_event_mapping_table_suffix(outfp);
-	fclose(outfp);
-}
-
-static int get_maxfds(void)
-{
-	struct rlimit rlim;
-
-	if (getrlimit(RLIMIT_NOFILE, &rlim) == 0)
-		return min(rlim.rlim_max / 2, (rlim_t)512);
-
-	return 512;
-}
-
-/*
- * nftw() doesn't let us pass an argument to the processing function,
- * so use a global variables.
- */
-static FILE *eventsfp;
-static char *mapfile;
-
-static int is_leaf_dir(const char *fpath)
-{
-	DIR *d;
-	struct dirent *dir;
-	int res = 1;
-
-	d = opendir(fpath);
-	if (!d)
-		return 0;
-
-	while ((dir = readdir(d)) != NULL) {
-		if (!strcmp(dir->d_name, ".") || !strcmp(dir->d_name, ".."))
-			continue;
-
-		if (dir->d_type == DT_DIR) {
-			res = 0;
-			break;
-		} else if (dir->d_type == DT_UNKNOWN) {
-			char path[PATH_MAX];
-			struct stat st;
-
-			sprintf(path, "%s/%s", fpath, dir->d_name);
-			if (stat(path, &st))
-				break;
-
-			if (S_ISDIR(st.st_mode)) {
-				res = 0;
-				break;
-			}
-		}
-	}
-
-	closedir(d);
-
-	return res;
-}
-
-static int is_json_file(const char *name)
-{
-	const char *suffix;
-
-	if (strlen(name) < 5)
-		return 0;
-
-	suffix = name + strlen(name) - 5;
-
-	if (strncmp(suffix, ".json", 5) == 0)
-		return 1;
-	return 0;
-}
-
-static int preprocess_arch_std_files(const char *fpath, const struct stat *sb,
-				int typeflag, struct FTW *ftwbuf)
-{
-	int level = ftwbuf->level;
-	int is_file = typeflag == FTW_F;
-
-	if (level == 1 && is_file && is_json_file(fpath))
-		return json_events(fpath, save_arch_std_events, (void *)sb);
-
-	return 0;
-}
-
-static int process_one_file(const char *fpath, const struct stat *sb,
-			    int typeflag, struct FTW *ftwbuf)
-{
-	char *tblname, *bname;
-	int is_dir  = typeflag == FTW_D;
-	int is_file = typeflag == FTW_F;
-	int level   = ftwbuf->level;
-	int err = 0;
-
-	if (level >= 2 && is_dir) {
-		int count = 0;
-		/*
-		 * For level 2 directory, bname will include parent name,
-		 * like vendor/platform. So search back from platform dir
-		 * to find this.
-		 * Something similar for level 3 directory, but we're a PMU
-		 * category folder, like vendor/platform/cpu.
-		 */
-		bname = (char *) fpath + ftwbuf->base - 2;
-		for (;;) {
-			if (*bname == '/')
-				count++;
-			if (count == level - 1)
-				break;
-			bname--;
-		}
-		bname++;
-	} else
-		bname = (char *) fpath + ftwbuf->base;
-
-	pr_debug("%s %d %7jd %-20s %s\n",
-		 is_file ? "f" : is_dir ? "d" : "x",
-		 level, sb->st_size, bname, fpath);
-
-	/* base dir or too deep */
-	if (level == 0 || level > 4)
-		return 0;
-
-
-	/* model directory, reset topic */
-	if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
-	    (level >= 2 && is_dir && is_leaf_dir(fpath))) {
-		if (close_table)
-			print_events_table_suffix(eventsfp);
-
-		/*
-		 * Drop file name suffix. Replace hyphens with underscores.
-		 * Fail if file name contains any alphanum characters besides
-		 * underscores.
-		 */
-		tblname = file_name_to_table_name(bname);
-		if (!tblname) {
-			pr_info("%s: Error determining table name for %s\n", prog,
-				bname);
-			return -1;
-		}
-
-		if (is_sys_dir(bname)) {
-			struct sys_event_table *sys_event_table;
-
-			sys_event_table = malloc(sizeof(*sys_event_table));
-			if (!sys_event_table)
-				return -1;
-
-			sys_event_table->soc_id = strdup(tblname);
-			if (!sys_event_table->soc_id) {
-				free(sys_event_table);
-				return -1;
-			}
-			list_add_tail(&sys_event_table->list,
-				      &sys_event_tables);
-		}
-
-		print_events_table_prefix(eventsfp, tblname);
-		return 0;
-	}
-
-	/*
-	 * Save the mapfile name for now. We will process mapfile
-	 * after processing all JSON files (so we can write out the
-	 * mapping table after all PMU events tables).
-	 *
-	 */
-	if (level == 1 && is_file) {
-		if (!strcmp(bname, "mapfile.csv")) {
-			mapfile = strdup(fpath);
-			return 0;
-		}
-		if (is_json_file(bname))
-			pr_debug("%s: ArchStd json is preprocessed %s\n", prog, fpath);
-		else
-			pr_info("%s: Ignoring file %s\n", prog, fpath);
-		return 0;
-	}
-
-	/*
-	 * If the file name does not have a .json extension,
-	 * ignore it. It could be a readme.txt for instance.
-	 */
-	if (is_file) {
-		if (!is_json_file(bname)) {
-			pr_info("%s: Ignoring file without .json suffix %s\n", prog,
-				fpath);
-			return 0;
-		}
-	}
-
-	if (level > 1 && add_topic(bname))
-		return -ENOMEM;
-
-	/*
-	 * Assume all other files are JSON files.
-	 *
-	 * If mapfile refers to 'power7_core.json', we create a table
-	 * named 'power7_core'. Any inconsistencies between the mapfile
-	 * and directory tree could result in build failure due to table
-	 * names not being found.
-	 *
-	 * At least for now, be strict with processing JSON file names.
-	 * i.e. if JSON file name cannot be mapped to C-style table name,
-	 * fail.
-	 */
-	if (is_file) {
-		struct perf_entry_data data = {
-			.topic = get_topic(),
-			.outfp = eventsfp,
-		};
-
-		err = json_events(fpath, print_events_table_entry, &data);
-
-		free(data.topic);
-	}
-
-	return err;
-}
-
-#ifndef PATH_MAX
-#define PATH_MAX	4096
-#endif
-
-/*
- * Starting in directory 'start_dirname', find the "mapfile.csv" and
- * the set of JSON files for the architecture 'arch'.
- *
- * From each JSON file, create a C-style "PMU events table" from the
- * JSON file (see struct pmu_event).
- *
- * From the mapfile, create a mapping between the CPU revisions and
- * PMU event tables (see struct pmu_events_map).
- *
- * Write out the PMU events tables and the mapping table to pmu-event.c.
- */
-int main(int argc, char *argv[])
-{
-	int rc, ret = 0, empty_map = 0;
-	int maxfds;
-	char ldirname[PATH_MAX];
-	const char *arch;
-	const char *output_file;
-	const char *start_dirname;
-	const char *err_string_ext = "";
-	struct stat stbuf;
-
-	prog = basename(argv[0]);
-	if (argc < 4) {
-		pr_err("Usage: %s <arch> <starting_dir> <output_file>\n", prog);
-		return 1;
-	}
-
-	arch = argv[1];
-	start_dirname = argv[2];
-	output_file = argv[3];
-
-	if (argc > 4)
-		verbose = atoi(argv[4]);
-
-	eventsfp = fopen(output_file, "w");
-	if (!eventsfp) {
-		pr_err("%s Unable to create required file %s (%s)\n",
-				prog, output_file, strerror(errno));
-		return 2;
-	}
-
-	sprintf(ldirname, "%s/%s", start_dirname, arch);
-
-	/* If architecture does not have any event lists, bail out */
-	if (stat(ldirname, &stbuf) < 0) {
-		pr_info("%s: Arch %s has no PMU event lists\n", prog, arch);
-		empty_map = 1;
-		goto err_close_eventsfp;
-	}
-
-	/* Include pmu-events.h first */
-	fprintf(eventsfp, "#include \"pmu-events/pmu-events.h\"\n");
-
-	/*
-	 * The mapfile allows multiple CPUids to point to the same JSON file,
-	 * so, not sure if there is a need for symlinks within the pmu-events
-	 * directory.
-	 *
-	 * For now, treat symlinks of JSON files as regular files and create
-	 * separate tables for each symlink (presumably, each symlink refers
-	 * to specific version of the CPU).
-	 */
-
-	maxfds = get_maxfds();
-	rc = nftw(ldirname, preprocess_arch_std_files, maxfds, 0);
-	if (rc)
-		goto err_processing_std_arch_event_dir;
-
-	rc = nftw(ldirname, process_one_file, maxfds, 0);
-	if (rc)
-		goto err_processing_dir;
-
-	sprintf(ldirname, "%s/test", start_dirname);
-
-	rc = nftw(ldirname, preprocess_arch_std_files, maxfds, 0);
-	if (rc)
-		goto err_processing_std_arch_event_dir;
-
-	rc = nftw(ldirname, process_one_file, maxfds, 0);
-	if (rc)
-		goto err_processing_dir;
-
-	if (close_table)
-		print_events_table_suffix(eventsfp);
-
-	if (!mapfile) {
-		pr_info("%s: No CPU->JSON mapping?\n", prog);
-		empty_map = 1;
-		goto err_close_eventsfp;
-	}
-
-	rc = process_mapfile(eventsfp, mapfile);
-	if (rc) {
-		pr_info("%s: Error processing mapfile %s\n", prog, mapfile);
-		/* Make build fail */
-		ret = 1;
-		goto err_close_eventsfp;
-	}
-
-	rc = process_system_event_tables(eventsfp);
-	fclose(eventsfp);
-	if (rc) {
-		ret = 1;
-		goto err_out;
-	}
-
-	free_arch_std_events();
-	free_sys_event_tables();
-	free(mapfile);
-	return 0;
-
-err_processing_std_arch_event_dir:
-	err_string_ext = " for std arch event";
-err_processing_dir:
-	if (verbose) {
-		pr_info("%s: Error walking file tree %s%s\n", prog, ldirname,
-			err_string_ext);
-		empty_map = 1;
-	} else if (rc < 0) {
-		ret = 1;
-	} else {
-		empty_map = 1;
-	}
-err_close_eventsfp:
-	fclose(eventsfp);
-	if (empty_map)
-		create_empty_mapping(output_file);
-err_out:
-	free_arch_std_events();
-	free_sys_event_tables();
-	free(mapfile);
-	return ret;
-}
diff --git a/tools/perf/pmu-events/jsmn.h b/tools/perf/pmu-events/jsmn.h
deleted file mode 100644
index 1bdfd55fff30..000000000000
--- a/tools/perf/pmu-events/jsmn.h
+++ /dev/null
@@ -1,68 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-#ifndef __JSMN_H_
-#define __JSMN_H_
-
-/*
- * JSON type identifier. Basic types are:
- *	o Object
- *	o Array
- *	o String
- *	o Other primitive: number, boolean (true/false) or null
- */
-typedef enum {
-	JSMN_PRIMITIVE = 0,
-	JSMN_OBJECT = 1,
-	JSMN_ARRAY = 2,
-	JSMN_STRING = 3
-} jsmntype_t;
-
-typedef enum {
-	/* Not enough tokens were provided */
-	JSMN_ERROR_NOMEM = -1,
-	/* Invalid character inside JSON string */
-	JSMN_ERROR_INVAL = -2,
-	/* The string is not a full JSON packet, more bytes expected */
-	JSMN_ERROR_PART = -3,
-	/* Everything was fine */
-	JSMN_SUCCESS = 0
-} jsmnerr_t;
-
-/*
- * JSON token description.
- * @param		type	type (object, array, string etc.)
- * @param		start	start position in JSON data string
- * @param		end		end position in JSON data string
- */
-typedef struct {
-	jsmntype_t type;
-	int start;
-	int end;
-	int size;
-} jsmntok_t;
-
-/*
- * JSON parser. Contains an array of token blocks available. Also stores
- * the string being parsed now and current position in that string
- */
-typedef struct {
-	unsigned int pos; /* offset in the JSON string */
-	int toknext; /* next token to allocate */
-	int toksuper; /* superior token node, e.g parent object or array */
-} jsmn_parser;
-
-/*
- * Create JSON parser over an array of tokens
- */
-void jsmn_init(jsmn_parser *parser);
-
-/*
- * Run JSON parser. It parses a JSON data string into and array of tokens,
- * each describing a single JSON object.
- */
-jsmnerr_t jsmn_parse(jsmn_parser *parser, const char *js,
-		     size_t len,
-		     jsmntok_t *tokens, unsigned int num_tokens);
-
-const char *jsmn_strerror(jsmnerr_t err);
-
-#endif /* __JSMN_H_ */
diff --git a/tools/perf/pmu-events/json.c b/tools/perf/pmu-events/json.c
deleted file mode 100644
index 0544398d6e2d..000000000000
--- a/tools/perf/pmu-events/json.c
+++ /dev/null
@@ -1,162 +0,0 @@
-/* Parse JSON files using the JSMN parser. */
-
-/*
- * Copyright (c) 2014, Intel Corporation
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- * 1. Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimer.
- *
- * 2. Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
- * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
- * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
- * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
- * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
- * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
- * OF THE POSSIBILITY OF SUCH DAMAGE.
-*/
-
-#include <stdlib.h>
-#include <string.h>
-#include <sys/mman.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <stdio.h>
-#include <errno.h>
-#include <unistd.h>
-#include "jsmn.h"
-#include "json.h"
-#include <linux/kernel.h>
-
-
-static char *mapfile(const char *fn, size_t *size)
-{
-	unsigned ps = sysconf(_SC_PAGESIZE);
-	struct stat st;
-	char *map = NULL;
-	int err;
-	int fd = open(fn, O_RDONLY);
-
-	if (fd < 0 && verbose > 0 && fn) {
-		pr_err("Error opening events file '%s': %s\n", fn,
-				strerror(errno));
-	}
-
-	if (fd < 0)
-		return NULL;
-	err = fstat(fd, &st);
-	if (err < 0)
-		goto out;
-	*size = st.st_size;
-	map = mmap(NULL,
-		   (st.st_size + ps - 1) & ~(ps - 1),
-		   PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0);
-	if (map == MAP_FAILED)
-		map = NULL;
-out:
-	close(fd);
-	return map;
-}
-
-static void unmapfile(char *map, size_t size)
-{
-	unsigned ps = sysconf(_SC_PAGESIZE);
-	munmap(map, roundup(size, ps));
-}
-
-/*
- * Parse json file using jsmn. Return array of tokens,
- * and mapped file. Caller needs to free array.
- */
-jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len)
-{
-	jsmn_parser parser;
-	jsmntok_t *tokens;
-	jsmnerr_t res;
-	unsigned sz;
-
-	*map = mapfile(fn, size);
-	if (!*map)
-		return NULL;
-	/* Heuristic */
-	sz = *size * 16;
-	tokens = malloc(sz);
-	if (!tokens)
-		goto error;
-	jsmn_init(&parser);
-	res = jsmn_parse(&parser, *map, *size, tokens,
-			 sz / sizeof(jsmntok_t));
-	if (res != JSMN_SUCCESS) {
-		pr_err("%s: json error %s\n", fn, jsmn_strerror(res));
-		goto error_free;
-	}
-	if (len)
-		*len = parser.toknext;
-	return tokens;
-error_free:
-	free(tokens);
-error:
-	unmapfile(*map, *size);
-	return NULL;
-}
-
-void free_json(char *map, size_t size, jsmntok_t *tokens)
-{
-	free(tokens);
-	unmapfile(map, size);
-}
-
-static int countchar(char *map, char c, int end)
-{
-	int i;
-	int count = 0;
-	for (i = 0; i < end; i++)
-		if (map[i] == c)
-			count++;
-	return count;
-}
-
-/* Return line number of a jsmn token */
-int json_line(char *map, jsmntok_t *t)
-{
-	return countchar(map, '\n', t->start) + 1;
-}
-
-static const char * const jsmn_types[] = {
-	[JSMN_PRIMITIVE] = "primitive",
-	[JSMN_ARRAY] = "array",
-	[JSMN_OBJECT] = "object",
-	[JSMN_STRING] = "string"
-};
-
-#define LOOKUP(a, i) ((i) < (sizeof(a)/sizeof(*(a))) ? ((a)[i]) : "?")
-
-/* Return type name of a jsmn token */
-const char *json_name(jsmntok_t *t)
-{
-	return LOOKUP(jsmn_types, t->type);
-}
-
-int json_len(jsmntok_t *t)
-{
-	return t->end - t->start;
-}
-
-/* Is string t equal to s? */
-int json_streq(char *map, jsmntok_t *t, const char *s)
-{
-	unsigned len = json_len(t);
-	return len == strlen(s) && !strncasecmp(map + t->start, s, len);
-}
diff --git a/tools/perf/pmu-events/json.h b/tools/perf/pmu-events/json.h
deleted file mode 100644
index fbcd5a0590ad..000000000000
--- a/tools/perf/pmu-events/json.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef JSON_H
-#define JSON_H 1
-
-#include "jsmn.h"
-
-jsmntok_t *parse_json(const char *fn, char **map, size_t *size, int *len);
-void free_json(char *map, size_t size, jsmntok_t *tokens);
-int json_line(char *map, jsmntok_t *t);
-const char *json_name(jsmntok_t *t);
-int json_streq(char *map, jsmntok_t *t, const char *s);
-int json_len(jsmntok_t *t);
-
-extern int verbose;
-
-#include <stdbool.h>
-
-extern int eprintf(int level, int var, const char *fmt, ...);
-#define pr_fmt(fmt)	fmt
-
-#define pr_err(fmt, ...) \
-	eprintf(0, verbose, pr_fmt(fmt), ##__VA_ARGS__)
-
-#define pr_info(fmt, ...) \
-	eprintf(1, verbose, pr_fmt(fmt), ##__VA_ARGS__)
-
-#define pr_debug(fmt, ...) \
-	eprintf(2, verbose, pr_fmt(fmt), ##__VA_ARGS__)
-
-#ifndef roundup
-#define roundup(x, y) (                                \
-{                                                      \
-        const typeof(y) __y = y;                       \
-        (((x) + (__y - 1)) / __y) * __y;               \
-}                                                      \
-)
-#endif
-
-#endif
-- 
2.36.0.512.ge40c2bad7a-goog


^ permalink raw reply related	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 5/7] perf jevents: Add python converter script
  2022-05-11 21:15 ` [PATCH v2 5/7] perf jevents: Add python converter script Ian Rogers
@ 2022-05-11 21:47   ` Andi Kleen
  2022-05-12  1:15     ` Ian Rogers
  0 siblings, 1 reply; 19+ messages in thread
From: Andi Kleen @ 2022-05-11 21:47 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, John Garry, Kan Liang, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian


On 5/11/2022 2:15 PM, Ian Rogers wrote:
> jevents.c is large, has a dependency on an old forked version of jsmn,
> and is challenging to work upon. A lot of jevents.c's complexity comes
> from needing to write json and csv parsing from first principles. In
> contrast python has this functionality in standard libraries and is
> already a build pre-requisite for tools like asciidoc (that builds all
> of the perf man pages).


Well I was still hoping at some point we could have run time loading of 
events. But yes if that's not

happening then this probably the right approach.

-Andi



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 5/7] perf jevents: Add python converter script
  2022-05-11 21:47   ` Andi Kleen
@ 2022-05-12  1:15     ` Ian Rogers
  0 siblings, 0 replies; 19+ messages in thread
From: Ian Rogers @ 2022-05-12  1:15 UTC (permalink / raw)
  To: Andi Kleen
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	John Garry, Kan Liang, Zhengjun Xing, Felix Fietkau, Qi Liu,
	Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar, Stephane Eranian

On Wed, May 11, 2022 at 2:47 PM Andi Kleen <ak@linux.intel.com> wrote:
>
>
> On 5/11/2022 2:15 PM, Ian Rogers wrote:
> > jevents.c is large, has a dependency on an old forked version of jsmn,
> > and is challenging to work upon. A lot of jevents.c's complexity comes
> > from needing to write json and csv parsing from first principles. In
> > contrast python has this functionality in standard libraries and is
> > already a build pre-requisite for tools like asciidoc (that builds all
> > of the perf man pages).
>
>
> Well I was still hoping at some point we could have run time loading of
> events. But yes if that's not
>
> happening then this probably the right approach.

Thanks Andi, some thinking out loud below:

The perf tool already discovers events from sysfs, so could we have
runtime loading by turning the sysfs mount point into a list of
directories?

I'm thinking that for consistency the jevents API inside of the perf
code should look more like sysfs. There's then a question of how we
resolve events, perhaps we can have a config value something like
"sysfs,runtime=/home/me/my-events,jevents" where we treat the list as
setting up something like a unionfs in the tool. So if you ask for
instructions.retired, the tool would first check sysfs, then any
runtime events specified sysfs like directories, then jevents (or a
different order if you set a different config value). If the event is
defined more than once for the same PMU then the ordering says which
one has priority. If a PMU isn't specified for an event then we search
all PMUs and for jevents you'd only see PMUs relevant to the cpuid(s).

I'm hoping something like this will allow hybrid to work without all
the "if...hybrid" code that's getting added, and with a behavior
that's obvious and will allow us to debug errors. Maybe a runtime
sysfs like directory implies too many files so we could make a json
file to replace that directory, or use a zip file, etc. I'd prefer it
if the mappings were as 1-to-1 as possible (ie have a new json format
whose structure matches sysfs) and so I wouldn't like to use the
current jevents.c code for it.

Thanks,
Ian

> -Andi
>
>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 6/7] perf jevents: Switch build to use jevents.py
  2022-05-11 21:15 ` [PATCH v2 6/7] perf jevents: Switch build to use jevents.py Ian Rogers
@ 2022-05-13 15:37   ` John Garry
  2022-05-13 15:58     ` Ian Rogers
  0 siblings, 1 reply; 19+ messages in thread
From: John Garry @ 2022-05-13 15:37 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Kan Liang, Andi Kleen, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian

On 11/05/2022 22:15, Ian Rogers wrote:
>    # jevents.py uses os.scandir and type hints present in Python 3.5 released in Sept. 2015.
> +    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 5) else "0")')

I think that many - like me - will have python 2.7, so now will find no 
pmu-events generated any longer after missing this message in the build :(

Maybe many will have python >= 3.5 - but I don't know...

 > + ifneq ($(JEVENTS_PYTHON_GOOD), 1)
 > + $(warning Python interpreter too old (older than 3.5) disabling 
jevent generation)
 > + NO_JEVENTS := 1

It is possible to flip NO_JEVENTS to be JEVENTS, i.e. no 
double-negatives, like NO_JEVENTS := 0

 > + endif

Thanks,
John

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 6/7] perf jevents: Switch build to use jevents.py
  2022-05-13 15:37   ` John Garry
@ 2022-05-13 15:58     ` Ian Rogers
  2022-05-17 10:32       ` John Garry
  0 siblings, 1 reply; 19+ messages in thread
From: Ian Rogers @ 2022-05-13 15:58 UTC (permalink / raw)
  To: John Garry
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau, Qi Liu,
	Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar, Stephane Eranian

On Fri, May 13, 2022 at 8:38 AM John Garry <john.garry@huawei.com> wrote:
>
> On 11/05/2022 22:15, Ian Rogers wrote:
> >    # jevents.py uses os.scandir and type hints present in Python 3.5 released in Sept. 2015.
> > +    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 5) else "0")')
>
> I think that many - like me - will have python 2.7, so now will find no
> pmu-events generated any longer after missing this message in the build :(
>
> Maybe many will have python >= 3.5 - but I don't know...

So Python 2 has been end-of-life for over 2 years now:
https://www.python.org/doc/sunset-python-2/
There have been a number of LKML patches upgrading python to version 3.

Python 3.5 has some very nice features of os.scandir and type hints,
so if I set the bar lower than this it hurts the code quality. It is
also at least 6 years old at this point, and so hopefully not
unreasonable for a distribution to have picked it up :-) Looking at
the change to C11 thread:
https://lore.kernel.org/lkml/20220228103142.3301082-1-arnd@kernel.org/
It seems the motivation for picking a language version is the features
it provides and compatibility. If we choose pre-Python 3.5 we get more
compatibility but we lose language features.

My feeling is that we shouldn't need to support things that are no
longer maintained (like Python 2) but I'm less clear if Python 3.5 is
sufficiently compatible for everyone's needs. I kind of hope so, hence
making the patches this way.

>  > + ifneq ($(JEVENTS_PYTHON_GOOD), 1)
>  > + $(warning Python interpreter too old (older than 3.5) disabling
> jevent generation)
>  > + NO_JEVENTS := 1
>
> It is possible to flip NO_JEVENTS to be JEVENTS, i.e. no
> double-negatives, like NO_JEVENTS := 0

Agreed that double negatives are bad. The NO_... pattern is kind of
throughout the make files and build files. I preferred the NO_... for
consistency but if there's a consensus I'm happy to change.

Thanks,
Ian

>  > + endif
>
> Thanks,
> John

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 6/7] perf jevents: Switch build to use jevents.py
  2022-05-13 15:58     ` Ian Rogers
@ 2022-05-17 10:32       ` John Garry
  2022-05-18  4:37         ` Ian Rogers
  0 siblings, 1 reply; 19+ messages in thread
From: John Garry @ 2022-05-17 10:32 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau, Qi Liu,
	Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar, Stephane Eranian

On 13/05/2022 16:58, Ian Rogers wrote:
> On Fri, May 13, 2022 at 8:38 AM John Garry <john.garry@huawei.com> wrote:
>>
>> On 11/05/2022 22:15, Ian Rogers wrote:
>>>     # jevents.py uses os.scandir and type hints present in Python 3.5 released in Sept. 2015.
>>> +    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 5) else "0")')
>>
>> I think that many - like me - will have python 2.7, so now will find no
>> pmu-events generated any longer after missing this message in the build :(
>>
>> Maybe many will have python >= 3.5 - but I don't know...
> 
> So Python 2 has been end-of-life for over 2 years now:
> https://www.python.org/doc/sunset-python-2/
> There have been a number of LKML patches upgrading python to version 3.
> 
> Python 3.5 has some very nice features of os.scandir and type hints,
> so if I set the bar lower than this it hurts the code quality. It is
> also at least 6 years old at this point, and so hopefully not
> unreasonable for a distribution to have picked it up :-) Looking at
> the change to C11 thread:
> https://lore.kernel.org/lkml/20220228103142.3301082-1-arnd@kernel.org/
> It seems the motivation for picking a language version is the features
> it provides and compatibility. If we choose pre-Python 3.5 we get more
> compatibility but we lose language features.
> 
> My feeling is that we shouldn't need to support things that are no
> longer maintained (like Python 2) but I'm less clear if Python 3.5 is
> sufficiently compatible for everyone's needs. I kind of hope so, hence
> making the patches this way.

Fine, I just think that you need to make this transition as seamless as 
possible, otherwise it can be judged as a regression.

For example, I have now python 3.6 (default) and 2.7 but it still 
doesn't seem to work:

john@localhost:~/acme/tools/perf> make
   BUILD:   Doing 'make -j4' parallel build
Warning: Kernel ABI header at 'tools/include/linux/coresight-pmu.h' 
differs from latest version at 'include/linux/coresight-pmu.h'
diff -u tools/include/linux/coresight-pmu.h include/linux/coresight-pmu.h
Makefile.config:593: No sys/sdt.h found, no SDT events are defined, 
please install systemtap-sdt-devel or systemtap-sdt-dev
Makefile.config:871: Python interpreter too old (older than 3.5) 
disabling jevent generation
Makefile.config:904: Old version of libbfd/binutils things like PE 
executable profiling will not be available
Makefile.config:1092: No openjdk development package found, please 
install JDK package, e.g. openjdk-8-jdk, java-1.8.0-openjdk-devel

Auto-detecting system features:
...                         dwarf: [ on  ]
...            dwarf_getlocations: [ on  ]
...                         glibc: [ on  ]
...                        libbfd: [ OFF ]
...                libbfd-buildid: [ OFF ]
...                        libcap: [ on  ]
...                        libelf: [ on  ]
...                       libnuma: [ on  ]
...        numa_num_possible_cpus: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...                     libcrypto: [ on  ]
...                     libunwind: [ on  ]
...            libdw-dwarf-unwind: [ on  ]
...                          zlib: [ on  ]
...                          lzma: [ on  ]
...                     get_cpuid: [ on  ]
...                           bpf: [ on  ]
...                        libaio: [ on  ]
...                       libzstd: [ on  ]
...        disassembler-four-args: [ on  ]


make[3]: Nothing to be done for 'install_headers'.
john@localhost:~/acme/tools/perf> python --version
Python 3.6.12
john@localhost:~/acme/tools/perf>

which I need to figure out...

> 
>>   > + ifneq ($(JEVENTS_PYTHON_GOOD), 1)
>>   > + $(warning Python interpreter too old (older than 3.5) disabling
>> jevent generation)
>>   > + NO_JEVENTS := 1
>>
>> It is possible to flip NO_JEVENTS to be JEVENTS, i.e. no
>> double-negatives, like NO_JEVENTS := 0
> 
> Agreed that double negatives are bad. The NO_... pattern is kind of
> throughout the make files and build files. I preferred the NO_... for
> consistency but if there's a consensus I'm happy to change.
> 

I have no strong preference. I just find that double negatives boggle 
the mind.

Thanks,
john

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 1/7] perf jevents: Append PMU description later
  2022-05-11 21:15 ` [PATCH v2 1/7] perf jevents: Append PMU description later Ian Rogers
@ 2022-05-17 13:28   ` John Garry
  0 siblings, 0 replies; 19+ messages in thread
From: John Garry @ 2022-05-17 13:28 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Kan Liang, Andi Kleen, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian

On 11/05/2022 22:15, Ian Rogers wrote:
> Append the PMU information from "Unit" to the description later. This
> avoids a problem when "Unit" appears early in a json event and the
> information prepends the description rather than being the expected
> suffix.
> 
> Update the pmu-events test so that expectations now match the improved
> output.
> 
> Signed-off-by: Ian Rogers<irogers@google.com>

Reviewed-by: John Garry <john.garry@huawei.com>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups
  2022-05-11 21:15 ` [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups Ian Rogers
@ 2022-05-17 13:30   ` John Garry
  0 siblings, 0 replies; 19+ messages in thread
From: John Garry @ 2022-05-17 13:30 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Kan Liang, Andi Kleen, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian

On 11/05/2022 22:15, Ian Rogers wrote:
> Remove unnecessary empty groups.
> 
> Signed-off-by: Ian Rogers<irogers@google.com>

I don't know the background as to why we had it like this but the change 
itself looks ok:
Reviewed-by: John Garry <john.garry@huawei.com>


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask
  2022-05-11 21:15 ` [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask Ian Rogers
@ 2022-05-17 13:31   ` John Garry
  0 siblings, 0 replies; 19+ messages in thread
From: John Garry @ 2022-05-17 13:31 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Kan Liang, Andi Kleen, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian

On 11/05/2022 22:15, Ian Rogers wrote:
> The event had two umasks with the umask of 3 being correct.
> Note: this change wasn't automatically generated as there is no CSV for
> Ivytown uncore events at:
> https://github.com/intel/event-converter-for-linux-perf
> 
> Signed-off-by: Ian Rogers<irogers@google.com>

Reviewed-by: John Garry <john.garry@huawei.com>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 4/7] perf jevents: Modify match field
  2022-05-11 21:15 ` [PATCH v2 4/7] perf jevents: Modify match field Ian Rogers
@ 2022-05-17 13:35   ` John Garry
  2022-05-23 13:09     ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 19+ messages in thread
From: John Garry @ 2022-05-17 13:35 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, Kan Liang, Andi Kleen, Zhengjun Xing,
	Felix Fietkau, Qi Liu, Like Xu, linux-kernel, linux-perf-users,
	Nick Forrington, Kajol Jain, James Clark, Andrew Kilroy,
	Paul A . Clarke, Will Deacon, Mathieu Poirier, ananth.narayan,
	ravi.bangoria, santosh.shukla, sandipan.das, Caleb Biggers,
	Perry Taylor, Kshipra Bopardikar
  Cc: Stephane Eranian

On 11/05/2022 22:15, Ian Rogers wrote:
> The match_field function looks for json values to append to the event
> string. As the C code processes these in order the output order matches
> that in the json dictionary. Python json readers read the entire
> dictionary and lose the ordering. To make the python and C output
> comparable make the C code first read the extra fields then append them
> to the event in an order not determined by their order in the file.
> 
> Modify the pmu-events test so that test expectations match the new
> order.
> 
> Signed-off-by: Ian Rogers<irogers@google.com>

Reviewed-by: John Garry <john.garry@huawei.com>

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 6/7] perf jevents: Switch build to use jevents.py
  2022-05-17 10:32       ` John Garry
@ 2022-05-18  4:37         ` Ian Rogers
  0 siblings, 0 replies; 19+ messages in thread
From: Ian Rogers @ 2022-05-18  4:37 UTC (permalink / raw)
  To: John Garry
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	Kan Liang, Andi Kleen, Zhengjun Xing, Felix Fietkau, Qi Liu,
	Like Xu, linux-kernel, linux-perf-users, Nick Forrington,
	Kajol Jain, James Clark, Andrew Kilroy, Paul A . Clarke,
	Will Deacon, Mathieu Poirier, ananth.narayan, ravi.bangoria,
	santosh.shukla, sandipan.das, Caleb Biggers, Perry Taylor,
	Kshipra Bopardikar, Stephane Eranian

On Tue, May 17, 2022 at 3:32 AM John Garry <john.garry@huawei.com> wrote:
>
> On 13/05/2022 16:58, Ian Rogers wrote:
> > On Fri, May 13, 2022 at 8:38 AM John Garry <john.garry@huawei.com> wrote:
> >>
> >> On 11/05/2022 22:15, Ian Rogers wrote:
> >>>     # jevents.py uses os.scandir and type hints present in Python 3.5 released in Sept. 2015.
> >>> +    JEVENTS_PYTHON_GOOD := $(shell $(PYTHON) -c 'import sys;print("1" if(sys.version_info.major >= 3 and sys.version_info.minor >= 5) else "0")')
> >>
> >> I think that many - like me - will have python 2.7, so now will find no
> >> pmu-events generated any longer after missing this message in the build :(
> >>
> >> Maybe many will have python >= 3.5 - but I don't know...
> >
> > So Python 2 has been end-of-life for over 2 years now:
> > https://www.python.org/doc/sunset-python-2/
> > There have been a number of LKML patches upgrading python to version 3.
> >
> > Python 3.5 has some very nice features of os.scandir and type hints,
> > so if I set the bar lower than this it hurts the code quality. It is
> > also at least 6 years old at this point, and so hopefully not
> > unreasonable for a distribution to have picked it up :-) Looking at
> > the change to C11 thread:
> > https://lore.kernel.org/lkml/20220228103142.3301082-1-arnd@kernel.org/
> > It seems the motivation for picking a language version is the features
> > it provides and compatibility. If we choose pre-Python 3.5 we get more
> > compatibility but we lose language features.
> >
> > My feeling is that we shouldn't need to support things that are no
> > longer maintained (like Python 2) but I'm less clear if Python 3.5 is
> > sufficiently compatible for everyone's needs. I kind of hope so, hence
> > making the patches this way.
>
> Fine, I just think that you need to make this transition as seamless as
> possible, otherwise it can be judged as a regression.

Agreed.

> For example, I have now python 3.6 (default) and 2.7 but it still
> doesn't seem to work:
>
> john@localhost:~/acme/tools/perf> make
>    BUILD:   Doing 'make -j4' parallel build
> Warning: Kernel ABI header at 'tools/include/linux/coresight-pmu.h'
> differs from latest version at 'include/linux/coresight-pmu.h'
> diff -u tools/include/linux/coresight-pmu.h include/linux/coresight-pmu.h
> Makefile.config:593: No sys/sdt.h found, no SDT events are defined,
> please install systemtap-sdt-devel or systemtap-sdt-dev
> Makefile.config:871: Python interpreter too old (older than 3.5)
> disabling jevent generation
> Makefile.config:904: Old version of libbfd/binutils things like PE
> executable profiling will not be available
> Makefile.config:1092: No openjdk development package found, please
> install JDK package, e.g. openjdk-8-jdk, java-1.8.0-openjdk-devel
>
> Auto-detecting system features:
> ...                         dwarf: [ on  ]
> ...            dwarf_getlocations: [ on  ]
> ...                         glibc: [ on  ]
> ...                        libbfd: [ OFF ]
> ...                libbfd-buildid: [ OFF ]
> ...                        libcap: [ on  ]
> ...                        libelf: [ on  ]
> ...                       libnuma: [ on  ]
> ...        numa_num_possible_cpus: [ on  ]
> ...                       libperl: [ on  ]
> ...                     libpython: [ on  ]
> ...                     libcrypto: [ on  ]
> ...                     libunwind: [ on  ]
> ...            libdw-dwarf-unwind: [ on  ]
> ...                          zlib: [ on  ]
> ...                          lzma: [ on  ]
> ...                     get_cpuid: [ on  ]
> ...                           bpf: [ on  ]
> ...                        libaio: [ on  ]
> ...                       libzstd: [ on  ]
> ...        disassembler-four-args: [ on  ]
>
>
> make[3]: Nothing to be done for 'install_headers'.
> john@localhost:~/acme/tools/perf> python --version
> Python 3.6.12
> john@localhost:~/acme/tools/perf>
>
> which I need to figure out...

Yep, I can't explain that :-) You could try something like adding a
$(warning ... or similar to the build and running:
python3 -c 'import sys; print(sys.version_info)'

the jevents.py uses f-strings and so I'll need to bump the version
from 3.5 to 3.6 in v2. Python 3.6 was released in December 2016.

Thanks,
Ian


> >
> >>   > + ifneq ($(JEVENTS_PYTHON_GOOD), 1)
> >>   > + $(warning Python interpreter too old (older than 3.5) disabling
> >> jevent generation)
> >>   > + NO_JEVENTS := 1
> >>
> >> It is possible to flip NO_JEVENTS to be JEVENTS, i.e. no
> >> double-negatives, like NO_JEVENTS := 0
> >
> > Agreed that double negatives are bad. The NO_... pattern is kind of
> > throughout the make files and build files. I preferred the NO_... for
> > consistency but if there's a consensus I'm happy to change.
> >
>
> I have no strong preference. I just find that double negatives boggle
> the mind.
>
> Thanks,
> john

^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: [PATCH v2 4/7] perf jevents: Modify match field
  2022-05-17 13:35   ` John Garry
@ 2022-05-23 13:09     ` Arnaldo Carvalho de Melo
  0 siblings, 0 replies; 19+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-05-23 13:09 UTC (permalink / raw)
  To: John Garry
  Cc: Ian Rogers, Peter Zijlstra, Ingo Molnar, Mark Rutland,
	Alexander Shishkin, Jiri Olsa, Namhyung Kim, Kan Liang,
	Andi Kleen, Zhengjun Xing, Felix Fietkau, Qi Liu, Like Xu,
	linux-kernel, linux-perf-users, Nick Forrington, Kajol Jain,
	James Clark, Andrew Kilroy, Paul A . Clarke, Will Deacon,
	Mathieu Poirier, ananth.narayan, ravi.bangoria, santosh.shukla,
	sandipan.das, Caleb Biggers, Perry Taylor, Kshipra Bopardikar,
	Stephane Eranian

Em Tue, May 17, 2022 at 02:35:14PM +0100, John Garry escreveu:
> On 11/05/2022 22:15, Ian Rogers wrote:
> > The match_field function looks for json values to append to the event
> > string. As the C code processes these in order the output order matches
> > that in the json dictionary. Python json readers read the entire
> > dictionary and lose the ordering. To make the python and C output
> > comparable make the C code first read the extra fields then append them
> > to the event in an order not determined by their order in the file.
> > 
> > Modify the pmu-events test so that test expectations match the new
> > order.
> > 
> > Signed-off-by: Ian Rogers<irogers@google.com>
> 
> Reviewed-by: John Garry <john.garry@huawei.com>


Applied up to here, will take a closer look at 5-7

- Arnaldo

^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2022-05-23 13:09 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-11 21:15 [PATCH v2 0/7] Rewrite jevents program in python Ian Rogers
2022-05-11 21:15 ` [PATCH v2 1/7] perf jevents: Append PMU description later Ian Rogers
2022-05-17 13:28   ` John Garry
2022-05-11 21:15 ` [PATCH v2 2/7] perf vendor events: Fix Alderlake metric groups Ian Rogers
2022-05-17 13:30   ` John Garry
2022-05-11 21:15 ` [PATCH v2 3/7] perf vendor events: Fix Ivytown UNC_M_ACT_COUNT.RD umask Ian Rogers
2022-05-17 13:31   ` John Garry
2022-05-11 21:15 ` [PATCH v2 4/7] perf jevents: Modify match field Ian Rogers
2022-05-17 13:35   ` John Garry
2022-05-23 13:09     ` Arnaldo Carvalho de Melo
2022-05-11 21:15 ` [PATCH v2 5/7] perf jevents: Add python converter script Ian Rogers
2022-05-11 21:47   ` Andi Kleen
2022-05-12  1:15     ` Ian Rogers
2022-05-11 21:15 ` [PATCH v2 6/7] perf jevents: Switch build to use jevents.py Ian Rogers
2022-05-13 15:37   ` John Garry
2022-05-13 15:58     ` Ian Rogers
2022-05-17 10:32       ` John Garry
2022-05-18  4:37         ` Ian Rogers
2022-05-11 21:15 ` [PATCH v2 7/7] perf jevents: Remove jevents.c Ian Rogers

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).