[PATCH v9 00/13] Don't compute events that won't be used in a metric.

All of lore.kernel.org
 help / color / mirror / Atom feed

* [PATCH v9 00/13] Don't compute events that won't be used in a metric.
@ 2021-09-23  7:46 Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 01/13] perf metric: Restructure struct expr_parse_ctx Ian Rogers
                   ` (14 more replies)
  0 siblings, 15 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

For a metric like:
  EVENT1 if #smt_on else EVENT2

currently EVENT1 and EVENT2 will be measured and then when the metric
is reported EVENT1 or EVENT2 will be printed depending on the value
from smt_on() during the expr parsing. Computing both events is
unnecessary and can lead to multiplexing as discussed in this thread:
https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/

This change modifies expression parsing so that constants are
considered when building the set of ids (events) and only events not
contributing to a constant value are measured.

v9. adds a missing memory allocation failure check in the pmu-metrics
    test.  A memory allocation failure in union returns NULL. Some
    parser debug on IDs is removed. "Modify code layout" is broken
    apart into 3 changes.  "Don't compute unused events" is broken
    apart into 4 changes with the tests merged into the change that
    adds the corresponding optimization. This is trying to address
    feedback from Jiri Olsa <jolsa@redhat.com>.  The unmodified
    patches have Reviewed-by: Andi Kleen <ak@linux.intel.com> added,
    although the overall patch set is largely the same as v8 which was
    fully reviewed-by.

v8. rebases, adds an ability to compute metrics with no events and
    further breaks apart the "Don't compute unused events" part of the
    change as requested by Jiri Olsa <jolsa@redhat.com>.

v7. fixes the fix to be in the correct patch.

v6. rebases and fixes issues raised by Namhyung Kim <namhyung@kernel.org>,
a memory leak and a function comment.

v5. uses macros to reduce boiler plate in patch 5/5 as suggested by
Andi Kleen <ak@linux.intel.com>.

v4. reduces references to BOTTOM/NAN in patch 5/5 by using utility
functions. It improves comments and fixes an unnecessary union in a
peephole optimization.

v3. fixes an assignment in patch 2/5. In patch 5/5 additional comments
are added and useless frees are replaced by asserts. A new peephole
optimization is added for the case CONST IF expr ELSE CONST, where the
the constants are identical, as we don't need to evaluate the IF
condition.

v2. is a rebase.

Ian Rogers (13):
  perf metric: Restructure struct expr_parse_ctx.
  perf metric: Use NAN for missing event IDs.
  perf expr: Remove unused headers and inline d_ratio
  perf expr: Separate token declataion from type
  perf expr: Use macros for operators
  perf expr: Move actions to the left.
  perf metric: Rename expr__find_other.
  perf metric: Add utilities to work on ids map.
  perf metric: Allow metrics with no events
  perf expr: Merge find_ids and regular parsing
  perf expr: Propagate constants for binary operations
  perf metric: Don't compute unused events
  perf metric: Avoid events for an 'if' constant result

 tools/perf/tests/expr.c       | 160 ++++++++++++-----
 tools/perf/tests/pmu-events.c |  54 +++---
 tools/perf/util/expr.c        | 121 +++++++++++--
 tools/perf/util/expr.h        |  20 ++-
 tools/perf/util/expr.l        |   9 -
 tools/perf/util/expr.y        | 325 +++++++++++++++++++++++++---------
 tools/perf/util/metricgroup.c | 145 ++++++++-------
 tools/perf/util/stat-shadow.c |  54 +++---
 8 files changed, 620 insertions(+), 268 deletions(-)

-- 
2.33.0.464.g1972c5931b-goog

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH v9 01/13] perf metric: Restructure struct expr_parse_ctx.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 02/13] perf metric: Use NAN for missing event IDs Ian Rogers
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

A later change to parsing the ids out (in expr__find_other) will
potentially drop hashmaps and so it is more convenient to move
expr_parse_ctx to have a hashmap pointer rather than a struct value. As
this pointer must be freed, rather than just going out of scope,
add expr__ctx_new and expr__ctx_free to manage expr_parse_ctx memory.
Adjust use of struct expr_parse_ctx accordingly.

Reviewed-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c       | 81 ++++++++++++++++++-----------------
 tools/perf/tests/pmu-events.c | 47 ++++++++++++--------
 tools/perf/util/expr.c        | 39 +++++++++++++----
 tools/perf/util/expr.h        |  5 ++-
 tools/perf/util/metricgroup.c | 44 ++++++++++---------
 tools/perf/util/stat-shadow.c | 50 +++++++++++++--------
 6 files changed, 159 insertions(+), 107 deletions(-)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 4d01051951cd..b0a3b5fd0c00 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -22,67 +22,70 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
 	const char *p;
 	double val;
 	int ret;
-	struct expr_parse_ctx ctx;
+	struct expr_parse_ctx *ctx;
 
-	expr__ctx_init(&ctx);
-	expr__add_id_val(&ctx, strdup("FOO"), 1);
-	expr__add_id_val(&ctx, strdup("BAR"), 2);
+	ctx = expr__ctx_new();
+	TEST_ASSERT_VAL("expr__ctx_new", ctx);
+	expr__add_id_val(ctx, strdup("FOO"), 1);
+	expr__add_id_val(ctx, strdup("BAR"), 2);
 
-	ret = test(&ctx, "1+1", 2);
-	ret |= test(&ctx, "FOO+BAR", 3);
-	ret |= test(&ctx, "(BAR/2)%2", 1);
-	ret |= test(&ctx, "1 - -4",  5);
-	ret |= test(&ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
-	ret |= test(&ctx, "1-1 | 1", 1);
-	ret |= test(&ctx, "1-1 & 1", 0);
-	ret |= test(&ctx, "min(1,2) + 1", 2);
-	ret |= test(&ctx, "max(1,2) + 1", 3);
-	ret |= test(&ctx, "1+1 if 3*4 else 0", 2);
-	ret |= test(&ctx, "1.1 + 2.1", 3.2);
-	ret |= test(&ctx, ".1 + 2.", 2.1);
-	ret |= test(&ctx, "d_ratio(1, 2)", 0.5);
-	ret |= test(&ctx, "d_ratio(2.5, 0)", 0);
-	ret |= test(&ctx, "1.1 < 2.2", 1);
-	ret |= test(&ctx, "2.2 > 1.1", 1);
-	ret |= test(&ctx, "1.1 < 1.1", 0);
-	ret |= test(&ctx, "2.2 > 2.2", 0);
-	ret |= test(&ctx, "2.2 < 1.1", 0);
-	ret |= test(&ctx, "1.1 > 2.2", 0);
+	ret = test(ctx, "1+1", 2);
+	ret |= test(ctx, "FOO+BAR", 3);
+	ret |= test(ctx, "(BAR/2)%2", 1);
+	ret |= test(ctx, "1 - -4",  5);
+	ret |= test(ctx, "(FOO-1)*2 + (BAR/2)%2 - -4",  5);
+	ret |= test(ctx, "1-1 | 1", 1);
+	ret |= test(ctx, "1-1 & 1", 0);
+	ret |= test(ctx, "min(1,2) + 1", 2);
+	ret |= test(ctx, "max(1,2) + 1", 3);
+	ret |= test(ctx, "1+1 if 3*4 else 0", 2);
+	ret |= test(ctx, "1.1 + 2.1", 3.2);
+	ret |= test(ctx, ".1 + 2.", 2.1);
+	ret |= test(ctx, "d_ratio(1, 2)", 0.5);
+	ret |= test(ctx, "d_ratio(2.5, 0)", 0);
+	ret |= test(ctx, "1.1 < 2.2", 1);
+	ret |= test(ctx, "2.2 > 1.1", 1);
+	ret |= test(ctx, "1.1 < 1.1", 0);
+	ret |= test(ctx, "2.2 > 2.2", 0);
+	ret |= test(ctx, "2.2 < 1.1", 0);
+	ret |= test(ctx, "1.1 > 2.2", 0);
 
-	if (ret)
+	if (ret) {
+		expr__ctx_free(ctx);
 		return ret;
+	}
 
 	p = "FOO/0";
-	ret = expr__parse(&val, &ctx, p, 1);
+	ret = expr__parse(&val, ctx, p, 1);
 	TEST_ASSERT_VAL("division by zero", ret == -1);
 
 	p = "BAR/";
-	ret = expr__parse(&val, &ctx, p, 1);
+	ret = expr__parse(&val, ctx, p, 1);
 	TEST_ASSERT_VAL("missing operand", ret == -1);
 
-	expr__ctx_clear(&ctx);
+	expr__ctx_clear(ctx);
 	TEST_ASSERT_VAL("find other",
 			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO",
-					 &ctx, 1) == 0);
-	TEST_ASSERT_VAL("find other", hashmap__size(&ctx.ids) == 3);
-	TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BAR",
+					 ctx, 1) == 0);
+	TEST_ASSERT_VAL("find other", hashmap__size(ctx->ids) == 3);
+	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BAR",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BAZ",
+	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BAZ",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BOZO",
+	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BOZO",
 						    (void **)&val_ptr));
 
-	expr__ctx_clear(&ctx);
+	expr__ctx_clear(ctx);
 	TEST_ASSERT_VAL("find other",
 			expr__find_other("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@",
-					 NULL, &ctx, 3) == 0);
-	TEST_ASSERT_VAL("find other", hashmap__size(&ctx.ids) == 2);
-	TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "EVENT1,param=3/",
+					 NULL, ctx, 3) == 0);
+	TEST_ASSERT_VAL("find other", hashmap__size(ctx->ids) == 2);
+	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "EVENT1,param=3/",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "EVENT2,param=3/",
+	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "EVENT2,param=3/",
 						    (void **)&val_ptr));
 
-	expr__ctx_clear(&ctx);
+	expr__ctx_free(ctx);
 
 	return 0;
 }
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index 43743cf719ef..ff3b2e5e6141 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -781,7 +781,7 @@ static int resolve_metric_simple(struct expr_parse_ctx *pctx,
 
 	do {
 		all = true;
-		hashmap__for_each_entry_safe((&pctx->ids), cur, cur_tmp, bkt) {
+		hashmap__for_each_entry_safe(pctx->ids, cur, cur_tmp, bkt) {
 			struct metric_ref *ref;
 			struct pmu_event *pe;
 
@@ -835,9 +835,14 @@ static int test_parsing(void)
 	struct pmu_event *pe;
 	int i, j, k;
 	int ret = 0;
-	struct expr_parse_ctx ctx;
+	struct expr_parse_ctx *ctx;
 	double result;
 
+	ctx = expr__ctx_new();
+	if (!ctx) {
+		pr_debug("expr__ctx_new failed");
+		return TEST_FAIL;
+	}
 	i = 0;
 	for (;;) {
 		map = &pmu_events_map[i++];
@@ -855,15 +860,15 @@ static int test_parsing(void)
 				break;
 			if (!pe->metric_expr)
 				continue;
-			expr__ctx_init(&ctx);
-			if (expr__find_other(pe->metric_expr, NULL, &ctx, 0)
+			expr__ctx_clear(ctx);
+			if (expr__find_other(pe->metric_expr, NULL, ctx, 0)
 				  < 0) {
 				expr_failure("Parse other failed", map, pe);
 				ret++;
 				continue;
 			}
 
-			if (resolve_metric_simple(&ctx, &compound_list, map,
+			if (resolve_metric_simple(ctx, &compound_list, map,
 						  pe->metric_name)) {
 				expr_failure("Could not resolve metrics", map, pe);
 				ret++;
@@ -876,27 +881,27 @@ static int test_parsing(void)
 			 * make them unique.
 			 */
 			k = 1;
-			hashmap__for_each_entry((&ctx.ids), cur, bkt)
-				expr__add_id_val(&ctx, strdup(cur->key), k++);
+			hashmap__for_each_entry(ctx->ids, cur, bkt)
+				expr__add_id_val(ctx, strdup(cur->key), k++);
 
-			hashmap__for_each_entry((&ctx.ids), cur, bkt) {
+			hashmap__for_each_entry(ctx->ids, cur, bkt) {
 				if (check_parse_cpu(cur->key, map == cpus_map,
 						   pe))
 					ret++;
 			}
 
 			list_for_each_entry_safe(metric, tmp, &compound_list, list) {
-				expr__add_ref(&ctx, &metric->metric_ref);
+				expr__add_ref(ctx, &metric->metric_ref);
 				free(metric);
 			}
 
-			if (expr__parse(&result, &ctx, pe->metric_expr, 0)) {
+			if (expr__parse(&result, ctx, pe->metric_expr, 0)) {
 				expr_failure("Parse failed", map, pe);
 				ret++;
 			}
-			expr__ctx_clear(&ctx);
 		}
 	}
+	expr__ctx_free(ctx);
 	/* TODO: fail when not ok */
 exit:
 	return ret == 0 ? TEST_OK : TEST_SKIP;
@@ -916,7 +921,7 @@ static struct test_metric metrics[] = {
 
 static int metric_parse_fake(const char *str)
 {
-	struct expr_parse_ctx ctx;
+	struct expr_parse_ctx *ctx;
 	struct hashmap_entry *cur;
 	double result;
 	int ret = -1;
@@ -925,8 +930,12 @@ static int metric_parse_fake(const char *str)
 
 	pr_debug("parsing '%s'\n", str);
 
-	expr__ctx_init(&ctx);
-	if (expr__find_other(str, NULL, &ctx, 0) < 0) {
+	ctx = expr__ctx_new();
+	if (!ctx) {
+		pr_debug("expr__ctx_new failed");
+		return TEST_FAIL;
+	}
+	if (expr__find_other(str, NULL, ctx, 0) < 0) {
 		pr_err("expr__find_other failed\n");
 		return -1;
 	}
@@ -937,23 +946,23 @@ static int metric_parse_fake(const char *str)
 	 * make them unique.
 	 */
 	i = 1;
-	hashmap__for_each_entry((&ctx.ids), cur, bkt)
-		expr__add_id_val(&ctx, strdup(cur->key), i++);
+	hashmap__for_each_entry(ctx->ids, cur, bkt)
+		expr__add_id_val(ctx, strdup(cur->key), i++);
 
-	hashmap__for_each_entry((&ctx.ids), cur, bkt) {
+	hashmap__for_each_entry(ctx->ids, cur, bkt) {
 		if (check_parse_fake(cur->key)) {
 			pr_err("check_parse_fake failed\n");
 			goto out;
 		}
 	}
 
-	if (expr__parse(&result, &ctx, str, 0))
+	if (expr__parse(&result, ctx, str, 0))
 		pr_err("expr__parse failed\n");
 	else
 		ret = 0;
 
 out:
-	expr__ctx_clear(&ctx);
+	expr__ctx_free(ctx);
 	return ret;
 }
 
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index a850fd0be3ee..7b1c06772a49 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -73,7 +73,7 @@ int expr__add_id(struct expr_parse_ctx *ctx, const char *id)
 	data_ptr->parent = ctx->parent;
 	data_ptr->kind = EXPR_ID_DATA__PARENT;
 
-	ret = hashmap__set(&ctx->ids, id, data_ptr,
+	ret = hashmap__set(ctx->ids, id, data_ptr,
 			   (const void **)&old_key, (void **)&old_data);
 	if (ret)
 		free(data_ptr);
@@ -95,7 +95,7 @@ int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)
 	data_ptr->val = val;
 	data_ptr->kind = EXPR_ID_DATA__VALUE;
 
-	ret = hashmap__set(&ctx->ids, id, data_ptr,
+	ret = hashmap__set(ctx->ids, id, data_ptr,
 			   (const void **)&old_key, (void **)&old_data);
 	if (ret)
 		free(data_ptr);
@@ -140,7 +140,7 @@ int expr__add_ref(struct expr_parse_ctx *ctx, struct metric_ref *ref)
 	data_ptr->ref.metric_expr = ref->metric_expr;
 	data_ptr->kind = EXPR_ID_DATA__REF;
 
-	ret = hashmap__set(&ctx->ids, name, data_ptr,
+	ret = hashmap__set(ctx->ids, name, data_ptr,
 			   (const void **)&old_key, (void **)&old_data);
 	if (ret)
 		free(data_ptr);
@@ -156,7 +156,7 @@ int expr__add_ref(struct expr_parse_ctx *ctx, struct metric_ref *ref)
 int expr__get_id(struct expr_parse_ctx *ctx, const char *id,
 		 struct expr_id_data **data)
 {
-	return hashmap__find(&ctx->ids, id, (void **)data) ? 0 : -1;
+	return hashmap__find(ctx->ids, id, (void **)data) ? 0 : -1;
 }
 
 int expr__resolve_id(struct expr_parse_ctx *ctx, const char *id,
@@ -205,15 +205,23 @@ void expr__del_id(struct expr_parse_ctx *ctx, const char *id)
 	struct expr_id_data *old_val = NULL;
 	char *old_key = NULL;
 
-	hashmap__delete(&ctx->ids, id,
+	hashmap__delete(ctx->ids, id,
 			(const void **)&old_key, (void **)&old_val);
 	free(old_key);
 	free(old_val);
 }
 
-void expr__ctx_init(struct expr_parse_ctx *ctx)
+struct expr_parse_ctx *expr__ctx_new(void)
 {
-	hashmap__init(&ctx->ids, key_hash, key_equal, NULL);
+	struct expr_parse_ctx *ctx;
+
+	ctx = malloc(sizeof(struct expr_parse_ctx));
+	if (!ctx)
+		return NULL;
+
+	ctx->ids = hashmap__new(key_hash, key_equal, NULL);
+	ctx->parent = NULL;
+	return ctx;
 }
 
 void expr__ctx_clear(struct expr_parse_ctx *ctx)
@@ -221,11 +229,24 @@ void expr__ctx_clear(struct expr_parse_ctx *ctx)
 	struct hashmap_entry *cur;
 	size_t bkt;
 
-	hashmap__for_each_entry((&ctx->ids), cur, bkt) {
+	hashmap__for_each_entry(ctx->ids, cur, bkt) {
+		free((char *)cur->key);
+		free(cur->value);
+	}
+	hashmap__clear(ctx->ids);
+}
+
+void expr__ctx_free(struct expr_parse_ctx *ctx)
+{
+	struct hashmap_entry *cur;
+	size_t bkt;
+
+	hashmap__for_each_entry(ctx->ids, cur, bkt) {
 		free((char *)cur->key);
 		free(cur->value);
 	}
-	hashmap__clear(&ctx->ids);
+	hashmap__free(ctx->ids);
+	free(ctx);
 }
 
 static int
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index 85df3e4771e4..5fa394f10418 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -19,7 +19,7 @@ struct expr_id {
 };
 
 struct expr_parse_ctx {
-	struct hashmap	 ids;
+	struct hashmap	*ids;
 	struct expr_id	*parent;
 };
 
@@ -30,8 +30,9 @@ struct expr_scanner_ctx {
 	int runtime;
 };
 
-void expr__ctx_init(struct expr_parse_ctx *ctx);
+struct expr_parse_ctx *expr__ctx_new(void);
 void expr__ctx_clear(struct expr_parse_ctx *ctx);
+void expr__ctx_free(struct expr_parse_ctx *ctx);
 void expr__del_id(struct expr_parse_ctx *ctx, const char *id);
 int expr__add_id(struct expr_parse_ctx *ctx, const char *id);
 int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val);
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 29b747ac31c1..b7924a2f1f45 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -118,7 +118,7 @@ struct metric_ref_node {
 
 struct metric {
 	struct list_head nd;
-	struct expr_parse_ctx pctx;
+	struct expr_parse_ctx *pctx;
 	const char *metric_name;
 	const char *metric_expr;
 	const char *metric_unit;
@@ -198,7 +198,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	struct evsel *ev, *current_leader = NULL;
 	struct expr_id_data *val_ptr;
 	int i = 0, matched_events = 0, events_to_match;
-	const int idnum = (int)hashmap__size(&pctx->ids);
+	const int idnum = (int)hashmap__size(pctx->ids);
 
 	/*
 	 * duration_time is always grouped separately, when events are grouped
@@ -206,7 +206,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	 * add it to metric_events at the end.
 	 */
 	if (!has_constraint &&
-	    hashmap__find(&pctx->ids, "duration_time", (void **)&val_ptr))
+	    hashmap__find(pctx->ids, "duration_time", (void **)&val_ptr))
 		events_to_match = idnum - 1;
 	else
 		events_to_match = idnum;
@@ -242,7 +242,7 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 		if (contains_event(metric_events, matched_events, ev->name))
 			continue;
 		/* Does this event belong to the parse context? */
-		if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr))
+		if (hashmap__find(pctx->ids, ev->name, (void **)&val_ptr))
 			metric_events[matched_events++] = ev;
 
 		if (matched_events == events_to_match)
@@ -322,12 +322,12 @@ static int metricgroup__setup_events(struct list_head *groups,
 		struct metric_ref *metric_refs = NULL;
 
 		metric_events = calloc(sizeof(void *),
-				hashmap__size(&m->pctx.ids) + 1);
+				hashmap__size(m->pctx->ids) + 1);
 		if (!metric_events) {
 			ret = -ENOMEM;
 			break;
 		}
-		evsel = find_evsel_group(perf_evlist, &m->pctx,
+		evsel = find_evsel_group(perf_evlist, m->pctx,
 					 metric_no_merge,
 					 m->has_constraint, metric_events,
 					 evlist_used);
@@ -693,7 +693,7 @@ static void metricgroup__add_metric_weak_group(struct strbuf *events,
 	size_t bkt;
 	bool no_group = true, has_duration = false;
 
-	hashmap__for_each_entry((&ctx->ids), cur, bkt) {
+	hashmap__for_each_entry(ctx->ids, cur, bkt) {
 		pr_debug("found event %s\n", (const char *)cur->key);
 		/*
 		 * Duration time maps to a software event and can make
@@ -724,7 +724,7 @@ static void metricgroup__add_metric_non_group(struct strbuf *events,
 	size_t bkt;
 	bool first = true;
 
-	hashmap__for_each_entry((&ctx->ids), cur, bkt) {
+	hashmap__for_each_entry(ctx->ids, cur, bkt) {
 		if (!first)
 			strbuf_addf(events, ",");
 		strbuf_addf(events, "%s", (const char *)cur->key);
@@ -799,7 +799,11 @@ static int __add_metric(struct list_head *metric_list,
 		if (!m)
 			return -ENOMEM;
 
-		expr__ctx_init(&m->pctx);
+		m->pctx = expr__ctx_new();
+		if (!m->pctx) {
+			free(m);
+			return -ENOMEM;
+		}
 		m->metric_name = pe->metric_name;
 		m->metric_expr = pe->metric_expr;
 		m->metric_unit = pe->unit;
@@ -847,15 +851,15 @@ static int __add_metric(struct list_head *metric_list,
 
 	/* Force all found IDs in metric to have us as parent ID. */
 	WARN_ON_ONCE(!parent);
-	m->pctx.parent = parent;
+	m->pctx->parent = parent;
 
 	/*
 	 * For both the parent and referenced metrics, we parse
 	 * all the metric's IDs and add it to the parent context.
 	 */
-	if (expr__find_other(pe->metric_expr, NULL, &m->pctx, runtime) < 0) {
+	if (expr__find_other(pe->metric_expr, NULL, m->pctx, runtime) < 0) {
 		if (m->metric_refs_cnt == 0) {
-			expr__ctx_clear(&m->pctx);
+			expr__ctx_free(m->pctx);
 			free(m);
 			*mp = NULL;
 		}
@@ -878,8 +882,8 @@ static int __add_metric(struct list_head *metric_list,
 		list_for_each_prev(pos, metric_list) {
 			struct metric *old = list_entry(pos, struct metric, nd);
 
-			if (hashmap__size(&m->pctx.ids) <=
-			    hashmap__size(&old->pctx.ids))
+			if (hashmap__size(m->pctx->ids) <=
+			    hashmap__size(old->pctx->ids))
 				break;
 		}
 		list_add(&m->nd, pos);
@@ -927,7 +931,7 @@ static int recursion_check(struct metric *m, const char *id, struct expr_id **pa
 	 * if we already processed 'id', if we did, it's recursion
 	 * and we fail.
 	 */
-	ret = expr__get_id(&m->pctx, id, &data);
+	ret = expr__get_id(m->pctx, id, &data);
 	if (ret)
 		return ret;
 
@@ -982,7 +986,7 @@ static int __resolve_metric(struct metric *m,
 	 */
 	do {
 		all = true;
-		hashmap__for_each_entry((&m->pctx.ids), cur, bkt) {
+		hashmap__for_each_entry(m->pctx->ids, cur, bkt) {
 			struct expr_id *parent;
 			struct pmu_event *pe;
 
@@ -996,7 +1000,7 @@ static int __resolve_metric(struct metric *m,
 
 			all = false;
 			/* The metric key itself needs to go out.. */
-			expr__del_id(&m->pctx, cur->key);
+			expr__del_id(m->pctx, cur->key);
 
 			/* ... and it gets resolved to the parent context. */
 			ret = add_metric(metric_list, pe, metric_no_group, &m, parent, ids);
@@ -1144,10 +1148,10 @@ static int metricgroup__add_metric(const char *metric, bool metric_no_group,
 
 		if (m->has_constraint) {
 			metricgroup__add_metric_non_group(events,
-							  &m->pctx);
+							  m->pctx);
 		} else {
 			metricgroup__add_metric_weak_group(events,
-							   &m->pctx);
+							   m->pctx);
 		}
 	}
 
@@ -1210,7 +1214,7 @@ static void metricgroup__free_metrics(struct list_head *metric_list)
 
 	list_for_each_entry_safe (m, tmp, metric_list, nd) {
 		metric__free_refs(m);
-		expr__ctx_clear(&m->pctx);
+		expr__ctx_free(m->pctx);
 		list_del_init(&m->nd);
 		free(m);
 	}
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index 34a7f5c1fff7..c9fa07e49e72 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -1,8 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
+#include <math.h>
 #include <stdio.h>
 #include "evsel.h"
 #include "stat.h"
 #include "color.h"
+#include "debug.h"
 #include "pmu.h"
 #include "rblist.h"
 #include "evlist.h"
@@ -370,12 +372,16 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list)
 {
 	struct evsel *counter, *leader, **metric_events, *oc;
 	bool found;
-	struct expr_parse_ctx ctx;
+	struct expr_parse_ctx *ctx;
 	struct hashmap_entry *cur;
 	size_t bkt;
 	int i;
 
-	expr__ctx_init(&ctx);
+	ctx = expr__ctx_new();
+	if (!ctx) {
+		pr_debug("expr__ctx_new failed");
+		return;
+	}
 	evlist__for_each_entry(evsel_list, counter) {
 		bool invalid = false;
 
@@ -383,25 +389,25 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list)
 		if (!counter->metric_expr)
 			continue;
 
-		expr__ctx_clear(&ctx);
+		expr__ctx_clear(ctx);
 		metric_events = counter->metric_events;
 		if (!metric_events) {
 			if (expr__find_other(counter->metric_expr,
 					     counter->name,
-					     &ctx, 1) < 0)
+					     ctx, 1) < 0)
 				continue;
 
 			metric_events = calloc(sizeof(struct evsel *),
-					       hashmap__size(&ctx.ids) + 1);
+					       hashmap__size(ctx->ids) + 1);
 			if (!metric_events) {
-				expr__ctx_clear(&ctx);
+				expr__ctx_free(ctx);
 				return;
 			}
 			counter->metric_events = metric_events;
 		}
 
 		i = 0;
-		hashmap__for_each_entry((&ctx.ids), cur, bkt) {
+		hashmap__for_each_entry(ctx->ids, cur, bkt) {
 			const char *metric_name = (const char *)cur->key;
 
 			found = false;
@@ -453,7 +459,7 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list)
 			counter->metric_expr = NULL;
 		}
 	}
-	expr__ctx_clear(&ctx);
+	expr__ctx_free(ctx);
 }
 
 static double runtime_stat_avg(struct runtime_stat *st,
@@ -818,7 +824,6 @@ static int prepare_metric(struct evsel **metric_events,
 	char *n, *pn;
 	int i, j, ret;
 
-	expr__ctx_init(pctx);
 	for (i = 0; metric_events[i]; i++) {
 		struct saved_value *v;
 		struct stats *stats;
@@ -880,17 +885,22 @@ static void generic_metric(struct perf_stat_config *config,
 			   struct runtime_stat *st)
 {
 	print_metric_t print_metric = out->print_metric;
-	struct expr_parse_ctx pctx;
+	struct expr_parse_ctx *pctx;
 	double ratio, scale;
 	int i;
 	void *ctxp = out->ctx;
 
-	i = prepare_metric(metric_events, metric_refs, &pctx, cpu, st);
-	if (i < 0)
+	pctx = expr__ctx_new();
+	if (!pctx)
 		return;
 
+	i = prepare_metric(metric_events, metric_refs, pctx, cpu, st);
+	if (i < 0) {
+		expr__ctx_free(pctx);
+		return;
+	}
 	if (!metric_events[i]) {
-		if (expr__parse(&ratio, &pctx, metric_expr, runtime) == 0) {
+		if (expr__parse(&ratio, pctx, metric_expr, runtime) == 0) {
 			char *unit;
 			char metric_bf[64];
 
@@ -926,22 +936,26 @@ static void generic_metric(struct perf_stat_config *config,
 			     (metric_name ? metric_name : name) : "", 0);
 	}
 
-	expr__ctx_clear(&pctx);
+	expr__ctx_free(pctx);
 }
 
 double test_generic_metric(struct metric_expr *mexp, int cpu, struct runtime_stat *st)
 {
-	struct expr_parse_ctx pctx;
+	struct expr_parse_ctx *pctx;
 	double ratio = 0.0;
 
-	if (prepare_metric(mexp->metric_events, mexp->metric_refs, &pctx, cpu, st) < 0)
+	pctx = expr__ctx_new();
+	if (!pctx)
+		return NAN;
+
+	if (prepare_metric(mexp->metric_events, mexp->metric_refs, pctx, cpu, st) < 0)
 		goto out;
 
-	if (expr__parse(&ratio, &pctx, mexp->metric_expr, 1))
+	if (expr__parse(&ratio, pctx, mexp->metric_expr, 1))
 		ratio = 0.0;
 
 out:
-	expr__ctx_clear(&pctx);
+	expr__ctx_free(pctx);
 	return ratio;
 }
 
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 02/13] perf metric: Use NAN for missing event IDs.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 01/13] perf metric: Restructure struct expr_parse_ctx Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 03/13] perf expr: Remove unused headers and inline d_ratio Ian Rogers
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

If during computing a metric an event (id) is missing the parsing
aborts. A later patch will make it so that events that aren't used in
the output are deliberately omitted, in which case we don't want the
abort. Modify the missing ID case to report NAN for these cases.

Reviewed-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index b2ada8f8309a..41c9cd4efadd 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -1,6 +1,7 @@
 /* Simple expression parser */
 %{
 #define YYDEBUG 1
+#include <math.h>
 #include <stdio.h>
 #include "util.h"
 #include "util/debug.h"
@@ -88,12 +89,10 @@ expr:	  NUMBER
 	| ID			{
 					struct expr_id_data *data;
 
-					if (expr__resolve_id(ctx, $1, &data)) {
-						free($1);
-						YYABORT;
-					}
+					$$ = NAN;
+					if (expr__resolve_id(ctx, $1, &data) == 0)
+						$$ = expr_id_data__value(data);
 
-					$$ = expr_id_data__value(data);
 					free($1);
 				}
 	| expr '|' expr		{ $$ = (long)$1 | (long)$3; }
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 03/13] perf expr: Remove unused headers and inline d_ratio
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 01/13] perf metric: Restructure struct expr_parse_ctx Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 02/13] perf metric: Use NAN for missing event IDs Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 04/13] perf expr: Separate token declataion from type Ian Rogers
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

No functional change. Inlining d_ratio makes it easier to special case
for constants in a later patch.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 22 +++++++---------------
 1 file changed, 7 insertions(+), 15 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 41c9cd4efadd..e6005450feae 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -2,23 +2,10 @@
 %{
 #define YYDEBUG 1
 #include <math.h>
-#include <stdio.h>
-#include "util.h"
 #include "util/debug.h"
-#include <stdlib.h> // strtod()
+#include "smt.h"
 #define IN_EXPR_Y 1
 #include "expr.h"
-#include "smt.h"
-#include <string.h>
-
-static double d_ratio(double val0, double val1)
-{
-	if (val1 == 0) {
-		return 0;
-	}
-	return  val0 / val1;
-}
-
 %}
 
 %define api.pure full
@@ -120,7 +107,12 @@ expr:	  NUMBER
 	| MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; }
 	| MAX '(' expr ',' expr ')' { $$ = $3 > $5 ? $3 : $5; }
 	| SMT_ON		 { $$ = smt_on() > 0; }
-	| D_RATIO '(' expr ',' expr ')' { $$ = d_ratio($3,$5); }
+	| D_RATIO '(' expr ',' expr ')' { if ($5 == 0) {
+						$$ = 0;
+					  } else {
+						$$ = $3 / $5;
+					  }
+					}
 	;
 
 %%
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 04/13] perf expr: Separate token declataion from type
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (2 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 03/13] perf expr: Remove unused headers and inline d_ratio Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 05/13] perf expr: Use macros for operators Ian Rogers
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

No functional change, so the type of expr remains <num>. A later patch
will change the computation to be an aggregate type and making this
change makes that later change smaller.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index e6005450feae..68b122e59b3f 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -20,11 +20,7 @@
 	char	*str;
 }
 
-%token EXPR_PARSE EXPR_OTHER EXPR_ERROR
-%token <num> NUMBER
-%token <str> ID
-%destructor { free ($$); } <str>
-%token MIN MAX IF ELSE SMT_ON D_RATIO
+%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR EXPR_PARSE EXPR_OTHER
 %left MIN MAX IF
 %left '|'
 %left '^'
@@ -33,6 +29,9 @@
 %left '-' '+'
 %left '*' '/' '%'
 %left NEG NOT
+%type <num> NUMBER
+%type <str> ID
+%destructor { free ($$); } <str>
 %type <num> expr if_expr
 
 %{
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 05/13] perf expr: Use macros for operators
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (3 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 04/13] perf expr: Separate token declataion from type Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 06/13] perf expr: Move actions to the left Ian Rogers
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

No functional change, switch the operators to use macros so that
additional complexity for constants can be added in a later change.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 68b122e59b3f..5535badeef0a 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -43,6 +43,12 @@ static void expr_error(double *final_val __maybe_unused,
 	pr_debug("%s\n", s);
 }
 
+#define BINARY_LONG_OP(RESULT, OP, LHS, RHS)				\
+	RESULT = (long)LHS OP (long)RHS;
+
+#define BINARY_OP(RESULT, OP, LHS, RHS)					\
+	RESULT = LHS OP RHS;
+
 %}
 %%
 
@@ -81,14 +87,14 @@ expr:	  NUMBER
 
 					free($1);
 				}
-	| expr '|' expr		{ $$ = (long)$1 | (long)$3; }
-	| expr '&' expr		{ $$ = (long)$1 & (long)$3; }
-	| expr '^' expr		{ $$ = (long)$1 ^ (long)$3; }
-	| expr '<' expr		{ $$ = $1 < $3; }
-	| expr '>' expr		{ $$ = $1 > $3; }
-	| expr '+' expr		{ $$ = $1 + $3; }
-	| expr '-' expr		{ $$ = $1 - $3; }
-	| expr '*' expr		{ $$ = $1 * $3; }
+	| expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
+	| expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
+	| expr '^' expr { BINARY_LONG_OP($$, ^, $1, $3); }
+	| expr '<' expr { BINARY_OP($$, <, $1, $3); }
+	| expr '>' expr { BINARY_OP($$, >, $1, $3); }
+	| expr '+' expr { BINARY_OP($$, +, $1, $3); }
+	| expr '-' expr { BINARY_OP($$, -, $1, $3); }
+	| expr '*' expr { BINARY_OP($$, *, $1, $3); }
 	| expr '/' expr		{ if ($3 == 0) {
 					pr_debug("division by zero\n");
 					YYABORT;
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 06/13] perf expr: Move actions to the left.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (4 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 05/13] perf expr: Use macros for operators Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 07/13] perf metric: Rename expr__find_other Ian Rogers
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

No functional change, just modifying whitespace. This creates additional
space for adding logic to actions in later changes.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 124 +++++++++++++++++++++++++----------------
 1 file changed, 75 insertions(+), 49 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 5535badeef0a..78cbe377eb0e 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -70,54 +70,80 @@ MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/
 '<' | '>' | D_RATIO
 
 all_expr: if_expr			{ *final_val = $1; }
-	;
-
-if_expr:
-	expr IF expr ELSE expr { $$ = $3 ? $1 : $5; }
-	| expr
-	;
-
-expr:	  NUMBER
-	| ID			{
-					struct expr_id_data *data;
-
-					$$ = NAN;
-					if (expr__resolve_id(ctx, $1, &data) == 0)
-						$$ = expr_id_data__value(data);
-
-					free($1);
-				}
-	| expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
-	| expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
-	| expr '^' expr { BINARY_LONG_OP($$, ^, $1, $3); }
-	| expr '<' expr { BINARY_OP($$, <, $1, $3); }
-	| expr '>' expr { BINARY_OP($$, >, $1, $3); }
-	| expr '+' expr { BINARY_OP($$, +, $1, $3); }
-	| expr '-' expr { BINARY_OP($$, -, $1, $3); }
-	| expr '*' expr { BINARY_OP($$, *, $1, $3); }
-	| expr '/' expr		{ if ($3 == 0) {
-					pr_debug("division by zero\n");
-					YYABORT;
-				  }
-				  $$ = $1 / $3;
-	                        }
-	| expr '%' expr		{ if ((long)$3 == 0) {
-					pr_debug("division by zero\n");
-					YYABORT;
-				  }
-				  $$ = (long)$1 % (long)$3;
-	                        }
-	| '-' expr %prec NEG	{ $$ = -$2; }
-	| '(' if_expr ')'	{ $$ = $2; }
-	| MIN '(' expr ',' expr ')' { $$ = $3 < $5 ? $3 : $5; }
-	| MAX '(' expr ',' expr ')' { $$ = $3 > $5 ? $3 : $5; }
-	| SMT_ON		 { $$ = smt_on() > 0; }
-	| D_RATIO '(' expr ',' expr ')' { if ($5 == 0) {
-						$$ = 0;
-					  } else {
-						$$ = $3 / $5;
-					  }
-					}
-	;
+
+if_expr: expr IF expr ELSE expr
+{
+	$$ = $3 ? $1 : $5;
+}
+| expr
+;
+
+expr: NUMBER
+{
+	$$ = $1;
+}
+| ID
+{
+	struct expr_id_data *data;
+
+	$$ = NAN;
+	if (expr__resolve_id(ctx, $1, &data) == 0)
+		$$ = expr_id_data__value(data);
+
+	free($1);
+}
+| expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
+| expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
+| expr '^' expr { BINARY_LONG_OP($$, ^, $1, $3); }
+| expr '<' expr { BINARY_OP($$, <, $1, $3); }
+| expr '>' expr { BINARY_OP($$, >, $1, $3); }
+| expr '+' expr { BINARY_OP($$, +, $1, $3); }
+| expr '-' expr { BINARY_OP($$, -, $1, $3); }
+| expr '*' expr { BINARY_OP($$, *, $1, $3); }
+| expr '/' expr
+{
+	if ($3 == 0) {
+		pr_debug("division by zero\n");
+		YYABORT;
+	}
+	$$ = $1 / $3;
+}
+| expr '%' expr
+{
+	if ((long)$3 == 0) {
+		pr_debug("division by zero\n");
+		YYABORT;
+	}
+	$$ = (long)$1 % (long)$3;
+}
+| D_RATIO '(' expr ',' expr ')'
+{
+	if ($5 == 0) {
+		$$ = 0;
+	} else {
+		$$ = $3 / $5;
+	}
+}
+| '-' expr %prec NEG
+{
+	$$ = -$2;
+}
+| '(' if_expr ')'
+{
+	$$ = $2;
+}
+| MIN '(' expr ',' expr ')'
+{
+	$$ = $3 < $5 ? $3 : $5;
+}
+| MAX '(' expr ',' expr ')'
+{
+	$$ = $3 > $5 ? $3 : $5;
+}
+| SMT_ON
+{
+	$$ = smt_on() > 0 ? 1.0 : 0.0;
+}
+;
 
 %%
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 07/13] perf metric: Rename expr__find_other.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (5 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 06/13] perf expr: Move actions to the left Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 08/13] perf metric: Add utilities to work on ids map Ian Rogers
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

A later change will remove the notion of other, rename the function to
expr__find_ids as this is what it populates.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c       | 26 +++++++++++++-------------
 tools/perf/tests/pmu-events.c | 11 +++++------
 tools/perf/util/expr.c        |  4 ++--
 tools/perf/util/expr.h        |  2 +-
 tools/perf/util/metricgroup.c |  2 +-
 tools/perf/util/stat-shadow.c |  6 +++---
 6 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index b0a3b5fd0c00..7ccb97c73347 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -64,25 +64,25 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
 	TEST_ASSERT_VAL("missing operand", ret == -1);
 
 	expr__ctx_clear(ctx);
-	TEST_ASSERT_VAL("find other",
-			expr__find_other("FOO + BAR + BAZ + BOZO", "FOO",
-					 ctx, 1) == 0);
-	TEST_ASSERT_VAL("find other", hashmap__size(ctx->ids) == 3);
-	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BAR",
+	TEST_ASSERT_VAL("find ids",
+			expr__find_ids("FOO + BAR + BAZ + BOZO", "FOO",
+					ctx, 1) == 0);
+	TEST_ASSERT_VAL("find ids", hashmap__size(ctx->ids) == 3);
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "BAR",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BAZ",
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "BAZ",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "BOZO",
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "BOZO",
 						    (void **)&val_ptr));
 
 	expr__ctx_clear(ctx);
-	TEST_ASSERT_VAL("find other",
-			expr__find_other("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@",
-					 NULL, ctx, 3) == 0);
-	TEST_ASSERT_VAL("find other", hashmap__size(ctx->ids) == 2);
-	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "EVENT1,param=3/",
+	TEST_ASSERT_VAL("find ids",
+			expr__find_ids("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@",
+					NULL, ctx, 3) == 0);
+	TEST_ASSERT_VAL("find ids", hashmap__size(ctx->ids) == 2);
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "EVENT1,param=3/",
 						    (void **)&val_ptr));
-	TEST_ASSERT_VAL("find other", hashmap__find(ctx->ids, "EVENT2,param=3/",
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "EVENT2,param=3/",
 						    (void **)&val_ptr));
 
 	expr__ctx_free(ctx);
diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c
index ff3b2e5e6141..73ec2123b99e 100644
--- a/tools/perf/tests/pmu-events.c
+++ b/tools/perf/tests/pmu-events.c
@@ -811,7 +811,7 @@ static int resolve_metric_simple(struct expr_parse_ctx *pctx,
 			ref->metric_expr = pe->metric_expr;
 			list_add_tail(&metric->list, compound_list);
 
-			rc = expr__find_other(pe->metric_expr, NULL, pctx, 0);
+			rc = expr__find_ids(pe->metric_expr, NULL, pctx, 0);
 			if (rc)
 				goto out_err;
 			break; /* The hashmap has been modified, so restart */
@@ -861,9 +861,8 @@ static int test_parsing(void)
 			if (!pe->metric_expr)
 				continue;
 			expr__ctx_clear(ctx);
-			if (expr__find_other(pe->metric_expr, NULL, ctx, 0)
-				  < 0) {
-				expr_failure("Parse other failed", map, pe);
+			if (expr__find_ids(pe->metric_expr, NULL, ctx, 0) < 0) {
+				expr_failure("Parse find ids failed", map, pe);
 				ret++;
 				continue;
 			}
@@ -935,8 +934,8 @@ static int metric_parse_fake(const char *str)
 		pr_debug("expr__ctx_new failed");
 		return TEST_FAIL;
 	}
-	if (expr__find_other(str, NULL, ctx, 0) < 0) {
-		pr_err("expr__find_other failed\n");
+	if (expr__find_ids(str, NULL, ctx, 0) < 0) {
+		pr_err("expr__find_ids failed\n");
 		return -1;
 	}
 
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index 7b1c06772a49..adf16bb7571a 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -288,8 +288,8 @@ int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
 	return __expr__parse(final_val, ctx, expr, EXPR_PARSE, runtime) ? -1 : 0;
 }
 
-int expr__find_other(const char *expr, const char *one,
-		     struct expr_parse_ctx *ctx, int runtime)
+int expr__find_ids(const char *expr, const char *one,
+		   struct expr_parse_ctx *ctx, int runtime)
 {
 	int ret = __expr__parse(NULL, ctx, expr, EXPR_OTHER, runtime);
 
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index 5fa394f10418..de109c2ab917 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -43,7 +43,7 @@ int expr__resolve_id(struct expr_parse_ctx *ctx, const char *id,
 		     struct expr_id_data **datap);
 int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
 		const char *expr, int runtime);
-int expr__find_other(const char *expr, const char *one,
+int expr__find_ids(const char *expr, const char *one,
 		struct expr_parse_ctx *ids, int runtime);
 
 double expr_id_data__value(const struct expr_id_data *data);
diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index b7924a2f1f45..046fb3fe1700 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -857,7 +857,7 @@ static int __add_metric(struct list_head *metric_list,
 	 * For both the parent and referenced metrics, we parse
 	 * all the metric's IDs and add it to the parent context.
 	 */
-	if (expr__find_other(pe->metric_expr, NULL, m->pctx, runtime) < 0) {
+	if (expr__find_ids(pe->metric_expr, NULL, m->pctx, runtime) < 0) {
 		if (m->metric_refs_cnt == 0) {
 			expr__ctx_free(m->pctx);
 			free(m);
diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c
index c9fa07e49e72..9bc841e09a0c 100644
--- a/tools/perf/util/stat-shadow.c
+++ b/tools/perf/util/stat-shadow.c
@@ -392,9 +392,9 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list)
 		expr__ctx_clear(ctx);
 		metric_events = counter->metric_events;
 		if (!metric_events) {
-			if (expr__find_other(counter->metric_expr,
-					     counter->name,
-					     ctx, 1) < 0)
+			if (expr__find_ids(counter->metric_expr,
+					   counter->name,
+					   ctx, 1) < 0)
 				continue;
 
 			metric_events = calloc(sizeof(struct evsel *),
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 08/13] perf metric: Add utilities to work on ids map.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (6 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 07/13] perf metric: Rename expr__find_other Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 09/13] perf metric: Allow metrics with no events Ian Rogers
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

Add utilities to new/free an ids hashmap, as well as to union. Add
testing of the union. Unioning hashmaps will be used when parsing the
metric, if a value is known then the hashmap is unnecessary, otherwise
we need to union together all the event ids to compute their values for
reporting.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c | 47 +++++++++++++++++++++++++++
 tools/perf/util/expr.c  | 71 ++++++++++++++++++++++++++++++++++++++---
 tools/perf/util/expr.h  | 12 +++++++
 3 files changed, 126 insertions(+), 4 deletions(-)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 7ccb97c73347..1c881bea7fca 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -6,6 +6,51 @@
 #include <string.h>
 #include <linux/zalloc.h>
 
+static int test_ids_union(void)
+{
+	struct hashmap *ids1, *ids2;
+
+	/* Empty union. */
+	ids1 = ids__new();
+	TEST_ASSERT_VAL("ids__new", ids1);
+	ids2 = ids__new();
+	TEST_ASSERT_VAL("ids__new", ids2);
+
+	ids1 = ids__union(ids1, ids2);
+	TEST_ASSERT_EQUAL("union", (int)hashmap__size(ids1), 0);
+
+	/* Union {foo, bar} against {}. */
+	ids2 = ids__new();
+	TEST_ASSERT_VAL("ids__new", ids2);
+
+	TEST_ASSERT_EQUAL("ids__insert", ids__insert(ids1, strdup("foo"), NULL), 0);
+	TEST_ASSERT_EQUAL("ids__insert", ids__insert(ids1, strdup("bar"), NULL), 0);
+
+	ids1 = ids__union(ids1, ids2);
+	TEST_ASSERT_EQUAL("union", (int)hashmap__size(ids1), 2);
+
+	/* Union {foo, bar} against {foo}. */
+	ids2 = ids__new();
+	TEST_ASSERT_VAL("ids__new", ids2);
+	TEST_ASSERT_EQUAL("ids__insert", ids__insert(ids2, strdup("foo"), NULL), 0);
+
+	ids1 = ids__union(ids1, ids2);
+	TEST_ASSERT_EQUAL("union", (int)hashmap__size(ids1), 2);
+
+	/* Union {foo, bar} against {bar,baz}. */
+	ids2 = ids__new();
+	TEST_ASSERT_VAL("ids__new", ids2);
+	TEST_ASSERT_EQUAL("ids__insert", ids__insert(ids2, strdup("bar"), NULL), 0);
+	TEST_ASSERT_EQUAL("ids__insert", ids__insert(ids2, strdup("baz"), NULL), 0);
+
+	ids1 = ids__union(ids1, ids2);
+	TEST_ASSERT_EQUAL("union", (int)hashmap__size(ids1), 3);
+
+	ids__free(ids1);
+
+	return 0;
+}
+
 static int test(struct expr_parse_ctx *ctx, const char *e, double val2)
 {
 	double val;
@@ -24,6 +69,8 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
 	int ret;
 	struct expr_parse_ctx *ctx;
 
+	TEST_ASSERT_EQUAL("ids_union", test_ids_union(), 0);
+
 	ctx = expr__ctx_new();
 	TEST_ASSERT_VAL("expr__ctx_new", ctx);
 	expr__add_id_val(ctx, strdup("FOO"), 1);
diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index adf16bb7571a..81101be51044 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -59,8 +59,29 @@ static bool key_equal(const void *key1, const void *key2,
 	return !strcmp((const char *)key1, (const char *)key2);
 }
 
-/* Caller must make sure id is allocated */
-int expr__add_id(struct expr_parse_ctx *ctx, const char *id)
+struct hashmap *ids__new(void)
+{
+	return hashmap__new(key_hash, key_equal, NULL);
+}
+
+void ids__free(struct hashmap *ids)
+{
+	struct hashmap_entry *cur;
+	size_t bkt;
+
+	if (ids == NULL)
+		return;
+
+	hashmap__for_each_entry(ids, cur, bkt) {
+		free((char *)cur->key);
+		free(cur->value);
+	}
+
+	hashmap__free(ids);
+}
+
+int ids__insert(struct hashmap *ids, const char *id,
+		struct expr_id *parent)
 {
 	struct expr_id_data *data_ptr = NULL, *old_data = NULL;
 	char *old_key = NULL;
@@ -70,10 +91,10 @@ int expr__add_id(struct expr_parse_ctx *ctx, const char *id)
 	if (!data_ptr)
 		return -ENOMEM;
 
-	data_ptr->parent = ctx->parent;
+	data_ptr->parent = parent;
 	data_ptr->kind = EXPR_ID_DATA__PARENT;
 
-	ret = hashmap__set(ctx->ids, id, data_ptr,
+	ret = hashmap__set(ids, id, data_ptr,
 			   (const void **)&old_key, (void **)&old_data);
 	if (ret)
 		free(data_ptr);
@@ -82,6 +103,48 @@ int expr__add_id(struct expr_parse_ctx *ctx, const char *id)
 	return ret;
 }
 
+struct hashmap *ids__union(struct hashmap *ids1, struct hashmap *ids2)
+{
+	size_t bkt;
+	struct hashmap_entry *cur;
+	int ret;
+	struct expr_id_data *old_data = NULL;
+	char *old_key = NULL;
+
+	if (!ids1)
+		return ids2;
+
+	if (!ids2)
+		return ids1;
+
+	if (hashmap__size(ids1) <  hashmap__size(ids2)) {
+		struct hashmap *tmp = ids1;
+
+		ids1 = ids2;
+		ids2 = tmp;
+	}
+	hashmap__for_each_entry(ids2, cur, bkt) {
+		ret = hashmap__set(ids1, cur->key, cur->value,
+				(const void **)&old_key, (void **)&old_data);
+		free(old_key);
+		free(old_data);
+
+		if (ret) {
+			hashmap__free(ids1);
+			hashmap__free(ids2);
+			return NULL;
+		}
+	}
+	hashmap__free(ids2);
+	return ids1;
+}
+
+/* Caller must make sure id is allocated */
+int expr__add_id(struct expr_parse_ctx *ctx, const char *id)
+{
+	return ids__insert(ctx->ids, id, ctx->parent);
+}
+
 /* Caller must make sure id is allocated */
 int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val)
 {
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index de109c2ab917..4ed186bd1f13 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -30,9 +30,19 @@ struct expr_scanner_ctx {
 	int runtime;
 };
 
+struct hashmap *ids__new(void);
+void ids__free(struct hashmap *ids);
+int ids__insert(struct hashmap *ids, const char *id, struct expr_id *parent);
+/*
+ * Union two sets of ids (hashmaps) and construct a third, freeing ids1 and
+ * ids2.
+ */
+struct hashmap *ids__union(struct hashmap *ids1, struct hashmap *ids2);
+
 struct expr_parse_ctx *expr__ctx_new(void);
 void expr__ctx_clear(struct expr_parse_ctx *ctx);
 void expr__ctx_free(struct expr_parse_ctx *ctx);
+
 void expr__del_id(struct expr_parse_ctx *ctx, const char *id);
 int expr__add_id(struct expr_parse_ctx *ctx, const char *id);
 int expr__add_id_val(struct expr_parse_ctx *ctx, const char *id, double val);
@@ -41,8 +51,10 @@ int expr__get_id(struct expr_parse_ctx *ctx, const char *id,
 		 struct expr_id_data **data);
 int expr__resolve_id(struct expr_parse_ctx *ctx, const char *id,
 		     struct expr_id_data **datap);
+
 int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
 		const char *expr, int runtime);
+
 int expr__find_ids(const char *expr, const char *one,
 		struct expr_parse_ctx *ids, int runtime);
 
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 09/13] perf metric: Allow metrics with no events
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (7 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 08/13] perf metric: Add utilities to work on ids map Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing Ian Rogers
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

A metric may be a constant value, for example, some SMT metrics are
constant 0 if #smt_on is 0. If we eliminate all the events then there is
no printing. Fix this by forcing metrics like this to have a
duration_time tool event, previously the metric would fail when parsing
the events with a parse error.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/metricgroup.c | 109 ++++++++++++++++++----------------
 1 file changed, 59 insertions(+), 50 deletions(-)

diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c
index 046fb3fe1700..34956977e907 100644
--- a/tools/perf/util/metricgroup.c
+++ b/tools/perf/util/metricgroup.c
@@ -198,65 +198,69 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist,
 	struct evsel *ev, *current_leader = NULL;
 	struct expr_id_data *val_ptr;
 	int i = 0, matched_events = 0, events_to_match;
-	const int idnum = (int)hashmap__size(pctx->ids);
+	int idnum = (int)hashmap__size(pctx->ids);
 
-	/*
-	 * duration_time is always grouped separately, when events are grouped
-	 * (ie has_constraint is false) then ignore it in the matching loop and
-	 * add it to metric_events at the end.
-	 */
-	if (!has_constraint &&
-	    hashmap__find(pctx->ids, "duration_time", (void **)&val_ptr))
-		events_to_match = idnum - 1;
-	else
-		events_to_match = idnum;
-
-	evlist__for_each_entry (perf_evlist, ev) {
+	if (idnum != 0) {
 		/*
-		 * Events with a constraint aren't grouped and match the first
-		 * events available.
+		 * duration_time is always grouped separately, when events are
+		 * grouped (ie has_constraint is false) then ignore it in the
+		 * matching loop and add it to metric_events at the end.
 		 */
-		if (has_constraint && ev->weak_group)
-			continue;
-		/* Ignore event if already used and merging is disabled. */
-		if (metric_no_merge && test_bit(ev->core.idx, evlist_used))
-			continue;
-		if (!has_constraint && !evsel__has_leader(ev, current_leader)) {
+		events_to_match = idnum;
+		if (!has_constraint && hashmap__find(pctx->ids, "duration_time", (void **)&val_ptr))
+			events_to_match--;
+
+		evlist__for_each_entry(perf_evlist, ev) {
+			/*
+			 * Events with a constraint aren't grouped and match the
+			 * first events available.
+			 */
+			if (has_constraint && ev->weak_group)
+				continue;
+			/* Ignore event if already used and merging is disabled. */
+			if (metric_no_merge && test_bit(ev->core.idx, evlist_used))
+				continue;
+			if (!has_constraint && !evsel__has_leader(ev, current_leader)) {
+				/*
+				 * Start of a new group, discard the whole match
+				 * and start again.
+				 */
+				matched_events = 0;
+				memset(metric_events, 0, sizeof(struct evsel *) * idnum);
+				current_leader = evsel__leader(ev);
+			}
 			/*
-			 * Start of a new group, discard the whole match and
-			 * start again.
+			 * Check for duplicate events with the same name. For
+			 * example, uncore_imc/cas_count_read/ will turn into 6
+			 * events per socket on skylakex. Only the first such
+			 * event is placed in metric_events. If events aren't
+			 * grouped then this also ensures that the same event in
+			 * different sibling groups aren't both added to
+			 * metric_events.
 			 */
-			matched_events = 0;
-			memset(metric_events, 0,
-				sizeof(struct evsel *) * idnum);
-			current_leader = evsel__leader(ev);
+			if (contains_event(metric_events, matched_events, ev->name))
+				continue;
+			/* Does this event belong to the parse context? */
+			if (hashmap__find(pctx->ids, ev->name, (void **)&val_ptr))
+				metric_events[matched_events++] = ev;
+
+			if (matched_events == events_to_match)
+				break;
 		}
+	} else {
 		/*
-		 * Check for duplicate events with the same name. For example,
-		 * uncore_imc/cas_count_read/ will turn into 6 events per socket
-		 * on skylakex. Only the first such event is placed in
-		 * metric_events. If events aren't grouped then this also
-		 * ensures that the same event in different sibling groups
-		 * aren't both added to metric_events.
+		 * There are no events to match, but we need to associate the
+		 * metric with an event for printing. A duration_time event was
+		 * parsed for this.
 		 */
-		if (contains_event(metric_events, matched_events, ev->name))
-			continue;
-		/* Does this event belong to the parse context? */
-		if (hashmap__find(pctx->ids, ev->name, (void **)&val_ptr))
-			metric_events[matched_events++] = ev;
-
-		if (matched_events == events_to_match)
-			break;
+		idnum = 1;
+		events_to_match = 0;
 	}
-
 	if (events_to_match != idnum) {
 		/* Add the first duration_time. */
-		evlist__for_each_entry(perf_evlist, ev) {
-			if (!strcmp(ev->name, "duration_time")) {
-				metric_events[matched_events++] = ev;
-				break;
-			}
-		}
+		ev = evlist__find_evsel_by_str(perf_evlist, "duration_time");
+		if (ev)
+			metric_events[matched_events++] = ev;
 	}
 
 	if (matched_events != idnum) {
@@ -320,9 +324,10 @@ static int metricgroup__setup_events(struct list_head *groups,
 	list_for_each_entry (m, groups, nd) {
 		struct evsel **metric_events;
 		struct metric_ref *metric_refs = NULL;
+		const size_t ids_size = hashmap__size(m->pctx->ids);
 
 		metric_events = calloc(sizeof(void *),
-				hashmap__size(m->pctx->ids) + 1);
+				ids_size == 0 ? 2 : ids_size + 1);
 		if (!metric_events) {
 			ret = -ENOMEM;
 			break;
@@ -1240,7 +1245,11 @@ static int parse_groups(struct evlist *perf_evlist, const char *str,
 		goto out;
 	pr_debug("adding %s\n", extra_events.buf);
 	bzero(&parse_error, sizeof(parse_error));
-	ret = __parse_events(perf_evlist, extra_events.buf, &parse_error, fake_pmu);
+	ret = __parse_events(perf_evlist,
+			extra_events.len > 0
+			? extra_events.buf
+			: "duration_time",
+			&parse_error, fake_pmu);
 	if (ret) {
 		parse_events_print_error(&parse_error, extra_events.buf);
 		goto out;
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (8 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 09/13] perf metric: Allow metrics with no events Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-28 20:56   ` Jiri Olsa
  2021-09-23  7:46 ` [PATCH v9 11/13] perf expr: Propagate constants for binary operations Ian Rogers
                   ` (4 subsequent siblings)
  14 siblings, 1 reply; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

Add a new option to parsing that the set of IDs being used should be
computed, this means every action needs to handle the compute_ids and
regular case. This means actions yield a new ids type is a set of ids or
the value being computed. Use the compute_ids case to replace find IDs
parsing.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.c |   9 +--
 tools/perf/util/expr.h |   1 -
 tools/perf/util/expr.l |   9 ---
 tools/perf/util/expr.y | 176 ++++++++++++++++++++++++++++++-----------
 4 files changed, 136 insertions(+), 59 deletions(-)

diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
index 81101be51044..db2445677c8c 100644
--- a/tools/perf/util/expr.c
+++ b/tools/perf/util/expr.c
@@ -314,10 +314,9 @@ void expr__ctx_free(struct expr_parse_ctx *ctx)
 
 static int
 __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
-	      int start, int runtime)
+	      bool compute_ids, int runtime)
 {
 	struct expr_scanner_ctx scanner_ctx = {
-		.start_token = start,
 		.runtime = runtime,
 	};
 	YY_BUFFER_STATE buffer;
@@ -337,7 +336,7 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
 	expr_set_debug(1, scanner);
 #endif
 
-	ret = expr_parse(val, ctx, scanner);
+	ret = expr_parse(val, ctx, compute_ids, scanner);
 
 	expr__flush_buffer(buffer, scanner);
 	expr__delete_buffer(buffer, scanner);
@@ -348,13 +347,13 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
 int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
 		const char *expr, int runtime)
 {
-	return __expr__parse(final_val, ctx, expr, EXPR_PARSE, runtime) ? -1 : 0;
+	return __expr__parse(final_val, ctx, expr, /*compute_ids=*/false, runtime) ? -1 : 0;
 }
 
 int expr__find_ids(const char *expr, const char *one,
 		   struct expr_parse_ctx *ctx, int runtime)
 {
-	int ret = __expr__parse(NULL, ctx, expr, EXPR_OTHER, runtime);
+	int ret = __expr__parse(NULL, ctx, expr, /*compute_ids=*/true, runtime);
 
 	if (one)
 		expr__del_id(ctx, one);
diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
index 4ed186bd1f13..b20513f0ae59 100644
--- a/tools/perf/util/expr.h
+++ b/tools/perf/util/expr.h
@@ -26,7 +26,6 @@ struct expr_parse_ctx {
 struct expr_id_data;
 
 struct expr_scanner_ctx {
-	int start_token;
 	int runtime;
 };
 
diff --git a/tools/perf/util/expr.l b/tools/perf/util/expr.l
index 13e5e3c75f56..702fdf6456ca 100644
--- a/tools/perf/util/expr.l
+++ b/tools/perf/util/expr.l
@@ -91,15 +91,6 @@ symbol		({spec}|{sym})+
 %%
 	struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
 
-	{
-		int start_token = sctx->start_token;
-
-		if (sctx->start_token) {
-			sctx->start_token = 0;
-			return start_token;
-		}
-	}
-
 d_ratio		{ return D_RATIO; }
 max		{ return MAX; }
 min		{ return MIN; }
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 78cbe377eb0e..6aeead54760a 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -1,6 +1,7 @@
 /* Simple expression parser */
 %{
 #define YYDEBUG 1
+#include <assert.h>
 #include <math.h>
 #include "util/debug.h"
 #include "smt.h"
@@ -12,15 +13,31 @@
 
 %parse-param { double *final_val }
 %parse-param { struct expr_parse_ctx *ctx }
+%parse-param { bool compute_ids }
 %parse-param {void *scanner}
 %lex-param {void* scanner}
 
 %union {
 	double	 num;
 	char	*str;
+	struct ids {
+		/*
+		 * When creating ids, holds the working set of event ids. NULL
+		 * implies the set is empty.
+		 */
+		struct hashmap *ids;
+		/*
+		 * The metric value. When not creating ids this is the value
+		 * read from a counter, a constant or some computed value. When
+		 * creating ids the value is either a constant or BOTTOM. NAN is
+		 * used as the special BOTTOM value, representing a "set of all
+		 * values" case.
+		 */
+		double val;
+	} ids;
 }
 
-%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR EXPR_PARSE EXPR_OTHER
+%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR
 %left MIN MAX IF
 %left '|'
 %left '^'
@@ -32,65 +49,109 @@
 %type <num> NUMBER
 %type <str> ID
 %destructor { free ($$); } <str>
-%type <num> expr if_expr
+%type <ids> expr if_expr
+%destructor { ids__free($$.ids); } <ids>
 
 %{
 static void expr_error(double *final_val __maybe_unused,
 		       struct expr_parse_ctx *ctx __maybe_unused,
+		       bool compute_ids __maybe_unused,
 		       void *scanner,
 		       const char *s)
 {
 	pr_debug("%s\n", s);
 }
 
+/*
+ * During compute ids, the special "bottom" value uses NAN to represent the set
+ * of all values. NAN is selected as it isn't a useful constant value.
+ */
+#define BOTTOM NAN
+
+static struct ids union_expr(struct ids ids1, struct ids ids2)
+{
+	struct ids result = {
+		.val = BOTTOM,
+		.ids = ids__union(ids1.ids, ids2.ids),
+	};
+	return result;
+}
+
 #define BINARY_LONG_OP(RESULT, OP, LHS, RHS)				\
-	RESULT = (long)LHS OP (long)RHS;
+	if (!compute_ids) {						\
+		RESULT.val = (long)LHS.val OP (long)RHS.val;		\
+		RESULT.ids = NULL;					\
+	} else {							\
+	        RESULT = union_expr(LHS, RHS);				\
+	}
 
 #define BINARY_OP(RESULT, OP, LHS, RHS)					\
-	RESULT = LHS OP RHS;
+	if (!compute_ids) {						\
+		RESULT.val = LHS.val OP RHS.val;			\
+		RESULT.ids = NULL;					\
+	} else {							\
+	        RESULT = union_expr(LHS, RHS);				\
+	}
 
 %}
 %%
 
-start:
-EXPR_PARSE all_expr
-|
-EXPR_OTHER all_other
-
-all_other: all_other other
-|
-
-other: ID
+start: if_expr
 {
-	expr__add_id(ctx, $1);
-}
-|
-MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
-|
-'<' | '>' | D_RATIO
+	if (compute_ids)
+		ctx->ids = ids__union($1.ids, ctx->ids);
 
-all_expr: if_expr			{ *final_val = $1; }
+	if (final_val)
+		*final_val = $1.val;
+}
+;
 
 if_expr: expr IF expr ELSE expr
 {
-	$$ = $3 ? $1 : $5;
+	if (!compute_ids) {
+		$$.ids = NULL;
+		if (fpclassify($3.val) == FP_ZERO) {
+			$$.val = $5.val;
+		} else {
+			$$.val = $1.val;
+		}
+	} else {
+		$$ = union_expr($1, union_expr($3, $5));
+	}
 }
 | expr
 ;
 
 expr: NUMBER
 {
-	$$ = $1;
+	$$.val = $1;
+	$$.ids = NULL;
 }
 | ID
 {
-	struct expr_id_data *data;
-
-	$$ = NAN;
-	if (expr__resolve_id(ctx, $1, &data) == 0)
-		$$ = expr_id_data__value(data);
-
-	free($1);
+	if (!compute_ids) {
+		/*
+		 * Compute the event's value from ID. If the ID isn't known then
+		 * it isn't used to compute the formula so set to NAN.
+		 */
+		struct expr_id_data *data;
+
+		$$.val = NAN;
+		if (expr__resolve_id(ctx, $1, &data) == 0)
+			$$.val = expr_id_data__value(data);
+
+		$$.ids = NULL;
+		free($1);
+	} else {
+		/*
+		 * Set the value to BOTTOM to show that any value is possible
+		 * when the event is computed. Create a set of just the ID.
+		 */
+		$$.val = BOTTOM;
+		$$.ids = ids__new();
+		if (!$$.ids || ids__insert($$.ids, $1, ctx->parent))
+			YYABORT;
+	}
 }
 | expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
 | expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
@@ -102,31 +163,47 @@ expr: NUMBER
 | expr '*' expr { BINARY_OP($$, *, $1, $3); }
 | expr '/' expr
 {
-	if ($3 == 0) {
-		pr_debug("division by zero\n");
-		YYABORT;
+	if (!compute_ids) {
+		if (fpclassify($3.val) == FP_ZERO) {
+			pr_debug("division by zero\n");
+			YYABORT;
+		}
+		$$.val = $1.val / $3.val;
+		$$.ids = NULL;
+	} else {
+		$$ = union_expr($1, $3);
 	}
-	$$ = $1 / $3;
 }
 | expr '%' expr
 {
-	if ((long)$3 == 0) {
-		pr_debug("division by zero\n");
-		YYABORT;
+	if (!compute_ids) {
+		if (fpclassify($3.val) == FP_ZERO) {
+			pr_debug("division by zero\n");
+			YYABORT;
+		}
+		$$.val = (long)$1.val % (long)$3.val;
+		$$.ids = NULL;
+	} else {
+		$$ = union_expr($1, $3);
 	}
-	$$ = (long)$1 % (long)$3;
 }
 | D_RATIO '(' expr ',' expr ')'
 {
-	if ($5 == 0) {
-		$$ = 0;
+	if (!compute_ids) {
+		$$.ids = NULL;
+		if (fpclassify($5.val) == FP_ZERO) {
+			$$.val = 0.0;
+		} else {
+			$$.val = $3.val / $5.val;
+		}
 	} else {
-		$$ = $3 / $5;
+		$$ = union_expr($3, $5);
 	}
 }
 | '-' expr %prec NEG
 {
-	$$ = -$2;
+	$$.val = -$2.val;
+	$$.ids = $2.ids;
 }
 | '(' if_expr ')'
 {
@@ -134,15 +211,26 @@ expr: NUMBER
 }
 | MIN '(' expr ',' expr ')'
 {
-	$$ = $3 < $5 ? $3 : $5;
+	if (!compute_ids) {
+		$$.val = $3.val < $5.val ? $3.val : $5.val;
+		$$.ids = NULL;
+	} else {
+		$$ = union_expr($3, $5);
+	}
 }
 | MAX '(' expr ',' expr ')'
 {
-	$$ = $3 > $5 ? $3 : $5;
+	if (!compute_ids) {
+		$$.val = $3.val > $5.val ? $3.val : $5.val;
+		$$.ids = NULL;
+	} else {
+		$$ = union_expr($3, $5);
+	}
 }
 | SMT_ON
 {
-	$$ = smt_on() > 0 ? 1.0 : 0.0;
+	$$.val = smt_on() > 0 ? 1.0 : 0.0;
+	$$.ids = NULL;
 }
 ;
 
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing
  2021-09-23  7:46 ` [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing Ian Rogers
@ 2021-09-28 20:56   ` Jiri Olsa
  2021-09-28 21:20     ` Ian Rogers
  0 siblings, 1 reply; 20+ messages in thread
From: Jiri Olsa @ 2021-09-28 20:56 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Namhyung Kim, linux-kernel,
	Andi Kleen, Jin Yao, John Garry, Paul Clarke, kajoljain,
	linux-perf-users, Stephane Eranian, Sandeep Dasgupta

On Thu, Sep 23, 2021 at 12:46:13AM -0700, Ian Rogers wrote:
> Add a new option to parsing that the set of IDs being used should be
> computed, this means every action needs to handle the compute_ids and
> regular case. This means actions yield a new ids type is a set of ids or
> the value being computed. Use the compute_ids case to replace find IDs
> parsing.
> 
> Signed-off-by: Ian Rogers <irogers@google.com>
> ---
>  tools/perf/util/expr.c |   9 +--
>  tools/perf/util/expr.h |   1 -
>  tools/perf/util/expr.l |   9 ---
>  tools/perf/util/expr.y | 176 ++++++++++++++++++++++++++++++-----------
>  4 files changed, 136 insertions(+), 59 deletions(-)
> 
> diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
> index 81101be51044..db2445677c8c 100644
> --- a/tools/perf/util/expr.c
> +++ b/tools/perf/util/expr.c
> @@ -314,10 +314,9 @@ void expr__ctx_free(struct expr_parse_ctx *ctx)
>  
>  static int
>  __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
> -	      int start, int runtime)
> +	      bool compute_ids, int runtime)
>  {
>  	struct expr_scanner_ctx scanner_ctx = {
> -		.start_token = start,
>  		.runtime = runtime,
>  	};
>  	YY_BUFFER_STATE buffer;
> @@ -337,7 +336,7 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
>  	expr_set_debug(1, scanner);
>  #endif
>  
> -	ret = expr_parse(val, ctx, scanner);
> +	ret = expr_parse(val, ctx, compute_ids, scanner);
>  
>  	expr__flush_buffer(buffer, scanner);
>  	expr__delete_buffer(buffer, scanner);
> @@ -348,13 +347,13 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
>  int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
>  		const char *expr, int runtime)
>  {
> -	return __expr__parse(final_val, ctx, expr, EXPR_PARSE, runtime) ? -1 : 0;
> +	return __expr__parse(final_val, ctx, expr, /*compute_ids=*/false, runtime) ? -1 : 0;
>  }
>  
>  int expr__find_ids(const char *expr, const char *one,
>  		   struct expr_parse_ctx *ctx, int runtime)
>  {
> -	int ret = __expr__parse(NULL, ctx, expr, EXPR_OTHER, runtime);
> +	int ret = __expr__parse(NULL, ctx, expr, /*compute_ids=*/true, runtime);
>  
>  	if (one)
>  		expr__del_id(ctx, one);
> diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
> index 4ed186bd1f13..b20513f0ae59 100644
> --- a/tools/perf/util/expr.h
> +++ b/tools/perf/util/expr.h
> @@ -26,7 +26,6 @@ struct expr_parse_ctx {
>  struct expr_id_data;
>  
>  struct expr_scanner_ctx {
> -	int start_token;
>  	int runtime;
>  };
>  
> diff --git a/tools/perf/util/expr.l b/tools/perf/util/expr.l
> index 13e5e3c75f56..702fdf6456ca 100644
> --- a/tools/perf/util/expr.l
> +++ b/tools/perf/util/expr.l
> @@ -91,15 +91,6 @@ symbol		({spec}|{sym})+
>  %%
>  	struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
>  
> -	{
> -		int start_token = sctx->start_token;
> -
> -		if (sctx->start_token) {
> -			sctx->start_token = 0;
> -			return start_token;
> -		}
> -	}
> -
>  d_ratio		{ return D_RATIO; }
>  max		{ return MAX; }
>  min		{ return MIN; }
> diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
> index 78cbe377eb0e..6aeead54760a 100644
> --- a/tools/perf/util/expr.y
> +++ b/tools/perf/util/expr.y
> @@ -1,6 +1,7 @@
>  /* Simple expression parser */
>  %{
>  #define YYDEBUG 1
> +#include <assert.h>
>  #include <math.h>
>  #include "util/debug.h"
>  #include "smt.h"
> @@ -12,15 +13,31 @@
>  
>  %parse-param { double *final_val }
>  %parse-param { struct expr_parse_ctx *ctx }
> +%parse-param { bool compute_ids }
>  %parse-param {void *scanner}
>  %lex-param {void* scanner}
>  
>  %union {
>  	double	 num;
>  	char	*str;
> +	struct ids {
> +		/*
> +		 * When creating ids, holds the working set of event ids. NULL
> +		 * implies the set is empty.
> +		 */
> +		struct hashmap *ids;
> +		/*
> +		 * The metric value. When not creating ids this is the value
> +		 * read from a counter, a constant or some computed value. When
> +		 * creating ids the value is either a constant or BOTTOM. NAN is
> +		 * used as the special BOTTOM value, representing a "set of all
> +		 * values" case.
> +		 */
> +		double val;
> +	} ids;
>  }
>  
> -%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR EXPR_PARSE EXPR_OTHER
> +%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR
>  %left MIN MAX IF
>  %left '|'
>  %left '^'
> @@ -32,65 +49,109 @@
>  %type <num> NUMBER
>  %type <str> ID
>  %destructor { free ($$); } <str>
> -%type <num> expr if_expr
> +%type <ids> expr if_expr
> +%destructor { ids__free($$.ids); } <ids>
>  
>  %{
>  static void expr_error(double *final_val __maybe_unused,
>  		       struct expr_parse_ctx *ctx __maybe_unused,
> +		       bool compute_ids __maybe_unused,
>  		       void *scanner,
>  		       const char *s)
>  {
>  	pr_debug("%s\n", s);
>  }
>  
> +/*
> + * During compute ids, the special "bottom" value uses NAN to represent the set
> + * of all values. NAN is selected as it isn't a useful constant value.
> + */
> +#define BOTTOM NAN
> +
> +static struct ids union_expr(struct ids ids1, struct ids ids2)
> +{
> +	struct ids result = {
> +		.val = BOTTOM,
> +		.ids = ids__union(ids1.ids, ids2.ids),

should we check for error in here?

jirka

> +	};
> +	return result;
> +}
> +
>  #define BINARY_LONG_OP(RESULT, OP, LHS, RHS)				\
> -	RESULT = (long)LHS OP (long)RHS;
> +	if (!compute_ids) {						\
> +		RESULT.val = (long)LHS.val OP (long)RHS.val;		\
> +		RESULT.ids = NULL;					\
> +	} else {							\
> +	        RESULT = union_expr(LHS, RHS);				\
> +	}
>  
>  #define BINARY_OP(RESULT, OP, LHS, RHS)					\
> -	RESULT = LHS OP RHS;
> +	if (!compute_ids) {						\
> +		RESULT.val = LHS.val OP RHS.val;			\
> +		RESULT.ids = NULL;					\
> +	} else {							\
> +	        RESULT = union_expr(LHS, RHS);				\
> +	}
>  
>  %}
>  %%
>  
> -start:
> -EXPR_PARSE all_expr
> -|
> -EXPR_OTHER all_other
> -
> -all_other: all_other other
> -|
> -
> -other: ID
> +start: if_expr
>  {
> -	expr__add_id(ctx, $1);
> -}
> -|
> -MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
> -|
> -'<' | '>' | D_RATIO
> +	if (compute_ids)
> +		ctx->ids = ids__union($1.ids, ctx->ids);
>  
> -all_expr: if_expr			{ *final_val = $1; }
> +	if (final_val)
> +		*final_val = $1.val;
> +}
> +;
>  
>  if_expr: expr IF expr ELSE expr
>  {
> -	$$ = $3 ? $1 : $5;
> +	if (!compute_ids) {
> +		$$.ids = NULL;
> +		if (fpclassify($3.val) == FP_ZERO) {
> +			$$.val = $5.val;
> +		} else {
> +			$$.val = $1.val;
> +		}
> +	} else {
> +		$$ = union_expr($1, union_expr($3, $5));
> +	}
>  }
>  | expr
>  ;
>  
>  expr: NUMBER
>  {
> -	$$ = $1;
> +	$$.val = $1;
> +	$$.ids = NULL;
>  }
>  | ID
>  {
> -	struct expr_id_data *data;
> -
> -	$$ = NAN;
> -	if (expr__resolve_id(ctx, $1, &data) == 0)
> -		$$ = expr_id_data__value(data);
> -
> -	free($1);
> +	if (!compute_ids) {
> +		/*
> +		 * Compute the event's value from ID. If the ID isn't known then
> +		 * it isn't used to compute the formula so set to NAN.
> +		 */
> +		struct expr_id_data *data;
> +
> +		$$.val = NAN;
> +		if (expr__resolve_id(ctx, $1, &data) == 0)
> +			$$.val = expr_id_data__value(data);
> +
> +		$$.ids = NULL;
> +		free($1);
> +	} else {
> +		/*
> +		 * Set the value to BOTTOM to show that any value is possible
> +		 * when the event is computed. Create a set of just the ID.
> +		 */
> +		$$.val = BOTTOM;
> +		$$.ids = ids__new();
> +		if (!$$.ids || ids__insert($$.ids, $1, ctx->parent))
> +			YYABORT;
> +	}
>  }
>  | expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
>  | expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
> @@ -102,31 +163,47 @@ expr: NUMBER
>  | expr '*' expr { BINARY_OP($$, *, $1, $3); }
>  | expr '/' expr
>  {
> -	if ($3 == 0) {
> -		pr_debug("division by zero\n");
> -		YYABORT;
> +	if (!compute_ids) {
> +		if (fpclassify($3.val) == FP_ZERO) {
> +			pr_debug("division by zero\n");
> +			YYABORT;
> +		}
> +		$$.val = $1.val / $3.val;
> +		$$.ids = NULL;
> +	} else {
> +		$$ = union_expr($1, $3);
>  	}
> -	$$ = $1 / $3;
>  }
>  | expr '%' expr
>  {
> -	if ((long)$3 == 0) {
> -		pr_debug("division by zero\n");
> -		YYABORT;
> +	if (!compute_ids) {
> +		if (fpclassify($3.val) == FP_ZERO) {
> +			pr_debug("division by zero\n");
> +			YYABORT;
> +		}
> +		$$.val = (long)$1.val % (long)$3.val;
> +		$$.ids = NULL;
> +	} else {
> +		$$ = union_expr($1, $3);
>  	}
> -	$$ = (long)$1 % (long)$3;
>  }
>  | D_RATIO '(' expr ',' expr ')'
>  {
> -	if ($5 == 0) {
> -		$$ = 0;
> +	if (!compute_ids) {
> +		$$.ids = NULL;
> +		if (fpclassify($5.val) == FP_ZERO) {
> +			$$.val = 0.0;
> +		} else {
> +			$$.val = $3.val / $5.val;
> +		}
>  	} else {
> -		$$ = $3 / $5;
> +		$$ = union_expr($3, $5);
>  	}
>  }
>  | '-' expr %prec NEG
>  {
> -	$$ = -$2;
> +	$$.val = -$2.val;
> +	$$.ids = $2.ids;
>  }
>  | '(' if_expr ')'
>  {
> @@ -134,15 +211,26 @@ expr: NUMBER
>  }
>  | MIN '(' expr ',' expr ')'
>  {
> -	$$ = $3 < $5 ? $3 : $5;
> +	if (!compute_ids) {
> +		$$.val = $3.val < $5.val ? $3.val : $5.val;
> +		$$.ids = NULL;
> +	} else {
> +		$$ = union_expr($3, $5);
> +	}
>  }
>  | MAX '(' expr ',' expr ')'
>  {
> -	$$ = $3 > $5 ? $3 : $5;
> +	if (!compute_ids) {
> +		$$.val = $3.val > $5.val ? $3.val : $5.val;
> +		$$.ids = NULL;
> +	} else {
> +		$$ = union_expr($3, $5);
> +	}
>  }
>  | SMT_ON
>  {
> -	$$ = smt_on() > 0 ? 1.0 : 0.0;
> +	$$.val = smt_on() > 0 ? 1.0 : 0.0;
> +	$$.ids = NULL;
>  }
>  ;
>  
> -- 
> 2.33.0.464.g1972c5931b-goog
> 


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing
  2021-09-28 20:56   ` Jiri Olsa
@ 2021-09-28 21:20     ` Ian Rogers
  0 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-28 21:20 UTC (permalink / raw)
  To: Jiri Olsa
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Namhyung Kim, linux-kernel,
	Andi Kleen, Jin Yao, John Garry, Paul Clarke, kajoljain,
	linux-perf-users, Stephane Eranian, Sandeep Dasgupta

On Tue, Sep 28, 2021 at 1:56 PM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Sep 23, 2021 at 12:46:13AM -0700, Ian Rogers wrote:
> > Add a new option to parsing that the set of IDs being used should be
> > computed, this means every action needs to handle the compute_ids and
> > regular case. This means actions yield a new ids type is a set of ids or
> > the value being computed. Use the compute_ids case to replace find IDs
> > parsing.
> >
> > Signed-off-by: Ian Rogers <irogers@google.com>
> > ---
> >  tools/perf/util/expr.c |   9 +--
> >  tools/perf/util/expr.h |   1 -
> >  tools/perf/util/expr.l |   9 ---
> >  tools/perf/util/expr.y | 176 ++++++++++++++++++++++++++++++-----------
> >  4 files changed, 136 insertions(+), 59 deletions(-)
> >
> > diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c
> > index 81101be51044..db2445677c8c 100644
> > --- a/tools/perf/util/expr.c
> > +++ b/tools/perf/util/expr.c
> > @@ -314,10 +314,9 @@ void expr__ctx_free(struct expr_parse_ctx *ctx)
> >
> >  static int
> >  __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
> > -           int start, int runtime)
> > +           bool compute_ids, int runtime)
> >  {
> >       struct expr_scanner_ctx scanner_ctx = {
> > -             .start_token = start,
> >               .runtime = runtime,
> >       };
> >       YY_BUFFER_STATE buffer;
> > @@ -337,7 +336,7 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
> >       expr_set_debug(1, scanner);
> >  #endif
> >
> > -     ret = expr_parse(val, ctx, scanner);
> > +     ret = expr_parse(val, ctx, compute_ids, scanner);
> >
> >       expr__flush_buffer(buffer, scanner);
> >       expr__delete_buffer(buffer, scanner);
> > @@ -348,13 +347,13 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr,
> >  int expr__parse(double *final_val, struct expr_parse_ctx *ctx,
> >               const char *expr, int runtime)
> >  {
> > -     return __expr__parse(final_val, ctx, expr, EXPR_PARSE, runtime) ? -1 : 0;
> > +     return __expr__parse(final_val, ctx, expr, /*compute_ids=*/false, runtime) ? -1 : 0;
> >  }
> >
> >  int expr__find_ids(const char *expr, const char *one,
> >                  struct expr_parse_ctx *ctx, int runtime)
> >  {
> > -     int ret = __expr__parse(NULL, ctx, expr, EXPR_OTHER, runtime);
> > +     int ret = __expr__parse(NULL, ctx, expr, /*compute_ids=*/true, runtime);
> >
> >       if (one)
> >               expr__del_id(ctx, one);
> > diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h
> > index 4ed186bd1f13..b20513f0ae59 100644
> > --- a/tools/perf/util/expr.h
> > +++ b/tools/perf/util/expr.h
> > @@ -26,7 +26,6 @@ struct expr_parse_ctx {
> >  struct expr_id_data;
> >
> >  struct expr_scanner_ctx {
> > -     int start_token;
> >       int runtime;
> >  };
> >
> > diff --git a/tools/perf/util/expr.l b/tools/perf/util/expr.l
> > index 13e5e3c75f56..702fdf6456ca 100644
> > --- a/tools/perf/util/expr.l
> > +++ b/tools/perf/util/expr.l
> > @@ -91,15 +91,6 @@ symbol             ({spec}|{sym})+
> >  %%
> >       struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner);
> >
> > -     {
> > -             int start_token = sctx->start_token;
> > -
> > -             if (sctx->start_token) {
> > -                     sctx->start_token = 0;
> > -                     return start_token;
> > -             }
> > -     }
> > -
> >  d_ratio              { return D_RATIO; }
> >  max          { return MAX; }
> >  min          { return MIN; }
> > diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
> > index 78cbe377eb0e..6aeead54760a 100644
> > --- a/tools/perf/util/expr.y
> > +++ b/tools/perf/util/expr.y
> > @@ -1,6 +1,7 @@
> >  /* Simple expression parser */
> >  %{
> >  #define YYDEBUG 1
> > +#include <assert.h>
> >  #include <math.h>
> >  #include "util/debug.h"
> >  #include "smt.h"
> > @@ -12,15 +13,31 @@
> >
> >  %parse-param { double *final_val }
> >  %parse-param { struct expr_parse_ctx *ctx }
> > +%parse-param { bool compute_ids }
> >  %parse-param {void *scanner}
> >  %lex-param {void* scanner}
> >
> >  %union {
> >       double   num;
> >       char    *str;
> > +     struct ids {
> > +             /*
> > +              * When creating ids, holds the working set of event ids. NULL
> > +              * implies the set is empty.
> > +              */
> > +             struct hashmap *ids;
> > +             /*
> > +              * The metric value. When not creating ids this is the value
> > +              * read from a counter, a constant or some computed value. When
> > +              * creating ids the value is either a constant or BOTTOM. NAN is
> > +              * used as the special BOTTOM value, representing a "set of all
> > +              * values" case.
> > +              */
> > +             double val;
> > +     } ids;
> >  }
> >
> > -%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR EXPR_PARSE EXPR_OTHER
> > +%token ID NUMBER MIN MAX IF ELSE SMT_ON D_RATIO EXPR_ERROR
> >  %left MIN MAX IF
> >  %left '|'
> >  %left '^'
> > @@ -32,65 +49,109 @@
> >  %type <num> NUMBER
> >  %type <str> ID
> >  %destructor { free ($$); } <str>
> > -%type <num> expr if_expr
> > +%type <ids> expr if_expr
> > +%destructor { ids__free($$.ids); } <ids>
> >
> >  %{
> >  static void expr_error(double *final_val __maybe_unused,
> >                      struct expr_parse_ctx *ctx __maybe_unused,
> > +                    bool compute_ids __maybe_unused,
> >                      void *scanner,
> >                      const char *s)
> >  {
> >       pr_debug("%s\n", s);
> >  }
> >
> > +/*
> > + * During compute ids, the special "bottom" value uses NAN to represent the set
> > + * of all values. NAN is selected as it isn't a useful constant value.
> > + */
> > +#define BOTTOM NAN
> > +
> > +static struct ids union_expr(struct ids ids1, struct ids ids2)
> > +{
> > +     struct ids result = {
> > +             .val = BOTTOM,
> > +             .ids = ids__union(ids1.ids, ids2.ids),
>
> should we check for error in here?

It is possible. The only way for the union to fail is with
out-of-memory, in which case it returns NULL which is the same as the
empty set. It doesn't leak memory in this case. We only use the union
operation when computing IDs. With the events missing from ids when
metric resolution happens it will fail complaining of missing events.
Metric resolution failing is non-fatal and the command will continue
to record the metrics that didn't fail.

So the current approach is to somewhat fail and carry on to record
metrics that can be created. A different approach would be to detect
an error and fail in the parser without recording any metrics. Rather
than fail in the parser it'd be less/cleaner code to abort in
ids_union, but that wouldn't be consistent with how errors are
currently propagated up.

My feeling is the current approach is a good one and that we never
really expect the out-of-memory case to occur.

Thanks,
Ian

> jirka
>
> > +     };
> > +     return result;
> > +}
> > +
> >  #define BINARY_LONG_OP(RESULT, OP, LHS, RHS)                         \
> > -     RESULT = (long)LHS OP (long)RHS;
> > +     if (!compute_ids) {                                             \
> > +             RESULT.val = (long)LHS.val OP (long)RHS.val;            \
> > +             RESULT.ids = NULL;                                      \
> > +     } else {                                                        \
> > +             RESULT = union_expr(LHS, RHS);                          \
> > +     }
> >
> >  #define BINARY_OP(RESULT, OP, LHS, RHS)                                      \
> > -     RESULT = LHS OP RHS;
> > +     if (!compute_ids) {                                             \
> > +             RESULT.val = LHS.val OP RHS.val;                        \
> > +             RESULT.ids = NULL;                                      \
> > +     } else {                                                        \
> > +             RESULT = union_expr(LHS, RHS);                          \
> > +     }
> >
> >  %}
> >  %%
> >
> > -start:
> > -EXPR_PARSE all_expr
> > -|
> > -EXPR_OTHER all_other
> > -
> > -all_other: all_other other
> > -|
> > -
> > -other: ID
> > +start: if_expr
> >  {
> > -     expr__add_id(ctx, $1);
> > -}
> > -|
> > -MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ','
> > -|
> > -'<' | '>' | D_RATIO
> > +     if (compute_ids)
> > +             ctx->ids = ids__union($1.ids, ctx->ids);
> >
> > -all_expr: if_expr                    { *final_val = $1; }
> > +     if (final_val)
> > +             *final_val = $1.val;
> > +}
> > +;
> >
> >  if_expr: expr IF expr ELSE expr
> >  {
> > -     $$ = $3 ? $1 : $5;
> > +     if (!compute_ids) {
> > +             $$.ids = NULL;
> > +             if (fpclassify($3.val) == FP_ZERO) {
> > +                     $$.val = $5.val;
> > +             } else {
> > +                     $$.val = $1.val;
> > +             }
> > +     } else {
> > +             $$ = union_expr($1, union_expr($3, $5));
> > +     }
> >  }
> >  | expr
> >  ;
> >
> >  expr: NUMBER
> >  {
> > -     $$ = $1;
> > +     $$.val = $1;
> > +     $$.ids = NULL;
> >  }
> >  | ID
> >  {
> > -     struct expr_id_data *data;
> > -
> > -     $$ = NAN;
> > -     if (expr__resolve_id(ctx, $1, &data) == 0)
> > -             $$ = expr_id_data__value(data);
> > -
> > -     free($1);
> > +     if (!compute_ids) {
> > +             /*
> > +              * Compute the event's value from ID. If the ID isn't known then
> > +              * it isn't used to compute the formula so set to NAN.
> > +              */
> > +             struct expr_id_data *data;
> > +
> > +             $$.val = NAN;
> > +             if (expr__resolve_id(ctx, $1, &data) == 0)
> > +                     $$.val = expr_id_data__value(data);
> > +
> > +             $$.ids = NULL;
> > +             free($1);
> > +     } else {
> > +             /*
> > +              * Set the value to BOTTOM to show that any value is possible
> > +              * when the event is computed. Create a set of just the ID.
> > +              */
> > +             $$.val = BOTTOM;
> > +             $$.ids = ids__new();
> > +             if (!$$.ids || ids__insert($$.ids, $1, ctx->parent))
> > +                     YYABORT;
> > +     }
> >  }
> >  | expr '|' expr { BINARY_LONG_OP($$, |, $1, $3); }
> >  | expr '&' expr { BINARY_LONG_OP($$, &, $1, $3); }
> > @@ -102,31 +163,47 @@ expr: NUMBER
> >  | expr '*' expr { BINARY_OP($$, *, $1, $3); }
> >  | expr '/' expr
> >  {
> > -     if ($3 == 0) {
> > -             pr_debug("division by zero\n");
> > -             YYABORT;
> > +     if (!compute_ids) {
> > +             if (fpclassify($3.val) == FP_ZERO) {
> > +                     pr_debug("division by zero\n");
> > +                     YYABORT;
> > +             }
> > +             $$.val = $1.val / $3.val;
> > +             $$.ids = NULL;
> > +     } else {
> > +             $$ = union_expr($1, $3);
> >       }
> > -     $$ = $1 / $3;
> >  }
> >  | expr '%' expr
> >  {
> > -     if ((long)$3 == 0) {
> > -             pr_debug("division by zero\n");
> > -             YYABORT;
> > +     if (!compute_ids) {
> > +             if (fpclassify($3.val) == FP_ZERO) {
> > +                     pr_debug("division by zero\n");
> > +                     YYABORT;
> > +             }
> > +             $$.val = (long)$1.val % (long)$3.val;
> > +             $$.ids = NULL;
> > +     } else {
> > +             $$ = union_expr($1, $3);
> >       }
> > -     $$ = (long)$1 % (long)$3;
> >  }
> >  | D_RATIO '(' expr ',' expr ')'
> >  {
> > -     if ($5 == 0) {
> > -             $$ = 0;
> > +     if (!compute_ids) {
> > +             $$.ids = NULL;
> > +             if (fpclassify($5.val) == FP_ZERO) {
> > +                     $$.val = 0.0;
> > +             } else {
> > +                     $$.val = $3.val / $5.val;
> > +             }
> >       } else {
> > -             $$ = $3 / $5;
> > +             $$ = union_expr($3, $5);
> >       }
> >  }
> >  | '-' expr %prec NEG
> >  {
> > -     $$ = -$2;
> > +     $$.val = -$2.val;
> > +     $$.ids = $2.ids;
> >  }
> >  | '(' if_expr ')'
> >  {
> > @@ -134,15 +211,26 @@ expr: NUMBER
> >  }
> >  | MIN '(' expr ',' expr ')'
> >  {
> > -     $$ = $3 < $5 ? $3 : $5;
> > +     if (!compute_ids) {
> > +             $$.val = $3.val < $5.val ? $3.val : $5.val;
> > +             $$.ids = NULL;
> > +     } else {
> > +             $$ = union_expr($3, $5);
> > +     }
> >  }
> >  | MAX '(' expr ',' expr ')'
> >  {
> > -     $$ = $3 > $5 ? $3 : $5;
> > +     if (!compute_ids) {
> > +             $$.val = $3.val > $5.val ? $3.val : $5.val;
> > +             $$.ids = NULL;
> > +     } else {
> > +             $$ = union_expr($3, $5);
> > +     }
> >  }
> >  | SMT_ON
> >  {
> > -     $$ = smt_on() > 0 ? 1.0 : 0.0;
> > +     $$.val = smt_on() > 0 ? 1.0 : 0.0;
> > +     $$.ids = NULL;
> >  }
> >  ;
> >
> > --
> > 2.33.0.464.g1972c5931b-goog
> >
>

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH v9 11/13] perf expr: Propagate constants for binary operations
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (9 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 12/13] perf metric: Don't compute unused events Ian Rogers
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

When we're computing ID values, if we have constant values then compute
the constant result. For example:
  1 + 2
Previously .val would be set to BOTTOM by union_expr, meaning that
all values are possible. With this change .val is set to 3.
Later changes will use the constant values to hopefully eliminate ID
values that don't need to be computed.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/util/expr.y | 63 ++++++++++++++++++++++++++++++------------
 1 file changed, 45 insertions(+), 18 deletions(-)

diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 6aeead54760a..5a295e385914 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -68,6 +68,12 @@ static void expr_error(double *final_val __maybe_unused,
  */
 #define BOTTOM NAN
 
+/* During computing ids, does val represent a constant (non-BOTTOM) value? */
+static bool is_const(double val)
+{
+	return isfinite(val);
+}
+
 static struct ids union_expr(struct ids ids1, struct ids ids2)
 {
 	struct ids result = {
@@ -77,8 +83,15 @@ static struct ids union_expr(struct ids ids1, struct ids ids2)
 	return result;
 }
 
+/*
+ * If we're not computing ids or $1 and $3 are constants, compute the new
+ * constant value using OP. Its invariant that there are no ids.  If computing
+ * ids for non-constants union the set of IDs that must be computed.
+ */
 #define BINARY_LONG_OP(RESULT, OP, LHS, RHS)				\
-	if (!compute_ids) {						\
+	if (!compute_ids || (is_const(LHS.val) && is_const(RHS.val))) { \
+		assert(LHS.ids == NULL);				\
+		assert(RHS.ids == NULL);				\
 		RESULT.val = (long)LHS.val OP (long)RHS.val;		\
 		RESULT.ids = NULL;					\
 	} else {							\
@@ -86,7 +99,9 @@ static struct ids union_expr(struct ids ids1, struct ids ids2)
 	}
 
 #define BINARY_OP(RESULT, OP, LHS, RHS)					\
-	if (!compute_ids) {						\
+	if (!compute_ids || (is_const(LHS.val) && is_const(RHS.val))) { \
+		assert(LHS.ids == NULL);				\
+		assert(RHS.ids == NULL);				\
 		RESULT.val = LHS.val OP RHS.val;			\
 		RESULT.ids = NULL;					\
 	} else {							\
@@ -163,40 +178,52 @@ expr: NUMBER
 | expr '*' expr { BINARY_OP($$, *, $1, $3); }
 | expr '/' expr
 {
-	if (!compute_ids) {
-		if (fpclassify($3.val) == FP_ZERO) {
-			pr_debug("division by zero\n");
-			YYABORT;
-		}
+	if (fpclassify($3.val) == FP_ZERO) {
+		pr_debug("division by zero\n");
+		YYABORT;
+	} else if (!compute_ids || (is_const($1.val) && is_const($3.val))) {
+		assert($1.ids == NULL);
+		assert($3.ids == NULL);
 		$$.val = $1.val / $3.val;
 		$$.ids = NULL;
 	} else {
+		/* LHS and/or RHS need computing from event IDs so union. */
 		$$ = union_expr($1, $3);
 	}
 }
 | expr '%' expr
 {
-	if (!compute_ids) {
-		if (fpclassify($3.val) == FP_ZERO) {
-			pr_debug("division by zero\n");
-			YYABORT;
-		}
+	if (fpclassify($3.val) == FP_ZERO) {
+		pr_debug("division by zero\n");
+		YYABORT;
+	} else if (!compute_ids || (is_const($1.val) && is_const($3.val))) {
+		assert($1.ids == NULL);
+		assert($3.ids == NULL);
 		$$.val = (long)$1.val % (long)$3.val;
 		$$.ids = NULL;
 	} else {
+		/* LHS and/or RHS need computing from event IDs so union. */
 		$$ = union_expr($1, $3);
 	}
 }
 | D_RATIO '(' expr ',' expr ')'
 {
-	if (!compute_ids) {
+	if (fpclassify($5.val) == FP_ZERO) {
+		/*
+		 * Division by constant zero always yields zero and no events
+		 * are necessary.
+		 */
+		assert($5.ids == NULL);
+		$$.val = 0.0;
+		$$.ids = NULL;
+		ids__free($3.ids);
+	} else if (!compute_ids || (is_const($3.val) && is_const($5.val))) {
+		assert($3.ids == NULL);
+		assert($5.ids == NULL);
+		$$.val = $3.val / $5.val;
 		$$.ids = NULL;
-		if (fpclassify($5.val) == FP_ZERO) {
-			$$.val = 0.0;
-		} else {
-			$$.val = $3.val / $5.val;
-		}
 	} else {
+		/* LHS and/or RHS need computing from event IDs so union. */
 		$$ = union_expr($3, $5);
 	}
 }
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 12/13] perf metric: Don't compute unused events
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (10 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 11/13] perf expr: Propagate constants for binary operations Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-23  7:46 ` [PATCH v9 13/13] perf metric: Avoid events for an 'if' constant result Ian Rogers
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

For a metric like:
  EVENT1 if #smt_on else EVENT2

currently EVENT1 and EVENT2 will be measured and then when the metric is
reported EVENT1 or EVENT2 will be printed depending on the value from
smt_on() during the expr parsing. Computing both events is unnecessary and
can lead to multiplexing as discussed in this thread:
https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/

If the input is constant to certain operators like:
 IDS1 if CONST else IDS2
then the result will be either IDS1 or IDS2 depending on CONST (which
may be evaluated from an entire expression), and so IDS1 or IDS2 may
be discarded avoiding events from being programmed.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c | 11 +++++++++++
 tools/perf/util/expr.y  | 30 +++++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 1c881bea7fca..287989321d2a 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 #include "util/debug.h"
 #include "util/expr.h"
+#include "util/smt.h"
 #include "tests.h"
 #include <stdlib.h>
 #include <string.h>
@@ -132,6 +133,16 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
 	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids, "EVENT2,param=3/",
 						    (void **)&val_ptr));
 
+	/* Only EVENT1 or EVENT2 need be measured depending on the value of smt_on. */
+	expr__ctx_clear(ctx);
+	TEST_ASSERT_VAL("find ids",
+			expr__find_ids("EVENT1 if #smt_on else EVENT2",
+				NULL, ctx, 0) == 0);
+	TEST_ASSERT_VAL("find ids", hashmap__size(ctx->ids) == 1);
+	TEST_ASSERT_VAL("find ids", hashmap__find(ctx->ids,
+						  smt_on() ? "EVENT1" : "EVENT2",
+						  (void **)&val_ptr));
+
 	expr__ctx_free(ctx);
 
 	return 0;
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 5a295e385914..5b878f044f22 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -123,14 +123,30 @@ start: if_expr
 
 if_expr: expr IF expr ELSE expr
 {
-	if (!compute_ids) {
-		$$.ids = NULL;
-		if (fpclassify($3.val) == FP_ZERO) {
-			$$.val = $5.val;
-		} else {
-			$$.val = $1.val;
-		}
+	if (fpclassify($3.val) == FP_ZERO) {
+		/*
+		 * The IF expression evaluated to 0 so treat as false, take the
+		 * ELSE and discard everything else.
+		 */
+		$$.val = $5.val;
+		$$.ids = $5.ids;
+		ids__free($1.ids);
+		ids__free($3.ids);
+	} else if (!compute_ids || is_const($3.val)) {
+		/*
+		 * If ids aren't computed then treat the expression as true. If
+		 * ids are being computed and the IF expr is a non-zero
+		 * constant, then also evaluate the true case.
+		 */
+		$$.val = $1.val;
+		$$.ids = $1.ids;
+		ids__free($3.ids);
+		ids__free($5.ids);
 	} else {
+		/*
+		 * Value is either the LHS or RHS and we need the IF expression
+		 * to compute it.
+		 */
 		$$ = union_expr($1, union_expr($3, $5));
 	}
 }
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH v9 13/13] perf metric: Avoid events for an 'if' constant result
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (11 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 12/13] perf metric: Don't compute unused events Ian Rogers
@ 2021-09-23  7:46 ` Ian Rogers
  2021-09-29  5:35 ` [PATCH v9 00/13] Don't compute events that won't be used in a metric Jiri Olsa
  2021-09-29 15:19 ` John Garry
  14 siblings, 0 replies; 20+ messages in thread
From: Ian Rogers @ 2021-09-23  7:46 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, John Garry, Paul Clarke,
	kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta, Ian Rogers

For a metric like:
  CONST if expr else CONST

if the values of CONST are identical then expr doesn't need evaluating,
and events, in order to compute a result.

Signed-off-by: Ian Rogers <irogers@google.com>
---
 tools/perf/tests/expr.c |  7 +++++++
 tools/perf/util/expr.y  | 10 ++++++++++
 2 files changed, 17 insertions(+)

diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c
index 287989321d2a..f1d8411fce12 100644
--- a/tools/perf/tests/expr.c
+++ b/tools/perf/tests/expr.c
@@ -143,6 +143,13 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)
 						  smt_on() ? "EVENT1" : "EVENT2",
 						  (void **)&val_ptr));
 
+	/* The expression is a constant 1.0 without needing to evaluate EVENT1. */
+	expr__ctx_clear(ctx);
+	TEST_ASSERT_VAL("find ids",
+			expr__find_ids("1.0 if EVENT1 > 100.0 else 1.0",
+			NULL, ctx, 0) == 0);
+	TEST_ASSERT_VAL("find ids", hashmap__size(ctx->ids) == 0);
+
 	expr__ctx_free(ctx);
 
 	return 0;
diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y
index 5b878f044f22..ba7d3b667fcb 100644
--- a/tools/perf/util/expr.y
+++ b/tools/perf/util/expr.y
@@ -142,6 +142,16 @@ if_expr: expr IF expr ELSE expr
 		$$.ids = $1.ids;
 		ids__free($3.ids);
 		ids__free($5.ids);
+	} else if ($1.val == $5.val) {
+		/*
+		 * LHS == RHS, so both are an identical constant. No need to
+		 * evaluate any events.
+		 */
+		$$.val = $1.val;
+		$$.ids = NULL;
+		ids__free($1.ids);
+		ids__free($3.ids);
+		ids__free($5.ids);
 	} else {
 		/*
 		 * Value is either the LHS or RHS and we need the IF expression
-- 
2.33.0.464.g1972c5931b-goog


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 00/13] Don't compute events that won't be used in a metric.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (12 preceding siblings ...)
  2021-09-23  7:46 ` [PATCH v9 13/13] perf metric: Avoid events for an 'if' constant result Ian Rogers
@ 2021-09-29  5:35 ` Jiri Olsa
  2021-09-29 15:19 ` John Garry
  14 siblings, 0 replies; 20+ messages in thread
From: Jiri Olsa @ 2021-09-29  5:35 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Namhyung Kim, linux-kernel,
	Andi Kleen, Jin Yao, John Garry, Paul Clarke, kajoljain,
	linux-perf-users, Stephane Eranian, Sandeep Dasgupta

On Thu, Sep 23, 2021 at 12:46:03AM -0700, Ian Rogers wrote:
> For a metric like:
>   EVENT1 if #smt_on else EVENT2
>     
> currently EVENT1 and EVENT2 will be measured and then when the metric
> is reported EVENT1 or EVENT2 will be printed depending on the value
> from smt_on() during the expr parsing. Computing both events is
> unnecessary and can lead to multiplexing as discussed in this thread:
> https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/
> 
> This change modifies expression parsing so that constants are
> considered when building the set of ids (events) and only events not
> contributing to a constant value are measured.
> 
> v9. adds a missing memory allocation failure check in the pmu-metrics
>     test.  A memory allocation failure in union returns NULL. Some
>     parser debug on IDs is removed. "Modify code layout" is broken
>     apart into 3 changes.  "Don't compute unused events" is broken
>     apart into 4 changes with the tests merged into the change that
>     adds the corresponding optimization. This is trying to address
>     feedback from Jiri Olsa <jolsa@redhat.com>.  The unmodified
>     patches have Reviewed-by: Andi Kleen <ak@linux.intel.com> added,
>     although the overall patch set is largely the same as v8 which was
>     fully reviewed-by.

Acked-by: Jiri Olsa <jolsa@redhat.com>

thanks,
jirka

> 
> v8. rebases, adds an ability to compute metrics with no events and
>     further breaks apart the "Don't compute unused events" part of the
>     change as requested by Jiri Olsa <jolsa@redhat.com>.
> 
> v7. fixes the fix to be in the correct patch.
> 
> v6. rebases and fixes issues raised by Namhyung Kim <namhyung@kernel.org>,
> a memory leak and a function comment.
> 
> v5. uses macros to reduce boiler plate in patch 5/5 as suggested by
> Andi Kleen <ak@linux.intel.com>.
> 
> v4. reduces references to BOTTOM/NAN in patch 5/5 by using utility
> functions. It improves comments and fixes an unnecessary union in a
> peephole optimization.
> 
> v3. fixes an assignment in patch 2/5. In patch 5/5 additional comments
> are added and useless frees are replaced by asserts. A new peephole
> optimization is added for the case CONST IF expr ELSE CONST, where the
> the constants are identical, as we don't need to evaluate the IF
> condition.
> 
> v2. is a rebase.
> 
> Ian Rogers (13):
>   perf metric: Restructure struct expr_parse_ctx.
>   perf metric: Use NAN for missing event IDs.
>   perf expr: Remove unused headers and inline d_ratio
>   perf expr: Separate token declataion from type
>   perf expr: Use macros for operators
>   perf expr: Move actions to the left.
>   perf metric: Rename expr__find_other.
>   perf metric: Add utilities to work on ids map.
>   perf metric: Allow metrics with no events
>   perf expr: Merge find_ids and regular parsing
>   perf expr: Propagate constants for binary operations
>   perf metric: Don't compute unused events
>   perf metric: Avoid events for an 'if' constant result
> 
>  tools/perf/tests/expr.c       | 160 ++++++++++++-----
>  tools/perf/tests/pmu-events.c |  54 +++---
>  tools/perf/util/expr.c        | 121 +++++++++++--
>  tools/perf/util/expr.h        |  20 ++-
>  tools/perf/util/expr.l        |   9 -
>  tools/perf/util/expr.y        | 325 +++++++++++++++++++++++++---------
>  tools/perf/util/metricgroup.c | 145 ++++++++-------
>  tools/perf/util/stat-shadow.c |  54 +++---
>  8 files changed, 620 insertions(+), 268 deletions(-)
> 
> -- 
> 2.33.0.464.g1972c5931b-goog
> 


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 00/13] Don't compute events that won't be used in a metric.
  2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
                   ` (13 preceding siblings ...)
  2021-09-29  5:35 ` [PATCH v9 00/13] Don't compute events that won't be used in a metric Jiri Olsa
@ 2021-09-29 15:19 ` John Garry
  2021-09-29 16:09   ` Ian Rogers
  14 siblings, 1 reply; 20+ messages in thread
From: John Garry @ 2021-09-29 15:19 UTC (permalink / raw)
  To: Ian Rogers, Peter Zijlstra, Ingo Molnar,
	Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
	Jiri Olsa, Namhyung Kim, linux-kernel, Andi Kleen, Jin Yao,
	Paul Clarke, kajoljain, linux-perf-users
  Cc: Stephane Eranian, Sandeep Dasgupta

On 23/09/2021 08:46, Ian Rogers wrote:
> For a metric like:
>    EVENT1 if #smt_on else EVENT2
>      
> currently EVENT1 and EVENT2 will be measured and then when the metric
> is reported EVENT1 or EVENT2 will be printed depending on the value
> from smt_on() during the expr parsing. Computing both events is
> unnecessary and can lead to multiplexing as discussed in this thread:
> https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/
> 
> This change modifies expression parsing so that constants are
> considered when building the set of ids (events) and only events not
> contributing to a constant value are measured.

Based on some testing on my arm64 platform, no regression seen, so:

Tested-by: John Garry <john.garry@huawei.com>


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 00/13] Don't compute events that won't be used in a metric.
  2021-09-29 15:19 ` John Garry
@ 2021-09-29 16:09   ` Ian Rogers
  2021-09-29 16:51     ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 20+ messages in thread
From: Ian Rogers @ 2021-09-29 16:09 UTC (permalink / raw)
  To: John Garry
  Cc: Peter Zijlstra, Ingo Molnar, Arnaldo Carvalho de Melo,
	Mark Rutland, Alexander Shishkin, Jiri Olsa, Namhyung Kim,
	linux-kernel, Andi Kleen, Jin Yao, Paul Clarke, kajoljain,
	linux-perf-users, Stephane Eranian, Sandeep Dasgupta

On Wed, Sep 29, 2021 at 8:16 AM John Garry <john.garry@huawei.com> wrote:
>
> On 23/09/2021 08:46, Ian Rogers wrote:
> > For a metric like:
> >    EVENT1 if #smt_on else EVENT2
> >
> > currently EVENT1 and EVENT2 will be measured and then when the metric
> > is reported EVENT1 or EVENT2 will be printed depending on the value
> > from smt_on() during the expr parsing. Computing both events is
> > unnecessary and can lead to multiplexing as discussed in this thread:
> > https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/
> >
> > This change modifies expression parsing so that constants are
> > considered when building the set of ids (events) and only events not
> > contributing to a constant value are measured.
>
> Based on some testing on my arm64 platform, no regression seen, so:
>
> Tested-by: John Garry <john.garry@huawei.com>

Awesome, much thanks Jiri, John, Andi for the reviews and testing!

Ian

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH v9 00/13] Don't compute events that won't be used in a metric.
  2021-09-29 16:09   ` Ian Rogers
@ 2021-09-29 16:51     ` Arnaldo Carvalho de Melo
  0 siblings, 0 replies; 20+ messages in thread
From: Arnaldo Carvalho de Melo @ 2021-09-29 16:51 UTC (permalink / raw)
  To: Ian Rogers
  Cc: John Garry, Peter Zijlstra, Ingo Molnar, Mark Rutland,
	Alexander Shishkin, Jiri Olsa, Namhyung Kim, linux-kernel,
	Andi Kleen, Jin Yao, Paul Clarke, kajoljain, linux-perf-users,
	Stephane Eranian, Sandeep Dasgupta

Em Wed, Sep 29, 2021 at 09:09:24AM -0700, Ian Rogers escreveu:
> On Wed, Sep 29, 2021 at 8:16 AM John Garry <john.garry@huawei.com> wrote:
> >
> > On 23/09/2021 08:46, Ian Rogers wrote:
> > > For a metric like:
> > >    EVENT1 if #smt_on else EVENT2
> > >
> > > currently EVENT1 and EVENT2 will be measured and then when the metric
> > > is reported EVENT1 or EVENT2 will be printed depending on the value
> > > from smt_on() during the expr parsing. Computing both events is
> > > unnecessary and can lead to multiplexing as discussed in this thread:
> > > https://lore.kernel.org/lkml/20201110100346.2527031-1-irogers@google.com/
> > >
> > > This change modifies expression parsing so that constants are
> > > considered when building the set of ids (events) and only events not
> > > contributing to a constant value are measured.
> >
> > Based on some testing on my arm64 platform, no regression seen, so:
> >
> > Tested-by: John Garry <john.garry@huawei.com>
> 
> Awesome, much thanks Jiri, John, Andi for the reviews and testing!

Thanks, applied.

- Arnaldo


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2021-09-29 16:51 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-23  7:46 [PATCH v9 00/13] Don't compute events that won't be used in a metric Ian Rogers
2021-09-23  7:46 ` [PATCH v9 01/13] perf metric: Restructure struct expr_parse_ctx Ian Rogers
2021-09-23  7:46 ` [PATCH v9 02/13] perf metric: Use NAN for missing event IDs Ian Rogers
2021-09-23  7:46 ` [PATCH v9 03/13] perf expr: Remove unused headers and inline d_ratio Ian Rogers
2021-09-23  7:46 ` [PATCH v9 04/13] perf expr: Separate token declataion from type Ian Rogers
2021-09-23  7:46 ` [PATCH v9 05/13] perf expr: Use macros for operators Ian Rogers
2021-09-23  7:46 ` [PATCH v9 06/13] perf expr: Move actions to the left Ian Rogers
2021-09-23  7:46 ` [PATCH v9 07/13] perf metric: Rename expr__find_other Ian Rogers
2021-09-23  7:46 ` [PATCH v9 08/13] perf metric: Add utilities to work on ids map Ian Rogers
2021-09-23  7:46 ` [PATCH v9 09/13] perf metric: Allow metrics with no events Ian Rogers
2021-09-23  7:46 ` [PATCH v9 10/13] perf expr: Merge find_ids and regular parsing Ian Rogers
2021-09-28 20:56   ` Jiri Olsa
2021-09-28 21:20     ` Ian Rogers
2021-09-23  7:46 ` [PATCH v9 11/13] perf expr: Propagate constants for binary operations Ian Rogers
2021-09-23  7:46 ` [PATCH v9 12/13] perf metric: Don't compute unused events Ian Rogers
2021-09-23  7:46 ` [PATCH v9 13/13] perf metric: Avoid events for an 'if' constant result Ian Rogers
2021-09-29  5:35 ` [PATCH v9 00/13] Don't compute events that won't be used in a metric Jiri Olsa
2021-09-29 15:19 ` John Garry
2021-09-29 16:09   ` Ian Rogers
2021-09-29 16:51     ` Arnaldo Carvalho de Melo

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.