From: John Garry <john.garry@huawei.com>
To: Ian Rogers <irogers@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@redhat.com>,
"Namhyung Kim" <namhyung@kernel.org>,
Kajol Jain <kjain@linux.ibm.com>,
linux-perf-users <linux-perf-users@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Stephane Eranian <eranian@google.com>
Subject: Re: [PATCH] perf test: Make metric testing more robust.
Date: Wed, 4 Aug 2021 18:18:56 +0100 [thread overview]
Message-ID: <3e853906-f758-8ad6-2297-27e4f097591a@huawei.com> (raw)
In-Reply-To: <CAP-5=fWWvGi8gGfwGORF-4y7V6G_gCxY7=SX2zo_cyxKxTEf+A@mail.gmail.com>
On 04/08/2021 15:55, Ian Rogers wrote:
>
>
> On Wed, Aug 4, 2021, 2:11 AM John Garry <john.garry@huawei.com
> <mailto:john.garry@huawei.com>> wrote:
>
> On 04/08/2021 08:25, Ian Rogers wrote:
> > When testing metric expressions we fake counter values from 1 going
> > upward. For some metrics this can yield negative values that are
> clipped
> > to zero, and then cause divide by zero failures. A workaround for
> this
> > case is to try a second time with counter values going in the
> opposite
> > direction.
> >
> > This case was seen in a metric like:
> > event1 / max(event2 - event3, 0)
>
> is this the standard method to make the metric evaluation fail when
> results are not as expected? In this example, event2 should be greater
> than event3 always. Dividing by max(x, 0) would seem a bit silly.
>
>
> I wouldn't say it was standard but it is in a metric a third party gave
> us.
I agree that making it more robust is a good thing. But masking bogus
expressions isn't great. After all, we're here to find them :)
> It would be possible to get the same test failure on more standard
> expressions, so it would be nice if these tests were more robust.
so something like this would fail also:
event1 / (event2 + event3 - 1 - event4)
assuming we have ascending values from 1 for event1. And this would seem
a valid expression.
Anyway, it would be nice if we could reject max(0, x) and any divide by
negative numbers, apart from your change.
Cheers,
john
> >
> > Signed-off-by: Ian Rogers <irogers@google.com
> <mailto:irogers@google.com>>
> > ---
> > tools/perf/tests/pmu-events.c | 32 ++++++++++++++++++++++++++------
> > 1 file changed, 26 insertions(+), 6 deletions(-)
> >
> > diff --git a/tools/perf/tests/pmu-events.c
> b/tools/perf/tests/pmu-events.c
> > index b8aff8fb50d8..6c1cd58605c1 100644
> > --- a/tools/perf/tests/pmu-events.c
> > +++ b/tools/perf/tests/pmu-events.c
> > @@ -600,8 +600,18 @@ static int test_parsing(void)
> > }
> >
> > if (expr__parse(&result, &ctx,
> pe->metric_expr, 0)) {
> > - expr_failure("Parse failed", map, pe);
> > - ret++;
> > + /*
> > + * Parsing failed, make numbers go
> from large to
> > + * small which can resolve divide
> by zero
> > + * issues.
> > + */
> > + k = 1024;
> > + hashmap__for_each_entry((&ctx.ids),
> cur, bkt)
> > + expr__add_id_val(&ctx,
> strdup(cur->key), k--);
> > + if (expr__parse(&result, &ctx,
> pe->metric_expr, 0)) {
> > + expr_failure("Parse
> failed", map, pe);
> > + ret++;
> > + }
> > }
> > expr__ctx_clear(&ctx);
> > }
> > @@ -656,10 +666,20 @@ static int metric_parse_fake(const char *str)
> > }
> > }
> >
> > - if (expr__parse(&result, &ctx, str, 0))
> > - pr_err("expr__parse failed\n");
> > - else
> > - ret = 0;
> > + ret = 0;
> > + if (expr__parse(&result, &ctx, str, 0)) {
> > + /*
> > + * Parsing failed, make numbers go from large to
> small which can
> > + * resolve divide by zero issues.
> > + */
> > + i = 1024;
> > + hashmap__for_each_entry((&ctx.ids), cur, bkt)
> > + expr__add_id_val(&ctx, strdup(cur->key), i--);
> > + if (expr__parse(&result, &ctx, str, 0)) {
> > + pr_err("expr__parse failed\n");
> > + ret = -1;
> > + }
> > + }
> >
> > out:
> > expr__ctx_clear(&ctx);
> >
>
next prev parent reply other threads:[~2021-08-04 17:19 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-04 7:25 [PATCH] perf test: Make metric testing more robust Ian Rogers
2021-08-04 9:11 ` John Garry
[not found] ` <CAP-5=fWWvGi8gGfwGORF-4y7V6G_gCxY7=SX2zo_cyxKxTEf+A@mail.gmail.com>
2021-08-04 17:18 ` John Garry [this message]
2021-09-16 0:14 ` Ian Rogers
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3e853906-f758-8ad6-2297-27e4f097591a@huawei.com \
--to=john.garry@huawei.com \
--cc=acme@kernel.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=eranian@google.com \
--cc=irogers@google.com \
--cc=jolsa@redhat.com \
--cc=kjain@linux.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).