LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Ian Rogers <irogers@google.com>
To: Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>, Andrii Nakryiko <andriin@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@chromium.org>, Kajol Jain <kjain@linux.ibm.com>,
	Andi Kleen <ak@linux.intel.com>,
	John Garry <john.garry@huawei.com>,
	Jin Yao <yao.jin@linux.intel.com>,
	Kan Liang <kan.liang@linux.intel.com>,
	Cong Wang <xiyou.wangcong@gmail.com>,
	Kim Phillips <kim.phillips@amd.com>,
	linux-kernel@vger.kernel.org
Cc: netdev@vger.kernel.org, bpf@vger.kernel.org,
	linux-perf-users@vger.kernel.org,
	Stephane Eranian <eranian@google.com>,
	Ian Rogers <irogers@google.com>
Subject: [RFC PATCH v2 00/23] Share events between metrics
Date: Thu,  7 May 2020 07:07:56 -0700
Message-ID: <20200507140819.126960-1-irogers@google.com> (raw)

Metric groups contain metrics. Metrics create groups of events to
ideally be scheduled together. Often metrics refer to the same events,
for example, a cache hit and cache miss rate. Using separate event
groups means these metrics are multiplexed at different times and the
counts don't sum to 100%. More multiplexing also decreases the
accuracy of the measurement.

This change orders metrics from groups or the command line, so that
the ones with the most events are set up first. Later metrics see if
groups already provide their events, and reuse them if
possible. Unnecessary events and groups are eliminated.

RFC because:
 - without this change events within a metric may get scheduled
   together, after they may appear as part of a larger group and be
   multiplexed at different times, lowering accuracy - however, less
   multiplexing may compensate for this.
 - libbpf's hashmap is used, however, libbpf is an optional
   requirement for building perf.
 - other things I'm not thinking of.

Thanks!

v2. is the entire patch set based on acme's perf/core tree and includes a
cherry-picks. Patch 13 was sent for review to the bpf maintainers here:
https://lore.kernel.org/lkml/20200506205257.8964-2-irogers@google.com/
v1. was based on the perf metrics fixes and test sent here:
https://lore.kernel.org/lkml/20200501173333.227162-1-irogers@google.com/

Andrii Nakryiko (1):
  libbpf: Fix memory leak and possible double-free in hashmap__clear

Ian Rogers (22):
  perf expr: unlimited escaped characters in a symbol
  perf metrics: fix parse errors in cascade lake metrics
  perf metrics: fix parse errors in skylake metrics
  perf expr: allow ',' to be an other token
  perf expr: increase max other
  perf expr: parse numbers as doubles
  perf expr: debug lex if debugging yacc
  perf metrics: fix parse errors in power8 metrics
  perf metrics: fix parse errors in power9 metrics
  perf expr: print a debug message for division by zero
  perf parse-events: expand add PMU error/verbose messages
  perf test: improve pmu event metric testing
  lib/bpf hashmap: increase portability
  perf expr: fix memory leaks in bison
  perf evsel: fix 2 memory leaks
  perf expr: migrate expr ids table to libbpf's hashmap
  perf metricgroup: change evlist_used to a bitmap
  perf metricgroup: free metric_events on error
  perf metricgroup: always place duration_time last
  perf metricgroup: delay events string creation
  perf metricgroup: order event groups by size
  perf metricgroup: remove duped metric group events

 tools/lib/bpf/hashmap.c                       |   7 +
 tools/lib/bpf/hashmap.h                       |   3 +-
 tools/perf/arch/x86/util/intel-pt.c           |  32 ++-
 .../arch/powerpc/power8/metrics.json          |   2 +-
 .../arch/powerpc/power9/metrics.json          |   2 +-
 .../arch/x86/cascadelakex/clx-metrics.json    |  10 +-
 .../arch/x86/skylakex/skx-metrics.json        |   4 +-
 tools/perf/tests/builtin-test.c               |   5 +
 tools/perf/tests/expr.c                       |  33 ++-
 tools/perf/tests/pmu-events.c                 | 158 +++++++++++-
 tools/perf/tests/pmu.c                        |   4 +-
 tools/perf/tests/tests.h                      |   2 +
 tools/perf/util/evsel.c                       |   2 +
 tools/perf/util/expr.c                        | 126 ++++-----
 tools/perf/util/expr.h                        |  22 +-
 tools/perf/util/expr.l                        |  16 +-
 tools/perf/util/expr.y                        |  41 ++-
 tools/perf/util/metricgroup.c                 | 242 +++++++++++-------
 tools/perf/util/parse-events.c                |  29 ++-
 tools/perf/util/pmu.c                         |  33 ++-
 tools/perf/util/pmu.h                         |   2 +-
 tools/perf/util/stat-shadow.c                 |  46 ++--
 22 files changed, 545 insertions(+), 276 deletions(-)

-- 
2.26.2.526.g744177e7f7-goog


             reply index

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-07 14:07 Ian Rogers [this message]
2020-05-07 14:07 ` [RFC PATCH v2 01/23] perf expr: unlimited escaped characters in a symbol Ian Rogers
2020-05-07 14:07 ` [RFC PATCH v2 02/23] perf metrics: fix parse errors in cascade lake metrics Ian Rogers
2020-05-07 14:07 ` [RFC PATCH v2 03/23] perf metrics: fix parse errors in skylake metrics Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 04/23] perf expr: allow ',' to be an other token Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 05/23] perf expr: increase max other Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 06/23] perf expr: parse numbers as doubles Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 07/23] perf expr: debug lex if debugging yacc Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 08/23] perf metrics: fix parse errors in power8 metrics Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 09/23] perf metrics: fix parse errors in power9 metrics Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 10/23] perf expr: print a debug message for division by zero Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 11/23] perf parse-events: expand add PMU error/verbose messages Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 12/23] perf test: improve pmu event metric testing Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 13/23] lib/bpf hashmap: increase portability Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 14/23] libbpf: Fix memory leak and possible double-free in hashmap__clear Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 15/23] perf expr: fix memory leaks in bison Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 16/23] perf evsel: fix 2 memory leaks Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 17/23] perf expr: migrate expr ids table to libbpf's hashmap Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 18/23] perf metricgroup: change evlist_used to a bitmap Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 19/23] perf metricgroup: free metric_events on error Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 20/23] perf metricgroup: always place duration_time last Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 21/23] perf metricgroup: delay events string creation Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 22/23] perf metricgroup: order event groups by size Ian Rogers
2020-05-07 14:08 ` [RFC PATCH v2 23/23] perf metricgroup: remove duped metric group events Ian Rogers
2020-10-02 11:57   ` Issue of metrics for multiple uncore PMUs (was Re: [RFC PATCH v2 23/23] perf metricgroup: remove duped metric group events) John Garry
2020-10-02 20:46     ` Ian Rogers
2020-10-05 10:03       ` John Garry
2020-10-05 16:28         ` Ian Rogers
2020-10-05 18:05           ` John Garry
2020-10-06 14:19             ` John Garry
2020-10-06 14:42               ` Ian Rogers

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200507140819.126960-1-irogers@google.com \
    --to=irogers@google.com \
    --cc=acme@kernel.org \
    --cc=ak@linux.intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=andriin@fb.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=eranian@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=john.garry@huawei.com \
    --cc=jolsa@redhat.com \
    --cc=kafai@fb.com \
    --cc=kan.liang@linux.intel.com \
    --cc=kim.phillips@amd.com \
    --cc=kjain@linux.ibm.com \
    --cc=kpsingh@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=songliubraving@fb.com \
    --cc=xiyou.wangcong@gmail.com \
    --cc=yao.jin@linux.intel.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git