Re: [RFC/PATCH 00/14] perf record: Add support to store data in directory

From: Stephane Eranian <eranian@google.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>,
	Alexey Budankov <alexey.budankov@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>, lkml <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Adrian Hunter <adrian.hunter@intel.com>,
	Andi Kleen <ak@linux.intel.com>
Subject: Re: [RFC/PATCH 00/14] perf record: Add support to store data in directory
Date: Mon, 11 Feb 2019 10:34:16 -0800	[thread overview]
Message-ID: <CABPqkBSkyHRaXN-oVLuJJ=aPEBfqOAunRUReHsxESnC=KKQuPg@mail.gmail.com> (raw)
In-Reply-To: <20190211101957.GB14253@krava>

Jiri,

On Mon, Feb 11, 2019 at 2:20 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Tue, Feb 05, 2019 at 02:37:27PM +0100, Jiri Olsa wrote:
> > On Mon, Feb 04, 2019 at 02:44:37PM -0800, Stephane Eranian wrote:
> > > Jiri,
> > >
> > > While you're looking at the output format, I think it would be good
> > > time to simplify the code handling perf.data file.
> > > Today, perf record can emit in two formats: file mode or pipe mode.
> > > This adds complexity in the code and
> > > is error prone as the file mode path is tested more than the pipe mode
> > > path. We have run into multiple issues with
> > > the pipe mode in recent years. There is no real reason why we need to
> > > maintain two formats. If I recall, the pipe format
> > > was introduced because on pipes you cannot lseek to update the headers
> > > and therefore some of the information present as tables
> > > updated on the fly needed to be generated as pseudo records by the
> > > tool. I believe that the pipe format covers all the needs and could
> > > supersede the file mode format. That would simplify code in perf
> > > record and eliminate the risk of errors when new headers
> > > are introduced.
> >
> > yep, I think we have almost all the features covered for pipe mode,
> > and we have all necessary events to describe events features
> >
> > so with some effort we could switch off the superfluos file header
> > and use only events to describe events ;-) make sense, I'll check
> > on it
>
> so following features are not synthesized:
>
>         FEAT_OPN(TRACING_DATA,  tracing_data,   false),
>         FEAT_OPN(BUILD_ID,      build_id,       false),
>         FEAT_OPN(BRANCH_STACK,  branch_stack,   false),
>         FEAT_OPN(AUXTRACE,      auxtrace,       false),
>         FEAT_OPN(STAT,          stat,           false),
>         FEAT_OPN(CACHE,         cache,          true),
>
What do you need for BRANCH_STACK?

> I think all could be added and worked around with exception
> of BUILD_ID, which we store at the end (after processing
> all data) and we need it early in the report phase
>
Buildids are injected after the fact via perf inject when in pipe mode.

> maybe it's time to re-think that buildid -> mmap event
> association again, because it's pain in current implementation
> as well
>
Sure, but what do you propose?

> looks like bpf code is actualy getting build ids and storing
> it for the callchains in kernel.. we can check if we can do
> something similar for mmap events
>
> jirka