From: Namhyung Kim <namhyung@kernel.org>
To: "Bayduraev, Alexey V" <alexey.v.bayduraev@linux.intel.com>
Cc: Alexei Budankov <abudankov@huawei.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Jiri Olsa <jolsa@redhat.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
linux-kernel <linux-kernel@vger.kernel.org>,
Andi Kleen <ak@linux.intel.com>,
Adrian Hunter <adrian.hunter@intel.com>,
Alexander Antonov <alexander.antonov@linux.intel.com>
Subject: Re: [PATCH v3 07/12] perf record: init data file at mmap buffer object
Date: Mon, 1 Mar 2021 20:44:24 +0900 [thread overview]
Message-ID: <CAM9d7chw-i7Vx+eOPDAdyh2MPQpW=t9ueGFqUH=UcyfsNi7dUg@mail.gmail.com> (raw)
In-Reply-To: <64b0859f-aad3-43fa-4e4c-81614d0c75e4@linux.intel.com>
Hello,
On Mon, Mar 1, 2021 at 8:16 PM Bayduraev, Alexey V
<alexey.v.bayduraev@linux.intel.com> wrote:
>
> Hi,
>
> On 20.11.2020 13:49, Namhyung Kim wrote:
> > On Mon, Nov 16, 2020 at 03:19:41PM +0300, Alexey Budankov wrote:
>
> <SNIP>
>
> >>
> >> @@ -1400,8 +1417,12 @@ static int record__mmap_read_evlist(struct record *rec, struct evlist *evlist,
> >> /*
> >> * Mark the round finished in case we wrote
> >> * at least one event.
> >> + *
> >> + * No need for round events in directory mode,
> >> + * because per-cpu maps and files have data
> >> + * sorted by kernel.
> >
> > But it's not just for single cpu since task can migrate so we need to
> > look at other cpu's data too. Thus we use the ordered events queue
> > and round events help to determine when to flush the data. Without
> > the round events, it'd consume huge amount of memory during report.
> >
> > If we separate tracking records and process them first, we should be
> > able to process samples immediately without sorting them in the
> > ordered event queue. This will save both cpu cycles and memory
> > footprint significantly IMHO.
> >
> > Thanks,
> > Namhyung
> >
>
> As far as I understand, to split tracing records (FORK/MMAP/COMM) into
> a separate file, we need to implement a runtime trace decoder on the
> perf-record side to recognize such tracing records coming from the kernel.
> Is that what you mean?
No, I meant separating the mmap buffers so that the record process
can save the data without decoding.
>
> IMHO this can be tricky to implement and adds some overhead that can lead
> to possible data loss. Do you have any other ideas how to optimize memory
> consumption on perf-report side without a runtime trace decoder?
> Maybe "round events" would somehow help in directory mode?
>
> BTW In our tool we use another approach: two-pass trace file loading.
> The first loads tracing records, the second loads samples.
Yeah, something like that. With the separated data, we can do it
more efficiently IMHO.
Thanks,
Namhyung
next prev parent reply other threads:[~2021-03-01 11:45 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-16 12:12 [PATCH v3 00/12] Introduce threaded trace streaming for basic perf record operation Alexey Budankov
2020-11-16 12:14 ` [PATCH v3 01/12] perf record: introduce thread affinity and mmap masks Alexey Budankov
2020-11-20 10:01 ` Namhyung Kim
2020-11-16 12:15 ` [PATCH v3 02/12] perf record: introduce thread specific data array Alexey Budankov
2020-11-20 10:14 ` Namhyung Kim
2020-11-16 12:16 ` [PATCH v3 03/12] perf record: introduce thread local variable Alexey Budankov
2020-11-20 10:20 ` Namhyung Kim
2020-11-16 12:17 ` [PATCH v3 04/12] perf record: stop threads in the end of trace streaming Alexey Budankov
2020-11-16 12:18 ` [PATCH v3 05/12] perf record: start threads in the beginning " Alexey Budankov
2020-11-16 12:18 ` [PATCH v3 06/12] perf record: introduce data file at mmap buffer object Alexey Budankov
2020-11-20 10:28 ` Namhyung Kim
2020-11-16 12:19 ` [PATCH v3 07/12] perf record: init " Alexey Budankov
2020-11-20 10:49 ` Namhyung Kim
2021-03-01 11:16 ` Bayduraev, Alexey V
2021-03-01 11:44 ` Namhyung Kim [this message]
2021-03-01 13:33 ` Bayduraev, Alexey V
2021-03-01 14:20 ` Namhyung Kim
2020-11-16 12:20 ` [PATCH v3 08/12] perf record: introduce --threads=<spec> command line option Alexey Budankov
2020-11-20 11:09 ` Namhyung Kim
2020-11-16 12:21 ` [PATCH v3 09/12] perf record: document parallel data streaming mode Alexey Budankov
2020-11-16 12:22 ` [PATCH v3 10/12] perf report: output data file name in raw trace dump Alexey Budankov
2020-11-16 12:22 ` [PATCH v3 11/12] perf session: load data directory files for analysis Alexey Budankov
2020-11-16 12:25 ` [PATCH v3 12/12] perf session: use reader functions to load perf data file Alexey Budankov
2020-11-20 9:45 ` [PATCH v3 00/12] Introduce threaded trace streaming for basic perf record operation Namhyung Kim
2020-12-15 15:05 ` Alexei Budankov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAM9d7chw-i7Vx+eOPDAdyh2MPQpW=t9ueGFqUH=UcyfsNi7dUg@mail.gmail.com' \
--to=namhyung@kernel.org \
--cc=abudankov@huawei.com \
--cc=acme@kernel.org \
--cc=adrian.hunter@intel.com \
--cc=ak@linux.intel.com \
--cc=alexander.antonov@linux.intel.com \
--cc=alexander.shishkin@linux.intel.com \
--cc=alexey.v.bayduraev@linux.intel.com \
--cc=jolsa@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).