linux-perf-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Ian Rogers <irogers@google.com>
Cc: Matthew Wilcox <willy@infradead.org>,
	Jiri Olsa <jolsa@kernel.org>, Alexei Starovoitov <ast@kernel.org>,
	Andrii Nakryiko <andrii@kernel.org>, Hao Luo <haoluo@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	bpf@vger.kernel.org, linux-mm@kvack.org,
	linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-perf-users@vger.kernel.org, Martin KaFai Lau <kafai@fb.com>,
	Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	KP Singh <kpsingh@chromium.org>,
	Stanislav Fomichev <sdf@google.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Namhyung Kim <namhyung@gmail.com>,
	Dave Chinner <david@fromorbit.com>
Subject: Re: [PATCHv3 bpf-next 0/9] mm/bpf/perf: Store build id in file object
Date: Thu, 16 Mar 2023 14:51:52 -0700	[thread overview]
Message-ID: <CAEf4BzYgyGTVv=cDwaW+DBke1uk_aLCg3CB_9W6+9tkS8Nyn_Q@mail.gmail.com> (raw)
In-Reply-To: <CAP-5=fVYriALLwF2FU1ZUtLuHndnvPw=3SctVqY6Uwex8JfscA@mail.gmail.com>

On Thu, Mar 16, 2023 at 10:50 AM Ian Rogers <irogers@google.com> wrote:
>
> On Thu, Mar 16, 2023 at 10:35 AM Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Thu, Mar 16, 2023 at 06:01:40PM +0100, Jiri Olsa wrote:
> > > hi,
> > > this patchset adds build id object pointer to struct file object.
> > >
> > > We have several use cases for build id to be used in BPF programs
> > > [2][3].
> >
> > Yes, you have use cases, but you never answered the question I asked:
> >
> > Is this going to be enabled by every distro kernel, or is it for special
> > use-cases where only people doing a very specialised thing who are
> > willing to build their own kernels will use it?
> >
> > Saying "hubble/tetragon" doesn't answer that question.  Maybe it does
> > to you, but I have no idea what that software is.
> >
> > Put it another way: how does this make *MY* life better?  Literally me.
> > How will it affect my life?
>
> So at Google we use build IDs for all profiling, I believe Meta is the
> same but obviously I can't speak for them. For BPF program stack

Yep, Meta is also capturing stack traces with build ID as well, if
possible. Build IDs help with profiling short-lived processes which
exit before the profiling session is done and user-space tooling is
able to collect /proc/<pid>/maps contents (which is what Ian is
referring to here). But also build ID allows to offload more of the
expensive stack symbolization process (converting raw memory addresses
into human readable function+offset+file path+line numbers
information) to dedicated remote servers, by allowing to cache and
reuse preprocessed DWARF/ELF information based on build ID.

I believe perf tool is also using build ID, so any tool relying on
perf capturing full and complete profiling data for system-wide
performance analysis would benefit as well.

Generally speaking, there is a whole ecosystem built on top of
assumption that binaries have build ID and profiling tooling is able
to provide more value if those build IDs are more reliably collected.
Which ultimately benefits the entire open-source ecosystem by allowing
people to spot issues (not necessarily just performance, it could be
correctness issues as well) more reliably, fix them, and benefit every
user.

> traces, using build ID + offset stack traces is preferable to perf's
> whole system synthesis of mmap events based on data held in
> /proc/pid/maps. Individual stack traces are larger, but you avoid the
> ever growing problem of coming up with some initial virtual memory
> state that will allow you to identify samples.
>
> This doesn't answer the question about how this will help you, but I
> expect over time you will see scalability issues and also want to use
> tools assuming build IDs are present and cheap to access.
>
> Thanks,
> Ian

  reply	other threads:[~2023-03-16 21:52 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-16 17:01 [PATCHv3 bpf-next 0/9] mm/bpf/perf: Store build id in file object Jiri Olsa
2023-03-16 17:01 ` [PATCHv3 bpf-next 1/9] mm: " Jiri Olsa
2023-03-16 22:07   ` Andrii Nakryiko
2023-03-16 17:01 ` [PATCHv3 bpf-next 2/9] perf: Use file object build id in perf_event_mmap_event Jiri Olsa
2023-03-16 17:01 ` [PATCHv3 bpf-next 3/9] bpf: Use file object build id in stackmap Jiri Olsa
2023-03-16 22:07   ` Andrii Nakryiko
2023-03-16 17:01 ` [PATCHv3 bpf-next 4/9] bpf: Switch BUILD_ID_SIZE_MAX to enum Jiri Olsa
2023-03-16 22:07   ` Andrii Nakryiko
2023-03-16 17:01 ` [PATCHv3 bpf-next 5/9] selftests/bpf: Add read_buildid function Jiri Olsa
2023-03-16 22:23   ` Andrii Nakryiko
2023-03-30 22:05     ` Jiri Olsa
2023-03-16 17:01 ` [PATCHv3 bpf-next 6/9] selftests/bpf: Add err.h header Jiri Olsa
2023-03-16 22:24   ` Andrii Nakryiko
2023-03-16 17:01 ` [PATCHv3 bpf-next 7/9] selftests/bpf: Replace extract_build_id with read_build_id Jiri Olsa
2023-03-16 17:01 ` [PATCHv3 bpf-next 8/9] selftests/bpf: Add iter_task_vma_buildid test Jiri Olsa
2023-03-16 22:31   ` Andrii Nakryiko
2023-03-16 17:01 ` [PATCHv3 bpf-next 9/9] selftests/bpf: Add file_build_id test Jiri Olsa
2023-03-16 19:59   ` Daniel Borkmann
2023-03-16 22:36   ` Andrii Nakryiko
2023-03-16 17:34 ` [PATCHv3 bpf-next 0/9] mm/bpf/perf: Store build id in file object Matthew Wilcox
2023-03-16 17:50   ` Ian Rogers
2023-03-16 21:51     ` Andrii Nakryiko [this message]
2023-03-17  3:51       ` Matthew Wilcox
2023-03-17 16:33         ` Andrii Nakryiko
2023-03-17 21:14           ` Al Viro
2023-03-17 21:21             ` Al Viro
2023-03-18  6:08               ` Andrii Nakryiko
2023-03-18  8:34                 ` Jiri Olsa
2023-03-18  8:33   ` Jiri Olsa
2023-03-18 15:16     ` Matthew Wilcox
2023-03-18 17:40       ` Jiri Olsa
2023-03-22 15:45       ` Arnaldo Carvalho de Melo
2023-03-31 18:19         ` Andrii Nakryiko
2023-03-31 18:36           ` Matthew Wilcox
2023-03-31 20:27             ` Andrii Nakryiko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAEf4BzYgyGTVv=cDwaW+DBke1uk_aLCg3CB_9W6+9tkS8Nyn_Q@mail.gmail.com' \
    --to=andrii.nakryiko@gmail.com \
    --cc=acme@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=david@fromorbit.com \
    --cc=haoluo@google.com \
    --cc=irogers@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kafai@fb.com \
    --cc=kpsingh@chromium.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@gmail.com \
    --cc=peterz@infradead.org \
    --cc=sdf@google.com \
    --cc=songliubraving@fb.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).