From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Kui-Feng Lee <kuifeng@fb.com>,
dwarves@vger.kernel.org,
Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>, bpf <bpf@vger.kernel.org>
Subject: Re: [PATCH dwarves v4 3/4] pahole: Use per-thread btf instances to avoid mutex locking.
Date: Thu, 27 Jan 2022 17:12:02 -0300 [thread overview]
Message-ID: <YfL8kjM30uHN3qxs@kernel.org> (raw)
In-Reply-To: <CAEf4BzYwOWJsfYMOLPt+cX=AB2pFSbcesH-6q_O-AqVT8=CnsQ@mail.gmail.com>
Em Wed, Jan 26, 2022 at 11:58:27AM -0800, Andrii Nakryiko escreveu:
> On Wed, Jan 26, 2022 at 11:21 AM Kui-Feng Lee <kuifeng@fb.com> wrote:
> >
> > Create an instance of btf for each worker thread, and add type info to
> > the local btf instance in the steal-function of pahole without mutex
> > acquiring. Once finished with all worker threads, merge all
> > per-thread btf instances to the primary btf instance.
> >
> > Signed-off-by: Kui-Feng Lee <kuifeng@fb.com>
> > ---
>
> There are still unnecessary casts and missing {} in the else branch,
> but I'll let Arnaldo decide or fix it up.
>
> Once this lands, can you please send kernel patch to use -j if pahole
> support it during the kernel build? See scripts/pahole-version.sh and
> scripts/link-vmlinux.sh for how pahole is set up and used in the
> kernel. Thanks!
I also tweaked this:
diff --git a/pahole.c b/pahole.c
index 8dcd6bf951fe1f93..1b2b19b2be45d30c 100644
--- a/pahole.c
+++ b/pahole.c
@@ -2887,13 +2887,13 @@ static enum load_steal_kind pahole_stealer(struct cu *cu,
static pthread_mutex_t btf_lock = PTHREAD_MUTEX_INITIALIZER;
struct btf_encoder *encoder;
+ pthread_mutex_lock(&btf_lock);
/*
* FIXME:
*
* This should be really done at main(), but since in the current codebase only at this
* point we'll have cu->elf setup...
*/
- pthread_mutex_lock(&btf_lock);
if (!btf_encoder) {
/*
* btf_encoder is the primary encoder.
⬢[acme@toolbox pahole]$
As moving it to after that comment will only make the patch a bit
larger, changing nothing.
+++ b/pahole.c
@@ -2900,11 +2900,9 @@ static enum load_steal_kind pahole_stealer(struct cu *cu,
* And, it is used by the thread
* create it.
*/
- btf_encoder = btf_encoder__new(cu, detached_btf_filename,
- conf_load->base_btf,
- skip_encoding_btf_vars,
- btf_encode_force,
- btf_gen_floats, global_verbose);
+ btf_encoder = btf_encoder__new(cu, detached_btf_filename, conf_load->base_btf, skip_encoding_btf_vars,
+ btf_encode_force, btf_gen_floats, global_verbose);
+
if (btf_encoder && thr_data) {
struct thread_data *thread = thr_data;
⬢[acme@toolbox pahole]$
i.e. cosmetic stuff to make the patch smaller by keeping preexisting
lines as-is.
And the missing {} Andrii noticed:
diff --git a/pahole.c b/pahole.c
index 7e2e37582f21c566..8c0a982f05c9ae3d 100644
--- a/pahole.c
+++ b/pahole.c
@@ -2937,8 +2937,9 @@ static enum load_steal_kind pahole_stealer(struct cu *cu,
thread->btf = btf_encoder__btf(thread->encoder);
}
encoder = thread->encoder;
- } else
+ } else {
encoder = btf_encoder;
+ }
if (btf_encoder__encode_cu(encoder, cu)) {
fprintf(stderr, "Encountered error while encoding BTF.\n");
⬢[acme@toolbox pahole]$
I'll look at the needless casts and push it to the 'next' and tmp.master
branches so that libbpf's CI can have a chance to test it.
I also added 'perf stat' results for 1.21 (the one in this fedora 34
workstation), 1.23 (parallel DWARF loading) and with your patches + the
libbpf update as a committer testing section, please add performance
numbers in in future work.
Thanks, applied!
- Arnaldo
next prev parent reply other threads:[~2022-01-27 20:14 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-26 19:20 [PATCH dwarves v4 0/4] Parallelize BTF type info generating of pahole Kui-Feng Lee
2022-01-26 19:20 ` [PATCH dwarves v4 1/4] dwarf_loader: Receive per-thread data on worker threads Kui-Feng Lee
2022-01-26 19:55 ` Andrii Nakryiko
2022-01-26 20:14 ` Kui-Feng Lee
2022-01-27 10:05 ` Arnaldo Carvalho de Melo
2022-03-08 23:45 ` Andrii Nakryiko
2022-03-09 19:24 ` Arnaldo Carvalho de Melo
2022-03-10 0:14 ` Andrii Nakryiko
2022-03-10 0:18 ` Andrii Nakryiko
2022-03-21 19:06 ` Kui-Feng Lee
2022-03-21 21:08 ` Arnaldo Carvalho de Melo
2022-03-14 15:38 ` Arnaldo Carvalho de Melo
2022-03-14 15:43 ` Arnaldo Carvalho de Melo
2022-01-26 19:20 ` [PATCH dwarves v4 2/4] dwarf_loader: Prepare and pass per-thread data to " Kui-Feng Lee
2022-01-26 19:55 ` Andrii Nakryiko
2022-01-26 19:20 ` [PATCH dwarves v4 3/4] pahole: Use per-thread btf instances to avoid mutex locking Kui-Feng Lee
2022-01-26 19:58 ` Andrii Nakryiko
2022-01-26 20:57 ` Kui-Feng Lee
2022-01-27 10:05 ` Arnaldo Carvalho de Melo
2022-01-27 20:12 ` Arnaldo Carvalho de Melo [this message]
2022-01-28 19:50 ` Arnaldo Carvalho de Melo
2022-02-01 6:56 ` Andrii Nakryiko
2022-02-02 0:16 ` Arnaldo Carvalho de Melo
2022-01-26 19:20 ` [PATCH dwarves v4 4/4] libbpf: Update libbpf to a new revision Kui-Feng Lee
2022-01-26 19:59 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YfL8kjM30uHN3qxs@kernel.org \
--to=acme@kernel.org \
--cc=andrii.nakryiko@gmail.com \
--cc=andrii@kernel.org \
--cc=arnaldo.melo@gmail.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=dwarves@vger.kernel.org \
--cc=kuifeng@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).