From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Alan Maguire <alan.maguire@oracle.com>,
Eduard Zingerman <eddyz87@gmail.com>
Cc: dwarves@vger.kernel.org, Jiri Olsa <jolsa@kernel.org>,
bpf@vger.kernel.org, kuifeng@fb.com, linux@weissschuh.net
Subject: Re: [PATCH dwarves 2/3] pahole: add reproducible_build to --btf_features
Date: Tue, 16 Apr 2024 17:44:58 -0300 [thread overview]
Message-ID: <Zh7jSqXRsQ5iOKSg@x1> (raw)
In-Reply-To: <a820dddb-54a0-47e0-9a6e-e12c6341babb@oracle.com>
On Tue, Apr 16, 2024 at 04:41:20PM +0100, Alan Maguire wrote:
> On 16/04/2024 16:24, Arnaldo Carvalho de Melo wrote:
> > On Tue, Apr 16, 2024 at 03:37:17PM +0100, Alan Maguire wrote:
> >> ...as a non-standard feature, so it will not be enabled for
> >> "--btf_features=all"
> >
> > How did you test this?
> >
> > ⬢[acme@toolbox pahole]$ pahole --btf_features_strict=bgasd
> > Feature 'bgasd' in 'bgasd' is not supported. Supported BTF features are:
> > encode_force,var,float,decl_tag,type_tag,enum64,optimized_func,consistent_func,reproducible_build
> > ⬢[acme@toolbox pahole]$ pahole -j --btf_features=all,reproducible_build --btf_encode_detached=vmlinux.btf.parallel.reproducible_build-via-btf_features vmlinux
> > ⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel.reproducible_build-via-btf_features > output.vmlinux.btf.parallel.reproducible_build-via-btf_features
> > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel.reproducible
> > ⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel.reproducible_build output.vmlinux.btf.parallel.reproducible_build-via-btf_features | head
> > --- output.vmlinux.btf.parallel.reproducible_build 2024-04-16 12:20:28.513462223 -0300
> > +++ output.vmlinux.btf.parallel.reproducible_build-via-btf_features 2024-04-16 12:23:37.792962930 -0300
> > @@ -265,7 +265,7 @@
> > 'target' type_id=33 bits_offset=32
> > 'key' type_id=43 bits_offset=64
> > [164] PTR '(anon)' type_id=163
> > -[165] PTR '(anon)' type_id=35751
> > +[165] PTR '(anon)' type_id=14983
> > [166] STRUCT 'static_key' size=16 vlen=2
> > 'enabled' type_id=88 bits_offset=0
> > ⬢[acme@toolbox pahole]$
> >
> > I'm double checking things now...
> The test worked for me on x86_64/aarch64. Did you test with patch 3
> applied? Because the test in its original state prior to patch 3 sets
With all patches applied:
⬢[acme@toolbox pahole]$ git log --oneline -5
ecd3b0852ab1f1ff (HEAD -> master) tests/reproducible_build: use --btf_features=all,reproducible_build
f9c8f5856b2aafea pahole: Add reproducible_build to --btf_features
0412978e8e6f8f76 pahole: Allow --btf_features to not participate in "all"
a9738ddc828d5ea0 tests/reproducible_build: Use --btf_features=all when encoding
d7edf9ae0388fb97 tests/reproducible_build: Validate the vmlinux file
⬢[acme@toolbox pahole]$
I made some mistake:
⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.serial --btf_features=all vmlinux
⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel --btf_features=all -j vmlinux
⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel--reproducible_build --btf_features=all --reproducible_build -j vmlinux
⬢[acme@toolbox pahole]$ pahole --btf_encode_detached=vmlinux.btf.parallel--btf_features=all,reproducible_build --btf_features=all,reproducible_build -j vmlinux
⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.serial > output.vmlinux.btf.serial
⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel > output.vmlinux.btf.parallel
⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel--reproducible_build > output.vmlinux.btf.parallel--reproducible_build
⬢[acme@toolbox pahole]$ bpftool btf dump file vmlinux.btf.parallel--btf_features=all,reproducible_build > output.vmlinux.btf.parallel--btf_features=all,reproducible_build
⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel | wc -l
629165
⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel--reproducible_build | wc -l
0
⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.serial output.vmlinux.btf.parallel--btf_features=all,reproducible_build | wc -l
0
⬢[acme@toolbox pahole]$ diff -u output.vmlinux.btf.parallel--reproducible_build output.vmlinux.btf.parallel--btf_features=all,reproducible_build | wc -l
0
⬢[acme@toolbox pahole]$
And of course:
⬢[acme@toolbox pahole]$ time tests/reproducible_build.sh vmlinux
Parallel reproducible DWARF Loading/Serial BTF encoding: Ok
real 1m25.241s
user 3m10.144s
sys 0m48.104s
⬢[acme@toolbox pahole]$
> --reproducible_build before setting --btf_features=all, you won't get a
> reproducible build since the command line is saying "enable reproducible
> builds but also enable standard features only"; the second action undoes
Makes sense.
> the first. switching to using --btf_features=all,reproducible_build
> fixes things for me.
Ok, all tested now, will push to the 'next' branch, waiting for Eduard's
review.
- Arnaldo
> > - Arnaldo
> >
> >> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> >> ---
> >> man-pages/pahole.1 | 8 ++++++++
> >> pahole.c | 1 +
> >> 2 files changed, 9 insertions(+)
> >>
> >> diff --git a/man-pages/pahole.1 b/man-pages/pahole.1
> >> index 2c08e97..64de343 100644
> >> --- a/man-pages/pahole.1
> >> +++ b/man-pages/pahole.1
> >> @@ -310,6 +310,14 @@ Encode BTF using the specified feature list, or specify 'all' for all standard f
> >> in different CUs.
> >> .fi
> >>
> >> +Supported non-standard features (not enabled for 'all')
> >> +
> >> +.nf
> >> + reproducible_build Ensure generated BTF is consistent every time;
> >> + without this parallel BTF encoding can result in
> >> + inconsistent BTF ids.
> >> +.fi
> >> +
> >> So for example, specifying \-\-btf_encode=var,enum64 will result in a BTF encoding that (as well as encoding basic BTF information) will contain variables and enum64 values.
> >>
> >> .TP
> >> diff --git a/pahole.c b/pahole.c
> >> index 890ef81..38cc636 100644
> >> --- a/pahole.c
> >> +++ b/pahole.c
> >> @@ -1286,6 +1286,7 @@ struct btf_feature {
> >> BTF_FEATURE(enum64, skip_encoding_btf_enum64, true, true),
> >> BTF_FEATURE(optimized_func, btf_gen_optimized, false, true),
> >> BTF_FEATURE(consistent_func, skip_encoding_btf_inconsistent_proto, false, true),
> >> + BTF_FEATURE(reproducible_build, reproducible_build, false, false),
> >> };
> >>
> >> #define BTF_MAX_FEATURE_STR 1024
> >
> >
> >
next prev parent reply other threads:[~2024-04-16 20:45 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-16 14:37 [PATCH dwarves 0/3] pahole: support nonstandard btf_features Alan Maguire
2024-04-16 14:37 ` [PATCH dwarves 1/3] pahole: allow --btf_features to not participate in "all" Alan Maguire
2024-04-19 0:06 ` Andrii Nakryiko
2024-04-19 9:44 ` Alan Maguire
2024-04-16 14:37 ` [PATCH dwarves 2/3] pahole: add reproducible_build to --btf_features Alan Maguire
2024-04-16 15:24 ` Arnaldo Carvalho de Melo
2024-04-16 15:41 ` Alan Maguire
2024-04-16 20:44 ` Arnaldo Carvalho de Melo [this message]
2024-04-16 14:37 ` [PATCH dwarves 3/3] tests/reproducible_build: use --btf_features=all,reproducible_build Alan Maguire
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Zh7jSqXRsQ5iOKSg@x1 \
--to=acme@kernel.org \
--cc=alan.maguire@oracle.com \
--cc=bpf@vger.kernel.org \
--cc=dwarves@vger.kernel.org \
--cc=eddyz87@gmail.com \
--cc=jolsa@kernel.org \
--cc=kuifeng@fb.com \
--cc=linux@weissschuh.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).