perf build broken seemingly due to libbpf changes, checking...

linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* perf build broken seemingly due to libbpf changes, checking...
@ 2022-01-06 19:34 Arnaldo Carvalho de Melo
  2022-01-06 19:44 ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-01-06 19:34 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: Song Liu, Jiri Olsa, Namhyung Kim, bpf, Linux Kernel Mailing List

After merging torvalds/master to perf/urgent I'm getting this:

util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
   25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
      |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
   37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
      | ^~~~~~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
   46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
      | ^~~~~~~~~~~~~~~~~~~~
util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
   55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
      | ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
make[4]: *** Waiting for unfinished jobs....
util/bpf_counter.c: In function ‘bpf_target_prog_name’:
util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
   82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
   82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
      |             ^
cc1: all warnings being treated as errors
make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1

I'm checking now...

BTW I test perf builds with:

make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python


-- 

- Arnaldo

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 19:34 perf build broken seemingly due to libbpf changes, checking Arnaldo Carvalho de Melo
@ 2022-01-06 19:44 ` Arnaldo Carvalho de Melo
  2022-01-06 19:48   ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-01-06 19:44 UTC (permalink / raw)
  To: Andrii Nakryiko, Ian Rogers
  Cc: Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby, Namhyung Kim,
	bpf, Linux Kernel Mailing List

Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> After merging torvalds/master to perf/urgent I'm getting this:
> 
> util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
>    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
>       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
>    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
>       | ^~~~~~~~~~~~~~~~~~~~~~~~
> util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
>    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
>       | ^~~~~~~~~~~~~~~~~~~~
> util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
>    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
>       | ^~~~~~~~~~~~~
> cc1: all warnings being treated as errors
> make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> make[4]: *** Waiting for unfinished jobs....
> util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
>    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
>       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
>    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
>       |             ^
> cc1: all warnings being treated as errors
> make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> 
> I'm checking now...
> 
> BTW I test perf builds with:
> 
> make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python

Nevermind, this was due to a patch by Ian Rogers I was testing,
bisecting get up to the last patch, since I had merged torvalds/master
today it got me to a wrong correlation, sorry for the disturbance.

For reference, this is the patch:

http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com

- Arnaldo

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 19:44 ` Arnaldo Carvalho de Melo
@ 2022-01-06 19:48   ` Arnaldo Carvalho de Melo
  2022-01-06 21:41     ` Ian Rogers
  0 siblings, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-01-06 19:48 UTC (permalink / raw)
  To: Ian Rogers, Andrii Nakryiko
  Cc: Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby, Namhyung Kim,
	bpf, Linux Kernel Mailing List

Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > After merging torvalds/master to perf/urgent I'm getting this:
> > 
> > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> >       | ^~~~~~~~~~~~~~~~~~~~
> > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> >       | ^~~~~~~~~~~~~
> > cc1: all warnings being treated as errors
> > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > make[4]: *** Waiting for unfinished jobs....
> > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> >       |             ^
> > cc1: all warnings being treated as errors
> > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > 
> > I'm checking now...
> > 
> > BTW I test perf builds with:
> > 
> > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> 
> Nevermind, this was due to a patch by Ian Rogers I was testing,
> bisecting get up to the last patch, since I had merged torvalds/master
> today it got me to a wrong correlation, sorry for the disturbance.
> 
> For reference, this is the patch:
> 
> http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com

Ian, I have libbpf-devel installed:

⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
libbpf-0.4.0-1.fc34.x86_64
libbpf-devel-0.4.0-1.fc34.x86_64
⬢[acme@toolbox perf]$

But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
system headers for all of the homies in tools/lib/.

I bet that if I remove the libbpf-devel package it works, yeah, just
tested. So we need to make those overrides dependent on using
LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
that is -Itools/lib/, using a more finegrained approach, right?

- Arnaldo

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 19:48   ` Arnaldo Carvalho de Melo
@ 2022-01-06 21:41     ` Ian Rogers
  2022-01-06 21:44       ` Andrii Nakryiko
  0 siblings, 1 reply; 11+ messages in thread
From: Ian Rogers @ 2022-01-06 21:41 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Andrii Nakryiko, Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby,
	Namhyung Kim, bpf, Linux Kernel Mailing List

On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > After merging torvalds/master to perf/urgent I'm getting this:
> > >
> > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > >       | ^~~~~~~~~~~~~~~~~~~~
> > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > >       | ^~~~~~~~~~~~~
> > > cc1: all warnings being treated as errors
> > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > make[4]: *** Waiting for unfinished jobs....
> > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > >       |             ^
> > > cc1: all warnings being treated as errors
> > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > >
> > > I'm checking now...
> > >
> > > BTW I test perf builds with:
> > >
> > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> >
> > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > bisecting get up to the last patch, since I had merged torvalds/master
> > today it got me to a wrong correlation, sorry for the disturbance.
> >
> > For reference, this is the patch:
> >
> > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
>
> Ian, I have libbpf-devel installed:
>
> ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> libbpf-0.4.0-1.fc34.x86_64
> libbpf-devel-0.4.0-1.fc34.x86_64
> ⬢[acme@toolbox perf]$
>
> But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> system headers for all of the homies in tools/lib/.
>
> I bet that if I remove the libbpf-devel package it works, yeah, just
> tested. So we need to make those overrides dependent on using
> LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> that is -Itools/lib/, using a more finegrained approach, right?

Ugh, this is messy. The -I for tools/lib is overloaded and being used
in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and
bpf/libbpf.h can be found. Likewise, for tools/perf/util/debug.c it is
used to pick up traceevent/event-parse.h.

Assuming  LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get
different combinations of:
libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken for debug.c,
-idirafter okay
libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken for debug.c,
-idirafter broken for bpf-event.c
libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but
headers mismatched, -idirafter okay
libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to link
bpf-event.c, -idirafter broken for bpf-event.c

As the choice of -I and -idirafter are binary then there will always
be a way to break the build. We could modify the build so that the
-I/-idirafter only applies to the affected C files. This postpones the
problem to when libbpf and libtracevent are in the same file, which
doesn't happen currently. I think if you want the dynamic behavior
then you need to use idirafter.

Thanks,
Ian

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 21:41     ` Ian Rogers
@ 2022-01-06 21:44       ` Andrii Nakryiko
  2022-01-06 22:04         ` Ian Rogers
  0 siblings, 1 reply; 11+ messages in thread
From: Andrii Nakryiko @ 2022-01-06 21:44 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Arnaldo Carvalho de Melo, Song Liu, Jiri Olsa, Steven Rostedt,
	Jiri Slaby, Namhyung Kim, bpf, Linux Kernel Mailing List,
	Quentin Monnet

On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
>
> On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > After merging torvalds/master to perf/urgent I'm getting this:
> > > >
> > > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > > >       | ^~~~~~~~~~~~~~~~~~~~
> > > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > > >       | ^~~~~~~~~~~~~
> > > > cc1: all warnings being treated as errors
> > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > > make[4]: *** Waiting for unfinished jobs....
> > > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > >       |             ^
> > > > cc1: all warnings being treated as errors
> > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > > >
> > > > I'm checking now...
> > > >
> > > > BTW I test perf builds with:
> > > >
> > > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> > >
> > > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > > bisecting get up to the last patch, since I had merged torvalds/master
> > > today it got me to a wrong correlation, sorry for the disturbance.
> > >
> > > For reference, this is the patch:
> > >
> > > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
> >
> > Ian, I have libbpf-devel installed:
> >
> > ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> > libbpf-0.4.0-1.fc34.x86_64
> > libbpf-devel-0.4.0-1.fc34.x86_64
> > ⬢[acme@toolbox perf]$
> >
> > But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> > system headers for all of the homies in tools/lib/.
> >
> > I bet that if I remove the libbpf-devel package it works, yeah, just
> > tested. So we need to make those overrides dependent on using
> > LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> > that is -Itools/lib/, using a more finegrained approach, right?
>
> Ugh, this is messy. The -I for tools/lib is overloaded and being used
> in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and

can you do `make install` for libbpf instead and have it install
headers into a dedicated target directory which can be added into -I
search path. Quentin did this for all the other libbpf users in kernel
tree (bpftool, resolve_btfids, etc) and it works great.


> bpf/libbpf.h can be found. Likewise, for tools/perf/util/debug.c it is
> used to pick up traceevent/event-parse.h.
>
> Assuming  LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get
> different combinations of:
> libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken for debug.c,
> -idirafter okay
> libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken for debug.c,
> -idirafter broken for bpf-event.c
> libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but
> headers mismatched, -idirafter okay
> libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to link
> bpf-event.c, -idirafter broken for bpf-event.c
>
> As the choice of -I and -idirafter are binary then there will always
> be a way to break the build. We could modify the build so that the
> -I/-idirafter only applies to the affected C files. This postpones the
> problem to when libbpf and libtracevent are in the same file, which
> doesn't happen currently. I think if you want the dynamic behavior
> then you need to use idirafter.
>
> Thanks,
> Ian

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 21:44       ` Andrii Nakryiko
@ 2022-01-06 22:04         ` Ian Rogers
  2022-01-07  3:30           ` Ian Rogers
  0 siblings, 1 reply; 11+ messages in thread
From: Ian Rogers @ 2022-01-06 22:04 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: Arnaldo Carvalho de Melo, Song Liu, Jiri Olsa, Steven Rostedt,
	Jiri Slaby, Namhyung Kim, bpf, Linux Kernel Mailing List,
	Quentin Monnet

On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> >
> > On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
> > <acme@kernel.org> wrote:
> > >
> > > Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > After merging torvalds/master to perf/urgent I'm getting this:
> > > > >
> > > > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > > > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > > > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > > > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > > > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > > > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > > > >       | ^~~~~~~~~~~~~~~~~~~~
> > > > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > > > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > > > >       | ^~~~~~~~~~~~~
> > > > > cc1: all warnings being treated as errors
> > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > > > make[4]: *** Waiting for unfinished jobs....
> > > > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > >       |             ^
> > > > > cc1: all warnings being treated as errors
> > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > > > >
> > > > > I'm checking now...
> > > > >
> > > > > BTW I test perf builds with:
> > > > >
> > > > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> > > >
> > > > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > > > bisecting get up to the last patch, since I had merged torvalds/master
> > > > today it got me to a wrong correlation, sorry for the disturbance.
> > > >
> > > > For reference, this is the patch:
> > > >
> > > > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
> > >
> > > Ian, I have libbpf-devel installed:
> > >
> > > ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> > > libbpf-0.4.0-1.fc34.x86_64
> > > libbpf-devel-0.4.0-1.fc34.x86_64
> > > ⬢[acme@toolbox perf]$
> > >
> > > But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> > > system headers for all of the homies in tools/lib/.
> > >
> > > I bet that if I remove the libbpf-devel package it works, yeah, just
> > > tested. So we need to make those overrides dependent on using
> > > LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> > > that is -Itools/lib/, using a more finegrained approach, right?
> >
> > Ugh, this is messy. The -I for tools/lib is overloaded and being used
> > in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and
>
> can you do `make install` for libbpf instead and have it install
> headers into a dedicated target directory which can be added into -I
> search path. Quentin did this for all the other libbpf users in kernel
> tree (bpftool, resolve_btfids, etc) and it works great.

This sounds good to me, and being able to borrow code from bpftool
should make writing it is straightforward. I'll try to find time to do
a patch, but I don't mind someone getting there before me :-)

Thanks,
Ian

> > bpf/libbpf.h can be found. Likewise, for tools/perf/util/debug.c it is
> > used to pick up traceevent/event-parse.h.
> >
> > Assuming  LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get
> > different combinations of:
> > libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken for debug.c,
> > -idirafter okay
> > libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken for debug.c,
> > -idirafter broken for bpf-event.c
> > libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but
> > headers mismatched, -idirafter okay
> > libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to link
> > bpf-event.c, -idirafter broken for bpf-event.c
> >
> > As the choice of -I and -idirafter are binary then there will always
> > be a way to break the build. We could modify the build so that the
> > -I/-idirafter only applies to the affected C files. This postpones the
> > problem to when libbpf and libtracevent are in the same file, which
> > doesn't happen currently. I think if you want the dynamic behavior
> > then you need to use idirafter.
> >
> > Thanks,
> > Ian

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-06 22:04         ` Ian Rogers
@ 2022-01-07  3:30           ` Ian Rogers
  2022-01-07 18:32             ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 11+ messages in thread
From: Ian Rogers @ 2022-01-07  3:30 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: Arnaldo Carvalho de Melo, Song Liu, Jiri Olsa, Steven Rostedt,
	Jiri Slaby, Namhyung Kim, bpf, Linux Kernel Mailing List,
	Quentin Monnet

On Thu, Jan 6, 2022 at 2:04 PM Ian Rogers <irogers@google.com> wrote:
>
> On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> > >
> > > On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
> > > <acme@kernel.org> wrote:
> > > >
> > > > Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > After merging torvalds/master to perf/urgent I'm getting this:
> > > > > >
> > > > > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > > > > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > > > > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > > > > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > > > > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > > > > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > > > > >       | ^~~~~~~~~~~~~~~~~~~~
> > > > > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > > > > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > > > > >       | ^~~~~~~~~~~~~
> > > > > > cc1: all warnings being treated as errors
> > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > > > > make[4]: *** Waiting for unfinished jobs....
> > > > > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > > > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > >       |             ^
> > > > > > cc1: all warnings being treated as errors
> > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > > > > >
> > > > > > I'm checking now...
> > > > > >
> > > > > > BTW I test perf builds with:
> > > > > >
> > > > > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> > > > >
> > > > > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > > > > bisecting get up to the last patch, since I had merged torvalds/master
> > > > > today it got me to a wrong correlation, sorry for the disturbance.
> > > > >
> > > > > For reference, this is the patch:
> > > > >
> > > > > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
> > > >
> > > > Ian, I have libbpf-devel installed:
> > > >
> > > > ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> > > > libbpf-0.4.0-1.fc34.x86_64
> > > > libbpf-devel-0.4.0-1.fc34.x86_64
> > > > ⬢[acme@toolbox perf]$
> > > >
> > > > But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> > > > system headers for all of the homies in tools/lib/.
> > > >
> > > > I bet that if I remove the libbpf-devel package it works, yeah, just
> > > > tested. So we need to make those overrides dependent on using
> > > > LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> > > > that is -Itools/lib/, using a more finegrained approach, right?
> > >
> > > Ugh, this is messy. The -I for tools/lib is overloaded and being used
> > > in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and
> >
> > can you do `make install` for libbpf instead and have it install
> > headers into a dedicated target directory which can be added into -I
> > search path. Quentin did this for all the other libbpf users in kernel
> > tree (bpftool, resolve_btfids, etc) and it works great.
>
> This sounds good to me, and being able to borrow code from bpftool
> should make writing it is straightforward. I'll try to find time to do
> a patch, but I don't mind someone getting there before me :-)

So tools/lib also provides subcmd, symbol and api. These will need
Makefiles to allow an install and likely the header file structure
altering. This seems like too big a fix for the next 5.16rc, wdyt?

Thanks,
Ian

> > > bpf/libbpf.h can be found. Likewise, for tools/perf/util/debug.c it is
> > > used to pick up traceevent/event-parse.h.
> > >
> > > Assuming  LIBBPF_DYNAMIC=1 and LIBTRACEEVENT_DYNAMIC=1 then we get
> > > different combinations of:
> > > libtraceevent >= 1.3 && libbpf >= 0.6 - -I is broken for debug.c,
> > > -idirafter okay
> > > libtraceevent >= 1.3 && libbpf < 0.6 - -I is broken for debug.c,
> > > -idirafter broken for bpf-event.c
> > > libtraceevent < 1.3 && libbpf >= 0.6 - -I should build okay but
> > > headers mismatched, -idirafter okay
> > > libtraceevent < 1.3 && libbpf < 0.6 - -I will fail to link
> > > bpf-event.c, -idirafter broken for bpf-event.c
> > >
> > > As the choice of -I and -idirafter are binary then there will always
> > > be a way to break the build. We could modify the build so that the
> > > -I/-idirafter only applies to the affected C files. This postpones the
> > > problem to when libbpf and libtracevent are in the same file, which
> > > doesn't happen currently. I think if you want the dynamic behavior
> > > then you need to use idirafter.
> > >
> > > Thanks,
> > > Ian

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-07  3:30           ` Ian Rogers
@ 2022-01-07 18:32             ` Arnaldo Carvalho de Melo
  2022-01-07 19:26               ` Ian Rogers
  0 siblings, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-01-07 18:32 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Andrii Nakryiko, Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby,
	Namhyung Kim, bpf, Linux Kernel Mailing List, Quentin Monnet

Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
> On Thu, Jan 6, 2022 at 2:04 PM Ian Rogers <irogers@google.com> wrote:
> >
> > On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> > > >
> > > > On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
> > > > <acme@kernel.org> wrote:
> > > > >
> > > > > Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > > After merging torvalds/master to perf/urgent I'm getting this:
> > > > > > >
> > > > > > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > > > > > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > > > > > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > > > > > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > > > > > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > > > > > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > > > > > >       | ^~~~~~~~~~~~~~~~~~~~
> > > > > > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > > > > > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > > > > > >       | ^~~~~~~~~~~~~
> > > > > > > cc1: all warnings being treated as errors
> > > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > > > > > make[4]: *** Waiting for unfinished jobs....
> > > > > > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > > > > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > > >       |             ^
> > > > > > > cc1: all warnings being treated as errors
> > > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > > > > > >
> > > > > > > I'm checking now...
> > > > > > >
> > > > > > > BTW I test perf builds with:
> > > > > > >
> > > > > > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> > > > > >
> > > > > > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > > > > > bisecting get up to the last patch, since I had merged torvalds/master
> > > > > > today it got me to a wrong correlation, sorry for the disturbance.
> > > > > >
> > > > > > For reference, this is the patch:
> > > > > >
> > > > > > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
> > > > >
> > > > > Ian, I have libbpf-devel installed:
> > > > >
> > > > > ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> > > > > libbpf-0.4.0-1.fc34.x86_64
> > > > > libbpf-devel-0.4.0-1.fc34.x86_64
> > > > > ⬢[acme@toolbox perf]$
> > > > >
> > > > > But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> > > > > system headers for all of the homies in tools/lib/.
> > > > >
> > > > > I bet that if I remove the libbpf-devel package it works, yeah, just
> > > > > tested. So we need to make those overrides dependent on using
> > > > > LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> > > > > that is -Itools/lib/, using a more finegrained approach, right?
> > > >
> > > > Ugh, this is messy. The -I for tools/lib is overloaded and being used
> > > > in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and
> > >
> > > can you do `make install` for libbpf instead and have it install
> > > headers into a dedicated target directory which can be added into -I
> > > search path. Quentin did this for all the other libbpf users in kernel
> > > tree (bpftool, resolve_btfids, etc) and it works great.
> >
> > This sounds good to me, and being able to borrow code from bpftool
> > should make writing it is straightforward. I'll try to find time to do
> > a patch, but I don't mind someone getting there before me :-)
> 
> So tools/lib also provides subcmd, symbol and api. These will need
> Makefiles to allow an install and likely the header file structure
> altering. This seems like too big a fix for the next 5.16rc, wdyt?

Right, I think the best thing is to revert the patch Jiri pointed out,
right?

- Arnaldo

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-07 18:32             ` Arnaldo Carvalho de Melo
@ 2022-01-07 19:26               ` Ian Rogers
  2022-01-07 20:18                 ` Arnaldo Carvalho de Melo
  0 siblings, 1 reply; 11+ messages in thread
From: Ian Rogers @ 2022-01-07 19:26 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Andrii Nakryiko, Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby,
	Namhyung Kim, bpf, Linux Kernel Mailing List, Quentin Monnet

On Fri, Jan 7, 2022 at 10:32 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
> > On Thu, Jan 6, 2022 at 2:04 PM Ian Rogers <irogers@google.com> wrote:
> > >
> > > On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> > > > >
> > > > > On Thu, Jan 6, 2022 at 11:48 AM Arnaldo Carvalho de Melo
> > > > > <acme@kernel.org> wrote:
> > > > > >
> > > > > > Em Thu, Jan 06, 2022 at 04:44:14PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > > Em Thu, Jan 06, 2022 at 04:34:46PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > > > After merging torvalds/master to perf/urgent I'm getting this:
> > > > > > > >
> > > > > > > > util/bpf-event.c:25:21: error: no previous prototype for ‘btf__load_from_kernel_by_id’ [-Werror=missing-prototypes]
> > > > > > > >    25 | struct btf * __weak btf__load_from_kernel_by_id(__u32 id)
> > > > > > > >       |                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > > util/bpf-event.c:37:1: error: no previous prototype for ‘bpf_object__next_program’ [-Werror=missing-prototypes]
> > > > > > > >    37 | bpf_object__next_program(const struct bpf_object *obj, struct bpf_program *prev)
> > > > > > > >       | ^~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > > util/bpf-event.c:46:1: error: no previous prototype for ‘bpf_object__next_map’ [-Werror=missing-prototypes]
> > > > > > > >    46 | bpf_object__next_map(const struct bpf_object *obj, const struct bpf_map *prev)
> > > > > > > >       | ^~~~~~~~~~~~~~~~~~~~
> > > > > > > > util/bpf-event.c:55:1: error: no previous prototype for ‘btf__raw_data’ [-Werror=missing-prototypes]
> > > > > > > >    55 | btf__raw_data(const struct btf *btf_ro, __u32 *size)
> > > > > > > >       | ^~~~~~~~~~~~~
> > > > > > > > cc1: all warnings being treated as errors
> > > > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf-event.o] Error 1
> > > > > > > > make[4]: *** Waiting for unfinished jobs....
> > > > > > > > util/bpf_counter.c: In function ‘bpf_target_prog_name’:
> > > > > > > > util/bpf_counter.c:82:15: error: implicit declaration of function ‘btf__load_from_kernel_by_id’ [-Werror=implicit-function-declaration]
> > > > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > > > >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > > > > > util/bpf_counter.c:82:13: error: assignment to ‘struct btf *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
> > > > > > > >    82 |         btf = btf__load_from_kernel_by_id(info_linear->info.btf_id);
> > > > > > > >       |             ^
> > > > > > > > cc1: all warnings being treated as errors
> > > > > > > > make[4]: *** [/var/home/acme/git/perf/tools/build/Makefile.build:96: /tmp/build/perf/util/bpf_counter.o] Error 1
> > > > > > > >
> > > > > > > > I'm checking now...
> > > > > > > >
> > > > > > > > BTW I test perf builds with:
> > > > > > > >
> > > > > > > > make -k BUILD_BPF_SKEL=1 CORESIGHT=1 PYTHON=python3 O=/tmp/build/perf -C tools/perf install-bin && git status && perf test python
> > > > > > >
> > > > > > > Nevermind, this was due to a patch by Ian Rogers I was testing,
> > > > > > > bisecting get up to the last patch, since I had merged torvalds/master
> > > > > > > today it got me to a wrong correlation, sorry for the disturbance.
> > > > > > >
> > > > > > > For reference, this is the patch:
> > > > > > >
> > > > > > > http://lore.kernel.org/lkml/20220106072627.476524-1-irogers@google.com
> > > > > >
> > > > > > Ian, I have libbpf-devel installed:
> > > > > >
> > > > > > ⬢[acme@toolbox perf]$ rpm -qa | grep libbpf
> > > > > > libbpf-0.4.0-1.fc34.x86_64
> > > > > > libbpf-devel-0.4.0-1.fc34.x86_64
> > > > > > ⬢[acme@toolbox perf]$
> > > > > >
> > > > > > But I'm not using LIBBPF_DYNAMIC=1, so you can't just give precedence to
> > > > > > system headers for all of the homies in tools/lib/.
> > > > > >
> > > > > > I bet that if I remove the libbpf-devel package it works, yeah, just
> > > > > > tested. So we need to make those overrides dependent on using
> > > > > > LIBBPF_DYNAMIC=1, LIBTRACEEVENT_DYNAMIC=1, etc and avoid the big hammer
> > > > > > that is -Itools/lib/, using a more finegrained approach, right?
> > > > >
> > > > > Ugh, this is messy. The -I for tools/lib is overloaded and being used
> > > > > in tools/perf/util/bpf-event.c so that bpf/bpf.h, bpf/btf.h and
> > > >
> > > > can you do `make install` for libbpf instead and have it install
> > > > headers into a dedicated target directory which can be added into -I
> > > > search path. Quentin did this for all the other libbpf users in kernel
> > > > tree (bpftool, resolve_btfids, etc) and it works great.
> > >
> > > This sounds good to me, and being able to borrow code from bpftool
> > > should make writing it is straightforward. I'll try to find time to do
> > > a patch, but I don't mind someone getting there before me :-)
> >
> > So tools/lib also provides subcmd, symbol and api. These will need
> > Makefiles to allow an install and likely the header file structure
> > altering. This seems like too big a fix for the next 5.16rc, wdyt?
>
> Right, I think the best thing is to revert the patch Jiri pointed out,
> right?

Your call. There is a latent bug that with LIBTRACEEVENT_DYNAMIC we
are using tools/lib/traceevent header files. Reverting the change
means we don't break because of this, but it means that people
building with LIBTRACEEVENT_DYNAMIC and newer libtraceevent (at least
my employer :-) ) lose logging. I can carry the change locally, so not
a big loss :-) There are a few issues stemming from this:

1) we've identified the current build is wrong for xxx_DYNAMIC options
as tools/lib versions headers always override
2) to address this we should make the tools/lib things proper
libraries like libbpf, libtraceevent, etc.
3) once we have proper libraries, we need to update the perf build to
build non-dynamic libraries then depend on the built/installed header
files

I expect at least some of this is going to break when testing on many
distributions as that just seems to be what always happens, and
changing the build in this significant way is going to have
implications. Doing this means that the code base is in better shape
and logging works. To counter some of the many distribution pain, do
you have a way to reproduce your testing? My OpenSuSE recipe is:

```
# Get the image
docker pull opensuse/tumbleweed
# Start it with an interactive bash shell and mounting the current
directory as /kernel-src
sudo docker run --privileged -it --net=host --env="DISPLAY" --mount
type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed
/bin/bash
# Install missing rpms
zypper install make gcc diffutils flex bison kernel-devel findutils
libelf-devel python3 kernel-kvmsmall-devel glibc-devel
# Go to /kernel-src and build into /tmp, etc.
```
But finding every distribution, every rpm, etc. is quite laborious.

Thanks,
Ian

> - Arnaldo

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-07 19:26               ` Ian Rogers
@ 2022-01-07 20:18                 ` Arnaldo Carvalho de Melo
  2022-01-08 22:38                   ` Ian Rogers
  0 siblings, 1 reply; 11+ messages in thread
From: Arnaldo Carvalho de Melo @ 2022-01-07 20:18 UTC (permalink / raw)
  To: Ian Rogers
  Cc: Andrii Nakryiko, Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby,
	Namhyung Kim, bpf, Linux Kernel Mailing List, Quentin Monnet

Em Fri, Jan 07, 2022 at 11:26:50AM -0800, Ian Rogers escreveu:
> On Fri, Jan 7, 2022 at 10:32 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
> > > On Thu, Jan 6, 2022 at 2:04 PM Ian Rogers <irogers@google.com> wrote:
> > > > On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> > > > > On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> > > So tools/lib also provides subcmd, symbol and api. These will need
> > > Makefiles to allow an install and likely the header file structure
> > > altering. This seems like too big a fix for the next 5.16rc, wdyt?

> > Right, I think the best thing is to revert the patch Jiri pointed out,
> > right?
 
> Your call. There is a latent bug that with LIBTRACEEVENT_DYNAMIC we
> are using tools/lib/traceevent header files. Reverting the change
> means we don't break because of this, but it means that people
> building with LIBTRACEEVENT_DYNAMIC and newer libtraceevent (at least
> my employer :-) ) lose logging. I can carry the change locally, so not
> a big loss :-)

Just for a while, we should get this fixed for v5.17, for v5.16 a revert
is required :-\

> There are a few issues stemming from this:
> 1) we've identified the current build is wrong for xxx_DYNAMIC options
> as tools/lib versions headers always override

yeap

> 2) to address this we should make the tools/lib things proper
> libraries like libbpf, libtraceevent, etc.

yeap

> 3) once we have proper libraries, we need to update the perf build to
> build non-dynamic libraries then depend on the built/installed header
> files

yeap
 
> I expect at least some of this is going to break when testing on many
> distributions as that just seems to be what always happens, and
> changing the build in this significant way is going to have
> implications. Doing this means that the code base is in better shape
> and logging works.

Right, its nice that I have a mechanism to test build in 80+ distro
versions/cross-build environments. :-)

> To counter some of the many distribution pain, do
> you have a way to reproduce your testing? My OpenSuSE recipe is:

So I'm adding this:

[perfbuilder@five tumbleweed]$ git diff rx_and_build.sh
diff --git a/opensuse/tumbleweed/rx_and_build.sh b/opensuse/tumbleweed/rx_and_build.sh
index fbc8845..0510ef1 100755
--- a/opensuse/tumbleweed/rx_and_build.sh
+++ b/opensuse/tumbleweed/rx_and_build.sh
@@ -11,11 +11,19 @@ build_perf_gcc() {
        make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBELF=1 -C tools/perf O=/tmp/build/perf || exit 1
        rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
        make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBBPF=1 -C tools/perf O=/tmp/build/perf || exit 1
+       [ -d /usr/include/traceevent/ ] && \
+       make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1
        set +o xtrace
 }

 build_perf_clang() {
        set -o xtrace
+
+       if [ ! $NO_BUILD_BPF_SKEL ] ; then
+               rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
+               make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf || exit 1
+       fi
+
        rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
        make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C tools/perf O=/tmp/build/perf CC=clang || exit 1
        rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
@@ -26,6 +34,8 @@ build_perf_clang() {
        make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf CC=clang || exit 1
        rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
        make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf || exit 1
+       [ -d /usr/include/traceevent/ ] && \
+       make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1
        set +o xtrace
 }

@@ -48,6 +58,7 @@ TARBALL=`basename $TARBALL_URL`
 xzcat $TARBALL | tar xvf -
 SRCDIR=`echo $TARBALL | sed -r 's/(.*).tar\..*/\1/g'`
 cd /git/$SRCDIR
+echo -n BUILD_TARBALL_HEAD=
 cat HEAD

 # print the version for dm to harvest and put in the status line
[perfbuilder@five tumbleweed]$

And I'll add this to 'make -C tools/perf build-test' too, what I have in
rx_and_build.sh is a limited set of 'build-test', building with it in
all containers take more time, it is possible if one does:

[perfbuilder@five ~]$ export BUILD_TARBALL=http://192.168.100.2/perf/perf-5.16.0-rc8.tar.xz
[perfbuilder@five ~]$ export BUILD_CMD='make -C tools/perf build-cmd'
[perfbuilder@five ~]$ dm

Then it will do it for gcc and for clang, takes a while...

My Dockerfile for tumbleweed, btw, is:

[perfbuilder@five tumbleweed]$ cat Dockerfile
# acmel/linux-perf-tools-build-opensuse:tumbleweed
FROM docker.io/opensuse/tumbleweed
MAINTAINER Arnaldo Carvalho de Melo <acme@kernel.org>
# shadow for groupadd
RUN zypper -n update && \
    zypper -n install shadow \
	make gcc gcc-c++ flex bison cpio \
	bc file findutils clang llvm curl tar xz \
	libunwind-devel openssl-devel slang-devel python-devel \
	systemtap-sdt-devel gtk2-devel xz-devel binutils-devel \
	libelf-devel libdw-devel audit-devel libzstd-devel \
	java-1_8_0-openjdk-headless libcap-devel \
	clang-devel llvm-devel perl libnuma-devel \
	libbpf-devel libtraceevent-devel \
	babeltrace-devel OpenCSD-devel cmake xmlto asciidoc && \
    zypper clean --all && \
    mkdir -m 777 -p /git /tmp/build/perf /tmp/build/objtool /tmp/build/linux && \
    groupadd -r perfbuilder && \
    useradd -m -r -g perfbuilder perfbuilder && \
    chown -R perfbuilder.perfbuilder /tmp/build/ /git/
USER perfbuilder
COPY rx_and_build.sh /
COPY .bash_profile /home/perfbuilder/
ENTRYPOINT ["/rx_and_build.sh"]
[perfbuilder@five tumbleweed]$
 
> ```
> # Get the image
> docker pull opensuse/tumbleweed
> # Start it with an interactive bash shell and mounting the current
> directory as /kernel-src
> sudo docker run --privileged -it --net=host --env="DISPLAY" --mount
> type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed
> /bin/bash
> # Install missing rpms
> zypper install make gcc diffutils flex bison kernel-devel findutils
> libelf-devel python3 kernel-kvmsmall-devel glibc-devel
> # Go to /kernel-src and build into /tmp, etc.
> ```
> But finding every distribution, every rpm, etc. is quite laborious.

Yeah, but I did it already for quite a few distros :-)

[perfbuilder@five linux-perf-tools-build]$ find . -name Dockerfile | nl
     1	./alpine/3.10/Dockerfile
     2	./alpine/3.11/Dockerfile
     3	./alpine/3.12/Dockerfile
     4	./alpine/3.13/Dockerfile
     5	./alpine/3.5/Dockerfile
     6	./alpine/3.6/Dockerfile
     7	./alpine/3.7/Dockerfile
     8	./alpine/3.8/Dockerfile
     9	./alpine/3.9/Dockerfile
    10	./alpine/edge/Dockerfile
    11	./alpine/3.14/Dockerfile
    12	./alpine/3.4/Dockerfile
    13	./alpine/3.15/Dockerfile
    14	./alt/p8/Dockerfile
    15	./alt/p9/Dockerfile
    16	./alt/sisyphus/Dockerfile
    17	./alt/p10/Dockerfile
    18	./amazonlinux/1/Dockerfile
    19	./amazonlinux/2/Dockerfile
    20	./android/end-of-life/r22b/arm/Dockerfile
    21	./android/end-of-life/r12b/arm/Dockerfile
    22	./android/end-of-life/r15c/arm/Dockerfile
    23	./archlinux/base/Dockerfile
    24	./centos/8/Dockerfile
    25	./centos/end-of-life/5/Dockerfile
    26	./centos/end-of-life/6/Dockerfile
    27	./centos/end-of-life/7/Dockerfile
    28	./centos/stream/Dockerfile
    29	./clearlinux/latest/Dockerfile
    30	./debian/10/Dockerfile
    31	./debian/9/Dockerfile
    32	./debian/end-of-life/7/Dockerfile
    33	./debian/end-of-life/8/Dockerfile
    34	./debian/experimental/x-arm64/Dockerfile
    35	./debian/experimental/x-mips/Dockerfile
    36	./debian/experimental/x-mips64/Dockerfile
    37	./debian/experimental/x-mipsel/Dockerfile
    38	./debian/experimental/Dockerfile
    39	./debian/11/Dockerfile
    40	./fedora/22/Dockerfile
    41	./fedora/23/Dockerfile
    42	./fedora/24/x-ARC-uClibc/Dockerfile
    43	./fedora/24/Dockerfile
    44	./fedora/25/Dockerfile
    45	./fedora/26/Dockerfile
    46	./fedora/27/Dockerfile
    47	./fedora/28/Dockerfile
    48	./fedora/29/Dockerfile
    49	./fedora/30/Dockerfile
    50	./fedora/31/Dockerfile
    51	./fedora/32/Dockerfile
    52	./fedora/33/Dockerfile
    53	./fedora/34/x-ARC-glibc/Dockerfile
    54	./fedora/34/x-ARC-uClibc/Dockerfile
    55	./fedora/34/Dockerfile
    56	./fedora/end-of-life/21/Dockerfile
    57	./fedora/end-of-life/20/Dockerfile
    58	./fedora/rawhide/Dockerfile
    59	./fedora/35/Dockerfile
    60	./gentoo/end-of-life/stage3-amd64/Dockerfile
    61	./gentoo/stage3/Dockerfile
    62	./mageia/5/Dockerfile
    63	./mageia/6/Dockerfile
    64	./mageia/7/Dockerfile
    65	./mageia/8/Dockerfile
    66	./manjaro/base/Dockerfile
    67	./openmandriva/cooker/Dockerfile
    68	./opensuse/15.0/Dockerfile
    69	./opensuse/15.1/Dockerfile
    70	./opensuse/15.2/Dockerfile
    71	./opensuse/15.3/Dockerfile
    72	./opensuse/end-of-life/13.2/Dockerfile
    73	./opensuse/end-of-life/42.1/Dockerfile
    74	./opensuse/end-of-life/42.2/Dockerfile
    75	./opensuse/end-of-life/42.3/Dockerfile
    76	./opensuse/tumbleweed/Dockerfile
    77	./opensuse/15.4/Dockerfile
    78	./oraclelinux/8/Dockerfile
    79	./oraclelinux/end-of-life/6/Dockerfile
    80	./oraclelinux/end-of-life/7/Dockerfile
    81	./rhel7/Dockerfile
    82	./ubuntu/16.04/x-arm/Dockerfile
    83	./ubuntu/16.04/x-arm64/Dockerfile
    84	./ubuntu/16.04/x-powerpc/Dockerfile
    85	./ubuntu/16.04/x-powerpc64/Dockerfile
    86	./ubuntu/16.04/x-powerpc64el/Dockerfile
    87	./ubuntu/16.04/x-s390/Dockerfile
    88	./ubuntu/16.04/Dockerfile
    89	./ubuntu/18.04/x-arm/Dockerfile
    90	./ubuntu/18.04/x-arm64/Dockerfile
    91	./ubuntu/18.04/x-m68k/Dockerfile
    92	./ubuntu/18.04/x-powerpc/Dockerfile
    93	./ubuntu/18.04/x-powerpc64/Dockerfile
    94	./ubuntu/18.04/x-powerpc64el/Dockerfile
    95	./ubuntu/18.04/x-riscv64/Dockerfile
    96	./ubuntu/18.04/x-s390/Dockerfile
    97	./ubuntu/18.04/x-sh4/Dockerfile
    98	./ubuntu/18.04/x-sparc64/Dockerfile
    99	./ubuntu/18.04/Dockerfile
   100	./ubuntu/20.04/x-powerpc64el/Dockerfile
   101	./ubuntu/20.04/x-s390/Dockerfile
   102	./ubuntu/20.04/Dockerfile
   103	./ubuntu/20.04/x-arm/Dockerfile
   104	./ubuntu/20.10/Dockerfile
   105	./ubuntu/21.04/Dockerfile
   106	./ubuntu/21.04/end-of-life/x-mips/Dockerfile
   107	./ubuntu/21.04/end-of-life/x-mips64/Dockerfile
   108	./ubuntu/end-of-life/15.04/Dockerfile
   109	./ubuntu/end-of-life/15.10/Dockerfile
   110	./ubuntu/end-of-life/16.10/Dockerfile
   111	./ubuntu/end-of-life/17.04/Dockerfile
   112	./ubuntu/end-of-life/17.10/Dockerfile
   113	./ubuntu/end-of-life/18.10/Dockerfile
   114	./ubuntu/end-of-life/19.04/Dockerfile
   115	./ubuntu/end-of-life/19.10/Dockerfile
   116	./ubuntu/end-of-life/19.10/x-alpha/Dockerfile
   117	./ubuntu/end-of-life/19.10/x-arm64/Dockerfile
   118	./ubuntu/end-of-life/19.10/x-hppa/Dockerfile
   119	./ubuntu/end-of-life/12.04/Dockerfile
   120	./ubuntu/end-of-life/14.04/end-of-life/x-linaro-arm64/Dockerfile
   121	./ubuntu/end-of-life/14.04/Dockerfile
   122	./ubuntu/21.10/Dockerfile
   123	./ubuntu/22.04/Dockerfile
   124	./almalinux/8/Dockerfile
   125	./rockylinux/8/Dockerfile
   126	./ubi8/Dockerfile
[perfbuilder@five linux-perf-tools-build]$

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: perf build broken seemingly due to libbpf changes, checking...
  2022-01-07 20:18                 ` Arnaldo Carvalho de Melo
@ 2022-01-08 22:38                   ` Ian Rogers
  0 siblings, 0 replies; 11+ messages in thread
From: Ian Rogers @ 2022-01-08 22:38 UTC (permalink / raw)
  To: Arnaldo Carvalho de Melo
  Cc: Andrii Nakryiko, Song Liu, Jiri Olsa, Steven Rostedt, Jiri Slaby,
	Namhyung Kim, bpf, Linux Kernel Mailing List, Quentin Monnet

On Fri, Jan 7, 2022 at 12:18 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Fri, Jan 07, 2022 at 11:26:50AM -0800, Ian Rogers escreveu:
> > On Fri, Jan 7, 2022 at 10:32 AM Arnaldo Carvalho de Melo <acme@kernel.org> wrote:
> > > Em Thu, Jan 06, 2022 at 07:30:34PM -0800, Ian Rogers escreveu:
> > > > On Thu, Jan 6, 2022 at 2:04 PM Ian Rogers <irogers@google.com> wrote:
> > > > > On Thu, Jan 6, 2022 at 1:44 PM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> > > > > > On Thu, Jan 6, 2022 at 1:42 PM Ian Rogers <irogers@google.com> wrote:
> > > > So tools/lib also provides subcmd, symbol and api. These will need
> > > > Makefiles to allow an install and likely the header file structure
> > > > altering. This seems like too big a fix for the next 5.16rc, wdyt?
>
> > > Right, I think the best thing is to revert the patch Jiri pointed out,
> > > right?
>
> > Your call. There is a latent bug that with LIBTRACEEVENT_DYNAMIC we
> > are using tools/lib/traceevent header files. Reverting the change
> > means we don't break because of this, but it means that people
> > building with LIBTRACEEVENT_DYNAMIC and newer libtraceevent (at least
> > my employer :-) ) lose logging. I can carry the change locally, so not
> > a big loss :-)
>
> Just for a while, we should get this fixed for v5.17, for v5.16 a revert
> is required :-\
>
> > There are a few issues stemming from this:
> > 1) we've identified the current build is wrong for xxx_DYNAMIC options
> > as tools/lib versions headers always override
>
> yeap
>
> > 2) to address this we should make the tools/lib things proper
> > libraries like libbpf, libtraceevent, etc.
>
> yeap
>
> > 3) once we have proper libraries, we need to update the perf build to
> > build non-dynamic libraries then depend on the built/installed header
> > files
>
> yeap
>
> > I expect at least some of this is going to break when testing on many
> > distributions as that just seems to be what always happens, and
> > changing the build in this significant way is going to have
> > implications. Doing this means that the code base is in better shape
> > and logging works.
>
> Right, its nice that I have a mechanism to test build in 80+ distro
> versions/cross-build environments. :-)
>
> > To counter some of the many distribution pain, do
> > you have a way to reproduce your testing? My OpenSuSE recipe is:
>
> So I'm adding this:
>
> [perfbuilder@five tumbleweed]$ git diff rx_and_build.sh
> diff --git a/opensuse/tumbleweed/rx_and_build.sh b/opensuse/tumbleweed/rx_and_build.sh
> index fbc8845..0510ef1 100755
> --- a/opensuse/tumbleweed/rx_and_build.sh
> +++ b/opensuse/tumbleweed/rx_and_build.sh
> @@ -11,11 +11,19 @@ build_perf_gcc() {
>         make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBELF=1 -C tools/perf O=/tmp/build/perf || exit 1
>         rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
>         make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" NO_LIBBPF=1 -C tools/perf O=/tmp/build/perf || exit 1
> +       [ -d /usr/include/traceevent/ ] && \
> +       make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1
>         set +o xtrace
>  }
>
>  build_perf_clang() {
>         set -o xtrace
> +
> +       if [ ! $NO_BUILD_BPF_SKEL ] ; then
> +               rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
> +               make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" BUILD_BPF_SKEL=1 -C tools/perf O=/tmp/build/perf || exit 1
> +       fi
> +
>         rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
>         make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" -C tools/perf O=/tmp/build/perf CC=clang || exit 1
>         rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
> @@ -26,6 +34,8 @@ build_perf_clang() {
>         make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf CC=clang || exit 1
>         rm -rf /tmp/build/perf ; mkdir /tmp/build/perf
>         make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBCLANGLLVM=1 -C tools/perf O=/tmp/build/perf || exit 1
> +       [ -d /usr/include/traceevent/ ] && \
> +       make $EXTRA_MAKE_ARGS ARCH=$ARCH CROSS_COMPILE=$CROSS_COMPILE EXTRA_CFLAGS="$EXTRA_CFLAGS" LIBTRACEEVENT_DYNAMIC=1 -C tools/perf O=/tmp/build/perf || exit 1
>         set +o xtrace
>  }
>
> @@ -48,6 +58,7 @@ TARBALL=`basename $TARBALL_URL`
>  xzcat $TARBALL | tar xvf -
>  SRCDIR=`echo $TARBALL | sed -r 's/(.*).tar\..*/\1/g'`
>  cd /git/$SRCDIR
> +echo -n BUILD_TARBALL_HEAD=
>  cat HEAD
>
>  # print the version for dm to harvest and put in the status line
> [perfbuilder@five tumbleweed]$
>
> And I'll add this to 'make -C tools/perf build-test' too, what I have in
> rx_and_build.sh is a limited set of 'build-test', building with it in
> all containers take more time, it is possible if one does:
>
> [perfbuilder@five ~]$ export BUILD_TARBALL=http://192.168.100.2/perf/perf-5.16.0-rc8.tar.xz
> [perfbuilder@five ~]$ export BUILD_CMD='make -C tools/perf build-cmd'
> [perfbuilder@five ~]$ dm
>
> Then it will do it for gcc and for clang, takes a while...
>
> My Dockerfile for tumbleweed, btw, is:
>
> [perfbuilder@five tumbleweed]$ cat Dockerfile
> # acmel/linux-perf-tools-build-opensuse:tumbleweed
> FROM docker.io/opensuse/tumbleweed
> MAINTAINER Arnaldo Carvalho de Melo <acme@kernel.org>
> # shadow for groupadd
> RUN zypper -n update && \
>     zypper -n install shadow \
>         make gcc gcc-c++ flex bison cpio \
>         bc file findutils clang llvm curl tar xz \
>         libunwind-devel openssl-devel slang-devel python-devel \
>         systemtap-sdt-devel gtk2-devel xz-devel binutils-devel \
>         libelf-devel libdw-devel audit-devel libzstd-devel \
>         java-1_8_0-openjdk-headless libcap-devel \
>         clang-devel llvm-devel perl libnuma-devel \
>         libbpf-devel libtraceevent-devel \
>         babeltrace-devel OpenCSD-devel cmake xmlto asciidoc && \
>     zypper clean --all && \
>     mkdir -m 777 -p /git /tmp/build/perf /tmp/build/objtool /tmp/build/linux && \
>     groupadd -r perfbuilder && \
>     useradd -m -r -g perfbuilder perfbuilder && \
>     chown -R perfbuilder.perfbuilder /tmp/build/ /git/
> USER perfbuilder
> COPY rx_and_build.sh /
> COPY .bash_profile /home/perfbuilder/
> ENTRYPOINT ["/rx_and_build.sh"]
> [perfbuilder@five tumbleweed]$
>
> > ```
> > # Get the image
> > docker pull opensuse/tumbleweed
> > # Start it with an interactive bash shell and mounting the current
> > directory as /kernel-src
> > sudo docker run --privileged -it --net=host --env="DISPLAY" --mount
> > type=bind,source="$(pwd)",target=/kernel-src opensuse/tumbleweed
> > /bin/bash
> > # Install missing rpms
> > zypper install make gcc diffutils flex bison kernel-devel findutils
> > libelf-devel python3 kernel-kvmsmall-devel glibc-devel
> > # Go to /kernel-src and build into /tmp, etc.
> > ```
> > But finding every distribution, every rpm, etc. is quite laborious.
>
> Yeah, but I did it already for quite a few distros :-)
>
> [perfbuilder@five linux-perf-tools-build]$ find . -name Dockerfile | nl
>      1  ./alpine/3.10/Dockerfile
>      2  ./alpine/3.11/Dockerfile
>      3  ./alpine/3.12/Dockerfile
>      4  ./alpine/3.13/Dockerfile
>      5  ./alpine/3.5/Dockerfile
>      6  ./alpine/3.6/Dockerfile
>      7  ./alpine/3.7/Dockerfile
>      8  ./alpine/3.8/Dockerfile
>      9  ./alpine/3.9/Dockerfile
>     10  ./alpine/edge/Dockerfile
>     11  ./alpine/3.14/Dockerfile
>     12  ./alpine/3.4/Dockerfile
>     13  ./alpine/3.15/Dockerfile
>     14  ./alt/p8/Dockerfile
>     15  ./alt/p9/Dockerfile
>     16  ./alt/sisyphus/Dockerfile
>     17  ./alt/p10/Dockerfile
>     18  ./amazonlinux/1/Dockerfile
>     19  ./amazonlinux/2/Dockerfile
>     20  ./android/end-of-life/r22b/arm/Dockerfile
>     21  ./android/end-of-life/r12b/arm/Dockerfile
>     22  ./android/end-of-life/r15c/arm/Dockerfile
>     23  ./archlinux/base/Dockerfile
>     24  ./centos/8/Dockerfile
>     25  ./centos/end-of-life/5/Dockerfile
>     26  ./centos/end-of-life/6/Dockerfile
>     27  ./centos/end-of-life/7/Dockerfile
>     28  ./centos/stream/Dockerfile
>     29  ./clearlinux/latest/Dockerfile
>     30  ./debian/10/Dockerfile
>     31  ./debian/9/Dockerfile
>     32  ./debian/end-of-life/7/Dockerfile
>     33  ./debian/end-of-life/8/Dockerfile
>     34  ./debian/experimental/x-arm64/Dockerfile
>     35  ./debian/experimental/x-mips/Dockerfile
>     36  ./debian/experimental/x-mips64/Dockerfile
>     37  ./debian/experimental/x-mipsel/Dockerfile
>     38  ./debian/experimental/Dockerfile
>     39  ./debian/11/Dockerfile
>     40  ./fedora/22/Dockerfile
>     41  ./fedora/23/Dockerfile
>     42  ./fedora/24/x-ARC-uClibc/Dockerfile
>     43  ./fedora/24/Dockerfile
>     44  ./fedora/25/Dockerfile
>     45  ./fedora/26/Dockerfile
>     46  ./fedora/27/Dockerfile
>     47  ./fedora/28/Dockerfile
>     48  ./fedora/29/Dockerfile
>     49  ./fedora/30/Dockerfile
>     50  ./fedora/31/Dockerfile
>     51  ./fedora/32/Dockerfile
>     52  ./fedora/33/Dockerfile
>     53  ./fedora/34/x-ARC-glibc/Dockerfile
>     54  ./fedora/34/x-ARC-uClibc/Dockerfile
>     55  ./fedora/34/Dockerfile
>     56  ./fedora/end-of-life/21/Dockerfile
>     57  ./fedora/end-of-life/20/Dockerfile
>     58  ./fedora/rawhide/Dockerfile
>     59  ./fedora/35/Dockerfile
>     60  ./gentoo/end-of-life/stage3-amd64/Dockerfile
>     61  ./gentoo/stage3/Dockerfile
>     62  ./mageia/5/Dockerfile
>     63  ./mageia/6/Dockerfile
>     64  ./mageia/7/Dockerfile
>     65  ./mageia/8/Dockerfile
>     66  ./manjaro/base/Dockerfile
>     67  ./openmandriva/cooker/Dockerfile
>     68  ./opensuse/15.0/Dockerfile
>     69  ./opensuse/15.1/Dockerfile
>     70  ./opensuse/15.2/Dockerfile
>     71  ./opensuse/15.3/Dockerfile
>     72  ./opensuse/end-of-life/13.2/Dockerfile
>     73  ./opensuse/end-of-life/42.1/Dockerfile
>     74  ./opensuse/end-of-life/42.2/Dockerfile
>     75  ./opensuse/end-of-life/42.3/Dockerfile
>     76  ./opensuse/tumbleweed/Dockerfile
>     77  ./opensuse/15.4/Dockerfile
>     78  ./oraclelinux/8/Dockerfile
>     79  ./oraclelinux/end-of-life/6/Dockerfile
>     80  ./oraclelinux/end-of-life/7/Dockerfile
>     81  ./rhel7/Dockerfile
>     82  ./ubuntu/16.04/x-arm/Dockerfile
>     83  ./ubuntu/16.04/x-arm64/Dockerfile
>     84  ./ubuntu/16.04/x-powerpc/Dockerfile
>     85  ./ubuntu/16.04/x-powerpc64/Dockerfile
>     86  ./ubuntu/16.04/x-powerpc64el/Dockerfile
>     87  ./ubuntu/16.04/x-s390/Dockerfile
>     88  ./ubuntu/16.04/Dockerfile
>     89  ./ubuntu/18.04/x-arm/Dockerfile
>     90  ./ubuntu/18.04/x-arm64/Dockerfile
>     91  ./ubuntu/18.04/x-m68k/Dockerfile
>     92  ./ubuntu/18.04/x-powerpc/Dockerfile
>     93  ./ubuntu/18.04/x-powerpc64/Dockerfile
>     94  ./ubuntu/18.04/x-powerpc64el/Dockerfile
>     95  ./ubuntu/18.04/x-riscv64/Dockerfile
>     96  ./ubuntu/18.04/x-s390/Dockerfile
>     97  ./ubuntu/18.04/x-sh4/Dockerfile
>     98  ./ubuntu/18.04/x-sparc64/Dockerfile
>     99  ./ubuntu/18.04/Dockerfile
>    100  ./ubuntu/20.04/x-powerpc64el/Dockerfile
>    101  ./ubuntu/20.04/x-s390/Dockerfile
>    102  ./ubuntu/20.04/Dockerfile
>    103  ./ubuntu/20.04/x-arm/Dockerfile
>    104  ./ubuntu/20.10/Dockerfile
>    105  ./ubuntu/21.04/Dockerfile
>    106  ./ubuntu/21.04/end-of-life/x-mips/Dockerfile
>    107  ./ubuntu/21.04/end-of-life/x-mips64/Dockerfile
>    108  ./ubuntu/end-of-life/15.04/Dockerfile
>    109  ./ubuntu/end-of-life/15.10/Dockerfile
>    110  ./ubuntu/end-of-life/16.10/Dockerfile
>    111  ./ubuntu/end-of-life/17.04/Dockerfile
>    112  ./ubuntu/end-of-life/17.10/Dockerfile
>    113  ./ubuntu/end-of-life/18.10/Dockerfile
>    114  ./ubuntu/end-of-life/19.04/Dockerfile
>    115  ./ubuntu/end-of-life/19.10/Dockerfile
>    116  ./ubuntu/end-of-life/19.10/x-alpha/Dockerfile
>    117  ./ubuntu/end-of-life/19.10/x-arm64/Dockerfile
>    118  ./ubuntu/end-of-life/19.10/x-hppa/Dockerfile
>    119  ./ubuntu/end-of-life/12.04/Dockerfile
>    120  ./ubuntu/end-of-life/14.04/end-of-life/x-linaro-arm64/Dockerfile
>    121  ./ubuntu/end-of-life/14.04/Dockerfile
>    122  ./ubuntu/21.10/Dockerfile
>    123  ./ubuntu/22.04/Dockerfile
>    124  ./almalinux/8/Dockerfile
>    125  ./rockylinux/8/Dockerfile
>    126  ./ubi8/Dockerfile
> [perfbuilder@five linux-perf-tools-build]$

Cool. Do you have this available externally?

Thanks,
Ian

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2022-01-08 22:39 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-06 19:34 perf build broken seemingly due to libbpf changes, checking Arnaldo Carvalho de Melo
2022-01-06 19:44 ` Arnaldo Carvalho de Melo
2022-01-06 19:48   ` Arnaldo Carvalho de Melo
2022-01-06 21:41     ` Ian Rogers
2022-01-06 21:44       ` Andrii Nakryiko
2022-01-06 22:04         ` Ian Rogers
2022-01-07  3:30           ` Ian Rogers
2022-01-07 18:32             ` Arnaldo Carvalho de Melo
2022-01-07 19:26               ` Ian Rogers
2022-01-07 20:18                 ` Arnaldo Carvalho de Melo
2022-01-08 22:38                   ` Ian Rogers

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).