[PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

When compiling bpftool on a system where the /usr/include/asm symlink
doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
the build fails with:

    CLANG    skeleton/profiler.bpf.o
  In file included from skeleton/profiler.bpf.c:4:
  In file included from /usr/include/linux/bpf.h:11:
  /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
  #include <asm/types.h>
           ^~~~~~~~~~~~~
  1 error generated.
  make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1

To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
path so <asm/types.h> can be found.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
---
 tools/bpf/bpftool/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 20a90d8450f8..3cc0644fd91e 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -120,7 +120,7 @@ $(OUTPUT)_bpftool: $(_OBJS) $(LIBBPF)
 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(_OBJS) $(LIBS)
 
 skeleton/profiler.bpf.o: skeleton/profiler.bpf.c
-	$(QUIET_CLANG)$(CLANG) -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
+	$(QUIET_CLANG)$(CLANG) -I/usr/include/$(shell uname -m)-linux-gnu -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
 
 profiler.skel.h: $(OUTPUT)_bpftool skeleton/profiler.bpf.o
 	$(QUIET_GEN)$(OUTPUT)./_bpftool gen skeleton skeleton/profiler.bpf.o > $@
-- 
2.25.1

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Toke Høiland-Jørgensen]

Tobias Klauser <tklauser@distanz.ch> writes:

> When compiling bpftool on a system where the /usr/include/asm symlink
> doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> the build fails with:
>
>     CLANG    skeleton/profiler.bpf.o
>   In file included from skeleton/profiler.bpf.c:4:
>   In file included from /usr/include/linux/bpf.h:11:
>   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
>   #include <asm/types.h>
>            ^~~~~~~~~~~~~
>   1 error generated.
>   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
>
> To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
> path so <asm/types.h> can be found.

Isn't the right thing here to just install gcc-multilib?

-Toke

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

On 2020-03-11 at 13:49:53 +0100, Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> Tobias Klauser <tklauser@distanz.ch> writes:
> 
> > When compiling bpftool on a system where the /usr/include/asm symlink
> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> > the build fails with:
> >
> >     CLANG    skeleton/profiler.bpf.o
> >   In file included from skeleton/profiler.bpf.c:4:
> >   In file included from /usr/include/linux/bpf.h:11:
> >   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
> >   #include <asm/types.h>
> >            ^~~~~~~~~~~~~
> >   1 error generated.
> >   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> >
> > To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
> > path so <asm/types.h> can be found.
> 
> Isn't the right thing here to just install gcc-multilib?

For a container build we would like to avoid installing gcc-multilib
which pulls in additional dependencies which are otherwise not needed to
build bpftool. This patch would allow that.

Tobias

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Toke Høiland-Jørgensen]

Tobias Klauser <tklauser@distanz.ch> writes:

> On 2020-03-11 at 13:49:53 +0100, Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>> Tobias Klauser <tklauser@distanz.ch> writes:
>> 
>> > When compiling bpftool on a system where the /usr/include/asm symlink
>> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
>> > the build fails with:
>> >
>> >     CLANG    skeleton/profiler.bpf.o
>> >   In file included from skeleton/profiler.bpf.c:4:
>> >   In file included from /usr/include/linux/bpf.h:11:
>> >   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
>> >   #include <asm/types.h>
>> >            ^~~~~~~~~~~~~
>> >   1 error generated.
>> >   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
>> >
>> > To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
>> > path so <asm/types.h> can be found.
>> 
>> Isn't the right thing here to just install gcc-multilib?
>
> For a container build we would like to avoid installing gcc-multilib
> which pulls in additional dependencies which are otherwise not needed to
> build bpftool. This patch would allow that.

Ah, right. Well, stating that use case in the commit message would have
been nice :)

I'm personally a little skeptical about having to add this (how many
weird build systems should we support?), but I can also see where you're
coming from. Up to the maintainers, I suppose...

-Toke

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

On 2020-03-11 at 15:24:23 +0100, Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> Tobias Klauser <tklauser@distanz.ch> writes:
> 
> > On 2020-03-11 at 13:49:53 +0100, Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> >> Tobias Klauser <tklauser@distanz.ch> writes:
> >> 
> >> > When compiling bpftool on a system where the /usr/include/asm symlink
> >> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> >> > the build fails with:
> >> >
> >> >     CLANG    skeleton/profiler.bpf.o
> >> >   In file included from skeleton/profiler.bpf.c:4:
> >> >   In file included from /usr/include/linux/bpf.h:11:
> >> >   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
> >> >   #include <asm/types.h>
> >> >            ^~~~~~~~~~~~~
> >> >   1 error generated.
> >> >   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> >> >
> >> > To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
> >> > path so <asm/types.h> can be found.
> >> 
> >> Isn't the right thing here to just install gcc-multilib?
> >
> > For a container build we would like to avoid installing gcc-multilib
> > which pulls in additional dependencies which are otherwise not needed to
> > build bpftool. This patch would allow that.
> 
> Ah, right. Well, stating that use case in the commit message would have
> been nice :)

Agree. Can also do a v2 if needed adding that rationale to the commit
message. There's anyway a typo in the patch subject
(s/iprofiler/profiler) which I could also fix :)

Tobias

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Song Liu]

> On Mar 11, 2020, at 5:34 AM, Tobias Klauser <tklauser@distanz.ch> wrote:
> 
> When compiling bpftool on a system where the /usr/include/asm symlink
> doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> the build fails with:
> 
>    CLANG    skeleton/profiler.bpf.o
>  In file included from skeleton/profiler.bpf.c:4:
>  In file included from /usr/include/linux/bpf.h:11:
>  /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
>  #include <asm/types.h>
>           ^~~~~~~~~~~~~
>  1 error generated.
>  make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> 
> To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
> path so <asm/types.h> can be found.
> 
> Signed-off-by: Tobias Klauser <tklauser@distanz.ch>

Looks good, with a nit below. 

Acked-by: Song Liu <songliubraving@fb.com>

> ---
> tools/bpf/bpftool/Makefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index 20a90d8450f8..3cc0644fd91e 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -120,7 +120,7 @@ $(OUTPUT)_bpftool: $(_OBJS) $(LIBBPF)
> 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(_OBJS) $(LIBS)
> 
> skeleton/profiler.bpf.o: skeleton/profiler.bpf.c
> -	$(QUIET_CLANG)$(CLANG) -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
> +	$(QUIET_CLANG)$(CLANG) -I/usr/include/$(shell uname -m)-linux-gnu -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@

Nit: this line is too long. It is better to break it into two lines. 

> 
> profiler.skel.h: $(OUTPUT)_bpftool skeleton/profiler.bpf.o
> 	$(QUIET_GEN)$(OUTPUT)./_bpftool gen skeleton skeleton/profiler.bpf.o > $@
> -- 
> 2.25.1
> 

Re: [PATCH] bpftool: fix iprofiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

On 2020-03-11 at 16:55:38 +0100, Song Liu <songliubraving@fb.com> wrote:
> 
> 
> > On Mar 11, 2020, at 5:34 AM, Tobias Klauser <tklauser@distanz.ch> wrote:
> > 
> > When compiling bpftool on a system where the /usr/include/asm symlink
> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> > the build fails with:
> > 
> >    CLANG    skeleton/profiler.bpf.o
> >  In file included from skeleton/profiler.bpf.c:4:
> >  In file included from /usr/include/linux/bpf.h:11:
> >  /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
> >  #include <asm/types.h>
> >           ^~~~~~~~~~~~~
> >  1 error generated.
> >  make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> > 
> > To fix this, add /usr/include/$(uname -m)-linux-gnu to the clang search
> > path so <asm/types.h> can be found.
> > 
> > Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
> 
> Looks good, with a nit below. 
> 
> Acked-by: Song Liu <songliubraving@fb.com>
> 
> > ---
> > tools/bpf/bpftool/Makefile | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> > index 20a90d8450f8..3cc0644fd91e 100644
> > --- a/tools/bpf/bpftool/Makefile
> > +++ b/tools/bpf/bpftool/Makefile
> > @@ -120,7 +120,7 @@ $(OUTPUT)_bpftool: $(_OBJS) $(LIBBPF)
> > 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(_OBJS) $(LIBS)
> > 
> > skeleton/profiler.bpf.o: skeleton/profiler.bpf.c
> > -	$(QUIET_CLANG)$(CLANG) -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
> > +	$(QUIET_CLANG)$(CLANG) -I/usr/include/$(shell uname -m)-linux-gnu -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
> 
> Nit: this line is too long. It is better to break it into two lines. 

Thanks. Will send a v2, also with Toke's feedback regarding commit
message included.

[PATCH bpf-next v2] bpftool: fix profiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

When compiling bpftool on a system where the /usr/include/asm symlink
doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
the build fails with:

    CLANG    skeleton/profiler.bpf.o
  In file included from skeleton/profiler.bpf.c:4:
  In file included from /usr/include/linux/bpf.h:11:
  /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
  #include <asm/types.h>
           ^~~~~~~~~~~~~
  1 error generated.
  make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1

In certain cases (e.g. for container builds), installing gcc-multilib
and all its dependencies - which are otherwise not needed to build
bpftool - unnecessarily increases the image size.

Thus, fix this by adding /usr/include/$(uname -m)-linux-gnu to the
clang search path so <asm/types.h> can be found.

Acked-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
---
 tools/bpf/bpftool/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 20a90d8450f8..db54e9bb873a 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -120,7 +120,8 @@ $(OUTPUT)_bpftool: $(_OBJS) $(LIBBPF)
 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(_OBJS) $(LIBS)
 
 skeleton/profiler.bpf.o: skeleton/profiler.bpf.c
-	$(QUIET_CLANG)$(CLANG) -I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
+	$(QUIET_CLANG)$(CLANG) -I/usr/include/$(shell uname -m)-linux-gnu \
+		-I$(srctree)/tools/lib -g -O2 -target bpf -c $< -o $@
 
 profiler.skel.h: $(OUTPUT)_bpftool skeleton/profiler.bpf.o
 	$(QUIET_GEN)$(OUTPUT)./_bpftool gen skeleton skeleton/profiler.bpf.o > $@
-- 
2.25.1

Re: [PATCH bpf-next v2] bpftool: fix profiler build on systems without /usr/include/asm symlink

[Alexei Starovoitov]

On Wed, Mar 11, 2020 at 05:14:59PM +0100, Tobias Klauser wrote:
> When compiling bpftool on a system where the /usr/include/asm symlink
> doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> the build fails with:
> 
>     CLANG    skeleton/profiler.bpf.o
>   In file included from skeleton/profiler.bpf.c:4:
>   In file included from /usr/include/linux/bpf.h:11:
>   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
>   #include <asm/types.h>
>            ^~~~~~~~~~~~~
>   1 error generated.

I think the issue is different.
profiler.bpf.c should have picked up
tools/include/uapi/linux/bpf.h (instead of global from /usr/inclde)
which should have included
tools/include/linux/types.h (instead of /usr/include/linux/types.h)

we also have a workaround for some cases:
./tools/testing/selftests/bpf/include/uapi/linux/types.h

>   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> 
> In certain cases (e.g. for container builds), installing gcc-multilib
> and all its dependencies - which are otherwise not needed to build
> bpftool - unnecessarily increases the image size.
> 
> Thus, fix this by adding /usr/include/$(uname -m)-linux-gnu to the
> clang search path so <asm/types.h> can be found.

In general perf builds fine on all sorts of distros and configs.
I think bpftool should use the same includes from tools/
and skeleton too.

Re: [PATCH bpf-next v2] bpftool: fix profiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

On 2020-03-11 at 18:26:50 +0100, Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> On Wed, Mar 11, 2020 at 05:14:59PM +0100, Tobias Klauser wrote:
> > When compiling bpftool on a system where the /usr/include/asm symlink
> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> > the build fails with:
> > 
> >     CLANG    skeleton/profiler.bpf.o
> >   In file included from skeleton/profiler.bpf.c:4:
> >   In file included from /usr/include/linux/bpf.h:11:
> >   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
> >   #include <asm/types.h>
> >            ^~~~~~~~~~~~~
> >   1 error generated.
> 
> I think the issue is different.
> profiler.bpf.c should have picked up
> tools/include/uapi/linux/bpf.h (instead of global from /usr/inclde)
> which should have included
> tools/include/linux/types.h (instead of /usr/include/linux/types.h)
> 
> we also have a workaround for some cases:
> ./tools/testing/selftests/bpf/include/uapi/linux/types.h

I just tried that. Unfortunately, this will pull in stdbool.h, stddef.h
and stdint.h libc header (via either linux/types.h header). This will
again require the libc6-dev-i386 package when building for the bpf
target. This is exactly the dependency that we'd like to avoid in
Cilium, also see https://github.com/cilium/cilium/pull/10204

I agree that we ideally shouldn't pull in the headers from the system,
but currently I don't see a way of doing that without still depending on
libc headers. Suggestions welcome...

> >   make: *** [Makefile:123: skeleton/profiler.bpf.o] Error 1
> > 
> > In certain cases (e.g. for container builds), installing gcc-multilib
> > and all its dependencies - which are otherwise not needed to build
> > bpftool - unnecessarily increases the image size.
> > 
> > Thus, fix this by adding /usr/include/$(uname -m)-linux-gnu to the
> > clang search path so <asm/types.h> can be found.
> 
> In general perf builds fine on all sorts of distros and configs.
> I think bpftool should use the same includes from tools/
> and skeleton too.

I'll check how perf builds do it but I suspect they will also depend on
libc headers. This is fine for userspace tools, but we'd like to avoid
it for bpf programs.

- Tobias

Re: [PATCH bpf-next v2] bpftool: fix profiler build on systems without /usr/include/asm symlink

[Tobias Klauser]

On 2020-03-11 at 18:26:50 +0100, Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> On Wed, Mar 11, 2020 at 05:14:59PM +0100, Tobias Klauser wrote:
> > When compiling bpftool on a system where the /usr/include/asm symlink
> > doesn't exist (e.g. on an Ubuntu system without gcc-multilib installed),
> > the build fails with:
> > 
> >     CLANG    skeleton/profiler.bpf.o
> >   In file included from skeleton/profiler.bpf.c:4:
> >   In file included from /usr/include/linux/bpf.h:11:
> >   /usr/include/linux/types.h:5:10: fatal error: 'asm/types.h' file not found
> >   #include <asm/types.h>
> >            ^~~~~~~~~~~~~
> >   1 error generated.
> 
> I think the issue is different.
> profiler.bpf.c should have picked up
> tools/include/uapi/linux/bpf.h (instead of global from /usr/inclde)
> which should have included
> tools/include/linux/types.h (instead of /usr/include/linux/types.h)
> 
> we also have a workaround for some cases:
> ./tools/testing/selftests/bpf/include/uapi/linux/types.h

Sorry, my earlier reply was wrong. Adding
tools/testing/selftests/bpf/include/uapi/ to the search path (and
tools/include/uapi/ in addition) makes this work. Will send a v3.