From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: "Toke Høiland-Jørgensen" <toke@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
Yonghong Song <yhs@fb.com>, Andrii Nakryiko <andriin@fb.com>,
Doug Ledford <dledford@redhat.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
"David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>,
Jesper Dangaard Brouer <brouer@redhat.com>,
John Fastabend <john.fastabend@gmail.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
Shuah Khan <shuah@kernel.org>,
Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
open list <linux-kernel@vger.kernel.org>,
linux-rdma@vger.kernel.org,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>,
clang-built-linux@googlegroups.com
Subject: Re: [PATCH bpf-next v4 09/10] selftests: Remove tools/lib/bpf from include path
Date: Sun, 19 Jan 2020 22:20:55 -0800 [thread overview]
Message-ID: <CAEf4Bzb9zUmhxTyYahJqySJzgfyB-zMEd+o4ybv=a8-t+iZS4w@mail.gmail.com> (raw)
In-Reply-To: <157926820677.1555735.5437255599683298212.stgit@toke.dk>
On Fri, Jan 17, 2020 at 5:43 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> From: Toke Høiland-Jørgensen <toke@redhat.com>
>
> To make sure no new files are introduced that doesn't include the bpf/
> prefix in its #include, remove tools/lib/bpf from the include path
> entirely.
>
> Instead, we introduce a new header files directory under the scratch tools/
> dir, and add a rule to run the 'install_headers' rule from libbpf to have a
> full set of consistent libbpf headers in $(OUTPUT)/tools/include/bpf, and
> then use $(OUTPUT)/tools/include as the include path for selftests.
>
> For consistency we also make sure we put all the scratch build files from
> other bpftool and libbpf into tools/build/, so everything stays within
> selftests/.
>
> Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
> ---
> tools/testing/selftests/bpf/.gitignore | 2 +
> tools/testing/selftests/bpf/Makefile | 49 +++++++++++++++++++++-----------
> 2 files changed, 33 insertions(+), 18 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
> index 1d14e3ab70be..8c9eac626996 100644
> --- a/tools/testing/selftests/bpf/.gitignore
> +++ b/tools/testing/selftests/bpf/.gitignore
> @@ -39,4 +39,4 @@ test_cpp
> /no_alu32
> /bpf_gcc
> /tools
> -bpf_helper_defs.h
can you please also drop libbpf.pc and libbpf.so.* rules from .gitignore?
> +
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 858d4e3369ad..ac0292a82fdc 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -20,8 +20,8 @@ CLANG ?= clang
> LLC ?= llc
> LLVM_OBJCOPY ?= llvm-objcopy
> BPF_GCC ?= $(shell command -v bpf-gcc;)
> -CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) -I$(LIBDIR) \
> - -I$(BPFDIR) -I$(GENDIR) -I$(TOOLSINCDIR) \
> +CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) \
extra space here
> + -I$(INCLUDE_DIR) -I$(GENDIR) -I$(LIBDIR) -I$(TOOLSINCDIR) \
> -Dbpf_prog_load=bpf_prog_test_load \
> -Dbpf_load_program=bpf_test_load_program
> LDLIBS += -lcap -lelf -lz -lrt -lpthread
> @@ -97,11 +97,15 @@ OVERRIDE_TARGETS := 1
> override define CLEAN
> $(call msg,CLEAN)
> $(RM) -r $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) $(EXTRA_CLEAN)
> - $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ clean
> endef
>
> include ../lib.mk
>
> +SCRATCH_DIR := $(OUTPUT)/tools
> +BUILD_DIR := $(SCRATCH_DIR)/build
> +INCLUDE_DIR := $(SCRATCH_DIR)/include
> +INCLUDE_BPF := $(INCLUDE_DIR)/bpf
> +
> # Define simple and short `make test_progs`, `make test_sysctl`, etc targets
> # to build individual tests.
> # NOTE: Semicolon at the end is critical to override lib.mk's default static
> @@ -120,7 +124,7 @@ $(OUTPUT)/urandom_read: urandom_read.c
> $(call msg,BINARY,,$@)
> $(CC) -o $@ $< -Wl,--build-id
>
> -$(OUTPUT)/test_stub.o: test_stub.c
> +$(OUTPUT)/test_stub.o: test_stub.c $(INCLUDE_BPF)
> $(call msg,CC,,$@)
> $(CC) -c $(CFLAGS) -o $@ $<
>
> @@ -133,7 +137,7 @@ $(OUTPUT)/runqslower: force
> $(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower \
> OUTPUT=$(OUTPUT)/tools/ VMLINUX_BTF=$(VMLINUX_BTF)
>
> -BPFOBJ := $(OUTPUT)/libbpf.a
> +BPFOBJ := $(BUILD_DIR)/libbpf/libbpf.a
>
> $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED): $(OUTPUT)/test_stub.o $(BPFOBJ)
>
> @@ -159,17 +163,28 @@ force:
> DEFAULT_BPFTOOL := $(OUTPUT)/tools/sbin/bpftool
> BPFTOOL ?= $(DEFAULT_BPFTOOL)
>
> -$(DEFAULT_BPFTOOL): force
> - $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
> +$(BUILD_DIR)/libbpf $(BUILD_DIR)/bpftool $(INCLUDE_DIR):
> + $(call msg,MKDIR,,$@)
> + mkdir -p $@
> +
> +$(DEFAULT_BPFTOOL): force | $(BUILD_DIR)/bpftool
> + $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \
slash alignment
> + OUTPUT=$(BUILD_DIR)/bpftool/ \
> prefix= DESTDIR=$(OUTPUT)/tools/ install
>
> -$(BPFOBJ): force
> - $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
> +$(BPFOBJ): force | $(BUILD_DIR)/libbpf
> + $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
> + OUTPUT=$(BUILD_DIR)/libbpf/
> +
> +$(INCLUDE_BPF): $(BPFOBJ) | $(INCLUDE_DIR)
> + $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) install_headers \
> + OUTPUT=$(BUILD_DIR)/libbpf/ DESTDIR=$(SCRATCH_DIR) prefix=
> +
> +BPF_HELPERS := $(or $(wildcard $(INCLUDE_BPF)/bpf_*.h),$(INCLUDE_BPF))
> +ifneq ($(BPF_HELPERS),$(INCLUDE_BPF))
> +$(BPF_HELPERS): $(INCLUDE_BPF)
> +endif
>
> -BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
> -$(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
> - $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
> - OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
>
I really-really didn't like this alternating dependency on directory
or a set of file, depending on current state of those temporary
directories. Then also this ugly check to avoid circular dependency.
All that seemed wrong. So I played a bit with how to achieve the same
differently, and here's what I came up with, which I like a bit
better. What do you think?
$(BPFOBJ): $(wildcard $(BPFDIR)/*.c $(BPFDIR)/*.h $(BPFDIR)/Makefile) \
../../../include/uapi/linux/bpf.h \
| $(INCLUDE_DIR) $(BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \
DESTDIR=$(SCRATCH_DIR) prefix= all install_headers
So, essentially, just make sure that we install local copies of
headers whenever libbpf.a needs to be re-built.
../../../include/uapi/linux/bpf.h ensures we don't miss uapi header
changes as well. Now anything that uses libbpf headers will need to
depend on $(BPFOBJ) and will automatically get up-to-date local copies
of headers.
This seems much simpler. Please give it a try, thanks!
> # Get Clang's default includes on this system, as opposed to those seen by
> # '-target bpf'. This fixes "missing" files on some architectures/distros,
> @@ -189,8 +204,8 @@ MENDIAN=$(if $(IS_LITTLE_ENDIAN),-mlittle-endian,-mbig-endian)
>
> CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG))
> BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \
> - -I$(OUTPUT) -I$(CURDIR) -I$(CURDIR)/include/uapi \
> - -I$(APIDIR) -I$(LIBDIR) -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
> + -I$(INCLUDE_DIR) -I$(CURDIR) -I$(CURDIR)/include/uapi \
> + -I$(APIDIR) -I$(abspath $(OUTPUT)/../usr/include)
>
> CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \
> -Wno-compare-distinct-pointer-types
> @@ -392,7 +407,7 @@ $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
> $(call msg,CXX,,$@)
> $(CXX) $(CFLAGS) $^ $(LDLIBS) -o $@
>
> -EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) \
> +EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) \
> prog_tests/tests.h map_tests/tests.h verifier/tests.h \
> feature \
> - $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc tools)
> + $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc)
>
next prev parent reply other threads:[~2020-01-20 6:21 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-17 13:36 [PATCH bpf-next v4 00/10] tools: Use consistent libbpf include paths everywhere Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 01/10] samples/bpf: Don't try to remove user's homedir on clean Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 02/10] tools/bpf/runqslower: Fix override option for VMLINUX_BTF Toke Høiland-Jørgensen
2020-01-17 21:30 ` Andrii Nakryiko
2020-01-20 11:04 ` Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 03/10] selftests: Pass VMLINUX_BTF to runqslower Makefile Toke Høiland-Jørgensen
2020-01-17 21:36 ` Andrii Nakryiko
2020-01-17 13:36 ` [PATCH bpf-next v4 04/10] tools/runqslower: Use consistent include paths for libbpf Toke Høiland-Jørgensen
2020-01-17 21:51 ` Andrii Nakryiko
2020-01-20 12:56 ` Toke Høiland-Jørgensen
2020-01-20 18:35 ` Andrii Nakryiko
2020-01-20 21:00 ` Andrii Nakryiko
2020-01-17 13:36 ` [PATCH bpf-next v4 05/10] selftests: " Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 06/10] bpftool: " Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 07/10] perf: " Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 08/10] samples/bpf: " Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 09/10] selftests: Remove tools/lib/bpf from include path Toke Høiland-Jørgensen
2020-01-20 6:20 ` Andrii Nakryiko [this message]
2020-01-20 11:06 ` Toke Høiland-Jørgensen
2020-01-17 13:36 ` [PATCH bpf-next v4 10/10] tools/runqslower: " Toke Høiland-Jørgensen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAEf4Bzb9zUmhxTyYahJqySJzgfyB-zMEd+o4ybv=a8-t+iZS4w@mail.gmail.com' \
--to=andrii.nakryiko@gmail.com \
--cc=acme@kernel.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=andriin@fb.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=clang-built-linux@googlegroups.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=john.fastabend@gmail.com \
--cc=jolsa@redhat.com \
--cc=kafai@fb.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=shuah@kernel.org \
--cc=songliubraving@fb.com \
--cc=toke@redhat.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).