* [PATCH bpf-next 0/2] selftests/bpf Makefile cleanup and fixes @ 2019-10-11 3:13 Andrii Nakryiko 2019-10-11 3:13 ` [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built Andrii Nakryiko 2019-10-11 3:13 ` [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection Andrii Nakryiko 0 siblings, 2 replies; 7+ messages in thread From: Andrii Nakryiko @ 2019-10-11 3:13 UTC (permalink / raw) To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko Patch #1 enforces libbpf build to have bpf_helper_defs.h ready before test BPF programs are built. Patch #2 drops obsolete BTF/pahole detection logic from Makefile. Andrii Nakryiko (2): selftests/bpf: enforce libbpf build before BPF programs are built selftests/bpf: remove obsolete pahole/BTF support detection tools/testing/selftests/bpf/Makefile | 47 +++++----------------------- 1 file changed, 8 insertions(+), 39 deletions(-) -- 2.17.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built 2019-10-11 3:13 [PATCH bpf-next 0/2] selftests/bpf Makefile cleanup and fixes Andrii Nakryiko @ 2019-10-11 3:13 ` Andrii Nakryiko 2019-10-11 20:22 ` Martin Lau 2019-10-11 3:13 ` [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection Andrii Nakryiko 1 sibling, 1 reply; 7+ messages in thread From: Andrii Nakryiko @ 2019-10-11 3:13 UTC (permalink / raw) To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko Given BPF programs rely on libbpf's bpf_helper_defs.h, which is auto-generated during libbpf build, libbpf build has to happen before we attempt progs/*.c build. Enforce it as order-only dependency. Fixes: 24f25763d6de ("libbpf: auto-generate list of BPF helper definitions") Signed-off-by: Andrii Nakryiko <andriin@fb.com> --- tools/testing/selftests/bpf/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 40552fb441e5..f958643d36da 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -256,7 +256,8 @@ ifeq ($(DWARF2BTF),y) $(BTF_PAHOLE) -J $@ endif -$(OUTPUT)/%.o: progs/%.c +# libbpf has to be built before BPF programs due to bpf_helper_defs.h +$(OUTPUT)/%.o: progs/%.c | $(BPFOBJ) ($(CLANG) $(BPF_CFLAGS) $(CLANG_CFLAGS) -O2 -target bpf -emit-llvm \ -c $< -o - || echo "clang failed") | \ $(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@ -- 2.17.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built 2019-10-11 3:13 ` [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built Andrii Nakryiko @ 2019-10-11 20:22 ` Martin Lau 0 siblings, 0 replies; 7+ messages in thread From: Martin Lau @ 2019-10-11 20:22 UTC (permalink / raw) To: Andrii Nakryiko Cc: bpf, netdev, Alexei Starovoitov, daniel, andrii.nakryiko, Kernel Team On Thu, Oct 10, 2019 at 08:13:17PM -0700, Andrii Nakryiko wrote: > Given BPF programs rely on libbpf's bpf_helper_defs.h, which is > auto-generated during libbpf build, libbpf build has to happen before > we attempt progs/*.c build. Enforce it as order-only dependency. Acked-by: Martin KaFai Lau <kafai@fb.com> ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection 2019-10-11 3:13 [PATCH bpf-next 0/2] selftests/bpf Makefile cleanup and fixes Andrii Nakryiko 2019-10-11 3:13 ` [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built Andrii Nakryiko @ 2019-10-11 3:13 ` Andrii Nakryiko 2019-10-11 16:21 ` [Potential Spoof] " Martin Lau 1 sibling, 1 reply; 7+ messages in thread From: Andrii Nakryiko @ 2019-10-11 3:13 UTC (permalink / raw) To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko Given lots of selftests won't work without recent enough Clang/LLVM that fully supports BTF, there is no point in maintaining outdated BTF support detection and fall-back to pahole logic. Just assume we have everything we need. Signed-off-by: Andrii Nakryiko <andriin@fb.com> --- tools/testing/selftests/bpf/Makefile | 44 ++++------------------------ 1 file changed, 6 insertions(+), 38 deletions(-) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index f958643d36da..d1770da2da70 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -15,8 +15,6 @@ endif CLANG ?= clang LLC ?= llc LLVM_OBJCOPY ?= llvm-objcopy -LLVM_READELF ?= llvm-readelf -BTF_PAHOLE ?= pahole BPF_GCC ?= $(shell command -v bpf-gcc;) CFLAGS += -g -Wall -O2 -I$(APIDIR) -I$(LIBDIR) -I$(BPFDIR) -I$(GENDIR) $(GENFLAGS) -I../../../include \ -Dbpf_prog_load=bpf_prog_test_load \ @@ -147,8 +145,9 @@ $(shell $(1) -v -E - </dev/null 2>&1 \ | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') endef CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG)) -BPF_CFLAGS = -I. -I./include/uapi -I../../../include/uapi \ - -I$(BPFDIR) -I$(OUTPUT)/../usr/include -D__TARGET_ARCH_$(SRCARCH) +BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH) \ + -I. -I./include/uapi -I../../../include/uapi \ + -I$(BPFDIR) -I$(OUTPUT)/../usr/include CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \ -Wno-compare-distinct-pointer-types @@ -162,28 +161,6 @@ $(OUTPUT)/test_stack_map.o: test_queue_stack_map.h $(OUTPUT)/flow_dissector_load.o: flow_dissector_load.h $(OUTPUT)/test_progs.o: flow_dissector_load.h -BTF_LLC_PROBE := $(shell $(LLC) -march=bpf -mattr=help 2>&1 | grep dwarfris) -BTF_PAHOLE_PROBE := $(shell $(BTF_PAHOLE) --help 2>&1 | grep BTF) -BTF_OBJCOPY_PROBE := $(shell $(LLVM_OBJCOPY) --help 2>&1 | grep -i 'usage.*llvm') -BTF_LLVM_PROBE := $(shell echo "int main() { return 0; }" | \ - $(CLANG) -target bpf -O2 -g -c -x c - -o ./llvm_btf_verify.o; \ - $(LLVM_READELF) -S ./llvm_btf_verify.o | grep BTF; \ - /bin/rm -f ./llvm_btf_verify.o) - -ifneq ($(BTF_LLVM_PROBE),) - BPF_CFLAGS += -g -else -ifneq ($(BTF_LLC_PROBE),) -ifneq ($(BTF_PAHOLE_PROBE),) -ifneq ($(BTF_OBJCOPY_PROBE),) - BPF_CFLAGS += -g - LLC_FLAGS += -mattr=dwarfris - DWARF2BTF = y -endif -endif -endif -endif - TEST_PROGS_CFLAGS := -I. -I$(OUTPUT) TEST_MAPS_CFLAGS := -I. -I$(OUTPUT) TEST_VERIFIER_CFLAGS := -I. -I$(OUTPUT) -Iverifier @@ -212,11 +189,8 @@ $(ALU32_BUILD_DIR)/%.o: progs/%.c $(ALU32_BUILD_DIR)/test_progs_32 \ | $(ALU32_BUILD_DIR) ($(CLANG) $(BPF_CFLAGS) $(CLANG_CFLAGS) -O2 -target bpf -emit-llvm \ -c $< -o - || echo "clang failed") | \ - $(LLC) -march=bpf -mattr=+alu32 -mcpu=$(CPU) $(LLC_FLAGS) \ + $(LLC) -march=bpf -mcpu=probe -mattr=+alu32 $(LLC_FLAGS) \ -filetype=obj -o $@ -ifeq ($(DWARF2BTF),y) - $(BTF_PAHOLE) -J $@ -endif endif ifneq ($(BPF_GCC),) @@ -251,19 +225,13 @@ endif $(OUTPUT)/test_xdp.o: progs/test_xdp.c ($(CLANG) $(BPF_CFLAGS) $(CLANG_CFLAGS) -O2 -emit-llvm -c $< -o - || \ echo "clang failed") | \ - $(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@ -ifeq ($(DWARF2BTF),y) - $(BTF_PAHOLE) -J $@ -endif + $(LLC) -march=bpf -mcpu=probe $(LLC_FLAGS) -filetype=obj -o $@ # libbpf has to be built before BPF programs due to bpf_helper_defs.h $(OUTPUT)/%.o: progs/%.c | $(BPFOBJ) ($(CLANG) $(BPF_CFLAGS) $(CLANG_CFLAGS) -O2 -target bpf -emit-llvm \ -c $< -o - || echo "clang failed") | \ - $(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@ -ifeq ($(DWARF2BTF),y) - $(BTF_PAHOLE) -J $@ -endif + $(LLC) -march=bpf -mcpu=probe $(LLC_FLAGS) -filetype=obj -o $@ PROG_TESTS_DIR = $(OUTPUT)/prog_tests $(PROG_TESTS_DIR): -- 2.17.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Potential Spoof] [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection 2019-10-11 3:13 ` [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection Andrii Nakryiko @ 2019-10-11 16:21 ` Martin Lau 2019-10-11 17:28 ` Andrii Nakryiko 0 siblings, 1 reply; 7+ messages in thread From: Martin Lau @ 2019-10-11 16:21 UTC (permalink / raw) To: Andrii Nakryiko Cc: bpf, netdev, Alexei Starovoitov, daniel, andrii.nakryiko, Kernel Team On Thu, Oct 10, 2019 at 08:13:18PM -0700, Andrii Nakryiko wrote: > Given lots of selftests won't work without recent enough Clang/LLVM that > fully supports BTF, there is no point in maintaining outdated BTF > support detection and fall-back to pahole logic. Just assume we have > everything we need. May be an error message to tell which llvm is needed? $(CPU) and $(PROBE) are no longer needed also? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Potential Spoof] [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection 2019-10-11 16:21 ` [Potential Spoof] " Martin Lau @ 2019-10-11 17:28 ` Andrii Nakryiko 2019-10-11 21:17 ` Daniel Borkmann 0 siblings, 1 reply; 7+ messages in thread From: Andrii Nakryiko @ 2019-10-11 17:28 UTC (permalink / raw) To: Martin Lau Cc: Andrii Nakryiko, bpf, netdev, Alexei Starovoitov, daniel, Kernel Team On Fri, Oct 11, 2019 at 9:21 AM Martin Lau <kafai@fb.com> wrote: > > On Thu, Oct 10, 2019 at 08:13:18PM -0700, Andrii Nakryiko wrote: > > Given lots of selftests won't work without recent enough Clang/LLVM that > > fully supports BTF, there is no point in maintaining outdated BTF > > support detection and fall-back to pahole logic. Just assume we have > > everything we need. > May be an error message to tell which llvm is needed? Not sure where we'd want this to be checked/printed. We don't do this today, so what I'm doing here is not really a regression. There is no single llvm version I'd want to pin down. For most tests LLVM w/ basic BTF support would be enough, for CO-RE stuff we need the latest Clang 10 (not yet released officially), though. So essentially the stance right now is that you need latest Clang built from sources to have all the tests compiled and I don't think it's easy to check for that. > > $(CPU) and $(PROBE) are no longer needed also? Good catch, removing them as well. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Potential Spoof] [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection 2019-10-11 17:28 ` Andrii Nakryiko @ 2019-10-11 21:17 ` Daniel Borkmann 0 siblings, 0 replies; 7+ messages in thread From: Daniel Borkmann @ 2019-10-11 21:17 UTC (permalink / raw) To: Andrii Nakryiko Cc: Martin Lau, Andrii Nakryiko, bpf, netdev, Alexei Starovoitov, Kernel Team On Fri, Oct 11, 2019 at 10:28:39AM -0700, Andrii Nakryiko wrote: > On Fri, Oct 11, 2019 at 9:21 AM Martin Lau <kafai@fb.com> wrote: > > On Thu, Oct 10, 2019 at 08:13:18PM -0700, Andrii Nakryiko wrote: > > > Given lots of selftests won't work without recent enough Clang/LLVM that > > > fully supports BTF, there is no point in maintaining outdated BTF > > > support detection and fall-back to pahole logic. Just assume we have > > > everything we need. > > May be an error message to tell which llvm is needed? > > Not sure where we'd want this to be checked/printed. We don't do this > today, so what I'm doing here is not really a regression. > There is no single llvm version I'd want to pin down. For most tests > LLVM w/ basic BTF support would be enough, for CO-RE stuff we need the > latest Clang 10 (not yet released officially), though. So essentially > the stance right now is that you need latest Clang built from sources > to have all the tests compiled and I don't think it's easy to check > for that. At some point once bpf-gcc gets more mature, we might need something more elaborate than just telling everyone to use latest clang/llvm from git, but so far that's our convention we have in place today. > > $(CPU) and $(PROBE) are no longer needed also? > > Good catch, removing them as well. Ok, expecting v2 then. Thanks, Daniel ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-10-11 21:18 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-10-11 3:13 [PATCH bpf-next 0/2] selftests/bpf Makefile cleanup and fixes Andrii Nakryiko 2019-10-11 3:13 ` [PATCH bpf-next 1/2] selftests/bpf: enforce libbpf build before BPF programs are built Andrii Nakryiko 2019-10-11 20:22 ` Martin Lau 2019-10-11 3:13 ` [PATCH bpf-next 2/2] selftests/bpf: remove obsolete pahole/BTF support detection Andrii Nakryiko 2019-10-11 16:21 ` [Potential Spoof] " Martin Lau 2019-10-11 17:28 ` Andrii Nakryiko 2019-10-11 21:17 ` Daniel Borkmann
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).