* [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets
2020-01-10 5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
@ 2020-01-10 5:17 ` Andrii Nakryiko
2020-01-10 5:17 ` [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir Andrii Nakryiko
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10 5:17 UTC (permalink / raw)
To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko
Libbpf's clean target should clean out generated files in $(OUTPUT) directory
and not make assumption that $(OUTPUT) directory is current working directory.
Selftest's Makefile should delegate cleaning of libbpf-generated files to
libbpf's Makefile. This ensures more robust clean up.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/Makefile | 9 +++++----
tools/testing/selftests/bpf/Makefile | 5 +++--
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index d87830e7ea63..db2afccde757 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -273,10 +273,11 @@ config-clean:
$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
clean:
- $(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
- *.o *~ *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) .*.d .*.cmd \
- *.pc LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
- $(SHARED_OBJDIR) $(STATIC_OBJDIR)
+ $(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
+ *~ .*.d .*.cmd LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
+ $(SHARED_OBJDIR) $(STATIC_OBJDIR) \
+ $(addprefix $(OUTPUT), \
+ *.o *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) *.pc)
$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index f1f949cd8ed9..cb9f18e4b98b 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -93,6 +93,7 @@ OVERRIDE_TARGETS := 1
override define CLEAN
$(call msg, CLEAN)
$(RM) -r $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) $(EXTRA_CLEAN)
+ $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ clean
endef
include ../lib.mk
@@ -377,5 +378,5 @@ $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) \
prog_tests/tests.h map_tests/tests.h verifier/tests.h \
- feature $(OUTPUT)/*.o $(OUTPUT)/no_alu32 $(OUTPUT)/bpf_gcc \
- tools *.skel.h
+ feature \
+ $(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc tools)
--
2.17.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir
2020-01-10 5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
2020-01-10 5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
@ 2020-01-10 5:17 ` Andrii Nakryiko
2020-01-10 5:17 ` [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output Andrii Nakryiko
2020-01-10 5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10 5:17 UTC (permalink / raw)
To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko
Reorder includes search path to ensure $(OUTPUT) and $(CURDIR) go before
libbpf's directory. Also fix bpf_helpers.h to include bpf_helper_defs.h in
such a way as to leverage includes search path. This allows selftests to not
use libbpf's local and potentially stale bpf_helper_defs.h. It's important
because selftests/bpf's Makefile only re-generates bpf_helper_defs.h in
seltests' output directory, not the one in libbpf's directory.
Also force regeneration of bpf_helper_defs.h when libbpf.a is updated to
reduce staleness.
Fixes: fa633a0f8919 ("libbpf: Fix build on read-only filesystems")
Reported-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/lib/bpf/bpf_helpers.h | 2 +-
tools/testing/selftests/bpf/Makefile | 12 ++++++------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
index f69cc208778a..050bb7bf5be6 100644
--- a/tools/lib/bpf/bpf_helpers.h
+++ b/tools/lib/bpf/bpf_helpers.h
@@ -2,7 +2,7 @@
#ifndef __BPF_HELPERS__
#define __BPF_HELPERS__
-#include "bpf_helper_defs.h"
+#include <bpf_helper_defs.h>
#define __uint(name, val) int (*name)[val]
#define __type(name, val) typeof(val) *name
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index cb9f18e4b98b..c0a18994db87 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -20,8 +20,8 @@ CLANG ?= clang
LLC ?= llc
LLVM_OBJCOPY ?= llvm-objcopy
BPF_GCC ?= $(shell command -v bpf-gcc;)
-CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(APIDIR) -I$(LIBDIR) -I$(BPFDIR) \
- -I$(GENDIR) -I$(TOOLSINCDIR) -I$(CURDIR) \
+CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) -I$(LIBDIR) \
+ -I$(BPFDIR) -I$(GENDIR) -I$(TOOLSINCDIR) \
-Dbpf_prog_load=bpf_prog_test_load \
-Dbpf_load_program=bpf_test_load_program
LDLIBS += -lcap -lelf -lz -lrt -lpthread
@@ -153,7 +153,7 @@ $(BPFOBJ): force
$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(OUTPUT)/bpf_helper_defs.h:
+$(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
# Get Clang's default includes on this system, as opposed to those seen by
@@ -174,8 +174,8 @@ MENDIAN=$(if $(IS_LITTLE_ENDIAN),-mlittle-endian,-mbig-endian)
CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG))
BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) \
- -I. -I./include/uapi -I$(APIDIR) \
- -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
+ -I$(OUTPUT) -I$(CURDIR) -I$(CURDIR)/include/uapi \
+ -I$(APIDIR) -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \
-Wno-compare-distinct-pointer-types
@@ -329,7 +329,7 @@ TRUNNER_EXTRA_SOURCES := test_progs.c cgroup_helpers.c trace_helpers.c \
TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read \
$(wildcard progs/btf_dump_test_case_*.c)
TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
-TRUNNER_BPF_CFLAGS := -I. -I$(OUTPUT) $(BPF_CFLAGS) $(CLANG_CFLAGS)
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
TRUNNER_BPF_LDFLAGS := -mattr=+alu32
$(eval $(call DEFINE_TEST_RUNNER,test_progs))
--
2.17.1
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output
2020-01-10 5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
2020-01-10 5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
2020-01-10 5:17 ` [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir Andrii Nakryiko
@ 2020-01-10 5:17 ` Andrii Nakryiko
2020-01-10 5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10 5:17 UTC (permalink / raw)
To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko
Further clean up Makefile output:
- hide "entering directory" messages;
- silvence sub-Make command echoing;
- succinct MKDIR messages.
Also remove few test binaries that are not produced anymore from .gitignore.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
tools/testing/selftests/bpf/.gitignore | 2 --
tools/testing/selftests/bpf/Makefile | 10 +++++++---
2 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 301ac12d5d69..1d14e3ab70be 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -22,11 +22,9 @@ get_cgroup_id_user
test_skb_cgroup_id_user
test_socket_cookie
test_cgroup_storage
-test_select_reuseport
test_flow_dissector
flow_dissector_load
test_netcnt
-test_section_names
test_tcpnotify_user
test_libbpf
test_tcp_check_syncookie_user
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index c0a18994db87..c28e67548f45 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -83,9 +83,12 @@ TEST_CUSTOM_PROGS = urandom_read
# $3 - target (assumed to be file); only file name will be emitted;
# $4 - optional extra arg, emitted as-is, if provided.
ifeq ($(V),1)
+Q =
msg =
else
+Q = @
msg = @$(info $(1)$(if $(2), [$(2)]) $(notdir $(3)))$(if $(4), $(4))
+MAKEFLAGS += --no-print-directory
endif
# override lib.mk's default rules
@@ -147,14 +150,14 @@ DEFAULT_BPFTOOL := $(OUTPUT)/tools/usr/local/sbin/bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
$(DEFAULT_BPFTOOL): force
- $(MAKE) -C $(BPFTOOLDIR) DESTDIR=$(OUTPUT)/tools install
+ $(Q)$(MAKE) -C $(BPFTOOLDIR) DESTDIR=$(OUTPUT)/tools install
$(BPFOBJ): force
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
+ $(Q)$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
$(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
- $(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
+ $(Q)$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
# Get Clang's default includes on this system, as opposed to those seen by
# '-target bpf'. This fixes "missing" files on some architectures/distros,
@@ -253,6 +256,7 @@ define DEFINE_TEST_RUNNER_RULES
ifeq ($($(TRUNNER_OUTPUT)-dir),)
$(TRUNNER_OUTPUT)-dir := y
$(TRUNNER_OUTPUT):
+ $$(call msg, MKDIR,,$$@)
mkdir -p $$@
endif
--
2.17.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf
2020-01-10 5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
` (2 preceding siblings ...)
2020-01-10 5:17 ` [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output Andrii Nakryiko
@ 2020-01-10 5:58 ` Alexei Starovoitov
2020-01-10 6:29 ` Andrii Nakryiko
3 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2020-01-10 5:58 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: bpf, Network Development, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Kernel Team
On Thu, Jan 9, 2020 at 9:17 PM Andrii Nakryiko <andriin@fb.com> wrote:
>
> Fix issues with bpf_helper_defs.h usage in selftests/bpf. As part of that, fix
> the way clean up is performed for libbpf and selftests/bpf. Some for Makefile
> output clean ups as well.
feature auto-detect and few unnecessary installs are now
happening even when make shouldn't be doing anything.
But overall it's still an improvement.
Applied. Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf
2020-01-10 5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
@ 2020-01-10 6:29 ` Andrii Nakryiko
0 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10 6:29 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Andrii Nakryiko, bpf, Network Development, Alexei Starovoitov,
Daniel Borkmann, Kernel Team
On Thu, Jan 9, 2020 at 9:58 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Jan 9, 2020 at 9:17 PM Andrii Nakryiko <andriin@fb.com> wrote:
> >
> > Fix issues with bpf_helper_defs.h usage in selftests/bpf. As part of that, fix
> > the way clean up is performed for libbpf and selftests/bpf. Some for Makefile
> > output clean ups as well.
>
> feature auto-detect and few unnecessary installs are now
> happening even when make shouldn't be doing anything.
> But overall it's still an improvement.
It's because of bpftool dependency. Don't see an easy way around that
yet, but I'll keep it on a back-burner...
> Applied. Thanks
^ permalink raw reply [flat|nested] 6+ messages in thread