bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf
@ 2020-01-10  5:17 Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
                   ` (3 more replies)
  0 siblings, 4 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10  5:17 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Fix issues with bpf_helper_defs.h usage in selftests/bpf. As part of that, fix
the way clean up is performed for libbpf and selftests/bpf. Some for Makefile
output clean ups as well.

Andrii Nakryiko (3):
  libbpf,selftests/bpf: fix clean targets
  selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir
  selftests/bpf: further clean up Makefile output

 tools/lib/bpf/Makefile                 |  9 +++++----
 tools/lib/bpf/bpf_helpers.h            |  2 +-
 tools/testing/selftests/bpf/.gitignore |  2 --
 tools/testing/selftests/bpf/Makefile   | 27 +++++++++++++++-----------
 4 files changed, 22 insertions(+), 18 deletions(-)

-- 
2.17.1


^ permalink raw reply	[flat|nested] 6+ messages in thread
* [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets
  2020-01-10  5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
@ 2020-01-10  5:17 ` Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir Andrii Nakryiko
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10  5:17 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Libbpf's clean target should clean out generated files in $(OUTPUT) directory
and not make assumption that $(OUTPUT) directory is current working directory.

Selftest's Makefile should delegate cleaning of libbpf-generated files to
libbpf's Makefile. This ensures more robust clean up.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/lib/bpf/Makefile               | 9 +++++----
 tools/testing/selftests/bpf/Makefile | 5 +++--
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index d87830e7ea63..db2afccde757 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -273,10 +273,11 @@ config-clean:
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
 clean:
-	$(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
-		*.o *~ *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) .*.d .*.cmd \
-		*.pc LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
-		$(SHARED_OBJDIR) $(STATIC_OBJDIR)
+	$(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS)		     \
+		*~ .*.d .*.cmd LIBBPF-CFLAGS $(BPF_HELPER_DEFS)		     \
+		$(SHARED_OBJDIR) $(STATIC_OBJDIR)			     \
+		$(addprefix $(OUTPUT),					     \
+			    *.o *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) *.pc)
 	$(call QUIET_CLEAN, core-gen) $(RM) $(OUTPUT)FEATURE-DUMP.libbpf
 
 
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index f1f949cd8ed9..cb9f18e4b98b 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -93,6 +93,7 @@ OVERRIDE_TARGETS := 1
 override define CLEAN
 	$(call msg,    CLEAN)
 	$(RM) -r $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) $(EXTRA_CLEAN)
+	$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ clean
 endef
 
 include ../lib.mk
@@ -377,5 +378,5 @@ $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
 
 EXTRA_CLEAN := $(TEST_CUSTOM_PROGS)					\
 	prog_tests/tests.h map_tests/tests.h verifier/tests.h		\
-	feature $(OUTPUT)/*.o $(OUTPUT)/no_alu32 $(OUTPUT)/bpf_gcc	\
-	tools *.skel.h
+	feature								\
+	$(addprefix $(OUTPUT)/,*.o *.skel.h no_alu32 bpf_gcc tools)
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir
  2020-01-10  5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
@ 2020-01-10  5:17 ` Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output Andrii Nakryiko
  2020-01-10  5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
  3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10  5:17 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Reorder includes search path to ensure $(OUTPUT) and $(CURDIR) go before
libbpf's directory. Also fix bpf_helpers.h to include bpf_helper_defs.h in
such a way as to leverage includes search path. This allows selftests to not
use libbpf's local and potentially stale bpf_helper_defs.h. It's important
because selftests/bpf's Makefile only re-generates bpf_helper_defs.h in
seltests' output directory, not the one in libbpf's directory.

Also force regeneration of bpf_helper_defs.h when libbpf.a is updated to
reduce staleness.

Fixes: fa633a0f8919 ("libbpf: Fix build on read-only filesystems")
Reported-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/lib/bpf/bpf_helpers.h          |  2 +-
 tools/testing/selftests/bpf/Makefile | 12 ++++++------
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h
index f69cc208778a..050bb7bf5be6 100644
--- a/tools/lib/bpf/bpf_helpers.h
+++ b/tools/lib/bpf/bpf_helpers.h
@@ -2,7 +2,7 @@
 #ifndef __BPF_HELPERS__
 #define __BPF_HELPERS__
 
-#include "bpf_helper_defs.h"
+#include <bpf_helper_defs.h>
 
 #define __uint(name, val) int (*name)[val]
 #define __type(name, val) typeof(val) *name
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index cb9f18e4b98b..c0a18994db87 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -20,8 +20,8 @@ CLANG		?= clang
 LLC		?= llc
 LLVM_OBJCOPY	?= llvm-objcopy
 BPF_GCC		?= $(shell command -v bpf-gcc;)
-CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(APIDIR) -I$(LIBDIR) -I$(BPFDIR)	\
-	  -I$(GENDIR) -I$(TOOLSINCDIR) -I$(CURDIR)			\
+CFLAGS += -g -Wall -O2 $(GENFLAGS) -I$(CURDIR) -I$(APIDIR) -I$(LIBDIR)  \
+	  -I$(BPFDIR) -I$(GENDIR) -I$(TOOLSINCDIR)			\
 	  -Dbpf_prog_load=bpf_prog_test_load				\
 	  -Dbpf_load_program=bpf_test_load_program
 LDLIBS += -lcap -lelf -lz -lrt -lpthread
@@ -153,7 +153,7 @@ $(BPFOBJ): force
 	$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
 
 BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
-$(OUTPUT)/bpf_helper_defs.h:
+$(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
 	$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
 
 # Get Clang's default includes on this system, as opposed to those seen by
@@ -174,8 +174,8 @@ MENDIAN=$(if $(IS_LITTLE_ENDIAN),-mlittle-endian,-mbig-endian)
 
 CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG))
 BPF_CFLAGS = -g -D__TARGET_ARCH_$(SRCARCH) $(MENDIAN) 			\
-	     -I. -I./include/uapi -I$(APIDIR)				\
-	     -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
+	     -I$(OUTPUT) -I$(CURDIR) -I$(CURDIR)/include/uapi		\
+	     -I$(APIDIR) -I$(BPFDIR) -I$(abspath $(OUTPUT)/../usr/include)
 
 CLANG_CFLAGS = $(CLANG_SYS_INCLUDES) \
 	       -Wno-compare-distinct-pointer-types
@@ -329,7 +329,7 @@ TRUNNER_EXTRA_SOURCES := test_progs.c cgroup_helpers.c trace_helpers.c	\
 TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read				\
 		       $(wildcard progs/btf_dump_test_case_*.c)
 TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
-TRUNNER_BPF_CFLAGS := -I. -I$(OUTPUT) $(BPF_CFLAGS) $(CLANG_CFLAGS)
+TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
 TRUNNER_BPF_LDFLAGS := -mattr=+alu32
 $(eval $(call DEFINE_TEST_RUNNER,test_progs))
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output
  2020-01-10  5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
  2020-01-10  5:17 ` [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir Andrii Nakryiko
@ 2020-01-10  5:17 ` Andrii Nakryiko
  2020-01-10  5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
  3 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10  5:17 UTC (permalink / raw)
  To: bpf, netdev, ast, daniel; +Cc: andrii.nakryiko, kernel-team, Andrii Nakryiko

Further clean up Makefile output:
- hide "entering directory" messages;
- silvence sub-Make command echoing;
- succinct MKDIR messages.

Also remove few test binaries that are not produced anymore from .gitignore.

Signed-off-by: Andrii Nakryiko <andriin@fb.com>
---
 tools/testing/selftests/bpf/.gitignore |  2 --
 tools/testing/selftests/bpf/Makefile   | 10 +++++++---
 2 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/bpf/.gitignore b/tools/testing/selftests/bpf/.gitignore
index 301ac12d5d69..1d14e3ab70be 100644
--- a/tools/testing/selftests/bpf/.gitignore
+++ b/tools/testing/selftests/bpf/.gitignore
@@ -22,11 +22,9 @@ get_cgroup_id_user
 test_skb_cgroup_id_user
 test_socket_cookie
 test_cgroup_storage
-test_select_reuseport
 test_flow_dissector
 flow_dissector_load
 test_netcnt
-test_section_names
 test_tcpnotify_user
 test_libbpf
 test_tcp_check_syncookie_user
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index c0a18994db87..c28e67548f45 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -83,9 +83,12 @@ TEST_CUSTOM_PROGS = urandom_read
 # $3 - target (assumed to be file); only file name will be emitted;
 # $4 - optional extra arg, emitted as-is, if provided.
 ifeq ($(V),1)
+Q =
 msg =
 else
+Q = @
 msg = @$(info $(1)$(if $(2), [$(2)]) $(notdir $(3)))$(if $(4), $(4))
+MAKEFLAGS += --no-print-directory
 endif
 
 # override lib.mk's default rules
@@ -147,14 +150,14 @@ DEFAULT_BPFTOOL := $(OUTPUT)/tools/usr/local/sbin/bpftool
 BPFTOOL ?= $(DEFAULT_BPFTOOL)
 
 $(DEFAULT_BPFTOOL): force
-	$(MAKE) -C $(BPFTOOLDIR) DESTDIR=$(OUTPUT)/tools install
+	$(Q)$(MAKE) -C $(BPFTOOLDIR) DESTDIR=$(OUTPUT)/tools install
 
 $(BPFOBJ): force
-	$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
+	$(Q)$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/
 
 BPF_HELPERS := $(OUTPUT)/bpf_helper_defs.h $(wildcard $(BPFDIR)/bpf_*.h)
 $(OUTPUT)/bpf_helper_defs.h: $(BPFOBJ)
-	$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
+	$(Q)$(MAKE) -C $(BPFDIR) OUTPUT=$(OUTPUT)/ $(OUTPUT)/bpf_helper_defs.h
 
 # Get Clang's default includes on this system, as opposed to those seen by
 # '-target bpf'. This fixes "missing" files on some architectures/distros,
@@ -253,6 +256,7 @@ define DEFINE_TEST_RUNNER_RULES
 ifeq ($($(TRUNNER_OUTPUT)-dir),)
 $(TRUNNER_OUTPUT)-dir := y
 $(TRUNNER_OUTPUT):
+	$$(call msg,      MKDIR,,$$@)
 	mkdir -p $$@
 endif
 
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf
  2020-01-10  5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
                   ` (2 preceding siblings ...)
  2020-01-10  5:17 ` [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output Andrii Nakryiko
@ 2020-01-10  5:58 ` Alexei Starovoitov
  2020-01-10  6:29   ` Andrii Nakryiko
  3 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2020-01-10  5:58 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: bpf, Network Development, Alexei Starovoitov, Daniel Borkmann,
	Andrii Nakryiko, Kernel Team

On Thu, Jan 9, 2020 at 9:17 PM Andrii Nakryiko <andriin@fb.com> wrote:
>
> Fix issues with bpf_helper_defs.h usage in selftests/bpf. As part of that, fix
> the way clean up is performed for libbpf and selftests/bpf. Some for Makefile
> output clean ups as well.

feature auto-detect and few unnecessary installs are now
happening even when make shouldn't be doing anything.
But overall it's still an improvement.
Applied. Thanks

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf
  2020-01-10  5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
@ 2020-01-10  6:29   ` Andrii Nakryiko
  0 siblings, 0 replies; 6+ messages in thread
From: Andrii Nakryiko @ 2020-01-10  6:29 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: Andrii Nakryiko, bpf, Network Development, Alexei Starovoitov,
	Daniel Borkmann, Kernel Team

On Thu, Jan 9, 2020 at 9:58 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Jan 9, 2020 at 9:17 PM Andrii Nakryiko <andriin@fb.com> wrote:
> >
> > Fix issues with bpf_helper_defs.h usage in selftests/bpf. As part of that, fix
> > the way clean up is performed for libbpf and selftests/bpf. Some for Makefile
> > output clean ups as well.
>
> feature auto-detect and few unnecessary installs are now
> happening even when make shouldn't be doing anything.
> But overall it's still an improvement.

It's because of bpftool dependency. Don't see an easy way around that
yet, but I'll keep it on a back-burner...

> Applied. Thanks

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-01-10  6:29 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-10  5:17 [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Andrii Nakryiko
2020-01-10  5:17 ` [PATCH bpf-next 1/3] libbpf,selftests/bpf: fix clean targets Andrii Nakryiko
2020-01-10  5:17 ` [PATCH bpf-next 2/3] selftests/bpf: ensure bpf_helper_defs.h are taken from selftests dir Andrii Nakryiko
2020-01-10  5:17 ` [PATCH bpf-next 3/3] selftests/bpf: further clean up Makefile output Andrii Nakryiko
2020-01-10  5:58 ` [PATCH bpf-next 0/3] Fix usage of bpf_helper_defs.h in selftests/bpf Alexei Starovoitov
2020-01-10  6:29   ` Andrii Nakryiko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).