[PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes

[PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes

[Jean-Philippe Brucker]

A few fixes for cross and out-of-tree build of bpftool and runqslower.
These changes allow to build for different target architectures, using
the same source tree.

Since [v2], I addressed Andrii's comments on patches 3 and 5, and added
patch 7 which fixes a build slowdown.

[v2] https://lore.kernel.org/bpf/20201109110929.1223538-1-jean-philippe@linaro.org/

Jean-Philippe Brucker (7):
  tools: Factor HOSTCC, HOSTLD, HOSTAR definitions
  tools/bpftool: Force clean of out-of-tree build
  tools/bpftool: Fix cross-build
  tools/runqslower: Use Makefile.include
  tools/runqslower: Enable out-of-tree build
  tools/runqslower: Build bpftool using HOSTCC
  tools/bpftool: Fix build slowdown

 tools/bpf/bpftool/Makefile        | 44 ++++++++++++++++++-------
 tools/bpf/resolve_btfids/Makefile |  9 -----
 tools/bpf/runqslower/Makefile     | 55 +++++++++++++++----------------
 tools/build/Makefile              |  4 ---
 tools/objtool/Makefile            |  9 -----
 tools/perf/Makefile.perf          |  4 ---
 tools/power/acpi/Makefile.config  |  1 -
 tools/scripts/Makefile.include    | 10 ++++++
 8 files changed, 70 insertions(+), 66 deletions(-)

-- 
2.29.1

[PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions

[Jean-Philippe Brucker]

Several Makefiles in tools/ need to define the host toolchain variables.
Move their definition to tools/scripts/Makefile.include

Acked-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v1: https://lore.kernel.org/bpf/20200827153629.3820891-2-jean-philippe@linaro.org/
v2: https://lore.kernel.org/bpf/20201109110929.1223538-2-jean-philippe@linaro.org/

Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Robert Moore <robert.moore@intel.com>
Cc: Erik Kaneda <erik.kaneda@intel.com>
Cc: "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>
Cc: Len Brown <lenb@kernel.org>
Cc: linux-acpi@vger.kernel.org
Cc: devel@acpica.org
---
 tools/bpf/resolve_btfids/Makefile |  9 ---------
 tools/build/Makefile              |  4 ----
 tools/objtool/Makefile            |  9 ---------
 tools/perf/Makefile.perf          |  4 ----
 tools/power/acpi/Makefile.config  |  1 -
 tools/scripts/Makefile.include    | 10 ++++++++++
 6 files changed, 10 insertions(+), 27 deletions(-)

diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index 66cb92136de4..bf656432ad73 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -18,15 +18,6 @@ else
 endif
 
 # always use the host compiler
-ifneq ($(LLVM),)
-HOSTAR  ?= llvm-ar
-HOSTCC  ?= clang
-HOSTLD  ?= ld.lld
-else
-HOSTAR  ?= ar
-HOSTCC  ?= gcc
-HOSTLD  ?= ld
-endif
 AR       = $(HOSTAR)
 CC       = $(HOSTCC)
 LD       = $(HOSTLD)
diff --git a/tools/build/Makefile b/tools/build/Makefile
index 722f1700d96a..bae48e6fa995 100644
--- a/tools/build/Makefile
+++ b/tools/build/Makefile
@@ -15,10 +15,6 @@ endef
 $(call allow-override,CC,$(CROSS_COMPILE)gcc)
 $(call allow-override,LD,$(CROSS_COMPILE)ld)
 
-HOSTCC ?= gcc
-HOSTLD ?= ld
-HOSTAR ?= ar
-
 export HOSTCC HOSTLD HOSTAR
 
 ifeq ($(V),1)
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 4ea9a833dde7..5cdb19036d7f 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -3,15 +3,6 @@ include ../scripts/Makefile.include
 include ../scripts/Makefile.arch
 
 # always use the host compiler
-ifneq ($(LLVM),)
-HOSTAR	?= llvm-ar
-HOSTCC	?= clang
-HOSTLD	?= ld.lld
-else
-HOSTAR	?= ar
-HOSTCC	?= gcc
-HOSTLD	?= ld
-endif
 AR	 = $(HOSTAR)
 CC	 = $(HOSTCC)
 LD	 = $(HOSTLD)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 7ce3f2e8b9c7..62f3deb1d3a8 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -175,10 +175,6 @@ endef
 
 LD += $(EXTRA_LDFLAGS)
 
-HOSTCC  ?= gcc
-HOSTLD  ?= ld
-HOSTAR  ?= ar
-
 PKG_CONFIG = $(CROSS_COMPILE)pkg-config
 LLVM_CONFIG ?= llvm-config
 
diff --git a/tools/power/acpi/Makefile.config b/tools/power/acpi/Makefile.config
index 54a2857c2510..331f6d30f472 100644
--- a/tools/power/acpi/Makefile.config
+++ b/tools/power/acpi/Makefile.config
@@ -54,7 +54,6 @@ INSTALL_SCRIPT = ${INSTALL_PROGRAM}
 CROSS = #/usr/i386-linux-uclibc/usr/bin/i386-uclibc-
 CROSS_COMPILE ?= $(CROSS)
 LD = $(CC)
-HOSTCC = gcc
 
 # check if compiler option is supported
 cc-supports = ${shell if $(CC) ${1} -S -o /dev/null -x c /dev/null > /dev/null 2>&1; then echo "$(1)"; fi;}
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index a7974638561c..1358e89cdf7d 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -59,6 +59,16 @@ $(call allow-override,LD,$(CROSS_COMPILE)ld)
 $(call allow-override,CXX,$(CROSS_COMPILE)g++)
 $(call allow-override,STRIP,$(CROSS_COMPILE)strip)
 
+ifneq ($(LLVM),)
+HOSTAR  ?= llvm-ar
+HOSTCC  ?= clang
+HOSTLD  ?= ld.lld
+else
+HOSTAR  ?= ar
+HOSTCC  ?= gcc
+HOSTLD  ?= ld
+endif
+
 ifeq ($(CC_NO_CLANG), 1)
 EXTRA_WARNINGS += -Wstrict-aliasing=3
 endif
-- 
2.29.1

[PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build

[Jean-Philippe Brucker]

Cleaning a partial build can fail if the output directory for libbpf
wasn't created:

$ make -C tools/bpf/bpftool O=/tmp/bpf clean
/bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist.  Stop.
make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2

As a result make never gets around to clearing the leftover objects. Add
the libbpf output directory as clean dependency to ensure clean always
succeeds (similarly to the "descend" macro). The directory is later
removed by the clean recipe.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 tools/bpf/bpftool/Makefile | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index f60e6ad3a1df..1358c093b812 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
 
 BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 
-$(LIBBPF): FORCE
-	$(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
+$(LIBBPF_OUTPUT):
+	$(QUIET_MKDIR)mkdir -p $@
+
+$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
 	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
 
-$(LIBBPF)-clean:
+$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
 	$(call QUIET_CLEAN, libbpf)
 	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
 
-- 
2.29.1

[PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build

[Jean-Philippe Brucker]

The bpftool build first creates an intermediate binary, executed on the
host, to generate skeletons required by the final build. When
cross-building bpftool for an architecture different from the host, the
intermediate binary should be built using the host compiler (gcc) and
the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).

Generate the intermediate objects into the bootstrap/ directory using
the host toolchain.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
---
 tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 1358c093b812..d566bced135e 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -19,24 +19,37 @@ BPF_DIR = $(srctree)/tools/lib/bpf/
 ifneq ($(OUTPUT),)
   LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
   LIBBPF_PATH = $(LIBBPF_OUTPUT)
+  BOOTSTRAP_OUTPUT = $(OUTPUT)/bootstrap/
 else
+  LIBBPF_OUTPUT =
   LIBBPF_PATH = $(BPF_DIR)
+  BOOTSTRAP_OUTPUT = $(CURDIR)/bootstrap/
 endif
 
 LIBBPF = $(LIBBPF_PATH)libbpf.a
+LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
+LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
 
 BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
 
-$(LIBBPF_OUTPUT):
+$(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
 	$(QUIET_MKDIR)mkdir -p $@
 
 $(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
 	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
 
+$(LIBBPF_BOOTSTRAP): FORCE | $(LIBBPF_BOOTSTRAP_OUTPUT)
+	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) \
+		ARCH= CC=$(HOSTCC) LD=$(HOSTLD) $@
+
 $(LIBBPF)-clean: $(LIBBPF_OUTPUT)
 	$(call QUIET_CLEAN, libbpf)
 	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
 
+$(LIBBPF_BOOTSTRAP)-clean: $(LIBBPF_BOOTSTRAP_OUTPUT)
+	$(call QUIET_CLEAN, libbpf-bootstrap)
+	$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) clean >/dev/null
+
 prefix ?= /usr/local
 bash_compdir ?= /usr/share/bash-completion/completions
 
@@ -94,6 +107,7 @@ CFLAGS += -DCOMPAT_NEED_REALLOCARRAY
 endif
 
 LIBS = $(LIBBPF) -lelf -lz
+LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
 ifeq ($(feature-libcap), 1)
 CFLAGS += -DUSE_LIBCAP
 LIBS += -lcap
@@ -120,9 +134,9 @@ CFLAGS += -DHAVE_LIBBFD_SUPPORT
 SRCS += $(BFD_SRCS)
 endif
 
-BPFTOOL_BOOTSTRAP := $(if $(OUTPUT),$(OUTPUT)bpftool-bootstrap,./bpftool-bootstrap)
+BPFTOOL_BOOTSTRAP := $(BOOTSTRAP_OUTPUT)bpftool
 
-BOOTSTRAP_OBJS = $(addprefix $(OUTPUT),main.o common.o json_writer.o gen.o btf.o)
+BOOTSTRAP_OBJS = $(addprefix $(BOOTSTRAP_OUTPUT),main.o common.o json_writer.o gen.o btf.o)
 OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
 
 VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)				\
@@ -169,12 +183,16 @@ $(OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
 
 $(OUTPUT)feature.o: | zdep
 
-$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF)
-	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) $(LIBS)
+$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF_BOOTSTRAP)
+	$(QUIET_LINK)$(HOSTCC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) \
+		$(LIBS_BOOTSTRAP)
 
 $(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
 
+$(BOOTSTRAP_OUTPUT)%.o: %.c | $(BOOTSTRAP_OUTPUT)
+	$(QUIET_CC)$(HOSTCC) $(CFLAGS) -c -MMD -o $@ $<
+
 $(OUTPUT)%.o: %.c
 	$(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
 
@@ -182,11 +200,11 @@ feature-detect-clean:
 	$(call QUIET_CLEAN, feature-detect)
 	$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
 
-clean: $(LIBBPF)-clean feature-detect-clean
+clean: $(LIBBPF)-clean $(LIBBPF_BOOTSTRAP)-clean feature-detect-clean
 	$(call QUIET_CLEAN, bpftool)
 	$(Q)$(RM) -- $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
-	$(Q)$(RM) -- $(BPFTOOL_BOOTSTRAP) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
-	$(Q)$(RM) -r -- $(OUTPUT)libbpf/
+	$(Q)$(RM) -- $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
+	$(Q)$(RM) -r -- $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT)
 	$(call QUIET_CLEAN, core-gen)
 	$(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.bpftool
 	$(Q)$(RM) -r -- $(OUTPUT)feature/
-- 
2.29.1

[PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include

[Jean-Philippe Brucker]

Makefile.include defines variables such as OUTPUT and CC for out-of-tree
build and cross-build. Include it into the runqslower Makefile and use
its $(QUIET*) helpers.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 tools/bpf/runqslower/Makefile | 24 +++++++++---------------
 1 file changed, 9 insertions(+), 15 deletions(-)

diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index fb1337d69868..bcc4a7396713 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -1,4 +1,6 @@
 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+include ../../scripts/Makefile.include
+
 OUTPUT := .output
 CLANG ?= clang
 LLC ?= llc
@@ -21,10 +23,8 @@ VMLINUX_BTF_PATH := $(or $(VMLINUX_BTF),$(firstword			       \
 abs_out := $(abspath $(OUTPUT))
 ifeq ($(V),1)
 Q =
-msg =
 else
 Q = @
-msg = @printf '  %-8s %s%s\n' "$(1)" "$(notdir $(2))" "$(if $(3), $(3))";
 MAKEFLAGS += --no-print-directory
 submake_extras := feature_display=0
 endif
@@ -37,12 +37,11 @@ all: runqslower
 runqslower: $(OUTPUT)/runqslower
 
 clean:
-	$(call msg,CLEAN)
+	$(call QUIET_CLEAN, runqslower)
 	$(Q)rm -rf $(OUTPUT) runqslower
 
 $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
-	$(call msg,BINARY,$@)
-	$(Q)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
+	$(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
 
 $(OUTPUT)/runqslower.o: runqslower.h $(OUTPUT)/runqslower.skel.h	      \
 			$(OUTPUT)/runqslower.bpf.o
@@ -50,31 +49,26 @@ $(OUTPUT)/runqslower.o: runqslower.h $(OUTPUT)/runqslower.skel.h	      \
 $(OUTPUT)/runqslower.bpf.o: $(OUTPUT)/vmlinux.h runqslower.h
 
 $(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(BPFTOOL)
-	$(call msg,GEN-SKEL,$@)
-	$(Q)$(BPFTOOL) gen skeleton $< > $@
+	$(QUIET_GEN)$(BPFTOOL) gen skeleton $< > $@
 
 $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
-	$(call msg,BPF,$@)
-	$(Q)$(CLANG) -g -O2 -target bpf $(INCLUDES)			      \
+	$(QUIET_GEN)$(CLANG) -g -O2 -target bpf $(INCLUDES)		      \
 		 -c $(filter %.c,$^) -o $@ &&				      \
 	$(LLVM_STRIP) -g $@
 
 $(OUTPUT)/%.o: %.c | $(OUTPUT)
-	$(call msg,CC,$@)
-	$(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
+	$(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
 
 $(OUTPUT):
-	$(call msg,MKDIR,$@)
-	$(Q)mkdir -p $(OUTPUT)
+	$(QUIET_MKDIR)mkdir -p $(OUTPUT)
 
 $(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
-	$(call msg,GEN,$@)
 	$(Q)if [ ! -e "$(VMLINUX_BTF_PATH)" ] ; then \
 		echo "Couldn't find kernel BTF; set VMLINUX_BTF to"	       \
 			"specify its location." >&2;			       \
 		exit 1;\
 	fi
-	$(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
+	$(QUIET_GEN)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
 
 $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)
 	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC)			       \
-- 
2.29.1

[PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build

[Jean-Philippe Brucker]

Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
wasn't already set by the user.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3:
* Drop clean recipe for bpftool and libbpf, since the whole output
  directories are removed by the clean recipe.
* Use ?= for $(OUTPUT)
---
 tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index bcc4a7396713..0fc4d4046193 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -1,15 +1,18 @@
 # SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
 include ../../scripts/Makefile.include
 
-OUTPUT := .output
+OUTPUT ?= $(abspath .output)/
+
 CLANG ?= clang
 LLC ?= llc
 LLVM_STRIP ?= llvm-strip
-DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
+BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
 BPFTOOL ?= $(DEFAULT_BPFTOOL)
 LIBBPF_SRC := $(abspath ../../lib/bpf)
-BPFOBJ := $(OUTPUT)/libbpf.a
-BPF_INCLUDE := $(OUTPUT)
+BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
+BPFOBJ := $(BPFOBJ_OUTPUT)libbpf.a
+BPF_INCLUDE := $(BPFOBJ_OUTPUT)
 INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../lib)        \
        -I$(abspath ../../include/uapi)
 CFLAGS := -g -Wall
@@ -20,7 +23,6 @@ VMLINUX_BTF_PATHS := /sys/kernel/btf/vmlinux /boot/vmlinux-$(KERNEL_REL)
 VMLINUX_BTF_PATH := $(or $(VMLINUX_BTF),$(firstword			       \
 					  $(wildcard $(VMLINUX_BTF_PATHS))))
 
-abs_out := $(abspath $(OUTPUT))
 ifeq ($(V),1)
 Q =
 else
@@ -38,7 +40,11 @@ runqslower: $(OUTPUT)/runqslower
 
 clean:
 	$(call QUIET_CLEAN, runqslower)
-	$(Q)rm -rf $(OUTPUT) runqslower
+	$(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
+	$(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
+	$(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
+	$(Q)$(RM) $(OUTPUT)runqslower
+	$(Q)$(RM) -r .output
 
 $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
 	$(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
@@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
 $(OUTPUT)/%.o: %.c | $(OUTPUT)
 	$(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
 
-$(OUTPUT):
-	$(QUIET_MKDIR)mkdir -p $(OUTPUT)
+$(OUTPUT) $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT):
+	$(QUIET_MKDIR)mkdir -p $@
 
 $(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
 	$(Q)if [ ! -e "$(VMLINUX_BTF_PATH)" ] ; then \
@@ -70,10 +76,8 @@ $(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
 	fi
 	$(QUIET_GEN)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
 
-$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)
-	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC)			       \
-		    OUTPUT=$(abspath $(dir $@))/ $(abspath $@)
+$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OUTPUT)
+	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
 
-$(DEFAULT_BPFTOOL):
-	$(Q)$(MAKE) $(submake_extras) -C ../bpftool			      \
-		    prefix= OUTPUT=$(abs_out)/ DESTDIR=$(abs_out) install
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
-- 
2.29.1

[PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC

[Jean-Philippe Brucker]

When cross building runqslower for an other architecture, the
intermediate bpftool used to generate a skeleton must be built using the
host toolchain. Pass HOSTCC and HOSTLD, defined in Makefile.include, to
the bpftool Makefile.

Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
 tools/bpf/runqslower/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index 0fc4d4046193..4d5ca54fcd4c 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -80,4 +80,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
 	$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
 
 $(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
-	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
+	$(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)   \
+		    CC=$(HOSTCC) LD=$(HOSTLD)
-- 
2.29.1

[PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown

[Jean-Philippe Brucker]

Commit ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to
make") changed BPFTOOL_VERSION to a recursively expanded variable,
forcing it to be recomputed on every expansion of CFLAGS and
dramatically slowing down the bpftool build. Restore BPFTOOL_VERSION as
a simply expanded variable, guarded by an ifeq().

Fixes: ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to make")
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3: new
---
 tools/bpf/bpftool/Makefile | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index d566bced135e..804ade95929f 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -30,7 +30,9 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
 LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
 LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
 
-BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
+ifeq ($(BPFTOOL_VERSION),)
+BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
+endif
 
 $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
 	$(QUIET_MKDIR)mkdir -p $@
-- 
2.29.1

Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Cleaning a partial build can fail if the output directory for libbpf
> wasn't created:
>
> $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist.  Stop.
> make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
>
> As a result make never gets around to clearing the leftover objects. Add
> the libbpf output directory as clean dependency to ensure clean always
> succeeds (similarly to the "descend" macro). The directory is later
> removed by the clean recipe.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
>  tools/bpf/bpftool/Makefile | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index f60e6ad3a1df..1358c093b812 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
>
>  BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
>
> -$(LIBBPF): FORCE
> -       $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> +$(LIBBPF_OUTPUT):
> +       $(QUIET_MKDIR)mkdir -p $@
> +
> +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
>         $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
>
> -$(LIBBPF)-clean:
> +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)

shouldn't this be `| $(LIBBPF_OUTPUT)` ?

>         $(call QUIET_CLEAN, libbpf)
>         $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
>
> --
> 2.29.1
>

Re: [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> The bpftool build first creates an intermediate binary, executed on the
> host, to generate skeletons required by the final build. When
> cross-building bpftool for an architecture different from the host, the
> intermediate binary should be built using the host compiler (gcc) and
> the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).
>
> Generate the intermediate objects into the bootstrap/ directory using
> the host toolchain.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
> ---

LGTM.

Acked-by: Andrii Nakryiko <andrii@kernel.org>

>  tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
>  1 file changed, 26 insertions(+), 8 deletions(-)
>

[...]

Re: [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Makefile.include defines variables such as OUTPUT and CC for out-of-tree
> build and cross-build. Include it into the runqslower Makefile and use
> its $(QUIET*) helpers.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---

Acked-by: Andrii Nakryiko <andrii@kernel.org>

>  tools/bpf/runqslower/Makefile | 24 +++++++++---------------
>  1 file changed, 9 insertions(+), 15 deletions(-)
>

[...]

Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> wasn't already set by the user.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3:
> * Drop clean recipe for bpftool and libbpf, since the whole output
>   directories are removed by the clean recipe.
> * Use ?= for $(OUTPUT)
> ---
>  tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
>  1 file changed, 18 insertions(+), 14 deletions(-)
>

[...]

>  clean:
>         $(call QUIET_CLEAN, runqslower)
> -       $(Q)rm -rf $(OUTPUT) runqslower
> +       $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> +       $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> +       $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> +       $(Q)$(RM) $(OUTPUT)runqslower
> +       $(Q)$(RM) -r .output

hard-coding .output here doesn't seem right, didn't all the other
lines clean up everything already?

>
>  $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
>         $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
>  $(OUTPUT)/%.o: %.c | $(OUTPUT)
>         $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
>

[...]

Re: [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> When cross building runqslower for an other architecture, the
> intermediate bpftool used to generate a skeleton must be built using the
> host toolchain. Pass HOSTCC and HOSTLD, defined in Makefile.include, to
> the bpftool Makefile.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---

Acked-by: Andrii Nakryiko <andrii@kernel.org>

>  tools/bpf/runqslower/Makefile | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
> index 0fc4d4046193..4d5ca54fcd4c 100644
> --- a/tools/bpf/runqslower/Makefile
> +++ b/tools/bpf/runqslower/Makefile
> @@ -80,4 +80,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
>         $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
>
>  $(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
> -       $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
> +       $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)   \
> +                   CC=$(HOSTCC) LD=$(HOSTLD)
> --
> 2.29.1
>

Re: [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:45 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Commit ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to
> make") changed BPFTOOL_VERSION to a recursively expanded variable,
> forcing it to be recomputed on every expansion of CFLAGS and
> dramatically slowing down the bpftool build. Restore BPFTOOL_VERSION as
> a simply expanded variable, guarded by an ifeq().
>
> Fixes: ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to make")
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---

Thanks a lot! I verified that it does restore the previous build speed.

Acked-by: Andrii Nakryiko <andrii@kernel.org>

> v3: new
> ---
>  tools/bpf/bpftool/Makefile | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index d566bced135e..804ade95929f 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -30,7 +30,9 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
>  LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
>  LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
>
> -BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> +ifeq ($(BPFTOOL_VERSION),)
> +BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> +endif
>
>  $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
>         $(QUIET_MKDIR)mkdir -p $@
> --
> 2.29.1
>

Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build

[Jean-Philippe Brucker]

On Tue, Nov 10, 2020 at 09:11:05PM -0800, Andrii Nakryiko wrote:
> On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> > wasn't already set by the user.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v3:
> > * Drop clean recipe for bpftool and libbpf, since the whole output
> >   directories are removed by the clean recipe.
> > * Use ?= for $(OUTPUT)
> > ---
> >  tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
> >  1 file changed, 18 insertions(+), 14 deletions(-)
> >
> 
> [...]
> 
> >  clean:
> >         $(call QUIET_CLEAN, runqslower)
> > -       $(Q)rm -rf $(OUTPUT) runqslower
> > +       $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> > +       $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> > +       $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> > +       $(Q)$(RM) $(OUTPUT)runqslower
> > +       $(Q)$(RM) -r .output
> 
> hard-coding .output here doesn't seem right, didn't all the other
> lines clean up everything already?

Yes, but to clean the source tree, the .output directory needs to be
removed. On the other hand when $(OUTPUT) is out-of-tree, we only want to
remove its content but not the dir itself, because it was created by the
user.

Thanks,
Jean

> 
> >
> >  $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
> >         $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> > @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
> >  $(OUTPUT)/%.o: %.c | $(OUTPUT)
> >         $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
> >
> 
> [...]

Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build

[Jean-Philippe Brucker]

On Tue, Nov 10, 2020 at 08:57:51PM -0800, Andrii Nakryiko wrote:
> On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > Cleaning a partial build can fail if the output directory for libbpf
> > wasn't created:
> >
> > $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> > /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> > tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist.  Stop.
> > make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
> >
> > As a result make never gets around to clearing the leftover objects. Add
> > the libbpf output directory as clean dependency to ensure clean always
> > succeeds (similarly to the "descend" macro). The directory is later
> > removed by the clean recipe.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> >  tools/bpf/bpftool/Makefile | 8 +++++---
> >  1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> > index f60e6ad3a1df..1358c093b812 100644
> > --- a/tools/bpf/bpftool/Makefile
> > +++ b/tools/bpf/bpftool/Makefile
> > @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
> >
> >  BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> >
> > -$(LIBBPF): FORCE
> > -       $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> > +$(LIBBPF_OUTPUT):
> > +       $(QUIET_MKDIR)mkdir -p $@
> > +
> > +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
> >         $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
> >
> > -$(LIBBPF)-clean:
> > +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
> 
> shouldn't this be `| $(LIBBPF_OUTPUT)` ?

It wouldn't have any effect here. Order-only prerequisites tell make to
only build the prerequisite before the target, but not to update the
target if the prerequisite was updated. Because $(LIBBPF)-clean is not a
file, the recipe is always run and adding the | doesn't make a difference.

Thanks,
Jean

Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build

[Andrii Nakryiko]

On Wed, Nov 11, 2020 at 12:55 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Tue, Nov 10, 2020 at 08:57:51PM -0800, Andrii Nakryiko wrote:
> > On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > Cleaning a partial build can fail if the output directory for libbpf
> > > wasn't created:
> > >
> > > $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> > > /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> > > tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist.  Stop.
> > > make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
> > >
> > > As a result make never gets around to clearing the leftover objects. Add
> > > the libbpf output directory as clean dependency to ensure clean always
> > > succeeds (similarly to the "descend" macro). The directory is later
> > > removed by the clean recipe.
> > >
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > ---
> > >  tools/bpf/bpftool/Makefile | 8 +++++---
> > >  1 file changed, 5 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> > > index f60e6ad3a1df..1358c093b812 100644
> > > --- a/tools/bpf/bpftool/Makefile
> > > +++ b/tools/bpf/bpftool/Makefile
> > > @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
> > >
> > >  BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> > >
> > > -$(LIBBPF): FORCE
> > > -       $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> > > +$(LIBBPF_OUTPUT):
> > > +       $(QUIET_MKDIR)mkdir -p $@
> > > +
> > > +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
> > >         $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
> > >
> > > -$(LIBBPF)-clean:
> > > +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
> >
> > shouldn't this be `| $(LIBBPF_OUTPUT)` ?
>
> It wouldn't have any effect here. Order-only prerequisites tell make to
> only build the prerequisite before the target, but not to update the
> target if the prerequisite was updated. Because $(LIBBPF)-clean is not a
> file, the recipe is always run and adding the | doesn't make a difference.

I know. I wanted it just for consistency, because everything after |
means "make sure it's completed, but it's not my direct input". But
I'm ok either way, it's not that important. Also $(LIBBPF)-clean and
$(LIBBPF_BOOTSTRAP)-clean should be .PHONY targets, but then again in
practice won't matter, because unlikely that we'll have such files.
Would be nice to follow up with a fix, but I'll apply your patch set
as is.

>
> Thanks,
> Jean

Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build

[Andrii Nakryiko]

On Wed, Nov 11, 2020 at 12:54 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Tue, Nov 10, 2020 at 09:11:05PM -0800, Andrii Nakryiko wrote:
> > On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> > > wasn't already set by the user.
> > >
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > ---
> > > v3:
> > > * Drop clean recipe for bpftool and libbpf, since the whole output
> > >   directories are removed by the clean recipe.
> > > * Use ?= for $(OUTPUT)
> > > ---
> > >  tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
> > >  1 file changed, 18 insertions(+), 14 deletions(-)
> > >
> >
> > [...]
> >
> > >  clean:
> > >         $(call QUIET_CLEAN, runqslower)
> > > -       $(Q)rm -rf $(OUTPUT) runqslower
> > > +       $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> > > +       $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> > > +       $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> > > +       $(Q)$(RM) $(OUTPUT)runqslower
> > > +       $(Q)$(RM) -r .output
> >
> > hard-coding .output here doesn't seem right, didn't all the other
> > lines clean up everything already?
>
> Yes, but to clean the source tree, the .output directory needs to be
> removed. On the other hand when $(OUTPUT) is out-of-tree, we only want to
> remove its content but not the dir itself, because it was created by the
> user.

Right, this is fine.

>
> Thanks,
> Jean
>
> >
> > >
> > >  $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
> > >         $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> > > @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
> > >  $(OUTPUT)/%.o: %.c | $(OUTPUT)
> > >         $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
> > >
> >
> > [...]

Re: [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes

[patchwork-bot+netdevbpf]

Hello:

This series was applied to bpf/bpf-next.git (refs/heads/master):

On Tue, 10 Nov 2020 17:43:04 +0100 you wrote:
> A few fixes for cross and out-of-tree build of bpftool and runqslower.
> These changes allow to build for different target architectures, using
> the same source tree.
> 
> Since [v2], I addressed Andrii's comments on patches 3 and 5, and added
> patch 7 which fixes a build slowdown.
> 
> [...]

Here is the summary with links:
  - [bpf-next,v3,1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions
    https://git.kernel.org/bpf/bpf-next/c/c8a950d0d3b9
  - [bpf-next,v3,2/7] tools/bpftool: Force clean of out-of-tree build
    https://git.kernel.org/bpf/bpf-next/c/9e8929fdbb9c
  - [bpf-next,v3,3/7] tools/bpftool: Fix cross-build
    https://git.kernel.org/bpf/bpf-next/c/8859b0da5aac
  - [bpf-next,v3,4/7] tools/runqslower: Use Makefile.include
    https://git.kernel.org/bpf/bpf-next/c/3290996e7133
  - [bpf-next,v3,5/7] tools/runqslower: Enable out-of-tree build
    https://git.kernel.org/bpf/bpf-next/c/85e59344d079
  - [bpf-next,v3,6/7] tools/runqslower: Build bpftool using HOSTCC
    https://git.kernel.org/bpf/bpf-next/c/2d9393fefb50
  - [bpf-next,v3,7/7] tools/bpftool: Fix build slowdown
    https://git.kernel.org/bpf/bpf-next/c/0639e5e97ad9

You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build

[Andrii Nakryiko]

On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> The bpftool build first creates an intermediate binary, executed on the
> host, to generate skeletons required by the final build. When
> cross-building bpftool for an architecture different from the host, the
> intermediate binary should be built using the host compiler (gcc) and
> the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).
>
> Generate the intermediate objects into the bootstrap/ directory using
> the host toolchain.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
> ---
>  tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
>  1 file changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index 1358c093b812..d566bced135e 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -19,24 +19,37 @@ BPF_DIR = $(srctree)/tools/lib/bpf/
>  ifneq ($(OUTPUT),)
>    LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
>    LIBBPF_PATH = $(LIBBPF_OUTPUT)
> +  BOOTSTRAP_OUTPUT = $(OUTPUT)/bootstrap/
>  else
> +  LIBBPF_OUTPUT =
>    LIBBPF_PATH = $(BPF_DIR)
> +  BOOTSTRAP_OUTPUT = $(CURDIR)/bootstrap/

This leaves behind bootstrap dir, which is not in .gitignore. Please
follow up with a fix. Thanks!

>  endif
>
>  LIBBPF = $(LIBBPF_PATH)libbpf.a
> +LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
> +LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
>

[...]