* [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes
@ 2020-11-10 16:43 Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions Jean-Philippe Brucker
` (7 more replies)
0 siblings, 8 replies; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
A few fixes for cross and out-of-tree build of bpftool and runqslower.
These changes allow to build for different target architectures, using
the same source tree.
Since [v2], I addressed Andrii's comments on patches 3 and 5, and added
patch 7 which fixes a build slowdown.
[v2] https://lore.kernel.org/bpf/20201109110929.1223538-1-jean-philippe@linaro.org/
Jean-Philippe Brucker (7):
tools: Factor HOSTCC, HOSTLD, HOSTAR definitions
tools/bpftool: Force clean of out-of-tree build
tools/bpftool: Fix cross-build
tools/runqslower: Use Makefile.include
tools/runqslower: Enable out-of-tree build
tools/runqslower: Build bpftool using HOSTCC
tools/bpftool: Fix build slowdown
tools/bpf/bpftool/Makefile | 44 ++++++++++++++++++-------
tools/bpf/resolve_btfids/Makefile | 9 -----
tools/bpf/runqslower/Makefile | 55 +++++++++++++++----------------
tools/build/Makefile | 4 ---
tools/objtool/Makefile | 9 -----
tools/perf/Makefile.perf | 4 ---
tools/power/acpi/Makefile.config | 1 -
tools/scripts/Makefile.include | 10 ++++++
8 files changed, 70 insertions(+), 66 deletions(-)
--
2.29.1
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build Jean-Philippe Brucker
` (6 subsequent siblings)
7 siblings, 0 replies; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker, Jiri Olsa, Rafael J . Wysocki,
Josh Poimboeuf, Peter Zijlstra, Ingo Molnar,
Arnaldo Carvalho de Melo, Mark Rutland, Alexander Shishkin,
Namhyung Kim, Robert Moore, Erik Kaneda, Len Brown, linux-acpi,
devel
Several Makefiles in tools/ need to define the host toolchain variables.
Move their definition to tools/scripts/Makefile.include
Acked-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v1: https://lore.kernel.org/bpf/20200827153629.3820891-2-jean-philippe@linaro.org/
v2: https://lore.kernel.org/bpf/20201109110929.1223538-2-jean-philippe@linaro.org/
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Robert Moore <robert.moore@intel.com>
Cc: Erik Kaneda <erik.kaneda@intel.com>
Cc: "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>
Cc: Len Brown <lenb@kernel.org>
Cc: linux-acpi@vger.kernel.org
Cc: devel@acpica.org
---
tools/bpf/resolve_btfids/Makefile | 9 ---------
tools/build/Makefile | 4 ----
tools/objtool/Makefile | 9 ---------
tools/perf/Makefile.perf | 4 ----
tools/power/acpi/Makefile.config | 1 -
tools/scripts/Makefile.include | 10 ++++++++++
6 files changed, 10 insertions(+), 27 deletions(-)
diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index 66cb92136de4..bf656432ad73 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -18,15 +18,6 @@ else
endif
# always use the host compiler
-ifneq ($(LLVM),)
-HOSTAR ?= llvm-ar
-HOSTCC ?= clang
-HOSTLD ?= ld.lld
-else
-HOSTAR ?= ar
-HOSTCC ?= gcc
-HOSTLD ?= ld
-endif
AR = $(HOSTAR)
CC = $(HOSTCC)
LD = $(HOSTLD)
diff --git a/tools/build/Makefile b/tools/build/Makefile
index 722f1700d96a..bae48e6fa995 100644
--- a/tools/build/Makefile
+++ b/tools/build/Makefile
@@ -15,10 +15,6 @@ endef
$(call allow-override,CC,$(CROSS_COMPILE)gcc)
$(call allow-override,LD,$(CROSS_COMPILE)ld)
-HOSTCC ?= gcc
-HOSTLD ?= ld
-HOSTAR ?= ar
-
export HOSTCC HOSTLD HOSTAR
ifeq ($(V),1)
diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile
index 4ea9a833dde7..5cdb19036d7f 100644
--- a/tools/objtool/Makefile
+++ b/tools/objtool/Makefile
@@ -3,15 +3,6 @@ include ../scripts/Makefile.include
include ../scripts/Makefile.arch
# always use the host compiler
-ifneq ($(LLVM),)
-HOSTAR ?= llvm-ar
-HOSTCC ?= clang
-HOSTLD ?= ld.lld
-else
-HOSTAR ?= ar
-HOSTCC ?= gcc
-HOSTLD ?= ld
-endif
AR = $(HOSTAR)
CC = $(HOSTCC)
LD = $(HOSTLD)
diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf
index 7ce3f2e8b9c7..62f3deb1d3a8 100644
--- a/tools/perf/Makefile.perf
+++ b/tools/perf/Makefile.perf
@@ -175,10 +175,6 @@ endef
LD += $(EXTRA_LDFLAGS)
-HOSTCC ?= gcc
-HOSTLD ?= ld
-HOSTAR ?= ar
-
PKG_CONFIG = $(CROSS_COMPILE)pkg-config
LLVM_CONFIG ?= llvm-config
diff --git a/tools/power/acpi/Makefile.config b/tools/power/acpi/Makefile.config
index 54a2857c2510..331f6d30f472 100644
--- a/tools/power/acpi/Makefile.config
+++ b/tools/power/acpi/Makefile.config
@@ -54,7 +54,6 @@ INSTALL_SCRIPT = ${INSTALL_PROGRAM}
CROSS = #/usr/i386-linux-uclibc/usr/bin/i386-uclibc-
CROSS_COMPILE ?= $(CROSS)
LD = $(CC)
-HOSTCC = gcc
# check if compiler option is supported
cc-supports = ${shell if $(CC) ${1} -S -o /dev/null -x c /dev/null > /dev/null 2>&1; then echo "$(1)"; fi;}
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index a7974638561c..1358e89cdf7d 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -59,6 +59,16 @@ $(call allow-override,LD,$(CROSS_COMPILE)ld)
$(call allow-override,CXX,$(CROSS_COMPILE)g++)
$(call allow-override,STRIP,$(CROSS_COMPILE)strip)
+ifneq ($(LLVM),)
+HOSTAR ?= llvm-ar
+HOSTCC ?= clang
+HOSTLD ?= ld.lld
+else
+HOSTAR ?= ar
+HOSTCC ?= gcc
+HOSTLD ?= ld
+endif
+
ifeq ($(CC_NO_CLANG), 1)
EXTRA_WARNINGS += -Wstrict-aliasing=3
endif
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 4:57 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build Jean-Philippe Brucker
` (5 subsequent siblings)
7 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
Cleaning a partial build can fail if the output directory for libbpf
wasn't created:
$ make -C tools/bpf/bpftool O=/tmp/bpf clean
/bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist. Stop.
make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
As a result make never gets around to clearing the leftover objects. Add
the libbpf output directory as clean dependency to ensure clean always
succeeds (similarly to the "descend" macro). The directory is later
removed by the clean recipe.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
tools/bpf/bpftool/Makefile | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index f60e6ad3a1df..1358c093b812 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
-$(LIBBPF): FORCE
- $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
+$(LIBBPF_OUTPUT):
+ $(QUIET_MKDIR)mkdir -p $@
+
+$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
-$(LIBBPF)-clean:
+$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
$(call QUIET_CLEAN, libbpf)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 5:04 ` Andrii Nakryiko
2020-11-11 20:50 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include Jean-Philippe Brucker
` (4 subsequent siblings)
7 siblings, 2 replies; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
The bpftool build first creates an intermediate binary, executed on the
host, to generate skeletons required by the final build. When
cross-building bpftool for an architecture different from the host, the
intermediate binary should be built using the host compiler (gcc) and
the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).
Generate the intermediate objects into the bootstrap/ directory using
the host toolchain.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
---
tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
1 file changed, 26 insertions(+), 8 deletions(-)
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index 1358c093b812..d566bced135e 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -19,24 +19,37 @@ BPF_DIR = $(srctree)/tools/lib/bpf/
ifneq ($(OUTPUT),)
LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
LIBBPF_PATH = $(LIBBPF_OUTPUT)
+ BOOTSTRAP_OUTPUT = $(OUTPUT)/bootstrap/
else
+ LIBBPF_OUTPUT =
LIBBPF_PATH = $(BPF_DIR)
+ BOOTSTRAP_OUTPUT = $(CURDIR)/bootstrap/
endif
LIBBPF = $(LIBBPF_PATH)libbpf.a
+LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
+LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
-$(LIBBPF_OUTPUT):
+$(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
$(QUIET_MKDIR)mkdir -p $@
$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
+$(LIBBPF_BOOTSTRAP): FORCE | $(LIBBPF_BOOTSTRAP_OUTPUT)
+ $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) \
+ ARCH= CC=$(HOSTCC) LD=$(HOSTLD) $@
+
$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
$(call QUIET_CLEAN, libbpf)
$(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
+$(LIBBPF_BOOTSTRAP)-clean: $(LIBBPF_BOOTSTRAP_OUTPUT)
+ $(call QUIET_CLEAN, libbpf-bootstrap)
+ $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_BOOTSTRAP_OUTPUT) clean >/dev/null
+
prefix ?= /usr/local
bash_compdir ?= /usr/share/bash-completion/completions
@@ -94,6 +107,7 @@ CFLAGS += -DCOMPAT_NEED_REALLOCARRAY
endif
LIBS = $(LIBBPF) -lelf -lz
+LIBS_BOOTSTRAP = $(LIBBPF_BOOTSTRAP) -lelf -lz
ifeq ($(feature-libcap), 1)
CFLAGS += -DUSE_LIBCAP
LIBS += -lcap
@@ -120,9 +134,9 @@ CFLAGS += -DHAVE_LIBBFD_SUPPORT
SRCS += $(BFD_SRCS)
endif
-BPFTOOL_BOOTSTRAP := $(if $(OUTPUT),$(OUTPUT)bpftool-bootstrap,./bpftool-bootstrap)
+BPFTOOL_BOOTSTRAP := $(BOOTSTRAP_OUTPUT)bpftool
-BOOTSTRAP_OBJS = $(addprefix $(OUTPUT),main.o common.o json_writer.o gen.o btf.o)
+BOOTSTRAP_OBJS = $(addprefix $(BOOTSTRAP_OUTPUT),main.o common.o json_writer.o gen.o btf.o)
OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \
@@ -169,12 +183,16 @@ $(OUTPUT)disasm.o: $(srctree)/kernel/bpf/disasm.c
$(OUTPUT)feature.o: | zdep
-$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF)
- $(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) $(LIBS)
+$(BPFTOOL_BOOTSTRAP): $(BOOTSTRAP_OBJS) $(LIBBPF_BOOTSTRAP)
+ $(QUIET_LINK)$(HOSTCC) $(CFLAGS) $(LDFLAGS) -o $@ $(BOOTSTRAP_OBJS) \
+ $(LIBS_BOOTSTRAP)
$(OUTPUT)bpftool: $(OBJS) $(LIBBPF)
$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJS) $(LIBS)
+$(BOOTSTRAP_OUTPUT)%.o: %.c | $(BOOTSTRAP_OUTPUT)
+ $(QUIET_CC)$(HOSTCC) $(CFLAGS) -c -MMD -o $@ $<
+
$(OUTPUT)%.o: %.c
$(QUIET_CC)$(CC) $(CFLAGS) -c -MMD -o $@ $<
@@ -182,11 +200,11 @@ feature-detect-clean:
$(call QUIET_CLEAN, feature-detect)
$(Q)$(MAKE) -C $(srctree)/tools/build/feature/ clean >/dev/null
-clean: $(LIBBPF)-clean feature-detect-clean
+clean: $(LIBBPF)-clean $(LIBBPF_BOOTSTRAP)-clean feature-detect-clean
$(call QUIET_CLEAN, bpftool)
$(Q)$(RM) -- $(OUTPUT)bpftool $(OUTPUT)*.o $(OUTPUT)*.d
- $(Q)$(RM) -- $(BPFTOOL_BOOTSTRAP) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
- $(Q)$(RM) -r -- $(OUTPUT)libbpf/
+ $(Q)$(RM) -- $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
+ $(Q)$(RM) -r -- $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT)
$(call QUIET_CLEAN, core-gen)
$(Q)$(RM) -- $(OUTPUT)FEATURE-DUMP.bpftool
$(Q)$(RM) -r -- $(OUTPUT)feature/
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
` (2 preceding siblings ...)
2020-11-10 16:43 ` [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 5:06 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build Jean-Philippe Brucker
` (3 subsequent siblings)
7 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
Makefile.include defines variables such as OUTPUT and CC for out-of-tree
build and cross-build. Include it into the runqslower Makefile and use
its $(QUIET*) helpers.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
tools/bpf/runqslower/Makefile | 24 +++++++++---------------
1 file changed, 9 insertions(+), 15 deletions(-)
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index fb1337d69868..bcc4a7396713 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -1,4 +1,6 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
+include ../../scripts/Makefile.include
+
OUTPUT := .output
CLANG ?= clang
LLC ?= llc
@@ -21,10 +23,8 @@ VMLINUX_BTF_PATH := $(or $(VMLINUX_BTF),$(firstword \
abs_out := $(abspath $(OUTPUT))
ifeq ($(V),1)
Q =
-msg =
else
Q = @
-msg = @printf ' %-8s %s%s\n' "$(1)" "$(notdir $(2))" "$(if $(3), $(3))";
MAKEFLAGS += --no-print-directory
submake_extras := feature_display=0
endif
@@ -37,12 +37,11 @@ all: runqslower
runqslower: $(OUTPUT)/runqslower
clean:
- $(call msg,CLEAN)
+ $(call QUIET_CLEAN, runqslower)
$(Q)rm -rf $(OUTPUT) runqslower
$(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
- $(call msg,BINARY,$@)
- $(Q)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
+ $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
$(OUTPUT)/runqslower.o: runqslower.h $(OUTPUT)/runqslower.skel.h \
$(OUTPUT)/runqslower.bpf.o
@@ -50,31 +49,26 @@ $(OUTPUT)/runqslower.o: runqslower.h $(OUTPUT)/runqslower.skel.h \
$(OUTPUT)/runqslower.bpf.o: $(OUTPUT)/vmlinux.h runqslower.h
$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(BPFTOOL)
- $(call msg,GEN-SKEL,$@)
- $(Q)$(BPFTOOL) gen skeleton $< > $@
+ $(QUIET_GEN)$(BPFTOOL) gen skeleton $< > $@
$(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
- $(call msg,BPF,$@)
- $(Q)$(CLANG) -g -O2 -target bpf $(INCLUDES) \
+ $(QUIET_GEN)$(CLANG) -g -O2 -target bpf $(INCLUDES) \
-c $(filter %.c,$^) -o $@ && \
$(LLVM_STRIP) -g $@
$(OUTPUT)/%.o: %.c | $(OUTPUT)
- $(call msg,CC,$@)
- $(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
+ $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
$(OUTPUT):
- $(call msg,MKDIR,$@)
- $(Q)mkdir -p $(OUTPUT)
+ $(QUIET_MKDIR)mkdir -p $(OUTPUT)
$(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
- $(call msg,GEN,$@)
$(Q)if [ ! -e "$(VMLINUX_BTF_PATH)" ] ; then \
echo "Couldn't find kernel BTF; set VMLINUX_BTF to" \
"specify its location." >&2; \
exit 1;\
fi
- $(Q)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
+ $(QUIET_GEN)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)
$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) \
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
` (3 preceding siblings ...)
2020-11-10 16:43 ` [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 5:11 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC Jean-Philippe Brucker
` (2 subsequent siblings)
7 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
wasn't already set by the user.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3:
* Drop clean recipe for bpftool and libbpf, since the whole output
directories are removed by the clean recipe.
* Use ?= for $(OUTPUT)
---
tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
1 file changed, 18 insertions(+), 14 deletions(-)
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index bcc4a7396713..0fc4d4046193 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -1,15 +1,18 @@
# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
include ../../scripts/Makefile.include
-OUTPUT := .output
+OUTPUT ?= $(abspath .output)/
+
CLANG ?= clang
LLC ?= llc
LLVM_STRIP ?= llvm-strip
-DEFAULT_BPFTOOL := $(OUTPUT)/sbin/bpftool
+BPFTOOL_OUTPUT := $(OUTPUT)bpftool/
+DEFAULT_BPFTOOL := $(BPFTOOL_OUTPUT)bpftool
BPFTOOL ?= $(DEFAULT_BPFTOOL)
LIBBPF_SRC := $(abspath ../../lib/bpf)
-BPFOBJ := $(OUTPUT)/libbpf.a
-BPF_INCLUDE := $(OUTPUT)
+BPFOBJ_OUTPUT := $(OUTPUT)libbpf/
+BPFOBJ := $(BPFOBJ_OUTPUT)libbpf.a
+BPF_INCLUDE := $(BPFOBJ_OUTPUT)
INCLUDES := -I$(OUTPUT) -I$(BPF_INCLUDE) -I$(abspath ../../lib) \
-I$(abspath ../../include/uapi)
CFLAGS := -g -Wall
@@ -20,7 +23,6 @@ VMLINUX_BTF_PATHS := /sys/kernel/btf/vmlinux /boot/vmlinux-$(KERNEL_REL)
VMLINUX_BTF_PATH := $(or $(VMLINUX_BTF),$(firstword \
$(wildcard $(VMLINUX_BTF_PATHS))))
-abs_out := $(abspath $(OUTPUT))
ifeq ($(V),1)
Q =
else
@@ -38,7 +40,11 @@ runqslower: $(OUTPUT)/runqslower
clean:
$(call QUIET_CLEAN, runqslower)
- $(Q)rm -rf $(OUTPUT) runqslower
+ $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
+ $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
+ $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
+ $(Q)$(RM) $(OUTPUT)runqslower
+ $(Q)$(RM) -r .output
$(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
$(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
@@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
$(OUTPUT)/%.o: %.c | $(OUTPUT)
$(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
-$(OUTPUT):
- $(QUIET_MKDIR)mkdir -p $(OUTPUT)
+$(OUTPUT) $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT):
+ $(QUIET_MKDIR)mkdir -p $@
$(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
$(Q)if [ ! -e "$(VMLINUX_BTF_PATH)" ] ; then \
@@ -70,10 +76,8 @@ $(OUTPUT)/vmlinux.h: $(VMLINUX_BTF_PATH) | $(OUTPUT) $(BPFTOOL)
fi
$(QUIET_GEN)$(BPFTOOL) btf dump file $(VMLINUX_BTF_PATH) format c > $@
-$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)
- $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) \
- OUTPUT=$(abspath $(dir $@))/ $(abspath $@)
+$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OUTPUT)
+ $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
-$(DEFAULT_BPFTOOL):
- $(Q)$(MAKE) $(submake_extras) -C ../bpftool \
- prefix= OUTPUT=$(abs_out)/ DESTDIR=$(abs_out) install
+$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
+ $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
` (4 preceding siblings ...)
2020-11-10 16:43 ` [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 5:11 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown Jean-Philippe Brucker
2020-11-11 20:40 ` [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes patchwork-bot+netdevbpf
7 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
When cross building runqslower for an other architecture, the
intermediate bpftool used to generate a skeleton must be built using the
host toolchain. Pass HOSTCC and HOSTLD, defined in Makefile.include, to
the bpftool Makefile.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
tools/bpf/runqslower/Makefile | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
index 0fc4d4046193..4d5ca54fcd4c 100644
--- a/tools/bpf/runqslower/Makefile
+++ b/tools/bpf/runqslower/Makefile
@@ -80,4 +80,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
$(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
$(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
- $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
+ $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
+ CC=$(HOSTCC) LD=$(HOSTLD)
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
` (5 preceding siblings ...)
2020-11-10 16:43 ` [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC Jean-Philippe Brucker
@ 2020-11-10 16:43 ` Jean-Philippe Brucker
2020-11-11 5:12 ` Andrii Nakryiko
2020-11-11 20:40 ` [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes patchwork-bot+netdevbpf
7 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-10 16:43 UTC (permalink / raw)
To: ast, daniel
Cc: bpf, kafai, songliubraving, yhs, andriin, john.fastabend,
kpsingh, Jean-Philippe Brucker
Commit ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to
make") changed BPFTOOL_VERSION to a recursively expanded variable,
forcing it to be recomputed on every expansion of CFLAGS and
dramatically slowing down the bpftool build. Restore BPFTOOL_VERSION as
a simply expanded variable, guarded by an ifeq().
Fixes: ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to make")
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
v3: new
---
tools/bpf/bpftool/Makefile | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index d566bced135e..804ade95929f 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -30,7 +30,9 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
-BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
+ifeq ($(BPFTOOL_VERSION),)
+BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
+endif
$(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
$(QUIET_MKDIR)mkdir -p $@
--
2.29.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build
2020-11-10 16:43 ` [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build Jean-Philippe Brucker
@ 2020-11-11 4:57 ` Andrii Nakryiko
2020-11-11 8:54 ` Jean-Philippe Brucker
0 siblings, 1 reply; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 4:57 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Cleaning a partial build can fail if the output directory for libbpf
> wasn't created:
>
> $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist. Stop.
> make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
>
> As a result make never gets around to clearing the leftover objects. Add
> the libbpf output directory as clean dependency to ensure clean always
> succeeds (similarly to the "descend" macro). The directory is later
> removed by the clean recipe.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> tools/bpf/bpftool/Makefile | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index f60e6ad3a1df..1358c093b812 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
>
> BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
>
> -$(LIBBPF): FORCE
> - $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> +$(LIBBPF_OUTPUT):
> + $(QUIET_MKDIR)mkdir -p $@
> +
> +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
> $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
>
> -$(LIBBPF)-clean:
> +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
shouldn't this be `| $(LIBBPF_OUTPUT)` ?
> $(call QUIET_CLEAN, libbpf)
> $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) clean >/dev/null
>
> --
> 2.29.1
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build
2020-11-10 16:43 ` [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build Jean-Philippe Brucker
@ 2020-11-11 5:04 ` Andrii Nakryiko
2020-11-11 20:50 ` Andrii Nakryiko
1 sibling, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 5:04 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> The bpftool build first creates an intermediate binary, executed on the
> host, to generate skeletons required by the final build. When
> cross-building bpftool for an architecture different from the host, the
> intermediate binary should be built using the host compiler (gcc) and
> the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).
>
> Generate the intermediate objects into the bootstrap/ directory using
> the host toolchain.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
> ---
LGTM.
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
> 1 file changed, 26 insertions(+), 8 deletions(-)
>
[...]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include
2020-11-10 16:43 ` [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include Jean-Philippe Brucker
@ 2020-11-11 5:06 ` Andrii Nakryiko
0 siblings, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 5:06 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Makefile.include defines variables such as OUTPUT and CC for out-of-tree
> build and cross-build. Include it into the runqslower Makefile and use
> its $(QUIET*) helpers.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> tools/bpf/runqslower/Makefile | 24 +++++++++---------------
> 1 file changed, 9 insertions(+), 15 deletions(-)
>
[...]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build
2020-11-10 16:43 ` [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build Jean-Philippe Brucker
@ 2020-11-11 5:11 ` Andrii Nakryiko
2020-11-11 8:53 ` Jean-Philippe Brucker
0 siblings, 1 reply; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 5:11 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> wasn't already set by the user.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3:
> * Drop clean recipe for bpftool and libbpf, since the whole output
> directories are removed by the clean recipe.
> * Use ?= for $(OUTPUT)
> ---
> tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
> 1 file changed, 18 insertions(+), 14 deletions(-)
>
[...]
> clean:
> $(call QUIET_CLEAN, runqslower)
> - $(Q)rm -rf $(OUTPUT) runqslower
> + $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> + $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> + $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> + $(Q)$(RM) $(OUTPUT)runqslower
> + $(Q)$(RM) -r .output
hard-coding .output here doesn't seem right, didn't all the other
lines clean up everything already?
>
> $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
> $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
> $(OUTPUT)/%.o: %.c | $(OUTPUT)
> $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
>
[...]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC
2020-11-10 16:43 ` [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC Jean-Philippe Brucker
@ 2020-11-11 5:11 ` Andrii Nakryiko
0 siblings, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 5:11 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> When cross building runqslower for an other architecture, the
> intermediate bpftool used to generate a skeleton must be built using the
> host toolchain. Pass HOSTCC and HOSTLD, defined in Makefile.include, to
> the bpftool Makefile.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> tools/bpf/runqslower/Makefile | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/runqslower/Makefile b/tools/bpf/runqslower/Makefile
> index 0fc4d4046193..4d5ca54fcd4c 100644
> --- a/tools/bpf/runqslower/Makefile
> +++ b/tools/bpf/runqslower/Makefile
> @@ -80,4 +80,5 @@ $(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(BPFOBJ_OU
> $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(BPFOBJ_OUTPUT) $@
>
> $(DEFAULT_BPFTOOL): | $(BPFTOOL_OUTPUT)
> - $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT)
> + $(Q)$(MAKE) $(submake_extras) -C ../bpftool OUTPUT=$(BPFTOOL_OUTPUT) \
> + CC=$(HOSTCC) LD=$(HOSTLD)
> --
> 2.29.1
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown
2020-11-10 16:43 ` [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown Jean-Philippe Brucker
@ 2020-11-11 5:12 ` Andrii Nakryiko
0 siblings, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 5:12 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:45 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> Commit ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to
> make") changed BPFTOOL_VERSION to a recursively expanded variable,
> forcing it to be recomputed on every expansion of CFLAGS and
> dramatically slowing down the bpftool build. Restore BPFTOOL_VERSION as
> a simply expanded variable, guarded by an ifeq().
>
> Fixes: ba2fd563b740 ("tools/bpftool: Support passing BPFTOOL_VERSION to make")
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
Thanks a lot! I verified that it does restore the previous build speed.
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> v3: new
> ---
> tools/bpf/bpftool/Makefile | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index d566bced135e..804ade95929f 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -30,7 +30,9 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
> LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
> LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
>
> -BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> +ifeq ($(BPFTOOL_VERSION),)
> +BPFTOOL_VERSION := $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> +endif
>
> $(LIBBPF_OUTPUT) $(BOOTSTRAP_OUTPUT) $(LIBBPF_BOOTSTRAP_OUTPUT):
> $(QUIET_MKDIR)mkdir -p $@
> --
> 2.29.1
>
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build
2020-11-11 5:11 ` Andrii Nakryiko
@ 2020-11-11 8:53 ` Jean-Philippe Brucker
2020-11-11 18:23 ` Andrii Nakryiko
0 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-11 8:53 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 09:11:05PM -0800, Andrii Nakryiko wrote:
> On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> > wasn't already set by the user.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > v3:
> > * Drop clean recipe for bpftool and libbpf, since the whole output
> > directories are removed by the clean recipe.
> > * Use ?= for $(OUTPUT)
> > ---
> > tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
> > 1 file changed, 18 insertions(+), 14 deletions(-)
> >
>
> [...]
>
> > clean:
> > $(call QUIET_CLEAN, runqslower)
> > - $(Q)rm -rf $(OUTPUT) runqslower
> > + $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> > + $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> > + $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> > + $(Q)$(RM) $(OUTPUT)runqslower
> > + $(Q)$(RM) -r .output
>
> hard-coding .output here doesn't seem right, didn't all the other
> lines clean up everything already?
Yes, but to clean the source tree, the .output directory needs to be
removed. On the other hand when $(OUTPUT) is out-of-tree, we only want to
remove its content but not the dir itself, because it was created by the
user.
Thanks,
Jean
>
> >
> > $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
> > $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> > @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
> > $(OUTPUT)/%.o: %.c | $(OUTPUT)
> > $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
> >
>
> [...]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build
2020-11-11 4:57 ` Andrii Nakryiko
@ 2020-11-11 8:54 ` Jean-Philippe Brucker
2020-11-11 18:22 ` Andrii Nakryiko
0 siblings, 1 reply; 20+ messages in thread
From: Jean-Philippe Brucker @ 2020-11-11 8:54 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 08:57:51PM -0800, Andrii Nakryiko wrote:
> On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> >
> > Cleaning a partial build can fail if the output directory for libbpf
> > wasn't created:
> >
> > $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> > /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> > tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist. Stop.
> > make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
> >
> > As a result make never gets around to clearing the leftover objects. Add
> > the libbpf output directory as clean dependency to ensure clean always
> > succeeds (similarly to the "descend" macro). The directory is later
> > removed by the clean recipe.
> >
> > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > ---
> > tools/bpf/bpftool/Makefile | 8 +++++---
> > 1 file changed, 5 insertions(+), 3 deletions(-)
> >
> > diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> > index f60e6ad3a1df..1358c093b812 100644
> > --- a/tools/bpf/bpftool/Makefile
> > +++ b/tools/bpf/bpftool/Makefile
> > @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
> >
> > BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> >
> > -$(LIBBPF): FORCE
> > - $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> > +$(LIBBPF_OUTPUT):
> > + $(QUIET_MKDIR)mkdir -p $@
> > +
> > +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
> > $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
> >
> > -$(LIBBPF)-clean:
> > +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
>
> shouldn't this be `| $(LIBBPF_OUTPUT)` ?
It wouldn't have any effect here. Order-only prerequisites tell make to
only build the prerequisite before the target, but not to update the
target if the prerequisite was updated. Because $(LIBBPF)-clean is not a
file, the recipe is always run and adding the | doesn't make a difference.
Thanks,
Jean
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build
2020-11-11 8:54 ` Jean-Philippe Brucker
@ 2020-11-11 18:22 ` Andrii Nakryiko
0 siblings, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 18:22 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Wed, Nov 11, 2020 at 12:55 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Tue, Nov 10, 2020 at 08:57:51PM -0800, Andrii Nakryiko wrote:
> > On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > Cleaning a partial build can fail if the output directory for libbpf
> > > wasn't created:
> > >
> > > $ make -C tools/bpf/bpftool O=/tmp/bpf clean
> > > /bin/sh: line 0: cd: /tmp/bpf/libbpf/: No such file or directory
> > > tools/scripts/Makefile.include:17: *** output directory "/tmp/bpf/libbpf/" does not exist. Stop.
> > > make: *** [Makefile:36: /tmp/bpf/libbpf/libbpf.a-clean] Error 2
> > >
> > > As a result make never gets around to clearing the leftover objects. Add
> > > the libbpf output directory as clean dependency to ensure clean always
> > > succeeds (similarly to the "descend" macro). The directory is later
> > > removed by the clean recipe.
> > >
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > ---
> > > tools/bpf/bpftool/Makefile | 8 +++++---
> > > 1 file changed, 5 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> > > index f60e6ad3a1df..1358c093b812 100644
> > > --- a/tools/bpf/bpftool/Makefile
> > > +++ b/tools/bpf/bpftool/Makefile
> > > @@ -27,11 +27,13 @@ LIBBPF = $(LIBBPF_PATH)libbpf.a
> > >
> > > BPFTOOL_VERSION ?= $(shell make -rR --no-print-directory -sC ../../.. kernelversion)
> > >
> > > -$(LIBBPF): FORCE
> > > - $(if $(LIBBPF_OUTPUT),@mkdir -p $(LIBBPF_OUTPUT))
> > > +$(LIBBPF_OUTPUT):
> > > + $(QUIET_MKDIR)mkdir -p $@
> > > +
> > > +$(LIBBPF): FORCE | $(LIBBPF_OUTPUT)
> > > $(Q)$(MAKE) -C $(BPF_DIR) OUTPUT=$(LIBBPF_OUTPUT) $(LIBBPF_OUTPUT)libbpf.a
> > >
> > > -$(LIBBPF)-clean:
> > > +$(LIBBPF)-clean: $(LIBBPF_OUTPUT)
> >
> > shouldn't this be `| $(LIBBPF_OUTPUT)` ?
>
> It wouldn't have any effect here. Order-only prerequisites tell make to
> only build the prerequisite before the target, but not to update the
> target if the prerequisite was updated. Because $(LIBBPF)-clean is not a
> file, the recipe is always run and adding the | doesn't make a difference.
I know. I wanted it just for consistency, because everything after |
means "make sure it's completed, but it's not my direct input". But
I'm ok either way, it's not that important. Also $(LIBBPF)-clean and
$(LIBBPF_BOOTSTRAP)-clean should be .PHONY targets, but then again in
practice won't matter, because unlikely that we'll have such files.
Would be nice to follow up with a fix, but I'll apply your patch set
as is.
>
> Thanks,
> Jean
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build
2020-11-11 8:53 ` Jean-Philippe Brucker
@ 2020-11-11 18:23 ` Andrii Nakryiko
0 siblings, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 18:23 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Wed, Nov 11, 2020 at 12:54 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> On Tue, Nov 10, 2020 at 09:11:05PM -0800, Andrii Nakryiko wrote:
> > On Tue, Nov 10, 2020 at 8:46 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > >
> > > Enable out-of-tree build for runqslower. Only set OUTPUT=.output if it
> > > wasn't already set by the user.
> > >
> > > Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> > > ---
> > > v3:
> > > * Drop clean recipe for bpftool and libbpf, since the whole output
> > > directories are removed by the clean recipe.
> > > * Use ?= for $(OUTPUT)
> > > ---
> > > tools/bpf/runqslower/Makefile | 32 ++++++++++++++++++--------------
> > > 1 file changed, 18 insertions(+), 14 deletions(-)
> > >
> >
> > [...]
> >
> > > clean:
> > > $(call QUIET_CLEAN, runqslower)
> > > - $(Q)rm -rf $(OUTPUT) runqslower
> > > + $(Q)$(RM) -r $(BPFOBJ_OUTPUT) $(BPFTOOL_OUTPUT)
> > > + $(Q)$(RM) $(OUTPUT)*.o $(OUTPUT)*.d
> > > + $(Q)$(RM) $(OUTPUT)*.skel.h $(OUTPUT)vmlinux.h
> > > + $(Q)$(RM) $(OUTPUT)runqslower
> > > + $(Q)$(RM) -r .output
> >
> > hard-coding .output here doesn't seem right, didn't all the other
> > lines clean up everything already?
>
> Yes, but to clean the source tree, the .output directory needs to be
> removed. On the other hand when $(OUTPUT) is out-of-tree, we only want to
> remove its content but not the dir itself, because it was created by the
> user.
Right, this is fine.
>
> Thanks,
> Jean
>
> >
> > >
> > > $(OUTPUT)/runqslower: $(OUTPUT)/runqslower.o $(BPFOBJ)
> > > $(QUIET_LINK)$(CC) $(CFLAGS) $^ -lelf -lz -o $@
> > > @@ -59,8 +65,8 @@ $(OUTPUT)/%.bpf.o: %.bpf.c $(BPFOBJ) | $(OUTPUT)
> > > $(OUTPUT)/%.o: %.c | $(OUTPUT)
> > > $(QUIET_CC)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@
> > >
> >
> > [...]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
` (6 preceding siblings ...)
2020-11-10 16:43 ` [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown Jean-Philippe Brucker
@ 2020-11-11 20:40 ` patchwork-bot+netdevbpf
7 siblings, 0 replies; 20+ messages in thread
From: patchwork-bot+netdevbpf @ 2020-11-11 20:40 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: ast, daniel, bpf, kafai, songliubraving, yhs, andriin,
john.fastabend, kpsingh
Hello:
This series was applied to bpf/bpf-next.git (refs/heads/master):
On Tue, 10 Nov 2020 17:43:04 +0100 you wrote:
> A few fixes for cross and out-of-tree build of bpftool and runqslower.
> These changes allow to build for different target architectures, using
> the same source tree.
>
> Since [v2], I addressed Andrii's comments on patches 3 and 5, and added
> patch 7 which fixes a build slowdown.
>
> [...]
Here is the summary with links:
- [bpf-next,v3,1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions
https://git.kernel.org/bpf/bpf-next/c/c8a950d0d3b9
- [bpf-next,v3,2/7] tools/bpftool: Force clean of out-of-tree build
https://git.kernel.org/bpf/bpf-next/c/9e8929fdbb9c
- [bpf-next,v3,3/7] tools/bpftool: Fix cross-build
https://git.kernel.org/bpf/bpf-next/c/8859b0da5aac
- [bpf-next,v3,4/7] tools/runqslower: Use Makefile.include
https://git.kernel.org/bpf/bpf-next/c/3290996e7133
- [bpf-next,v3,5/7] tools/runqslower: Enable out-of-tree build
https://git.kernel.org/bpf/bpf-next/c/85e59344d079
- [bpf-next,v3,6/7] tools/runqslower: Build bpftool using HOSTCC
https://git.kernel.org/bpf/bpf-next/c/2d9393fefb50
- [bpf-next,v3,7/7] tools/bpftool: Fix build slowdown
https://git.kernel.org/bpf/bpf-next/c/0639e5e97ad9
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build
2020-11-10 16:43 ` [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build Jean-Philippe Brucker
2020-11-11 5:04 ` Andrii Nakryiko
@ 2020-11-11 20:50 ` Andrii Nakryiko
1 sibling, 0 replies; 20+ messages in thread
From: Andrii Nakryiko @ 2020-11-11 20:50 UTC (permalink / raw)
To: Jean-Philippe Brucker
Cc: Alexei Starovoitov, Daniel Borkmann, bpf, Martin Lau, Song Liu,
Yonghong Song, Andrii Nakryiko, john fastabend, KP Singh
On Tue, Nov 10, 2020 at 8:44 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
>
> The bpftool build first creates an intermediate binary, executed on the
> host, to generate skeletons required by the final build. When
> cross-building bpftool for an architecture different from the host, the
> intermediate binary should be built using the host compiler (gcc) and
> the final bpftool using the cross compiler (e.g. aarch64-linux-gnu-gcc).
>
> Generate the intermediate objects into the bootstrap/ directory using
> the host toolchain.
>
> Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
> ---
> v3: Always set LIBBPF_OUTPUT. Tidy the clean recipe.
> ---
> tools/bpf/bpftool/Makefile | 34 ++++++++++++++++++++++++++--------
> 1 file changed, 26 insertions(+), 8 deletions(-)
>
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index 1358c093b812..d566bced135e 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -19,24 +19,37 @@ BPF_DIR = $(srctree)/tools/lib/bpf/
> ifneq ($(OUTPUT),)
> LIBBPF_OUTPUT = $(OUTPUT)/libbpf/
> LIBBPF_PATH = $(LIBBPF_OUTPUT)
> + BOOTSTRAP_OUTPUT = $(OUTPUT)/bootstrap/
> else
> + LIBBPF_OUTPUT =
> LIBBPF_PATH = $(BPF_DIR)
> + BOOTSTRAP_OUTPUT = $(CURDIR)/bootstrap/
This leaves behind bootstrap dir, which is not in .gitignore. Please
follow up with a fix. Thanks!
> endif
>
> LIBBPF = $(LIBBPF_PATH)libbpf.a
> +LIBBPF_BOOTSTRAP_OUTPUT = $(BOOTSTRAP_OUTPUT)libbpf/
> +LIBBPF_BOOTSTRAP = $(LIBBPF_BOOTSTRAP_OUTPUT)libbpf.a
>
[...]
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-11-11 20:50 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-10 16:43 [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 1/7] tools: Factor HOSTCC, HOSTLD, HOSTAR definitions Jean-Philippe Brucker
2020-11-10 16:43 ` [PATCH bpf-next v3 2/7] tools/bpftool: Force clean of out-of-tree build Jean-Philippe Brucker
2020-11-11 4:57 ` Andrii Nakryiko
2020-11-11 8:54 ` Jean-Philippe Brucker
2020-11-11 18:22 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 3/7] tools/bpftool: Fix cross-build Jean-Philippe Brucker
2020-11-11 5:04 ` Andrii Nakryiko
2020-11-11 20:50 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 4/7] tools/runqslower: Use Makefile.include Jean-Philippe Brucker
2020-11-11 5:06 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 5/7] tools/runqslower: Enable out-of-tree build Jean-Philippe Brucker
2020-11-11 5:11 ` Andrii Nakryiko
2020-11-11 8:53 ` Jean-Philippe Brucker
2020-11-11 18:23 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 6/7] tools/runqslower: Build bpftool using HOSTCC Jean-Philippe Brucker
2020-11-11 5:11 ` Andrii Nakryiko
2020-11-10 16:43 ` [PATCH bpf-next v3 7/7] tools/bpftool: Fix build slowdown Jean-Philippe Brucker
2020-11-11 5:12 ` Andrii Nakryiko
2020-11-11 20:40 ` [PATCH bpf-next v3 0/7] tools/bpftool: Some build fixes patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).