* [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
@ 2021-09-08 21:32 Andrii Nakryiko
2021-09-09 12:58 ` Daniel Borkmann
2021-09-09 21:40 ` patchwork-bot+netdevbpf
0 siblings, 2 replies; 10+ messages in thread
From: Andrii Nakryiko @ 2021-09-08 21:32 UTC (permalink / raw)
To: bpf, ast, daniel, quentin; +Cc: andrii, kernel-team
From: Quentin Monnet <quentin@isovalent.com>
Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
the deprecation of two API functions. This macro marks functions as deprecated
when libbpf's version reaches the values passed as an argument.
As part of this change libbpf_version.h header is added with recorded major
(LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
They are now part of libbpf public API and can be relied upon by user code.
libbpf_version.h is installed system-wide along other libbpf public headers.
Due to this new build-time auto-generated header, in-kernel applications
relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
include libbpf's output directory as part of a list of include search paths.
Better fix would be to use libbpf's make_install target to install public API
headers, but that clean up is left out as a future improvement. The build
changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
problems were detected.
Note that because of the constraints of the C preprocessor we have to write
a few lines of macro magic for each version used to prepare deprecation (0.6
for now).
Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
btf__get_from_id() and btf__load(), which are replaced by
btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
[0] Closes: https://github.com/libbpf/libbpf/issues/278
Co-developed-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Quentin Monnet <quentin@isovalent.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
---
v2->v3:
- adding `sleep 10` revealed two more missing dependencies in resolve_btfids
and selftest/bpf's bench, which were fixed (BPF CI);
v1->v2:
- fix bpf_preload build by adding dependency for iterators/iterators.o on
libbpf.a generation (caught by BPF CI);
kernel/bpf/preload/Makefile | 7 +++++--
tools/bpf/bpftool/Makefile | 4 ++++
tools/bpf/resolve_btfids/Makefile | 6 ++++--
tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
tools/lib/bpf/btf.h | 2 ++
tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
tools/testing/selftests/bpf/Makefile | 4 ++--
7 files changed, 53 insertions(+), 13 deletions(-)
diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
index 1951332dd15f..ac29d4e9a384 100644
--- a/kernel/bpf/preload/Makefile
+++ b/kernel/bpf/preload/Makefile
@@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
$(LIBBPF_A):
$(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
-userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
+userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
+ -I $(srctree)/tools/include/uapi \
-I $(srctree)/tools/lib/ -Wno-unused-result
userprogs := bpf_preload_umd
-clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
+clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
+
+$(obj)/iterators/iterators.o: $(LIBBPF_A)
bpf_preload_umd-objs := iterators/iterators.o
bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index d73232be1e99..06aa1616dabe 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -60,6 +60,7 @@ CFLAGS += -W -Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers
CFLAGS += $(filter-out -Wswitch-enum -Wnested-externs,$(EXTRA_WARNINGS))
CFLAGS += -DPACKAGE='"bpftool"' -D__EXPORTED_HEADERS__ \
-I$(if $(OUTPUT),$(OUTPUT),.) \
+ $(if $(LIBBPF_OUTPUT),-I$(LIBBPF_OUTPUT)) \
-I$(srctree)/kernel/bpf/ \
-I$(srctree)/tools/include \
-I$(srctree)/tools/include/uapi \
@@ -137,7 +138,10 @@ endif
BPFTOOL_BOOTSTRAP := $(BOOTSTRAP_OUTPUT)bpftool
BOOTSTRAP_OBJS = $(addprefix $(BOOTSTRAP_OUTPUT),main.o common.o json_writer.o gen.o btf.o xlated_dumper.o btf_dumper.o disasm.o)
+$(BOOTSTRAP_OBJS): $(LIBBPF_BOOTSTRAP)
+
OBJS = $(patsubst %.c,$(OUTPUT)%.o,$(SRCS)) $(OUTPUT)disasm.o
+$(OBJS): $(LIBBPF)
VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \
$(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \
diff --git a/tools/bpf/resolve_btfids/Makefile b/tools/bpf/resolve_btfids/Makefile
index bb9fa8de7e62..edc0c329cf74 100644
--- a/tools/bpf/resolve_btfids/Makefile
+++ b/tools/bpf/resolve_btfids/Makefile
@@ -26,6 +26,7 @@ LIBBPF_SRC := $(srctree)/tools/lib/bpf/
SUBCMD_SRC := $(srctree)/tools/lib/subcmd/
BPFOBJ := $(OUTPUT)/libbpf/libbpf.a
+LIBBPF_OUT := $(abspath $(dir $(BPFOBJ)))/
SUBCMDOBJ := $(OUTPUT)/libsubcmd/libsubcmd.a
BINARY := $(OUTPUT)/resolve_btfids
@@ -41,11 +42,12 @@ $(SUBCMDOBJ): fixdep FORCE | $(OUTPUT)/libsubcmd
$(Q)$(MAKE) -C $(SUBCMD_SRC) OUTPUT=$(abspath $(dir $@))/ $(abspath $@)
$(BPFOBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf
- $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(abspath $(dir $@))/ $(abspath $@)
+ $(Q)$(MAKE) $(submake_extras) -C $(LIBBPF_SRC) OUTPUT=$(LIBBPF_OUT) $(abspath $@)
CFLAGS := -g \
-I$(srctree)/tools/include \
-I$(srctree)/tools/include/uapi \
+ -I$(LIBBPF_OUT) \
-I$(LIBBPF_SRC) \
-I$(SUBCMD_SRC)
@@ -54,7 +56,7 @@ LIBS = -lelf -lz
export srctree OUTPUT CFLAGS Q
include $(srctree)/tools/build/Makefile.include
-$(BINARY_IN): fixdep FORCE | $(OUTPUT)
+$(BINARY_IN): $(BPFOBJ) fixdep FORCE | $(OUTPUT)
$(Q)$(MAKE) $(build)=resolve_btfids
$(BINARY): $(BPFOBJ) $(SUBCMDOBJ) $(BINARY_IN)
diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
index 74c3b73a5fbe..dab21e0c7cc2 100644
--- a/tools/lib/bpf/Makefile
+++ b/tools/lib/bpf/Makefile
@@ -8,7 +8,8 @@ VERSION_SCRIPT := libbpf.map
LIBBPF_VERSION := $(shell \
grep -oE '^LIBBPF_([0-9.]+)' $(VERSION_SCRIPT) | \
sort -rV | head -n1 | cut -d'_' -f2)
-LIBBPF_MAJOR_VERSION := $(firstword $(subst ., ,$(LIBBPF_VERSION)))
+LIBBPF_MAJOR_VERSION := $(word 1,$(subst ., ,$(LIBBPF_VERSION)))
+LIBBPF_MINOR_VERSION := $(word 2,$(subst ., ,$(LIBBPF_VERSION)))
MAKEFLAGS += --no-print-directory
@@ -59,7 +60,8 @@ ifndef VERBOSE
VERBOSE = 0
endif
-INCLUDES = -I. -I$(srctree)/tools/include -I$(srctree)/tools/include/uapi
+INCLUDES = -I$(if $(OUTPUT),$(OUTPUT),.) \
+ -I$(srctree)/tools/include -I$(srctree)/tools/include/uapi
export prefix libdir src obj
@@ -111,7 +113,9 @@ SHARED_OBJDIR := $(OUTPUT)sharedobjs/
STATIC_OBJDIR := $(OUTPUT)staticobjs/
BPF_IN_SHARED := $(SHARED_OBJDIR)libbpf-in.o
BPF_IN_STATIC := $(STATIC_OBJDIR)libbpf-in.o
+VERSION_HDR := $(OUTPUT)libbpf_version.h
BPF_HELPER_DEFS := $(OUTPUT)bpf_helper_defs.h
+BPF_GENERATED := $(BPF_HELPER_DEFS) $(VERSION_HDR)
LIB_TARGET := $(addprefix $(OUTPUT),$(LIB_TARGET))
LIB_FILE := $(addprefix $(OUTPUT),$(LIB_FILE))
@@ -136,7 +140,7 @@ all: fixdep
all_cmd: $(CMD_TARGETS) check
-$(BPF_IN_SHARED): force $(BPF_HELPER_DEFS)
+$(BPF_IN_SHARED): force $(BPF_GENERATED)
@(test -f ../../include/uapi/linux/bpf.h -a -f ../../../include/uapi/linux/bpf.h && ( \
(diff -B ../../include/uapi/linux/bpf.h ../../../include/uapi/linux/bpf.h >/dev/null) || \
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/bpf.h' differs from latest version at 'include/uapi/linux/bpf.h'" >&2 )) || true
@@ -154,13 +158,19 @@ $(BPF_IN_SHARED): force $(BPF_HELPER_DEFS)
echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"
-$(BPF_IN_STATIC): force $(BPF_HELPER_DEFS)
+$(BPF_IN_STATIC): force $(BPF_GENERATED)
$(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)
$(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
$(QUIET_GEN)$(srctree)/scripts/bpf_doc.py --header \
--file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS)
+$(VERSION_HDR): force
+ $(QUIET_GEN)echo "/* This file was auto-generated. */" > $@
+ @echo "" >> $@
+ @echo "#define LIBBPF_MAJOR_VERSION $(LIBBPF_MAJOR_VERSION)" >> $@
+ @echo "#define LIBBPF_MINOR_VERSION $(LIBBPF_MINOR_VERSION)" >> $@
+
$(OUTPUT)libbpf.so: $(OUTPUT)libbpf.so.$(LIBBPF_VERSION)
$(OUTPUT)libbpf.so.$(LIBBPF_VERSION): $(BPF_IN_SHARED) $(VERSION_SCRIPT)
@@ -224,10 +234,10 @@ install_lib: all_cmd
cp -fpR $(LIB_FILE) $(DESTDIR)$(libdir_SQ)
INSTALL_HEADERS = bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h xsk.h \
- bpf_helpers.h $(BPF_HELPER_DEFS) bpf_tracing.h \
+ bpf_helpers.h $(BPF_GENERATED) bpf_tracing.h \
bpf_endian.h bpf_core_read.h skel_internal.h
-install_headers: $(BPF_HELPER_DEFS)
+install_headers: $(BPF_GENERATED)
$(call QUIET_INSTALL, headers) \
$(foreach hdr,$(INSTALL_HEADERS), \
$(call do_install,$(hdr),$(prefix)/include/bpf,644);)
@@ -240,7 +250,7 @@ install: install_lib install_pkgconfig install_headers
clean:
$(call QUIET_CLEAN, libbpf) $(RM) -rf $(CMD_TARGETS) \
- *~ .*.d .*.cmd LIBBPF-CFLAGS $(BPF_HELPER_DEFS) \
+ *~ .*.d .*.cmd LIBBPF-CFLAGS $(BPF_GENERATED) \
$(SHARED_OBJDIR) $(STATIC_OBJDIR) \
$(addprefix $(OUTPUT), \
*.o *.a *.so *.so.$(LIBBPF_MAJOR_VERSION) *.pc)
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index 4a711f990904..f2e2fab950b7 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -50,9 +50,11 @@ LIBBPF_API struct btf *libbpf_find_kernel_btf(void);
LIBBPF_API struct btf *btf__load_from_kernel_by_id(__u32 id);
LIBBPF_API struct btf *btf__load_from_kernel_by_id_split(__u32 id, struct btf *base_btf);
+LIBBPF_DEPRECATED_SINCE(0, 6, "use btf__load_from_kernel_by_id instead")
LIBBPF_API int btf__get_from_id(__u32 id, struct btf **btf);
LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf);
+LIBBPF_DEPRECATED_SINCE(0, 6, "use btf__load_into_kernel instead")
LIBBPF_API int btf__load(struct btf *btf);
LIBBPF_API int btf__load_into_kernel(struct btf *btf);
LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,
diff --git a/tools/lib/bpf/libbpf_common.h b/tools/lib/bpf/libbpf_common.h
index 947d8bd8a7bb..36ac77f2bea2 100644
--- a/tools/lib/bpf/libbpf_common.h
+++ b/tools/lib/bpf/libbpf_common.h
@@ -10,6 +10,7 @@
#define __LIBBPF_LIBBPF_COMMON_H
#include <string.h>
+#include "libbpf_version.h"
#ifndef LIBBPF_API
#define LIBBPF_API __attribute__((visibility("default")))
@@ -17,6 +18,24 @@
#define LIBBPF_DEPRECATED(msg) __attribute__((deprecated(msg)))
+/* Mark a symbol as deprecated when libbpf version is >= {major}.{minor} */
+#define LIBBPF_DEPRECATED_SINCE(major, minor, msg) \
+ __LIBBPF_MARK_DEPRECATED_ ## major ## _ ## minor \
+ (LIBBPF_DEPRECATED("libbpf v" # major "." # minor "+: " msg))
+
+#define __LIBBPF_CURRENT_VERSION_GEQ(major, minor) \
+ (LIBBPF_MAJOR_VERSION > (major) || \
+ (LIBBPF_MAJOR_VERSION == (major) && LIBBPF_MINOR_VERSION >= (minor)))
+
+/* Add checks for other versions below when planning deprecation of API symbols
+ * with the LIBBPF_DEPRECATED_SINCE macro.
+ */
+#if __LIBBPF_CURRENT_VERSION_GEQ(0, 6)
+#define __LIBBPF_MARK_DEPRECATED_0_6(X) X
+#else
+#define __LIBBPF_MARK_DEPRECATED_0_6(X)
+#endif
+
/* Helper macro to declare and initialize libbpf options struct
*
* This dance with uninitialized declaration, followed by memset to zero,
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 866531c08e4f..1a4d30ff3275 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -512,14 +512,14 @@ $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
$(Q)$(CXX) $(CFLAGS) $(filter %.a %.o %.cpp,$^) $(LDLIBS) -o $@
# Benchmark runner
-$(OUTPUT)/bench_%.o: benchs/bench_%.c bench.h
+$(OUTPUT)/bench_%.o: benchs/bench_%.c bench.h $(BPFOBJ)
$(call msg,CC,,$@)
$(Q)$(CC) $(CFLAGS) -c $(filter %.c,$^) $(LDLIBS) -o $@
$(OUTPUT)/bench_rename.o: $(OUTPUT)/test_overhead.skel.h
$(OUTPUT)/bench_trigger.o: $(OUTPUT)/trigger_bench.skel.h
$(OUTPUT)/bench_ringbufs.o: $(OUTPUT)/ringbuf_bench.skel.h \
$(OUTPUT)/perfbuf_bench.skel.h
-$(OUTPUT)/bench.o: bench.h testing_helpers.h
+$(OUTPUT)/bench.o: bench.h testing_helpers.h $(BPFOBJ)
$(OUTPUT)/bench: LDLIBS += -lm
$(OUTPUT)/bench: $(OUTPUT)/bench.o $(OUTPUT)/testing_helpers.o \
$(OUTPUT)/bench_count.o \
--
2.30.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-08 21:32 [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations Andrii Nakryiko
@ 2021-09-09 12:58 ` Daniel Borkmann
2021-09-09 16:37 ` Andrii Nakryiko
2021-09-09 21:40 ` patchwork-bot+netdevbpf
1 sibling, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2021-09-09 12:58 UTC (permalink / raw)
To: Andrii Nakryiko, bpf, ast, quentin; +Cc: kernel-team
On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> From: Quentin Monnet <quentin@isovalent.com>
>
> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> the deprecation of two API functions. This macro marks functions as deprecated
> when libbpf's version reaches the values passed as an argument.
>
> As part of this change libbpf_version.h header is added with recorded major
> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> They are now part of libbpf public API and can be relied upon by user code.
> libbpf_version.h is installed system-wide along other libbpf public headers.
>
> Due to this new build-time auto-generated header, in-kernel applications
> relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> include libbpf's output directory as part of a list of include search paths.
> Better fix would be to use libbpf's make_install target to install public API
> headers, but that clean up is left out as a future improvement. The build
> changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> problems were detected.
>
> Note that because of the constraints of the C preprocessor we have to write
> a few lines of macro magic for each version used to prepare deprecation (0.6
> for now).
>
> Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> btf__get_from_id() and btf__load(), which are replaced by
> btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
>
> [0] Closes: https://github.com/libbpf/libbpf/issues/278
>
> Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> ---
> v2->v3:
> - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> and selftest/bpf's bench, which were fixed (BPF CI);
> v1->v2:
> - fix bpf_preload build by adding dependency for iterators/iterators.o on
> libbpf.a generation (caught by BPF CI);
>
> kernel/bpf/preload/Makefile | 7 +++++--
> tools/bpf/bpftool/Makefile | 4 ++++
> tools/bpf/resolve_btfids/Makefile | 6 ++++--
> tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> tools/lib/bpf/btf.h | 2 ++
> tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> tools/testing/selftests/bpf/Makefile | 4 ++--
> 7 files changed, 53 insertions(+), 13 deletions(-)
>
> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> index 1951332dd15f..ac29d4e9a384 100644
> --- a/kernel/bpf/preload/Makefile
> +++ b/kernel/bpf/preload/Makefile
> @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> $(LIBBPF_A):
> $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
>
> -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> + -I $(srctree)/tools/include/uapi \
> -I $(srctree)/tools/lib/ -Wno-unused-result
>
> userprogs := bpf_preload_umd
>
> -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> +
> +$(obj)/iterators/iterators.o: $(LIBBPF_A)
>
> bpf_preload_umd-objs := iterators/iterators.o
> bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
One small issue I ran into by accident while testing:
[root@linux bpf-next]# make -j8 kernel/bpf/
SYNC include/config/auto.conf.cmd
DESCEND objtool
CALL scripts/atomic/check-atomics.sh
CALL scripts/checksyscalls.sh
CC kernel/bpf/syscall.o
AR kernel/bpf/preload/built-in.a
CC [M] kernel/bpf/preload/bpf_preload_kern.o
CC [U] kernel/bpf/preload/iterators/iterators.o
In file included from ./tools/lib/bpf/libbpf.h:20,
from kernel/bpf/preload/iterators/iterators.c:10:
./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
13 | #include "libbpf_version.h"
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
make: *** [Makefile:1872: kernel] Error 2
For me it was the case where tools/lib/bpf/ was already built _before_ this patch
was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
can then be reproduced. I'd assume that as-is, this would affect many folks on update.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-09 12:58 ` Daniel Borkmann
@ 2021-09-09 16:37 ` Andrii Nakryiko
2021-09-09 18:08 ` Andrii Nakryiko
0 siblings, 1 reply; 10+ messages in thread
From: Andrii Nakryiko @ 2021-09-09 16:37 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Andrii Nakryiko, bpf, Alexei Starovoitov, Quentin Monnet, Kernel Team
On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> > From: Quentin Monnet <quentin@isovalent.com>
> >
> > Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> > the deprecation of two API functions. This macro marks functions as deprecated
> > when libbpf's version reaches the values passed as an argument.
> >
> > As part of this change libbpf_version.h header is added with recorded major
> > (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> > They are now part of libbpf public API and can be relied upon by user code.
> > libbpf_version.h is installed system-wide along other libbpf public headers.
> >
> > Due to this new build-time auto-generated header, in-kernel applications
> > relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> > include libbpf's output directory as part of a list of include search paths.
> > Better fix would be to use libbpf's make_install target to install public API
> > headers, but that clean up is left out as a future improvement. The build
> > changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> > explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> > problems were detected.
> >
> > Note that because of the constraints of the C preprocessor we have to write
> > a few lines of macro magic for each version used to prepare deprecation (0.6
> > for now).
> >
> > Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> > btf__get_from_id() and btf__load(), which are replaced by
> > btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> > starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> >
> > [0] Closes: https://github.com/libbpf/libbpf/issues/278
> >
> > Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> > Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> > v2->v3:
> > - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> > and selftest/bpf's bench, which were fixed (BPF CI);
> > v1->v2:
> > - fix bpf_preload build by adding dependency for iterators/iterators.o on
> > libbpf.a generation (caught by BPF CI);
> >
> > kernel/bpf/preload/Makefile | 7 +++++--
> > tools/bpf/bpftool/Makefile | 4 ++++
> > tools/bpf/resolve_btfids/Makefile | 6 ++++--
> > tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> > tools/lib/bpf/btf.h | 2 ++
> > tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> > tools/testing/selftests/bpf/Makefile | 4 ++--
> > 7 files changed, 53 insertions(+), 13 deletions(-)
> >
> > diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> > index 1951332dd15f..ac29d4e9a384 100644
> > --- a/kernel/bpf/preload/Makefile
> > +++ b/kernel/bpf/preload/Makefile
> > @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> > $(LIBBPF_A):
> > $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> >
> > -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> > +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> > + -I $(srctree)/tools/include/uapi \
> > -I $(srctree)/tools/lib/ -Wno-unused-result
> >
> > userprogs := bpf_preload_umd
> >
> > -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > +
> > +$(obj)/iterators/iterators.o: $(LIBBPF_A)
> >
> > bpf_preload_umd-objs := iterators/iterators.o
> > bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
>
> One small issue I ran into by accident while testing:
>
> [root@linux bpf-next]# make -j8 kernel/bpf/
> SYNC include/config/auto.conf.cmd
> DESCEND objtool
> CALL scripts/atomic/check-atomics.sh
> CALL scripts/checksyscalls.sh
> CC kernel/bpf/syscall.o
> AR kernel/bpf/preload/built-in.a
> CC [M] kernel/bpf/preload/bpf_preload_kern.o
> CC [U] kernel/bpf/preload/iterators/iterators.o
> In file included from ./tools/lib/bpf/libbpf.h:20,
> from kernel/bpf/preload/iterators/iterators.c:10:
> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
> 13 | #include "libbpf_version.h"
> | ^~~~~~~~~~~~~~~~~~
> compilation terminated.
> make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
> make[3]: *** Waiting for unfinished jobs....
> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> make: *** [Makefile:1872: kernel] Error 2
>
> For me it was the case where tools/lib/bpf/ was already built _before_ this patch
> was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
> can then be reproduced. I'd assume that as-is, this would affect many folks on update.
We had a similar issue even before these changes with resolve_btfids
build, because Kbuild doesn't record dependency on libbpf build
properly. I'll see how hard it is to record that in a non-intrusive
way for both resolve_btfids and preload/iterators, because doing `make
resolve_btfids_clean` or, worse, `make clean` isn't great.
>
> Thanks,
> Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-09 16:37 ` Andrii Nakryiko
@ 2021-09-09 18:08 ` Andrii Nakryiko
2021-09-09 21:43 ` Daniel Borkmann
0 siblings, 1 reply; 10+ messages in thread
From: Andrii Nakryiko @ 2021-09-09 18:08 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Andrii Nakryiko, bpf, Alexei Starovoitov, Quentin Monnet, Kernel Team
On Thu, Sep 9, 2021 at 9:37 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >
> > On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> > > From: Quentin Monnet <quentin@isovalent.com>
> > >
> > > Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> > > the deprecation of two API functions. This macro marks functions as deprecated
> > > when libbpf's version reaches the values passed as an argument.
> > >
> > > As part of this change libbpf_version.h header is added with recorded major
> > > (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> > > They are now part of libbpf public API and can be relied upon by user code.
> > > libbpf_version.h is installed system-wide along other libbpf public headers.
> > >
> > > Due to this new build-time auto-generated header, in-kernel applications
> > > relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> > > include libbpf's output directory as part of a list of include search paths.
> > > Better fix would be to use libbpf's make_install target to install public API
> > > headers, but that clean up is left out as a future improvement. The build
> > > changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> > > explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> > > problems were detected.
> > >
> > > Note that because of the constraints of the C preprocessor we have to write
> > > a few lines of macro magic for each version used to prepare deprecation (0.6
> > > for now).
> > >
> > > Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> > > btf__get_from_id() and btf__load(), which are replaced by
> > > btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> > > starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> > >
> > > [0] Closes: https://github.com/libbpf/libbpf/issues/278
> > >
> > > Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> > > Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> > > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > ---
> > > v2->v3:
> > > - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> > > and selftest/bpf's bench, which were fixed (BPF CI);
> > > v1->v2:
> > > - fix bpf_preload build by adding dependency for iterators/iterators.o on
> > > libbpf.a generation (caught by BPF CI);
> > >
> > > kernel/bpf/preload/Makefile | 7 +++++--
> > > tools/bpf/bpftool/Makefile | 4 ++++
> > > tools/bpf/resolve_btfids/Makefile | 6 ++++--
> > > tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> > > tools/lib/bpf/btf.h | 2 ++
> > > tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> > > tools/testing/selftests/bpf/Makefile | 4 ++--
> > > 7 files changed, 53 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> > > index 1951332dd15f..ac29d4e9a384 100644
> > > --- a/kernel/bpf/preload/Makefile
> > > +++ b/kernel/bpf/preload/Makefile
> > > @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> > > $(LIBBPF_A):
> > > $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> > >
> > > -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> > > +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> > > + -I $(srctree)/tools/include/uapi \
> > > -I $(srctree)/tools/lib/ -Wno-unused-result
> > >
> > > userprogs := bpf_preload_umd
> > >
> > > -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > > +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > > +
> > > +$(obj)/iterators/iterators.o: $(LIBBPF_A)
> > >
> > > bpf_preload_umd-objs := iterators/iterators.o
> > > bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
> >
> > One small issue I ran into by accident while testing:
> >
> > [root@linux bpf-next]# make -j8 kernel/bpf/
> > SYNC include/config/auto.conf.cmd
> > DESCEND objtool
> > CALL scripts/atomic/check-atomics.sh
> > CALL scripts/checksyscalls.sh
> > CC kernel/bpf/syscall.o
> > AR kernel/bpf/preload/built-in.a
> > CC [M] kernel/bpf/preload/bpf_preload_kern.o
> > CC [U] kernel/bpf/preload/iterators/iterators.o
> > In file included from ./tools/lib/bpf/libbpf.h:20,
> > from kernel/bpf/preload/iterators/iterators.c:10:
> > ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
> > 13 | #include "libbpf_version.h"
> > | ^~~~~~~~~~~~~~~~~~
> > compilation terminated.
> > make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
> > make[3]: *** Waiting for unfinished jobs....
> > make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> > make[2]: *** Waiting for unfinished jobs....
> > make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> > make: *** [Makefile:1872: kernel] Error 2
> >
> > For me it was the case where tools/lib/bpf/ was already built _before_ this patch
> > was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
> > can then be reproduced. I'd assume that as-is, this would affect many folks on update.
>
> We had a similar issue even before these changes with resolve_btfids
> build, because Kbuild doesn't record dependency on libbpf build
> properly. I'll see how hard it is to record that in a non-intrusive
> way for both resolve_btfids and preload/iterators, because doing `make
> resolve_btfids_clean` or, worse, `make clean` isn't great.
>
Hm... I can't repro this. Both preload/iterators and resolve_btfids
have $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)
dependency which will trigger libbpf rebuild if any of libbpf source
files changes. I added `sleep 1000` and everything under kernel/bpf is
blocked on this until libbpf build completes. For full kernel build it
also shouldn't happen because we build resolve_btfids first before
proceeding to building other files and resolve_btfids should trigger
libbpf build.
I assume `make clean kernel/bpf` fixed the issue for you in that case?
Also, how did you build tools/lib/bpf before running `make -j8 kernel/bpf`?
Honestly, hard to tell how this happens without being able to repro
this. So far everything seems to be working for me. I've tried with
and without the KBUILD_OUTPUT envvar set. I also tried O=<path>.
> >
> > Thanks,
> > Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-08 21:32 [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations Andrii Nakryiko
2021-09-09 12:58 ` Daniel Borkmann
@ 2021-09-09 21:40 ` patchwork-bot+netdevbpf
1 sibling, 0 replies; 10+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-09-09 21:40 UTC (permalink / raw)
To: Andrii Nakryiko; +Cc: bpf, ast, daniel, quentin, kernel-team
Hello:
This patch was applied to bpf/bpf-next.git (refs/heads/master):
On Wed, 8 Sep 2021 14:32:26 -0700 you wrote:
> From: Quentin Monnet <quentin@isovalent.com>
>
> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> the deprecation of two API functions. This macro marks functions as deprecated
> when libbpf's version reaches the values passed as an argument.
>
> As part of this change libbpf_version.h header is added with recorded major
> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> They are now part of libbpf public API and can be relied upon by user code.
> libbpf_version.h is installed system-wide along other libbpf public headers.
>
> [...]
Here is the summary with links:
- [v3,bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
https://git.kernel.org/bpf/bpf-next/c/0b46b7550560
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-09 18:08 ` Andrii Nakryiko
@ 2021-09-09 21:43 ` Daniel Borkmann
2021-09-09 22:31 ` Andrii Nakryiko
0 siblings, 1 reply; 10+ messages in thread
From: Daniel Borkmann @ 2021-09-09 21:43 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Andrii Nakryiko, bpf, Alexei Starovoitov, Quentin Monnet, Kernel Team
On 9/9/21 8:08 PM, Andrii Nakryiko wrote:
> On Thu, Sep 9, 2021 at 9:37 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>> On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>>> On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
>>>> From: Quentin Monnet <quentin@isovalent.com>
>>>>
>>>> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
>>>> the deprecation of two API functions. This macro marks functions as deprecated
>>>> when libbpf's version reaches the values passed as an argument.
>>>>
>>>> As part of this change libbpf_version.h header is added with recorded major
>>>> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
>>>> They are now part of libbpf public API and can be relied upon by user code.
>>>> libbpf_version.h is installed system-wide along other libbpf public headers.
>>>>
>>>> Due to this new build-time auto-generated header, in-kernel applications
>>>> relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
>>>> include libbpf's output directory as part of a list of include search paths.
>>>> Better fix would be to use libbpf's make_install target to install public API
>>>> headers, but that clean up is left out as a future improvement. The build
>>>> changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
>>>> explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
>>>> problems were detected.
>>>>
>>>> Note that because of the constraints of the C preprocessor we have to write
>>>> a few lines of macro magic for each version used to prepare deprecation (0.6
>>>> for now).
>>>>
>>>> Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
>>>> btf__get_from_id() and btf__load(), which are replaced by
>>>> btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
>>>> starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
>>>>
>>>> [0] Closes: https://github.com/libbpf/libbpf/issues/278
>>>>
>>>> Co-developed-by: Quentin Monnet <quentin@isovalent.com>
>>>> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
>>>> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
>>>> ---
>>>> v2->v3:
>>>> - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
>>>> and selftest/bpf's bench, which were fixed (BPF CI);
>>>> v1->v2:
>>>> - fix bpf_preload build by adding dependency for iterators/iterators.o on
>>>> libbpf.a generation (caught by BPF CI);
>>>>
>>>> kernel/bpf/preload/Makefile | 7 +++++--
>>>> tools/bpf/bpftool/Makefile | 4 ++++
>>>> tools/bpf/resolve_btfids/Makefile | 6 ++++--
>>>> tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
>>>> tools/lib/bpf/btf.h | 2 ++
>>>> tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
>>>> tools/testing/selftests/bpf/Makefile | 4 ++--
>>>> 7 files changed, 53 insertions(+), 13 deletions(-)
>>>>
>>>> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
>>>> index 1951332dd15f..ac29d4e9a384 100644
>>>> --- a/kernel/bpf/preload/Makefile
>>>> +++ b/kernel/bpf/preload/Makefile
>>>> @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
>>>> $(LIBBPF_A):
>>>> $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
>>>>
>>>> -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
>>>> +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
>>>> + -I $(srctree)/tools/include/uapi \
>>>> -I $(srctree)/tools/lib/ -Wno-unused-result
>>>>
>>>> userprogs := bpf_preload_umd
>>>>
>>>> -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
>>>> +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
>>>> +
>>>> +$(obj)/iterators/iterators.o: $(LIBBPF_A)
>>>>
>>>> bpf_preload_umd-objs := iterators/iterators.o
>>>> bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
>>>
>>> One small issue I ran into by accident while testing:
>>>
>>> [root@linux bpf-next]# make -j8 kernel/bpf/
>>> SYNC include/config/auto.conf.cmd
>>> DESCEND objtool
>>> CALL scripts/atomic/check-atomics.sh
>>> CALL scripts/checksyscalls.sh
>>> CC kernel/bpf/syscall.o
>>> AR kernel/bpf/preload/built-in.a
>>> CC [M] kernel/bpf/preload/bpf_preload_kern.o
>>> CC [U] kernel/bpf/preload/iterators/iterators.o
>>> In file included from ./tools/lib/bpf/libbpf.h:20,
>>> from kernel/bpf/preload/iterators/iterators.c:10:
>>> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
>>> 13 | #include "libbpf_version.h"
>>> | ^~~~~~~~~~~~~~~~~~
>>> compilation terminated.
>>> make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
>>> make[3]: *** Waiting for unfinished jobs....
>>> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
>>> make[2]: *** Waiting for unfinished jobs....
>>> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
>>> make: *** [Makefile:1872: kernel] Error 2
>>>
>>> For me it was the case where tools/lib/bpf/ was already built _before_ this patch
>>> was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
>>> can then be reproduced. I'd assume that as-is, this would affect many folks on update.
>>
>> We had a similar issue even before these changes with resolve_btfids
>> build, because Kbuild doesn't record dependency on libbpf build
>> properly. I'll see how hard it is to record that in a non-intrusive
>> way for both resolve_btfids and preload/iterators, because doing `make
>> resolve_btfids_clean` or, worse, `make clean` isn't great.
>
> Hm... I can't repro this. Both preload/iterators and resolve_btfids
> have $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)
> dependency which will trigger libbpf rebuild if any of libbpf source
> files changes. I added `sleep 1000` and everything under kernel/bpf is
> blocked on this until libbpf build completes. For full kernel build it
> also shouldn't happen because we build resolve_btfids first before
> proceeding to building other files and resolve_btfids should trigger
> libbpf build.
>
> I assume `make clean kernel/bpf` fixed the issue for you in that case?
No, fixing on that node was done via re-building libbpf via tools/lib/bpf/.
> Also, how did you build tools/lib/bpf before running `make -j8 kernel/bpf`?
Basically just i) cd tools/lib/bpf, ii) make clean, iii) make, then applying
the patch, and then running make -j8 kernel/bpf. I just tried on a different
build node with newer user space, and there it indeed triggers a rebuild of
libbpf. On my laptop where I was running into this it didn't. I would suspect
a potential diff in make's behavior (not seen in GNU Make 4.3, seen in GNU
Make 4.2.1, fwiw). Anyway, for this niche case, manual rebuild of tools/lib/bpf
does it ... given it's neither seen in your case nor on newer uspace, I took
it in, lets see how it goes. Thanks!
> Honestly, hard to tell how this happens without being able to repro
> this. So far everything seems to be working for me. I've tried with
> and without the KBUILD_OUTPUT envvar set. I also tried O=<path>.
>
>>>
>>> Thanks,
>>> Daniel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-09 21:43 ` Daniel Borkmann
@ 2021-09-09 22:31 ` Andrii Nakryiko
2021-09-13 14:28 ` sunyucong
0 siblings, 1 reply; 10+ messages in thread
From: Andrii Nakryiko @ 2021-09-09 22:31 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Andrii Nakryiko, bpf, Alexei Starovoitov, Quentin Monnet, Kernel Team
On Thu, Sep 9, 2021 at 2:43 PM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 9/9/21 8:08 PM, Andrii Nakryiko wrote:
> > On Thu, Sep 9, 2021 at 9:37 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >> On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>> On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> >>>> From: Quentin Monnet <quentin@isovalent.com>
> >>>>
> >>>> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> >>>> the deprecation of two API functions. This macro marks functions as deprecated
> >>>> when libbpf's version reaches the values passed as an argument.
> >>>>
> >>>> As part of this change libbpf_version.h header is added with recorded major
> >>>> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> >>>> They are now part of libbpf public API and can be relied upon by user code.
> >>>> libbpf_version.h is installed system-wide along other libbpf public headers.
> >>>>
> >>>> Due to this new build-time auto-generated header, in-kernel applications
> >>>> relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> >>>> include libbpf's output directory as part of a list of include search paths.
> >>>> Better fix would be to use libbpf's make_install target to install public API
> >>>> headers, but that clean up is left out as a future improvement. The build
> >>>> changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> >>>> explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> >>>> problems were detected.
> >>>>
> >>>> Note that because of the constraints of the C preprocessor we have to write
> >>>> a few lines of macro magic for each version used to prepare deprecation (0.6
> >>>> for now).
> >>>>
> >>>> Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> >>>> btf__get_from_id() and btf__load(), which are replaced by
> >>>> btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> >>>> starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> >>>>
> >>>> [0] Closes: https://github.com/libbpf/libbpf/issues/278
> >>>>
> >>>> Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> >>>> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> >>>> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> >>>> ---
> >>>> v2->v3:
> >>>> - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> >>>> and selftest/bpf's bench, which were fixed (BPF CI);
> >>>> v1->v2:
> >>>> - fix bpf_preload build by adding dependency for iterators/iterators.o on
> >>>> libbpf.a generation (caught by BPF CI);
> >>>>
> >>>> kernel/bpf/preload/Makefile | 7 +++++--
> >>>> tools/bpf/bpftool/Makefile | 4 ++++
> >>>> tools/bpf/resolve_btfids/Makefile | 6 ++++--
> >>>> tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> >>>> tools/lib/bpf/btf.h | 2 ++
> >>>> tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> >>>> tools/testing/selftests/bpf/Makefile | 4 ++--
> >>>> 7 files changed, 53 insertions(+), 13 deletions(-)
> >>>>
> >>>> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> >>>> index 1951332dd15f..ac29d4e9a384 100644
> >>>> --- a/kernel/bpf/preload/Makefile
> >>>> +++ b/kernel/bpf/preload/Makefile
> >>>> @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> >>>> $(LIBBPF_A):
> >>>> $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> >>>>
> >>>> -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> >>>> +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> >>>> + -I $(srctree)/tools/include/uapi \
> >>>> -I $(srctree)/tools/lib/ -Wno-unused-result
> >>>>
> >>>> userprogs := bpf_preload_umd
> >>>>
> >>>> -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> >>>> +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> >>>> +
> >>>> +$(obj)/iterators/iterators.o: $(LIBBPF_A)
> >>>>
> >>>> bpf_preload_umd-objs := iterators/iterators.o
> >>>> bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
> >>>
> >>> One small issue I ran into by accident while testing:
> >>>
> >>> [root@linux bpf-next]# make -j8 kernel/bpf/
> >>> SYNC include/config/auto.conf.cmd
> >>> DESCEND objtool
> >>> CALL scripts/atomic/check-atomics.sh
> >>> CALL scripts/checksyscalls.sh
> >>> CC kernel/bpf/syscall.o
> >>> AR kernel/bpf/preload/built-in.a
> >>> CC [M] kernel/bpf/preload/bpf_preload_kern.o
> >>> CC [U] kernel/bpf/preload/iterators/iterators.o
> >>> In file included from ./tools/lib/bpf/libbpf.h:20,
> >>> from kernel/bpf/preload/iterators/iterators.c:10:
> >>> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
> >>> 13 | #include "libbpf_version.h"
> >>> | ^~~~~~~~~~~~~~~~~~
> >>> compilation terminated.
> >>> make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
> >>> make[3]: *** Waiting for unfinished jobs....
> >>> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> >>> make[2]: *** Waiting for unfinished jobs....
> >>> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> >>> make: *** [Makefile:1872: kernel] Error 2
> >>>
> >>> For me it was the case where tools/lib/bpf/ was already built _before_ this patch
> >>> was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
> >>> can then be reproduced. I'd assume that as-is, this would affect many folks on update.
> >>
> >> We had a similar issue even before these changes with resolve_btfids
> >> build, because Kbuild doesn't record dependency on libbpf build
> >> properly. I'll see how hard it is to record that in a non-intrusive
> >> way for both resolve_btfids and preload/iterators, because doing `make
> >> resolve_btfids_clean` or, worse, `make clean` isn't great.
> >
> > Hm... I can't repro this. Both preload/iterators and resolve_btfids
> > have $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)
> > dependency which will trigger libbpf rebuild if any of libbpf source
> > files changes. I added `sleep 1000` and everything under kernel/bpf is
> > blocked on this until libbpf build completes. For full kernel build it
> > also shouldn't happen because we build resolve_btfids first before
> > proceeding to building other files and resolve_btfids should trigger
> > libbpf build.
> >
> > I assume `make clean kernel/bpf` fixed the issue for you in that case?
>
> No, fixing on that node was done via re-building libbpf via tools/lib/bpf/.
>
> > Also, how did you build tools/lib/bpf before running `make -j8 kernel/bpf`?
>
> Basically just i) cd tools/lib/bpf, ii) make clean, iii) make, then applying
> the patch, and then running make -j8 kernel/bpf. I just tried on a different
> build node with newer user space, and there it indeed triggers a rebuild of
> libbpf. On my laptop where I was running into this it didn't. I would suspect
> a potential diff in make's behavior (not seen in GNU Make 4.3, seen in GNU
> Make 4.2.1, fwiw). Anyway, for this niche case, manual rebuild of tools/lib/bpf
> does it ... given it's neither seen in your case nor on newer uspace, I took
> it in, lets see how it goes. Thanks!
Yeah, thanks! Please let me know if you do run into this again
somewhere, though.
>
> > Honestly, hard to tell how this happens without being able to repro
> > this. So far everything seems to be working for me. I've tried with
> > and without the KBUILD_OUTPUT envvar set. I also tried O=<path>.
> >
> >>>
> >>> Thanks,
> >>> Daniel
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-09 22:31 ` Andrii Nakryiko
@ 2021-09-13 14:28 ` sunyucong
2021-09-13 18:13 ` Andrii Nakryiko
0 siblings, 1 reply; 10+ messages in thread
From: sunyucong @ 2021-09-13 14:28 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Daniel Borkmann, Andrii Nakryiko, bpf, Alexei Starovoitov,
Quentin Monnet, Kernel Team
I ran into the same issue
DESCEND objtool
DESCEND bpf/resolve_btfids
CALL scripts/atomic/check-atomics.sh
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
CC kernel/rseq.o
CC [U] kernel/bpf/preload/iterators/iterators.o
In file included from ./tools/lib/bpf/libbpf.h:20,
from kernel/bpf/preload/iterators/iterators.c:10:
./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h:
No such file or directory
13 | #include "libbpf_version.h"
| ^~~~~~~~~~~~~~~~~~
compilation terminated.
make[3]: *** [scripts/Makefile.userprogs:43:
kernel/bpf/preload/iterators/iterators.o] Error 1
make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:1872: kernel] Error 2
make: *** Waiting for unfinished jobs....
And I tried `make clean` in tools/lib/bpf , which didn't solve the
issue, had to do a full clean. To me it seems we should add
"libbpf_version.h" along with LIBBPF_A in kernel/bpf/preload/Makefile
Cheers.
On Thu, Sep 9, 2021 at 6:48 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Sep 9, 2021 at 2:43 PM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >
> > On 9/9/21 8:08 PM, Andrii Nakryiko wrote:
> > > On Thu, Sep 9, 2021 at 9:37 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > >> On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > >>> On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> > >>>> From: Quentin Monnet <quentin@isovalent.com>
> > >>>>
> > >>>> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> > >>>> the deprecation of two API functions. This macro marks functions as deprecated
> > >>>> when libbpf's version reaches the values passed as an argument.
> > >>>>
> > >>>> As part of this change libbpf_version.h header is added with recorded major
> > >>>> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> > >>>> They are now part of libbpf public API and can be relied upon by user code.
> > >>>> libbpf_version.h is installed system-wide along other libbpf public headers.
> > >>>>
> > >>>> Due to this new build-time auto-generated header, in-kernel applications
> > >>>> relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> > >>>> include libbpf's output directory as part of a list of include search paths.
> > >>>> Better fix would be to use libbpf's make_install target to install public API
> > >>>> headers, but that clean up is left out as a future improvement. The build
> > >>>> changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> > >>>> explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> > >>>> problems were detected.
> > >>>>
> > >>>> Note that because of the constraints of the C preprocessor we have to write
> > >>>> a few lines of macro magic for each version used to prepare deprecation (0.6
> > >>>> for now).
> > >>>>
> > >>>> Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> > >>>> btf__get_from_id() and btf__load(), which are replaced by
> > >>>> btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> > >>>> starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> > >>>>
> > >>>> [0] Closes: https://github.com/libbpf/libbpf/issues/278
> > >>>>
> > >>>> Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> > >>>> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> > >>>> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > >>>> ---
> > >>>> v2->v3:
> > >>>> - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> > >>>> and selftest/bpf's bench, which were fixed (BPF CI);
> > >>>> v1->v2:
> > >>>> - fix bpf_preload build by adding dependency for iterators/iterators.o on
> > >>>> libbpf.a generation (caught by BPF CI);
> > >>>>
> > >>>> kernel/bpf/preload/Makefile | 7 +++++--
> > >>>> tools/bpf/bpftool/Makefile | 4 ++++
> > >>>> tools/bpf/resolve_btfids/Makefile | 6 ++++--
> > >>>> tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> > >>>> tools/lib/bpf/btf.h | 2 ++
> > >>>> tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> > >>>> tools/testing/selftests/bpf/Makefile | 4 ++--
> > >>>> 7 files changed, 53 insertions(+), 13 deletions(-)
> > >>>>
> > >>>> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> > >>>> index 1951332dd15f..ac29d4e9a384 100644
> > >>>> --- a/kernel/bpf/preload/Makefile
> > >>>> +++ b/kernel/bpf/preload/Makefile
> > >>>> @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> > >>>> $(LIBBPF_A):
> > >>>> $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> > >>>>
> > >>>> -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> > >>>> +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> > >>>> + -I $(srctree)/tools/include/uapi \
> > >>>> -I $(srctree)/tools/lib/ -Wno-unused-result
> > >>>>
> > >>>> userprogs := bpf_preload_umd
> > >>>>
> > >>>> -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > >>>> +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > >>>> +
> > >>>> +$(obj)/iterators/iterators.o: $(LIBBPF_A)
> > >>>>
> > >>>> bpf_preload_umd-objs := iterators/iterators.o
> > >>>> bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
> > >>>
> > >>> One small issue I ran into by accident while testing:
> > >>>
> > >>> [root@linux bpf-next]# make -j8 kernel/bpf/
> > >>> SYNC include/config/auto.conf.cmd
> > >>> DESCEND objtool
> > >>> CALL scripts/atomic/check-atomics.sh
> > >>> CALL scripts/checksyscalls.sh
> > >>> CC kernel/bpf/syscall.o
> > >>> AR kernel/bpf/preload/built-in.a
> > >>> CC [M] kernel/bpf/preload/bpf_preload_kern.o
> > >>> CC [U] kernel/bpf/preload/iterators/iterators.o
> > >>> In file included from ./tools/lib/bpf/libbpf.h:20,
> > >>> from kernel/bpf/preload/iterators/iterators.c:10:
> > >>> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
> > >>> 13 | #include "libbpf_version.h"
> > >>> | ^~~~~~~~~~~~~~~~~~
> > >>> compilation terminated.
> > >>> make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
> > >>> make[3]: *** Waiting for unfinished jobs....
> > >>> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> > >>> make[2]: *** Waiting for unfinished jobs....
> > >>> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> > >>> make: *** [Makefile:1872: kernel] Error 2
> > >>>
> > >>> For me it was the case where tools/lib/bpf/ was already built _before_ this patch
> > >>> was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
> > >>> can then be reproduced. I'd assume that as-is, this would affect many folks on update.
> > >>
> > >> We had a similar issue even before these changes with resolve_btfids
> > >> build, because Kbuild doesn't record dependency on libbpf build
> > >> properly. I'll see how hard it is to record that in a non-intrusive
> > >> way for both resolve_btfids and preload/iterators, because doing `make
> > >> resolve_btfids_clean` or, worse, `make clean` isn't great.
> > >
> > > Hm... I can't repro this. Both preload/iterators and resolve_btfids
> > > have $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)
> > > dependency which will trigger libbpf rebuild if any of libbpf source
> > > files changes. I added `sleep 1000` and everything under kernel/bpf is
> > > blocked on this until libbpf build completes. For full kernel build it
> > > also shouldn't happen because we build resolve_btfids first before
> > > proceeding to building other files and resolve_btfids should trigger
> > > libbpf build.
> > >
> > > I assume `make clean kernel/bpf` fixed the issue for you in that case?
> >
> > No, fixing on that node was done via re-building libbpf via tools/lib/bpf/.
> >
> > > Also, how did you build tools/lib/bpf before running `make -j8 kernel/bpf`?
> >
> > Basically just i) cd tools/lib/bpf, ii) make clean, iii) make, then applying
> > the patch, and then running make -j8 kernel/bpf. I just tried on a different
> > build node with newer user space, and there it indeed triggers a rebuild of
> > libbpf. On my laptop where I was running into this it didn't. I would suspect
> > a potential diff in make's behavior (not seen in GNU Make 4.3, seen in GNU
> > Make 4.2.1, fwiw). Anyway, for this niche case, manual rebuild of tools/lib/bpf
> > does it ... given it's neither seen in your case nor on newer uspace, I took
> > it in, lets see how it goes. Thanks!
>
> Yeah, thanks! Please let me know if you do run into this again
> somewhere, though.
>
> >
> > > Honestly, hard to tell how this happens without being able to repro
> > > this. So far everything seems to be working for me. I've tried with
> > > and without the KBUILD_OUTPUT envvar set. I also tried O=<path>.
> > >
> > >>>
> > >>> Thanks,
> > >>> Daniel
> >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-13 14:28 ` sunyucong
@ 2021-09-13 18:13 ` Andrii Nakryiko
2021-09-13 19:32 ` Alexei Starovoitov
0 siblings, 1 reply; 10+ messages in thread
From: Andrii Nakryiko @ 2021-09-13 18:13 UTC (permalink / raw)
To: sunyucong
Cc: Daniel Borkmann, Andrii Nakryiko, bpf, Alexei Starovoitov,
Quentin Monnet, Kernel Team
On Mon, Sep 13, 2021 at 7:29 AM sunyucong@gmail.com <sunyucong@gmail.com> wrote:
>
> I ran into the same issue
>
> DESCEND objtool
> DESCEND bpf/resolve_btfids
> CALL scripts/atomic/check-atomics.sh
> CALL scripts/checksyscalls.sh
> CHK include/generated/compile.h
> CC kernel/rseq.o
> CC [U] kernel/bpf/preload/iterators/iterators.o
> In file included from ./tools/lib/bpf/libbpf.h:20,
> from kernel/bpf/preload/iterators/iterators.c:10:
> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h:
> No such file or directory
> 13 | #include "libbpf_version.h"
> | ^~~~~~~~~~~~~~~~~~
> compilation terminated.
> make[3]: *** [scripts/Makefile.userprogs:43:
> kernel/bpf/preload/iterators/iterators.o] Error 1
> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> make[1]: *** Waiting for unfinished jobs....
> make: *** [Makefile:1872: kernel] Error 2
> make: *** Waiting for unfinished jobs....
>
>
> And I tried `make clean` in tools/lib/bpf , which didn't solve the
make clean in tools/lib/bpf won't help, you'd need to clean kernel/bpf/preload
> issue, had to do a full clean. To me it seems we should add
> "libbpf_version.h" along with LIBBPF_A in kernel/bpf/preload/Makefile
LIBBPF_A should force generation of libbpf_version.h, so there is no
need to add libbpf_version.h explicitly. I wonder if maybe adding
FORCE would help, like this:
$(LIBBPF_A): FORCE
...
If you can still repro this, can you please try that? You might need
to also have
.PHONY: FORCE
FORCE:
in that file for this to work. It should force calling into libbpf's
Makefile, even if Kbuild believes LIBBPF_A is up to date.
>
> Cheers.
>
> On Thu, Sep 9, 2021 at 6:48 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Sep 9, 2021 at 2:43 PM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > >
> > > On 9/9/21 8:08 PM, Andrii Nakryiko wrote:
> > > > On Thu, Sep 9, 2021 at 9:37 AM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > >> On Thu, Sep 9, 2021 at 5:58 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > > >>> On 9/8/21 11:32 PM, Andrii Nakryiko wrote:
> > > >>>> From: Quentin Monnet <quentin@isovalent.com>
> > > >>>>
> > > >>>> Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> > > >>>> the deprecation of two API functions. This macro marks functions as deprecated
> > > >>>> when libbpf's version reaches the values passed as an argument.
> > > >>>>
> > > >>>> As part of this change libbpf_version.h header is added with recorded major
> > > >>>> (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> > > >>>> They are now part of libbpf public API and can be relied upon by user code.
> > > >>>> libbpf_version.h is installed system-wide along other libbpf public headers.
> > > >>>>
> > > >>>> Due to this new build-time auto-generated header, in-kernel applications
> > > >>>> relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> > > >>>> include libbpf's output directory as part of a list of include search paths.
> > > >>>> Better fix would be to use libbpf's make_install target to install public API
> > > >>>> headers, but that clean up is left out as a future improvement. The build
> > > >>>> changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> > > >>>> explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> > > >>>> problems were detected.
> > > >>>>
> > > >>>> Note that because of the constraints of the C preprocessor we have to write
> > > >>>> a few lines of macro magic for each version used to prepare deprecation (0.6
> > > >>>> for now).
> > > >>>>
> > > >>>> Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> > > >>>> btf__get_from_id() and btf__load(), which are replaced by
> > > >>>> btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> > > >>>> starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> > > >>>>
> > > >>>> [0] Closes: https://github.com/libbpf/libbpf/issues/278
> > > >>>>
> > > >>>> Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> > > >>>> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> > > >>>> Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > > >>>> ---
> > > >>>> v2->v3:
> > > >>>> - adding `sleep 10` revealed two more missing dependencies in resolve_btfids
> > > >>>> and selftest/bpf's bench, which were fixed (BPF CI);
> > > >>>> v1->v2:
> > > >>>> - fix bpf_preload build by adding dependency for iterators/iterators.o on
> > > >>>> libbpf.a generation (caught by BPF CI);
> > > >>>>
> > > >>>> kernel/bpf/preload/Makefile | 7 +++++--
> > > >>>> tools/bpf/bpftool/Makefile | 4 ++++
> > > >>>> tools/bpf/resolve_btfids/Makefile | 6 ++++--
> > > >>>> tools/lib/bpf/Makefile | 24 +++++++++++++++++-------
> > > >>>> tools/lib/bpf/btf.h | 2 ++
> > > >>>> tools/lib/bpf/libbpf_common.h | 19 +++++++++++++++++++
> > > >>>> tools/testing/selftests/bpf/Makefile | 4 ++--
> > > >>>> 7 files changed, 53 insertions(+), 13 deletions(-)
> > > >>>>
> > > >>>> diff --git a/kernel/bpf/preload/Makefile b/kernel/bpf/preload/Makefile
> > > >>>> index 1951332dd15f..ac29d4e9a384 100644
> > > >>>> --- a/kernel/bpf/preload/Makefile
> > > >>>> +++ b/kernel/bpf/preload/Makefile
> > > >>>> @@ -10,12 +10,15 @@ LIBBPF_OUT = $(abspath $(obj))
> > > >>>> $(LIBBPF_A):
> > > >>>> $(Q)$(MAKE) -C $(LIBBPF_SRCS) O=$(LIBBPF_OUT)/ OUTPUT=$(LIBBPF_OUT)/ $(LIBBPF_OUT)/libbpf.a
> > > >>>>
> > > >>>> -userccflags += -I $(srctree)/tools/include/ -I $(srctree)/tools/include/uapi \
> > > >>>> +userccflags += -I$(LIBBPF_OUT) -I $(srctree)/tools/include/ \
> > > >>>> + -I $(srctree)/tools/include/uapi \
> > > >>>> -I $(srctree)/tools/lib/ -Wno-unused-result
> > > >>>>
> > > >>>> userprogs := bpf_preload_umd
> > > >>>>
> > > >>>> -clean-files := $(userprogs) bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > > >>>> +clean-files := $(userprogs) libbpf_version.h bpf_helper_defs.h FEATURE-DUMP.libbpf staticobjs/ feature/
> > > >>>> +
> > > >>>> +$(obj)/iterators/iterators.o: $(LIBBPF_A)
> > > >>>>
> > > >>>> bpf_preload_umd-objs := iterators/iterators.o
> > > >>>> bpf_preload_umd-userldlibs := $(LIBBPF_A) -lelf -lz
> > > >>>
> > > >>> One small issue I ran into by accident while testing:
> > > >>>
> > > >>> [root@linux bpf-next]# make -j8 kernel/bpf/
> > > >>> SYNC include/config/auto.conf.cmd
> > > >>> DESCEND objtool
> > > >>> CALL scripts/atomic/check-atomics.sh
> > > >>> CALL scripts/checksyscalls.sh
> > > >>> CC kernel/bpf/syscall.o
> > > >>> AR kernel/bpf/preload/built-in.a
> > > >>> CC [M] kernel/bpf/preload/bpf_preload_kern.o
> > > >>> CC [U] kernel/bpf/preload/iterators/iterators.o
> > > >>> In file included from ./tools/lib/bpf/libbpf.h:20,
> > > >>> from kernel/bpf/preload/iterators/iterators.c:10:
> > > >>> ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
> > > >>> 13 | #include "libbpf_version.h"
> > > >>> | ^~~~~~~~~~~~~~~~~~
> > > >>> compilation terminated.
> > > >>> make[3]: *** [scripts/Makefile.userprogs:43: kernel/bpf/preload/iterators/iterators.o] Error 1
> > > >>> make[3]: *** Waiting for unfinished jobs....
> > > >>> make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> > > >>> make[2]: *** Waiting for unfinished jobs....
> > > >>> make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> > > >>> make: *** [Makefile:1872: kernel] Error 2
> > > >>>
> > > >>> For me it was the case where tools/lib/bpf/ was already built _before_ this patch
> > > >>> was applied, then I applied it, and just ran make -j8 kernel/bpf/ where the above
> > > >>> can then be reproduced. I'd assume that as-is, this would affect many folks on update.
> > > >>
> > > >> We had a similar issue even before these changes with resolve_btfids
> > > >> build, because Kbuild doesn't record dependency on libbpf build
> > > >> properly. I'll see how hard it is to record that in a non-intrusive
> > > >> way for both resolve_btfids and preload/iterators, because doing `make
> > > >> resolve_btfids_clean` or, worse, `make clean` isn't great.
> > > >
> > > > Hm... I can't repro this. Both preload/iterators and resolve_btfids
> > > > have $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile)
> > > > dependency which will trigger libbpf rebuild if any of libbpf source
> > > > files changes. I added `sleep 1000` and everything under kernel/bpf is
> > > > blocked on this until libbpf build completes. For full kernel build it
> > > > also shouldn't happen because we build resolve_btfids first before
> > > > proceeding to building other files and resolve_btfids should trigger
> > > > libbpf build.
> > > >
> > > > I assume `make clean kernel/bpf` fixed the issue for you in that case?
> > >
> > > No, fixing on that node was done via re-building libbpf via tools/lib/bpf/.
> > >
> > > > Also, how did you build tools/lib/bpf before running `make -j8 kernel/bpf`?
> > >
> > > Basically just i) cd tools/lib/bpf, ii) make clean, iii) make, then applying
> > > the patch, and then running make -j8 kernel/bpf. I just tried on a different
> > > build node with newer user space, and there it indeed triggers a rebuild of
> > > libbpf. On my laptop where I was running into this it didn't. I would suspect
> > > a potential diff in make's behavior (not seen in GNU Make 4.3, seen in GNU
> > > Make 4.2.1, fwiw). Anyway, for this niche case, manual rebuild of tools/lib/bpf
> > > does it ... given it's neither seen in your case nor on newer uspace, I took
> > > it in, lets see how it goes. Thanks!
> >
> > Yeah, thanks! Please let me know if you do run into this again
> > somewhere, though.
> >
> > >
> > > > Honestly, hard to tell how this happens without being able to repro
> > > > this. So far everything seems to be working for me. I've tried with
> > > > and without the KBUILD_OUTPUT envvar set. I also tried O=<path>.
> > > >
> > > >>>
> > > >>> Thanks,
> > > >>> Daniel
> > >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations
2021-09-13 18:13 ` Andrii Nakryiko
@ 2021-09-13 19:32 ` Alexei Starovoitov
0 siblings, 0 replies; 10+ messages in thread
From: Alexei Starovoitov @ 2021-09-13 19:32 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: sunyucong, Daniel Borkmann, Andrii Nakryiko, bpf,
Alexei Starovoitov, Quentin Monnet, Kernel Team
On Mon, Sep 13, 2021 at 11:13 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Sep 13, 2021 at 7:29 AM sunyucong@gmail.com <sunyucong@gmail.com> wrote:
> >
> > I ran into the same issue
> >
> > DESCEND objtool
> > DESCEND bpf/resolve_btfids
> > CALL scripts/atomic/check-atomics.sh
> > CALL scripts/checksyscalls.sh
> > CHK include/generated/compile.h
> > CC kernel/rseq.o
> > CC [U] kernel/bpf/preload/iterators/iterators.o
> > In file included from ./tools/lib/bpf/libbpf.h:20,
> > from kernel/bpf/preload/iterators/iterators.c:10:
> > ./tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h:
> > No such file or directory
> > 13 | #include "libbpf_version.h"
> > | ^~~~~~~~~~~~~~~~~~
> > compilation terminated.
> > make[3]: *** [scripts/Makefile.userprogs:43:
> > kernel/bpf/preload/iterators/iterators.o] Error 1
> > make[2]: *** [scripts/Makefile.build:540: kernel/bpf/preload] Error 2
> > make[1]: *** [scripts/Makefile.build:540: kernel/bpf] Error 2
> > make[1]: *** Waiting for unfinished jobs....
> > make: *** [Makefile:1872: kernel] Error 2
> > make: *** Waiting for unfinished jobs....
> >
> >
> > And I tried `make clean` in tools/lib/bpf , which didn't solve the
>
> make clean in tools/lib/bpf won't help, you'd need to clean kernel/bpf/preload
>
> > issue, had to do a full clean. To me it seems we should add
> > "libbpf_version.h" along with LIBBPF_A in kernel/bpf/preload/Makefile
>
> LIBBPF_A should force generation of libbpf_version.h, so there is no
> need to add libbpf_version.h explicitly. I wonder if maybe adding
> FORCE would help, like this:
>
> $(LIBBPF_A): FORCE
> ...
>
> If you can still repro this, can you please try that? You might need
> to also have
>
> .PHONY: FORCE
> FORCE:
>
> in that file for this to work. It should force calling into libbpf's
> Makefile, even if Kbuild believes LIBBPF_A is up to date.
I've hit this issue as well and could easily repro.
The (LIBBPF_A): FORCE fixed it for me and I pushed this one liner to bpf-next.
The later hunk wasn't necessary.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-09-13 19:32 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-08 21:32 [PATCH v3 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations Andrii Nakryiko
2021-09-09 12:58 ` Daniel Borkmann
2021-09-09 16:37 ` Andrii Nakryiko
2021-09-09 18:08 ` Andrii Nakryiko
2021-09-09 21:43 ` Daniel Borkmann
2021-09-09 22:31 ` Andrii Nakryiko
2021-09-13 14:28 ` sunyucong
2021-09-13 18:13 ` Andrii Nakryiko
2021-09-13 19:32 ` Alexei Starovoitov
2021-09-09 21:40 ` patchwork-bot+netdevbpf
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.