Re: [PATCH v2 bpf-next] libbpf: add LIBBPF_DEPRECATED_SINCE macro for scheduling API deprecations

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Wed, Sep 8, 2021 at 6:01 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 9/8/21 8:55 AM, Andrii Nakryiko wrote:
> > From: Quentin Monnet <quentin@isovalent.com>
> >
> > Introduce a macro LIBBPF_DEPRECATED_SINCE(major, minor, message) to prepare
> > the deprecation of two API functions. This macro marks functions as deprecated
> > when libbpf's version reaches the values passed as an argument.
> >
> > As part of this change libbpf_version.h header is added with recorded major
> > (LIBBPF_MAJOR_VERSION) and minor (LIBBPF_MINOR_VERSION) libbpf version macros.
> > They are now part of libbpf public API and can be relied upon by user code.
> > libbpf_version.h is installed system-wide along other libbpf public headers.
> >
> > Due to this new build-time auto-generated header, in-kernel applications
> > relying on libbpf (resolve_btfids, bpftool, bpf_preload) are updated to
> > include libbpf's output directory as part of a list of include search paths.
> > Better fix would be to use libbpf's make_install target to install public API
> > headers, but that clean up is left out as a future improvement. The build
> > changes were tested by building kernel (with KBUILD_OUTPUT and O= specified
> > explicitly), bpftool, libbpf, selftests/bpf, and resolve_btfids builds. No
> > problems were detected.
> >
> > Note that because of the constraints of the C preprocessor we have to write
> > a few lines of macro magic for each version used to prepare deprecation (0.6
> > for now).
> >
> > Also, use LIBBPF_DEPRECATED_SINCE() to schedule deprecation of
> > btf__get_from_id() and btf__load(), which are replaced by
> > btf__load_from_kernel_by_id() and btf__load_into_kernel(), respectively,
> > starting from future libbpf v0.6. This is part of libbpf 1.0 effort ([0]).
> >
> >    [0] Closes: https://github.com/libbpf/libbpf/issues/278
> >
> > Co-developed-by: Quentin Monnet <quentin@isovalent.com>
> > Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> > Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
> > ---
> >
> > v1->v2:
> >    - fix bpf_preload build by adding dependency for iterators/iterators.o on
> >      libbpf.a generation (caught by BPF CI);
> [...]
> > @@ -136,7 +140,7 @@ all: fixdep
> >
> >   all_cmd: $(CMD_TARGETS) check
> >
> > -$(BPF_IN_SHARED): force $(BPF_HELPER_DEFS)
> > +$(BPF_IN_SHARED): force $(BPF_GENERATED)
> >       @(test -f ../../include/uapi/linux/bpf.h -a -f ../../../include/uapi/linux/bpf.h && ( \
> >       (diff -B ../../include/uapi/linux/bpf.h ../../../include/uapi/linux/bpf.h >/dev/null) || \
> >       echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/bpf.h' differs from latest version at 'include/uapi/linux/bpf.h'" >&2 )) || true
> > @@ -154,13 +158,19 @@ $(BPF_IN_SHARED): force $(BPF_HELPER_DEFS)
> >       echo "Warning: Kernel ABI header at 'tools/include/uapi/linux/if_xdp.h' differs from latest version at 'include/uapi/linux/if_xdp.h'" >&2 )) || true
> >       $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(SHARED_OBJDIR) CFLAGS="$(CFLAGS) $(SHLIB_FLAGS)"
> >
> > -$(BPF_IN_STATIC): force $(BPF_HELPER_DEFS)
> > +$(BPF_IN_STATIC): force $(BPF_GENERATED)
> >       $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR)
> >
> >   $(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h
> >       $(QUIET_GEN)$(srctree)/scripts/bpf_doc.py --header \
> >               --file $(srctree)/tools/include/uapi/linux/bpf.h > $(BPF_HELPER_DEFS)
> >
> > +$(VERSION_HDR): force
> > +     $(QUIET_GEN)echo "/* This file was auto-generated. */" > $@
> > +     @echo "" >> $@
> > +     @echo "#define LIBBPF_MAJOR_VERSION $(LIBBPF_MAJOR_VERSION)" >> $@
> > +     @echo "#define LIBBPF_MINOR_VERSION $(LIBBPF_MINOR_VERSION)" >> $@
> > +
>
> Looks like CI caught a different issue this time with v2 where it cannot find libbpf_version.h:

Yep, I'll keep fixing this. Couldn't repro this locally. But I'm also
going to add a long sleep in libbpf_version.h generation to force this
kind of issue more easily. Sorry for the noise.

>
>    [...]
>      CC       bench_count.o
>      GEN     /home/runner/work/bpf/bpf/tools/testing/selftests/bpf/bpf-helpers.rst
>    In file included from /home/runner/work/bpf/bpf/tools/lib/bpf/bpf.h:31,
>                     from /home/runner/work/bpf/bpf/tools/testing/selftests/bpf/bench.h:8,
>                     from benchs/bench_count.c:3:
>    /home/runner/work/bpf/bpf/tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
>       13 | #include "libbpf_version.h"
>          |          ^~~~~~~~~~~~~~~~~~
>    compilation terminated.
>    make: *** [Makefile:517: /home/runner/work/bpf/bpf/tools/testing/selftests/bpf/bench_count.o] Error 1
>    make: *** Waiting for unfinished jobs....
>      GEN     /home/runner/work/bpf/bpf/tools/testing/selftests/bpf/bpf-syscall.rst
>    In file included from /home/runner/work/bpf/bpf/tools/lib/bpf/bpf.h:31,
>                     from bench.h:8,
>                     from bench.c:13:
>    /home/runner/work/bpf/bpf/tools/lib/bpf/libbpf_common.h:13:10: fatal error: libbpf_version.h: No such file or directory
>       13 | #include "libbpf_version.h"
>          |          ^~~~~~~~~~~~~~~~~~
>    compilation terminated.
>    make: *** [Makefile:162: /home/runner/work/bpf/bpf/tools/testing/selftests/bpf/bench.o] Error 1
>
> Thanks,
> Daniel