Re: [PATCHv3 iproute2-next 0/5] iproute2: add libbpf support

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Thu, Nov 5, 2020 at 6:05 AM Jamal Hadi Salim <jhs@mojatatu.com> wrote:
>
> On 2020-11-04 10:19 p.m., David Ahern wrote:
>
> [..]
> >
> > User experience keeps getting brought up, but I also keep reading the
> > stance that BPF users can not expect a consistent experience unless they
> > are constantly chasing latest greatest versions of *ALL* S/W related to
> > BPF. That is not a realistic expectation for users. Distributions exist
> > for a reason. They solve real packaging problems.
> >
> > As libbpf and bpf in general reach a broader audience, the requirements
> > to use, deploy and even tryout BPF features needs to be more user
> > friendly and that starts with maintainers of the BPF code and how they
> > approach extensions and features. Telling libbpf consumers to make
> > libbpf a submodule of their project and update the reference point every
> > time a new release comes out is not user friendly.
> >
> > Similarly, it is not realistic or user friendly to *require* general
> > Linux users to constantly chase latest versions of llvm, clang, dwarves,
> > bcc, bpftool, libbpf, (I am sure I am missing more), and, by extension
> > of what you want here, iproute2 just to upgrade their production kernel
> > to say v5.10, the next LTS, or to see what relevant new ebpf features
> > exists in the new kernel. As a specific example BTF extensions are added
> > in a way that is all or nothing. Meaning, you want to compile kernel
> > version X with CONFIG_DEBUG_INFO_BTF enabled, update your toolchain.
> > Sure, you are using the latest LTS of $distro, and it worked fine with
> > kernel version X-1 last week, but now compile fails completely unless
> > the pahole version is updated. Horrible user experience. Again, just an
> > example and one I brought up in July. I am sure there more.
> >
>
>
> 2cents feedback from a dabbler in ebpf on user experience:
>
> What David described above *has held me back*.
> Over time it seems things have gotten better with libbpf
> (although a few times i find myself copying includes from the
> latest iproute into libbpf). I ended up just doing static links.
> The idea of upgrading clang/llvm every 2 months i revisit ebpf is
> the most painful. At times code that used to compile just fine
> earlier doesnt anymore. There's a minor issue of requiring i install

Do you have a specific example of something that stopped compiling?
I'm not saying that can't happen, but we definitely try hard to avoid
any regressions. I might be forgetting something, but I don't recall
the situation when something would stop compiling just due to newer
libbpf.

> kernel headers every time i want to run something in samples, etc
> but i am probably lacking knowledge on how to ease the pain in that
> regard.
>
> I find the loader and associated tooling in iproute2/tc to be quiet
> stable (not shiny but works everytime).
> And for that reason i often find myself sticking to just tc instead
> of toying with other areas.

That's the part that others on this thread mentioned is bit rotting?
Doesn't seem like everyone is happy about that, though. Stopping any
development definitely makes things stable by definition. BPF and
libbpf try to be stable while not stagnating, which is harder than
just stopping any development, unfortunately.

> Slight tangent:
> One thing that would help libbpf adoption is to include an examples/
> directory. Put a bunch of sample apps for tc, probes, xdp etc.
> And have them compile outside of the kernel. Maybe useful Makefiles
> that people can cutnpaste from. Every time you add a new feature
> put some sample code in the examples.

That's what tools/testing/selftests/bpf in kernel source are for. It's
not the greatest showcase of examples, but all the new features have a
test demonstrating its usage. I do agree about having simple Makefiles
and we do have that at [0]. I'm also about to do another sample repo
with a lot of things pre-setup, for tinkering and using that as a
bootstrap for BPF development with libbpf.

  [0] https://github.com/iovisor/bcc/tree/master/libbpf-tools

>
> cheers,
> jamal