bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Daniel Borkmann <daniel@iogearbox.net>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Hangbin Liu <haliu@redhat.com>
Cc: "David Ahern" <dsahern@gmail.com>,
	"Andrii Nakryiko" <andrii.nakryiko@gmail.com>,
	"Stephen Hemminger" <stephen@networkplumber.org>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"David Miller" <davem@davemloft.net>,
	"Jesper Dangaard Brouer" <brouer@redhat.com>,
	Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
	"Jiri Benc" <jbenc@redhat.com>,
	"Andrii Nakryiko" <andrii@kernel.org>,
	"Toke Høiland-Jørgensen" <toke@redhat.com>
Subject: Re: [PATCHv3 iproute2-next 0/5] iproute2: add libbpf support
Date: Wed, 4 Nov 2020 11:21:51 +0100	[thread overview]
Message-ID: <2e8ba0be-51bf-9060-e1f7-2148fbaf0f1d@iogearbox.net> (raw)
In-Reply-To: <20201104031145.nmtggnzomfee4fma@ast-mbp.dhcp.thefacebook.com>

On 11/4/20 4:11 AM, Alexei Starovoitov wrote:
> On Wed, Nov 04, 2020 at 10:17:30AM +0800, Hangbin Liu wrote:
>> On Tue, Nov 03, 2020 at 02:55:54PM -0800, Alexei Starovoitov wrote:
>>>> The scope of bpf in iproute2 is tiny - a few tc modules (and VRF but it
>>>> does not need libbpf) which is a small subset of the functionality and
>>>> commands within the package.
>>>
>>> When Hangbin sent this patch set I got excited that finally tc command
>>> will start working with the latest bpf elf files.
>>> Currently "tc" supports 4 year old files which caused plenty of pain to bpf users.
>>> I got excited, but now I've realized that this patch set will make it worse.
>>> The bpf support in "tc" command instead of being obviously old and obsolete
>>> will be sort-of working with unpredictable delay between released kernel
>>> and released iproute2 version. The iproute2 release that suppose to match kernel
>>> release will be meaningless.
>>> More so, the upgrade of shared libbpf.so can make older iproute2/tc to do
>>> something new and unpredictable.
>>> The user experience will be awful. Not only the users won't know
>>> what to expect out of 'tc' command they won't have a way to debug it.
>>> All of it because iproute2 build will take system libbpf and link it
>>> as shared library by default.
>>> So I think iproute2 must not use libbpf. If I could remove bpf support
>>> from iproute2 I would do so as well.
>>> The current state of iproute2 is hurting bpf ecosystem and proposed
>>> libbpf+iproute2 integration will make it worse.
>>
>> Please take it easy. IMHO, it always very hard to make a perfect solution.
>> From development side, it's easier and could get latest features by using
>> libbpf as submodule. But we need to take care of users, backward
>> compatibility, distros policy etc.
>>
>> I like using iproute2 to load bpf objs. But it's not standardized and too old
>> to load the new BTF defined objs. I think all of us like to improve it by
>> using libbpf. But users and distros are slowly. Some user are still using
>> `ifconfig`. Distros have policies to link the shared .so, etc. We have to
>> compromise on something.
>>
>> Our purpose is to push the user to use new features. As this patchset
>> does, push users to try libbpf instead of legacy code. But this need time.
> 
> My problem with iproute2 picking random libbpf is unpredictability.
> Such roll of dice gives no confidence to users on what is expected to work.
> bpf_hello_world.o will load, but that's it.
> What is going to work with this or that version of "tc" command? No one knows.
> The user will do 'tc -V'. Does version mean anything from bpf loading pov?
> It's not. The user will do "ldd `which tc`" and then what?
> Such bpf support in "tc" is worse than the current one.
> At least the current one is predictably old.

User experience will be crappy and predictability worse, agree on that. For libbpf
it's the same as with rest of iproute2 code in that features are developed along
with the kernel. Distros so far are more or less used to upgrade iproute2 along
with new kernel releases though it's not the first time that some major ones have
been shipping old iproute2 for several releases until we pinged them to finally
get their act together to upgrade. With libbpf dynamically linked it's one more
moving target and it's not clear whether distros will upgrade with same cadence
as iproute2 (or even add libbpf as dependency to their packaging). Only option
users might have if they were to rely on iproute2 and to have predictability is
to ship their stuff via container with current libbpf approach which is probably
not the goal of this set.

> There are alternatives though.
> Forking the whole iproute2 because of "tc" is pointless, of course.
> My 'proposal' was a fire starter because people are too stubborn to
> realize that their long term believes could be incorrect until the fire is burning.
> "bpftool prog load" can load any kind of elf. It cannot operate on qdiscs
> and shouldn't do qdisc manipulations, but may be we can combine them into pipe
> of some sort. Like "bpftool prog load file.o | tc filter ... bpf pipe"
> I think that would be better long term. It will be predictable.

We've been thinking about 'bpftool prog load' as well given we build it right of
the kernel tree and bpftool + libbpf are predictable since they are both built out
of the same git tree and with latest features. I don't think it needs to pipe, it
would be enough to just specify where the loaded progs should be pinned in bpf fs
and tc/ip(xdp) already has the option to pick fd of the entry point from pinned
file. But should be doable as well to just pass fd via pipe to avoid later potential
cleanup. Either way I think it makes sense to do 'bpftool prog load' regardless
since it's generic and useful also for other (non-networking) prog types that can
be attached elsewhere in the system.

> When we release new version of libbpf it goes through rigorous testing.
> bpftool gets a lot of test coverage as well.
> iproute2 with shared libbpf will get nothing. It's the same random roll of dice.
> New libbpf may or may not break iproute2. That's awful user experience.
> So iproute2 has to use git submodule with particular libbpf sha.

Alternatively, you have an uapi sync script already in order to not rely on the
distro system headers installed by the distro and to copy latest uapi ones from
kernel tree. Could as well be extended to have similar fixed built-in situation as
with the current lib/bpf.c in iproute2. Back in the days when developing lib/bpf.c,
it was explicitly done as built-in for iproute2 so that it doesn't take years for
users to actually get to the point where they can realistically make use of it. If
we were to extend the internal lib/bpf.c to similar feature state as libbpf today,
how is that different in the bigger picture compared to sync or submodule... so far
noone complained about lib/bpf.c.

> Then libbpf release process can incorporate proper testing of libbpf
> and iproute2 combination.
> Or iproute2 should stay as-is with obsolete bpf support.
> 
> Few years from now the situation could be different and shared libbpf would
> be the most appropriate choice. But that day is not today.

Yep, for libbpf to be in same situation as libelf or libmnl basically feature
development would have to pretty much come to a stop so that even minor or exotic
distros get to a point where they ship same libbpf version as major distros where
then users can start to rely on the base feature set for developing programs
against it.

Thanks,
Daniel

  parent reply	other threads:[~2020-11-04 10:22 UTC|newest]

Thread overview: 167+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-23  3:38 [PATCH iproute2-next 0/5] iproute2: add libbpf support Hangbin Liu
2020-10-23  3:38 ` [PATCH iproute2-next 1/5] configure: add check_libbpf() for later " Hangbin Liu
2020-10-23  3:38 ` [PATCH iproute2-next 2/5] lib: rename bpf.c to bpf_legacy.c Hangbin Liu
2020-10-23  3:38 ` [PATCH iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-10-23 14:34   ` David Ahern
2020-10-25 15:13     ` Toke Høiland-Jørgensen
2020-10-25 22:12       ` David Ahern
2020-10-26  8:56         ` Hangbin Liu
2020-10-26 15:15           ` David Ahern
2020-10-27  2:58             ` Hangbin Liu
2020-10-24  0:21   ` Andrii Nakryiko
2020-10-25 15:11     ` Toke Høiland-Jørgensen
2020-10-26  8:10     ` Hangbin Liu
2020-10-23  3:38 ` [PATCH iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-10-23  3:38 ` [PATCH iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-10-28 13:25 ` [PATCHv2 iproute2-next 0/5] iproute2: add libbpf support Hangbin Liu
2020-10-28 13:25   ` [PATCHv2 iproute2-next 1/5] configure: add check_libbpf() for later " Hangbin Liu
2020-10-28 13:25   ` [PATCHv2 iproute2-next 2/5] lib: rename bpf.c to bpf_legacy.c Hangbin Liu
2020-10-28 13:25   ` [PATCHv2 iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-10-28 13:25   ` [PATCHv2 iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-10-28 13:25   ` [PATCHv2 iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-10-28 21:17   ` [PATCHv2 iproute2-next 0/5] iproute2: add libbpf support Alexei Starovoitov
2020-10-28 23:02   ` David Ahern
2020-10-29  2:06     ` Hangbin Liu
2020-10-29  2:20       ` David Ahern
2020-10-29  2:45         ` Hangbin Liu
2020-10-29  3:00           ` David Ahern
2020-10-29  3:17             ` Hangbin Liu
2020-10-29 10:26             ` Hangbin Liu
2020-10-29 10:51               ` Toke Høiland-Jørgensen
2020-10-29  2:27       ` Andrii Nakryiko
2020-10-29  2:33         ` David Ahern
2020-10-29  2:46           ` Andrii Nakryiko
2020-10-29  2:34         ` Stephen Hemminger
2020-10-29  2:50           ` Andrii Nakryiko
2020-10-29 11:38             ` Jesper Dangaard Brouer
2020-10-29 20:30               ` Andrii Nakryiko
2020-10-29  2:33       ` Stephen Hemminger
2020-10-29 15:11   ` [PATCHv3 " Hangbin Liu
2020-10-29 15:11     ` [PATCHv3 iproute2-next 1/5] configure: add check_libbpf() for later " Hangbin Liu
2020-10-29 15:26       ` Toke Høiland-Jørgensen
2020-11-02 15:37       ` David Ahern
2020-11-03  5:54         ` Hangbin Liu
2020-11-03 17:32           ` David Ahern
2020-11-04  8:51             ` Hangbin Liu
2020-11-04 11:09               ` Toke Høiland-Jørgensen
2020-11-04 11:40                 ` Hangbin Liu
2020-10-29 15:11     ` [PATCHv3 iproute2-next 2/5] lib: rename bpf.c to bpf_legacy.c Hangbin Liu
2020-10-29 15:11     ` [PATCHv3 iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-11-02 15:41       ` David Ahern
2020-11-03  5:48         ` Hangbin Liu
2020-11-03 17:19           ` David Ahern
2020-11-04  8:22         ` Hangbin Liu
2020-11-05  2:33           ` David Ahern
2020-11-05  7:51             ` Hangbin Liu
2020-11-05 15:25               ` David Ahern
2020-11-05 15:57                 ` Toke Høiland-Jørgensen
2020-11-05 16:02                   ` David Ahern
2020-11-06  0:56                     ` Hangbin Liu
2020-11-06  0:41                 ` Hangbin Liu
2020-10-29 15:11     ` [PATCHv3 iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-10-29 15:11     ` [PATCHv3 iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-11-02 15:47     ` [PATCHv3 iproute2-next 0/5] iproute2: add libbpf support David Ahern
2020-11-03  6:58       ` Andrii Nakryiko
2020-11-03  8:42         ` Jiri Benc
2020-11-03 17:45           ` David Ahern
2020-11-03 17:48           ` Alexei Starovoitov
2020-11-03  8:46         ` Daniel Borkmann
2020-11-03 17:35           ` David Ahern
2020-11-03 17:47             ` Alexei Starovoitov
2020-11-03 18:23               ` Stephen Hemminger
2020-11-03 22:32               ` David Ahern
2020-11-03 22:55                 ` Alexei Starovoitov
2020-11-04  1:40                   ` David Ahern
2020-11-04  2:45                     ` Alexei Starovoitov
2020-11-04  9:28                       ` Jiri Benc
2020-11-05  2:39                         ` David Ahern
2020-11-04  2:17                   ` Hangbin Liu
2020-11-04  3:11                     ` Alexei Starovoitov
2020-11-04 10:01                       ` Jiri Benc
2020-11-04 10:21                       ` Daniel Borkmann [this message]
2020-11-04 11:20                         ` Toke Høiland-Jørgensen
2020-11-04 13:12                           ` Daniel Borkmann
2020-11-04 19:17                             ` Jakub Kicinski
2020-11-04 20:43                               ` Andrii Nakryiko
2020-11-04 22:24                                 ` Toke Høiland-Jørgensen
2020-11-05 20:14                                   ` Andrii Nakryiko
2020-11-05  3:48                                 ` David Ahern
2020-11-05 20:53                                   ` Andrii Nakryiko
2020-11-05  3:19                         ` David Ahern
2020-11-05 14:05                           ` Jamal Hadi Salim
2020-11-05 21:01                             ` Andrii Nakryiko
2020-11-06 15:27                               ` Jamal Hadi Salim
2020-11-06 21:25                                 ` Andrii Nakryiko
2020-11-10 12:47                             ` Edward Cree
2020-11-11  0:53                               ` Alexei Starovoitov
2020-11-11 11:31                                 ` Edward Cree
2020-11-11 18:08                                   ` Alexei Starovoitov
2020-11-05 20:45                           ` Andrii Nakryiko
2020-11-06  9:00                             ` Jiri Benc
2020-11-06 21:07                               ` Andrii Nakryiko
2020-11-04 21:15                       ` Edward Cree
2020-11-04 22:10                         ` Alexei Starovoitov
2020-11-04 22:35                           ` Toke Høiland-Jørgensen
2020-11-04 23:05                           ` Edward Cree
2020-11-05 20:19                             ` Andrii Nakryiko
2020-11-06  8:44                               ` Jiri Benc
2020-11-06 20:57                                 ` Andrii Nakryiko
2020-11-06 21:04                                   ` Alexei Starovoitov
2020-11-06 23:25                                     ` Stephen Hemminger
2020-11-06 23:30                                       ` Andrii Nakryiko
2020-11-07  0:41                                         ` Stephen Hemminger
2020-11-07  1:07                                           ` Andrii Nakryiko
2020-11-06 23:38                                       ` David Ahern
2020-11-09  1:45                                         ` Alexei Starovoitov
2020-11-10  4:09                                           ` David Ahern
2020-11-11  0:47                                             ` Alexei Starovoitov
2020-11-11 11:02                                               ` Toke Høiland-Jørgensen
2020-11-11 15:06                                                 ` Daniel Borkmann
2020-11-11 16:33                                                   ` David Ahern
2020-11-12 22:36                                                   ` Toke Høiland-Jørgensen
2020-11-12 23:20                                                     ` Daniel Borkmann
2020-11-13  0:04                                                       ` Stephen Hemminger
2020-11-13  0:40                                                         ` Alexei Starovoitov
2020-11-13  3:55                                                       ` David Ahern
2020-11-09  7:07     ` [PATCHv4 " Hangbin Liu
2020-11-09  7:07       ` [PATCHv4 iproute2-next 1/5] configure: add check_libbpf() for later " Hangbin Liu
2020-11-14  3:26         ` David Ahern
2020-11-16  4:30           ` Hangbin Liu
2020-11-16  4:33             ` David Ahern
2020-11-09  7:07       ` [PATCHv4 iproute2-next 2/5] lib: rename bpf.c to bpf_legacy.c Hangbin Liu
2020-11-14  3:24         ` David Ahern
2020-11-16  3:55           ` Hangbin Liu
2020-11-09  7:08       ` [PATCHv4 iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-11-09  7:08       ` [PATCHv4 iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-11-09  7:08       ` [PATCHv4 iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-11-16  6:53       ` [PATCHv5 iproute2-next 0/5] iproute2: add libbpf support Hangbin Liu
2020-11-16  6:53         ` [PATCHv5 iproute2-next 1/5] configure: add check_libbpf() for later " Hangbin Liu
2020-11-16  6:53         ` [PATCHv5 iproute2-next 2/5] lib: rename bpf.c to bpf_legacy.c Hangbin Liu
2020-11-16  6:53         ` [PATCHv5 iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-11-16  6:53         ` [PATCHv5 iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-11-16  6:53         ` [PATCHv5 iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-11-16  7:19         ` [PATCHv5 iproute2-next 0/5] iproute2: add libbpf support Alexei Starovoitov
2020-11-16 14:54           ` Jesper Dangaard Brouer
2020-11-16 23:29             ` Toke Høiland-Jørgensen
2020-11-17  2:37             ` Alexei Starovoitov
2020-11-17  3:19               ` Hangbin Liu
2020-11-17 18:27                 ` Alexei Starovoitov
2020-11-17 11:56               ` Edward Cree
2020-11-17  3:38             ` David Ahern
2020-11-17 18:19               ` Alexei Starovoitov
2020-11-16 16:45           ` Stephen Hemminger
2020-11-23 13:11         ` [PATCHv6 " Hangbin Liu
2020-11-23 13:11           ` [PATCHv6 iproute2-next 1/5] iproute2: add check_libbpf() and get_libbpf_version() Hangbin Liu
2020-11-23 13:11           ` [PATCHv6 iproute2-next 2/5] lib: make ipvrf able to use libbpf and fix function name conflicts Hangbin Liu
2020-11-23 13:11           ` [PATCHv6 iproute2-next 3/5] lib: add libbpf support Hangbin Liu
2020-11-23 13:12           ` [PATCHv6 iproute2-next 4/5] examples/bpf: move struct bpf_elf_map defined maps to legacy folder Hangbin Liu
2020-11-23 13:12           ` [PATCHv6 iproute2-next 5/5] examples/bpf: add bpf examples with BTF defined maps Hangbin Liu
2020-11-25  5:28           ` [PATCHv6 iproute2-next 0/5] iproute2: add libbpf support David Ahern
2020-11-25  5:30           ` patchwork-bot+netdevbpf
2020-11-29  6:16 ` [PATCH " Stephen Hemminger
2020-11-29  6:22   ` Greg KH
2020-11-30 11:39     ` Michal Kubecek
2020-11-29 17:33   ` Alexei Starovoitov
2020-11-29 19:41   ` David Ahern
2020-11-30 11:04     ` Toke Høiland-Jørgensen
2020-12-01 14:22     ` Jesper Dangaard Brouer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2e8ba0be-51bf-9060-e1f7-2148fbaf0f1d@iogearbox.net \
    --to=daniel@iogearbox.net \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrii.nakryiko@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=davem@davemloft.net \
    --cc=dsahern@gmail.com \
    --cc=haliu@redhat.com \
    --cc=jbenc@redhat.com \
    --cc=kafai@fb.com \
    --cc=netdev@vger.kernel.org \
    --cc=songliubraving@fb.com \
    --cc=stephen@networkplumber.org \
    --cc=toke@redhat.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).