bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: "Kumar Kartikeya Dwivedi" <memxor@gmail.com>,
	bpf <bpf@vger.kernel.org>,
	"Jesper Dangaard Brouer" <brouer@redhat.com>,
	"Toke Høiland-Jørgensen" <toke@redhat.com>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Andrii Nakryiko" <andrii@kernel.org>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"KP Singh" <kpsingh@kernel.org>, "Shuah Khan" <shuah@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	"Jakub Kicinski" <kuba@kernel.org>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"Peter Zijlstra" <peterz@infradead.org>,
	"open list" <linux-kernel@vger.kernel.org>,
	Networking <netdev@vger.kernel.org>,
	"open list:KERNEL SELFTEST FRAMEWORK"
	<linux-kselftest@vger.kernel.org>
Subject: Re: [PATCH bpf-next 5/5] libbpf: add selftests for TC-BPF API
Date: Sun, 28 Mar 2021 19:38:42 -0700	[thread overview]
Message-ID: <CAEf4BzZaWjVhfkr7vizir7PfbcsaN99yEwOoqKi32V4X17f0Ng@mail.gmail.com> (raw)
In-Reply-To: <20210329014044.fkmusoeaqs2hjiek@ast-mbp>

On Sun, Mar 28, 2021 at 6:40 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat, Mar 27, 2021 at 09:32:58PM -0700, Andrii Nakryiko wrote:
> > > I think it's better to start with new library for tc/xdp and have
> > > libbpf as a dependency on that new lib.
> > > For example we can add it as subdir in tools/lib/bpf/.
> > >
> > > Similarly I think integerating static linking into libbpf was a mistake.
> > > It should be a sub library as well.
> > >
> > > If we end up with core libbpf and ten sublibs for tc, xdp, af_xdp, linking,
> > > whatever else the users would appreciate that we don't shove single libbpf
> > > to them with a ton of features that they might never use.
> >
> > What's the concern exactly? The size of the library? Having 10
> > micro-libraries has its own set of downsides,
>
> specifically?

You didn't answer my question, but from what you write below I assume
libbpf size is your main concern?

As for downsides, I'm sure I'm not yet seeing all of the problems
we'll encounter when splitting libbpf into 10 pieces. But as a user,
having to figure out which libraries I need to use is a big hassle.
E.g., for XDP application using ringbuf, I'll need libbpfelf,
libbpftrace, libbpfnet, which implicitly also would depend on
libsysbpf, libbtf, libbpfutil, I assume. So having to list 3 vs 1
library is already annoying, but when statically linking I'd need to
specify all 6. I'd very much rather know that it has to be -lbpf at it
will provide me with all the basics (and it's already -lz and -lelf in
static linking scenario, which I wish we could get rid of).

>
> > I'm not convinced that's
> > a better situation for end users. And would certainly cause more
> > hassle for libbpf developers and packagers.
>
> For developers and packagers.. yes.
> For users.. quite the opposite.

See above. I don't know which hassle is libbpf for users today. You
were implying code size used for functionality users might not use
(e.g., linker). Libbpf is a very small library, <300KB. There are
users building tools for constrained embedded systems that use libbpf.
There are/were various problems mentioned, but the size of libbpf
wasn't yet one of them. We should certainly watch the code bloat, but
we are not yet at the point where library is too big for users to be
turned off. In shared library case it's even less of a concern.

> The skel gen and static linking must be split out before the next libbpf release.
> Not a single application linked with libbpf is going to use those pieces.
> bpftool is one and only that needs them. Hence forcing libbpf users
> to increase their .text with a dead code is a selfish call of libbpf
> developers and packagers. The user's priorities must come first.
>
> > And what did you include in "core libbpf"?
>
> I would take this opportunity to split libbpf into maintainable pieces:
> - libsysbpf - sys_bpf wrappers (pretty much tools/lib/bpf/bpf.c)
> - libbpfutil - hash, strset

strset and hash are internal data structures, I never intended to
expose them through public APIs. I haven't investigated, but if we
have a separate shared library (libbpfutil), I imagine we won't be
able to hide those APIs, right?

> - libbtf - BTF read/write
> - libbpfelf - ELF parsing, CORE, ksym, kconfig
> - libbpfskel - skeleton gen used by bpftool only

skeleton generation is already part of bpftool, there is no need to
split anything out

> - libbpflink - linker used by bpftool only
> - libbpfnet - networking attachment via netlink including TC and XDP
> - libbpftrace - perfbuf, ringbuf

ringbuf and perfbuf are both very small code-wise, and are used in
majority of BPF applications anyways

> - libxdp - Toke's xdp chaining
> - libxsk - af_xdp logic
>

Now, if we look at libbpf .o files, we can approximately see what
functionality is using most code:

File                Size Percent

bpf.o              17800    4.88
bpf_prog_linfo.o    2952    0.81
btf_dump.o         20472    5.61
btf.o              58160   15.93
hashmap.o           4056    1.11
libbpf_errno.o      2912    0.80
libbpf.o          190072   52.06
libbpf_probes.o     6696    1.83
linker.o           29408    8.05
netlink.o           5944    1.63
nlattr.o            2744    0.75
ringbuf.o           6128    1.68
str_error.o         1640    0.45
strset.o            3656    1.00
xsk.o              12456    3.41

Total             365096  100.00

so libbpf.o which has mostly bpf_object open/load logic and CO-RE take
more than half already. And it depends on still more stuff in btf,
hashmap, bpf, libbpf_probes, errno. But the final code size is even
smaller, because libbpf.so is just 285128 bytes (not 365096 as implied
by the table above), so even these numbers are pessimistic.

linker.o, which is about 8% of the code right now, but is also
actually taking less than 29KB, because when I remove linker.o and
re-compile, the final libbpf.so goes from 285128 to 267576 = 17552
reduction. Even if it grows 2x, I'd still say it's not a big deal.

One reason to keep BPF linker in libbpf is that it is not only bpftool
that would be using it. Our libbpf Rust bindings
 is implementing its own BPF skeleton generation, and we'd like to use
linker APIs to support static linking when using libbpf-rs without
depending on bpftool. So having it in libbpf and not in bpftool is
good when you consider the wider ecosystem.

But again, let's just reflect for a second that we are talking about
the library that takes less than 300KB total. It would be also
interesting to experiment with LTO and its effect on final binaries
when statically linking against libbpf. I haven't tried yet, though.


> In the future the stack trace symbolization code can come
> into libbpftrace or be a part of its own lib.
> My upcoming loader program and signed prog generation logic
> can be part of libbpfskel.

  reply	other threads:[~2021-03-29  2:40 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-25 11:59 [PATCH bpf-next 0/5] libbpf: Add TC-BPF API Kumar Kartikeya Dwivedi
2021-03-25 11:59 ` [PATCH bpf-next 1/5] tools pkt_cls.h: sync with kernel sources Kumar Kartikeya Dwivedi
2021-03-26 23:25   ` Andrii Nakryiko
2021-03-27  3:54     ` Kumar Kartikeya Dwivedi
2021-03-27  3:58       ` Andrii Nakryiko
2021-03-25 12:00 ` [PATCH bpf-next 2/5] libbpf: add helpers for preparing netlink attributes Kumar Kartikeya Dwivedi
2021-03-26 23:52   ` Andrii Nakryiko
2021-03-25 12:00 ` [PATCH bpf-next 3/5] libbpf: add low level TC-BPF API Kumar Kartikeya Dwivedi
2021-03-28  4:42   ` Andrii Nakryiko
2021-03-28  8:11     ` Kumar Kartikeya Dwivedi
2021-03-30 20:39       ` Andrii Nakryiko
2021-03-30 21:11         ` Toke Høiland-Jørgensen
2021-03-31  9:32           ` Kumar Kartikeya Dwivedi
2021-03-30 21:25         ` Daniel Borkmann
2021-03-30 23:30           ` Alexei Starovoitov
2021-03-31  9:44           ` Kumar Kartikeya Dwivedi
2021-04-02  0:19             ` Daniel Borkmann
2021-04-02 15:27               ` Kumar Kartikeya Dwivedi
2021-04-02 18:32                 ` Alexei Starovoitov
2021-04-02 19:08                   ` Kumar Kartikeya Dwivedi
2021-04-03 17:47                     ` Alexei Starovoitov
2021-04-05 17:27                       ` Andrii Nakryiko
2021-04-06 10:06                         ` Toke Høiland-Jørgensen
2021-04-14  0:47                           ` Andrii Nakryiko
2021-04-14 10:58                             ` Toke Høiland-Jørgensen
2021-04-14 22:22                               ` Andrii Nakryiko
2021-04-14 22:51                                 ` Toke Høiland-Jørgensen
2021-04-14 23:19                                   ` Andrii Nakryiko
2021-04-14 23:32                                     ` Daniel Borkmann
2021-04-14 23:58                                       ` Andrii Nakryiko
2021-04-15 22:10                                         ` Daniel Borkmann
2021-04-15 22:22                                           ` Andrii Nakryiko
2021-04-15 23:10                                             ` Daniel Borkmann
2021-04-16  9:01                                               ` Toke Høiland-Jørgensen
2021-04-15 15:57                                     ` Toke Høiland-Jørgensen
2021-04-15 21:09                                       ` Andrii Nakryiko
2021-04-05 17:21                 ` Andrii Nakryiko
2021-04-06 19:05                   ` Kumar Kartikeya Dwivedi
2021-03-31  9:51           ` Toke Høiland-Jørgensen
2021-03-29 11:46   ` Vlad Buslov
2021-03-29 12:32     ` Toke Høiland-Jørgensen
2021-03-29 12:49       ` Vlad Buslov
2021-03-25 12:00 ` [PATCH bpf-next 4/5] libbpf: add high " Kumar Kartikeya Dwivedi
2021-03-25 12:00 ` [PATCH bpf-next 5/5] libbpf: add selftests for " Kumar Kartikeya Dwivedi
2021-03-27  2:15   ` Alexei Starovoitov
2021-03-27 15:17     ` Toke Høiland-Jørgensen
2021-03-29  1:26       ` Alexei Starovoitov
2021-03-29  1:45         ` Kumar Kartikeya Dwivedi
2021-03-28  4:32     ` Andrii Nakryiko
2021-03-29  1:40       ` Alexei Starovoitov
2021-03-29  2:38         ` Andrii Nakryiko [this message]
2021-03-30  3:28           ` Alexei Starovoitov
2021-03-30 20:28             ` Andrii Nakryiko
2021-03-30 23:27               ` Alexei Starovoitov
2021-03-29  9:56         ` Toke Høiland-Jørgensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAEf4BzZaWjVhfkr7vizir7PfbcsaN99yEwOoqKi32V4X17f0Ng@mail.gmail.com \
    --to=andrii.nakryiko@gmail.com \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=hawk@kernel.org \
    --cc=john.fastabend@gmail.com \
    --cc=kafai@fb.com \
    --cc=kpsingh@kernel.org \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=memxor@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=peterz@infradead.org \
    --cc=shuah@kernel.org \
    --cc=songliubraving@fb.com \
    --cc=toke@redhat.com \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).