From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
Kumar Kartikeya Dwivedi <memxor@gmail.com>,
Daniel Borkmann <daniel@iogearbox.net>, bpf <bpf@vger.kernel.org>,
Jesper Dangaard Brouer <brouer@redhat.com>,
Alexei Starovoitov <ast@kernel.org>,
Andrii Nakryiko <andrii@kernel.org>,
Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
Yonghong Song <yhs@fb.com>,
John Fastabend <john.fastabend@gmail.com>,
KP Singh <kpsingh@kernel.org>, Shuah Khan <shuah@kernel.org>,
"David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>,
Jesper Dangaard Brouer <hawk@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
open list <linux-kernel@vger.kernel.org>,
Networking <netdev@vger.kernel.org>,
"open list:KERNEL SELFTEST FRAMEWORK"
<linux-kselftest@vger.kernel.org>
Subject: Re: [PATCH bpf-next 3/5] libbpf: add low level TC-BPF API
Date: Wed, 14 Apr 2021 12:58:50 +0200 [thread overview]
Message-ID: <874kg9m8t1.fsf@toke.dk> (raw)
In-Reply-To: <CAEf4BzaOJ-WD3A13B2uCrsE2yrctAL8QtJ8TuXHLeP+tm98pbA@mail.gmail.com>
Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
> On Tue, Apr 6, 2021 at 3:06 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>>
>> Andrii Nakryiko <andrii.nakryiko@gmail.com> writes:
>>
>> > On Sat, Apr 3, 2021 at 10:47 AM Alexei Starovoitov
>> > <alexei.starovoitov@gmail.com> wrote:
>> >>
>> >> On Sat, Apr 03, 2021 at 12:38:06AM +0530, Kumar Kartikeya Dwivedi wrote:
>> >> > On Sat, Apr 03, 2021 at 12:02:14AM IST, Alexei Starovoitov wrote:
>> >> > > On Fri, Apr 2, 2021 at 8:27 AM Kumar Kartikeya Dwivedi <memxor@gmail.com> wrote:
>> >> > > > [...]
>> >> > >
>> >> > > All of these things are messy because of tc legacy. bpf tried to follow tc style
>> >> > > with cls and act distinction and it didn't quite work. cls with
>> >> > > direct-action is the only
>> >> > > thing that became mainstream while tc style attach wasn't really addressed.
>> >> > > There were several incidents where tc had tens of thousands of progs attached
>> >> > > because of this attach/query/index weirdness described above.
>> >> > > I think the only way to address this properly is to introduce bpf_link style of
>> >> > > attaching to tc. Such bpf_link would support ingress/egress only.
>> >> > > direction-action will be implied. There won't be any index and query
>> >> > > will be obvious.
>> >> >
>> >> > Note that we already have bpf_link support working (without support for pinning
>> >> > ofcourse) in a limited way. The ifindex, protocol, parent_id, priority, handle,
>> >> > chain_index tuple uniquely identifies a filter, so we stash this in the bpf_link
>> >> > and are able to operate on the exact filter during release.
>> >>
>> >> Except they're not unique. The library can stash them, but something else
>> >> doing detach via iproute2 or their own netlink calls will detach the prog.
>> >> This other app can attach to the same spot a different prog and now
>> >> bpf_link__destroy will be detaching somebody else prog.
>> >>
>> >> > > So I would like to propose to take this patch set a step further from
>> >> > > what Daniel said:
>> >> > > int bpf_tc_attach(prog_fd, ifindex, {INGRESS,EGRESS}):
>> >> > > and make this proposed api to return FD.
>> >> > > To detach from tc ingress/egress just close(fd).
>> >> >
>> >> > You mean adding an fd-based TC API to the kernel?
>> >>
>> >> yes.
>> >
>> > I'm totally for bpf_link-based TC attachment.
>> >
>> > But I think *also* having "legacy" netlink-based APIs will allow
>> > applications to handle older kernels in a much nicer way without extra
>> > dependency on iproute2. We have a similar situation with kprobe, where
>> > currently libbpf only supports "modern" fd-based attachment, but users
>> > periodically ask questions and struggle to figure out issues on older
>> > kernels that don't support new APIs.
>>
>> +1; I am OK with adding a new bpf_link-based way to attach TC programs,
>> but we still need to support the netlink API in libbpf.
>>
>> > So I think we'd have to support legacy TC APIs, but I agree with
>> > Alexei and Daniel that we should keep it to the simplest and most
>> > straightforward API of supporting direction-action attachments and
>> > setting up qdisc transparently (if I'm getting all the terminology
>> > right, after reading Quentin's blog post). That coincidentally should
>> > probably match how bpf_link-based TC API will look like, so all that
>> > can be abstracted behind a single bpf_link__attach_tc() API as well,
>> > right? That's the plan for dealing with kprobe right now, btw. Libbpf
>> > will detect the best available API and transparently fall back (maybe
>> > with some warning for awareness, due to inherent downsides of legacy
>> > APIs: no auto-cleanup being the most prominent one).
>>
>> Yup, SGTM: Expose both in the low-level API (in bpf.c), and make the
>> high-level API auto-detect. That way users can also still use the
>> netlink attach function if they don't want the fd-based auto-close
>> behaviour of bpf_link.
>
> So I thought a bit more about this, and it feels like the right move
> would be to expose only higher-level TC BPF API behind bpf_link. It
> will keep the API complexity and amount of APIs that libbpf will have
> to support to the minimum, and will keep the API itself simple:
> direct-attach with the minimum amount of input arguments. By not
> exposing low-level APIs we also table the whole bpf_tc_cls_attach_id
> design discussion, as we now can keep as much info as needed inside
> bpf_link_tc (which will embed bpf_link internally as well) to support
> detachment and possibly some additional querying, if needed.
But then there would be no way for the caller to explicitly select a
mechanism? I.e., if I write a BPF program using this mechanism targeting
a 5.12 kernel, I'll get netlink attachment, which can stick around when
I do bpf_link__disconnect(). But then if the kernel gets upgraded to
support bpf_link for TC programs I'll suddenly transparently get
bpf_link and the attachments will go away unless I pin them. This
seems... less than ideal?
If we expose the low-level API I can elect to just use this if I know I
want netlink behaviour, but if bpf_program__attach_tc() is the only API
available it would at least need a flag to enforce one mode or the other
(I can see someone wanting to enforce kernel bpf_link semantics as well,
so a flag for either mode seems reasonable?).
-Toke
next prev parent reply other threads:[~2021-04-14 10:59 UTC|newest]
Thread overview: 55+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-25 11:59 [PATCH bpf-next 0/5] libbpf: Add TC-BPF API Kumar Kartikeya Dwivedi
2021-03-25 11:59 ` [PATCH bpf-next 1/5] tools pkt_cls.h: sync with kernel sources Kumar Kartikeya Dwivedi
2021-03-26 23:25 ` Andrii Nakryiko
2021-03-27 3:54 ` Kumar Kartikeya Dwivedi
2021-03-27 3:58 ` Andrii Nakryiko
2021-03-25 12:00 ` [PATCH bpf-next 2/5] libbpf: add helpers for preparing netlink attributes Kumar Kartikeya Dwivedi
2021-03-26 23:52 ` Andrii Nakryiko
2021-03-25 12:00 ` [PATCH bpf-next 3/5] libbpf: add low level TC-BPF API Kumar Kartikeya Dwivedi
2021-03-28 4:42 ` Andrii Nakryiko
2021-03-28 8:11 ` Kumar Kartikeya Dwivedi
2021-03-30 20:39 ` Andrii Nakryiko
2021-03-30 21:11 ` Toke Høiland-Jørgensen
2021-03-31 9:32 ` Kumar Kartikeya Dwivedi
2021-03-30 21:25 ` Daniel Borkmann
2021-03-30 23:30 ` Alexei Starovoitov
2021-03-31 9:44 ` Kumar Kartikeya Dwivedi
2021-04-02 0:19 ` Daniel Borkmann
2021-04-02 15:27 ` Kumar Kartikeya Dwivedi
2021-04-02 18:32 ` Alexei Starovoitov
2021-04-02 19:08 ` Kumar Kartikeya Dwivedi
2021-04-03 17:47 ` Alexei Starovoitov
2021-04-05 17:27 ` Andrii Nakryiko
2021-04-06 10:06 ` Toke Høiland-Jørgensen
2021-04-14 0:47 ` Andrii Nakryiko
2021-04-14 10:58 ` Toke Høiland-Jørgensen [this message]
2021-04-14 22:22 ` Andrii Nakryiko
2021-04-14 22:51 ` Toke Høiland-Jørgensen
2021-04-14 23:19 ` Andrii Nakryiko
2021-04-14 23:32 ` Daniel Borkmann
2021-04-14 23:58 ` Andrii Nakryiko
2021-04-15 22:10 ` Daniel Borkmann
2021-04-15 22:22 ` Andrii Nakryiko
2021-04-15 23:10 ` Daniel Borkmann
2021-04-16 9:01 ` Toke Høiland-Jørgensen
2021-04-15 15:57 ` Toke Høiland-Jørgensen
2021-04-15 21:09 ` Andrii Nakryiko
2021-04-05 17:21 ` Andrii Nakryiko
2021-04-06 19:05 ` Kumar Kartikeya Dwivedi
2021-03-31 9:51 ` Toke Høiland-Jørgensen
2021-03-29 11:46 ` Vlad Buslov
2021-03-29 12:32 ` Toke Høiland-Jørgensen
2021-03-29 12:49 ` Vlad Buslov
2021-03-25 12:00 ` [PATCH bpf-next 4/5] libbpf: add high " Kumar Kartikeya Dwivedi
2021-03-25 12:00 ` [PATCH bpf-next 5/5] libbpf: add selftests for " Kumar Kartikeya Dwivedi
2021-03-27 2:15 ` Alexei Starovoitov
2021-03-27 15:17 ` Toke Høiland-Jørgensen
2021-03-29 1:26 ` Alexei Starovoitov
2021-03-29 1:45 ` Kumar Kartikeya Dwivedi
2021-03-28 4:32 ` Andrii Nakryiko
2021-03-29 1:40 ` Alexei Starovoitov
2021-03-29 2:38 ` Andrii Nakryiko
2021-03-30 3:28 ` Alexei Starovoitov
2021-03-30 20:28 ` Andrii Nakryiko
2021-03-30 23:27 ` Alexei Starovoitov
2021-03-29 9:56 ` Toke Høiland-Jørgensen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=874kg9m8t1.fsf@toke.dk \
--to=toke@redhat.com \
--cc=alexei.starovoitov@gmail.com \
--cc=andrii.nakryiko@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=hawk@kernel.org \
--cc=john.fastabend@gmail.com \
--cc=kafai@fb.com \
--cc=kpsingh@kernel.org \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=memxor@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=shuah@kernel.org \
--cc=songliubraving@fb.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).