Re: [PATCH bpf-next v2 2/7] bpf: Add fd-based tcx multi-prog infra with link support

From: Jamal Hadi Salim <jhs@mojatatu.com>
To: Daniel Borkmann <daniel@iogearbox.net>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	ast@kernel.org, andrii@kernel.org,  martin.lau@linux.dev,
	razor@blackwall.org, sdf@google.com,  john.fastabend@gmail.com,
	kuba@kernel.org, dxu@dxuuu.xyz, joe@cilium.io,  toke@kernel.org,
	davem@davemloft.net, bpf@vger.kernel.org,
	 netdev@vger.kernel.org, lmb@isovalent.com
Subject: Re: [PATCH bpf-next v2 2/7] bpf: Add fd-based tcx multi-prog infra with link support
Date: Thu, 6 Jul 2023 09:31:42 -0400	[thread overview]
Message-ID: <CAM0EoM=-Kk1K04wYgsiARPfqLx1a2kkq92haU9dZPP7P2mgh8g@mail.gmail.com> (raw)
In-Reply-To: <b147aa2d-6aa5-6336-1484-41c7c1032ecd@iogearbox.net>

On Wed, Jul 5, 2023 at 3:34 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 7/5/23 12:38 AM, Jamal Hadi Salim wrote:
> > On Tue, Jul 4, 2023 at 6:01 PM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >> On 7/4/23 11:36 PM, Jamal Hadi Salim wrote:
> >>> On Thu, Jun 8, 2023 at 5:25 PM Andrii Nakryiko
> >>> <andrii.nakryiko@gmail.com> wrote:
> >>>> On Thu, Jun 8, 2023 at 12:46 PM Jamal Hadi Salim <jhs@mojatatu.com> wrote:
> >>>>> On Thu, Jun 8, 2023 at 6:12 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >>>>>> On 6/8/23 3:25 AM, Jamal Hadi Salim wrote:
> >> [...]
> >>>>>> BPF links are supported for XDP today, just tc BPF is one of the few
> >>>>>> remainders where it is not the case, hence the work of this series. What
> >>>>>> XDP lacks today however is multi-prog support. With the bpf_mprog concept
> >>>>>> that could be addressed with that common/uniform api (and Andrii expressed
> >>>>>> interest in integrating this also for cgroup progs), so yes, various hook
> >>>>>> points/program types could benefit from it.
> >>>>>
> >>>>> Is there some sample XDP related i could look at?  Let me describe our
> >>>>> use case: lets say we load an ebpf program foo attached to XDP of a
> >>>>> netdev  and then something further upstream in the stack is consuming
> >>>>> the results of that ebpf XDP program. For some reason someone, at some
> >>>>> point, decides to replace the XDP prog with a different one - and the
> >>>>> new prog does a very different thing. Could we stop the replacement
> >>>>> with the link mechanism you describe? i.e the program is still loaded
> >>>>> but is no longer attached to the netdev.
> >>>>
> >>>> If you initially attached an XDP program using BPF link api
> >>>> (LINK_CREATE command in bpf() syscall), then subsequent attachment to
> >>>> the same interface (of a new link or program with BPF_PROG_ATTACH)
> >>>> will fail until the current BPF link is detached through closing its
> >>>> last fd.
> >>>
> >>> So this works as advertised. The problem is however not totally solved
> >>> because it seems we need a process that's alive to hold the ownership.
> >>> If we had a daemon then that would solve it i think (we dont).
> >>> Alternatively,  you pin the link. The pinning part can be
> >>> circumvented, unless i misunderstood i,e anybody with the right
> >>> permissions can remove it.
> >>>
> >>> Am I missing something?
> >>
> >> It would be either of those depending on the use case, and for pinning
> >> removal, it would require right permissions/acls. Keep in mind that for
> >> your application you can also use your own bpffs mount, so you don't
> >> need to use the default /sys/fs/bpf one in hostns.
> >
> > This helps for sure - doesnt 100% solve it. It would really be nice if
> > we could tie in a kerberos-like ticketing system for ownership of the
> > mount or something even more fine grained like a link. Doesnt have to
> > be kerberos but anything that would allow a digest of some verifiable
> > credentials/token to be handed to the kernel for authorization...
>
> What is your use-case, you don't want anyone except your own orchestration
> application to access it, so any kind of ACLs, LSM policies or making the
> mount only available to your container are not enough in this scenario you
> have in mind?

It should work - it's not even a shared environment (unlike the
situation you have to deal with). I think i got overly paranoid
because we  have gone through a couple of debug cases where an
installed parser (using ip) in  XDP (with a tc prog consuming the
results) was accidentally replaced (and the tc side had expectations
built on the removed prog). i.e end goal is two or more programs, in
this case, one running in XDP and another at TC are interdependent; if
you touch one you affect the other.
In a shared environment it could be problematic because all you need
is root access to remove things.
If you have a second factor authentication etc then someone has to be
both root and has more secret knowledge to displace things.

But: I do have some ulterior motive where the authentication could be
used at policy level as well. Example someone can read-only a tc rule
whereas someone else can read, update or even delete the rule.

> I think the closest to that is probably the prototype which Lorenz recently
> built where the user space application's digest is validated via IMA [0].

This may be sufficient for the atomicity requirement if we can lock
things into our own ebpf fs. I will take a look - thanks.

cheers,
jamal
>    [0] http://vger.kernel.org/bpfconf2023_material/Lorenz_Bauer_-_BPF_signing_using_fsverity_and_LSM_gatekeeper.pdf
>        https://github.com/isovalent/bpf-verity