Selectively delay loading of eBPF program.

Selectively delay loading of eBPF program.

[Dominic]

Hi, I have multiple eBPF programs compiled into a single skeleton file
and I need a way to delay loading of one of the programs.

I am aware of `bpf_program__set_autoload()` API but once an object is
loaded using `bpf_object__load()`, there are no APIs to selectively
load a program (bpf_prog_load() has been deprecated). Calling
bpf_object__load() again fails.

Wondering if there are any options to achieve the above mentioned behavior.

Thanks & Regards,
Dominic

Re: Selectively delay loading of eBPF program.

[Alan Maguire]

On 07/03/2023 09:52, Dominic wrote:
> Hi, I have multiple eBPF programs compiled into a single skeleton file
> and I need a way to delay loading of one of the programs.
> 
> I am aware of `bpf_program__set_autoload()` API but once an object is
> loaded using `bpf_object__load()`, there are no APIs to selectively
> load a program (bpf_prog_load() has been deprecated). Calling
> bpf_object__load() again fails.
> 
> Wondering if there are any options to achieve the above mentioned behavior.
>

I ran into a similar problem recently; in my case, the problem
was that one of the functions that one of my BPF programs attached
to could be inlined on some kernel versions.  As a result the
whole skeleton would fail on auto-attach. If that is the
problem you are facing, you can try a full load/attach and
if that fails, start again - you'll need to destroy the
skeleton and go back to the open if I remember correctly -
and mark the problematic program using bpf_program__set_autoload()
to false the second time round.

Failing that, is separating into two skeletons and using
bpf_map__reuse_fd() to share fds across both skeletons 
feasible?

If not, can you provide more details on why the delayed load
is required - that might help us figure out a solution. One
thing to figure out - is it definitely delayed load you need,
or just delayed attach? Thanks!

Alan
 
> Thanks & Regards,
> Dominic

Re: Selectively delay loading of eBPF program.

[Dominic]

Thanks for the reply Alan.

One of the eBPF program needs to attach to the TC qdisc. If the
associated userspace program gets killed (for whatever reason), the
eBPF program attached to TC will continue to be loaded & attached.
When the userspace program restarts, it will load the required eBPF TC
program only if not already loaded. (But it will load all others that
were attached to lets say tracepoint).

This can be achieved by bpf_program__set_autoload() but that requires
all the logic to decide which program to load and skip at one place. I
was looking at a way for each component to make its own decision and
load only its own set of programs. Another option is to have two
seperate skeleton files.

Thanks & Regards,
Dominic

On Tue, Mar 7, 2023 at 3:46 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 07/03/2023 09:52, Dominic wrote:
> > Hi, I have multiple eBPF programs compiled into a single skeleton file
> > and I need a way to delay loading of one of the programs.
> >
> > I am aware of `bpf_program__set_autoload()` API but once an object is
> > loaded using `bpf_object__load()`, there are no APIs to selectively
> > load a program (bpf_prog_load() has been deprecated). Calling
> > bpf_object__load() again fails.
> >
> > Wondering if there are any options to achieve the above mentioned behavior.
> >
>
> I ran into a similar problem recently; in my case, the problem
> was that one of the functions that one of my BPF programs attached
> to could be inlined on some kernel versions.  As a result the
> whole skeleton would fail on auto-attach. If that is the
> problem you are facing, you can try a full load/attach and
> if that fails, start again - you'll need to destroy the
> skeleton and go back to the open if I remember correctly -
> and mark the problematic program using bpf_program__set_autoload()
> to false the second time round.
>
> Failing that, is separating into two skeletons and using
> bpf_map__reuse_fd() to share fds across both skeletons
> feasible?
>
> If not, can you provide more details on why the delayed load
> is required - that might help us figure out a solution. One
> thing to figure out - is it definitely delayed load you need,
> or just delayed attach? Thanks!
>
> Alan
>
> > Thanks & Regards,
> > Dominic

Re: Selectively delay loading of eBPF program.

[Andrii Nakryiko]

On Tue, Mar 7, 2023 at 3:07 AM Dominic <d.dropify@gmail.com> wrote:
>
> Thanks for the reply Alan.
>
> One of the eBPF program needs to attach to the TC qdisc. If the
> associated userspace program gets killed (for whatever reason), the
> eBPF program attached to TC will continue to be loaded & attached.
> When the userspace program restarts, it will load the required eBPF TC
> program only if not already loaded. (But it will load all others that
> were attached to lets say tracepoint).
>
> This can be achieved by bpf_program__set_autoload() but that requires
> all the logic to decide which program to load and skip at one place. I
> was looking at a way for each component to make its own decision and
> load only its own set of programs. Another option is to have two
> seperate skeleton files.
>

please don't top post on the mailing list

As for your question, libbpf currently doesn't support loading
individual programs after bpf_object (or skeleton for that matter) was
loaded. This might change in the future, but for now this is the
approach we stick to.

I'd try to organize code such that decision about what needs to be
loaded was done before bpf_object__load(), even if it's done across
multiple components (can you poll each of them somehow?)


> Thanks & Regards,
> Dominic
>
> On Tue, Mar 7, 2023 at 3:46 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> >
> > On 07/03/2023 09:52, Dominic wrote:
> > > Hi, I have multiple eBPF programs compiled into a single skeleton file
> > > and I need a way to delay loading of one of the programs.
> > >
> > > I am aware of `bpf_program__set_autoload()` API but once an object is
> > > loaded using `bpf_object__load()`, there are no APIs to selectively
> > > load a program (bpf_prog_load() has been deprecated). Calling
> > > bpf_object__load() again fails.
> > >
> > > Wondering if there are any options to achieve the above mentioned behavior.
> > >
> >
> > I ran into a similar problem recently; in my case, the problem
> > was that one of the functions that one of my BPF programs attached
> > to could be inlined on some kernel versions.  As a result the
> > whole skeleton would fail on auto-attach. If that is the
> > problem you are facing, you can try a full load/attach and
> > if that fails, start again - you'll need to destroy the
> > skeleton and go back to the open if I remember correctly -
> > and mark the problematic program using bpf_program__set_autoload()
> > to false the second time round.
> >
> > Failing that, is separating into two skeletons and using
> > bpf_map__reuse_fd() to share fds across both skeletons
> > feasible?
> >
> > If not, can you provide more details on why the delayed load
> > is required - that might help us figure out a solution. One
> > thing to figure out - is it definitely delayed load you need,
> > or just delayed attach? Thanks!
> >
> > Alan
> >
> > > Thanks & Regards,
> > > Dominic