Re: [RFC PATCH bpf-next] bpf: Introduce bpf_timer

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Jamal Hadi Salim <jhs@mojatatu.com>
Cc: Cong Wang <xiyou.wangcong@gmail.com>,
	David Miller <davem@davemloft.net>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	John Fastabend <john.fastabend@gmail.com>,
	Lorenz Bauer <lmb@cloudflare.com>,
	Linux Kernel Network Developers <netdev@vger.kernel.org>,
	bpf <bpf@vger.kernel.org>, kernel-team <kernel-team@fb.com>,
	Pedro Tammela <pctammela@gmail.com>
Subject: Re: [RFC PATCH bpf-next] bpf: Introduce bpf_timer
Date: Tue, 25 May 2021 15:08:04 -0700	[thread overview]
Message-ID: <CAADnVQJq37Xi2bHBG5L+DmMq6dJvFUCE3tt+uC-oAKX3WxcCQg@mail.gmail.com> (raw)
In-Reply-To: <27dae780-b66b-4ee9-cff1-a3257e42070e@mojatatu.com>

On Tue, May 25, 2021 at 2:09 PM Jamal Hadi Salim <jhs@mojatatu.com> wrote:
>
> On 2021-05-25 3:57 p.m., Alexei Starovoitov wrote:
> > On Tue, May 25, 2021 at 12:35 PM Jamal Hadi Salim <jhs@mojatatu.com> wrote:
>
> [..]
> > The outcome of the last bpf office hours was a general agreement
> > that we need new hooks in map update/delete operations
> > (including auto-delete by LRU) that will trigger a bpf subprog.
>
> This is certainly a useful feature (for other reasons as well).
> Does this include create/update/delete issued from user space?

Right. Any kind of update/delete and create is a subset of update.
The lookup is not included (yet or may be ever) since it doesn't
have deterministic start/end points.
The prog can do a lookup and update values in place while
holding on the element until prog execution ends.

While update/delete have precise points in hash/lru/lpm maps.
Array is a different story.

> > It might look very similar to the timer callback that is part of this patch,
> > but instead of being called by the timer the LRU logic will call it.
> > This way the subprog can transfer the data stored in the
> > about-to-be-deleted map element into some other map or pass
> > to user space via ringbuf or do any other logic.
> >
>
> The challenge we have in this case is LRU makes the decision
> which entry to victimize. We do have some entries we want to
> keep longer - even if they are not seeing a lot of activity.

Right. That's certainly an argument to make LRU eviction
logic programmable.
John/Joe/Daniel proposed it as a concept long ago.
Design ideas are in demand to make further progress here :)

> You could just notify user space to re-add the entry but then
> you have sync challenges.
> The timers do provide us a way to implement custom GC.

My point is that time is always going to be a heuristic that will
break under certain traffic conditions.
I recommend to focus development effort on creating
building blocks that are truly great instead of reimplementing
old ideas in bpf with all of their shortcomings.

> So a question (which may have already been discussed),
> assuming the following setup:
> - 2 programs a) Ingress b) egress
> - sharing a conntrack map which and said map pinned.
> - a timer prog (with a map with just timers;
>     even a single timer would be enough in some cases).
>
> ingress and egress do std stuff like create/update
> timer prog does the deletes. For simplicity sake assume
> we just have one timer that does a foreach and iterates
> all entries.
>
> What happens when both ingress and egress are ejected?

What is 'ejected'? Like a CD? ;)
I think you mean 'detached' ?
and then, I assume, the user space doesn't hold to prog FD?
The kernel can choose to do different things with the timer here.
One option is to cancel the outstanding timers and unload
.text where the timer callback lives.
Another option is to let the timer stay armed and auto unload
.text of bpf function when it finishes executing.
If timer callback decides to re-arm itself it can continue
executing indefinitely.
This patch is doing the latter.
There could be a combination of both options.
All options have their pros/cons.