Re: [PATCH v2 bpf-next 03/17] bpf: Introduce BPF trampoline

From: Song Liu <liu.song.a23@gmail.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Song Liu <songliubraving@fb.com>,
	Alexei Starovoitov <ast@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"daniel@iogearbox.net" <daniel@iogearbox.net>,
	"x86@kernel.org" <x86@kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"bpf@vger.kernel.org" <bpf@vger.kernel.org>,
	Kernel Team <Kernel-team@fb.com>
Subject: Re: [PATCH v2 bpf-next 03/17] bpf: Introduce BPF trampoline
Date: Thu, 7 Nov 2019 17:10:18 -0800	[thread overview]
Message-ID: <CAPhsuW4gYU=HJTe2ueDXhiyY__V1ZBF1ZEhCasHb5m8XgkTtww@mail.gmail.com> (raw)
In-Reply-To: <20191108000941.r4umt2624o3j45p7@ast-mbp.dhcp.thefacebook.com>

On Thu, Nov 7, 2019 at 4:10 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Thu, Nov 07, 2019 at 11:16:20PM +0000, Song Liu wrote:
> >
> >
> > > On Nov 7, 2019, at 3:09 PM, Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Thu, Nov 07, 2019 at 11:07:21PM +0000, Song Liu wrote:
> > >>
> > >>
> > >>>
> > >>>
> > >>>>> +
> > >>>>> +static int bpf_trampoline_update(struct bpf_prog *prog)
> > >>>>
> > >>>> Seems argument "prog" is not used at all?
> > >>>
> > >>> like one below ? ;)
> > >> e... I was really dumb... sorry..
> > >>
> > >> Maybe we should just pass the tr in?
> > >
> > > that would be imbalanced.
> >
> > Hmm.. what do you mean by imbalanced?
>
> I take it back. Yeah. It can be tr.
>
> >
> > >
> > >>>
> > >>>>> +{
> > >>>>> +       struct bpf_trampoline *tr = prog->aux->trampoline;
> > >>>>> +       void *old_image = tr->image + ((tr->selector + 1) & 1) * PAGE_SIZE/2;
> > >>>>> +       void *new_image = tr->image + (tr->selector & 1) * PAGE_SIZE/2;
> > >>>>> +       if (err)
> > >>>>> +               goto out;
> > >>>>> +       tr->selector++;
> > >>>>
> > >>>> Shall we do selector-- for unlink?
> > >>>
> > >>> It's a bit flip. I think it would be more confusing with --
> > >>
> > >> Right.. Maybe should use int instead of u64 for selector?
> > >
> > > No, since int can overflow.
> >
> > I guess it is OK to overflow, no?
>
> overflow is not ok, since transition 0->1 should use nop->call patching
> whereas 1->2, 2->3 should use call->call.
>
> In my initial implementation (one I didn't share with anyone) I had
> trampoline_mutex taken inside bpf_trampoline_update(). And multiple link()
> operation were allowed. The idea was to attach multiple progs and update
> trampoline once. But then I realized that I cannot do that since 'unlink +
> update' where only 'update' is taking lock will not guarantee success. Since
> other 'link' operations can race and 'update' can potentially fail in
> arch_prepare_bpf_trampoline() due to new things that 'link' brought in. In that
> version (since there several fentry/fexit progs can come in at once) I used
> separate 'selector' ticker to pick the side of the page. Once I realized the
> issue (to guarantee that unlink+update == always success) I moved mutex all the
> way to unlink and link and left 'selector' as-is. Just now I realized that
> 'selector' can be removed.  fentry_cnt + fexit_cnt can be used instead. This
> sum of counters will change 1 bit at a time. Am I right?

Yeah, I think fentry_cnt + fexit_cnt is cleaner.

Thanks,
Song