libxsk move from libbpf to libxdp

libxsk move from libbpf to libxdp

[Andrii Nakryiko]

Hey guys,

It's been a while since we chatted about libxsk move. I believe last
time we were already almost ready to recommend libxdp for this, but
I'd like to double-check. Can one of you please own [0], validate that
whatever APIs are provided by libxdp are equivalent to what libbpf
provides, and start marking xdk.h APIs as deprecated? Thanks!

  [0] https://github.com/libbpf/libbpf/issues/270

-- Andrii

Re: libxsk move from libbpf to libxdp

[Magnus Karlsson]

On Fri, Oct 22, 2021 at 7:49 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> Hey guys,
>
> It's been a while since we chatted about libxsk move. I believe last
> time we were already almost ready to recommend libxdp for this, but
> I'd like to double-check. Can one of you please own [0], validate that
> whatever APIs are provided by libxdp are equivalent to what libbpf
> provides, and start marking xdk.h APIs as deprecated? Thanks!

Resending since Gmail had jumped out of plain text mode again.

No problem, I will own this. I will verify the APIs are the same then
submit a patch marking the ones in libbpf's xsk.h as deprecated.

One question is what to do with the samples and the selftests for xsk.
They currently rely on libbpf's xsk support. Two options that I see:

1: Require libxdp on the system. Do not try to compile the xsk samples
and selftests if libxdp is not available so the rest of the bpf
samples and selftests are not impacted.
2: Provide a standalone mock-up file of xsk.c and xsk.h that samples
and selftests could use.

I prefer #1 as it is better for the long-term. #2 means I would have
to maintain that mock-up file as libxdp features are added. Sounds
like double the amount of work to me. Thoughts?

/Magnus

>   [0] https://github.com/libbpf/libbpf/issues/270
>
> -- Andrii

Re: libxsk move from libbpf to libxdp

[Toke Høiland-Jørgensen]

Magnus Karlsson <magnus.karlsson@gmail.com> writes:

> On Fri, Oct 22, 2021 at 7:49 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> Hey guys,
>>
>> It's been a while since we chatted about libxsk move. I believe last
>> time we were already almost ready to recommend libxdp for this, but
>> I'd like to double-check. Can one of you please own [0], validate that
>> whatever APIs are provided by libxdp are equivalent to what libbpf
>> provides, and start marking xdk.h APIs as deprecated? Thanks!
>
> Resending since Gmail had jumped out of plain text mode again.
>
> No problem, I will own this. I will verify the APIs are the same then
> submit a patch marking the ones in libbpf's xsk.h as deprecated.
>
> One question is what to do with the samples and the selftests for xsk.
> They currently rely on libbpf's xsk support. Two options that I see:
>
> 1: Require libxdp on the system. Do not try to compile the xsk samples
> and selftests if libxdp is not available so the rest of the bpf
> samples and selftests are not impacted.
> 2: Provide a standalone mock-up file of xsk.c and xsk.h that samples
> and selftests could use.
>
> I prefer #1 as it is better for the long-term. #2 means I would have
> to maintain that mock-up file as libxdp features are added. Sounds
> like double the amount of work to me. Thoughts?

I agree #1 is preferable of those two. Another option is to move the
samples to the xdp-tools repo instead? Doesn't work for selftests, of
course; if it's acceptable to conditionally-compile the XSK tests
depending on system library availability that would be fine by me...

I pinged the Debian maintainer of libbpf to see if I can get him to pick
up libxdp as well, or sponsor me to maintain it. Should make the
transition smoother; guess I also need to get hold of the OpenSuse
people.

-Toke

Re: libxsk move from libbpf to libxdp

[Andrii Nakryiko]

On Mon, Oct 25, 2021 at 9:03 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Magnus Karlsson <magnus.karlsson@gmail.com> writes:
>
> > On Fri, Oct 22, 2021 at 7:49 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >>
> >> Hey guys,
> >>
> >> It's been a while since we chatted about libxsk move. I believe last
> >> time we were already almost ready to recommend libxdp for this, but
> >> I'd like to double-check. Can one of you please own [0], validate that
> >> whatever APIs are provided by libxdp are equivalent to what libbpf
> >> provides, and start marking xdk.h APIs as deprecated? Thanks!
> >
> > Resending since Gmail had jumped out of plain text mode again.
> >
> > No problem, I will own this. I will verify the APIs are the same then
> > submit a patch marking the ones in libbpf's xsk.h as deprecated.
> >
> > One question is what to do with the samples and the selftests for xsk.
> > They currently rely on libbpf's xsk support. Two options that I see:
> >
> > 1: Require libxdp on the system. Do not try to compile the xsk samples
> > and selftests if libxdp is not available so the rest of the bpf
> > samples and selftests are not impacted.
> > 2: Provide a standalone mock-up file of xsk.c and xsk.h that samples
> > and selftests could use.
> >
> > I prefer #1 as it is better for the long-term. #2 means I would have
> > to maintain that mock-up file as libxdp features are added. Sounds
> > like double the amount of work to me. Thoughts?
>
> I agree #1 is preferable of those two. Another option is to move the
> samples to the xdp-tools repo instead? Doesn't work for selftests, of
> course; if it's acceptable to conditionally-compile the XSK tests
> depending on system library availability that would be fine by me...

Seems like the only thing that uses xsk.h is xdpxceiver.c which is
tested through test_xsk.sh. It's not part of test_progs and so isn't
run regularly by BPF CI or maintainers. It makes sense to me to move
such test closer to the library it's supposed to be testing (i.e.,
libxdp)?

>
> I pinged the Debian maintainer of libbpf to see if I can get him to pick
> up libxdp as well, or sponsor me to maintain it. Should make the
> transition smoother; guess I also need to get hold of the OpenSuse
> people.
>
> -Toke
>

Re: libxsk move from libbpf to libxdp

[Magnus Karlsson]

On Tue, Oct 26, 2021 at 6:18 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Mon, Oct 25, 2021 at 9:03 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
> >
> > Magnus Karlsson <magnus.karlsson@gmail.com> writes:
> >
> > > On Fri, Oct 22, 2021 at 7:49 PM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > >>
> > >> Hey guys,
> > >>
> > >> It's been a while since we chatted about libxsk move. I believe last
> > >> time we were already almost ready to recommend libxdp for this, but
> > >> I'd like to double-check. Can one of you please own [0], validate that
> > >> whatever APIs are provided by libxdp are equivalent to what libbpf
> > >> provides, and start marking xdk.h APIs as deprecated? Thanks!
> > >
> > > Resending since Gmail had jumped out of plain text mode again.
> > >
> > > No problem, I will own this. I will verify the APIs are the same then
> > > submit a patch marking the ones in libbpf's xsk.h as deprecated.
> > >
> > > One question is what to do with the samples and the selftests for xsk.
> > > They currently rely on libbpf's xsk support. Two options that I see:
> > >
> > > 1: Require libxdp on the system. Do not try to compile the xsk samples
> > > and selftests if libxdp is not available so the rest of the bpf
> > > samples and selftests are not impacted.
> > > 2: Provide a standalone mock-up file of xsk.c and xsk.h that samples
> > > and selftests could use.
> > >
> > > I prefer #1 as it is better for the long-term. #2 means I would have
> > > to maintain that mock-up file as libxdp features are added. Sounds
> > > like double the amount of work to me. Thoughts?
> >
> > I agree #1 is preferable of those two. Another option is to move the
> > samples to the xdp-tools repo instead? Doesn't work for selftests, of
> > course; if it's acceptable to conditionally-compile the XSK tests
> > depending on system library availability that would be fine by me...
>
> Seems like the only thing that uses xsk.h is xdpxceiver.c which is
> tested through test_xsk.sh. It's not part of test_progs and so isn't
> run regularly by BPF CI or maintainers. It makes sense to me to move
> such test closer to the library it's supposed to be testing (i.e.,
> libxdp)?

xdpxceiver.c tests kernel functionality, not libxdp functionality,
though it does use libxdp (and libbpf) to make the implementation
simpler. So it should remain here and use strategy #1. libxdp tests
are on another level and should definitely go into the libxdp repo.
The xsk samples in samples/bpf/, we could just stop developing/retire
(or even remove) in the Linux repo and move them to the xdp-tools
repo. They just show how to use the xsk.h api:s and it makes more
sense to have them together with libxdp.

> >
> > I pinged the Debian maintainer of libbpf to see if I can get him to pick
> > up libxdp as well, or sponsor me to maintain it. Should make the
> > transition smoother; guess I also need to get hold of the OpenSuse
> > people.
> >
> > -Toke
> >

Re: libxsk move from libbpf to libxdp

[Toke Høiland-Jørgensen]

Magnus Karlsson <magnus.karlsson@gmail.com> writes:

> On Tue, Oct 26, 2021 at 6:18 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Mon, Oct 25, 2021 at 9:03 AM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>> >
>> > Magnus Karlsson <magnus.karlsson@gmail.com> writes:
>> >
>> > > On Fri, Oct 22, 2021 at 7:49 PM Andrii Nakryiko
>> > > <andrii.nakryiko@gmail.com> wrote:
>> > >>
>> > >> Hey guys,
>> > >>
>> > >> It's been a while since we chatted about libxsk move. I believe last
>> > >> time we were already almost ready to recommend libxdp for this, but
>> > >> I'd like to double-check. Can one of you please own [0], validate that
>> > >> whatever APIs are provided by libxdp are equivalent to what libbpf
>> > >> provides, and start marking xdk.h APIs as deprecated? Thanks!
>> > >
>> > > Resending since Gmail had jumped out of plain text mode again.
>> > >
>> > > No problem, I will own this. I will verify the APIs are the same then
>> > > submit a patch marking the ones in libbpf's xsk.h as deprecated.
>> > >
>> > > One question is what to do with the samples and the selftests for xsk.
>> > > They currently rely on libbpf's xsk support. Two options that I see:
>> > >
>> > > 1: Require libxdp on the system. Do not try to compile the xsk samples
>> > > and selftests if libxdp is not available so the rest of the bpf
>> > > samples and selftests are not impacted.
>> > > 2: Provide a standalone mock-up file of xsk.c and xsk.h that samples
>> > > and selftests could use.
>> > >
>> > > I prefer #1 as it is better for the long-term. #2 means I would have
>> > > to maintain that mock-up file as libxdp features are added. Sounds
>> > > like double the amount of work to me. Thoughts?
>> >
>> > I agree #1 is preferable of those two. Another option is to move the
>> > samples to the xdp-tools repo instead? Doesn't work for selftests, of
>> > course; if it's acceptable to conditionally-compile the XSK tests
>> > depending on system library availability that would be fine by me...
>>
>> Seems like the only thing that uses xsk.h is xdpxceiver.c which is
>> tested through test_xsk.sh. It's not part of test_progs and so isn't
>> run regularly by BPF CI or maintainers. It makes sense to me to move
>> such test closer to the library it's supposed to be testing (i.e.,
>> libxdp)?
>
> xdpxceiver.c tests kernel functionality, not libxdp functionality,
> though it does use libxdp (and libbpf) to make the implementation
> simpler. So it should remain here and use strategy #1. libxdp tests
> are on another level and should definitely go into the libxdp repo.
> The xsk samples in samples/bpf/, we could just stop developing/retire
> (or even remove) in the Linux repo and move them to the xdp-tools
> repo. They just show how to use the xsk.h api:s and it makes more
> sense to have them together with libxdp.

SGTM :)

-Toke