libbpf distro packaging

libbpf distro packaging

[Julia Kartseva]

I would like to bring up libbpf publishing discussion started at [1].
The present state of things is that libbpf is built from kernel tree, e.g. [2]
For Debian and [3] for Fedora whereas the better way would be having a
package built from github mirror. The advantages of the latter:
- Consistent, ABI matching versioning across distros
- The mirror has integration tests
- No need in kernel tree to build a package
- Changes can be merged directly to github w/o waiting them to be merged
through bpf-next -> net-next -> main
There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
Any comments regarding the spec itself can be posted there.
In the future it may be used as a source of truth.
Please consider switching libbpf packaging to the github mirror instead
of the kernel tree.
Thanks

[1] https://lists.iovisor.org/g/iovisor-dev/message/1521
[2] https://packages.debian.org/sid/libbpf4.19
[3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html
[4] https://github.com/libbpf/libbpf/pull/64

Re: libbpf distro packaging

[Jiri Olsa]

On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
> I would like to bring up libbpf publishing discussion started at [1].
> The present state of things is that libbpf is built from kernel tree, e.g. [2]
> For Debian and [3] for Fedora whereas the better way would be having a
> package built from github mirror. The advantages of the latter:
> - Consistent, ABI matching versioning across distros
> - The mirror has integration tests
> - No need in kernel tree to build a package
> - Changes can be merged directly to github w/o waiting them to be merged
> through bpf-next -> net-next -> main
> There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
> Any comments regarding the spec itself can be posted there.
> In the future it may be used as a source of truth.
> Please consider switching libbpf packaging to the github mirror instead
> of the kernel tree.
> Thanks
> 
> [1] https://lists.iovisor.org/g/iovisor-dev/message/1521
> [2] https://packages.debian.org/sid/libbpf4.19
> [3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html
> [4] https://github.com/libbpf/libbpf/pull/64

hi,
Fedora has libbpf as kernel-tools subpackage, so I think
we'd need to create new package and deprecate the current

but I like the ABI stability by using github .. how's actually
the sync (in both directions) with kernel sources going on?

thanks,
jirka

Re: libbpf distro packaging

[Daniel Borkmann]

On 8/13/19 2:24 PM, Jiri Olsa wrote:
> On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
>> I would like to bring up libbpf publishing discussion started at [1].
>> The present state of things is that libbpf is built from kernel tree, e.g. [2]
>> For Debian and [3] for Fedora whereas the better way would be having a
>> package built from github mirror. The advantages of the latter:
>> - Consistent, ABI matching versioning across distros
>> - The mirror has integration tests
>> - No need in kernel tree to build a package
>> - Changes can be merged directly to github w/o waiting them to be merged
>> through bpf-next -> net-next -> main
>> There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
>> Any comments regarding the spec itself can be posted there.
>> In the future it may be used as a source of truth.
>> Please consider switching libbpf packaging to the github mirror instead
>> of the kernel tree.
>> Thanks
>>
>> [1] https://lists.iovisor.org/g/iovisor-dev/message/1521
>> [2] https://packages.debian.org/sid/libbpf4.19
>> [3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html
>> [4] https://github.com/libbpf/libbpf/pull/64
> 
> hi,
> Fedora has libbpf as kernel-tools subpackage, so I think
> we'd need to create new package and deprecate the current
> 
> but I like the ABI stability by using github .. how's actually
> the sync (in both directions) with kernel sources going on?

The upstream kernel's tools/lib/bpf/ is always source of truth. Meaning, changes need
to make it upstream first and they are later synced into the GH stand-alone repo.

Thanks,
Daniel

Re: libbpf distro packaging

[Andrii Nakryiko]

On Tue, Aug 13, 2019 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
> > I would like to bring up libbpf publishing discussion started at [1].
> > The present state of things is that libbpf is built from kernel tree, e.g. [2]
> > For Debian and [3] for Fedora whereas the better way would be having a
> > package built from github mirror. The advantages of the latter:
> > - Consistent, ABI matching versioning across distros
> > - The mirror has integration tests
> > - No need in kernel tree to build a package
> > - Changes can be merged directly to github w/o waiting them to be merged
> > through bpf-next -> net-next -> main
> > There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
> > Any comments regarding the spec itself can be posted there.
> > In the future it may be used as a source of truth.
> > Please consider switching libbpf packaging to the github mirror instead
> > of the kernel tree.
> > Thanks
> >
> > [1] https://lists.iovisor.org/g/iovisor-dev/message/1521
> > [2] https://packages.debian.org/sid/libbpf4.19
> > [3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html
> > [4] https://github.com/libbpf/libbpf/pull/64
>
> hi,
> Fedora has libbpf as kernel-tools subpackage, so I think
> we'd need to create new package and deprecate the current
>
> but I like the ABI stability by using github .. how's actually
> the sync (in both directions) with kernel sources going on?

Sync is always in one direction, from kernel sources into Github repo.
Right now it's triggered by a human (usually me), but we are using a
script that automates entire process (see
https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh).
It cherry-pick relevant commits from kernel, transforms them to match
Github's file layout and re-applies those changes to Github repo.

There is never a sync from Github back to kernel, but Github repo
contains some extra stuff that's not in kernel. E.g., the script I
mentioned, plus Github's Makefile is different, because it can't rely
on kernel's kbuild setup.

>
> thanks,
> jirka

Re: libbpf distro packaging

[Andrii Nakryiko]

On Tue, Aug 13, 2019 at 7:14 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
>
> On 8/13/19 2:24 PM, Jiri Olsa wrote:
> > On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
> >> I would like to bring up libbpf publishing discussion started at [1].
> >> The present state of things is that libbpf is built from kernel tree, e.g. [2]
> >> For Debian and [3] for Fedora whereas the better way would be having a
> >> package built from github mirror. The advantages of the latter:
> >> - Consistent, ABI matching versioning across distros
> >> - The mirror has integration tests
> >> - No need in kernel tree to build a package
> >> - Changes can be merged directly to github w/o waiting them to be merged
> >> through bpf-next -> net-next -> main
> >> There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
> >> Any comments regarding the spec itself can be posted there.
> >> In the future it may be used as a source of truth.
> >> Please consider switching libbpf packaging to the github mirror instead
> >> of the kernel tree.
> >> Thanks
> >>
> >> [1] https://lists.iovisor.org/g/iovisor-dev/message/1521
> >> [2] https://packages.debian.org/sid/libbpf4.19
> >> [3] http://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/l/libbpf-5.3.0-0.rc2.git0.1.fc31.x86_64.html
> >> [4] https://github.com/libbpf/libbpf/pull/64
> >
> > hi,
> > Fedora has libbpf as kernel-tools subpackage, so I think
> > we'd need to create new package and deprecate the current
> >
> > but I like the ABI stability by using github .. how's actually
> > the sync (in both directions) with kernel sources going on?
>
> The upstream kernel's tools/lib/bpf/ is always source of truth. Meaning, changes need
> to make it upstream first and they are later synced into the GH stand-alone repo.

As I mentioned in reply to Jiri, kernel's tools/lib/bpf are the source
of truth for the sources of libbpf itself, but Github has some extra
stuff necessary to make libbpf work/build in isolation from kernel.
Plus some administrative stuff (e.g., sync script).

So if this spec is geared towards Github layout and for use with
Github projection of libbpf, maybe it makes more sense to keep it in
Github only? Is that spec going to be useful in kernel sources? Or
will it just create more confusion on why it's there?

Plus it will make it easier to synchronize version bumping/tagging of
new release on Github.

>
> Thanks,
> Daniel

Re: libbpf distro packaging

[Julia Kartseva]

On 8/19/19, 11:08 AM, "Julia Kartseva" <hex@fb.com> wrote:

    On 8/13/19, 11:24 AM, "Andrii Nakryiko" <andrii.nakryiko@gmail.com> wrote:
    
        On Tue, Aug 13, 2019 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
        >
        > On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
        > > I would like to bring up libbpf publishing discussion started at [1].
        > > The present state of things is that libbpf is built from kernel tree, e.g. [2]
        > > For Debian and [3] for Fedora whereas the better way would be having a
        > > package built from github mirror. The advantages of the latter:
        > > - Consistent, ABI matching versioning across distros
        > > - The mirror has integration tests
        > > - No need in kernel tree to build a package
        > > - Changes can be merged directly to github w/o waiting them to be merged
        > > through bpf-next -> net-next -> main
        > > There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
        > > Any comments regarding the spec itself can be posted there.
        > > In the future it may be used as a source of truth.
        > > Please consider switching libbpf packaging to the github mirror instead
        > > of the kernel tree.
        > > Thanks
        > >
        > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.iovisor.org_g_iovisor-2Ddev_message_1521&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=dYAc2jLhFg0wtCZ_ms2HF5bWANoHzA3UMug5TNCeBtE&e= 
        > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__packages.debian.org_sid_libbpf4.19&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=lq1MpF-bt6y6ZEtFc57eT-BO_wMBx8uUBACJooWbUYk&e= 
        > > [3] https://urldefense.proofpoint.com/v2/url?u=http-3A__rpmfind.net_linux_RPM_fedora_devel_rawhide_x86-5F64_l_libbpf-2D5.3.0-2D0.rc2.git0.1.fc31.x86-5F64.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=NoolYHL57G2KhzE768iWdy6v5LD2GfJQyqPmtjy196E&e= 
        > > [4] https://github.com/libbpf/libbpf/pull/64
        >
        > hi,
        > Fedora has libbpf as kernel-tools subpackage, so I think
        > we'd need to create new package and deprecate the current
        >
        > but I like the ABI stability by using github .. how's actually
        > the sync (in both directions) with kernel sources going on?
        
        Sync is always in one direction, from kernel sources into Github repo.
        Right now it's triggered by a human (usually me), but we are using a
        script that automates entire process (see
        https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh).
        It cherry-pick relevant commits from kernel, transforms them to match
        Github's file layout and re-applies those changes to Github repo.
        
        There is never a sync from Github back to kernel, but Github repo
        contains some extra stuff that's not in kernel. E.g., the script I
        mentioned, plus Github's Makefile is different, because it can't rely
        on kernel's kbuild setup.

Hi Jiri,
I'm curious if you have any comments regarding sync procedure described
By Andrii. Or if there is anything else you'd like us to address so Fedora
can be switched to libbpf built from the github mirror?

Thank you
        
        >
        > thanks,
        > jirka
        

Re: libbpf distro packaging

[Jiri Olsa]

On Tue, Aug 20, 2019 at 10:27:23PM +0000, Julia Kartseva wrote:
> 
> 
> On 8/19/19, 11:08 AM, "Julia Kartseva" <hex@fb.com> wrote:
> 
>     On 8/13/19, 11:24 AM, "Andrii Nakryiko" <andrii.nakryiko@gmail.com> wrote:
>     
>         On Tue, Aug 13, 2019 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
>         >
>         > On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
>         > > I would like to bring up libbpf publishing discussion started at [1].
>         > > The present state of things is that libbpf is built from kernel tree, e.g. [2]
>         > > For Debian and [3] for Fedora whereas the better way would be having a
>         > > package built from github mirror. The advantages of the latter:
>         > > - Consistent, ABI matching versioning across distros
>         > > - The mirror has integration tests
>         > > - No need in kernel tree to build a package
>         > > - Changes can be merged directly to github w/o waiting them to be merged
>         > > through bpf-next -> net-next -> main
>         > > There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
>         > > Any comments regarding the spec itself can be posted there.
>         > > In the future it may be used as a source of truth.
>         > > Please consider switching libbpf packaging to the github mirror instead
>         > > of the kernel tree.
>         > > Thanks
>         > >
>         > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.iovisor.org_g_iovisor-2Ddev_message_1521&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=dYAc2jLhFg0wtCZ_ms2HF5bWANoHzA3UMug5TNCeBtE&e= 
>         > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__packages.debian.org_sid_libbpf4.19&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=lq1MpF-bt6y6ZEtFc57eT-BO_wMBx8uUBACJooWbUYk&e= 
>         > > [3] https://urldefense.proofpoint.com/v2/url?u=http-3A__rpmfind.net_linux_RPM_fedora_devel_rawhide_x86-5F64_l_libbpf-2D5.3.0-2D0.rc2.git0.1.fc31.x86-5F64.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=NoolYHL57G2KhzE768iWdy6v5LD2GfJQyqPmtjy196E&e= 
>         > > [4] https://github.com/libbpf/libbpf/pull/64
>         >
>         > hi,
>         > Fedora has libbpf as kernel-tools subpackage, so I think
>         > we'd need to create new package and deprecate the current
>         >
>         > but I like the ABI stability by using github .. how's actually
>         > the sync (in both directions) with kernel sources going on?
>         
>         Sync is always in one direction, from kernel sources into Github repo.
>         Right now it's triggered by a human (usually me), but we are using a
>         script that automates entire process (see
>         https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh).
>         It cherry-pick relevant commits from kernel, transforms them to match
>         Github's file layout and re-applies those changes to Github repo.
>         
>         There is never a sync from Github back to kernel, but Github repo
>         contains some extra stuff that's not in kernel. E.g., the script I
>         mentioned, plus Github's Makefile is different, because it can't rely
>         on kernel's kbuild setup.
> 
> Hi Jiri,
> I'm curious if you have any comments regarding sync procedure described
> By Andrii. Or if there is anything else you'd like us to address so Fedora
> can be switched to libbpf built from the github mirror?

hi,
yea, I think it's ok.. just need to check the implications
for rhel packaging and I'll let you know

jirka

Re: libbpf distro packaging

[Jiri Olsa]

On Wed, Aug 21, 2019 at 11:09:06PM +0200, Jiri Olsa wrote:
> On Tue, Aug 20, 2019 at 10:27:23PM +0000, Julia Kartseva wrote:
> > 
> > 
> > On 8/19/19, 11:08 AM, "Julia Kartseva" <hex@fb.com> wrote:
> > 
> >     On 8/13/19, 11:24 AM, "Andrii Nakryiko" <andrii.nakryiko@gmail.com> wrote:
> >     
> >         On Tue, Aug 13, 2019 at 5:26 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >         >
> >         > On Mon, Aug 12, 2019 at 07:04:12PM +0000, Julia Kartseva wrote:
> >         > > I would like to bring up libbpf publishing discussion started at [1].
> >         > > The present state of things is that libbpf is built from kernel tree, e.g. [2]
> >         > > For Debian and [3] for Fedora whereas the better way would be having a
> >         > > package built from github mirror. The advantages of the latter:
> >         > > - Consistent, ABI matching versioning across distros
> >         > > - The mirror has integration tests
> >         > > - No need in kernel tree to build a package
> >         > > - Changes can be merged directly to github w/o waiting them to be merged
> >         > > through bpf-next -> net-next -> main
> >         > > There is a PR introducing a libbpf.spec which can be used as a starting point: [4]
> >         > > Any comments regarding the spec itself can be posted there.
> >         > > In the future it may be used as a source of truth.
> >         > > Please consider switching libbpf packaging to the github mirror instead
> >         > > of the kernel tree.
> >         > > Thanks
> >         > >
> >         > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.iovisor.org_g_iovisor-2Ddev_message_1521&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=dYAc2jLhFg0wtCZ_ms2HF5bWANoHzA3UMug5TNCeBtE&e= 
> >         > > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__packages.debian.org_sid_libbpf4.19&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=lq1MpF-bt6y6ZEtFc57eT-BO_wMBx8uUBACJooWbUYk&e= 
> >         > > [3] https://urldefense.proofpoint.com/v2/url?u=http-3A__rpmfind.net_linux_RPM_fedora_devel_rawhide_x86-5F64_l_libbpf-2D5.3.0-2D0.rc2.git0.1.fc31.x86-5F64.html&d=DwIBaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=zUrDY_Sp_5PqcGtRQPNeDA&m=prYVDiu3-aH1o2PWH4ZcP7lEQRCQAcTwcWPrJrtaroQ&s=NoolYHL57G2KhzE768iWdy6v5LD2GfJQyqPmtjy196E&e= 
> >         > > [4] https://github.com/libbpf/libbpf/pull/64
> >         >
> >         > hi,
> >         > Fedora has libbpf as kernel-tools subpackage, so I think
> >         > we'd need to create new package and deprecate the current
> >         >
> >         > but I like the ABI stability by using github .. how's actually
> >         > the sync (in both directions) with kernel sources going on?
> >         
> >         Sync is always in one direction, from kernel sources into Github repo.
> >         Right now it's triggered by a human (usually me), but we are using a
> >         script that automates entire process (see
> >         https://github.com/libbpf/libbpf/blob/master/scripts/sync-kernel.sh).
> >         It cherry-pick relevant commits from kernel, transforms them to match
> >         Github's file layout and re-applies those changes to Github repo.
> >         
> >         There is never a sync from Github back to kernel, but Github repo
> >         contains some extra stuff that's not in kernel. E.g., the script I
> >         mentioned, plus Github's Makefile is different, because it can't rely
> >         on kernel's kbuild setup.
> > 
> > Hi Jiri,
> > I'm curious if you have any comments regarding sync procedure described
> > By Andrii. Or if there is anything else you'd like us to address so Fedora
> > can be switched to libbpf built from the github mirror?
> 
> hi,
> yea, I think it's ok.. just need to check the implications
> for rhel packaging and I'll let you know

btw, the libbpf GH repo tag v0.0.4 has 0.0.3 version set in Makefile:

  VERSION = 0
  PATCHLEVEL = 0
  EXTRAVERSION = 3

current code takes version from libbpf.map so it's fine,
but would be great to start from 0.0.5 so we don't need to
bother with rpm patches.. is 0.0.5 planned soon?

thanks,
jirka

Re: libbpf distro packaging

[Alexei Starovoitov]

On 8/23/19 2:22 AM, Jiri Olsa wrote:
> btw, the libbpf GH repo tag v0.0.4 has 0.0.3 version set in Makefile:
> 
>    VERSION = 0
>    PATCHLEVEL = 0
>    EXTRAVERSION = 3
> 
> current code takes version from libbpf.map so it's fine,
> but would be great to start from 0.0.5 so we don't need to
> bother with rpm patches.. is 0.0.5 planned soon?

Technically we can bump it at any time.
The goal was to bump it only when new kernel is released
to capture a collection of new APIs in a given 0.0.X release.
So that libbpf versions are synchronized with kernel versions
in some what loose way.
In this case we can make an exception and bump it now.

Re: libbpf distro packaging

[Jiri Olsa]

On Fri, Aug 23, 2019 at 04:00:01PM +0000, Alexei Starovoitov wrote:
> On 8/23/19 2:22 AM, Jiri Olsa wrote:
> > btw, the libbpf GH repo tag v0.0.4 has 0.0.3 version set in Makefile:
> > 
> >    VERSION = 0
> >    PATCHLEVEL = 0
> >    EXTRAVERSION = 3
> > 
> > current code takes version from libbpf.map so it's fine,
> > but would be great to start from 0.0.5 so we don't need to
> > bother with rpm patches.. is 0.0.5 planned soon?
> 
> Technically we can bump it at any time.
> The goal was to bump it only when new kernel is released
> to capture a collection of new APIs in a given 0.0.X release.
> So that libbpf versions are synchronized with kernel versions
> in some what loose way.
> In this case we can make an exception and bump it now.

I see, I dont think it's worth of the exception now,
the patch is simple or we'll start with 0.0.3

jirka

Re: libbpf distro packaging

[Julia Kartseva]

On 8/25/19, 11:42 PM, "Jiri Olsa" <jolsa@redhat.com> wrote:

> On Fri, Aug 23, 2019 at 04:00:01PM +0000, Alexei Starovoitov wrote:
> > 
> > Technically we can bump it at any time.
> > The goal was to bump it only when new kernel is released
> > to capture a collection of new APIs in a given 0.0.X release.
> > So that libbpf versions are synchronized with kernel versions
> > in some what loose way.
> > In this case we can make an exception and bump it now.
>
> I see, I dont think it's worth of the exception now,
> the patch is simple or we'll start with 0.0.3

PR introducing 0.0.5 ABI was merged:
https://github.com/libbpf/libbpf/commit/476e158
Jiri, you'd like to avoid patching, you can start w/ 0.0.5.
Also if you're planning to use *.spec from libbpf as a source of truth,
It may be enhanced by syncing spec and ABI versions, similar to
https://github.com/libbpf/libbpf/commit/d60f568

Re: libbpf distro packaging

[Jiri Olsa]

On Tue, Aug 27, 2019 at 10:30:24PM +0000, Julia Kartseva wrote:
> On 8/25/19, 11:42 PM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> 
> > On Fri, Aug 23, 2019 at 04:00:01PM +0000, Alexei Starovoitov wrote:
> > > 
> > > Technically we can bump it at any time.
> > > The goal was to bump it only when new kernel is released
> > > to capture a collection of new APIs in a given 0.0.X release.
> > > So that libbpf versions are synchronized with kernel versions
> > > in some what loose way.
> > > In this case we can make an exception and bump it now.
> >
> > I see, I dont think it's worth of the exception now,
> > the patch is simple or we'll start with 0.0.3
> 
> PR introducing 0.0.5 ABI was merged:
> https://github.com/libbpf/libbpf/commit/476e158
> Jiri, you'd like to avoid patching, you can start w/ 0.0.5.
> Also if you're planning to use *.spec from libbpf as a source of truth,
> It may be enhanced by syncing spec and ABI versions, similar to
> https://github.com/libbpf/libbpf/commit/d60f568

cool, anyway I started with v0.0.3 ;-) I'll update
to latest once we are merged in

the spec/srpm is currently under Fedora review:
  https://bugzilla.redhat.com/show_bug.cgi?id=1745478

you can check it in here:
  http://people.redhat.com/~jolsa/libbpf/v2/

I think it's little different from what you have,
but not in the essential parts

jirka

Re: libbpf distro packaging

[Jiri Olsa]

On Wed, Aug 28, 2019 at 09:12:37AM +0200, Jiri Olsa wrote:
> On Tue, Aug 27, 2019 at 10:30:24PM +0000, Julia Kartseva wrote:
> > On 8/25/19, 11:42 PM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> > 
> > > On Fri, Aug 23, 2019 at 04:00:01PM +0000, Alexei Starovoitov wrote:
> > > > 
> > > > Technically we can bump it at any time.
> > > > The goal was to bump it only when new kernel is released
> > > > to capture a collection of new APIs in a given 0.0.X release.
> > > > So that libbpf versions are synchronized with kernel versions
> > > > in some what loose way.
> > > > In this case we can make an exception and bump it now.
> > >
> > > I see, I dont think it's worth of the exception now,
> > > the patch is simple or we'll start with 0.0.3
> > 
> > PR introducing 0.0.5 ABI was merged:
> > https://github.com/libbpf/libbpf/commit/476e158
> > Jiri, you'd like to avoid patching, you can start w/ 0.0.5.
> > Also if you're planning to use *.spec from libbpf as a source of truth,
> > It may be enhanced by syncing spec and ABI versions, similar to
> > https://github.com/libbpf/libbpf/commit/d60f568
> 
> cool, anyway I started with v0.0.3 ;-) I'll update
> to latest once we are merged in
> 
> the spec/srpm is currently under Fedora review:
>   https://bugzilla.redhat.com/show_bug.cgi?id=1745478
> 
> you can check it in here:
>   http://people.redhat.com/~jolsa/libbpf/v2/
> 
> I think it's little different from what you have,
> but not in the essential parts

heya,
FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available

jirka

Re: libbpf distro packaging

[Julia Kartseva]

Thank you Jiri, that's great news.

Adding Marco D'Itri.
Marco, I wonder if you are OK with the rationale for libbpf packaging
from GH mirror? Can we proceed with switching Debian package as well
just like we discussed offline at ASG?
The bug report for Fedora: [1]
Thank you

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1745478

> On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
>
> heya,
> FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available

Re: libbpf distro packaging

[Julia Kartseva]

Hi Jiri, 

v0.0.5 is out: [1] which brings questions regarding further maintenance 
of rpm. 
Who'll maintain the most recent version of rpm up-to-date? 
Are you looking into making the procedure automated and do you need any 
help from the side of libbpf devs if so? In particular, we can have the *.spec file
in GH mirror synchronized with the most recent tag so you can take it from the 
mirror along with tarball.
Thanks! 

[1] https://github.com/libbpf/libbpf/releases/tag/v0.0.5

On 9/30/19, 11:18 AM, "Julia Kartseva" <hex@fb.com> wrote:

> Thank you Jiri, that's great news.
>
> > On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> >
> > heya,
> > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available

Re: libbpf distro packaging

[Jiri Olsa]

On Thu, Oct 03, 2019 at 12:50:08AM +0000, Julia Kartseva wrote:
> Hi Jiri, 
> 
> v0.0.5 is out: [1] which brings questions regarding further maintenance 
> of rpm. 
> Who'll maintain the most recent version of rpm up-to-date? 

I will do that on fedora/rhel side now

> Are you looking into making the procedure automated and do you need any 
> help from the side of libbpf devs if so? In particular, we can have the *.spec file
> in GH mirror synchronized with the most recent tag so you can take it from the 
> mirror along with tarball.

some notification of new tag/sync would be great

the spec file update is not a big deal because I need to do its
changelog update anyway.. but I can put the fedora rpm spec in
GH repo for reference, I will do a pull request for that

jirka

> Thanks! 
> 
> [1] https://github.com/libbpf/libbpf/releases/tag/v0.0.5
> 
> On 9/30/19, 11:18 AM, "Julia Kartseva" <hex@fb.com> wrote:
> 
> > Thank you Jiri, that's great news.
> >
> > > On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> > >
> > > heya,
> > > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> > > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
> 
> 
> 

Re: libbpf distro packaging

[Andrii Nakryiko]

On Thu, Oct 3, 2019 at 4:10 AM Jiri Olsa <jolsa@redhat.com> wrote:
>
> On Thu, Oct 03, 2019 at 12:50:08AM +0000, Julia Kartseva wrote:
> > Hi Jiri,
> >
> > v0.0.5 is out: [1] which brings questions regarding further maintenance
> > of rpm.
> > Who'll maintain the most recent version of rpm up-to-date?
>
> I will do that on fedora/rhel side now
>
> > Are you looking into making the procedure automated and do you need any
> > help from the side of libbpf devs if so? In particular, we can have the *.spec file
> > in GH mirror synchronized with the most recent tag so you can take it from the
> > mirror along with tarball.
>
> some notification of new tag/sync would be great

Hey Jiri! You can watch Github repo for new releases, see
https://help.github.com/en/articles/watching-and-unwatching-releases-for-a-repository.

>
> the spec file update is not a big deal because I need to do its
> changelog update anyway.. but I can put the fedora rpm spec in
> GH repo for reference, I will do a pull request for that
>
> jirka
>
> > Thanks!
> >
> > [1] https://github.com/libbpf/libbpf/releases/tag/v0.0.5
> >
> > On 9/30/19, 11:18 AM, "Julia Kartseva" <hex@fb.com> wrote:
> >
> > > Thank you Jiri, that's great news.
> > >
> > > > On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> > > >
> > > > heya,
> > > > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> > > > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
> >
> >
> >

Re: libbpf distro packaging

[Jiri Olsa]

On Thu, Oct 03, 2019 at 09:24:21AM -0700, Andrii Nakryiko wrote:
> On Thu, Oct 3, 2019 at 4:10 AM Jiri Olsa <jolsa@redhat.com> wrote:
> >
> > On Thu, Oct 03, 2019 at 12:50:08AM +0000, Julia Kartseva wrote:
> > > Hi Jiri,
> > >
> > > v0.0.5 is out: [1] which brings questions regarding further maintenance
> > > of rpm.
> > > Who'll maintain the most recent version of rpm up-to-date?
> >
> > I will do that on fedora/rhel side now
> >
> > > Are you looking into making the procedure automated and do you need any
> > > help from the side of libbpf devs if so? In particular, we can have the *.spec file
> > > in GH mirror synchronized with the most recent tag so you can take it from the
> > > mirror along with tarball.
> >
> > some notification of new tag/sync would be great
> 
> Hey Jiri! You can watch Github repo for new releases, see
> https://help.github.com/en/articles/watching-and-unwatching-releases-for-a-repository.

cool, I was hoping there's something like this

thanks,
jirka

> 
> >
> > the spec file update is not a big deal because I need to do its
> > changelog update anyway.. but I can put the fedora rpm spec in
> > GH repo for reference, I will do a pull request for that
> >
> > jirka
> >
> > > Thanks!
> > >
> > > [1] https://github.com/libbpf/libbpf/releases/tag/v0.0.5
> > >
> > > On 9/30/19, 11:18 AM, "Julia Kartseva" <hex@fb.com> wrote:
> > >
> > > > Thank you Jiri, that's great news.
> > > >
> > > > > On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> > > > >
> > > > > heya,
> > > > > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> > > > > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
> > >
> > >
> > >

Re: libbpf distro packaging

[Julia Kartseva]

On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:

> heya,
> FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
>
> jirka

Hi Jiri,

I wonder what are the steps to make libbpf available for CentOS {7|8} as well?
One (likely the quickest) way to do that is to publish it to Fedora's EPEL [1].

I have a little concern about dependencies, namely elfutils-libelf-devel and 
elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if 
dependencies from another repo are fine.

Thoughts? Thanks!

[1] https://fedoraproject.org/wiki/EPEL
[2] http://mirror.centos.org/centos/7/os/x86_64/Packages/

Re: libbpf distro packaging

[Jiri Olsa]

On Mon, Oct 07, 2019 at 12:25:51AM +0000, Julia Kartseva wrote:
> On 9/30/19, 4:13 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> 
> > heya,
> > FYI we got it through.. there's libbpf-0.0.3 available on fedora 30/31/32
> > I'll update to 0.0.5 version as soon as there's the v0.0.5 tag available
> >
> > jirka
> 
> Hi Jiri,
> 
> I wonder what are the steps to make libbpf available for CentOS {7|8} as well?
> One (likely the quickest) way to do that is to publish it to Fedora's EPEL [1].
> 
> I have a little concern about dependencies, namely elfutils-libelf-devel and 
> elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if 
> dependencies from another repo are fine.
> 
> Thoughts? Thanks!

I think that should be ok, I'll ask around and let you know

jirka

Re: libbpf distro packaging

[Julia Kartseva]

Hi Jiri,

systemd folks published libbpf CentOS 7 package in systemd corp repo: [1],
so guess that proves that deps from other repo are fine.

Rebuild is fairly simple: [2]

[1] https://copr.fedorainfracloud.org/coprs/mrc0mmand/systemd-centos-ci/build/1053694/
[2] https://github.com/systemd/systemd/pull/13744#issuecomment-541168076

On 10/8/19, 12:40 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
>
> On Mon, Oct 07, 2019 at 12:25:51AM +0000, Julia Kartseva wrote:
> > 
> > I wonder what are the steps to make libbpf available for CentOS {7|8} as well?
> > One (likely the quickest) way to do that is to publish it to Fedora's EPEL [1].
> > 
> > I have a little concern about dependencies, namely elfutils-libelf-devel and 
> > elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if 
> > dependencies from another repo are fine.
> > 
> > Thoughts? Thanks!
>
> I think that should be ok, I'll ask around and let you know
>
> jirka

Re: libbpf distro packaging

[Jiri Olsa]

On Fri, Oct 11, 2019 at 09:14:19PM +0000, Julia Kartseva wrote:
> Hi Jiri,
> 
> systemd folks published libbpf CentOS 7 package in systemd corp repo: [1],
> so guess that proves that deps from other repo are fine.

yea, actualy got request for that:
  https://bugzilla.redhat.com/show_bug.cgi?id=1762219

jirka

> 
> Rebuild is fairly simple: [2]
> 
> [1] https://copr.fedorainfracloud.org/coprs/mrc0mmand/systemd-centos-ci/build/1053694/
> [2] https://github.com/systemd/systemd/pull/13744#issuecomment-541168076
> 
> On 10/8/19, 12:40 AM, "Jiri Olsa" <jolsa@redhat.com> wrote:
> >
> > On Mon, Oct 07, 2019 at 12:25:51AM +0000, Julia Kartseva wrote:
> > > 
> > > I wonder what are the steps to make libbpf available for CentOS {7|8} as well?
> > > One (likely the quickest) way to do that is to publish it to Fedora's EPEL [1].
> > > 
> > > I have a little concern about dependencies, namely elfutils-libelf-devel and 
> > > elfutils-devel are sourced directly by CentOS repos, e.g. [2], not sure if 
> > > dependencies from another repo are fine.
> > > 
> > > Thoughts? Thanks!
> >
> > I think that should be ok, I'll ask around and let you know
> >
> > jirka
> 

Re: libbpf distro packaging

[Julia Kartseva]

Hi Jiri,

1. v. 0.0.6 is out [1], could you please package it?
2. we might need a small spec update due to zlib is made an explicit
dependency in [2]. zlib should be listed in BuildRequires: section of the
spec so it's consistent with libbpf.pc
3. Do you plan to address the bug report [3] for CentOS? Namely rebuilding
Fedora's RPM and publishing to EPEL repo?

Thanks

[1] https://github.com/libbpf/libbpf/releases/tag/v0.0.6
[2] https://lore.kernel.org/bpf/20191216183830.3972964-1-andriin@fb.com/
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1762219

On 10/16/19 3:01 AM, Jiri Olsa wrote:
> On Fri, Oct 11, 2019 at 09:14:19PM +0000, Julia Kartseva wrote:
>> Hi Jiri,
>>
>> systemd folks published libbpf CentOS 7 package in systemd corp repo: [1],
>> so guess that proves that deps from other repo are fine.
> 
> yea, actualy got request for that:
>   https://bugzilla.redhat.com/show_bug.cgi?id=1762219
> 
> jirka
> 

Re: libbpf distro packaging

[Jiri Olsa]

On Thu, Dec 19, 2019 at 09:37:23PM +0000, Julia Kartseva wrote:
> Hi Jiri,
> 
> 1. v. 0.0.6 is out [1], could you please package it?
> 2. we might need a small spec update due to zlib is made an explicit
> dependency in [2]. zlib should be listed in BuildRequires: section of the
> spec so it's consistent with libbpf.pc

sure, it's ok for rawhide, in fedora 31/30 we still don't have
latest headers packaged

> 3. Do you plan to address the bug report [3] for CentOS? Namely rebuilding
> Fedora's RPM and publishing to EPEL repo?

I did not get any answers on who would do that internally,
so I'm afreaid I'll have to do it, but let me ask again first ;-)

jirka

Re: libbpf distro packaging

[Julia Kartseva]

On 12/20/19 5:58 AM, Jiri Olsa wrote:
> On Thu, Dec 19, 2019 at 09:37:23PM +0000, Julia Kartseva wrote:
>> Hi Jiri,
>>
>> 1. v. 0.0.6 is out [1], could you please package it?
>> 2. we might need a small spec update due to zlib is made an explicit
>> dependency in [2]. zlib should be listed in BuildRequires: section of the
>> spec so it's consistent with libbpf.pc
> 
> sure, it's ok for rawhide, in fedora 31/30 we still don't have
> latest headers packaged
> 
>> 3. Do you plan to address the bug report [3] for CentOS? Namely rebuilding
>> Fedora's RPM and publishing to EPEL repo?
> 
> I did not get any answers on who would do that internally,
> so I'm afreaid I'll have to do it, but let me ask again first ;-)
>

Bump :)
Jiri, any volunteers for Fedora EPEL 7|8 packaging?
systemd folks would appreciate it.
Thanks!

> jirka
> 

Re: libbpf distro packaging

[Jiri Olsa]

On Wed, Mar 04, 2020 at 04:22:35PM -0800, Julia Kartseva wrote:
> 
> 
> On 12/20/19 5:58 AM, Jiri Olsa wrote:
> > On Thu, Dec 19, 2019 at 09:37:23PM +0000, Julia Kartseva wrote:
> >> Hi Jiri,
> >>
> >> 1. v. 0.0.6 is out [1], could you please package it?
> >> 2. we might need a small spec update due to zlib is made an explicit
> >> dependency in [2]. zlib should be listed in BuildRequires: section of the
> >> spec so it's consistent with libbpf.pc
> > 
> > sure, it's ok for rawhide, in fedora 31/30 we still don't have
> > latest headers packaged
> > 
> >> 3. Do you plan to address the bug report [3] for CentOS? Namely rebuilding
> >> Fedora's RPM and publishing to EPEL repo?
> > 
> > I did not get any answers on who would do that internally,
> > so I'm afreaid I'll have to do it, but let me ask again first ;-)
> >
> 
> Bump :)
> Jiri, any volunteers for Fedora EPEL 7|8 packaging?
> systemd folks would appreciate it.
> Thanks!

hi,
yep.. we've got Cestmir as 'volunteer' for that (cc-ed) ;-) I can see
there's already epel8 branch for libbpf created.. but not sure what's
the actual rpm status

jirka

Re: libbpf distro packaging

[Jiri Olsa]

On Thu, Mar 05, 2020 at 03:18:12PM +0100, Jiri Olsa wrote:
> On Wed, Mar 04, 2020 at 04:22:35PM -0800, Julia Kartseva wrote:
> > 
> > 
> > On 12/20/19 5:58 AM, Jiri Olsa wrote:
> > > On Thu, Dec 19, 2019 at 09:37:23PM +0000, Julia Kartseva wrote:
> > >> Hi Jiri,
> > >>
> > >> 1. v. 0.0.6 is out [1], could you please package it?
> > >> 2. we might need a small spec update due to zlib is made an explicit
> > >> dependency in [2]. zlib should be listed in BuildRequires: section of the
> > >> spec so it's consistent with libbpf.pc
> > > 
> > > sure, it's ok for rawhide, in fedora 31/30 we still don't have
> > > latest headers packaged
> > > 
> > >> 3. Do you plan to address the bug report [3] for CentOS? Namely rebuilding
> > >> Fedora's RPM and publishing to EPEL repo?
> > > 
> > > I did not get any answers on who would do that internally,
> > > so I'm afreaid I'll have to do it, but let me ask again first ;-)
> > >
> > 
> > Bump :)
> > Jiri, any volunteers for Fedora EPEL 7|8 packaging?
> > systemd folks would appreciate it.
> > Thanks!
> 
> hi,
> yep.. we've got Cestmir as 'volunteer' for that (cc-ed) ;-) I can see
> there's already epel8 branch for libbpf created.. but not sure what's
> the actual rpm status

so I did some more checking and libbpf is automatically pulled into
centos 8, it's just at the moment there's some bug preventing that.. 
it is going to be fixed shortly ;-)

as for centos 7, what is the target user there? which version of libbpf
would you need in there?

jirka

Re: libbpf distro packaging

[Julia Kartseva]

On 3/10/20 7:57 AM, Jiri Olsa wrote:
> On Thu, Mar 05, 2020 at 03:18:12PM +0100, Jiri Olsa wrote:
> 
> so I did some more checking and libbpf is automatically pulled into
> centos 8, it's just at the moment there's some bug preventing that.. 
> it is going to be fixed shortly ;-)
> 
> as for centos 7, what is the target user there? which version of libbpf
> would you need in there?
> 
> jirka
>
Hi, that's great news!
Nothing prevents us from having the latest v0.0.7 [1] in CentOS 7 :)
Can updates for CentOS 7 and 8 be synced so the have the same libbpf version?

[1] https://github.com/libbpf/libbpf/releases/tag/v0.0.7

Re: libbpf distro packaging

[Jiri Olsa]

On Tue, Mar 10, 2020 at 10:18:12AM -0700, Julia Kartseva wrote:
> 
> 
> On 3/10/20 7:57 AM, Jiri Olsa wrote:
> > On Thu, Mar 05, 2020 at 03:18:12PM +0100, Jiri Olsa wrote:
> > 
> > so I did some more checking and libbpf is automatically pulled into
> > centos 8, it's just at the moment there's some bug preventing that.. 
> > it is going to be fixed shortly ;-)
> > 
> > as for centos 7, what is the target user there? which version of libbpf
> > would you need in there?
> > 
> > jirka
> >
> Hi, that's great news!
> Nothing prevents us from having the latest v0.0.7 [1] in CentOS 7 :)

that's just half true.. while libbpf is ok, libbpf-devel needs uapi
headers to define all the stuff that's used in libbpf's headers

  Example:
	$ echo "#include <bpf/xsk.h>" | gcc -x c - 
	In file included from <stdin>:1:
	/usr/include/bpf/xsk.h: In function ‘xsk_ring_prod__needs_wakeup’:
	/usr/include/bpf/xsk.h:82:21: error: ‘XDP_RING_NEED_WAKEUP’ undeclared (first use in this function)
	   82 |  return *r->flags & XDP_RING_NEED_WAKEUP;
	      |                     ^~~~~~~~~~~~~~~~~~~~
	/usr/include/bpf/xsk.h:82:21: note: each undeclared identifier is reported only once for each function it appears in
	/usr/include/bpf/xsk.h: In function ‘xsk_umem__extract_addr’:
	/usr/include/bpf/xsk.h:173:16: error: ‘XSK_UNALIGNED_BUF_ADDR_MASK’ undeclared (first use in this function)
	  173 |  return addr & XSK_UNALIGNED_BUF_ADDR_MASK;
	      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
	/usr/include/bpf/xsk.h: In function ‘xsk_umem__extract_offset’:
	/usr/include/bpf/xsk.h:178:17: error: ‘XSK_UNALIGNED_BUF_OFFSET_SHIFT’ undeclared (first use in this function)
	  178 |  return addr >> XSK_UNALIGNED_BUF_OFFSET_SHIFT;
	      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I'll need to check the state of rhel7 kernel headers, but that was
very early backport and I think headers are far behind

jirka

> Can updates for CentOS 7 and 8 be synced so the have the same libbpf version?
> 
> [1] https://github.com/libbpf/libbpf/releases/tag/v0.0.7
>