Getting the latest and greatest Linux perf features on every Debian kernel

Getting the latest and greatest Linux perf features on every Debian kernel

[Ian Rogers]

(Apologies, resending in plain-text to allow linux-perf-users to
follow along - the 1st e-mail bounced)

Hi,

Debian currently tries to match the Linux perf tool to the version of
the kernel that it is being run upon. Reaching out to Ben Hutchings,
he explained to me that this was done back in 2010 due to kernel and
Linux perf incompatibilities. This was likely the case, but it was a
bug in the Linux perf tool that should have been fixed. It is the goal
of the tool to be backward compatible. A problem with matching the
tool to the kernel version is that users miss out on new features and
fixes (this topic came up in a recent interview of the maintainer
Arnaldo Carvalho de Melo [1]).

Ben Hutchings informs me that making it so that Debian ships the
latest Linux perf tool requires updates both to the linux-base and
linux source packages. The Linux perf tool also has many other often
optional dependencies, like libunwind, libbpf, libpfm4, libtraceevent,
etc. In general, having the dependency will unlock more features.
Linux tools has its own copies of libbpf and libtraceevent, and so
these may pose some versioning issues.

I think it'd be great to get Debian shipping the latest version of
Linux perf for its users. Hopefully we can agree to change how Debian
packages perf currently and then work out the best way to package and
keep it up-to-date. I look forward to everyone's help and input.

Thanks in advance,
Ian Rogers

[1] https://youtu.be/aUDtN0qjxD0?t=5869

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Arnaldo Carvalho de Melo]

Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> Hi,
 
> Debian currently tries to match the Linux perf tool to the version of
> the kernel that it is being run upon. Reaching out to Ben Hutchings,
> he explained to me that this was done back in 2010 due to kernel and
> Linux perf incompatibilities. This was likely the case, but it was a
> bug in the Linux perf tool that should have been fixed. It is the goal
> of the tool to be backward compatible. A problem with matching the
> tool to the kernel version is that users miss out on new features and
> fixes (this topic came up in a recent interview of the maintainer
> Arnaldo Carvalho de Melo [1]).
 
> Ben Hutchings informs me that making it so that Debian ships the
> latest Linux perf tool requires updates both to the linux-base and
> linux source packages. The Linux perf tool also has many other often
> optional dependencies, like libunwind, libbpf, libpfm4, libtraceevent,
> etc. In general, having the dependency will unlock more features.
> Linux tools has its own copies of libbpf and libtraceevent, and so
> these may pose some versioning issues.

We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package, which
currently is going thru some growing pains as libbpf is 0.x, with
several APIs being deprecated, others renamed, and that has been a
source of friction, but should be past us with v1.0. Till then the perf
codebase is being adjusted to allow it to be seamlessly built with the
in-kernel version and with whatever libbpf-devel the distro has.
 
> I think it'd be great to get Debian shipping the latest version of
> Linux perf for its users. Hopefully we can agree to change how Debian
> packages perf currently and then work out the best way to package and
> keep it up-to-date. I look forward to everyone's help and input.

I also keep the tarballs available at:

https://mirrors.edge.kernel.org/pub/linux/kernel/tools/perf/

Where there are instructions on how to build this detached tarball.

I regularly build perf on lots of distros, including:

  debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 , clang version 3.8.1-24 (tags/RELEASE_381/final)
  debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)
  debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian clang version 11.0.1-2
  debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang version 11.1.0-4
  debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0
  debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110
  debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian 10.2.1-6) 10.2.1 20210110
  debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0

Ditto for ubuntu.

A complete list can be found in each pull request I send to Linus, see
in the last one:

  https://lore.kernel.org/all/20211107204947.1394255-1-acme@kernel.org/

Search for BUILD_TARBALL.

There is also the output for 'make -C tools/perf build-test' that tests
building with lots of combinations of optional libraries, for instance,
with LIBBPF_DYNAMIC=1, etc.

- Arnaldo

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ian Rogers]

Hi,

I'd love to package Linux perf as per Arnaldo's prepared tarball for
Debian. If I could get help on the Debian side it would be great! The
actions that are needed are:

1) package Arnaldo's tarball for Debian, presumably the build in the
sources for linux-perf-5.10 will be a good starting point. The package
will be linux-perf and replace the current metapackage of the same
name.

2) create an updated linux-base that removes /usr/bin/perf

3) upload the updated linux-base and linux-perf packages for
maintainer approval to https://mentors.debian.net/

4) some how get the previous Debian linux-perf-[45].* packages removed

After this, when Arnaldo sends out a new tarball we can update the
linux-perf package. libbpf and libtraceevent are currently part of
linux/tools, but it would be better to depend on the distribution
versions, which can be a future update to the packaging.

I'd love some feedback on this plan, whether linux-base/linux-perf
have a special process, etc. Documentation wise I've seen there is a
lot, but:
https://mentors.debian.net/intro-maintainers/
looks fairly to the point.

Thanks,
Ian



On Fri, Nov 12, 2021 at 6:39 AM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > Hi,
>
> > Debian currently tries to match the Linux perf tool to the version of
> > the kernel that it is being run upon. Reaching out to Ben Hutchings,
> > he explained to me that this was done back in 2010 due to kernel and
> > Linux perf incompatibilities. This was likely the case, but it was a
> > bug in the Linux perf tool that should have been fixed. It is the goal
> > of the tool to be backward compatible. A problem with matching the
> > tool to the kernel version is that users miss out on new features and
> > fixes (this topic came up in a recent interview of the maintainer
> > Arnaldo Carvalho de Melo [1]).
>
> > Ben Hutchings informs me that making it so that Debian ships the
> > latest Linux perf tool requires updates both to the linux-base and
> > linux source packages. The Linux perf tool also has many other often
> > optional dependencies, like libunwind, libbpf, libpfm4, libtraceevent,
> > etc. In general, having the dependency will unlock more features.
> > Linux tools has its own copies of libbpf and libtraceevent, and so
> > these may pose some versioning issues.
>
> We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package, which
> currently is going thru some growing pains as libbpf is 0.x, with
> several APIs being deprecated, others renamed, and that has been a
> source of friction, but should be past us with v1.0. Till then the perf
> codebase is being adjusted to allow it to be seamlessly built with the
> in-kernel version and with whatever libbpf-devel the distro has.
>
> > I think it'd be great to get Debian shipping the latest version of
> > Linux perf for its users. Hopefully we can agree to change how Debian
> > packages perf currently and then work out the best way to package and
> > keep it up-to-date. I look forward to everyone's help and input.
>
> I also keep the tarballs available at:
>
> https://mirrors.edge.kernel.org/pub/linux/kernel/tools/perf/
>
> Where there are instructions on how to build this detached tarball.
>
> I regularly build perf on lots of distros, including:
>
>   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 , clang version 3.8.1-24 (tags/RELEASE_381/final)
>   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)
>   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian clang version 11.0.1-2
>   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang version 11.1.0-4
>   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0
>   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110
>   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian 10.2.1-6) 10.2.1 20210110
>   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0
>
> Ditto for ubuntu.
>
> A complete list can be found in each pull request I send to Linus, see
> in the last one:
>
>   https://lore.kernel.org/all/20211107204947.1394255-1-acme@kernel.org/
>
> Search for BUILD_TARBALL.
>
> There is also the output for 'make -C tools/perf build-test' that tests
> building with lots of combinations of optional libraries, for instance,
> with LIBBPF_DYNAMIC=1, etc.
>
> - Arnaldo

RE: Getting the latest and greatest Linux perf features on every Debian kernel

[Moessbauer, Felix]

Hi,

> -----Original Message-----
> From: Ian Rogers <irogers@google.com>
> Sent: Monday, November 22, 2021 3:18 AM
> To: debian-kernel@lists.debian.org; waldi@debian.org
> Cc: linux-perf-users <linux-perf-users@vger.kernel.org>; Arnaldo Carvalho de
> Melo <acme@kernel.org>; ben@decadent.org.uk; Jiri Olsa <jolsa@kernel.org>;
> Namhyung Kim <namhyung@kernel.org>
> Subject: Re: Getting the latest and greatest Linux perf features on every Debian
> kernel
> 
> Hi,
> 
> I'd love to package Linux perf as per Arnaldo's prepared tarball for Debian. If I
> could get help on the Debian side it would be great! The actions that are needed
> are:
> 
> 1) package Arnaldo's tarball for Debian, presumably the build in the sources for
> linux-perf-5.10 will be a good starting point. The package will be linux-perf and
> replace the current metapackage of the same name.

The current packaging in Debian does not include the perf python bindings.
Today these bindings are used by more and more tools (e.g. tuned).
For that, we packaged both a versioned module (linux-perf-<...>) and a python wrapper to load it using 'import perf' (linux-base).
These patches are currently under review at Debian salsa [1].

All this would be way simpler if perf bindings do not have to be versioned anymore.
But please also consider that still a lot of people are using 4.19er kernels (don't know since when versioning is no longer required).

[1] https://salsa.debian.org/kernel-team/linux/-/merge_requests/425

> 
> 2) create an updated linux-base that removes /usr/bin/perf
> 
> 3) upload the updated linux-base and linux-perf packages for maintainer
> approval to
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> debian.net%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C
> 4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d5
> 5a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJWI
> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> 0&amp;sdata=80jfnANxTuGYzw0vErMUQOlpdUWFrDNP5n9ktaVTBaQ%3D&amp
> ;reserved=0
> 
> 4) some how get the previous Debian linux-perf-[45].* packages removed

This can easily be achieved using the Conflicts field with a linux-base version that is older than your patched one.

> 
> After this, when Arnaldo sends out a new tarball we can update the linux-perf
> package. libbpf and libtraceevent are currently part of linux/tools, but it would
> be better to depend on the distribution versions, which can be a future update to
> the packaging.
> 
> I'd love some feedback on this plan, whether linux-base/linux-perf have a special
> process, etc. Documentation wise I've seen there is a lot, but:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> debian.net%2Fintro-
> maintainers%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7
> C4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d
> 55a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJ
> WIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> 3000&amp;sdata=Y0L%2Bn%2B0yJqGVnp%2FhCfBVD9V3bZ%2Bffb48vy85jg92M
> oU%3D&amp;reserved=0
> looks fairly to the point.

I really like the plan and I'm also willing to support here.
But please also note that I'm not a Debian sponsor or maintainer of the mentioned packages.

Felix

> 
> Thanks,
> Ian
> 
> 
> 
> On Fri, Nov 12, 2021 at 6:39 AM Arnaldo Carvalho de Melo <acme@kernel.org>
> wrote:
> >
> > Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > > Hi,
> >
> > > Debian currently tries to match the Linux perf tool to the version
> > > of the kernel that it is being run upon. Reaching out to Ben
> > > Hutchings, he explained to me that this was done back in 2010 due to
> > > kernel and Linux perf incompatibilities. This was likely the case,
> > > but it was a bug in the Linux perf tool that should have been fixed.
> > > It is the goal of the tool to be backward compatible. A problem with
> > > matching the tool to the kernel version is that users miss out on
> > > new features and fixes (this topic came up in a recent interview of
> > > the maintainer Arnaldo Carvalho de Melo [1]).
> >
> > > Ben Hutchings informs me that making it so that Debian ships the
> > > latest Linux perf tool requires updates both to the linux-base and
> > > linux source packages. The Linux perf tool also has many other often
> > > optional dependencies, like libunwind, libbpf, libpfm4,
> > > libtraceevent, etc. In general, having the dependency will unlock more
> features.
> > > Linux tools has its own copies of libbpf and libtraceevent, and so
> > > these may pose some versioning issues.
> >
> > We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package,
> > which currently is going thru some growing pains as libbpf is 0.x,
> > with several APIs being deprecated, others renamed, and that has been
> > a source of friction, but should be past us with v1.0. Till then the
> > perf codebase is being adjusted to allow it to be seamlessly built
> > with the in-kernel version and with whatever libbpf-devel the distro has.
> >
> > > I think it'd be great to get Debian shipping the latest version of
> > > Linux perf for its users. Hopefully we can agree to change how
> > > Debian packages perf currently and then work out the best way to
> > > package and keep it up-to-date. I look forward to everyone's help and input.
> >
> > I also keep the tarballs available at:
> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmirr
> >
> ors.edge.kernel.org%2Fpub%2Flinux%2Fkernel%2Ftools%2Fperf%2F&amp;data=
> >
> 04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2de08d
> 9ad5
> >
> e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637731443001
> 828013%
> >
> 7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> I6Ik
> >
> 1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=A2TTg%2BElfRv9bslXw6YmEA1
> R8Iu08Z
> > hFyv6juVhRj24%3D&amp;reserved=0
> >
> > Where there are instructions on how to build this detached tarball.
> >
> > I regularly build perf on lots of distros, including:
> >
> >   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 ,
> clang version 3.8.1-24 (tags/RELEASE_381/final)
> >   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-
> 8+deb10u2 (tags/RELEASE_701/final)
> >   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian
> clang version 11.0.1-2
> >   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang
> version 11.1.0-4
> >   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9)
> 11.2.0
> >   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6)
> 10.2.1 20210110
> >   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian
> 10.2.1-6) 10.2.1 20210110
> >   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9)
> 11.2.0
> >
> > Ditto for ubuntu.
> >
> > A complete list can be found in each pull request I send to Linus, see
> > in the last one:
> >
> >
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > .kernel.org%2Fall%2F20211107204947.1394255-1-
> acme%40kernel.org%2F&amp;
> >
> data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2
> de08
> >
> d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6377314
> 4300182
> >
> 8013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> iLCJBT
> >
> iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=foKAv3e5lm5du1yoLqhzIVf
> vfc7
> > RkZqRAxmfTFkID9U%3D&amp;reserved=0
> >
> > Search for BUILD_TARBALL.
> >
> > There is also the output for 'make -C tools/perf build-test' that
> > tests building with lots of combinations of optional libraries, for
> > instance, with LIBBPF_DYNAMIC=1, etc.
> >
> > - Arnaldo

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ian Rogers]

On Mon, Nov 22, 2021 at 4:54 AM Moessbauer, Felix
<felix.moessbauer@siemens.com> wrote:
>
> Hi,
>
> > -----Original Message-----
> > From: Ian Rogers <irogers@google.com>
> > Sent: Monday, November 22, 2021 3:18 AM
> > To: debian-kernel@lists.debian.org; waldi@debian.org
> > Cc: linux-perf-users <linux-perf-users@vger.kernel.org>; Arnaldo Carvalho de
> > Melo <acme@kernel.org>; ben@decadent.org.uk; Jiri Olsa <jolsa@kernel.org>;
> > Namhyung Kim <namhyung@kernel.org>
> > Subject: Re: Getting the latest and greatest Linux perf features on every Debian
> > kernel
> >
> > Hi,
> >
> > I'd love to package Linux perf as per Arnaldo's prepared tarball for Debian. If I
> > could get help on the Debian side it would be great! The actions that are needed
> > are:
> >
> > 1) package Arnaldo's tarball for Debian, presumably the build in the sources for
> > linux-perf-5.10 will be a good starting point. The package will be linux-perf and
> > replace the current metapackage of the same name.
>
> The current packaging in Debian does not include the perf python bindings.
> Today these bindings are used by more and more tools (e.g. tuned).
> For that, we packaged both a versioned module (linux-perf-<...>) and a python wrapper to load it using 'import perf' (linux-base).
> These patches are currently under review at Debian salsa [1].
>
> All this would be way simpler if perf bindings do not have to be versioned anymore.
> But please also consider that still a lot of people are using 4.19er kernels (don't know since when versioning is no longer required).

Thanks Felix! Is there a specific bug I can look at which describes
what is broken with later versions of perf an 4.19? Perhaps we can add
a workaround for this age of kernel in the command. It is certainly
nobody's intent to have some users without a workable tool. Fwiw, I do
a lot of testing with an older 4.15 based kernel and I'm not aware of
issues.

> [1] https://salsa.debian.org/kernel-team/linux/-/merge_requests/425
>
> >
> > 2) create an updated linux-base that removes /usr/bin/perf
> >
> > 3) upload the updated linux-base and linux-perf packages for maintainer
> > approval to
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> > debian.net%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C
> > 4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d5
> > 5a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJWI
> > joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> > 0&amp;sdata=80jfnANxTuGYzw0vErMUQOlpdUWFrDNP5n9ktaVTBaQ%3D&amp
> > ;reserved=0
> >
> > 4) some how get the previous Debian linux-perf-[45].* packages removed
>
> This can easily be achieved using the Conflicts field with a linux-base version that is older than your patched one.

Can I use Conflicts to say that the linux-perf package, that will
provide /usr/bin/perf, will conflict with earlier linux-base releases?
I'm concerned that if linux-base conflicts with linux-perf and
linux-perf conflicts with linux-base, it is quite easy to make
something that becomes wedged :-)

> >
> > After this, when Arnaldo sends out a new tarball we can update the linux-perf
> > package. libbpf and libtraceevent are currently part of linux/tools, but it would
> > be better to depend on the distribution versions, which can be a future update to
> > the packaging.
> >
> > I'd love some feedback on this plan, whether linux-base/linux-perf have a special
> > process, etc. Documentation wise I've seen there is a lot, but:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> > debian.net%2Fintro-
> > maintainers%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7
> > C4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d
> > 55a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJ
> > WIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > 3000&amp;sdata=Y0L%2Bn%2B0yJqGVnp%2FhCfBVD9V3bZ%2Bffb48vy85jg92M
> > oU%3D&amp;reserved=0
> > looks fairly to the point.
>
> I really like the plan and I'm also willing to support here.
> But please also note that I'm not a Debian sponsor or maintainer of the mentioned packages.

Thanks!
Ian

> Felix
>
> >
> > Thanks,
> > Ian
> >
> >
> >
> > On Fri, Nov 12, 2021 at 6:39 AM Arnaldo Carvalho de Melo <acme@kernel.org>
> > wrote:
> > >
> > > Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > > > Hi,
> > >
> > > > Debian currently tries to match the Linux perf tool to the version
> > > > of the kernel that it is being run upon. Reaching out to Ben
> > > > Hutchings, he explained to me that this was done back in 2010 due to
> > > > kernel and Linux perf incompatibilities. This was likely the case,
> > > > but it was a bug in the Linux perf tool that should have been fixed.
> > > > It is the goal of the tool to be backward compatible. A problem with
> > > > matching the tool to the kernel version is that users miss out on
> > > > new features and fixes (this topic came up in a recent interview of
> > > > the maintainer Arnaldo Carvalho de Melo [1]).
> > >
> > > > Ben Hutchings informs me that making it so that Debian ships the
> > > > latest Linux perf tool requires updates both to the linux-base and
> > > > linux source packages. The Linux perf tool also has many other often
> > > > optional dependencies, like libunwind, libbpf, libpfm4,
> > > > libtraceevent, etc. In general, having the dependency will unlock more
> > features.
> > > > Linux tools has its own copies of libbpf and libtraceevent, and so
> > > > these may pose some versioning issues.
> > >
> > > We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package,
> > > which currently is going thru some growing pains as libbpf is 0.x,
> > > with several APIs being deprecated, others renamed, and that has been
> > > a source of friction, but should be past us with v1.0. Till then the
> > > perf codebase is being adjusted to allow it to be seamlessly built
> > > with the in-kernel version and with whatever libbpf-devel the distro has.
> > >
> > > > I think it'd be great to get Debian shipping the latest version of
> > > > Linux perf for its users. Hopefully we can agree to change how
> > > > Debian packages perf currently and then work out the best way to
> > > > package and keep it up-to-date. I look forward to everyone's help and input.
> > >
> > > I also keep the tarballs available at:
> > >
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmirr
> > >
> > ors.edge.kernel.org%2Fpub%2Flinux%2Fkernel%2Ftools%2Fperf%2F&amp;data=
> > >
> > 04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2de08d
> > 9ad5
> > >
> > e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637731443001
> > 828013%
> > >
> > 7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik
> > >
> > 1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=A2TTg%2BElfRv9bslXw6YmEA1
> > R8Iu08Z
> > > hFyv6juVhRj24%3D&amp;reserved=0
> > >
> > > Where there are instructions on how to build this detached tarball.
> > >
> > > I regularly build perf on lots of distros, including:
> > >
> > >   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 ,
> > clang version 3.8.1-24 (tags/RELEASE_381/final)
> > >   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-
> > 8+deb10u2 (tags/RELEASE_701/final)
> > >   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian
> > clang version 11.0.1-2
> > >   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang
> > version 11.1.0-4
> > >   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9)
> > 11.2.0
> > >   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6)
> > 10.2.1 20210110
> > >   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian
> > 10.2.1-6) 10.2.1 20210110
> > >   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9)
> > 11.2.0
> > >
> > > Ditto for ubuntu.
> > >
> > > A complete list can be found in each pull request I send to Linus, see
> > > in the last one:
> > >
> > >
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore
> > > .kernel.org%2Fall%2F20211107204947.1394255-1-
> > acme%40kernel.org%2F&amp;
> > >
> > data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2
> > de08
> > >
> > d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6377314
> > 4300182
> > >
> > 8013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> > iLCJBT
> > >
> > iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=foKAv3e5lm5du1yoLqhzIVf
> > vfc7
> > > RkZqRAxmfTFkID9U%3D&amp;reserved=0
> > >
> > > Search for BUILD_TARBALL.
> > >
> > > There is also the output for 'make -C tools/perf build-test' that
> > > tests building with lots of combinations of optional libraries, for
> > > instance, with LIBBPF_DYNAMIC=1, etc.
> > >
> > > - Arnaldo

RE: Getting the latest and greatest Linux perf features on every Debian kernel

[Moessbauer, Felix]

> -----Original Message-----
> From: Ian Rogers <irogers@google.com>
> Sent: Monday, November 22, 2021 5:57 PM
> To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>
> Cc: debian-kernel@lists.debian.org; waldi@debian.org; linux-perf-users <linux-
> perf-users@vger.kernel.org>; Arnaldo Carvalho de Melo <acme@kernel.org>;
> ben@decadent.org.uk; Jiri Olsa <jolsa@kernel.org>; Namhyung Kim
> <namhyung@kernel.org>; Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>;
> Schild, Henning (T RDA IOT SES-DE) <henning.schild@siemens.com>; Schmidt,
> Adriaan (T RDA IOT SES-DE) <adriaan.schmidt@siemens.com>
> Subject: Re: Getting the latest and greatest Linux perf features on every Debian
> kernel
> 
> On Mon, Nov 22, 2021 at 4:54 AM Moessbauer, Felix
> <felix.moessbauer@siemens.com> wrote:
> >
> > Hi,
> >
> > > -----Original Message-----
> > > From: Ian Rogers <irogers@google.com>
> > > Sent: Monday, November 22, 2021 3:18 AM
> > > To: debian-kernel@lists.debian.org; waldi@debian.org
> > > Cc: linux-perf-users <linux-perf-users@vger.kernel.org>; Arnaldo
> > > Carvalho de Melo <acme@kernel.org>; ben@decadent.org.uk; Jiri Olsa
> > > <jolsa@kernel.org>; Namhyung Kim <namhyung@kernel.org>
> > > Subject: Re: Getting the latest and greatest Linux perf features on
> > > every Debian kernel
> > >
> > > Hi,
> > >
> > > I'd love to package Linux perf as per Arnaldo's prepared tarball for
> > > Debian. If I could get help on the Debian side it would be great!
> > > The actions that are needed
> > > are:
> > >
> > > 1) package Arnaldo's tarball for Debian, presumably the build in the
> > > sources for
> > > linux-perf-5.10 will be a good starting point. The package will be
> > > linux-perf and replace the current metapackage of the same name.
> >
> > The current packaging in Debian does not include the perf python bindings.
> > Today these bindings are used by more and more tools (e.g. tuned).
> > For that, we packaged both a versioned module (linux-perf-<...>) and a python
> wrapper to load it using 'import perf' (linux-base).
> > These patches are currently under review at Debian salsa [1].
> >
> > All this would be way simpler if perf bindings do not have to be versioned
> anymore.
> > But please also consider that still a lot of people are using 4.19er kernels (don't
> know since when versioning is no longer required).
> 
> Thanks Felix! Is there a specific bug I can look at which describes what is broken
> with later versions of perf an 4.19? Perhaps we can add a workaround for this
> age of kernel in the command. It is certainly nobody's intent to have some users
> without a workable tool. Fwiw, I do a lot of testing with an older 4.15 based
> kernel and I'm not aware of issues.

Actually I don't know of any bugs, but just wanted to point out that this kernel should still be supported.
If that works with a modern perf out of the box, that would be even better.

> > >
> > >
> > > 2) create an updated linux-base that removes /usr/bin/perf
> > >
> > > 3) upload the updated linux-base and linux-perf packages for
> > > maintainer approval to
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> > >
> debian.net%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C
> > >
> 4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d5
> > >
> 5a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJWI
> > >
> joiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C300
> > >
> 0&amp;sdata=80jfnANxTuGYzw0vErMUQOlpdUWFrDNP5n9ktaVTBaQ%3D&amp
> > > ;reserved=0
> > >
> > > 4) some how get the previous Debian linux-perf-[45].* packages
> > > removed
> >
> > This can easily be achieved using the Conflicts field with a linux-base version
> that is older than your patched one.
> 
> Can I use Conflicts to say that the linux-perf package, that will provide
> /usr/bin/perf, will conflict with earlier linux-base releases?
> I'm concerned that if linux-base conflicts with linux-perf and linux-perf conflicts
> with linux-base, it is quite easy to make something that becomes wedged :-)

For details, have a look here: https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces
You could start with the packaging (as part of the Debian Linux source package) and create a MR at Salsa [1].
Then I can have a look over it and implement the transition / replacement.

[1] https://salsa.debian.org/kernel-team/linux

Felix
 
> 
> > >
> > > After this, when Arnaldo sends out a new tarball we can update the
> > > linux-perf package. libbpf and libtraceevent are currently part of
> > > linux/tools, but it would be better to depend on the distribution
> > > versions, which can be a future update to the packaging.
> > >
> > > I'd love some feedback on this plan, whether linux-base/linux-perf
> > > have a special process, etc. Documentation wise I've seen there is a lot, but:
> > >
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmentors.
> > > debian.net%2Fintro-
> > >
> maintainers%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7
> > >
> C4dbf17b41e4542b4c2de08d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d
> > >
> 55a%7C1%7C0%7C637731443001828013%7CUnknown%7CTWFpbGZsb3d8eyJ
> > >
> WIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
> > >
> 3000&amp;sdata=Y0L%2Bn%2B0yJqGVnp%2FhCfBVD9V3bZ%2Bffb48vy85jg92M
> > > oU%3D&amp;reserved=0
> > > looks fairly to the point.
> >
> > I really like the plan and I'm also willing to support here.
> > But please also note that I'm not a Debian sponsor or maintainer of the
> mentioned packages.
> 
> Thanks!
> Ian
> 
> > Felix
> >
> > >
> > > Thanks,
> > > Ian
> > >
> > >
> > >
> > > On Fri, Nov 12, 2021 at 6:39 AM Arnaldo Carvalho de Melo
> > > <acme@kernel.org>
> > > wrote:
> > > >
> > > > Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > > > > Hi,
> > > >
> > > > > Debian currently tries to match the Linux perf tool to the
> > > > > version of the kernel that it is being run upon. Reaching out to
> > > > > Ben Hutchings, he explained to me that this was done back in
> > > > > 2010 due to kernel and Linux perf incompatibilities. This was
> > > > > likely the case, but it was a bug in the Linux perf tool that should have
> been fixed.
> > > > > It is the goal of the tool to be backward compatible. A problem
> > > > > with matching the tool to the kernel version is that users miss
> > > > > out on new features and fixes (this topic came up in a recent
> > > > > interview of the maintainer Arnaldo Carvalho de Melo [1]).
> > > >
> > > > > Ben Hutchings informs me that making it so that Debian ships the
> > > > > latest Linux perf tool requires updates both to the linux-base
> > > > > and linux source packages. The Linux perf tool also has many
> > > > > other often optional dependencies, like libunwind, libbpf,
> > > > > libpfm4, libtraceevent, etc. In general, having the dependency
> > > > > will unlock more
> > > features.
> > > > > Linux tools has its own copies of libbpf and libtraceevent, and
> > > > > so these may pose some versioning issues.
> > > >
> > > > We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package,
> > > > which currently is going thru some growing pains as libbpf is 0.x,
> > > > with several APIs being deprecated, others renamed, and that has
> > > > been a source of friction, but should be past us with v1.0. Till
> > > > then the perf codebase is being adjusted to allow it to be
> > > > seamlessly built with the in-kernel version and with whatever libbpf-devel
> the distro has.
> > > >
> > > > > I think it'd be great to get Debian shipping the latest version
> > > > > of Linux perf for its users. Hopefully we can agree to change
> > > > > how Debian packages perf currently and then work out the best
> > > > > way to package and keep it up-to-date. I look forward to everyone's help
> and input.
> > > >
> > > > I also keep the tarballs available at:
> > > >
> > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > mirr
> > > >
> > >
> ors.edge.kernel.org%2Fpub%2Flinux%2Fkernel%2Ftools%2Fperf%2F&amp;dat
> > > a=
> > > >
> > >
> 04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2de08d
> > > 9ad5
> > > >
> > >
> e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637731443001
> > > 828013%
> > > >
> > >
> 7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > > I6Ik
> > > >
> > >
> 1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=A2TTg%2BElfRv9bslXw6YmEA1
> > > R8Iu08Z
> > > > hFyv6juVhRj24%3D&amp;reserved=0
> > > >
> > > > Where there are instructions on how to build this detached tarball.
> > > >
> > > > I regularly build perf on lots of distros, including:
> > > >
> > > >   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 ,
> > > clang version 3.8.1-24 (tags/RELEASE_381/final)
> > > >   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-
> > > 8+deb10u2 (tags/RELEASE_701/final)
> > > >   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 ,
> Debian
> > > clang version 11.0.1-2
> > > >   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian
> clang
> > > version 11.1.0-4
> > > >   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian
> 11.2.0-9)
> > > 11.2.0
> > > >   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6)
> > > 10.2.1 20210110
> > > >   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian
> > > 10.2.1-6) 10.2.1 20210110
> > > >   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-
> 9)
> > > 11.2.0
> > > >
> > > > Ditto for ubuntu.
> > > >
> > > > A complete list can be found in each pull request I send to Linus,
> > > > see in the last one:
> > > >
> > > >
> > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2F
> > > > lore
> > > > .kernel.org%2Fall%2F20211107204947.1394255-1-
> > > acme%40kernel.org%2F&amp;
> > > >
> > >
> data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C4dbf17b41e4542b4c2
> > > de08
> > > >
> > >
> d9ad5e58c3%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C6377314
> > > 4300182
> > > >
> > >
> 8013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI
> > > iLCJBT
> > > >
> > >
> iI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=foKAv3e5lm5du1yoLqhzIVf
> > > vfc7
> > > > RkZqRAxmfTFkID9U%3D&amp;reserved=0
> > > >
> > > > Search for BUILD_TARBALL.
> > > >
> > > > There is also the output for 'make -C tools/perf build-test' that
> > > > tests building with lots of combinations of optional libraries,
> > > > for instance, with LIBBPF_DYNAMIC=1, etc.
> > > >
> > > > - Arnaldo

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 4011 bytes --]

On Fri, 2021-11-12 at 11:39 -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Nov 11, 2021 at 10:34:33PM -0800, Ian Rogers escreveu:
> > Hi,
>  
> > Debian currently tries to match the Linux perf tool to the version of
> > the kernel that it is being run upon. Reaching out to Ben Hutchings,
> > he explained to me that this was done back in 2010 due to kernel and
> > Linux perf incompatibilities. This was likely the case, but it was a
> > bug in the Linux perf tool that should have been fixed.

My understanding at the time was that the perf developers weren't
trying to provide backward compatibility.  If I hadn't though that, I
wouldn't have bothered with the wrapper and would have forwarded any
bug reports we got about compatibility breaks.

> > It is the goal
> > of the tool to be backward compatible. A problem with matching the
> > tool to the kernel version is that users miss out on new features and
> > fixes (this topic came up in a recent interview of the maintainer
> > Arnaldo Carvalho de Melo [1]).
>  
> > Ben Hutchings informs me that making it so that Debian ships the
> > latest Linux perf tool requires updates both to the linux-base and
> > linux source packages. The Linux perf tool also has many other often
> > optional dependencies, like libunwind, libbpf, libpfm4, libtraceevent,
> > etc. In general, having the dependency will unlock more features.
> > Linux tools has its own copies of libbpf and libtraceevent, and so
> > these may pose some versioning issues.

The libraries themselves are statically linked, so there's no conflict
there, but the libtraceevent plugins have the potential to conflict.

> We can use LIBBPF_DYNAMIC=1 to use the distro libbpf-dev package, which
> currently is going thru some growing pains as libbpf is 0.x, with
> several APIs being deprecated, others renamed, and that has been a
> source of friction, but should be past us with v1.0. Till then the perf
> codebase is being adjusted to allow it to be seamlessly built with the
> in-kernel version and with whatever libbpf-devel the distro has.
>  
> > I think it'd be great to get Debian shipping the latest version of
> > Linux perf for its users. Hopefully we can agree to change how Debian
> > packages perf currently and then work out the best way to package and
> > keep it up-to-date. I look forward to everyone's help and input.
> 
> I also keep the tarballs available at:
> 
> https://mirrors.edge.kernel.org/pub/linux/kernel/tools/perf/
> 
> Where there are instructions on how to build this detached tarball.
> 
> I regularly build perf on lots of distros, including:
> 
>   debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 , clang version 3.8.1-24 (tags/RELEASE_381/final)
>   debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0 , clang version 7.0.1-8+deb10u2 (tags/RELEASE_701/final)
>   debian:11                     : Ok   gcc (Debian 10.2.1-6) 10.2.1 20210110 , Debian clang version 11.0.1-2
>   debian:experimental           : Ok   gcc (Debian 11.2.0-10) 11.2.0 , Debian clang version 11.1.0-4
>   debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0
>   debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 10.2.1-6) 10.2.1 20210110
>   debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian 10.2.1-6) 10.2.1 20210110
>   debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 11.2.0-9) 11.2.0
[...]

I'm glad to hear that.

But whether perf tools can be built in different environments is
orthogonal to the current questions for Debian's packaging, which are:

- Do they run correctly on older kernel version?  (I think you're
  claiming they do.)
- Is there a reason to prefer building from those separate tarballs,
  rather than from kernel releases?  (I don't think there is.)


Ben.

-- 
Ben Hutchings
I say we take off; nuke the site from orbit.
It's the only way to be sure.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1278 bytes --]

On Sun, 2021-11-21 at 18:18 -0800, Ian Rogers wrote:
> Hi,
> 
> I'd love to package Linux perf as per Arnaldo's prepared tarball for
> Debian. If I could get help on the Debian side it would be great! The
> actions that are needed are:
> 
> 1) package Arnaldo's tarball for Debian, presumably the build in the
> sources for linux-perf-5.10 will be a good starting point. The package
> will be linux-perf and replace the current metapackage of the same
> name.

I don't think this buys us anything over building from the kernel tree
(i.e. the linux source package), but will wait to see what Arnaldo
says.

> 2) create an updated linux-base that removes /usr/bin/perf
> 
> 3) upload the updated linux-base and linux-perf packages for
> maintainer approval to https://mentors.debian.net/

linux-base is maintained by the kernel team on salsa.debian.org, where
you can send merge requests.  If linux-perf is to be a separate source
package, I think it should be maintained in the same way.

> 4) some how get the previous Debian linux-perf-[45].* packages removed
[...]

They will be removed from the Debian archive through a periodic
cleanup.

Ben.

-- 
Ben Hutchings
I say we take off; nuke the site from orbit.
It's the only way to be sure.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Arnaldo Carvalho de Melo]

Em Mon, Jan 10, 2022 at 12:54:31AM +0100, Ben Hutchings escreveu:
> On Sun, 2021-11-21 at 18:18 -0800, Ian Rogers wrote:
> > Hi,
> > 
> > I'd love to package Linux perf as per Arnaldo's prepared tarball for
> > Debian. If I could get help on the Debian side it would be great! The
> > actions that are needed are:
> > 
> > 1) package Arnaldo's tarball for Debian, presumably the build in the
> > sources for linux-perf-5.10 will be a good starting point. The package
> > will be linux-perf and replace the current metapackage of the same
> > name.
> 
> I don't think this buys us anything over building from the kernel tree
> (i.e. the linux source package), but will wait to see what Arnaldo
> says.

Building perf from the kernel sources is fine, but usually the kernel
packaging takes more time to be completed, more tests, etc and if
someone wants to move the perf tools side of things faster, then using
the tarball may be an option.

Another workflow using the tarball allows is to have different packagers
for the kernel and for the perf tools.

But either way is fine, its just a way to allow more ways to package
perf, to try the latest (or an older one) on an arbitrary kernel or
system with different versions of the libraries it links to.

- Arnaldo
 
> > 2) create an updated linux-base that removes /usr/bin/perf
> > 
> > 3) upload the updated linux-base and linux-perf packages for
> > maintainer approval to https://mentors.debian.net/
> 
> linux-base is maintained by the kernel team on salsa.debian.org, where
> you can send merge requests.  If linux-perf is to be a separate source
> package, I think it should be maintained in the same way.
> 
> > 4) some how get the previous Debian linux-perf-[45].* packages removed
> [...]
> 
> They will be removed from the Debian archive through a periodic
> cleanup.
> 
> Ben.
> 
> -- 
> Ben Hutchings
> I say we take off; nuke the site from orbit.
> It's the only way to be sure.

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ian Rogers]

So I wanted to say a BIG thanks to Ben as I see in the 5.16 changelog
that the perf wrapper script for Debian is now gone!
https://metadata.ftp-master.debian.org/changelogs//main/l/linux/linux_6.1~rc8-1~exp1_changelog

```
linux (5.16.3-1~exp1) experimental; urgency=medium
...
  * linux-perf: Build a single unversioned package of perf, as it is no
    longer necessary to match the kernel version
```

I notice from trying the experimental version the features are:
```
perf version 6.1.0-rc8
                dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
   dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
                glibc: [ on  ]  # HAVE_GLIBC_SUPPORT
        syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
               libbfd: [ OFF ]  # HAVE_LIBBFD_SUPPORT
           debuginfod: [ OFF ]  # HAVE_DEBUGINFOD_SUPPORT
               libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
              libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
              libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
            libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
             libslang: [ on  ]  # HAVE_SLANG_SUPPORT
            libcrypto: [ OFF ]  # HAVE_LIBCRYPTO_SUPPORT
            libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
   libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
                 zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                 lzma: [ on  ]  # HAVE_LZMA_SUPPORT
            get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                  bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                  aio: [ on  ]  # HAVE_AIO_SUPPORT
                 zstd: [ OFF ]  # HAVE_ZSTD_SUPPORT
              libpfm4: [ OFF ]  # HAVE_LIBPFM
```
I believe this is a step up as previously libpython was disabled.

I had a few things to ask wrt this thread and getting the best
possible Debian experience:
 - in Linux 6.2 the libtraceevent will need to become an external
library dependency

 - are there issues with debuginfod/zstd enablement? libbfd has
licensing issues. libcrypto is only used by the JIT support, but still
may be nice to enable.

- wrt packaging, the kernel has a deb-pkg build target, should we move
the debian packaging to something like a perf-deb-pkg similar to the
perf-tar-src-pkg target? This thread was previously proposing using
the source releases Arnaldo makes, this would be possible for a
deb-pkg target if the logic were contained in the source tar ball. I
believe what is currently being built is from a kernel tree, but
perhaps a build target adds consistency with the kernel, allows
contributors to update the Debian related files and serves as an
example for similar packaging of other kernel tools, like bpftool.

 - are there some residual wrapper artifacts in the package, for example:
```
$ dpkg -L linux-perf
/.
/usr
/usr/bin
/usr/bin/perf
package diverts others to: /usr/bin/perf.wrapper
...
package diverts others to: /usr/share/bash-completion/completions/perf.wrapper
...
package diverts others to: /usr/share/man/man1/perf.wrapper.1.gz
...
```

Another BIG thank you to Ben!

Ian

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Andi Kleen]

Ian Rogers <irogers@google.com> writes:

> So I wanted to say a BIG thanks to Ben as I see in the 5.16 changelog
> that the perf wrapper script for Debian is now gone!
> https://metadata.ftp-master.debian.org/changelogs//main/l/linux/linux_6.1~rc8-1~exp1_changelog

Halleluja! This wrapper literally is the singlest biggest usability problem with
perf with real users. Hopefully that change gets picked up everywhere as
soon possible.

-Andi

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Moessbauer, Felix]

On Sun, 2022-12-25 at 03:51 -0800, Andi Kleen wrote:
> Ian Rogers <irogers@google.com> writes:
> 
> > So I wanted to say a BIG thanks to Ben as I see in the 5.16
> > changelog
> > that the perf wrapper script for Debian is now gone!

As already said: Thank you Ben!

> >  
> > https://metadata.ftp-master.debian.org/changelogs//main/l/linux/linux_6.1~rc8-1~exp1_changelog
> 
> Halleluja! This wrapper literally is the singlest biggest usability
> problem with
> perf with real users. Hopefully that change gets picked up everywhere
> as
> soon possible.

One important thing to notice: when using the bullseye backports
kernel, you should also use the linux-perf from backports.
This package uses a diversion to replace the perf wrapper
(/usr/bin/perf) which still is provided by linux-base. Very neat!

Long story short: Everything works properly when using the linux-perf
from backports (including the python bindings).

Felix

> 
> -Andi
> 

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1542 bytes --]

On Thu, 2022-12-22 at 17:43 -0800, Ian Rogers wrote:
[...]
> I had a few things to ask wrt this thread and getting the best
> possible Debian experience:
>  - in Linux 6.2 the libtraceevent will need to become an external
> library dependency

This shouldn't be a problem; it's already packaged separately in Debian
and we can build-depend on that.

>  - are there issues with debuginfod/zstd enablement? libbfd has
> licensing issues. libcrypto is only used by the JIT support, but still
> may be nice to enable.

- debuginfod is licensed under GPLv3+, so has the same issue as libbfd
- libzstd is licensed under GPLv2 (with most of its code also
permissively licensed)
- libcrypto is now licensed under Apache 2.0, which is not compatible
with GPLv2

So I think only libzstd can be enabled in official packages.

[...]
>  - are there some residual wrapper artifacts in the package, for example:
> ```
> $ dpkg -L linux-perf
> /.
> /usr
> /usr/bin
> /usr/bin/perf
> package diverts others to: /usr/bin/perf.wrapper
> ...
> package diverts others to: /usr/share/bash-completion/completions/perf.wrapper
> ...
> package diverts others to: /usr/share/man/man1/perf.wrapper.1.gz
> ...
> ```
[...]

These diversions are present to avoid conflicts with the wrappers in
older versions of linux-base.  We can remove them after the next stable
release.

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
                               A fail-safe circuit will destroy others.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ian Rogers]

On Mon, Dec 26, 2022 at 11:15 AM Ben Hutchings <ben@decadent.org.uk> wrote:
>
> On Thu, 2022-12-22 at 17:43 -0800, Ian Rogers wrote:
> [...]
> > I had a few things to ask wrt this thread and getting the best
> > possible Debian experience:
> >  - in Linux 6.2 the libtraceevent will need to become an external
> > library dependency
>
> This shouldn't be a problem; it's already packaged separately in Debian
> and we can build-depend on that.
>
> >  - are there issues with debuginfod/zstd enablement? libbfd has
> > licensing issues. libcrypto is only used by the JIT support, but still
> > may be nice to enable.
>
> - debuginfod is licensed under GPLv3+, so has the same issue as libbfd
> - libzstd is licensed under GPLv2 (with most of its code also
> permissively licensed)
> - libcrypto is now licensed under Apache 2.0, which is not compatible
> with GPLv2
>
> So I think only libzstd can be enabled in official packages.
>
> [...]
> >  - are there some residual wrapper artifacts in the package, for example:
> > ```
> > $ dpkg -L linux-perf
> > /.
> > /usr
> > /usr/bin
> > /usr/bin/perf
> > package diverts others to: /usr/bin/perf.wrapper
> > ...
> > package diverts others to: /usr/share/bash-completion/completions/perf.wrapper
> > ...
> > package diverts others to: /usr/share/man/man1/perf.wrapper.1.gz
> > ...
> > ```
> [...]
>
> These diversions are present to avoid conflicts with the wrappers in
> older versions of linux-base.  We can remove them after the next stable
> release.
>
> Ben.

Thanks again Ben! With the releases, the plan is to follow the kernel
releases? We could potentially be more aggressive with the perf tool
given its backward compatibility. That would benefit people using
older kernels who want the latest perf tool bug fixes and features.

Ian

> --
> Ben Hutchings
> Klipstein's 4th Law of Prototyping and Production:
>                                A fail-safe circuit will destroy others.

Re: Getting the latest and greatest Linux perf features on every Debian kernel

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

On Mon, 2023-01-09 at 17:32 -0800, Ian Rogers wrote:
[...]
> Thanks again Ben! With the releases, the plan is to follow the kernel
> releases? We could potentially be more aggressive with the perf tool
> given its backward compatibility. That would benefit people using
> older kernels who want the latest perf tool bug fixes and features.
> 

Yes, we continue to build perf from the same source package as our
kernel images.  Users of a stable release can still install newer
versions of perf and/or the kernel from the corresponding -backports
suite without upgrading everything elese.

Ben.

-- 
Ben Hutchings
The program is absolutely right; therefore, the computer must be wrong.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]