* pahole and libdwarves linking/versioning issues @ 2020-11-11 0:14 Andrii Nakryiko 2020-11-11 0:28 ` Arnaldo 2020-11-13 11:45 ` Arnaldo Carvalho de Melo 0 siblings, 2 replies; 4+ messages in thread From: Andrii Nakryiko @ 2020-11-11 0:14 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, dwarves Cc: Alexei Starovoitov, Kernel Team, Roman Gushchin Hey Arnaldo, We use pahole pretty extensively here at Facebook, as you might imagine, given it's a required tool to compile Linux kernel with BTF. There are a few problems we periodically run into, all of which seems to be related to the fact that pahole is just a thin wrapper binary on top of libdwarves shared library. One very common problem is when we upgrade the dwarves package from one minor version to another one. Typically people would do yum upgrade dwarves And that would succeed to update the dwarves package (say 1.13 to 1.16). But when you run pahole --version You'll still see 1.13 (and still have 1.13 functionality, of course)... So what happens is that pahole *the binary* was updated, but libdwarves1 package, which is a dependency of dwarves package, wasn't updated. And the actual version string all pretty much all of the functionality is coming from libdwarves1. So users have to yum upgrade libdwarves1 explicitly. Which is, of course, not an obvious and frustrating experience. Also, whenever pahole is using some new API from libdwarves and libdwarves is outdated, that results in spectacular failure during dynamic linking time. Which makes sense because libdwarves doesn't maintain API stability per se and is supposed to match 1:1 w/ pahole version. But it all causes real logistics issues. So I have a few questions: 1. Is there anything on the RPM spec side that could be done to trigger libdwarves1 update when dwarves is updated and keep them in sync down to the minor version? 2. If not, can we add an option of statically linking libdwarves into pahole and other tools? That would eliminate all the problems altogether at the expense of a pretty negligible slight increase in the binary sizes. Thanks! -- Andrii ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: pahole and libdwarves linking/versioning issues 2020-11-11 0:14 pahole and libdwarves linking/versioning issues Andrii Nakryiko @ 2020-11-11 0:28 ` Arnaldo 2020-11-13 11:45 ` Arnaldo Carvalho de Melo 1 sibling, 0 replies; 4+ messages in thread From: Arnaldo @ 2020-11-11 0:28 UTC (permalink / raw) To: Andrii Nakryiko, Arnaldo Carvalho de Melo, dwarves Cc: Alexei Starovoitov, Kernel Team, Roman Gushchin On November 10, 2020 9:14:16 PM GMT-03:00, Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote: >Hey Arnaldo, > >We use pahole pretty extensively here at Facebook, as you might >imagine, given it's a required tool to compile Linux kernel with BTF. > >There are a few problems we periodically run into, all of which seems >to be related to the fact that pahole is just a thin wrapper binary on >top of libdwarves shared library. > >One very common problem is when we upgrade the dwarves package from >one minor version to another one. Typically people would do > >yum upgrade dwarves I'll reply to this more thoroughly tomorrow morning. But yes, the spec file um the pahole git repo has a explicit Requires: tag to ask for the latest libdwarves. - Arnaldo > >And that would succeed to update the dwarves package (say 1.13 to >1.16). But when you run > >pahole --version > >You'll still see 1.13 (and still have 1.13 functionality, of -- Sent from my Android device with K-9 Mail. Please excuse my brevity. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: pahole and libdwarves linking/versioning issues 2020-11-11 0:14 pahole and libdwarves linking/versioning issues Andrii Nakryiko 2020-11-11 0:28 ` Arnaldo @ 2020-11-13 11:45 ` Arnaldo Carvalho de Melo 2020-12-04 23:11 ` Andrii Nakryiko 1 sibling, 1 reply; 4+ messages in thread From: Arnaldo Carvalho de Melo @ 2020-11-13 11:45 UTC (permalink / raw) To: Andrii Nakryiko; +Cc: dwarves, Alexei Starovoitov, Kernel Team, Roman Gushchin Em Tue, Nov 10, 2020 at 04:14:16PM -0800, Andrii Nakryiko escreveu: > Hey Arnaldo, Hi Andrii, > We use pahole pretty extensively here at Facebook, as you might > imagine, given it's a required tool to compile Linux kernel with BTF. > There are a few problems we periodically run into, all of which seems > to be related to the fact that pahole is just a thin wrapper binary on > top of libdwarves shared library. > One very common problem is when we upgrade the dwarves package from > one minor version to another one. Typically people would do > yum upgrade dwarves > And that would succeed to update the dwarves package (say 1.13 to > 1.16). But when you run > pahole --version > You'll still see 1.13 (and still have 1.13 functionality, of > course)... So what happens is that pahole *the binary* was updated, > but libdwarves1 package, which is a dependency of dwarves package, > wasn't updated. And the actual version string all pretty much all of > the functionality is coming from libdwarves1. So users have to > yum upgrade libdwarves1 > explicitly. Which is, of course, not an obvious and frustrating experience. > Also, whenever pahole is using some new API from libdwarves and > libdwarves is outdated, that results in spectacular failure during > dynamic linking time. Which makes sense because libdwarves doesn't > maintain API stability per se and is supposed to match 1:1 w/ pahole > version. But it all causes real logistics issues. > So I have a few questions: > 1. Is there anything on the RPM spec side that could be done to > trigger libdwarves1 update when dwarves is updated and keep them in > sync down to the minor version? So, there is this in the reference .spec file that is in pahole's git repo: [acme@five pahole]$ egrep '^(Requires|Name)' rpm/SPECS/dwarves.spec Name: dwarves Requires: %{libname}%{libver} = %{version}-%{release} Requires: %{libname}%{libver} = %{version}-%{release} [acme@five pahole]$ egrep '^(Requires|Name|%package)' rpm/SPECS/dwarves.spec Name: dwarves Requires: %{libname}%{libver} = %{version}-%{release} %package -n %{libname}%{libver} %package -n %{libname}%{libver}-devel Requires: %{libname}%{libver} = %{version}-%{release} [acme@five pahole]$ Someone else also reported this recently... Fedora has: [acme@five pahole]$ rpm -qR dwarves | grep ^libdwarves1 libdwarves1 = 1.17-4.fc33 [acme@five pahole]$ So it includes both ${version} and %{release}, which should get it in lockstep. Having said that I'm not aware of anything using libdwarves and it was intended just to share those among the tools that comes in the 'dwarves' package, so I should have really made it internal, installed in /usr/libexec/dwarves/ :-\ > 2. If not, can we add an option of statically linking libdwarves into > pahole and other tools? That would eliminate all the problems > altogether at the expense of a pretty negligible slight increase in > the binary sizes. Yeah, this looks like the way to proceed, i.e. we statically link it while keeping the .so around in case someone uses it, while adding some warning that that mode is deprecated and is going away. - Arnaldo ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: pahole and libdwarves linking/versioning issues 2020-11-13 11:45 ` Arnaldo Carvalho de Melo @ 2020-12-04 23:11 ` Andrii Nakryiko 0 siblings, 0 replies; 4+ messages in thread From: Andrii Nakryiko @ 2020-12-04 23:11 UTC (permalink / raw) To: Arnaldo Carvalho de Melo, Matt Mullins Cc: dwarves, Alexei Starovoitov, Kernel Team, Roman Gushchin On Fri, Nov 13, 2020 at 3:45 AM Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote: > > Em Tue, Nov 10, 2020 at 04:14:16PM -0800, Andrii Nakryiko escreveu: > > Hey Arnaldo, > > Hi Andrii, > > > We use pahole pretty extensively here at Facebook, as you might > > imagine, given it's a required tool to compile Linux kernel with BTF. > > > There are a few problems we periodically run into, all of which seems > > to be related to the fact that pahole is just a thin wrapper binary on > > top of libdwarves shared library. > > > One very common problem is when we upgrade the dwarves package from > > one minor version to another one. Typically people would do > > > yum upgrade dwarves > > > And that would succeed to update the dwarves package (say 1.13 to > > 1.16). But when you run > > > pahole --version > > > You'll still see 1.13 (and still have 1.13 functionality, of > > course)... So what happens is that pahole *the binary* was updated, > > but libdwarves1 package, which is a dependency of dwarves package, > > wasn't updated. And the actual version string all pretty much all of > > the functionality is coming from libdwarves1. So users have to > > > yum upgrade libdwarves1 > > > explicitly. Which is, of course, not an obvious and frustrating experience. > > > Also, whenever pahole is using some new API from libdwarves and > > libdwarves is outdated, that results in spectacular failure during > > dynamic linking time. Which makes sense because libdwarves doesn't > > maintain API stability per se and is supposed to match 1:1 w/ pahole > > version. But it all causes real logistics issues. > > > So I have a few questions: > > 1. Is there anything on the RPM spec side that could be done to > > trigger libdwarves1 update when dwarves is updated and keep them in > > sync down to the minor version? > > So, there is this in the reference .spec file that is in pahole's git > repo: > > [acme@five pahole]$ egrep '^(Requires|Name)' rpm/SPECS/dwarves.spec > Name: dwarves > Requires: %{libname}%{libver} = %{version}-%{release} > Requires: %{libname}%{libver} = %{version}-%{release} > [acme@five pahole]$ egrep '^(Requires|Name|%package)' rpm/SPECS/dwarves.spec > Name: dwarves > Requires: %{libname}%{libver} = %{version}-%{release} > %package -n %{libname}%{libver} > %package -n %{libname}%{libver}-devel > Requires: %{libname}%{libver} = %{version}-%{release} Ok, so I would have never noticed it, but Matt Mullins has an eagle eye! This Requirement applies only to libdwarves1-devel package! %package -n %{libname}%{libver} doesn't have this requirement! Arnaldo, if you can roll it into 1.19 along the changelog timestamp fix, that would be awesome! > [acme@five pahole]$ > > Someone else also reported this recently... Fedora has: > > [acme@five pahole]$ rpm -qR dwarves | grep ^libdwarves1 > libdwarves1 = 1.17-4.fc33 > [acme@five pahole]$ > > So it includes both ${version} and %{release}, which should get it in > lockstep. > > Having said that I'm not aware of anything using libdwarves and it was > intended just to share those among the tools that comes in the > 'dwarves' package, so I should have really made it internal, installed > in /usr/libexec/dwarves/ :-\ > > > 2. If not, can we add an option of statically linking libdwarves into > > pahole and other tools? That would eliminate all the problems > > altogether at the expense of a pretty negligible slight increase in > > the binary sizes. > > Yeah, this looks like the way to proceed, i.e. we statically link it > while keeping the .so around in case someone uses it, while adding some > warning that that mode is deprecated and is going away. > > - Arnaldo ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-12-04 23:11 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-11 0:14 pahole and libdwarves linking/versioning issues Andrii Nakryiko 2020-11-11 0:28 ` Arnaldo 2020-11-13 11:45 ` Arnaldo Carvalho de Melo 2020-12-04 23:11 ` Andrii Nakryiko
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).