Re: [PATCH] libbpf: fix readelf output parsing on powerpc with recent binutils

From: Justin Forbes <jmforbes@linuxtx.org>
To: Thadeu Lima de Souza Cascardo <cascardo@canonical.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>,
	Song Liu <songliubraving@fb.com>,
	Andrii Nakryiko <andriin@fb.com>,
	Alexei Starovoitov <ast@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	"open list:BPF (Safe dynamic programs and tools)" 
	<netdev@vger.kernel.org>, Yonghong Song <yhs@fb.com>,
	"open list:BPF (Safe dynamic programs and tools)" 
	<bpf@vger.kernel.org>,
	linuxppc-dev@lists.ozlabs.org, Martin KaFai Lau <kafai@fb.com>,
	Aurelien Jarno <aurelien@aurel32.net>,
	debian-kernel@lists.debian.org
Subject: Re: [PATCH] libbpf: fix readelf output parsing on powerpc with recent binutils
Date: Wed, 11 Dec 2019 10:52:46 -0600	[thread overview]
Message-ID: <CAFxkdAp9OGjJS1Sdny+TiG2+zU4n0Nj+ZVrZt5J6iVsS_zqqcw@mail.gmail.com> (raw)
In-Reply-To: <20191211160133.GB4580@calabresa>

On Wed, Dec 11, 2019 at 10:01 AM Thadeu Lima de Souza Cascardo
<cascardo@canonical.com> wrote:
>
> On Wed, Dec 11, 2019 at 09:33:53AM -0600, Justin Forbes wrote:
> > On Tue, Dec 10, 2019 at 4:26 PM Thadeu Lima de Souza Cascardo
> > <cascardo@canonical.com> wrote:
> > >
> > > On Tue, Dec 10, 2019 at 12:58:33PM -0600, Justin Forbes wrote:
> > > > On Mon, Dec 2, 2019 at 3:37 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> > > > >
> > > > > On Mon, Dec 02, 2019 at 04:53:26PM +1100, Michael Ellerman wrote:
> > > > > > Aurelien Jarno <aurelien@aurel32.net> writes:
> > > > > > > On powerpc with recent versions of binutils, readelf outputs an extra
> > > > > > > field when dumping the symbols of an object file. For example:
> > > > > > >
> > > > > > >     35: 0000000000000838    96 FUNC    LOCAL  DEFAULT [<localentry>: 8]     1 btf_is_struct
> > > > > > >
> > > > > > > The extra "[<localentry>: 8]" prevents the GLOBAL_SYM_COUNT variable to
> > > > > > > be computed correctly and causes the checkabi target to fail.
> > > > > > >
> > > > > > > Fix that by looking for the symbol name in the last field instead of the
> > > > > > > 8th one. This way it should also cope with future extra fields.
> > > > > > >
> > > > > > > Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
> > > > > > > ---
> > > > > > >  tools/lib/bpf/Makefile | 4 ++--
> > > > > > >  1 file changed, 2 insertions(+), 2 deletions(-)
> > > > > >
> > > > > > Thanks for fixing that, it's been on my very long list of test failures
> > > > > > for a while.
> > > > > >
> > > > > > Tested-by: Michael Ellerman <mpe@ellerman.id.au>
> > > > >
> > > > > Looks good & also continues to work on x86. Applied, thanks!
> > > >
> > > > This actually seems to break horribly on PPC64le with binutils 2.33.1
> > > > resulting in:
> > > > Warning: Num of global symbols in sharedobjs/libbpf-in.o (32) does NOT
> > > > match with num of versioned symbols in libbpf.so (184). Please make
> > > > sure all LIBBPF_API symbols are versioned in libbpf.map.
> > > >
> > > > This is the only arch that fails, with x86/arm/aarch64/s390 all
> > > > building fine.  Reverting this patch allows successful build across
> > > > all arches.
> > > >
> > > > Justin
> > >
> > > Well, I ended up debugging this same issue and had the same fix as Jarno's when
> > > I noticed his fix was already applied.
> > >
> > > I just installed a system with the latest binutils, 2.33.1, and it still breaks
> > > without such fix. Can you tell what is the output of the following command on
> > > your system?
> > >
> > > readelf -s --wide tools/lib/bpf/sharedobjs/libbpf-in.o | cut -d "@" -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' | awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $0}'
> > >
> >
> > readelf -s --wide tools/lib/bpf/sharedobjs/libbpf-in.o | cut -d "@"
> > -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' | awk '/GLOBAL/ && /DEFAULT/ &&
> > !/UND/ {print $0}'
> >    373: 00000000000141bc  1376 FUNC    GLOBAL DEFAULT    1
> > libbpf_num_possible_cpus [<localentry>: 8]
> >    375: 000000000001869c   176 FUNC    GLOBAL DEFAULT    1 btf__free
> > [<localentry>: 8]
> [...]
>
> This is a patch on binutils carried by Fedora:
>
> https://src.fedoraproject.org/rpms/binutils/c/b8265c46f7ddae23a792ee8306fbaaeacba83bf8
>
> " b8265c Have readelf display extra symbol information at the end of the line. "
>
> It has the following comment:
>
> # FIXME:    The proper fix would be to update the scripts that are expecting
> #           a fixed output from readelf.  But it seems that some of them are
> #           no longer being maintained.
>
> This commit is from 2017, had it been on binutils upstream, maybe the situation
> right now would be different.
>
> Honestly, it seems the best way out is to filter the other information in the
> libbpf Makefile.
>
> Does the following patch work for you?
>
>
> diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile
> index 56ce6292071b..e6f99484d7d5 100644
> --- a/tools/lib/bpf/Makefile
> +++ b/tools/lib/bpf/Makefile
> @@ -145,6 +145,7 @@ PC_FILE             := $(addprefix $(OUTPUT),$(PC_FILE))
>
>  GLOBAL_SYM_COUNT = $(shell readelf -s --wide $(BPF_IN_SHARED) | \
>                            cut -d "@" -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' | \
> +                          sed 's/\[.*\]//' | \
>                            awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$8}' | \
>                            sort -u | wc -l)
>  VERSIONED_SYM_COUNT = $(shell readelf -s --wide $(OUTPUT)libbpf.so | \
> @@ -217,6 +218,7 @@ check_abi: $(OUTPUT)libbpf.so
>                      "versioned in $(VERSION_SCRIPT)." >&2;              \
>                 readelf -s --wide $(OUTPUT)libbpf-in.o |                 \
>                     cut -d "@" -f1 | sed 's/_v[0-9]_[0-9]_[0-9].*//' |   \
> +                   sed 's/\[.*\]//' |                                   \
>                     awk '/GLOBAL/ && /DEFAULT/ && !/UND/ {print $$8}'|   \
>                     sort -u > $(OUTPUT)libbpf_global_syms.tmp;           \
>                 readelf -s --wide $(OUTPUT)libbpf.so |                   \

This patch was against the older version, but when updated for current
5.5-rc1, it does indeed work for me.

Thanks,
Justin