Re: [PATCH bpf-next] kbuild: add ability to generate BTF type info for vmlinux

From: Arnaldo Carvalho de Melo <acme@redhat.com>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>,
	andrii.nakryiko@gmail.com, bpf@vger.kernel.org,
	netdev@vger.kernel.org, kernel-team@fb.com,
	Andrii Nakryiko <andriin@fb.com>,
	Masahiro Yamada <yamada.masahiro@socionext.com>,
	Alexei Starovoitov <ast@fb.com>, Yonghong Song <yhs@fb.com>,
	Martin KaFai Lau <kafai@fb.com>
Subject: Re: [PATCH bpf-next] kbuild: add ability to generate BTF type info for vmlinux
Date: Wed, 3 Apr 2019 09:05:06 -0300	[thread overview]
Message-ID: <20190403120506.GK2219@redhat.com> (raw)
In-Reply-To: <20190403091258.GA24875@krava>

Em Wed, Apr 03, 2019 at 11:12:58AM +0200, Jiri Olsa escreveu:
> On Wed, Apr 03, 2019 at 11:04:28AM +0200, Daniel Borkmann wrote:
> > On 04/03/2019 10:46 AM, Jiri Olsa wrote:
> > > On Tue, Apr 02, 2019 at 09:49:50AM -0700, andrii.nakryiko@gmail.com wrote:
> > >> From: Andrii Nakryiko <andriin@fb.com>
> > >>
> > >> This patch adds new config option to trigger generation of BTF type
> > >> information from DWARF debuginfo for vmlinux and kernel modules through
> > >> pahole, which in turn relies on libbpf for btf_dedup() algorithm.
> > >>
> > >> The intent is to record compact type information of all types used
> > >> inside kernel, including all the structs/unions/typedefs/etc. This
> > >> enables BPF's compile-once-run-everywhere ([0]) approach, in which
> > >> tracing programs that are inspecting kernel's internal data (e.g.,
> > >> struct task_struct) can be compiled on a system running some kernel
> > >> version, but would be possible to run on other kernel versions (and
> > >> configurations) without recompilation, even if the layout of structs
> > >> changed and/or some of the fields were added, removed, or renamed.
> > >>
> > >> This is only possible if BPF loader can get kernel type info to adjust
> > >> all the offsets correctly. This patch is a first time in this direction,
> > >> making sure that BTF type info is part of Linux kernel image in
> > >> non-loadable ELF section.
> > >>
> > >> BTF deduplication ([1]) algorithm typically provides 100x savings
> > >> compared to DWARF data, so resulting .BTF section is not big as is
> > >> typically about 2MB in size.
> > > 
> > > hi,
> > > I'm using the latest pahole from git tree:
> > >   https://github.com/acmel/dwarves.git
> > > 
> > > and getting pahole crash:
> > > 
> > >   LD      vmlinux
> > >   BTF     vmlinux
> > > die__process_inline_expansion: DW_TAG_label (0xa) @ <0x3eef8> not handled!
> > > scripts/link-vmlinux.sh: line 96: 31222 Segmentation fault      (core dumped) LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
> > > make[1]: *** [/home/jolsa/linux/Makefile:1025: vmlinux] Error 139
> > > make: *** [Makefile:170: sub-make] Error 2
> > > 
> > > is there some other source/dependency I'm missing?
> > 
> > Yesterday night, I've tested with [0] but seems to have the same HEAD as
> > the github repo you pointed out. Seems the above is coming from pahole's
> > __die__process_tag() bailing out with default for DW_TAG_label?
> > 
> > On my side worked fine:
> 
> hum, I also had to change the version of pahole in the patch to allow
> version v1.12, because both latest pahole sources are on version 1.12,
> did u have to do that? looks like there's v1.13 somewhere ;-)

I'll get the --reorganize algorithm simplified to cope with some
internal changes needed for the BTF encoding/decoding and tag v1.13
later today, max late this week. Then I'll go back adding the bitfield
reorganization bits that will take a bit more work.

Hopefully this way we get something to work with the Kbuild changes and
have progress in that front.

I'll look at the DW_TAG_label case too, please send me me in PVT the
vmlinux file with that problem so that I can analyse it.

> jirka
> 
> SNIP
> 
> > 
> >   [0] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/