Dwarves Archive on lore.kernel.org
 help / color / Atom feed
From: Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
Cc: Arnaldo Carvalho de Melo
	<arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Alexei Starovoitov
	<alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Daniel Borkmann <daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org>,
	Oleg Rombakh <olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Martin Lau <kafai-b10kYP2dOMg@public.gmane.org>,
	dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH v4] btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.
Date: Mon, 29 Jun 2020 13:25:52 -0700
Message-ID: <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA@mail.gmail.com> (raw)
In-Reply-To: <CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Fri, Jun 19, 2020 at 1:30 PM Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
>
> Hi, Andrii,
>
> I agree that we'd better put a hold on this patch until we find out
> the reason for the 'unconventional' symbols. I'll _try_ to figure it
> out, but not able to fully commit my time on this patch. I thought I'd
> better publish this patch as the DATASEC and VARs are generated
> correctly in format, so that anyone can use it to generate the vmlinux
> and continue the development on libbpf based on your ksym work (i.e.
> typed ksyms).
>

Hey Hao,

It's a pity that you are willing to drop this while being half-step
away from doing this properly. See my other reply, there is something
fishy with variable__name() and variable__type_size() and how it
calculates/caches values. ELF itself has all the data (based ELF
symbol data) and it seems to be correct. And I was correct to be
suspicious about just filtering out such variables, because in my case
you'd filter out a good chunk of variables for no good reason: 74
variables, which is a 26% of all per-CPU variables in my kernel.

It's up to you, of course, but it would be nice to fix it up (just
switching to elf_sym__size() and elf_sym__name() would be fine, you'd
just need to make sure to add elf_sym_name() result into a string
buffer; unless Arnaldo has some better alternatives) and publish
complete v5, that would get merged into pahole. After that, building
on my .ksym work in libbpf should get you to what you need from BPF
pretty quickly (plus some kernel-side logic to recognize these per-CPU
variables).

> Hao
>
> On Fri, Jun 19, 2020 at 12:58 PM Andrii Nakryiko
> <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> >
> > On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> > >
> > > On SMP systems, the global percpu variables are placed in a special
> > > '.data..percpu' section, which is stored in a segment whose initial
> > > address is set to 0, the addresses of per-CPU variables are relative
> > > positive addresses [1].
> > >
> > > This patch extracts these variables from vmlinux and places them with
> > > their type information in BTF. More specifically, when BTF is encoded,
> > > we find the index of the '.data..percpu' section and then traverse
> > > the symbol table to find those global objects which are in this section.
> > > For each of these objects, we push a BTF_KIND_VAR into the types buffer,
> > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the
> > > CUs have finished processing, we push a BTF_KIND_DATASEC into the
> > > btfe->types buffer, followed by the percpu_secinfo's content.
> > >
> > > In a v5.7-rc7 linux kernel, I was able to extract 291 such variables.
> > > The build time overhead is small and the space overhead is also small.
> > >
> > > Testing:
> > >
> > > - vmlinux size has increased by ~12kb.
> > >   Before:
> > >    $ readelf -SW vmlinux | grep BTF
> > >    [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
> > >   After:
> > >    $ pahole -J vmlinux
> > >    $ readelf -SW vmlinux  | grep BTF
> > >    [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00
> > >
> > > - Common global percpu VARs and DATASEC are found in BTF section.
> > >   $ bpftool btf dump file vmlinux | grep runqueues
> > >   [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
> > >
> > >   $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
> > >   [17592] STRUCT 'cpu_stopper' size=72 vlen=5
> > >   [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
> > >
> > >   $ bpftool btf dump file vmlinux | grep ' DATASEC '
> > >   [63652] DATASEC '.data..percpu' size=0 vlen=294
> >
> > probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?
> >

[...]

  parent reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-18  7:48 Hao Luo
     [not found] ` <20200618074853.133675-1-haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2020-06-19 19:58   ` Andrii Nakryiko
     [not found]     ` <CAEf4BzbE2K7uyJ-2d3N+NLrV5OKS0HMm5vKuxMz5bfZkE_ANzw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-19 20:30       ` Hao Luo
     [not found]         ` <CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 20:25           ` Andrii Nakryiko [this message]
     [not found]             ` <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 21:13               ` Hao Luo
     [not found]                 ` <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30  1:02                   ` Arnaldo Carvalho de Melo
2020-06-29 20:20       ` Andrii Nakryiko
     [not found]         ` <CAEf4Bzbiu8G5UK9TDBRFYGwVk4A4QWoLurZsFKhs439gBXaDCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30 11:59           ` Arnaldo Carvalho de Melo
     [not found]             ` <20200630115927.GL29008-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-07-03  8:13               ` Hao Luo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA@mail.gmail.com \
    --to=andrii.nakryiko-re5jqeeqqe8avxtiumwx3w@public.gmane.org \
    --cc=alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org \
    --cc=dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=kafai-b10kYP2dOMg@public.gmane.org \
    --cc=olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Dwarves Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dwarves/0 dwarves/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dwarves dwarves/ https://lore.kernel.org/dwarves \
		dwarves@vger.kernel.org
	public-inbox-index dwarves

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dwarves


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git