Dwarves Archive on lore.kernel.org
 help / color / Atom feed
From: Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
To: Andrii Nakryiko
	<andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Arnaldo Carvalho de Melo
	<arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Alexei Starovoitov
	<alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Daniel Borkmann <daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org>,
	Oleg Rombakh <olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Martin Lau <kafai-b10kYP2dOMg@public.gmane.org>,
	dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH v4] btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.
Date: Mon, 29 Jun 2020 14:13:20 -0700
Message-ID: <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ@mail.gmail.com> (raw)
In-Reply-To: <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

Hi Andrii,

Thanks for taking a look! I can post a v5 based on your thorough and
detailed analysis this week. Some of my other work on sched consumed
most of my bandwidth last week and I very appreciate your help! I
apologize if it is felt I dropped this patch and will follow up within
this week.

Hao

On Mon, Jun 29, 2020 at 1:26 PM Andrii Nakryiko
<andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> On Fri, Jun 19, 2020 at 1:30 PM Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> >
> > Hi, Andrii,
> >
> > I agree that we'd better put a hold on this patch until we find out
> > the reason for the 'unconventional' symbols. I'll _try_ to figure it
> > out, but not able to fully commit my time on this patch. I thought I'd
> > better publish this patch as the DATASEC and VARs are generated
> > correctly in format, so that anyone can use it to generate the vmlinux
> > and continue the development on libbpf based on your ksym work (i.e.
> > typed ksyms).
> >
>
> Hey Hao,
>
> It's a pity that you are willing to drop this while being half-step
> away from doing this properly. See my other reply, there is something
> fishy with variable__name() and variable__type_size() and how it
> calculates/caches values. ELF itself has all the data (based ELF
> symbol data) and it seems to be correct. And I was correct to be
> suspicious about just filtering out such variables, because in my case
> you'd filter out a good chunk of variables for no good reason: 74
> variables, which is a 26% of all per-CPU variables in my kernel.
>
> It's up to you, of course, but it would be nice to fix it up (just
> switching to elf_sym__size() and elf_sym__name() would be fine, you'd
> just need to make sure to add elf_sym_name() result into a string
> buffer; unless Arnaldo has some better alternatives) and publish
> complete v5, that would get merged into pahole. After that, building
> on my .ksym work in libbpf should get you to what you need from BPF
> pretty quickly (plus some kernel-side logic to recognize these per-CPU
> variables).
>
> > Hao
> >
> > On Fri, Jun 19, 2020 at 12:58 PM Andrii Nakryiko
> > <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > >
> > > On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> > > >
> > > > On SMP systems, the global percpu variables are placed in a special
> > > > '.data..percpu' section, which is stored in a segment whose initial
> > > > address is set to 0, the addresses of per-CPU variables are relative
> > > > positive addresses [1].
> > > >
> > > > This patch extracts these variables from vmlinux and places them with
> > > > their type information in BTF. More specifically, when BTF is encoded,
> > > > we find the index of the '.data..percpu' section and then traverse
> > > > the symbol table to find those global objects which are in this section.
> > > > For each of these objects, we push a BTF_KIND_VAR into the types buffer,
> > > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the
> > > > CUs have finished processing, we push a BTF_KIND_DATASEC into the
> > > > btfe->types buffer, followed by the percpu_secinfo's content.
> > > >
> > > > In a v5.7-rc7 linux kernel, I was able to extract 291 such variables.
> > > > The build time overhead is small and the space overhead is also small.
> > > >
> > > > Testing:
> > > >
> > > > - vmlinux size has increased by ~12kb.
> > > >   Before:
> > > >    $ readelf -SW vmlinux | grep BTF
> > > >    [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
> > > >   After:
> > > >    $ pahole -J vmlinux
> > > >    $ readelf -SW vmlinux  | grep BTF
> > > >    [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00
> > > >
> > > > - Common global percpu VARs and DATASEC are found in BTF section.
> > > >   $ bpftool btf dump file vmlinux | grep runqueues
> > > >   [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
> > > >
> > > >   $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
> > > >   [17592] STRUCT 'cpu_stopper' size=72 vlen=5
> > > >   [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
> > > >
> > > >   $ bpftool btf dump file vmlinux | grep ' DATASEC '
> > > >   [63652] DATASEC '.data..percpu' size=0 vlen=294
> > >
> > > probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?
> > >
>
> [...]

  parent reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-18  7:48 Hao Luo
     [not found] ` <20200618074853.133675-1-haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2020-06-19 19:58   ` Andrii Nakryiko
     [not found]     ` <CAEf4BzbE2K7uyJ-2d3N+NLrV5OKS0HMm5vKuxMz5bfZkE_ANzw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-19 20:30       ` Hao Luo
     [not found]         ` <CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 20:25           ` Andrii Nakryiko
     [not found]             ` <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 21:13               ` Hao Luo [this message]
     [not found]                 ` <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30  1:02                   ` Arnaldo Carvalho de Melo
2020-06-29 20:20       ` Andrii Nakryiko
     [not found]         ` <CAEf4Bzbiu8G5UK9TDBRFYGwVk4A4QWoLurZsFKhs439gBXaDCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30 11:59           ` Arnaldo Carvalho de Melo
     [not found]             ` <20200630115927.GL29008-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-07-03  8:13               ` Hao Luo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ@mail.gmail.com \
    --to=haoluo-hpiqsd4aklfqt0dzr+alfa@public.gmane.org \
    --cc=alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org \
    --cc=dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=kafai-b10kYP2dOMg@public.gmane.org \
    --cc=olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Dwarves Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dwarves/0 dwarves/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dwarves dwarves/ https://lore.kernel.org/dwarves \
		dwarves@vger.kernel.org
	public-inbox-index dwarves

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dwarves


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git