Dwarves Archive on lore.kernel.org
 help / color / Atom feed
From: Arnaldo Carvalho de Melo <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
To: Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Andrii Nakryiko
	<andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Cc: Alexei Starovoitov
	<alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Daniel Borkmann <daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org>,
	Oleg Rombakh <olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Martin Lau <kafai-b10kYP2dOMg@public.gmane.org>,
	dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH v4] btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.
Date: Mon, 29 Jun 2020 22:02:31 -0300
Message-ID: <5839EE93-F4F3-4210-9095-13CADC1822E8@gmail.com> (raw)
In-Reply-To: <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>



On June 29, 2020 6:13:20 PM GMT-03:00, Hao Luo <haoluo@google.com> wrote:
>Hi Andrii,
>
>Thanks for taking a look! I can post a v5 based on your thorough and
>detailed analysis this week. Some of my other work on sched consumed
>most of my bandwidth last week and I very appreciate your help! I
>apologize if it is felt I dropped this patch and will follow up within
>this week.

I'll also try and look at this tomorrow.

- Arnaldo

>
>Hao
>
>On Mon, Jun 29, 2020 at 1:26 PM Andrii Nakryiko
><andrii.nakryiko@gmail.com> wrote:
>>
>> On Fri, Jun 19, 2020 at 1:30 PM Hao Luo <haoluo@google.com> wrote:
>> >
>> > Hi, Andrii,
>> >
>> > I agree that we'd better put a hold on this patch until we find out
>> > the reason for the 'unconventional' symbols. I'll _try_ to figure
>it
>> > out, but not able to fully commit my time on this patch. I thought
>I'd
>> > better publish this patch as the DATASEC and VARs are generated
>> > correctly in format, so that anyone can use it to generate the
>vmlinux
>> > and continue the development on libbpf based on your ksym work
>(i.e.
>> > typed ksyms).
>> >
>>
>> Hey Hao,
>>
>> It's a pity that you are willing to drop this while being half-step
>> away from doing this properly. See my other reply, there is something
>> fishy with variable__name() and variable__type_size() and how it
>> calculates/caches values. ELF itself has all the data (based ELF
>> symbol data) and it seems to be correct. And I was correct to be
>> suspicious about just filtering out such variables, because in my
>case
>> you'd filter out a good chunk of variables for no good reason: 74
>> variables, which is a 26% of all per-CPU variables in my kernel.
>>
>> It's up to you, of course, but it would be nice to fix it up (just
>> switching to elf_sym__size() and elf_sym__name() would be fine, you'd
>> just need to make sure to add elf_sym_name() result into a string
>> buffer; unless Arnaldo has some better alternatives) and publish
>> complete v5, that would get merged into pahole. After that, building
>> on my .ksym work in libbpf should get you to what you need from BPF
>> pretty quickly (plus some kernel-side logic to recognize these
>per-CPU
>> variables).
>>
>> > Hao
>> >
>> > On Fri, Jun 19, 2020 at 12:58 PM Andrii Nakryiko
>> > <andrii.nakryiko@gmail.com> wrote:
>> > >
>> > > On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <haoluo@google.com>
>wrote:
>> > > >
>> > > > On SMP systems, the global percpu variables are placed in a
>special
>> > > > '.data..percpu' section, which is stored in a segment whose
>initial
>> > > > address is set to 0, the addresses of per-CPU variables are
>relative
>> > > > positive addresses [1].
>> > > >
>> > > > This patch extracts these variables from vmlinux and places
>them with
>> > > > their type information in BTF. More specifically, when BTF is
>encoded,
>> > > > we find the index of the '.data..percpu' section and then
>traverse
>> > > > the symbol table to find those global objects which are in this
>section.
>> > > > For each of these objects, we push a BTF_KIND_VAR into the
>types buffer,
>> > > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When
>all the
>> > > > CUs have finished processing, we push a BTF_KIND_DATASEC into
>the
>> > > > btfe->types buffer, followed by the percpu_secinfo's content.
>> > > >
>> > > > In a v5.7-rc7 linux kernel, I was able to extract 291 such
>variables.
>> > > > The build time overhead is small and the space overhead is also
>small.
>> > > >
>> > > > Testing:
>> > > >
>> > > > - vmlinux size has increased by ~12kb.
>> > > >   Before:
>> > > >    $ readelf -SW vmlinux | grep BTF
>> > > >    [25] .BTF              PROGBITS        ffffffff821a905c
>13a905c 2d2bf8 00
>> > > >   After:
>> > > >    $ pahole -J vmlinux
>> > > >    $ readelf -SW vmlinux  | grep BTF
>> > > >    [25] .BTF              PROGBITS        ffffffff821a905c
>13a905c 2d5bca 00
>> > > >
>> > > > - Common global percpu VARs and DATASEC are found in BTF
>section.
>> > > >   $ bpftool btf dump file vmlinux | grep runqueues
>> > > >   [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
>> > > >
>> > > >   $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
>> > > >   [17592] STRUCT 'cpu_stopper' size=72 vlen=5
>> > > >   [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
>> > > >
>> > > >   $ bpftool btf dump file vmlinux | grep ' DATASEC '
>> > > >   [63652] DATASEC '.data..percpu' size=0 vlen=294
>> > >
>> > > probably forgot to update the example, I'd imagine size wouldn't
>be 0 anymore?
>> > >
>>
>> [...]

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

  parent reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-18  7:48 Hao Luo
     [not found] ` <20200618074853.133675-1-haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
2020-06-19 19:58   ` Andrii Nakryiko
     [not found]     ` <CAEf4BzbE2K7uyJ-2d3N+NLrV5OKS0HMm5vKuxMz5bfZkE_ANzw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-19 20:30       ` Hao Luo
     [not found]         ` <CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 20:25           ` Andrii Nakryiko
     [not found]             ` <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-29 21:13               ` Hao Luo
     [not found]                 ` <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30  1:02                   ` Arnaldo Carvalho de Melo [this message]
2020-06-29 20:20       ` Andrii Nakryiko
     [not found]         ` <CAEf4Bzbiu8G5UK9TDBRFYGwVk4A4QWoLurZsFKhs439gBXaDCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2020-06-30 11:59           ` Arnaldo Carvalho de Melo
     [not found]             ` <20200630115927.GL29008-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
2020-07-03  8:13               ` Hao Luo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5839EE93-F4F3-4210-9095-13CADC1822E8@gmail.com \
    --to=arnaldo.melo-re5jqeeqqe8avxtiumwx3w@public.gmane.org \
    --cc=alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org \
    --cc=dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=kafai-b10kYP2dOMg@public.gmane.org \
    --cc=olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Dwarves Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dwarves/0 dwarves/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dwarves dwarves/ https://lore.kernel.org/dwarves \
		dwarves@vger.kernel.org
	public-inbox-index dwarves

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dwarves


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git