From: Arnaldo Carvalho de Melo <arnaldo.melo-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> To: Hao Luo <haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Andrii Nakryiko <andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> Cc: Alexei Starovoitov <alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Daniel Borkmann <daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org>, Oleg Rombakh <olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Martin Lau <kafai-b10kYP2dOMg@public.gmane.org>, dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Subject: Re: [PATCH v4] btf_encoder: Teach pahole to store percpu variables in vmlinux BTF. Date: Mon, 29 Jun 2020 22:02:31 -0300 Message-ID: <5839EE93-F4F3-4210-9095-13CADC1822E8@gmail.com> (raw) In-Reply-To: <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> On June 29, 2020 6:13:20 PM GMT-03:00, Hao Luo <email@example.com> wrote: >Hi Andrii, > >Thanks for taking a look! I can post a v5 based on your thorough and >detailed analysis this week. Some of my other work on sched consumed >most of my bandwidth last week and I very appreciate your help! I >apologize if it is felt I dropped this patch and will follow up within >this week. I'll also try and look at this tomorrow. - Arnaldo > >Hao > >On Mon, Jun 29, 2020 at 1:26 PM Andrii Nakryiko ><firstname.lastname@example.org> wrote: >> >> On Fri, Jun 19, 2020 at 1:30 PM Hao Luo <email@example.com> wrote: >> > >> > Hi, Andrii, >> > >> > I agree that we'd better put a hold on this patch until we find out >> > the reason for the 'unconventional' symbols. I'll _try_ to figure >it >> > out, but not able to fully commit my time on this patch. I thought >I'd >> > better publish this patch as the DATASEC and VARs are generated >> > correctly in format, so that anyone can use it to generate the >vmlinux >> > and continue the development on libbpf based on your ksym work >(i.e. >> > typed ksyms). >> > >> >> Hey Hao, >> >> It's a pity that you are willing to drop this while being half-step >> away from doing this properly. See my other reply, there is something >> fishy with variable__name() and variable__type_size() and how it >> calculates/caches values. ELF itself has all the data (based ELF >> symbol data) and it seems to be correct. And I was correct to be >> suspicious about just filtering out such variables, because in my >case >> you'd filter out a good chunk of variables for no good reason: 74 >> variables, which is a 26% of all per-CPU variables in my kernel. >> >> It's up to you, of course, but it would be nice to fix it up (just >> switching to elf_sym__size() and elf_sym__name() would be fine, you'd >> just need to make sure to add elf_sym_name() result into a string >> buffer; unless Arnaldo has some better alternatives) and publish >> complete v5, that would get merged into pahole. After that, building >> on my .ksym work in libbpf should get you to what you need from BPF >> pretty quickly (plus some kernel-side logic to recognize these >per-CPU >> variables). >> >> > Hao >> > >> > On Fri, Jun 19, 2020 at 12:58 PM Andrii Nakryiko >> > <firstname.lastname@example.org> wrote: >> > > >> > > On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <email@example.com> >wrote: >> > > > >> > > > On SMP systems, the global percpu variables are placed in a >special >> > > > '.data..percpu' section, which is stored in a segment whose >initial >> > > > address is set to 0, the addresses of per-CPU variables are >relative >> > > > positive addresses . >> > > > >> > > > This patch extracts these variables from vmlinux and places >them with >> > > > their type information in BTF. More specifically, when BTF is >encoded, >> > > > we find the index of the '.data..percpu' section and then >traverse >> > > > the symbol table to find those global objects which are in this >section. >> > > > For each of these objects, we push a BTF_KIND_VAR into the >types buffer, >> > > > and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When >all the >> > > > CUs have finished processing, we push a BTF_KIND_DATASEC into >the >> > > > btfe->types buffer, followed by the percpu_secinfo's content. >> > > > >> > > > In a v5.7-rc7 linux kernel, I was able to extract 291 such >variables. >> > > > The build time overhead is small and the space overhead is also >small. >> > > > >> > > > Testing: >> > > > >> > > > - vmlinux size has increased by ~12kb. >> > > > Before: >> > > > $ readelf -SW vmlinux | grep BTF >> > > >  .BTF PROGBITS ffffffff821a905c >13a905c 2d2bf8 00 >> > > > After: >> > > > $ pahole -J vmlinux >> > > > $ readelf -SW vmlinux | grep BTF >> > > >  .BTF PROGBITS ffffffff821a905c >13a905c 2d5bca 00 >> > > > >> > > > - Common global percpu VARs and DATASEC are found in BTF >section. >> > > > $ bpftool btf dump file vmlinux | grep runqueues >> > > >  VAR 'runqueues' type_id=13725, linkage=global-alloc >> > > > >> > > > $ bpftool btf dump file vmlinux | grep 'cpu_stopper' >> > > >  STRUCT 'cpu_stopper' size=72 vlen=5 >> > > >  VAR 'cpu_stopper' type_id=17592, linkage=static >> > > > >> > > > $ bpftool btf dump file vmlinux | grep ' DATASEC ' >> > > >  DATASEC '.data..percpu' size=0 vlen=294 >> > > >> > > probably forgot to update the example, I'd imagine size wouldn't >be 0 anymore? >> > > >> >> [...] -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
next prev parent reply index Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-18 7:48 Hao Luo [not found] ` <20200618074853.133675-1-haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> 2020-06-19 19:58 ` Andrii Nakryiko [not found] ` <CAEf4BzbE2K7uyJ-2d3N+NLrV5OKS0HMm5vKuxMz5bfZkE_ANzw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-06-19 20:30 ` Hao Luo [not found] ` <CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-06-29 20:25 ` Andrii Nakryiko [not found] ` <CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-06-29 21:13 ` Hao Luo [not found] ` <CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-06-30 1:02 ` Arnaldo Carvalho de Melo [this message] 2020-06-29 20:20 ` Andrii Nakryiko [not found] ` <CAEf4Bzbiu8G5UK9TDBRFYGwVk4A4QWoLurZsFKhs439gBXaDCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2020-06-30 11:59 ` Arnaldo Carvalho de Melo [not found] ` <20200630115927.GL29008-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> 2020-07-03 8:13 ` Hao Luo
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=5839EE93-F4F3-4210-9095-13CADC1822E8@gmail.com \ --firstname.lastname@example.org \ --cc=alexei.starovoitov-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \ --cc=andrii.nakryiko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \ --cc=daniel-FeC+5ew28dpmcu3hnIyYJQ@public.gmane.org \ --cc=dwarves-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \ --cc=kafai-b10kYP2dOMg@public.gmane.org \ --cc=olegrom-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Dwarves Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/dwarves/0 dwarves/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dwarves dwarves/ https://lore.kernel.org/dwarves \ email@example.com public-inbox-index dwarves Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.dwarves AGPL code for this site: git clone https://public-inbox.org/public-inbox.git