bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Saeed Mahameed <saeedm@nvidia.com>
To: "andrii.nakryiko@gmail.com" <andrii.nakryiko@gmail.com>,
	"brouer@redhat.com" <brouer@redhat.com>
Cc: "aspsk2@gmail.com" <aspsk2@gmail.com>, "ast@fb.com" <ast@fb.com>,
	"dsahern@kernel.org" <dsahern@kernel.org>,
	"daniel@iogearbox.net" <daniel@iogearbox.net>,
	"kernel-team@fb.com" <kernel-team@fb.com>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"bpf@vger.kernel.org" <bpf@vger.kernel.org>,
	"andrii@kernel.org" <andrii@kernel.org>
Subject: Re: [PATCH v2 bpf-next 00/11] libbpf: split BTF support
Date: Thu, 5 Nov 2020 19:38:17 +0000	[thread overview]
Message-ID: <105c48d56a550af6e0008b4b5867eb51764d41c9.camel@nvidia.com> (raw)
In-Reply-To: <CAEf4BzYOcQt1dv2f5UmVqCGWJVqM95DoUAumH+sRuXW3rzejMg@mail.gmail.com>

On Thu, 2020-11-05 at 11:16 -0800, Andrii Nakryiko wrote:
> > > This split approach is necessary if we are to have a reasonably-
> > > sized kernel
> > > module BTFs. By deduping each kernel module's BTF individually,
> > > resulting
> > > module BTFs contain copies of a lot of kernel types that are
> > > already present
> > > in vmlinux BTF. Even those single copies result in a big BTF size
> > > bloat. On my
> > > kernel configuration with 700 modules built, non-split BTF
> > > approach results in
> > > 115MBs of BTFs across all modules. With split BTF deduplication
> > > approach,
> > > total size is down to 5.2MBs total, which is on part with vmlinux
> > > BTF (at
> > > around 4MBs). This seems reasonable and practical. As to why we'd
> > > need kernel
> > > module BTFs, that should be pretty obvious to anyone using BPF at
> > > this point,
> > > as it allows all the BTF-powered features to be used with kernel
> > > modules:
> > > tp_btf, fentry/fexit/fmod_ret, lsm, bpf_iter, etc.
> > I love to see this work going forward.
> 
> 
> Thanks.
> 
> 
> 
> > My/Our (+Saeed +Ahern) use-case is for NIC-driver kernel modules. 
> > I
> > want drivers to define a BTF struct that describe a meta-data area
> > that
> > can be consumed/used by XDP, also available during xdp_frame to SKB
> > transition, which happens in net-core. So, I hope BTF-IDs are also
> > "available" from core kernel code?
> 
> 
> I'll probably need a more specific example to understand what exactly
> 
> you are asking and how you see everything working together, sorry.
> 
> 

BTF-IDs can be made available for kernel/drivers, I've wrote a small
patch for this a while ago.

https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=6c1cb83629226889d6fadd3ba694e827fca3e247

So the basic use case is that :
1- driver kernel/registers a BTF format (one or more).
2- Userland queries driver's registered BTF to be able understand the
kernel/driver buffers format.

driver example of using this infrastructure:
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=9c24657d6cb3a7852c2e948dc9782f3f39b60104

User Queries driver's XDP metadata BTF format:
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=6a117e2d9196f58de7cf067741e84ec242af27f6

Dump it as C header style 
https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp_metadata3&id=
8bd99626879bff28379707ac3a2c3bb94fd5b410

And then use it in your XDP program to parse packets meta data passed
from this specific driver. ( i mean no real parsing is required, you
just point to the meta data buffer with the metadata btf formatted C
strucuter).


> 
> If you are asking about support for using BTF_ID_LIST() macro in a
> 
> kernel module, then right now we don't call resolve_btfids on
> modules,
> 
> so it's not supported there yet. It's trivial to add, but we'll
> 
> probably need to teach resolve_btfids to understand split BTF. We can
> 
> do that separately after the basic "infra" lands, though.



  reply	other threads:[~2020-11-05 19:38 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-05  4:33 [PATCH v2 bpf-next 00/11] libbpf: split BTF support Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 01/11] libbpf: factor out common operations in BTF writing APIs Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 02/11] selftest/bpf: relax btf_dedup test checks Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 03/11] libbpf: unify and speed up BTF string deduplication Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 04/11] libbpf: implement basic split BTF support Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 05/11] selftests/bpf: add split BTF basic test Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 06/11] selftests/bpf: add checking of raw type dump in BTF writer APIs selftests Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 07/11] libbpf: fix BTF data layout checks and allow empty BTF Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 08/11] libbpf: support BTF dedup of split BTFs Andrii Nakryiko
2020-11-05  4:33 ` [PATCH v2 bpf-next 09/11] libbpf: accomodate DWARF/compiler bug with duplicated identical arrays Andrii Nakryiko
2020-11-05  4:34 ` [PATCH v2 bpf-next 10/11] selftests/bpf: add split BTF dedup selftests Andrii Nakryiko
2020-11-05  4:34 ` [PATCH v2 bpf-next 11/11] tools/bpftool: add bpftool support for split BTF Andrii Nakryiko
2020-11-05  9:52 ` [PATCH v2 bpf-next 00/11] libbpf: split BTF support Jesper Dangaard Brouer
2020-11-05 19:16   ` Andrii Nakryiko
2020-11-05 19:38     ` Saeed Mahameed [this message]
2020-11-05 20:02       ` Andrii Nakryiko
2020-11-06  2:50 ` patchwork-bot+netdevbpf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=105c48d56a550af6e0008b4b5867eb51764d41c9.camel@nvidia.com \
    --to=saeedm@nvidia.com \
    --cc=andrii.nakryiko@gmail.com \
    --cc=andrii@kernel.org \
    --cc=aspsk2@gmail.com \
    --cc=ast@fb.com \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=dsahern@kernel.org \
    --cc=kernel-team@fb.com \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).