From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
john fastabend <john.fastabend@gmail.com>,
bpf <bpf@vger.kernel.org>, Kernel Team <kernel-team@fb.com>
Subject: Re: [PATCH v4 bpf-next 14/22] libbpf: Generate loader program out of BPF ELF file.
Date: Tue, 11 May 2021 16:22:52 -0700 [thread overview]
Message-ID: <CAEf4BzZzOP6ai0Yf_PbF+At4-UXu0E_8bKyXL4Qj9T6ohxvofQ@mail.gmail.com> (raw)
In-Reply-To: <20210508034837.64585-15-alexei.starovoitov@gmail.com>
On Fri, May 7, 2021 at 8:49 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> From: Alexei Starovoitov <ast@kernel.org>
>
> The BPF program loading process performed by libbpf is quite complex
> and consists of the following steps:
> "open" phase:
> - parse elf file and remember relocations, sections
> - collect externs and ksyms including their btf_ids in prog's BTF
> - patch BTF datasec (since llvm couldn't do it)
> - init maps (old style map_def, BTF based, global data map, kconfig map)
> - collect relocations against progs and maps
> "load" phase:
> - probe kernel features
> - load vmlinux BTF
> - resolve externs (kconfig and ksym)
> - load program BTF
> - init struct_ops
> - create maps
> - apply CO-RE relocations
> - patch ld_imm64 insns with src_reg=PSEUDO_MAP, PSEUDO_MAP_VALUE, PSEUDO_BTF_ID
> - reposition subprograms and adjust call insns
> - sanitize and load progs
>
> During this process libbpf does sys_bpf() calls to load BTF, create maps,
> populate maps and finally load programs.
> Instead of actually doing the syscalls generate a trace of what libbpf
> would have done and represent it as the "loader program".
> The "loader program" consists of single map with:
> - union bpf_attr(s)
> - BTF bytes
> - map value bytes
> - insns bytes
> and single bpf program that passes bpf_attr(s) and data into bpf_sys_bpf() helper.
> Executing such "loader program" via bpf_prog_test_run() command will
> replay the sequence of syscalls that libbpf would have done which will result
> the same maps created and programs loaded as specified in the elf file.
> The "loader program" removes libelf and majority of libbpf dependency from
> program loading process.
>
> kconfig, typeless ksym, struct_ops and CO-RE are not supported yet.
>
> The order of relocate_data and relocate_calls had to change, so that
> bpf_gen__prog_load() can see all relocations for a given program with
> correct insn_idx-es.
>
> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> ---
LGTM.
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> tools/lib/bpf/Build | 2 +-
> tools/lib/bpf/bpf_gen_internal.h | 40 ++
> tools/lib/bpf/gen_loader.c | 657 +++++++++++++++++++++++++++++++
> tools/lib/bpf/libbpf.c | 224 +++++++++--
> tools/lib/bpf/libbpf.h | 12 +
> tools/lib/bpf/libbpf.map | 1 +
> tools/lib/bpf/libbpf_internal.h | 2 +
> tools/lib/bpf/skel_internal.h | 116 ++++++
> 8 files changed, 1022 insertions(+), 32 deletions(-)
> create mode 100644 tools/lib/bpf/bpf_gen_internal.h
> create mode 100644 tools/lib/bpf/gen_loader.c
> create mode 100644 tools/lib/bpf/skel_internal.h
>
[...]
next prev parent reply other threads:[~2021-05-11 23:23 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-08 3:48 [PATCH v4 bpf-next 00/22] bpf: syscall program, FD array, loader program, light skeleton Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 01/22] bpf: Introduce bpf_sys_bpf() helper and program type Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 02/22] bpf: Introduce bpfptr_t user/kernel pointer Alexei Starovoitov
2021-05-11 22:23 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 03/22] bpf: Prepare bpf syscall to be used from kernel and user space Alexei Starovoitov
2021-05-11 22:31 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 04/22] libbpf: Support for syscall program type Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 05/22] selftests/bpf: Test " Alexei Starovoitov
2021-05-11 22:36 ` Andrii Nakryiko
2021-05-11 22:44 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 06/22] bpf: Make btf_load command to be bpfptr_t compatible Alexei Starovoitov
2021-05-11 22:41 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 07/22] selftests/bpf: Test for btf_load command Alexei Starovoitov
2021-05-11 22:45 ` Andrii Nakryiko
2021-05-12 4:04 ` Alexei Starovoitov
2021-05-12 18:00 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 08/22] bpf: Introduce fd_idx Alexei Starovoitov
2021-05-11 22:47 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 09/22] libbpf: Support for fd_idx Alexei Starovoitov
2021-05-11 22:57 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 10/22] bpf: Add bpf_btf_find_by_name_kind() helper Alexei Starovoitov
2021-05-11 23:02 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 11/22] bpf: Add bpf_sys_close() helper Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 12/22] libbpf: Change the order of data and text relocations Alexei Starovoitov
2021-05-11 23:06 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 13/22] libbpf: Add bpf_object pointer to kernel_supports() Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 14/22] libbpf: Generate loader program out of BPF ELF file Alexei Starovoitov
2021-05-11 23:22 ` Andrii Nakryiko [this message]
2021-05-08 3:48 ` [PATCH v4 bpf-next 15/22] libbpf: Use fd_array only with gen_loader Alexei Starovoitov
2021-05-11 23:24 ` Andrii Nakryiko
2021-05-12 4:17 ` Alexei Starovoitov
2021-05-12 18:11 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 16/22] libbpf: Cleanup temp FDs when intermediate sys_bpf fails Alexei Starovoitov
2021-05-11 23:34 ` Andrii Nakryiko
2021-05-12 4:33 ` Alexei Starovoitov
2021-05-08 3:48 ` [PATCH v4 bpf-next 17/22] libbpf: Introduce bpf_map__get_initial_value() Alexei Starovoitov
2021-05-11 23:39 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 18/22] bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command Alexei Starovoitov
2021-05-12 4:17 ` Andrii Nakryiko
2021-05-12 18:43 ` Alexei Starovoitov
2021-05-12 18:55 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 19/22] selftests/bpf: Convert few tests to light skeleton Alexei Starovoitov
2021-05-12 4:19 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 20/22] selftests/bpf: Convert atomics test " Alexei Starovoitov
2021-05-12 4:21 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 21/22] selftests/bpf: Convert test printk to use rodata Alexei Starovoitov
2021-05-12 4:23 ` Andrii Nakryiko
2021-05-08 3:48 ` [PATCH v4 bpf-next 22/22] selftests/bpf: Convert test trace_printk to lskel Alexei Starovoitov
2021-05-12 4:24 ` Andrii Nakryiko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAEf4BzZzOP6ai0Yf_PbF+At4-UXu0E_8bKyXL4Qj9T6ohxvofQ@mail.gmail.com \
--to=andrii.nakryiko@gmail.com \
--cc=alexei.starovoitov@gmail.com \
--cc=andrii@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=john.fastabend@gmail.com \
--cc=kernel-team@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).