From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: "Mauricio Vásquez" <mauricio@kinvolk.io>
Cc: Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Andrii Nakryiko <andrii@kernel.org>,
Quentin Monnet <quentin@isovalent.com>,
Rafael David Tinoco <rafaeldtinoco@gmail.com>,
Lorenzo Fontana <lorenzo.fontana@elastic.co>,
Leonardo Di Donato <leonardo.didonato@elastic.co>
Subject: Re: [PATCH bpf-next v5 2/9] bpftool: Add gen min_core_btf command
Date: Wed, 2 Feb 2022 09:58:16 -0800 [thread overview]
Message-ID: <CAEf4BzY3_GZD8C754nb5P_+btFEsmK5QHC-qoHqJqAdSMNKcsQ@mail.gmail.com> (raw)
In-Reply-To: <20220128223312.1253169-3-mauricio@kinvolk.io>
On Fri, Jan 28, 2022 at 2:33 PM Mauricio Vásquez <mauricio@kinvolk.io> wrote:
>
> This command is implemented under the "gen" command in bpftool and the
> syntax is the following:
>
> $ bpftool gen min_core_btf INPUT OUTPUT OBJECT(S)
>
> INPUT can be either a single BTF file or a folder containing BTF files,
> when it's a folder, a BTF file is generated for each BTF file contained
> in this folder. OUTPUT is the file (or folder) where generated files are
> stored and OBJECT(S) is the list of bpf objects we want to generate the
> BTF file(s) for (each generated BTF file contains all the types needed
> by all the objects).
>
> Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
> Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
> Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
> Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
> ---
> tools/bpf/bpftool/bash-completion/bpftool | 6 +-
> tools/bpf/bpftool/gen.c | 112 +++++++++++++++++++++-
> 2 files changed, 114 insertions(+), 4 deletions(-)
>
> diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
> index 493753a4962e..958e1fd71b5c 100644
> --- a/tools/bpf/bpftool/bash-completion/bpftool
> +++ b/tools/bpf/bpftool/bash-completion/bpftool
> @@ -1003,9 +1003,13 @@ _bpftool()
> ;;
> esac
> ;;
> + min_core_btf)
> + _filedir
> + return 0
> + ;;
> *)
> [[ $prev == $object ]] && \
> - COMPREPLY=( $( compgen -W 'object skeleton help' -- "$cur" ) )
> + COMPREPLY=( $( compgen -W 'object skeleton help min_core_btf' -- "$cur" ) )
> ;;
> esac
> ;;
> diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
> index 8f78c27d41f0..7db31b0f265f 100644
> --- a/tools/bpf/bpftool/gen.c
> +++ b/tools/bpf/bpftool/gen.c
> @@ -5,6 +5,7 @@
> #define _GNU_SOURCE
> #endif
> #include <ctype.h>
> +#include <dirent.h>
> #include <errno.h>
> #include <fcntl.h>
> #include <linux/err.h>
> @@ -1084,6 +1085,7 @@ static int do_help(int argc, char **argv)
> fprintf(stderr,
> "Usage: %1$s %2$s object OUTPUT_FILE INPUT_FILE [INPUT_FILE...]\n"
> " %1$s %2$s skeleton FILE [name OBJECT_NAME]\n"
> + " %1$s %2$s min_core_btf INPUT OUTPUT OBJECT(S)\n"
OBJECTS(S) should be OBJECT... for this "CLI notation", no?
> " %1$s %2$s help\n"
> "\n"
> " " HELP_SPEC_OPTIONS " |\n"
> @@ -1094,10 +1096,114 @@ static int do_help(int argc, char **argv)
> return 0;
> }
>
> +/* Create BTF file for a set of BPF objects */
> +static int btfgen(const char *src_btf, const char *dst_btf, const char *objspaths[])
> +{
> + return -EOPNOTSUPP;
> +}
> +
> +static int do_min_core_btf(int argc, char **argv)
> +{
> + char src_btf_path[PATH_MAX], dst_btf_path[PATH_MAX];
> + bool input_is_file, output_is_file = true;
> + const char *input, *output;
> + const char **objs = NULL;
> + struct dirent *dir;
> + struct stat st;
> + DIR *d = NULL;
> + int i, err;
> +
> + if (!REQ_ARGS(3)) {
> + usage();
> + return -1;
> + }
> +
> + input = GET_ARG();
> + if (stat(input, &st) < 0) {
> + p_err("failed to stat %s: %s", input, strerror(errno));
> + return -errno;
> + }
> +
> + if ((st.st_mode & S_IFMT) != S_IFDIR && (st.st_mode & S_IFMT) != S_IFREG) {
> + p_err("file type not valid: %s", input);
> + return -EINVAL;
> + }
> +
> + input_is_file = (st.st_mode & S_IFMT) == S_IFREG;
move before if and use input_is_file in the if itself instead of
duplicating all the S_IFREG flags?
> +
> + output = GET_ARG();
> + if (stat(output, &st) == 0 && (st.st_mode & S_IFMT) == S_IFDIR)
> + output_is_file = false;
if stat() succeeds but it's neither directory or file, should be an
error, right?
> +
> + objs = (const char **) malloc((argc + 1) * sizeof(*objs));
calloc() seems to be better suited for this (and zero-intialization is
nice for safety and to avoid objs[argc] = NULL after the loop below)
> + if (!objs) {
> + p_err("failed to allocate array for object names");
> + return -ENOMEM;
> + }
> +
> + i = 0;
> + while (argc > 0)
> + objs[i++] = GET_ARG();
for (i = 0; i < argc; i++) ?
> +
> + objs[i] = NULL;
> +
> + /* single BTF file */
> + if (input_is_file) {
> + p_info("Processing source BTF file: %s", input);
> +
> + if (output_is_file) {
> + err = btfgen(input, output, objs);
> + goto out;
> + }
> + snprintf(dst_btf_path, sizeof(dst_btf_path), "%s/%s", output,
> + basename(input));
> + err = btfgen(input, dst_btf_path, objs);
> + goto out;
> + }
> +
> + if (output_is_file) {
> + p_err("can't have just one file as output");
> + err = -EINVAL;
> + goto out;
> + }
> +
> + /* directory with BTF files */
> + d = opendir(input);
> + if (!d) {
> + p_err("error opening input dir: %s", strerror(errno));
> + err = -errno;
> + goto out;
> + }
> +
> + while ((dir = readdir(d)) != NULL) {
> + if (dir->d_type != DT_REG)
> + continue;
> +
> + if (strncmp(dir->d_name + strlen(dir->d_name) - 4, ".btf", 4))
> + continue;
this whole handling of input directory feels a bit icky, tbh... maybe
we should require explicit listing of input files always. In CLI
invocation those could be separated by "keywords", something like
this:
bpftool gen min_core_btf <output> inputs <file1> <file2> .... objects
<obj1> <obj2> ...
a bit of a downside is that you can't have a file named "inputs" or
"objects", but that seems extremely unlikely? Quentin, any opinion as
well?
I'm mainly off put by a bit random ".btf" naming convention, the
DT_REG skipping, etc.
Another cleaner alternative from POV of bpftool (but might be less
convenient for users) is to use @file convention to specify a file
that contains a list of files. So
bpftool gen min_core_btf <output> @btf_filelist.txt @obj_filelist.txt
would take lists of inputs and outputs from respective files?
But actually, let's take a step back again. Why should there be
multiple inputs and outputs? I can see why multiple objects are
mandatory (you have an application that has multiple BPF objects used
internally). But processing single vmlinux BTF at a time seems
absolutely fine. I don't buy that CO-RE relo processing is that slow
to require optimized batch processing.
I might have asked this before, sorry, but the duration between each
iteration of btfgen is pretty long and I'm losing the context.
> +
> + snprintf(src_btf_path, sizeof(src_btf_path), "%s%s", input, dir->d_name);
> + snprintf(dst_btf_path, sizeof(dst_btf_path), "%s%s", output, dir->d_name);
> +
> + p_info("Processing source BTF file: %s", src_btf_path);
> +
> + err = btfgen(src_btf_path, dst_btf_path, objs);
> + if (err)
> + goto out;
> + }
> +
> +out:
> + free(objs);
> + if (d)
> + closedir(d);
> + return err;
> +}
> +
> static const struct cmd cmds[] = {
> - { "object", do_object },
> - { "skeleton", do_skeleton },
> - { "help", do_help },
> + { "object", do_object },
> + { "skeleton", do_skeleton },
> + { "min_core_btf", do_min_core_btf},
> + { "help", do_help },
> { 0 }
> };
>
> --
> 2.25.1
>
next prev parent reply other threads:[~2022-02-02 17:58 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-28 22:33 [PATCH bpf-next v5 0/9] libbpf: Implement BTFGen Mauricio Vásquez
2022-01-28 22:33 ` [PATCH bpf-next v5 1/9] libbpf: Implement changes needed for BTFGen in bpftool Mauricio Vásquez
2022-02-01 20:57 ` Quentin Monnet
2022-02-03 16:08 ` Mauricio Vásquez Bernal
2022-02-02 18:54 ` Andrii Nakryiko
2022-02-02 19:02 ` Andrii Nakryiko
2022-02-03 16:09 ` Mauricio Vásquez Bernal
2022-01-28 22:33 ` [PATCH bpf-next v5 2/9] bpftool: Add gen min_core_btf command Mauricio Vásquez
2022-02-02 17:58 ` Andrii Nakryiko [this message]
2022-02-03 16:07 ` Mauricio Vásquez Bernal
2022-02-03 17:21 ` Andrii Nakryiko
2022-01-28 22:33 ` [PATCH bpf-next v5 3/9] bpftool: Implement btf_save_raw() Mauricio Vásquez
2022-02-02 18:48 ` Andrii Nakryiko
2022-02-03 16:07 ` Mauricio Vásquez Bernal
2022-02-03 17:23 ` Andrii Nakryiko
2022-01-28 22:33 ` [PATCH bpf-next v5 4/9] bpftool: Add struct definitions and helpers for BTFGen Mauricio Vásquez
2022-02-02 18:54 ` Andrii Nakryiko
2022-02-03 16:08 ` Mauricio Vásquez Bernal
2022-02-03 17:24 ` Andrii Nakryiko
2022-01-28 22:33 ` [PATCH bpf-next v5 5/9] bpftool: Implement btfgen() Mauricio Vásquez
2022-02-01 20:57 ` Quentin Monnet
2022-02-03 19:10 ` Mauricio Vásquez Bernal
2022-02-02 19:14 ` Andrii Nakryiko
2022-02-03 16:09 ` Mauricio Vásquez Bernal
2022-01-28 22:33 ` [PATCH bpf-next v5 6/9] bpftool: Implement relocations recording for BTFGen Mauricio Vásquez
2022-02-02 19:31 ` Andrii Nakryiko
2022-02-03 16:40 ` Mauricio Vásquez Bernal
2022-02-03 17:30 ` Andrii Nakryiko
2022-02-04 6:20 ` Rafael David Tinoco
2022-02-04 18:41 ` Andrii Nakryiko
2022-02-02 22:55 ` Andrii Nakryiko
2022-02-04 19:44 ` Mauricio Vásquez Bernal
2022-01-28 22:33 ` [PATCH bpf-next v5 7/9] bpftool: Implement btfgen_get_btf() Mauricio Vásquez
2022-02-02 19:36 ` Andrii Nakryiko
2022-02-03 16:10 ` Mauricio Vásquez Bernal
2022-02-03 17:31 ` Andrii Nakryiko
2022-01-28 22:33 ` [PATCH bpf-next v5 8/9] bpftool: gen min_core_btf explanation and examples Mauricio Vásquez
2022-02-01 20:57 ` Quentin Monnet
2022-01-28 22:33 ` [PATCH bpf-next v5 9/9] selftest/bpf: Implement tests for bpftool gen min_core_btf Mauricio Vásquez
2022-01-28 23:23 ` Mauricio Vásquez Bernal
2022-02-01 20:58 ` Quentin Monnet
2022-02-02 19:50 ` Andrii Nakryiko
2022-02-03 21:17 ` Mauricio Vásquez Bernal
2022-02-04 20:05 ` Andrii Nakryiko
2022-02-01 20:57 ` Quentin Monnet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAEf4BzY3_GZD8C754nb5P_+btFEsmK5QHC-qoHqJqAdSMNKcsQ@mail.gmail.com \
--to=andrii.nakryiko@gmail.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=leonardo.didonato@elastic.co \
--cc=lorenzo.fontana@elastic.co \
--cc=mauricio@kinvolk.io \
--cc=netdev@vger.kernel.org \
--cc=quentin@isovalent.com \
--cc=rafaeldtinoco@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).