[PATCH bpf-next 0/6] libbpf: rename btf__get_from_id() and btf__load() APIs, support split BTF

[PATCH bpf-next 0/6] libbpf: rename btf__get_from_id() and btf__load() APIs, support split BTF

[Quentin Monnet]

As part of the effort to move towards a v1.0 for libbpf [0], this set
improves some confusing function names related to BTF loading from and to
the kernel:

- btf__load() becomes btf__load_into_kernel().
- btf__get_from_id becomes btf__load_from_kernel_by_id().
- A new version btf__load_from_kernel_by_id_split() extends the former to
  add support for split BTF.

The old functions are not removed yet, but marked as deprecated.

The last patch is a trivial change to bpftool to add support for dumping
split BTF objects by referencing them by their id (and not only by their
BTF path).

[0] https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Quentin Monnet (6):
  libbpf: rename btf__load() as btf__load_into_kernel()
  libbpf: rename btf__get_from_id() as btf__load_from_kernel_by_id()
  tools: replace btf__get_from_id() with btf__load_from_kernel_by_id()
  libbpf: explicitly mark btf__load() and btf__get_from_id() as
    deprecated
  libbpf: add split BTF support for btf__load_from_kernel_by_id()
  tools: bpftool: support dumping split BTF by id

 tools/bpf/bpftool/btf.c                      |  2 +-
 tools/bpf/bpftool/btf_dumper.c               |  2 +-
 tools/bpf/bpftool/map.c                      |  4 ++--
 tools/bpf/bpftool/prog.c                     |  6 +++---
 tools/lib/bpf/btf.c                          | 15 ++++++++++++---
 tools/lib/bpf/btf.h                          | 10 ++++++++--
 tools/lib/bpf/libbpf.c                       |  4 ++--
 tools/lib/bpf/libbpf.map                     |  7 +++++++
 tools/perf/util/bpf-event.c                  |  4 ++--
 tools/perf/util/bpf_counter.c                |  2 +-
 tools/testing/selftests/bpf/prog_tests/btf.c |  2 +-
 11 files changed, 40 insertions(+), 18 deletions(-)

-- 
2.30.2

[PATCH bpf-next 1/6] libbpf: rename btf__load() as btf__load_into_kernel()

[Quentin Monnet]

As part of the effort to move towards a v1.0 for libbpf, rename
btf__load() function, used to "upload" BTF information into the kernel,
and rename it instead as btf__load_into_kernel().

This new name better reflects what the function does, and should be less
confusing.

References:

- https://github.com/libbpf/libbpf/issues/278
- https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/lib/bpf/btf.c      | 3 ++-
 tools/lib/bpf/btf.h      | 1 +
 tools/lib/bpf/libbpf.c   | 2 +-
 tools/lib/bpf/libbpf.map | 5 +++++
 4 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b46760b93bb4..7e0de560490e 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -1180,7 +1180,7 @@ int btf__finalize_data(struct bpf_object *obj, struct btf *btf)
 
 static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian);
 
-int btf__load(struct btf *btf)
+int btf__load_into_kernel(struct btf *btf)
 {
 	__u32 log_buf_size = 0, raw_size;
 	char *log_buf = NULL;
@@ -1228,6 +1228,7 @@ int btf__load(struct btf *btf)
 	free(log_buf);
 	return libbpf_err(err);
 }
+int btf__load(struct btf *) __attribute__((alias("btf__load_into_kernel")));
 
 int btf__fd(const struct btf *btf)
 {
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index b54f1c3ebd57..b36f1b2805dc 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -46,6 +46,7 @@ LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_b
 
 LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf);
 LIBBPF_API int btf__load(struct btf *btf);
+LIBBPF_API int btf__load_into_kernel(struct btf *btf);
 LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,
 				   const char *type_name);
 LIBBPF_API __s32 btf__find_by_name_kind(const struct btf *btf,
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 88b99401040c..d8b7c7750402 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -2768,7 +2768,7 @@ static int bpf_object__sanitize_and_load_btf(struct bpf_object *obj)
 		 */
 		btf__set_fd(kern_btf, 0);
 	} else {
-		err = btf__load(kern_btf);
+		err = btf__load_into_kernel(kern_btf);
 	}
 	if (sanitize) {
 		if (!err) {
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 944c99d1ded3..d42f20b0e9e4 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -375,3 +375,8 @@ LIBBPF_0.5.0 {
 		bpf_object__gen_loader;
 		libbpf_set_strict_mode;
 } LIBBPF_0.4.0;
+
+LIBBPF_0.6.0 {
+	global:
+		btf__load_into_kernel;
+} LIBBPF_0.5.0;
-- 
2.30.2

[PATCH bpf-next 2/6] libbpf: rename btf__get_from_id() as btf__load_from_kernel_by_id()

[Quentin Monnet]

Rename function btf__get_from_id() as btf__load_from_kernel_by_id() to
better indicate what the function does.

The other tools calling the deprecated btf__get_from_id() function will
be updated in a future commit.

References:

- https://github.com/libbpf/libbpf/issues/278
- https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/lib/bpf/btf.c      | 4 +++-
 tools/lib/bpf/btf.h      | 1 +
 tools/lib/bpf/libbpf.c   | 2 +-
 tools/lib/bpf/libbpf.map | 1 +
 4 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 7e0de560490e..05b63b63083a 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -1383,7 +1383,7 @@ struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf)
 	return btf;
 }
 
-int btf__get_from_id(__u32 id, struct btf **btf)
+int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
 {
 	struct btf *res;
 	int err, btf_fd;
@@ -1404,6 +1404,8 @@ int btf__get_from_id(__u32 id, struct btf **btf)
 	*btf = res;
 	return 0;
 }
+int btf__get_from_id(__u32, struct btf **)
+	__attribute__((alias("btf__load_from_kernel_by_id")));
 
 int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
 			 __u32 expected_key_size, __u32 expected_value_size,
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index b36f1b2805dc..0bd9d3952d19 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -68,6 +68,7 @@ LIBBPF_API const void *btf__get_raw_data(const struct btf *btf, __u32 *size);
 LIBBPF_API const char *btf__name_by_offset(const struct btf *btf, __u32 offset);
 LIBBPF_API const char *btf__str_by_offset(const struct btf *btf, __u32 offset);
 LIBBPF_API int btf__get_from_id(__u32 id, struct btf **btf);
+LIBBPF_API int btf__load_from_kernel_by_id(__u32 id, struct btf **btf);
 LIBBPF_API int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
 				    __u32 expected_key_size,
 				    __u32 expected_value_size,
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index d8b7c7750402..e54fa1e57d48 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -9571,7 +9571,7 @@ static int libbpf_find_prog_btf_id(const char *name, __u32 attach_prog_fd)
 		pr_warn("The target program doesn't have BTF\n");
 		goto out;
 	}
-	if (btf__get_from_id(info->btf_id, &btf)) {
+	if (btf__load_from_kernel_by_id(info->btf_id, &btf)) {
 		pr_warn("Failed to get BTF of the program\n");
 		goto out;
 	}
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index d42f20b0e9e4..a687cc63cd80 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -378,5 +378,6 @@ LIBBPF_0.5.0 {
 
 LIBBPF_0.6.0 {
 	global:
+		btf__load_from_kernel_by_id;
 		btf__load_into_kernel;
 } LIBBPF_0.5.0;
-- 
2.30.2

[PATCH bpf-next 3/6] tools: replace btf__get_from_id() with btf__load_from_kernel_by_id()

[Quentin Monnet]

Replace the calls to deprecated function btf__get_from_id() with calls
to btf__load_from_kernel_by_id() in tools/ (bpftool, perf, selftests).

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/bpf/bpftool/btf.c                      | 2 +-
 tools/bpf/bpftool/btf_dumper.c               | 2 +-
 tools/bpf/bpftool/map.c                      | 4 ++--
 tools/bpf/bpftool/prog.c                     | 6 +++---
 tools/perf/util/bpf-event.c                  | 4 ++--
 tools/perf/util/bpf_counter.c                | 2 +-
 tools/testing/selftests/bpf/prog_tests/btf.c | 2 +-
 7 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 385d5c955cf3..2296e8eba0ff 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -580,7 +580,7 @@ static int do_dump(int argc, char **argv)
 	}
 
 	if (!btf) {
-		err = btf__get_from_id(btf_id, &btf);
+		err = btf__load_from_kernel_by_id(btf_id, &btf);
 		if (err) {
 			p_err("get btf by id (%u): %s", btf_id, strerror(err));
 			goto done;
diff --git a/tools/bpf/bpftool/btf_dumper.c b/tools/bpf/bpftool/btf_dumper.c
index 7ca54d046362..92db1fccda49 100644
--- a/tools/bpf/bpftool/btf_dumper.c
+++ b/tools/bpf/bpftool/btf_dumper.c
@@ -65,7 +65,7 @@ static int dump_prog_id_as_func_ptr(const struct btf_dumper *d,
 	info = &prog_info->info;
 
 	if (!info->btf_id || !info->nr_func_info ||
-	    btf__get_from_id(info->btf_id, &prog_btf))
+	    btf__load_from_kernel_by_id(info->btf_id, &prog_btf))
 		goto print;
 	finfo = u64_to_ptr(info->func_info);
 	func_type = btf__type_by_id(prog_btf, finfo->type_id);
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 09ae0381205b..69ced1af0ab1 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -807,7 +807,7 @@ static struct btf *get_map_kv_btf(const struct bpf_map_info *info)
 	} else if (info->btf_value_type_id) {
 		int err;
 
-		err = btf__get_from_id(info->btf_id, &btf);
+		err = btf__load_from_kernel_by_id(info->btf_id, &btf);
 		if (err || !btf) {
 			p_err("failed to get btf");
 			btf = err ? ERR_PTR(err) : ERR_PTR(-ESRCH);
@@ -1042,7 +1042,7 @@ static void print_key_value(struct bpf_map_info *info, void *key,
 	struct btf *btf = NULL;
 	int err;
 
-	err = btf__get_from_id(info->btf_id, &btf);
+	err = btf__load_from_kernel_by_id(info->btf_id, &btf);
 	if (err) {
 		p_err("failed to get btf");
 		return;
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index cc48726740ad..663828f96358 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -263,7 +263,7 @@ static void show_prog_metadata(int fd, __u32 num_maps)
 	if (!value)
 		return;
 
-	err = btf__get_from_id(map_info.btf_id, &btf);
+	err = btf__load_from_kernel_by_id(map_info.btf_id, &btf);
 	if (err || !btf)
 		goto out_free;
 
@@ -646,7 +646,7 @@ prog_dump(struct bpf_prog_info *info, enum dump_mode mode,
 		member_len = info->xlated_prog_len;
 	}
 
-	if (info->btf_id && btf__get_from_id(info->btf_id, &btf)) {
+	if (info->btf_id && btf__load_from_kernel_by_id(info->btf_id, &btf)) {
 		p_err("failed to get btf");
 		return -1;
 	}
@@ -2013,7 +2013,7 @@ static char *profile_target_name(int tgt_fd)
 	}
 
 	if (info_linear->info.btf_id == 0 ||
-	    btf__get_from_id(info_linear->info.btf_id, &btf)) {
+	    btf__load_from_kernel_by_id(info_linear->info.btf_id, &btf)) {
 		p_err("prog FD %d doesn't have valid btf", tgt_fd);
 		goto out;
 	}
diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index cdecda1ddd36..5e0aa7d379f0 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -223,7 +223,7 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_session *session,
 			free(info_linear);
 			return -1;
 		}
-		if (btf__get_from_id(info->btf_id, &btf)) {
+		if (btf__load_from_kernel_by_id(info->btf_id, &btf)) {
 			pr_debug("%s: failed to get BTF of id %u, aborting\n", __func__, info->btf_id);
 			err = -1;
 			btf = NULL;
@@ -478,7 +478,7 @@ static void perf_env__add_bpf_info(struct perf_env *env, u32 id)
 	if (btf_id == 0)
 		goto out;
 
-	if (btf__get_from_id(btf_id, &btf)) {
+	if (btf__load_from_kernel_by_id(btf_id, &btf)) {
 		pr_debug("%s: failed to get BTF of id %u, aborting\n",
 			 __func__, btf_id);
 		goto out;
diff --git a/tools/perf/util/bpf_counter.c b/tools/perf/util/bpf_counter.c
index 5ed674a2f55e..9b9d24016772 100644
--- a/tools/perf/util/bpf_counter.c
+++ b/tools/perf/util/bpf_counter.c
@@ -85,7 +85,7 @@ static char *bpf_target_prog_name(int tgt_fd)
 	}
 
 	if (info_linear->info.btf_id == 0 ||
-	    btf__get_from_id(info_linear->info.btf_id, &btf)) {
+	    btf__load_from_kernel_by_id(info_linear->info.btf_id, &btf)) {
 		pr_debug("prog FD %d doesn't have valid btf\n", tgt_fd);
 		goto out;
 	}
diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
index 857e3f26086f..60e0be02931d 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf.c
@@ -4350,7 +4350,7 @@ static void do_test_file(unsigned int test_num)
 		goto done;
 	}
 
-	err = btf__get_from_id(info.btf_id, &btf);
+	err = btf__load_from_kernel_by_id(info.btf_id, &btf);
 	if (CHECK(err, "cannot get btf from kernel, err: %d", err))
 		goto done;
 
-- 
2.30.2

[PATCH bpf-next 4/6] libbpf: explicitly mark btf__load() and btf__get_from_id() as deprecated

[Quentin Monnet]

Now that we have replacement functions for btf__load() and
btf__get_from_id(), with names that better reflect what the functions
do, and that we have updated the different tools in the repository
calling the legacy functions, let's explicitly mark those as deprecated.

References:

- https://github.com/libbpf/libbpf/issues/278
- https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/lib/bpf/btf.h | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index 0bd9d3952d19..522277b16a88 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -45,7 +45,8 @@ LIBBPF_API struct btf *btf__parse_raw(const char *path);
 LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf);
 
 LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf);
-LIBBPF_API int btf__load(struct btf *btf);
+LIBBPF_API LIBBPF_DEPRECATED("the name was confusing and will be removed in the future libbpf versions, please use btf__load_into_kernel() instead")
+int btf__load(struct btf *btf);
 LIBBPF_API int btf__load_into_kernel(struct btf *btf);
 LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,
 				   const char *type_name);
@@ -67,7 +68,8 @@ LIBBPF_API void btf__set_fd(struct btf *btf, int fd);
 LIBBPF_API const void *btf__get_raw_data(const struct btf *btf, __u32 *size);
 LIBBPF_API const char *btf__name_by_offset(const struct btf *btf, __u32 offset);
 LIBBPF_API const char *btf__str_by_offset(const struct btf *btf, __u32 offset);
-LIBBPF_API int btf__get_from_id(__u32 id, struct btf **btf);
+LIBBPF_API LIBBPF_DEPRECATED("the name was confusing and will be removed in the future libbpf versions, please use btf__load_from_kernel_by_id() instead")
+int btf__get_from_id(__u32 id, struct btf **btf);
 LIBBPF_API int btf__load_from_kernel_by_id(__u32 id, struct btf **btf);
 LIBBPF_API int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
 				    __u32 expected_key_size,
-- 
2.30.2

[PATCH bpf-next 5/6] libbpf: add split BTF support for btf__load_from_kernel_by_id()

[Quentin Monnet]

Add a new API function btf__load_from_kernel_by_id_split(), which takes
a pointer to a base BTF object in order to support split BTF objects
when retrieving BTF information from the kernel.

Reference: https://github.com/libbpf/libbpf/issues/314

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/lib/bpf/btf.c      | 10 ++++++++--
 tools/lib/bpf/btf.h      |  2 ++
 tools/lib/bpf/libbpf.map |  1 +
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 05b63b63083a..15967dd80ffb 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -1383,7 +1383,8 @@ struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf)
 	return btf;
 }
 
-int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
+int btf__load_from_kernel_by_id_split(__u32 id, struct btf **btf,
+				      struct btf *base_btf)
 {
 	struct btf *res;
 	int err, btf_fd;
@@ -1393,7 +1394,7 @@ int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
 	if (btf_fd < 0)
 		return libbpf_err(-errno);
 
-	res = btf_get_from_fd(btf_fd, NULL);
+	res = btf_get_from_fd(btf_fd, base_btf);
 	err = libbpf_get_error(res);
 
 	close(btf_fd);
@@ -1407,6 +1408,11 @@ int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
 int btf__get_from_id(__u32, struct btf **)
 	__attribute__((alias("btf__load_from_kernel_by_id")));
 
+int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
+{
+	return btf__load_from_kernel_by_id_split(id, btf, NULL);
+}
+
 int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
 			 __u32 expected_key_size, __u32 expected_value_size,
 			 __u32 *key_type_id, __u32 *value_type_id)
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index 522277b16a88..62291d3cc9c6 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -71,6 +71,8 @@ LIBBPF_API const char *btf__str_by_offset(const struct btf *btf, __u32 offset);
 LIBBPF_API LIBBPF_DEPRECATED("the name was confusing and will be removed in the future libbpf versions, please use btf__load_from_kernel_by_id() instead")
 int btf__get_from_id(__u32 id, struct btf **btf);
 LIBBPF_API int btf__load_from_kernel_by_id(__u32 id, struct btf **btf);
+LIBBPF_API int btf__load_from_kernel_by_id_split(__u32 id, struct btf **btf,
+						 struct btf *base_btf);
 LIBBPF_API int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
 				    __u32 expected_key_size,
 				    __u32 expected_value_size,
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index a687cc63cd80..f8420a6d7872 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -379,5 +379,6 @@ LIBBPF_0.5.0 {
 LIBBPF_0.6.0 {
 	global:
 		btf__load_from_kernel_by_id;
+		btf__load_from_kernel_by_id_split;
 		btf__load_into_kernel;
 } LIBBPF_0.5.0;
-- 
2.30.2

[PATCH bpf-next 6/6] tools: bpftool: support dumping split BTF by id

[Quentin Monnet]

Split BTF objects are typically BTF objects for kernel modules, which
are incrementally built on top of kernel BTF instead of redefining all
kernel symbols they need. We can use bpftool with its -B command-line
option to dump split BTF objects. It works well when the handle provided
for the BTF object to dump is a "path" to the BTF object, typically
under /sys/kernel/btf, because bpftool internally calls
btf__parse_split() which can take a "base_btf" pointer and resolve the
BTF reconstruction (although in that case, the "-B" option is
unnecessary because bpftool performs autodetection).

However, it did not work so far when passing the BTF object through its
id, because bpftool would call btf__get_from_id() which did not provide
a way to pass a "base_btf" pointer.

In other words, the following works:

    # bpftool btf dump file /sys/kernel/btf/i2c_smbus -B /sys/kernel/btf/vmlinux

But this was not possible:

    # bpftool btf dump id 6 -B /sys/kernel/btf/vmlinux

The libbpf API has recently changed, and btf__get_from_id() has been
deprecated in favour of btf__load_from_kernel_by_id() and its version
with support for split BTF, btf__load_from_kernel_by_id_split(). Let's
update bpftool to make it able to dump the BTF object in the second case
as well.

Signed-off-by: Quentin Monnet <quentin@isovalent.com>
---
 tools/bpf/bpftool/btf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 2296e8eba0ff..b77a59225f5b 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -580,7 +580,7 @@ static int do_dump(int argc, char **argv)
 	}
 
 	if (!btf) {
-		err = btf__load_from_kernel_by_id(btf_id, &btf);
+		err = btf__load_from_kernel_by_id_split(btf_id, &btf, base_btf);
 		if (err) {
 			p_err("get btf by id (%u): %s", btf_id, strerror(err));
 			goto done;
-- 
2.30.2

RE: [PATCH bpf-next 0/6] libbpf: rename btf__get_from_id() and btf__load() APIs, support split BTF

[John Fastabend]

Quentin Monnet wrote:
> As part of the effort to move towards a v1.0 for libbpf [0], this set
> improves some confusing function names related to BTF loading from and to
> the kernel:
> 
> - btf__load() becomes btf__load_into_kernel().
> - btf__get_from_id becomes btf__load_from_kernel_by_id().
> - A new version btf__load_from_kernel_by_id_split() extends the former to
>   add support for split BTF.
> 
> The old functions are not removed yet, but marked as deprecated.
> 
> The last patch is a trivial change to bpftool to add support for dumping
> split BTF objects by referencing them by their id (and not only by their
> BTF path).
> 
> [0] https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis
> 
> Quentin Monnet (6):
>   libbpf: rename btf__load() as btf__load_into_kernel()
>   libbpf: rename btf__get_from_id() as btf__load_from_kernel_by_id()
>   tools: replace btf__get_from_id() with btf__load_from_kernel_by_id()
>   libbpf: explicitly mark btf__load() and btf__get_from_id() as
>     deprecated
>   libbpf: add split BTF support for btf__load_from_kernel_by_id()
>   tools: bpftool: support dumping split BTF by id
> 
>  tools/bpf/bpftool/btf.c                      |  2 +-
>  tools/bpf/bpftool/btf_dumper.c               |  2 +-
>  tools/bpf/bpftool/map.c                      |  4 ++--
>  tools/bpf/bpftool/prog.c                     |  6 +++---
>  tools/lib/bpf/btf.c                          | 15 ++++++++++++---
>  tools/lib/bpf/btf.h                          | 10 ++++++++--
>  tools/lib/bpf/libbpf.c                       |  4 ++--
>  tools/lib/bpf/libbpf.map                     |  7 +++++++
>  tools/perf/util/bpf-event.c                  |  4 ++--
>  tools/perf/util/bpf_counter.c                |  2 +-
>  tools/testing/selftests/bpf/prog_tests/btf.c |  2 +-
>  11 files changed, 40 insertions(+), 18 deletions(-)
> 
> -- 
> 2.30.2
> 

For the series.

Acked-by: John Fastabend <john.fastabend@gmail.com>

Re: [PATCH bpf-next 1/6] libbpf: rename btf__load() as btf__load_into_kernel()

[Andrii Nakryiko]

On Wed, Jul 14, 2021 at 7:15 AM Quentin Monnet <quentin@isovalent.com> wrote:
>
> As part of the effort to move towards a v1.0 for libbpf, rename
> btf__load() function, used to "upload" BTF information into the kernel,
> and rename it instead as btf__load_into_kernel().
>
> This new name better reflects what the function does, and should be less
> confusing.
>
> References:
>
> - https://github.com/libbpf/libbpf/issues/278
> - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis
>
> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> ---
>  tools/lib/bpf/btf.c      | 3 ++-
>  tools/lib/bpf/btf.h      | 1 +
>  tools/lib/bpf/libbpf.c   | 2 +-
>  tools/lib/bpf/libbpf.map | 5 +++++
>  4 files changed, 9 insertions(+), 2 deletions(-)
>

[...]

> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index 944c99d1ded3..d42f20b0e9e4 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -375,3 +375,8 @@ LIBBPF_0.5.0 {
>                 bpf_object__gen_loader;
>                 libbpf_set_strict_mode;
>  } LIBBPF_0.4.0;
> +
> +LIBBPF_0.6.0 {

we haven't released v0.5 yet, so this will go into 0.5, probably

> +       global:
> +               btf__load_into_kernel;
> +} LIBBPF_0.5.0;
> --
> 2.30.2
>

Re: [PATCH bpf-next 2/6] libbpf: rename btf__get_from_id() as btf__load_from_kernel_by_id()

[Andrii Nakryiko]

On Wed, Jul 14, 2021 at 7:15 AM Quentin Monnet <quentin@isovalent.com> wrote:
>
> Rename function btf__get_from_id() as btf__load_from_kernel_by_id() to
> better indicate what the function does.
>
> The other tools calling the deprecated btf__get_from_id() function will
> be updated in a future commit.
>
> References:
>
> - https://github.com/libbpf/libbpf/issues/278
> - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis
>
> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> ---
>  tools/lib/bpf/btf.c      | 4 +++-
>  tools/lib/bpf/btf.h      | 1 +
>  tools/lib/bpf/libbpf.c   | 2 +-
>  tools/lib/bpf/libbpf.map | 1 +
>  4 files changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 7e0de560490e..05b63b63083a 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -1383,7 +1383,7 @@ struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf)
>         return btf;
>  }
>
> -int btf__get_from_id(__u32 id, struct btf **btf)
> +int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)

we can't change existing btf__get_from_id(), but for the new
btf__load_from_kernel_by_id() let's keep them in line with other BTF
"constructor" APIs (btf__new and btf__parse) and return resulting
struct btf, instead of passing it in output argument. With the new
libbpf error reporting strategy (NULL on error + errno for those who
care about specific error code), it has a better usability and will
just be more consistent.

>  {
>         struct btf *res;
>         int err, btf_fd;

Re: [PATCH bpf-next 4/6] libbpf: explicitly mark btf__load() and btf__get_from_id() as deprecated

[Andrii Nakryiko]

On Wed, Jul 14, 2021 at 7:15 AM Quentin Monnet <quentin@isovalent.com> wrote:
>
> Now that we have replacement functions for btf__load() and
> btf__get_from_id(), with names that better reflect what the functions
> do, and that we have updated the different tools in the repository
> calling the legacy functions, let's explicitly mark those as deprecated.
>
> References:
>
> - https://github.com/libbpf/libbpf/issues/278
> - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis
>
> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> ---
>  tools/lib/bpf/btf.h | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
> index 0bd9d3952d19..522277b16a88 100644
> --- a/tools/lib/bpf/btf.h
> +++ b/tools/lib/bpf/btf.h
> @@ -45,7 +45,8 @@ LIBBPF_API struct btf *btf__parse_raw(const char *path);
>  LIBBPF_API struct btf *btf__parse_raw_split(const char *path, struct btf *base_btf);
>
>  LIBBPF_API int btf__finalize_data(struct bpf_object *obj, struct btf *btf);
> -LIBBPF_API int btf__load(struct btf *btf);
> +LIBBPF_API LIBBPF_DEPRECATED("the name was confusing and will be removed in the future libbpf versions, please use btf__load_into_kernel() instead")
> +int btf__load(struct btf *btf);

So I learned from my previous attempts to deprecate libbpf API to
never do that in the same release that introduces a replacement API.
So I'll postpone this change till the next libbpf release, otherwise
it causes too much churn and frustration for existing users.

And I'd probably keep the message short, like "use
btf__load_into_kernel() instead".

>  LIBBPF_API int btf__load_into_kernel(struct btf *btf);
>  LIBBPF_API __s32 btf__find_by_name(const struct btf *btf,
>                                    const char *type_name);
> @@ -67,7 +68,8 @@ LIBBPF_API void btf__set_fd(struct btf *btf, int fd);
>  LIBBPF_API const void *btf__get_raw_data(const struct btf *btf, __u32 *size);
>  LIBBPF_API const char *btf__name_by_offset(const struct btf *btf, __u32 offset);
>  LIBBPF_API const char *btf__str_by_offset(const struct btf *btf, __u32 offset);
> -LIBBPF_API int btf__get_from_id(__u32 id, struct btf **btf);
> +LIBBPF_API LIBBPF_DEPRECATED("the name was confusing and will be removed in the future libbpf versions, please use btf__load_from_kernel_by_id() instead")
> +int btf__get_from_id(__u32 id, struct btf **btf);
>  LIBBPF_API int btf__load_from_kernel_by_id(__u32 id, struct btf **btf);
>  LIBBPF_API int btf__get_map_kv_tids(const struct btf *btf, const char *map_name,
>                                     __u32 expected_key_size,
> --
> 2.30.2
>

Re: [PATCH bpf-next 5/6] libbpf: add split BTF support for btf__load_from_kernel_by_id()

[Andrii Nakryiko]

On Wed, Jul 14, 2021 at 7:15 AM Quentin Monnet <quentin@isovalent.com> wrote:
>
> Add a new API function btf__load_from_kernel_by_id_split(), which takes
> a pointer to a base BTF object in order to support split BTF objects
> when retrieving BTF information from the kernel.
>
> Reference: https://github.com/libbpf/libbpf/issues/314
>
> Signed-off-by: Quentin Monnet <quentin@isovalent.com>
> ---
>  tools/lib/bpf/btf.c      | 10 ++++++++--
>  tools/lib/bpf/btf.h      |  2 ++
>  tools/lib/bpf/libbpf.map |  1 +
>  3 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 05b63b63083a..15967dd80ffb 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -1383,7 +1383,8 @@ struct btf *btf_get_from_fd(int btf_fd, struct btf *base_btf)
>         return btf;
>  }
>
> -int btf__load_from_kernel_by_id(__u32 id, struct btf **btf)
> +int btf__load_from_kernel_by_id_split(__u32 id, struct btf **btf,
> +                                     struct btf *base_btf)

here all those struct btf pointers are even more confusing, let's
return the resulting struct btf * as a direct result value

>  {
>         struct btf *res;
>         int err, btf_fd;

[...]

Re: [PATCH bpf-next 1/6] libbpf: rename btf__load() as btf__load_into_kernel()

[Quentin Monnet]

2021-07-15 21:32 UTC-0700 ~ Andrii Nakryiko <andrii.nakryiko@gmail.com>

> we haven't released v0.5 yet, so this will go into 0.5, probably

Oh my, I still haven't done the mental switch since the old release
model. v2 is coming, with this and your other points hopefully
addressed. Thanks a lot for the review!

Quentin