* [PATCH bpf-next v5 01/10] bpf: btf: Introduce BPF Type Format (BTF)
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
@ 2018-04-18 22:55 ` Martin KaFai Lau
2018-04-18 22:55 ` [PATCH bpf-next v5 02/10] bpf: btf: Validate type reference Martin KaFai Lau
` (10 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:55 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch introduces BPF type Format (BTF).
BTF (BPF Type Format) is the meta data format which describes
the data types of BPF program/map. Hence, it basically focus
on the C programming language which the modern BPF is primary
using. The first use case is to provide a generic pretty print
capability for a BPF map.
BTF has its root from CTF (Compact C-Type format). To simplify
the handling of BTF data, BTF removes the differences between
small and big type/struct-member. Hence, BTF consistently uses u32
instead of supporting both "one u16" and "two u32 (+padding)" in
describing type and struct-member.
It also raises the number of types (and functions) limit
from 0x7fff to 0x7fffffff.
Due to the above changes, the format is not compatible to CTF.
Hence, BTF starts with a new BTF_MAGIC and version number.
This patch does the first verification pass to the BTF. The first
pass checks:
1. meta-data size (e.g. It does not go beyond the total btf's size)
2. name_offset is valid
3. Each BTF_KIND (e.g. int, enum, struct....) does its
own check of its meta-data.
Some other checks, like checking a struct's member is referring
to a valid type, can only be done in the second pass. The second
verification pass will be implemented in the next patch.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/uapi/linux/btf.h | 130 +++++++
kernel/bpf/Makefile | 1 +
kernel/bpf/btf.c | 915 +++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 1046 insertions(+)
create mode 100644 include/uapi/linux/btf.h
create mode 100644 kernel/bpf/btf.c
diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
new file mode 100644
index 000000000000..74a30b1090df
--- /dev/null
+++ b/include/uapi/linux/btf.h
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* Copyright (c) 2018 Facebook */
+#ifndef _UAPI__LINUX_BTF_H__
+#define _UAPI__LINUX_BTF_H__
+
+#include <linux/types.h>
+
+#define BTF_MAGIC 0xeB9F
+#define BTF_MAGIC_SWAP 0x9FeB
+#define BTF_VERSION 1
+#define BTF_FLAGS_COMPR 0x01
+
+struct btf_header {
+ __u16 magic;
+ __u8 version;
+ __u8 flags;
+
+ __u32 parent_label;
+ __u32 parent_name;
+
+ /* All offsets are in bytes relative to the end of this header */
+ __u32 label_off; /* offset of label section */
+ __u32 object_off; /* offset of data object section*/
+ __u32 func_off; /* offset of function section */
+ __u32 type_off; /* offset of type section */
+ __u32 str_off; /* offset of string section */
+ __u32 str_len; /* length of string section */
+};
+
+/* Max # of type identifier */
+#define BTF_MAX_TYPE 0x7fffffff
+/* Max offset into the string section */
+#define BTF_MAX_NAME_OFFSET 0x7fffffff
+/* Max # of struct/union/enum members or func args */
+#define BTF_MAX_VLEN 0xffff
+
+/* The type id is referring to a parent BTF */
+#define BTF_TYPE_PARENT(id) (((id) >> 31) & 0x1)
+#define BTF_TYPE_ID(id) ((id) & BTF_MAX_TYPE)
+
+/* String is in the ELF string section */
+#define BTF_STR_TBL_ELF_ID(ref) (((ref) >> 31) & 0x1)
+#define BTF_STR_OFFSET(ref) ((ref) & BTF_MAX_NAME_OFFSET)
+
+struct btf_type {
+ __u32 name;
+ /* "info" bits arrangement
+ * bits 0-15: vlen (e.g. # of struct's members)
+ * bits 16-23: unused
+ * bits 24-28: kind (e.g. int, ptr, array...etc)
+ * bits 29-30: unused
+ * bits 31: root
+ */
+ __u32 info;
+ /* "size" is used by INT, ENUM, STRUCT and UNION.
+ * "size" tells the size of the type it is describing.
+ *
+ * "type" is used by PTR, TYPEDEF, VOLATILE, CONST and RESTRICT.
+ * "type" is a type_id referring to another type.
+ */
+ union {
+ __u32 size;
+ __u32 type;
+ };
+};
+
+#define BTF_INFO_KIND(info) (((info) >> 24) & 0x1f)
+#define BTF_INFO_ISROOT(info) (!!(((info) >> 24) & 0x80))
+#define BTF_INFO_VLEN(info) ((info) & 0xffff)
+
+#define BTF_KIND_UNKN 0 /* Unknown */
+#define BTF_KIND_INT 1 /* Integer */
+#define BTF_KIND_PTR 2 /* Pointer */
+#define BTF_KIND_ARRAY 3 /* Array */
+#define BTF_KIND_STRUCT 4 /* Struct */
+#define BTF_KIND_UNION 5 /* Union */
+#define BTF_KIND_ENUM 6 /* Enumeration */
+#define BTF_KIND_FWD 7 /* Forward */
+#define BTF_KIND_TYPEDEF 8 /* Typedef */
+#define BTF_KIND_VOLATILE 9 /* Volatile */
+#define BTF_KIND_CONST 10 /* Const */
+#define BTF_KIND_RESTRICT 11 /* Restrict */
+#define BTF_KIND_MAX 11
+#define NR_BTF_KINDS 12
+
+/* For some specific BTF_KIND, "struct btf_type" is immediately
+ * followed by extra data.
+ */
+
+/* BTF_KIND_INT is followed by a u32 and the following
+ * is the 32 bits arrangement:
+ */
+#define BTF_INT_ENCODING(VAL) (((VAL) & 0xff000000) >> 24)
+#define BTF_INT_OFFSET(VAL) (((VAL & 0x00ff0000)) >> 16)
+#define BTF_INT_BITS(VAL) ((VAL) & 0x0000ffff)
+
+/* Attributes stored in the BTF_INT_ENCODING */
+#define BTF_INT_SIGNED 0x1
+#define BTF_INT_CHAR 0x2
+#define BTF_INT_BOOL 0x4
+#define BTF_INT_VARARGS 0x8
+
+/* BTF_KIND_ENUM is followed by multiple "struct btf_enum".
+ * The exact number of btf_enum is stored in the vlen (of the
+ * info in "struct btf_type").
+ */
+struct btf_enum {
+ __u32 name;
+ __s32 val;
+};
+
+/* BTF_KIND_ARRAY is followed by one "struct btf_array" */
+struct btf_array {
+ __u32 type;
+ __u32 index_type;
+ __u32 nelems;
+};
+
+/* BTF_KIND_STRUCT and BTF_KIND_UNION are followed
+ * by multiple "struct btf_member". The exact number
+ * of btf_member is stored in the vlen (of the info in
+ * "struct btf_type").
+ */
+struct btf_member {
+ __u32 name;
+ __u32 type;
+ __u32 offset; /* offset in bits */
+};
+
+#endif /* _UAPI__LINUX_BTF_H__ */
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index a713fd23ec88..35c485fa9ea3 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -4,6 +4,7 @@ obj-y := core.o
obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o
obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o
obj-$(CONFIG_BPF_SYSCALL) += disasm.o
+obj-$(CONFIG_BPF_SYSCALL) += btf.o
ifeq ($(CONFIG_NET),y)
obj-$(CONFIG_BPF_SYSCALL) += devmap.o
obj-$(CONFIG_BPF_SYSCALL) += cpumap.o
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
new file mode 100644
index 000000000000..26e9ed7cea5f
--- /dev/null
+++ b/kernel/bpf/btf.c
@@ -0,0 +1,915 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+
+#include <uapi/linux/btf.h>
+#include <uapi/linux/types.h>
+#include <linux/compiler.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/kernel.h>
+#include <linux/bpf_verifier.h>
+#include <linux/btf.h>
+
+/* BTF (BPF Type Format) is the meta data format which describes
+ * the data types of BPF program/map. Hence, it basically focus
+ * on the C programming language which the modern BPF is primary
+ * using.
+ *
+ * ELF Section:
+ * ~~~~~~~~~~~
+ * The BTF data is stored under the ".BTF" ELF section
+ *
+ * struct btf_type:
+ * ~~~~~~~~~~~~~~~
+ * Each 'struct btf_type' object describes a C data type.
+ * Depending on the type it is describing, a 'struct btf_type'
+ * object may be followed by more data. F.e.
+ * To describe an array, 'struct btf_type' is followed by
+ * 'struct btf_array'.
+ *
+ * 'struct btf_type' and any extra data following it are
+ * 4 bytes aligned.
+ *
+ * Type section:
+ * ~~~~~~~~~~~~~
+ * The BTF type section contains a list of 'struct btf_type' objects.
+ * Each one describes a C type. Recall from the above section
+ * that a 'struct btf_type' object could be immediately followed by extra
+ * data in order to desribe some particular C types.
+ *
+ * type_id:
+ * ~~~~~~~
+ * Each btf_type object is identified by a type_id. The type_id
+ * is implicitly implied by the location of the btf_type object in
+ * the BTF type section. The first one has type_id 1. The second
+ * one has type_id 2...etc. Hence, an earlier btf_type has
+ * a smaller type_id.
+ *
+ * A btf_type object may refer to another btf_type object by using
+ * type_id (i.e. the "type" in the "struct btf_type").
+ *
+ * NOTE that we cannot assume any reference-order.
+ * A btf_type object can refer to an earlier btf_type object
+ * but it can also refer to a later btf_type object.
+ *
+ * For example, to describe "const void *". A btf_type
+ * object describing "const" may refer to another btf_type
+ * object describing "void *". This type-reference is done
+ * by specifying type_id:
+ *
+ * [1] CONST (anon) type_id=2
+ * [2] PTR (anon) type_id=0
+ *
+ * The above is the btf_verifier debug log:
+ * - Each line started with "[?]" is a btf_type object
+ * - [?] is the type_id of the btf_type object.
+ * - CONST/PTR is the BTF_KIND_XXX
+ * - "(anon)" is the name of the type. It just
+ * happens that CONST and PTR has no name.
+ * - type_id=XXX is the 'u32 type' in btf_type
+ *
+ * NOTE: "void" has type_id 0
+ *
+ * String section:
+ * ~~~~~~~~~~~~~~
+ * The BTF string section contains the names used by the type section.
+ * Each string is referred by an "offset" from the beginning of the
+ * string section.
+ *
+ * Each string is '\0' terminated.
+ *
+ * The first character in the string section must be '\0'
+ * which is used to mean 'anonymous'. Some btf_type may not
+ * have a name.
+ */
+
+/* BTF verification:
+ *
+ * To verify BTF data, two passes are needed.
+ *
+ * Pass #1
+ * ~~~~~~~
+ * The first pass is to collect all btf_type objects to
+ * an array: "btf->types".
+ *
+ * Depending on the C type that a btf_type is describing,
+ * a btf_type may be followed by extra data. We don't know
+ * how many btf_type is there, and more importantly we don't
+ * know where each btf_type is located in the type section.
+ *
+ * Without knowing the location of each type_id, most verifications
+ * cannot be done. e.g. an earlier btf_type may refer to a later
+ * btf_type (recall the "const void *" above), so we cannot
+ * check this type-reference in the first pass.
+ *
+ * In the first pass, it still does some verifications (e.g.
+ * checking the name is a valid offset to the string section).
+ */
+
+#define BITS_PER_U64 (sizeof(u64) * BITS_PER_BYTE)
+#define BITS_PER_BYTE_MASK (BITS_PER_BYTE - 1)
+#define BITS_PER_BYTE_MASKED(bits) ((bits) & BITS_PER_BYTE_MASK)
+#define BITS_ROUNDDOWN_BYTES(bits) ((bits) >> 3)
+#define BITS_ROUNDUP_BYTES(bits) \
+ (BITS_ROUNDDOWN_BYTES(bits) + !!BITS_PER_BYTE_MASKED(bits))
+
+/* 16MB for 64k structs and each has 16 members and
+ * a few MB spaces for the string section.
+ * The hard limit is S32_MAX.
+ */
+#define BTF_MAX_SIZE (16 * 1024 * 1024)
+/* 64k. We can raise it later. The hard limit is S32_MAX. */
+#define BTF_MAX_NR_TYPES 65535
+
+#define for_each_member(i, struct_type, member) \
+ for (i = 0, member = btf_type_member(struct_type); \
+ i < btf_type_vlen(struct_type); \
+ i++, member++)
+
+struct btf {
+ union {
+ struct btf_header *hdr;
+ void *data;
+ };
+ struct btf_type **types;
+ const char *strings;
+ void *nohdr_data;
+ u32 nr_types;
+ u32 types_size;
+ u32 data_size;
+};
+
+struct btf_verifier_env {
+ struct btf *btf;
+ struct bpf_verifier_log log;
+ u32 log_type_id;
+};
+
+static const char * const btf_kind_str[NR_BTF_KINDS] = {
+ [BTF_KIND_UNKN] = "UNKNOWN",
+ [BTF_KIND_INT] = "INT",
+ [BTF_KIND_PTR] = "PTR",
+ [BTF_KIND_ARRAY] = "ARRAY",
+ [BTF_KIND_STRUCT] = "STRUCT",
+ [BTF_KIND_UNION] = "UNION",
+ [BTF_KIND_ENUM] = "ENUM",
+ [BTF_KIND_FWD] = "FWD",
+ [BTF_KIND_TYPEDEF] = "TYPEDEF",
+ [BTF_KIND_VOLATILE] = "VOLATILE",
+ [BTF_KIND_CONST] = "CONST",
+ [BTF_KIND_RESTRICT] = "RESTRICT",
+};
+
+struct btf_kind_operations {
+ s32 (*check_meta)(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left);
+ void (*log_details)(struct btf_verifier_env *env,
+ const struct btf_type *t);
+};
+
+static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS];
+static struct btf_type btf_void;
+
+static const char *btf_int_encoding_str(u8 encoding)
+{
+ if (encoding == 0)
+ return "(none)";
+ else if (encoding == BTF_INT_SIGNED)
+ return "SIGNED";
+ else if (encoding == BTF_INT_CHAR)
+ return "CHAR";
+ else if (encoding == BTF_INT_BOOL)
+ return "BOOL";
+ else if (encoding == BTF_INT_VARARGS)
+ return "VARARGS";
+ else
+ return "UNKN";
+}
+
+static u16 btf_type_vlen(const struct btf_type *t)
+{
+ return BTF_INFO_VLEN(t->info);
+}
+
+static u32 btf_type_int(const struct btf_type *t)
+{
+ return *(u32 *)(t + 1);
+}
+
+static const struct btf_array *btf_type_array(const struct btf_type *t)
+{
+ return (const struct btf_array *)(t + 1);
+}
+
+static const struct btf_member *btf_type_member(const struct btf_type *t)
+{
+ return (const struct btf_member *)(t + 1);
+}
+
+static const struct btf_enum *btf_type_enum(const struct btf_type *t)
+{
+ return (const struct btf_enum *)(t + 1);
+}
+
+static const struct btf_kind_operations *btf_type_ops(const struct btf_type *t)
+{
+ return kind_ops[BTF_INFO_KIND(t->info)];
+}
+
+static bool btf_name_offset_valid(const struct btf *btf, u32 offset)
+{
+ return !BTF_STR_TBL_ELF_ID(offset) &&
+ BTF_STR_OFFSET(offset) < btf->hdr->str_len;
+}
+
+static const char *btf_name_by_offset(const struct btf *btf, u32 offset)
+{
+ if (!BTF_STR_OFFSET(offset))
+ return "(anon)";
+ else if (BTF_STR_OFFSET(offset) < btf->hdr->str_len)
+ return &btf->strings[BTF_STR_OFFSET(offset)];
+ else
+ return "(invalid-name-offset)";
+}
+
+__printf(2, 3) static void __btf_verifier_log(struct bpf_verifier_log *log,
+ const char *fmt, ...)
+{
+ va_list args;
+
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+}
+
+__printf(2, 3) static void btf_verifier_log(struct btf_verifier_env *env,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+}
+
+__printf(4, 5) static void __btf_verifier_log_type(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ bool log_details,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ u8 kind = BTF_INFO_KIND(t->info);
+ struct btf *btf = env->btf;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ __btf_verifier_log(log, "[%u] %s %s%s",
+ env->log_type_id,
+ btf_kind_str[kind],
+ btf_name_by_offset(btf, t->name),
+ log_details ? " " : "");
+
+ if (log_details)
+ btf_type_ops(t)->log_details(env, t);
+
+ if (fmt && *fmt) {
+ __btf_verifier_log(log, " ");
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+ }
+
+ __btf_verifier_log(log, "\n");
+}
+
+#define btf_verifier_log_type(env, t, ...) \
+ __btf_verifier_log_type((env), (t), true, __VA_ARGS__)
+#define btf_verifier_log_basic(env, t, ...) \
+ __btf_verifier_log_type((env), (t), false, __VA_ARGS__)
+
+__printf(4, 5)
+static void btf_verifier_log_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ struct btf *btf = env->btf;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ __btf_verifier_log(log, "\t%s type_id=%u bits_offset=%u",
+ btf_name_by_offset(btf, member->name),
+ member->type, member->offset);
+
+ if (fmt && *fmt) {
+ __btf_verifier_log(log, " ");
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+ }
+
+ __btf_verifier_log(log, "\n");
+}
+
+static void btf_verifier_log_hdr(struct btf_verifier_env *env)
+{
+ struct bpf_verifier_log *log = &env->log;
+ const struct btf *btf = env->btf;
+ const struct btf_header *hdr;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ hdr = btf->hdr;
+ __btf_verifier_log(log, "magic: 0x%x\n", hdr->magic);
+ __btf_verifier_log(log, "version: %u\n", hdr->version);
+ __btf_verifier_log(log, "flags: 0x%x\n", hdr->flags);
+ __btf_verifier_log(log, "parent_label: %u\n", hdr->parent_label);
+ __btf_verifier_log(log, "parent_name: %u\n", hdr->parent_name);
+ __btf_verifier_log(log, "label_off: %u\n", hdr->label_off);
+ __btf_verifier_log(log, "object_off: %u\n", hdr->object_off);
+ __btf_verifier_log(log, "func_off: %u\n", hdr->func_off);
+ __btf_verifier_log(log, "type_off: %u\n", hdr->type_off);
+ __btf_verifier_log(log, "str_off: %u\n", hdr->str_off);
+ __btf_verifier_log(log, "str_len: %u\n", hdr->str_len);
+ __btf_verifier_log(log, "btf_total_size: %u\n", btf->data_size);
+}
+
+static int btf_add_type(struct btf_verifier_env *env, struct btf_type *t)
+{
+ struct btf *btf = env->btf;
+
+ /* < 2 because +1 for btf_void which is always in btf->types[0].
+ * btf_void is not accounted in btf->nr_types because btf_void
+ * does not come from the BTF file.
+ */
+ if (btf->types_size - btf->nr_types < 2) {
+ /* Expand 'types' array */
+
+ struct btf_type **new_types;
+ u32 expand_by, new_size;
+
+ if (btf->types_size == BTF_MAX_NR_TYPES) {
+ btf_verifier_log(env, "Exceeded max num of types");
+ return -E2BIG;
+ }
+
+ expand_by = max_t(u32, btf->types_size >> 2, 16);
+ new_size = min_t(u32, BTF_MAX_NR_TYPES,
+ btf->types_size + expand_by);
+
+ new_types = kvzalloc(new_size * sizeof(*new_types),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!new_types)
+ return -ENOMEM;
+
+ if (btf->nr_types == 0)
+ new_types[0] = &btf_void;
+ else
+ memcpy(new_types, btf->types,
+ sizeof(*btf->types) * (btf->nr_types + 1));
+
+ kvfree(btf->types);
+ btf->types = new_types;
+ btf->types_size = new_size;
+ }
+
+ btf->types[++(btf->nr_types)] = t;
+
+ return 0;
+}
+
+static void btf_free(struct btf *btf)
+{
+ kvfree(btf->types);
+ kvfree(btf->data);
+ kfree(btf);
+}
+
+static void btf_verifier_env_free(struct btf_verifier_env *env)
+{
+ kfree(env);
+}
+
+static s32 btf_int_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ u32 int_data, nr_bits, meta_needed = sizeof(int_data);
+ u16 encoding;
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ int_data = btf_type_int(t);
+ nr_bits = BTF_INT_BITS(int_data) + BTF_INT_OFFSET(int_data);
+
+ if (nr_bits > BITS_PER_U64) {
+ btf_verifier_log_type(env, t, "nr_bits exceeds %zu",
+ BITS_PER_U64);
+ return -EINVAL;
+ }
+
+ if (BITS_ROUNDUP_BYTES(nr_bits) > t->size) {
+ btf_verifier_log_type(env, t, "nr_bits exceeds type_size");
+ return -EINVAL;
+ }
+
+ encoding = BTF_INT_ENCODING(int_data);
+ if (encoding &&
+ encoding != BTF_INT_SIGNED &&
+ encoding != BTF_INT_CHAR &&
+ encoding != BTF_INT_BOOL &&
+ encoding != BTF_INT_VARARGS) {
+ btf_verifier_log_type(env, t, "Unsupported encoding");
+ return -ENOTSUPP;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static void btf_int_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ int int_data = btf_type_int(t);
+
+ btf_verifier_log(env,
+ "size=%u bits_offset=%u nr_bits=%u encoding=%s",
+ t->size, BTF_INT_OFFSET(int_data),
+ BTF_INT_BITS(int_data),
+ btf_int_encoding_str(BTF_INT_ENCODING(int_data)));
+}
+
+static const struct btf_kind_operations int_ops = {
+ .check_meta = btf_int_check_meta,
+ .log_details = btf_int_log,
+};
+
+static int btf_ref_type_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (BTF_TYPE_PARENT(t->type)) {
+ btf_verifier_log_type(env, t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return 0;
+}
+
+static void btf_ref_type_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "type_id=%u", t->type);
+}
+
+static struct btf_kind_operations modifier_ops = {
+ .check_meta = btf_ref_type_check_meta,
+ .log_details = btf_ref_type_log,
+};
+
+static struct btf_kind_operations ptr_ops = {
+ .check_meta = btf_ref_type_check_meta,
+ .log_details = btf_ref_type_log,
+};
+
+static struct btf_kind_operations fwd_ops = {
+ .check_meta = btf_ref_type_check_meta,
+ .log_details = btf_ref_type_log,
+};
+
+static s32 btf_array_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_array *array = btf_type_array(t);
+ u32 meta_needed = sizeof(*array);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ /* We are a little forgiving on array->index_type since
+ * the kernel is not using it.
+ */
+ /* Array elem cannot be in type void,
+ * so !array->type is not allowed.
+ */
+ if (!array->type || BTF_TYPE_PARENT(array->type)) {
+ btf_verifier_log_type(env, t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static void btf_array_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ const struct btf_array *array = btf_type_array(t);
+
+ btf_verifier_log(env, "type_id=%u index_type_id=%u nr_elems=%u",
+ array->type, array->index_type, array->nelems);
+}
+
+static struct btf_kind_operations array_ops = {
+ .check_meta = btf_array_check_meta,
+ .log_details = btf_array_log,
+};
+
+static s32 btf_struct_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ bool is_union = BTF_INFO_KIND(t->info) == BTF_KIND_UNION;
+ const struct btf_member *member;
+ struct btf *btf = env->btf;
+ u32 struct_size = t->size;
+ u32 meta_needed;
+ u16 i;
+
+ meta_needed = btf_type_vlen(t) * sizeof(*member);
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ for_each_member(i, t, member) {
+ if (!btf_name_offset_valid(btf, member->name)) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid member name_offset:%u",
+ member->name);
+ return -EINVAL;
+ }
+
+ /* A member cannot be in type void */
+ if (!member->type || BTF_TYPE_PARENT(member->type)) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (is_union && member->offset) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid member bits_offset");
+ return -EINVAL;
+ }
+
+ if (BITS_ROUNDUP_BYTES(member->offset) > struct_size) {
+ btf_verifier_log_member(env, t, member,
+ "Memmber bits_offset exceeds its struct size");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_member(env, t, member, NULL);
+ }
+
+ return meta_needed;
+}
+
+static void btf_struct_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
+}
+
+static struct btf_kind_operations struct_ops = {
+ .check_meta = btf_struct_check_meta,
+ .log_details = btf_struct_log,
+};
+
+static s32 btf_enum_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_enum *enums = btf_type_enum(t);
+ struct btf *btf = env->btf;
+ u16 i, nr_enums;
+ u32 meta_needed;
+
+ nr_enums = btf_type_vlen(t);
+ meta_needed = nr_enums * sizeof(*enums);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (t->size != sizeof(int)) {
+ btf_verifier_log_type(env, t, "Expected size:%zu",
+ sizeof(int));
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ for (i = 0; i < nr_enums; i++) {
+ if (!btf_name_offset_valid(btf, enums[i].name)) {
+ btf_verifier_log(env, "\tInvalid name_offset:%u",
+ enums[i].name);
+ return -EINVAL;
+ }
+
+ btf_verifier_log(env, "\t%s val=%d\n",
+ btf_name_by_offset(btf, enums[i].name),
+ enums[i].val);
+ }
+
+ return meta_needed;
+}
+
+static void btf_enum_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
+}
+
+static struct btf_kind_operations enum_ops = {
+ .check_meta = btf_enum_check_meta,
+ .log_details = btf_enum_log,
+};
+
+static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = {
+ [BTF_KIND_INT] = &int_ops,
+ [BTF_KIND_PTR] = &ptr_ops,
+ [BTF_KIND_ARRAY] = &array_ops,
+ [BTF_KIND_STRUCT] = &struct_ops,
+ [BTF_KIND_UNION] = &struct_ops,
+ [BTF_KIND_ENUM] = &enum_ops,
+ [BTF_KIND_FWD] = &fwd_ops,
+ [BTF_KIND_TYPEDEF] = &modifier_ops,
+ [BTF_KIND_VOLATILE] = &modifier_ops,
+ [BTF_KIND_CONST] = &modifier_ops,
+ [BTF_KIND_RESTRICT] = &modifier_ops,
+};
+
+static s32 btf_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ u32 saved_meta_left = meta_left;
+ s32 var_meta_size;
+
+ if (meta_left < sizeof(*t)) {
+ btf_verifier_log(env, "[%u] meta_left:%u meta_needed:%zu",
+ env->log_type_id, meta_left, sizeof(*t));
+ return -EINVAL;
+ }
+ meta_left -= sizeof(*t);
+
+ if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
+ BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
+ btf_verifier_log(env, "[%u] Invalid kind:%u",
+ env->log_type_id, BTF_INFO_KIND(t->info));
+ return -EINVAL;
+ }
+
+ if (!btf_name_offset_valid(env->btf, t->name)) {
+ btf_verifier_log(env, "[%u] Invalid name_offset:%u",
+ env->log_type_id, t->name);
+ return -EINVAL;
+ }
+
+ var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
+ if (var_meta_size < 0)
+ return var_meta_size;
+
+ meta_left -= var_meta_size;
+
+ return saved_meta_left - meta_left;
+}
+
+static int btf_check_all_metas(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ struct btf_header *hdr;
+ void *cur, *end;
+
+ hdr = btf->hdr;
+ cur = btf->nohdr_data + hdr->type_off;
+ end = btf->nohdr_data + hdr->str_off;
+
+ env->log_type_id = 1;
+ while (cur < end) {
+ struct btf_type *t = cur;
+ s32 meta_size;
+
+ meta_size = btf_check_meta(env, t, end - cur);
+ if (meta_size < 0)
+ return meta_size;
+
+ btf_add_type(env, t);
+ cur += meta_size;
+ env->log_type_id++;
+ }
+
+ return 0;
+}
+
+static int btf_parse_type_sec(struct btf_verifier_env *env)
+{
+ return btf_check_all_metas(env);
+}
+
+static int btf_parse_str_sec(struct btf_verifier_env *env)
+{
+ const struct btf_header *hdr;
+ struct btf *btf = env->btf;
+ const char *start, *end;
+
+ hdr = btf->hdr;
+ start = btf->nohdr_data + hdr->str_off;
+ end = start + hdr->str_len;
+
+ if (!hdr->str_len || hdr->str_len - 1 > BTF_MAX_NAME_OFFSET ||
+ start[0] || end[-1]) {
+ btf_verifier_log(env, "Invalid string section");
+ return -EINVAL;
+ }
+
+ btf->strings = start;
+
+ return 0;
+}
+
+static int btf_parse_hdr(struct btf_verifier_env *env)
+{
+ const struct btf_header *hdr;
+ struct btf *btf = env->btf;
+ u32 meta_left;
+
+ if (btf->data_size < sizeof(*hdr)) {
+ btf_verifier_log(env, "btf_header not found");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_hdr(env);
+
+ hdr = btf->hdr;
+ if (hdr->magic != BTF_MAGIC) {
+ btf_verifier_log(env, "Invalid magic");
+ return -EINVAL;
+ }
+
+ if (hdr->version != BTF_VERSION) {
+ btf_verifier_log(env, "Unsupported version");
+ return -ENOTSUPP;
+ }
+
+ if (hdr->flags) {
+ btf_verifier_log(env, "Unsupported flags");
+ return -ENOTSUPP;
+ }
+
+ meta_left = btf->data_size - sizeof(*hdr);
+ if (!meta_left) {
+ btf_verifier_log(env, "No data");
+ return -EINVAL;
+ }
+
+ if (meta_left < hdr->type_off || hdr->str_off <= hdr->type_off ||
+ /* Type section must align to 4 bytes */
+ hdr->type_off & (sizeof(u32) - 1)) {
+ btf_verifier_log(env, "Invalid type_off");
+ return -EINVAL;
+ }
+
+ if (meta_left < hdr->str_off ||
+ meta_left - hdr->str_off < hdr->str_len) {
+ btf_verifier_log(env, "Invalid str_off or str_len");
+ return -EINVAL;
+ }
+
+ btf->nohdr_data = btf->hdr + 1;
+
+ return 0;
+}
+
+static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
+ u32 log_level, char __user *log_ubuf, u32 log_size)
+{
+ struct btf_verifier_env *env = NULL;
+ struct bpf_verifier_log *log;
+ struct btf *btf = NULL;
+ u8 *data;
+ int err;
+
+ if (btf_data_size > BTF_MAX_SIZE)
+ return ERR_PTR(-E2BIG);
+
+ env = kzalloc(sizeof(*env), GFP_KERNEL | __GFP_NOWARN);
+ if (!env)
+ return ERR_PTR(-ENOMEM);
+
+ log = &env->log;
+ if (log_level || log_ubuf || log_size) {
+ /* user requested verbose verifier output
+ * and supplied buffer to store the verification trace
+ */
+ log->level = log_level;
+ log->ubuf = log_ubuf;
+ log->len_total = log_size;
+
+ /* log attributes have to be sane */
+ if (log->len_total < 128 || log->len_total > UINT_MAX >> 8 ||
+ !log->level || !log->ubuf) {
+ err = -EINVAL;
+ goto errout;
+ }
+ }
+
+ btf = kzalloc(sizeof(*btf), GFP_KERNEL | __GFP_NOWARN);
+ if (!btf) {
+ err = -ENOMEM;
+ goto errout;
+ }
+
+ data = kvmalloc(btf_data_size, GFP_KERNEL | __GFP_NOWARN);
+ if (!data) {
+ err = -ENOMEM;
+ goto errout;
+ }
+
+ btf->data = data;
+ btf->data_size = btf_data_size;
+
+ if (copy_from_user(data, btf_data, btf_data_size)) {
+ err = -EFAULT;
+ goto errout;
+ }
+
+ env->btf = btf;
+
+ err = btf_parse_hdr(env);
+ if (err)
+ goto errout;
+
+ err = btf_parse_str_sec(env);
+ if (err)
+ goto errout;
+
+ err = btf_parse_type_sec(env);
+ if (err)
+ goto errout;
+
+ if (!err && log->level && bpf_verifier_log_full(log)) {
+ err = -ENOSPC;
+ goto errout;
+ }
+
+ if (!err) {
+ btf_verifier_env_free(env);
+ return btf;
+ }
+
+errout:
+ btf_verifier_env_free(env);
+ if (btf)
+ btf_free(btf);
+ return ERR_PTR(err);
+}
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 02/10] bpf: btf: Validate type reference
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
2018-04-18 22:55 ` [PATCH bpf-next v5 01/10] bpf: btf: Introduce BPF Type Format (BTF) Martin KaFai Lau
@ 2018-04-18 22:55 ` Martin KaFai Lau
2018-04-18 22:55 ` [PATCH bpf-next v5 03/10] bpf: btf: Check members of struct/union Martin KaFai Lau
` (9 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:55 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
After collecting all btf_type in the first pass in an earlier patch,
the second pass (in this patch) can validate the reference types
(e.g. the referring type does exist and it does not refer to itself).
While checking the reference type, it also gathers other information (e.g.
the size of an array). This info will be useful in checking the
struct's members in a later patch. They will also be useful in doing
pretty print later.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/linux/btf.h | 37 +++
kernel/bpf/btf.c | 666 +++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 702 insertions(+), 1 deletion(-)
create mode 100644 include/linux/btf.h
diff --git a/include/linux/btf.h b/include/linux/btf.h
new file mode 100644
index 000000000000..f14b60368753
--- /dev/null
+++ b/include/linux/btf.h
@@ -0,0 +1,37 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+
+#ifndef _LINUX_BTF_H
+#define _LINUX_BTF_H 1
+
+#include <linux/types.h>
+
+struct btf;
+struct btf_type;
+
+/* Figure out the size of a type_id. If type_id is a modifier
+ * (e.g. const), it will be resolved to find out the type with size.
+ *
+ * For example:
+ * In describing "const void *", type_id is "const" and "const"
+ * refers to "void *". The return type will be "void *".
+ *
+ * If type_id is a simple "int", then return type will be "int".
+ *
+ * @btf: struct btf object
+ * @type_id: Find out the size of type_id. The type_id of the return
+ * type is set to *type_id.
+ * @ret_size: It can be NULL. If not NULL, the size of the return
+ * type is set to *ret_size.
+ * Return: The btf_type (resolved to another type with size info if needed).
+ * NULL is returned if type_id itself does not have size info
+ * (e.g. void) or it cannot be resolved to another type that
+ * has size info.
+ * *type_id and *ret_size will not be changed in the
+ * NULL return case.
+ */
+const struct btf_type *btf_type_id_size(const struct btf *btf,
+ u32 *type_id,
+ u32 *ret_size);
+
+#endif
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 26e9ed7cea5f..18bf266ceeda 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -105,6 +105,50 @@
*
* In the first pass, it still does some verifications (e.g.
* checking the name is a valid offset to the string section).
+ *
+ * Pass #2
+ * ~~~~~~~
+ * The main focus is to resolve a btf_type that is referring
+ * to another type.
+ *
+ * We have to ensure the referring type:
+ * 1) does exist in the BTF (i.e. in btf->types[])
+ * 2) does not cause a loop:
+ * struct A {
+ * struct B b;
+ * };
+ *
+ * struct B {
+ * struct A a;
+ * };
+ *
+ * btf_type_needs_resolve() decides if a btf_type needs
+ * to be resolved.
+ *
+ * The needs_resolve type implements the "resolve()" ops which
+ * essentially does a DFS and detects backedge.
+ *
+ * During resolve (or DFS), different C types have different
+ * "RESOLVED" conditions.
+ *
+ * When resolving a BTF_KIND_STRUCT, we need to resolve all its
+ * members because a member is always referring to another
+ * type. A struct's member can be treated as "RESOLVED" if
+ * it is referring to a BTF_KIND_PTR. Otherwise, the
+ * following valid C struct would be rejected:
+ *
+ * struct A {
+ * int m;
+ * struct A *a;
+ * };
+ *
+ * When resolving a BTF_KIND_PTR, it needs to keep resolving if
+ * it is referring to another BTF_KIND_PTR. Otherwise, we cannot
+ * detect a pointer loop, e.g.:
+ * BTF_KIND_CONST -> BTF_KIND_PTR -> BTF_KIND_CONST -> BTF_KIND_PTR +
+ * ^ |
+ * +-----------------------------------------+
+ *
*/
#define BITS_PER_U64 (sizeof(u64) * BITS_PER_BYTE)
@@ -127,12 +171,19 @@
i < btf_type_vlen(struct_type); \
i++, member++)
+#define for_each_member_from(i, from, struct_type, member) \
+ for (i = from, member = btf_type_member(struct_type) + from; \
+ i < btf_type_vlen(struct_type); \
+ i++, member++)
+
struct btf {
union {
struct btf_header *hdr;
void *data;
};
struct btf_type **types;
+ u32 *resolved_ids;
+ u32 *resolved_sizes;
const char *strings;
void *nohdr_data;
u32 nr_types;
@@ -140,10 +191,42 @@ struct btf {
u32 data_size;
};
+enum verifier_phase {
+ CHECK_META,
+ CHECK_TYPE,
+};
+
+struct resolve_vertex {
+ const struct btf_type *t;
+ u32 type_id;
+ u16 next_member;
+};
+
+enum visit_state {
+ NOT_VISITED,
+ VISITED,
+ RESOLVED,
+};
+
+enum resolve_mode {
+ RESOLVE_TBD, /* To Be Determined */
+ RESOLVE_PTR, /* Resolving for Pointer */
+ RESOLVE_STRUCT_OR_ARRAY, /* Resolving for struct/union
+ * or array
+ */
+};
+
+#define MAX_RESOLVE_DEPTH 32
+
struct btf_verifier_env {
struct btf *btf;
+ u8 *visit_states;
+ struct resolve_vertex stack[MAX_RESOLVE_DEPTH];
struct bpf_verifier_log log;
u32 log_type_id;
+ u32 top_stack;
+ enum verifier_phase phase;
+ enum resolve_mode resolve_mode;
};
static const char * const btf_kind_str[NR_BTF_KINDS] = {
@@ -165,6 +248,8 @@ struct btf_kind_operations {
s32 (*check_meta)(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left);
+ int (*resolve)(struct btf_verifier_env *env,
+ const struct resolve_vertex *v);
void (*log_details)(struct btf_verifier_env *env,
const struct btf_type *t);
};
@@ -172,6 +257,101 @@ struct btf_kind_operations {
static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS];
static struct btf_type btf_void;
+static bool btf_type_is_modifier(const struct btf_type *t)
+{
+ /* Some of them is not strictly a C modifier
+ * but they are grouped into the same bucket
+ * for BTF concern:
+ * A type (t) that refers to another
+ * type through t->type AND its size cannot
+ * be determined without following the t->type.
+ *
+ * ptr does not fall into this bucket
+ * because its size is always sizeof(void *).
+ */
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ return true;
+ }
+
+ return false;
+}
+
+static bool btf_type_is_void(const struct btf_type *t)
+{
+ /* void => no type and size info.
+ * Hence, FWD is also treated as void.
+ */
+ return t == &btf_void || BTF_INFO_KIND(t->info) == BTF_KIND_FWD;
+}
+
+static bool btf_type_is_void_or_null(const struct btf_type *t)
+{
+ return !t || btf_type_is_void(t);
+}
+
+/* union is only a special case of struct:
+ * all its offsetof(member) == 0
+ */
+static bool btf_type_is_struct(const struct btf_type *t)
+{
+ u8 kind = BTF_INFO_KIND(t->info);
+
+ return kind == BTF_KIND_STRUCT || kind == BTF_KIND_UNION;
+}
+
+static bool btf_type_is_array(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info) == BTF_KIND_ARRAY;
+}
+
+static bool btf_type_is_ptr(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info) == BTF_KIND_PTR;
+}
+
+static bool btf_type_is_int(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info) == BTF_KIND_INT;
+}
+
+/* What types need to be resolved?
+ *
+ * btf_type_is_modifier() is an obvious one.
+ *
+ * btf_type_is_struct() because its member refers to
+ * another type (through member->type).
+
+ * btf_type_is_array() because its element (array->type)
+ * refers to another type. Array can be thought of a
+ * special case of struct while array just has the same
+ * member-type repeated by array->nelems of times.
+ */
+static bool btf_type_needs_resolve(const struct btf_type *t)
+{
+ return btf_type_is_modifier(t) ||
+ btf_type_is_ptr(t) ||
+ btf_type_is_struct(t) ||
+ btf_type_is_array(t);
+}
+
+/* t->size can be used */
+static bool btf_type_has_size(const struct btf_type *t)
+{
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_INT:
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ return true;
+ }
+
+ return false;
+}
+
static const char *btf_int_encoding_str(u8 encoding)
{
if (encoding == 0)
@@ -234,6 +414,14 @@ static const char *btf_name_by_offset(const struct btf *btf, u32 offset)
return "(invalid-name-offset)";
}
+static const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id)
+{
+ if (type_id > btf->nr_types)
+ return NULL;
+
+ return btf->types[type_id];
+}
+
__printf(2, 3) static void __btf_verifier_log(struct bpf_verifier_log *log,
const char *fmt, ...)
{
@@ -308,6 +496,15 @@ static void btf_verifier_log_member(struct btf_verifier_env *env,
if (!bpf_verifier_log_needed(log))
return;
+ /* The CHECK_META phase already did a btf dump.
+ *
+ * If member is logged again, it must hit an error in
+ * parsing this member. It is useful to print out which
+ * struct this member belongs to.
+ */
+ if (env->phase != CHECK_META)
+ btf_verifier_log_type(env, struct_type, NULL);
+
__btf_verifier_log(log, "\t%s type_id=%u bits_offset=%u",
btf_name_by_offset(btf, member->name),
member->type, member->offset);
@@ -393,15 +590,183 @@ static int btf_add_type(struct btf_verifier_env *env, struct btf_type *t)
static void btf_free(struct btf *btf)
{
kvfree(btf->types);
+ kvfree(btf->resolved_sizes);
+ kvfree(btf->resolved_ids);
kvfree(btf->data);
kfree(btf);
}
+static int env_resolve_init(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ u32 nr_types = btf->nr_types;
+ u32 *resolved_sizes = NULL;
+ u32 *resolved_ids = NULL;
+ u8 *visit_states = NULL;
+
+ /* +1 for btf_void */
+ resolved_sizes = kvzalloc((nr_types + 1) * sizeof(*resolved_sizes),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!resolved_sizes)
+ goto nomem;
+
+ resolved_ids = kvzalloc((nr_types + 1) * sizeof(*resolved_ids),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!resolved_ids)
+ goto nomem;
+
+ visit_states = kvzalloc((nr_types + 1) * sizeof(*visit_states),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!visit_states)
+ goto nomem;
+
+ btf->resolved_sizes = resolved_sizes;
+ btf->resolved_ids = resolved_ids;
+ env->visit_states = visit_states;
+
+ return 0;
+
+nomem:
+ kvfree(resolved_sizes);
+ kvfree(resolved_ids);
+ kvfree(visit_states);
+ return -ENOMEM;
+}
+
static void btf_verifier_env_free(struct btf_verifier_env *env)
{
+ kvfree(env->visit_states);
kfree(env);
}
+static bool env_type_is_resolve_sink(const struct btf_verifier_env *env,
+ const struct btf_type *next_type)
+{
+ switch (env->resolve_mode) {
+ case RESOLVE_TBD:
+ /* int, enum or void is a sink */
+ return !btf_type_needs_resolve(next_type);
+ case RESOLVE_PTR:
+ /* int, enum, void, struct or array is a sink for ptr */
+ return !btf_type_is_modifier(next_type) &&
+ !btf_type_is_ptr(next_type);
+ case RESOLVE_STRUCT_OR_ARRAY:
+ /* int, enum, void or ptr is a sink for struct and array */
+ return !btf_type_is_modifier(next_type) &&
+ !btf_type_is_array(next_type) &&
+ !btf_type_is_struct(next_type);
+ default:
+ BUG_ON(1);
+ }
+}
+
+static bool env_type_is_resolved(const struct btf_verifier_env *env,
+ u32 type_id)
+{
+ return env->visit_states[type_id] == RESOLVED;
+}
+
+static int env_stack_push(struct btf_verifier_env *env,
+ const struct btf_type *t, u32 type_id)
+{
+ struct resolve_vertex *v;
+
+ if (env->top_stack == MAX_RESOLVE_DEPTH)
+ return -E2BIG;
+
+ if (env->visit_states[type_id] != NOT_VISITED)
+ return -EEXIST;
+
+ env->visit_states[type_id] = VISITED;
+
+ v = &env->stack[env->top_stack++];
+ v->t = t;
+ v->type_id = type_id;
+ v->next_member = 0;
+
+ if (env->resolve_mode == RESOLVE_TBD) {
+ if (btf_type_is_ptr(t))
+ env->resolve_mode = RESOLVE_PTR;
+ else if (btf_type_is_struct(t) || btf_type_is_array(t))
+ env->resolve_mode = RESOLVE_STRUCT_OR_ARRAY;
+ }
+
+ return 0;
+}
+
+static void env_stack_set_next_member(struct btf_verifier_env *env,
+ u16 next_member)
+{
+ env->stack[env->top_stack - 1].next_member = next_member;
+}
+
+static void env_stack_pop_resolved(struct btf_verifier_env *env,
+ u32 resolved_type_id,
+ u32 resolved_size)
+{
+ u32 type_id = env->stack[--(env->top_stack)].type_id;
+ struct btf *btf = env->btf;
+
+ btf->resolved_sizes[type_id] = resolved_size;
+ btf->resolved_ids[type_id] = resolved_type_id;
+ env->visit_states[type_id] = RESOLVED;
+}
+
+static const struct resolve_vertex *env_stack_peak(struct btf_verifier_env *env)
+{
+ return env->top_stack ? &env->stack[env->top_stack - 1] : NULL;
+}
+
+/* The input param "type_id" must point to a needs_resolve type */
+static const struct btf_type *btf_type_id_resolve(const struct btf *btf,
+ u32 *type_id)
+{
+ *type_id = btf->resolved_ids[*type_id];
+ return btf_type_by_id(btf, *type_id);
+}
+
+const struct btf_type *btf_type_id_size(const struct btf *btf,
+ u32 *type_id, u32 *ret_size)
+{
+ const struct btf_type *size_type;
+ u32 size_type_id = *type_id;
+ u32 size = 0;
+
+ size_type = btf_type_by_id(btf, size_type_id);
+ if (btf_type_is_void_or_null(size_type))
+ return NULL;
+
+ if (btf_type_has_size(size_type)) {
+ size = size_type->size;
+ } else if (btf_type_is_array(size_type)) {
+ size = btf->resolved_sizes[size_type_id];
+ } else if (btf_type_is_ptr(size_type)) {
+ size = sizeof(void *);
+ } else {
+ if (WARN_ON_ONCE(!btf_type_is_modifier(size_type)))
+ return NULL;
+
+ size = btf->resolved_sizes[size_type_id];
+ size_type_id = btf->resolved_ids[size_type_id];
+ size_type = btf_type_by_id(btf, size_type_id);
+ if (btf_type_is_void(size_type))
+ return NULL;
+ }
+
+ *type_id = size_type_id;
+ if (ret_size)
+ *ret_size = size;
+
+ return size_type;
+}
+
+static int btf_df_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ btf_verifier_log_basic(env, v->t, "Unsupported resolve");
+ return -EINVAL;
+}
+
static s32 btf_int_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -464,6 +829,7 @@ static void btf_int_log(struct btf_verifier_env *env,
static const struct btf_kind_operations int_ops = {
.check_meta = btf_int_check_meta,
+ .resolve = btf_df_resolve,
.log_details = btf_int_log,
};
@@ -486,6 +852,104 @@ static int btf_ref_type_check_meta(struct btf_verifier_env *env,
return 0;
}
+static int btf_modifier_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *t = v->t;
+ const struct btf_type *next_type;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+ u32 next_type_size = 0;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ /* "typedef void new_void", "const void"...etc */
+ if (btf_type_is_void(next_type))
+ goto resolved;
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ /* Figure out the resolved next_type_id with size.
+ * They will be stored in the current modifier's
+ * resolved_ids and resolved_sizes such that it can
+ * save us a few type-following when we use it later (e.g. in
+ * pretty print).
+ */
+ if (!btf_type_id_size(btf, &next_type_id, &next_type_size) &&
+ !btf_type_is_void(btf_type_id_resolve(btf, &next_type_id))) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+resolved:
+ env_stack_pop_resolved(env, next_type_id, next_type_size);
+
+ return 0;
+}
+
+static int btf_ptr_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *next_type;
+ const struct btf_type *t = v->t;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+ u32 next_type_size = 0;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ /* "void *" */
+ if (btf_type_is_void(next_type))
+ goto resolved;
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ /* If the modifier was RESOLVED during RESOLVE_STRUCT_OR_ARRAY,
+ * the modifier may have stopped resolving when it was resolved
+ * to a ptr (last-resolved-ptr).
+ *
+ * We now need to continue from the last-resolved-ptr to
+ * ensure the last-resolved-ptr will not referring back to
+ * the currenct ptr (t).
+ */
+ if (btf_type_is_modifier(next_type)) {
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id;
+
+ resolved_type_id = next_type_id;
+ resolved_type = btf_type_id_resolve(btf, &resolved_type_id);
+
+ if (btf_type_is_ptr(resolved_type) &&
+ !env_type_is_resolve_sink(env, resolved_type) &&
+ !env_type_is_resolved(env, resolved_type_id))
+ return env_stack_push(env, resolved_type,
+ resolved_type_id);
+ }
+
+ if (!btf_type_id_size(btf, &next_type_id, &next_type_size) &&
+ !btf_type_is_void(btf_type_id_resolve(btf, &next_type_id))) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+resolved:
+ env_stack_pop_resolved(env, next_type_id, 0);
+
+ return 0;
+}
+
static void btf_ref_type_log(struct btf_verifier_env *env,
const struct btf_type *t)
{
@@ -494,16 +958,19 @@ static void btf_ref_type_log(struct btf_verifier_env *env,
static struct btf_kind_operations modifier_ops = {
.check_meta = btf_ref_type_check_meta,
+ .resolve = btf_modifier_resolve,
.log_details = btf_ref_type_log,
};
static struct btf_kind_operations ptr_ops = {
.check_meta = btf_ref_type_check_meta,
+ .resolve = btf_ptr_resolve,
.log_details = btf_ref_type_log,
};
static struct btf_kind_operations fwd_ops = {
.check_meta = btf_ref_type_check_meta,
+ .resolve = btf_df_resolve,
.log_details = btf_ref_type_log,
};
@@ -542,6 +1009,61 @@ static s32 btf_array_check_meta(struct btf_verifier_env *env,
return meta_needed;
}
+static int btf_array_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_array *array = btf_type_array(v->t);
+ const struct btf_type *elem_type;
+ u32 elem_type_id = array->type;
+ struct btf *btf = env->btf;
+ u32 elem_size;
+
+ elem_type = btf_type_by_id(btf, elem_type_id);
+ if (btf_type_is_void_or_null(elem_type)) {
+ btf_verifier_log_type(env, v->t,
+ "Invalid elem");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, elem_type) &&
+ !env_type_is_resolved(env, elem_type_id))
+ return env_stack_push(env, elem_type, elem_type_id);
+
+ elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
+ if (!elem_type) {
+ btf_verifier_log_type(env, v->t, "Invalid elem");
+ return -EINVAL;
+ }
+
+ if (btf_type_is_int(elem_type)) {
+ int int_type_data = btf_type_int(elem_type);
+ u16 nr_bits = BTF_INT_BITS(int_type_data);
+ u16 nr_bytes = BITS_ROUNDUP_BYTES(nr_bits);
+
+ /* Put more restriction on array of int. The int cannot
+ * be a bit field and it must be either u8/u16/u32/u64.
+ */
+ if (BITS_PER_BYTE_MASKED(nr_bits) ||
+ BTF_INT_OFFSET(int_type_data) ||
+ (nr_bytes != sizeof(u8) && nr_bytes != sizeof(u16) &&
+ nr_bytes != sizeof(u32) && nr_bytes != sizeof(u64))) {
+ btf_verifier_log_type(env, v->t,
+ "Invalid array of int");
+ return -EINVAL;
+ }
+ }
+
+ if (array->nelems && elem_size > U32_MAX / array->nelems) {
+ btf_verifier_log_type(env, v->t,
+ "Array size overflows U32_MAX");
+ return -EINVAL;
+ }
+
+ env_stack_pop_resolved(env, elem_type_id, elem_size * array->nelems);
+
+ return 0;
+}
+
static void btf_array_log(struct btf_verifier_env *env,
const struct btf_type *t)
{
@@ -553,6 +1075,7 @@ static void btf_array_log(struct btf_verifier_env *env,
static struct btf_kind_operations array_ops = {
.check_meta = btf_array_check_meta,
+ .resolve = btf_array_resolve,
.log_details = btf_array_log,
};
@@ -610,6 +1133,50 @@ static s32 btf_struct_check_meta(struct btf_verifier_env *env,
return meta_needed;
}
+static int btf_struct_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_member *member;
+ u16 i;
+
+ /* Before continue resolving the next_member,
+ * ensure the last member is indeed resolved to a
+ * type with size info.
+ */
+ if (v->next_member) {
+ const struct btf_member *last_member;
+ u16 last_member_type_id;
+
+ last_member = btf_type_member(v->t) + v->next_member - 1;
+ last_member_type_id = last_member->type;
+ if (WARN_ON_ONCE(!env_type_is_resolved(env,
+ last_member_type_id)))
+ return -EINVAL;
+ }
+
+ for_each_member_from(i, v->next_member, v->t, member) {
+ u32 member_type_id = member->type;
+ const struct btf_type *member_type = btf_type_by_id(env->btf,
+ member_type_id);
+
+ if (btf_type_is_void_or_null(member_type)) {
+ btf_verifier_log_member(env, v->t, member,
+ "Invalid member");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, member_type) &&
+ !env_type_is_resolved(env, member_type_id)) {
+ env_stack_set_next_member(env, i + 1);
+ return env_stack_push(env, member_type, member_type_id);
+ }
+ }
+
+ env_stack_pop_resolved(env, 0, 0);
+
+ return 0;
+}
+
static void btf_struct_log(struct btf_verifier_env *env,
const struct btf_type *t)
{
@@ -618,6 +1185,7 @@ static void btf_struct_log(struct btf_verifier_env *env,
static struct btf_kind_operations struct_ops = {
.check_meta = btf_struct_check_meta,
+ .resolve = btf_struct_resolve,
.log_details = btf_struct_log,
};
@@ -671,6 +1239,7 @@ static void btf_enum_log(struct btf_verifier_env *env,
static struct btf_kind_operations enum_ops = {
.check_meta = btf_enum_check_meta,
+ .resolve = btf_df_resolve,
.log_details = btf_enum_log,
};
@@ -751,9 +1320,104 @@ static int btf_check_all_metas(struct btf_verifier_env *env)
return 0;
}
+static int btf_resolve(struct btf_verifier_env *env,
+ const struct btf_type *t, u32 type_id)
+{
+ const struct resolve_vertex *v;
+ int err = 0;
+
+ env->resolve_mode = RESOLVE_TBD;
+ env_stack_push(env, t, type_id);
+ while (!err && (v = env_stack_peak(env))) {
+ env->log_type_id = v->type_id;
+ err = btf_type_ops(v->t)->resolve(env, v);
+ }
+
+ env->log_type_id = type_id;
+ if (err == -E2BIG)
+ btf_verifier_log_type(env, t,
+ "Exceeded max resolving depth:%u",
+ MAX_RESOLVE_DEPTH);
+ else if (err == -EEXIST)
+ btf_verifier_log_type(env, t, "Loop detected");
+
+ return err;
+}
+
+static bool btf_resolve_valid(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 type_id)
+{
+ struct btf *btf = env->btf;
+
+ if (!env_type_is_resolved(env, type_id))
+ return false;
+
+ if (btf_type_is_struct(t))
+ return !btf->resolved_ids[type_id] &&
+ !btf->resolved_sizes[type_id];
+
+ if (btf_type_is_modifier(t) || btf_type_is_ptr(t)) {
+ t = btf_type_id_resolve(btf, &type_id);
+ return t && !btf_type_is_modifier(t);
+ }
+
+ if (btf_type_is_array(t)) {
+ const struct btf_array *array = btf_type_array(t);
+ const struct btf_type *elem_type;
+ u32 elem_type_id = array->type;
+ u32 elem_size;
+
+ elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
+ return elem_type && !btf_type_is_modifier(elem_type) &&
+ (array->nelems * elem_size ==
+ btf->resolved_sizes[type_id]);
+ }
+
+ return false;
+}
+
+static int btf_check_all_types(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ u32 type_id;
+ int err;
+
+ err = env_resolve_init(env);
+ if (err)
+ return err;
+
+ env->phase++;
+ for (type_id = 1; type_id <= btf->nr_types; type_id++) {
+ const struct btf_type *t = btf_type_by_id(btf, type_id);
+
+ env->log_type_id = type_id;
+ if (btf_type_needs_resolve(t) &&
+ !env_type_is_resolved(env, type_id)) {
+ err = btf_resolve(env, t, type_id);
+ if (err)
+ return err;
+ }
+
+ if (btf_type_needs_resolve(t) &&
+ !btf_resolve_valid(env, t, type_id)) {
+ btf_verifier_log_type(env, t, "Invalid resolve state");
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
static int btf_parse_type_sec(struct btf_verifier_env *env)
{
- return btf_check_all_metas(env);
+ int err;
+
+ err = btf_check_all_metas(env);
+ if (err)
+ return err;
+
+ return btf_check_all_types(env);
}
static int btf_parse_str_sec(struct btf_verifier_env *env)
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 03/10] bpf: btf: Check members of struct/union
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
2018-04-18 22:55 ` [PATCH bpf-next v5 01/10] bpf: btf: Introduce BPF Type Format (BTF) Martin KaFai Lau
2018-04-18 22:55 ` [PATCH bpf-next v5 02/10] bpf: btf: Validate type reference Martin KaFai Lau
@ 2018-04-18 22:55 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 04/10] bpf: btf: Add pretty print capability for data with BTF type info Martin KaFai Lau
` (8 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:55 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch checks a few things of struct's members:
1) It has a valid size (e.g. a "const void" is invalid)
2) A member's size (+ its member's offset) does not exceed
the containing struct's size.
3) The member's offset satisfies the alignment requirement
The above can only be done after the needs_resolve member's type
is resolved. Hence, the above is done together in
btf_struct_resolve().
Each possible member's type (e.g. int, enum, modifier...) implements
the check_member() ops which will be called from btf_struct_resolve().
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
kernel/bpf/btf.c | 205 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 205 insertions(+)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 18bf266ceeda..4e31249f6c61 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -250,6 +250,10 @@ struct btf_kind_operations {
u32 meta_left);
int (*resolve)(struct btf_verifier_env *env,
const struct resolve_vertex *v);
+ int (*check_member)(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type);
void (*log_details)(struct btf_verifier_env *env,
const struct btf_type *t);
};
@@ -760,6 +764,16 @@ const struct btf_type *btf_type_id_size(const struct btf *btf,
return size_type;
}
+static int btf_df_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ btf_verifier_log_basic(env, struct_type,
+ "Unsupported check_member");
+ return -EINVAL;
+}
+
static int btf_df_resolve(struct btf_verifier_env *env,
const struct resolve_vertex *v)
{
@@ -767,6 +781,44 @@ static int btf_df_resolve(struct btf_verifier_env *env,
return -EINVAL;
}
+static int btf_int_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 int_data = btf_type_int(member_type);
+ u32 struct_bits_off = member->offset;
+ u32 struct_size = struct_type->size;
+ u32 nr_copy_bits;
+ u32 bytes_offset;
+
+ if (U32_MAX - struct_bits_off < BTF_INT_OFFSET(int_data)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "bits_offset exceeds U32_MAX");
+ return -EINVAL;
+ }
+
+ struct_bits_off += BTF_INT_OFFSET(int_data);
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ nr_copy_bits = BTF_INT_BITS(int_data) +
+ BITS_PER_BYTE_MASKED(struct_bits_off);
+
+ if (nr_copy_bits > BITS_PER_U64) {
+ btf_verifier_log_member(env, struct_type, member,
+ "nr_copy_bits exceeds 64");
+ return -EINVAL;
+ }
+
+ if (struct_size < bytes_offset ||
+ struct_size - bytes_offset < BITS_ROUNDUP_BYTES(nr_copy_bits)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static s32 btf_int_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -830,9 +882,61 @@ static void btf_int_log(struct btf_verifier_env *env,
static const struct btf_kind_operations int_ops = {
.check_meta = btf_int_check_meta,
.resolve = btf_df_resolve,
+ .check_member = btf_int_check_member,
.log_details = btf_int_log,
};
+static int btf_modifier_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id = member->type;
+ struct btf_member resolved_member;
+ struct btf *btf = env->btf;
+
+ resolved_type = btf_type_id_size(btf, &resolved_type_id, NULL);
+ if (!resolved_type) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member");
+ return -EINVAL;
+ }
+
+ resolved_member = *member;
+ resolved_member.type = resolved_type_id;
+
+ return btf_type_ops(resolved_type)->check_member(env, struct_type,
+ &resolved_member,
+ resolved_type);
+}
+
+static int btf_ptr_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_size, struct_bits_off, bytes_offset;
+
+ struct_size = struct_type->size;
+ struct_bits_off = member->offset;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ if (struct_size - bytes_offset < sizeof(void *)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int btf_ref_type_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -959,21 +1063,53 @@ static void btf_ref_type_log(struct btf_verifier_env *env,
static struct btf_kind_operations modifier_ops = {
.check_meta = btf_ref_type_check_meta,
.resolve = btf_modifier_resolve,
+ .check_member = btf_modifier_check_member,
.log_details = btf_ref_type_log,
};
static struct btf_kind_operations ptr_ops = {
.check_meta = btf_ref_type_check_meta,
.resolve = btf_ptr_resolve,
+ .check_member = btf_ptr_check_member,
.log_details = btf_ref_type_log,
};
static struct btf_kind_operations fwd_ops = {
.check_meta = btf_ref_type_check_meta,
.resolve = btf_df_resolve,
+ .check_member = btf_df_check_member,
.log_details = btf_ref_type_log,
};
+static int btf_array_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+ u32 array_type_id, array_size;
+ struct btf *btf = env->btf;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ array_type_id = member->type;
+ btf_type_id_size(btf, &array_type_id, &array_size);
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < array_size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static s32 btf_array_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -1076,9 +1212,35 @@ static void btf_array_log(struct btf_verifier_env *env,
static struct btf_kind_operations array_ops = {
.check_meta = btf_array_check_meta,
.resolve = btf_array_resolve,
+ .check_member = btf_array_check_member,
.log_details = btf_array_log,
};
+static int btf_struct_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < member_type->size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static s32 btf_struct_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -1137,6 +1299,7 @@ static int btf_struct_resolve(struct btf_verifier_env *env,
const struct resolve_vertex *v)
{
const struct btf_member *member;
+ int err;
u16 i;
/* Before continue resolving the next_member,
@@ -1144,6 +1307,7 @@ static int btf_struct_resolve(struct btf_verifier_env *env,
* type with size info.
*/
if (v->next_member) {
+ const struct btf_type *last_member_type;
const struct btf_member *last_member;
u16 last_member_type_id;
@@ -1152,6 +1316,14 @@ static int btf_struct_resolve(struct btf_verifier_env *env,
if (WARN_ON_ONCE(!env_type_is_resolved(env,
last_member_type_id)))
return -EINVAL;
+
+ last_member_type = btf_type_by_id(env->btf,
+ last_member_type_id);
+ err = btf_type_ops(last_member_type)->check_member(env, v->t,
+ last_member,
+ last_member_type);
+ if (err)
+ return err;
}
for_each_member_from(i, v->next_member, v->t, member) {
@@ -1170,6 +1342,12 @@ static int btf_struct_resolve(struct btf_verifier_env *env,
env_stack_set_next_member(env, i + 1);
return env_stack_push(env, member_type, member_type_id);
}
+
+ err = btf_type_ops(member_type)->check_member(env, v->t,
+ member,
+ member_type);
+ if (err)
+ return err;
}
env_stack_pop_resolved(env, 0, 0);
@@ -1186,9 +1364,35 @@ static void btf_struct_log(struct btf_verifier_env *env,
static struct btf_kind_operations struct_ops = {
.check_meta = btf_struct_check_meta,
.resolve = btf_struct_resolve,
+ .check_member = btf_struct_check_member,
.log_details = btf_struct_log,
};
+static int btf_enum_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < sizeof(int)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static s32 btf_enum_check_meta(struct btf_verifier_env *env,
const struct btf_type *t,
u32 meta_left)
@@ -1240,6 +1444,7 @@ static void btf_enum_log(struct btf_verifier_env *env,
static struct btf_kind_operations enum_ops = {
.check_meta = btf_enum_check_meta,
.resolve = btf_df_resolve,
+ .check_member = btf_enum_check_member,
.log_details = btf_enum_log,
};
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 04/10] bpf: btf: Add pretty print capability for data with BTF type info
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (2 preceding siblings ...)
2018-04-18 22:55 ` [PATCH bpf-next v5 03/10] bpf: btf: Check members of struct/union Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 05/10] bpf: btf: Add BPF_BTF_LOAD command Martin KaFai Lau
` (7 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch adds pretty print capability for data with BTF type info.
The current usage is to allow pretty print for a BPF map.
The next few patches will allow a read() on a pinned map with BTF
type info for its key and value.
This patch uses the seq_printf() infra.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/linux/btf.h | 2 +
kernel/bpf/btf.c | 198 ++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 200 insertions(+)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index f14b60368753..d8bdab0280ba 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -33,5 +33,7 @@ struct btf_type;
const struct btf_type *btf_type_id_size(const struct btf *btf,
u32 *type_id,
u32 *ret_size);
+void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
+ struct seq_file *m);
#endif
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 4e31249f6c61..10ee41589da2 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -3,6 +3,7 @@
#include <uapi/linux/btf.h>
#include <uapi/linux/types.h>
+#include <linux/seq_file.h>
#include <linux/compiler.h>
#include <linux/errno.h>
#include <linux/slab.h>
@@ -256,6 +257,9 @@ struct btf_kind_operations {
const struct btf_type *member_type);
void (*log_details)(struct btf_verifier_env *env,
const struct btf_type *t);
+ void (*seq_show)(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offsets,
+ struct seq_file *m);
};
static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS];
@@ -781,6 +785,13 @@ static int btf_df_resolve(struct btf_verifier_env *env,
return -EINVAL;
}
+static void btf_df_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offsets,
+ struct seq_file *m)
+{
+ seq_printf(m, "<unsupported kind:%u>", BTF_INFO_KIND(t->info));
+}
+
static int btf_int_check_member(struct btf_verifier_env *env,
const struct btf_type *struct_type,
const struct btf_member *member,
@@ -879,11 +890,96 @@ static void btf_int_log(struct btf_verifier_env *env,
btf_int_encoding_str(BTF_INT_ENCODING(int_data)));
}
+static void btf_int_bits_seq_show(const struct btf *btf,
+ const struct btf_type *t,
+ void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ u32 int_data = btf_type_int(t);
+ u16 nr_bits = BTF_INT_BITS(int_data);
+ u16 total_bits_offset;
+ u16 nr_copy_bytes;
+ u16 nr_copy_bits;
+ u8 nr_upper_bits;
+ union {
+ u64 u64_num;
+ u8 u8_nums[8];
+ } print_num;
+
+ total_bits_offset = bits_offset + BTF_INT_OFFSET(int_data);
+ data += BITS_ROUNDDOWN_BYTES(total_bits_offset);
+ bits_offset = BITS_PER_BYTE_MASKED(total_bits_offset);
+ nr_copy_bits = nr_bits + bits_offset;
+ nr_copy_bytes = BITS_ROUNDUP_BYTES(nr_copy_bits);
+
+ print_num.u64_num = 0;
+ memcpy(&print_num.u64_num, data, nr_copy_bytes);
+
+ /* Ditch the higher order bits */
+ nr_upper_bits = BITS_PER_BYTE_MASKED(nr_copy_bits);
+ if (nr_upper_bits) {
+ /* We need to mask out some bits of the upper byte. */
+ u8 mask = (1 << nr_upper_bits) - 1;
+
+ print_num.u8_nums[nr_copy_bytes - 1] &= mask;
+ }
+
+ print_num.u64_num >>= bits_offset;
+
+ seq_printf(m, "0x%llx", print_num.u64_num);
+}
+
+static void btf_int_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ u32 int_data = btf_type_int(t);
+ u8 encoding = BTF_INT_ENCODING(int_data);
+ bool sign = encoding & BTF_INT_SIGNED;
+ u32 nr_bits = BTF_INT_BITS(int_data);
+
+ if (bits_offset || BTF_INT_OFFSET(int_data) ||
+ BITS_PER_BYTE_MASKED(nr_bits)) {
+ btf_int_bits_seq_show(btf, t, data, bits_offset, m);
+ return;
+ }
+
+ switch (nr_bits) {
+ case 64:
+ if (sign)
+ seq_printf(m, "%lld", *(s64 *)data);
+ else
+ seq_printf(m, "%llu", *(u64 *)data);
+ break;
+ case 32:
+ if (sign)
+ seq_printf(m, "%d", *(s32 *)data);
+ else
+ seq_printf(m, "%u", *(u32 *)data);
+ break;
+ case 16:
+ if (sign)
+ seq_printf(m, "%d", *(s16 *)data);
+ else
+ seq_printf(m, "%u", *(u16 *)data);
+ break;
+ case 8:
+ if (sign)
+ seq_printf(m, "%d", *(s8 *)data);
+ else
+ seq_printf(m, "%u", *(u8 *)data);
+ break;
+ default:
+ btf_int_bits_seq_show(btf, t, data, bits_offset, m);
+ }
+}
+
static const struct btf_kind_operations int_ops = {
.check_meta = btf_int_check_meta,
.resolve = btf_df_resolve,
.check_member = btf_int_check_member,
.log_details = btf_int_log,
+ .seq_show = btf_int_seq_show,
};
static int btf_modifier_check_member(struct btf_verifier_env *env,
@@ -1054,6 +1150,24 @@ static int btf_ptr_resolve(struct btf_verifier_env *env,
return 0;
}
+static void btf_modifier_seq_show(const struct btf *btf,
+ const struct btf_type *t,
+ u32 type_id, void *data,
+ u8 bits_offset, struct seq_file *m)
+{
+ t = btf_type_id_resolve(btf, &type_id);
+
+ btf_type_ops(t)->seq_show(btf, t, type_id, data, bits_offset, m);
+}
+
+static void btf_ptr_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ /* It is a hashed value */
+ seq_printf(m, "%p", *(void **)data);
+}
+
static void btf_ref_type_log(struct btf_verifier_env *env,
const struct btf_type *t)
{
@@ -1065,6 +1179,7 @@ static struct btf_kind_operations modifier_ops = {
.resolve = btf_modifier_resolve,
.check_member = btf_modifier_check_member,
.log_details = btf_ref_type_log,
+ .seq_show = btf_modifier_seq_show,
};
static struct btf_kind_operations ptr_ops = {
@@ -1072,6 +1187,7 @@ static struct btf_kind_operations ptr_ops = {
.resolve = btf_ptr_resolve,
.check_member = btf_ptr_check_member,
.log_details = btf_ref_type_log,
+ .seq_show = btf_ptr_seq_show,
};
static struct btf_kind_operations fwd_ops = {
@@ -1079,6 +1195,7 @@ static struct btf_kind_operations fwd_ops = {
.resolve = btf_df_resolve,
.check_member = btf_df_check_member,
.log_details = btf_ref_type_log,
+ .seq_show = btf_df_seq_show,
};
static int btf_array_check_member(struct btf_verifier_env *env,
@@ -1209,11 +1326,36 @@ static void btf_array_log(struct btf_verifier_env *env,
array->type, array->index_type, array->nelems);
}
+static void btf_array_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ const struct btf_array *array = btf_type_array(t);
+ const struct btf_kind_operations *elem_ops;
+ const struct btf_type *elem_type;
+ u32 i, elem_size, elem_type_id;
+
+ elem_type_id = array->type;
+ elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
+ elem_ops = btf_type_ops(elem_type);
+ seq_puts(m, "[");
+ for (i = 0; i < array->nelems; i++) {
+ if (i)
+ seq_puts(m, ",");
+
+ elem_ops->seq_show(btf, elem_type, elem_type_id, data,
+ bits_offset, m);
+ data += elem_size;
+ }
+ seq_puts(m, "]");
+}
+
static struct btf_kind_operations array_ops = {
.check_meta = btf_array_check_meta,
.resolve = btf_array_resolve,
.check_member = btf_array_check_member,
.log_details = btf_array_log,
+ .seq_show = btf_array_seq_show,
};
static int btf_struct_check_member(struct btf_verifier_env *env,
@@ -1361,11 +1503,39 @@ static void btf_struct_log(struct btf_verifier_env *env,
btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
}
+static void btf_struct_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ const char *seq = BTF_INFO_KIND(t->info) == BTF_KIND_UNION ? "|" : ",";
+ const struct btf_member *member;
+ u32 i;
+
+ seq_puts(m, "{");
+ for_each_member(i, t, member) {
+ const struct btf_type *member_type = btf_type_by_id(btf,
+ member->type);
+ u32 member_offset = member->offset;
+ u32 bytes_offset = BITS_ROUNDDOWN_BYTES(member_offset);
+ u8 bits8_offset = BITS_PER_BYTE_MASKED(member_offset);
+ const struct btf_kind_operations *ops;
+
+ if (i)
+ seq_puts(m, seq);
+
+ ops = btf_type_ops(member_type);
+ ops->seq_show(btf, member_type, member->type,
+ data + bytes_offset, bits8_offset, m);
+ }
+ seq_puts(m, "}");
+}
+
static struct btf_kind_operations struct_ops = {
.check_meta = btf_struct_check_meta,
.resolve = btf_struct_resolve,
.check_member = btf_struct_check_member,
.log_details = btf_struct_log,
+ .seq_show = btf_struct_seq_show,
};
static int btf_enum_check_member(struct btf_verifier_env *env,
@@ -1441,11 +1611,31 @@ static void btf_enum_log(struct btf_verifier_env *env,
btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
}
+static void btf_enum_seq_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct seq_file *m)
+{
+ const struct btf_enum *enums = btf_type_enum(t);
+ u32 i, nr_enums = btf_type_vlen(t);
+ int v = *(int *)data;
+
+ for (i = 0; i < nr_enums; i++) {
+ if (v == enums[i].val) {
+ seq_printf(m, "%s",
+ btf_name_by_offset(btf, enums[i].name));
+ return;
+ }
+ }
+
+ seq_printf(m, "%d", v);
+}
+
static struct btf_kind_operations enum_ops = {
.check_meta = btf_enum_check_meta,
.resolve = btf_df_resolve,
.check_member = btf_enum_check_member,
.log_details = btf_enum_log,
+ .seq_show = btf_enum_seq_show,
};
static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = {
@@ -1782,3 +1972,11 @@ static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
btf_free(btf);
return ERR_PTR(err);
}
+
+void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
+ struct seq_file *m)
+{
+ const struct btf_type *t = btf_type_by_id(btf, type_id);
+
+ btf_type_ops(t)->seq_show(btf, t, type_id, obj, 0, m);
+}
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 05/10] bpf: btf: Add BPF_BTF_LOAD command
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (3 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 04/10] bpf: btf: Add pretty print capability for data with BTF type info Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 06/10] bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd Martin KaFai Lau
` (6 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch adds a BPF_BTF_LOAD command which
1) loads and verifies the BTF (implemented in earlier patches)
2) returns a BTF fd to userspace. In the next patch, the
BTF fd can be specified during BPF_MAP_CREATE.
It currently limits to CAP_SYS_ADMIN.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/linux/btf.h | 4 +++
include/uapi/linux/bpf.h | 9 +++++++
kernel/bpf/btf.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++
kernel/bpf/syscall.c | 17 ++++++++++++
4 files changed, 97 insertions(+)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index d8bdab0280ba..a7c7072535ea 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -8,7 +8,11 @@
struct btf;
struct btf_type;
+union bpf_attr;
+void btf_put(struct btf *btf);
+int btf_new_fd(const union bpf_attr *attr);
+struct btf *btf_get_by_fd(int fd);
/* Figure out the size of a type_id. If type_id is a modifier
* (e.g. const), it will be resolved to find out the type with size.
*
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index c5ec89732a8d..c7d75f18521b 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -95,6 +95,7 @@ enum bpf_cmd {
BPF_OBJ_GET_INFO_BY_FD,
BPF_PROG_QUERY,
BPF_RAW_TRACEPOINT_OPEN,
+ BPF_BTF_LOAD,
};
enum bpf_map_type {
@@ -363,6 +364,14 @@ union bpf_attr {
__u64 name;
__u32 prog_fd;
} raw_tracepoint;
+
+ struct { /* anonymous struct for BPF_BTF_LOAD */
+ __aligned_u64 btf;
+ __aligned_u64 btf_log_buf;
+ __u32 btf_size;
+ __u32 btf_log_size;
+ __u32 btf_log_level;
+ };
} __attribute__((aligned(8)));
/* BPF helper function descriptions:
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 10ee41589da2..2322340694cf 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -7,6 +7,8 @@
#include <linux/compiler.h>
#include <linux/errno.h>
#include <linux/slab.h>
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
#include <linux/uaccess.h>
#include <linux/kernel.h>
#include <linux/bpf_verifier.h>
@@ -190,6 +192,7 @@ struct btf {
u32 nr_types;
u32 types_size;
u32 data_size;
+ refcount_t refcnt;
};
enum verifier_phase {
@@ -604,6 +607,17 @@ static void btf_free(struct btf *btf)
kfree(btf);
}
+static void btf_get(struct btf *btf)
+{
+ refcount_inc(&btf->refcnt);
+}
+
+void btf_put(struct btf *btf)
+{
+ if (btf && refcount_dec_and_test(&btf->refcnt))
+ btf_free(btf);
+}
+
static int env_resolve_init(struct btf_verifier_env *env)
{
struct btf *btf = env->btf;
@@ -1963,6 +1977,7 @@ static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
if (!err) {
btf_verifier_env_free(env);
+ btf_get(btf);
return btf;
}
@@ -1980,3 +1995,55 @@ void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
btf_type_ops(t)->seq_show(btf, t, type_id, obj, 0, m);
}
+
+static int btf_release(struct inode *inode, struct file *filp)
+{
+ btf_put(filp->private_data);
+ return 0;
+}
+
+static const struct file_operations btf_fops = {
+ .release = btf_release,
+};
+
+int btf_new_fd(const union bpf_attr *attr)
+{
+ struct btf *btf;
+ int fd;
+
+ btf = btf_parse(u64_to_user_ptr(attr->btf),
+ attr->btf_size, attr->btf_log_level,
+ u64_to_user_ptr(attr->btf_log_buf),
+ attr->btf_log_size);
+ if (IS_ERR(btf))
+ return PTR_ERR(btf);
+
+ fd = anon_inode_getfd("btf", &btf_fops, btf,
+ O_RDONLY | O_CLOEXEC);
+ if (fd < 0)
+ btf_put(btf);
+
+ return fd;
+}
+
+struct btf *btf_get_by_fd(int fd)
+{
+ struct btf *btf;
+ struct fd f;
+
+ f = fdget(fd);
+
+ if (!f.file)
+ return ERR_PTR(-EBADF);
+
+ if (f.file->f_op != &btf_fops) {
+ fdput(f);
+ return ERR_PTR(-EINVAL);
+ }
+
+ btf = f.file->private_data;
+ btf_get(btf);
+ fdput(f);
+
+ return btf;
+}
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 4ca46df19c9a..cd8ebadc66eb 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -11,6 +11,7 @@
*/
#include <linux/bpf.h>
#include <linux/bpf_trace.h>
+#include <linux/btf.h>
#include <linux/syscalls.h>
#include <linux/slab.h>
#include <linux/sched/signal.h>
@@ -2023,6 +2024,19 @@ static int bpf_obj_get_info_by_fd(const union bpf_attr *attr,
return err;
}
+#define BPF_BTF_LOAD_LAST_FIELD btf_log_level
+
+static int bpf_btf_load(const union bpf_attr *attr)
+{
+ if (CHECK_ATTR(BPF_BTF_LOAD))
+ return -EINVAL;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ return btf_new_fd(attr);
+}
+
SYSCALL_DEFINE3(bpf, int, cmd, union bpf_attr __user *, uattr, unsigned int, size)
{
union bpf_attr attr = {};
@@ -2103,6 +2117,9 @@ SYSCALL_DEFINE3(bpf, int, cmd, union bpf_attr __user *, uattr, unsigned int, siz
case BPF_RAW_TRACEPOINT_OPEN:
err = bpf_raw_tracepoint_open(&attr);
break;
+ case BPF_BTF_LOAD:
+ err = bpf_btf_load(&attr);
+ break;
default:
err = -EINVAL;
break;
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 06/10] bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (4 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 05/10] bpf: btf: Add BPF_BTF_LOAD command Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 07/10] bpf: btf: Add pretty print support to the basic arraymap Martin KaFai Lau
` (5 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch adds BPF_OBJ_GET_INFO_BY_FD support to BTF fd.
The original BTF data, which was used to create the BTF fd during
the earlier BPF_BTF_LOAD call, will be returned.
The userspace is expected to allocate buffer
to info.info and the buffer size is set to info.info_len before
calling BPF_OBJ_GET_INFO_BY_FD.
The original BTF data is copied to the userspace buffer (info.info).
Only upto the user's specified info.info_len will be copied.
The original BTF data size is set to info.info_len. The userspace
needs to check if it is bigger than its allocated buffer size.
If it is, the userspace should realloc with the kernel-returned
info.info_len and call the BPF_OBJ_GET_INFO_BY_FD again.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/linux/btf.h | 5 +++++
kernel/bpf/btf.c | 17 ++++++++++++++++-
kernel/bpf/syscall.c | 2 ++
3 files changed, 23 insertions(+), 1 deletion(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index a7c7072535ea..a966dc6d61ee 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -10,9 +10,14 @@ struct btf;
struct btf_type;
union bpf_attr;
+extern const struct file_operations btf_fops;
+
void btf_put(struct btf *btf);
int btf_new_fd(const union bpf_attr *attr);
struct btf *btf_get_by_fd(int fd);
+int btf_get_info_by_fd(const struct btf *btf,
+ const union bpf_attr *attr,
+ union bpf_attr __user *uattr);
/* Figure out the size of a type_id. If type_id is a modifier
* (e.g. const), it will be resolved to find out the type with size.
*
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 2322340694cf..eb56ac760547 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -2002,7 +2002,7 @@ static int btf_release(struct inode *inode, struct file *filp)
return 0;
}
-static const struct file_operations btf_fops = {
+const struct file_operations btf_fops = {
.release = btf_release,
};
@@ -2047,3 +2047,18 @@ struct btf *btf_get_by_fd(int fd)
return btf;
}
+
+int btf_get_info_by_fd(const struct btf *btf,
+ const union bpf_attr *attr,
+ union bpf_attr __user *uattr)
+{
+ void __user *udata = u64_to_user_ptr(attr->info.info);
+ u32 copy_len = min_t(u32, btf->data_size,
+ attr->info.info_len);
+
+ if (copy_to_user(udata, btf->data, copy_len) ||
+ put_user(btf->data_size, &uattr->info.info_len))
+ return -EFAULT;
+
+ return 0;
+}
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index cd8ebadc66eb..0a4924a0a8da 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -2017,6 +2017,8 @@ static int bpf_obj_get_info_by_fd(const union bpf_attr *attr,
else if (f.file->f_op == &bpf_map_fops)
err = bpf_map_get_info_by_fd(f.file->private_data, attr,
uattr);
+ else if (f.file->f_op == &btf_fops)
+ err = btf_get_info_by_fd(f.file->private_data, attr, uattr);
else
err = -EINVAL;
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 07/10] bpf: btf: Add pretty print support to the basic arraymap
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (5 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 06/10] bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 08/10] bpf: btf: Sync bpf.h and btf.h to tools/ Martin KaFai Lau
` (4 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch adds pretty print support to the basic arraymap.
Support for other bpf maps can be added later.
This patch adds new attrs to the BPF_MAP_CREATE command to allow
specifying the btf_fd, btf_key_id and btf_value_id. The
BPF_MAP_CREATE can then associate the btf to the map if
the creating map supports BTF.
A BTF supported map needs to implement two new map ops,
map_seq_show_elem() and map_check_btf(). This patch has
implemented these new map ops for the basic arraymap.
It also adds file_operations, bpffs_map_fops, to the pinned
map such that the pinned map can be opened and read.
After that, the user has an intuitive way to do
"cat bpffs/pathto/a-pinned-map" instead of getting
an error.
bpffs_map_fops should not be extended further to support
other operations. Other operations (e.g. write/key-lookup...)
should be realized by the userspace tools (e.g. bpftool) through
the BPF_OBJ_GET_INFO_BY_FD, map's lookup/update interface...etc.
Follow up patches will allow the userspace to obtain
the BTF from a map-fd.
Here is a sample output when reading a pinned arraymap
with the following map's value:
struct map_value {
int count_a;
int count_b;
};
cat /sys/fs/bpf/pinned_array_map:
0: {1,2}
1: {3,4}
2: {5,6}
...
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
include/linux/bpf.h | 20 +++++-
include/uapi/linux/bpf.h | 3 +
kernel/bpf/arraymap.c | 50 +++++++++++++++
kernel/bpf/inode.c | 156 ++++++++++++++++++++++++++++++++++++++++++++++-
kernel/bpf/syscall.c | 32 +++++++++-
5 files changed, 254 insertions(+), 7 deletions(-)
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 95a7abd0ee92..ee5275e7d4df 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -22,6 +22,8 @@ struct perf_event;
struct bpf_prog;
struct bpf_map;
struct sock;
+struct seq_file;
+struct btf;
/* map is generic key/value storage optionally accesible by eBPF programs */
struct bpf_map_ops {
@@ -43,10 +45,14 @@ struct bpf_map_ops {
void (*map_fd_put_ptr)(void *ptr);
u32 (*map_gen_lookup)(struct bpf_map *map, struct bpf_insn *insn_buf);
u32 (*map_fd_sys_lookup_elem)(void *ptr);
+ void (*map_seq_show_elem)(struct bpf_map *map, void *key,
+ struct seq_file *m);
+ int (*map_check_btf)(const struct bpf_map *map, const struct btf *btf,
+ u32 key_type_id, u32 value_type_id);
};
struct bpf_map {
- /* 1st cacheline with read-mostly members of which some
+ /* The first two cachelines with read-mostly members of which some
* are also accessed in fast-path (e.g. ops, max_entries).
*/
const struct bpf_map_ops *ops ____cacheline_aligned;
@@ -62,10 +68,13 @@ struct bpf_map {
u32 pages;
u32 id;
int numa_node;
+ u32 btf_key_id;
+ u32 btf_value_id;
+ struct btf *btf;
bool unpriv_array;
- /* 7 bytes hole */
+ /* 55 bytes hole */
- /* 2nd cacheline with misc members to avoid false sharing
+ /* The 3rd and 4th cacheline with misc members to avoid false sharing
* particularly with refcounting.
*/
struct user_struct *user ____cacheline_aligned;
@@ -100,6 +109,11 @@ static inline struct bpf_offloaded_map *map_to_offmap(struct bpf_map *map)
return container_of(map, struct bpf_offloaded_map, map);
}
+static inline bool bpf_map_support_seq_show(const struct bpf_map *map)
+{
+ return map->ops->map_seq_show_elem && map->ops->map_check_btf;
+}
+
extern const struct bpf_map_ops bpf_map_offload_ops;
/* function argument constraints */
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index c7d75f18521b..2c010a6b25e7 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -280,6 +280,9 @@ union bpf_attr {
*/
char map_name[BPF_OBJ_NAME_LEN];
__u32 map_ifindex; /* ifindex of netdev to create on */
+ __u32 btf_fd; /* fd pointing to a BTF type data */
+ __u32 btf_key_id; /* BTF type_id of the key */
+ __u32 btf_value_id; /* BTF type_id of the value */
};
struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */
diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c
index 14750e7c5ee4..02a189339381 100644
--- a/kernel/bpf/arraymap.c
+++ b/kernel/bpf/arraymap.c
@@ -11,11 +11,13 @@
* General Public License for more details.
*/
#include <linux/bpf.h>
+#include <linux/btf.h>
#include <linux/err.h>
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/filter.h>
#include <linux/perf_event.h>
+#include <uapi/linux/btf.h>
#include "map_in_map.h"
@@ -336,6 +338,52 @@ static void array_map_free(struct bpf_map *map)
bpf_map_area_free(array);
}
+static void array_map_seq_show_elem(struct bpf_map *map, void *key,
+ struct seq_file *m)
+{
+ void *value;
+
+ rcu_read_lock();
+
+ value = array_map_lookup_elem(map, key);
+ if (!value) {
+ rcu_read_unlock();
+ return;
+ }
+
+ seq_printf(m, "%u: ", *(u32 *)key);
+ btf_type_seq_show(map->btf, map->btf_value_id, value, m);
+ seq_puts(m, "\n");
+
+ rcu_read_unlock();
+}
+
+static int array_map_check_btf(const struct bpf_map *map, const struct btf *btf,
+ u32 btf_key_id, u32 btf_value_id)
+{
+ const struct btf_type *key_type, *value_type;
+ u32 key_size, value_size;
+ u32 int_data;
+
+ key_type = btf_type_id_size(btf, &btf_key_id, &key_size);
+ if (!key_type || BTF_INFO_KIND(key_type->info) != BTF_KIND_INT)
+ return -EINVAL;
+
+ int_data = *(u32 *)(key_type + 1);
+ /* bpf array can only take a u32 key. This check makes
+ * sure that the btf matches the attr used during map_create.
+ */
+ if (BTF_INT_BITS(int_data) != 32 || key_size != 4 ||
+ BTF_INT_OFFSET(int_data))
+ return -EINVAL;
+
+ value_type = btf_type_id_size(btf, &btf_value_id, &value_size);
+ if (!value_type || value_size > map->value_size)
+ return -EINVAL;
+
+ return 0;
+}
+
const struct bpf_map_ops array_map_ops = {
.map_alloc_check = array_map_alloc_check,
.map_alloc = array_map_alloc,
@@ -345,6 +393,8 @@ const struct bpf_map_ops array_map_ops = {
.map_update_elem = array_map_update_elem,
.map_delete_elem = array_map_delete_elem,
.map_gen_lookup = array_map_gen_lookup,
+ .map_seq_show_elem = array_map_seq_show_elem,
+ .map_check_btf = array_map_check_btf,
};
const struct bpf_map_ops percpu_array_map_ops = {
diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
index bf6da59ae0d0..a41343009ccc 100644
--- a/kernel/bpf/inode.c
+++ b/kernel/bpf/inode.c
@@ -150,8 +150,154 @@ static int bpf_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode)
return 0;
}
+struct map_iter {
+ void *key;
+ bool done;
+};
+
+static struct map_iter *map_iter(struct seq_file *m)
+{
+ return m->private;
+}
+
+static struct bpf_map *seq_file_to_map(struct seq_file *m)
+{
+ return file_inode(m->file)->i_private;
+}
+
+static void map_iter_free(struct map_iter *iter)
+{
+ if (iter) {
+ kfree(iter->key);
+ kfree(iter);
+ }
+}
+
+static struct map_iter *map_iter_alloc(struct bpf_map *map)
+{
+ struct map_iter *iter;
+
+ iter = kzalloc(sizeof(*iter), GFP_KERNEL | __GFP_NOWARN);
+ if (!iter)
+ goto error;
+
+ iter->key = kzalloc(map->key_size, GFP_KERNEL | __GFP_NOWARN);
+ if (!iter->key)
+ goto error;
+
+ return iter;
+
+error:
+ map_iter_free(iter);
+ return NULL;
+}
+
+static void *map_seq_next(struct seq_file *m, void *v, loff_t *pos)
+{
+ struct bpf_map *map = seq_file_to_map(m);
+ void *key = map_iter(m)->key;
+
+ if (map_iter(m)->done)
+ return NULL;
+
+ if (unlikely(v == SEQ_START_TOKEN))
+ goto done;
+
+ if (map->ops->map_get_next_key(map, key, key)) {
+ map_iter(m)->done = true;
+ return NULL;
+ }
+
+done:
+ ++(*pos);
+ return key;
+}
+
+static void *map_seq_start(struct seq_file *m, loff_t *pos)
+{
+ if (map_iter(m)->done)
+ return NULL;
+
+ return *pos ? map_iter(m)->key : SEQ_START_TOKEN;
+}
+
+static void map_seq_stop(struct seq_file *m, void *v)
+{
+}
+
+static int map_seq_show(struct seq_file *m, void *v)
+{
+ struct bpf_map *map = seq_file_to_map(m);
+ void *key = map_iter(m)->key;
+
+ if (unlikely(v == SEQ_START_TOKEN)) {
+ seq_puts(m, "# WARNING!! The output is for debug purpose only\n");
+ seq_puts(m, "# WARNING!! The output format will change\n");
+ } else {
+ map->ops->map_seq_show_elem(map, key, m);
+ }
+
+ return 0;
+}
+
+static const struct seq_operations bpffs_map_seq_ops = {
+ .start = map_seq_start,
+ .next = map_seq_next,
+ .show = map_seq_show,
+ .stop = map_seq_stop,
+};
+
+static int bpffs_map_open(struct inode *inode, struct file *file)
+{
+ struct bpf_map *map = inode->i_private;
+ struct map_iter *iter;
+ struct seq_file *m;
+ int err;
+
+ iter = map_iter_alloc(map);
+ if (!iter)
+ return -ENOMEM;
+
+ err = seq_open(file, &bpffs_map_seq_ops);
+ if (err) {
+ map_iter_free(iter);
+ return err;
+ }
+
+ m = file->private_data;
+ m->private = iter;
+
+ return 0;
+}
+
+static int bpffs_map_release(struct inode *inode, struct file *file)
+{
+ struct seq_file *m = file->private_data;
+
+ map_iter_free(map_iter(m));
+
+ return seq_release(inode, file);
+}
+
+/* bpffs_map_fops should only implement the basic
+ * read operation for a BPF map. The purpose is to
+ * provide a simple user intuitive way to do
+ * "cat bpffs/pathto/a-pinned-map".
+ *
+ * Other operations (e.g. write, lookup...) should be realized by
+ * the userspace tools (e.g. bpftool) through the
+ * BPF_OBJ_GET_INFO_BY_FD and the map's lookup/update
+ * interface.
+ */
+static const struct file_operations bpffs_map_fops = {
+ .open = bpffs_map_open,
+ .read = seq_read,
+ .release = bpffs_map_release,
+};
+
static int bpf_mkobj_ops(struct dentry *dentry, umode_t mode, void *raw,
- const struct inode_operations *iops)
+ const struct inode_operations *iops,
+ const struct file_operations *fops)
{
struct inode *dir = dentry->d_parent->d_inode;
struct inode *inode = bpf_get_inode(dir->i_sb, dir, mode);
@@ -159,6 +305,7 @@ static int bpf_mkobj_ops(struct dentry *dentry, umode_t mode, void *raw,
return PTR_ERR(inode);
inode->i_op = iops;
+ inode->i_fop = fops;
inode->i_private = raw;
bpf_dentry_finalize(dentry, inode, dir);
@@ -167,12 +314,15 @@ static int bpf_mkobj_ops(struct dentry *dentry, umode_t mode, void *raw,
static int bpf_mkprog(struct dentry *dentry, umode_t mode, void *arg)
{
- return bpf_mkobj_ops(dentry, mode, arg, &bpf_prog_iops);
+ return bpf_mkobj_ops(dentry, mode, arg, &bpf_prog_iops, NULL);
}
static int bpf_mkmap(struct dentry *dentry, umode_t mode, void *arg)
{
- return bpf_mkobj_ops(dentry, mode, arg, &bpf_map_iops);
+ struct bpf_map *map = arg;
+
+ return bpf_mkobj_ops(dentry, mode, arg, &bpf_map_iops,
+ map->btf ? &bpffs_map_fops : NULL);
}
static struct dentry *
diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
index 0a4924a0a8da..fe23dc5a3ec4 100644
--- a/kernel/bpf/syscall.c
+++ b/kernel/bpf/syscall.c
@@ -27,6 +27,7 @@
#include <linux/cred.h>
#include <linux/timekeeping.h>
#include <linux/ctype.h>
+#include <linux/btf.h>
#define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PROG_ARRAY || \
(map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \
@@ -251,6 +252,7 @@ static void bpf_map_free_deferred(struct work_struct *work)
bpf_map_uncharge_memlock(map);
security_bpf_map_free(map);
+ btf_put(map->btf);
/* implementation dependent freeing */
map->ops->map_free(map);
}
@@ -416,7 +418,7 @@ static int bpf_obj_name_cpy(char *dst, const char *src)
return 0;
}
-#define BPF_MAP_CREATE_LAST_FIELD map_ifindex
+#define BPF_MAP_CREATE_LAST_FIELD btf_value_id
/* called via syscall */
static int map_create(union bpf_attr *attr)
{
@@ -450,6 +452,33 @@ static int map_create(union bpf_attr *attr)
atomic_set(&map->refcnt, 1);
atomic_set(&map->usercnt, 1);
+ if (bpf_map_support_seq_show(map) &&
+ (attr->btf_key_id || attr->btf_value_id)) {
+ struct btf *btf;
+
+ if (!attr->btf_key_id || !attr->btf_value_id) {
+ err = -EINVAL;
+ goto free_map_nouncharge;
+ }
+
+ btf = btf_get_by_fd(attr->btf_fd);
+ if (IS_ERR(btf)) {
+ err = PTR_ERR(btf);
+ goto free_map_nouncharge;
+ }
+
+ err = map->ops->map_check_btf(map, btf, attr->btf_key_id,
+ attr->btf_value_id);
+ if (err) {
+ btf_put(btf);
+ goto free_map_nouncharge;
+ }
+
+ map->btf = btf;
+ map->btf_key_id = attr->btf_key_id;
+ map->btf_value_id = attr->btf_value_id;
+ }
+
err = security_bpf_map_alloc(map);
if (err)
goto free_map_nouncharge;
@@ -482,6 +511,7 @@ static int map_create(union bpf_attr *attr)
free_map_sec:
security_bpf_map_free(map);
free_map_nouncharge:
+ btf_put(map->btf);
map->ops->map_free(map);
return err;
}
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 08/10] bpf: btf: Sync bpf.h and btf.h to tools/
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (6 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 07/10] bpf: btf: Add pretty print support to the basic arraymap Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-18 22:56 ` [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf Martin KaFai Lau
` (3 subsequent siblings)
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch sync up the bpf.h and btf.h to tools/
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
tools/include/uapi/linux/bpf.h | 12 ++++
tools/include/uapi/linux/btf.h | 130 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 142 insertions(+)
create mode 100644 tools/include/uapi/linux/btf.h
diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h
index 9d07465023a2..33107232e1f0 100644
--- a/tools/include/uapi/linux/bpf.h
+++ b/tools/include/uapi/linux/bpf.h
@@ -95,6 +95,7 @@ enum bpf_cmd {
BPF_OBJ_GET_INFO_BY_FD,
BPF_PROG_QUERY,
BPF_RAW_TRACEPOINT_OPEN,
+ BPF_BTF_LOAD,
};
enum bpf_map_type {
@@ -279,6 +280,9 @@ union bpf_attr {
*/
char map_name[BPF_OBJ_NAME_LEN];
__u32 map_ifindex; /* ifindex of netdev to create on */
+ __u32 btf_fd; /* fd pointing to a BTF type data */
+ __u32 btf_key_id; /* BTF type_id of the key */
+ __u32 btf_value_id; /* BTF type_id of the value */
};
struct { /* anonymous struct used by BPF_MAP_*_ELEM commands */
@@ -363,6 +367,14 @@ union bpf_attr {
__u64 name;
__u32 prog_fd;
} raw_tracepoint;
+
+ struct { /* anonymous struct for BPF_BTF_LOAD */
+ __aligned_u64 btf;
+ __aligned_u64 btf_log_buf;
+ __u32 btf_size;
+ __u32 btf_log_size;
+ __u32 btf_log_level;
+ };
} __attribute__((aligned(8)));
/* BPF helper function descriptions:
diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
new file mode 100644
index 000000000000..74a30b1090df
--- /dev/null
+++ b/tools/include/uapi/linux/btf.h
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/* Copyright (c) 2018 Facebook */
+#ifndef _UAPI__LINUX_BTF_H__
+#define _UAPI__LINUX_BTF_H__
+
+#include <linux/types.h>
+
+#define BTF_MAGIC 0xeB9F
+#define BTF_MAGIC_SWAP 0x9FeB
+#define BTF_VERSION 1
+#define BTF_FLAGS_COMPR 0x01
+
+struct btf_header {
+ __u16 magic;
+ __u8 version;
+ __u8 flags;
+
+ __u32 parent_label;
+ __u32 parent_name;
+
+ /* All offsets are in bytes relative to the end of this header */
+ __u32 label_off; /* offset of label section */
+ __u32 object_off; /* offset of data object section*/
+ __u32 func_off; /* offset of function section */
+ __u32 type_off; /* offset of type section */
+ __u32 str_off; /* offset of string section */
+ __u32 str_len; /* length of string section */
+};
+
+/* Max # of type identifier */
+#define BTF_MAX_TYPE 0x7fffffff
+/* Max offset into the string section */
+#define BTF_MAX_NAME_OFFSET 0x7fffffff
+/* Max # of struct/union/enum members or func args */
+#define BTF_MAX_VLEN 0xffff
+
+/* The type id is referring to a parent BTF */
+#define BTF_TYPE_PARENT(id) (((id) >> 31) & 0x1)
+#define BTF_TYPE_ID(id) ((id) & BTF_MAX_TYPE)
+
+/* String is in the ELF string section */
+#define BTF_STR_TBL_ELF_ID(ref) (((ref) >> 31) & 0x1)
+#define BTF_STR_OFFSET(ref) ((ref) & BTF_MAX_NAME_OFFSET)
+
+struct btf_type {
+ __u32 name;
+ /* "info" bits arrangement
+ * bits 0-15: vlen (e.g. # of struct's members)
+ * bits 16-23: unused
+ * bits 24-28: kind (e.g. int, ptr, array...etc)
+ * bits 29-30: unused
+ * bits 31: root
+ */
+ __u32 info;
+ /* "size" is used by INT, ENUM, STRUCT and UNION.
+ * "size" tells the size of the type it is describing.
+ *
+ * "type" is used by PTR, TYPEDEF, VOLATILE, CONST and RESTRICT.
+ * "type" is a type_id referring to another type.
+ */
+ union {
+ __u32 size;
+ __u32 type;
+ };
+};
+
+#define BTF_INFO_KIND(info) (((info) >> 24) & 0x1f)
+#define BTF_INFO_ISROOT(info) (!!(((info) >> 24) & 0x80))
+#define BTF_INFO_VLEN(info) ((info) & 0xffff)
+
+#define BTF_KIND_UNKN 0 /* Unknown */
+#define BTF_KIND_INT 1 /* Integer */
+#define BTF_KIND_PTR 2 /* Pointer */
+#define BTF_KIND_ARRAY 3 /* Array */
+#define BTF_KIND_STRUCT 4 /* Struct */
+#define BTF_KIND_UNION 5 /* Union */
+#define BTF_KIND_ENUM 6 /* Enumeration */
+#define BTF_KIND_FWD 7 /* Forward */
+#define BTF_KIND_TYPEDEF 8 /* Typedef */
+#define BTF_KIND_VOLATILE 9 /* Volatile */
+#define BTF_KIND_CONST 10 /* Const */
+#define BTF_KIND_RESTRICT 11 /* Restrict */
+#define BTF_KIND_MAX 11
+#define NR_BTF_KINDS 12
+
+/* For some specific BTF_KIND, "struct btf_type" is immediately
+ * followed by extra data.
+ */
+
+/* BTF_KIND_INT is followed by a u32 and the following
+ * is the 32 bits arrangement:
+ */
+#define BTF_INT_ENCODING(VAL) (((VAL) & 0xff000000) >> 24)
+#define BTF_INT_OFFSET(VAL) (((VAL & 0x00ff0000)) >> 16)
+#define BTF_INT_BITS(VAL) ((VAL) & 0x0000ffff)
+
+/* Attributes stored in the BTF_INT_ENCODING */
+#define BTF_INT_SIGNED 0x1
+#define BTF_INT_CHAR 0x2
+#define BTF_INT_BOOL 0x4
+#define BTF_INT_VARARGS 0x8
+
+/* BTF_KIND_ENUM is followed by multiple "struct btf_enum".
+ * The exact number of btf_enum is stored in the vlen (of the
+ * info in "struct btf_type").
+ */
+struct btf_enum {
+ __u32 name;
+ __s32 val;
+};
+
+/* BTF_KIND_ARRAY is followed by one "struct btf_array" */
+struct btf_array {
+ __u32 type;
+ __u32 index_type;
+ __u32 nelems;
+};
+
+/* BTF_KIND_STRUCT and BTF_KIND_UNION are followed
+ * by multiple "struct btf_member". The exact number
+ * of btf_member is stored in the vlen (of the info in
+ * "struct btf_type").
+ */
+struct btf_member {
+ __u32 name;
+ __u32 type;
+ __u32 offset; /* offset in bits */
+};
+
+#endif /* _UAPI__LINUX_BTF_H__ */
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (7 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 08/10] bpf: btf: Sync bpf.h and btf.h to tools/ Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-05-09 22:17 ` libbpf backward compatibility (was: [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf) Jakub Kicinski
2018-04-18 22:56 ` [PATCH bpf-next v5 10/10] bpf: btf: Add BTF tests Martin KaFai Lau
` (2 subsequent siblings)
11 siblings, 1 reply; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
If the ".BTF" elf section exists, libbpf will try to create
a btf_fd (through BPF_BTF_LOAD). If that fails, it will still
continue loading the bpf prog/map without the BTF.
If the bpf_object has a BTF loaded, it will create a map with the btf_fd.
libbpf will try to figure out the btf_key_id and btf_value_id of a map by
finding the BTF type with name "<map_name>_key" and "<map_name>_value".
If they cannot be found, it will continue without using the BTF.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
tools/lib/bpf/Build | 2 +-
tools/lib/bpf/bpf.c | 92 +++++++++---
tools/lib/bpf/bpf.h | 16 +++
tools/lib/bpf/btf.c | 374 +++++++++++++++++++++++++++++++++++++++++++++++++
tools/lib/bpf/btf.h | 22 +++
tools/lib/bpf/libbpf.c | 148 +++++++++++++++++--
tools/lib/bpf/libbpf.h | 3 +
7 files changed, 626 insertions(+), 31 deletions(-)
create mode 100644 tools/lib/bpf/btf.c
create mode 100644 tools/lib/bpf/btf.h
diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build
index 64c679d67109..6070e655042d 100644
--- a/tools/lib/bpf/Build
+++ b/tools/lib/bpf/Build
@@ -1 +1 @@
-libbpf-y := libbpf.o bpf.o nlattr.o
+libbpf-y := libbpf.o bpf.o nlattr.o btf.o
diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c
index acbb3f8b3bec..76b36cc16e7f 100644
--- a/tools/lib/bpf/bpf.c
+++ b/tools/lib/bpf/bpf.c
@@ -73,43 +73,76 @@ static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr,
return syscall(__NR_bpf, cmd, attr, size);
}
-int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
- int key_size, int value_size, int max_entries,
- __u32 map_flags, int node)
+int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr)
{
- __u32 name_len = name ? strlen(name) : 0;
+ __u32 name_len = create_attr->name ? strlen(create_attr->name) : 0;
union bpf_attr attr;
memset(&attr, '\0', sizeof(attr));
- attr.map_type = map_type;
- attr.key_size = key_size;
- attr.value_size = value_size;
- attr.max_entries = max_entries;
- attr.map_flags = map_flags;
- memcpy(attr.map_name, name, min(name_len, BPF_OBJ_NAME_LEN - 1));
+ attr.map_type = create_attr->map_type;
+ attr.key_size = create_attr->key_size;
+ attr.value_size = create_attr->value_size;
+ attr.max_entries = create_attr->max_entries;
+ attr.map_flags = create_attr->map_flags;
+ memcpy(attr.map_name, create_attr->name,
+ min(name_len, BPF_OBJ_NAME_LEN - 1));
+ attr.numa_node = create_attr->numa_node;
+ attr.btf_fd = create_attr->btf_fd;
+ attr.btf_key_id = create_attr->btf_key_id;
+ attr.btf_value_id = create_attr->btf_value_id;
+
+ return sys_bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
+}
+int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
+ int key_size, int value_size, int max_entries,
+ __u32 map_flags, int node)
+{
+ struct bpf_create_map_attr map_attr = {};
+
+ map_attr.name = name;
+ map_attr.map_type = map_type;
+ map_attr.map_flags = map_flags;
+ map_attr.key_size = key_size;
+ map_attr.value_size = value_size;
+ map_attr.max_entries = max_entries;
if (node >= 0) {
- attr.map_flags |= BPF_F_NUMA_NODE;
- attr.numa_node = node;
+ map_attr.numa_node = node;
+ map_attr.map_flags |= BPF_F_NUMA_NODE;
}
- return sys_bpf(BPF_MAP_CREATE, &attr, sizeof(attr));
+ return bpf_create_map_xattr(&map_attr);
}
int bpf_create_map(enum bpf_map_type map_type, int key_size,
int value_size, int max_entries, __u32 map_flags)
{
- return bpf_create_map_node(map_type, NULL, key_size, value_size,
- max_entries, map_flags, -1);
+ struct bpf_create_map_attr map_attr = {};
+
+ map_attr.map_type = map_type;
+ map_attr.map_flags = map_flags;
+ map_attr.key_size = key_size;
+ map_attr.value_size = value_size;
+ map_attr.max_entries = max_entries;
+
+ return bpf_create_map_xattr(&map_attr);
}
int bpf_create_map_name(enum bpf_map_type map_type, const char *name,
int key_size, int value_size, int max_entries,
__u32 map_flags)
{
- return bpf_create_map_node(map_type, name, key_size, value_size,
- max_entries, map_flags, -1);
+ struct bpf_create_map_attr map_attr = {};
+
+ map_attr.name = name;
+ map_attr.map_type = map_type;
+ map_attr.map_flags = map_flags;
+ map_attr.key_size = key_size;
+ map_attr.value_size = value_size;
+ map_attr.max_entries = max_entries;
+
+ return bpf_create_map_xattr(&map_attr);
}
int bpf_create_map_in_map_node(enum bpf_map_type map_type, const char *name,
@@ -573,3 +606,28 @@ int bpf_set_link_xdp_fd(int ifindex, int fd, __u32 flags)
close(sock);
return ret;
}
+
+int bpf_load_btf(void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size,
+ bool do_log)
+{
+ union bpf_attr attr = {};
+ int fd;
+
+ attr.btf = ptr_to_u64(btf);
+ attr.btf_size = btf_size;
+
+retry:
+ if (do_log && log_buf && log_buf_size) {
+ attr.btf_log_level = 1;
+ attr.btf_log_size = log_buf_size;
+ attr.btf_log_buf = ptr_to_u64(log_buf);
+ }
+
+ fd = sys_bpf(BPF_BTF_LOAD, &attr, sizeof(attr));
+ if (fd == -1 && !do_log && log_buf && log_buf_size) {
+ do_log = true;
+ goto retry;
+ }
+
+ return fd;
+}
diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
index 39f6a0d64a3b..01bda076310f 100644
--- a/tools/lib/bpf/bpf.h
+++ b/tools/lib/bpf/bpf.h
@@ -26,6 +26,20 @@
#include <linux/bpf.h>
#include <stddef.h>
+struct bpf_create_map_attr {
+ const char *name;
+ enum bpf_map_type map_type;
+ __u32 map_flags;
+ __u32 key_size;
+ __u32 value_size;
+ __u32 max_entries;
+ __u32 numa_node;
+ __u32 btf_fd;
+ __u32 btf_key_id;
+ __u32 btf_value_id;
+};
+
+int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr);
int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
int key_size, int value_size, int max_entries,
__u32 map_flags, int node);
@@ -87,4 +101,6 @@ int bpf_obj_get_info_by_fd(int prog_fd, void *info, __u32 *info_len);
int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags,
__u32 *attach_flags, __u32 *prog_ids, __u32 *prog_cnt);
int bpf_raw_tracepoint_open(const char *name, int prog_fd);
+int bpf_load_btf(void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size,
+ bool do_log);
#endif
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
new file mode 100644
index 000000000000..58b6255abc7a
--- /dev/null
+++ b/tools/lib/bpf/btf.c
@@ -0,0 +1,374 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <unistd.h>
+#include <errno.h>
+#include <linux/err.h>
+#include <linux/btf.h>
+#include "btf.h"
+#include "bpf.h"
+
+#define elog(fmt, ...) { if (err_log) err_log(fmt, ##__VA_ARGS__); }
+#define max(a, b) ((a) > (b) ? (a) : (b))
+#define min(a, b) ((a) < (b) ? (a) : (b))
+
+#define BTF_MAX_NR_TYPES 65535
+
+static struct btf_type btf_void;
+
+struct btf {
+ union {
+ struct btf_header *hdr;
+ void *data;
+ };
+ struct btf_type **types;
+ const char *strings;
+ void *nohdr_data;
+ uint32_t nr_types;
+ uint32_t types_size;
+ uint32_t data_size;
+ int fd;
+};
+
+static const char *btf_name_by_offset(const struct btf *btf, uint32_t offset)
+{
+ if (!BTF_STR_TBL_ELF_ID(offset) &&
+ BTF_STR_OFFSET(offset) < btf->hdr->str_len)
+ return &btf->strings[BTF_STR_OFFSET(offset)];
+ else
+ return NULL;
+}
+
+static int btf_add_type(struct btf *btf, struct btf_type *t)
+{
+ if (btf->types_size - btf->nr_types < 2) {
+ struct btf_type **new_types;
+ u32 expand_by, new_size;
+
+ if (btf->types_size == BTF_MAX_NR_TYPES)
+ return -E2BIG;
+
+ expand_by = max(btf->types_size >> 2, 16);
+ new_size = min(BTF_MAX_NR_TYPES, btf->types_size + expand_by);
+
+ new_types = realloc(btf->types, sizeof(*new_types) * new_size);
+ if (!new_types)
+ return -ENOMEM;
+
+ if (btf->nr_types == 0)
+ new_types[0] = &btf_void;
+
+ btf->types = new_types;
+ btf->types_size = new_size;
+ }
+
+ btf->types[++(btf->nr_types)] = t;
+
+ return 0;
+}
+
+static int btf_parse_hdr(struct btf *btf, btf_print_fn_t err_log)
+{
+ const struct btf_header *hdr = btf->hdr;
+ u32 meta_left;
+
+ if (btf->data_size < sizeof(struct btf_header)) {
+ elog("BTF header not found\n");
+ return -EINVAL;
+ }
+
+ if (hdr->magic != BTF_MAGIC) {
+ elog("Invalid BTF magic:%x\n", hdr->magic);
+ return -EINVAL;
+ }
+
+ if (hdr->version != BTF_VERSION) {
+ elog("Unsupported BTF version:%u\n", hdr->version);
+ return -ENOTSUP;
+ }
+
+ if (hdr->flags) {
+ elog("Unsupported BTF flags:%x\n", hdr->flags);
+ return -ENOTSUP;
+ }
+
+ meta_left = btf->data_size - sizeof(*hdr);
+ if (!meta_left) {
+ elog("BTF has no data\n");
+ return -EINVAL;
+ }
+
+ if (meta_left < hdr->type_off) {
+ elog("Invalid BTF type section offset:%u\n", hdr->type_off);
+ return -EINVAL;
+ }
+
+ if (meta_left < hdr->str_off) {
+ elog("Invalid BTF string section offset:%u\n", hdr->str_off);
+ return -EINVAL;
+ }
+
+ if (hdr->type_off >= hdr->str_off) {
+ elog("BTF type section offset >= string section offset. No type?\n");
+ return -EINVAL;
+ }
+
+ if (hdr->type_off & 0x02) {
+ elog("BTF type section is not aligned to 4 bytes\n");
+ return -EINVAL;
+ }
+
+ btf->nohdr_data = btf->hdr + 1;
+
+ return 0;
+}
+
+static int btf_parse_str_sec(struct btf *btf, btf_print_fn_t err_log)
+{
+ const struct btf_header *hdr = btf->hdr;
+ const char *start = btf->nohdr_data + hdr->str_off;
+ const char *end = start + btf->hdr->str_len;
+
+ if (!hdr->str_len || hdr->str_len - 1 > BTF_MAX_NAME_OFFSET ||
+ start[0] || end[-1]) {
+ elog("Invalid BTF string section\n");
+ return -EINVAL;
+ }
+
+ btf->strings = start;
+
+ return 0;
+}
+
+static int btf_parse_type_sec(struct btf *btf, btf_print_fn_t err_log)
+{
+ struct btf_header *hdr = btf->hdr;
+ void *nohdr_data = btf->nohdr_data;
+ void *next_type = nohdr_data + hdr->type_off;
+ void *end_type = nohdr_data + hdr->str_off;
+
+ while (next_type < end_type) {
+ struct btf_type *t = next_type;
+ uint16_t vlen = BTF_INFO_VLEN(t->info);
+ int err;
+
+ next_type += sizeof(*t);
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_INT:
+ next_type += sizeof(int);
+ break;
+ case BTF_KIND_ARRAY:
+ next_type += sizeof(struct btf_array);
+ break;
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ next_type += vlen * sizeof(struct btf_member);
+ break;
+ case BTF_KIND_ENUM:
+ next_type += vlen * sizeof(struct btf_enum);
+ break;
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_PTR:
+ case BTF_KIND_FWD:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ break;
+ default:
+ elog("Unsupported BTF_KIND:%u\n",
+ BTF_INFO_KIND(t->info));
+ return -EINVAL;
+ }
+
+ err = btf_add_type(btf, t);
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static const struct btf_type *btf_type_by_id(const struct btf *btf,
+ uint32_t type_id)
+{
+ if (type_id > btf->nr_types)
+ return NULL;
+
+ return btf->types[type_id];
+}
+
+static bool btf_type_is_void(const struct btf_type *t)
+{
+ return t == &btf_void || BTF_INFO_KIND(t->info) == BTF_KIND_FWD;
+}
+
+static bool btf_type_is_void_or_null(const struct btf_type *t)
+{
+ return !t || btf_type_is_void(t);
+}
+
+static int64_t btf_type_size(const struct btf_type *t)
+{
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_INT:
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ return t->size;
+ case BTF_KIND_PTR:
+ return sizeof(void *);
+ default:
+ return -EINVAL;
+ }
+}
+
+#define MAX_RESOLVE_DEPTH 32
+
+int64_t btf__resolve_size(const struct btf *btf, uint32_t type_id)
+{
+ const struct btf_array *array;
+ const struct btf_type *t;
+ uint32_t nelems = 1;
+ int64_t size = -1;
+ int i;
+
+ t = btf_type_by_id(btf, type_id);
+ for (i = 0; i < MAX_RESOLVE_DEPTH && !btf_type_is_void_or_null(t);
+ i++) {
+ size = btf_type_size(t);
+ if (size >= 0)
+ break;
+
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ type_id = t->type;
+ break;
+ case BTF_KIND_ARRAY:
+ array = (const struct btf_array *)(t + 1);
+ if (nelems && array->nelems > UINT32_MAX / nelems)
+ return -E2BIG;
+ nelems *= array->nelems;
+ type_id = array->type;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ t = btf_type_by_id(btf, type_id);
+ }
+
+ if (size < 0)
+ return -EINVAL;
+
+ if (nelems && size > UINT32_MAX / nelems)
+ return -E2BIG;
+
+ return nelems * size;
+}
+
+int32_t btf__find_by_name(const struct btf *btf, const char *type_name)
+{
+ uint32_t i;
+
+ if (!strcmp(type_name, "void"))
+ return 0;
+
+ for (i = 1; i <= btf->nr_types; i++) {
+ const struct btf_type *t = btf->types[i];
+ const char *name = btf_name_by_offset(btf, t->name);
+
+ if (name && !strcmp(type_name, name))
+ return i;
+ }
+
+ return -ENOENT;
+}
+
+void btf__free(struct btf *btf)
+{
+ if (!btf)
+ return;
+
+ if (btf->fd != -1)
+ close(btf->fd);
+
+ free(btf->data);
+ free(btf->types);
+ free(btf);
+}
+
+struct btf *btf__new(uint8_t *data, uint32_t size,
+ btf_print_fn_t err_log)
+{
+ uint32_t log_buf_size = 0;
+ char *log_buf = NULL;
+ struct btf *btf;
+ int err;
+
+ btf = calloc(1, sizeof(struct btf));
+ if (!btf)
+ return ERR_PTR(-ENOMEM);
+
+ btf->fd = -1;
+
+ if (err_log) {
+ log_buf = malloc(BPF_LOG_BUF_SIZE);
+ if (!log_buf) {
+ err = -ENOMEM;
+ goto done;
+ }
+ *log_buf = 0;
+ log_buf_size = BPF_LOG_BUF_SIZE;
+ }
+
+ btf->data = malloc(size);
+ if (!btf->data) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ memcpy(btf->data, data, size);
+ btf->data_size = size;
+
+ btf->fd = bpf_load_btf(btf->data, btf->data_size,
+ log_buf, log_buf_size, false);
+
+ if (btf->fd == -1) {
+ err = -errno;
+ elog("Error loading BTF: %s(%d)\n", strerror(errno), errno);
+ if (log_buf && *log_buf)
+ elog("%s\n", log_buf);
+ goto done;
+ }
+
+ err = btf_parse_hdr(btf, err_log);
+ if (err)
+ goto done;
+
+ err = btf_parse_str_sec(btf, err_log);
+ if (err)
+ goto done;
+
+ err = btf_parse_type_sec(btf, err_log);
+
+done:
+ free(log_buf);
+
+ if (err) {
+ btf__free(btf);
+ return ERR_PTR(err);
+ }
+
+ return btf;
+}
+
+int btf__fd(const struct btf *btf)
+{
+ return btf->fd;
+}
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
new file mode 100644
index 000000000000..74bb344035bb
--- /dev/null
+++ b/tools/lib/bpf/btf.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+
+#ifndef __BPF_BTF_H
+#define __BPF_BTF_H
+
+#include <stdint.h>
+
+#define BTF_ELF_SEC ".BTF"
+
+struct btf;
+
+typedef int (*btf_print_fn_t)(const char *, ...)
+ __attribute__((format(printf, 1, 2)));
+
+void btf__free(struct btf *btf);
+struct btf *btf__new(uint8_t *data, uint32_t size, btf_print_fn_t err_log);
+int32_t btf__find_by_name(const struct btf *btf, const char *type_name);
+int64_t btf__resolve_size(const struct btf *btf, uint32_t type_id);
+int btf__fd(const struct btf *btf);
+
+#endif
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 3d35bacf656f..6513e0b08795 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -45,6 +45,7 @@
#include "libbpf.h"
#include "bpf.h"
+#include "btf.h"
#ifndef EM_BPF
#define EM_BPF 247
@@ -212,6 +213,8 @@ struct bpf_map {
char *name;
size_t offset;
struct bpf_map_def def;
+ uint32_t btf_key_id;
+ uint32_t btf_value_id;
void *priv;
bpf_map_clear_priv_t clear_priv;
};
@@ -256,6 +259,8 @@ struct bpf_object {
*/
struct list_head list;
+ struct btf *btf;
+
void *priv;
bpf_object_clear_priv_t clear_priv;
@@ -819,7 +824,15 @@ static int bpf_object__elf_collect(struct bpf_object *obj)
data->d_size);
else if (strcmp(name, "maps") == 0)
obj->efile.maps_shndx = idx;
- else if (sh.sh_type == SHT_SYMTAB) {
+ else if (strcmp(name, BTF_ELF_SEC) == 0) {
+ obj->btf = btf__new(data->d_buf, data->d_size,
+ __pr_debug);
+ if (IS_ERR(obj->btf)) {
+ pr_warning("Error loading ELF section %s: %ld. Ignored and continue.\n",
+ BTF_ELF_SEC, PTR_ERR(obj->btf));
+ obj->btf = NULL;
+ }
+ } else if (sh.sh_type == SHT_SYMTAB) {
if (obj->efile.symbols) {
pr_warning("bpf: multiple SYMTAB in %s\n",
obj->path);
@@ -996,33 +1009,126 @@ bpf_program__collect_reloc(struct bpf_program *prog, GElf_Shdr *shdr,
return 0;
}
+static int bpf_map_find_btf_info(struct bpf_map *map, const struct btf *btf)
+{
+ struct bpf_map_def *def = &map->def;
+ const size_t max_name = 256;
+ int64_t key_size, value_size;
+ int32_t key_id, value_id;
+ char name[max_name];
+
+ /* Find key type by name from BTF */
+ if (snprintf(name, max_name, "%s_key", map->name) == max_name) {
+ pr_warning("map:%s length of BTF key_type:%s_key is too long\n",
+ map->name, map->name);
+ return -EINVAL;
+ }
+
+ key_id = btf__find_by_name(btf, name);
+ if (key_id < 0) {
+ pr_debug("map:%s key_type:%s cannot be found in BTF\n",
+ map->name, name);
+ return key_id;
+ }
+
+ key_size = btf__resolve_size(btf, key_id);
+ if (key_size < 0) {
+ pr_warning("map:%s key_type:%s cannot get the BTF type_size\n",
+ map->name, name);
+ return key_size;
+ }
+
+ if (def->key_size != key_size) {
+ pr_warning("map:%s key_type:%s has BTF type_size:%ld != key_size:%u\n",
+ map->name, name, key_size, def->key_size);
+ return -EINVAL;
+ }
+
+ /* Find value type from BTF */
+ if (snprintf(name, max_name, "%s_value", map->name) == max_name) {
+ pr_warning("map:%s length of BTF value_type:%s_value is too long\n",
+ map->name, map->name);
+ return -EINVAL;
+ }
+
+ value_id = btf__find_by_name(btf, name);
+ if (value_id < 0) {
+ pr_debug("map:%s value_type:%s cannot be found in BTF\n",
+ map->name, name);
+ return value_id;
+ }
+
+ value_size = btf__resolve_size(btf, value_id);
+ if (value_size < 0) {
+ pr_warning("map:%s value_type:%s cannot get the BTF type_size\n",
+ map->name, name);
+ return value_size;
+ }
+
+ if (def->value_size != value_size) {
+ pr_warning("map:%s value_type:%s has BTF type_size:%ld != value_size:%u\n",
+ map->name, name, value_size, def->value_size);
+ return -EINVAL;
+ }
+
+ map->btf_key_id = key_id;
+ map->btf_value_id = value_id;
+
+ return 0;
+}
+
static int
bpf_object__create_maps(struct bpf_object *obj)
{
+ struct bpf_create_map_attr create_attr = {};
unsigned int i;
+ int err;
for (i = 0; i < obj->nr_maps; i++) {
- struct bpf_map_def *def = &obj->maps[i].def;
- int *pfd = &obj->maps[i].fd;
-
- *pfd = bpf_create_map_name(def->type,
- obj->maps[i].name,
- def->key_size,
- def->value_size,
- def->max_entries,
- def->map_flags);
+ struct bpf_map *map = &obj->maps[i];
+ struct bpf_map_def *def = &map->def;
+ int *pfd = &map->fd;
+
+ create_attr.name = map->name;
+ create_attr.map_type = def->type;
+ create_attr.map_flags = def->map_flags;
+ create_attr.key_size = def->key_size;
+ create_attr.value_size = def->value_size;
+ create_attr.max_entries = def->max_entries;
+ create_attr.btf_fd = 0;
+ create_attr.btf_key_id = 0;
+ create_attr.btf_value_id = 0;
+
+ if (obj->btf && !bpf_map_find_btf_info(map, obj->btf)) {
+ create_attr.btf_fd = btf__fd(obj->btf);
+ create_attr.btf_key_id = map->btf_key_id;
+ create_attr.btf_value_id = map->btf_value_id;
+ }
+
+ *pfd = bpf_create_map_xattr(&create_attr);
+ if (*pfd < 0 && create_attr.btf_key_id) {
+ pr_warning("Error in bpf_create_map_xattr(%s):%s(%d). Retrying without BTF.\n",
+ map->name, strerror(errno), errno);
+ create_attr.btf_fd = 0;
+ create_attr.btf_key_id = 0;
+ create_attr.btf_value_id = 0;
+ map->btf_key_id = 0;
+ map->btf_value_id = 0;
+ *pfd = bpf_create_map_xattr(&create_attr);
+ }
+
if (*pfd < 0) {
size_t j;
- int err = *pfd;
+ err = *pfd;
pr_warning("failed to create map (name: '%s'): %s\n",
- obj->maps[i].name,
+ map->name,
strerror(errno));
for (j = 0; j < i; j++)
zclose(obj->maps[j].fd);
return err;
}
- pr_debug("create map %s: fd=%d\n", obj->maps[i].name, *pfd);
+ pr_debug("create map %s: fd=%d\n", map->name, *pfd);
}
return 0;
@@ -1641,6 +1747,7 @@ void bpf_object__close(struct bpf_object *obj)
bpf_object__elf_finish(obj);
bpf_object__unload(obj);
+ btf__free(obj->btf);
for (i = 0; i < obj->nr_maps; i++) {
zfree(&obj->maps[i].name);
@@ -1692,6 +1799,11 @@ unsigned int bpf_object__kversion(struct bpf_object *obj)
return obj ? obj->kern_version : 0;
}
+int bpf_object__btf_fd(const struct bpf_object *obj)
+{
+ return obj->btf ? btf__fd(obj->btf) : -1;
+}
+
int bpf_object__set_priv(struct bpf_object *obj, void *priv,
bpf_object_clear_priv_t clear_priv)
{
@@ -1937,6 +2049,16 @@ const char *bpf_map__name(struct bpf_map *map)
return map ? map->name : NULL;
}
+uint32_t bpf_map__btf_key_id(const struct bpf_map *map)
+{
+ return map ? map->btf_key_id : 0;
+}
+
+uint32_t bpf_map__btf_value_id(const struct bpf_map *map)
+{
+ return map ? map->btf_value_id : 0;
+}
+
int bpf_map__set_priv(struct bpf_map *map, void *priv,
bpf_map_clear_priv_t clear_priv)
{
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 8b242486b464..d6ac4fa6f472 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -78,6 +78,7 @@ int bpf_object__load(struct bpf_object *obj);
int bpf_object__unload(struct bpf_object *obj);
const char *bpf_object__name(struct bpf_object *obj);
unsigned int bpf_object__kversion(struct bpf_object *obj);
+int bpf_object__btf_fd(const struct bpf_object *obj);
struct bpf_object *bpf_object__next(struct bpf_object *prev);
#define bpf_object__for_each_safe(pos, tmp) \
@@ -241,6 +242,8 @@ bpf_map__next(struct bpf_map *map, struct bpf_object *obj);
int bpf_map__fd(struct bpf_map *map);
const struct bpf_map_def *bpf_map__def(struct bpf_map *map);
const char *bpf_map__name(struct bpf_map *map);
+uint32_t bpf_map__btf_key_id(const struct bpf_map *map);
+uint32_t bpf_map__btf_value_id(const struct bpf_map *map);
typedef void (*bpf_map_clear_priv_t)(struct bpf_map *, void *);
int bpf_map__set_priv(struct bpf_map *map, void *priv,
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* libbpf backward compatibility (was: [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf)
2018-04-18 22:56 ` [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf Martin KaFai Lau
@ 2018-05-09 22:17 ` Jakub Kicinski
2018-05-09 22:20 ` Jakub Kicinski
0 siblings, 1 reply; 38+ messages in thread
From: Jakub Kicinski @ 2018-05-09 22:17 UTC (permalink / raw)
To: Martin KaFai Lau, Alexei Starovoitov, Daniel Borkmann, Andrey Ignatov
Cc: netdev, kernel-team, David Beckett
On Wed, 18 Apr 2018 15:56:05 -0700, Martin KaFai Lau wrote:
> diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
> index 39f6a0d64a3b..01bda076310f 100644
> --- a/tools/lib/bpf/bpf.h
> +++ b/tools/lib/bpf/bpf.h
> @@ -26,6 +26,20 @@
> #include <linux/bpf.h>
> #include <stddef.h>
>
> +struct bpf_create_map_attr {
> + const char *name;
> + enum bpf_map_type map_type;
> + __u32 map_flags;
> + __u32 key_size;
> + __u32 value_size;
> + __u32 max_entries;
> + __u32 numa_node;
> + __u32 btf_fd;
> + __u32 btf_key_id;
> + __u32 btf_value_id;
> +};
> +
> +int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr);
> int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
> int key_size, int value_size, int max_entries,
> __u32 map_flags, int node);
> @@ -87,4 +101,6 @@ int bpf_obj_get_info_by_fd(int prog_fd, void *info, __u32 *info_len);
> int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags,
> __u32 *attach_flags, __u32 *prog_ids, __u32 *prog_cnt);
> int bpf_raw_tracepoint_open(const char *name, int prog_fd);
> +int bpf_load_btf(void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size,
> + bool do_log);
> #endif
Does libbpf have to provide backward compatibility? Are the function
prototypes not supposed to change?
Recently a number of *_xattr functions were added, these are nice for
limiting the number of parameters passed to functions, but they don't
buy us anything in terms of extensibility unless we can grow the
structures. Should structure size be passed in as one of parameters?
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: libbpf backward compatibility (was: [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf)
2018-05-09 22:17 ` libbpf backward compatibility (was: [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf) Jakub Kicinski
@ 2018-05-09 22:20 ` Jakub Kicinski
0 siblings, 0 replies; 38+ messages in thread
From: Jakub Kicinski @ 2018-05-09 22:20 UTC (permalink / raw)
To: Martin KaFai Lau, Alexei Starovoitov, Daniel Borkmann, Andrey Ignatov
Cc: netdev, kernel-team, David Beckett
On Wed, 9 May 2018 15:17:12 -0700, Jakub Kicinski wrote:
> On Wed, 18 Apr 2018 15:56:05 -0700, Martin KaFai Lau wrote:
> > diff --git a/tools/lib/bpf/bpf.h b/tools/lib/bpf/bpf.h
> > index 39f6a0d64a3b..01bda076310f 100644
> > --- a/tools/lib/bpf/bpf.h
> > +++ b/tools/lib/bpf/bpf.h
> > @@ -26,6 +26,20 @@
> > #include <linux/bpf.h>
> > #include <stddef.h>
> >
> > +struct bpf_create_map_attr {
> > + const char *name;
> > + enum bpf_map_type map_type;
> > + __u32 map_flags;
> > + __u32 key_size;
> > + __u32 value_size;
> > + __u32 max_entries;
> > + __u32 numa_node;
> > + __u32 btf_fd;
> > + __u32 btf_key_id;
> > + __u32 btf_value_id;
> > +};
> > +
> > +int bpf_create_map_xattr(const struct bpf_create_map_attr *create_attr);
> > int bpf_create_map_node(enum bpf_map_type map_type, const char *name,
> > int key_size, int value_size, int max_entries,
> > __u32 map_flags, int node);
> > @@ -87,4 +101,6 @@ int bpf_obj_get_info_by_fd(int prog_fd, void *info, __u32 *info_len);
> > int bpf_prog_query(int target_fd, enum bpf_attach_type type, __u32 query_flags,
> > __u32 *attach_flags, __u32 *prog_ids, __u32 *prog_cnt);
> > int bpf_raw_tracepoint_open(const char *name, int prog_fd);
> > +int bpf_load_btf(void *btf, __u32 btf_size, char *log_buf, __u32 log_buf_size,
> > + bool do_log);
> > #endif
>
> Does libbpf have to provide backward compatibility? Are the function
> prototypes not supposed to change?
>
> Recently a number of *_xattr functions were added, these are nice for
> limiting the number of parameters passed to functions, but they don't
> buy us anything in terms of extensibility unless we can grow the
> structures. Should structure size be passed in as one of parameters?
Or is the backward compatibility not at the binary level so we can grow
the structures at will?
^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH bpf-next v5 10/10] bpf: btf: Add BTF tests
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (8 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 09/10] bpf: btf: Add BTF support to libbpf Martin KaFai Lau
@ 2018-04-18 22:56 ` Martin KaFai Lau
2018-04-19 19:40 ` [PATCH bpf-next v5 00/10] BTF: BPF Type Format Arnaldo Carvalho de Melo
2018-04-19 23:57 ` Daniel Borkmann
11 siblings, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-18 22:56 UTC (permalink / raw)
To: netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
This patch tests the BTF loading, map_create with BTF
and the changes in libbpf.
-r: Raw tests that test raw crafted BTF data
-f: Test LLVM compiled bpf prog with BTF data
-g: Test BPF_OBJ_GET_INFO_BY_FD for btf_fd
-p: Test pretty print
The tools/testing/selftests/bpf/Makefile will probe
for BTF support in llc and pahole before generating
debug info (-g) and convert them to BTF. You can supply
the BTF supported binary through the following make variables:
LLC, BTF_PAHOLE and LLVM_OBJCOPY.
LLC: The lastest llc with -mattr=dwarfris support for the bpf target.
It is only in the master of the llvm repo for now.
BTF_PAHOLE: The modified pahole with BTF support:
https://github.com/iamkafai/pahole/tree/btf
To add a BTF section: "pahole -J bpf_prog.o"
LLVM_OBJCOPY: Any llvm-objcopy should do
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Alexei Starovoitov <ast@fb.com>
---
tools/testing/selftests/bpf/Makefile | 26 +-
tools/testing/selftests/bpf/test_btf.c | 1669 ++++++++++++++++++++++++++
tools/testing/selftests/bpf/test_btf_haskv.c | 48 +
tools/testing/selftests/bpf/test_btf_nokv.c | 43 +
4 files changed, 1783 insertions(+), 3 deletions(-)
create mode 100644 tools/testing/selftests/bpf/test_btf.c
create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 0a315ddabbf4..8d954926de91 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -24,14 +24,15 @@ urandom_read: urandom_read.c
# Order correspond to 'make run_tests' order
TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \
test_align test_verifier_log test_dev_cgroup test_tcpbpf_user \
- test_sock test_sock_addr
+ test_sock test_sock_addr test_btf
TEST_GEN_FILES = test_pkt_access.o test_xdp.o test_l4lb.o test_tcp_estats.o test_obj_id.o \
test_pkt_md_access.o test_xdp_redirect.o test_xdp_meta.o sockmap_parse_prog.o \
sockmap_verdict_prog.o dev_cgroup.o sample_ret0.o test_tracepoint.o \
test_l4lb_noinline.o test_xdp_noinline.o test_stacktrace_map.o \
sample_map_ret0.o test_tcpbpf_kern.o test_stacktrace_build_id.o \
- sockmap_tcp_msg_prog.o connect4_prog.o connect6_prog.o
+ sockmap_tcp_msg_prog.o connect4_prog.o connect6_prog.o \
+ test_btf_haskv.o test_btf_nokv.o
# Order correspond to 'make run_tests' order
TEST_PROGS := test_kmod.sh \
@@ -66,6 +67,8 @@ $(BPFOBJ): force
CLANG ?= clang
LLC ?= llc
+LLVM_OBJCOPY ?= llvm-objcopy
+BTF_PAHOLE ?= pahole
PROBE := $(shell $(LLC) -march=bpf -mcpu=probe -filetype=null /dev/null 2>&1)
@@ -83,9 +86,26 @@ CLANG_FLAGS = -I. -I./include/uapi -I../../../include/uapi \
$(OUTPUT)/test_l4lb_noinline.o: CLANG_FLAGS += -fno-inline
$(OUTPUT)/test_xdp_noinline.o: CLANG_FLAGS += -fno-inline
+BTF_LLC_PROBE := $(shell $(LLC) -march=bpf -mattr=help |& grep dwarfris)
+BTF_PAHOLE_PROBE := $(shell $(BTF_PAHOLE) --help |& grep BTF)
+BTF_OBJCOPY_PROBE := $(shell $(LLVM_OBJCOPY) --version |& grep LLVM)
+
+ifneq ($(BTF_LLC_PROBE),)
+ifneq ($(BTF_PAHOLE_PROBE),)
+ifneq ($(BTF_OBJCOPY_PROBE),)
+ CLANG_FLAGS += -g
+ LLC_FLAGS += -mattr=dwarfris
+ DWARF2BTF = y
+endif
+endif
+endif
+
$(OUTPUT)/%.o: %.c
$(CLANG) $(CLANG_FLAGS) \
-O2 -target bpf -emit-llvm -c $< -o - | \
- $(LLC) -march=bpf -mcpu=$(CPU) -filetype=obj -o $@
+ $(LLC) -march=bpf -mcpu=$(CPU) $(LLC_FLAGS) -filetype=obj -o $@
+ifeq ($(DWARF2BTF),y)
+ $(BTF_PAHOLE) -J $@
+endif
EXTRA_CLEAN := $(TEST_CUSTOM_PROGS)
diff --git a/tools/testing/selftests/bpf/test_btf.c b/tools/testing/selftests/bpf/test_btf.c
new file mode 100644
index 000000000000..7b39b1f712a1
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_btf.c
@@ -0,0 +1,1669 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+
+#include <linux/bpf.h>
+#include <linux/btf.h>
+#include <linux/err.h>
+#include <bpf/bpf.h>
+#include <sys/resource.h>
+#include <libelf.h>
+#include <gelf.h>
+#include <string.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <stdarg.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <errno.h>
+#include <bpf/libbpf.h>
+#include <bpf/btf.h>
+
+#include "bpf_rlimit.h"
+
+#define min(a, b) ((a) < (b) ? (a) : (b))
+#define __printf(a, b) __attribute__((format(printf, a, b)))
+
+__printf(1, 2)
+static int __base_pr(const char *format, ...)
+{
+ va_list args;
+ int err;
+
+ va_start(args, format);
+ err = vfprintf(stderr, format, args);
+ va_end(args);
+ return err;
+}
+
+#define BTF_INFO_ENC(kind, root, vlen) \
+ ((!!(root) << 31) | ((kind) << 24) | ((vlen) & BTF_MAX_VLEN))
+
+#define BTF_TYPE_ENC(name, info, size_or_type) \
+ (name), (info), (size_or_type)
+
+#define BTF_INT_ENC(encoding, bits_offset, nr_bits) \
+ ((encoding) << 24 | (bits_offset) << 16 | (nr_bits))
+#define BTF_TYPE_INT_ENC(name, encoding, bits_offset, bits, sz) \
+ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_INT, 0, 0), sz), \
+ BTF_INT_ENC(encoding, bits_offset, bits)
+
+#define BTF_ARRAY_ENC(type, index_type, nr_elems) \
+ (type), (index_type), (nr_elems)
+#define BTF_TYPE_ARRAY_ENC(type, index_type, nr_elems) \
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_ARRAY, 0, 0), 0), \
+ BTF_ARRAY_ENC(type, index_type, nr_elems)
+
+#define BTF_MEMBER_ENC(name, type, bits_offset) \
+ (name), (type), (bits_offset)
+#define BTF_ENUM_ENC(name, val) (name), (val)
+
+#define BTF_TYPEDEF_ENC(name, type) \
+ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_TYPEDEF, 0, 0), type)
+
+#define BTF_PTR_ENC(name, type) \
+ BTF_TYPE_ENC(name, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), type)
+
+#define BTF_END_RAW 0xdeadbeef
+#define NAME_TBD 0xdeadb33f
+
+#define MAX_NR_RAW_TYPES 1024
+#define BTF_LOG_BUF_SIZE 65535
+
+#ifndef ARRAY_SIZE
+# define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0]))
+#endif
+
+static struct args {
+ unsigned int raw_test_num;
+ unsigned int file_test_num;
+ unsigned int get_info_test_num;
+ bool raw_test;
+ bool file_test;
+ bool get_info_test;
+ bool pprint_test;
+ bool always_log;
+} args;
+
+static char btf_log_buf[BTF_LOG_BUF_SIZE];
+
+static struct btf_header hdr_tmpl = {
+ .magic = BTF_MAGIC,
+ .version = BTF_VERSION,
+};
+
+struct btf_raw_test {
+ const char *descr;
+ const char *str_sec;
+ const char *map_name;
+ __u32 raw_types[MAX_NR_RAW_TYPES];
+ __u32 str_sec_size;
+ enum bpf_map_type map_type;
+ __u32 key_size;
+ __u32 value_size;
+ __u32 key_id;
+ __u32 value_id;
+ __u32 max_entries;
+ bool btf_load_err;
+ bool map_create_err;
+ int type_off_delta;
+ int str_off_delta;
+ int str_len_delta;
+};
+
+static struct btf_raw_test raw_tests[] = {
+/* enum E {
+ * E0,
+ * E1,
+ * };
+ *
+ * struct A {
+ * int m;
+ * unsigned long long n;
+ * char o;
+ * [3 bytes hole]
+ * int p[8];
+ * int q[4][8];
+ * enum E r;
+ * };
+ */
+{
+ .descr = "struct test #1",
+ .raw_types = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ /* unsigned long long */
+ BTF_TYPE_INT_ENC(0, 0, 0, 64, 8), /* [2] */
+ /* char */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 8, 1), /* [3] */
+ /* int[8] */
+ BTF_TYPE_ARRAY_ENC(1, 1, 8), /* [4] */
+ /* struct A { */ /* [5] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 6), 180),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* unsigned long long n;*/
+ BTF_MEMBER_ENC(NAME_TBD, 3, 96),/* char o; */
+ BTF_MEMBER_ENC(NAME_TBD, 4, 128),/* int p[8] */
+ BTF_MEMBER_ENC(NAME_TBD, 6, 384),/* int q[4][8] */
+ BTF_MEMBER_ENC(NAME_TBD, 7, 1408), /* enum E r */
+ /* } */
+ /* int[4][8] */
+ BTF_TYPE_ARRAY_ENC(4, 1, 4), /* [6] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_ENUM, 0, 2), sizeof(int)),
+ BTF_ENUM_ENC(NAME_TBD, 0),
+ BTF_ENUM_ENC(NAME_TBD, 1),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0n\0o\0p\0q\0r\0E\0E0\0E1",
+ .str_sec_size = sizeof("\0A\0m\0n\0o\0p\0q\0r\0E\0E0\0E1"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "struct_test1_map",
+ .key_size = sizeof(int),
+ .value_size = 180,
+ .key_id = 1,
+ .value_id = 5,
+ .max_entries = 4,
+},
+
+/* typedef struct b Struct_B;
+ *
+ * struct A {
+ * int m;
+ * struct b n[4];
+ * const Struct_B o[4];
+ * };
+ *
+ * struct B {
+ * int m;
+ * int n;
+ * };
+ */
+{
+ .descr = "struct test #2",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* struct b [4] */ /* [2] */
+ BTF_TYPE_ARRAY_ENC(4, 1, 4),
+
+ /* struct A { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 3), 68),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* struct B n[4] */
+ BTF_MEMBER_ENC(NAME_TBD, 8, 288),/* const Struct_B o[4];*/
+ /* } */
+
+ /* struct B { */ /* [4] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 8),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 1, 32),/* int n; */
+ /* } */
+
+ /* const int */ /* [5] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 1),
+ /* typedef struct b Struct_B */ /* [6] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_TYPEDEF, 0, 0), 4),
+ /* const Struct_B */ /* [7] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 6),
+ /* const Struct_B [4] */ /* [8] */
+ BTF_TYPE_ARRAY_ENC(7, 1, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0n\0o\0B\0m\0n\0Struct_B",
+ .str_sec_size = sizeof("\0A\0m\0n\0o\0B\0m\0n\0Struct_B"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "struct_test2_map",
+ .key_size = sizeof(int),
+ .value_size = 68,
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+},
+
+/* Test member exceeds the size of struct.
+ *
+ * struct A {
+ * int m;
+ * int n;
+ * };
+ */
+{
+ .descr = "size check test #1",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* struct A { */ /* [2] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), sizeof(int) * 2 - 1),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* int n; */
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0n",
+ .str_sec_size = sizeof("\0A\0m\0n"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "size_check1_map",
+ .key_size = sizeof(int),
+ .value_size = 1,
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* Test member exeeds the size of struct
+ *
+ * struct A {
+ * int m;
+ * int n[2];
+ * };
+ */
+{
+ .descr = "size check test #2",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, sizeof(int)),
+ /* int[2] */ /* [2] */
+ BTF_TYPE_ARRAY_ENC(1, 1, 2),
+ /* struct A { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), sizeof(int) * 3 - 1),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* int n[2]; */
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0n",
+ .str_sec_size = sizeof("\0A\0m\0n"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "size_check2_map",
+ .key_size = sizeof(int),
+ .value_size = 1,
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+ .btf_load_err = true,
+
+},
+
+/* Test member exeeds the size of struct
+ *
+ * struct A {
+ * int m;
+ * void *n;
+ * };
+ */
+{
+ .descr = "size check test #3",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, sizeof(int)),
+ /* void* */ /* [2] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 0),
+ /* struct A { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), sizeof(int) + sizeof(void *) - 1),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* void *n; */
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0n",
+ .str_sec_size = sizeof("\0A\0m\0n"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "size_check3_map",
+ .key_size = sizeof(int),
+ .value_size = 1,
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* Test member exceeds the size of struct
+ *
+ * enum E {
+ * E0,
+ * E1,
+ * };
+ *
+ * struct A {
+ * int m;
+ * enum E n;
+ * };
+ */
+{
+ .descr = "size check test #4",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, sizeof(int)),
+ /* enum E { */ /* [2] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_ENUM, 0, 2), sizeof(int)),
+ BTF_ENUM_ENC(NAME_TBD, 0),
+ BTF_ENUM_ENC(NAME_TBD, 1),
+ /* } */
+ /* struct A { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), sizeof(int) * 2 - 1),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* enum E n; */
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0E\0E0\0E1\0A\0m\0n",
+ .str_sec_size = sizeof("\0E\0E0\0E1\0A\0m\0n"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "size_check4_map",
+ .key_size = sizeof(int),
+ .value_size = 1,
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* typedef const void * const_void_ptr;
+ * struct A {
+ * const_void_ptr m;
+ * };
+ */
+{
+ .descr = "void test #1",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* const void */ /* [2] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 0),
+ /* const void* */ /* [3] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 2),
+ /* typedef const void * const_void_ptr */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 3),
+ /* struct A { */ /* [4] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), sizeof(void *)),
+ /* const_void_ptr m; */
+ BTF_MEMBER_ENC(NAME_TBD, 3, 0),
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0const_void_ptr\0A\0m",
+ .str_sec_size = sizeof("\0const_void_ptr\0A\0m"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "void_test1_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *),
+ .key_id = 1,
+ .value_id = 4,
+ .max_entries = 4,
+},
+
+/* struct A {
+ * const void m;
+ * };
+ */
+{
+ .descr = "void test #2",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* const void */ /* [2] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 0),
+ /* struct A { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), 8),
+ /* const void m; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 0),
+ /* } */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m",
+ .str_sec_size = sizeof("\0A\0m"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "void_test2_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *),
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* typedef const void * const_void_ptr;
+ * const_void_ptr[4]
+ */
+{
+ .descr = "void test #3",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* const void */ /* [2] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 0),
+ /* const void* */ /* [3] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 2),
+ /* typedef const void * const_void_ptr */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 3),
+ /* const_void_ptr[4] */ /* [4] */
+ BTF_TYPE_ARRAY_ENC(3, 1, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0const_void_ptr",
+ .str_sec_size = sizeof("\0const_void_ptr"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "void_test3_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *) * 4,
+ .key_id = 1,
+ .value_id = 4,
+ .max_entries = 4,
+},
+
+/* const void[4] */
+{
+ .descr = "void test #4",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* const void */ /* [2] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 0),
+ /* const void[4] */ /* [3] */
+ BTF_TYPE_ARRAY_ENC(2, 1, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m",
+ .str_sec_size = sizeof("\0A\0m"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "void_test4_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *) * 4,
+ .key_id = 1,
+ .value_id = 3,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* Array_A <------------------+
+ * elem_type == Array_B |
+ * | |
+ * | |
+ * Array_B <-------- + |
+ * elem_type == Array A --+
+ */
+{
+ .descr = "loop test #1",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* Array_A */ /* [2] */
+ BTF_TYPE_ARRAY_ENC(3, 1, 8),
+ /* Array_B */ /* [3] */
+ BTF_TYPE_ARRAY_ENC(2, 1, 8),
+ BTF_END_RAW,
+ },
+ .str_sec = "",
+ .str_sec_size = sizeof(""),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test1_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(sizeof(int) * 8),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* typedef is _before_ the BTF type of Array_A and Array_B
+ *
+ * typedef Array_B int_array;
+ *
+ * Array_A <------------------+
+ * elem_type == int_array |
+ * | |
+ * | |
+ * Array_B <-------- + |
+ * elem_type == Array_A --+
+ */
+{
+ .descr = "loop test #2",
+ .raw_types = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ /* typedef Array_B int_array */
+ BTF_TYPEDEF_ENC(1, 4), /* [2] */
+ /* Array_A */
+ BTF_TYPE_ARRAY_ENC(2, 1, 8), /* [3] */
+ /* Array_B */
+ BTF_TYPE_ARRAY_ENC(3, 1, 8), /* [4] */
+
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int_array\0",
+ .str_sec_size = sizeof("\0int_array"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test2_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(sizeof(int) * 8),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* Array_A <------------------+
+ * elem_type == Array_B |
+ * | |
+ * | |
+ * Array_B <-------- + |
+ * elem_type == Array_A --+
+ */
+{
+ .descr = "loop test #3",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* Array_A */ /* [2] */
+ BTF_TYPE_ARRAY_ENC(3, 1, 8),
+ /* Array_B */ /* [3] */
+ BTF_TYPE_ARRAY_ENC(2, 1, 8),
+
+ BTF_END_RAW,
+ },
+ .str_sec = "",
+ .str_sec_size = sizeof(""),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test3_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(sizeof(int) * 8),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* typedef is _between_ the BTF type of Array_A and Array_B
+ *
+ * typedef Array_B int_array;
+ *
+ * Array_A <------------------+
+ * elem_type == int_array |
+ * | |
+ * | |
+ * Array_B <-------- + |
+ * elem_type == Array_A --+
+ */
+{
+ .descr = "loop test #4",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* Array_A */ /* [2] */
+ BTF_TYPE_ARRAY_ENC(3, 1, 8),
+ /* typedef Array_B int_array */ /* [3] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4),
+ /* Array_B */ /* [4] */
+ BTF_TYPE_ARRAY_ENC(2, 1, 8),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int_array\0",
+ .str_sec_size = sizeof("\0int_array"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test4_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(sizeof(int) * 8),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* typedef struct B Struct_B
+ *
+ * struct A {
+ * int x;
+ * Struct_B y;
+ * };
+ *
+ * struct B {
+ * int x;
+ * struct A y;
+ * };
+ */
+{
+ .descr = "loop test #5",
+ .raw_types = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ /* struct A */ /* [2] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 8),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int x; */
+ BTF_MEMBER_ENC(NAME_TBD, 3, 32),/* Struct_B y; */
+ /* typedef struct B Struct_B */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4), /* [3] */
+ /* struct B */ /* [4] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 8),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int x; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* struct A y; */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0x\0y\0Struct_B\0B\0x\0y",
+ .str_sec_size = sizeof("\0A\0x\0y\0Struct_B\0B\0x\0y"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test5_map",
+ .key_size = sizeof(int),
+ .value_size = 8,
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+/* struct A {
+ * int x;
+ * struct A array_a[4];
+ * };
+ */
+{
+ .descr = "loop test #6",
+ .raw_types = {
+ /* int */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [1] */
+ BTF_TYPE_ARRAY_ENC(3, 1, 4), /* [2] */
+ /* struct A */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 2), 8),
+ BTF_MEMBER_ENC(NAME_TBD, 1, 0), /* int x; */
+ BTF_MEMBER_ENC(NAME_TBD, 2, 32),/* struct A array_a[4]; */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0x\0y",
+ .str_sec_size = sizeof("\0A\0x\0y"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test6_map",
+ .key_size = sizeof(int),
+ .value_size = 8,
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+{
+ .descr = "loop test #7",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* struct A { */ /* [2] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), sizeof(void *)),
+ /* const void *m; */
+ BTF_MEMBER_ENC(NAME_TBD, 3, 0),
+ /* CONST type_id=3 */ /* [3] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 4),
+ /* PTR type_id=2 */ /* [4] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 3),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m",
+ .str_sec_size = sizeof("\0A\0m"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test7_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+{
+ .descr = "loop test #8",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ /* struct A { */ /* [2] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), sizeof(void *)),
+ /* const void *m; */
+ BTF_MEMBER_ENC(NAME_TBD, 4, 0),
+ /* struct B { */ /* [3] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 1), sizeof(void *)),
+ /* const void *n; */
+ BTF_MEMBER_ENC(NAME_TBD, 6, 0),
+ /* CONST type_id=5 */ /* [4] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 5),
+ /* PTR type_id=6 */ /* [5] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 6),
+ /* CONST type_id=7 */ /* [6] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_CONST, 0, 0), 7),
+ /* PTR type_id=4 */ /* [7] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_PTR, 0, 0), 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0A\0m\0B\0n",
+ .str_sec_size = sizeof("\0A\0m\0B\0n"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "loop_test8_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(void *),
+ .key_id = 1,
+ .value_id = 2,
+ .max_entries = 4,
+ .btf_load_err = true,
+},
+
+{
+ .descr = "type_off == str_off",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .type_off_delta = sizeof(struct btf_type) + sizeof(int) + sizeof("\0int"),
+},
+
+{
+ .descr = "Unaligned type_off",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .type_off_delta = 1,
+},
+
+{
+ .descr = "str_off beyonds btf size",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .str_off_delta = sizeof("\0int") + 1,
+},
+
+{
+ .descr = "str_len beyonds btf size",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .str_len_delta = 1,
+},
+
+{
+ .descr = "String section does not end with null",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .str_len_delta = -1,
+},
+
+{
+ .descr = "Empty string section",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "\0int",
+ .str_sec_size = sizeof("\0int"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "hdr_test_map",
+ .key_size = sizeof(int),
+ .value_size = sizeof(int),
+ .key_id = 1,
+ .value_id = 1,
+ .max_entries = 4,
+ .btf_load_err = true,
+ .str_len_delta = 0 - (int)sizeof("\0int"),
+},
+
+}; /* struct btf_raw_test raw_tests[] */
+
+static const char *get_next_str(const char *start, const char *end)
+{
+ return start < end - 1 ? start + 1 : NULL;
+}
+
+static int get_type_sec_size(const __u32 *raw_types)
+{
+ int i;
+
+ for (i = MAX_NR_RAW_TYPES - 1;
+ i >= 0 && raw_types[i] != BTF_END_RAW;
+ i--)
+ ;
+
+ return i < 0 ? i : i * sizeof(raw_types[0]);
+}
+
+static void *btf_raw_create(const struct btf_header *hdr,
+ const __u32 *raw_types,
+ const char *str,
+ unsigned int str_sec_size,
+ unsigned int *btf_size)
+{
+ const char *next_str = str, *end_str = str + str_sec_size;
+ unsigned int size_needed, offset;
+ struct btf_header *ret_hdr;
+ int i, type_sec_size;
+ uint32_t *ret_types;
+ void *raw_btf;
+
+ type_sec_size = get_type_sec_size(raw_types);
+ if (type_sec_size < 0) {
+ fprintf(stderr, "Cannot get nr_raw_types\n");
+ return NULL;
+ }
+
+ size_needed = sizeof(*hdr) + type_sec_size + str_sec_size;
+ raw_btf = malloc(size_needed);
+ if (!raw_btf) {
+ fprintf(stderr, "Cannot allocate memory for raw_btf\n");
+ return NULL;
+ }
+
+ /* Copy header */
+ memcpy(raw_btf, hdr, sizeof(*hdr));
+ offset = sizeof(*hdr);
+
+ /* Copy type section */
+ ret_types = raw_btf + offset;
+ for (i = 0; i < type_sec_size / sizeof(raw_types[0]); i++) {
+ if (raw_types[i] == NAME_TBD) {
+ next_str = get_next_str(next_str, end_str);
+ if (!next_str) {
+ fprintf(stderr, "Error in getting next_str\n");
+ free(raw_btf);
+ return NULL;
+ }
+ ret_types[i] = next_str - str;
+ next_str += strlen(next_str);
+ } else {
+ ret_types[i] = raw_types[i];
+ }
+ }
+ offset += type_sec_size;
+
+ /* Copy string section */
+ memcpy(raw_btf + offset, str, str_sec_size);
+
+ ret_hdr = (struct btf_header *)raw_btf;
+ ret_hdr->str_off = type_sec_size;
+ ret_hdr->str_len = str_sec_size;
+
+ *btf_size = size_needed;
+
+ return raw_btf;
+}
+
+static int do_test_raw(unsigned int test_num)
+{
+ struct btf_raw_test *test = &raw_tests[test_num - 1];
+ struct bpf_create_map_attr create_attr = {};
+ int map_fd = -1, btf_fd = -1;
+ unsigned int raw_btf_size;
+ struct btf_header *hdr;
+ void *raw_btf;
+ int err;
+
+ fprintf(stderr, "BTF raw test[%u] (%s): ", test_num, test->descr);
+ raw_btf = btf_raw_create(&hdr_tmpl,
+ test->raw_types,
+ test->str_sec,
+ test->str_sec_size,
+ &raw_btf_size);
+
+ if (!raw_btf)
+ return -1;
+
+ hdr = raw_btf;
+
+ hdr->type_off = (int)hdr->type_off + test->type_off_delta;
+ hdr->str_off = (int)hdr->str_off + test->str_off_delta;
+ hdr->str_len = (int)hdr->str_len + test->str_len_delta;
+
+ *btf_log_buf = '\0';
+ btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
+ btf_log_buf, BTF_LOG_BUF_SIZE,
+ args.always_log);
+ free(raw_btf);
+
+ err = ((btf_fd == -1) != test->btf_load_err);
+ if (err)
+ fprintf(stderr, "btf_load_err:%d btf_fd:%d\n",
+ test->btf_load_err, btf_fd);
+
+ if (err || btf_fd == -1)
+ goto done;
+
+ create_attr.name = test->map_name;
+ create_attr.map_type = test->map_type;
+ create_attr.key_size = test->key_size;
+ create_attr.value_size = test->value_size;
+ create_attr.max_entries = test->max_entries;
+ create_attr.btf_fd = btf_fd;
+ create_attr.btf_key_id = test->key_id;
+ create_attr.btf_value_id = test->value_id;
+
+ map_fd = bpf_create_map_xattr(&create_attr);
+
+ err = ((map_fd == -1) != test->map_create_err);
+ if (err)
+ fprintf(stderr, "map_create_err:%d map_fd:%d\n",
+ test->map_create_err, map_fd);
+
+done:
+ if (!err)
+ fprintf(stderr, "OK\n");
+
+ if (*btf_log_buf && (err || args.always_log))
+ fprintf(stderr, "%s\n", btf_log_buf);
+
+ if (btf_fd != -1)
+ close(btf_fd);
+ if (map_fd != -1)
+ close(map_fd);
+
+ return err;
+}
+
+static int test_raw(void)
+{
+ unsigned int i;
+ int err = 0;
+
+ if (args.raw_test_num)
+ return do_test_raw(args.raw_test_num);
+
+ for (i = 1; i <= ARRAY_SIZE(raw_tests); i++)
+ err |= do_test_raw(i);
+
+ return err;
+}
+
+struct btf_get_info_test {
+ const char *descr;
+ const char *str_sec;
+ __u32 raw_types[MAX_NR_RAW_TYPES];
+ __u32 str_sec_size;
+ int info_size_delta;
+};
+
+const struct btf_get_info_test get_info_tests[] = {
+{
+ .descr = "== raw_btf_size+1",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "",
+ .str_sec_size = sizeof(""),
+ .info_size_delta = 1,
+},
+{
+ .descr = "== raw_btf_size-3",
+ .raw_types = {
+ /* int */ /* [1] */
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4),
+ BTF_END_RAW,
+ },
+ .str_sec = "",
+ .str_sec_size = sizeof(""),
+ .info_size_delta = -3,
+},
+};
+
+static int do_test_get_info(unsigned int test_num)
+{
+ const struct btf_get_info_test *test = &get_info_tests[test_num - 1];
+ unsigned int raw_btf_size, user_btf_size, expected_nbytes;
+ uint8_t *raw_btf = NULL, *user_btf = NULL;
+ int btf_fd = -1, err;
+
+ fprintf(stderr, "BTF GET_INFO_BY_ID test[%u] (%s): ",
+ test_num, test->descr);
+
+ raw_btf = btf_raw_create(&hdr_tmpl,
+ test->raw_types,
+ test->str_sec,
+ test->str_sec_size,
+ &raw_btf_size);
+
+ if (!raw_btf)
+ return -1;
+
+ *btf_log_buf = '\0';
+
+ user_btf = malloc(raw_btf_size);
+ if (!user_btf) {
+ fprintf(stderr, "Cannot allocate memory for user_btf\n");
+ err = -1;
+ goto done;
+ }
+
+ btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
+ btf_log_buf, BTF_LOG_BUF_SIZE,
+ args.always_log);
+ if (btf_fd == -1) {
+ fprintf(stderr, "bpf_load_btf:%s(%d)\n",
+ strerror(errno), errno);
+ err = -1;
+ goto done;
+ }
+
+ user_btf_size = (int)raw_btf_size + test->info_size_delta;
+ expected_nbytes = min(raw_btf_size, user_btf_size);
+ if (raw_btf_size > expected_nbytes)
+ memset(user_btf + expected_nbytes, 0xff,
+ raw_btf_size - expected_nbytes);
+
+ err = bpf_obj_get_info_by_fd(btf_fd, user_btf, &user_btf_size);
+ if (err || user_btf_size != raw_btf_size ||
+ memcmp(raw_btf, user_btf, expected_nbytes)) {
+ fprintf(stderr,
+ "err:%d(errno:%d) raw_btf_size:%u user_btf_size:%u expected_nbytes:%u memcmp:%d\n",
+ err, errno,
+ raw_btf_size, user_btf_size, expected_nbytes,
+ memcmp(raw_btf, user_btf, expected_nbytes));
+ err = -1;
+ goto done;
+ }
+
+ while (expected_nbytes < raw_btf_size) {
+ fprintf(stderr, "%u...", expected_nbytes);
+ if (user_btf[expected_nbytes++] != 0xff) {
+ fprintf(stderr, "!= 0xff\n");
+ err = -1;
+ goto done;
+ }
+ }
+
+ fprintf(stderr, "OK\n");
+
+done:
+ if (*btf_log_buf && (err || args.always_log))
+ fprintf(stderr, "%s\n", btf_log_buf);
+
+ free(raw_btf);
+ free(user_btf);
+
+ if (btf_fd != -1)
+ close(btf_fd);
+
+ return err;
+}
+
+static int test_get_info(void)
+{
+ unsigned int i;
+ int err = 0;
+
+ if (args.get_info_test_num)
+ return do_test_get_info(args.get_info_test_num);
+
+ for (i = 1; i <= ARRAY_SIZE(get_info_tests); i++)
+ err |= do_test_get_info(i);
+
+ return err;
+}
+
+struct btf_file_test {
+ const char *file;
+ bool btf_kv_notfound;
+};
+
+static struct btf_file_test file_tests[] = {
+{
+ .file = "test_btf_haskv.o",
+},
+{
+ .file = "test_btf_nokv.o",
+ .btf_kv_notfound = true,
+},
+};
+
+static int file_has_btf_elf(const char *fn)
+{
+ Elf_Scn *scn = NULL;
+ GElf_Ehdr ehdr;
+ int elf_fd;
+ Elf *elf;
+ int ret;
+
+ if (elf_version(EV_CURRENT) == EV_NONE) {
+ fprintf(stderr, "Failed to init libelf\n");
+ return -1;
+ }
+
+ elf_fd = open(fn, O_RDONLY);
+ if (elf_fd == -1) {
+ fprintf(stderr, "Cannot open file %s: %s(%d)\n",
+ fn, strerror(errno), errno);
+ return -1;
+ }
+
+ elf = elf_begin(elf_fd, ELF_C_READ, NULL);
+ if (!elf) {
+ fprintf(stderr, "Failed to read ELF from %s. %s\n", fn,
+ elf_errmsg(elf_errno()));
+ ret = -1;
+ goto done;
+ }
+
+ if (!gelf_getehdr(elf, &ehdr)) {
+ fprintf(stderr, "Failed to get EHDR from %s\n", fn);
+ ret = -1;
+ goto done;
+ }
+
+ while ((scn = elf_nextscn(elf, scn))) {
+ const char *sh_name;
+ GElf_Shdr sh;
+
+ if (gelf_getshdr(scn, &sh) != &sh) {
+ fprintf(stderr,
+ "Failed to get section header from %s\n", fn);
+ ret = -1;
+ goto done;
+ }
+
+ sh_name = elf_strptr(elf, ehdr.e_shstrndx, sh.sh_name);
+ if (!strcmp(sh_name, BTF_ELF_SEC)) {
+ ret = 1;
+ goto done;
+ }
+ }
+
+ ret = 0;
+
+done:
+ close(elf_fd);
+ elf_end(elf);
+ return ret;
+}
+
+static int do_test_file(unsigned int test_num)
+{
+ const struct btf_file_test *test = &file_tests[test_num - 1];
+ struct bpf_object *obj = NULL;
+ struct bpf_program *prog;
+ struct bpf_map *map;
+ int err;
+
+ fprintf(stderr, "BTF libbpf test[%u] (%s): ", test_num,
+ test->file);
+
+ err = file_has_btf_elf(test->file);
+ if (err == -1)
+ return err;
+
+ if (err == 0) {
+ fprintf(stderr, "SKIP. No ELF %s found\n", BTF_ELF_SEC);
+ return 0;
+ }
+
+ obj = bpf_object__open(test->file);
+ if (IS_ERR(obj))
+ return PTR_ERR(obj);
+
+ err = bpf_object__btf_fd(obj);
+ if (err == -1) {
+ fprintf(stderr, "bpf_object__btf_fd: -1\n");
+ goto done;
+ }
+
+ prog = bpf_program__next(NULL, obj);
+ if (!prog) {
+ fprintf(stderr, "Cannot find bpf_prog\n");
+ err = -1;
+ goto done;
+ }
+
+ bpf_program__set_type(prog, BPF_PROG_TYPE_TRACEPOINT);
+ err = bpf_object__load(obj);
+ if (err < 0) {
+ fprintf(stderr, "bpf_object__load: %d\n", err);
+ goto done;
+ }
+
+ map = bpf_object__find_map_by_name(obj, "btf_map");
+ if (!map) {
+ fprintf(stderr, "btf_map not found\n");
+ err = -1;
+ goto done;
+ }
+
+ err = (bpf_map__btf_key_id(map) == 0 || bpf_map__btf_value_id(map) == 0)
+ != test->btf_kv_notfound;
+ if (err) {
+ fprintf(stderr,
+ "btf_kv_notfound:%u btf_key_id:%u btf_value_id:%u\n",
+ test->btf_kv_notfound,
+ bpf_map__btf_key_id(map),
+ bpf_map__btf_value_id(map));
+ goto done;
+ }
+
+ fprintf(stderr, "OK\n");
+
+done:
+ bpf_object__close(obj);
+ return err;
+}
+
+static int test_file(void)
+{
+ unsigned int i;
+ int err = 0;
+
+ if (args.file_test_num)
+ return do_test_file(args.file_test_num);
+
+ for (i = 1; i <= ARRAY_SIZE(file_tests); i++)
+ err |= do_test_file(i);
+
+ return err;
+}
+
+const char *pprint_enum_str[] = {
+ "ENUM_ZERO",
+ "ENUM_ONE",
+ "ENUM_TWO",
+ "ENUM_THREE",
+};
+
+struct pprint_mapv {
+ uint32_t ui32;
+ uint16_t ui16;
+ /* 2 bytes hole */
+ int32_t si32;
+ uint32_t unused_bits2a:2,
+ bits28:28,
+ unused_bits2b:2;
+ union {
+ uint64_t ui64;
+ uint8_t ui8a[8];
+ };
+ enum {
+ ENUM_ZERO,
+ ENUM_ONE,
+ ENUM_TWO,
+ ENUM_THREE,
+ } aenum;
+};
+
+static struct btf_raw_test pprint_test = {
+ .descr = "BTF pretty print test #1",
+ .raw_types = {
+ /* unsighed char */ /* [1] */
+ BTF_TYPE_INT_ENC(NAME_TBD, 0, 0, 8, 1),
+ /* unsigned short */ /* [2] */
+ BTF_TYPE_INT_ENC(NAME_TBD, 0, 0, 16, 2),
+ /* unsigned int */ /* [3] */
+ BTF_TYPE_INT_ENC(NAME_TBD, 0, 0, 32, 4),
+ /* int */ /* [4] */
+ BTF_TYPE_INT_ENC(NAME_TBD, BTF_INT_SIGNED, 0, 32, 4),
+ /* unsigned long long */ /* [5] */
+ BTF_TYPE_INT_ENC(NAME_TBD, 0, 0, 64, 8),
+ /* 2 bits */ /* [6] */
+ BTF_TYPE_INT_ENC(0, 0, 0, 2, 2),
+ /* 28 bits */ /* [7] */
+ BTF_TYPE_INT_ENC(0, 0, 0, 28, 4),
+ /* uint8_t[8] */ /* [8] */
+ BTF_TYPE_ARRAY_ENC(9, 3, 8),
+ /* typedef unsigned char uint8_t */ /* [9] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 1),
+ /* typedef unsigned short uint16_t */ /* [10] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 2),
+ /* typedef unsigned int uint32_t */ /* [11] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 3),
+ /* typedef int int32_t */ /* [12] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 4),
+ /* typedef unsigned long long uint64_t *//* [13] */
+ BTF_TYPEDEF_ENC(NAME_TBD, 5),
+ /* union (anon) */ /* [14] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_UNION, 0, 2), 8),
+ BTF_MEMBER_ENC(NAME_TBD, 13, 0),/* uint64_t ui64; */
+ BTF_MEMBER_ENC(NAME_TBD, 8, 0), /* uint8_t ui8a[8]; */
+ /* enum (anon) */ /* [15] */
+ BTF_TYPE_ENC(0, BTF_INFO_ENC(BTF_KIND_ENUM, 0, 4), 4),
+ BTF_ENUM_ENC(NAME_TBD, 0),
+ BTF_ENUM_ENC(NAME_TBD, 1),
+ BTF_ENUM_ENC(NAME_TBD, 2),
+ BTF_ENUM_ENC(NAME_TBD, 3),
+ /* struct pprint_mapv */ /* [16] */
+ BTF_TYPE_ENC(NAME_TBD, BTF_INFO_ENC(BTF_KIND_STRUCT, 0, 8), 28),
+ BTF_MEMBER_ENC(NAME_TBD, 11, 0), /* uint32_t ui32 */
+ BTF_MEMBER_ENC(NAME_TBD, 10, 32), /* uint16_t ui16 */
+ BTF_MEMBER_ENC(NAME_TBD, 12, 64), /* int32_t si32 */
+ BTF_MEMBER_ENC(NAME_TBD, 6, 96), /* unused_bits2a */
+ BTF_MEMBER_ENC(NAME_TBD, 7, 98), /* bits28 */
+ BTF_MEMBER_ENC(NAME_TBD, 6, 126), /* unused_bits2b */
+ BTF_MEMBER_ENC(0, 14, 128), /* union (anon) */
+ BTF_MEMBER_ENC(NAME_TBD, 15, 192), /* aenum */
+ BTF_END_RAW,
+ },
+ .str_sec = "\0unsigned char\0unsigned short\0unsigned int\0int\0unsigned long long\0uint8_t\0uint16_t\0uint32_t\0int32_t\0uint64_t\0ui64\0ui8a\0ENUM_ZERO\0ENUM_ONE\0ENUM_TWO\0ENUM_THREE\0pprint_mapv\0ui32\0ui16\0si32\0unused_bits2a\0bits28\0unused_bits2b\0aenum",
+ .str_sec_size = sizeof("\0unsigned char\0unsigned short\0unsigned int\0int\0unsigned long long\0uint8_t\0uint16_t\0uint32_t\0int32_t\0uint64_t\0ui64\0ui8a\0ENUM_ZERO\0ENUM_ONE\0ENUM_TWO\0ENUM_THREE\0pprint_mapv\0ui32\0ui16\0si32\0unused_bits2a\0bits28\0unused_bits2b\0aenum"),
+ .map_type = BPF_MAP_TYPE_ARRAY,
+ .map_name = "pprint_test",
+ .key_size = sizeof(unsigned int),
+ .value_size = sizeof(struct pprint_mapv),
+ .key_id = 3, /* unsigned int */
+ .value_id = 16, /* struct pprint_mapv */
+ .max_entries = 128 * 1024,
+};
+
+static void set_pprint_mapv(struct pprint_mapv *v, uint32_t i)
+{
+ v->ui32 = i;
+ v->si32 = -i;
+ v->unused_bits2a = 3;
+ v->bits28 = i;
+ v->unused_bits2b = 3;
+ v->ui64 = i;
+ v->aenum = i & 0x03;
+}
+
+static int test_pprint(void)
+{
+ const struct btf_raw_test *test = &pprint_test;
+ struct bpf_create_map_attr create_attr = {};
+ int map_fd = -1, btf_fd = -1;
+ struct pprint_mapv mapv = {};
+ unsigned int raw_btf_size;
+ char expected_line[255];
+ FILE *pin_file = NULL;
+ char pin_path[255];
+ size_t line_len = 0;
+ char *line = NULL;
+ unsigned int key;
+ uint8_t *raw_btf;
+ ssize_t nread;
+ int err;
+
+ fprintf(stderr, "%s......", test->descr);
+ raw_btf = btf_raw_create(&hdr_tmpl, test->raw_types,
+ test->str_sec, test->str_sec_size,
+ &raw_btf_size);
+
+ if (!raw_btf)
+ return -1;
+
+ *btf_log_buf = '\0';
+ btf_fd = bpf_load_btf(raw_btf, raw_btf_size,
+ btf_log_buf, BTF_LOG_BUF_SIZE,
+ args.always_log);
+ free(raw_btf);
+
+ if (btf_fd == -1) {
+ err = -1;
+ fprintf(stderr, "bpf_load_btf: %s(%d)\n",
+ strerror(errno), errno);
+ goto done;
+ }
+
+ create_attr.name = test->map_name;
+ create_attr.map_type = test->map_type;
+ create_attr.key_size = test->key_size;
+ create_attr.value_size = test->value_size;
+ create_attr.max_entries = test->max_entries;
+ create_attr.btf_fd = btf_fd;
+ create_attr.btf_key_id = test->key_id;
+ create_attr.btf_value_id = test->value_id;
+
+ map_fd = bpf_create_map_xattr(&create_attr);
+ if (map_fd == -1) {
+ err = -1;
+ fprintf(stderr, "bpf_creat_map_btf: %s(%d)\n",
+ strerror(errno), errno);
+ goto done;
+ }
+
+ if (snprintf(pin_path, sizeof(pin_path), "%s/%s",
+ "/sys/fs/bpf", test->map_name) == sizeof(pin_path)) {
+ err = -1;
+ fprintf(stderr, "pin_path is too long\n");
+ goto done;
+ }
+
+ err = bpf_obj_pin(map_fd, pin_path);
+ if (err) {
+ fprintf(stderr, "Cannot pin to %s. %s(%d).\n", pin_path,
+ strerror(errno), errno);
+ goto done;
+ }
+
+ for (key = 0; key < test->max_entries; key++) {
+ set_pprint_mapv(&mapv, key);
+ bpf_map_update_elem(map_fd, &key, &mapv, 0);
+ }
+
+ pin_file = fopen(pin_path, "r");
+ if (!pin_file) {
+ err = -1;
+ fprintf(stderr, "fopen(%s): %s(%d)\n", pin_path,
+ strerror(errno), errno);
+ goto done;
+ }
+
+ /* Skip lines start with '#' */
+ while ((nread = getline(&line, &line_len, pin_file)) > 0 &&
+ *line == '#')
+ ;
+
+ if (nread <= 0) {
+ err = -1;
+ fprintf(stderr, "Unexpected EOF\n");
+ goto done;
+ }
+
+ key = 0;
+ do {
+ ssize_t nexpected_line;
+
+ set_pprint_mapv(&mapv, key);
+ nexpected_line = snprintf(expected_line, sizeof(expected_line),
+ "%u: {%u,0,%d,0x%x,0x%x,0x%x,{%lu|[%u,%u,%u,%u,%u,%u,%u,%u]},%s}\n",
+ key,
+ mapv.ui32, mapv.si32,
+ mapv.unused_bits2a, mapv.bits28, mapv.unused_bits2b,
+ mapv.ui64,
+ mapv.ui8a[0], mapv.ui8a[1], mapv.ui8a[2], mapv.ui8a[3],
+ mapv.ui8a[4], mapv.ui8a[5], mapv.ui8a[6], mapv.ui8a[7],
+ pprint_enum_str[mapv.aenum]);
+
+ if (nexpected_line == sizeof(expected_line)) {
+ err = -1;
+ fprintf(stderr, "expected_line is too long\n");
+ goto done;
+ }
+
+ if (strcmp(expected_line, line)) {
+ err = -1;
+ fprintf(stderr, "unexpected pprint output\n");
+ fprintf(stderr, "expected: %s", expected_line);
+ fprintf(stderr, " read: %s", line);
+ goto done;
+ }
+
+ nread = getline(&line, &line_len, pin_file);
+ } while (++key < test->max_entries && nread > 0);
+
+ if (key < test->max_entries) {
+ err = -1;
+ fprintf(stderr, "Unexpected EOF\n");
+ goto done;
+ }
+
+ if (nread > 0) {
+ err = -1;
+ fprintf(stderr, "Unexpected extra pprint output: %s\n", line);
+ goto done;
+ }
+
+ err = 0;
+
+done:
+ if (!err)
+ fprintf(stderr, "OK\n");
+ if (*btf_log_buf && (err || args.always_log))
+ fprintf(stderr, "%s\n", btf_log_buf);
+ if (btf_fd != -1)
+ close(btf_fd);
+ if (map_fd != -1)
+ close(map_fd);
+ if (pin_file)
+ fclose(pin_file);
+ unlink(pin_path);
+ free(line);
+
+ return err;
+}
+
+static void usage(const char *cmd)
+{
+ fprintf(stderr, "Usage: %s [-l] [[-r test_num (1 - %zu)] | [-g test_num (1 - %zu)] | [-f test_num (1 - %zu)] | [-p]]\n",
+ cmd, ARRAY_SIZE(raw_tests), ARRAY_SIZE(get_info_tests),
+ ARRAY_SIZE(file_tests));
+}
+
+static int parse_args(int argc, char **argv)
+{
+ const char *optstr = "lpf:r:g:";
+ int opt;
+
+ while ((opt = getopt(argc, argv, optstr)) != -1) {
+ switch (opt) {
+ case 'l':
+ args.always_log = true;
+ break;
+ case 'f':
+ args.file_test_num = atoi(optarg);
+ args.file_test = true;
+ break;
+ case 'r':
+ args.raw_test_num = atoi(optarg);
+ args.raw_test = true;
+ break;
+ case 'g':
+ args.get_info_test_num = atoi(optarg);
+ args.get_info_test = true;
+ break;
+ case 'p':
+ args.pprint_test = true;
+ break;
+ case 'h':
+ usage(argv[0]);
+ exit(0);
+ default:
+ usage(argv[0]);
+ return -1;
+ }
+ }
+
+ if (args.raw_test_num &&
+ (args.raw_test_num < 1 ||
+ args.raw_test_num > ARRAY_SIZE(raw_tests))) {
+ fprintf(stderr, "BTF raw test number must be [1 - %zu]\n",
+ ARRAY_SIZE(raw_tests));
+ return -1;
+ }
+
+ if (args.file_test_num &&
+ (args.file_test_num < 1 ||
+ args.file_test_num > ARRAY_SIZE(file_tests))) {
+ fprintf(stderr, "BTF file test number must be [1 - %zu]\n",
+ ARRAY_SIZE(file_tests));
+ return -1;
+ }
+
+ if (args.get_info_test_num &&
+ (args.get_info_test_num < 1 ||
+ args.get_info_test_num > ARRAY_SIZE(get_info_tests))) {
+ fprintf(stderr, "BTF get info test number must be [1 - %zu]\n",
+ ARRAY_SIZE(get_info_tests));
+ return -1;
+ }
+
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ int err = 0;
+
+ err = parse_args(argc, argv);
+ if (err)
+ return err;
+
+ if (args.always_log)
+ libbpf_set_print(__base_pr, __base_pr, __base_pr);
+
+ if (args.raw_test)
+ err |= test_raw();
+
+ if (args.get_info_test)
+ err |= test_get_info();
+
+ if (args.file_test)
+ err |= test_file();
+
+ if (args.pprint_test)
+ err |= test_pprint();
+
+ if (args.raw_test || args.get_info_test || args.file_test ||
+ args.pprint_test)
+ return err;
+
+ err |= test_raw();
+ err |= test_get_info();
+ err |= test_file();
+
+ return err;
+}
diff --git a/tools/testing/selftests/bpf/test_btf_haskv.c b/tools/testing/selftests/bpf/test_btf_haskv.c
new file mode 100644
index 000000000000..8c7ca096ecf2
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_btf_haskv.c
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+int _version SEC("version") = 1;
+
+struct ipv_counts {
+ unsigned int v4;
+ unsigned int v6;
+};
+
+typedef int btf_map_key;
+typedef struct ipv_counts btf_map_value;
+btf_map_key dumm_key;
+btf_map_value dummy_value;
+
+struct bpf_map_def SEC("maps") btf_map = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct ipv_counts),
+ .max_entries = 4,
+};
+
+struct dummy_tracepoint_args {
+ unsigned long long pad;
+ struct sock *sock;
+};
+
+SEC("dummy_tracepoint")
+int _dummy_tracepoint(struct dummy_tracepoint_args *arg)
+{
+ struct ipv_counts *counts;
+ int key = 0;
+
+ if (!arg->sock)
+ return 0;
+
+ counts = bpf_map_lookup_elem(&btf_map, &key);
+ if (!counts)
+ return 0;
+
+ counts->v6++;
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/test_btf_nokv.c b/tools/testing/selftests/bpf/test_btf_nokv.c
new file mode 100644
index 000000000000..0ed8e088eebf
--- /dev/null
+++ b/tools/testing/selftests/bpf/test_btf_nokv.c
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+#include <linux/bpf.h>
+#include "bpf_helpers.h"
+
+int _version SEC("version") = 1;
+
+struct ipv_counts {
+ unsigned int v4;
+ unsigned int v6;
+};
+
+struct bpf_map_def SEC("maps") btf_map = {
+ .type = BPF_MAP_TYPE_ARRAY,
+ .key_size = sizeof(int),
+ .value_size = sizeof(struct ipv_counts),
+ .max_entries = 4,
+};
+
+struct dummy_tracepoint_args {
+ unsigned long long pad;
+ struct sock *sock;
+};
+
+SEC("dummy_tracepoint")
+int _dummy_tracepoint(struct dummy_tracepoint_args *arg)
+{
+ struct ipv_counts *counts;
+ int key = 0;
+
+ if (!arg->sock)
+ return 0;
+
+ counts = bpf_map_lookup_elem(&btf_map, &key);
+ if (!counts)
+ return 0;
+
+ counts->v6++;
+
+ return 0;
+}
+
+char _license[] SEC("license") = "GPL";
--
2.9.5
^ permalink raw reply related [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (9 preceding siblings ...)
2018-04-18 22:56 ` [PATCH bpf-next v5 10/10] bpf: btf: Add BTF tests Martin KaFai Lau
@ 2018-04-19 19:40 ` Arnaldo Carvalho de Melo
2018-04-19 20:58 ` Martin KaFai Lau
2018-06-05 21:25 ` Martin KaFai Lau
2018-04-19 23:57 ` Daniel Borkmann
11 siblings, 2 replies; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-04-19 19:40 UTC (permalink / raw)
To: Martin KaFai Lau; +Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> This patch introduces BPF Type Format (BTF).
>
> BTF (BPF Type Format) is the meta data format which describes
> the data types of BPF program/map. Hence, it basically focus
> on the C programming language which the modern BPF is primary
> using. The first use case is to provide a generic pretty print
> capability for a BPF map.
>
> A modified pahole that can convert dwarf to BTF is here:
> https://github.com/iamkafai/pahole/tree/btf
> (Arnaldo, there is some BTF_KIND numbering changes on
> Apr 18th, d61426c1571)
Thanks for letting me know, I'm starting to look at this,
- Arnaldo
> Please see individual patch for details.
>
> v5:
> - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> currently used. They can be added in the future.
> Some bpf_df_xxx() are removed together.
> - Add comment in patch 7 to clarify that the new bpffs_map_fops
> should not be extended further.
>
> v4:
> - Fix warning (remove unneeded semicolon)
> - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> patch 1. Caught by W=1.
>
> v3:
> - Rebase to bpf-next
> - Fix sparse warning (by adding static)
> - Add BTF header logging: btf_verifier_log_hdr()
> - Fix the alignment test on btf->type_off
> - Add tests for the BTF header
> - Lower the max BTF size to 16MB. It should be enough
> for some time. We could raise it later if it would
> be needed.
>
> v2:
> - Use kvfree where needed in patch 1 and 2
> - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> in patch 1
> - Fix an incorrect goto target in map_create() during
> the btf-error-path in patch 7
> - re-org some local vars to keep the rev xmas tree in btf.c
>
> Martin KaFai Lau (10):
> bpf: btf: Introduce BPF Type Format (BTF)
> bpf: btf: Validate type reference
> bpf: btf: Check members of struct/union
> bpf: btf: Add pretty print capability for data with BTF type info
> bpf: btf: Add BPF_BTF_LOAD command
> bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> bpf: btf: Add pretty print support to the basic arraymap
> bpf: btf: Sync bpf.h and btf.h to tools/
> bpf: btf: Add BTF support to libbpf
> bpf: btf: Add BTF tests
>
> include/linux/bpf.h | 20 +-
> include/linux/btf.h | 48 +
> include/uapi/linux/bpf.h | 12 +
> include/uapi/linux/btf.h | 130 ++
> kernel/bpf/Makefile | 1 +
> kernel/bpf/arraymap.c | 50 +
> kernel/bpf/btf.c | 2064 ++++++++++++++++++++++++++
> kernel/bpf/inode.c | 156 +-
> kernel/bpf/syscall.c | 51 +-
> tools/include/uapi/linux/bpf.h | 12 +
> tools/include/uapi/linux/btf.h | 130 ++
> tools/lib/bpf/Build | 2 +-
> tools/lib/bpf/bpf.c | 92 +-
> tools/lib/bpf/bpf.h | 16 +
> tools/lib/bpf/btf.c | 374 +++++
> tools/lib/bpf/btf.h | 22 +
> tools/lib/bpf/libbpf.c | 148 +-
> tools/lib/bpf/libbpf.h | 3 +
> tools/testing/selftests/bpf/Makefile | 26 +-
> tools/testing/selftests/bpf/test_btf.c | 1669 +++++++++++++++++++++
> tools/testing/selftests/bpf/test_btf_haskv.c | 48 +
> tools/testing/selftests/bpf/test_btf_nokv.c | 43 +
> 22 files changed, 5076 insertions(+), 41 deletions(-)
> create mode 100644 include/linux/btf.h
> create mode 100644 include/uapi/linux/btf.h
> create mode 100644 kernel/bpf/btf.c
> create mode 100644 tools/include/uapi/linux/btf.h
> create mode 100644 tools/lib/bpf/btf.c
> create mode 100644 tools/lib/bpf/btf.h
> create mode 100644 tools/testing/selftests/bpf/test_btf.c
> create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
>
> --
> 2.9.5
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-04-19 19:40 ` [PATCH bpf-next v5 00/10] BTF: BPF Type Format Arnaldo Carvalho de Melo
@ 2018-04-19 20:58 ` Martin KaFai Lau
2018-06-05 21:25 ` Martin KaFai Lau
1 sibling, 0 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-04-19 20:58 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> > This patch introduces BPF Type Format (BTF).
> >
> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map. Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using. The first use case is to provide a generic pretty print
> > capability for a BPF map.
> >
> > A modified pahole that can convert dwarf to BTF is here:
> > https://github.com/iamkafai/pahole/tree/btf
> > (Arnaldo, there is some BTF_KIND numbering changes on
> > Apr 18th, d61426c1571)
>
> Thanks for letting me know, I'm starting to look at this,
Thanks for reviewing. Feel free to comment directly on the github diff.
Also, I think it may make sense to wait for the kernel pieces to land
first.
>
> - Arnaldo
>
> > Please see individual patch for details.
> >
> > v5:
> > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> > currently used. They can be added in the future.
> > Some bpf_df_xxx() are removed together.
> > - Add comment in patch 7 to clarify that the new bpffs_map_fops
> > should not be extended further.
> >
> > v4:
> > - Fix warning (remove unneeded semicolon)
> > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> > patch 1. Caught by W=1.
> >
> > v3:
> > - Rebase to bpf-next
> > - Fix sparse warning (by adding static)
> > - Add BTF header logging: btf_verifier_log_hdr()
> > - Fix the alignment test on btf->type_off
> > - Add tests for the BTF header
> > - Lower the max BTF size to 16MB. It should be enough
> > for some time. We could raise it later if it would
> > be needed.
> >
> > v2:
> > - Use kvfree where needed in patch 1 and 2
> > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> > in patch 1
> > - Fix an incorrect goto target in map_create() during
> > the btf-error-path in patch 7
> > - re-org some local vars to keep the rev xmas tree in btf.c
> >
> > Martin KaFai Lau (10):
> > bpf: btf: Introduce BPF Type Format (BTF)
> > bpf: btf: Validate type reference
> > bpf: btf: Check members of struct/union
> > bpf: btf: Add pretty print capability for data with BTF type info
> > bpf: btf: Add BPF_BTF_LOAD command
> > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> > bpf: btf: Add pretty print support to the basic arraymap
> > bpf: btf: Sync bpf.h and btf.h to tools/
> > bpf: btf: Add BTF support to libbpf
> > bpf: btf: Add BTF tests
> >
> > include/linux/bpf.h | 20 +-
> > include/linux/btf.h | 48 +
> > include/uapi/linux/bpf.h | 12 +
> > include/uapi/linux/btf.h | 130 ++
> > kernel/bpf/Makefile | 1 +
> > kernel/bpf/arraymap.c | 50 +
> > kernel/bpf/btf.c | 2064 ++++++++++++++++++++++++++
> > kernel/bpf/inode.c | 156 +-
> > kernel/bpf/syscall.c | 51 +-
> > tools/include/uapi/linux/bpf.h | 12 +
> > tools/include/uapi/linux/btf.h | 130 ++
> > tools/lib/bpf/Build | 2 +-
> > tools/lib/bpf/bpf.c | 92 +-
> > tools/lib/bpf/bpf.h | 16 +
> > tools/lib/bpf/btf.c | 374 +++++
> > tools/lib/bpf/btf.h | 22 +
> > tools/lib/bpf/libbpf.c | 148 +-
> > tools/lib/bpf/libbpf.h | 3 +
> > tools/testing/selftests/bpf/Makefile | 26 +-
> > tools/testing/selftests/bpf/test_btf.c | 1669 +++++++++++++++++++++
> > tools/testing/selftests/bpf/test_btf_haskv.c | 48 +
> > tools/testing/selftests/bpf/test_btf_nokv.c | 43 +
> > 22 files changed, 5076 insertions(+), 41 deletions(-)
> > create mode 100644 include/linux/btf.h
> > create mode 100644 include/uapi/linux/btf.h
> > create mode 100644 kernel/bpf/btf.c
> > create mode 100644 tools/include/uapi/linux/btf.h
> > create mode 100644 tools/lib/bpf/btf.c
> > create mode 100644 tools/lib/bpf/btf.h
> > create mode 100644 tools/testing/selftests/bpf/test_btf.c
> > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> >
> > --
> > 2.9.5
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-04-19 19:40 ` [PATCH bpf-next v5 00/10] BTF: BPF Type Format Arnaldo Carvalho de Melo
2018-04-19 20:58 ` Martin KaFai Lau
@ 2018-06-05 21:25 ` Martin KaFai Lau
2018-06-06 12:33 ` Arnaldo Carvalho de Melo
2018-06-07 13:54 ` Arnaldo Carvalho de Melo
1 sibling, 2 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-06-05 21:25 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> > This patch introduces BPF Type Format (BTF).
> >
> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map. Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using. The first use case is to provide a generic pretty print
> > capability for a BPF map.
> >
> > A modified pahole that can convert dwarf to BTF is here:
> > https://github.com/iamkafai/pahole/tree/btf
> > (Arnaldo, there is some BTF_KIND numbering changes on
> > Apr 18th, d61426c1571)
>
> Thanks for letting me know, I'm starting to look at this,
Hi Arnaldo,
Do you have a chance to take a look and pull it? The kernel
changes will be in 4.18, so it will be handy if it is available in
the pahole repository.
[ btw, the latest commit (1 commit) should be 94a11b59e592 ].
Thanks,
Martin
>
> - Arnaldo
>
> > Please see individual patch for details.
> >
> > v5:
> > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> > currently used. They can be added in the future.
> > Some bpf_df_xxx() are removed together.
> > - Add comment in patch 7 to clarify that the new bpffs_map_fops
> > should not be extended further.
> >
> > v4:
> > - Fix warning (remove unneeded semicolon)
> > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> > patch 1. Caught by W=1.
> >
> > v3:
> > - Rebase to bpf-next
> > - Fix sparse warning (by adding static)
> > - Add BTF header logging: btf_verifier_log_hdr()
> > - Fix the alignment test on btf->type_off
> > - Add tests for the BTF header
> > - Lower the max BTF size to 16MB. It should be enough
> > for some time. We could raise it later if it would
> > be needed.
> >
> > v2:
> > - Use kvfree where needed in patch 1 and 2
> > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> > in patch 1
> > - Fix an incorrect goto target in map_create() during
> > the btf-error-path in patch 7
> > - re-org some local vars to keep the rev xmas tree in btf.c
> >
> > Martin KaFai Lau (10):
> > bpf: btf: Introduce BPF Type Format (BTF)
> > bpf: btf: Validate type reference
> > bpf: btf: Check members of struct/union
> > bpf: btf: Add pretty print capability for data with BTF type info
> > bpf: btf: Add BPF_BTF_LOAD command
> > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> > bpf: btf: Add pretty print support to the basic arraymap
> > bpf: btf: Sync bpf.h and btf.h to tools/
> > bpf: btf: Add BTF support to libbpf
> > bpf: btf: Add BTF tests
> >
> > include/linux/bpf.h | 20 +-
> > include/linux/btf.h | 48 +
> > include/uapi/linux/bpf.h | 12 +
> > include/uapi/linux/btf.h | 130 ++
> > kernel/bpf/Makefile | 1 +
> > kernel/bpf/arraymap.c | 50 +
> > kernel/bpf/btf.c | 2064 ++++++++++++++++++++++++++
> > kernel/bpf/inode.c | 156 +-
> > kernel/bpf/syscall.c | 51 +-
> > tools/include/uapi/linux/bpf.h | 12 +
> > tools/include/uapi/linux/btf.h | 130 ++
> > tools/lib/bpf/Build | 2 +-
> > tools/lib/bpf/bpf.c | 92 +-
> > tools/lib/bpf/bpf.h | 16 +
> > tools/lib/bpf/btf.c | 374 +++++
> > tools/lib/bpf/btf.h | 22 +
> > tools/lib/bpf/libbpf.c | 148 +-
> > tools/lib/bpf/libbpf.h | 3 +
> > tools/testing/selftests/bpf/Makefile | 26 +-
> > tools/testing/selftests/bpf/test_btf.c | 1669 +++++++++++++++++++++
> > tools/testing/selftests/bpf/test_btf_haskv.c | 48 +
> > tools/testing/selftests/bpf/test_btf_nokv.c | 43 +
> > 22 files changed, 5076 insertions(+), 41 deletions(-)
> > create mode 100644 include/linux/btf.h
> > create mode 100644 include/uapi/linux/btf.h
> > create mode 100644 kernel/bpf/btf.c
> > create mode 100644 tools/include/uapi/linux/btf.h
> > create mode 100644 tools/lib/bpf/btf.c
> > create mode 100644 tools/lib/bpf/btf.h
> > create mode 100644 tools/testing/selftests/bpf/test_btf.c
> > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> >
> > --
> > 2.9.5
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-05 21:25 ` Martin KaFai Lau
@ 2018-06-06 12:33 ` Arnaldo Carvalho de Melo
2018-06-07 13:54 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-06 12:33 UTC (permalink / raw)
To: Martin KaFai Lau; +Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> > > This patch introduces BPF Type Format (BTF).
> > >
> > > BTF (BPF Type Format) is the meta data format which describes
> > > the data types of BPF program/map. Hence, it basically focus
> > > on the C programming language which the modern BPF is primary
> > > using. The first use case is to provide a generic pretty print
> > > capability for a BPF map.
> > >
> > > A modified pahole that can convert dwarf to BTF is here:
> > > https://github.com/iamkafai/pahole/tree/btf
> > > (Arnaldo, there is some BTF_KIND numbering changes on
> > > Apr 18th, d61426c1571)
> >
> > Thanks for letting me know, I'm starting to look at this,
> Hi Arnaldo,
>
> Do you have a chance to take a look and pull it? The kernel
> changes will be in 4.18, so it will be handy if it is available in
> the pahole repository.
>
> [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
Got sidetracked, will get back to it later today.
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-05 21:25 ` Martin KaFai Lau
2018-06-06 12:33 ` Arnaldo Carvalho de Melo
@ 2018-06-07 13:54 ` Arnaldo Carvalho de Melo
2018-06-07 14:03 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-07 13:54 UTC (permalink / raw)
To: Martin KaFai Lau; +Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu:
> > > This patch introduces BPF Type Format (BTF).
> > >
> > > BTF (BPF Type Format) is the meta data format which describes
> > > the data types of BPF program/map. Hence, it basically focus
> > > on the C programming language which the modern BPF is primary
> > > using. The first use case is to provide a generic pretty print
> > > capability for a BPF map.
> > >
> > > A modified pahole that can convert dwarf to BTF is here:
> > > https://github.com/iamkafai/pahole/tree/btf
> > > (Arnaldo, there is some BTF_KIND numbering changes on
> > > Apr 18th, d61426c1571)
> >
> > Thanks for letting me know, I'm starting to look at this,
> Hi Arnaldo,
>
> Do you have a chance to take a look and pull it? The kernel
> changes will be in 4.18, so it will be handy if it is available in
> the pahole repository.
>
> [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
Yeah, the one I had before had:
It also raises the number of types (and functions) limit from 0x7fff to
0x7fffffff.
----
And on this last one I see that:
/* Max # of type identifier */
-#define BTF_MAX_TYPE 0x7fffffff
+#define BTF_MAX_TYPE 0x0000ffff
/* Max offset into the string section */
-#define BTF_MAX_NAME_OFFSET 0x7fffffff
+#define BTF_MAX_NAME_OFFSET 0x0000ffff
So somehow (still reading) you'll be able to get more space, if we find
necessary, to have more types and names, ok.
Continuing...
- Arnaldo
> >
> > - Arnaldo
> >
> > > Please see individual patch for details.
> > >
> > > v5:
> > > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> > > currently used. They can be added in the future.
> > > Some bpf_df_xxx() are removed together.
> > > - Add comment in patch 7 to clarify that the new bpffs_map_fops
> > > should not be extended further.
> > >
> > > v4:
> > > - Fix warning (remove unneeded semicolon)
> > > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> > > patch 1. Caught by W=1.
> > >
> > > v3:
> > > - Rebase to bpf-next
> > > - Fix sparse warning (by adding static)
> > > - Add BTF header logging: btf_verifier_log_hdr()
> > > - Fix the alignment test on btf->type_off
> > > - Add tests for the BTF header
> > > - Lower the max BTF size to 16MB. It should be enough
> > > for some time. We could raise it later if it would
> > > be needed.
> > >
> > > v2:
> > > - Use kvfree where needed in patch 1 and 2
> > > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> > > in patch 1
> > > - Fix an incorrect goto target in map_create() during
> > > the btf-error-path in patch 7
> > > - re-org some local vars to keep the rev xmas tree in btf.c
> > >
> > > Martin KaFai Lau (10):
> > > bpf: btf: Introduce BPF Type Format (BTF)
> > > bpf: btf: Validate type reference
> > > bpf: btf: Check members of struct/union
> > > bpf: btf: Add pretty print capability for data with BTF type info
> > > bpf: btf: Add BPF_BTF_LOAD command
> > > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd
> > > bpf: btf: Add pretty print support to the basic arraymap
> > > bpf: btf: Sync bpf.h and btf.h to tools/
> > > bpf: btf: Add BTF support to libbpf
> > > bpf: btf: Add BTF tests
> > >
> > > include/linux/bpf.h | 20 +-
> > > include/linux/btf.h | 48 +
> > > include/uapi/linux/bpf.h | 12 +
> > > include/uapi/linux/btf.h | 130 ++
> > > kernel/bpf/Makefile | 1 +
> > > kernel/bpf/arraymap.c | 50 +
> > > kernel/bpf/btf.c | 2064 ++++++++++++++++++++++++++
> > > kernel/bpf/inode.c | 156 +-
> > > kernel/bpf/syscall.c | 51 +-
> > > tools/include/uapi/linux/bpf.h | 12 +
> > > tools/include/uapi/linux/btf.h | 130 ++
> > > tools/lib/bpf/Build | 2 +-
> > > tools/lib/bpf/bpf.c | 92 +-
> > > tools/lib/bpf/bpf.h | 16 +
> > > tools/lib/bpf/btf.c | 374 +++++
> > > tools/lib/bpf/btf.h | 22 +
> > > tools/lib/bpf/libbpf.c | 148 +-
> > > tools/lib/bpf/libbpf.h | 3 +
> > > tools/testing/selftests/bpf/Makefile | 26 +-
> > > tools/testing/selftests/bpf/test_btf.c | 1669 +++++++++++++++++++++
> > > tools/testing/selftests/bpf/test_btf_haskv.c | 48 +
> > > tools/testing/selftests/bpf/test_btf_nokv.c | 43 +
> > > 22 files changed, 5076 insertions(+), 41 deletions(-)
> > > create mode 100644 include/linux/btf.h
> > > create mode 100644 include/uapi/linux/btf.h
> > > create mode 100644 kernel/bpf/btf.c
> > > create mode 100644 tools/include/uapi/linux/btf.h
> > > create mode 100644 tools/lib/bpf/btf.c
> > > create mode 100644 tools/lib/bpf/btf.h
> > > create mode 100644 tools/testing/selftests/bpf/test_btf.c
> > > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c
> > > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c
> > >
> > > --
> > > 2.9.5
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 13:54 ` Arnaldo Carvalho de Melo
@ 2018-06-07 14:03 ` Arnaldo Carvalho de Melo
2018-06-07 19:05 ` Martin KaFai Lau
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-07 14:03 UTC (permalink / raw)
To: Martin KaFai Lau; +Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
So, the commit log message for the pahole patch is non-existent:
https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
we should do better in describing what is done and how, I'm staring
with a message you sent to the kernel part:
--
This patch introduces BPF Type Format (BTF).
BTF (BPF Type Format) is the meta data format which describes
the data types of BPF program/map. Hence, it basically focus
on the C programming language which the modern BPF is primary
using. The first use case is to provide a generic pretty print
capability for a BPF map.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 14:03 ` Arnaldo Carvalho de Melo
@ 2018-06-07 19:05 ` Martin KaFai Lau
2018-06-07 19:30 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 38+ messages in thread
From: Martin KaFai Lau @ 2018-06-07 19:05 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team
On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
>
> So, the commit log message for the pahole patch is non-existent:
>
> https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
>
> we should do better in describing what is done and how, I'm staring
> with a message you sent to the kernel part:
>
> --
> This patch introduces BPF Type Format (BTF).
>
> BTF (BPF Type Format) is the meta data format which describes
> the data types of BPF program/map. Hence, it basically focus
> on the C programming language which the modern BPF is primary
> using. The first use case is to provide a generic pretty print
> capability for a BPF map.
> --
I will add details in the next github respin/push.
>
> Now I'm going to do the step-by-step guide on testing the feature just
> introduced, and will try to convert from dwarf to BTF and back, compare
> the pahole output for types encoded in DWARF and BTF, etc.
>
> If you have something ressembling this already, please share.
The pahole only has the encoder part. I tested with the verbose output
from the "pahole -V -J". Loading the btf to the kernel is also tested.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 19:05 ` Martin KaFai Lau
@ 2018-06-07 19:30 ` Arnaldo Carvalho de Melo
2018-06-07 20:07 ` Martin KaFai Lau
2018-06-15 11:20 ` Bo YU
0 siblings, 2 replies; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-07 19:30 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
> > So, the commit log message for the pahole patch is non-existent:
> > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
> > we should do better in describing what is done and how, I'm staring
> > with a message you sent to the kernel part:
> > --
> > This patch introduces BPF Type Format (BTF).
> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map. Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using. The first use case is to provide a generic pretty print
> > capability for a BPF map.
> I will add details in the next github respin/push.
Ok, but I can do that if there is nothing else to do in the code at this
stage :-)
> > Now I'm going to do the step-by-step guide on testing the feature just
> > introduced, and will try to convert from dwarf to BTF and back, compare
> > the pahole output for types encoded in DWARF and BTF, etc.
> > If you have something ressembling this already, please share.
> The pahole only has the encoder part. I tested with the verbose output
> from the "pahole -V -J". Loading the btf to the kernel is also tested.
Ok, so here it goes my first stab at testing, using perf's BPF
integration:
[root@jouet bpf]# cat hello.c
#include "stdio.h"
int syscall_enter(openat)(void *ctx)
{
puts("Hello, world\n");
return 0;
}
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
0.017 ( ): __bpf_stdout__:Hello, world
0.019 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3
0.053 ( ): __bpf_stdout__:Hello, world
0.055 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3
0.354 ( 0.012 ms): touch/28147 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
0.411 ( ): __bpf_stdout__:Hello, world
0.412 ( 0.198 ms): touch/28147 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
[root@jouet bpf]#
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), not stripped
[root@jouet bpf]# pahole --btf_encode hello.o
pahole: hello.o: No debugging information found
[root@jouet bpf]#
[root@jouet bpf]# readelf -s hello.o
Symbol table '.symtab' contains 5 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 7 __bpf_stdout__
2: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 5 _license
3: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 6 _version
4: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 3 syscall_enter_openat
[root@jouet bpf]#
[root@jouet bpf]# readelf -SW hello.o
There are 10 section headers, starting at offset 0x1f8:
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .strtab STRTAB 0000000000000000 000178 00007f 00 0 0 1
[ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
[ 3] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
[ 4] .relsyscalls:sys_enter_openat REL 0000000000000000 000168 000010 10 9 3 8
[ 5] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
[ 6] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
[ 7] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
[ 8] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
[ 9] .symtab SYMTAB 0000000000000000 0000f0 000078 18 1 1 8
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)
[root@jouet bpf]#
Humm, lemme try something, add -g to clang-opt:
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
clang-opt = -g
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
0.185 ( ): __bpf_stdout__:Hello, world
0.188 ( 0.009 ms): touch/19670 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3
0.219 ( ): __bpf_stdout__:Hello, world
0.220 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3
0.480 ( 0.095 ms): touch/19670 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
0.624 ( ): __bpf_stdout__:Hello, world
0.626 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
[root@jouet bpf]#
Much better:
[root@jouet bpf]# readelf -SW hello.o
There are 25 section headers, starting at offset 0xc70:
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .strtab STRTAB 0000000000000000 000b50 000119 00 0 0 1
[ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
[ 3] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
[ 4] .relsyscalls:sys_enter_openat REL 0000000000000000 000910 000010 10 24 3 8
[ 5] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
[ 6] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
[ 7] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
[ 8] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
[ 9] .debug_str PROGBITS 0000000000000000 0000ee 00010e 01 MS 0 0 1
[10] .debug_loc PROGBITS 0000000000000000 0001fc 000023 00 0 0 1
[11] .debug_abbrev PROGBITS 0000000000000000 00021f 0000e3 00 0 0 1
[12] .debug_info PROGBITS 0000000000000000 000302 00015e 00 0 0 1
[13] .rel.debug_info REL 0000000000000000 000920 0001e0 10 24 12 8
[14] .debug_ranges PROGBITS 0000000000000000 000460 000030 00 0 0 1
[15] .debug_macinfo PROGBITS 0000000000000000 000490 000001 00 0 0 1
[16] .debug_pubnames PROGBITS 0000000000000000 000491 00006e 00 0 0 1
[17] .rel.debug_pubnames REL 0000000000000000 000b00 000010 10 24 16 8
[18] .debug_pubtypes PROGBITS 0000000000000000 0004ff 00005a 00 0 0 1
[19] .rel.debug_pubtypes REL 0000000000000000 000b10 000010 10 24 18 8
[20] .debug_frame PROGBITS 0000000000000000 000560 000028 00 0 0 8
[21] .rel.debug_frame REL 0000000000000000 000b20 000020 10 24 20 8
[22] .debug_line PROGBITS 0000000000000000 000588 00006e 00 0 0 1
[23] .rel.debug_line REL 0000000000000000 000b40 000010 10 24 22 8
[24] .symtab SYMTAB 0000000000000000 0005f8 000318 18 1 29 8
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)
[root@jouet bpf]#
[root@jouet bpf]# readelf -s hello.o
Symbol table '.symtab' contains 33 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 NOTYPE LOCAL DEFAULT 9
2: 000000000000002d 0 NOTYPE LOCAL DEFAULT 9
3: 0000000000000044 0 NOTYPE LOCAL DEFAULT 9
4: 0000000000000053 0 NOTYPE LOCAL DEFAULT 9
5: 000000000000005c 0 NOTYPE LOCAL DEFAULT 9
6: 0000000000000061 0 NOTYPE LOCAL DEFAULT 9
7: 000000000000006a 0 NOTYPE LOCAL DEFAULT 9
8: 0000000000000073 0 NOTYPE LOCAL DEFAULT 9
9: 0000000000000077 0 NOTYPE LOCAL DEFAULT 9
10: 0000000000000086 0 NOTYPE LOCAL DEFAULT 9
11: 000000000000008b 0 NOTYPE LOCAL DEFAULT 9
12: 0000000000000098 0 NOTYPE LOCAL DEFAULT 9
13: 00000000000000a1 0 NOTYPE LOCAL DEFAULT 9
14: 00000000000000ac 0 NOTYPE LOCAL DEFAULT 9
15: 00000000000000b8 0 NOTYPE LOCAL DEFAULT 9
16: 00000000000000c4 0 NOTYPE LOCAL DEFAULT 9
17: 00000000000000d6 0 NOTYPE LOCAL DEFAULT 9
18: 00000000000000e8 0 NOTYPE LOCAL DEFAULT 9
19: 00000000000000fd 0 NOTYPE LOCAL DEFAULT 9
20: 0000000000000101 0 NOTYPE LOCAL DEFAULT 9
21: 0000000000000107 0 NOTYPE LOCAL DEFAULT 9
22: 0000000000000000 0 SECTION LOCAL DEFAULT 3
23: 0000000000000000 0 SECTION LOCAL DEFAULT 10
24: 0000000000000000 0 SECTION LOCAL DEFAULT 11
25: 0000000000000000 0 SECTION LOCAL DEFAULT 12
26: 0000000000000000 0 SECTION LOCAL DEFAULT 14
27: 0000000000000000 0 SECTION LOCAL DEFAULT 20
28: 0000000000000000 0 SECTION LOCAL DEFAULT 22
29: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 7 __bpf_stdout__
30: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 5 _license
31: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 6 _version
32: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 3 syscall_enter_openat
[root@jouet bpf]#
Progress:
[root@jouet bpf]# pahole --btf_encode hello.o
sh: llvm-objcopy: command not found
Failed to encode BTF
[root@jouet bpf]#
[acme@jouet perf]$ llvm-
llvm-ar llvm-cov llvm-dis llvm-lib llvm-modextract llvm-profdata llvm-split
llvm-as llvm-c-test llvm-dlltool llvm-link llvm-mt llvm-ranlib llvm-stress
llvm-bcanalyzer llvm-cvtres llvm-dsymutil llvm-lto llvm-nm llvm-readelf llvm-strings
llvm-cat llvm-cxxdump llvm-dwarfdump llvm-lto2 llvm-objdump llvm-readobj llvm-symbolizer
llvm-config llvm-cxxfilt llvm-dwp llvm-mc llvm-opt-report llvm-rtdyld llvm-tblgen
llvm-config-64 llvm-diff llvm-extract llvm-mcmarkup llvm-pdbutil llvm-size llvm-xray
[acme@jouet perf]$ llvm-
So this must be available in a newer llvm version? Which one?
[root@jouet bpf]# clang -v
clang version 5.0.1 (tags/RELEASE_501/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[root@jouet bpf]#
[acme@jouet perf]$ rpm -q clang llvm
clang-5.0.1-5.fc27.x86_64
llvm-5.0.1-6.fc27.x86_64
[acme@jouet perf]$
[root@jouet bpf]# clang -v
clang version 5.0.1 (tags/RELEASE_501/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
[root@jouet bpf]#
[acme@jouet perf]$ rpm -q clang llvm
clang-5.0.1-5.fc27.x86_64
llvm-5.0.1-6.fc27.x86_64
[acme@jouet perf]$
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 19:30 ` Arnaldo Carvalho de Melo
@ 2018-06-07 20:07 ` Martin KaFai Lau
2018-06-07 20:25 ` Arnaldo Carvalho de Melo
2018-06-12 20:31 ` Arnaldo Carvalho de Melo
2018-06-15 11:20 ` Bo YU
1 sibling, 2 replies; 38+ messages in thread
From: Martin KaFai Lau @ 2018-06-07 20:07 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
> > On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
>
> > > So, the commit log message for the pahole patch is non-existent:
>
> > > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
>
> > > we should do better in describing what is done and how, I'm staring
> > > with a message you sent to the kernel part:
> > > --
> > > This patch introduces BPF Type Format (BTF).
>
> > > BTF (BPF Type Format) is the meta data format which describes
> > > the data types of BPF program/map. Hence, it basically focus
> > > on the C programming language which the modern BPF is primary
> > > using. The first use case is to provide a generic pretty print
> > > capability for a BPF map.
>
> > I will add details in the next github respin/push.
>
> Ok, but I can do that if there is nothing else to do in the code at this
> stage :-)
>
> > > Now I'm going to do the step-by-step guide on testing the feature just
> > > introduced, and will try to convert from dwarf to BTF and back, compare
> > > the pahole output for types encoded in DWARF and BTF, etc.
>
> > > If you have something ressembling this already, please share.
>
> > The pahole only has the encoder part. I tested with the verbose output
> > from the "pahole -V -J". Loading the btf to the kernel is also tested.
>
> Ok, so here it goes my first stab at testing, using perf's BPF
> integration:
>
> [root@jouet bpf]# cat hello.c
> #include "stdio.h"
>
> int syscall_enter(openat)(void *ctx)
> {
> puts("Hello, world\n");
> return 0;
> }
> [root@jouet bpf]# cat ~/.perfconfig
> [llvm]
> dump-obj = true
> [root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
> LLVM: dumping hello.o
> 0.017 ( ): __bpf_stdout__:Hello, world
> 0.019 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3
> 0.053 ( ): __bpf_stdout__:Hello, world
> 0.055 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3
> 0.354 ( 0.012 ms): touch/28147 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
> 0.411 ( ): __bpf_stdout__:Hello, world
> 0.412 ( 0.198 ms): touch/28147 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
> [root@jouet bpf]#
> [root@jouet bpf]# file hello.o
> hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), not stripped
> [root@jouet bpf]# pahole --btf_encode hello.o
> pahole: hello.o: No debugging information found
> [root@jouet bpf]#
>
> [root@jouet bpf]# readelf -s hello.o
>
> Symbol table '.symtab' contains 5 entries:
> Num: Value Size Type Bind Vis Ndx Name
> 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
> 1: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 7 __bpf_stdout__
> 2: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 5 _license
> 3: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 6 _version
> 4: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 3 syscall_enter_openat
> [root@jouet bpf]#
> [root@jouet bpf]# readelf -SW hello.o
> There are 10 section headers, starting at offset 0x1f8:
>
> Section Headers:
> [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> [ 0] NULL 0000000000000000 000000 000000 00 0 0 0
> [ 1] .strtab STRTAB 0000000000000000 000178 00007f 00 0 0 1
> [ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
> [ 3] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
> [ 4] .relsyscalls:sys_enter_openat REL 0000000000000000 000168 000010 10 9 3 8
> [ 5] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
> [ 6] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
> [ 7] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
> [ 8] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
> [ 9] .symtab SYMTAB 0000000000000000 0000f0 000078 18 1 1 8
> Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> p (processor specific)
> [root@jouet bpf]#
>
> Humm, lemme try something, add -g to clang-opt:
>
> [root@jouet bpf]# cat ~/.perfconfig
> [llvm]
> dump-obj = true
> clang-opt = -g
> [root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
> LLVM: dumping hello.o
> 0.185 ( ): __bpf_stdout__:Hello, world
> 0.188 ( 0.009 ms): touch/19670 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3
> 0.219 ( ): __bpf_stdout__:Hello, world
> 0.220 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3
> 0.480 ( 0.095 ms): touch/19670 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
> 0.624 ( ): __bpf_stdout__:Hello, world
> 0.626 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
> [root@jouet bpf]# file hello.o
> hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
> [root@jouet bpf]#
>
> Much better:
>
> [root@jouet bpf]# readelf -SW hello.o
> There are 25 section headers, starting at offset 0xc70:
>
> Section Headers:
> [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> [ 0] NULL 0000000000000000 000000 000000 00 0 0 0
> [ 1] .strtab STRTAB 0000000000000000 000b50 000119 00 0 0 1
> [ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
> [ 3] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
> [ 4] .relsyscalls:sys_enter_openat REL 0000000000000000 000910 000010 10 24 3 8
> [ 5] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
> [ 6] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
> [ 7] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
> [ 8] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
> [ 9] .debug_str PROGBITS 0000000000000000 0000ee 00010e 01 MS 0 0 1
> [10] .debug_loc PROGBITS 0000000000000000 0001fc 000023 00 0 0 1
> [11] .debug_abbrev PROGBITS 0000000000000000 00021f 0000e3 00 0 0 1
> [12] .debug_info PROGBITS 0000000000000000 000302 00015e 00 0 0 1
> [13] .rel.debug_info REL 0000000000000000 000920 0001e0 10 24 12 8
> [14] .debug_ranges PROGBITS 0000000000000000 000460 000030 00 0 0 1
> [15] .debug_macinfo PROGBITS 0000000000000000 000490 000001 00 0 0 1
> [16] .debug_pubnames PROGBITS 0000000000000000 000491 00006e 00 0 0 1
> [17] .rel.debug_pubnames REL 0000000000000000 000b00 000010 10 24 16 8
> [18] .debug_pubtypes PROGBITS 0000000000000000 0004ff 00005a 00 0 0 1
> [19] .rel.debug_pubtypes REL 0000000000000000 000b10 000010 10 24 18 8
> [20] .debug_frame PROGBITS 0000000000000000 000560 000028 00 0 0 8
> [21] .rel.debug_frame REL 0000000000000000 000b20 000020 10 24 20 8
> [22] .debug_line PROGBITS 0000000000000000 000588 00006e 00 0 0 1
> [23] .rel.debug_line REL 0000000000000000 000b40 000010 10 24 22 8
> [24] .symtab SYMTAB 0000000000000000 0005f8 000318 18 1 29 8
> Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> p (processor specific)
> [root@jouet bpf]#
>
> [root@jouet bpf]# readelf -s hello.o
>
> Symbol table '.symtab' contains 33 entries:
> Num: Value Size Type Bind Vis Ndx Name
> 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
> 1: 0000000000000000 0 NOTYPE LOCAL DEFAULT 9
> 2: 000000000000002d 0 NOTYPE LOCAL DEFAULT 9
> 3: 0000000000000044 0 NOTYPE LOCAL DEFAULT 9
> 4: 0000000000000053 0 NOTYPE LOCAL DEFAULT 9
> 5: 000000000000005c 0 NOTYPE LOCAL DEFAULT 9
> 6: 0000000000000061 0 NOTYPE LOCAL DEFAULT 9
> 7: 000000000000006a 0 NOTYPE LOCAL DEFAULT 9
> 8: 0000000000000073 0 NOTYPE LOCAL DEFAULT 9
> 9: 0000000000000077 0 NOTYPE LOCAL DEFAULT 9
> 10: 0000000000000086 0 NOTYPE LOCAL DEFAULT 9
> 11: 000000000000008b 0 NOTYPE LOCAL DEFAULT 9
> 12: 0000000000000098 0 NOTYPE LOCAL DEFAULT 9
> 13: 00000000000000a1 0 NOTYPE LOCAL DEFAULT 9
> 14: 00000000000000ac 0 NOTYPE LOCAL DEFAULT 9
> 15: 00000000000000b8 0 NOTYPE LOCAL DEFAULT 9
> 16: 00000000000000c4 0 NOTYPE LOCAL DEFAULT 9
> 17: 00000000000000d6 0 NOTYPE LOCAL DEFAULT 9
> 18: 00000000000000e8 0 NOTYPE LOCAL DEFAULT 9
> 19: 00000000000000fd 0 NOTYPE LOCAL DEFAULT 9
> 20: 0000000000000101 0 NOTYPE LOCAL DEFAULT 9
> 21: 0000000000000107 0 NOTYPE LOCAL DEFAULT 9
> 22: 0000000000000000 0 SECTION LOCAL DEFAULT 3
> 23: 0000000000000000 0 SECTION LOCAL DEFAULT 10
> 24: 0000000000000000 0 SECTION LOCAL DEFAULT 11
> 25: 0000000000000000 0 SECTION LOCAL DEFAULT 12
> 26: 0000000000000000 0 SECTION LOCAL DEFAULT 14
> 27: 0000000000000000 0 SECTION LOCAL DEFAULT 20
> 28: 0000000000000000 0 SECTION LOCAL DEFAULT 22
> 29: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 7 __bpf_stdout__
> 30: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 5 _license
> 31: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 6 _version
> 32: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 3 syscall_enter_openat
> [root@jouet bpf]#
>
> Progress:
>
> [root@jouet bpf]# pahole --btf_encode hello.o
> sh: llvm-objcopy: command not found
> Failed to encode BTF
> [root@jouet bpf]#
>
> [acme@jouet perf]$ llvm-
> llvm-ar llvm-cov llvm-dis llvm-lib llvm-modextract llvm-profdata llvm-split
> llvm-as llvm-c-test llvm-dlltool llvm-link llvm-mt llvm-ranlib llvm-stress
> llvm-bcanalyzer llvm-cvtres llvm-dsymutil llvm-lto llvm-nm llvm-readelf llvm-strings
> llvm-cat llvm-cxxdump llvm-dwarfdump llvm-lto2 llvm-objdump llvm-readobj llvm-symbolizer
> llvm-config llvm-cxxfilt llvm-dwp llvm-mc llvm-opt-report llvm-rtdyld llvm-tblgen
> llvm-config-64 llvm-diff llvm-extract llvm-mcmarkup llvm-pdbutil llvm-size llvm-xray
> [acme@jouet perf]$ llvm-
>
> So this must be available in a newer llvm version? Which one?
I should have put in the details in my last email or
in the commit message, my bad.
1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
LLC_FLAGS needed to compile the bpf prog. It requires a new
"-mattr=dwarf" llc option which was added to the future
llvm 7.0.
Hence, I have been using the llvm's master in github which
also has the llvm-objcopy.
2. The kernel's btf part only focus on the BPF map.
Hence, the testing bpf program should have the map's key
and map's value. e.g. tools/testing/selftests/bpf/test_btf_haskv.c
>
> [root@jouet bpf]# clang -v
> clang version 5.0.1 (tags/RELEASE_501/final)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
> Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
> Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
> Candidate multilib: .;@m64
> Candidate multilib: 32;@m32
> Selected multilib: .;@m64
> [root@jouet bpf]#
>
> [acme@jouet perf]$ rpm -q clang llvm
> clang-5.0.1-5.fc27.x86_64
> llvm-5.0.1-6.fc27.x86_64
> [acme@jouet perf]$
> [root@jouet bpf]# clang -v
> clang version 5.0.1 (tags/RELEASE_501/final)
> Target: x86_64-unknown-linux-gnu
> Thread model: posix
> InstalledDir: /usr/bin
> Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
> Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
> Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
> Candidate multilib: .;@m64
> Candidate multilib: 32;@m32
> Selected multilib: .;@m64
> [root@jouet bpf]#
>
> [acme@jouet perf]$ rpm -q clang llvm
> clang-5.0.1-5.fc27.x86_64
> llvm-5.0.1-6.fc27.x86_64
> [acme@jouet perf]$
>
> - Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 20:07 ` Martin KaFai Lau
@ 2018-06-07 20:25 ` Arnaldo Carvalho de Melo
2018-06-12 20:31 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-07 20:25 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > So this must be available in a newer llvm version? Which one?
> I should have put in the details in my last email or
> in the commit message, my bad.
> 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> LLC_FLAGS needed to compile the bpf prog. It requires a new
> "-mattr=dwarf" llc option which was added to the future
> llvm 7.0.
> Hence, I have been using the llvm's master in github which
> also has the llvm-objcopy.
> 2. The kernel's btf part only focus on the BPF map.
> Hence, the testing bpf program should have the map's key
> and map's value. e.g. tools/testing/selftests/bpf/test_btf_haskv.c
Thanks for the version required to test this, but where is this
test_btf_haskv.c file? Which tree? net-next?
Ok, just pulled torvalds/master and there it is. Gotcha.
struct bpf_map_def SEC("maps") __bpf_stdout__ = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(u32),
.max_entries = __NR_CPUS__,
};
This map is in the above hello.c example, but I guess its way too simple
:-)
Ok, I'll test this at home in another machine where I have the llvm's
git repo.
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 20:07 ` Martin KaFai Lau
2018-06-07 20:25 ` Arnaldo Carvalho de Melo
@ 2018-06-12 20:31 ` Arnaldo Carvalho de Melo
2018-06-12 20:41 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-12 20:31 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > So this must be available in a newer llvm version? Which one?
> I should have put in the details in my last email or
> in the commit message, my bad.
> 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> LLC_FLAGS needed to compile the bpf prog. It requires a new
> "-mattr=dwarf" llc option which was added to the future
> llvm 7.0.
> Hence, I have been using the llvm's master in github which
> also has the llvm-objcopy.
> 2. The kernel's btf part only focus on the BPF map.
> Hence, the testing bpf program should have the map's key
> and map's value. e.g. tools/testing/selftests/bpf/test_btf_haskv.c
So, with llvm and clang HEAD I get:
[root@jouet bpf]# pahole -J hello.o
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
[root@jouet bpf]# llvm-readelf -s hello.o
There are 26 section headers, starting at offset 0xe30:
Section Headers:
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 0] NULL 0000000000000000 000000 000000 00 0 0 0
[ 1] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
[ 2] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
[ 3] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
[ 4] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
[ 5] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
[ 6] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
[ 7] .debug_str PROGBITS 0000000000000000 0000ee 00010e 01 MS 0 0 1
[ 8] .debug_loc PROGBITS 0000000000000000 0001fc 000023 00 0 0 1
[ 9] .debug_abbrev PROGBITS 0000000000000000 00021f 0000e3 00 0 0 1
[10] .debug_info PROGBITS 0000000000000000 000302 00015e 00 0 0 1
[11] .debug_ranges PROGBITS 0000000000000000 000460 000030 00 0 0 1
[12] .debug_macinfo PROGBITS 0000000000000000 000490 000001 00 0 0 1
[13] .debug_pubnames PROGBITS 0000000000000000 000491 00006e 00 0 0 1
[14] .debug_pubtypes PROGBITS 0000000000000000 0004ff 00005a 00 0 0 1
[15] .debug_frame PROGBITS 0000000000000000 000560 000028 00 0 0 8
[16] .debug_line PROGBITS 0000000000000000 000588 00006e 00 0 0 1
[17] .symtab SYMTAB 0000000000000000 0005f8 000318 18 24 29 8
[18] .relsyscalls:sys_enter_openat REL 0000000000000000 000910 000010 10 17 2 8
[19] .rel.debug_info REL 0000000000000000 000920 0001e0 10 17 10 8
[20] .rel.debug_pubnames REL 0000000000000000 000b00 000010 10 17 13 8
[21] .rel.debug_pubtypes REL 0000000000000000 000b10 000010 10 17 14 8
[22] .rel.debug_frame REL 0000000000000000 000b20 000020 10 17 15 8
[23] .rel.debug_line REL 0000000000000000 000b40 000010 10 17 16 8
[24] .strtab STRTAB 0000000000000000 000b50 00018e 00 0 0 1
[25] .BTF PROGBITS 0000000000000000 000cde 00014e 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
[root@jouet bpf]#
[root@jouet bpf]# pahole hello.o
struct clang version 5.0.1 (tags/RELEASE_501/final) {
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 0 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 4 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 8 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Ok, I guess I saw this case in the llvm/clang git logs, so this one was
generated with the older clang, will regenerate and add that "-mattr=dwarf"
part.
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-12 20:31 ` Arnaldo Carvalho de Melo
@ 2018-06-12 20:41 ` Arnaldo Carvalho de Melo
2018-06-13 23:26 ` Martin KaFai Lau
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-12 20:41 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > > So this must be available in a newer llvm version? Which one?
>
> > I should have put in the details in my last email or
> > in the commit message, my bad.
>
> > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > LLC_FLAGS needed to compile the bpf prog. It requires a new
> > "-mattr=dwarf" llc option which was added to the future
> > llvm 7.0.
> [root@jouet bpf]# pahole hello.o
> struct clang version 5.0.1 (tags/RELEASE_501/final) {
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 0 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 4 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 8 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 12 4 */
>
> /* size: 16, cachelines: 1, members: 4 */
> /* last cacheline: 16 bytes */
> };
> [root@jouet bpf]#
>
> Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> generated with the older clang, will regenerate and add that "-mattr=dwarf"
> part.
[root@jouet bpf]# pahole hello.o
struct clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34) {
clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78 clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 0 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78 clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 4 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78 clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 8 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78 clang version 7.0.0 (http://llvm.org/git/clang.git 8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Ideas?
[root@jouet bpf]# trace -e open*,hello.c
clang-6.0: error: unknown argument: '-mattr=dwarf'
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [<options>] [<command>]
or: perf trace [<options>] -- <command> [<options>]
or: perf trace record [<options>] [<command>]
or: perf trace record [<options>] -- <command> [<options>]
-e, --event <event> event/syscall selector. use 'perf list' to list available events
[root@jouet bpf]#
[root@jouet bpf]# trace -v -e open*,hello.c
bpf: builtin compilation failed: -95, try external compiler
Kernel build dir is set to /lib/modules/4.17.0-rc5/build
set env: KBUILD_DIR=/lib/modules/4.17.0-rc5/build
unset env: KBUILD_OPTS
include option is set to -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x41100
set env: CLANG_EXEC=/usr/local/bin/clang
set env: CLANG_OPTIONS=-g -mattr=dwarf
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: PERF_BPF_INC_OPTIONS=-I/home/acme/lib/include/perf/bpf
set env: WORKING_DIR=/lib/modules/4.17.0-rc5/build
set env: CLANG_SOURCE=/home/acme/bpf/hello.c
llvm compiling command template: $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS $PERF_BPF_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o -
llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
clang-6.0: error: unknown argument: '-mattr=dwarf'
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when compiling BPF scriptlet
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-12 20:41 ` Arnaldo Carvalho de Melo
@ 2018-06-13 23:26 ` Martin KaFai Lau
2018-06-14 15:03 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 38+ messages in thread
From: Martin KaFai Lau @ 2018-06-13 23:26 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
On Tue, Jun 12, 2018 at 05:41:26PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > So this must be available in a newer llvm version? Which one?
> >
> > > I should have put in the details in my last email or
> > > in the commit message, my bad.
> >
> > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > "-mattr=dwarf" llc option which was added to the future
> > > llvm 7.0.
>
> > [root@jouet bpf]# pahole hello.o
> > struct clang version 5.0.1 (tags/RELEASE_501/final) {
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 0 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 4 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 8 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 12 4 */
> >
> > /* size: 16, cachelines: 1, members: 4 */
> > /* last cacheline: 16 bytes */
> > };
> > [root@jouet bpf]#
> >
> > Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> > generated with the older clang, will regenerate and add that "-mattr=dwarf"
> > part.
>
> [root@jouet bpf]# pahole hello.o
> struct clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34) {
> clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78 clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 0 4 */
> clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78 clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 4 4 */
> clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78 clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 8 4 */
> clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78 clang version 7.0.0 (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg&e= 8c873daccce7ee5339b9fd82c81fe02b73543b65) (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4&m=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI&s=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg&e= 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 12 4 */
>
> /* size: 16, cachelines: 1, members: 4 */
> /* last cacheline: 16 bytes */
> };
That means the "-mattr=dwarf" is not effective.
Can you share your clang and llc command to create hello.o?
> [root@jouet bpf]#
>
> Ideas?
>
> [root@jouet bpf]# trace -e open*,hello.c
> clang-6.0: error: unknown argument: '-mattr=dwarf'
> ERROR: unable to compile hello.c
> Hint: Check error message shown above.
> Hint: You can also pre-compile it into .o using:
> clang -target bpf -O2 -c hello.c
> with proper -I and -D options.
> event syntax error: 'hello.c'
> \___ Failed to load hello.c from source: Error when compiling BPF scriptlet
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
> Usage: perf trace [<options>] [<command>]
> or: perf trace [<options>] -- <command> [<options>]
> or: perf trace record [<options>] [<command>]
> or: perf trace record [<options>] -- <command> [<options>]
>
> -e, --event <event> event/syscall selector. use 'perf list' to list available events
> [root@jouet bpf]#
>
> [root@jouet bpf]# trace -v -e open*,hello.c
> bpf: builtin compilation failed: -95, try external compiler
> Kernel build dir is set to /lib/modules/4.17.0-rc5/build
> set env: KBUILD_DIR=/lib/modules/4.17.0-rc5/build
> unset env: KBUILD_OPTS
> include option is set to -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
> set env: NR_CPUS=4
> set env: LINUX_VERSION_CODE=0x41100
> set env: CLANG_EXEC=/usr/local/bin/clang
> set env: CLANG_OPTIONS=-g -mattr=dwarf
> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
> set env: PERF_BPF_INC_OPTIONS=-I/home/acme/lib/include/perf/bpf
> set env: WORKING_DIR=/lib/modules/4.17.0-rc5/build
> set env: CLANG_SOURCE=/home/acme/bpf/hello.c
> llvm compiling command template: $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS $PERF_BPF_INC_OPTIONS -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c "$CLANG_SOURCE" -target bpf -O2 -o -
> llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
> clang-6.0: error: unknown argument: '-mattr=dwarf'
> ERROR: unable to compile hello.c
> Hint: Check error message shown above.
> Hint: You can also pre-compile it into .o using:
> clang -target bpf -O2 -c hello.c
> with proper -I and -D options.
> event syntax error: 'hello.c'
> \___ Failed to load hello.c from source: Error when compiling BPF scriptlet
>
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-13 23:26 ` Martin KaFai Lau
@ 2018-06-14 15:03 ` Arnaldo Carvalho de Melo
2018-06-14 16:22 ` Martin KaFai Lau
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-14 15:03 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Wed, Jun 13, 2018 at 04:26:38PM -0700, Martin KaFai Lau escreveu:
> On Tue, Jun 12, 2018 at 05:41:26PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > > > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > > So this must be available in a newer llvm version? Which one?
> > > > I should have put in the details in my last email or
> > > > in the commit message, my bad.
> > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > > LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > > "-mattr=dwarf" llc option which was added to the future
> > > > llvm 7.0.
> > > [root@jouet bpf]# pahole hello.o
> > > struct clang version 5.0.1 (tags/RELEASE_501/final) {
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 0 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 4 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 8 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1 (tags/RELEASE_501/final); /* 12 4 */
> > > /* size: 16, cachelines: 1, members: 4 */
> > > /* last cacheline: 16 bytes */
> > > };
> > > [root@jouet bpf]#
> > >
> > > Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> > > generated with the older clang, will regenerate and add that "-mattr=dwarf"
> > > part.
> > [root@jouet bpf]# pahole hello.o
> > struct clang version 7.0.0 <SNIP>
<SNIP>
> > /* size: 16, cachelines: 1, members: 4 */
> > /* last cacheline: 16 bytes */
> > };
> That means the "-mattr=dwarf" is not effective.
> Can you share your clang and llc command to create hello.o?
I tried it, but it didn't work, see:
[root@jouet bpf]# cat hello.c
#include "stdio.h"
int syscall_enter(openat)(void *ctx)
{
puts("Hello, world\n");
return 0;
}
[root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
clang-6.0: error: unknown argument: '-mattr=dwarf'
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [<options>] [<command>]
or: perf trace [<options>] -- <command> [<options>]
or: perf trace record [<options>] [<command>]
or: perf trace record [<options>] -- <command> [<options>]
-e, --event <event> event/syscall selector. use 'perf list' to list available events
[root@jouet bpf]#
The full command line with that is:
[root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
set env: CLANG_OPTIONS=-g -mattr=dwarf
llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
clang-6.0: error: unknown argument: '-mattr=dwarf'
[root@jouet bpf]#
This is with these llvm and clang trees:
[root@jouet llvm]# git log --oneline -5
98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument comdat globals on COFF targets
6ad988b5998 [DAGCombiner] clean up comments; NFC
a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
1503b9f6fe8 [x86] add tests for node-level FMF; NFC
4a49826736f [x86] regenerate test checks; NFC
[root@jouet llvm]#
[root@jouet llvm]# cd tools/clang/
[root@jouet clang]# git log --oneline -5
8c873daccc (HEAD -> master, origin/master, origin/HEAD) [X86] Add builtins for vpermq/vpermpd instructions to enable target feature checking.
a344be6ba4 [X86] Change immediate type for some builtins from char to int.
dcdd53793e [CUDA] Fix emission of constant strings in sections
a90c85acaf [X86] Add builtins for shufps and shufpd to enable target feature and immediate range checking.
ff71c0eccc [X86] Add builtins for pshufd, pshuflw, and pshufhw to enable target feature and immediate range checking.
[root@jouet clang]#
[root@jouet clang]# git log | grep mattr=dwarf
[root@jouet clang]# cd -
/home/acme/git.tmp/git/llvm
[root@jouet llvm]# git log | grep mattr=dwarf
bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
This patch introduces a new flag -mattr=dwarfris
[root@jouet llvm]#
Humm, so its -mattr=dwarfris and not -attr=dwarf?
Didn't help :-\
commit 0e0047f8c9ada2f0fe0c5f01579a80e2455b8df5
Author: Yonghong Song <yhs@fb.com>
Date: Thu Mar 1 23:04:59 2018 +0000
bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
Commit e4507fb8c94b ("bpf: disable DwarfUsesRelocationsAcrossSections")
disables MCAsmInfo DwarfUsesRelocationsAcrossSections unconditionally
so that dwarf will not use cross section (between dwarf and symbol table)
relocations. This new debug format enables pahole to dump structures
correctly as libdwarves.so does not have BPF backend support yet.
This new debug format, however, breaks bcc (https://github.com/iovisor/bcc)
source debug output as llvm in-memory Dwarf support has some issues to
handle it. More specifically, with DwarfUsesRelocationsAcrossSections
disabled, JIT compiler does not generate .debug_abbrev and Dwarf
DIE (debug info entry) processing is not happy about this.
This patch introduces a new flag -mattr=dwarfris
(dwarf relocation in section) to disable DwarfUsesRelocationsAcrossSections.
DwarfUsesRelocationsAcrossSections is true by default.
Signed-off-by: Yonghong Song <yhs@fb.com>
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@326505 91177308-0d34-0410-b5e6-96231b3b80d8
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 15:03 ` Arnaldo Carvalho de Melo
@ 2018-06-14 16:22 ` Martin KaFai Lau
2018-06-14 17:18 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 38+ messages in thread
From: Martin KaFai Lau @ 2018-06-14 16:22 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
On Thu, Jun 14, 2018 at 12:03:34PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > > > LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > > > "-mattr=dwarf" llc option which was added to the future
> > > > > llvm 7.0.
[ ... ]
> I tried it, but it didn't work, see:
>
> [root@jouet bpf]# cat hello.c
> #include "stdio.h"
>
> int syscall_enter(openat)(void *ctx)
> {
> puts("Hello, world\n");
> return 0;
> }
> [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
> clang-6.0: error: unknown argument: '-mattr=dwarf'
"-mattr=dwarf" is currently a llc only option.
tools/testing/selftests/bpf/Makefile has example on how to pipe clang to llc.
e.g.:
clang -g -O2 -target bpf -emit-llvm -c hello.c -o - | llc -march=bpf -mcpu=generic -mattr=dwarfris -filetype=obj -o hello.o
> ERROR: unable to compile hello.c
> Hint: Check error message shown above.
> Hint: You can also pre-compile it into .o using:
> clang -target bpf -O2 -c hello.c
> with proper -I and -D options.
> event syntax error: 'hello.c'
> \___ Failed to load hello.c from source: Error when compiling BPF scriptlet
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
> Usage: perf trace [<options>] [<command>]
> or: perf trace [<options>] -- <command> [<options>]
> or: perf trace record [<options>] [<command>]
> or: perf trace record [<options>] -- <command> [<options>]
>
> -e, --event <event> event/syscall selector. use 'perf list' to list available events
> [root@jouet bpf]#
>
> The full command line with that is:
>
> [root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
> set env: CLANG_OPTIONS=-g -mattr=dwarf
> llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
> clang-6.0: error: unknown argument: '-mattr=dwarf'
> [root@jouet bpf]#
>
> This is with these llvm and clang trees:
>
> [root@jouet llvm]# git log --oneline -5
> 98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument comdat globals on COFF targets
> 6ad988b5998 [DAGCombiner] clean up comments; NFC
> a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
> 1503b9f6fe8 [x86] add tests for node-level FMF; NFC
> 4a49826736f [x86] regenerate test checks; NFC
> [root@jouet llvm]#
>
> [root@jouet llvm]# cd tools/clang/
> [root@jouet clang]# git log --oneline -5
> 8c873daccc (HEAD -> master, origin/master, origin/HEAD) [X86] Add builtins for vpermq/vpermpd instructions to enable target feature checking.
> a344be6ba4 [X86] Change immediate type for some builtins from char to int.
> dcdd53793e [CUDA] Fix emission of constant strings in sections
> a90c85acaf [X86] Add builtins for shufps and shufpd to enable target feature and immediate range checking.
> ff71c0eccc [X86] Add builtins for pshufd, pshuflw, and pshufhw to enable target feature and immediate range checking.
> [root@jouet clang]#
>
> [root@jouet clang]# git log | grep mattr=dwarf
> [root@jouet clang]# cd -
> /home/acme/git.tmp/git/llvm
> [root@jouet llvm]# git log | grep mattr=dwarf
> bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
> This patch introduces a new flag -mattr=dwarfris
> [root@jouet llvm]#
>
> Humm, so its -mattr=dwarfris and not -attr=dwarf?
>
> Didn't help :-\
>
> commit 0e0047f8c9ada2f0fe0c5f01579a80e2455b8df5
> Author: Yonghong Song <yhs@fb.com>
> Date: Thu Mar 1 23:04:59 2018 +0000
>
> bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
>
> Commit e4507fb8c94b ("bpf: disable DwarfUsesRelocationsAcrossSections")
> disables MCAsmInfo DwarfUsesRelocationsAcrossSections unconditionally
> so that dwarf will not use cross section (between dwarf and symbol table)
> relocations. This new debug format enables pahole to dump structures
> correctly as libdwarves.so does not have BPF backend support yet.
>
> This new debug format, however, breaks bcc (https://github.com/iovisor/bcc)
> source debug output as llvm in-memory Dwarf support has some issues to
> handle it. More specifically, with DwarfUsesRelocationsAcrossSections
> disabled, JIT compiler does not generate .debug_abbrev and Dwarf
> DIE (debug info entry) processing is not happy about this.
>
> This patch introduces a new flag -mattr=dwarfris
> (dwarf relocation in section) to disable DwarfUsesRelocationsAcrossSections.
> DwarfUsesRelocationsAcrossSections is true by default.
>
> Signed-off-by: Yonghong Song <yhs@fb.com>
>
> git-svn-id: https://urldefense.proofpoint.com/v2/url?u=https-3A__llvm.org_svn_llvm-2Dproject_llvm_trunk-40326505&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=VQnoQ7LvghIj0gVEaiQSUw&m=LO28-RE-2ZJTXto_gff4BgnxXkbUq8d2CEz1jD_wDl4&s=VCR3pGVfY54-OsZ3BqRsOr3FF5JVyltwbnbzu30_4EY&e= 91177308-0d34-0410-b5e6-96231b3b80d8
>
>
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 16:22 ` Martin KaFai Lau
@ 2018-06-14 17:18 ` Arnaldo Carvalho de Melo
2018-06-14 17:21 ` Alexei Starovoitov
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-14 17:18 UTC (permalink / raw)
To: Martin KaFai Lau
Cc: netdev, Alexei Starovoitov, Daniel Borkmann, kernel-team,
Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 14, 2018 at 09:22:27AM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 14, 2018 at 12:03:34PM -0300, Arnaldo Carvalho de Melo wrote:
>
> > > > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > > > > LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > > > > "-mattr=dwarf" llc option which was added to the future
> > > > > > llvm 7.0.
>
> [ ... ]
>
> > I tried it, but it didn't work, see:
> >
> > [root@jouet bpf]# cat hello.c
> > #include "stdio.h"
> >
> > int syscall_enter(openat)(void *ctx)
> > {
> > puts("Hello, world\n");
> > return 0;
> > }
> > [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
> > clang-6.0: error: unknown argument: '-mattr=dwarf'
> "-mattr=dwarf" is currently a llc only option.
>
> tools/testing/selftests/bpf/Makefile has example on how to pipe clang to llc.
> e.g.:
> clang -g -O2 -target bpf -emit-llvm -c hello.c -o - | llc -march=bpf -mcpu=generic -mattr=dwarfris -filetype=obj -o hello.o
Ok, so I'll probably add a llvm.opts .perfconfig entry that, if present
will tell tools/perf/util/llvm-utils.c that piping the output of clang
to llvm, so that we can use llvm specific options, needs to be done.
Probably, for the time being I'll check for -g in llvm.clang-opt and if
it is there, set up the piping...
Just out of curiosity, is there any plan to have this as a clang option?
Just to finish this thing here, lemme try a slightly modified version of
your command line:
[root@jouet bpf]# clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -emit-llvm -O2 -o - | llc -march=bpf -mcpu=generic -mattr=dwarfris -filetype=obj -o hello2.o
[root@jouet bpf]#
[root@jouet bpf]# file hello2.o
hello2.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
[root@jouet bpf]# pahole hello2.o
struct bpf_map_def {
unsigned int type; /* 0 4 */
unsigned int key_size; /* 4 4 */
unsigned int value_size; /* 8 4 */
unsigned int max_entries; /* 12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Finally works, thanks.
Thanks,
- Arnaldo
> > ERROR: unable to compile hello.c
> > Hint: Check error message shown above.
> > Hint: You can also pre-compile it into .o using:
> > clang -target bpf -O2 -c hello.c
> > with proper -I and -D options.
> > event syntax error: 'hello.c'
> > \___ Failed to load hello.c from source: Error when compiling BPF scriptlet
> >
> > (add -v to see detail)
> > Run 'perf list' for a list of valid events
> >
> > Usage: perf trace [<options>] [<command>]
> > or: perf trace [<options>] -- <command> [<options>]
> > or: perf trace record [<options>] [<command>]
> > or: perf trace record [<options>] -- <command> [<options>]
> >
> > -e, --event <event> event/syscall selector. use 'perf list' to list available events
> > [root@jouet bpf]#
> >
> > The full command line with that is:
> >
> > [root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
> > set env: CLANG_OPTIONS=-g -mattr=dwarf
> > llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
> > clang-6.0: error: unknown argument: '-mattr=dwarf'
> > [root@jouet bpf]#
> >
> > This is with these llvm and clang trees:
> >
> > [root@jouet llvm]# git log --oneline -5
> > 98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument comdat globals on COFF targets
> > 6ad988b5998 [DAGCombiner] clean up comments; NFC
> > a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
> > 1503b9f6fe8 [x86] add tests for node-level FMF; NFC
> > 4a49826736f [x86] regenerate test checks; NFC
> > [root@jouet llvm]#
> >
> > [root@jouet llvm]# cd tools/clang/
> > [root@jouet clang]# git log --oneline -5
> > 8c873daccc (HEAD -> master, origin/master, origin/HEAD) [X86] Add builtins for vpermq/vpermpd instructions to enable target feature checking.
> > a344be6ba4 [X86] Change immediate type for some builtins from char to int.
> > dcdd53793e [CUDA] Fix emission of constant strings in sections
> > a90c85acaf [X86] Add builtins for shufps and shufpd to enable target feature and immediate range checking.
> > ff71c0eccc [X86] Add builtins for pshufd, pshuflw, and pshufhw to enable target feature and immediate range checking.
> > [root@jouet clang]#
> >
> > [root@jouet clang]# git log | grep mattr=dwarf
> > [root@jouet clang]# cd -
> > /home/acme/git.tmp/git/llvm
> > [root@jouet llvm]# git log | grep mattr=dwarf
> > bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
> > This patch introduces a new flag -mattr=dwarfris
> > [root@jouet llvm]#
> >
> > Humm, so its -mattr=dwarfris and not -attr=dwarf?
> >
> > Didn't help :-\
> >
> > commit 0e0047f8c9ada2f0fe0c5f01579a80e2455b8df5
> > Author: Yonghong Song <yhs@fb.com>
> > Date: Thu Mar 1 23:04:59 2018 +0000
> >
> > bpf: introduce -mattr=dwarfris to disable DwarfUsesRelocationsAcrossSections
> >
> > Commit e4507fb8c94b ("bpf: disable DwarfUsesRelocationsAcrossSections")
> > disables MCAsmInfo DwarfUsesRelocationsAcrossSections unconditionally
> > so that dwarf will not use cross section (between dwarf and symbol table)
> > relocations. This new debug format enables pahole to dump structures
> > correctly as libdwarves.so does not have BPF backend support yet.
> >
> > This new debug format, however, breaks bcc (https://github.com/iovisor/bcc)
> > source debug output as llvm in-memory Dwarf support has some issues to
> > handle it. More specifically, with DwarfUsesRelocationsAcrossSections
> > disabled, JIT compiler does not generate .debug_abbrev and Dwarf
> > DIE (debug info entry) processing is not happy about this.
> >
> > This patch introduces a new flag -mattr=dwarfris
> > (dwarf relocation in section) to disable DwarfUsesRelocationsAcrossSections.
> > DwarfUsesRelocationsAcrossSections is true by default.
> >
> > Signed-off-by: Yonghong Song <yhs@fb.com>
> >
> > git-svn-id: https://urldefense.proofpoint.com/v2/url?u=https-3A__llvm.org_svn_llvm-2Dproject_llvm_trunk-40326505&d=DwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=VQnoQ7LvghIj0gVEaiQSUw&m=LO28-RE-2ZJTXto_gff4BgnxXkbUq8d2CEz1jD_wDl4&s=VCR3pGVfY54-OsZ3BqRsOr3FF5JVyltwbnbzu30_4EY&e= 91177308-0d34-0410-b5e6-96231b3b80d8
> >
> >
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 17:18 ` Arnaldo Carvalho de Melo
@ 2018-06-14 17:21 ` Alexei Starovoitov
2018-06-14 17:41 ` Arnaldo Carvalho de Melo
2018-06-14 17:47 ` Arnaldo Carvalho de Melo
0 siblings, 2 replies; 38+ messages in thread
From: Alexei Starovoitov @ 2018-06-14 17:21 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Martin KaFai Lau
Cc: netdev, Daniel Borkmann, kernel-team, Wang Nan, Jiri Olsa,
Namhyung Kim, Ingo Molnar
On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
>
> Just out of curiosity, is there any plan to have this as a clang option?
I think
clang ... -mllvm -mattr=dwarfris
should work.
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 17:21 ` Alexei Starovoitov
@ 2018-06-14 17:41 ` Arnaldo Carvalho de Melo
2018-06-14 17:47 ` Arnaldo Carvalho de Melo
1 sibling, 0 replies; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-14 17:41 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Martin KaFai Lau, netdev, Daniel Borkmann, kernel-team, Wang Nan,
Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu:
> On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
> >
> > Just out of curiosity, is there any plan to have this as a clang option?
>
> I think
> clang ... -mllvm -mattr=dwarfris
thanks, trying...
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 17:21 ` Alexei Starovoitov
2018-06-14 17:41 ` Arnaldo Carvalho de Melo
@ 2018-06-14 17:47 ` Arnaldo Carvalho de Melo
2018-06-14 18:00 ` Arnaldo Carvalho de Melo
1 sibling, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-14 17:47 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Martin KaFai Lau, netdev, Daniel Borkmann, kernel-team, Wang Nan,
Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu:
> On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
> > Just out of curiosity, is there any plan to have this as a clang option?
> I think
> clang ... -mllvm -mattr=dwarfris
> should work.
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
clang-opt = -g -mllvm -mattr=dwarfris
[root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'?
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when compiling BPF scriptlet
(add -v to see detail)
<SNIP>
[root@jouet bpf]#
[root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai |& grep clang
clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'?
clang -target bpf -O2 -c hello.c
[root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep clang
set env: CLANG_EXEC=/usr/local/bin/clang
llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mllvm -mattr=dwarfris -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help'
clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'?
clang -target bpf -O2 -c hello.c
[root@jouet bpf]#
The message "(LLVM option parsing)" implies what you suggest, but didn't
worked :-\
-mllvm <value> Additional arguments to forward to LLVM's option processing
Almost there tho :-\
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 17:47 ` Arnaldo Carvalho de Melo
@ 2018-06-14 18:00 ` Arnaldo Carvalho de Melo
2018-06-15 4:56 ` Yonghong Song
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-14 18:00 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Martin KaFai Lau, netdev, Daniel Borkmann, kernel-team, Wang Nan,
Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 14, 2018 at 02:47:59PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu:
> > On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
> > > Just out of curiosity, is there any plan to have this as a clang option?
> > I think
> > clang ... -mllvm -mattr=dwarfris
> > should work.
> The message "(LLVM option parsing)" implies what you suggest, but didn't
> worked :-\
> -mllvm <value> Additional arguments to forward to LLVM's option processing
> Almost there tho :-\
So I thought that this -mattr=dwarfris would be available only after I
set the target, because I tried 'llc -mattr=help' and dwarfris wasn't
there:
[acme@jouet perf]$ llc -mattr=help |& grep dwarf
[acme@jouet perf]$
Only after I set the arch it appears:
[acme@jouet perf]$ llc -march=bpf -mattr=help |& grep dwarf
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
[acme@jouet perf]$
But even after moving the '-mllvm -mattr=dwarfris' to after '-target
bpf' it still can't grok it :-\
/usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -mllvm -mattr=dwarfris -O2 -o hello.o
So onlye with 'clang ... -target bpf -emit-llvm -O2 -o - | llc -march=bpf -mattr=dwarfris ...'
things work as we expect.
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-14 18:00 ` Arnaldo Carvalho de Melo
@ 2018-06-15 4:56 ` Yonghong Song
2018-06-15 14:24 ` Arnaldo Carvalho de Melo
0 siblings, 1 reply; 38+ messages in thread
From: Yonghong Song @ 2018-06-15 4:56 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo, Alexei Starovoitov
Cc: Martin KaFai Lau, netdev, Daniel Borkmann, kernel-team, Wang Nan,
Jiri Olsa, Namhyung Kim, Ingo Molnar
On 6/14/18 11:00 AM, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jun 14, 2018 at 02:47:59PM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu:
>>> On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
>>>> Just out of curiosity, is there any plan to have this as a clang option?
>
>>> I think
>>> clang ... -mllvm -mattr=dwarfris
>>> should work.
>
>> The message "(LLVM option parsing)" implies what you suggest, but didn't
>> worked :-\
>
>> -mllvm <value> Additional arguments to forward to LLVM's option processing
>
>> Almost there tho :-\
>
> So I thought that this -mattr=dwarfris would be available only after I
> set the target, because I tried 'llc -mattr=help' and dwarfris wasn't
> there:
>
> [acme@jouet perf]$ llc -mattr=help |& grep dwarf
> [acme@jouet perf]$
>
> Only after I set the arch it appears:
>
> [acme@jouet perf]$ llc -march=bpf -mattr=help |& grep dwarf
> dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
> dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
> dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
> [acme@jouet perf]$
>
> But even after moving the '-mllvm -mattr=dwarfris' to after '-target
> bpf' it still can't grok it :-\
>
> /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -mllvm -mattr=dwarfris -O2 -o hello.o
>
> So onlye with 'clang ... -target bpf -emit-llvm -O2 -o - | llc -march=bpf -mattr=dwarfris ...'
> things work as we expect.
Right. Currently, the only way to use option -mattr=dwarfris is through
llc. The "clang -mllvm -mattr=dwarfris" won't work since
-mllvm <value> Additional arguments to forward to LLVM's option
processing
and -mattr=dwarfris is not in LLVM auto option processing system.
Those options, in llvm source code, typically have a pattern like below:
===
static cl::opt<unsigned> MemCmpEqZeroNumLoadsPerBlock(
"memcmp-num-loads-per-block", cl::Hidden, cl::init(1),
cl::desc("The number of loads per basic block for inline expansion of "
"memcmp that is only being compared against zero."));
===
I really want to get rid of this option as well. To make pahole work
with the default default format, I need to add bpf support to
libdwfl in elfutils repo. I will work on that.
> - Arnaldo
>
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-15 4:56 ` Yonghong Song
@ 2018-06-15 14:24 ` Arnaldo Carvalho de Melo
2018-06-15 16:06 ` Yonghong Song
0 siblings, 1 reply; 38+ messages in thread
From: Arnaldo Carvalho de Melo @ 2018-06-15 14:24 UTC (permalink / raw)
To: Yonghong Song
Cc: Alexei Starovoitov, Martin KaFai Lau, netdev, Daniel Borkmann,
kernel-team, Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Em Thu, Jun 14, 2018 at 09:56:43PM -0700, Yonghong Song escreveu:
> I really want to get rid of this option as well. To make pahole work
> with the default default format, I need to add bpf support to
> libdwfl in elfutils repo. I will work on that.
Right, I haven't looked into detail, but perhaps we can do like we do in
tools/perf/ where we add a feature test to check if some function is
present in a library (elfutils even) and if so, use it, otherwise, use a
copy that we carry in pahole.git.
For instance:
tools/perf/util/symbol-elf.c
#ifndef HAVE_ELF_GETPHDRNUM_SUPPORT
static int elf_getphdrnum(Elf *elf, size_t *dst)
{
GElf_Ehdr gehdr;
GElf_Ehdr *ehdr;
ehdr = gelf_getehdr(elf, &gehdr);
if (!ehdr)
return -1;
*dst = ehdr->e_phnum;
return 0;
}
#endif
And we have a feature test to check if that is present, simple one, if
that builds and links, we have it, then the tools build Makefile magic
will end up defining HAVE_ELF_GETPHDRNUM_SUPPORT and our copy doesn't
get included, using what is in elfutils:
[acme@jouet perf]$ cat tools/build/feature/test-libelf-getphdrnum.c
// SPDX-License-Identifier: GPL-2.0
#include <libelf.h>
int main(void)
{
size_t dst;
return elf_getphdrnum(0, &dst);
}
[acme@jouet perf]$
[acme@jouet perf]$ grep elf /tmp/build/perf/FEATURE-DUMP
feature-libelf=1
feature-libelf-getphdrnum=1
feature-libelf-gelf_getnote=1
feature-libelf-getshdrstrndx=1
feature-libelf-mmap=1
[acme@jouet perf]$
This way a new pahole version won't get to wait till places where it
gets built have these new functions and we stop using it as soon as the
library get it.
- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-15 14:24 ` Arnaldo Carvalho de Melo
@ 2018-06-15 16:06 ` Yonghong Song
0 siblings, 0 replies; 38+ messages in thread
From: Yonghong Song @ 2018-06-15 16:06 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Alexei Starovoitov, Martin KaFai Lau, netdev, Daniel Borkmann,
kernel-team, Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
On 6/15/18 7:24 AM, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jun 14, 2018 at 09:56:43PM -0700, Yonghong Song escreveu:
>> I really want to get rid of this option as well. To make pahole work
>> with the default default format, I need to add bpf support to
>> libdwfl in elfutils repo. I will work on that.
>
> Right, I haven't looked into detail, but perhaps we can do like we do in
> tools/perf/ where we add a feature test to check if some function is
> present in a library (elfutils even) and if so, use it, otherwise, use a
> copy that we carry in pahole.git.
>
> For instance:
>
> tools/perf/util/symbol-elf.c
>
> #ifndef HAVE_ELF_GETPHDRNUM_SUPPORT
> static int elf_getphdrnum(Elf *elf, size_t *dst)
> {
> GElf_Ehdr gehdr;
> GElf_Ehdr *ehdr;
>
> ehdr = gelf_getehdr(elf, &gehdr);
> if (!ehdr)
> return -1;
>
> *dst = ehdr->e_phnum;
>
> return 0;
> }
> #endif
>
> And we have a feature test to check if that is present, simple one, if
> that builds and links, we have it, then the tools build Makefile magic
> will end up defining HAVE_ELF_GETPHDRNUM_SUPPORT and our copy doesn't
> get included, using what is in elfutils:
>
> [acme@jouet perf]$ cat tools/build/feature/test-libelf-getphdrnum.c
> // SPDX-License-Identifier: GPL-2.0
> #include <libelf.h>
>
> int main(void)
> {
> size_t dst;
>
> return elf_getphdrnum(0, &dst);
> }
> [acme@jouet perf]$
>
> [acme@jouet perf]$ grep elf /tmp/build/perf/FEATURE-DUMP
> feature-libelf=1
> feature-libelf-getphdrnum=1
> feature-libelf-gelf_getnote=1
> feature-libelf-getshdrstrndx=1
> feature-libelf-mmap=1
> [acme@jouet perf]$
>
> This way a new pahole version won't get to wait till places where it
> gets built have these new functions and we stop using it as soon as the
> library get it.
Agreed that later on we can use feature testing to check pahole output
is good or not without dwarfris option. If it is good, compilation does
not need this option and we could deprecate dwarfris option and
eventually remove it once BPF support is in all major elf libraries.
BTW, I have pushed the following commit
https://reviews.llvm.org/rL334839
to clang so now you can compile with attribute dwarfris directly with
clang -tartget bpf.
-bash-4.2$ cat t.c
struct tt {
int a;
char b;
int c;
};
int test(struct tt *a) {
return a->a;
}
-bash-4.2$ clang -target bpf -O2 -g -c -Xclang -target-feature -Xclang
+dwarfris t.c
-bash-4.2$ llvm-objdump -S -d t.o
t.o: file format ELF64-BPF
Disassembly of section .text:
test:
; int test(struct tt *a) {
0: 61 10 00 00 00 00 00 00 r0 = *(u32 *)(r1 + 0)
; return a->a;
1: 95 00 00 00 00 00 00 00 exit
-bash-4.2$ pahole t.o
struct tt {
int a; /* 0 4 */
char b; /* 4 1 */
/* XXX 3 bytes hole, try to pack */
int c; /* 8 4 */
/* size: 12, cachelines: 1, members: 3 */
/* sum members: 9, holes: 1, sum holes: 3 */
/* last cacheline: 12 bytes */
};
-bash-4.2$
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-06-07 19:30 ` Arnaldo Carvalho de Melo
2018-06-07 20:07 ` Martin KaFai Lau
@ 2018-06-15 11:20 ` Bo YU
1 sibling, 0 replies; 38+ messages in thread
From: Bo YU @ 2018-06-15 11:20 UTC (permalink / raw)
To: Arnaldo Carvalho de Melo
Cc: Martin KaFai Lau, netdev, Alexei Starovoitov, Daniel Borkmann,
kernel-team, Wang Nan, Jiri Olsa, Namhyung Kim, Ingo Molnar
Hi,
On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
>Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
>dump-obj = true
>clang-opt = -g
>[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
>LLVM: dumping hello.o
> 0.185 ( ): __bpf_stdout__:Hello, world
> 0.188 ( 0.009 ms): touch/19670 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3
> 0.219 ( ): __bpf_stdout__:Hello, world
> 0.220 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3
> 0.480 ( 0.095 ms): touch/19670 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
> 0.624 ( ): __bpf_stdout__:Hello, world
> 0.626 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
>[root@jouet bpf]# file hello.o
>hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped
>[root@jouet bpf]#
>
>Much better:
>
>[root@jouet bpf]# readelf -SW hello.o
>There are 25 section headers, starting at offset 0xc70:
>
>Section Headers:
> [Nr] Name Type Address Off Size ES Flg Lk Inf Al
> [ 0] NULL 0000000000000000 000000 000000 00 0 0 0
> [ 1] .strtab STRTAB 0000000000000000 000b50 000119 00 0 0 1
> [ 2] .text PROGBITS 0000000000000000 000040 000000 00 AX 0 0 4
> [ 3] syscalls:sys_enter_openat PROGBITS 0000000000000000 000040 000088 00 AX 0 0 8
> [ 4] .relsyscalls:sys_enter_openat REL 0000000000000000 000910 000010 10 24 3 8
> [ 5] license PROGBITS 0000000000000000 0000c8 000004 00 WA 0 0 1
> [ 6] version PROGBITS 0000000000000000 0000cc 000004 00 WA 0 0 4
> [ 7] maps PROGBITS 0000000000000000 0000d0 000010 00 WA 0 0 4
> [ 8] .rodata.str1.1 PROGBITS 0000000000000000 0000e0 00000e 01 AMS 0 0 1
> [ 9] .debug_str PROGBITS 0000000000000000 0000ee 00010e 01 MS 0 0 1
> [10] .debug_loc PROGBITS 0000000000000000 0001fc 000023 00 0 0 1
> [11] .debug_abbrev PROGBITS 0000000000000000 00021f 0000e3 00 0 0 1
> [12] .debug_info PROGBITS 0000000000000000 000302 00015e 00 0 0 1
> [13] .rel.debug_info REL 0000000000000000 000920 0001e0 10 24 12 8
> [14] .debug_ranges PROGBITS 0000000000000000 000460 000030 00 0 0 1
> [15] .debug_macinfo PROGBITS 0000000000000000 000490 000001 00 0 0 1
> [16] .debug_pubnames PROGBITS 0000000000000000 000491 00006e 00 0 0 1
> [17] .rel.debug_pubnames REL 0000000000000000 000b00 000010 10 24 16 8
> [18] .debug_pubtypes PROGBITS 0000000000000000 0004ff 00005a 00 0 0 1
> [19] .rel.debug_pubtypes REL 0000000000000000 000b10 000010 10 24 18 8
> [20] .debug_frame PROGBITS 0000000000000000 000560 000028 00 0 0 8
> [21] .rel.debug_frame REL 0000000000000000 000b20 000020 10 24 20 8
> [22] .debug_line PROGBITS 0000000000000000 000588 00006e 00 0 0 1
> [23] .rel.debug_line REL 0000000000000000 000b40 000010 10 24 22 8
> [24] .symtab SYMTAB 0000000000000000 0005f8 000318 18 1 29 8
>Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> p (processor specific)
>[root@jouet bpf]#
>
>[root@jouet bpf]# readelf -s hello.o
>
>Symbol table '.symtab' contains 33 entries:
> Num: Value Size Type Bind Vis Ndx Name
> 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
> 1: 0000000000000000 0 NOTYPE LOCAL DEFAULT 9
> 2: 000000000000002d 0 NOTYPE LOCAL DEFAULT 9
> 3: 0000000000000044 0 NOTYPE LOCAL DEFAULT 9
> 4: 0000000000000053 0 NOTYPE LOCAL DEFAULT 9
> 5: 000000000000005c 0 NOTYPE LOCAL DEFAULT 9
> 6: 0000000000000061 0 NOTYPE LOCAL DEFAULT 9
> 7: 000000000000006a 0 NOTYPE LOCAL DEFAULT 9
> 8: 0000000000000073 0 NOTYPE LOCAL DEFAULT 9
> 9: 0000000000000077 0 NOTYPE LOCAL DEFAULT 9
> 10: 0000000000000086 0 NOTYPE LOCAL DEFAULT 9
> 11: 000000000000008b 0 NOTYPE LOCAL DEFAULT 9
> 12: 0000000000000098 0 NOTYPE LOCAL DEFAULT 9
> 13: 00000000000000a1 0 NOTYPE LOCAL DEFAULT 9
> 14: 00000000000000ac 0 NOTYPE LOCAL DEFAULT 9
> 15: 00000000000000b8 0 NOTYPE LOCAL DEFAULT 9
> 16: 00000000000000c4 0 NOTYPE LOCAL DEFAULT 9
> 17: 00000000000000d6 0 NOTYPE LOCAL DEFAULT 9
> 18: 00000000000000e8 0 NOTYPE LOCAL DEFAULT 9
> 19: 00000000000000fd 0 NOTYPE LOCAL DEFAULT 9
> 20: 0000000000000101 0 NOTYPE LOCAL DEFAULT 9
> 21: 0000000000000107 0 NOTYPE LOCAL DEFAULT 9
> 22: 0000000000000000 0 SECTION LOCAL DEFAULT 3
> 23: 0000000000000000 0 SECTION LOCAL DEFAULT 10
> 24: 0000000000000000 0 SECTION LOCAL DEFAULT 11
> 25: 0000000000000000 0 SECTION LOCAL DEFAULT 12
> 26: 0000000000000000 0 SECTION LOCAL DEFAULT 14
> 27: 0000000000000000 0 SECTION LOCAL DEFAULT 20
> 28: 0000000000000000 0 SECTION LOCAL DEFAULT 22
> 29: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 7 __bpf_stdout__
> 30: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 5 _license
> 31: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 6 _version
> 32: 0000000000000000 0 NOTYPE GLOBAL DEFAULT 3 syscall_enter_openat
>[root@jouet bpf]#
>
>Progress:
>
>[root@jouet bpf]# pahole --btf_encode hello.o
>sh: llvm-objcopy: command not found
>Failed to encode BTF
>[root@jouet bpf]#
>
>[acme@jouet perf]$ llvm-
>llvm-ar llvm-cov llvm-dis llvm-lib llvm-modextract llvm-profdata llvm-split
>llvm-as llvm-c-test llvm-dlltool llvm-link llvm-mt llvm-ranlib llvm-stress
>llvm-bcanalyzer llvm-cvtres llvm-dsymutil llvm-lto llvm-nm llvm-readelf llvm-strings
>llvm-cat llvm-cxxdump llvm-dwarfdump llvm-lto2 llvm-objdump llvm-readobj llvm-symbolizer
>llvm-config llvm-cxxfilt llvm-dwp llvm-mc llvm-opt-report llvm-rtdyld llvm-tblgen
>llvm-config-64 llvm-diff llvm-extract llvm-mcmarkup llvm-pdbutil llvm-size llvm-xray
>[acme@jouet perf]$ llvm-
Sorry, maybe i make a noise for the thread.
Could you share how you put the output above in mutt's reply mode?
Thank you in advance!
>
>So this must be available in a newer llvm version? Which one?
>
>[root@jouet bpf]# clang -v
>clang version 5.0.1 (tags/RELEASE_501/final)
>Target: x86_64-unknown-linux-gnu
>Thread model: posix
>InstalledDir: /usr/bin
>Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
>Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
>Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
>Candidate multilib: .;@m64
>Candidate multilib: 32;@m32
>Selected multilib: .;@m64
>[root@jouet bpf]#
>
>[acme@jouet perf]$ rpm -q clang llvm
>clang-5.0.1-5.fc27.x86_64
>llvm-5.0.1-6.fc27.x86_64
>[acme@jouet perf]$
>[root@jouet bpf]# clang -v
>clang version 5.0.1 (tags/RELEASE_501/final)
>Target: x86_64-unknown-linux-gnu
>Thread model: posix
>InstalledDir: /usr/bin
>Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
>Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/7
>Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/7
>Candidate multilib: .;@m64
>Candidate multilib: 32;@m32
>Selected multilib: .;@m64
>[root@jouet bpf]#
>
>[acme@jouet perf]$ rpm -q clang llvm
>clang-5.0.1-5.fc27.x86_64
>llvm-5.0.1-6.fc27.x86_64
>[acme@jouet perf]$
>
>- Arnaldo
^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
2018-04-18 22:55 [PATCH bpf-next v5 00/10] BTF: BPF Type Format Martin KaFai Lau
` (10 preceding siblings ...)
2018-04-19 19:40 ` [PATCH bpf-next v5 00/10] BTF: BPF Type Format Arnaldo Carvalho de Melo
@ 2018-04-19 23:57 ` Daniel Borkmann
11 siblings, 0 replies; 38+ messages in thread
From: Daniel Borkmann @ 2018-04-19 23:57 UTC (permalink / raw)
To: Martin KaFai Lau, netdev
Cc: Alexei Starovoitov, kernel-team, Arnaldo Carvalho de Melo
On 04/19/2018 12:55 AM, Martin KaFai Lau wrote:
> This patch introduces BPF Type Format (BTF).
>
> BTF (BPF Type Format) is the meta data format which describes
> the data types of BPF program/map. Hence, it basically focus
> on the C programming language which the modern BPF is primary
> using. The first use case is to provide a generic pretty print
> capability for a BPF map.
>
> A modified pahole that can convert dwarf to BTF is here:
> https://github.com/iamkafai/pahole/tree/btf
> (Arnaldo, there is some BTF_KIND numbering changes on
> Apr 18th, d61426c1571)
>
> Please see individual patch for details.
>
> v5:
> - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not
> currently used. They can be added in the future.
> Some bpf_df_xxx() are removed together.
> - Add comment in patch 7 to clarify that the new bpffs_map_fops
> should not be extended further.
>
> v4:
> - Fix warning (remove unneeded semicolon)
> - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in
> patch 1. Caught by W=1.
>
> v3:
> - Rebase to bpf-next
> - Fix sparse warning (by adding static)
> - Add BTF header logging: btf_verifier_log_hdr()
> - Fix the alignment test on btf->type_off
> - Add tests for the BTF header
> - Lower the max BTF size to 16MB. It should be enough
> for some time. We could raise it later if it would
> be needed.
>
> v2:
> - Use kvfree where needed in patch 1 and 2
> - Also consider BTF_INT_OFFSET() in the btf_int_check_meta()
> in patch 1
> - Fix an incorrect goto target in map_create() during
> the btf-error-path in patch 7
> - re-org some local vars to keep the rev xmas tree in btf.c
Series applied to bpf-next, thanks Martin. As discussed please follow up
with the bpftool patches.
Thanks,
Daniel
^ permalink raw reply [flat|nested] 38+ messages in thread