[PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Yonghong Song]

btf_type_tag is a new llvm type attribute which is used similar
to kernel __user/__rcu attributes. The format of btf_type_tag looks like
  __attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
  #define __tag1 __attribute__((btf_type_tag("tag1")))
  #define __tag2 __attribute__((btf_type_tag("tag2")))
  int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind
BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically
converts llvm dwarf btf_type_tag attributes to
BTF types. Please see individual patches for details.

Changelog:
  v1 -> v2:
     - reorg an if condition to reduce nesting level.
     - add more comments to explain how to chain type tag types.

Yonghong Song (4):
  libbpf: sync with latest libbpf repo
  dutil: move DW_TAG_LLVM_annotation definition to dutil.h
  dwarf_loader: support btf_type_tag attribute
  btf_encoder: support btf_type_tag attribute

 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)

-- 
2.30.2

[PATCH dwarves v2 1/4] libbpf: sync with latest libbpf repo

[Yonghong Song]

Sync up to commit 94a49850c5ee ("Makefile: enforce gnu89 standard").
This is needed to support BTF_KIND_TYPE_TAG.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 lib/bpf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/bpf b/lib/bpf
index f05791d..94a4985 160000
--- a/lib/bpf
+++ b/lib/bpf
@@ -1 +1 @@
-Subproject commit f05791d8cfcbbf9092b4099b9d011bb72e241ef8
+Subproject commit 94a49850c5ee61ea02dfcbabf48013391e8cecdf
-- 
2.30.2

[PATCH dwarves v2 2/4] dutil: move DW_TAG_LLVM_annotation definition to dutil.h

[Yonghong Song]

Move DW_TAG_LLVM_annotation definition from dwarf_load.c to
dutil.h as it will be used later for btf_encoder.c.
There is no functionality change for this patch.

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 dutil.h        | 4 ++++
 dwarf_loader.c | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/dutil.h b/dutil.h
index 25691ea..e45bba0 100644
--- a/dutil.h
+++ b/dutil.h
@@ -31,6 +31,10 @@
 
 #define roundup(x,y) ((((x) + ((y) - 1)) / (y)) * (y))
 
+#ifndef DW_TAG_LLVM_annotation
+#define DW_TAG_LLVM_annotation 0x6000
+#endif
+
 static inline __attribute__((const)) bool is_power_of_2(unsigned long n)
 {
         return (n != 0 && ((n & (n - 1)) == 0));
diff --git a/dwarf_loader.c b/dwarf_loader.c
index cf005da..1b07a62 100644
--- a/dwarf_loader.c
+++ b/dwarf_loader.c
@@ -52,10 +52,6 @@
 #define DW_OP_addrx 0xa1
 #endif
 
-#ifndef DW_TAG_LLVM_annotation
-#define DW_TAG_LLVM_annotation 0x6000
-#endif
-
 static pthread_mutex_t libdw__lock = PTHREAD_MUTEX_INITIALIZER;
 
 static uint32_t hashtags__bits = 12;
-- 
2.30.2

[PATCH dwarves v2 3/4] dwarf_loader: support btf_type_tag attribute

[Yonghong Song]

LLVM patches ([1] for clang, [2] and [3] for BPF backend)
added support for btf_type_tag attributes. The following is
an example:
  [$ ~] cat t.c
  #define __tag1 __attribute__((btf_type_tag("tag1")))
  #define __tag2 __attribute__((btf_type_tag("tag2")))
  int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
  [$ ~] clang -O2 -g -c t.c
  [$ ~] llvm-dwarfdump --debug-info t.o
  t.o:    file format elf64-x86-64
  ...
  0x0000001e:   DW_TAG_variable
                  DW_AT_name      ("g")
                  DW_AT_type      (0x00000033 "int **")
                  DW_AT_external  (true)
                  DW_AT_decl_file ("/home/yhs/t.c")
                  DW_AT_decl_line (3)
                  DW_AT_location  (DW_OP_addr 0x0)
  0x00000033:   DW_TAG_pointer_type
                  DW_AT_type      (0x0000004b "int *")
  0x00000038:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag1")
  0x00000041:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag2")
  0x0000004a:     NULL
  0x0000004b:   DW_TAG_pointer_type
                  DW_AT_type      (0x0000005a "int")
  0x00000050:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag1")
  0x00000059:     NULL
  0x0000005a:   DW_TAG_base_type
                  DW_AT_name      ("int")
                  DW_AT_encoding  (DW_ATE_signed)
                  DW_AT_byte_size (0x04)
  0x00000061:   NULL

From the above example, you can see that DW_TAG_pointer_type
may contain one or more DW_TAG_LLVM_annotation btf_type_tag tags.
If DW_TAG_LLVM_annotation tags are present inside
DW_TAG_pointer_type, for BTF encoding, pahole will need
to follow [3] to generate a type chain like
  var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int

This patch implemented dwarf_loader support. If a pointer type
contains DW_TAG_LLVM_annotation tags, a new type
btf_type_tag_ptr_type will be created which will store
the pointer tag itself and all DW_TAG_LLVM_annotation tags.
During recoding stage, the type chain will be formed properly
based on the above example.

An option "--skip_encoding_btf_type_tag" is added to disable
this new functionality.

  [1] https://reviews.llvm.org/D111199
  [2] https://reviews.llvm.org/D113222
  [3] https://reviews.llvm.org/D113496
---
 dwarf_loader.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++--
 dwarves.h      |  33 +++++++++++-
 pahole.c       |   8 +++
 3 files changed, 173 insertions(+), 4 deletions(-)

diff --git a/dwarf_loader.c b/dwarf_loader.c
index 1b07a62..e30b03c 100644
--- a/dwarf_loader.c
+++ b/dwarf_loader.c
@@ -1206,6 +1206,89 @@ static struct tag *die__create_new_tag(Dwarf_Die *die, struct cu *cu)
 	return tag;
 }
 
+static struct btf_type_tag_ptr_type *die__create_new_btf_type_tag_ptr_type(Dwarf_Die *die, struct cu *cu)
+{
+	struct btf_type_tag_ptr_type *tag;
+
+	tag  = tag__alloc_with_spec(cu, sizeof(struct btf_type_tag_ptr_type));
+	if (tag == NULL)
+		return NULL;
+
+	tag__init(&tag->tag, cu, die);
+	tag->tag.has_btf_type_tag = true;
+	INIT_LIST_HEAD(&tag->tags);
+	return tag;
+}
+
+static struct btf_type_tag_type *die__create_new_btf_type_tag_type(Dwarf_Die *die, struct cu *cu,
+								   struct conf_load *conf)
+{
+	struct btf_type_tag_type *tag;
+
+	tag  = tag__alloc_with_spec(cu, sizeof(struct btf_type_tag_type));
+	if (tag == NULL)
+		return NULL;
+
+	tag__init(&tag->tag, cu, die);
+	tag->value = attr_string(die, DW_AT_const_value, conf);
+	return tag;
+}
+
+static struct tag *die__create_new_pointer_tag(Dwarf_Die *die, struct cu *cu,
+					       struct conf_load *conf)
+{
+	struct btf_type_tag_ptr_type *tag = NULL;
+	struct btf_type_tag_type *annot;
+	Dwarf_Die *cdie, child;
+	const char *name;
+	uint32_t id;
+
+	/* If no child tags or skipping btf_type_tag encoding, just create a new tag
+	 * and return
+	 */
+	if (!dwarf_haschildren(die) || dwarf_child(die, &child) != 0 ||
+	    conf->skip_encoding_btf_type_tag)
+		return tag__new(die, cu);
+
+	/* Otherwise, check DW_TAG_LLVM_annotation child tags */
+	cdie = &child;
+	do {
+		if (dwarf_tag(cdie) != DW_TAG_LLVM_annotation)
+			continue;
+
+		/* Only check btf_type_tag annotations */
+		name = attr_string(cdie, DW_AT_name, conf);
+		if (strcmp(name, "btf_type_tag") != 0)
+			continue;
+
+		if (tag == NULL) {
+			/* Create a btf_type_tag_ptr type. */
+			tag = die__create_new_btf_type_tag_ptr_type(die, cu);
+			if (!tag)
+				return NULL;
+		}
+
+		/* Create a btf_type_tag type for this annotation. */
+		annot = die__create_new_btf_type_tag_type(cdie, cu, conf);
+		if (annot == NULL)
+			return NULL;
+
+		if (cu__table_add_tag(cu, &annot->tag, &id) < 0)
+			return NULL;
+
+		struct dwarf_tag *dtag = annot->tag.priv;
+		dtag->small_id = id;
+		cu__hash(cu, &annot->tag);
+
+		/* For a list of DW_TAG_LLVM_annotation like tag1 -> tag2 -> tag3,
+		 * the tag->tags contains tag3 -> tag2 -> tag1.
+		 */
+		list_add(&annot->node, &tag->tags);
+	} while (dwarf_siblingof(cdie, cdie) == 0);
+
+	return tag ? &tag->tag : tag__new(die, cu);
+}
+
 static struct tag *die__create_new_ptr_to_member_type(Dwarf_Die *die,
 						      struct cu *cu)
 {
@@ -1903,12 +1986,13 @@ static struct tag *__die__process_tag(Dwarf_Die *die, struct cu *cu,
 	case DW_TAG_const_type:
 	case DW_TAG_imported_declaration:
 	case DW_TAG_imported_module:
-	case DW_TAG_pointer_type:
 	case DW_TAG_reference_type:
 	case DW_TAG_restrict_type:
 	case DW_TAG_unspecified_type:
 	case DW_TAG_volatile_type:
 		tag = die__create_new_tag(die, cu);		break;
+	case DW_TAG_pointer_type:
+		tag = die__create_new_pointer_tag(die, cu, conf);	break;
 	case DW_TAG_ptr_to_member_type:
 		tag = die__create_new_ptr_to_member_type(die, cu); break;
 	case DW_TAG_enumeration_type:
@@ -2192,6 +2276,45 @@ static void lexblock__recode_dwarf_types(struct lexblock *tag, struct cu *cu)
 	}
 }
 
+static void dwarf_cu__recode_btf_type_tag_ptr(struct btf_type_tag_ptr_type *tag,
+					      uint32_t pointee_type)
+{
+	struct btf_type_tag_type *annot;
+	struct dwarf_tag *annot_dtag;
+	struct tag *prev_tag;
+
+	/* Given source like
+	 *   int tag1 tag2 tag3 *p;
+	 * the tag->tags contains tag3 -> tag2 -> tag1, the final type chain looks like:
+	 *   pointer -> tag3 -> tag2 -> tag1 -> pointee
+	 *
+	 * Basically it means
+	 *   - '*' applies to "int tag1 tag2 tag3"
+	 *   - tag3 applies to "int tag1 tag2"
+	 *   - tag2 applies to "int tag1"
+	 *   - tag1 applies to "int"
+	 *
+	 * This also makes final source code (format c) easier as we can do
+	 *   emit for "tag3 -> tag2 -> tag1 -> int"
+	 *   emit '*'
+	 *
+	 * For 'tag3 -> tag2 -> tag1 -> int":
+	 *   emit for "tag2 -> tag1 -> int"
+	 *   emit tag3
+	 *
+	 * Eventually we can get the source code like
+	 *   int tag1 tag2 tag3 *p;
+	 * and this matches the user/kernel code.
+	 */
+	prev_tag = &tag->tag;
+	list_for_each_entry(annot, &tag->tags, node) {
+		annot_dtag = annot->tag.priv;
+		prev_tag->type = annot_dtag->small_id;
+		prev_tag = &annot->tag;
+	}
+	prev_tag->type = pointee_type;
+}
+
 static int tag__recode_dwarf_type(struct tag *tag, struct cu *cu)
 {
 	struct dwarf_tag *dtag = tag->priv;
@@ -2301,7 +2424,10 @@ static int tag__recode_dwarf_type(struct tag *tag, struct cu *cu)
 	}
 
 	if (dtag->type.off == 0) {
-		tag->type = 0; /* void */
+		if (tag->tag != DW_TAG_pointer_type || !tag->has_btf_type_tag)
+			tag->type = 0; /* void */
+		else
+			dwarf_cu__recode_btf_type_tag_ptr(tag__btf_type_tag_ptr(tag), 0);
 		return 0;
 	}
 
@@ -2313,7 +2439,11 @@ check_type:
 		return 0;
 	}
 out:
-	tag->type = dtype->small_id;
+	if (tag->tag != DW_TAG_pointer_type || !tag->has_btf_type_tag)
+		tag->type = dtype->small_id;
+	else
+		dwarf_cu__recode_btf_type_tag_ptr(tag__btf_type_tag_ptr(tag), dtype->small_id);
+
 	return 0;
 }
 
diff --git a/dwarves.h b/dwarves.h
index 0d3e204..4425d3c 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -63,6 +63,7 @@ struct conf_load {
 	bool			ptr_table_stats;
 	bool			skip_encoding_btf_decl_tag;
 	bool			skip_missing;
+	bool			skip_encoding_btf_type_tag;
 	uint8_t			hashtable_bits;
 	uint8_t			max_hashtable_bits;
 	uint16_t		kabi_prefix_len;
@@ -413,6 +414,7 @@ struct tag {
 	uint16_t	 tag;
 	bool		 visited;
 	bool		 top_level;
+	bool		 has_btf_type_tag;
 	uint16_t	 recursivity_level;
 	void		 *priv;
 };
@@ -533,7 +535,8 @@ static inline int tag__is_tag_type(const struct tag *tag)
 	       tag->tag == DW_TAG_restrict_type ||
 	       tag->tag == DW_TAG_subroutine_type ||
 	       tag->tag == DW_TAG_unspecified_type ||
-	       tag->tag == DW_TAG_volatile_type;
+	       tag->tag == DW_TAG_volatile_type ||
+	       tag->tag == DW_TAG_LLVM_annotation;
 }
 
 static inline const char *tag__decl_file(const struct tag *tag,
@@ -606,6 +609,34 @@ struct llvm_annotation {
 	struct list_head	node;
 };
 
+/** struct btf_type_tag_type - representing a btf_type_tag annotation
+ *
+ * @tag   - DW_TAG_LLVM_annotation tag
+ * @value - btf_type_tag value string
+ * @node  - list_head node
+ */
+struct btf_type_tag_type {
+	struct tag		tag;
+	const char		*value;
+	struct list_head	node;
+};
+
+/** The struct btf_type_tag_ptr_type - type containing both pointer type and
+ *  its btf_type_tag annotations
+ *
+ * @tag  - pointer type tag
+ * @tags - btf_type_tag annotations for the pointer type
+ */
+struct btf_type_tag_ptr_type {
+	struct tag		tag;
+	struct list_head 	tags;
+};
+
+static inline struct btf_type_tag_ptr_type *tag__btf_type_tag_ptr(struct tag *tag)
+{
+	return (struct btf_type_tag_ptr_type *)tag;
+}
+
 /** struct namespace - base class for enums, structs, unions, typedefs, etc
  *
  * @tags - class_member, enumerators, etc
diff --git a/pahole.c b/pahole.c
index 5fc1cca..f3a51cb 100644
--- a/pahole.c
+++ b/pahole.c
@@ -1126,6 +1126,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;
 #define ARGP_devel_stats	   330
 #define ARGP_skip_encoding_btf_decl_tag 331
 #define ARGP_skip_missing          332
+#define ARGP_skip_encoding_btf_type_tag 333
 
 static const struct argp_option pahole__options[] = {
 	{
@@ -1506,6 +1507,11 @@ static const struct argp_option pahole__options[] = {
 		.key  = ARGP_skip_missing,
 		.doc = "skip missing types passed to -C rather than stop",
 	},
+	{
+		.name = "skip_encoding_btf_type_tag",
+		.key  = ARGP_skip_encoding_btf_type_tag,
+		.doc  = "Do not encode TAGs in BTF."
+	},
 	{
 		.name = NULL,
 	}
@@ -1658,6 +1664,8 @@ static error_t pahole__options_parser(int key, char *arg,
 		conf_load.skip_encoding_btf_decl_tag = true;	break;
 	case ARGP_skip_missing:
 		conf_load.skip_missing = true;          break;
+	case ARGP_skip_encoding_btf_type_tag:
+		conf_load.skip_encoding_btf_type_tag = true;	break;
 	default:
 		return ARGP_ERR_UNKNOWN;
 	}
-- 
2.30.2

[PATCH dwarves v2 4/4] btf_encoder: support btf_type_tag attribute

[Yonghong Song]

The following is an example.
  [$ ~] cat t.c
  #define __tag1 __attribute__((btf_type_tag("tag1")))
  #define __tag2 __attribute__((btf_type_tag("tag2")))
  int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
  [$ ~] clang -O2 -g -c t.c
  [$ ~] pahole -JV t.o
  Found per-CPU symbol 'g' at address 0x0
  Found 1 per-CPU variables!
  File t.o:
  [1] TYPE_TAG tag1 type_id=5
  [2] TYPE_TAG tag2 type_id=1
  [3] PTR (anon) type_id=2
  [4] TYPE_TAG tag1 type_id=6
  [5] PTR (anon) type_id=4
  [6] INT int size=4 nr_bits=32 encoding=SIGNED
  search cu 't.c' for percpu global variables.
  Variable 'g' from CU 't.c' at address 0x0 encoded
  [7] VAR g type=3 linkage=1
  [8] DATASEC .data..percpu size=8 vlen=1
          type=7 offset=0 size=8
  [$ ~]

You can see for the source
  int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
the following type chain is generated:
  var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int

The following shows pahole option "--skip_encoding_btf_type_tag"
can prevent BTF_KIND_TYPE_TAG generation.
  [$ ~] pahole -JV t.o --skip_encoding_btf_type_tag
  Found per-CPU symbol 'g' at address 0x0
  Found 1 per-CPU variables!
  File t.o:
  [1] PTR (anon) type_id=2
  [2] PTR (anon) type_id=3
  [3] INT int size=4 nr_bits=32 encoding=SIGNED
  search cu 't.c' for percpu global variables.
  Variable 'g' from CU 't.c' at address 0x0 encoded
  [4] VAR g type=1 linkage=1
  [5] DATASEC .data..percpu size=8 vlen=1
          type=4 offset=0 size=8
  [$ ~]

Signed-off-by: Yonghong Song <yhs@fb.com>
---
 btf_encoder.c | 7 +++++++
 dwarves.h     | 5 +++++
 2 files changed, 12 insertions(+)

diff --git a/btf_encoder.c b/btf_encoder.c
index 117656e..9d015f3 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -143,6 +143,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = {
 	[BTF_KIND_DATASEC]      = "DATASEC",
 	[BTF_KIND_FLOAT]        = "FLOAT",
 	[BTF_KIND_DECL_TAG]     = "DECL_TAG",
+	[BTF_KIND_TYPE_TAG]     = "TYPE_TAG",
 };
 
 static const char *btf__printable_name(const struct btf *btf, uint32_t offset)
@@ -393,6 +394,9 @@ static int32_t btf_encoder__add_ref_type(struct btf_encoder *encoder, uint16_t k
 	case BTF_KIND_TYPEDEF:
 		id = btf__add_typedef(btf, name, type);
 		break;
+	case BTF_KIND_TYPE_TAG:
+		id = btf__add_type_tag(btf, name, type);
+		break;
 	case BTF_KIND_FWD:
 		id = btf__add_fwd(btf, name, kind_flag);
 		break;
@@ -862,6 +866,9 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag,
 	case DW_TAG_typedef:
 		name = namespace__name(tag__namespace(tag));
 		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPEDEF, ref_type_id, name, false);
+	case DW_TAG_LLVM_annotation:
+		name = tag__btf_type_tag(tag)->value;
+		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPE_TAG, ref_type_id, name, false);
 	case DW_TAG_structure_type:
 	case DW_TAG_union_type:
 	case DW_TAG_class_type:
diff --git a/dwarves.h b/dwarves.h
index 4425d3c..52d162d 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -637,6 +637,11 @@ static inline struct btf_type_tag_ptr_type *tag__btf_type_tag_ptr(struct tag *ta
 	return (struct btf_type_tag_ptr_type *)tag;
 }
 
+static inline struct btf_type_tag_type *tag__btf_type_tag(struct tag *tag)
+{
+	return (struct btf_type_tag_type *)tag;
+}
+
 /** struct namespace - base class for enums, structs, unions, typedefs, etc
  *
  * @tags - class_member, enumerators, etc
-- 
2.30.2

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Andrii Nakryiko]

On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>
> btf_type_tag is a new llvm type attribute which is used similar
> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>   __attribute__((btf_type_tag("tag1")))
> For the case where the attribute applied to a pointee like
>   #define __tag1 __attribute__((btf_type_tag("tag1")))
>   #define __tag2 __attribute__((btf_type_tag("tag2")))
>   int __tag1 * __tag1 __tag2 *g;
> the information will be encoded in dwarf.
>
> In BTF, the attribute is encoded as a new kind
> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>
> The patch added support in pahole, specifically
> converts llvm dwarf btf_type_tag attributes to
> BTF types. Please see individual patches for details.
>
> Changelog:
>   v1 -> v2:
>      - reorg an if condition to reduce nesting level.
>      - add more comments to explain how to chain type tag types.
>
> Yonghong Song (4):
>   libbpf: sync with latest libbpf repo
>   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>   dwarf_loader: support btf_type_tag attribute
>   btf_encoder: support btf_type_tag attribute
>

I thought that v1 was already applied, but either way LGTM. I'm not
super familiar with the DWARF loader parts, so I mostly just read it
very superficially :)

Acked-by: Andrii Nakryiko <andrii@kernel.org>


>  btf_encoder.c  |   7 +++
>  dutil.h        |   4 ++
>  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>  dwarves.h      |  38 +++++++++++++-
>  lib/bpf        |   2 +-
>  pahole.c       |   8 +++
>  6 files changed, 190 insertions(+), 9 deletions(-)
>
> --
> 2.30.2
>

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Arnaldo Carvalho de Melo]

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> >
> > btf_type_tag is a new llvm type attribute which is used similar
> > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> >   __attribute__((btf_type_tag("tag1")))
> > For the case where the attribute applied to a pointee like
> >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> >   int __tag1 * __tag1 __tag2 *g;
> > the information will be encoded in dwarf.
> >
> > In BTF, the attribute is encoded as a new kind
> > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> >
> > The patch added support in pahole, specifically
> > converts llvm dwarf btf_type_tag attributes to
> > BTF types. Please see individual patches for details.
> >
> > Changelog:
> >   v1 -> v2:
> >      - reorg an if condition to reduce nesting level.
> >      - add more comments to explain how to chain type tag types.
> >
> > Yonghong Song (4):
> >   libbpf: sync with latest libbpf repo
> >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> >   dwarf_loader: support btf_type_tag attribute
> >   btf_encoder: support btf_type_tag attribute
> >
> 
> I thought that v1 was already applied, but either way LGTM. I'm not

To the next branch, and the libbpf pahole CI is failing, since a few
days, can you please take a look?

> super familiar with the DWARF loader parts, so I mostly just read it
> very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and
tested it with llvm-project HEAD.

> Acked-by: Andrii Nakryiko <andrii@kernel.org>

Adding it to the csets.

Thanks!

- Arnaldo
 
> 
> >  btf_encoder.c  |   7 +++
> >  dutil.h        |   4 ++
> >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> >  dwarves.h      |  38 +++++++++++++-
> >  lib/bpf        |   2 +-
> >  pahole.c       |   8 +++
> >  6 files changed, 190 insertions(+), 9 deletions(-)
> >
> > --
> > 2.30.2
> >

-- 

- Arnaldo

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Yonghong Song]

On 11/23/21 3:36 PM, Arnaldo Carvalho de Melo wrote:
> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
>> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>>>
>>> btf_type_tag is a new llvm type attribute which is used similar
>>> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>>>    __attribute__((btf_type_tag("tag1")))
>>> For the case where the attribute applied to a pointee like
>>>    #define __tag1 __attribute__((btf_type_tag("tag1")))
>>>    #define __tag2 __attribute__((btf_type_tag("tag2")))
>>>    int __tag1 * __tag1 __tag2 *g;
>>> the information will be encoded in dwarf.
>>>
>>> In BTF, the attribute is encoded as a new kind
>>> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>>>
>>> The patch added support in pahole, specifically
>>> converts llvm dwarf btf_type_tag attributes to
>>> BTF types. Please see individual patches for details.
>>>
>>> Changelog:
>>>    v1 -> v2:
>>>       - reorg an if condition to reduce nesting level.
>>>       - add more comments to explain how to chain type tag types.
>>>
>>> Yonghong Song (4):
>>>    libbpf: sync with latest libbpf repo
>>>    dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>>>    dwarf_loader: support btf_type_tag attribute
>>>    btf_encoder: support btf_type_tag attribute
>>>
>>
>> I thought that v1 was already applied, but either way LGTM. I'm not
> 
> To the next branch, and the libbpf pahole CI is failing, since a few
> days, can you please take a look?
> 
>> super familiar with the DWARF loader parts, so I mostly just read it
>> very superficially :)
> 
> I replaced the patches that changed, re-added the S-o-B for Yonghong and
> tested it with llvm-project HEAD.

Thanks Arnaldo!

> 
>> Acked-by: Andrii Nakryiko <andrii@kernel.org>
> 
> Adding it to the csets.
> 
> Thanks!
> 
> - Arnaldo
>   
>>
>>>   btf_encoder.c  |   7 +++
>>>   dutil.h        |   4 ++
>>>   dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>>>   dwarves.h      |  38 +++++++++++++-
>>>   lib/bpf        |   2 +-
>>>   pahole.c       |   8 +++
>>>   6 files changed, 190 insertions(+), 9 deletions(-)
>>>
>>> --
>>> 2.30.2
>>>
> 

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Andrii Nakryiko]

On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
>
> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> > On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> > >
> > > btf_type_tag is a new llvm type attribute which is used similar
> > > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> > >   __attribute__((btf_type_tag("tag1")))
> > > For the case where the attribute applied to a pointee like
> > >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> > >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> > >   int __tag1 * __tag1 __tag2 *g;
> > > the information will be encoded in dwarf.
> > >
> > > In BTF, the attribute is encoded as a new kind
> > > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> > >
> > > The patch added support in pahole, specifically
> > > converts llvm dwarf btf_type_tag attributes to
> > > BTF types. Please see individual patches for details.
> > >
> > > Changelog:
> > >   v1 -> v2:
> > >      - reorg an if condition to reduce nesting level.
> > >      - add more comments to explain how to chain type tag types.
> > >
> > > Yonghong Song (4):
> > >   libbpf: sync with latest libbpf repo
> > >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> > >   dwarf_loader: support btf_type_tag attribute
> > >   btf_encoder: support btf_type_tag attribute
> > >
> >
> > I thought that v1 was already applied, but either way LGTM. I'm not
>
> To the next branch, and the libbpf pahole CI is failing, since a few
> days, can you please take a look?

We've had Clang regression which Yonghong fixed very quickly, but then
we were blocked on Clang nightly builds being broken for days. Seems
like we got a new Clang today, so hopefully libbpf CI will be back to
green again.

>
> > super familiar with the DWARF loader parts, so I mostly just read it
> > very superficially :)
>
> I replaced the patches that changed, re-added the S-o-B for Yonghong and
> tested it with llvm-project HEAD.
>
> > Acked-by: Andrii Nakryiko <andrii@kernel.org>
>
> Adding it to the csets.
>
> Thanks!
>
> - Arnaldo
>
> >
> > >  btf_encoder.c  |   7 +++
> > >  dutil.h        |   4 ++
> > >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> > >  dwarves.h      |  38 +++++++++++++-
> > >  lib/bpf        |   2 +-
> > >  pahole.c       |   8 +++
> > >  6 files changed, 190 insertions(+), 9 deletions(-)
> > >
> > > --
> > > 2.30.2
> > >
>
> --
>
> - Arnaldo

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Arnaldo Carvalho de Melo]

Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:
> On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
> <acme@kernel.org> wrote:
> >
> > Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
> > > On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
> > > >
> > > > btf_type_tag is a new llvm type attribute which is used similar
> > > > to kernel __user/__rcu attributes. The format of btf_type_tag looks like
> > > >   __attribute__((btf_type_tag("tag1")))
> > > > For the case where the attribute applied to a pointee like
> > > >   #define __tag1 __attribute__((btf_type_tag("tag1")))
> > > >   #define __tag2 __attribute__((btf_type_tag("tag2")))
> > > >   int __tag1 * __tag1 __tag2 *g;
> > > > the information will be encoded in dwarf.
> > > >
> > > > In BTF, the attribute is encoded as a new kind
> > > > BTF_KIND_TYPE_TAG and latest bpf-next supports it.
> > > >
> > > > The patch added support in pahole, specifically
> > > > converts llvm dwarf btf_type_tag attributes to
> > > > BTF types. Please see individual patches for details.
> > > >
> > > > Changelog:
> > > >   v1 -> v2:
> > > >      - reorg an if condition to reduce nesting level.
> > > >      - add more comments to explain how to chain type tag types.
> > > >
> > > > Yonghong Song (4):
> > > >   libbpf: sync with latest libbpf repo
> > > >   dutil: move DW_TAG_LLVM_annotation definition to dutil.h
> > > >   dwarf_loader: support btf_type_tag attribute
> > > >   btf_encoder: support btf_type_tag attribute
> > > >
> > >
> > > I thought that v1 was already applied, but either way LGTM. I'm not
> >
> > To the next branch, and the libbpf pahole CI is failing, since a few
> > days, can you please take a look?
> 
> We've had Clang regression which Yonghong fixed very quickly, but then
> we were blocked on Clang nightly builds being broken for days. Seems
> like we got a new Clang today, so hopefully libbpf CI will be back to
> green again.

It is back to green, so I moved next to master, i.e. this series is now
on master.

- Arnaldo
 
> >
> > > super familiar with the DWARF loader parts, so I mostly just read it
> > > very superficially :)
> >
> > I replaced the patches that changed, re-added the S-o-B for Yonghong and
> > tested it with llvm-project HEAD.
> >
> > > Acked-by: Andrii Nakryiko <andrii@kernel.org>
> >
> > Adding it to the csets.
> >
> > Thanks!
> >
> > - Arnaldo
> >
> > >
> > > >  btf_encoder.c  |   7 +++
> > > >  dutil.h        |   4 ++
> > > >  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
> > > >  dwarves.h      |  38 +++++++++++++-
> > > >  lib/bpf        |   2 +-
> > > >  pahole.c       |   8 +++
> > > >  6 files changed, 190 insertions(+), 9 deletions(-)
> > > >
> > > > --
> > > > 2.30.2
> > > >
> >
> > --
> >
> > - Arnaldo

-- 

- Arnaldo

Re: [PATCH dwarves v2 0/4] btf: support btf_type_tag attribute

[Yonghong Song]

On 11/25/21 1:33 PM, Arnaldo Carvalho de Melo wrote:
> Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:
>> On Tue, Nov 23, 2021 at 3:36 PM Arnaldo Carvalho de Melo
>> <acme@kernel.org> wrote:
>>>
>>> Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:
>>>> On Mon, Nov 22, 2021 at 8:56 PM Yonghong Song <yhs@fb.com> wrote:
>>>>>
>>>>> btf_type_tag is a new llvm type attribute which is used similar
>>>>> to kernel __user/__rcu attributes. The format of btf_type_tag looks like
>>>>>    __attribute__((btf_type_tag("tag1")))
>>>>> For the case where the attribute applied to a pointee like
>>>>>    #define __tag1 __attribute__((btf_type_tag("tag1")))
>>>>>    #define __tag2 __attribute__((btf_type_tag("tag2")))
>>>>>    int __tag1 * __tag1 __tag2 *g;
>>>>> the information will be encoded in dwarf.
>>>>>
>>>>> In BTF, the attribute is encoded as a new kind
>>>>> BTF_KIND_TYPE_TAG and latest bpf-next supports it.
>>>>>
>>>>> The patch added support in pahole, specifically
>>>>> converts llvm dwarf btf_type_tag attributes to
>>>>> BTF types. Please see individual patches for details.
>>>>>
>>>>> Changelog:
>>>>>    v1 -> v2:
>>>>>       - reorg an if condition to reduce nesting level.
>>>>>       - add more comments to explain how to chain type tag types.
>>>>>
>>>>> Yonghong Song (4):
>>>>>    libbpf: sync with latest libbpf repo
>>>>>    dutil: move DW_TAG_LLVM_annotation definition to dutil.h
>>>>>    dwarf_loader: support btf_type_tag attribute
>>>>>    btf_encoder: support btf_type_tag attribute
>>>>>
>>>>
>>>> I thought that v1 was already applied, but either way LGTM. I'm not
>>>
>>> To the next branch, and the libbpf pahole CI is failing, since a few
>>> days, can you please take a look?
>>
>> We've had Clang regression which Yonghong fixed very quickly, but then
>> we were blocked on Clang nightly builds being broken for days. Seems
>> like we got a new Clang today, so hopefully libbpf CI will be back to
>> green again.
> 
> It is back to green, so I moved next to master, i.e. this series is now
> on master.

Arnaldo,

Is it possible that we could cut a pahole release in the near future?
My kernel using btf_type_tag patch ([1]) needs a pahole version check. 
In old pahole versions, a warning about pointer type subtags not 
supported will be emitted and btf_type_tag attributes (as pointer type 
subtags) will be ignored.

   [1] https://lore.kernel.org/bpf/20211118184810.1846996-1-yhs@fb.com/

Thanks!

> 
> - Arnaldo
>   
>>>
>>>> super familiar with the DWARF loader parts, so I mostly just read it
>>>> very superficially :)
>>>
>>> I replaced the patches that changed, re-added the S-o-B for Yonghong and
>>> tested it with llvm-project HEAD.
>>>
>>>> Acked-by: Andrii Nakryiko <andrii@kernel.org>
>>>
>>> Adding it to the csets.
>>>
>>> Thanks!
>>>
>>> - Arnaldo
>>>
>>>>
>>>>>   btf_encoder.c  |   7 +++
>>>>>   dutil.h        |   4 ++
>>>>>   dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
>>>>>   dwarves.h      |  38 +++++++++++++-
>>>>>   lib/bpf        |   2 +-
>>>>>   pahole.c       |   8 +++
>>>>>   6 files changed, 190 insertions(+), 9 deletions(-)
>>>>>
>>>>> --
>>>>> 2.30.2
>>>>>
>>>
>>> --
>>>
>>> - Arnaldo
>