Re: [PATCH bpf-next v2 00/11] bpf: add support for new btf kind BTF_KIND_TAG

[Yonghong Song <yhs@fb.com>]

On 12/17/21 5:44 PM, Alexei Starovoitov wrote:
> On Fri, Dec 17, 2021 at 11:40:10AM +0100, Jose E. Marchesi wrote:
>>
>> 2) The need for DWARF to convey free-text tags on certain elements, such
>>     as members of struct types.
>>
>>     The motivation for this was originally the way the Linux kernel
>>     generates its BTF information, using pahole, using DWARF as a source.
>>     As we discussed in our last exchange on this topic, this is
>>     accidental, i.e. if the kernel switched to generate BTF directly from
>>     the compiler and the linker could merge/deduplicate BTF, there would
>>     be no need for using DWARF to act as the "unwilling conveyer" of this
>>     information.  There are additional benefits of this second approach.
>>     Thats why we didn't plan to add these extended DWARF DIEs to GCC.
>>
>>     However, it now seems that a DWARF consumer, the drgn project, would
>>     also benefit from having such a support in DWARF to distinguish
>>     between different kind of pointers.
> 
> drgn can use .percpu section in vmlinux for global percpu vars.
> For pointers the annotation is indeed necessary.
> 
>>     So it seems to me that now we have two use-cases for adding support
>>     for these free-text tags to DWARF, as a proper extension to the
>>     format, strictly unrelated to BTF, BPF or even the kernel, since:
>>     - This is not kernel specific.
>>     - This is not directly related to BTF.
>>     - This is not directly related to BPF.
> 
> __percpu annotation is kernel specific.
> __user and __rcu are kernel specific too.
> Only BPF and BTF can meaningfully consume all three.
> drgn can consume __percpu.
> 
> In that sense if GCC follows LLVM and emits compiler specific DWARF tag
> pahole can convert it to the same BTF regardless whether kernel
> was compiled with clang or gcc.
> drgn can consume dwarf generated by clang or gcc as well even when BTF
> is not there. That is the fastest way forward.
> 
> In that sense it would be nice to have common DWARF tag for pointer
> annotations, but it's not mandatory. The time is the most valuable asset.
> Implementing GCC specific DWARF tag doesn't require committee voting
> and the mailing list bikeshedding.
> 
>> 3) Addition of C-family language-level constructions to specify
>>     free-text tags on certain language elements, such as struct fields.
>>
>>     These are the attributes, or built-ins or whatever syntax.
>>
>>     Note that, strictly speaking:
>>     - This is orthogonal to both DWARF and BTF, and any other supported
>>       debugging format, which may or may not be expressive enough to
>>       convey the free-form text tag.
>>     - This is not specific to BPF.
>>
>>     Therefore I would avoid any reference to BTF or BPF in the attribute
>>     names.  Something like `__attribute__((btf_tag("arbitrary_str")))'
>>     makes very little sense to me; the attribute name ought to be more
>>     generic.
> 
> Let's agree to disagree.
> When BPF ISA was designed we didn't go to Intel, Arm, Mips, etc in order to
> come up with the best ISA that would JIT to those architectures the best
> possible way. Same thing with btf_tag. Today it is specific to BTF and BPF
> only. Hence it's called this way. Whenever actual users will appear that need
> free-text tags on a struct field then and only then will be the time to discuss
> generic tag name. Just because "free-text tag on a struct field" sounds generic
> it doesn't mean that it has any use case beyond what we're using it for in BPF
> land. It goes back to the point of coding now instead of talking about coding.
> If gcc wants to call it __attribute__((my_precious_gcc_tag("arbitrary_str")))
> go ahead and code it this way. The include/linux/compiler.h can accommodate it.

Just want to add a little bit context for this. In the beginning when we
proposed to add the attribute, we named as a generic name like 'tag' (or 
something like that). But eventually upstream suggested 'btf_tag' since
the use case we proposed is for bpf. At that point, we don't know
drgn use cases yet. Even with that, the use cases are still just for
linux kernel.

At that time, some *similar* use cases did came up, e.g., for
swift<->C++ conversion encoding ("tag name", "attribute info") for
attributes in the source code, will help a lot. But they will use a 
different "tag name" than btf_tag to differentiate.