update on llvm BTF support

[Yonghong Song <yhs@fb.com>]

Hi,

This is an update of llvm BTF support.

The patches corresponding to 
https://reviews.llvm.org/{D55755,D55756,D55752} have been accepted and 
merged.
The llvm BTF support will be in llvm 8.0 whose release
time is around next March.

So with latest llvm/clang trunk, add -g to your compilation
flag, .BTF and .BTF.ext will be generated.
If you are curious about encoding, -S will produce an
assembly code including .BTF/.BTF.ext.

Thanks!

Yonghong

On 12/17/18 2:45 PM, Yonghong Song wrote:
> Hi,
> 
> As many of you already knew, latest bpf-next/net-next
> kernel added supports for func_info and line_info.
>     https://www.mail-archive.com/netdev@vger.kernel.org/msg255580.html
>     https://www.mail-archive.com/netdev@vger.kernel.org/msg257914.html
>     https://www.mail-archive.com/netdev@vger.kernel.org/msg272974.html
> 
> The func info and line info enables better ksym, correlation of
> xlated/jited code with source code, and better source-annotated
> verifier log. perf will be enhanced later to pick up
> such debug info so better information can be displayed for hot
> bpf programs.
> 
> Patch https://reviews.llvm.org/D53736
> =====================================
> 
> Currently, the func info and line info are generated by LLVM
> unmerged patch https://reviews.llvm.org/D53736. As stated in
> the commit message of D53726, the main advantage of implementing
> in LLVM is:
>     . better integration/deployment as no extra tools are needed.
>     . bpf JIT based compilation (like bcc, bpftrace, etc.) can get
>       BTF without much extra effort.
>     . BTF line_info needs selective source codes, which can be
>       easily retrieved when inside the compiler.
> 
> Not in the above list is the future work related to
> compile once and run everywhere (https://lwn.net/Articles/773198/),
> which needs strong type. Such strong type is only available
> in llvm IR. So implementation will have to be in llvm as well.
> 
> The llvm patch D53736 has been submitted on Oct 25, 2018.
> The patch implemented BTF as a first class debug info, meaning
> in a similar way to that Dwarf or CodeView debug info is generated.
> The implementation is mainly placed under
>     llvm/lib/CodeGen/AsmPrinter
> directory. Since BTF currently is only used by BPF, cmake
> file is changed to allow compilation of main BTFDebug.c only
> if BPF is in the llvm build target list. Otherwise, a
> stub implementation is used.
> 
> After initial review by Adrian Prantl where all comments
> are addressed, the patch became dormand since Nov 16, except some
> minor stylish review by Fangrui Song.
> 
> Patch https://reviews.llvm.org/{D55755,D55756,D55752}
> =====================================================
> 
> During review of D53736, on Nov 16, I commented it was possible
> to move main file BTFDebug.cpp to lib/Target/BPF
> directory if moving some DebugInfo private headers
> to llvm:include/llvm/CodeGen directory. But I never
> got an answer whether this is a good idea or not.
> 
> Since review has dragged so long, to help facilitate discussion,
> last weekend, I went ahead and implemented this scheme
> with three patches in phabricator:
>     https://reviews.llvm.org/D55755: refactor to move three DebugInfo
>        private files into llvm:include/llvm/CodeGen directory.
>     https://reviews.llvm.org/D55756: expose some private AsmPrinter
>        members to protected so target extended AsmPrinter can use
>        it to add their own DebugInfo handlers.
>     https://reviews.llvm.org/D55752
>        everything else is inside BPF directory.
> 
> This approach kept BTF completely under lib/Target/BPF
> if patches D55755 and D55756 are implemented to enable
> target specific debug info handlers.
> 
> Recap of two Approaches
> =======================
> 
> To recap,
>     https://reviews.llvm.org/D53736:
>        everything in the llvm core, BTF is a "first-class" debug info.
>        lib/Target/BPF unchanged.
>     https://reviews.llvm.org/{D55755,D55756,D55752}:
>        Some changes related DebugInfo to allow target specific
>        debug info handlers.
>        Major changes will be under /lib/Target/BPF.
> At this point, it is not clear which approach will be accepted
> by the community.
> 
> Going Forward
> =============
> 
> We will continue to engage with LLVM community for integrating
> BTF generation. Any help we can get from you should be helpful
> and will be really appreciated.
> 
> Thanks,
> 
> Yonghong
>