Re: [PATCH bpf-next] docs/bpf: add llvm_reloc.rst to explain llvm bpf relocations

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Mon, May 24, 2021 at 11:01 AM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 5/24/21 10:23 AM, Andrii Nakryiko wrote:
> > On Sat, May 22, 2021 at 9:39 AM Yonghong Song <yhs@fb.com> wrote:
> >>
> >> LLVM upstream commit https://reviews.llvm.org/D102712
> >> made some changes to bpf relocations to make them
> >> llvm linker lld friendly. The scope of
> >> existing relocations R_BPF_64_{64,32} is narrowed
> >> and new relocations R_BPF_64_{ABS32,ABS64,NODYLD32}
> >> are introduced.
> >>
> >> Let us add some documentation about llvm bpf
> >> relocations so people can understand how to resolve
> >> them properly in their respective tools.
> >>
> >> Cc: John Fastabend <john.fastabend@gmail.com>
> >> Cc: Lorenz Bauer <lmb@cloudflare.com>
> >> Signed-off-by: Yonghong Song <yhs@fb.com>
> >> ---
> >>   Documentation/bpf/index.rst      |   1 +
> >>   Documentation/bpf/llvm_reloc.rst | 168 +++++++++++++++++++++++++++++++
> >>   2 files changed, 169 insertions(+)
> >>   create mode 100644 Documentation/bpf/llvm_reloc.rst
> >>
> >> diff --git a/Documentation/bpf/index.rst b/Documentation/bpf/index.rst
> >> index a702f67dd45f..93e8cf12a6d4 100644
> >> --- a/Documentation/bpf/index.rst
> >> +++ b/Documentation/bpf/index.rst
> >> @@ -84,6 +84,7 @@ Other
> >>      :maxdepth: 1
> >>
> >>      ringbuf
> >> +   llvm_reloc
> >>
> >>   .. Links:
> >>   .. _networking-filter: ../networking/filter.rst
> >> diff --git a/Documentation/bpf/llvm_reloc.rst b/Documentation/bpf/llvm_reloc.rst
> >> new file mode 100644
> >> index 000000000000..bc62bce591b1
> >> --- /dev/null
> >> +++ b/Documentation/bpf/llvm_reloc.rst
> >> @@ -0,0 +1,168 @@
> >> +.. SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause)
> >> +
> >> +====================
> >> +BPF LLVM Relocations
> >> +====================
> >> +
> >> +This document describes LLVM BPF backend relocation types.
> >> +
> >> +Relocation Record
> >> +=================
> >> +
> >> +LLVM BPF backend records each relocation with the following 16-byte
> >> +ELF structure::
> >> +
> >> +  typedef struct
> >> +  {
> >> +    Elf64_Addr    r_offset;  // Offset from the beginning of section.
> >> +    Elf64_Xword   r_info;    // Relocation type and symbol index.
> >> +  } Elf64_Rel;
> >> +
> >> +For static function/variable references, the symbol often refers to
> >> +the section itself which has a value of 0. To identify actual static
> >> +function/variable, its section offset or some computation result
> >> +based on section offset is written to the original insn/data buffer,
> >> +which is called ``IA`` (implicit addend) below.  For global
> >> +function/variables, the symbol refers to actual global and the implicit
> >> +addend is 0.
> >> +
> >> +Different Relocation Types
> >> +==========================
> >> +
> >> +Six relocation types are supported. The following is an overview and
> >> +``S`` represents the value of the symbol in the symbol table::
> >> +
> >> +  Enum  ELF Reloc Type     Description      BitSize  Offset        Calculation
> >> +  0     R_BPF_NONE         None
> >> +  1     R_BPF_64_64        ld_imm64 insn    32       r_offset + 4  S + IA
> >
> > There are cases where we set all 64-bits of ld_imm64 (e.g., extern
> > ksym, global variables). Or those will be a different relocation now
> > (R_BPF_64_ABS64?). If not, I think BitSize 64 is more correct here.
>
> It is still R_BPF_64_64. In llvm, we have restriction that section
> offset must be <= UINT32_MAX, and that is why only 32bit is used
> to find the actual symbol in symbol table. 32bit permits 4GB section
> which should enough in practice for a bpf program.
>
> libbpf or tools can write to full 64bits of imm values of ld_imm64 insn.
>

Ok, sounds good.

> The name is a little bit misleading, but it has become part of ABI
> and lives in /usr/include/elf.h and we are not able to change it
> any more.
>
> >
> > Looking at LLVM diff I haven't found a test for global variables (at
> > least I didn't realize it was there), so double-checking here (and it
> > might be a good idea to have an explicit test for global variables?)
>
> We have llvm/test/CodeGen/BPF/reloc.ll and
> llvm/test/CodeGen/BPF/reloc-btf.ll covering R_BPF_64_ABS64. But I think
> I can enhance
> llvm/test/CodeGen/BPF/reloc-2.ll to cover an explicit global variable case.
>

Great, thanks.

> >
> >> +  2     R_BPF_64_ABS64     normal data      64       r_offset      S + IA
> >> +  3     R_BPF_64_ABS32     normal data      32       r_offset      S + IA
> >> +  4     R_BPF_64_NODYLD32  .BTF[.ext] data  32       r_offset      S + IA
> >> +  10    R_BPF_64_32        call insn        32       r_offset + 4  (S + IA) / 8 - 1
> >> +
> >> +For example, ``R_BPF_64_64`` relocation type is used for ``ld_imm64`` instruction.
> >> +The actual to-be-relocated data is stored at ``r_offset + 4`` and the read/write
> >> +data bitsize is 32 (4 bytes). The relocation can be resolved with
> >> +the symbol value plus implicit addend.
> >> +
> >> +In another case, ``R_BPF_64_ABS64`` relocation type is used for normal 64-bit data.
> >> +The actual to-be-relocated data is stored at ``r_offset`` and the read/write data
> >> +bitsize is 64 (8 bytes). The relocation can be resolved with
> >> +the symbol value plus implicit addend.
> >> +

[...]