Re: How to disassemble a BPF program?

From: Jakub Sitnicki <jakub@cloudflare.com>
To: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>,
	bpf <bpf@vger.kernel.org>, Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>
Subject: Re: How to disassemble a BPF program?
Date: Mon, 25 Apr 2022 15:48:26 +0200	[thread overview]
Message-ID: <87tuah6ziv.fsf@cloudflare.com> (raw)
In-Reply-To: <CAEf4Bzbugg4dy_2J=cFKYYQEJx-irF-cRZvkkwCx4QQwXm5OpA@mail.gmail.com>

On Wed, Apr 20, 2022 at 09:48 AM -07, Andrii Nakryiko wrote:
> On Wed, Apr 20, 2022 at 4:38 AM Tetsuo Handa
> <penguin-kernel@i-love.sakura.ne.jp> wrote:
>>
>> Ping?
>>
>> Since how to fix this "current top five crasher" bug depends on how a kernel
>> socket is created via BPF program, this bug wants help from BPF developers.
>
> If the BPF program is loaded/verified successfully, the easiest way to
> go about this would be to prevent repro from proceeding right after
> successful validation (e.g, do scanf()) and then use bpftool to find
> that program's ID and dump disassembly while that program is in the
> kernel.
>
> $ sudo bpftool prog show
> ...
> 654439: cgroup_skb  tag 6deef7357e7b4530  gpl
>         loaded_at 2022-04-20T06:14:08-0700  uid 0
>         xlated 64B  jited 54B  memlock 4096B
>         pids systemd(1)
>
> $ sudo bpftool prog dump xlat id 654439
>    0: (bf) r6 = r1
>    1: (69) r7 = *(u16 *)(r6 +176)
>    2: (b4) w8 = 0
>    3: (44) w8 |= 2
>    4: (b7) r0 = 1
>    5: (55) if r8 != 0x2 goto pc+1
>    6: (b7) r0 = 0
>    7: (95) exit
>
> Hope that helps. I don't know any tool that allows to disassemble raw
> bytes into BPF assembly. Normally I use llvm-objdump to disassemble
> well-formed BPF ELF files. Not sure if you can wrange llvm-objdump to
> disassemble raw bytes without ELF file itself.

You can disassemble raw BPF binaries with GNU objdump, but the assembly
mnemonics are different:

$ sudo bpftool prog dump xlated id 77
   0: (bf) r6 = r1
   1: (69) r7 = *(u16 *)(r6 +176)
   2: (b4) w8 = 0
   3: (44) w8 |= 2
   4: (b7) r0 = 1
   5: (55) if r8 != 0x2 goto pc+1
   6: (b7) r0 = 0
   7: (95) exit
$ sudo bpftool prog dump xlated id 77 file prog.bin
$ sudo objdump -D -b binary -m bpf prog.bin

prog.bin:     file format binary

Disassembly of section .data:

0000000000000000 <.data>:
   0:   bf 16 00 00 00 00 00 00         mov %r6,%r1
   8:   69 67 b0 00 00 00 00 00         ldxh %r7,[%r6+0xb0]
  10:   b4 08 00 00 00 00 00 00         mov32 %r8,0
  18:   44 08 00 00 02 00 00 00         or32 %r8,2
  20:   b7 00 00 00 01 00 00 00         mov %r0,1
  28:   55 08 01 00 02 00 00 00         jne %r8,2,1
  30:   b7 00 00 00 00 00 00 00         mov %r0,0
  38:   95 00 00 00 00 00 00 00         exit
$