Re: [PATCH bpf v2] bpf, x64: implement retpoline for tail call

[Alexei Starovoitov <alexei.starovoitov@gmail.com>]

On Thu, Feb 22, 2018 at 03:12:53PM +0100, Daniel Borkmann wrote:
> Implement a retpoline [0] for the BPF tail call JIT'ing that converts
> the indirect jump via jmp %rax that is used to make the long jump into
> another JITed BPF image. Since this is subject to speculative execution,
> we need to control the transient instruction sequence here as well
> when CONFIG_RETPOLINE is set, and direct it into a pause + lfence loop.
> The latter aligns also with what gcc / clang emits (e.g. [1]).
> 
> JIT dump after patch:
> 
>   # bpftool p d x i 1
>    0: (18) r2 = map[id:1]
>    2: (b7) r3 = 0
>    3: (85) call bpf_tail_call#12
>    4: (b7) r0 = 2
>    5: (95) exit
> 
> With CONFIG_RETPOLINE:
> 
>   # bpftool p d j i 1
>   [...]
>   33:	cmp    %edx,0x24(%rsi)
>   36:	jbe    0x0000000000000072  |*
>   38:	mov    0x24(%rbp),%eax
>   3e:	cmp    $0x20,%eax
>   41:	ja     0x0000000000000072  |
>   43:	add    $0x1,%eax
>   46:	mov    %eax,0x24(%rbp)
>   4c:	mov    0x90(%rsi,%rdx,8),%rax
>   54:	test   %rax,%rax
>   57:	je     0x0000000000000072  |
>   59:	mov    0x28(%rax),%rax
>   5d:	add    $0x25,%rax
>   61:	callq  0x000000000000006d  |+
>   66:	pause                      |
>   68:	lfence                     |
>   6b:	jmp    0x0000000000000066  |
>   6d:	mov    %rax,(%rsp)         |
>   71:	retq                       |
>   72:	mov    $0x2,%eax
>   [...]
> 
>   * relative fall-through jumps in error case
>   + retpoline for indirect jump
> 
> Without CONFIG_RETPOLINE:
> 
>   # bpftool p d j i 1
>   [...]
>   33:	cmp    %edx,0x24(%rsi)
>   36:	jbe    0x0000000000000063  |*
>   38:	mov    0x24(%rbp),%eax
>   3e:	cmp    $0x20,%eax
>   41:	ja     0x0000000000000063  |
>   43:	add    $0x1,%eax
>   46:	mov    %eax,0x24(%rbp)
>   4c:	mov    0x90(%rsi,%rdx,8),%rax
>   54:	test   %rax,%rax
>   57:	je     0x0000000000000063  |
>   59:	mov    0x28(%rax),%rax
>   5d:	add    $0x25,%rax
>   61:	jmpq   *%rax               |-
>   63:	mov    $0x2,%eax
>   [...]
> 
>   * relative fall-through jumps in error case
>   - plain indirect jump as before
> 
>   [0] https://support.google.com/faqs/answer/7625886
>   [1] https://github.com/gcc-mirror/gcc/commit/a31e654fa107be968b802786d747e962c2fcdb2b
> 
> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
> ---
>  v1 -> v2:
>   - Moved into nospec-branch.h as suggested by Eric, thanks!

Applied to bpf tree, Thanks Daniel.