getting bpf_tail_call to work with bpf function calls. Was: [RFC PATCH bpf-next 0/1] bpf, x64: optimize JIT prologue/epilogue generation

[Alexei Starovoitov <alexei.starovoitov@gmail.com>]

On Wed, May 13, 2020 at 01:58:55PM +0200, Maciej Fijalkowski wrote:
> 
> So to me, if we would like to get rid of maxing out stack space, then we
> would have to do some dancing for preserving the tail call counter - keep
> it in some unused register? Or epilogue would pop it from stack to some
> register and target program's prologue would push it to stack from that
> register (I am making this up probably). And rbp/rsp would need to be
> created/destroyed during the program-to-program transition that happens
> via tailcall. That would mean also more instructions.

How about the following:
The prologue will look like:
nop5
xor eax,eax  // two new bytes if bpf_tail_call() is used in this function
push rbp
mov rbp, rsp
sub rsp, rounded_stack_depth
push rax // zero init tail_call counter
variable number of push rbx,r13,r14,r15

Then bpf_tail_call will pop variable number rbx,..
and final 'pop rax'
Then 'add rsp, size_of_current_stack_frame'
jmp to next function and skip over 'nop5; xor eax,eax; push rpb; mov rbp, rsp'

This way new function will set its own stack size and will init tail call
counter with whatever value the parent had.

If next function doesn't use bpf_tail_call it won't have 'xor eax,eax'.
Instead it would need to have 'nop2' in there.
That's the only downside I see.
Any other ideas?