Is is possible to get the function calling stack in an fentry bpf program?

Is is possible to get the function calling stack in an fentry bpf program?

[刘畅]

Hi all

I attached an fentry eBPF program to a kernel function, i.e., tcp_transmit_skb(). I want to implement different logic in the bpf program for different calling stack cases, e.g., __tcp_retransmit_skb()->tcp_transmit_skb() and tcp_write_xmit()->tcp_transmit_skb(). I know that I can access stack traces using the bpf_get_stack() helper function. However, in the fentry eBPF program, I don't know the value of the RSP and RBP register, which means I can not locate the return address even if I can get the stack traces. I want to know if there's any way that I can get the return address and thus get the function calling stack in an fentry bpf program. 

I'd be appreciate if you can help me.

Chang Liu
Tsinghua University, China

Re: Is is possible to get the function calling stack in an fentry bpf program?

[Alan Maguire]

On 19/09/2023 14:55, 刘畅 wrote:
> Hi all
> 
> I attached an fentry eBPF program to a kernel function, i.e., tcp_transmit_skb(). I want to implement different logic in the bpf program for different calling stack cases, e.g., __tcp_retransmit_skb()->tcp_transmit_skb() and tcp_write_xmit()->tcp_transmit_skb(). I know that I can access stack traces using the bpf_get_stack() helper function. However, in the fentry eBPF program, I don't know the value of the RSP and RBP register, which means I can not locate the return address even if I can get the stack traces. I want to know if there's any way that I can get the return address and thus get the function calling stack in an fentry bpf program. 
> 
> I'd be appreciate if you can help me.
>

Note that fentry programs can access the ctx pointer; it's just that the
BPF_PROG() macro hides it.  For example, see

tools/testing/selftests/bpf/progs/test_bpf_cookie.c

...where we have:

SEC("fentry/bpf_fentry_test1")
int BPF_PROG(fentry_test1, int a)
{
        update(ctx, &fentry_res);
        return 0;
}

SEC("fexit/bpf_fentry_test1")
int BPF_PROG(fexit_test1, int a, int ret)
{
        update(ctx, &fexit_res);
        return 0;
}

Once you have ctx, you should (I think) be able to call bpf_get_stack().
If you've got a limited number of function callers you could probably
search for their addresses in the stack data returned without having to
parse stack frame boundaries.

Another less tricky option though is to trace the calling functions too,
and use a hashmap or similar to map from skb to caller in the calling
functions. Then when you reach the function of interest you can check
the map to see which caller added the map entry for that skb.

Alan

> Chang Liu
> Tsinghua University, China

Re: Is is possible to get the function calling stack in an fentry bpf program?

[Kui-Feng Lee]

On 9/19/23 06:55, 刘畅 wrote:
> Hi all
> 
> I attached an fentry eBPF program to a kernel function, i.e., tcp_transmit_skb(). I want to implement different logic in the bpf program for different calling stack cases, e.g., __tcp_retransmit_skb()->tcp_transmit_skb() and tcp_write_xmit()->tcp_transmit_skb(). I know that I can access stack traces using the bpf_get_stack() helper function. However, in the fentry eBPF program, I don't know the value of the RSP and RBP register, which means I can not locate the return address even if I can get the stack traces. I want to know if there's any way that I can get the return address and thus get the function calling stack in an fentry bpf program.
> 
> I'd be appreciate if you can help me.
> 
> Chang Liu
> Tsinghua University, China

Once you get stack returned by bpf_get_stack(), it is an array of
addresses. For example,

  __u64 buf[256];
  bpf_get_stack(ctx, buf, 256, 0);

buf[0], buf[1], ... will be addresses of caller sites from most inner.

Re: Re: Is is possible to get the function calling stack in an fentry bpf program?

[刘畅]

Thank you Kui-Feng and Alan. I misunderstood the usage of bpf_stack_get(). I thought it would retrieve all data from the stack, including local variables. Now, I can get the following calling stack in my fentry bpf program:

ffffffffc07c9090 (bpf_prog_12dc2796861890d0_bpf_fentry___tcp_transmit_skb)
       |
ffffffffc07c9090 (bpf_prog_12dc2796861890d0_bpf_fentry___tcp_transmit_skb)
       |
ffffffffc465c000 (bpf_trampoline_6442564494_0)
       |
ffffffff85dd15a5 (__tcp_transmit_skb)
       |
ffffffff85dd15a5 (__tcp_push_pending_frames)
       |
ffffffff85dba8c9 (tcp_push)
       |
    ......

But the first two addresses look quite strange (bpf_fentry___tcp_transmit_skb is the name of my fentry bpf program), they are the same and look like they belong to the attached fentry bpf program. I don't know why this happens. Regardless, I can get the function that calls __tcp_transmit_skb() now.

> -----Original Messages-----
> From: "Kui-Feng Lee" <sinquersw@gmail.com>
> Sent Time: 2023-09-20 00:22:11 (Wednesday)
> To: "刘畅" <chang-liu22@mails.tsinghua.edu.cn>, bpf@vger.kernel.org
> Cc: 
> Subject: Re: Is is possible to get the function calling stack in an fentry bpf program?
> 
> 
> 
> On 9/19/23 06:55, 刘畅 wrote:
> > Hi all
> > 
> > I attached an fentry eBPF program to a kernel function, i.e., tcp_transmit_skb(). I want to implement different logic in the bpf program for different calling stack cases, e.g., __tcp_retransmit_skb()->tcp_transmit_skb() and tcp_write_xmit()->tcp_transmit_skb(). I know that I can access stack traces using the bpf_get_stack() helper function. However, in the fentry eBPF program, I don't know the value of the RSP and RBP register, which means I can not locate the return address even if I can get the stack traces. I want to know if there's any way that I can get the return address and thus get the function calling stack in an fentry bpf program.
> > 
> > I'd be appreciate if you can help me.
> > 
> > Chang Liu
> > Tsinghua University, China
> 
> Once you get stack returned by bpf_get_stack(), it is an array of
> addresses. For example,
> 
>   __u64 buf[256];
>   bpf_get_stack(ctx, buf, 256, 0);
> 
> buf[0], buf[1], ... will be addresses of caller sites from most inner.