Re: [RFC PATCH bpf-next 0/3] bpf: add tracing for XDP programs using the BPF_PROG_TEST_RUN API

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Yonghong Song <yhs@fb.com>
Cc: Eelco Chaudron <echaudro@redhat.com>, bpf <bpf@vger.kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Network Development <netdev@vger.kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Andrii Nakryiko <andriin@fb.com>
Subject: Re: [RFC PATCH bpf-next 0/3] bpf: add tracing for XDP programs using the BPF_PROG_TEST_RUN API
Date: Sun, 19 Apr 2020 15:54:32 -0700	[thread overview]
Message-ID: <CAADnVQ+SCu97cF5Li6nBBCkshjF45U-nPEO5jO8DQrY5PqPqyg@mail.gmail.com> (raw)
In-Reply-To: <fefda00a-1a08-3a53-efbc-93c36292b77d@fb.com>

On Sun, Apr 19, 2020 at 12:02 AM Yonghong Song <yhs@fb.com> wrote:
>
>
>
> On 4/16/20 5:45 AM, Eelco Chaudron wrote:
> >
> >
> > On 23 Mar 2020, at 23:47, Yonghong Song wrote:
> >
> >> On 3/18/20 6:06 AM, Eelco Chaudron wrote:
> >>> I sent out this RFC to get an idea if the approach suggested here
> >>> would be something other people would also like to see. In addition,
> >>> this cover letter mentions some concerns and questions that need
> >>> answers before we can move to an acceptable implementation.
> >>>
> >>> This patch adds support for tracing eBPF XDP programs that get
> >>> executed using the __BPF_PROG_RUN syscall. This is done by switching
> >>> from JIT (if enabled) to executing the program using the interpreter
> >>> and record each executed instruction.

sorry for delay. I only noticed these patches after Yonghong replied.

I think hacking interpreter to pr_info after every instruction is too
much of a hack.
Not working with JIT-ed code is imo red flag for the approach as well.
When every insn is spamming the logs the only use case I can see
is to feed the test program with one packet and read thousand lines dump.
Even that is quite user unfriendly.

How about enabling kprobe in JITed code instead?
Then if you really need to trap and print regs for every instruction you can
still do so by placing kprobe on every JITed insn.
But in reality I think few kprobes in the prog will be enough
to debug the program and XDP prog may still process millions of packets
because your kprobe could be in error path and the user may want to
capture only specific things when it triggers.
kprobe bpf prog will execute in such case and it can capture necessary
state from xdp prog, from packet or from maps that xdp prog is using.
Some sort of bpf-gdb would be needed in user space.
Obviously people shouldn't be writing such kprob-bpf progs that debug
other bpf progs by hand. bpf-gdb should be able to generate them automatically.