Re: 5.15 ftrace bug when enabling function_graph

[Ivan Babrou <ivan@cloudflare.com>]

On Fri, Aug 5, 2022 at 3:25 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Fri, 5 Aug 2022 13:14:47 -0700
> Ivan Babrou <ivan@cloudflare.com> wrote:
>
> > On Fri, Aug 5, 2022 at 11:53 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > >
> > > On Fri, 5 Aug 2022 11:37:56 -0700
> > > Ivan Babrou <ivan@cloudflare.com> wrote:
> > >
> > > > root@foo:~# echo 'bpf_dispatcher_*_func' >
> > > > /sys/kernel/tracing/set_ftrace_notrace
> > > >
> > > > root@foo:~# trace-cmd record -p function_graph -l handle_mm_fault -P 3367417
> > > > plugin 'function_graph'
> > > > Hit Ctrl^C to stop recording
> > >
> > >
> > > > [1459709.204253] ------------[ ftrace bug ]------------
> > > > [1459709.204257] ftrace failed to modify
> > > > [1459709.204262] [<ffffffffac3ea960>] bpf_dispatcher_xdp_func+0x0/0x10
> > > > [1459709.204275]  actual:   ffffffe9:ffffff9b:76:ffffffb0:14
> > > > [1459709.204302] Setting ftrace call site to call ftrace function
> > > > [1459709.204305] ftrace record flags: 10000001
> > > > [1459709.204309]  (1)
> > > >                   expected tramp: ffffffffabc4bcb0
> > > > [1459709.204325] ------------[ cut here ]------------
> > >
> > > Did you reboot since the error happened? Because the command you showed
> > > should not have touched bpf_dispatcher_xdp_func.
> >
> > I did reboot, but I can reproduce if you have things for me to try.
>
> Let's see if this works manually.
>
> Reboot again.
>
> Make sure the above "ftrace bug" is not there.
>
> If not already mounted, as root:
>
>  # mount -t tracefs tracefs /sys/krenel/tracing
>  # cd /sys/kernel/tracing
>  # cat enabled_functions
>
> Let me know what is there.
>
>  # echo handle_mm_fault > set_ftrace_filter
>  # cat set_ftrace_filter
>
> to make sure that handle_mm_fault is listed.
>
>  # echo function_graph > current_tracer
>
> If it fails again, let me know. It should not, but if it does, it needs to
> be investigated.

It does not fail this way. It also doesn't fail when I run perf:

perf ftrace -N 'bpf_dispatcher_*_func' -G handle_mm_fault -a -- sleep 1

It does fail without "-N 'bpf_dispatcher_*_func'".

I suppose there's something about trace-cmd that makes it fail for me.