Re: [Phishing Risk] [PATCH] kprobe: fix kretprobe stack backtrace

From: Masami Hiramatsu <mhiramat@kernel.org>
To: Muchun Song <songmuchun@bytedance.com>
Cc: Qiang Wang <wangqiang.wq.frank@bytedance.com>,
	"Naveen N . Rao" <naveen.n.rao@linux.ibm.com>,
	Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
	David Miller <davem@davemloft.net>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>,
	Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>,
	Yonghong Song <yhs@fb.com>,
	John Fastabend <john.fastabend@gmail.com>,
	kpsingh@kernel.org, LKML <linux-kernel@vger.kernel.org>,
	Networking <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
	Chengming Zhou <zhouchengming@bytedance.com>,
	Xiongchun duan <duanxiongchun@bytedance.com>
Subject: Re: [Phishing Risk] [PATCH] kprobe: fix kretprobe stack backtrace
Date: Mon, 28 Jun 2021 22:34:30 +0900	[thread overview]
Message-ID: <20210628223430.8f903ea25977f5568ec1df6e@kernel.org> (raw)
In-Reply-To: <CAMZfGtWPi4CuVOtmUpy2N9J_mvp+5=gSAFvqV1nmvDKP+CAvQA@mail.gmail.com>

On Fri, 25 Jun 2021 18:24:12 +0800
Muchun Song <songmuchun@bytedance.com> wrote:

> On Fri, Jun 25, 2021 at 4:49 PM Qiang Wang
> <wangqiang.wq.frank@bytedance.com> wrote:
> >
> > We found that we couldn't get the correct kernel stack from
> > kretprobe. For example:
> >
> > bpftrace -e 'kr:submit_bio {print(kstack)}'
> > Attaching 1 probe...
> >
> >         kretprobe_trampoline+0
> >
> >         kretprobe_trampoline+0
> >
> > The problem is caused by the wrong instruction register which
> > points to the address of kretprobe_trampoline in regs.
> > So we set the real return address in instruction register.
> > Finally, we tested and successfully fixed it.
> >
> > bpftrace -e 'kr:submit_bio {print(kstack)}'
> > Attaching 1 probe...
> >
> >         ext4_mpage_readpages+475
> >         read_pages+139
> >         page_cache_ra_unbounded+417
> >         filemap_get_pages+245
> >         filemap_read+169
> >         __kernel_read+327
> >         bprm_execve+648
> >         do_execveat_common.isra.39+409
> >         __x64_sys_execve+50
> >         do_syscall_64+54
> >         entry_SYSCALL_64_after_hwframe+68
> >
> > Reported-by: Chengming Zhou <zhouchengming@bytedance.com>
> > Signed-off-by: Qiang Wang <wangqiang.wq.frank@bytedance.com>
> 
> Seems like a bug. Maybe we should add a "Fixes" tag here.

No, that is not a bug in the kretprobes. If you carefully check
kretprobes provided the rp->addr as ip address. BPF just did not
use it.

Anyway, I already made a same patch in the below series.

https://lore.kernel.org/bpf/162400000592.506599.4695807810528866713.stgit@devnote2/

and you can see that the series is including below 3 patches for that change.

https://lore.kernel.org/bpf/162399997853.506599.13701157683968161733.stgit@devnote2/
https://lore.kernel.org/bpf/162399998747.506599.1115560529431673586.stgit@devnote2/
https://lore.kernel.org/bpf/162399999702.506599.16339931387573094059.stgit@devnote2/

Without these patches, this change will break other arch.

Thanks,

> 
> > ---
> >  kernel/kprobes.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> > index 745f08fdd..1130381ca 100644
> > --- a/kernel/kprobes.c
> > +++ b/kernel/kprobes.c
> > @@ -1899,6 +1899,9 @@ unsigned long __kretprobe_trampoline_handler(struct pt_regs *regs,
> >         current->kretprobe_instances.first = node->next;
> >         node->next = NULL;
> >
> > +       /* Kretprobe handler expects address is the real return address */
> > +       instruction_pointer_set(regs, (unsigned long)correct_ret_addr);
> > +
> >         /* Run them..  */
> >         while (first) {
> >                 ri = container_of(first, struct kretprobe_instance, llist);
> > --
> > 2.20.1
> >

-- 
Masami Hiramatsu <mhiramat@kernel.org>