Re: Why do kprobes and uprobes singlestep?

From: Masami Hiramatsu <mhiramat@kernel.org>
To: Andy Lutomirski <luto@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
	"David S. Miller" <davem@davemloft.net>, X86 ML <x86@kernel.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Why do kprobes and uprobes singlestep?
Date: Wed, 24 Feb 2021 10:17:56 +0900	[thread overview]
Message-ID: <20210224101756.bbdf95b9b6dfc982bff21324@kernel.org> (raw)
In-Reply-To: <CALCETrXzXv-V3A3SpN_Pdj_PNG8Gw0AVsZD7+VO-q_xCAu2T2A@mail.gmail.com>

On Tue, 23 Feb 2021 15:24:19 -0800
Andy Lutomirski <luto@kernel.org> wrote:

> A while back, I let myself be convinced that kprobes genuinely need to
> single-step the kernel on occasion, and I decided that this sucked but
> I could live with it.  it would, however, be Really Really Nice (tm)
> if we could have a rule that anyone running x86 Linux who single-steps
> the kernel (e.g. kgdb and nothing else) gets to keep all the pieces
> when the system falls apart around them.  Specifically, if we don't
> allow kernel single-stepping and if we suitably limit kernel
> instruction breakpoints (the latter isn't actually a major problem),
> then we don't really really need to use IRET to return to the kernel,
> and that means we can avoid some massive NMI nastiness.

Would you mean using "pop regs + popf + ret" instead of IRET after
int3 handled for avoiding IRET releasing the NMI mask? Yeah, it is
possible. I don't complain about that.

However, what is the relationship between the IRET and single-stepping?
I think we can do same thing in do_debug...

> But I was contemplating the code, and I'm no longer convinced.
> Uprobes seem to single-step user code for no discernable reason.
> (They want to trap after executing an out of line instruction, AFAICT.
> Surely INT3 or even CALL after the out-of-line insn would work as well
> or better.)  Why does kprobe single-step?  I spend a while staring at
> the code, and it was entirely unclear to me what the purpose of the
> single-step is.

For kprobes, there are 2 major reasons for (still relaying on) single stepping.
One is to provide post_handler, another is executing the original code,
which is replaced by int3, without modifying code nor emulation.
Indeed, most of the instructions actually not depends on the ip register,
in that case (and user doesn't set post_handler), kprobe already skips
single stepping (a.k.a. kprobe booster, jump back to the kernel code after
executing out-of-line instruction.)
However, since some instructions, e.g. jump, call and ret, changes the ip
register (and stack), we have to do a fixup afterwards. 

But yes, it is possible to emulate, as same as arm/arm64 does. I just
concern about side-effects of the emulation, need to be carefully
implemented.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>