From: Masami Hiramatsu <mhiramat@kernel.org>
To: Andy Lutomirski <luto@kernel.org>
Cc: Oleg Nesterov <oleg@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
LKML <linux-kernel@vger.kernel.org>,
Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
"David S. Miller" <davem@davemloft.net>, X86 ML <x86@kernel.org>,
Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: Why do kprobes and uprobes singlestep?
Date: Thu, 25 Feb 2021 18:11:14 +0900 [thread overview]
Message-ID: <20210225181114.e8e92e2fc2a204219b1bd28d@kernel.org> (raw)
In-Reply-To: <CALCETrVUBd5CuAh5TRTFqbCE2mYCiBvqrPouTicC0pyO7A6GWw@mail.gmail.com>
On Wed, 24 Feb 2021 22:03:12 -0800
Andy Lutomirski <luto@kernel.org> wrote:
> On Wed, Feb 24, 2021 at 6:22 PM Masami Hiramatsu <mhiramat@kernel.org> wrote:
> >
> > On Wed, 24 Feb 2021 11:45:10 -0800
> > Andy Lutomirski <luto@kernel.org> wrote:
> >
> > > On Tue, Feb 23, 2021 at 5:18 PM Masami Hiramatsu <mhiramat@kernel.org> wrote:
> > > >
> > > > On Tue, 23 Feb 2021 15:24:19 -0800
> > > > Andy Lutomirski <luto@kernel.org> wrote:
> > > >
> > > > > A while back, I let myself be convinced that kprobes genuinely need to
> > > > > single-step the kernel on occasion, and I decided that this sucked but
> > > > > I could live with it. it would, however, be Really Really Nice (tm)
> > > > > if we could have a rule that anyone running x86 Linux who single-steps
> > > > > the kernel (e.g. kgdb and nothing else) gets to keep all the pieces
> > > > > when the system falls apart around them. Specifically, if we don't
> > > > > allow kernel single-stepping and if we suitably limit kernel
> > > > > instruction breakpoints (the latter isn't actually a major problem),
> > > > > then we don't really really need to use IRET to return to the kernel,
> > > > > and that means we can avoid some massive NMI nastiness.
> > > >
> > > > Would you mean using "pop regs + popf + ret" instead of IRET after
> > > > int3 handled for avoiding IRET releasing the NMI mask? Yeah, it is
> > > > possible. I don't complain about that.
> > >
> > > Yes, more or less.
> > >
> > > >
> > > > However, what is the relationship between the IRET and single-stepping?
> > > > I think we can do same thing in do_debug...
> > >
> > > Because there is no way to single-step without using IRET. POPF; RET
> > > will trap after RET and you won't make forward progress.
> >
> > Ah, indeed. "POPF; RET" is not atomically exceute.
> >
> > > > > But I was contemplating the code, and I'm no longer convinced.
> > > > > Uprobes seem to single-step user code for no discernable reason.
> > > > > (They want to trap after executing an out of line instruction, AFAICT.
> > > > > Surely INT3 or even CALL after the out-of-line insn would work as well
> > > > > or better.) Why does kprobe single-step? I spend a while staring at
> > > > > the code, and it was entirely unclear to me what the purpose of the
> > > > > single-step is.
> > > >
> > > > For kprobes, there are 2 major reasons for (still relaying on) single stepping.
> > > > One is to provide post_handler, another is executing the original code,
> > > > which is replaced by int3, without modifying code nor emulation.
> > >
> > > I don't follow. Suppose we execute out of line. If we originally have:
> > >
> > > INSN
> > >
> > > we replace it with:
> > >
> > > INT3
> > >
> > > and we have, out of line:
> > >
> > > INSN [but with displacement modified if it's RIP-relative]
> > >
> > > right now, we single-step the out of line copy. But couldn't we instead do:
> > >
> > > INSN [but with displacement modified if it's RIP-relative]
> > > INT3
> >
> > If the INSN is "jmp +127", it will skip the INT3. So those instructions
> > must be identified and emulated. We did it already in the arm64 (see commit
> > 7ee31a3aa8f4 ("arm64: kprobes: Use BRK instead of single-step when executing
> > instructions out-of-line")), because arm64 already emulated the branch
> > instructions. I have to check x86 insns can be emulated without side-effects.
>
> Off the top of my head:
>
> JMP changes RIP but has no other side effects. Jcc is the same except
> that the condition needs checking, which would be a bit tedious.
>
> CALL changes RIP and does a push but has no other side effects. We
> have special infrastructure to emulate CALL from int3 context:
> int3_emulate_call().
Yeah, I remember that a gap was introduced for int3_emulate_call().
These helps me to implement emulation.
>
> RET pops and changes RIP. No other side effects.
>
> RET imm is rare. I don't think it occurs in the kernel at all.
>
> LRET is rare. I don't think kprobe needs to support it.
>
> JMP FAR and CALL FAR are rare. I see no reason to support them.
I see those are rare, but supporting those is not hard.
>
> IRET is rare, and trying to kprobe it seems likely to cause a
> disaster, although it's within the realm of possibility that the IRET
> in sync_core() could work.
Agreed. Iret should not be probed.
> > > or even
> > >
> > > INSN [but with displacement modified if it's RIP-relative]
> > > JMP kprobe_post_handler
> >
> > This needs a sequence of push-regs etc. ;)
> >
> > >
> > > and avoid single-stepping?
> > >
> > > I guess I see the point for CALL, JMP and RET, but it seems like we
> > > could emulate those cases instead fairly easily.
> >
> > OK, let's try to do it. I think it should be possible because even in the
> > current code, resume fixup code (adjust IP register) works only for a few
> > groups of instructions.
>
> I suspect that emulating them would give a nice performance boost,
> too. Single-stepping is very slow on x86.
Yeah, that's same on arm64. Jean reported eliminating single-step
gained the performance.
>
> I should let you know, though, that I might have found a sneaky
> alternative solution to handling NMIs, so this is a bit lower priority
> from my perspective than I thought it was. I'm not quite 100%
> convinced my idea works, but I'll play with it.
Does that involve kprobes? Anyway, I'll try to remove single-step by
emulation and int3.
Thank you,
>
> --Andy
>
> >
> > Thank you,
> >
> > --
> > Masami Hiramatsu <mhiramat@kernel.org>
--
Masami Hiramatsu <mhiramat@kernel.org>
next prev parent reply other threads:[~2021-02-25 9:15 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-23 23:24 Why do kprobes and uprobes singlestep? Andy Lutomirski
2021-02-24 1:17 ` Masami Hiramatsu
2021-02-24 19:45 ` Andy Lutomirski
2021-02-25 2:22 ` Masami Hiramatsu
2021-02-25 6:03 ` Andy Lutomirski
2021-02-25 9:11 ` Masami Hiramatsu [this message]
2021-03-01 14:08 ` [RFC PATCH 0/1] x86/kprobes: Remoev single-step trap from x86 kprobes Masami Hiramatsu
2021-03-01 14:08 ` [RFC PATCH 1/1] x86/kprobes: Use int3 instead of debug trap for single-step Masami Hiramatsu
2021-03-02 8:06 ` Peter Zijlstra
2021-03-02 8:38 ` Peter Zijlstra
2021-03-02 8:41 ` Peter Zijlstra
2021-03-02 8:54 ` Peter Zijlstra
2021-03-02 12:51 ` Masami Hiramatsu
2021-03-02 13:58 ` Peter Zijlstra
2021-03-02 15:25 ` [PATCH -tip 0/3] x86/kprobes: Remoev single-step trap from x86 kprobes Masami Hiramatsu
2021-03-02 15:25 ` [PATCH -tip 1/3] x86/kprobes: Retrieve correct opcode for group instruction Masami Hiramatsu
2021-03-23 15:15 ` [tip: x86/core] " tip-bot2 for Masami Hiramatsu
2021-03-02 15:25 ` [PATCH -tip 2/3] x86/kprobes: Identify far indirect JMP correctly Masami Hiramatsu
2021-03-23 15:15 ` [tip: x86/core] " tip-bot2 for Masami Hiramatsu
2021-03-02 15:25 ` [PATCH -tip 3/3] x86/kprobes: Use int3 instead of debug trap for single-step Masami Hiramatsu
2021-03-23 15:15 ` [tip: x86/core] " tip-bot2 for Masami Hiramatsu
2021-03-17 14:55 ` [PATCH -tip 0/3] x86/kprobes: Remoev single-step trap from x86 kprobes Masami Hiramatsu
2021-03-17 16:26 ` Peter Zijlstra
2021-03-17 17:45 ` Andy Lutomirski
2021-02-25 9:59 ` Why do kprobes and uprobes singlestep? Peter Zijlstra
2021-03-01 16:51 ` Oleg Nesterov
2021-03-02 1:36 ` Andy Lutomirski
2021-03-02 20:24 ` Alexei Starovoitov
2021-03-02 21:02 ` Andy Lutomirski
2021-03-03 1:22 ` Alexei Starovoitov
2021-03-03 1:46 ` Andy Lutomirski
2021-03-03 2:18 ` Alexei Starovoitov
2021-03-03 13:27 ` Oleg Nesterov
2021-03-03 18:11 ` Daniel Xu
2021-03-03 19:14 ` Andy Lutomirski
2021-03-02 20:25 ` Oleg Nesterov
2021-03-02 20:35 ` Andy Lutomirski
2021-03-02 20:28 ` Oleg Nesterov
2021-03-02 2:22 ` Masami Hiramatsu
2021-03-02 2:48 ` Andy Lutomirski
2021-03-02 20:31 ` Oleg Nesterov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210225181114.e8e92e2fc2a204219b1bd28d@kernel.org \
--to=mhiramat@kernel.org \
--cc=andrew.cooper3@citrix.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).