Ptrace documentation, draft #1

Ptrace documentation, draft #1

[Denys Vlasenko]

Hi Tejun,

On Monday 09 May 2011 12:09, Tejun Heo wrote:
> Hmmm... I was thinking about writing a proper ptrace API doc with
> example programs under Documentation/.  It's userland visible API so
> it shouldn't change all that much and the amount of necessary
> documentation would be too much for comments.

Since I am not helping you much in kernel ptrace hacking,
I can at least help you with the docs.

Ptrace discussions repeatedly display a higher than average amount
of misunderstanding and confusion. New ptrace users and even people
who already worked with it are repeatedly confused by details
which are not documented anywhere and knowledge about which exists
mostly in the brains of strace/gdb/other_such_tools developers.

This document is meant as a brain dump of this knowledge.
It assumes that the reader has basic understanding what ptrace is.

Tejun, Oleg, Jan, please comment. If some paragraphs are unclear,
post your version of the whole paragraph(s).

I am especially interested in adding info on ptrace quirks which
I forgot or don't know yet - Jan, you must know quite a bit
of those from your gdb work.

Tejun, Oleg - "2. Linux kernel implementation" section is for you
to fill with a brain dump of kernel side of the story.

======================================================================
======================================================================
======================================================================

Ptrace API (ab)uses standard Unix parent/child signaling over waitpid.
An unfortunate effect of it is that resulting API is complex and has
subtle quirks. This document aims to describe these quirks.

It is split into two parts. First part focuses exclusively on
userspace-visible API and behavior. Second section describes kernel
internals of ptrace.



		1. Userspace API.

(Note to editors: in this section, do not use kernel concepts and terms
which are not observable through userspace API and user-visible
behavior. Use section 2 for that.)

Debugged processes (tracees) first need to be attached to the debugging
process (tracer). Attachment and subsequent commands are per-thread: in
multi-threaded process, every thread can be individually attached to a
(potentially different) tracer, or left not attached and thus not
debugged. Therefore, "tracee" always means "(one) thread", never "a
(possibly multi-threaded) process". Ptrace commands are always sent to
a specific tracee using ptrace(PTRACE_foo, pid, ...), where pid is a
TID of the corresponding Linux thread.

After attachment, each tracee can be in two states: running or stopped.

There are many kinds of states when tracee is stopped, and in ptrace
discussions they are often conflated. Therefore, it is important to use
precise terms.

In this document, any stopped state in which tracee is ready to accept
ptrace commands from the tracer is called ptrace-stop. Ptrace-stops can
be further subdivided into signal-delivery-stop, group-stop,
syscall-stop and so on. They are described in detail later.


	1.x Death under ptrace.

When a (possibly multi-threaded) process receives a killing signal (a
signal set to SIG_DFL and whose default action is to kill the process),
all threads exit. Tracees report their death to the tracer(s). This is
not a ptrace-stop (because tracer can't query tracee status such as
register contents, cannot restart tracee etc) but the notification
about this event is delivered through waitpid API similarly to
ptrace-stop.

Note that killing signal will first cause signal-delivery-stop (on one
tracee only), and only after it is injected by tracer (or after it was
dispatched to a thread which isn't traced), death from signal will
happen on ALL tracees within multi-threaded process.

SIGKILL operates similarly, with exceptions. No signal-delivery-stop is
generated for SIGKILL and therefore tracer can't suppress it. SIGKILL
kills even within syscalls (syscall-exit-stop is not generated prior to
death by SIGKILL). The net effect is that SIGKILL always kills the
process (all its threads), even if some threads of the process are
ptraced.

Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).

??? Does it kill only the tracee or the whole process? What is exit
    status? Will tracer see the death? Other tracers?

When tracee executes exit syscall, it reports its death to its tracer.
Other threads are not affected.

When any thread executes exit_group syscall, every tracee reports its
death to its tracer.

??? Is it true that *every* thread reports death?
    IIRC it isn't... one does (leader? what if there is no leader?)
    and the rest simply disappear.

Tracer cannot assume that ptrace-stopped tracee exists. There are many
scenarios when tracee may die while stopped (such as SIGKILL).
Therefore, tracer must always be prepared to handle ESRCH error on any
ptrace operation. Unfortunately, the same error is returned if tracee
exists but is not ptrace-stopped (for commands which require stopped
tracee). Tracer needs to keep track of stopped/running state, and
interpret ESRCH as "tracee died unexpectedly" only if it knows that
tracee has been observed to enter ptrace-stop.

??? is there a recommended usage of waitpid(WNOHANG) to check whether
    tracee is dead or alive?


	1.x Stopped states.

When running tracee enters ptrace-stop, it notifies its tracer using
waitpid API. Tracer should use waitpid family of syscalls to wait for
tracee to stop. Most of this document assumes that tracer waits with:

	pid = waitpid(pid_or_minus_1, &status, __WALL);

Ptrace-stopped tracees are reported as returns with pid > 0 and
WIFSTOPPED(status) == true.

??? any pitfalls with WNOHANG (I remember that there are bugs in this
    area)? effects of WSTOPPED, WEXITED, WCONTINUED bits? Are they ok?
    waitid usage? WNOWAIT?

	1.x.x Signal-delivery-stop

When (possibly multi-threaded) process receives any signal except
SIGKILL, kernel selects a thread which handles the signal (if signal is
generated with tgkill, thread selection is done by user). If selected
thread is traced, it enters signal-delivery-stop. By this point, signal
is not yet delivered to the process, and can be suppressed by tracer.
If tracer doesn't suppress the signal, it passes signal to tracee in
the next ptrace request. This is called "signal injection" and will be
described later. Note that if signal is blocked, signal-delivery-stop
doesn't happen until signal is unblocked, with the usual exception that
SIGSTOP can't be blocked.

Signal-delivery-stop is observed by tracer as waitpid returning with
WIFSTOPPED(status) == true, WSTOPSIG(status) == signal. If
WSTOPSIG(status) == SIGTRAP, this may be a different kind of
ptrace-stop - see syscall-stop section below for details. If
WSTOPSIG(status) == stopping signal, this may be a group-stop - see
below.

Kernel delivers an extra SIGTRAP to tracee after execve syscall
returns. This is an ordinary signal (similar to one generated by kill
-TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.

??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
    by looking at its siginfo?

Usually, tracer (for example, strace) would not want to show this extra
post-execve SIGTRAP signal to the user, and would suppress its delivery
to the tracee (if SIGTRAP is set to SIG_DFL, it is a killing signal).

	1.x.x Signal injection and suppression.

After signal-delivery-stop is observed by tracer, PTRACE_GETSIGINFO can
be used to retrieve corresponding siginfo_t structure.
PTRACE_SETSIGINFO may be used to modify it.

Tracer should restart tracee with
	ptrace(PTRACE_rest, pid, 0, sig)
call, where PTRACE_rest is one of the restarting ptrace ops. If sig is
0, then signal is not delivered and has no effect. Otherwise, signal
sig is delivered. This operation is called "signal injection", to
distinguish it from signal delivery which causes signal-delivery-stop.

Note that sig value may be different from WSTOPSIG(status) value -
tracer can cause a different signal to be injected. If before
restarting PTRACE_SETSIGINFO command was used to alter injected
signal's siginfo_t, si_signo field and sig parameter in restarting
command must match.

??? Are syscalls interrupted by signals which are suppressed by tracer?
    If yes, document it here

Note that restarting ptrace commands issued in ptrace-stops other than
signal-delivery-stop do NOT inject a signal, even if sig is nonzero. No
error is reported either. This is a cause of confusion among ptrace
users. One typical scenario is that tracer observes group-stop,
mistakes it for signal-delivery-stop, restarts tracee with
ptrace(PTRACE_rest, pid, 0, stopsig) with the intention of injecting
stopsig, but stopsig gets ignored and tracee continues to run.

??? TODO: there *are* some other ptrace-stops which can take sig.
    Document them.

SIGCONT signal has a side effect of waking up (all threads of)
group-stopped process. This side effect happens before
signal-delivery-stop. Tracer can't suppress this side-effect (it can
only suppress signal injection, which only causes SIGCONT handler to
not be executed in the tracee, if such handler is installed). In fact,
waking up from group-stop may be followed by signal-delivery-stop for
signal(s) *other than* SIGCONT, if they were pending when SIGCONT was
delivered. IOW: SIGCONT may be not the first signal observed by the
tracee after it was sent.

Stopping signals cause (all threads of) process to enter group-stop.
This side effect happens after signal injection, and therefore can be
suppressed by tracer.

	1.x.x Group-stop

When a (possibly multi-threaded) process receives a stopping signal,
all threads stop. If some threads are traced, they enter a group-stop.
Note that stopping signal will first cause signal-delivery-stop (on one
tracee only), and only after it is injected by tracer (or after it was
dispatched to a thread which isn't thraced), group-stop will be
initiated on ALL tracees within multi-threaded process. As usual, every
tracee reports its group-stop to corresponding tracer.

Group-stop is observed by tracer as waitpid returning with the
following result: WIFSTOPPED(status) is true, WSTOPSIG(status) is the
delivered signal. The same result is returned by some other classes of
ptrace-stops, therefore the recommended practice is to perform
	ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo)
call. The call can be avoided if signal number is not SIGSTOP, SIGTSTP,
SIGTTIN or SIGTTOU - only these four signals are stopping signals. If
tracer sees something else, it can't be group-stop. Otherwise, tracer
needs to call PTRACE_GETSIGINFO. If PTRACE_GETSIGINFO fails, then it is
a group-stop. If it succeeds, it's a signal-delivery-stop.

As of kernel 2.6.38, after tracer sees tracee ptrace-stop and until it
restarts or kills it, tracee will not run, and will not send
notifications (except SIGKILL death) to tracer, even if tracer enters
into another waitpid call.

Currently, it causes problem with transparent handling of stopping
signals: if tracer restarts tracee after group-stop, SIGSTOP is
effectively ignored: tracee doesn't remain stopped, it runs. If tracer
doesn't restart tracee before entering into next waitpid, future
SIGCONT will not be reported to the tracer. Which would make SIGCONT to
have no effect.

??? ...at least no effect on this tracee - how will other threads
    be affected?

	1.x.x Syscall-stops

If tracee was restarted by PTRACE_SYSCALL, tracee enters
syscall-enter-stop just prior to entering any syscall. If tracer
restarts it with PTRACE_SYSCALL, tracee enters syscall-exit-stop when
syscall is finished, or if it is interrupted by a signal. (That is,
signal-delivery-stop never happens between syscall-enter-stop and
syscall-exit-stop, it happens after syscall-exit-stop). Other
possibilities are that tracee may exit (if it entered exit or
exit_group syscall), be killed by SIGKILL, or die when other threads'
actions terminate all threads of the process (such as execve syscall).

??? how such death-because-of-other-thread is reported?

Syscall-enter-stop and syscall-exit-stop are observed by tracer as
waitpid returning with the following result: WIFSTOPPED(status) is
true, WSTOPSIG(status) == SIGTRAP. If PTRACE_O_TRACESYSGOOD option was
set by tracer, then WSTOPSIG(status) == (SIGTRAP | 0x80). Otherwise, it
is impossible to distinguish them from signal-delivery-stop with
SIGTRAP.

Syscall-enter-stop and syscall-exit-stop are indistinguishable by
tracer. Tracer needs to keep track of the sequence of ptrace-stops in
order to not misinterpret syscall-enter-stop as syscall-exit-stop or
vice versa. The rule is that syscall-enter-stop is always followed by
syscall-exit-stop or tracee's death - no other kinds of ptrace-stop can
occur in between.

??? What will happen if trace uses *NOT* PTRACE_SYSCALL to restart
    tracee after syscall-enter-stop?

??? what PTRACE_GETSIGINFO returns on syscall stops?

	1.x.x SINGLESTEP, SYSEMU, SYSEMU_SINGLESTEP

??? document PTRACE_SINGLESTEP, PTRACE_SYSEMU, PTRACE_SYSEMU_SINGLESTEP

	1.x.x PTRACE_EVENT-stops

If tracer sets TRACE_O_TRACEfoo options, tracee will enter ptrace-stops
called PTRACE_EVENT-stops.

PTRACE_EVENT-stops are observed by tracer as waitpid returning with the
following result: WIFSTOPPED(status) is true, WSTOPSIG(status) ==
SIGTRAP. Additional bit is set in a higher byte of status word: value
((status >> 8) & 0xffff) will be (SIGTRAP | PTRACE_EVENT_foo << 8).

PTRACE_EVENT_VFORK - stop after return from vfork/clone+CLONE_VFORK ???
continuing it will make it wait for child to exit/exec, right?
PTRACE_EVENT_FORK - stop after return from fork/clone+SIGCHLD
PTRACE_EVENT_CLONE - stop after return from clone
PTRACE_EVENT_VFORK_DONE - stop after vfork child unblocks this tracee
	For all four: stop occurs in parent, not in new thread;
	PTRACE_GETEVENTMSG can be used to retrieve new thread's tid.
PTRACE_EVENT_EXEC - stop after return from exec PTRACE_EVENT_EXIT -
stop before exit
	PTRACE_GETEVENTMSG returns exit status.
	Registers can be examined (unlike when "real" exit happens).
??? needs to be PTRACE_CONTed to finish exit, or not?

??? what PTRACE_GETSIGINFO returns on PTRACE_EVENT-stops?


	1.x Informational and restarting ptrace commands.

Most ptrace commands (all except ATTACH, TRACEME, KILL, SETOPTIONS)
require tracee to be in ptrace-stop, otherwise they fail with ESRCH.

When tracee is in ptrace-stop, tracer can read and write data to tracee
using informational commands. They leave tracee in ptrace-stopped state.

longv = ptrace(PTRACE_PEEKTEXT/PTRACE_PEEKDATA/PTRACE_PEEKUSER, pid,
addr, 0);
	ptrace(PTRACE_POKETEXT/PTRACE_POKEDATA/PTRACE_POKEUSER, pid,
addr, long_val);
	ptrace(PTRACE_GETREGS/PTRACE_GETFPREGS, pid, 0, &struct);
	ptrace(PTRACE_SETREGS/PTRACE_SETFPREGS, pid, 0, &struct);
	ptrace(PTRACE_GETSIGINFO, pid, 0, &siginfo);
	ptrace(PTRACE_SETSIGINFO, pid, 0, &siginfo);
	ptrace(PTRACE_GETEVENTMSG, pid, 0, &long_var);

Note that some errors are not reported. For example, setting siginfo
may have no effect in some ptrace-stops, yet the call will succeed
(return 0 and not set errno).

Another group of ptrace commands makes ptrace-stopped tracee run. They
all have the form:
	ptrace(PTRACE_cmd, pid, 0, sig);
where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
signal to be injected. Otherwise, sig is ignored.


	1.x Attaching and detaching

A thread can be attached to using ptrace(PTRACE_ATTACH, pid, 0, 0)
call. This also sends SIGSTOP to this thread. If tracer wants this
SIGSTOP to have no effect, it needs to suppress it. Note that if other
signals are concurrently sent to this thread during attach, tracer may
see tracee enter signal-delivery-stop with other signal(s) first! The
usual practice is to reinject these signals until SIGSTOP is seen, then
suppress SIGSTOP injection. The design bug here is that attach and
concurrent SIGSTOP race and SIGSTOP may be lost.

??? Is there a bug/misfeature that attaching interrupts some syscalls,
    such as nanosleep? Document it. (I guess even suppressed SIGSTOP
    causes syscall to return prematurely).

ptrace(PTRACE_TRACEME, 0, 0, 0) request turns current thread into a
tracee. It continues to run (doesn't enter ptrace-stop). A common
practice is follow ptrace(PTRACE_TRACEME) with raise(SIGSTOP) and allow
parent (which is our tracer now) to observe our signal-delivery-stop.

If PTRACE_O_TRACE[V]FORK or PTRACE_O_TRACECLONE options are in effect,
then children created by (vfork or clone(CLONE_VFORK)), (fork or
clone(SIGCHLD)) and (other kinds of clone) respectively are
automatically attached to the same tracer which traced their parent.
SIGSTOP is delivered to them, causing them to enter
signal-delivery-stop as they exit syscall which created them.

Detaching of tracee is performed by ptrace(PTRACE_DETACH, pid, 0, sig).
PTRACE_DETACH is a restarting operation, therefore it requires tracee
to be in ptrace-stop. If tracee is in signal-delivery-stop, signal can
be injected. Othervice, sig parameter is ignored.

If tracee is running when tracer wants to detach it, the usual solution
is to send SIGSTOP (using tgkill, to mkae sure it goes to the correct
thread), wait for tracee to stop in signal-delivery-stop for SIGSTOP
and then detach it (suppressing SIGSTOP injection). Design bug is that
this can race with concurrent SIGSTOPs. Another complication is that
tracee may enter other ptrace-stops and needs to be restarted and
waited for again, until SIGSTOP is seen. Yet another complication is to
be sure that tracee is not already group-stopped, because no signal
delivery happens while it is - not even SIGSTOP.

??? is above accurate?

If tracer dies, all tracees are automatically detached.

??? are they restarted if they were in some ptrace-stop?
    Even those which were in group-stop?
    Is signal injected if they were in signal-delivery-stop?


	1.x Real parent

Ptrace API (ab)uses standard Unix parent/child signaling over waitpid.
This used to cause real parent of the process to stop receiving several
kinds of waitpid notifications when child process is traced by some
other process.

Many of these bugs have been fixed, but as of 2.6.38 several still
exist.

As of 2.6.38, the following is believed to work correctly:

- exit/death by signal is reported both to tracer and to real parent.
If they are the same process, the report is sent only once.

- ??? add more docs

Following bugs still exist:

- group-stop notifications are sent to tracer, but not to real parent.

- ??? add more known bugs here



	2. Linux kernel implementation

TODO

Re: Ptrace documentation, draft #1

[Tejun Heo]

Hello,

On Sun, May 15, 2011 at 10:35:32PM +0200, Denys Vlasenko wrote:
> Hi Tejun,
> 
> On Monday 09 May 2011 12:09, Tejun Heo wrote:
> > Hmmm... I was thinking about writing a proper ptrace API doc with
> > example programs under Documentation/.  It's userland visible API so
> > it shouldn't change all that much and the amount of necessary
> > documentation would be too much for comments.
> 
> Since I am not helping you much in kernel ptrace hacking,
> I can at least help you with the docs.

Oh, I love you.  :-)

I'm a bit backed up at the moment but will get to this later this
week.

Thank you very much.

-- 
tejun

Re: Ptrace documentation, draft #1

[Oleg Nesterov]

Denys, thanks for doing this.

On 05/15, Denys Vlasenko wrote:
>
> 	1.x Death under ptrace.
>
> When a (possibly multi-threaded) process receives a killing signal (a
> signal set to SIG_DFL and whose default action is to kill the process),
> all threads exit. Tracees report their death to the tracer(s). This is
> not a ptrace-stop (because tracer can't query tracee status such as
> register contents, cannot restart tracee etc) but the notification
> about this event is delivered through waitpid API similarly to
> ptrace-stop.

Note: currently a killed PT_TRACE_EXIT tracee can stop and report
PTRACE_EVENT_EXIT before it actually exits. I'd say this is wrong and
should be fixed.

Another problem: the tracee can silently "disappear" during exec,
if it was the group leader and exec is called by its sub-thread.
Unfortunately, this is not easy to fix. The new leader inherits
the same pid. In fact, the old thread can "disappear", exactly
because it changes its pid.

IOW. If the old leader was traced - it disappears. If the new leader
is traced, it continues to be traced but it changes its pid, so it
is visible as the old leader to the tracer.

> Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).

Oh, no. This is more or less equivalent to PTRACE_CONT(SIGKILL) except
PTRACE_KILL doesn't return the error if the tracee is not stopped.

I'd say: do not use PTRACE_KILL, never. If the tracer wants to kill
the tracee - kill or tkill should be used.

> When any thread executes exit_group syscall, every tracee reports its
> death to its tracer.
>
> ??? Is it true that *every* thread reports death?

Yes, if you mean do_wait() as above.

> ??? is there a recommended usage of waitpid(WNOHANG) to check whether
>     tracee is dead or alive?

I don't think so. It can be dying but not dead yet, so WNOHANG | WEXITED
will fail.

> Kernel delivers an extra SIGTRAP to tracee after execve syscall
> returns. This is an ordinary signal (similar to one generated by kill
> -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
> is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
>
> ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
>     by looking at its siginfo?

Afaics no. Well, except .si_pid shows that the signal was sent by the
tracing process to itself.

I'd say it is better to assume nobody sends SIGTRAP to the tracee.
Even if the tracer could filter out the "real" signals, SIGTRAP doesn't
queue.

> ??? Are syscalls interrupted by signals which are suppressed by tracer?
>     If yes, document it here

Please reiterate, can't understand.

> Note that restarting ptrace commands issued in ptrace-stops other than
> signal-delivery-stop do NOT inject a signal, even if sig is nonzero. No
> error is reported either. This is a cause of confusion among ptrace
> users.

Yes. Except syscall entry/exit. But in this case SET_SIGINFO doesn't work
to add more confusion ;)

> As of kernel 2.6.38, after tracer sees tracee ptrace-stop and until it
> restarts or kills it, tracee will not run,

Well, this is not exactly true. Initially the tracee sleeps in TASK_STOPPED
and thus it can be woken by SIGCONT. But the first ptrace request changes
turns this state into TASK_TRACED.

This was already changed by the pending patches.


> If tracee was restarted by PTRACE_SYSCALL, tracee enters
> syscall-enter-stop just prior to entering any syscall. If tracer
> restarts it with PTRACE_SYSCALL, tracee enters syscall-exit-stop when
> syscall is finished, or if it is interrupted by a signal. (That is,
> signal-delivery-stop never happens between syscall-enter-stop and
> syscall-exit-stop, it happens after syscall-exit-stop).

This is true. But, just in case, please note that PTRACE_EVENT_EXEC
or PTRACE_EVENT_{FORK,CLONE,etc} can be reported in between.

> ??? how such death-because-of-other-thread is reported?

again, can't understand the question.

> Syscall-enter-stop and syscall-exit-stop are indistinguishable by
> tracer.

Almost... at least on x86 rax = -ENOSYS in syscall-enter-stop.

> ??? What will happen if trace uses *NOT* PTRACE_SYSCALL to restart
>     tracee after syscall-enter-stop?

Nothing special. syscall-exit-stop won't be reported.

> ??? what PTRACE_GETSIGINFO returns on syscall stops?

The same info as any other ptrace_notify(). Except si_code which you
already described: SIGTRAP, or SIGTRAP | 0x80 if PT_TRACESYSGOOD.

> stop before exit
> 	PTRACE_GETEVENTMSG returns exit status.
> 	Registers can be examined (unlike when "real" exit happens).
> ??? needs to be PTRACE_CONTed to finish exit, or not?

Yes.

> ??? what PTRACE_GETSIGINFO returns on PTRACE_EVENT-stops?

The contents is always the same,

	si_signo = SIGTRAP;
	si_pid	= tid_of_the_tracee;
	si_uid	= uid;

But si_code = (event << 8) | SIGTRAP and depends on reported event.

> Most ptrace commands (all except ATTACH, TRACEME, KILL, SETOPTIONS)
> require tracee to be in ptrace-stop, otherwise they fail with ESRCH.

SETOPTIONS needs the stopped tracee as well.

> 	ptrace(PTRACE_cmd, pid, 0, sig);
> where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
> SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
> signal to be injected. Otherwise, sig is ignored.

There is another special case. If the tracee single-stepps into the
signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
But ptrace(PTRACE, ..., sig) doesn't inject after that.

> ??? Is there a bug/misfeature that attaching interrupts some syscalls,
>     such as nanosleep? Document it. (I guess even suppressed SIGSTOP
>     causes syscall to return prematurely).

Yes, this is the "normal" signal and thus it can interrupt syscalls.

> If tracer dies, all tracees are automatically detached.
>
> ??? are they restarted if they were in some ptrace-stop?

yes,

>     Even those which were in group-stop?

well, the current code is buggy, but it tries to leave it stopped.

>     Is signal injected if they were in signal-delivery-stop?

Yes, The tracee resumes and handles the previously reported signal.

> As of 2.6.38, the following is believed to work correctly:
>
> - exit/death by signal is reported both to tracer and to real parent.

First to the tracer. Once it does do_wait(), we notify the real parent.
And of course, the real parent is not notified about the exiting threads.

There is additional complication with the group-leader. If it is traced
and exits, do_wait(WEXITED) doesn't work (until all threads exit) for
the tracer. Should be changed, I think.

Oleg.

Re: Ptrace documentation, draft #1

[Tejun Heo]

Hello,

I haven't read the whole thing yet but wanna comment on some.

On Mon, May 16, 2011 at 05:31:22PM +0200, Oleg Nesterov wrote:
> > Kernel delivers an extra SIGTRAP to tracee after execve syscall
> > returns. This is an ordinary signal (similar to one generated by kill
> > -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
> > is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
> >
> > ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
> >     by looking at its siginfo?
> 
> Afaics no. Well, except .si_pid shows that the signal was sent by the
> tracing process to itself.

If you do GETSIGINFO and look at si->si_code, user generated signals
can't have non-zero value there (assuming SETSIGINFO doesn't allow
uploading __SI_TRAP siginfo), so, if si->si_code contains SIGTRAP |
PTRACE_EVENT_* << 8, I think it reliably identifies a specific trap.
We probably want to add more PTRACE_EVENT_* so that each trap site can
be identified uniquely.  Note that the si_code is also returned as
exit_code on wait(2) and combined with CLD_TRAPPED ptrace traps should
be reliably identifiable.

> > Note that restarting ptrace commands issued in ptrace-stops other than
> > signal-delivery-stop do NOT inject a signal, even if sig is nonzero. No
> > error is reported either. This is a cause of confusion among ptrace
> > users.
> 
> Yes. Except syscall entry/exit. But in this case SET_SIGINFO doesn't work
> to add more confusion ;)

Yeah, I hate these signal injections.  Maybe we can remove these if
SEIZED?  Tracer might as well just do tkill().  I don't really see the
point of the signal injection feature.

> > Syscall-enter-stop and syscall-exit-stop are indistinguishable by
> > tracer.
> 
> Almost... at least on x86 rax = -ENOSYS in syscall-enter-stop.

We can add PTRACE_EVENT_* values if SEIZED.

> > 	ptrace(PTRACE_cmd, pid, 0, sig);
> > where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
> > SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
> > signal to be injected. Otherwise, sig is ignored.
> 
> There is another special case. If the tracee single-stepps into the
> signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
> But ptrace(PTRACE, ..., sig) doesn't inject after that.

Ditto as PTRACE_SYSCALL.  I think resuming from signal delivery trap
should be the only place where @data means signo.

> >     Is signal injected if they were in signal-delivery-stop?
> 
> Yes, The tracee resumes and handles the previously reported signal.

Please note that this one really isn't an injection.  It's just given
a chance to either suppress or alter a signal which is being
delivered.  It's not creating a new one.

Thank you.

-- 
tejun

Re: Ptrace documentation, draft #1

[Oleg Nesterov]

On 05/16, Tejun Heo wrote:
>
> On Mon, May 16, 2011 at 05:31:22PM +0200, Oleg Nesterov wrote:
> > > Kernel delivers an extra SIGTRAP to tracee after execve syscall
> > > returns. This is an ordinary signal (similar to one generated by kill
> > > -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
> > > is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
> > >
> > > ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
> > >     by looking at its siginfo?
> >
> > Afaics no. Well, except .si_pid shows that the signal was sent by the
> > tracing process to itself.
>
> If you do GETSIGINFO and look at si->si_code, user generated signals
> can't have non-zero value there

Hmm. The can? sys_kill() sets si_code = 0, but tkill() or queueinfo()
can pass any si_code < 0. Also, the kernel can generate the signal
with si_code > 0.

> so, if si->si_code contains SIGTRAP |
> PTRACE_EVENT_* << 8,

But in this case (without PT_TRACE_EXEC) the tracee simply sends SIGTRAP
to itself. It will be reported later like a normal signal.

> > Yes. Except syscall entry/exit. But in this case SET_SIGINFO doesn't work
> > to add more confusion ;)
>
> Yeah, I hate these signal injections.

Welcome to the club!

> Maybe we can remove these if
> SEIZED?

Heh... I am not sure.

> > > 	ptrace(PTRACE_cmd, pid, 0, sig);
> > > where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
> > > SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
> > > signal to be injected. Otherwise, sig is ignored.
> >
> > There is another special case. If the tracee single-stepps into the
> > signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
> > But ptrace(PTRACE, ..., sig) doesn't inject after that.
>
> Ditto as PTRACE_SYSCALL.

Hmm. What do you mean? tracehook_report_syscall_exit() can skip
ptrace_report_syscall() if step, but this is another story. And in this
case the tracee sends the real signal to itself, unlike
tracehook_signal_handler() which does ptrace_notify().

> I think resuming from signal delivery trap
> should be the only place where @data means signo.

Agreed, it would be more clean/simple/understandable.

> > >     Is signal injected if they were in signal-delivery-stop?
> >
> > Yes, The tracee resumes and handles the previously reported signal.
>
> Please note that this one really isn't an injection.  It's just given
> a chance to either suppress or alter a signal which is being
> delivered.  It's not creating a new one.

Yes, agreed, we do not create the new one.

Oleg.

Re: Ptrace documentation, draft #1

[Tejun Heo]

Hello,

On Mon, May 16, 2011 at 06:53:52PM +0200, Oleg Nesterov wrote:
> > If you do GETSIGINFO and look at si->si_code, user generated signals
> > can't have non-zero value there
> 
> Hmm. The can? sys_kill() sets si_code = 0, but tkill() or queueinfo()
> can pass any si_code < 0. Also, the kernel can generate the signal
> with si_code > 0.

Yeah, sure, not non-zero then, but it's still distinguishible, no?

> > so, if si->si_code contains SIGTRAP | PTRACE_EVENT_* << 8,
> 
> But in this case (without PT_TRACE_EXEC) the tracee simply sends SIGTRAP
> to itself. It will be reported later like a normal signal.

Ah, okay, I missed the context.  I thought we were talking about
GETSIGINFO after traps not the extra SIGTRAP after execve.  Sorry
about that.  :-)

> > Maybe we can remove these if SEIZED?
> 
> Heh... I am not sure.

Yeah, I'm not sure either.  I think the most reasonable solution would
be leaving them alone but strongly discourage their use.

> Hmm. What do you mean? tracehook_report_syscall_exit() can skip
> ptrace_report_syscall() if step, but this is another story. And in this
> case the tracee sends the real signal to itself, unlike
> tracehook_signal_handler() which does ptrace_notify().

Please disregard.  Still was fixated on GETSIGINFO.

Thanks.

-- 
tejun

Re: Ptrace documentation, draft #1

[Oleg Nesterov]

On 05/16, Tejun Heo wrote:
>
> On Mon, May 16, 2011 at 06:53:52PM +0200, Oleg Nesterov wrote:
> > > If you do GETSIGINFO and look at si->si_code, user generated signals
> > > can't have non-zero value there
> >
> > Hmm. The can? sys_kill() sets si_code = 0, but tkill() or queueinfo()
> > can pass any si_code < 0. Also, the kernel can generate the signal
> > with si_code > 0.
>
> Yeah, sure, not non-zero then, but it's still distinguishible, no?

Hmm. Now it is me who is missing context. And probably I misunderstood
the original question:

	Kernel delivers an extra SIGTRAP to tracee after execve syscall
	returns. This is an ordinary signal
	...
	??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
	    by looking at its siginfo?

I meant, we can't distinguish this SIGTRAP. But if PTRACE_O_TRACEEXEC is
set then the tracee reports PTRACE_EVENT_EXEC, this is probably
distinguishible. IIUC, si_code > 0 if the tracee reports an event.

Oleg.

Re: Ptrace documentation, draft #1

[Denys Vlasenko]

On Mon, May 16, 2011 at 5:31 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> On 05/15, Denys Vlasenko wrote:
>>
>>       1.x Death under ptrace.
>>
>> When a (possibly multi-threaded) process receives a killing signal (a
>> signal set to SIG_DFL and whose default action is to kill the process),
>> all threads exit. Tracees report their death to the tracer(s). This is
>> not a ptrace-stop (because tracer can't query tracee status such as
>> register contents, cannot restart tracee etc) but the notification
>> about this event is delivered through waitpid API similarly to
>> ptrace-stop.
>
> Note: currently a killed PT_TRACE_EXIT tracee can stop and report
> PTRACE_EVENT_EXIT before it actually exits. I'd say this is wrong and
> should be fixed.

Yes, I assumed this is normal.
Or do you mean that *killed* tracee (that is, by signal) also stops there?

> Another problem: the tracee can silently "disappear" during exec,
> if it was the group leader and exec is called by its sub-thread.
> Unfortunately, this is not easy to fix. The new leader inherits
> the same pid. In fact, the old thread can "disappear", exactly
> because it changes its pid.
>
> IOW. If the old leader was traced - it disappears. If the new leader
> is traced, it continues to be traced but it changes its pid, so it
> is visible as the old leader to the tracer.

This should be so much fun for ptrace users :)
I am documenting this.

>> Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).
>
> Oh, no. This is more or less equivalent to PTRACE_CONT(SIGKILL) except
> PTRACE_KILL doesn't return the error if the tracee is not stopped.
>
> I'd say: do not use PTRACE_KILL, never. If the tracer wants to kill
> the tracee - kill or tkill should be used.

Regardless. We need to tell users what to expect after they do PTRACE_KILL.
Should they expect that every tracee report WIFSIGNALED(status)
and WTERMSIG(status) = SIGKILL? Can they get stale, buffered waitpid results
before they get this last one?


>> When any thread executes exit_group syscall, every tracee reports its
>> death to its tracer.
>>
>> ??? Is it true that *every* thread reports death?
>
> Yes, if you mean do_wait() as above.

And will PTRACE_EVENT_EXIT happen for *every* tracee (which has it configured)?

>> Kernel delivers an extra SIGTRAP to tracee after execve syscall
>> returns. This is an ordinary signal (similar to one generated by kill
>> -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
>> is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
>>
>> ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
>>     by looking at its siginfo?
>
> Afaics no. Well, except .si_pid shows that the signal was sent by the
> tracing process to itself.

What about si_code? Is it set to SI_KERNEL for this signal?
AFAIR userspace can't create signals with such si_code.

> I'd say it is better to assume nobody sends SIGTRAP to the tracee.
> Even if the tracer could filter out the "real" signals, SIGTRAP doesn't
> queue.

Yes, I understand that the race with real SIGTRAPs is not fixable.
I mostly look for a way for tracer to say "aha, this is that pesky
SIGTRAP from execve, ignore it". One way is to set PTRACE_O_TRACEEXEC.
Is GETSIGINFO another?


>> ??? Are syscalls interrupted by signals which are suppressed by tracer?
>>     If yes, document it here
>
> Please reiterate, can't understand.

Let's say tracee is in nanosleep. Then some signal arrives,
but tracer decides to ignore it. In tracer:

waitpid: WIFSTOPPED, WSTOPSIG = some_sig  <===
ptrace(PTRACE_CONT, pid, 0, 0)  ===>

will this interrupt nanosleep in tracee?


>> Note that restarting ptrace commands issued in ptrace-stops other than
>> signal-delivery-stop do NOT inject a signal, even if sig is nonzero. No
>> error is reported either. This is a cause of confusion among ptrace
>> users.
>
> Yes. Except syscall entry/exit. But in this case SET_SIGINFO doesn't work
> to add more confusion ;)

I rewrote it like this:

Note that restarting ptrace commands issued in ptrace-stops other than
signal-delivery-stop is not guaranteed to inject a signal, even if sig
is nonzero.
No error is reported, nonzero sig may simply be ignored.
Ptrace users should not try to "create new signal" this way: use
tgkill(2) instead.


>> As of kernel 2.6.38, after tracer sees tracee ptrace-stop and until it
>> restarts or kills it, tracee will not run,
>
> Well, this is not exactly true. Initially the tracee sleeps in TASK_STOPPED
> and thus it can be woken by SIGCONT. But the first ptrace request changes
> turns this state into TASK_TRACED.

> This was already changed by the pending patches.

This is an extremely subtle point, and is not really a part of API "as
designed":
how tracer can know that it's a group-stop *without* performing GETSIGINFO?
And after it performed GETSIGINFO, it "destroyed" TASK_STOPPED...
so knowing that it *was* TASK_STOPPED, which *could have been* woken up
by SIGCONT does absolutely no good for the poor tracer, it's too late!

It looks insane.

I propose to not document it, as you guys plan to fix this thing for good.


>> If tracee was restarted by PTRACE_SYSCALL, tracee enters
>> syscall-enter-stop just prior to entering any syscall. If tracer
>> restarts it with PTRACE_SYSCALL, tracee enters syscall-exit-stop when
>> syscall is finished, or if it is interrupted by a signal. (That is,
>> signal-delivery-stop never happens between syscall-enter-stop and
>> syscall-exit-stop, it happens after syscall-exit-stop).
>
> This is true. But, just in case, please note that PTRACE_EVENT_EXEC
> or PTRACE_EVENT_{FORK,CLONE,etc} can be reported in between.

Aha, so PTRACE_EVENT-stops happen "within" the syscall?
Meaning, between syscall-enter-stop and syscall-exit-stop?

> But si_code = (event << 8) | SIGTRAP and depends on reported event.

Aha! I sort-of expected SI_KERNEL there...

>>       ptrace(PTRACE_cmd, pid, 0, sig);
>> where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
>> SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
>> signal to be injected. Otherwise, sig is ignored.
>
> There is another special case. If the tracee single-stepps into the
> signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
> But ptrace(PTRACE, ..., sig) doesn't inject after that.

This is part of missing doc about PTRACE_SINGLESTEP.
>From what you are saying it looks like PTRACE_SINGLESTEP
implies PTRACE_SYSCALL behavior: "report syscall-stops".


>> As of 2.6.38, the following is believed to work correctly:
>>
>> - exit/death by signal is reported both to tracer and to real parent.
>
> First to the tracer. Once it does do_wait(), we notify the real parent.
> And of course, the real parent is not notified about the exiting threads.
>
> There is additional complication with the group-leader. If it is traced
> and exits, do_wait(WEXITED) doesn't work (until all threads exit) for
> the tracer. Should be changed, I think.

Updated this part.

-- 
vda

Re: Ptrace documentation, draft #1

[Denys Vlasenko]

On Mon, May 16, 2011 at 6:53 PM, Oleg Nesterov <oleg@redhat.com> wrote:
>> I think resuming from signal delivery trap
>> should be the only place where @data means signo.
>
> Agreed, it would be more clean/simple/understandable.

I think ptrace API never promised that it will be possible
to "create new signal" by passing non-zero data - only
that the already pending signal can be passed, altered
or suppressed. If there *are* cases where "create new signal"
currently works, it's a quirk.

I changed doc to reflect this. If we tighten it up so that we can state
that all such attempts will be ignored, we can update docs.

-- 
vda

Re: Ptrace documentation, draft #1

[Oleg Nesterov]

On 05/18, Denys Vlasenko wrote:
>
> On Mon, May 16, 2011 at 5:31 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > Note: currently a killed PT_TRACE_EXIT tracee can stop and report
> > PTRACE_EVENT_EXIT before it actually exits. I'd say this is wrong and
> > should be fixed.
>
> Yes, I assumed this is normal.
> Or do you mean that *killed* tracee (that is, by signal) also stops there?

Yes.

> >> Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).
> >
> > Oh, no. This is more or less equivalent to PTRACE_CONT(SIGKILL) except
> > PTRACE_KILL doesn't return the error if the tracee is not stopped.
> >
> > I'd say: do not use PTRACE_KILL, never. If the tracer wants to kill
> > the tracee - kill or tkill should be used.
>
> Regardless. We need to tell users what to expect after they do PTRACE_KILL.

Once again, PTRACE_KILL == ptrace(PTRACE_CONT, SIGKILL), except it
doesn't return the error if the tracee is not stopped.

OTOH, it does the unconditional wakeup, but I don't think we should
document this bug.

> >> When any thread executes exit_group syscall, every tracee reports its
> >> death to its tracer.
> >>
> >> ??? Is it true that *every* thread reports death?
> >
> > Yes, if you mean do_wait() as above.
>
> And will PTRACE_EVENT_EXIT happen for *every* tracee (which has it configured)?

Oh. This depends on /dev/random. Most probably the exiting tracee
dequeues the (implicit) SIGKILL and report PTRACE_EVENT_EXIT. Oh,
unless arch_ptrace_stop_needed() is true. But it can exit on its own
or deque another fatal signal, then it won't stop because of
fatal_signal_pending().

In short: this should be fixed. We already discussed this a bit (many
times ;), first of all we should define the correct behaviour. If you
ask me, personally I think PTRACE_EVENT_EXIT should be always reported
unless the task was explicitly killed by SIGKILL. But this is not clear.

> >> Kernel delivers an extra SIGTRAP to tracee after execve syscall
> >> returns. This is an ordinary signal (similar to one generated by kill
> >> -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
> >> is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
> >>
> >> ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
> >>     by looking at its siginfo?
> >
> > Afaics no. Well, except .si_pid shows that the signal was sent by the
> > tracing process to itself.
>
> What about si_code? Is it set to SI_KERNEL for this signal?

No, SI_USER.

> > I'd say it is better to assume nobody sends SIGTRAP to the tracee.
> > Even if the tracer could filter out the "real" signals, SIGTRAP doesn't
> > queue.
>
> Yes, I understand that the race with real SIGTRAPs is not fixable.
> I mostly look for a way for tracer to say "aha, this is that pesky
> SIGTRAP from execve, ignore it". One way is to set PTRACE_O_TRACEEXEC.

Yes.

> Is GETSIGINFO another?

How? We simply send SIGTRAP as if it was sent by kill(), the tracee
will dequeue this signal and report later.

> >> ??? Are syscalls interrupted by signals which are suppressed by tracer?
> >>     If yes, document it here
> >
> > Please reiterate, can't understand.
>
> Let's say tracee is in nanosleep. Then some signal arrives,

note that the tracee is already interrupted here, sys_nanosleep()
returns ERESTART_RESTARTBLOCK.

> but tracer decides to ignore it. In tracer:
>
> waitpid: WIFSTOPPED, WSTOPSIG = some_sig  <===
> ptrace(PTRACE_CONT, pid, 0, 0)  ===>
>
> will this interrupt nanosleep in tracee?

Yes and no. Once again, the tracee already returned from sys_nanosleep,
but it will restart this syscall (actually, it will do sys_restart_syscall)
and continue to sleep.

> >> As of kernel 2.6.38, after tracer sees tracee ptrace-stop and until it
> >> restarts or kills it, tracee will not run,
> >
> > Well, this is not exactly true. Initially the tracee sleeps in TASK_STOPPED
> > and thus it can be woken by SIGCONT. But the first ptrace request changes
> > turns this state into TASK_TRACED.
>
> > This was already changed by the pending patches.
>
> This is an extremely subtle point, and is not really a part of API "as
> designed":

I think this was never designed ;)

> I propose to not document it, as you guys plan to fix this thing for good.

Agreed.

> >> If tracee was restarted by PTRACE_SYSCALL, tracee enters
> >> syscall-enter-stop just prior to entering any syscall. If tracer
> >> restarts it with PTRACE_SYSCALL, tracee enters syscall-exit-stop when
> >> syscall is finished, or if it is interrupted by a signal. (That is,
> >> signal-delivery-stop never happens between syscall-enter-stop and
> >> syscall-exit-stop, it happens after syscall-exit-stop).
> >
> > This is true. But, just in case, please note that PTRACE_EVENT_EXEC
> > or PTRACE_EVENT_{FORK,CLONE,etc} can be reported in between.
>
> Aha, so PTRACE_EVENT-stops happen "within" the syscall?
> Meaning, between syscall-enter-stop and syscall-exit-stop?

Yes.

> >>       ptrace(PTRACE_cmd, pid, 0, sig);
> >> where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
> >> SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
> >> signal to be injected. Otherwise, sig is ignored.
> >
> > There is another special case. If the tracee single-stepps into the
> > signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
> > But ptrace(PTRACE, ..., sig) doesn't inject after that.
>
> This is part of missing doc about PTRACE_SINGLESTEP.
> From what you are saying it looks like PTRACE_SINGLESTEP
> implies PTRACE_SYSCALL behavior: "report syscall-stops".

Hmm. Why do you think so?

Oleg.

Re: Ptrace documentation, draft #1

[Denys Vlasenko]

On Thu, May 19, 2011 at 9:49 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> On 05/18, Denys Vlasenko wrote:
>>
>> On Mon, May 16, 2011 at 5:31 PM, Oleg Nesterov <oleg@redhat.com> wrote:
>> >
>> > Note: currently a killed PT_TRACE_EXIT tracee can stop and report
>> > PTRACE_EVENT_EXIT before it actually exits. I'd say this is wrong and
>> > should be fixed.
>>
>> Yes, I assumed this is normal.
>> Or do you mean that *killed* tracee (that is, by signal) also stops there?
>
> Yes.

Thanks, noted.

>> >> Tracer can kill a tracee with ptrace(PTRACE_KILL, pid, 0, 0).
>> >
>> > Oh, no. This is more or less equivalent to PTRACE_CONT(SIGKILL) except
>> > PTRACE_KILL doesn't return the error if the tracee is not stopped.
>> >
>> > I'd say: do not use PTRACE_KILL, never. If the tracer wants to kill
>> > the tracee - kill or tkill should be used.
>>
>> Regardless. We need to tell users what to expect after they do PTRACE_KILL.
>
> Once again, PTRACE_KILL == ptrace(PTRACE_CONT, SIGKILL), except it
> doesn't return the error if the tracee is not stopped.

Oleg, this doesn't explain the resulting behavior in terms understandable
to mere mortals. *What will happen* when user does ptrace(PTRACE_KILL)?

Yes, it's obvious that the tracee gets SIGKILLed, but will it report WIFSIGNALED
or not? Userspace folks won't be 100.00% sure if we won't be exact about it.

They may think "hmm... maybe this PTRACE_KILL thing is so powerful it makes
it unnecessary to waitpid for the nuked process?", which actualy isn't such
a stupid hupothesis - if tracer itself PTRACE_KILL's tracee, it doesn't want
to know about it anymore, so why should it waitpid for it?


>> >> When any thread executes exit_group syscall, every tracee reports its
>> >> death to its tracer.
>> >>
>> >> ??? Is it true that *every* thread reports death?
>> >
>> > Yes, if you mean do_wait() as above.
>>
>> And will PTRACE_EVENT_EXIT happen for *every* tracee (which has it configured)?
>
> Oh. This depends on /dev/random. Most probably the exiting tracee
> dequeues the (implicit) SIGKILL and report PTRACE_EVENT_EXIT. Oh,
> unless arch_ptrace_stop_needed() is true. But it can exit on its own
> or deque another fatal signal, then it won't stop because of
> fatal_signal_pending().
>
> In short: this should be fixed. We already discussed this a bit (many
> times ;), first of all we should define the correct behaviour. If you
> ask me, personally I think PTRACE_EVENT_EXIT should be always reported
> unless the task was explicitly killed by SIGKILL. But this is not clear.

Documented with "KNOWN BUG:" tag.


>> >> Kernel delivers an extra SIGTRAP to tracee after execve syscall
>> >> returns. This is an ordinary signal (similar to one generated by kill
>> >> -TRAP), not a special kind of ptrace-stop. If PTRACE_O_TRACEEXEC option
>> >> is in effect, a PTRACE_EVENT_EXEC-stop is generated instead.
>> >>
>> >> ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
>> >>     by looking at its siginfo?
>> >
>> > Afaics no. Well, except .si_pid shows that the signal was sent by the
>> > tracing process to itself.
>>
>> What about si_code? Is it set to SI_KERNEL for this signal?
>
> No, SI_USER.

This is stupid. This signal is sent by kernel. Why is it flagged as "from user"?
Maybe we should change it?

(BTW, where is it generated in the kernel source? I found
PTRACE_EVENT_EXEC generation, but failed to find
"old-school SIGTRAP" generation code...)


>> >> ??? Are syscalls interrupted by signals which are suppressed by tracer?
>> >>     If yes, document it here
>> >
>> > Please reiterate, can't understand.
>>
>> Let's say tracee is in nanosleep. Then some signal arrives,
>
> note that the tracee is already interrupted here, sys_nanosleep()
> returns ERESTART_RESTARTBLOCK.
>
>> but tracer decides to ignore it. In tracer:
>>
>> waitpid: WIFSTOPPED, WSTOPSIG = some_sig  <===
>> ptrace(PTRACE_CONT, pid, 0, 0)  ===>
>>
>> will this interrupt nanosleep in tracee?
>
> Yes and no. Once again, the tracee already returned from sys_nanosleep,
> but it will restart this syscall (actually, it will do sys_restart_syscall)
> and continue to sleep.

Documented as such.


>> >>       ptrace(PTRACE_cmd, pid, 0, sig);
>> >> where cmd is CONT, DETACH, SYSCALL, SINGLESTEP, SYSEMU,
>> >> SYSEMU_SINGLESTEP. If tracee is in signal-delivery-stop, sig is the
>> >> signal to be injected. Otherwise, sig is ignored.
>> >
>> > There is another special case. If the tracee single-stepps into the
>> > signal handler, it reports SIGTRAP as if it recieved this SIGNAL.
>> > But ptrace(PTRACE, ..., sig) doesn't inject after that.
>>
>> This is part of missing doc about PTRACE_SINGLESTEP.
>> From what you are saying it looks like PTRACE_SINGLESTEP
>> implies PTRACE_SYSCALL behavior: "report syscall-stops".
>
> Hmm. Why do you think so?

I am totally unfamiliar with PTRACE_SINGLESTEP.


Thanks! Expect draft #3 soon.
-- 
vda

Re: Ptrace documentation, draft #1

[Oleg Nesterov]

On 05/20, Denys Vlasenko wrote:
>
> On Thu, May 19, 2011 at 9:49 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> > Once again, PTRACE_KILL == ptrace(PTRACE_CONT, SIGKILL), except it
> > doesn't return the error if the tracee is not stopped.
>
> Oleg, this doesn't explain the resulting behavior in terms understandable
> to mere mortals. *What will happen* when user does ptrace(PTRACE_KILL)?
>
> Yes, it's obvious that the tracee gets SIGKILLed,

No, it is not necessarily killed. Once again, it is equivalent to PTRACE_CONT
except it returns 0 (but does nothing) if the tracee is not stopped.

"does nothing" is not 100% true, it does wake_up_process() but this shouldn't
be documented, this should be fixed.

> >> >> ??? can this SIGTRAP be distinguished from "real" user-generated SIGTRAP
> >> >>     by looking at its siginfo?
> >> >
> >> > Afaics no. Well, except .si_pid shows that the signal was sent by the
> >> > tracing process to itself.
> >>
> >> What about si_code? Is it set to SI_KERNEL for this signal?
> >
> > No, SI_USER.
>
> This is stupid. This signal is sent by kernel. Why is it flagged as "from user"?
> Maybe we should change it?

Well, may be... I dunno actually. Hmm. On a second thought you are probably
right. ptrace_report_syscall() does send_sig() too, but it sets SI_KERNEL.
I am starting to think, at least they should use the same si_code.

> (BTW, where is it generated in the kernel source? I found
> PTRACE_EVENT_EXEC generation, but failed to find
> "old-school SIGTRAP" generation code...)

tracehook_report_exec()->send_sig(SIGTRAP).

Oleg.

ptrace_resume->wake_up_process (Was: Ptrace documentation, draft #1)

[Oleg Nesterov]

On 05/23, Oleg Nesterov wrote:
>
> "does nothing" is not 100% true, it does wake_up_process() but this shouldn't
> be documented, this should be fixed.

In fact ptrace_resume()->wake_up_process() is obviously wrong anyway,
I think the patch below makes sense even for 2.6.40.

But it is much worse in PTRACE_KILL case. Just for example,

	int main(void)
	{
		int child, status;

		child = fork();
		if (!child) {
			int ret;

			assert(ptrace(PTRACE_TRACEME, 0,0,0) == 0);

			ret = pause();
			printf("pause: %d %m\n", ret);

			return 0x23;
		}

		sleep(1);
		assert(ptrace(PTRACE_KILL, child, 0,0) == 0);

		assert(child == wait(&status));
		printf("wait: %x\n", status);

		return 0;
	}

leaks -ERESTARTNOHAND. Yes, we should probably fix sys_pause() as well,
it should check signal_pending(). But we shouldn't allow to wake up the
tracee in unknown state/path.

Can't understand why this wasn't fixed before... I always knew this looks
wrong, but I never sent the patch. Probably because I never understood
the original reason for wake_up_process...

Oleg.

--- a/kernel/ptrace.c
+++ b/kernel/ptrace.c
@@ -706,7 +706,7 @@ static int ptrace_resume(struct task_str
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, TASK_TRACED);
 
 	return 0;
 }

Re: ptrace_resume->wake_up_process (Was: Ptrace documentation, draft #1)

[Linus Torvalds]

On Mon, May 23, 2011 at 7:10 AM, Oleg Nesterov <oleg@redhat.com> wrote:
>
> In fact ptrace_resume()->wake_up_process() is obviously wrong anyway,
> I think the patch below makes sense even for 2.6.40.

Send me a signed-off version with a somewhat cleaned-up commit
message, and we can make that happen.

Or a pull request ..

                  Linus

[PATCH 0/2] Was: ptrace_resume->wake_up_process

[Oleg Nesterov]

On 05/23, Oleg Nesterov wrote:
>
> On 05/23, Oleg Nesterov wrote:
> >
> > "does nothing" is not 100% true, it does wake_up_process() but this shouldn't
> > be documented, this should be fixed.
>
> In fact ptrace_resume()->wake_up_process() is obviously wrong anyway,

So, I am going to add these 2 simple fixes to ptrace branch if they
pass the review. Imho 1/2 makes sense for 2.6.40.

Oleg.

[PATCH 1/2] ptrace: ptrace_resume() shouldn't wake up !TASK_TRACED thread

[Oleg Nesterov]

It is not clear why ptrace_resume() does wake_up_process(). Unless the
caller is PTRACE_KILL the tracee should be TASK_TRACED so we can use
wake_up_state(__TASK_TRACED). If sys_ptrace() races with SIGKILL we do
not need the extra and potentionally spurious wakeup.

If the caller is PTRACE_KILL, wake_up_process() is even more wrong.
The tracee can sleep in any state in any place, and if we have a buggy
code which doesn't handle a spurious wakeup correctly PTRACE_KILL can
be used to exploit it. For example:

	int main(void)
	{
		int child, status;

		child = fork();
		if (!child) {
			int ret;

			assert(ptrace(PTRACE_TRACEME, 0,0,0) == 0);

			ret = pause();
			printf("pause: %d %m\n", ret);

			return 0x23;
		}

		sleep(1);
		assert(ptrace(PTRACE_KILL, child, 0,0) == 0);

		assert(child == wait(&status));
		printf("wait: %x\n", status);

		return 0;
	}

prints "pause: -1 Unknown error 514", -ERESTARTNOHAND leaks to the
userland. In this case sys_pause() is buggy as well and should be
fixed.

I do not know what was the original rationality behind PTRACE_KILL.
The man page is simply wrong and afaics it was always wrong. Imho
it should be deprecated, or may be it should do send_sig(SIGKILL)
as Denys suggests, but in any case I do not think that the current
behaviour was intentional.

Note: there is another problem, ptrace_resume() changes ->exit_code
and this can race with SIGKILL too. Eventually we should change to
not use ->exit_code.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
---

 kernel/ptrace.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

--- sigprocmask/kernel/ptrace.c~ptrace_resume_wakeup	2011-05-23 18:09:48.000000000 +0200
+++ sigprocmask/kernel/ptrace.c	2011-05-23 18:20:18.000000000 +0200
@@ -561,7 +561,7 @@ static int ptrace_resume(struct task_str
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, __TASK_TRACED);
 
 	return 0;
 }

[PATCH 2/2] signal: sys_pause() should check signal_pending()

[Oleg Nesterov]

ERESTART* is always wrong without TIF_SIGPENDING. Teach sys_pause()
to handle the spurious wakeup correctly.

Signed-off-by: Oleg Nesterov <oleg@redhat.com>
---

 kernel/signal.c |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- sigprocmask/kernel/signal.c~pause_ck_signal_pending	2011-05-23 18:09:49.000000000 +0200
+++ sigprocmask/kernel/signal.c	2011-05-23 18:52:24.000000000 +0200
@@ -3023,8 +3023,10 @@ SYSCALL_DEFINE2(signal, int, sig, __sigh
 
 SYSCALL_DEFINE0(pause)
 {
-	current->state = TASK_INTERRUPTIBLE;
-	schedule();
+	while (!signal_pending(current)) {
+		current->state = TASK_INTERRUPTIBLE;
+		schedule();
+	}
 	return -ERESTARTNOHAND;
 }
 

Re: ptrace_resume->wake_up_process (Was: Ptrace documentation, draft #1)

[Oleg Nesterov]

On 05/23, Linus Torvalds wrote:
>
> On Mon, May 23, 2011 at 7:10 AM, Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > In fact ptrace_resume()->wake_up_process() is obviously wrong anyway,
> > I think the patch below makes sense even for 2.6.40.
>
> Send me a signed-off version with a somewhat cleaned-up commit
> message, and we can make that happen.

Ah, I sent the patch a minute ago.

> Or a pull request ..

Whatever is more convenient for you. If everyone agree with this change,
I'll add it to ptrace branch and send [GIT PULL] tomorrow.

Oleg.

[GIT PULL] PTRACE_KILL/wakeup fix for v2.6.40

[Oleg Nesterov]

On 05/23, Oleg Nesterov wrote:
>
> On 05/23, Linus Torvalds wrote:
> >
> > Or a pull request ..
>
> Whatever is more convenient for you. If everyone agree with this change,
> I'll add it to ptrace branch and send [GIT PULL] tomorrow.

OK, nobody replied, I assume there are no objections...

Please pull from

  git://git.kernel.org/pub/scm/linux/kernel/git/oleg/misc.git for-2.6.40

Oleg Nesterov (2):
      ptrace: ptrace_resume() shouldn't wake up !TASK_TRACED thread
      signal: sys_pause() should check signal_pending()

 kernel/ptrace.c |    2 +-
 kernel/signal.c |    6 ++++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/kernel/ptrace.c b/kernel/ptrace.c
index 7a81fc0..2df1157 100644
--- a/kernel/ptrace.c
+++ b/kernel/ptrace.c
@@ -562,7 +562,7 @@ static int ptrace_resume(struct task_struct *child, long request,
 	}
 
 	child->exit_code = data;
-	wake_up_process(child);
+	wake_up_state(child, __TASK_TRACED);
 
 	return 0;
 }
diff --git a/kernel/signal.c b/kernel/signal.c
index ad5e818..86c32b8 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -3023,8 +3023,10 @@ SYSCALL_DEFINE2(signal, int, sig, __sighandler_t, handler)
 
 SYSCALL_DEFINE0(pause)
 {
-	current->state = TASK_INTERRUPTIBLE;
-	schedule();
+	while (!signal_pending(current)) {
+		current->state = TASK_INTERRUPTIBLE;
+		schedule();
+	}
 	return -ERESTARTNOHAND;
 }