Re: execve-under-ptrace API bug (was Re: Ptrace documentation, draft #3)

[Tejun Heo <tj@kernel.org>]

Hello, Denys.

On Mon, May 30, 2011 at 05:28:17AM +0200, Denys Vlasenko wrote:
> On Wednesday 25 May 2011 16:32, Tejun Heo wrote:
> > > 	1.x execve under ptrace.
> > > 
> > ...
> > >   ** we get death notification: leader died: **
> > >  PID0 exit(0)                            = ?
> > >   ** we get syscall-entry-stop in thread 1: **
> > >  PID1 execve("/bin/foo", "foo" <unfinished ...>
> > >   ** we get syscall-entry-stop in thread 2: **
> > >  PID2 execve("/bin/bar", "bar" <unfinished ...>
> > >   ** we get PTRACE_EVENT_EXEC for PID0, we issue PTRACE_SYSCALL **
> > >   ** we get syscall-exit-stop for PID0: **
> > >  PID0 <... execve resumed> )             = 0
> > > 
> > > ??? Question: WHICH execve succeeded? Can tracer figure it out?
> > 
> > Hmmm... I don't know.  Maybe we can set ptrace message to the original
> > tid?
> 
> The problem with execve is bigger than merely reporting this pid.
>
> Consider how strace tracks its tracees. Currently, it remembers
> their pids - sometimes by remembering clone's return values!
> This is hopelessly broken wrt pid namespaces.

I'm not too familiar with pid namespaces but don't all threads of the
same process belong to the same namespace?  I don't think strace would
need to track pids all the time.  It just needs to store pids of
in-flight exec's and match it on exec completion.  I'm probably
missing something but why wouldn't that work?

> This works (I have a patch against a somewhat older strace),
> but now in light of this "interesting" execve-under-ptrace
> behavior it appears to have a flaw: all threads except the
> execve'ing one disappear without any notification to strace,
> therefore strace doesn't know which tracee data ("struct tcb"
> in strace-speak) need to be dropped!
> 
> I am not sure current strace handles this correctly either.
> I will be very surprised if it does.
> 
> I think the API needs fixing. Tracee must never disappear like that
> on execve (or in any other case). They must always deliver a
> WIFEXITED or WIFSIGNALED notification, allowing tracer to know
> that they are gone. We probably also need to document how are these
> "I died on execve" notifications are ordered wrt PTRACE_EVENT_EXEC
> stop in execve-ing thread.

A problem is that by the time de-threading is in progress, it's
already too deep and there's no way back and the exec'ing thread has
to wait for completion in uninterruptible sleeps - ie. it expects
de-threading to finish in finite amount of time and to achieve that it
basically sends SIGKILL to all other threads.  If we introduce a trap
in de-threading itself, we can easily end up with an unkillable
task.

> Ideas?

But, if necessary, I can think of two other ways,

1. Don't allow more than one thread in the same group enter exec(2)
   path at all.  It's not like parallel execution of exec(2) buys us
   anything anyway.  One thing to be careful about is that binfmt code
   may recurse.

2. Add another trap point right before de-threading commences.  It can
   still back out if de-threading hasn't started yet.  We'll still
   need to add explicit synchronization there but the window would be
   much smaller.

Thanks.

-- 
tejun