Unkillable processes due to PTRACE_TRACEME again

Unkillable processes due to PTRACE_TRACEME again

[Dmitry Vyukov]

Hello,

There was an issue discussed a year ago which leads to
unkillalble/unwaitable zombie processes due to PTRACE_TRACEME:
https://groups.google.com/d/msg/syzkaller/uGzwvhlCXAw/E-cfY2ejAgAJ
and I though it has been fixed by "wait/ptrace: assume __WALL if the
child is traced":
https://groups.google.com/d/msg/syzkaller/caeB1ebZWAs/gcbvcM2HDAAJ

I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
the same unwaitable zombie processes.

Reproducer is:

// autogenerated by syzkaller (http://github.com/google/syzkaller)
#include <pthread.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/syscall.h>
#include <sys/wait.h>
#include <sys/ptrace.h>
#include <signal.h>

void *thr(void *arg)
{
  ptrace(PTRACE_TRACEME, 0, 0, 0);
}

int main()
{
  int pid = fork();
  if (pid == 0) {
    pthread_t th;
    pthread_create(&th, 0, thr, 0);
    usleep(100000);
    exit(0);
  }
  usleep(200000);
  kill(pid, SIGKILL);
  int status = 0;
  waitpid(pid, &status, __WALL);
  return 0;
}

It creates the following process tree:

root     15730  0.0  0.0   1156     4 pts/0    S+   18:33   0:00  |
   \_ ./a.out
root     15731  0.0  0.0      0     0 pts/0    Zl+  18:33   0:00  |
       \_ [a.out] <defunct>

Both threads of the child processes terminated:

# ls -l /proc/15731/task/
total 0
dr-xr-xr-x 7 root root 0 Dec  2 18:33 15731
dr-xr-xr-x 7 root root 0 Dec  2 18:33 15732
root@dvyukov-z840:~# cat /proc/15731/status
Name: a.out
State: Z (zombie)
Tgid: 15731
Ngid: 0
Pid: 15731
PPid: 15730
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 0
Groups: 0
NStgid: 15731
NSpid: 15731
NSpgid: 15730
NSsid: 6670
Threads: 2
SigQ: 1/3098
SigPnd: 0000000000000000
ShdPnd: 0000000000000100
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000

# cat /proc/15732/status
Name: a.out
State: Z (zombie)
Tgid: 15731
Ngid: 0
Pid: 15732
PPid: 15730
TracerPid: 15730
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 0
Groups: 0
NStgid: 15731
NSpid: 15732
NSpgid: 15730
NSsid: 6670
Threads: 2
SigQ: 1/3098
SigPnd: 0000000000000000
ShdPnd: 0000000000000100
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: 0000000180000000
CapInh: 0000000000000000
CapPrm: 0000003fffffffff
CapEff: 0000003fffffffff
CapBnd: 0000003fffffffff
CapAmb: 0000000000000000


The parent process is blocked in waitpid(__WALL):

# cat /proc/15730/stack
[<ffffffff8140fbbb>] do_wait+0x34b/0xb30 kernel/exit.c:1574
[<     inline     >] SYSC_wait4 kernel/exit.c:1684
[<ffffffff814143bd>] SyS_wait4+0x20d/0x340 kernel/exit.c:1653
[<ffffffff81009a24>] do_syscall_64+0x2f4/0x940 arch/x86/entry/common.c:280
[<ffffffff881a3dcd>] entry_SYSCALL64_slow_path+0x25/0x25
arch/x86/entry/entry_64.S:251


Shouldn't __WALL cause the wait to return?

Again, it's easy to change the repro so that the process first
reparents to init, and then does PTRACE_TRACEME. Which will cause a
forever hanged processes.

Thanks

Re: Unkillable processes due to PTRACE_TRACEME again

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 774 bytes --]

On Fri 2016-12-02 19:48:40, Dmitry Vyukov wrote:
> Hello,
> 
> There was an issue discussed a year ago which leads to
> unkillalble/unwaitable zombie processes due to PTRACE_TRACEME:
> https://groups.google.com/d/msg/syzkaller/uGzwvhlCXAw/E-cfY2ejAgAJ
> and I though it has been fixed by "wait/ptrace: assume __WALL if the
> child is traced":
> https://groups.google.com/d/msg/syzkaller/caeB1ebZWAs/gcbvcM2HDAAJ
> 
> I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see

"Now on"?

> the same unwaitable zombie processes.

Well.. I guess git bisect is one option...?

Best regards,
								Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Unkillable processes due to PTRACE_TRACEME again

[Dmitry Vyukov]

On Fri, Dec 2, 2016 at 10:02 PM, Pavel Machek <pavel@ucw.cz> wrote:
> On Fri 2016-12-02 19:48:40, Dmitry Vyukov wrote:
>> Hello,
>>
>> There was an issue discussed a year ago which leads to
>> unkillalble/unwaitable zombie processes due to PTRACE_TRACEME:
>> https://groups.google.com/d/msg/syzkaller/uGzwvhlCXAw/E-cfY2ejAgAJ
>> and I though it has been fixed by "wait/ptrace: assume __WALL if the
>> child is traced":
>> https://groups.google.com/d/msg/syzkaller/caeB1ebZWAs/gcbvcM2HDAAJ
>>
>> I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
>
> "Now on"?

Yes.

>> the same unwaitable zombie processes.
>
> Well.. I guess git bisect is one option...?

I've checked on:

commit 91c4e8ea8f05916df0c8a6f383508ac7c9e10dba
Author: Oleg Nesterov <oleg@redhat.com>
Date:   Mon May 23 16:23:53 2016 -0700
    wait: allow sys_waitid() to accept __WNOTHREAD/__WCLONE/__WALL

and the program hangs the same way. So it seems that it was just never fixed.

Re: Unkillable processes due to PTRACE_TRACEME again

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1375 bytes --]

Hi!

You cc-ed me on initial mail, I replied, and you removed me from cc. Ouch.

On Sat 2016-12-03 16:55:49, Dmitry Vyukov wrote:
> On Fri, Dec 2, 2016 at 10:02 PM, Pavel Machek <pavel@ucw.cz> wrote:
> > On Fri 2016-12-02 19:48:40, Dmitry Vyukov wrote:
> >> Hello,
> >>
> >> There was an issue discussed a year ago which leads to
> >> unkillalble/unwaitable zombie processes due to PTRACE_TRACEME:
> >> https://groups.google.com/d/msg/syzkaller/uGzwvhlCXAw/E-cfY2ejAgAJ
> >> and I though it has been fixed by "wait/ptrace: assume __WALL if the
> >> child is traced":
> >> https://groups.google.com/d/msg/syzkaller/caeB1ebZWAs/gcbvcM2HDAAJ
> >>
> >> I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
> >
> > "Now on"?
> 
> Yes.
> 
> >> the same unwaitable zombie processes.
> >
> > Well.. I guess git bisect is one option...?
> 
> I've checked on:
> 
> commit 91c4e8ea8f05916df0c8a6f383508ac7c9e10dba
> Author: Oleg Nesterov <oleg@redhat.com>
> Date:   Mon May 23 16:23:53 2016 -0700
>     wait: allow sys_waitid() to accept __WNOTHREAD/__WCLONE/__WALL
> 
> and the program hangs the same way. So it seems that it was just never fixed.

Ouch #2. Oleg, any ideas?

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Unkillable processes due to PTRACE_TRACEME again

[Oleg Nesterov]

On 12/02, Dmitry Vyukov wrote:
>
> I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
> the same unwaitable zombie processes.

This is another thing, and notabug. This is how ptrace works,

> void *thr(void *arg)
> {
>   ptrace(PTRACE_TRACEME, 0, 0, 0);
> }
>
> int main()
> {
>   int pid = fork();
>   if (pid == 0) {
>     pthread_t th;
>     pthread_create(&th, 0, thr, 0);
>     usleep(100000);
>     exit(0);
>   }
>   usleep(200000);
>   kill(pid, SIGKILL);
>   int status = 0;
>   waitpid(pid, &status, __WALL);

waitpid(pid) hangs because you need to reap the sub-thread first.

Oleg.

Re: Unkillable processes due to PTRACE_TRACEME again

[Oleg Nesterov]

On 12/05, Oleg Nesterov wrote:
>
> On 12/02, Dmitry Vyukov wrote:
> >
> > I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
> > the same unwaitable zombie processes.
>
> This is another thing, and notabug. This is how ptrace works,
>
> > void *thr(void *arg)
> > {
> >   ptrace(PTRACE_TRACEME, 0, 0, 0);
> > }
> >
> > int main()
> > {
> >   int pid = fork();
> >   if (pid == 0) {
> >     pthread_t th;
> >     pthread_create(&th, 0, thr, 0);
> >     usleep(100000);
> >     exit(0);
> >   }
> >   usleep(200000);
> >   kill(pid, SIGKILL);
> >   int status = 0;
> >   waitpid(pid, &status, __WALL);
>
> waitpid(pid) hangs because you need to reap the sub-thread first.

I'm afraid I wasn't clear...

So the child process has 2 threads, the leader thread L and the sub-thread T.
waitpid(pid == L->pid) will block until all the threads go away, but since T is
traced it won't autoreap, the tracer should do waitpid(T->pid) first to reap
this zombie. waitpid(-1) should work too.

Oleg.

Re: Unkillable processes due to PTRACE_TRACEME again

[Dmitry Vyukov]

On Mon, Dec 5, 2016 at 12:00 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> On 12/05, Oleg Nesterov wrote:
>>
>> On 12/02, Dmitry Vyukov wrote:
>> >
>> > I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
>> > the same unwaitable zombie processes.
>>
>> This is another thing, and notabug. This is how ptrace works,
>>
>> > void *thr(void *arg)
>> > {
>> >   ptrace(PTRACE_TRACEME, 0, 0, 0);
>> > }
>> >
>> > int main()
>> > {
>> >   int pid = fork();
>> >   if (pid == 0) {
>> >     pthread_t th;
>> >     pthread_create(&th, 0, thr, 0);
>> >     usleep(100000);
>> >     exit(0);
>> >   }
>> >   usleep(200000);
>> >   kill(pid, SIGKILL);
>> >   int status = 0;
>> >   waitpid(pid, &status, __WALL);
>>
>> waitpid(pid) hangs because you need to reap the sub-thread first.
>
> I'm afraid I wasn't clear...
>
> So the child process has 2 threads, the leader thread L and the sub-thread T.
> waitpid(pid == L->pid) will block until all the threads go away, but since T is
> traced it won't autoreap, the tracer should do waitpid(T->pid) first to reap
> this zombie. waitpid(-1) should work too.

Do you mean that I need to replace:
 waitpid(pid, &status, __WALL);
with:
 while (waitpid(-1, &status, __WALL) != pid) {}
?
It seems to work. But want to make sure.

Re: Unkillable processes due to PTRACE_TRACEME again

[Oleg Nesterov]

On 12/05, Dmitry Vyukov wrote:
>
> On Mon, Dec 5, 2016 at 12:00 PM, Oleg Nesterov <oleg@redhat.com> wrote:
> > On 12/05, Oleg Nesterov wrote:
> >>
> >> On 12/02, Dmitry Vyukov wrote:
> >> >
> >> > I am not on 2caceb3294a78c389b462e7e236a4e744a53a474 (Dec 1). And see
> >> > the same unwaitable zombie processes.
> >>
> >> This is another thing, and notabug. This is how ptrace works,
> >>
> >> > void *thr(void *arg)
> >> > {
> >> >   ptrace(PTRACE_TRACEME, 0, 0, 0);
> >> > }
> >> >
> >> > int main()
> >> > {
> >> >   int pid = fork();
> >> >   if (pid == 0) {
> >> >     pthread_t th;
> >> >     pthread_create(&th, 0, thr, 0);
> >> >     usleep(100000);
> >> >     exit(0);
> >> >   }
> >> >   usleep(200000);
> >> >   kill(pid, SIGKILL);
> >> >   int status = 0;
> >> >   waitpid(pid, &status, __WALL);
> >>
> >> waitpid(pid) hangs because you need to reap the sub-thread first.
> >
> > I'm afraid I wasn't clear...
> >
> > So the child process has 2 threads, the leader thread L and the sub-thread T.
> > waitpid(pid == L->pid) will block until all the threads go away, but since T is
> > traced it won't autoreap, the tracer should do waitpid(T->pid) first to reap
> > this zombie. waitpid(-1) should work too.
> 
> Do you mean that I need to replace:
>  waitpid(pid, &status, __WALL);
> with:
>  while (waitpid(-1, &status, __WALL) != pid) {}
> ?

Yes. Or, if you knew the pid of the traced thread you could do

	// need to do this first, the traced sub-thread won't autoreap,
	// and the leader which represents the whole process is not reapable
	// until all other threads go away
	waitpid(tracee_pid, &status, __WALL);

	waitpid(pid, &status, __WALL);

Oleg.