[uml-devel] How to reliably kill all vmlinux processes for a single VM?

[uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

I found by experimentation that killing (SIGTERM) the first vmlinux
process only kills part of a UML virtual machine.  There are still
vmlinux processes (or threads?) running.

Compare the process listings below before and after sending the
SIGTERM signal to the head process (25356).

Is there a way to reliably kill a single UML VM?  Note that "killall
vmlinux" isn't a suitable answer because I don't want to kill every
UML instance on the host.  Also using process groups is difficult
because it stops ^C (in the parent process) from killing the VM.

I'm thinking perhaps naming the processes using umid=<unique> and
iterating over the process table ...

Rich.

Before:

25356 pts/4    S      0:01 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25367 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25388 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25437 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3

After:

25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3



-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[richard -rw- weinberger]

On Sun, Aug 11, 2013 at 7:48 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
> I found by experimentation that killing (SIGTERM) the first vmlinux
> process only kills part of a UML virtual machine.  There are still
> vmlinux processes (or threads?) running.
>
> Compare the process listings below before and after sending the
> SIGTERM signal to the head process (25356).
>
> Is there a way to reliably kill a single UML VM?  Note that "killall
> vmlinux" isn't a suitable answer because I don't want to kill every
> UML instance on the host.  Also using process groups is difficult
> because it stops ^C (in the parent process) from killing the VM.
>
> I'm thinking perhaps naming the processes using umid=<unique> and
> iterating over the process table ...

Can you try SIGINT?
SIGTERM should work but I fear you found an issue.

Why do you need to kill UML?
Can't you shutdown it?

Thanks,
//richard

> Rich.
>
> Before:
>
> 25356 pts/4    S      0:01 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25367 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25388 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25437 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>
> After:
>
> 25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
> 25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>
>
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> virt-df lists disk usage of guests without needing to install any
> software inside the virtual machine.  Supports Linux and Windows.
> http://people.redhat.com/~rjones/virt-df/
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> User-mode-linux-devel mailing list
> User-mode-linux-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel



-- 
Thanks,
//richard

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[richard -rw- weinberger]

On Mon, Aug 12, 2013 at 9:37 AM, richard -rw- weinberger
<richard.weinberger@gmail.com> wrote:
> On Sun, Aug 11, 2013 at 7:48 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
>> I found by experimentation that killing (SIGTERM) the first vmlinux
>> process only kills part of a UML virtual machine.  There are still
>> vmlinux processes (or threads?) running.
>>
>> Compare the process listings below before and after sending the
>> SIGTERM signal to the head process (25356).
>>
>> Is there a way to reliably kill a single UML VM?  Note that "killall
>> vmlinux" isn't a suitable answer because I don't want to kill every
>> UML instance on the host.  Also using process groups is difficult
>> because it stops ^C (in the parent process) from killing the VM.
>>
>> I'm thinking perhaps naming the processes using umid=<unique> and
>> iterating over the process table ...
>
> Can you try SIGINT?
> SIGTERM should work but I fear you found an issue.

Found the root cause, patch is on the way.

> Why do you need to kill UML?
> Can't you shutdown it?
>
> Thanks,
> //richard
>
>> Rich.
>>
>> Before:
>>
>> 25356 pts/4    S      0:01 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25367 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25388 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25437 pts/4    t      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>>
>> After:
>>
>> 25364 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25365 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>> 25366 pts/4    S      0:00 /home/rjones/d/linux/vmlinux mem=500M initrd=/home/rjones/d/libguestfs/tmp/.guestfs-1000/initrd.25291 init=/init guestfs_noreboot=1 rw panic=1 TERM=xterm-256color selinux=0 ubd0=/tmp/test1.img ubd1=/home/rjones/d/libguestfs/tmp/libguestfsUtAePE/cow0 root=/dev/ubdb ssl3=fd:6 guestfs_channel=/dev/ttyS3
>>
>>
>>
>> --
>> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
>> virt-df lists disk usage of guests without needing to install any
>> software inside the virtual machine.  Supports Linux and Windows.
>> http://people.redhat.com/~rjones/virt-df/
>>
>> ------------------------------------------------------------------------------
>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>> It's a free troubleshooting tool designed for production.
>> Get down to code-level detail for bottlenecks, with <2% overhead.
>> Download for free and get started troubleshooting in minutes.
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> User-mode-linux-devel mailing list
>> User-mode-linux-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel
>
>
>
> --
> Thanks,
> //richard



-- 
Thanks,
//richard

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

On Mon, Aug 12, 2013 at 10:15:24AM +0200, richard -rw- weinberger wrote:
> Found the root cause, patch is on the way.

I can test patches if you CC me on them.

> > Why do you need to kill UML?
> > Can't you shutdown it?

In the qemu/KVM case it's safe to kill the qemu process, as qemu
catches the signal and shuts down safely (in particular: it properly
flushes any writes in flight).  I just copied that same code from the
qemu backend to the UML backend.  The libguestfs test suite includes a
test to determine if data is flushed properly on shutdown.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

On Mon, Aug 12, 2013 at 09:53:00AM +0100, Richard W.M. Jones wrote:
> On Mon, Aug 12, 2013 at 10:15:24AM +0200, richard -rw- weinberger wrote:
> > Found the root cause, patch is on the way.
> 
> I can test patches if you CC me on them.

I'm still available to test patches :-)  Didn't see anything on
this list nor on LKML.

In particular I'm having a problem where it looks as if vmlinux is
sending a signal to its parent process on shutdown.

I can reliably reproduce this, although not with anything very
minimal.  But if you run the libguestfs test suite like this you'll
see it:

  make -C tests/regressions check TESTS=rhbz914931

The parent process (a C program which for unrelated reasons is called
'rhbz914931') receives a SIGTERM.  I checked the siginfo struct for
this signal, and it appears to come from the vmlinux main process,
which should not be happening.

So .. possibly there's something awry with how vmlinux delivers
signals to its child processes on shutdown which is causing it both to
miss out some children, and to kill other unrelated processes.  Or
maybe this is just a coincidence.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming blog: http://rwmj.wordpress.com
Fedora now supports 80 OCaml packages (the OPEN alternative to F#)

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard Weinberger]

Am 13.08.2013 12:36, schrieb Richard W.M. Jones:
> On Mon, Aug 12, 2013 at 09:53:00AM +0100, Richard W.M. Jones wrote:
>> On Mon, Aug 12, 2013 at 10:15:24AM +0200, richard -rw- weinberger wrote:
>>> Found the root cause, patch is on the way.
>>
>> I can test patches if you CC me on them.
> 
> I'm still available to test patches :-)  Didn't see anything on
> this list nor on LKML.

I'm currently on the road...

> In particular I'm having a problem where it looks as if vmlinux is
> sending a signal to its parent process on shutdown.
>

Really?
If so, why does it not kill my shell if I run it directly?

Thanks,
//richard

> I can reliably reproduce this, although not with anything very
> minimal.  But if you run the libguestfs test suite like this you'll
> see it:
> 
>   make -C tests/regressions check TESTS=rhbz914931
> 
> The parent process (a C program which for unrelated reasons is called
> 'rhbz914931') receives a SIGTERM.  I checked the siginfo struct for
> this signal, and it appears to come from the vmlinux main process,
> which should not be happening.
> 
> So .. possibly there's something awry with how vmlinux delivers
> signals to its child processes on shutdown which is causing it both to
> miss out some children, and to kill other unrelated processes.  Or
> maybe this is just a coincidence.
> 
> Rich.
> 


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

On Tue, Aug 13, 2013 at 12:49:25PM +0200, Richard Weinberger wrote:
> Am 13.08.2013 12:36, schrieb Richard W.M. Jones:
> > In particular I'm having a problem where it looks as if vmlinux is
> > sending a signal to its parent process on shutdown.
> 
> Really?
> If so, why does it not kill my shell if I run it directly?

I've no idea.  Perhaps you never send SIGTERM to the vmlinux process,
or maybe this is just a very strange bug.  Although it happens
reliably in two tests from the test suite, it doesn't happen in any of
the other tests (about 100 of them).

Perhaps I'm wrong about the nature of the bug and the signal is coming
from somewhere else.  Is there a way that a process inside the guest
could send a signal to a process on the host?  (I would hope not, but
have to ask in case something in the guest is going rogue).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

OK, I understand what's going wrong and how to reproduce the
signal-to-parent problem.  It does look like a bug in vmlinux.

It happens if the init process (PID 1) inside the VM gets a segfault.

In libguestfs we can force that easily, as there is a test path for
exercising segfaults in our init process:

----------------------------------------------------------------------
#!/bin/bash -
export LIBGUESTFS_BACKEND=uml
export LIBGUESTFS_QEMU=/home/rjones/d/linux/vmlinux
export LIBGUESTFS_DEBUG=1
export LIBGUESTFS_TRACE=1
./run strace -o /tmp/strace.log ./fish/guestfish -a /dev/null <<EOF
run
get-pid
debug segv 1
!sleep 5
EOF
----------------------------------------------------------------------

The output of this and the strace log file is here:

http://oirase.annexia.org/tmp/uml-signal/

output.log.txt shows the PID of vmlinux is 6856.

strace.log.txt shows that the parent process gets terminated by
SIGTERM *from* PID 6856:

--- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=6856, si_uid=1000} ---
+++ killed by SIGTERM +++

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

On Tue, Aug 13, 2013 at 01:04:01PM +0100, Richard W.M. Jones wrote:
> OK, I understand what's going wrong and how to reproduce the
> signal-to-parent problem.  It does look like a bug in vmlinux.
> 
> It happens if the init process (PID 1) inside the VM gets a segfault.

To add an additional data-point, applying the attached patch causes
the rogue SIGTERM-to-parent signal to disappear.  So at least we know
where it comes from!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

[-- Attachment #2: 0001-COMMENT-OUT-KILL-SIGTERM.patch --]
[-- Type: text/plain, Size: 700 bytes --]

>>From 73e2153039803d6f8cec7425ae501f857d9523a9 Mon Sep 17 00:00:00 2001
From: "Richard W.M. Jones" <rjones@redhat.com>
Date: Tue, 13 Aug 2013 13:24:32 +0100
Subject: [PATCH] COMMENT OUT KILL SIGTERM

---
 arch/um/os-Linux/util.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/um/os-Linux/util.c b/arch/um/os-Linux/util.c
index 492ef5e..a0f308e 100644
--- a/arch/um/os-Linux/util.c
+++ b/arch/um/os-Linux/util.c
@@ -108,7 +108,7 @@ void os_dump_core(void)
 	 */
 
 	signal(SIGTERM, SIG_IGN);
-	kill(0, SIGTERM);
+	//kill(0, SIGTERM);
 	/*
 	 * Most of the other processes associated with this UML are
 	 * likely sTopped, so give them a SIGCONT so they see the
-- 
1.8.3.1


[-- Attachment #3: Type: text/plain, Size: 405 bytes --]

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk

[-- Attachment #4: Type: text/plain, Size: 194 bytes --]

_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[richard -rw- weinberger]

On Tue, Aug 13, 2013 at 2:25 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
> On Tue, Aug 13, 2013 at 01:04:01PM +0100, Richard W.M. Jones wrote:
>> OK, I understand what's going wrong and how to reproduce the
>> signal-to-parent problem.  It does look like a bug in vmlinux.
>>
>> It happens if the init process (PID 1) inside the VM gets a segfault.
>
> To add an additional data-point, applying the attached patch causes
> the rogue SIGTERM-to-parent signal to disappear.  So at least we know
> where it comes from!

I think we should call setsid() at UML startup.
Thanks for reporting that issue.

BTW: You know that UML has a management console?
http://user-mode-linux.sourceforge.net/old/mconsole.html

Killing UML by SIGTERM is not nice. :D

-- 
Thanks,
//richard

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] How to reliably kill all vmlinux processes for a single VM?

[Richard W.M. Jones]

On Thu, Aug 15, 2013 at 07:34:04PM +0200, richard -rw- weinberger wrote:
> On Tue, Aug 13, 2013 at 2:25 PM, Richard W.M. Jones <rjones@redhat.com> wrote:
> > On Tue, Aug 13, 2013 at 01:04:01PM +0100, Richard W.M. Jones wrote:
> >> OK, I understand what's going wrong and how to reproduce the
> >> signal-to-parent problem.  It does look like a bug in vmlinux.
> >>
> >> It happens if the init process (PID 1) inside the VM gets a segfault.
> >
> > To add an additional data-point, applying the attached patch causes
> > the rogue SIGTERM-to-parent signal to disappear.  So at least we know
> > where it comes from!
> 
> I think we should call setsid() at UML startup.
> Thanks for reporting that issue.
> 
> BTW: You know that UML has a management console?
> http://user-mode-linux.sourceforge.net/old/mconsole.html

Yup, using the management console could be a good idea.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead. 
Download for free and get started troubleshooting in minutes. 
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel