[Qemu-devel] VM Hung issue

[Qemu-devel] VM Hung issue

[Umar Draz]

Hello,

I am running qemu-kvm on CentOS 7.3, I have few vms of CentOS and Windows
2012 servers.

Randomly they are hung, but the status of virtual machine keep running, vnc
and network not responding.

I have checked all /var/log/messages and /var/log/libvirt/qemu/vm.log but
nothing found anything wrong.

Here is the following packages list regarding qemu and libvirt.

libvirt-daemon-driver-qemu-2.0.0-10.el7_3.2.x86_64
libvirt-2.0.0-10.el7_3.2.x86_64
ipxe-roms-qemu-20160127-5.git6366fa7a.el7.noarch
qemu-img-1.5.3-126.el7.x86_64
qemu-guest-agent-2.5.0-3.el7.x86_64
qemu-kvm-1.5.3-126.el7.x86_64
qemu-kvm-common-1.5.3-126.el7.x86_64

Would you please help what should I do more so I can find the issue.

-- Umar

Re: [Qemu-devel] VM Hung issue

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 603 bytes --]

On Wed, Jan 18, 2017 at 01:51:26PM +0500, Umar Draz wrote:
> I am running qemu-kvm on CentOS 7.3, I have few vms of CentOS and Windows
> 2012 servers.
> 
> Randomly they are hung, but the status of virtual machine keep running, vnc
> and network not responding.

It's not clear whether the guest is hung or QEMU is hung.

Does the VNC client connect successfully?  If the client fails to
connect then QEMU is hung.  If the client connects but the guest doesn't
respond to input then only the guest is hanging.

If QEMU is hung please post a backtrace:

  # gstack $PID_OF_QEMU

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] VM Hung issue

[Umar Draz]

Hello Stefan,

Well yes, whenever vm hung, then there is nothing on vnc display, and
network of that vm is also down. I can not ping the vm that time.

Mostly my Windows VMs are affected due to this and sometime same thing
happen with Linux vm.

On Wed, Jan 18, 2017 at 3:29 PM, Stefan Hajnoczi <stefanha@gmail.com> wrote:

> On Wed, Jan 18, 2017 at 01:51:26PM +0500, Umar Draz wrote:
> > I am running qemu-kvm on CentOS 7.3, I have few vms of CentOS and Windows
> > 2012 servers.
> >
> > Randomly they are hung, but the status of virtual machine keep running,
> vnc
> > and network not responding.
>
> It's not clear whether the guest is hung or QEMU is hung.
>
> Does the VNC client connect successfully?  If the client fails to
> connect then QEMU is hung.  If the client connects but the guest doesn't
> respond to input then only the guest is hanging.
>
> If QEMU is hung please post a backtrace:
>
>   # gstack $PID_OF_QEMU
>
> Stefan
>

Re: [Qemu-devel] VM Hung issue

[Stefan Hajnoczi]

On Wed, Jan 18, 2017 at 10:46 AM, Umar Draz <unix.co@gmail.com> wrote:
> Well yes, whenever vm hung, then there is nothing on vnc display, and
> network of that vm is also down. I can not ping the vm that time.

Are you saying the VNC client connects successfully to a hung VM but
the display is blank?

Stefan

Re: [Qemu-devel] VM Hung issue

[Umar Draz]

Yes the display is blank, and network also not responding, it means vm
somewhere hung.

On Thu, Jan 19, 2017 at 9:26 PM, Stefan Hajnoczi <stefanha@gmail.com> wrote:

> On Wed, Jan 18, 2017 at 10:46 AM, Umar Draz <unix.co@gmail.com> wrote:
> > Well yes, whenever vm hung, then there is nothing on vnc display, and
> > network of that vm is also down. I can not ping the vm that time.
>
> Are you saying the VNC client connects successfully to a hung VM but
> the display is blank?
>
> Stefan
>

Re: [Qemu-devel] VM Hung issue

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 405 bytes --]

On Thu, Jan 19, 2017 at 10:22:57PM +0500, Umar Draz wrote:
> Yes the display is blank, and network also not responding, it means vm
> somewhere hung.

Try "gstack $PID_OF_QEMU" when the VM is hung.  This will output a
backtrace for all threads in QEMU.  You might need to install the gdb
package to get gstack.

Please post the backtrace.  Maybe we'll be able to identify a deadlock
or other bug.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]