[Qemu-devel] [Bug 1490853] [NEW] qemu windows guest hangs on 100% cpu usage

* [Qemu-devel] [Bug 1490853] [NEW] qemu windows guest hangs on 100% cpu usage
@ 2015-09-01  7:27 yebiaoxiang
  2015-09-01  7:46 ` [Qemu-devel] [Bug 1490853] " yebiaoxiang
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: yebiaoxiang @ 2015-09-01  7:27 UTC (permalink / raw)
  To: qemu-devel

Public bug reported:

hi:
I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit.
When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage.
I try to rebuild kvm and add some debug info , I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION.
It seems like all the vcpu always in spinlock waiting. I'm not sure whether it's qemu's bug or kvm's.
Any help would be appreciated.

How reproducible:
WinXP: seems always.
WinServer2008: rare.

Steps to Reproduce:
winXP: 1. hot reboot the xp guest os,  hot reboot is necessary.
WinServer2008: not sure, I didn't do anything, it just happened.

The different between WinXP and WInServer2008:
1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine.
2. When WinServer2008 hangs,  the vnc show the last screen and the screen won't change anything include system time.
3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs.

qemu version:
QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard
host info:
Ubuntu 12.04 LTS \n \l
Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux

 qemu command line (guest OS XP):
root      7124 1178  7.6 7750360 3761644 ?     Sl   14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

 all qemu thread (guest OS XP):
root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124
top - 14:37:05 up 7 days,  4:07,  1 user,  load average: 10.71, 10.90, 10.19
Tasks:  14 total,  12 running,   2 sleeping,   0 stopped,   0 zombie
Cpu(s): 38.8%us, 11.2%sy,  0.0%ni, 50.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  49159888k total, 35665128k used, 13494760k free,   436312k buffers
Swap:  8803324k total,        0k used,  8803324k free, 28595100k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND
 7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm
 7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm
 7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm
 7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm
 7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm
 7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm
 7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm
 7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm
 7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm
 7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm
 7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm
 7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm
 7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm
 7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm

all thread's kernel stack (guest OS XP):
root@cvknode2026:/proc/7124/task# cat 7130/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7132/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7133/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7135/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel]
[<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
[<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
[<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
[<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
[<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7137/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7126/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7127/stack
[<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel]
[<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7128/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7129/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7131/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7134/stack
[<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel]
[<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
[<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
[<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
[<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
[<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
[<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7136/stack
[<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7124/stack
[<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70
[<ffffffff811d678a>] do_sys_poll+0x50a/0x590
[<ffffffff811d68eb>] SyS_poll+0x6b/0x100
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvknode2026:/proc/7124/task# cat 7139/stack
[<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
[<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
[<ffffffff810ddc14>] do_futex+0x334/0xb70
[<ffffffff810de592>] SyS_futex+0x142/0x1a0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff

 qemu command line (guest OS WinServer2008):
root     25258  996 21.5 21174412 14181580 ?   Sl   Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so
cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus
=pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser
ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

 all qemu thread (guest OS WinServer2008):
 top -d 1 -H -p 25258
top - 14:53:37 up 24 days, 21:27,  2 users,  load average: 19.12, 20.56, 20.20
Tasks:  14 total,  13 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s): 48.1%us, 18.2%sy,  0.0%ni, 33.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65674944k total, 64651012k used,  1023932k free,   194608k buffers
Swap:  8803324k total,  4140324k used,  4663000k free,   363712k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND
25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm
25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm
25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm
25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm
25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm
25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm
25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm
25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm
25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm
25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm
25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm
25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm
25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm
25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm

all thread's kernel stack (guest OS WinServer2008):
root@cvk11:/proc/25258/task# cat 25281/stack
[<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
[<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm]
[<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
[<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
[<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
[<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25284/stack
[<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm]
[<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm]
[<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25294/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25287/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25299/stack
[<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel]
[<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
[<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
[<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
[<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
[<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
[<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25258/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25269/stack
[<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
[<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
[<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
[<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
[<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
[<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
[<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25301/stack
[<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25292/stack
[<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25297/stack
[<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25272/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25277/stack
[<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25290/stack
[<ffffffffffffffff>] 0xffffffffffffffff
root@cvk11:/proc/25258/task# cat 25314/stack
[<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
[<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
[<ffffffff810ddc14>] do_futex+0x334/0xb70
[<ffffffff810de592>] SyS_futex+0x142/0x1a0
[<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
[<ffffffffffffffff>] 0xffffffffffffffff

** Affects: qemu
     Importance: Undecided
         Status: New

** Tags: 100 cpu hangs kvm qemu usage windows

** Description changed:

  hi:
- I have two VM , one is winXP Prefessional SP3 32bit, another on is WindowsServer2008 Enterprise SP2 64bit.
- When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage. 
+ I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit.
+ When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage.
  I try to rebuild kvm and add some debug info , I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION.
  It seems like all the vcpu always in spinlock waiting. I not sure it's qemu's bug or kvm's.
  Any help would be appreciated.

  How reproducible:
  WinXP: seems always.
  WinServer2008: rare.

  Steps to Reproduce:
  winXP: 1. hot reboot the xp guest os,  hot reboot is necessary.
  WinServer2008: not sure, I didn't do anything, it just happened.

  The different between WinXP and WInServer2008:
- 1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine. 
+ 1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine.
  2. When WinServer2008 hangs,  the vnc show the last screen and the screen won't change anything include system time.
  3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs.

  qemu version:
  QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard
  host info:
  Ubuntu 12.04 LTS \n \l
  Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux

- 
-  qemu command line (guest OS XP):
+  qemu command line (guest OS XP):
  root      7124 1178  7.6 7750360 3761644 ?     Sl   14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

- 
-  all qemu thread (guest OS XP):
+  all qemu thread (guest OS XP):
  root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124
  top - 14:37:05 up 7 days,  4:07,  1 user,  load average: 10.71, 10.90, 10.19
  Tasks:  14 total,  12 running,   2 sleeping,   0 stopped,   0 zombie
  Cpu(s): 38.8%us, 11.2%sy,  0.0%ni, 50.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  49159888k total, 35665128k used, 13494760k free,   436312k buffers
  Swap:  8803324k total,        0k used,  8803324k free, 28595100k cached

-   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND                                                  
-  7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm                                                      
-  7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm                                                      
-  7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm                                                      
-  7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm                                                      
-  7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm                                                      
-  7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm                                                      
-  7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm                                                      
-  7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm                                                      
-  7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm                                                      
-  7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm                                                      
-  7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm                                                      
-  7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm                                                      
-  7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm                                                      
-  7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm       
+   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND
+  7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm
+  7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm
+  7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm
+  7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm
+  7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm
+  7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm
+  7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm
+  7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm
+  7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm
+  7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm
+  7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm
+  7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm
+  7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm
+  7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm

  all thread's kernel stack (guest OS XP):
  root@cvknode2026:/proc/7124/task# cat 7130/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7132/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7133/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7135/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7137/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7126/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7127/stack
  [<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7128/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7129/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7131/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7134/stack
  [<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7136/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7124/stack
  [<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70
  [<ffffffff811d678a>] do_sys_poll+0x50a/0x590
  [<ffffffff811d68eb>] SyS_poll+0x6b/0x100
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7139/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

-  qemu command line (guest OS WinServer2008):
+  qemu command line (guest OS WinServer2008):
  root     25258  996 21.5 21174412 14181580 ?   Sl   Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so
  cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus
  =pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser
  ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

-  all qemu thread (guest OS WinServer2008):
-  top -d 1 -H -p 25258
+  all qemu thread (guest OS WinServer2008):
+  top -d 1 -H -p 25258
  top - 14:53:37 up 24 days, 21:27,  2 users,  load average: 19.12, 20.56, 20.20
  Tasks:  14 total,  13 running,   1 sleeping,   0 stopped,   0 zombie
  Cpu(s): 48.1%us, 18.2%sy,  0.0%ni, 33.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  65674944k total, 64651012k used,  1023932k free,   194608k buffers
  Swap:  8803324k total,  4140324k used,  4663000k free,   363712k cached

-   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND                                
- 25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm                                    
- 25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm                                    
- 25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm                                    
- 25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm                                    
- 25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm                                    
- 25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm                                    
- 25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm                                    
- 25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm                                    
- 25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm                                    
- 25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm                                    
- 25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm                                    
- 25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm                                    
- 25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm                                    
- 25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm   
+   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND
+ 25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm
+ 25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm
+ 25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm
+ 25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm
+ 25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm
+ 25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm
+ 25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm
+ 25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm
+ 25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm
+ 25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm
+ 25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm
+ 25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm
+ 25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm
+ 25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm

  all thread's kernel stack (guest OS WinServer2008):
  root@cvk11:/proc/25258/task# cat 25281/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25284/stack
  [<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm]
  [<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm]
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25294/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25287/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25299/stack
  [<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25258/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25269/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25301/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25292/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25297/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25272/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25277/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25290/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25314/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

** Description changed:

  hi:
  I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit.
  When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage.
  I try to rebuild kvm and add some debug info , I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION.
- It seems like all the vcpu always in spinlock waiting. I not sure it's qemu's bug or kvm's.
+ It seems like all the vcpu always in spinlock waiting. I'm not sure it's qemu's bug or kvm's.
  Any help would be appreciated.

  How reproducible:
  WinXP: seems always.
  WinServer2008: rare.

  Steps to Reproduce:
  winXP: 1. hot reboot the xp guest os,  hot reboot is necessary.
  WinServer2008: not sure, I didn't do anything, it just happened.

  The different between WinXP and WInServer2008:
  1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine.
  2. When WinServer2008 hangs,  the vnc show the last screen and the screen won't change anything include system time.
  3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs.

  qemu version:
  QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard
  host info:
  Ubuntu 12.04 LTS \n \l
  Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux

   qemu command line (guest OS XP):
  root      7124 1178  7.6 7750360 3761644 ?     Sl   14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

   all qemu thread (guest OS XP):
  root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124
  top - 14:37:05 up 7 days,  4:07,  1 user,  load average: 10.71, 10.90, 10.19
  Tasks:  14 total,  12 running,   2 sleeping,   0 stopped,   0 zombie
  Cpu(s): 38.8%us, 11.2%sy,  0.0%ni, 50.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  49159888k total, 35665128k used, 13494760k free,   436312k buffers
  Swap:  8803324k total,        0k used,  8803324k free, 28595100k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND
   7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm
   7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm
   7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm
   7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm
   7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm
   7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm
   7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm
   7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm
   7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm
   7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm
   7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm
   7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm
   7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm
   7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm

  all thread's kernel stack (guest OS XP):
  root@cvknode2026:/proc/7124/task# cat 7130/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7132/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7133/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7135/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7137/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7126/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7127/stack
  [<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7128/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7129/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7131/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7134/stack
  [<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7136/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7124/stack
  [<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70
  [<ffffffff811d678a>] do_sys_poll+0x50a/0x590
  [<ffffffff811d68eb>] SyS_poll+0x6b/0x100
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7139/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

   qemu command line (guest OS WinServer2008):
  root     25258  996 21.5 21174412 14181580 ?   Sl   Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so
  cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus
  =pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser
  ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

   all qemu thread (guest OS WinServer2008):
   top -d 1 -H -p 25258
  top - 14:53:37 up 24 days, 21:27,  2 users,  load average: 19.12, 20.56, 20.20
  Tasks:  14 total,  13 running,   1 sleeping,   0 stopped,   0 zombie
  Cpu(s): 48.1%us, 18.2%sy,  0.0%ni, 33.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  65674944k total, 64651012k used,  1023932k free,   194608k buffers
  Swap:  8803324k total,  4140324k used,  4663000k free,   363712k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND
  25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm
  25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm
  25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm
  25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm
  25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm
  25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm
  25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm
  25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm
  25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm
  25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm
  25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm
  25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm
  25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm
  25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm

  all thread's kernel stack (guest OS WinServer2008):
  root@cvk11:/proc/25258/task# cat 25281/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25284/stack
  [<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm]
  [<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm]
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25294/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25287/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25299/stack
  [<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25258/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25269/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25301/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25292/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25297/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25272/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25277/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25290/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25314/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

** Description changed:

  hi:
  I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit.
  When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage.
  I try to rebuild kvm and add some debug info , I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION.
- It seems like all the vcpu always in spinlock waiting. I'm not sure it's qemu's bug or kvm's.
+ It seems like all the vcpu always in spinlock waiting. I'm not sure whether it's qemu's bug or kvm's.
  Any help would be appreciated.

  How reproducible:
  WinXP: seems always.
  WinServer2008: rare.

  Steps to Reproduce:
  winXP: 1. hot reboot the xp guest os,  hot reboot is necessary.
  WinServer2008: not sure, I didn't do anything, it just happened.

  The different between WinXP and WInServer2008:
  1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine.
  2. When WinServer2008 hangs,  the vnc show the last screen and the screen won't change anything include system time.
  3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs.

  qemu version:
  QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard
  host info:
  Ubuntu 12.04 LTS \n \l
  Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux

   qemu command line (guest OS XP):
  root      7124 1178  7.6 7750360 3761644 ?     Sl   14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

   all qemu thread (guest OS XP):
  root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124
  top - 14:37:05 up 7 days,  4:07,  1 user,  load average: 10.71, 10.90, 10.19
  Tasks:  14 total,  12 running,   2 sleeping,   0 stopped,   0 zombie
  Cpu(s): 38.8%us, 11.2%sy,  0.0%ni, 50.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  49159888k total, 35665128k used, 13494760k free,   436312k buffers
  Swap:  8803324k total,        0k used,  8803324k free, 28595100k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND
   7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm
   7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm
   7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm
   7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm
   7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm
   7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm
   7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm
   7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm
   7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm
   7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm
   7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm
   7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm
   7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm
   7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm

  all thread's kernel stack (guest OS XP):
  root@cvknode2026:/proc/7124/task# cat 7130/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7132/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7133/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7135/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7137/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7126/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7127/stack
  [<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7128/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7129/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7131/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7134/stack
  [<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7136/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7124/stack
  [<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70
  [<ffffffff811d678a>] do_sys_poll+0x50a/0x590
  [<ffffffff811d68eb>] SyS_poll+0x6b/0x100
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7139/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

   qemu command line (guest OS WinServer2008):
  root     25258  996 21.5 21174412 14181580 ?   Sl   Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so
  cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus
  =pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser
  ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

   all qemu thread (guest OS WinServer2008):
   top -d 1 -H -p 25258
  top - 14:53:37 up 24 days, 21:27,  2 users,  load average: 19.12, 20.56, 20.20
  Tasks:  14 total,  13 running,   1 sleeping,   0 stopped,   0 zombie
  Cpu(s): 48.1%us, 18.2%sy,  0.0%ni, 33.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  65674944k total, 64651012k used,  1023932k free,   194608k buffers
  Swap:  8803324k total,  4140324k used,  4663000k free,   363712k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND
  25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm
  25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm
  25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm
  25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm
  25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm
  25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm
  25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm
  25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm
  25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm
  25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm
  25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm
  25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm
  25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm
  25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm

  all thread's kernel stack (guest OS WinServer2008):
  root@cvk11:/proc/25258/task# cat 25281/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25284/stack
  [<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm]
  [<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm]
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25294/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25287/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25299/stack
  [<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25258/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25269/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25301/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25292/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25297/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25272/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25277/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25290/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25314/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1490853

Title:
  qemu windows guest hangs on 100% cpu usage

Status in QEMU:
  New

Bug description:
  hi:
  I have two VM , one is winXP Prefessional SP3 32bit, another one is WindowsServer2008 Enterprise SP2 64bit.
  When I hot reboot winXP in guest OS, it'll hangs on progress bar, and all the vcpu thread in qemu is 100% usage.
  I try to rebuild kvm and add some debug info , I found the cpu exit reason is EXIT_REASON_PAUSE_INSTRUCTION.
  It seems like all the vcpu always in spinlock waiting. I'm not sure whether it's qemu's bug or kvm's.
  Any help would be appreciated.

  How reproducible:
  WinXP: seems always.
  WinServer2008: rare.

  Steps to Reproduce:
  winXP: 1. hot reboot the xp guest os,  hot reboot is necessary.
  WinServer2008: not sure, I didn't do anything, it just happened.

  The different between WinXP and WInServer2008:
  1. When WinXP hangs, the boot progress bar is rolling, I think that vnc is work fine.
  2. When WinServer2008 hangs,  the vnc show the last screen and the screen won't change anything include system time.
  3. When the VM hangs , if I execute "virsh suspend vm-name" and "virsh resume vm-name", the WinServer2008 will change to normal , and work fine not hangs anymore. But WinXP not change anything, still hangs.

  qemu version:
  QEMU emulator version 1.5.0, Copyright (c) 2003-2008 Fabrice Bellard
  host info:
  Ubuntu 12.04 LTS \n \l
  Linux cvknode2026 3.13.6 #1 SMP Fri Dec 12 09:17:35 CST 2014 x86_64 x86_64 x86_64 GNU/Linux

   qemu command line (guest OS XP):
  root      7124 1178  7.6 7750360 3761644 ?     Sl   14:02 435:23 /usr/bin/kvm -name x -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 6144 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid d3832129-f77d-4b21-bbf7-fd337f53e572 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/x.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive file=/vms/images/sn1-of-ff.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=directsync -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=24,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=0c:da:41:1d:f8:40,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/x.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:0 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

   all qemu thread (guest OS XP):
  root@cvknode2026:/proc/7124/task# top -d 1 -H -p 7124
  top - 14:37:05 up 7 days,  4:07,  1 user,  load average: 10.71, 10.90, 10.19
  Tasks:  14 total,  12 running,   2 sleeping,   0 stopped,   0 zombie
  Cpu(s): 38.8%us, 11.2%sy,  0.0%ni, 50.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  49159888k total, 35665128k used, 13494760k free,   436312k buffers
  Swap:  8803324k total,        0k used,  8803324k free, 28595100k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P SWAP WCHAN     COMMAND
   7130 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.48  3 3.8g -         kvm
   7132 root      20   0 7568m 3.6g 6628 R  101  7.7  33:43.13  1 3.8g -         kvm
   7133 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.70  6 3.8g -         kvm
   7135 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.33 11 3.8g -         kvm
   7137 root      20   0 7568m 3.6g 6628 R  101  7.7  33:42.59 17 3.8g -         kvm
   7126 root      20   0 7568m 3.6g 6628 R  100  7.7  34:06.76  4 3.8g -         kvm
   7127 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.14  8 3.8g -         kvm
   7128 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64 13 3.8g -         kvm
   7129 root      20   0 7568m 3.6g 6628 R  100  7.7  33:43.64  7 3.8g -         kvm
   7131 root      20   0 7568m 3.6g 6628 R  100  7.7  33:44.24 10 3.8g -         kvm
   7134 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.47 12 3.8g -         kvm
   7136 root      20   0 7568m 3.6g 6628 R  100  7.7  33:42.16  2 3.8g -         kvm
   7124 root      20   0 7568m 3.6g 6628 S    1  7.7   0:30.65 14 3.8g poll_sche kvm
   7139 root      20   0 7568m 3.6g 6628 S    0  7.7   0:01.71 14 3.8g futex_wai kvm

  all thread's kernel stack (guest OS XP):
  root@cvknode2026:/proc/7124/task# cat 7130/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7132/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7133/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7135/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa02b6788>] vmx_vcpu_run+0x88/0x760 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7137/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7126/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7127/stack
  [<ffffffffa02b74f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7128/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7129/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7131/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7134/stack
  [<ffffffffa02b74fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa02ba0d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa0413aec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa0414188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa03fc042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7136/stack
  [<ffffffffa02b1fa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7124/stack
  [<ffffffff811d50c9>] poll_schedule_timeout+0x49/0x70
  [<ffffffff811d678a>] do_sys_poll+0x50a/0x590
  [<ffffffff811d68eb>] SyS_poll+0x6b/0x100
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvknode2026:/proc/7124/task# cat 7139/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

   qemu command line (guest OS WinServer2008):
  root     25258  996 21.5 21174412 14181580 ?   Sl   Aug27 73740:11 /usr/bin/kvm -name zjx_1-clone -S -machine pc-i440fx-1.5,accel=kvm,usb=off,system=windows -cpu qemu64,hv_relaxed,hv_spinlocks=0x2000 -m 16384 -smp 12,maxcpus=72,sockets=12,cores=6,threads=1 -uuid 8c8b9abf-e9a6-4c3e-93cd-137a9550e593 -no-user-config -nodefaults -chardev so
  cket,id=charmonitor,path=/var/lib/libvirt/qemu/zjx_1-clone.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -no-hpet -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device usb-ehci,id=ehci,bus=pci.0,addr=0x4 -device virtio-serial-pci,id=virtio-serial0,bus
  =pci.0,addr=0x5 -drive file=/vms/aaa/zjx_1-clone.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=directsync -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/vms/isos/virtio-win2008R2.vfd,if=none,id=drive-fdc0-0-0,readonly=on,format=raw,cache=directsync -global isa-fdc.driveA=drive-fdc0-0-0 -drive if=none,id=drive-ide0-1-1,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1,bootindex=2 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0c:da:41:1d:b6:47,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-ser
  ial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/zjx_1-clone.agent,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0,bus=usb.0 -vnc 0.0.0.0:3 -device VGA,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7

   all qemu thread (guest OS WinServer2008):
   top -d 1 -H -p 25258
  top - 14:53:37 up 24 days, 21:27,  2 users,  load average: 19.12, 20.56, 20.20
  Tasks:  14 total,  13 running,   1 sleeping,   0 stopped,   0 zombie
  Cpu(s): 48.1%us, 18.2%sy,  0.0%ni, 33.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
  Mem:  65674944k total, 64651012k used,  1023932k free,   194608k buffers
  Swap:  8803324k total,  4140324k used,  4663000k free,   363712k cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+   P WCHAN     COMMAND
  25281 root      20   0 20.2g  13g 4020 R  157 21.6   5864:12 14 -         kvm
  25284 root      20   0 20.2g  13g 4020 R  155 21.6   5863:02  4 -         kvm
  25294 root      20   0 20.2g  13g 4020 R  153 21.6   5851:59  3 -         kvm
  25287 root      20   0 20.2g  13g 4020 R  152 21.6   5861:20 15 -         kvm
  25299 root      20   0 20.2g  13g 4020 R  152 21.6   5847:14  1 -         kvm
  25258 root      20   0 20.2g  13g 4020 R  122 21.6   3372:41 13 -         kvm
  25269 root      20   0 20.2g  13g 4020 R  101 21.6   5929:42  5 -         kvm
  25301 root      20   0 20.2g  13g 4020 R  101 21.6   5847:26 10 -         kvm
  25292 root      20   0 20.2g  13g 4020 R  100 21.6   5853:18  7 -         kvm
  25297 root      20   0 20.2g  13g 4020 R  100 21.6   5843:37 16 -         kvm
  25272 root      20   0 20.2g  13g 4020 R   98 21.6   5872:52  2 -         kvm
  25277 root      20   0 20.2g  13g 4020 R   93 21.6   5878:21  0 -         kvm
  25290 root      20   0 20.2g  13g 4020 R   51 21.6   5863:15  8 -         kvm
  25314 root      20   0 20.2g  13g 4020 S    0 21.6   0:41.42  1 futex_wai kvm

  all thread's kernel stack (guest OS WinServer2008):
  root@cvk11:/proc/25258/task# cat 25281/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062cbb4>] __vcpu_run+0x704/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25284/stack
  [<ffffffffa0613537>] kvm_vcpu_yield_to+0x47/0xa0 [kvm]
  [<ffffffffa06136ab>] kvm_vcpu_on_spin+0x11b/0x150 [kvm]
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25294/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25287/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25299/stack
  [<ffffffffa03d34f6>] handle_pause+0x16/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25258/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25269/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffa03d60d4>] vmx_handle_exit+0x94/0x8b0 [kvm_intel]
  [<ffffffffa062caec>] __vcpu_run+0x63c/0xc30 [kvm]
  [<ffffffffa062d188>] kvm_arch_vcpu_ioctl_run+0xa8/0x270 [kvm]
  [<ffffffffa0615042>] kvm_vcpu_ioctl+0x512/0x6d0 [kvm]
  [<ffffffff811d4326>] do_vfs_ioctl+0x86/0x4f0
  [<ffffffff811d4821>] SyS_ioctl+0x91/0xb0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25301/stack
  [<ffffffffa03d34fe>] handle_pause+0x1e/0x30 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25292/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25297/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25272/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25277/stack
  [<ffffffffa03cdfa3>] clear_atomic_switch_msr+0x133/0x170 [kvm_intel]
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25290/stack
  [<ffffffffffffffff>] 0xffffffffffffffff
  root@cvk11:/proc/25258/task# cat 25314/stack
  [<ffffffff810daf77>] futex_wait_queue_me+0xd7/0x150
  [<ffffffff810dc087>] futex_wait+0x1a7/0x2c0
  [<ffffffff810ddc14>] do_futex+0x334/0xb70
  [<ffffffff810de592>] SyS_futex+0x142/0x1a0
  [<ffffffff817610ad>] system_call_fastpath+0x1a/0x1f
  [<ffffffffffffffff>] 0xffffffffffffffff

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1490853/+subscriptions

^ permalink raw reply	[flat|nested] 5+ messages in thread