Unexpected host I/O load

Unexpected host I/O load

[Brian Conry]

Hi,

I don't know if this is a bug, but it sure feels like one.  I've been
unable to find any evidence that anyone else has ever had this issue,
but that could be because most of the articles out there are about
guest I/O issues instead of host I/O issues.

My problem is that when running a kvm guest my host system experiences
a high I/O load even when the guest is not doing any I/O.  I've been
measuring this after doing a "shutdown -H now" on the guest OS to
unmount all drives, deactivate swap, and halt the kernel while leaving
the VM "on".

With my current setup I'm getting around 400 kB_wrtn/s (as reported by
iostat) with the guest in the halted state.  This goes away if I kill
or "kill -STOP" the qemu process.

CPU model: Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz, four cores

qemu-kvm version: qemu-kvm 1.0.1

host kernel version/arch: 3.3.8-gentoo, x86_64

guest: Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)

qemu command (started by virt-manager): /usr/bin/qemu-system-x86_64
--enable-kvm -S -M pc-1.0 -cpu
core2duo,+lahf_lm,+rdtscp,+avx,+osxsave,+xsave,+aes,+tsc-deadline,+popcnt,+x2apic,+sse4.2,+sse4.1,+pdcm,+xtpr,+cx16,+tm2,+est,+smx,+vmx,+ds_cpl,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+ds
-enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name test
-uuid 7960d870-f184-4c6e-04cc-0281c376f6b4 -nodefconfig -nodefaults
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/test.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
-no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
-drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
-drive file=/var/lib/libvirt/images/test.img,if=none,id=drive-virtio-disk0,format=raw,cache=none
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-netdev tap,fd=17,id=hostnet0 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:34:1c:4f,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -vnc 127.0.0.1:0 -vga cirrus
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

Note that the VM does /not/ start when I execute that command by hand,
and I can't find a way to get virt-manager to include it.  As such, I
do not know how things behave if I add the "-no-kvm-irqchip",
"-no-kvm-pit", or "-no-kvm" switches.

With kernel version 3.2.12-gentoo (alxo x86_64) I was seeing "only"
around 200 kB_wrtn/s reported by iostat.

Here are the KVM-related options from my kernel .config (both kernels
are the same):
CONFIG_KVM_CLOCK=y
CONFIG_KVM_GUEST=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_APIC_ARCHITECTURE=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
# CONFIG_KVM_AMD is not set
# CONFIG_KVM_MMU_AUDIT is not set

Thanks,
Brian

Re: Unexpected host I/O load

[Brian Conry]

After some additional troubleshooting under the guidance of a friend,
this appears to be a libvirt issue.

I have opened the following bug for it:
https://bugzilla.redhat.com/show_bug.cgi?id=841918.

Thanks to any who spent time looking at this.

Brian