From mboxrd@z Thu Jan 1 00:00:00 1970 From: Erik Jacobson Subject: Re: slow guest performance with build load, looking for ideas Date: Wed, 1 Jul 2009 16:41:28 -0500 Message-ID: <20090701214128.GA1380@sgi.com> References: <20090612210443.GA21840@sgi.com> <4A34C3D2.9020009@redhat.com> <20090618230744.GA19307@sgi.com> <4A477B8C.2010502@redhat.com> <20090628190500.GA7347@sgi.com> <4A47E06A.7040700@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Erik Jacobson , kvm@vger.kernel.org To: Avi Kivity Return-path: Received: from relay3.sgi.com ([192.48.156.57]:56649 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750914AbZGAVl0 (ORCPT ); Wed, 1 Jul 2009 17:41:26 -0400 Content-Disposition: inline In-Reply-To: <4A47E06A.7040700@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: I wanted to post in to the thread the lastest test run. Avi Kivity provided some ideas to try. I had mixed luck. I'd like to try this again if we have any thoughts on the vpid/ept issue, or any other ideas for drilling down on this. Avi Kivity mentioned LVM in the thread. I continued to just export the whole /dev/sdb to the guest. I'm happy to try LVM in some form if we think it would help? As indicated, I still had trouble locating information about ept and vpid (see below). Several Fedora11 packages were updated in both host and guest since the last run, so we're at current F11+updates. I don't know enough about some of these kvm settings to do much beyond what I'm told to try. System hardware: * Same machines as used before, extensive system detail posted earlier in the thread. * Same Nehalem based XE270 system as before * Hyperthreading disabled * System was the same as before. Host has 8 cores, 2 sockets, and is Nehalem. (Intel(R) Xeon(R) CPU X5570 @ 2.93GHz) * root and workarea disks are nothing special no LVM used. * 8gb host memory System Settings: * chkconfig ntpd off * service ntpd stop * $ cat /sys/devices/system/clocksource/clocksource0/current_clocksource kvm-clock * ensured kvm_stat was available on the host * I could NOT find vpid and ept parameters on the host. They weren't here: /sys/module/kvm_intel/parameters nor here /sys/module/kvm/parameters So the check for those parameters resulted in no information. Didn't see them elsewhere either: # pwd /sys # find . -name vpid -print # find . -name ept -print * Version information: kernel host and guest: 2.6.29.5-191.fc11.x86_64 kvm: qemu-kvm-0.10.5-3.fc11.x86_64, qemu-system-x86-0.10.5-3.fc11.x86_64 * Build area disk is the whole /dev/sdb drive exported to the guest. I did not use LVM. * Root is a raw disk image, pre-allocated * Host and guest are fedora11 with all current updates applied. * 8 cpu, 4gb memory exported to guest. * All disks exported virtio I had done some stuff to set up the test including a build I didn't count. GUEST time (make -j12&& make -j12 modules), work area disk no cache param -------------------------------------------------------------------------- kvm_stat output BEFORE running this test: kvm statistics efer_reload 13 0 exits 27145076 1142 fpu_reload 1298729 0 halt_exits 2152011 189 halt_wakeup 494689 123 host_state_reload 4998646 837 hypercalls 0 0 insn_emulation 10165593 302 insn_emulation_fail 0 0 invlpg 0 0 io_exits 2096834 643 irq_exits 6469071 8 irq_injections 4765189 190 irq_window 279385 0 largepages 0 0 mmio_exits 0 0 mmu_cache_miss 18670 0 mmu_flooded 0 0 mmu_pde_zapped 0 0 mmu_pte_updated 0 0 mmu_pte_write 10440 0 mmu_recycled 0 0 qemu-kvm command: /usr/bin/qemu-kvm -M pc -m 4096 -smp 8 -name f11-test -uuid b7b4b7e4-9c07-22aa-0c95-d5c8a24176c5 -monitor pty -pidfile /var/run/libvirt/qemu//f11-test.pid -drive file=/var/lib/libvirt/images/f11-test.img,if=virtio,index=0,boot=on -drive file=/dev/sdb,if=virtio,index=1 -net nic,macaddr=54:52:00:46:48:0e,model=virtio -net user -serial pty -parallel none -usb -usbdevice tablet -vnc cct201:1 -soundhw es1370 -redir tcp:5555::22 test run timing: real 12m36.165s user 27m28.976s sys 8m32.245s kvm_stat output after this test run kvm statistics efer_reload 13 0 exits 47097981 2003 fpu_reload 2168308 0 halt_exits 3378761 301 halt_wakeup 707171 241 host_state_reload 7545990 1538 hypercalls 0 0 insn_emulation 17809066 462 insn_emulation_fail 0 0 invlpg 0 0 io_exits 2801221 1232 irq_exits 11959063 7 irq_injections 8395980 304 irq_window 531641 3 largepages 0 0 mmio_exits 0 0 mmu_cache_miss 28419 0 mmu_flooded 0 0 mmu_pde_zapped 0 0 mmu_pte_updated 0 0 mmu_pte_write 10440 0 mmu_recycled 7193 0 GUEST time (make -j12&& make -j12 modules), work area disk, cache=none ----------------------------------------------------------------------- qemu-kvm command: /usr/bin/qemu-kvm -M pc -m 4096 -smp 8 -name f11-test -uuid b7b4b7e4-9c07-22aa-0c95-d5c8a24176c5 -monitor pty -pidfile /var/run/libvirt/qemu//f11-test.pid -drive file=/var/lib/libvirt/images/f11-test.img,if=virtio,index=0,boot=on -drive file=/dev/sdb,if=virtio,index=1,cache=none -net nic,macaddr=54:52:00:46:48:0e,model=virtio -net user -serial pty -parallel none -usb -usbdevice tablet -vnc cct201:1 -soundhw es1370 -redir tcp:5555::22 kvm_stat output BEFORE running this test: kvm statistics efer_reload 13 0 exits 1384042 1097 fpu_reload 168399 0 halt_exits 142790 179 halt_wakeup 60526 123 host_state_reload 561970 827 hypercalls 0 0 insn_emulation 553390 269 insn_emulation_fail 0 0 invlpg 0 0 io_exits 390616 643 irq_exits 42529 5 irq_injections 165009 180 irq_window 8618 1 largepages 0 0 mmio_exits 0 0 mmu_cache_miss 1664 0 mmu_flooded 0 0 mmu_pde_zapped 0 0 mmu_pte_updated 0 0 mmu_pte_write 5000 0 mmu_recycled 0 0 test run timing: real 11m25.250s user 29m43.453s sys 8m49.591s kvm_stat output after this test run kvm statistics efer_reload 13 0 exits 23320965 1095 fpu_reload 1354712 0 halt_exits 1510807 180 halt_wakeup 385195 123 host_state_reload 3541625 823 hypercalls 0 0 insn_emulation 9056031 272 insn_emulation_fail 0 0 invlpg 0 0 io_exits 1318308 643 irq_exits 5553479 0 irq_injections 4108969 180 irq_window 279961 0 largepages 0 0 mmio_exits 0 0 mmu_cache_miss 14225 0 mmu_flooded 0 0 mmu_pde_zapped 0 0 mmu_pte_updated 0 0 mmu_pte_write 5000 0 mmu_recycled 0 0 HOST time (make -j12&& make -j12 modules) with no guest running ---------------------------------------------------------------- real 6m50.936s user 29m12.051s sys 5m50.867s