From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pekka Enberg Subject: Re: [PATCH 0/9] kvm tools, qcow: Improve QCOW performance Date: Sun, 10 Jul 2011 21:38:01 +0300 Message-ID: References: <1310216563-17503-1-git-send-email-penberg@kernel.org> <20110710171543.GA19513@elte.hu> <20110710181755.GB21375@elte.hu> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: kvm@vger.kernel.org, Asias He , Cyrill Gorcunov , Prasad Joshi , Sasha Levin To: Ingo Molnar Return-path: Received: from mail-vx0-f174.google.com ([209.85.220.174]:37023 "EHLO mail-vx0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756476Ab1GJSiC convert rfc822-to-8bit (ORCPT ); Sun, 10 Jul 2011 14:38:02 -0400 Received: by vxb39 with SMTP id 39so2282495vxb.19 for ; Sun, 10 Jul 2011 11:38:01 -0700 (PDT) In-Reply-To: <20110710181755.GB21375@elte.hu> Sender: kvm-owner@vger.kernel.org List-ID: On Sun, Jul 10, 2011 at 9:17 PM, Ingo Molnar wrote: > > * Pekka Enberg wrote: > >> Hi Ingo, >> >> * Pekka Enberg wrote: >> >> This series fixes QCOW locking issues and implements delayed meta= data writeout. >> >> This improves performance of writeout to QCOW2 images that don't = have clusters >> >> and L2 tables allocated on-disk. >> >> >> >> I tested the series by running >> >> >> >> =A0 mount -t ext4 /dev/vdb /mnt >> >> =A0 dd if=3D/dev/zero of=3D/mnt/tmp >> >> >> >> in the guest multiple times for fresly generated QCOW2 image: >> >> >> >> =A0 dd if=3D/dev/zero of=3Dfs.ext4 bs=3D1024k count=3D512 && mkfs= =2Eext4 -F fs.ext4 && qemu-img convert -O qcow2 fs.ext4 fs.qcow2 >> >> >> >> which causes worst-case behavior for the current code. >> >> >> >> Before: >> >> >> >> =A0 [ seekwatcher: http://userweb.kernel.org/~penberg/kvm-qcow-de= layed/kvm-qcow2-master.png ] >> >> >> >> =A0 511229952 bytes (511 MB) copied, 19.906 s, 25.7 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 20.3168 s, 25.2 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 20.8078 s, 24.6 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 21.0889 s, 24.2 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 20.7833 s, 24.6 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 20.7536 s, 24.6 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 20.0312 s, 25.5 MB/s >> >> >> >> After: >> >> >> >> =A0 [ seekwatcher: http://userweb.kernel.org/~penberg/kvm-qcow-de= layed/kvm-qcow2-delayed.png ] >> >> >> >> =A0 511229952 bytes (511 MB) copied, 7.68312 s, 66.5 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 7.54065 s, 67.8 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 9.34749 s, 54.7 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 9.2421 s, 55.3 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 9.9364 s, 51.5 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 10.0337 s, 51.0 MB/s >> >> =A0 511229952 bytes (511 MB) copied, 9.39502 s, 54.4 MB/s >> >> On Sun, Jul 10, 2011 at 8:15 PM, Ingo Molnar wrote: >> > Just wondering, how does Qemu perform on the same system using the >> > same image, with comparable settings? >> >> Freshly built from qemu-kvm.git: >> >> $ /home/penberg/qemu-kvm/x86_64-softmmu/qemu-system-x86_64 --version >> QEMU emulator version 0.14.50 (qemu-kvm-devel), Copyright (c) >> 2003-2008 Fabrice Bellard >> >> Tests were run with this configuration: >> >> $ /home/penberg/qemu-kvm/x86_64-softmmu/qemu-system-x86_64 -kernel >> /boot/vmlinuz-3.0.0-rc5+ -drive >> file=3D/home/penberg/images/debian_squeeze_amd64_standard.img,if=3Dv= irtio,boot=3Don >> -drive file=3Dfs.qcow2,if=3Dvirtio -nographic -m 320 -smp 2 -append >> "root=3D/dev/vda1 console=3DttyS0 init=3D/root/iobench-write" >> >> Not sure if that's 100% comparable settings but anyway. The results >> looks as follows: >> >> =A0 511229952 bytes (511 MB) copied, 12.5543 s, 40.7 MB/s >> =A0 511229952 bytes (511 MB) copied, 9.50382 s, 53.8 MB/s >> =A0 511229952 bytes (511 MB) copied, 12.1092 s, 42.2 MB/s >> =A0 511229952 bytes (511 MB) copied, 13.2981 s, 38.4 MB/s >> =A0 511229952 bytes (511 MB) copied, 11.3314 s, 45.1 MB/s >> =A0 511229952 bytes (511 MB) copied, 12.7505 s, 40.1 MB/s >> =A0 511229952 bytes (511 MB) copied, 11.2921 s, 45.3 MB/s >> >> This is what I'd expect as tools/kvm has much more relaxed sync() >> guarantees than qemu-kvm. We treat all writes to QCOW2 images as >> volatile until VIRTIO_BLK_T_FLUSH is issued. Furthemore, for this >> particular (special case) load, it's pretty much append-only to the >> backing file which is why QCOW is so close to raw image performance >> here. > > Pretty impressive numbers! > > To relax Qemu's caching guarantees you can append ,cache=3Dwriteback = to > your -drive option, i.e. something like: > > =A0-drive file=3D/dev/shm/test.qcow2,cache=3Dwriteback,if=3Dvirtio > > Does that improve the Qemu results? Yes, it seems so: 511229952 bytes (511 MB) copied, 10.0879 s, 50.7 MB/s 511229952 bytes (511 MB) copied, 4.92686 s, 104 MB/s 511229952 bytes (511 MB) copied, 13.1955 s, 38.7 MB/s 511229952 bytes (511 MB) copied, 10.7322 s, 47.6 MB/s 511229952 bytes (511 MB) copied, 9.46115 s, 54.0 MB/s 511229952 bytes (511 MB) copied, 14.9963 s, 34.1 MB/s 511229952 bytes (511 MB) copied, 11.1701 s, 45.8 MB/s The numbers seem much more unstable from run to run with 'writeback' so it's pretty difficult to say how much it helps. I'm doing 'drop_caches' after image creation so I don't quite understand why they are so unstable. Pekka