From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Snitzer Subject: Re: [RFC]QEMU disk I/O limits Date: Tue, 31 May 2011 16:48:42 -0400 Message-ID: <20110531204842.GA16832@redhat.com> References: <20110530050923.GF18832@f12.cn.ibm.com> <20110531134537.GE16382@redhat.com> <4DE4F230.2040203@us.ibm.com> <20110531140402.GF16382@redhat.com> <4DE4FA5B.1090804@codemonkey.ws> <20110531175955.GI16382@redhat.com> <4DE535F3.6040400@codemonkey.ws> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Vivek Goyal , kwolf@redhat.com, stefanha@linux.vnet.ibm.com, kvm@vger.kernel.org, guijianfeng@cn.fujitsu.com, qemu-devel@nongnu.org, wuzhy@cn.ibm.com, herbert@gondor.hengli.com.au, Joe Thornber , Zhi Yong Wu , luowenj@cn.ibm.com, zhanx@cn.ibm.com, zhaoyang@cn.ibm.com, llim@redhat.com, Ryan A Harper To: Anthony Liguori Return-path: Received: from mx1.redhat.com ([209.132.183.28]:23399 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932093Ab1EaUuj (ORCPT ); Tue, 31 May 2011 16:50:39 -0400 Content-Disposition: inline In-Reply-To: <4DE535F3.6040400@codemonkey.ws> Sender: kvm-owner@vger.kernel.org List-ID: On Tue, May 31 2011 at 2:39pm -0400, Anthony Liguori wrote: > On 05/31/2011 12:59 PM, Vivek Goyal wrote: > >On Tue, May 31, 2011 at 09:25:31AM -0500, Anthony Liguori wrote: > >>On 05/31/2011 09:04 AM, Vivek Goyal wrote: > >>>On Tue, May 31, 2011 at 08:50:40AM -0500, Anthony Liguori wrote: > >>>>On 05/31/2011 08:45 AM, Vivek Goyal wrote: > >>>>>On Mon, May 30, 2011 at 01:09:23PM +0800, Zhi Yong Wu wrote: > >>>>>>Hello, all, > >>>>>> > >>>>>> I have prepared to work on a feature called "Disk I/O limits" for qemu-kvm projeect. > >>>>>> This feature will enable the user to cap disk I/O amount > performed by a VM.It is important for some storage resources to be > shared among multi-VMs. As you've known, if some of VMs are doing > excessive disk I/O, they will hurt the performance of other VMs. > >>>>>> > >>>>> > >>>>>Hi Zhiyong, > >>>>> > >>>>>Why not use kernel blkio controller for this and why reinvent the wheel > >>>>>and implement the feature again in qemu? > >>>> > >>>>blkio controller only works for block devices. It doesn't work when > >>>>using files. > >>> > >>>So can't we comeup with something to easily determine which device backs > >>>up this file? Though that will still not work for NFS backed storage > >>>though. > >> > >>Right. > >> > >>Additionally, in QEMU, we can rate limit based on concepts that make > >>sense to a guest. We can limit the actual I/O ops visible to the > >>guest which means that we'll get consistent performance regardless > >>of whether the backing file is qcow2, raw, LVM, or raw over NFS. > >> > > > >Are you referring to merging taking place which can change the definition > >of IOPS as seen by guest? > > No, with qcow2, it may take multiple real IOPs for what the guest > sees as an IOP. > > That's really the main argument I'm making here. The only entity > that knows what a guest IOP corresponds to is QEMU. On the backend, > it may end up being a network request, multiple BIOs to physical > disks, file access, etc. Couldn't QEMU give a hint to the kernel about the ratio of guest IOP to real IOPs? Or is QEMU blind to the real IOPs that correspond to a guest IOP? If QEMU is trully blind to the amount of real IOPs then couldn't QEMU driven throttling cause physical resources to be oversubscribed (underestimating the backend work it is creating)? Mike From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=33496 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QRVtp-0002G2-L7 for qemu-devel@nongnu.org; Tue, 31 May 2011 16:50:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QRVte-000332-7i for qemu-devel@nongnu.org; Tue, 31 May 2011 16:50:40 -0400 Received: from mx1.redhat.com ([209.132.183.28]:32081) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QRVtd-0002ln-O1 for qemu-devel@nongnu.org; Tue, 31 May 2011 16:50:38 -0400 Date: Tue, 31 May 2011 16:48:42 -0400 From: Mike Snitzer Message-ID: <20110531204842.GA16832@redhat.com> References: <20110530050923.GF18832@f12.cn.ibm.com> <20110531134537.GE16382@redhat.com> <4DE4F230.2040203@us.ibm.com> <20110531140402.GF16382@redhat.com> <4DE4FA5B.1090804@codemonkey.ws> <20110531175955.GI16382@redhat.com> <4DE535F3.6040400@codemonkey.ws> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4DE535F3.6040400@codemonkey.ws> Subject: Re: [Qemu-devel] [RFC]QEMU disk I/O limits List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anthony Liguori Cc: kwolf@redhat.com, stefanha@linux.vnet.ibm.com, kvm@vger.kernel.org, guijianfeng@cn.fujitsu.com, qemu-devel@nongnu.org, wuzhy@cn.ibm.com, herbert@gondor.hengli.com.au, Joe Thornber , Zhi Yong Wu , luowenj@cn.ibm.com, zhanx@cn.ibm.com, zhaoyang@cn.ibm.com, llim@redhat.com, Ryan A Harper , Vivek Goyal On Tue, May 31 2011 at 2:39pm -0400, Anthony Liguori wrote: > On 05/31/2011 12:59 PM, Vivek Goyal wrote: > >On Tue, May 31, 2011 at 09:25:31AM -0500, Anthony Liguori wrote: > >>On 05/31/2011 09:04 AM, Vivek Goyal wrote: > >>>On Tue, May 31, 2011 at 08:50:40AM -0500, Anthony Liguori wrote: > >>>>On 05/31/2011 08:45 AM, Vivek Goyal wrote: > >>>>>On Mon, May 30, 2011 at 01:09:23PM +0800, Zhi Yong Wu wrote: > >>>>>>Hello, all, > >>>>>> > >>>>>> I have prepared to work on a feature called "Disk I/O limits" for qemu-kvm projeect. > >>>>>> This feature will enable the user to cap disk I/O amount > performed by a VM.It is important for some storage resources to be > shared among multi-VMs. As you've known, if some of VMs are doing > excessive disk I/O, they will hurt the performance of other VMs. > >>>>>> > >>>>> > >>>>>Hi Zhiyong, > >>>>> > >>>>>Why not use kernel blkio controller for this and why reinvent the wheel > >>>>>and implement the feature again in qemu? > >>>> > >>>>blkio controller only works for block devices. It doesn't work when > >>>>using files. > >>> > >>>So can't we comeup with something to easily determine which device backs > >>>up this file? Though that will still not work for NFS backed storage > >>>though. > >> > >>Right. > >> > >>Additionally, in QEMU, we can rate limit based on concepts that make > >>sense to a guest. We can limit the actual I/O ops visible to the > >>guest which means that we'll get consistent performance regardless > >>of whether the backing file is qcow2, raw, LVM, or raw over NFS. > >> > > > >Are you referring to merging taking place which can change the definition > >of IOPS as seen by guest? > > No, with qcow2, it may take multiple real IOPs for what the guest > sees as an IOP. > > That's really the main argument I'm making here. The only entity > that knows what a guest IOP corresponds to is QEMU. On the backend, > it may end up being a network request, multiple BIOs to physical > disks, file access, etc. Couldn't QEMU give a hint to the kernel about the ratio of guest IOP to real IOPs? Or is QEMU blind to the real IOPs that correspond to a guest IOP? If QEMU is trully blind to the amount of real IOPs then couldn't QEMU driven throttling cause physical resources to be oversubscribed (underestimating the backend work it is creating)? Mike