From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [Qemu-devel] [RFC]QEMU disk I/O limits Date: Tue, 31 May 2011 13:39:47 -0500 Message-ID: <4DE535F3.6040400@codemonkey.ws> References: <20110530050923.GF18832@f12.cn.ibm.com> <20110531134537.GE16382@redhat.com> <4DE4F230.2040203@us.ibm.com> <20110531140402.GF16382@redhat.com> <4DE4FA5B.1090804@codemonkey.ws> <20110531175955.GI16382@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kwolf@redhat.com, stefanha@linux.vnet.ibm.com, kvm@vger.kernel.org, guijianfeng@cn.fujitsu.com, Mike Snitzer , qemu-devel@nongnu.org, wuzhy@cn.ibm.com, herbert@gondor.hengli.com.au, Joe Thornber , Zhi Yong Wu , luowenj@cn.ibm.com, zhanx@cn.ibm.com, zhaoyang@cn.ibm.com, llim@redhat.com, Ryan A Harper To: Vivek Goyal Return-path: Received: from mail-gy0-f174.google.com ([209.85.160.174]:38561 "EHLO mail-gy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757713Ab1EaSju (ORCPT ); Tue, 31 May 2011 14:39:50 -0400 Received: by gyd10 with SMTP id 10so1884301gyd.19 for ; Tue, 31 May 2011 11:39:50 -0700 (PDT) In-Reply-To: <20110531175955.GI16382@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 05/31/2011 12:59 PM, Vivek Goyal wrote: > On Tue, May 31, 2011 at 09:25:31AM -0500, Anthony Liguori wrote: >> On 05/31/2011 09:04 AM, Vivek Goyal wrote: >>> On Tue, May 31, 2011 at 08:50:40AM -0500, Anthony Liguori wrote: >>>> On 05/31/2011 08:45 AM, Vivek Goyal wrote: >>>>> On Mon, May 30, 2011 at 01:09:23PM +0800, Zhi Yong Wu wrote: >>>>>> Hello, all, >>>>>> >>>>>> I have prepared to work on a feature called "Disk I/O limits" for qemu-kvm projeect. >>>>>> This feature will enable the user to cap disk I/O amount performed by a VM.It is important for some storage resources to be shared among multi-VMs. As you've known, if some of VMs are doing excessive disk I/O, they will hurt the performance of other VMs. >>>>>> >>>>> >>>>> Hi Zhiyong, >>>>> >>>>> Why not use kernel blkio controller for this and why reinvent the wheel >>>>> and implement the feature again in qemu? >>>> >>>> blkio controller only works for block devices. It doesn't work when >>>> using files. >>> >>> So can't we comeup with something to easily determine which device backs >>> up this file? Though that will still not work for NFS backed storage >>> though. >> >> Right. >> >> Additionally, in QEMU, we can rate limit based on concepts that make >> sense to a guest. We can limit the actual I/O ops visible to the >> guest which means that we'll get consistent performance regardless >> of whether the backing file is qcow2, raw, LVM, or raw over NFS. >> > > Are you referring to merging taking place which can change the definition > of IOPS as seen by guest? No, with qcow2, it may take multiple real IOPs for what the guest sees as an IOP. That's really the main argument I'm making here. The only entity that knows what a guest IOP corresponds to is QEMU. On the backend, it may end up being a network request, multiple BIOs to physical disks, file access, etc. That's why QEMU is the right place to do the throttling for this use case. That doesn't mean device level throttling isn't useful but just that for virtualization, it makes more sense to do it in QEMU. Regards, Anthony Liguori From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:59992) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QRTr8-0003k9-EV for qemu-devel@nongnu.org; Tue, 31 May 2011 14:39:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QRTr5-0003Kc-OB for qemu-devel@nongnu.org; Tue, 31 May 2011 14:39:54 -0400 Received: from mail-yw0-f45.google.com ([209.85.213.45]:63985) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QRTr5-0003KP-BY for qemu-devel@nongnu.org; Tue, 31 May 2011 14:39:51 -0400 Received: by ywl41 with SMTP id 41so2301712ywl.4 for ; Tue, 31 May 2011 11:39:50 -0700 (PDT) Message-ID: <4DE535F3.6040400@codemonkey.ws> Date: Tue, 31 May 2011 13:39:47 -0500 From: Anthony Liguori MIME-Version: 1.0 References: <20110530050923.GF18832@f12.cn.ibm.com> <20110531134537.GE16382@redhat.com> <4DE4F230.2040203@us.ibm.com> <20110531140402.GF16382@redhat.com> <4DE4FA5B.1090804@codemonkey.ws> <20110531175955.GI16382@redhat.com> In-Reply-To: <20110531175955.GI16382@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC]QEMU disk I/O limits List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Vivek Goyal Cc: kwolf@redhat.com, stefanha@linux.vnet.ibm.com, Mike Snitzer , guijianfeng@cn.fujitsu.com, qemu-devel@nongnu.org, wuzhy@cn.ibm.com, herbert@gondor.hengli.com.au, Joe Thornber , Zhi Yong Wu , luowenj@cn.ibm.com, kvm@vger.kernel.org, zhanx@cn.ibm.com, zhaoyang@cn.ibm.com, llim@redhat.com, Ryan A Harper On 05/31/2011 12:59 PM, Vivek Goyal wrote: > On Tue, May 31, 2011 at 09:25:31AM -0500, Anthony Liguori wrote: >> On 05/31/2011 09:04 AM, Vivek Goyal wrote: >>> On Tue, May 31, 2011 at 08:50:40AM -0500, Anthony Liguori wrote: >>>> On 05/31/2011 08:45 AM, Vivek Goyal wrote: >>>>> On Mon, May 30, 2011 at 01:09:23PM +0800, Zhi Yong Wu wrote: >>>>>> Hello, all, >>>>>> >>>>>> I have prepared to work on a feature called "Disk I/O limits" for qemu-kvm projeect. >>>>>> This feature will enable the user to cap disk I/O amount performed by a VM.It is important for some storage resources to be shared among multi-VMs. As you've known, if some of VMs are doing excessive disk I/O, they will hurt the performance of other VMs. >>>>>> >>>>> >>>>> Hi Zhiyong, >>>>> >>>>> Why not use kernel blkio controller for this and why reinvent the wheel >>>>> and implement the feature again in qemu? >>>> >>>> blkio controller only works for block devices. It doesn't work when >>>> using files. >>> >>> So can't we comeup with something to easily determine which device backs >>> up this file? Though that will still not work for NFS backed storage >>> though. >> >> Right. >> >> Additionally, in QEMU, we can rate limit based on concepts that make >> sense to a guest. We can limit the actual I/O ops visible to the >> guest which means that we'll get consistent performance regardless >> of whether the backing file is qcow2, raw, LVM, or raw over NFS. >> > > Are you referring to merging taking place which can change the definition > of IOPS as seen by guest? No, with qcow2, it may take multiple real IOPs for what the guest sees as an IOP. That's really the main argument I'm making here. The only entity that knows what a guest IOP corresponds to is QEMU. On the backend, it may end up being a network request, multiple BIOs to physical disks, file access, etc. That's why QEMU is the right place to do the throttling for this use case. That doesn't mean device level throttling isn't useful but just that for virtualization, it makes more sense to do it in QEMU. Regards, Anthony Liguori