From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [Qemu-devel] Better qemu/kvm defaults (was Re: [RFC PATCH 0/4] Gang scheduling in CFS) Date: Tue, 03 Jan 2012 09:48:51 -0600 Message-ID: <4F032363.5000700@codemonkey.ws> References: <20111219083141.32311.9429.stgit@abhimanyu.in.ibm.com> <20111219112326.GA15090@elte.hu> <87sjke1a53.fsf@abhimanyu.in.ibm.com> <4EF1B85F.7060105@redhat.com> <877h1o9dp7.fsf@linux.vnet.ibm.com> <20111223103620.GD4749@elte.hu> <4EF701C7.9080907@redhat.com> <87vcp4t45p.fsf@linux.vnet.ibm.com> <4EF838BD.60406@redhat.com> <4EFC903C.3030509@redhat.com> <4EFC9277.9040604@codemonkey.ws> <4F003268.90906@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Anthony Liguori , Nikunj A Dadhania , Avi Kivity , kvm-devel , qemu-devel To: dlaor@redhat.com Return-path: Received: from mail-gx0-f174.google.com ([209.85.161.174]:53523 "EHLO mail-gx0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753827Ab2ACPs5 (ORCPT ); Tue, 3 Jan 2012 10:48:57 -0500 Received: by ggdk6 with SMTP id k6so9845484ggd.19 for ; Tue, 03 Jan 2012 07:48:57 -0800 (PST) In-Reply-To: <4F003268.90906@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 01/01/2012 04:16 AM, Dor Laor wrote: > On 12/29/2011 06:16 PM, Anthony Liguori wrote: >> On 12/29/2011 10:07 AM, Dor Laor wrote: >>> On 12/26/2011 11:05 AM, Avi Kivity wrote: >>>> On 12/26/2011 05:14 AM, Nikunj A Dadhania wrote: >>>>>> >>>>>> btw you can get an additional speedup by enabling x2apic, for >>>>>> default_send_IPI_mask_logical(). >>>>>> >>>>> In the host? >>>>> >>>> >>>> In the host, for the guest: >>>> >>>> qemu -cpu ...,+x2apic >>>> >>> >>> It seems to me that we should improve our default flags. >>> So many times users fail to submit the proper huge command-line >>> options that we >>> require. Honestly, we can't blame them, there are so many flags and so >>> many use >>> cases its just too hard to get it right for humans. >>> >>> I propose a basic idea and folks are welcome to discuss it: >>> >>> 1. Improve qemu/kvm defaults >>> Break the current backward compatibility (but add a --default- >>> backward-compat-mode) and set better values for: >>> - rtc slew time >> >> What do you specifically mean? > > -rtc localtime,driftfix=slew We can just set this for pc-1.1. I don't see any real harm in doing that. >>> - cache=none >> >> I'm not sure I see this as a "better default" particularly since >> O_DIRECT fails on certain file systems. I think we really need to let >> WCE be toggable from the guest and then have a caching mode independent >> of WCE. We then need some heuristics to only enable cache=off when we >> know it's safe. > > cache=none is still faster then it has the FS support. > qemu can test-run O_DIRECT and fall back to cache mode or just test the > filesystem capabilities. I think a safer approach is to white list based on the results from fstat but regardless, we need WCE to be toggable first since I'm fairly certain you wouldn't want directsync to become the default :-) >>> - x2apic, maybe enhance qemu64 or move to -cpu host? >> >> Alex posted a patch for this. I'm planning on merging it although so far >> no one has chimed up either way. >> >>> - aio=native|threads (auto-sense?) >> >> aio=native is unsafe to default because linux-aio is just fubar. It >> falls back to synchronous I/O if the underlying filesystem doesn't >> support aio. There's no way in userspace to problem if it's actually >> supported or not either... > > Can we test-run this too? Nope. We need a kernel interface that reports aio capabilities. > Maybe as a separate qemu mode or even binary that > given a qemu cmdline, it will try to suggest better parameters? We could potentially whitelist to enable linux-aio where we know it's safe. >>> - use virtio devices by default >> >> I don't think this is realistic since appropriately licensed signed >> virtio drivers do not exist for Windows. (Please note the phrase >> "appropriately licensed signed"). > > What's the percentage of qemu invocation w/ windows guest and a short cmd line? I'm not really sure. > My hunch is that plain short cmdline indicates a developer and probably they'll > use linux guest. I've thought about how we could fix this and what I've come up with in the past is something a little different. We could enable the guest to choose which type of hardware is presented to it. Essentially, qemu -net nic,model=guests-pick When using 'guests-pick', we initially present the most compatible network model (rtl8139, for instance). We would provide a paravirtual channel (guest-agent?) that could be used to enumerate which models were available and let guest decide which model to use for the next reboot. You could also enable immediate switch over using hot plug. > >> >>> - more? >>> >>> Different defaults may be picked automatically when TCG|KVM used. >>> >>> 2. External hardening configuration file kept in qemu.git >>> For non qemu/kvm specific definitions like the io scheduler we >>> should maintain a script in our tree that sets/sense the optimal >>> settings of the host kernel (maybe similar one for the guest). >> >> What are "appropriate host settings" and why aren't we suggesting that >> distros and/or upstream just set them by default? > > It's hard to set the right default for a distribution since the same distro > should optimize for various usages of the same OS. For example, Fedora has > tuned-adm w/ available profiles: > - desktop-powersave > - server-powersave > - enterprise-storage > - spindown-disk > - laptop-battery-powersave > - default > - throughput-performance > - latency-performance > - laptop-ac-powersave > > We need to keep on recommending the best profile for virtualization, for Fedora > I think it either enterprise-storage and maybe throughput-performance. I think that's more of a distro. It might be worth referring to in our documentation but I'm not sure it's something we can do much about. Regards, Anthony Liguori > > If we have a such a script, it can call the matching tuned profile instead of > tweaking every /sys option. > >> >> Regards, >> >> Anthony Liguori >> >>> HTH, >>> Dor >>> >> >> > > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:58040) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ri6bi-0005Rq-Vy for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:49:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Ri6bh-0000fQ-RY for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:48:58 -0500 Received: from mail-gx0-f173.google.com ([209.85.161.173]:54526) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Ri6bh-0000fM-LX for qemu-devel@nongnu.org; Tue, 03 Jan 2012 10:48:57 -0500 Received: by ggnk1 with SMTP id k1so11719938ggn.4 for ; Tue, 03 Jan 2012 07:48:57 -0800 (PST) Message-ID: <4F032363.5000700@codemonkey.ws> Date: Tue, 03 Jan 2012 09:48:51 -0600 From: Anthony Liguori MIME-Version: 1.0 References: <20111219083141.32311.9429.stgit@abhimanyu.in.ibm.com> <20111219112326.GA15090@elte.hu> <87sjke1a53.fsf@abhimanyu.in.ibm.com> <4EF1B85F.7060105@redhat.com> <877h1o9dp7.fsf@linux.vnet.ibm.com> <20111223103620.GD4749@elte.hu> <4EF701C7.9080907@redhat.com> <87vcp4t45p.fsf@linux.vnet.ibm.com> <4EF838BD.60406@redhat.com> <4EFC903C.3030509@redhat.com> <4EFC9277.9040604@codemonkey.ws> <4F003268.90906@redhat.com> In-Reply-To: <4F003268.90906@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Better qemu/kvm defaults (was Re: [RFC PATCH 0/4] Gang scheduling in CFS) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: dlaor@redhat.com Cc: qemu-devel , Anthony Liguori , Avi Kivity , Nikunj A Dadhania , kvm-devel On 01/01/2012 04:16 AM, Dor Laor wrote: > On 12/29/2011 06:16 PM, Anthony Liguori wrote: >> On 12/29/2011 10:07 AM, Dor Laor wrote: >>> On 12/26/2011 11:05 AM, Avi Kivity wrote: >>>> On 12/26/2011 05:14 AM, Nikunj A Dadhania wrote: >>>>>> >>>>>> btw you can get an additional speedup by enabling x2apic, for >>>>>> default_send_IPI_mask_logical(). >>>>>> >>>>> In the host? >>>>> >>>> >>>> In the host, for the guest: >>>> >>>> qemu -cpu ...,+x2apic >>>> >>> >>> It seems to me that we should improve our default flags. >>> So many times users fail to submit the proper huge command-line >>> options that we >>> require. Honestly, we can't blame them, there are so many flags and so >>> many use >>> cases its just too hard to get it right for humans. >>> >>> I propose a basic idea and folks are welcome to discuss it: >>> >>> 1. Improve qemu/kvm defaults >>> Break the current backward compatibility (but add a --default- >>> backward-compat-mode) and set better values for: >>> - rtc slew time >> >> What do you specifically mean? > > -rtc localtime,driftfix=slew We can just set this for pc-1.1. I don't see any real harm in doing that. >>> - cache=none >> >> I'm not sure I see this as a "better default" particularly since >> O_DIRECT fails on certain file systems. I think we really need to let >> WCE be toggable from the guest and then have a caching mode independent >> of WCE. We then need some heuristics to only enable cache=off when we >> know it's safe. > > cache=none is still faster then it has the FS support. > qemu can test-run O_DIRECT and fall back to cache mode or just test the > filesystem capabilities. I think a safer approach is to white list based on the results from fstat but regardless, we need WCE to be toggable first since I'm fairly certain you wouldn't want directsync to become the default :-) >>> - x2apic, maybe enhance qemu64 or move to -cpu host? >> >> Alex posted a patch for this. I'm planning on merging it although so far >> no one has chimed up either way. >> >>> - aio=native|threads (auto-sense?) >> >> aio=native is unsafe to default because linux-aio is just fubar. It >> falls back to synchronous I/O if the underlying filesystem doesn't >> support aio. There's no way in userspace to problem if it's actually >> supported or not either... > > Can we test-run this too? Nope. We need a kernel interface that reports aio capabilities. > Maybe as a separate qemu mode or even binary that > given a qemu cmdline, it will try to suggest better parameters? We could potentially whitelist to enable linux-aio where we know it's safe. >>> - use virtio devices by default >> >> I don't think this is realistic since appropriately licensed signed >> virtio drivers do not exist for Windows. (Please note the phrase >> "appropriately licensed signed"). > > What's the percentage of qemu invocation w/ windows guest and a short cmd line? I'm not really sure. > My hunch is that plain short cmdline indicates a developer and probably they'll > use linux guest. I've thought about how we could fix this and what I've come up with in the past is something a little different. We could enable the guest to choose which type of hardware is presented to it. Essentially, qemu -net nic,model=guests-pick When using 'guests-pick', we initially present the most compatible network model (rtl8139, for instance). We would provide a paravirtual channel (guest-agent?) that could be used to enumerate which models were available and let guest decide which model to use for the next reboot. You could also enable immediate switch over using hot plug. > >> >>> - more? >>> >>> Different defaults may be picked automatically when TCG|KVM used. >>> >>> 2. External hardening configuration file kept in qemu.git >>> For non qemu/kvm specific definitions like the io scheduler we >>> should maintain a script in our tree that sets/sense the optimal >>> settings of the host kernel (maybe similar one for the guest). >> >> What are "appropriate host settings" and why aren't we suggesting that >> distros and/or upstream just set them by default? > > It's hard to set the right default for a distribution since the same distro > should optimize for various usages of the same OS. For example, Fedora has > tuned-adm w/ available profiles: > - desktop-powersave > - server-powersave > - enterprise-storage > - spindown-disk > - laptop-battery-powersave > - default > - throughput-performance > - latency-performance > - laptop-ac-powersave > > We need to keep on recommending the best profile for virtualization, for Fedora > I think it either enterprise-storage and maybe throughput-performance. I think that's more of a distro. It might be worth referring to in our documentation but I'm not sure it's something we can do much about. Regards, Anthony Liguori > > If we have a such a script, it can call the matching tuned profile instead of > tweaking every /sys option. > >> >> Regards, >> >> Anthony Liguori >> >>> HTH, >>> Dor >>> >> >> > >