From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [Qemu-devel] [PATCH] vhost: force vhost off for non-MSI guests Date: Fri, 21 Jan 2011 08:40:36 -0600 Message-ID: <4D399AE4.2000307@codemonkey.ws> References: <20110120153521.GA24357@redhat.com> <4D38583D.9010602@codemonkey.ws> <20110120160718.GA24652@redhat.com> <4D38D208.4090403@codemonkey.ws> <1295573746.2931.59.camel@x201> <20110121095503.GD26070@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Alex Williamson , kvm@vger.kernel.org, Juan Quintela , Jes.Sorensen@redhat.com, jasowang@redhat.com, qemu-devel@nongnu.org To: "Michael S. Tsirkin" Return-path: Received: from mail-qy0-f181.google.com ([209.85.216.181]:59432 "EHLO mail-qy0-f181.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752795Ab1AUOkg (ORCPT ); Fri, 21 Jan 2011 09:40:36 -0500 Received: by qyk12 with SMTP id 12so1988891qyk.19 for ; Fri, 21 Jan 2011 06:40:36 -0800 (PST) In-Reply-To: <20110121095503.GD26070@redhat.com> Sender: kvm-owner@vger.kernel.org List-ID: On 01/21/2011 03:55 AM, Michael S. Tsirkin wrote: > On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote: > >> On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote: >> >>> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote: >>> >>>> On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote: >>>> >>>> >>>>> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote: >>>>> >>>>> >>>>>> When MSI is off, each interrupt needs to be bounced through the io >>>>>> thread when it's set/cleared, so vhost-net causes more context switches and >>>>>> higher CPU utilization than userspace virtio which handles networking in >>>>>> the same thread. >>>>>> >>>>>> We'll need to fix this by adding level irq support in kvm irqfd, >>>>>> for now disable vhost-net in these configurations. >>>>>> >>>>>> Signed-off-by: Michael S. Tsirkin >>>>>> >>>>>> >>>>> I actually think this should be a terminal error. The user asks for >>>>> vhost-net, if we cannot enable it, we should exit. >>>>> >>>>> Or we should warn the user that they should expect bad performance. >>>>> Silently doing something that the user has explicitly asked us not >>>>> to do is not a good behavior. >>>>> >>>>> Regards, >>>>> >>>>> Anthony Liguori >>>>> >>>>> >>>> The issue is that user has no control of the guest, and can not know >>>> whether the guest enables MSI. So what you ask for will just make >>>> some guests fail, and others fail sometimes. >>>> The user also has no way to know that version X of kvm does not expose a >>>> way to inject level interrupts with irqfd. >>>> >>>> We could have *another* flag that says "use vhost where it helps" but >>>> then I think this is what everyone wants to do, anyway, and libvirt >>>> already sets vhost=on so I prefer redefining the meaning of an existing >>>> flag. >>>> >>>> >>> In the very least, there needs to be a vhost=force. >>> >>> Having some sort of friendly default policy is fine but we need to >>> provide a mechanism for a user to have the final say. If you want to >>> redefine vhost=on to really mean, use the friendly default, that's fine >>> by me, but only if the vhost=force option exists. >>> >>> I actually would think libvirt would want to use vhost=force. Debugging >>> with vhost=on is going to be a royal pain in the ass if a user reports >>> bad performance. Given the libvirt XML, you can't actually tell from >>> the guest and the XML whether or not vhost was actually in use or not. >>> >> If we add a force option, let's please distinguish hotplug from VM >> creation time. The latter can abort. Hotplug should print an error and >> fail the initfn. >> > It can't abort at init - MSI is disabled at init, it needs to be enabled > by the guest later. And aborting the guest in the middle of the run > is a very bad idea. > > What vhostforce=true will do is force vhost backend to be used even if > it is slower. > vhost=on,vhostforce=false use vhost if we think it will improve performance vhost=on,vhostforce=true always use vhost vhost=off,vhostforce=* do not use vhost Regards, Anthony Liguori > >> Thanks, >> >> Alex >> > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=42135 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PgIAI-0001Ty-1y for qemu-devel@nongnu.org; Fri, 21 Jan 2011 09:40:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PgIAG-00025a-NN for qemu-devel@nongnu.org; Fri, 21 Jan 2011 09:40:37 -0500 Received: from mail-qw0-f45.google.com ([209.85.216.45]:61949) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PgIAG-00025U-J0 for qemu-devel@nongnu.org; Fri, 21 Jan 2011 09:40:36 -0500 Received: by qwk4 with SMTP id 4so1848901qwk.4 for ; Fri, 21 Jan 2011 06:40:36 -0800 (PST) Message-ID: <4D399AE4.2000307@codemonkey.ws> Date: Fri, 21 Jan 2011 08:40:36 -0600 From: Anthony Liguori MIME-Version: 1.0 Subject: Re: [Qemu-devel] [PATCH] vhost: force vhost off for non-MSI guests References: <20110120153521.GA24357@redhat.com> <4D38583D.9010602@codemonkey.ws> <20110120160718.GA24652@redhat.com> <4D38D208.4090403@codemonkey.ws> <1295573746.2931.59.camel@x201> <20110121095503.GD26070@redhat.com> In-Reply-To: <20110121095503.GD26070@redhat.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Michael S. Tsirkin" Cc: kvm@vger.kernel.org, Juan Quintela , Jes.Sorensen@redhat.com, jasowang@redhat.com, qemu-devel@nongnu.org, Alex Williamson On 01/21/2011 03:55 AM, Michael S. Tsirkin wrote: > On Thu, Jan 20, 2011 at 06:35:46PM -0700, Alex Williamson wrote: > >> On Thu, 2011-01-20 at 18:23 -0600, Anthony Liguori wrote: >> >>> On 01/20/2011 10:07 AM, Michael S. Tsirkin wrote: >>> >>>> On Thu, Jan 20, 2011 at 09:43:57AM -0600, Anthony Liguori wrote: >>>> >>>> >>>>> On 01/20/2011 09:35 AM, Michael S. Tsirkin wrote: >>>>> >>>>> >>>>>> When MSI is off, each interrupt needs to be bounced through the io >>>>>> thread when it's set/cleared, so vhost-net causes more context switches and >>>>>> higher CPU utilization than userspace virtio which handles networking in >>>>>> the same thread. >>>>>> >>>>>> We'll need to fix this by adding level irq support in kvm irqfd, >>>>>> for now disable vhost-net in these configurations. >>>>>> >>>>>> Signed-off-by: Michael S. Tsirkin >>>>>> >>>>>> >>>>> I actually think this should be a terminal error. The user asks for >>>>> vhost-net, if we cannot enable it, we should exit. >>>>> >>>>> Or we should warn the user that they should expect bad performance. >>>>> Silently doing something that the user has explicitly asked us not >>>>> to do is not a good behavior. >>>>> >>>>> Regards, >>>>> >>>>> Anthony Liguori >>>>> >>>>> >>>> The issue is that user has no control of the guest, and can not know >>>> whether the guest enables MSI. So what you ask for will just make >>>> some guests fail, and others fail sometimes. >>>> The user also has no way to know that version X of kvm does not expose a >>>> way to inject level interrupts with irqfd. >>>> >>>> We could have *another* flag that says "use vhost where it helps" but >>>> then I think this is what everyone wants to do, anyway, and libvirt >>>> already sets vhost=on so I prefer redefining the meaning of an existing >>>> flag. >>>> >>>> >>> In the very least, there needs to be a vhost=force. >>> >>> Having some sort of friendly default policy is fine but we need to >>> provide a mechanism for a user to have the final say. If you want to >>> redefine vhost=on to really mean, use the friendly default, that's fine >>> by me, but only if the vhost=force option exists. >>> >>> I actually would think libvirt would want to use vhost=force. Debugging >>> with vhost=on is going to be a royal pain in the ass if a user reports >>> bad performance. Given the libvirt XML, you can't actually tell from >>> the guest and the XML whether or not vhost was actually in use or not. >>> >> If we add a force option, let's please distinguish hotplug from VM >> creation time. The latter can abort. Hotplug should print an error and >> fail the initfn. >> > It can't abort at init - MSI is disabled at init, it needs to be enabled > by the guest later. And aborting the guest in the middle of the run > is a very bad idea. > > What vhostforce=true will do is force vhost backend to be used even if > it is slower. > vhost=on,vhostforce=false use vhost if we think it will improve performance vhost=on,vhostforce=true always use vhost vhost=off,vhostforce=* do not use vhost Regards, Anthony Liguori > >> Thanks, >> >> Alex >> >