From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:52599) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UGCHr-0001KL-4t for qemu-devel@nongnu.org; Thu, 14 Mar 2013 13:49:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UGCHn-0001y4-MP for qemu-devel@nongnu.org; Thu, 14 Mar 2013 13:49:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:26752) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UGCHn-0001xf-F8 for qemu-devel@nongnu.org; Thu, 14 Mar 2013 13:49:51 -0400 Date: Thu, 14 Mar 2013 19:50:04 +0200 From: "Michael S. Tsirkin" Message-ID: <20130314175003.GD29411@redhat.com> References: <6ce47933-c4df-4a64-9b94-69922ea3ef9a@mailpro> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <6ce47933-c4df-4a64-9b94-69922ea3ef9a@mailpro> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] slow virtio network with vhost=on and multiple cores List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alexandre DERUMIER Cc: Peter Lieven , Stefan Hajnoczi , qemu-devel@nongnu.org, Davide Guerri , Jan Kiszka , Peter Lieven , Dietmar Maurer Could be one of the bugs fixed in guest drivers since 2.6.32. For example, 39d321577405e8e269fd238b278aaf2425fa788a ? Does it help if you try a more recent guest? On Thu, Mar 14, 2013 at 11:43:30AM +0100, Alexandre DERUMIER wrote: > I don't think it's fixed in 1.3 or 1.4, some proxmox users have reporte= d again this bug with guest kernel 2.6.32. (proxmox host is rhel 6.3 kern= el + qemu 1.4) >=20 >=20 >=20 > ----- Mail original ----- >=20 > De: "Davide Guerri" > =C0: "Alexandre DERUMIER" > Cc: "Peter Lieven" , "Michael S. Tsirkin" , "Stefan Hajnoczi" , qemu-devel@nongnu.org, "Jan Ki= szka" , "Peter Lieven" , "Diet= mar Maurer" > Envoy=E9: Jeudi 14 Mars 2013 10:22:22 > Objet: Re: [Qemu-devel] slow virtio network with vhost=3Don and multipl= e cores >=20 > I'd like to reopen this thread because this problem is still here and i= t's really annoying. > Another possible work-around is to pin the virtio nic irq to one virtua= l CPU (with /proc/smp_affinity) but (of course) this is still sub-optimal. >=20 > Here are some graphs showing the performance of a heavy loaded machine = after and before a live migration from KVM-1.0 to KVM-1.2. > Host is a Ubuntu 12.10 Kernel 3.5, guest is a Ubuntu 10.04.4 LTS Kernel= 2.6.32 >=20 > History > ------- > CPU time: http://s1299.beta.photobucket.com/user/dguerri/media/CPU-10da= ys_zps4a088ff0.png.html > CPU Load: http://s1299.beta.photobucket.com/user/dguerri/media/Load-10d= ays_zpsff27f212.png.html > NET pps: http://s1299.beta.photobucket.com/user/dguerri/media/pps-10day= s_zps003dd039.png.html > NET Mbps: http://s1299.beta.photobucket.com/user/dguerri/media/Mbps-10d= ays_zpsfc3cba8c.png.html >=20 > Current > ------- > CPU time: http://s1299.beta.photobucket.com/user/dguerri/media/CPU-2day= s_zpsd362cac6.png.html > CPU Load: http://s1299.beta.photobucket.com/user/dguerri/media/load-2da= ys_zpsd4b7b50d.png.html > NET pps: http://s1299.beta.photobucket.com/user/dguerri/media/pps-2days= _zps8f5458c9.png.html > NET Mbps: http://s1299.beta.photobucket.com/user/dguerri/media/Mbps-2da= ys_zps299338b9.png.html >=20 > Red arrows indicate the kvm version change. > Black arrows indicate when I "pinned" the virtio NIC IRQ to only one CP= U (that machine has 2 virtual cores). > As can be seen the performance penalty is rather high: that machine was= almost unusable! >=20 > Does version 1.3 fixes this issue? >=20 > Could someone with the required knowledge look into this, please? > Please, this is a very nasty bug because I guess I'm not the only one w= ho is unable to upgrade all the machines with a (not-so) old kernel... :) >=20 > Thanks! >=20 > Davide Guerri. >=20 >=20 >=20 >=20 > On 09/dic/2012, at 19:38, Alexandre DERUMIER wrot= e: >=20 > >>> Have you had any further progress on this regression/problem? > > > > Hi Peter, > > I didn't re-tested myself, > > but a proxmox user who's have the problem with qemu-kvm 1.2, with win= dows guest and linux guest, > > don't have the problem anymore with qemu 1.3. > > > > http://forum.proxmox.com/threads/12157-Win2003R2-in-KVM-VM-is-slow-in= -PVE-2-2-when-multiply-CPU-cores-allowed > > > > I'll try to redone test myself this week > > > > Regards, > > > > Alexandre > > > > ----- Mail original ----- > > > > De: "Peter Lieven" > > =C0: "Alexandre DERUMIER" > > Cc: "Dietmar Maurer" , "Stefan Hajnoczi" , "Jan Kiszka" , qemu-devel@nongnu.org, "= Michael S. Tsirkin" , "Peter Lieven" > > Envoy=E9: Lundi 3 D=E9cembre 2012 12:23:11 > > Objet: Re: [Qemu-devel] slow virtio network with vhost=3Don and multi= ple cores > > > > > > Am 16.11.2012 um 12:00 schrieb Alexandre DERUMIER :=20 > > > >>>> While trying to reproduce the bug, we just detected that it depend= s on the hardware (mainboard) you run on. > >>>> > >>>> Sigh :-/ > >> > >> Hi, > >> > >> I can reproduce the bug on all my dell servers,differents generation= (R710 (intel),R815 (amd), 2950 (intel). > >> > >> They all use broadcom bnx2 network card (don't know if it can be rel= ated) > >> > >> host kernel : rhel 63 with 2.6.32 kernel > >> > >> guest kernel : 2.6.32 (debian squeeze, ubuntu). > >> > >> No problem with guest kernel 3.2 > > > > Have you had any further progress on this regression/problem? > > > > Thanks, > > Peter > > > >> > >> > >> > >> > >> ----- Mail original ----- > >> > >> De: "Dietmar Maurer" > >> =C0: "Peter Lieven" > >> Cc: "Stefan Hajnoczi" , "Peter Lieven" , "Jan Kiszka" , qemu-devel@nongnu.org, "Michael = S. Tsirkin" > >> Envoy=E9: Vendredi 16 Novembre 2012 11:44:26 > >> Objet: Re: [Qemu-devel] slow virtio network with vhost=3Don and mult= iple cores > >> > >>>> I only tested with RHEL6.3 kernel on host. > >>> > >>> can you check if there is a difference on interrupt delivery betwee= n those > >>> two? > >>> > >>> cat /proc/interrupts should be sufficient after some traffic has fl= own. > >> > >> While trying to reproduce the bug, we just detected that it depends = on the hardware (mainboard) you run on. > >> > >> Sigh :-/ > > > >