HVM dropped packets

HVM dropped packets

[Nathan Stratton]

I am having no issues with para guests, but HVM guests all show dropped 
packets.

I am runing Xen 3.4.1 on linux 2.6.31-rc6, but also have tried older xen 
and kernels.

Also, not sure why I have a vif and a tap for my HVM guests, only have vif 
for para.

[root@xen1 config]# ifconfig tap19.0
tap19.0   Link encap:Ethernet  HWaddr B6:B7:E1:71:09:3C
           inet6 addr: fe80::b4b7:e1ff:fe71:93c/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:34507 errors:0 dropped:0 overruns:0 frame:0
           TX packets:43841 errors:0 dropped:0 overruns:645 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:16847714 (16.0 MiB)  TX bytes:22711872 (21.6 MiB)

[root@xen1 config]# ifconfig tap19.1
tap19.1   Link encap:Ethernet  HWaddr 76:C7:74:AF:CA:EC
           inet6 addr: fe80::74c7:74ff:feaf:caec/64 Scope:Link
           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
           RX packets:36601 errors:0 dropped:0 overruns:0 frame:0
           TX packets:36018 errors:0 dropped:0 overruns:457 carrier:0
           collisions:0 txqueuelen:500
           RX bytes:18152679 (17.3 MiB)  TX bytes:17744980 (16.9 MiB)

[root@xen1 config]# ifconfig vif19.0
vif19.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:2908 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[root@xen1 config]# ifconfig vif19.1
vif19.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
           TX packets:0 errors:0 dropped:181 overruns:0 carrier:0
           collisions:0 txqueuelen:32
           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)


[root@xen1 config]# more proxy0.hou-para
import os, re
arch = os.uname()[4]
if re.search('64', arch):
     arch_libdir = 'lib64'
else:
     arch_libdir = 'lib'

kernel = "/usr/lib/xen/boot/hvmloader"

builder='hvm'

name = "proxy0.hou"
memory = 2048
vcpus=4

disk   = [ 'file:/share/xen/images/proxy0.hou.img,xvda,w', ]
vif    = [ 'mac=00:11:25:00:1E:01, type=ioemu, bridge=private',
            'mac=00:11:25:00:1E:02, type=ioemu, bridge=public' ]

uuid   = '00000014-1111-2222-3333-200809231904'

device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'

sdl=0
opengl=1
vnc=1
vncpasswd=''
stdvga=0
serial='pty'



><>
Nathan Stratton                                CTO, BlinkMind, Inc.
nathan at robotics.net                         nathan at blinkmind.com
http://www.robotics.net                        http://www.blinkmind.com

Re: HVM dropped packets

[Pasi Kärkkäinen]

On Tue, Aug 25, 2009 at 01:30:34PM -0500, Nathan Stratton wrote:
> 
> I am having no issues with para guests, but HVM guests all show dropped 
> packets.
> 

Try changing the Qemu emulated NIC to Intel e1000. Maybe that helps with the
performance.

Or then use PV drivers in the guest.

-- Pasi

> I am runing Xen 3.4.1 on linux 2.6.31-rc6, but also have tried older xen 
> and kernels.
> 
> Also, not sure why I have a vif and a tap for my HVM guests, only have vif 
> for para.
> 
> [root@xen1 config]# ifconfig tap19.0
> tap19.0   Link encap:Ethernet  HWaddr B6:B7:E1:71:09:3C
>           inet6 addr: fe80::b4b7:e1ff:fe71:93c/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:34507 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:43841 errors:0 dropped:0 overruns:645 carrier:0
>           collisions:0 txqueuelen:500
>           RX bytes:16847714 (16.0 MiB)  TX bytes:22711872 (21.6 MiB)
> 
> [root@xen1 config]# ifconfig tap19.1
> tap19.1   Link encap:Ethernet  HWaddr 76:C7:74:AF:CA:EC
>           inet6 addr: fe80::74c7:74ff:feaf:caec/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:36601 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:36018 errors:0 dropped:0 overruns:457 carrier:0
>           collisions:0 txqueuelen:500
>           RX bytes:18152679 (17.3 MiB)  TX bytes:17744980 (16.9 MiB)
> 
> [root@xen1 config]# ifconfig vif19.0
> vif19.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:0 errors:0 dropped:2908 overruns:0 carrier:0
>           collisions:0 txqueuelen:32
>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> 
> [root@xen1 config]# ifconfig vif19.1
> vif19.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:0 errors:0 dropped:181 overruns:0 carrier:0
>           collisions:0 txqueuelen:32
>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> 
> 
> [root@xen1 config]# more proxy0.hou-para
> import os, re
> arch = os.uname()[4]
> if re.search('64', arch):
>     arch_libdir = 'lib64'
> else:
>     arch_libdir = 'lib'
> 
> kernel = "/usr/lib/xen/boot/hvmloader"
> 
> builder='hvm'
> 
> name = "proxy0.hou"
> memory = 2048
> vcpus=4
> 
> disk   = [ 'file:/share/xen/images/proxy0.hou.img,xvda,w', ]
> vif    = [ 'mac=00:11:25:00:1E:01, type=ioemu, bridge=private',
>            'mac=00:11:25:00:1E:02, type=ioemu, bridge=public' ]
> 
> uuid   = '00000014-1111-2222-3333-200809231904'
> 
> device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
> 
> sdl=0
> opengl=1
> vnc=1
> vncpasswd=''
> stdvga=0
> serial='pty'
> 
> 
> 
> ><>
> Nathan Stratton                                CTO, BlinkMind, Inc.
> nathan at robotics.net                         nathan at blinkmind.com
> http://www.robotics.net                        http://www.blinkmind.com
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: HVM dropped packets

[Nathan Stratton]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3686 bytes --]

On Tue, 25 Aug 2009, Pasi Kärkkäinen wrote:

> On Tue, Aug 25, 2009 at 01:30:34PM -0500, Nathan Stratton wrote:
>>
>> I am having no issues with para guests, but HVM guests all show dropped
>> packets.
>
> Try changing the Qemu emulated NIC to Intel e1000. Maybe that helps with the
> performance.

Same

> Or then use PV drivers in the guest.

Can't, its a HVM guest that I don't have control over.

> -- Pasi
>
>> I am runing Xen 3.4.1 on linux 2.6.31-rc6, but also have tried older xen
>> and kernels.
>>
>> Also, not sure why I have a vif and a tap for my HVM guests, only have vif
>> for para.
>>
>> [root@xen1 config]# ifconfig tap19.0
>> tap19.0   Link encap:Ethernet  HWaddr B6:B7:E1:71:09:3C
>>           inet6 addr: fe80::b4b7:e1ff:fe71:93c/64 Scope:Link
>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>           RX packets:34507 errors:0 dropped:0 overruns:0 frame:0
>>           TX packets:43841 errors:0 dropped:0 overruns:645 carrier:0
>>           collisions:0 txqueuelen:500
>>           RX bytes:16847714 (16.0 MiB)  TX bytes:22711872 (21.6 MiB)
>>
>> [root@xen1 config]# ifconfig tap19.1
>> tap19.1   Link encap:Ethernet  HWaddr 76:C7:74:AF:CA:EC
>>           inet6 addr: fe80::74c7:74ff:feaf:caec/64 Scope:Link
>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>           RX packets:36601 errors:0 dropped:0 overruns:0 frame:0
>>           TX packets:36018 errors:0 dropped:0 overruns:457 carrier:0
>>           collisions:0 txqueuelen:500
>>           RX bytes:18152679 (17.3 MiB)  TX bytes:17744980 (16.9 MiB)
>>
>> [root@xen1 config]# ifconfig vif19.0
>> vif19.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
>>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>           TX packets:0 errors:0 dropped:2908 overruns:0 carrier:0
>>           collisions:0 txqueuelen:32
>>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>
>> [root@xen1 config]# ifconfig vif19.1
>> vif19.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
>>           inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
>>           UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
>>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>           TX packets:0 errors:0 dropped:181 overruns:0 carrier:0
>>           collisions:0 txqueuelen:32
>>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
>>
>>
>> [root@xen1 config]# more proxy0.hou-para
>> import os, re
>> arch = os.uname()[4]
>> if re.search('64', arch):
>>     arch_libdir = 'lib64'
>> else:
>>     arch_libdir = 'lib'
>>
>> kernel = "/usr/lib/xen/boot/hvmloader"
>>
>> builder='hvm'
>>
>> name = "proxy0.hou"
>> memory = 2048
>> vcpus=4
>>
>> disk   = [ 'file:/share/xen/images/proxy0.hou.img,xvda,w', ]
>> vif    = [ 'mac=00:11:25:00:1E:01, type=ioemu, bridge=private',
>>            'mac=00:11:25:00:1E:02, type=ioemu, bridge=public' ]
>>
>> uuid   = '00000014-1111-2222-3333-200809231904'
>>
>> device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
>>
>> sdl=0
>> opengl=1
>> vnc=1
>> vncpasswd=''
>> stdvga=0
>> serial='pty'
>>
>>
>>
>>> <>
>> Nathan Stratton                                CTO, BlinkMind, Inc.
>> nathan at robotics.net                         nathan at blinkmind.com
>> http://www.robotics.net                        http://www.blinkmind.com
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: HVM dropped packets

[Pasi Kärkkäinen]

On Tue, Aug 25, 2009 at 02:49:40PM -0500, Nathan Stratton wrote:
> On Tue, 25 Aug 2009, Pasi Kärkkäinen wrote:
> 
> >On Tue, Aug 25, 2009 at 01:30:34PM -0500, Nathan Stratton wrote:
> >>
> >>I am having no issues with para guests, but HVM guests all show dropped
> >>packets.
> >
> >Try changing the Qemu emulated NIC to Intel e1000. Maybe that helps with 
> >the
> >performance.
> 
> Same
> 
> >Or then use PV drivers in the guest.
> 
> Can't, its a HVM guest that I don't have control over.
>

Well I guess then your only option is to use stubdoms.

Beware there's a bug (qemu memleak) in xen 3.4.1 related to stubdoms and heavy io 
on hvm guests in general: http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html

-- Pasi
 
> >-- Pasi
> >
> >>I am runing Xen 3.4.1 on linux 2.6.31-rc6, but also have tried older xen
> >>and kernels.
> >>
> >>Also, not sure why I have a vif and a tap for my HVM guests, only have vif
> >>for para.
> >>
> >>[root@xen1 config]# ifconfig tap19.0
> >>tap19.0   Link encap:Ethernet  HWaddr B6:B7:E1:71:09:3C
> >>          inet6 addr: fe80::b4b7:e1ff:fe71:93c/64 Scope:Link
> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>          RX packets:34507 errors:0 dropped:0 overruns:0 frame:0
> >>          TX packets:43841 errors:0 dropped:0 overruns:645 carrier:0
> >>          collisions:0 txqueuelen:500
> >>          RX bytes:16847714 (16.0 MiB)  TX bytes:22711872 (21.6 MiB)
> >>
> >>[root@xen1 config]# ifconfig tap19.1
> >>tap19.1   Link encap:Ethernet  HWaddr 76:C7:74:AF:CA:EC
> >>          inet6 addr: fe80::74c7:74ff:feaf:caec/64 Scope:Link
> >>          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >>          RX packets:36601 errors:0 dropped:0 overruns:0 frame:0
> >>          TX packets:36018 errors:0 dropped:0 overruns:457 carrier:0
> >>          collisions:0 txqueuelen:500
> >>          RX bytes:18152679 (17.3 MiB)  TX bytes:17744980 (16.9 MiB)
> >>
> >>[root@xen1 config]# ifconfig vif19.0
> >>vif19.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
> >>          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
> >>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >>          TX packets:0 errors:0 dropped:2908 overruns:0 carrier:0
> >>          collisions:0 txqueuelen:32
> >>          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> >>
> >>[root@xen1 config]# ifconfig vif19.1
> >>vif19.1   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
> >>          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
> >>          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
> >>          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> >>          TX packets:0 errors:0 dropped:181 overruns:0 carrier:0
> >>          collisions:0 txqueuelen:32
> >>          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> >>
> >>
> >>[root@xen1 config]# more proxy0.hou-para
> >>import os, re
> >>arch = os.uname()[4]
> >>if re.search('64', arch):
> >>    arch_libdir = 'lib64'
> >>else:
> >>    arch_libdir = 'lib'
> >>
> >>kernel = "/usr/lib/xen/boot/hvmloader"
> >>
> >>builder='hvm'
> >>
> >>name = "proxy0.hou"
> >>memory = 2048
> >>vcpus=4
> >>
> >>disk   = [ 'file:/share/xen/images/proxy0.hou.img,xvda,w', ]
> >>vif    = [ 'mac=00:11:25:00:1E:01, type=ioemu, bridge=private',
> >>           'mac=00:11:25:00:1E:02, type=ioemu, bridge=public' ]
> >>
> >>uuid   = '00000014-1111-2222-3333-200809231904'
> >>
> >>device_model = '/usr/' + arch_libdir + '/xen/bin/qemu-dm'
> >>
> >>sdl=0
> >>opengl=1
> >>vnc=1
> >>vncpasswd=''
> >>stdvga=0
> >>serial='pty'
> >>
> >>
> >>
> >>><>
> >>Nathan Stratton                                CTO, BlinkMind, Inc.
> >>nathan at robotics.net                         nathan at blinkmind.com
> >>http://www.robotics.net                        http://www.blinkmind.com
> >>
> >>_______________________________________________
> >>Xen-devel mailing list
> >>Xen-devel@lists.xensource.com
> >>http://lists.xensource.com/xen-devel
> >

Re: HVM dropped packets

[Nathan Stratton]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 442 bytes --]

On Tue, 25 Aug 2009, Pasi Kärkkäinen wrote:

> Well I guess then your only option is to use stubdoms.

Any way to figure out what causes the overruns on tap and drops on vif? If 
I am using tap for HVM why is there even a vif?

> Beware there's a bug (qemu memleak) in xen 3.4.1 related to stubdoms and heavy io
> on hvm guests in general: http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html

Thanks.

-Nathan

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: HVM dropped packets

[Pasi Kärkkäinen]

On Tue, Aug 25, 2009 at 03:52:23PM -0500, Nathan Stratton wrote:
> On Tue, 25 Aug 2009, Pasi Kärkkäinen wrote:
> 
> >Well I guess then your only option is to use stubdoms.
> 
> Any way to figure out what causes the overruns on tap and drops on vif? If 
> I am using tap for HVM why is there even a vif?
> 

How's your CPU usage? 

I bet the drops are caused by the low performance of Qemu nic _emulation_.

-- Pasi

> >Beware there's a bug (qemu memleak) in xen 3.4.1 related to stubdoms and 
> >heavy io
> >on hvm guests in general: 
> >http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html
> 
> Thanks.
> 
> -Nathan

Re: HVM dropped packets

[Nathan Stratton]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 244 bytes --]

On Wed, 26 Aug 2009, Pasi Kärkkäinen wrote:

> How's your CPU usage?

Low this is a 8 core Quaid Xeon 5510.

> I bet the drops are caused by the low performance of Qemu nic _emulation_.

Does one emulate better then the rest?

-Nathan

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: HVM dropped packets

[Keir Fraser]

On 26/08/2009 08:45, "Nathan Stratton" <nathan@robotics.net> wrote:

>> I bet the drops are caused by the low performance of Qemu nic _emulation_.
> 
> Does one emulate better then the rest?

e1000 is probably faster than the rest due to a more efficient hardware
interface better suited to emulation. You may still get receiver overruns
though (which I think was the source of the drops you were seeing?). That's
simply the host receiving packets faster than they can get stuffed through
the emulated interface.

 -- Keir

Re: HVM dropped packets

[Stefano Stabellini]

On Wed, 26 Aug 2009, Keir Fraser wrote:
> On 26/08/2009 08:45, "Nathan Stratton" <nathan@robotics.net> wrote:
> 
> >> I bet the drops are caused by the low performance of Qemu nic _emulation_.
> > 
> > Does one emulate better then the rest?
> 
> e1000 is probably faster than the rest due to a more efficient hardware
> interface better suited to emulation. You may still get receiver overruns
> though (which I think was the source of the drops you were seeing?). That's
> simply the host receiving packets faster than they can get stuffed through
> the emulated interface.

ON this note KVM people stated that the emulated e1000 gives 90% of virtio
performances.