About ping latency in SR-IOV

About ping latency in SR-IOV

[XiaYubin]

Hi, Greg, Eddie,

I found that in SR-IOV + HVM environment, ping latency depends on ping interval.

The NIC is Intel 82576 (1000Mbps), server uses xen-4.0.1, domain-0 is
kernel-2.6.32
and HVM is kernel-2.6.31. The server and client are connected through
a 1000Mbps switch.

The data is as following:

# ping -i 1 192.168.0.21

PING 192.168.0.21 (192.168.0.21) 56(84) bytes of data.
64 bytes from 192.168.0.21: icmp_seq=1 ttl=64 time=0.117 ms
64 bytes from 192.168.0.21: icmp_seq=2 ttl=64 time=0.100 ms
64 bytes from 192.168.0.21: icmp_seq=3 ttl=64 time=0.102 ms
64 bytes from 192.168.0.21: icmp_seq=4 ttl=64 time=0.100 ms
64 bytes from 192.168.0.21: icmp_seq=5 ttl=64 time=0.105 ms

--- 192.168.0.21 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3998ms
rtt min/avg/max/mdev = 0.100/0.104/0.117/0.014 ms

# ping -i 0 192.168.0.21

PING 192.168.0.21 (192.168.0.21) 56(84) bytes of data.
64 bytes from 192.168.0.21: icmp_seq=1 ttl=64 time=0.114 ms
64 bytes from 192.168.0.21: icmp_seq=2 ttl=64 time=0.418 ms
64 bytes from 192.168.0.21: icmp_seq=3 ttl=64 time=0.455 ms
64 bytes from 192.168.0.21: icmp_seq=4 ttl=64 time=0.456 ms
64 bytes from 192.168.0.21: icmp_seq=5 ttl=64 time=0.455 ms

--- 192.168.0.21 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 1ms
rtt min/avg/max/mdev = 0.114/0.379/0.456/0.135 ms, ipg/ewma 0.400/0.252 ms

The ping latency of PF (in domain-0) is about 0.08ms in both tests.
Is this caused by something related to interrupt throttle configuration?
If so, can we adjust its threshold?

Thanks!

--
Yubin

RE: About ping latency in SR-IOV

[Rose, Gregory V]

Yes, it is dependent on the Interrupt Throttling setting.  Use modinfo on the driver or check the readme for more information on how to use the module parameter.

- Greg

> -----Original Message-----
> From: XiaYubin [mailto:xiayubin@gmail.com]
> Sent: Friday, October 29, 2010 1:31 AM
> To: Rose, Gregory V; Dong, Eddie
> Cc: xen-devel@lists.xensource.com
> Subject: About ping latency in SR-IOV
> 
> Hi, Greg, Eddie,
> 
> I found that in SR-IOV + HVM environment, ping latency depends on ping
> interval.
> 
> The NIC is Intel 82576 (1000Mbps), server uses xen-4.0.1, domain-0 is
> kernel-2.6.32
> and HVM is kernel-2.6.31. The server and client are connected through
> a 1000Mbps switch.
> 
> The data is as following:
> 
> # ping -i 1 192.168.0.21
> 
> PING 192.168.0.21 (192.168.0.21) 56(84) bytes of data.
> 64 bytes from 192.168.0.21: icmp_seq=1 ttl=64 time=0.117 ms
> 64 bytes from 192.168.0.21: icmp_seq=2 ttl=64 time=0.100 ms
> 64 bytes from 192.168.0.21: icmp_seq=3 ttl=64 time=0.102 ms
> 64 bytes from 192.168.0.21: icmp_seq=4 ttl=64 time=0.100 ms
> 64 bytes from 192.168.0.21: icmp_seq=5 ttl=64 time=0.105 ms
> 
> --- 192.168.0.21 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 3998ms
> rtt min/avg/max/mdev = 0.100/0.104/0.117/0.014 ms
> 
> # ping -i 0 192.168.0.21
> 
> PING 192.168.0.21 (192.168.0.21) 56(84) bytes of data.
> 64 bytes from 192.168.0.21: icmp_seq=1 ttl=64 time=0.114 ms
> 64 bytes from 192.168.0.21: icmp_seq=2 ttl=64 time=0.418 ms
> 64 bytes from 192.168.0.21: icmp_seq=3 ttl=64 time=0.455 ms
> 64 bytes from 192.168.0.21: icmp_seq=4 ttl=64 time=0.456 ms
> 64 bytes from 192.168.0.21: icmp_seq=5 ttl=64 time=0.455 ms
> 
> --- 192.168.0.21 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 1ms
> rtt min/avg/max/mdev = 0.114/0.379/0.456/0.135 ms, ipg/ewma 0.400/0.252 ms
> 
> The ping latency of PF (in domain-0) is about 0.08ms in both tests.
> Is this caused by something related to interrupt throttle configuration?
> If so, can we adjust its threshold?
> 
> Thanks!
> 
> --
> Yubin