receive: use gro call instead of plain call

receive: use gro call instead of plain call

[Lonnie Abelbeck]

For certain lower-end x86 boxes I test, I noticed WG 0.0.20180708 w/NAPI =
actually slowed down receive performance.

Jason recently added "receive: use gro call instead of plain call" [1] =
commit, which made a big performance improvement.

Here is a test on a PC Engines APU2 ...

pbx4: AMD GX-412TC SOC (1 GHz, 4-core)
      Ethernet controller: Intel Corporation I210

pbx:  Intel(R) Core(TM) i3-6100U CPU @ 2.30GHz
      Ethernet controller: Intel Corporation I211

# uname -a
Linux pbx4 3.16.57-astlinux #1 SMP PREEMPT Mon Jul 9 17:25:31 CDT 2018 =
x86_64 GNU/Linux

pbx ~ # iperf3 -s

pbx4 ~ # iperf3 -c 10.4.0.10 -P2 -R
-- 0.0.20180625 --
[SUM]   0.00-10.04  sec   466 MBytes   390 Mbits/sec  162             =
sender
[SUM]   0.00-10.00  sec   463 MBytes   389 Mbits/sec                  =
receiver
-- 0.0.20180708 --
[SUM]   0.00-10.04  sec   301 MBytes   252 Mbits/sec   30             =
sender
[SUM]   0.00-10.00  sec   300 MBytes   251 Mbits/sec                  =
receiver
-- 0.0.20180708 w/napi_gro_receive patch --
[SUM]   0.00-10.04  sec   596 MBytes   498 Mbits/sec   92             =
sender
[SUM]   0.00-10.00  sec   594 MBytes   498 Mbits/sec                  =
receiver
--

Kudos Jason !

Lonnie

[1] =
https://git.zx2c4.com/WireGuard/commit/?id=3D95951af7249912a4356b9a03cf3ad=
dc7e3f8f724=

Re: receive: use gro call instead of plain call

[Roman Mamedov]

On Fri, 13 Jul 2018 08:49:45 -0500
Lonnie Abelbeck <lists@lonnie.abelbeck.com> wrote:

> For certain lower-end x86 boxes I test, I noticed WG 0.0.20180708 w/NAPI actually slowed down receive performance.
> 
> Jason recently added "receive: use gro call instead of plain call" [1] commit, which made a big performance improvement.

Yes I'm also seeing about 20% higher performance with this patch (from 1.3-1.4
to 1.6 Gbit on same-host VMs). This is awesome!

...and... if I switch TCP Congestion Control from bbr to illinois on sender, I
now get 2.0 Gbit. WTF. :)

Lonnie, which one do you use on your hosts? Try with illinois (it was the best
choice among all of them from my tests before bbr) and bbr (this one tends to
get a bit lower speeds overall, but ramps up so much faster at the start).

-- 
With respect,
Roman

Re: receive: use gro call instead of plain call

[Lonnie Abelbeck]

> On Jul 13, 2018, at 11:15 AM, Roman Mamedov <rm@romanrm.net> wrote:
>=20
> On Fri, 13 Jul 2018 08:49:45 -0500
> Lonnie Abelbeck <lists@lonnie.abelbeck.com> wrote:
>=20
>> For certain lower-end x86 boxes I test, I noticed WG 0.0.20180708 =
w/NAPI actually slowed down receive performance.
>>=20
>> Jason recently added "receive: use gro call instead of plain call" =
[1] commit, which made a big performance improvement.
>=20
> Yes I'm also seeing about 20% higher performance with this patch (from =
1.3-1.4
> to 1.6 Gbit on same-host VMs). This is awesome!

Hi Roman, thanks for the followup ...


> ...and... if I switch TCP Congestion Control from bbr to illinois on =
sender, I
> now get 2.0 Gbit. WTF. :)

> Lonnie, which one do you use on your hosts?

We are using Linux 3.16.57 and CONFIG_TCP_CONG_ADVANCED=3Dn, so standard =
stuff ...

pbx ~ # sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control =3D cubic reno

pbx ~ # sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control =3D cubic


Lonnie=

Re: receive: use gro call instead of plain call

[Jason A. Donenfeld]

Hey Lonnie,

Wow, thanks for benching that (and watching the git repo for the
experiment-of-the-day). This performance increase certainly exceeds my
expectations; I'm quite pleased it's working so well.

You might, by the way, be interested in using Flent for testing. It's
slightly more scientific, and its developer hangs out on this mailing
list.

https://flent.org/

Jason

Re: receive: use gro call instead of plain call

[Lonnie Abelbeck]

> On Jul 13, 2018, at 1:27 PM, Jason A. Donenfeld <Jason@zx2c4.com> =
wrote:
>=20
> Hey Lonnie,
>=20
> Wow, thanks for benching that (and watching the git repo for the
> experiment-of-the-day). This performance increase certainly exceeds my
> expectations; I'm quite pleased it's working so well.

Another huge improvement ...

pbx3: Intel(R) Celeron(R) CPU  N2930  @ 1.83GHz
      Ethernet controller: Intel Corporation I211

pbx:  Intel(R) Core(TM) i3-6100U CPU @ 2.30GHz
      Ethernet controller: Intel Corporation I211


pbx ~ # iperf3 -s

pbx3 ~ # iperf3 -c 10.4.0.10 -P2 -R
-- 0.0.20180625 --
[SUM]   0.00-10.03  sec   940 MBytes   786 Mbits/sec  251             =
sender
[SUM]   0.00-10.00  sec   933 MBytes   782 Mbits/sec                  =
receiver
-- 0.0.20180708 --
[SUM]   0.00-10.05  sec   730 MBytes   609 Mbits/sec  186             =
sender
[SUM]   0.00-10.01  sec   726 MBytes   608 Mbits/sec                  =
receiver
-- 0.0.20180708 w/napi_gro_receive patch --
[SUM]   0.00-10.04  sec   973 MBytes   813 Mbits/sec  289             =
sender
[SUM]   0.00-10.00  sec   969 MBytes   813 Mbits/sec                  =
receiver
--

Lonnie