r8169: Ethernet speed regressions with RTL8168gu in ThinkPad E480

r8169: Ethernet speed regressions with RTL8168gu in ThinkPad E480

[Rafał Miłecki]

I've ThinkPad E480 with i3-8130U CPU and 10ec:8168 detected as:
[    8.458515] r8169 0000:03:00.0 eth0: RTL8168gu/8111gu, 8c:16:45:5d:f2:c2, XID 509, IRQ 136
[    8.458521] r8169 0000:03:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[   12.272352] Generic FE-GE Realtek PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC)

It's connected to WAN port of BCM4708 based home router running OpenWrt
with kernel 5.4.113. I run "iperf -s" on this E480 for network testing.

***************

After upgrading kernel from 4.12 to 5.18 I noticed Ethernet speed
regression. I bisected it down to two commits:


commit 6b839b6cf9eada30b086effb51e5d6076bafc761
Author: Heiner Kallweit <hkallweit1@gmail.com>
Date:   Thu Oct 18 19:56:01 2018 +0200

     r8169: fix NAPI handling under high load

(introduced in 4.19 and dropped speed by 20 Mb/s = 5%)


commit 288ac524cf70a8e7ed851a61ed2a9744039dae8d
Author: Heiner Kallweit <hkallweit1@gmail.com>
Date:   Sat Mar 30 17:13:24 2019 +0100

     r8169: disable default rx interrupt coalescing on RTL8168

(introduced in 5.1 and dropped speed by 60 Mb/s = 15%)

***************

Is that possible to fix / rework r8169 to provide original higher
Ethernet speed out of the box?

Honestly I'd expect this i3-8130U to handle 1 Gb/s traffic or more and
my guess is that the real bottleneck is my home router here (slow
BCM4708 SoC CPU). Still it seems like r8169 can be a bottleneck on top
of bottleneck.

***************

v5.18
334 Mbits/sec

v5.10
336 Mbits/sec

v5.1 + git revert 288ac524cf70a8e7ed851a61ed2a9744039dae8d
397 Mbits/sec (back to medium speed)

v5.1
335 Mbits/sec (-60 Mb/s)

v5.0
395 Mbits/sec

***************

v4.19 + git revert 6b839b6cf9eada30b086effb51e5d6076bafc761
414 Mbits/sec (back to high speed)

v4.19
395 Mbits/sec (-20 Mb/s)

v4.18
415 Mbits/sec

v4.12
415 Mbits/sec

Re: r8169: Ethernet speed regressions with RTL8168gu in ThinkPad E480

[Heiner Kallweit]

On 30.05.2022 16:10, Rafał Miłecki wrote:
> I've ThinkPad E480 with i3-8130U CPU and 10ec:8168 detected as:
> [    8.458515] r8169 0000:03:00.0 eth0: RTL8168gu/8111gu, 8c:16:45:5d:f2:c2, XID 509, IRQ 136
> [    8.458521] r8169 0000:03:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
> [   12.272352] Generic FE-GE Realtek PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC)
> 
> It's connected to WAN port of BCM4708 based home router running OpenWrt
> with kernel 5.4.113. I run "iperf -s" on this E480 for network testing.
> 
> ***************
> 
> After upgrading kernel from 4.12 to 5.18 I noticed Ethernet speed
> regression. I bisected it down to two commits:
> 
> 
> commit 6b839b6cf9eada30b086effb51e5d6076bafc761
> Author: Heiner Kallweit <hkallweit1@gmail.com>
> Date:   Thu Oct 18 19:56:01 2018 +0200
> 
>     r8169: fix NAPI handling under high load
> 
> (introduced in 4.19 and dropped speed by 20 Mb/s = 5%)
> 
> 
> commit 288ac524cf70a8e7ed851a61ed2a9744039dae8d
> Author: Heiner Kallweit <hkallweit1@gmail.com>
> Date:   Sat Mar 30 17:13:24 2019 +0100
> 
>     r8169: disable default rx interrupt coalescing on RTL8168
> 
> (introduced in 5.1 and dropped speed by 60 Mb/s = 15%)
> 
> ***************
> 
> Is that possible to fix / rework r8169 to provide original higher
> Ethernet speed out of the box?
> 
With RTL8168g and RTL8168h I get >900Mbps in both directions.
Both referenced changes are needed. What you could try:

Enable TSO:
ethtool -K eth0 sg on tso on

As replacement for hw irq coalescing:
echo 20000 > /sys/class/net/eth0/gro_flush_timeout
echo 1 > /sys/class/net/eth0/napi_defer_hard_irqs

Or use ethtool to (re-)enable hw irq coalescing.

> Honestly I'd expect this i3-8130U to handle 1 Gb/s traffic or more and
> my guess is that the real bottleneck is my home router here (slow
> BCM4708 SoC CPU). Still it seems like r8169 can be a bottleneck on top
> of bottleneck.
> 
> ***************
> 
> v5.18
> 334 Mbits/sec
> 
> v5.10
> 336 Mbits/sec
> 
> v5.1 + git revert 288ac524cf70a8e7ed851a61ed2a9744039dae8d
> 397 Mbits/sec (back to medium speed)
> 
> v5.1
> 335 Mbits/sec (-60 Mb/s)
> 
> v5.0
> 395 Mbits/sec
> 
> ***************
> 
> v4.19 + git revert 6b839b6cf9eada30b086effb51e5d6076bafc761
> 414 Mbits/sec (back to high speed)
> 
> v4.19
> 395 Mbits/sec (-20 Mb/s)
> 
> v4.18
> 415 Mbits/sec
> 
> v4.12
> 415 Mbits/sec