Re: r8169: Ethernet speed regressions with RTL8168gu in ThinkPad E480

From: Heiner Kallweit <hkallweit1@gmail.com>
To: "Rafał Miłecki" <zajec5@gmail.com>,
	"Network Development" <netdev@vger.kernel.org>,
	nic_swsd@realtek.com
Subject: Re: r8169: Ethernet speed regressions with RTL8168gu in ThinkPad E480
Date: Mon, 30 May 2022 21:21:21 +0200	[thread overview]
Message-ID: <81e63fc9-ac8c-cb35-4572-c808ddab997d@gmail.com> (raw)
In-Reply-To: <5a04080f-a2eb-6597-091e-6b31c4df1661@gmail.com>

On 30.05.2022 16:10, Rafał Miłecki wrote:
> I've ThinkPad E480 with i3-8130U CPU and 10ec:8168 detected as:
> [    8.458515] r8169 0000:03:00.0 eth0: RTL8168gu/8111gu, 8c:16:45:5d:f2:c2, XID 509, IRQ 136
> [    8.458521] r8169 0000:03:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
> [   12.272352] Generic FE-GE Realtek PHY r8169-0-300:00: attached PHY driver (mii_bus:phy_addr=r8169-0-300:00, irq=MAC)
> 
> It's connected to WAN port of BCM4708 based home router running OpenWrt
> with kernel 5.4.113. I run "iperf -s" on this E480 for network testing.
> 
> ***************
> 
> After upgrading kernel from 4.12 to 5.18 I noticed Ethernet speed
> regression. I bisected it down to two commits:
> 
> 
> commit 6b839b6cf9eada30b086effb51e5d6076bafc761
> Author: Heiner Kallweit <hkallweit1@gmail.com>
> Date:   Thu Oct 18 19:56:01 2018 +0200
> 
>     r8169: fix NAPI handling under high load
> 
> (introduced in 4.19 and dropped speed by 20 Mb/s = 5%)
> 
> 
> commit 288ac524cf70a8e7ed851a61ed2a9744039dae8d
> Author: Heiner Kallweit <hkallweit1@gmail.com>
> Date:   Sat Mar 30 17:13:24 2019 +0100
> 
>     r8169: disable default rx interrupt coalescing on RTL8168
> 
> (introduced in 5.1 and dropped speed by 60 Mb/s = 15%)
> 
> ***************
> 
> Is that possible to fix / rework r8169 to provide original higher
> Ethernet speed out of the box?
> 
With RTL8168g and RTL8168h I get >900Mbps in both directions.
Both referenced changes are needed. What you could try:

Enable TSO:
ethtool -K eth0 sg on tso on

As replacement for hw irq coalescing:
echo 20000 > /sys/class/net/eth0/gro_flush_timeout
echo 1 > /sys/class/net/eth0/napi_defer_hard_irqs

Or use ethtool to (re-)enable hw irq coalescing.

> Honestly I'd expect this i3-8130U to handle 1 Gb/s traffic or more and
> my guess is that the real bottleneck is my home router here (slow
> BCM4708 SoC CPU). Still it seems like r8169 can be a bottleneck on top
> of bottleneck.
> 
> ***************
> 
> v5.18
> 334 Mbits/sec
> 
> v5.10
> 336 Mbits/sec
> 
> v5.1 + git revert 288ac524cf70a8e7ed851a61ed2a9744039dae8d
> 397 Mbits/sec (back to medium speed)
> 
> v5.1
> 335 Mbits/sec (-60 Mb/s)
> 
> v5.0
> 395 Mbits/sec
> 
> ***************
> 
> v4.19 + git revert 6b839b6cf9eada30b086effb51e5d6076bafc761
> 414 Mbits/sec (back to high speed)
> 
> v4.19
> 395 Mbits/sec (-20 Mb/s)
> 
> v4.18
> 415 Mbits/sec
> 
> v4.12
> 415 Mbits/sec