[PATCH net-next] r8169: disable TSO on a single version of RTL8168c to fix performance

[PATCH net-next] r8169: disable TSO on a single version of RTL8168c to fix performance

[Corinna Vinschen]

During performance testing, I found that one of my r8169 NICs suffered
a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected
throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
"r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
special-casing the 8168evl as per the patch below.  This fixed the
performance problem for me.

Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
---
 drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index d8fcdb9db8d1..1de11ac05bd6 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
 	}
 
-	/* RTL8168e-vl has a HW issue with TSO */
-	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+	/*
+	 * RTL8168e-vl and one RTL8168c variant are known to have a
+	 * HW issue with TSO.
+	 */
+	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
+	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
 		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
-- 
2.20.1

Re: [PATCH net-next] r8169: disable TSO on a single version of RTL8168c to fix performance

[Heiner Kallweit]

On 18.11.2019 10:55, Corinna Vinschen wrote:
> During performance testing, I found that one of my r8169 NICs suffered
> a major performance loss, a 8168c model.
> 
> Running netperf's TCP_STREAM test didn't return the expected
> throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
> enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
> throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
> test (either one of 8169s, 8168e, 8168f).
> 
> Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
> "r8169: enable HW csum and TSO" as the culprit.
> 
> I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
> special-casing the 8168evl as per the patch below.  This fixed the
> performance problem for me.
> 
> Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
> Signed-off-by: Corinna Vinschen <vinschen@redhat.com>

Thanks for reporting and the fix. Just two small nits:
- fix should be annotated "net", not "net-next"
- comment blocks in net subsystem don't have /* on a separate line

Apart from that:
Reviewed-by: Heiner Kallweit <hkallweit1@gmail.com>

Out of curiosity: Did you test also with iperf3? If yes,
do you see the same issue?

> ---
>  drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
> index d8fcdb9db8d1..1de11ac05bd6 100644
> --- a/drivers/net/ethernet/realtek/r8169_main.c
> +++ b/drivers/net/ethernet/realtek/r8169_main.c
> @@ -6952,8 +6952,12 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
>  		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
>  	}
>  
> -	/* RTL8168e-vl has a HW issue with TSO */
> -	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
> +	/*

not net style comment

> +	 * RTL8168e-vl and one RTL8168c variant are known to have a
> +	 * HW issue with TSO.
> +	 */
> +	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
> +	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
>  		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
>  		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
>  		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
> 

[PATCH net] r8169: disable TSO on a single version of RTL8168c to fix performance

[Corinna Vinschen]

During performance testing, I found that one of my r8169 NICs suffered
a major performance loss, a 8168c model.

Running netperf's TCP_STREAM test didn't return the expected
throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
test (either one of 8169s, 8168e, 8168f).

Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
"r8169: enable HW csum and TSO" as the culprit.

I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
special-casing the 8168evl as per the patch below.  This fixed the
performance problem for me.

Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
---
 drivers/net/ethernet/realtek/r8169_main.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/realtek/r8169_main.c b/drivers/net/ethernet/realtek/r8169_main.c
index d8fcdb9db8d1..1de11ac05bd6 100644
--- a/drivers/net/ethernet/realtek/r8169_main.c
+++ b/drivers/net/ethernet/realtek/r8169_main.c
@@ -6952,8 +6952,11 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 		dev->gso_max_segs = RTL_GSO_MAX_SEGS_V1;
 	}
 
-	/* RTL8168e-vl has a HW issue with TSO */
-	if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+	/* RTL8168e-vl and one RTL8168c variant are known to have a
+	 * HW issue with TSO.
+	 */
+	if (tp->mac_version == RTL_GIGA_MAC_VER_34 ||
+	    tp->mac_version == RTL_GIGA_MAC_VER_22) {
 		dev->vlan_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->hw_features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
 		dev->features &= ~(NETIF_F_ALL_TSO | NETIF_F_SG);
-- 
2.20.1

Re: [PATCH net-next] r8169: disable TSO on a single version of RTL8168c to fix performance

[Corinna Vinschen]

On Nov 18 20:33, Heiner Kallweit wrote:
> On 18.11.2019 10:55, Corinna Vinschen wrote:
> > During performance testing, I found that one of my r8169 NICs suffered
> > a major performance loss, a 8168c model.
> > 
> > Running netperf's TCP_STREAM test didn't return the expected
> > throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
> > enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
> > throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
> > test (either one of 8169s, 8168e, 8168f).
> > 
> > Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
> > "r8169: enable HW csum and TSO" as the culprit.
> > 
> > I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
> > special-casing the 8168evl as per the patch below.  This fixed the
> > performance problem for me.
> > 
> > Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
> > Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
> 
> Thanks for reporting and the fix. Just two small nits:
> - fix should be annotated "net", not "net-next"
> - comment blocks in net subsystem don't have /* on a separate line

See my v2 patch.

> Apart from that:
> Reviewed-by: Heiner Kallweit <hkallweit1@gmail.com>
> 
> Out of curiosity: Did you test also with iperf3? If yes,
> do you see the same issue?

I didn't test with iperf3 originally, but I did so now.  The results are
the same.  941 Mbits/sec vs. 23.3 Mbits/sec.


Thanks,
Corinna

Re: [PATCH net] r8169: disable TSO on a single version of RTL8168c to fix performance

[Heiner Kallweit]

On 19.11.2019 10:09, Corinna Vinschen wrote:
> During performance testing, I found that one of my r8169 NICs suffered
> a major performance loss, a 8168c model.
> 
> Running netperf's TCP_STREAM test didn't return the expected
> throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
> enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
> throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
> test (either one of 8169s, 8168e, 8168f).
> 
> Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
> "r8169: enable HW csum and TSO" as the culprit.
> 
> I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
> special-casing the 8168evl as per the patch below.  This fixed the
> performance problem for me.
> 
> Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
> Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
> ---

Reviewed-by: Heiner Kallweit <hkallweit1@gmail.com>

Re: [PATCH net] r8169: disable TSO on a single version of RTL8168c to fix performance

[David Miller]

From: Corinna Vinschen <vinschen@redhat.com>
Date: Tue, 19 Nov 2019 10:09:39 +0100

> During performance testing, I found that one of my r8169 NICs suffered
> a major performance loss, a 8168c model.
> 
> Running netperf's TCP_STREAM test didn't return the expected
> throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
> enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
> throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
> test (either one of 8169s, 8168e, 8168f).
> 
> Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
> "r8169: enable HW csum and TSO" as the culprit.
> 
> I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
> special-casing the 8168evl as per the patch below.  This fixed the
> performance problem for me.
> 
> Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
> Signed-off-by: Corinna Vinschen <vinschen@redhat.com>

Applied, but it would be really nice to know why this is happening
instead of just turning it off completely.

Re: [PATCH net] r8169: disable TSO on a single version of RTL8168c to fix performance

[Heiner Kallweit]

On 20.11.2019 01:42, David Miller wrote:
> From: Corinna Vinschen <vinschen@redhat.com>
> Date: Tue, 19 Nov 2019 10:09:39 +0100
> 
>> During performance testing, I found that one of my r8169 NICs suffered
>> a major performance loss, a 8168c model.
>>
>> Running netperf's TCP_STREAM test didn't return the expected
>> throughput of > 900 Mb/s, but rather only about 22 Mb/s.  Strange
>> enough, running the TCP_MAERTS and UDP_STREAM tests all returned with
>> throughput > 900 Mb/s, as did TCP_STREAM with the other r8169 NICs I can
>> test (either one of 8169s, 8168e, 8168f).
>>
>> Bisecting turned up commit 93681cd7d94f83903cb3f0f95433d10c28a7e9a5,
>> "r8169: enable HW csum and TSO" as the culprit.
>>
>> I added my 8168c version, RTL_GIGA_MAC_VER_22, to the code
>> special-casing the 8168evl as per the patch below.  This fixed the
>> performance problem for me.
>>
>> Fixes: 93681cd7d94f ("r8169: enable HW csum and TSO")
>> Signed-off-by: Corinna Vinschen <vinschen@redhat.com>
> 
> Applied, but it would be really nice to know why this is happening
> instead of just turning it off completely.
> 
For RTL8168e-vl Realtek confirmed a HW issue, for this RTL8168c version
supposedly it's the same as other chip versions are working fine with TSO.