All of lore.kernel.org
 help / color / mirror / Atom feed
From: Shangshu Qian <qianshangshu_1997@outlook.com>
To: intel-wired-lan@osuosl.org
Subject: [Intel-wired-lan] PROBLEM: [e1000e] 10% throughput drop on I219-LM after the buffer overrun fix even with TSO&GSO disabled
Date: Fri, 17 Jul 2020 03:37:21 +0000	[thread overview]
Message-ID: <SJ0PR07MB7584416FB7A07FD506B8237F9C7C0@SJ0PR07MB7584.namprd07.prod.outlook.com> (raw)
In-Reply-To: <SJ0PR07MB7584B6DC5998006092D4D8859C7C0@SJ0PR07MB7584.namprd07.prod.outlook.com>

[1.] One line summary of the problem:

[e1000e] 10% throughput drop on I219-LM after the buffer overrun fix even with TSO&GSO disabled

[2.] Full description of the problem/report:

With e1000e 3.6.2-k and 3.8.4-NAPI driver, which include the fix for the buffer overrun problem (https://github.com/torvalds/linux/commit/b10effb92e272051dd1ec0d7be56bf9ca85ab927, discussion: https://www.spinics.net/lists/netdev/msg460589.html), I219-LM network card have a 10% throughput drop in the iperf3 test (with TSO and GSO disabled) compared with the driver without such patch, and the client is initiating the stream in such test (iperf3 client mode without -R option). If the server is initiating the stream (with -R option in iperf3 client mode), the performance is not impacted.

That is to say, disabling TSO and GSO as suggested in that patch still have performance impact on the TCP stream, and the throughput drops about 10%.

I tried to rollback the patch introduced to fix the buffer overrun problem in the 3.8.4-NAPI driver, with TSO and GSO enabled, iperf3 test still cannot max out my 1Gbps uplink. However, with TSO and GSO disabled, 1Gbps uplink can be fully saturated.

I'll attach iperf3 test results for all these situations later in this email.

[3.] Keywords

Network driver, e1000e

[4.] Kernel information
[4.1.] Kernel version (from /proc/version):

Linux version 5.4.0-31-generic (buildd at lgw01-amd64-059) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #35-Ubuntu SMP Thu May 7 20:20:34 UTC 2020

[4.2.] Kernel .config file:

https://kernel.ubuntu.com/~kernel-ppa/config/focal/linux/5.4.0-31.35/amd64-config.flavour.generic

[5.] Most recent kernel version which did not have the bug:

V4.14.2

[6.] Environment

00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (2) I219-LM (rev 31) 

driver: e1000e
version: 3.2.6-k
firmware-version: 0.8-4
expansion-rom-version:
bus-info: 0000:00:1f.6
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: no

iperf3:
iperf 3.7 (cJSON 1.5.2)
Linux Hetzner 5.4.0-31-generic #35-Ubuntu SMP Thu May 7 20:20:34 UTC 2020 x86_64
Optional features available: CPU affinity setting, IPv6 flow label, SCTP, TCP congestion algorithm setting, sendfile / zerocopy, socket pacing, authentication

[X.] Other notes, patches, fixes, workarounds:

3.6.2-k w/ TSO & GSO

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 120 MBytes?? 101 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 119 MBytes? 99.4 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 120 MBytes?? 101 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 119 MBytes? 99.4 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 120 MBytes?? 101 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 119 MBytes? 99.4 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 120 MBytes?? 101 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 119 MBytes? 99.5 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 120 MBytes?? 101 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 119 MBytes? 99.5 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec?? 601 MBytes?? 504 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec?? 594 MBytes?? 497 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.02? sec?? 351 MBytes?? 293 Mbits/sec? 134???????????? sender
[? 5]?? 0.00-10.00? sec?? 344 MBytes?? 289 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 204 MBytes?? 171 Mbits/sec? 331???????????? sender
[? 7]?? 0.00-10.00? sec?? 201 MBytes?? 168 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 174 MBytes?? 146 Mbits/sec? 226???????????? sender
[? 9]?? 0.00-10.00? sec?? 171 MBytes?? 144 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 177 MBytes?? 148 Mbits/sec? 226???????????? sender
[ 11]?? 0.00-10.00? sec?? 173 MBytes?? 145 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 164 MBytes?? 138 Mbits/sec? 229???????????? sender
[ 13]?? 0.00-10.00? sec?? 161 MBytes?? 135 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.04 GBytes?? 896 Mbits/sec? 1146?????? ??????sender
[SUM]?? 0.00-10.00? sec? 1.03 GBytes?? 881 Mbits/sec????????????????? receiver

3.6.2-k w/o TSO & GSO

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 190 MBytes?? 159 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 188 MBytes?? 157 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 190 MBytes?? 160 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 188 MBytes?? 157 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 191 MBytes?? 160 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 188 MBytes?? 158 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 190 MBytes?? 160 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 188 MBytes?? 158 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 191 MBytes?? 160 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 188 MBytes?? 157 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec?? 953 MBytes?? 799 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec?? 940 MBytes?? 787 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.02? sec?? 256 MBytes?? 214 Mbits/sec? 128???????????? sender
[? 5]?? 0.00-10.00? sec?? 252 MBytes?? 211 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 238 MBytes?? 199 Mbits/sec? 119???????????? sender
[? 7]?? 0.00-10.00? sec?? 234 MBytes?? 196 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 166 MBytes?? 139 Mbits/sec? 207???????????? sender
[? 9]?? 0.00-10.00? sec?? 162 MBytes?? 136 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 222 MBytes?? 186 Mbits/sec? 316???????????? sender
[ 11]?? 0.00-10.00? sec?? 219 MBytes?? 184 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 225 MBytes?? 189 Mbits/sec? 138???????????? sender
[ 13]?? 0.00-10.00? sec?? 222 MBytes?? 186 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.08 GBytes?? 927 Mbits/sec? 908??????? ?????sender
[SUM]?? 0.00-10.00? sec? 1.06 GBytes?? 913 Mbits/sec????????????????? receiver

3.8.4-NAPI w/ TSO & GSO

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 120 MBytes?? 100 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 117 MBytes? 97.5 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 115 MBytes? 96.4 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 114 MBytes? 95.1 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 118 MBytes? 98.9 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 117 MBytes? 97.6 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 118 MBytes? 99.3 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 117 MBytes? 97.5 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 118 MBytes? 99.3 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 116 MBytes? 97.5 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec?? 589 MBytes?? 494 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec?? 580 MBytes?? 485 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.02? sec?? 289 MBytes?? 241 Mbits/sec? 187???????????? sender
[? 5]?? 0.00-10.00? sec?? 282 MBytes?? 236 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 355 MBytes?? 297 Mbits/sec? 324???????????? sender
[? 7]?? 0.00-10.00? sec?? 350 MBytes?? 293 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 144 MBytes?? 121 Mbits/sec? 239???????????? sender
[? 9]?? 0.00-10.00? sec?? 141 MBytes?? 118 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 136 MBytes?? 113 Mbits/sec? 286???????????? sender
[ 11]?? 0.00-10.00? sec?? 133 MBytes?? 111 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 188 MBytes?? 157 Mbits/sec? 156???????????? sender
[ 13]?? 0.00-10.00? sec?? 185 MBytes?? 155 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.09 GBytes?? 930 Mbits/sec? 1192?????? ??????sender
[SUM]?? 0.00-10.00? sec? 1.06 GBytes?? 914 Mbits/sec????????????????? receiver

3.8.4-NAPI w/o TSO & GSO

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 194 MBytes?? 163 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 192 MBytes?? 160 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 194 MBytes?? 163 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 192 MBytes?? 160 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 195 MBytes?? 163 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 192 MBytes?? 160 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 195 MBytes?? 163 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 192 MBytes?? 160 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 195 MBytes?? 163 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 192 MBytes?? 161 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec?? 973 MBytes?? 816 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec?? 958 MBytes?? 802 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.02? sec?? 499 MBytes?? 418 Mbits/sec? 1208???????????? sender
[? 5]?? 0.00-10.00? sec?? 487 MBytes?? 409 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 149 MBytes?? 125 Mbits/sec? 262???????????? sender
[? 7]?? 0.00-10.00? sec?? 146 MBytes?? 122 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 158 MBytes?? 132 Mbits/sec? 199???????????? sender
[? 9]?? 0.00-10.00? sec?? 154 MBytes?? 129 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 179 MBytes?? 149 Mbits/sec? 209???????????? sender
[ 11]?? 0.00-10.00? sec?? 175 MBytes?? 147 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 128 MBytes?? 107 Mbits/sec? 386???????????? sender
[ 13]?? 0.00-10.00? sec?? 125 MBytes?? 105 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.09 GBytes?? 931 Mbits/sec? 2264?????? ??????sender
[SUM]?? 0.00-10.00? sec? 1.06 GBytes?? 912 Mbits/sec????????????????? receiver

3.8.4-NAPI w/ TSO & GSO (Without buffer overrun patch)

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 204 MBytes?? 171 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 201 MBytes?? 169 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 203 MBytes?? 170 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 201 MBytes?? 168 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 205 MBytes?? 172 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 202 MBytes?? 169 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 204 MBytes?? 171 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 201 MBytes?? 169 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 204 MBytes?? 171 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 200 MBytes?? 168 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec? 1019 MBytes?? 855 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec? 1006 MBytes?? 842 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[? 5]?? 0.00-10.02? sec?? 281 MBytes?? 235 Mbits/sec? 128???????????? sender
[? 5]?? 0.00-10.00? sec?? 277 MBytes?? 232 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 267 MBytes?? 223 Mbits/sec? 154???????????? sender
[? 7]?? 0.00-10.00? sec?? 263 MBytes?? 221 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 170 MBytes?? 142 Mbits/sec? 154???????????? sender
[? 9]?? 0.00-10.00? sec?? 166 MBytes?? 139 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 161 MBytes?? 135 Mbits/sec? 211???????????? sender
[ 11]?? 0.00-10.00? sec?? 158 MBytes?? 132 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 219 MBytes?? 183 Mbits/sec? 116???????????? sender
[ 13]?? 0.00-10.00? sec?? 216 MBytes?? 181 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.07 GBytes?? 919 Mbits/sec? 763???????????? sender
[SUM]?? 0.00-10.00? sec? 1.05 GBytes?? 906 Mbits/sec?????????????? ???receiver

3.8.4-NAPI w/o TSO & GSO (Without buffer overrun patch)

iperf3 -c bouygues.iperf.fr -d -P 5

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.00? sec?? 220 MBytes?? 184 Mbits/sec??? 0???????????? sender
[? 5]?? 0.00-10.02? sec?? 218 MBytes?? 182 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.00? sec?? 220 MBytes?? 185 Mbits/sec??? 0???????????? sender
[? 7]?? 0.00-10.02? sec?? 217 MBytes?? 182 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.00? sec?? 216 MBytes?? 182 Mbits/sec??? 0???????????? sender
[? 9]?? 0.00-10.02? sec?? 213 MBytes?? 179 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.00? sec?? 220 MBytes?? 185 Mbits/sec??? 0???????????? sender
[ 11]?? 0.00-10.02? sec?? 218 MBytes?? 182 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.00? sec?? 221 MBytes?? 185 Mbits/sec??? 0???????????? sender
[ 13]?? 0.00-10.02? sec?? 218 MBytes?? 182 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.00? sec? 1.07 GBytes?? 920 Mbits/sec??? 0??????? ?????sender
[SUM]?? 0.00-10.02? sec? 1.06 GBytes?? 907 Mbits/sec????????????????? receiver

iperf3 -c bouygues.iperf.fr -d -P 5 -R

[ ID] Interval?????????? Transfer???? Bitrate???????? Retr
[? 5]?? 0.00-10.02? sec?? 259 MBytes?? 217 Mbits/sec?? 67???????????? sender
[? 5]?? 0.00-10.00? sec?? 255 MBytes?? 214 Mbits/sec????????????????? receiver
[? 7]?? 0.00-10.02? sec?? 256 MBytes?? 214 Mbits/sec? 168???????????? sender
[? 7]?? 0.00-10.00? sec?? 252 MBytes?? 212 Mbits/sec????????????????? receiver
[? 9]?? 0.00-10.02? sec?? 260 MBytes?? 217 Mbits/sec? 189???????????? sender
[? 9]?? 0.00-10.00? sec?? 256 MBytes?? 215 Mbits/sec????????????????? receiver
[ 11]?? 0.00-10.02? sec?? 165 MBytes?? 138 Mbits/sec? 237???????????? sender
[ 11]?? 0.00-10.00? sec?? 161 MBytes?? 135 Mbits/sec????????????????? receiver
[ 13]?? 0.00-10.02? sec?? 172 MBytes?? 144 Mbits/sec? 220???????????? sender
[ 13]?? 0.00-10.00? sec?? 169 MBytes?? 142 Mbits/sec????????????????? receiver
[SUM]?? 0.00-10.02? sec? 1.08 GBytes?? 930 Mbits/sec? 881??????? ?????sender
[SUM]?? 0.00-10.00? sec? 1.07 GBytes?? 917 Mbits/sec????????????????? receiver

  reply	other threads:[~2020-07-17  3:37 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-17  3:28 [Intel-wired-lan] PROBLEM: [e1000e] 10% throughput drop on I219-LM after the buffer overrun fix even with TSO&GSO disabled Shangshu Qian
2020-07-17  3:37 ` Shangshu Qian [this message]
2020-07-17  3:50   ` qianshangshu.1997
2020-07-17  3:50     ` [Intel-wired-lan] " qianshangshu.1997

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=SJ0PR07MB7584416FB7A07FD506B8237F9C7C0@SJ0PR07MB7584.namprd07.prod.outlook.com \
    --to=qianshangshu_1997@outlook.com \
    --cc=intel-wired-lan@osuosl.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.