All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ian Kumlien <ian.kumlien@gmail.com>
To: Linux Kernel Network Developers <netdev@vger.kernel.org>
Subject: NAT performance issue 944mbit -> ~40mbit
Date: Sat, 11 Jul 2020 17:53:25 +0200	[thread overview]
Message-ID: <CAA85sZvKNXCo5bB5a6kKmsOUAiw+_daAVaSYqNW6QbSBJ0TcyQ@mail.gmail.com> (raw)

Hi,

I first detected this with 5.7.6 but it seems to apply as far back as 5.6.1...
(so, 5.7.8 client -> nat (5.6.1 -> 5.8-rc4 -> server 5.7.7)

It seems to me that the window size doesn't advance, so i did revert
the tcp: grow window for OOO packets only for SACK flows [1]
but it did no difference...

I have a 384 MB tcpdump of a iperf3 session that starts low and then
actually starts to get the bandwidth...
I do use BBR - I have tried with cubic... it didn't help  - the NAT
machine does use fq but changing it doesn't seem to yield any other
results.

Doing -P10 gives you the bandwith and can sometimes break the
stalemate but you'll end up back with the lower transfer speed again.
(it only seems to apply to NAT - the machine is a: A2SDi-12C-HLN4F and
has handled this without problems in the past...)


[1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v5.7.8&id=bf780119617797b5690e999e59a64ad79a572374

First iperf3 as a reference:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   945 Mbits/sec    0    814 KBytes
[  5]   1.00-2.00   sec   109 MBytes   912 Mbits/sec    0    806 KBytes
[  5]   2.00-3.00   sec   112 MBytes   944 Mbits/sec   31    792 KBytes
[  5]   3.00-4.00   sec   101 MBytes   849 Mbits/sec   31   1.18 MBytes
[  5]   4.00-5.00   sec   108 MBytes   902 Mbits/sec    0    783 KBytes
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec   31    778 KBytes
[  5]   6.00-7.00   sec   111 MBytes   933 Mbits/sec   93    772 KBytes
[  5]   7.00-8.00   sec   112 MBytes   944 Mbits/sec    0    778 KBytes
[  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec   60    778 KBytes
[  5]   9.00-10.00  sec   111 MBytes   933 Mbits/sec   92    814 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.07 GBytes   923 Mbits/sec  338             sender
[  5]   0.00-10.01  sec  1.07 GBytes   919 Mbits/sec                  receiver

After that:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.77 MBytes  40.0 Mbits/sec    0   42.4 KBytes
[  5]   1.00-2.00   sec  4.10 MBytes  34.4 Mbits/sec    0   84.8 KBytes
[  5]   2.00-3.00   sec  4.60 MBytes  38.6 Mbits/sec    0   87.7 KBytes
[  5]   3.00-4.00   sec  4.23 MBytes  35.4 Mbits/sec    0   42.4 KBytes
[  5]   4.00-5.00   sec  4.23 MBytes  35.4 Mbits/sec    0   42.4 KBytes
[  5]   5.00-6.00   sec  4.47 MBytes  37.5 Mbits/sec    0   76.4 KBytes
[  5]   6.00-7.00   sec  5.47 MBytes  45.9 Mbits/sec    0   67.9 KBytes
[  5]   7.00-8.00   sec  4.66 MBytes  39.1 Mbits/sec    0   67.9 KBytes
[  5]   8.00-9.00   sec  4.35 MBytes  36.5 Mbits/sec    0   82.0 KBytes
[  5]   9.00-10.00  sec  4.66 MBytes  39.1 Mbits/sec    0    139 KBytes
- - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  45.5 MBytes  38.2 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  45.0 MBytes  37.8 Mbits/sec                  receiver

You even get some:
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  5.38 MBytes  45.2 Mbits/sec    0   42.4 KBytes
[  5]   1.00-2.00   sec  7.08 MBytes  59.4 Mbits/sec    0    535 KBytes
[  5]   2.00-3.00   sec   108 MBytes   907 Mbits/sec    0    778 KBytes
[  5]   3.00-4.00   sec   111 MBytes   933 Mbits/sec    0    814 KBytes
[  5]   4.00-5.00   sec  91.2 MBytes   765 Mbits/sec    0    829 KBytes
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec    0    783 KBytes
[  5]   6.00-7.00   sec   111 MBytes   933 Mbits/sec    0    769 KBytes
[  5]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0    778 KBytes
[  5]   8.00-9.00   sec   112 MBytes   944 Mbits/sec    0    809 KBytes
[  5]   9.00-10.00  sec   110 MBytes   923 Mbits/sec    0    823 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   879 MBytes   738 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   875 MBytes   734 Mbits/sec                  receiver

             reply	other threads:[~2020-07-11 15:53 UTC|newest]

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-11 15:53 Ian Kumlien [this message]
2020-07-15 20:05 ` NAT performance issue 944mbit -> ~40mbit Ian Kumlien
2020-07-15 20:05   ` [Intel-wired-lan] " Ian Kumlien
2020-07-15 20:31   ` Jakub Kicinski
2020-07-15 20:31     ` [Intel-wired-lan] " Jakub Kicinski
2020-07-15 21:02     ` Ian Kumlien
2020-07-15 21:02       ` [Intel-wired-lan] " Ian Kumlien
2020-07-15 21:12       ` Ian Kumlien
2020-07-15 21:12         ` [Intel-wired-lan] " Ian Kumlien
2020-07-15 21:40         ` Jakub Kicinski
2020-07-15 21:40           ` [Intel-wired-lan] " Jakub Kicinski
2020-07-15 21:59           ` Ian Kumlien
2020-07-15 21:59             ` [Intel-wired-lan] " Ian Kumlien
2020-07-15 22:32             ` Alexander Duyck
2020-07-15 22:32               ` Alexander Duyck
2020-07-15 22:51               ` Ian Kumlien
2020-07-15 22:51                 ` Ian Kumlien
2020-07-15 23:41                 ` Alexander Duyck
2020-07-15 23:41                   ` Alexander Duyck
2020-07-15 23:59                   ` Ian Kumlien
2020-07-15 23:59                     ` Ian Kumlien
2020-07-16 15:18                     ` Alexander Duyck
2020-07-16 15:18                       ` Alexander Duyck
2020-07-16 15:51                       ` Ian Kumlien
2020-07-16 19:47                       ` Ian Kumlien
2020-07-16 19:47                         ` Ian Kumlien
2020-07-17  0:09                         ` Alexander Duyck
2020-07-17  0:09                           ` Alexander Duyck
2020-07-17 13:45                           ` Ian Kumlien
2020-07-17 13:45                             ` Ian Kumlien
2020-07-24 12:01                             ` Ian Kumlien
2020-07-24 12:01                               ` Ian Kumlien
2020-07-24 12:33                               ` Ian Kumlien
2020-07-24 12:33                                 ` Ian Kumlien
2020-07-24 14:56                                 ` Alexander Duyck
2020-07-24 14:56                                   ` Alexander Duyck
     [not found]                                   ` <CAA85sZsEG_SCC4GLb8xaUsESrzZyAwF0qmse6sJ=e1QkK9DVsQ@mail.gmail.com>
     [not found]                                     ` <CAKgT0UcY4FwAFf0BXv7vc_5ram7YkFXda78PWkdEFgMLsitvWA@mail.gmail.com>
     [not found]                                       ` <CAA85sZs_PSsyZhvdKBCoAGxoZvaQFhQ6j7qoA7y8ffjs2RqEGw@mail.gmail.com>
2020-07-24 21:50                                         ` Alexander Duyck
2020-07-24 22:41                                           ` Ian Kumlien
2020-07-24 22:49                                             ` Ian Kumlien
2020-07-24 23:08                                               ` Ian Kumlien
2020-07-25  0:13                                                 ` Ian Kumlien
2020-07-25  0:45                                                 ` Alexander Duyck
2020-07-25  1:03                                                   ` Ian Kumlien
2020-07-25 13:53                                                     ` Ian Kumlien
2020-07-25 17:43                                                       ` Alexander Duyck
2020-07-25 18:56                                                         ` Ian Kumlien
2020-07-25 19:35                                                           ` Ian Kumlien
2020-07-25 20:10                                                             ` Alexander Duyck
2020-07-25 20:16                                                               ` Ian Kumlien
2020-07-25 17:30                                                     ` Alexander Duyck
2020-07-25 18:52                                                       ` Ian Kumlien

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAA85sZvKNXCo5bB5a6kKmsOUAiw+_daAVaSYqNW6QbSBJ0TcyQ@mail.gmail.com \
    --to=ian.kumlien@gmail.com \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.