From: Wolfgang Walter <linux@stwm.de>
To: netdev@vger.kernel.org
Cc: Florian Westphal <fw@strlen.de>,
Steffen Klassert <steffen.klassert@secunet.com>,
David Miller <davem@redhat.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Regression: kernel 4.14 an later very slow with many ipsec tunnels
Date: Thu, 13 Sep 2018 13:30:12 +0200 [thread overview]
Message-ID: <5781211.jtEvgqSZyO@stwm.de> (raw)
Hello,
thanks to the fix from Steffen Klassert I could now run 4.14.69 + his patch
and 4.18.7 + his patch without oopsing immediately.
But I found that those kernels perform very bad. They perform so bad that they
are unusable for our router with about 3000 ipsec tunnels (tunnel mode network
<-> network).
With 4.9. (and all other kernels I used in the last 10 years with much less
potent hardware) I never had an comparable performance issue with networking.
4.18.7 is better then 4.14.69 but still remains unusuable for us.
Even with very little traffic all 8 cores are working 100% in ksoftirqd. As
soon as there is real traffic network gets rather unusuable.
Latency of packets goes from between 0.1ms to 1ms up to 100ms to 500ms (4.14)
or between 15ms to 90ms (4.18).
Throughput also suffers a lot.
I have a simple test I run after every upgrade. This test basically copies
with scp large files to 60 different remote locations (involving ipsec),
limited to 1GBit/s combined, and in paralled I ping from different networks
over this router to machines in other networks of this router (no ipsec-
tunnels involved).
With 4.9 and earlier copying needs about 2 minutes and the pings all remain
under 2ms roundtrip.
With 4.14 copying these files needs more than one our. The roundtrip time of
the ping is > 1 second.
With 4.18 this is much better, copying needs around 6 minutes and ping
roundtrip is between 30ms and 180ms. But even that is much worse then 4.9.
I think this dramatic loss in performance is due to the removal of the flow
cache. I propose to revert that for 4.14. I also propose to revert it for the
next longterm kernel if no other solution is found bringing back 4.9
performance (at least about the same order of magnitude).
Maybe it should generally be reverted until a solution to the problem exists.
Regards,
--
Wolfgang Walter
Studentenwerk München
Anstalt des öffentlichen Rechts
next reply other threads:[~2018-09-13 11:36 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-13 11:30 Wolfgang Walter [this message]
2018-09-13 13:58 ` Regression: kernel 4.14 an later very slow with many ipsec tunnels Florian Westphal
2018-09-13 15:46 ` Wolfgang Walter
2018-09-13 16:38 ` Florian Westphal
2018-09-13 17:23 ` David Miller
2018-09-13 21:03 ` Florian Westphal
2018-09-13 21:12 ` David Miller
2018-09-14 5:06 ` Steffen Klassert
2018-09-14 5:54 ` Florian Westphal
2018-09-14 6:01 ` Steffen Klassert
2018-09-14 8:01 ` Christophe Gouault
2018-09-14 11:49 ` Wolfgang Walter
2018-10-02 14:45 ` Wolfgang Walter
2018-10-02 14:56 ` Florian Westphal
2018-10-02 17:34 ` Wolfgang Walter
2018-10-02 21:35 ` Florian Westphal
2018-10-04 13:57 ` Wolfgang Walter
2018-10-25 9:38 ` Wolfgang Walter
2018-10-25 17:34 ` David Miller
2018-10-25 19:24 ` Florian Westphal
2018-10-26 12:18 ` Wolfgang Walter
2018-10-25 22:45 ` Florian Westphal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5781211.jtEvgqSZyO@stwm.de \
--to=linux@stwm.de \
--cc=davem@redhat.com \
--cc=fw@strlen.de \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=steffen.klassert@secunet.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).