From: Baptiste Jonglez <baptiste@bitsofnetworks.org>
To: David Ahern <dsahern@gmail.com>
Cc: Alarig Le Lay <alarig@swordarmor.fr>,
netdev@vger.kernel.org, jack@basilfillan.uk,
Vincent Bernat <bernat@debian.org>, Oliver <bird-o@sernet.de>
Subject: Re: IPv6 regression introduced by commit 3b6761d18bc11f2af2a6fc494e9026d39593f22c
Date: Mon, 28 Sep 2020 08:48:00 +0200 [thread overview]
Message-ID: <20200928064800.GA1132636@tuxmachine.localdomain> (raw)
In-Reply-To: <66345b05-7864-ced2-7f3c-493260be39f7@gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2176 bytes --]
On 27-09-20, David Ahern wrote:
> On 9/27/20 9:10 AM, Baptiste Jonglez wrote:
> > On 27-09-20, Baptiste Jonglez wrote:
> >> 1) failing IPv6 neighbours, what Alarig reported. We are seeing this
> >> on a full-view BGP router with rather low amount of IPv6 traffic
> >> (around 10-20 Mbps)
> >
> > Ok, I found a quick way to reproduce this issue:
> >
> > # for net in {1..9999}; do ip -6 route add 2001:db8:ffff:${net}::/64 via fe80::4242 dev lo; done
> >
> > and then:
> >
> > # for net in {1..9999}; do ping -c1 2001:db8:ffff:${net}::1; done
> >
> > This quickly gets to a situation where ping fails early with:
> >
> > ping: connect: Network is unreachable
> >
> > At this point, IPv6 connectivity is broken. The kernel is no longer
> > replying to IPv6 neighbor solicitation from other hosts on local
> > networks.
> >
> > When this happens, the "fib_rt_alloc" field from /proc/net/rt6_stats
> > is roughly equal to net.ipv6.route.max_size (a bit more in my tests).
> >
> > Interestingly, the system appears to stay in this broken state
> > indefinitely, even without trying to send new IPv6 traffic. The
> > fib_rt_alloc statistics does not decrease.
> >
>
> fib_rt_alloc is incremented by calls to ip6_dst_alloc. Each of your
> 9,999 pings is to a unique address and hence causes a dst to be
> allocated and the counter to be incremented. It is never decremented.
> That is standard operating procedure.
Ok, then this is a change in behaviour. Here is a graph of fib_rt_alloc
on a busy router (IPv6 full view, moderate IPv6 traffic) with 4.9 kernel:
https://files.polyno.me/tmp/rt6_stats_fib_rt_alloc_4.9.png
It varies quite a lot and stays around 50, so clearly it can be
decremented in regular operation.
On 4.19 and later, it does seem to be decremented only when a route is
removed (ip -6 route delete). Here is the same graph on a router with a
4.19 kernel and a large net.ipv6.route.max_size:
https://files.polyno.me/tmp/rt6_stats_fib_rt_alloc_4.19.png
Overall, do you mean that fib_rt_alloc is a red herring and is not a good
marker of the issue?
Thanks,
Baptiste
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2020-09-28 6:48 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-05 8:17 IPv6 regression introduced by commit 3b6761d18bc11f2af2a6fc494e9026d39593f22c Alarig Le Lay
2020-03-08 0:52 ` David Ahern
2020-03-08 10:57 ` Alarig Le Lay
2020-03-09 2:15 ` David Ahern
2020-03-09 8:59 ` Fabian Grünbichler
2020-03-09 10:47 ` Alarig Le Lay
2020-03-09 11:35 ` Fabian Grünbichler
2020-03-10 10:35 ` Alarig Le Lay
2020-03-10 15:27 ` David Ahern
2020-03-29 14:09 ` Alarig Le Lay
2020-09-27 15:35 ` Baptiste Jonglez
2020-09-27 16:10 ` Baptiste Jonglez
2020-09-28 3:38 ` David Ahern
2020-09-28 5:39 ` Vincent Bernat
2020-09-28 6:48 ` Baptiste Jonglez [this message]
2020-09-29 3:39 ` David Ahern
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200928064800.GA1132636@tuxmachine.localdomain \
--to=baptiste@bitsofnetworks.org \
--cc=alarig@swordarmor.fr \
--cc=bernat@debian.org \
--cc=bird-o@sernet.de \
--cc=dsahern@gmail.com \
--cc=jack@basilfillan.uk \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).