From: Jonathan Maxwell <jmaxwell37@gmail.com>
To: David Ahern <dsahern@kernel.org>
Cc: Paolo Abeni <pabeni@redhat.com>,
davem@davemloft.net, edumazet@google.com, kuba@kernel.org,
yoshfuji@linux-ipv6.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [net-next] ipv6: fix routing cache overflow for raw sockets
Date: Wed, 21 Dec 2022 15:31:09 +1100 [thread overview]
Message-ID: <CAGHK07D2Dy4zFGHqwdyg+nsRC_iL4ArWTPk7L2ndA2PaLfOMYQ@mail.gmail.com> (raw)
In-Reply-To: <CAGHK07BehyHXoS+27=cfZoKz4XNTcJjyB5us33sNS7P+_fudHQ@mail.gmail.com>
On Wed, Dec 21, 2022 at 8:55 AM Jonathan Maxwell <jmaxwell37@gmail.com> wrote:
>
> On Wed, Dec 21, 2022 at 2:10 AM David Ahern <dsahern@kernel.org> wrote:
> >
> > On 12/20/22 5:35 AM, Paolo Abeni wrote:
> > > On Mon, 2022-12-19 at 10:48 +1100, Jon Maxwell wrote:
> > >> Sending Ipv6 packets in a loop via a raw socket triggers an issue where a
> > >> route is cloned by ip6_rt_cache_alloc() for each packet sent. This quickly
> > >> consumes the Ipv6 max_size threshold which defaults to 4096 resulting in
> > >> these warnings:
> > >>
> > >> [1] 99.187805] dst_alloc: 7728 callbacks suppressed
> > >> [2] Route cache is full: consider increasing sysctl net.ipv6.route.max_size.
> > >> .
> > >> .
> > >> [300] Route cache is full: consider increasing sysctl net.ipv6.route.max_size.
> > >
> > > If I read correctly, the maximum number of dst that the raw socket can
> > > use this way is limited by the number of packets it allows via the
> > > sndbuf limit, right?
> > >
> > > Are other FLOWI_FLAG_KNOWN_NH users affected, too? e.g. nf_dup_ipv6,
> > > ipvs, seg6?
> > >
> > > @DavidA: why do we need to create RTF_CACHE clones for KNOWN_NH flows?
> > >
> > > Thanks,
> > >
> > > Paolo
> > >
> >
> > If I recall the details correctly: that sysctl limit was added back when
> > ipv6 routes were managed as dst_entries and there was a desire to allow
> > an admin to limit the memory consumed. At this point in time, IPv6 is
> > more inline with IPv4 - a separate struct for fib entries from dst
> > entries. That "Route cache is full" message is now out of date since
> > this is dst_entries which have a gc mechanism.
> >
> > IPv4 does not limit the number of dst_entries that can be allocated
> > (ip_rt_max_size is the sysctl variable behind the ipv4 version of
> > max_size and it is a no-op). IPv6 can probably do the same here?
> >
>
> diff --git a/net/ipv6/route.c b/net/ipv6/route.c
> index dbc224023977..701aba7feaf5 100644
> --- a/net/ipv6/route.c
> +++ b/net/ipv6/route.c
> @@ -6470,7 +6470,7 @@ static int __net_init ip6_route_net_init(struct net *net)
> #endif
>
> net->ipv6.sysctl.flush_delay = 0;
> - net->ipv6.sysctl.ip6_rt_max_size = 4096;
> + net->ipv6.sysctl.ip6_rt_max_size = INT_MAX;
> net->ipv6.sysctl.ip6_rt_gc_min_interval = HZ / 2;
> net->ipv6.sysctl.ip6_rt_gc_timeout = 60*HZ;
> net->ipv6.sysctl.ip6_rt_gc_interval = 30*HZ;
>
> The above patch resolved it for the Ipv6 reproducer.
>
> Would that be sufficient?
>
Otherwise if you prefer to make Ipv6 behaviour similar to IPv4.
Rather than upping max_size.
Here is prototype patch that removes the max_size check for Ipv6:
diff --git a/include/net/dst_ops.h b/include/net/dst_ops.h
index 88ff7bb2bb9b..632086b2f644 100644
--- a/include/net/dst_ops.h
+++ b/include/net/dst_ops.h
@@ -16,7 +16,7 @@ struct dst_ops {
unsigned short family;
unsigned int gc_thresh;
- int (*gc)(struct dst_ops *ops);
+ void (*gc)(struct dst_ops *ops);
struct dst_entry * (*check)(struct dst_entry *, __u32 cookie);
unsigned int (*default_advmss)(const struct dst_entry *);
unsigned int (*mtu)(const struct dst_entry *);
diff --git a/net/core/dst.c b/net/core/dst.c
index 497ef9b3fc6a..dcb85267bc4c 100644
--- a/net/core/dst.c
+++ b/net/core/dst.c
@@ -82,12 +82,8 @@ void *dst_alloc(struct dst_ops *ops, struct net_device *dev,
if (ops->gc &&
!(flags & DST_NOCOUNT) &&
- dst_entries_get_fast(ops) > ops->gc_thresh) {
- if (ops->gc(ops)) {
- pr_notice_ratelimited("Route cache is full:
consider increasing sysctl net.ipv6.route.max_size.\n");
- return NULL;
- }
- }
+ dst_entries_get_fast(ops) > ops->gc_thresh)
+ ops->gc(ops);
dst = kmem_cache_alloc(ops->kmem_cachep, GFP_ATOMIC);
if (!dst)
diff --git a/net/ipv6/route.c b/net/ipv6/route.c
index dbc224023977..8db7c5436da4 100644
--- a/net/ipv6/route.c
+++ b/net/ipv6/route.c
@@ -91,7 +91,7 @@ static struct dst_entry *ip6_negative_advice(struct
dst_entry *);
static void ip6_dst_destroy(struct dst_entry *);
static void ip6_dst_ifdown(struct dst_entry *,
struct net_device *dev, int how);
-static int ip6_dst_gc(struct dst_ops *ops);
+static void ip6_dst_gc(struct dst_ops *ops);
static int ip6_pkt_discard(struct sk_buff *skb);
static int ip6_pkt_discard_out(struct net *net, struct
sock *sk, struct sk_buff *skb);
@@ -3295,32 +3295,21 @@ struct dst_entry *icmp6_dst_alloc(struct
net_device *dev,
return dst;
}
-static int ip6_dst_gc(struct dst_ops *ops)
+static void ip6_dst_gc(struct dst_ops *ops)
{
struct net *net = container_of(ops, struct net, ipv6.ip6_dst_ops);
- int rt_min_interval = net->ipv6.sysctl.ip6_rt_gc_min_interval;
- int rt_max_size = net->ipv6.sysctl.ip6_rt_max_size;
int rt_elasticity = net->ipv6.sysctl.ip6_rt_gc_elasticity;
int rt_gc_timeout = net->ipv6.sysctl.ip6_rt_gc_timeout;
- unsigned long rt_last_gc = net->ipv6.ip6_rt_last_gc;
int entries;
entries = dst_entries_get_fast(ops);
- if (entries > rt_max_size)
- entries = dst_entries_get_slow(ops);
-
- if (time_after(rt_last_gc + rt_min_interval, jiffies) &&
- entries <= rt_max_size)
- goto out;
net->ipv6.ip6_rt_gc_expire++;
fib6_run_gc(net->ipv6.ip6_rt_gc_expire, net, true);
entries = dst_entries_get_slow(ops);
if (entries < ops->gc_thresh)
net->ipv6.ip6_rt_gc_expire = rt_gc_timeout>>1;
-out:
net->ipv6.ip6_rt_gc_expire -= net->ipv6.ip6_rt_gc_expire>>rt_elasticity;
- return entries > rt_max_size;
}
static int ip6_nh_lookup_table(struct net *net, struct fib6_config *cfg,
> > I do not believe the suggested flag is the right change.
>
> Regards
>
> Jon
next prev parent reply other threads:[~2022-12-21 4:31 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-18 23:48 [net-next] ipv6: fix routing cache overflow for raw sockets Jon Maxwell
2022-12-20 12:35 ` Paolo Abeni
2022-12-20 15:10 ` David Ahern
2022-12-20 21:55 ` Jonathan Maxwell
2022-12-21 4:31 ` Jonathan Maxwell [this message]
2022-12-22 5:39 ` Jonathan Maxwell
2022-12-22 16:17 ` David Ahern
2022-12-22 22:36 ` Jonathan Maxwell
2022-12-20 15:17 ` Julian Anastasov
2022-12-20 15:41 ` Julian Anastasov
2022-12-20 21:48 ` Jonathan Maxwell
2022-12-23 20:28 ` Andrea Mayer
2022-12-24 7:38 ` Jonathan Maxwell
2023-01-02 23:59 ` Jonathan Maxwell
2023-01-03 16:07 ` Andrea Mayer
2023-01-06 23:26 ` Andrea Mayer
2023-01-07 23:46 ` Jonathan Maxwell
2023-01-08 17:34 ` Andrea Mayer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAGHK07D2Dy4zFGHqwdyg+nsRC_iL4ArWTPk7L2ndA2PaLfOMYQ@mail.gmail.com \
--to=jmaxwell37@gmail.com \
--cc=davem@davemloft.net \
--cc=dsahern@kernel.org \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=yoshfuji@linux-ipv6.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).