All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Ahern <dsahern@gmail.com>
To: Dmitry Safonov <dima@arista.com>, linux-kernel@vger.kernel.org
Cc: Alexander Duyck <alexander.h.duyck@linux.intel.com>,
	Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
	Ido Schimmel <idosch@mellanox.com>,
	netdev@vger.kernel.org
Subject: Re: [RFC 4/4] net/ipv4/fib: Don't synchronise_rcu() every 512Kb
Date: Tue, 26 Mar 2019 09:39:31 -0600	[thread overview]
Message-ID: <2f911647-f35f-13c2-8177-2fb93147b0fa@gmail.com> (raw)
In-Reply-To: <20190326153026.24493-5-dima@arista.com>

On 3/26/19 9:30 AM, Dmitry Safonov wrote:
> Fib trie has a hard-coded sync_pages limit to call synchronise_rcu().
> The limit is 128 pages or 512Kb (considering common case with 4Kb
> pages).
> 
> Unfortunately, at Arista we have use-scenarios with full view software
> forwarding. At the scale of 100K and more routes even on 2 core boxes
> the hard-coded limit starts actively shooting in the leg: lockup
> detector notices that rtnl_lock is held for seconds.
> First reason is previously broken MAX_WORK, that didn't limit pending
> balancing work. While fixing it, I've noticed that the bottle-neck is
> actually in the number of synchronise_rcu() calls.
> 
> I've tried to fix it with a patch to decrement number of tnodes in rcu
> callback, but it hasn't much affected performance.
> 
> One possible way to "fix" it - provide another sysctl to control
> sync_pages, but in my POV it's nasty - exposing another realisation
> detail into user-space.

well, that was accepted last week. ;-)

commit 9ab948a91b2c2abc8e82845c0e61f4b1683e3a4f
Author: David Ahern <dsahern@gmail.com>
Date:   Wed Mar 20 09:18:59 2019 -0700

    ipv4: Allow amount of dirty memory from fib resizing to be controllable


Can you see how that change (should backport easily) affects your test
case? From my perspective 16MB was the sweet spot.

  reply	other threads:[~2019-03-26 15:39 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-03-26 15:30 [RFC 0/4] net/fib: Speed up trie rebalancing for full view Dmitry Safonov
2019-03-26 15:30 ` [RFC 1/4] net/ipv4/fib: Remove run-time check in tnode_alloc() Dmitry Safonov
2019-04-01 15:40   ` Alexander Duyck
2019-04-01 15:55     ` Dmitry Safonov
2019-04-01 17:50       ` Alexander Duyck
2019-04-04 16:33         ` Dmitry Safonov
2019-03-26 15:30 ` [RFC 2/4] net/fib: Provide fib_balance_budget sysctl Dmitry Safonov
2019-04-01 18:09   ` Alexander Duyck
2019-04-04 18:31     ` Dmitry Safonov
2019-03-26 15:30 ` [RFC 3/4] net/fib: Check budget before should_{inflate,halve}() Dmitry Safonov
2019-04-01 18:20   ` Alexander Duyck
2019-03-26 15:30 ` [RFC 4/4] net/ipv4/fib: Don't synchronise_rcu() every 512Kb Dmitry Safonov
2019-03-26 15:39   ` David Ahern [this message]
2019-03-26 17:15     ` Dmitry Safonov
2019-03-26 17:57       ` David Ahern
2019-03-26 18:17         ` Dmitry Safonov
2019-03-26 23:14     ` Dmitry Safonov
2019-03-27  3:33       ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=2f911647-f35f-13c2-8177-2fb93147b0fa@gmail.com \
    --to=dsahern@gmail.com \
    --cc=alexander.h.duyck@linux.intel.com \
    --cc=davem@davemloft.net \
    --cc=dima@arista.com \
    --cc=edumazet@google.com \
    --cc=idosch@mellanox.com \
    --cc=kuznet@ms2.inr.ac.ru \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=yoshfuji@linux-ipv6.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.