From: Uladzislau Rezki <urezki@gmail.com>
To: "Joel Fernandes (Google)" <joel@joelfernandes.org>
Cc: linux-kernel@vger.kernel.org, kernel-team@android.com,
kernel-team@lge.com, Byungchul Park <byungchul.park@lge.com>,
Davidlohr Bueso <dave@stgolabs.net>,
Josh Triplett <josh@joshtriplett.org>,
Lai Jiangshan <jiangshanlai@gmail.com>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
max.byungchul.park@gmail.com,
"Paul E. McKenney" <paulmck@linux.ibm.com>,
Rao Shoaib <rao.shoaib@oracle.com>,
rcu@vger.kernel.org, Steven Rostedt <rostedt@goodmis.org>
Subject: Re: [PATCH v4 1/2] rcu/tree: Add basic support for kfree_rcu() batching
Date: Wed, 18 Sep 2019 11:58:11 +0200 [thread overview]
Message-ID: <20190918095811.GA25821@pc636> (raw)
In-Reply-To: <20190814160411.58591-1-joel@joelfernandes.org>
> Recently a discussion about stability and performance of a system
> involving a high rate of kfree_rcu() calls surfaced on the list [1]
> which led to another discussion how to prepare for this situation.
>
> This patch adds basic batching support for kfree_rcu(). It is "basic"
> because we do none of the slab management, dynamic allocation, code
> moving or any of the other things, some of which previous attempts did
> [2]. These fancier improvements can be follow-up patches and there are
> different ideas being discussed in those regards. This is an effort to
> start simple, and build up from there. In the future, an extension to
> use kfree_bulk and possibly per-slab batching could be done to further
> improve performance due to cache-locality and slab-specific bulk free
> optimizations. By using an array of pointers, the worker thread
> processing the work would need to read lesser data since it does not
> need to deal with large rcu_head(s) any longer.
>
> Torture tests follow in the next patch and show improvements of around
> 5x reduction in number of grace periods on a 16 CPU system. More
> details and test data are in that patch.
>
> There is an implication with rcu_barrier() with this patch. Since the
> kfree_rcu() calls can be batched, and may not be handed yet to the RCU
> machinery in fact, the monitor may not have even run yet to do the
> queue_rcu_work(), there seems no easy way of implementing rcu_barrier()
> to wait for those kfree_rcu()s that are already made. So this means a
> kfree_rcu() followed by an rcu_barrier() does not imply that memory will
> be freed once rcu_barrier() returns.
>
> Another implication is higher active memory usage (although not
> run-away..) until the kfree_rcu() flooding ends, in comparison to
> without batching. More details about this are in the second patch which
> adds an rcuperf test.
>
> Finally, in the near future we will get rid of kfree_rcu() special casing
> within RCU such as in rcu_do_batch and switch everything to just
> batching. Currently we don't do that since timer subsystem is not yet up
> and we cannot schedule the kfree_rcu() monitor as the timer subsystem's
> lock are not initialized. That would also mean getting rid of
> kfree_call_rcu_nobatch() entirely.
>
Hello, Joel.
First of all thank you for improving it. I also noticed a high pressure
on RCU-machinery during performing some vmalloc tests when kfree_rcu()
flood occurred. Therefore i got rid of using kfree_rcu() there.
I have just a small question related to workloads and performance evaluation.
Are you aware of any specific workloads which benefit from it for example
mobile area, etc? I am asking because i think about backporting of it and
reuse it on our kernel.
Thank you!
--
Vlad Rezki
next prev parent reply other threads:[~2019-09-18 9:58 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-14 16:04 [PATCH v4 1/2] rcu/tree: Add basic support for kfree_rcu() batching Joel Fernandes (Google)
2019-08-14 16:04 ` [PATCH v4 2/2] rcuperf: Add kfree_rcu() performance Tests Joel Fernandes (Google)
2019-08-14 22:58 ` Paul E. McKenney
2019-08-19 19:33 ` Joel Fernandes
2019-08-19 22:23 ` Paul E. McKenney
2019-08-19 23:51 ` Joel Fernandes
2019-08-20 2:50 ` Paul E. McKenney
2019-08-21 0:27 ` Joel Fernandes
2019-08-21 0:31 ` Joel Fernandes
2019-08-21 0:44 ` Paul E. McKenney
2019-08-21 0:51 ` Joel Fernandes
2019-08-16 16:43 ` [PATCH v4 1/2] rcu/tree: Add basic support for kfree_rcu() batching Paul E. McKenney
2019-08-16 17:44 ` Joel Fernandes
2019-08-16 19:16 ` Paul E. McKenney
2019-08-17 1:32 ` Joel Fernandes
2019-08-17 3:56 ` Paul E. McKenney
2019-08-17 4:30 ` Joel Fernandes
2019-08-17 5:20 ` Paul E. McKenney
2019-08-17 5:53 ` Joel Fernandes
2019-08-17 21:45 ` Paul E. McKenney
2019-09-18 9:58 ` Uladzislau Rezki [this message]
2019-09-30 20:16 ` Joel Fernandes
2019-10-01 11:27 ` Uladzislau Rezki
2019-10-04 17:20 ` Joel Fernandes
2019-10-08 16:23 ` Uladzislau Rezki
2019-12-10 9:53 ` Uladzislau Rezki
2019-12-11 23:46 ` Paul E. McKenney
2019-12-16 12:06 ` Uladzislau Rezki
2019-12-12 5:27 ` Joel Fernandes
2019-12-16 12:46 ` Uladzislau Rezki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190918095811.GA25821@pc636 \
--to=urezki@gmail.com \
--cc=byungchul.park@lge.com \
--cc=dave@stgolabs.net \
--cc=jiangshanlai@gmail.com \
--cc=joel@joelfernandes.org \
--cc=josh@joshtriplett.org \
--cc=kernel-team@android.com \
--cc=kernel-team@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=max.byungchul.park@gmail.com \
--cc=paulmck@linux.ibm.com \
--cc=rao.shoaib@oracle.com \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).