From: "Joel Fernandes (Google)" <joel@joelfernandes.org>
To: rcu@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, rushikesh.s.kadam@intel.com,
urezki@gmail.com, neeraj.iitr10@gmail.com, frederic@kernel.org,
paulmck@kernel.org, rostedt@goodmis.org, youssefesmat@google.com,
surenb@google.com,
"Joel Fernandes (Google)" <joel@joelfernandes.org>
Subject: [PATCH v8 06/13] percpu-refcount: Use call_rcu_flush() for atomic switch
Date: Tue, 11 Oct 2022 18:01:35 +0000 [thread overview]
Message-ID: <20221011180142.2742289-7-joel@joelfernandes.org> (raw)
In-Reply-To: <20221011180142.2742289-1-joel@joelfernandes.org>
call_rcu() changes to save power will slow down the percpu refcounter's
"per-CPU to atomic switch" path. The primitive uses RCU when switching to
atomic mode. The enqueued async callback wakes up waiters waiting in the
percpu_ref_switch_waitq. Due to this, per-CPU refcount users will slow down,
such as blk_pre_runtime_suspend().
Use the call_rcu_flush() API instead which reverts to the old behavior.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
---
lib/percpu-refcount.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/percpu-refcount.c b/lib/percpu-refcount.c
index e5c5315da274..65c58a029297 100644
--- a/lib/percpu-refcount.c
+++ b/lib/percpu-refcount.c
@@ -230,7 +230,8 @@ static void __percpu_ref_switch_to_atomic(struct percpu_ref *ref,
percpu_ref_noop_confirm_switch;
percpu_ref_get(ref); /* put after confirmation */
- call_rcu(&ref->data->rcu, percpu_ref_switch_to_atomic_rcu);
+ call_rcu_flush(&ref->data->rcu,
+ percpu_ref_switch_to_atomic_rcu);
}
static void __percpu_ref_switch_to_percpu(struct percpu_ref *ref)
--
2.38.0.rc1.362.ged0d419d3c-goog
next prev parent reply other threads:[~2022-10-11 18:02 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-11 18:01 [PATCH v8 00/13] rcu: call_rcu() power improvements Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 01/13] rcu: Fix missing nocb gp wake on rcu_barrier() Joel Fernandes (Google)
2022-10-14 14:21 ` Paul E. McKenney
2022-10-14 14:40 ` Frederic Weisbecker
2022-10-14 15:03 ` Paul E. McKenney
2022-10-14 15:19 ` Joel Fernandes
2022-10-14 15:46 ` Paul E. McKenney
2022-10-14 20:47 ` Frederic Weisbecker
2022-10-16 15:16 ` Paul E. McKenney
2022-10-16 15:56 ` Joel Fernandes
2022-10-11 18:01 ` [PATCH v8 02/13] rcu: Make call_rcu() lazy to save power Joel Fernandes (Google)
2022-10-11 23:10 ` Frederic Weisbecker
2022-10-12 19:54 ` Joel Fernandes
2022-10-14 15:45 ` Paul E. McKenney
2022-10-11 18:01 ` [PATCH v8 03/13] rcu: Refactor code a bit in rcu_nocb_do_flush_bypass() Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 04/13] rcu: shrinker for lazy rcu Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 05/13] rcuscale: Add laziness and kfree tests Joel Fernandes (Google)
2022-10-11 18:01 ` Joel Fernandes (Google) [this message]
2022-10-11 18:01 ` [PATCH v8 07/13] rcu/sync: Use call_rcu_flush() instead of call_rcu Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 08/13] rcu/rcuscale: Use call_rcu_flush() for async reader test Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 09/13] rcu/rcutorture: Use call_rcu_flush() where needed Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 10/13] scsi/scsi_error: Use call_rcu_flush() instead of call_rcu() Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 11/13] workqueue: Make queue_rcu_work() use call_rcu_flush() Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 12/13] rxrpc: Use call_rcu_flush() instead of call_rcu() Joel Fernandes (Google)
2022-10-11 18:01 ` [PATCH v8 13/13] rcu/debug: Add wake-up debugging for lazy callbacks Joel Fernandes (Google)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221011180142.2742289-7-joel@joelfernandes.org \
--to=joel@joelfernandes.org \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=neeraj.iitr10@gmail.com \
--cc=paulmck@kernel.org \
--cc=rcu@vger.kernel.org \
--cc=rostedt@goodmis.org \
--cc=rushikesh.s.kadam@intel.com \
--cc=surenb@google.com \
--cc=urezki@gmail.com \
--cc=youssefesmat@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).