From: Peter Zijlstra <peterz@infradead.org>
To: longman@redhat.com, mingo@redhat.com, will@kernel.org
Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, boqun.feng@gmail.com
Subject: [PATCH 1/6] locking/rwsem: Minor code refactoring in rwsem_mark_wake()
Date: Thu, 23 Feb 2023 13:26:43 +0100 [thread overview]
Message-ID: <20230223123319.367721619@infradead.org> (raw)
In-Reply-To: 20230223122642.491637862@infradead.org
From: Waiman Long <longman@redhat.com>
Rename "oldcount" to "count" as it is not always old count value.
Also make some minor code refactoring to reduce indentation. There
is no functional change.
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20230216210933.1169097-2-longman@redhat.com
---
kernel/locking/rwsem.c | 44 ++++++++++++++++++++++----------------------
1 file changed, 22 insertions(+), 22 deletions(-)
--- a/kernel/locking/rwsem.c
+++ b/kernel/locking/rwsem.c
@@ -40,7 +40,7 @@
*
* When the rwsem is reader-owned and a spinning writer has timed out,
* the nonspinnable bit will be set to disable optimistic spinning.
-
+ *
* When a writer acquires a rwsem, it puts its task_struct pointer
* into the owner field. It is cleared after an unlock.
*
@@ -413,7 +413,7 @@ static void rwsem_mark_wake(struct rw_se
struct wake_q_head *wake_q)
{
struct rwsem_waiter *waiter, *tmp;
- long oldcount, woken = 0, adjustment = 0;
+ long count, woken = 0, adjustment = 0;
struct list_head wlist;
lockdep_assert_held(&sem->wait_lock);
@@ -424,22 +424,23 @@ static void rwsem_mark_wake(struct rw_se
*/
waiter = rwsem_first_waiter(sem);
- if (waiter->type == RWSEM_WAITING_FOR_WRITE) {
- if (wake_type == RWSEM_WAKE_ANY) {
- /*
- * Mark writer at the front of the queue for wakeup.
- * Until the task is actually later awoken later by
- * the caller, other writers are able to steal it.
- * Readers, on the other hand, will block as they
- * will notice the queued writer.
- */
- wake_q_add(wake_q, waiter->task);
- lockevent_inc(rwsem_wake_writer);
- }
+ if (waiter->type != RWSEM_WAITING_FOR_WRITE)
+ goto wake_readers;
- return;
+ if (wake_type == RWSEM_WAKE_ANY) {
+ /*
+ * Mark writer at the front of the queue for wakeup.
+ * Until the task is actually later awoken later by
+ * the caller, other writers are able to steal it.
+ * Readers, on the other hand, will block as they
+ * will notice the queued writer.
+ */
+ wake_q_add(wake_q, waiter->task);
+ lockevent_inc(rwsem_wake_writer);
}
+ return;
+wake_readers:
/*
* No reader wakeup if there are too many of them already.
*/
@@ -455,15 +456,15 @@ static void rwsem_mark_wake(struct rw_se
struct task_struct *owner;
adjustment = RWSEM_READER_BIAS;
- oldcount = atomic_long_fetch_add(adjustment, &sem->count);
- if (unlikely(oldcount & RWSEM_WRITER_MASK)) {
+ count = atomic_long_fetch_add(adjustment, &sem->count);
+ if (unlikely(count & RWSEM_WRITER_MASK)) {
/*
* When we've been waiting "too" long (for writers
* to give up the lock), request a HANDOFF to
* force the issue.
*/
if (time_after(jiffies, waiter->timeout)) {
- if (!(oldcount & RWSEM_FLAG_HANDOFF)) {
+ if (!(count & RWSEM_FLAG_HANDOFF)) {
adjustment -= RWSEM_FLAG_HANDOFF;
lockevent_inc(rwsem_rlock_handoff);
}
@@ -524,21 +525,21 @@ static void rwsem_mark_wake(struct rw_se
adjustment = woken * RWSEM_READER_BIAS - adjustment;
lockevent_cond_inc(rwsem_wake_reader, woken);
- oldcount = atomic_long_read(&sem->count);
+ count = atomic_long_read(&sem->count);
if (list_empty(&sem->wait_list)) {
/*
* Combined with list_move_tail() above, this implies
* rwsem_del_waiter().
*/
adjustment -= RWSEM_FLAG_WAITERS;
- if (oldcount & RWSEM_FLAG_HANDOFF)
+ if (count & RWSEM_FLAG_HANDOFF)
adjustment -= RWSEM_FLAG_HANDOFF;
} else if (woken) {
/*
* When we've woken a reader, we no longer need to force
* writers to give up the lock and we can clear HANDOFF.
*/
- if (oldcount & RWSEM_FLAG_HANDOFF)
+ if (count & RWSEM_FLAG_HANDOFF)
adjustment -= RWSEM_FLAG_HANDOFF;
}
@@ -844,7 +845,6 @@ static bool rwsem_optimistic_spin(struct
* Try to acquire the lock
*/
taken = rwsem_try_write_lock_unqueued(sem);
-
if (taken)
break;
next prev parent reply other threads:[~2023-02-23 12:34 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-23 12:26 [PATCH 0/6] locking/rwsem: Rework writer wakeup and handoff Peter Zijlstra
2023-02-23 12:26 ` Peter Zijlstra [this message]
2023-02-23 12:26 ` [PATCH 2/6] locking/rwsem: Enforce queueing when HANDOFF Peter Zijlstra
2023-02-23 12:26 ` [PATCH 3/6] locking/rwsem: Rework writer wakeup Peter Zijlstra
2023-02-23 21:38 ` Waiman Long
2023-02-26 11:58 ` Peter Zijlstra
2023-02-26 12:00 ` Peter Zijlstra
2023-02-26 21:31 ` Waiman Long
2023-02-26 11:59 ` Peter Zijlstra
2023-02-26 15:04 ` Peter Zijlstra
2023-02-26 16:51 ` Peter Zijlstra
2023-02-27 0:22 ` Waiman Long
2023-02-27 10:31 ` Peter Zijlstra
2023-02-27 20:16 ` Waiman Long
2023-03-20 8:12 ` Peter Zijlstra
2023-03-20 17:36 ` Waiman Long
2023-02-23 12:26 ` [PATCH 4/6] locking/rwsem: Split out rwsem_reader_wake() Peter Zijlstra
2023-02-23 12:26 ` [PATCH 5/6] locking/rwsem: Unify wait loop Peter Zijlstra
2023-02-23 19:31 ` Boqun Feng
2023-02-24 1:33 ` Boqun Feng
2023-02-26 12:01 ` Peter Zijlstra
2023-02-26 18:22 ` Boqun Feng
2023-02-23 22:45 ` Waiman Long
2023-02-26 16:15 ` Peter Zijlstra
2023-02-23 12:26 ` [PATCH 6/6] locking/rwsem: Use the force Peter Zijlstra
2023-02-24 1:19 ` [PATCH 0/6] locking/rwsem: Rework writer wakeup and handoff Waiman Long
2023-02-24 11:55 ` Jiri Wiesner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230223123319.367721619@infradead.org \
--to=peterz@infradead.org \
--cc=boqun.feng@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mingo@redhat.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).