From: Peter Zijlstra <peterz@infradead.org>
To: Waiman Long <longman@redhat.com>
Cc: Ingo Molnar <mingo@redhat.com>, Will Deacon <will.deacon@arm.com>,
Thomas Gleixner <tglx@linutronix.de>,
Borislav Petkov <bp@alien8.de>, "H. Peter Anvin" <hpa@zytor.com>,
linux-kernel@vger.kernel.org, x86@kernel.org,
Davidlohr Bueso <dave@stgolabs.net>,
Linus Torvalds <torvalds@linux-foundation.org>,
Tim Chen <tim.c.chen@linux.intel.com>,
huang ying <huang.ying.caritas@gmail.com>
Subject: Re: [PATCH-tip v7 09/20] locking/rwsem: Always release wait_lock before waking up tasks
Date: Fri, 3 May 2019 15:37:17 +0200 [thread overview]
Message-ID: <20190503133717.GG2623@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20190428212557.13482-10-longman@redhat.com>
On Sun, Apr 28, 2019 at 05:25:46PM -0400, Waiman Long wrote:
> + /*
> + * This waiter may have become first in the wait
> + * list after re-acquring the wait_lock. The
> + * rwsem_first_waiter() test in the main while
> + * loop below will correctly detect that. We do
> + * need to reload count to perform proper trylock
> + * and avoid missed wakeup.
> + */
> + count = atomic_long_read(&sem->count);
> + }
> } else {
> count = atomic_long_add_return(RWSEM_FLAG_WAITERS, &sem->count);
> }
I've been eyeing that count usage for the past few patches, and this
here makes me think we should get rid of it.
--- a/kernel/locking/rwsem.c
+++ b/kernel/locking/rwsem.c
@@ -400,13 +400,14 @@ static void __rwsem_mark_wake(struct rw_
* If wstate is WRITER_HANDOFF, it will make sure that either the handoff
* bit is set or the lock is acquired with handoff bit cleared.
*/
-static inline bool rwsem_try_write_lock(long count, struct rw_semaphore *sem,
+static inline bool rwsem_try_write_lock(struct rw_semaphore *sem,
enum writer_wait_state wstate)
{
- long new;
+ long count, new;
lockdep_assert_held(&sem->wait_lock);
+ count = atomic_long_read(&sem->count);
do {
bool has_handoff = !!(count & RWSEM_FLAG_HANDOFF);
@@ -760,25 +761,16 @@ rwsem_down_write_slowpath(struct rw_sema
wake_up_q(&wake_q);
wake_q_init(&wake_q); /* Used again, reinit */
raw_spin_lock_irq(&sem->wait_lock);
- /*
- * This waiter may have become first in the wait
- * list after re-acquring the wait_lock. The
- * rwsem_first_waiter() test in the main while
- * loop below will correctly detect that. We do
- * need to reload count to perform proper trylock
- * and avoid missed wakeup.
- */
- count = atomic_long_read(&sem->count);
}
} else {
- count = atomic_long_add_return(RWSEM_FLAG_WAITERS, &sem->count);
+ atomic_long_or(RWSEM_FLAG_WAITERS, &sem->count);
}
wait:
/* wait until we successfully acquire the lock */
set_current_state(state);
for (;;) {
- if (rwsem_try_write_lock(count, sem, wstate))
+ if (rwsem_try_write_lock(sem, wstate))
break;
raw_spin_unlock_irq(&sem->wait_lock);
@@ -819,7 +811,6 @@ rwsem_down_write_slowpath(struct rw_sema
}
raw_spin_lock_irq(&sem->wait_lock);
- count = atomic_long_read(&sem->count);
}
__set_current_state(TASK_RUNNING);
list_del(&waiter.list);
next prev parent reply other threads:[~2019-05-03 13:37 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-28 21:25 [PATCH-tip v7 00/20] locking/rwsem: Rwsem rearchitecture part 2 Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 01/20] locking/rwsem: Prevent decrement of reader count before increment Waiman Long
2019-05-03 12:06 ` Peter Zijlstra
2019-05-03 13:32 ` Waiman Long
2019-05-07 7:07 ` [tip:locking/urgent] " tip-bot for Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 02/20] locking/rwsem: Make owner available even if !CONFIG_RWSEM_SPIN_ON_OWNER Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 03/20] locking/rwsem: Remove rwsem_wake() wakeup optimization Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 04/20] locking/rwsem: Implement a new locking scheme Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 05/20] locking/rwsem: Merge rwsem.h and rwsem-xadd.c into rwsem.c Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 06/20] locking/rwsem: Code cleanup after files merging Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 07/20] locking/rwsem: Make rwsem_spin_on_owner() return owner state Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 08/20] locking/rwsem: Implement lock handoff to prevent lock starvation Waiman Long
2019-05-03 13:10 ` Peter Zijlstra
2019-05-03 13:57 ` Waiman Long
2019-05-03 14:37 ` David Laight
2019-04-28 21:25 ` [PATCH-tip v7 09/20] locking/rwsem: Always release wait_lock before waking up tasks Waiman Long
2019-05-03 13:37 ` Peter Zijlstra [this message]
2019-05-03 13:56 ` Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 10/20] locking/rwsem: More optimal RT task handling of null owner Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 11/20] locking/rwsem: Wake up almost all readers in wait queue Waiman Long
2019-05-03 16:51 ` Peter Zijlstra
2019-05-03 17:15 ` Waiman Long
2019-05-06 11:49 ` Peter Zijlstra
2019-04-28 21:25 ` [PATCH-tip v7 12/20] locking/rwsem: Clarify usage of owner's nonspinaable bit Waiman Long
2019-05-03 15:21 ` Peter Zijlstra
2019-05-03 15:26 ` Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 13/20] locking/rwsem: Enable readers spinning on writer Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 14/20] locking/rwsem: Enable time-based spinning on reader-owned rwsem Waiman Long
2019-05-06 15:47 ` Peter Zijlstra
2019-04-28 21:25 ` [PATCH-tip v7 15/20] locking/rwsem: Adaptive disabling of reader optimistic spinning Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 16/20] locking/rwsem: Add more rwsem owner access helpers Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 17/20] locking/rwsem: Guard against making count negative Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 18/20] locking/rwsem: Merge owner into count on x86-64 Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 19/20] locking/rwsem: Remove redundant computation of writer lock word Waiman Long
2019-04-28 21:25 ` [PATCH-tip v7 20/20] locking/rwsem: Disable preemption in down_read*() if owner in count Waiman Long
2019-04-28 22:46 ` [PATCH-tip v7 00/20] locking/rwsem: Rwsem rearchitecture part 2 Linus Torvalds
2019-04-28 23:12 ` Waiman Long
2019-04-28 23:19 ` Waiman Long
2019-04-29 0:10 ` Linus Torvalds
2019-04-29 0:27 ` Waiman Long
2019-04-29 2:41 ` Linus Torvalds
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190503133717.GG2623@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=bp@alien8.de \
--cc=dave@stgolabs.net \
--cc=hpa@zytor.com \
--cc=huang.ying.caritas@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mingo@redhat.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=torvalds@linux-foundation.org \
--cc=will.deacon@arm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).