* [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
@ 2021-04-25 20:06 Waiman Long
2021-04-26 8:08 ` Peter Zijlstra
0 siblings, 1 reply; 7+ messages in thread
From: Waiman Long @ 2021-04-25 20:06 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng
Cc: linux-kernel, Borislav Petkov, Ali Saidi, Steve Capper, Waiman Long
Make the code more readable by replacing the atomic_cmpxchg_acquire()
by an equivalent atomic_try_cmpxchg_acquire() and change atomic_add()
to atomic_or().
For architectures that use qrwlock, I do not find one that has an
atomic_add() defined but not an atomic_or(). I guess it should be fine
by changing atomic_add() to atomic_or(). I add a comment to state that
we can change it back to atomic_add() if there is an architecture that
has a more performant atomic_add() than an atomic_or().
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/locking/qrwlock.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index b94f3831e963..a1fa0f636b07 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -60,18 +60,26 @@ EXPORT_SYMBOL(queued_read_lock_slowpath);
*/
void queued_write_lock_slowpath(struct qrwlock *lock)
{
- int cnts;
+ int cnts = 0;
/* Put the writer into the wait queue */
arch_spin_lock(&lock->wait_lock);
/* Try to acquire the lock directly if no reader is present */
if (!atomic_read(&lock->cnts) &&
- (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
+ atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
goto unlock;
- /* Set the waiting flag to notify readers that a writer is pending */
- atomic_add(_QW_WAITING, &lock->cnts);
+ /*
+ * Set the waiting flag to notify readers that a writer is pending
+ *
+ * As only one writer who is the wait_lock owner can set the waiting
+ * flag which will be cleared later on when acquiring the write lock,
+ * we can easily replace atomic_or() by an atomic_add() if there is
+ * an architecture where an atomic_add() performs better than an
+ * atomic_or().
+ */
+ atomic_or(_QW_WAITING, &lock->cnts);
/* When no more readers or writers, set the locked flag */
do {
--
2.18.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
2021-04-25 20:06 [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup Waiman Long
@ 2021-04-26 8:08 ` Peter Zijlstra
2021-04-26 14:54 ` Waiman Long
0 siblings, 1 reply; 7+ messages in thread
From: Peter Zijlstra @ 2021-04-26 8:08 UTC (permalink / raw)
To: Waiman Long
Cc: Ingo Molnar, Will Deacon, Boqun Feng, linux-kernel,
Borislav Petkov, Ali Saidi, Steve Capper
On Sun, Apr 25, 2021 at 04:06:37PM -0400, Waiman Long wrote:
> void queued_write_lock_slowpath(struct qrwlock *lock)
> {
> - int cnts;
> + int cnts = 0;
>
> /* Put the writer into the wait queue */
> arch_spin_lock(&lock->wait_lock);
>
> /* Try to acquire the lock directly if no reader is present */
> if (!atomic_read(&lock->cnts) &&
> - (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
> + atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
> goto unlock;
Would not something like:
if (!(cnts = atomic_read(&lock->cnts)) &&
atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED)
goto unlock;
Be clearer?
>
> - /* Set the waiting flag to notify readers that a writer is pending */
> - atomic_add(_QW_WAITING, &lock->cnts);
> + /*
> + * Set the waiting flag to notify readers that a writer is pending
> + *
> + * As only one writer who is the wait_lock owner can set the waiting
> + * flag which will be cleared later on when acquiring the write lock,
> + * we can easily replace atomic_or() by an atomic_add() if there is
> + * an architecture where an atomic_add() performs better than an
> + * atomic_or().
That might be a little overboard on the comment, but sure :-) I don't
think there's any arch that doesn't have atomic_or(), like I wrote
elsewhere, the one that's often an issue is atomic_fetch_or().
> + */
> + atomic_or(_QW_WAITING, &lock->cnts);
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
2021-04-26 8:08 ` Peter Zijlstra
@ 2021-04-26 14:54 ` Waiman Long
0 siblings, 0 replies; 7+ messages in thread
From: Waiman Long @ 2021-04-26 14:54 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Ingo Molnar, Will Deacon, Boqun Feng, linux-kernel,
Borislav Petkov, Ali Saidi, Steve Capper
On 4/26/21 4:08 AM, Peter Zijlstra wrote:
> On Sun, Apr 25, 2021 at 04:06:37PM -0400, Waiman Long wrote:
>
>> void queued_write_lock_slowpath(struct qrwlock *lock)
>> {
>> - int cnts;
>> + int cnts = 0;
>>
>> /* Put the writer into the wait queue */
>> arch_spin_lock(&lock->wait_lock);
>>
>> /* Try to acquire the lock directly if no reader is present */
>> if (!atomic_read(&lock->cnts) &&
>> - (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
>> + atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
>> goto unlock;
> Would not something like:
>
> if (!(cnts = atomic_read(&lock->cnts)) &&
> atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED)
> goto unlock;
>
> Be clearer?
That works for me too. It is equivalent anyway.
>
>>
>> - /* Set the waiting flag to notify readers that a writer is pending */
>> - atomic_add(_QW_WAITING, &lock->cnts);
>> + /*
>> + * Set the waiting flag to notify readers that a writer is pending
>> + *
>> + * As only one writer who is the wait_lock owner can set the waiting
>> + * flag which will be cleared later on when acquiring the write lock,
>> + * we can easily replace atomic_or() by an atomic_add() if there is
>> + * an architecture where an atomic_add() performs better than an
>> + * atomic_or().
> That might be a little overboard on the comment, but sure :-) I don't
> think there's any arch that doesn't have atomic_or(), like I wrote
> elsewhere, the one that's often an issue is atomic_fetch_or().
>
I was not sure as I didn't look at other archs that hadn't used qrwlock
yet. Given what you said, I will remove the comment.
Cheers,
Longman
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
2021-04-27 7:56 ` Will Deacon
@ 2021-05-04 9:19 ` Peter Zijlstra
0 siblings, 0 replies; 7+ messages in thread
From: Peter Zijlstra @ 2021-05-04 9:19 UTC (permalink / raw)
To: Will Deacon
Cc: Waiman Long, Ingo Molnar, Boqun Feng, linux-kernel,
Borislav Petkov, Ali Saidi, Steve Capper
On Tue, Apr 27, 2021 at 08:56:59AM +0100, Will Deacon wrote:
> On Mon, Apr 26, 2021 at 02:50:17PM -0400, Waiman Long wrote:
> > Make the code more readable by replacing the atomic_cmpxchg_acquire()
> > by an equivalent atomic_try_cmpxchg_acquire() and change atomic_add()
> > to atomic_or().
> >
> > For architectures that use qrwlock, I do not find one that has an
> > atomic_add() defined but not an atomic_or(). I guess it should be fine
> > by changing atomic_add() to atomic_or().
> >
> > Note that the previous use of atomic_add() isn't wrong as only one
> > writer that is the wait_lock owner can set the waiting flag and the
> > flag will be cleared later on when acquiring the write lock.
>
> Right, there's no functional change here.
>
> > Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> > Signed-off-by: Waiman Long <longman@redhat.com>
> Acked-by: Will Deacon <will@kernel.org>
Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
2021-04-26 18:50 Waiman Long
2021-04-26 18:51 ` Waiman Long
@ 2021-04-27 7:56 ` Will Deacon
2021-05-04 9:19 ` Peter Zijlstra
1 sibling, 1 reply; 7+ messages in thread
From: Will Deacon @ 2021-04-27 7:56 UTC (permalink / raw)
To: Waiman Long
Cc: Peter Zijlstra, Ingo Molnar, Boqun Feng, linux-kernel,
Borislav Petkov, Ali Saidi, Steve Capper
On Mon, Apr 26, 2021 at 02:50:17PM -0400, Waiman Long wrote:
> Make the code more readable by replacing the atomic_cmpxchg_acquire()
> by an equivalent atomic_try_cmpxchg_acquire() and change atomic_add()
> to atomic_or().
>
> For architectures that use qrwlock, I do not find one that has an
> atomic_add() defined but not an atomic_or(). I guess it should be fine
> by changing atomic_add() to atomic_or().
>
> Note that the previous use of atomic_add() isn't wrong as only one
> writer that is the wait_lock owner can set the waiting flag and the
> flag will be cleared later on when acquiring the write lock.
Right, there's no functional change here.
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
> kernel/locking/qrwlock.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
> index b94f3831e963..ec36b73f4733 100644
> --- a/kernel/locking/qrwlock.c
> +++ b/kernel/locking/qrwlock.c
> @@ -66,12 +66,12 @@ void queued_write_lock_slowpath(struct qrwlock *lock)
> arch_spin_lock(&lock->wait_lock);
>
> /* Try to acquire the lock directly if no reader is present */
> - if (!atomic_read(&lock->cnts) &&
> - (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
> + if (!(cnts = atomic_read(&lock->cnts)) &&
> + atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
> goto unlock;
>
> /* Set the waiting flag to notify readers that a writer is pending */
> - atomic_add(_QW_WAITING, &lock->cnts);
> + atomic_or(_QW_WAITING, &lock->cnts);
Acked-by: Will Deacon <will@kernel.org>
Will
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
2021-04-26 18:50 Waiman Long
@ 2021-04-26 18:51 ` Waiman Long
2021-04-27 7:56 ` Will Deacon
1 sibling, 0 replies; 7+ messages in thread
From: Waiman Long @ 2021-04-26 18:51 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng
Cc: linux-kernel, Borislav Petkov, Ali Saidi, Steve Capper
On 4/26/21 2:50 PM, Waiman Long wrote:
> Make the code more readable by replacing the atomic_cmpxchg_acquire()
> by an equivalent atomic_try_cmpxchg_acquire() and change atomic_add()
> to atomic_or().
>
> For architectures that use qrwlock, I do not find one that has an
> atomic_add() defined but not an atomic_or(). I guess it should be fine
> by changing atomic_add() to atomic_or().
>
> Note that the previous use of atomic_add() isn't wrong as only one
> writer that is the wait_lock owner can set the waiting flag and the
> flag will be cleared later on when acquiring the write lock.
>
> Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: Waiman Long <longman@redhat.com>
> ---
> kernel/locking/qrwlock.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
> index b94f3831e963..ec36b73f4733 100644
> --- a/kernel/locking/qrwlock.c
> +++ b/kernel/locking/qrwlock.c
> @@ -66,12 +66,12 @@ void queued_write_lock_slowpath(struct qrwlock *lock)
> arch_spin_lock(&lock->wait_lock);
>
> /* Try to acquire the lock directly if no reader is present */
> - if (!atomic_read(&lock->cnts) &&
> - (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
> + if (!(cnts = atomic_read(&lock->cnts)) &&
> + atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
> goto unlock;
>
> /* Set the waiting flag to notify readers that a writer is pending */
> - atomic_add(_QW_WAITING, &lock->cnts);
> + atomic_or(_QW_WAITING, &lock->cnts);
>
> /* When no more readers or writers, set the locked flag */
> do {
Sorry, I missed the "v2" in the title.
Cheers,
Longman
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup
@ 2021-04-26 18:50 Waiman Long
2021-04-26 18:51 ` Waiman Long
2021-04-27 7:56 ` Will Deacon
0 siblings, 2 replies; 7+ messages in thread
From: Waiman Long @ 2021-04-26 18:50 UTC (permalink / raw)
To: Peter Zijlstra, Ingo Molnar, Will Deacon, Boqun Feng
Cc: linux-kernel, Borislav Petkov, Ali Saidi, Steve Capper, Waiman Long
Make the code more readable by replacing the atomic_cmpxchg_acquire()
by an equivalent atomic_try_cmpxchg_acquire() and change atomic_add()
to atomic_or().
For architectures that use qrwlock, I do not find one that has an
atomic_add() defined but not an atomic_or(). I guess it should be fine
by changing atomic_add() to atomic_or().
Note that the previous use of atomic_add() isn't wrong as only one
writer that is the wait_lock owner can set the waiting flag and the
flag will be cleared later on when acquiring the write lock.
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/locking/qrwlock.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index b94f3831e963..ec36b73f4733 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -66,12 +66,12 @@ void queued_write_lock_slowpath(struct qrwlock *lock)
arch_spin_lock(&lock->wait_lock);
/* Try to acquire the lock directly if no reader is present */
- if (!atomic_read(&lock->cnts) &&
- (atomic_cmpxchg_acquire(&lock->cnts, 0, _QW_LOCKED) == 0))
+ if (!(cnts = atomic_read(&lock->cnts)) &&
+ atomic_try_cmpxchg_acquire(&lock->cnts, &cnts, _QW_LOCKED))
goto unlock;
/* Set the waiting flag to notify readers that a writer is pending */
- atomic_add(_QW_WAITING, &lock->cnts);
+ atomic_or(_QW_WAITING, &lock->cnts);
/* When no more readers or writers, set the locked flag */
do {
--
2.18.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-05-04 9:20 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-25 20:06 [PATCH] locking/qrwlock: queued_write_lock_slowpath() cleanup Waiman Long
2021-04-26 8:08 ` Peter Zijlstra
2021-04-26 14:54 ` Waiman Long
2021-04-26 18:50 Waiman Long
2021-04-26 18:51 ` Waiman Long
2021-04-27 7:56 ` Will Deacon
2021-05-04 9:19 ` Peter Zijlstra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).