From: Waiman Long <longman@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>, Will Deacon <will.deacon@arm.com>
Cc: linux-kernel@vger.kernel.org, Waiman Long <longman@redhat.com>
Subject: [PATCH v2 5/5] locking/lockdep: Call lock_release() after releasing the lock
Date: Tue, 2 Oct 2018 16:19:20 -0400 [thread overview]
Message-ID: <1538511560-10090-6-git-send-email-longman@redhat.com> (raw)
In-Reply-To: <1538511560-10090-1-git-send-email-longman@redhat.com>
Currently, lock_acquire() is called before acquiring the lock and
lock_release() is called before the releasing the lock. As a result,
the execution time of lock_release() is added to the lock hold time
reducing locking throughput, especially for spinlocks and rwlocks which
tend to have a much shorter lock hold time.
As lock_release() is not going to update any shared data that needs
protection from the lock, we don't actually need to call it before
releasing the lock. So the lock_release() calls are now postponed to
after releasing the lock for spinlocks and rwlocks.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/rwlock_api_smp.h | 16 ++++++++--------
include/linux/spinlock_api_smp.h | 8 ++++----
2 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/include/linux/rwlock_api_smp.h b/include/linux/rwlock_api_smp.h
index 86ebb4bf9c6e..b026940a0962 100644
--- a/include/linux/rwlock_api_smp.h
+++ b/include/linux/rwlock_api_smp.h
@@ -215,63 +215,63 @@ static inline void __raw_write_lock(rwlock_t *lock)
static inline void __raw_write_unlock(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_write_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
preempt_enable();
}
static inline void __raw_read_unlock(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_read_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
preempt_enable();
}
static inline void
__raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_read_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
local_irq_restore(flags);
preempt_enable();
}
static inline void __raw_read_unlock_irq(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_read_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
local_irq_enable();
preempt_enable();
}
static inline void __raw_read_unlock_bh(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_read_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
__local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
static inline void __raw_write_unlock_irqrestore(rwlock_t *lock,
unsigned long flags)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_write_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
local_irq_restore(flags);
preempt_enable();
}
static inline void __raw_write_unlock_irq(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_write_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
local_irq_enable();
preempt_enable();
}
static inline void __raw_write_unlock_bh(rwlock_t *lock)
{
- rwlock_release(&lock->dep_map, 1, _RET_IP_);
do_raw_write_unlock(lock);
+ rwlock_release(&lock->dep_map, 1, _RET_IP_);
__local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
diff --git a/include/linux/spinlock_api_smp.h b/include/linux/spinlock_api_smp.h
index 42dfab89e740..fcb84df0678b 100644
--- a/include/linux/spinlock_api_smp.h
+++ b/include/linux/spinlock_api_smp.h
@@ -147,32 +147,32 @@ static inline void __raw_spin_lock(raw_spinlock_t *lock)
static inline void __raw_spin_unlock(raw_spinlock_t *lock)
{
- spin_release(&lock->dep_map, 1, _RET_IP_);
do_raw_spin_unlock(lock);
+ spin_release(&lock->dep_map, 1, _RET_IP_);
preempt_enable();
}
static inline void __raw_spin_unlock_irqrestore(raw_spinlock_t *lock,
unsigned long flags)
{
- spin_release(&lock->dep_map, 1, _RET_IP_);
do_raw_spin_unlock(lock);
+ spin_release(&lock->dep_map, 1, _RET_IP_);
local_irq_restore(flags);
preempt_enable();
}
static inline void __raw_spin_unlock_irq(raw_spinlock_t *lock)
{
- spin_release(&lock->dep_map, 1, _RET_IP_);
do_raw_spin_unlock(lock);
+ spin_release(&lock->dep_map, 1, _RET_IP_);
local_irq_enable();
preempt_enable();
}
static inline void __raw_spin_unlock_bh(raw_spinlock_t *lock)
{
- spin_release(&lock->dep_map, 1, _RET_IP_);
do_raw_spin_unlock(lock);
+ spin_release(&lock->dep_map, 1, _RET_IP_);
__local_bh_enable_ip(_RET_IP_, SOFTIRQ_LOCK_OFFSET);
}
--
2.18.0
prev parent reply other threads:[~2018-10-02 20:20 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-02 20:19 [PATCH v2 0/5] locking/lockdep: Improve lockdep performance Waiman Long
2018-10-02 20:19 ` [PATCH v2 1/5] locking/lockdep: Remove add_chain_cache_classes() Waiman Long
2018-10-03 7:30 ` [tip:locking/core] " tip-bot for Waiman Long
2018-10-02 20:19 ` [PATCH v2 2/5] locking/lockdep: Eliminate redundant irqs check in __lock_acquire() Waiman Long
2018-10-03 7:31 ` [tip:locking/core] locking/lockdep: Eliminate redundant IRQs " tip-bot for Waiman Long
2018-10-02 20:19 ` [PATCH v2 3/5] locking/lockdep: Add a faster path in __lock_release() Waiman Long
2018-10-03 7:31 ` [tip:locking/core] " tip-bot for Waiman Long
2018-10-02 20:19 ` [PATCH v2 4/5] locking/lockdep: Make class->ops a percpu counter Waiman Long
2018-10-03 7:48 ` Peter Zijlstra
2018-10-03 7:54 ` Ingo Molnar
2018-10-03 8:24 ` Peter Zijlstra
2018-10-03 13:57 ` Waiman Long
2018-10-03 17:07 ` Waiman Long
2018-10-04 10:14 ` Ingo Molnar
2018-10-04 13:05 ` Waiman Long
2018-10-09 10:54 ` [tip:locking/core] locking/lockdep: Make class->ops a percpu counter and move it under CONFIG_DEBUG_LOCKDEP=y tip-bot for Waiman Long
2018-10-02 20:19 ` Waiman Long [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1538511560-10090-6-git-send-email-longman@redhat.com \
--to=longman@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).