From: Peter Zijlstra <peterz@infradead.org>
To: Namhyung Kim <namhyung@kernel.org>
Cc: Ingo Molnar <mingo@kernel.org>, Will Deacon <will@kernel.org>,
Waiman Long <longman@redhat.com>,
Boqun Feng <boqun.feng@gmail.com>,
LKML <linux-kernel@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Steven Rostedt <rostedt@goodmis.org>,
Byungchul Park <byungchul.park@lge.com>,
"Paul E. McKenney" <paulmck@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Arnd Bergmann <arnd@arndb.de>, Radoslaw Burny <rburny@google.com>,
linux-arch@vger.kernel.org, bpf@vger.kernel.org,
Hyeonggon Yoo <42.hyeyoo@gmail.com>
Subject: Re: [PATCH 2/2] locking: Apply contention tracepoints in the slow path
Date: Mon, 28 Mar 2022 13:29:21 +0200 [thread overview]
Message-ID: <20220328112921.GZ8939@worktop.programming.kicks-ass.net> (raw)
In-Reply-To: <20220322185709.141236-3-namhyung@kernel.org>
On Tue, Mar 22, 2022 at 11:57:09AM -0700, Namhyung Kim wrote:
> Adding the lock contention tracepoints in various lock function slow
> paths. Note that each arch can define spinlock differently, I only
> added it only to the generic qspinlock for now.
>
> Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
> Signed-off-by: Namhyung Kim <namhyung@kernel.org>
> ---
> kernel/locking/mutex.c | 3 +++
> kernel/locking/percpu-rwsem.c | 3 +++
> kernel/locking/qrwlock.c | 9 +++++++++
> kernel/locking/qspinlock.c | 5 +++++
> kernel/locking/rtmutex.c | 11 +++++++++++
> kernel/locking/rwbase_rt.c | 3 +++
> kernel/locking/rwsem.c | 9 +++++++++
> kernel/locking/semaphore.c | 15 ++++++++++++++-
> 8 files changed, 57 insertions(+), 1 deletion(-)
I had conflicts in rwsem.c due to Waiman's patches, but that was simple
enough to resolve. However, I had a good look at the other sites and
ended up with the below...
Yes, I know I'm the one that suggested the percpu thing, but upon
looking again it missed the largest part of percpu_down_write(), which
very much includes that RCU grace period and waiting for the readers to
bugger off
Also, rwbase_rt was missing the entire READ side -- yes, I see that's
also covered by the rtmuex.c part, but that's on a different address and
with different flags, and it's very confusing to not have it annotated.
Anyway, I'll queue this patch with the below folded in for post -rc1.
---
--- a/kernel/locking/percpu-rwsem.c
+++ b/kernel/locking/percpu-rwsem.c
@@ -155,7 +155,6 @@ static void percpu_rwsem_wait(struct per
}
spin_unlock_irq(&sem->waiters.lock);
- trace_contention_begin(sem, LCB_F_PERCPU | (reader ? LCB_F_READ : LCB_F_WRITE));
while (wait) {
set_current_state(TASK_UNINTERRUPTIBLE);
if (!smp_load_acquire(&wq_entry.private))
@@ -163,7 +162,6 @@ static void percpu_rwsem_wait(struct per
schedule();
}
__set_current_state(TASK_RUNNING);
- trace_contention_end(sem, 0);
}
bool __sched __percpu_down_read(struct percpu_rw_semaphore *sem, bool try)
@@ -174,9 +172,11 @@ bool __sched __percpu_down_read(struct p
if (try)
return false;
+ trace_contention_begin(sem, LCB_F_PERCPU | LCB_F_READ);
preempt_enable();
percpu_rwsem_wait(sem, /* .reader = */ true);
preempt_disable();
+ trace_contention_end(sem, 0);
return true;
}
@@ -219,6 +219,7 @@ void __sched percpu_down_write(struct pe
{
might_sleep();
rwsem_acquire(&sem->dep_map, 0, 0, _RET_IP_);
+ trace_contention_begin(sem, LCB_F_PERCPU | LCB_F_WRITE);
/* Notify readers to take the slow path. */
rcu_sync_enter(&sem->rss);
@@ -240,6 +241,7 @@ void __sched percpu_down_write(struct pe
/* Wait for all active readers to complete. */
rcuwait_wait_event(&sem->writer, readers_active_check(sem), TASK_UNINTERRUPTIBLE);
+ trace_contention_end(sem, 0);
}
EXPORT_SYMBOL_GPL(percpu_down_write);
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -35,7 +35,7 @@ void queued_read_lock_slowpath(struct qr
}
atomic_sub(_QR_BIAS, &lock->cnts);
- trace_contention_begin(lock, LCB_F_READ | LCB_F_SPIN);
+ trace_contention_begin(lock, LCB_F_SPIN | LCB_F_READ);
/*
* Put the reader into the wait queue
@@ -67,7 +67,7 @@ void queued_write_lock_slowpath(struct q
{
int cnts;
- trace_contention_begin(lock, LCB_F_WRITE | LCB_F_SPIN);
+ trace_contention_begin(lock, LCB_F_SPIN | LCB_F_WRITE);
/* Put the writer into the wait queue */
arch_spin_lock(&lock->wait_lock);
--- a/kernel/locking/rwbase_rt.c
+++ b/kernel/locking/rwbase_rt.c
@@ -112,6 +112,8 @@ static int __sched __rwbase_read_lock(st
* Reader2 to call up_read(), which might be unbound.
*/
+ trace_contention_begin(rwb, LCB_F_RT | LCB_F_READ);
+
/*
* For rwlocks this returns 0 unconditionally, so the below
* !ret conditionals are optimized out.
@@ -130,6 +132,8 @@ static int __sched __rwbase_read_lock(st
raw_spin_unlock_irq(&rtm->wait_lock);
if (!ret)
rwbase_rtmutex_unlock(rtm);
+
+ trace_contention_end(rwb, ret);
return ret;
}
@@ -247,7 +251,7 @@ static int __sched rwbase_write_lock(str
goto out_unlock;
rwbase_set_and_save_current_state(state);
- trace_contention_begin(rwb, LCB_F_WRITE | LCB_F_RT);
+ trace_contention_begin(rwb, LCB_F_RT | LCB_F_WRITE);
for (;;) {
/* Optimized out for rwlocks */
if (rwbase_signal_pending_state(state, current)) {
next prev parent reply other threads:[~2022-03-28 11:38 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-22 18:57 [PATCH 0/2] locking: Add new lock contention tracepoints (v4) Namhyung Kim
2022-03-22 18:57 ` [PATCH 1/2] locking: Add lock contention tracepoints Namhyung Kim
2022-03-22 18:57 ` [PATCH 2/2] locking: Apply contention tracepoints in the slow path Namhyung Kim
2022-03-28 11:29 ` Peter Zijlstra [this message]
2022-03-28 17:41 ` Namhyung Kim
2022-03-28 11:39 ` Peter Zijlstra
2022-03-28 17:48 ` Namhyung Kim
2022-03-30 11:08 ` Peter Zijlstra
2022-03-30 19:03 ` Namhyung Kim
2022-03-31 11:59 ` Peter Zijlstra
2022-04-01 6:26 ` Namhyung Kim
2022-04-01 9:25 ` Peter Zijlstra
-- strict thread matches above, loose matches on Subject: below --
2022-03-16 22:45 [PATCH 0/2] locking: Add new lock contention tracepoints (v3) Namhyung Kim
2022-03-16 22:45 ` [PATCH 2/2] locking: Apply contention tracepoints in the slow path Namhyung Kim
2022-03-17 13:45 ` Mathieu Desnoyers
2022-03-17 16:10 ` Steven Rostedt
2022-03-17 16:43 ` Mathieu Desnoyers
2022-03-18 21:34 ` Namhyung Kim
2022-03-17 18:19 ` Hyeonggon Yoo
2022-03-18 21:43 ` Namhyung Kim
2022-03-18 12:55 ` Boqun Feng
2022-03-18 13:24 ` Hyeonggon Yoo
2022-03-18 13:28 ` Hyeonggon Yoo
2022-03-18 16:43 ` Peter Zijlstra
2022-03-18 21:55 ` Namhyung Kim
2022-03-18 22:07 ` Steven Rostedt
2022-03-19 0:11 ` Namhyung Kim
2022-03-22 5:31 ` Namhyung Kim
2022-03-22 12:59 ` Steven Rostedt
2022-03-22 16:39 ` Namhyung Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220328112921.GZ8939@worktop.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=42.hyeyoo@gmail.com \
--cc=arnd@arndb.de \
--cc=boqun.feng@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=byungchul.park@lge.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mathieu.desnoyers@efficios.com \
--cc=mingo@kernel.org \
--cc=namhyung@kernel.org \
--cc=paulmck@kernel.org \
--cc=rburny@google.com \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).