bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/2] locking: Add new lock contention tracepoints (v3)
@ 2022-03-16 22:45 Namhyung Kim
  2022-03-16 22:45 ` [PATCH 1/2] locking: Add lock contention tracepoints Namhyung Kim
                   ` (2 more replies)
  0 siblings, 3 replies; 36+ messages in thread
From: Namhyung Kim @ 2022-03-16 22:45 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Will Deacon, Waiman Long, Boqun Feng
  Cc: LKML, Thomas Gleixner, Steven Rostedt, Byungchul Park,
	Paul E. McKenney, Mathieu Desnoyers, Arnd Bergmann,
	Radoslaw Burny, linux-arch, bpf

Hello,

There have been some requests for low-overhead kernel lock contention
monitoring.  The kernel has CONFIG_LOCK_STAT to provide such an infra
either via /proc/lock_stat or tracepoints directly.

However it's not light-weight and hard to be used in production.  So
I'm trying to add new tracepoints for lock contention and using them
as a base to build a new monitoring system.

* Changes in v3
 - move the tracepoints deeper in the slow path
 - remove the caller ip
 - don't use task state in the flags

* Changes in v2
 - do not use lockdep infrastructure
 - add flags argument to lock:contention_begin tracepoint

I added a flags argument in the contention_begin to classify locks in
question.  It can tell whether it's a spinlock, reader-writer lock or
a mutex.  With stacktrace, users can identify which lock is contended.

The patch 01 added the tracepoints and move the definition to the
mutex.c file so that it can see the tracepoints without lockdep.

The patch 02 actually installs the tracepoints in the locking code.
To minimize the overhead, they were added in the slow path of the code
separately.  As spinlocks are defined in the arch headers, I couldn't
handle them all.  I've just added it to generic queued spinlock and
rwlocks only.  Each arch can add the tracepoints later.

This series base on the current tip/locking/core and you get it from
'locking/tracepoint-v3' branch in my tree at:

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git


Thanks,
Namhyung


Namhyung Kim (2):
  locking: Add lock contention tracepoints
  locking: Apply contention tracepoints in the slow path

 include/trace/events/lock.h   | 54 +++++++++++++++++++++++++++++++++--
 kernel/locking/lockdep.c      |  1 -
 kernel/locking/mutex.c        |  6 ++++
 kernel/locking/percpu-rwsem.c |  3 ++
 kernel/locking/qrwlock.c      |  9 ++++++
 kernel/locking/qspinlock.c    |  5 ++++
 kernel/locking/rtmutex.c      | 11 +++++++
 kernel/locking/rwbase_rt.c    |  3 ++
 kernel/locking/rwsem.c        |  9 ++++++
 kernel/locking/semaphore.c    | 14 ++++++++-
 10 files changed, 110 insertions(+), 5 deletions(-)


base-commit: cd27ccfc727e99352321c0c75012ab9c5a90321e
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 36+ messages in thread
* [PATCH 0/2] locking: Add new lock contention tracepoints (v4)
@ 2022-03-22 18:57 Namhyung Kim
  2022-03-22 18:57 ` [PATCH 2/2] locking: Apply contention tracepoints in the slow path Namhyung Kim
  0 siblings, 1 reply; 36+ messages in thread
From: Namhyung Kim @ 2022-03-22 18:57 UTC (permalink / raw)
  To: Peter Zijlstra, Ingo Molnar, Will Deacon, Waiman Long, Boqun Feng
  Cc: LKML, Thomas Gleixner, Steven Rostedt, Byungchul Park,
	Paul E. McKenney, Mathieu Desnoyers, Arnd Bergmann,
	Radoslaw Burny, linux-arch, bpf

Hello,

There have been some requests for low-overhead kernel lock contention
monitoring.  The kernel has CONFIG_LOCK_STAT to provide such an infra
either via /proc/lock_stat or tracepoints directly.

However it's not light-weight and hard to be used in production.  So
I'm trying to add new tracepoints for lock contention and using them
as a base to build a new monitoring system.

* Changes in v4
 - use __print_flags in the TP_printk()
 - reworked __down_common for semaphore
 - add Tested-by from Hyeonggon Yoo
 
* Changes in v3
 - move the tracepoints deeper in the slow path
 - remove the caller ip
 - don't use task state in the flags
 - add 'ret' field to the contention end tracepoint

* Changes in v2
 - do not use lockdep infrastructure
 - add flags argument to lock:contention_begin tracepoint

I added a flags argument in the contention_begin to classify locks in
question.  It can tell whether it's a spinlock, reader-writer lock or
a mutex.  With stacktrace, users can identify which lock is contended.

The patch 01 added the tracepoints and move the definition to the
mutex.c file so that it can see the tracepoints without lockdep.

The patch 02 actually installs the tracepoints in the locking code.
To minimize the overhead, they were added in the slow path of the code
separately.  As spinlocks are defined in the arch headers, I couldn't
handle them all.  I've just added it to generic queued spinlock and
rwlocks only.  Each arch can add the tracepoints later.

This series base on the current tip/locking/core and you get it from
'locking/tracepoint-v4' branch in my tree at:

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git


Thanks,
Namhyung


Namhyung Kim (2):
  locking: Add lock contention tracepoints
  locking: Apply contention tracepoints in the slow path

 include/trace/events/lock.h   | 61 +++++++++++++++++++++++++++++++++--
 kernel/locking/lockdep.c      |  1 -
 kernel/locking/mutex.c        |  6 ++++
 kernel/locking/percpu-rwsem.c |  3 ++
 kernel/locking/qrwlock.c      |  9 ++++++
 kernel/locking/qspinlock.c    |  5 +++
 kernel/locking/rtmutex.c      | 11 +++++++
 kernel/locking/rwbase_rt.c    |  3 ++
 kernel/locking/rwsem.c        |  9 ++++++
 kernel/locking/semaphore.c    | 15 ++++++++-
 10 files changed, 118 insertions(+), 5 deletions(-)


base-commit: cd27ccfc727e99352321c0c75012ab9c5a90321e
-- 
2.35.1.894.gb6a874cedc-goog


^ permalink raw reply	[flat|nested] 36+ messages in thread

end of thread, other threads:[~2022-04-01  9:26 UTC | newest]

Thread overview: 36+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-16 22:45 [PATCH 0/2] locking: Add new lock contention tracepoints (v3) Namhyung Kim
2022-03-16 22:45 ` [PATCH 1/2] locking: Add lock contention tracepoints Namhyung Kim
2022-03-17  2:31   ` Steven Rostedt
2022-03-17 13:32   ` Mathieu Desnoyers
2022-03-17 16:07     ` Steven Rostedt
2022-03-17 16:37       ` Mathieu Desnoyers
2022-03-18 20:58         ` Namhyung Kim
2022-03-16 22:45 ` [PATCH 2/2] locking: Apply contention tracepoints in the slow path Namhyung Kim
2022-03-17 13:45   ` Mathieu Desnoyers
2022-03-17 16:10     ` Steven Rostedt
2022-03-17 16:43       ` Mathieu Desnoyers
2022-03-18 21:34     ` Namhyung Kim
2022-03-17 18:19   ` Hyeonggon Yoo
2022-03-18 21:43     ` Namhyung Kim
2022-03-18 12:55   ` Boqun Feng
2022-03-18 13:24     ` Hyeonggon Yoo
2022-03-18 13:28       ` Hyeonggon Yoo
2022-03-18 16:43     ` Peter Zijlstra
2022-03-18 21:55       ` Namhyung Kim
2022-03-18 22:07         ` Steven Rostedt
2022-03-19  0:11           ` Namhyung Kim
2022-03-22  5:31             ` Namhyung Kim
2022-03-22 12:59               ` Steven Rostedt
2022-03-22 16:39                 ` Namhyung Kim
2022-03-17 17:32 ` [PATCH 0/2] locking: Add new lock contention tracepoints (v3) Hyeonggon Yoo
2022-03-18 21:12   ` Namhyung Kim
2022-03-22 18:57 [PATCH 0/2] locking: Add new lock contention tracepoints (v4) Namhyung Kim
2022-03-22 18:57 ` [PATCH 2/2] locking: Apply contention tracepoints in the slow path Namhyung Kim
2022-03-28 11:29   ` Peter Zijlstra
2022-03-28 17:41     ` Namhyung Kim
2022-03-28 11:39   ` Peter Zijlstra
2022-03-28 17:48     ` Namhyung Kim
2022-03-30 11:08       ` Peter Zijlstra
2022-03-30 19:03         ` Namhyung Kim
2022-03-31 11:59           ` Peter Zijlstra
2022-04-01  6:26             ` Namhyung Kim
2022-04-01  9:25               ` Peter Zijlstra

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).