All of lore.kernel.org
 help / color / mirror / Atom feed
From: Boqun Feng <boqun.feng@gmail.com>
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: linux-kernel@vger.kernel.org, rcu@vger.kernel.org,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Frederic Weisbecker <frederic@kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	Joel Fernandes <joel@joelfernandes.org>,
	John Ogness <john.ogness@linutronix.de>,
	Josh Triplett <josh@joshtriplett.org>,
	Lai Jiangshan <jiangshanlai@gmail.com>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	Neeraj Upadhyay <quic_neeraju@quicinc.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Waiman Long <longman@redhat.com>, Will Deacon <will@kernel.org>,
	Zqiang <qiang.zhang1211@gmail.com>
Subject: Re: [RFC PATCH] srcu: Use try-lock lockdep annotation for NMI-safe access.
Date: Wed, 27 Sep 2023 23:06:09 -0700	[thread overview]
Message-ID: <ZRUX0YUrXfepRGKE@Boquns-Mac-mini.home> (raw)
In-Reply-To: <20230927160231.XRCDDSK4@linutronix.de>

On Wed, Sep 27, 2023 at 06:02:31PM +0200, Sebastian Andrzej Siewior wrote:
> It is claimed that srcu_read_lock_nmisafe() NMI-safe. However it
> triggers a lockdep if used from NMI because lockdep expects a deadlock
> since nothing disables NMIs while the lock is acquired.
> 
> Use a try-lock annotation for srcu_read_lock_nmisafe() to avoid lockdep
> complains if used from NMI.
> 
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> ---
> 
> The splat:
> | ================================
> | WARNING: inconsistent lock state
> | 6.6.0-rc3-rt5+ #85 Not tainted
> | --------------------------------
> | inconsistent {INITIAL USE} -> {IN-NMI} usage.
> | swapper/0/0 [HC1[1]:SC0[0]:HE0:SE1] takes:
> | ffffffff828e6c90 (console_srcu){....}-{0:0}, at: console_srcu_read_lock+0x3a/0x50
> | {INITIAL USE} state was registered at:
> …
> |        CPU0
> |        ----
> |   lock(console_srcu);
> |   <Interrupt>
> |     lock(console_srcu);
> |
> |  *** DEADLOCK ***
> |
> 
> My guess is that trylock annotation should not apply to
> rcu_lock_acquire(). This would distinguish it from from non-NMI safe
> srcu_read_lock_nmisafe() and NMI check in rcu_read_unlock() is only
> there to survive if accidentally used in-NMI.

I think this is a "side-effect" of commit f0f44752f5f6 ("rcu: Annotate
SRCU's update-side lockdep dependencies"). In verify_lock_unused(), i.e.
the checking for NMI lock usages, the logic is that

1)	read lock usages in NMI conflicts with write lock usage in
	normal context (i.e. LOCKF_USED)

2)	write lock usage in NMI conflicts with read and write lock usage
	in normal context (i.e. LOCKF_USED | LOCKF_USED_READ)

before that commit, only read-side of SRCU is annotated, in other words,
SRCU only has read lock usage from lockdep PoV, but after that commit,
we annotate synchronize_srcu() as a write lock usage, so that we can
detect deadlocks between *normal* srcu_read_lock() and
synchronize_srcu(), however the side effect is now SRCU has a write lock
usage from lockdep PoV.

Actually in the above commit, I explicitly leave
srcu_read_lock_nmisafe() alone since its locking rules may be different
compared to srcu_read_lock(). In lockdep terms, srcu_read_lock_nmisafe()
is a !check read lock and srcu_read_lock() is a check read lock. Maybe
instead of using the trylock trick, we change lockdep to igore !check
locks for NMI context detection? Untested code as below:

diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index e85b5ad3e206..1af8d44e5eb4 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -5727,8 +5727,9 @@ void lock_acquire(struct lockdep_map *lock, unsigned int subclass,
                return;

        if (unlikely(!lockdep_enabled())) {
+               /* Only do NMI context checking if it's a check lock */
                /* XXX allow trylock from NMI ?!? */
-               if (lockdep_nmi() && !trylock) {
+               if (check && lockdep_nmi() && !trylock) {
                        struct held_lock hlock;

                        hlock.acquire_ip = ip;

Peter, thoughts?

Of course, either way, we need

Fixes: f0f44752f5f6 ("rcu: Annotate SRCU's update-side lockdep dependencies")

Regards,
Boqun

> 
>  include/linux/rcupdate.h | 6 ++++++
>  include/linux/srcu.h     | 2 +-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
> index 5e5f920ade909..44aab5c0bd2c1 100644
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -303,6 +303,11 @@ static inline void rcu_lock_acquire(struct lockdep_map *map)
>  	lock_acquire(map, 0, 0, 2, 0, NULL, _THIS_IP_);
>  }
>  
> +static inline void rcu_try_lock_acquire(struct lockdep_map *map)
> +{
> +	lock_acquire(map, 0, 1, 2, 0, NULL, _THIS_IP_);
> +}
> +
>  static inline void rcu_lock_release(struct lockdep_map *map)
>  {
>  	lock_release(map, _THIS_IP_);
> @@ -317,6 +322,7 @@ int rcu_read_lock_any_held(void);
>  #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */
>  
>  # define rcu_lock_acquire(a)		do { } while (0)
> +# define rcu_try_lock_acquire(a)	do { } while (0)
>  # define rcu_lock_release(a)		do { } while (0)
>  
>  static inline int rcu_read_lock_held(void)
> diff --git a/include/linux/srcu.h b/include/linux/srcu.h
> index 127ef3b2e6073..236610e4a8fa5 100644
> --- a/include/linux/srcu.h
> +++ b/include/linux/srcu.h
> @@ -229,7 +229,7 @@ static inline int srcu_read_lock_nmisafe(struct srcu_struct *ssp) __acquires(ssp
>  
>  	srcu_check_nmi_safety(ssp, true);
>  	retval = __srcu_read_lock_nmisafe(ssp);
> -	rcu_lock_acquire(&ssp->dep_map);
> +	rcu_try_lock_acquire(&ssp->dep_map);
>  	return retval;
>  }
>  
> -- 
> 2.40.1
> 

  reply	other threads:[~2023-09-28  6:06 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-27 16:02 [RFC PATCH] srcu: Use try-lock lockdep annotation for NMI-safe access Sebastian Andrzej Siewior
2023-09-28  6:06 ` Boqun Feng [this message]
2023-09-28  6:33   ` Sebastian Andrzej Siewior
2023-09-28  8:05   ` Peter Zijlstra
2023-09-28  8:09   ` Peter Zijlstra
2023-09-28 14:54     ` Boqun Feng
2023-09-28 15:29       ` Peter Zijlstra
2023-09-28 17:09         ` Boqun Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZRUX0YUrXfepRGKE@Boquns-Mac-mini.home \
    --to=boqun.feng@gmail.com \
    --cc=bigeasy@linutronix.de \
    --cc=frederic@kernel.org \
    --cc=jiangshanlai@gmail.com \
    --cc=joel@joelfernandes.org \
    --cc=john.ogness@linutronix.de \
    --cc=josh@joshtriplett.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mingo@redhat.com \
    --cc=paulmck@kernel.org \
    --cc=peterz@infradead.org \
    --cc=qiang.zhang1211@gmail.com \
    --cc=quic_neeraju@quicinc.com \
    --cc=rcu@vger.kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.