From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Joel Fernandes <joelaf@google.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
Joel Fernandes <joel@joelfernandes.org>,
LKML <linux-kernel@vger.kernel.org>,
"Cc: Android Kernel" <kernel-team@android.com>,
Boqun Feng <boqun.feng@gmail.com>,
Byungchul Park <byungchul.park@lge.com>,
Ingo Molnar <mingo@redhat.com>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Namhyung Kim <namhyung@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Thomas Glexiner <tglx@linutronix.de>,
Tom Zanussi <tom.zanussi@linux.intel.com>
Subject: Re: [PATCH v12 3/3] tracing: Centralize preemptirq tracepoints and unify their usage
Date: Wed, 8 Aug 2018 07:49:22 -0700 [thread overview]
Message-ID: <20180808144922.GN24813@linux.vnet.ibm.com> (raw)
In-Reply-To: <CAJWu+oopkDMe9AtuiRqAubuPk_H=NSifTDpWLdgCTEvR6zZiMw@mail.gmail.com>
On Wed, Aug 08, 2018 at 07:10:53AM -0700, Joel Fernandes wrote:
> On Wed, Aug 8, 2018 at 6:00 AM, Paul E. McKenney
> <paulmck@linux.vnet.ibm.com> wrote:
> > On Tue, Aug 07, 2018 at 08:53:54PM -0700, Joel Fernandes wrote:
> >> On Tue, Aug 7, 2018 at 8:44 PM, Joel Fernandes <joelaf@google.com> wrote:
> >> > Hi Steve,
> >> >
> >> > On Tue, Aug 7, 2018 at 7:28 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> >> [...]
> >> >>> @@ -171,8 +174,7 @@ extern void syscall_unregfunc(void);
> >> >>> } while ((++it_func_ptr)->func); \
> >> >>> } \
> >> >>> \
> >> >>> - if (rcuidle) \
> >> >>> - srcu_read_unlock_notrace(&tracepoint_srcu, idx);\
> >> >>> + srcu_read_unlock_notrace(ss, idx); \
> >> >>
> >> >> Hmm, why do we have the two different srcu handles?
> >> >
> >> > Because if the memory operations happening on the normal SRCU handle
> >> > (during srcu_read_lock) is interrupted by NMI, then the other handle
> >> > (devoted to NMI) could be used instead and not bother the interrupted
> >> > handle. Does that makes sense?
> >> >
> >> > When I talked to Paul few months ago about SRCU from NMI context, he
> >> > mentioned the per-cpu memory operations during srcu_read_lock can be
> >> > NMI interrupted, that's why we added that warning.
> >>
> >> So I looked more closely, __srcu_read_lock on 2 different handles may
> >> still be doing a this_cpu_inc on the same location..
> >> (sp->sda->srcu_lock_count). :-(
> >>
> >> Paul any ideas on how to solve this?
> >
> > You lost me on this one. When you said "2 different handles", I assumed
> > that you meant two different values of "sp", which would have two
> > different addresses for &sp->sda->srcu_lock_count. What am I missing?
>
> Thanks a lot for the reply.
> I thought "sda" is the same for different srcu_struct(s). May be it
> was too late for me in the night, that's why I thought so? Which makes
> no sense now that I think of it.
I know that feeling! ;-)
> In that case based on what you're saying, the patch I sent to using
> different srcu_struct for NMI is still good I guess...
As long as you wait for both SRCU grace periods. Hmmm... Maybe that means
that there is still a use for synchronize_rcu_mult():
void call_srcu_nmi(struct rcu_head *rhp, rcu_callback_t func)
{
call_srcu(&trace_srcu_struct_nmi, rhp, func);
}
void call_srcu_nonmi(struct rcu_head *rhp, rcu_callback_t func)
{
call_srcu(&trace_srcu_struct_nonmi, rhp, func);
}
...
/* Wait concurrently on the two grace periods. */
synchronize_rcu_mult(call_srcu_nmi, call_srcu_nonmi);
On the other hand, I bet that doing this is just fine in your use case:
synchronize_srcu(&trace_srcu_struct_nmi);
synchronize_srcu(&trace_srcu_struct_nonmi);
But please note that synchronize_rcu_mult() is no more in my -rcu tree,
so if you do want it please let me know (and please let me know why it
is important).
> >> It does start to seem like a show stopper :-(
> >
> > I suppose that an srcu_read_lock_nmi() and srcu_read_unlock_nmi() could
> > be added, which would do atomic ops on sp->sda->srcu_lock_count. Not sure
> > whether this would be fast enough to be useful, but easy to provide:
> >
> > int __srcu_read_lock_nmi(struct srcu_struct *sp) /* UNTESTED. */
> > {
> > int idx;
> >
> > idx = READ_ONCE(sp->srcu_idx) & 0x1;
> > atomic_inc(&sp->sda->srcu_lock_count[idx]);
> > smp_mb__after_atomic(); /* B */ /* Avoid leaking critical section. */
> > return idx;
> > }
> >
> > void __srcu_read_unlock_nmi(struct srcu_struct *sp, int idx)
> > {
> > smp_mb__before_atomic(); /* C */ /* Avoid leaking critical section. */
> > atomic_inc(&sp->sda->srcu_unlock_count[idx]);
> > }
> >
> > With appropriate adjustments to also allow Tiny RCU to also work.
> >
> > Note that you have to use _nmi() everywhere, not just in NMI handlers.
> > In fact, the NMI handlers are the one place you -don't- need to use
> > _nmi(), strangely enough.
> >
> > Might be worth a try -- smp_mb__{before,after}_atomic() is a no-op on
> > some architectures, for example.
>
> Continuing Steve's question on regular interrupts, do we need to use
> this atomic_inc API for regular interrupts as well?
If NMIs use one srcu_struct and non-NMI uses another, the current
srcu_read_lock() and srcu_read_unlock() will work just fine. If any given
srcu_struct needs both NMI and non-NMI readers, then we really do need
__srcu_read_lock_nmi() and __srcu_read_unlock_nmi() for that srcu_struct.
Thanx, Paul
next prev parent reply other threads:[~2018-08-08 14:49 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-30 22:24 [PATCH v12 0/3] tracing: Centralize preemptirq tracepoints and unify their usage Joel Fernandes
2018-07-30 22:24 ` [PATCH v12 1/3] lockdep: use this_cpu_ptr instead of get_cpu_var stats Joel Fernandes
2018-07-30 22:24 ` [PATCH v12 2/3] tracepoint: Make rcuidle tracepoint callers use SRCU Joel Fernandes
2018-07-30 23:10 ` Steven Rostedt
2018-08-10 15:35 ` Steven Rostedt
2018-08-10 16:32 ` Steven Rostedt
2018-07-30 22:24 ` [PATCH v12 3/3] tracing: Centralize preemptirq tracepoints and unify their usage Joel Fernandes
2018-08-06 19:50 ` Steven Rostedt
2018-08-07 0:43 ` Joel Fernandes
2018-08-07 1:43 ` Steven Rostedt
2018-08-07 13:33 ` Joel Fernandes
2018-08-07 13:49 ` Steven Rostedt
2018-08-07 14:10 ` Joel Fernandes
2018-08-07 14:34 ` Steven Rostedt
2018-08-07 14:48 ` Joel Fernandes
2018-08-07 15:09 ` Steven Rostedt
2018-08-07 15:24 ` Joel Fernandes
2018-08-07 23:45 ` Steven Rostedt
2018-08-07 23:54 ` Joel Fernandes
2018-08-08 0:48 ` Steven Rostedt
2018-08-08 1:17 ` Joel Fernandes
2018-08-08 1:55 ` Steven Rostedt
2018-08-08 2:13 ` Joel Fernandes
2018-08-08 2:28 ` Steven Rostedt
2018-08-08 3:44 ` Joel Fernandes
2018-08-08 3:53 ` Joel Fernandes
2018-08-08 5:06 ` Joel Fernandes
2018-08-08 12:46 ` Steven Rostedt
2018-08-08 13:03 ` Paul E. McKenney
2018-08-08 13:07 ` Steven Rostedt
2018-08-08 14:33 ` Paul E. McKenney
2018-08-08 14:49 ` Steven Rostedt
2018-08-08 15:05 ` Paul E. McKenney
2018-08-08 15:23 ` Steven Rostedt
2018-08-08 16:02 ` Paul E. McKenney
2018-08-08 16:24 ` Steven Rostedt
2018-08-08 17:21 ` Paul E. McKenney
2018-08-08 13:00 ` Paul E. McKenney
2018-08-08 14:10 ` Joel Fernandes
2018-08-08 14:49 ` Paul E. McKenney [this message]
2018-08-08 19:24 ` Joel Fernandes
2018-08-08 20:18 ` Paul E. McKenney
2018-08-08 22:15 ` Joel Fernandes
2018-08-08 22:47 ` Paul E. McKenney
2018-08-09 12:18 ` joel
2018-08-08 14:27 ` Steven Rostedt
2018-08-08 14:42 ` Paul E. McKenney
2018-08-08 15:27 ` Steven Rostedt
2018-08-08 16:03 ` Paul E. McKenney
2018-08-02 14:55 ` [PATCH v12 0/3] " Masami Hiramatsu
2018-08-03 2:57 ` Joel Fernandes
2018-08-03 7:23 ` Masami Hiramatsu
2018-08-04 4:51 ` Joel Fernandes
2018-08-05 16:46 ` Joel Fernandes
2018-08-06 2:07 ` Masami Hiramatsu
2018-08-06 15:24 ` Joel Fernandes
2018-08-03 7:34 ` Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180808144922.GN24813@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=boqun.feng@gmail.com \
--cc=byungchul.park@lge.com \
--cc=joel@joelfernandes.org \
--cc=joelaf@google.com \
--cc=kernel-team@android.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=tglx@linutronix.de \
--cc=tom.zanussi@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).