* [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
@ 2017-05-18 13:38 Steven Rostedt
2017-05-18 13:48 ` Miroslav Benes
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Steven Rostedt @ 2017-05-18 13:38 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: LKML, Ingo Molnar, Thomas Gleixner
From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
As stack tracing now requires "rcu watching", force RCU to be watching when
recording a stack trace.
Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
Changes since v1:
My testing discovered that the stack trace can be called with
interrupts enabled, which is a no no to have when calling
rcu_irq_enter(). When interrupts are enabled, as with being in an
NMI, RCU will also be watching.
kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index fcc9a2d..34a98ba 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
int pc)
{
- __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
+ struct ring_buffer *buffer = tr->trace_buffer.buffer;
+
+ /*
+ * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
+ * Also, RCU is always enabled when interrupts are.
+ */
+ if (!irqs_disabled() || in_nmi()) {
+ __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+ return;
+ }
+
+ /*
+ * It is possible that a function is being traced in a
+ * location that RCU is not watching. A call to
+ * rcu_irq_enter() will make sure that it is, but there's
+ * a few internal rcu functions that could be traced
+ * where that wont work either. In those cases, we just
+ * do nothing.
+ */
+ if (unlikely(rcu_irq_enter_disabled()))
+ return;
+
+ rcu_irq_enter();
+ __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
+ rcu_irq_exit();
}
/**
--
2.9.3
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 13:38 [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace Steven Rostedt
@ 2017-05-18 13:48 ` Miroslav Benes
2017-05-18 14:04 ` Steven Rostedt
2017-05-18 14:50 ` Steven Rostedt
2017-05-18 15:47 ` Paul E. McKenney
2 siblings, 1 reply; 10+ messages in thread
From: Miroslav Benes @ 2017-05-18 13:48 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Paul E. McKenney, LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017, Steven Rostedt wrote:
>
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
>
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
>
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
>
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>
> Changes since v1:
>
> My testing discovered that the stack trace can be called with
> interrupts enabled, which is a no no to have when calling
> rcu_irq_enter(). When interrupts are enabled, as with being in an
> NMI, RCU will also be watching.
Would rcu_irq_enter_irqson() help then? This is what Petr used in a live
patching handler.
Your solution works too, of course. Just asking if I am not missing
something.
Thanks,
Miroslav
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 13:48 ` Miroslav Benes
@ 2017-05-18 14:04 ` Steven Rostedt
2017-05-18 18:18 ` Miroslav Benes
0 siblings, 1 reply; 10+ messages in thread
From: Steven Rostedt @ 2017-05-18 14:04 UTC (permalink / raw)
To: Miroslav Benes; +Cc: Paul E. McKenney, LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017 15:48:55 +0200 (CEST)
Miroslav Benes <mbenes@suse.cz> wrote:
> On Thu, 18 May 2017, Steven Rostedt wrote:
>
> >
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> >
> > As stack tracing now requires "rcu watching", force RCU to be watching when
> > recording a stack trace.
> >
> > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> >
> > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> >
> > Changes since v1:
> >
> > My testing discovered that the stack trace can be called with
> > interrupts enabled, which is a no no to have when calling
> > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > NMI, RCU will also be watching.
>
> Would rcu_irq_enter_irqson() help then? This is what Petr used in a live
> patching handler.
>
Yes, that could work too, but I wanted to avoid disabling interrupts if
we didn't have to.
> Your solution works too, of course. Just asking if I am not missing
> something.
>
Nope, I was just trying to keep the overhead down. As this can be
called by every event enabled, as well as functions being traced. I
figured that local_save_irqs() is faster than a pair of
local_irq_save()/ local_irq_restore() calls.
-- Steve
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 13:38 [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace Steven Rostedt
2017-05-18 13:48 ` Miroslav Benes
@ 2017-05-18 14:50 ` Steven Rostedt
2017-05-18 15:47 ` Paul E. McKenney
2 siblings, 0 replies; 10+ messages in thread
From: Steven Rostedt @ 2017-05-18 14:50 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017 09:38:09 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
>
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
>
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
>
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Paul,
Can you give me an ack for this version too.
Thanks!
-- Steve
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>
> Changes since v1:
>
> My testing discovered that the stack trace can be called with
> interrupts enabled, which is a no no to have when calling
> rcu_irq_enter(). When interrupts are enabled, as with being in an
> NMI, RCU will also be watching.
>
> kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index fcc9a2d..34a98ba 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> int pc)
> {
> - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> +
> + /*
> + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> + * Also, RCU is always enabled when interrupts are.
> + */
> + if (!irqs_disabled() || in_nmi()) {
> + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> + return;
> + }
> +
> + /*
> + * It is possible that a function is being traced in a
> + * location that RCU is not watching. A call to
> + * rcu_irq_enter() will make sure that it is, but there's
> + * a few internal rcu functions that could be traced
> + * where that wont work either. In those cases, we just
> + * do nothing.
> + */
> + if (unlikely(rcu_irq_enter_disabled()))
> + return;
> +
> + rcu_irq_enter();
> + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> + rcu_irq_exit();
> }
>
> /**
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 13:38 [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace Steven Rostedt
2017-05-18 13:48 ` Miroslav Benes
2017-05-18 14:50 ` Steven Rostedt
@ 2017-05-18 15:47 ` Paul E. McKenney
2017-05-18 16:39 ` Steven Rostedt
2 siblings, 1 reply; 10+ messages in thread
From: Paul E. McKenney @ 2017-05-18 15:47 UTC (permalink / raw)
To: Steven Rostedt; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
>
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
>
> As stack tracing now requires "rcu watching", force RCU to be watching when
> recording a stack trace.
>
> Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
>
> Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>
> Changes since v1:
>
> My testing discovered that the stack trace can be called with
> interrupts enabled, which is a no no to have when calling
> rcu_irq_enter(). When interrupts are enabled, as with being in an
> NMI, RCU will also be watching.
>
> kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index fcc9a2d..34a98ba 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> int pc)
> {
> - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> +
> + /*
> + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> + * Also, RCU is always enabled when interrupts are.
> + */
> + if (!irqs_disabled() || in_nmi()) {
You lost me on this one. RCU might not be watching if irqs are
enabled, for example, in the idle loop. What am I missing here?
Thanx, Paul
> + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> + return;
> + }
> +
> + /*
> + * It is possible that a function is being traced in a
> + * location that RCU is not watching. A call to
> + * rcu_irq_enter() will make sure that it is, but there's
> + * a few internal rcu functions that could be traced
> + * where that wont work either. In those cases, we just
> + * do nothing.
> + */
> + if (unlikely(rcu_irq_enter_disabled()))
> + return;
> +
> + rcu_irq_enter();
> + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> + rcu_irq_exit();
> }
>
> /**
> --
> 2.9.3
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 15:47 ` Paul E. McKenney
@ 2017-05-18 16:39 ` Steven Rostedt
2017-05-18 18:45 ` Paul E. McKenney
0 siblings, 1 reply; 10+ messages in thread
From: Steven Rostedt @ 2017-05-18 16:39 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017 08:47:11 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> >
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> >
> > As stack tracing now requires "rcu watching", force RCU to be watching when
> > recording a stack trace.
> >
> > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> >
> > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> >
> > Changes since v1:
> >
> > My testing discovered that the stack trace can be called with
> > interrupts enabled, which is a no no to have when calling
> > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > NMI, RCU will also be watching.
> >
> > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > 1 file changed, 25 insertions(+), 1 deletion(-)
> >
> > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > index fcc9a2d..34a98ba 100644
> > --- a/kernel/trace/trace.c
> > +++ b/kernel/trace/trace.c
> > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > int pc)
> > {
> > - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > +
> > + /*
> > + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > + * Also, RCU is always enabled when interrupts are.
> > + */
> > + if (!irqs_disabled() || in_nmi()) {
>
> You lost me on this one. RCU might not be watching if irqs are
> enabled, for example, in the idle loop. What am I missing here?
>
Hmm, no, maybe I'm missing something :-/
OK, so if we trace in the idle loop, rcu may not be watching, so I may
need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(
-- Steve
> Thanx, Paul
>
> > + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > + return;
> > + }
> > +
> > + /*
> > + * It is possible that a function is being traced in a
> > + * location that RCU is not watching. A call to
> > + * rcu_irq_enter() will make sure that it is, but there's
> > + * a few internal rcu functions that could be traced
> > + * where that wont work either. In those cases, we just
> > + * do nothing.
> > + */
> > + if (unlikely(rcu_irq_enter_disabled()))
> > + return;
> > +
> > + rcu_irq_enter();
> > + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > + rcu_irq_exit();
> > }
> >
> > /**
> > --
> > 2.9.3
> >
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 14:04 ` Steven Rostedt
@ 2017-05-18 18:18 ` Miroslav Benes
0 siblings, 0 replies; 10+ messages in thread
From: Miroslav Benes @ 2017-05-18 18:18 UTC (permalink / raw)
To: Steven Rostedt; +Cc: Paul E. McKenney, LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017, Steven Rostedt wrote:
> On Thu, 18 May 2017 15:48:55 +0200 (CEST)
> Miroslav Benes <mbenes@suse.cz> wrote:
>
> > On Thu, 18 May 2017, Steven Rostedt wrote:
> >
> > >
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > >
> > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > recording a stack trace.
> > >
> > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > >
> > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > >
> > > Changes since v1:
> > >
> > > My testing discovered that the stack trace can be called with
> > > interrupts enabled, which is a no no to have when calling
> > > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > NMI, RCU will also be watching.
> >
> > Would rcu_irq_enter_irqson() help then? This is what Petr used in a live
> > patching handler.
> >
>
> Yes, that could work too, but I wanted to avoid disabling interrupts if
> we didn't have to.
Ok, that makes sense.
> > Your solution works too, of course. Just asking if I am not missing
> > something.
> >
>
> Nope, I was just trying to keep the overhead down. As this can be
> called by every event enabled, as well as functions being traced. I
> figured that local_save_irqs() is faster than a pair of
> local_irq_save()/ local_irq_restore() calls.
(noticed Paul's reply)... yeah, it'd great. Damn, this is mindblowing.
Thanks,
Miroslav
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 16:39 ` Steven Rostedt
@ 2017-05-18 18:45 ` Paul E. McKenney
2017-05-18 19:30 ` Steven Rostedt
0 siblings, 1 reply; 10+ messages in thread
From: Paul E. McKenney @ 2017-05-18 18:45 UTC (permalink / raw)
To: Steven Rostedt; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> On Thu, 18 May 2017 08:47:11 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
>
> > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> > >
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > >
> > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > recording a stack trace.
> > >
> > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > >
> > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > >
> > > Changes since v1:
> > >
> > > My testing discovered that the stack trace can be called with
> > > interrupts enabled, which is a no no to have when calling
> > > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > NMI, RCU will also be watching.
> > >
> > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > > 1 file changed, 25 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > index fcc9a2d..34a98ba 100644
> > > --- a/kernel/trace/trace.c
> > > +++ b/kernel/trace/trace.c
> > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > > void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > > int pc)
> > > {
> > > - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > +
> > > + /*
> > > + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > + * Also, RCU is always enabled when interrupts are.
> > > + */
> > > + if (!irqs_disabled() || in_nmi()) {
> >
> > You lost me on this one. RCU might not be watching if irqs are
> > enabled, for example, in the idle loop. What am I missing here?
> >
>
> Hmm, no, maybe I'm missing something :-/
>
> OK, so if we trace in the idle loop, rcu may not be watching, so I may
> need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(
If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching()
returns true, you are golden.
Thanx, Paul
> -- Steve
>
>
> > Thanx, Paul
> >
> > > + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > > + return;
> > > + }
> > > +
> > > + /*
> > > + * It is possible that a function is being traced in a
> > > + * location that RCU is not watching. A call to
> > > + * rcu_irq_enter() will make sure that it is, but there's
> > > + * a few internal rcu functions that could be traced
> > > + * where that wont work either. In those cases, we just
> > > + * do nothing.
> > > + */
> > > + if (unlikely(rcu_irq_enter_disabled()))
> > > + return;
> > > +
> > > + rcu_irq_enter();
> > > + __ftrace_trace_stack(buffer, flags, skip, pc, NULL);
> > > + rcu_irq_exit();
> > > }
> > >
> > > /**
> > > --
> > > 2.9.3
> > >
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 18:45 ` Paul E. McKenney
@ 2017-05-18 19:30 ` Steven Rostedt
2017-05-18 19:50 ` Paul E. McKenney
0 siblings, 1 reply; 10+ messages in thread
From: Steven Rostedt @ 2017-05-18 19:30 UTC (permalink / raw)
To: Paul E. McKenney; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, 18 May 2017 11:45:28 -0700
"Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> > On Thu, 18 May 2017 08:47:11 -0700
> > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> >
> > > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> > > >
> > > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > >
> > > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > > recording a stack trace.
> > > >
> > > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > >
> > > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > > ---
> > > >
> > > > Changes since v1:
> > > >
> > > > My testing discovered that the stack trace can be called with
> > > > interrupts enabled, which is a no no to have when calling
> > > > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > > NMI, RCU will also be watching.
> > > >
> > > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > > > 1 file changed, 25 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > > index fcc9a2d..34a98ba 100644
> > > > --- a/kernel/trace/trace.c
> > > > +++ b/kernel/trace/trace.c
> > > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > > > void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > > > int pc)
> > > > {
> > > > - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > > + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > > +
> > > > + /*
> > > > + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > > + * Also, RCU is always enabled when interrupts are.
> > > > + */
> > > > + if (!irqs_disabled() || in_nmi()) {
> > >
> > > You lost me on this one. RCU might not be watching if irqs are
> > > enabled, for example, in the idle loop. What am I missing here?
> > >
> >
> > Hmm, no, maybe I'm missing something :-/
> >
> > OK, so if we trace in the idle loop, rcu may not be watching, so I may
> > need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(
>
> If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching()
> returns true, you are golden.
>
Hmm, so I can just replace the entire if with:
if (rcu_is_watching())
because it should be watching in NMI context, correct?
-- Steve
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace
2017-05-18 19:30 ` Steven Rostedt
@ 2017-05-18 19:50 ` Paul E. McKenney
0 siblings, 0 replies; 10+ messages in thread
From: Paul E. McKenney @ 2017-05-18 19:50 UTC (permalink / raw)
To: Steven Rostedt; +Cc: LKML, Ingo Molnar, Thomas Gleixner
On Thu, May 18, 2017 at 03:30:26PM -0400, Steven Rostedt wrote:
> On Thu, 18 May 2017 11:45:28 -0700
> "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
>
> > On Thu, May 18, 2017 at 12:39:14PM -0400, Steven Rostedt wrote:
> > > On Thu, 18 May 2017 08:47:11 -0700
> > > "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote:
> > >
> > > > On Thu, May 18, 2017 at 09:38:09AM -0400, Steven Rostedt wrote:
> > > > >
> > > > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > > >
> > > > > As stack tracing now requires "rcu watching", force RCU to be watching when
> > > > > recording a stack trace.
> > > > >
> > > > > Link: http://lkml.kernel.org/r/20170512172449.879684501@goodmis.org
> > > > >
> > > > > Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
> > > > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > > > ---
> > > > >
> > > > > Changes since v1:
> > > > >
> > > > > My testing discovered that the stack trace can be called with
> > > > > interrupts enabled, which is a no no to have when calling
> > > > > rcu_irq_enter(). When interrupts are enabled, as with being in an
> > > > > NMI, RCU will also be watching.
> > > > >
> > > > > kernel/trace/trace.c | 26 +++++++++++++++++++++++++-
> > > > > 1 file changed, 25 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> > > > > index fcc9a2d..34a98ba 100644
> > > > > --- a/kernel/trace/trace.c
> > > > > +++ b/kernel/trace/trace.c
> > > > > @@ -2568,7 +2568,31 @@ static inline void ftrace_trace_stack(struct trace_array *tr,
> > > > > void __trace_stack(struct trace_array *tr, unsigned long flags, int skip,
> > > > > int pc)
> > > > > {
> > > > > - __ftrace_trace_stack(tr->trace_buffer.buffer, flags, skip, pc, NULL);
> > > > > + struct ring_buffer *buffer = tr->trace_buffer.buffer;
> > > > > +
> > > > > + /*
> > > > > + * When an NMI triggers, RCU is enabled via rcu_nmi_enter()
> > > > > + * Also, RCU is always enabled when interrupts are.
> > > > > + */
> > > > > + if (!irqs_disabled() || in_nmi()) {
> > > >
> > > > You lost me on this one. RCU might not be watching if irqs are
> > > > enabled, for example, in the idle loop. What am I missing here?
> > > >
> > >
> > > Hmm, no, maybe I'm missing something :-/
> > >
> > > OK, so if we trace in the idle loop, rcu may not be watching, so I may
> > > need to byte the bullet and use the rcu_irq_enter_irqon() instead :-(
> >
> > If rcu_is_watching() returns false, yes. Of course, if rcu_is_watching()
> > returns true, you are golden.
> >
>
> Hmm, so I can just replace the entire if with:
>
> if (rcu_is_watching())
>
> because it should be watching in NMI context, correct?
Yes, and that would be even better! ;-)
However, this assumes that there is no possibility of this being called
during the time that in_nmi() returns true but RCU is not aware of the
NMI. (Yes, we did go through this a bit earlier, just my usual paranoia
showing...)
Thanx, Paul
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2017-05-18 19:51 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-18 13:38 [PATCH v2] tracing: Make sure RCU is watching before calling a stack trace Steven Rostedt
2017-05-18 13:48 ` Miroslav Benes
2017-05-18 14:04 ` Steven Rostedt
2017-05-18 18:18 ` Miroslav Benes
2017-05-18 14:50 ` Steven Rostedt
2017-05-18 15:47 ` Paul E. McKenney
2017-05-18 16:39 ` Steven Rostedt
2017-05-18 18:45 ` Paul E. McKenney
2017-05-18 19:30 ` Steven Rostedt
2017-05-18 19:50 ` Paul E. McKenney
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).