* [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu
@ 2021-06-28 15:49 Steven Rostedt
2021-06-28 15:58 ` Paul E. McKenney
2021-06-28 16:25 ` Daniel Bristot de Oliveira
0 siblings, 2 replies; 3+ messages in thread
From: Steven Rostedt @ 2021-06-28 15:49 UTC (permalink / raw)
To: LKML; +Cc: Daniel Bristot de Oliveira, Paul E. McKenney, Ingo Molnar
From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
ftracetest triggered:
INFO: rcu_tasks detected stalls on tasks:
00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7
task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000
Call Trace:
? asm_sysvec_apic_timer_interrupt+0x12/0x20
? asm_sysvec_apic_timer_interrupt+0x12/0x20
? trace_hardirqs_on+0x2b/0xe0
? asm_sysvec_apic_timer_interrupt+0x12/0x20
? trace_clock_local+0xc/0x20
? osnoise_main+0x10e/0x450
? trace_softirq_entry_callback+0x50/0x50
? kthread+0x153/0x170
? __kthread_bind_mask+0x60/0x60
? ret_from_fork+0x22/0x30
While running osnoise tracer with other tracers that rely on
synchronize_rcu_tasks(), where that just hung.
The reason is that osnoise_main() never schedules out if the interval
is less than 1, and this will cause synchronize_rcu_tasks() to never
return.
Fixes: bce29ac9ce0bb ("trace: Add osnoise tracer")
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
kernel/trace/trace_osnoise.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
index 38aa5e208ffd..556d530af805 100644
--- a/kernel/trace/trace_osnoise.c
+++ b/kernel/trace/trace_osnoise.c
@@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)
* differently from hwlat_detector, the osnoise tracer can run
* without a pause because preemption is on.
*/
- if (interval < 1)
+ if (interval < 1) {
+ /* Let synchronize_rcu_tasks() make progress */
+ cond_resched_tasks_rcu_qs();
continue;
+ }
if (msleep_interruptible(interval))
break;
--
2.29.2
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu
2021-06-28 15:49 [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu Steven Rostedt
@ 2021-06-28 15:58 ` Paul E. McKenney
2021-06-28 16:25 ` Daniel Bristot de Oliveira
1 sibling, 0 replies; 3+ messages in thread
From: Paul E. McKenney @ 2021-06-28 15:58 UTC (permalink / raw)
To: Steven Rostedt; +Cc: LKML, Daniel Bristot de Oliveira, Ingo Molnar
On Mon, Jun 28, 2021 at 11:49:53AM -0400, Steven Rostedt wrote:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
>
> ftracetest triggered:
>
> INFO: rcu_tasks detected stalls on tasks:
> 00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7
> task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000
> Call Trace:
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? trace_hardirqs_on+0x2b/0xe0
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? trace_clock_local+0xc/0x20
> ? osnoise_main+0x10e/0x450
> ? trace_softirq_entry_callback+0x50/0x50
> ? kthread+0x153/0x170
> ? __kthread_bind_mask+0x60/0x60
> ? ret_from_fork+0x22/0x30
>
> While running osnoise tracer with other tracers that rely on
> synchronize_rcu_tasks(), where that just hung.
>
> The reason is that osnoise_main() never schedules out if the interval
> is less than 1, and this will cause synchronize_rcu_tasks() to never
> return.
>
> Fixes: bce29ac9ce0bb ("trace: Add osnoise tracer")
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
That looks like something I would do!
Acked-by: Paul E. McKenney <paulmck@kernel.org>
> ---
> kernel/trace/trace_osnoise.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
> index 38aa5e208ffd..556d530af805 100644
> --- a/kernel/trace/trace_osnoise.c
> +++ b/kernel/trace/trace_osnoise.c
> @@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)
> * differently from hwlat_detector, the osnoise tracer can run
> * without a pause because preemption is on.
> */
> - if (interval < 1)
> + if (interval < 1) {
> + /* Let synchronize_rcu_tasks() make progress */
> + cond_resched_tasks_rcu_qs();
> continue;
> + }
>
> if (msleep_interruptible(interval))
> break;
> --
> 2.29.2
>
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu
2021-06-28 15:49 [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu Steven Rostedt
2021-06-28 15:58 ` Paul E. McKenney
@ 2021-06-28 16:25 ` Daniel Bristot de Oliveira
1 sibling, 0 replies; 3+ messages in thread
From: Daniel Bristot de Oliveira @ 2021-06-28 16:25 UTC (permalink / raw)
To: Steven Rostedt, LKML; +Cc: Paul E. McKenney, Ingo Molnar
On 6/28/21 5:49 PM, Steven Rostedt wrote:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
>
> ftracetest triggered:
>
> INFO: rcu_tasks detected stalls on tasks:
> 00000000b92b832d: .. nvcsw: 1/1 holdout: 1 idle_cpu: -1/7
> task:osnoise/7 state:R running task stack: 0 pid: 2133 ppid: 2 flags:0x00004000
> Call Trace:
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? trace_hardirqs_on+0x2b/0xe0
> ? asm_sysvec_apic_timer_interrupt+0x12/0x20
> ? trace_clock_local+0xc/0x20
> ? osnoise_main+0x10e/0x450
> ? trace_softirq_entry_callback+0x50/0x50
> ? kthread+0x153/0x170
> ? __kthread_bind_mask+0x60/0x60
> ? ret_from_fork+0x22/0x30
>
> While running osnoise tracer with other tracers that rely on
> synchronize_rcu_tasks(), where that just hung.
>
> The reason is that osnoise_main() never schedules out if the interval
> is less than 1,
and there is no other task setting need resched (it has a regular cond_resched()
on the run_osnoise() loop).
and this will cause synchronize_rcu_tasks() to never
> return.
>
> Fixes: bce29ac9ce0bb ("trace: Add osnoise tracer")
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Thanks!
-- Daniel
> ---
> kernel/trace/trace_osnoise.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace_osnoise.c b/kernel/trace/trace_osnoise.c
> index 38aa5e208ffd..556d530af805 100644
> --- a/kernel/trace/trace_osnoise.c
> +++ b/kernel/trace/trace_osnoise.c
> @@ -1216,8 +1216,11 @@ static int osnoise_main(void *data)
> * differently from hwlat_detector, the osnoise tracer can run
> * without a pause because preemption is on.
> */
> - if (interval < 1)
> + if (interval < 1) {
> + /* Let synchronize_rcu_tasks() make progress */
> + cond_resched_tasks_rcu_qs();
> continue;
> + }
>
> if (msleep_interruptible(interval))
> break;
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-06-28 16:25 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-28 15:49 [PATCH] tracing: Have osnoise_main() add a quiescent state for task rcu Steven Rostedt
2021-06-28 15:58 ` Paul E. McKenney
2021-06-28 16:25 ` Daniel Bristot de Oliveira
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.