From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754286AbbFRV5a (ORCPT <rfc822;w@1wt.eu>);
	Thu, 18 Jun 2015 17:57:30 -0400
Received: from mail-ie0-f169.google.com ([209.85.223.169]:33214 "EHLO
	mail-ie0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751807AbbFRV5W (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Thu, 18 Jun 2015 17:57:22 -0400
MIME-Version: 1.0
In-Reply-To: <558223B2.5030204@codeaurora.org>
References: <CALAqxLVvPoawx47LoHg+ozRHeB2ZgtiG-YQ48XAK2rqWjZZQPw@mail.gmail.com>
	<20150617003154.GV3913@linux.vnet.ibm.com>
	<5580C239.4060707@codeaurora.org>
	<20150617044643.GW3913@linux.vnet.ibm.com>
	<558223B2.5030204@codeaurora.org>
Date: Thu, 18 Jun 2015 14:57:21 -0700
Message-ID: <CALAqxLXsVwSB0_hN7-WoFFUfN6Vah_yZxVyQBZ59Yc_Gz9rt+g@mail.gmail.com>
Subject: Re: Suspicious RCU usage at boot w/ arm ipi trace events?
From: John Stultz <john.stultz@linaro.org>
To: Stephen Boyd <sboyd@codeaurora.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>,
        Nicolas Pitre <nicolas.pitre@linaro.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        lkml <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jun 17, 2015 at 6:49 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> On 06/16/2015 09:46 PM, Paul E. McKenney wrote:
>> On Tue, Jun 16, 2015 at 05:41:29PM -0700, Stephen Boyd wrote:
>>>
>>> The tracepoint 'trace_ipi_entry' in handle_IPI()  is using RCU and we
>>> haven't called irq_enter() yet at the point. Does this tracepoint need
>>> to have _rcuidle() added to it?
>> Yes, I believe that would fix this problem.
>>
>
> Ok... here's the patch. I see the problem on my device and applying this
> patch fixes it.
>
> ----8<----
>
> From: Stephen Boyd <sboyd@codeaurora.org>
> Subject: [PATCH] ARM: smp: Silence suspicious RCU usage with ipi tracepoints
>
> John Stultz reports an RCU splat on boot with ARM ipi trace
> events enabled.
>
> ===============================
> [ INFO: suspicious RCU usage. ]
> 4.1.0-rc7-00033-gb5bed2f #153 Not tainted
> -------------------------------
> include/trace/events/ipi.h:68 suspicious rcu_dereference_check() usage!
>
> other info that might help us debug this:
>
> RCU used illegally from idle CPU!
> rcu_scheduler_active = 1, debug_locks = 0
> RCU used illegally from extended quiescent state!
> no locks held by swapper/0/0.
>
> stack backtrace:
> CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.1.0-rc7-00033-gb5bed2f #153
> Hardware name: Qualcomm (Flattened Device Tree)
> [<c0216b08>] (unwind_backtrace) from [<c02136e8>] (show_stack+0x10/0x14)
> [<c02136e8>] (show_stack) from [<c075e678>] (dump_stack+0x70/0xbc)
> [<c075e678>] (dump_stack) from [<c0215a80>] (handle_IPI+0x428/0x604)
> [<c0215a80>] (handle_IPI) from [<c020942c>] (gic_handle_irq+0x54/0x5c)
> [<c020942c>] (gic_handle_irq) from [<c0766604>] (__irq_svc+0x44/0x7c)
> Exception stack(0xc09f3f48 to 0xc09f3f90)
> 3f40:                   00000001 00000001 00000000 c09f73b8 c09f4528 c0a5de9c
> 3f60: c076b4f0 00000000 00000000 c09ef108 c0a5cec1 00000001 00000000 c09f3f90
> 3f80: c026bf60 c0210ab8 20000113 ffffffff
> [<c0766604>] (__irq_svc) from [<c0210ab8>] (arch_cpu_idle+0x20/0x3c)
> [<c0210ab8>] (arch_cpu_idle) from [<c02647f0>] (cpu_startup_entry+0x2c0/0x5dc)
> [<c02647f0>] (cpu_startup_entry) from [<c099bc1c>] (start_kernel+0x358/0x3c4)
> [<c099bc1c>] (start_kernel) from [<8020807c>] (0x8020807c)
>
> At this point in the IPI handling path we haven't called
> irq_enter() yet, so RCU doesn't know that we're about to exit
> idle and properly warns that we're using RCU from an idle CPU.
> Use trace_ipi_entry_rcuidle() instead of trace_ipi_entry() so
> that RCU is informed about our exit from idle.
>
> Reported-by: John Stultz <john.stultz@linaro.org>
> Fixes: 365ec7b17327 "ARM: add IPI tracepoints"
> Signed-off-by: Stephen Boyd <sboyd@codeaurora.org>
> ---
>  arch/arm/kernel/smp.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index 13a91d390832..03eb8a446dca 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -589,7 +589,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
>         struct pt_regs *old_regs = set_irq_regs(regs);
>
>         if ((unsigned)ipinr < NR_IPI) {
> -               trace_ipi_entry(ipi_types[ipinr]);
> +               trace_ipi_entry_rcuidle(ipi_types[ipinr]);
>                 __inc_irq_stat(cpu, ipi_irqs[ipinr]);
>         }
>
> @@ -648,7 +648,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
>         }
>
>         if ((unsigned)ipinr < NR_IPI)
> -               trace_ipi_exit(ipi_types[ipinr]);
> +               trace_ipi_exit_rcuidle(ipi_types[ipinr]);
>         set_irq_regs(old_regs);
>  }
>

Tested-by: John Stultz <john.stultz@linaro.org>

Thanks for fixing this up!
-john