From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1755593AbcJPNjY (ORCPT <rfc822;w@1wt.eu>);
        Sun, 16 Oct 2016 09:39:24 -0400
Received: from bombadil.infradead.org ([198.137.202.9]:34984 "EHLO
        bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1751756AbcJPNjQ (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Sun, 16 Oct 2016 09:39:16 -0400
Date: Sun, 16 Oct 2016 15:39:11 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Wanpeng Li <kernellwp@gmail.com>
Cc: linux-kernel@vger.kernel.org, Wanpeng Li <wanpeng.li@hotmail.com>,
        Ingo Molnar <mingo@kernel.org>, Mike Galbraith <efault@gmx.de>,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH] x86/smp: Add irq_enter/exit() in
 smp_reschedule_interrupt()
Message-ID: <20161016133911.GI3142@twins.programming.kicks-ass.net>
References: <1476409733-5133-1-git-send-email-wanpeng.li@hotmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1476409733-5133-1-git-send-email-wanpeng.li@hotmail.com>
User-Agent: Mutt/1.5.23.1 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Oct 14, 2016 at 09:48:53AM +0800, Wanpeng Li wrote:
>  ===============================
>  [ INFO: suspicious RCU usage. ]
>  4.8.0+ #24 Not tainted
>  -------------------------------
>  ./arch/x86/include/asm/msr-trace.h:47 suspicious rcu_dereference_check() usage!
>  
>  other info that might help us debug this:
>  
>  
>  RCU used illegally from idle CPU!
>  rcu_scheduler_active = 1, debug_locks = 0
>  RCU used illegally from extended quiescent state!
>  no locks held by swapper/1/0.
>  
>  stack backtrace:
>  CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.8.0+ #24
>  Hardware name: Dell Inc. OptiPlex 7020/0F5C5X, BIOS A03 01/08/2015
>   ffff90285de03f58 ffffffff9d44a0c9 ffff90285ca5d100 0000000000000001
>   ffff90285de03f88 ffffffff9d0ebd67 ffff902845165410 000000000000080b
>   0000000000000000 0000000000000000 ffff90285de03fb8 ffffffff9d492b95
>  Call Trace:
>   <IRQ>  [<ffffffff9d44a0c9>] dump_stack+0x99/0xd0
>   [<ffffffff9d0ebd67>] lockdep_rcu_suspicious+0xe7/0x120
>   [<ffffffff9d492b95>] do_trace_write_msr+0x135/0x140
>   [<ffffffff9d06f860>] native_write_msr+0x20/0x30
>   [<ffffffff9d065fad>] native_apic_msr_eoi_write+0x1d/0x30
>   [<ffffffff9d05bd1d>] smp_reschedule_interrupt+0x1d/0x30
>   [<ffffffff9d8daec6>] reschedule_interrupt+0x96/0xa0
>   <EOI>  [<ffffffff9d732634>] ? cpuidle_enter_state+0xe4/0x360
>   [<ffffffff9d73261f>] ? cpuidle_enter_state+0xcf/0x360
>   [<ffffffff9d7328e7>] cpuidle_enter+0x17/0x20
>   [<ffffffff9d0e1a73>] call_cpuidle+0x23/0x50
>   [<ffffffff9d0e1d0c>] cpu_startup_entry+0x15c/0x280
>   [<ffffffff9d05ce64>] start_secondary+0x154/0x180
> 
> Reschedule interrupt may be called in cpu idle state. This causes lockdep 
> check warning above. 
> 
> Add irq_enter/exit() in smp_reschedule_interrupt(), irq_enter() tells the RCU 
> subsystems to end the extended quiescent state, so the following trace call in 
> ack_APIC_irq() works correctly.
> 
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Mike Galbraith <efault@gmx.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Wanpeng Li <wanpeng.li@hotmail.com>
> ---
>  arch/x86/kernel/smp.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c
> index 658777c..ac2ee87 100644
> --- a/arch/x86/kernel/smp.c
> +++ b/arch/x86/kernel/smp.c
> @@ -259,8 +259,10 @@ static inline void __smp_reschedule_interrupt(void)
>  
>  __visible void smp_reschedule_interrupt(struct pt_regs *regs)
>  {
> +	irq_enter();
>  	ack_APIC_irq();
>  	__smp_reschedule_interrupt();
> +	irq_exit();

Urgh, I really hate this...

So now we're making a very frequent interrupt slower because of debug
code :/