From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63B62C433B4 for ; Fri, 23 Apr 2021 10:53:37 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B975161459 for ; Fri, 23 Apr 2021 10:53:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B975161459 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dOVIGi0L05lev9AJCfanl/MFJRr2dlws+3P81lcNwyw=; b=dO5lW4/sqF25oN4g+EjeEdb8h Uyk1ICFNs41kXmrsqFt1IV5pTBXhsicsyGpClY+OsXvdnYm8YTW2qfoVX9cmKTYUy4RQjYjccylCG f78m0zvlwgFHQwRjK5YZ/ZJ/ybgI38WfHVPmr7d7vTTf8d6ZSTqOYbDXs85U5YkASB+dSJWaUaWJC mPE8ODq8c/IjKPbX+aNArOPd2gDR60qqi5oxV/QlqqnN5udS8Jk0lHgG8RGGx/iMILeNcr7ZZYYG9 D4DkMxRZkkn7oeD8Gankhgl8G6XNNLWkujwjGW10KlXQxhY/u/HKrj9nzxccbiU7+THA0hy0vLCyf nmriSscYg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lZtOx-001LDs-8I; Fri, 23 Apr 2021 10:51:13 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lZtOt-001LDk-R5 for linux-arm-kernel@desiato.infradead.org; Fri, 23 Apr 2021 10:51:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=p02NyHgjvmqHLGK3MWNHsTDePUOWZrcwMLA03O7nXaw=; b=FpSSBSOVywjGcCvSKEnH6sojeQ yJ+BBe+sIQrvKeSFOc/u9yXgryMC9kWnetycAJYOEnWkMSXy5cDHjyFQXBBjbBYwYUqY9NR+XcqII ojqPDSHeJ0FhVBm9EAAXAYc4T84zNzZUyLtq7ez/qS7fe80AC627ban5u/vAdRjtfE57XhJwOeQyN PiFplqAB7VH6t1LWiYNhFtBbeB5OZhiq9q08+E9Hr2Bn1r5ScU8Nte41gxiwjZONopLYAoPoQgiGR /rFlXzoGMDh3PZBTHscAw1ShTEEWP1hS5xaPMn/xcbq5jCYMd0tQyz2eZJAPQWWB+EEYAXxK1bVbA c5eMpv+Q==; Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lZtOq-00ELKd-L7 for linux-arm-kernel@lists.infradead.org; Fri, 23 Apr 2021 10:51:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C68C211D4; Fri, 23 Apr 2021 03:50:58 -0700 (PDT) Received: from C02TD0UTHF1T.local (unknown [10.57.26.173]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6A3943F694; Fri, 23 Apr 2021 03:50:57 -0700 (PDT) Date: Fri, 23 Apr 2021 11:50:51 +0100 From: Mark Rutland To: He Ying Cc: tglx@linutronix.de, maz@kernel.org, julien.thierry.kdev@gmail.com, catalin.marinas@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: Re: [PATCH v2] irqchip/gic-v3: Do not enable irqs when handling spurious interrups Message-ID: <20210423105051.GA83097@C02TD0UTHF1T.local> References: <20210423083516.170111-1-heying24@huawei.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210423083516.170111-1-heying24@huawei.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210423_035104_812221_58E05676 X-CRM114-Status: GOOD ( 30.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Apr 23, 2021 at 04:35:16AM -0400, He Ying wrote: > We found this problem in our kernel src tree: > > [ 14.816231] ------------[ cut here ]------------ > [ 14.816231] kernel BUG at irq.c:99! > [ 14.816232] Internal error: Oops - BUG: 0 [#1] SMP > [ 14.816232] Process swapper/0 (pid: 0, stack limit = 0x(____ptrval____)) > [ 14.816233] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O 4.19.95.aarch64 #14 > [ 14.816233] Hardware name: evb (DT) > [ 14.816234] pstate: 80400085 (Nzcv daIf +PAN -UAO) > [ 14.816234] pc : asm_nmi_enter+0x94/0x98 > [ 14.816235] lr : asm_nmi_enter+0x18/0x98 > [ 14.816235] sp : ffff000008003c50 > [ 14.816235] pmr_save: 00000070 > [ 14.816237] x29: ffff000008003c50 x28: ffff0000095f56c0 > [ 14.816238] x27: 0000000000000000 x26: ffff000008004000 > [ 14.816239] x25: 00000000015e0000 x24: ffff8008fb916000 > [ 14.816240] x23: 0000000020400005 x22: ffff0000080817cc > [ 14.816241] x21: ffff000008003da0 x20: 0000000000000060 > [ 14.816242] x19: 00000000000003ff x18: ffffffffffffffff > [ 14.816243] x17: 0000000000000008 x16: 003d090000000000 > [ 14.816244] x15: ffff0000095ea6c8 x14: ffff8008fff5ab40 > [ 14.816244] x13: ffff8008fff58b9d x12: 0000000000000000 > [ 14.816245] x11: ffff000008c8a200 x10: 000000008e31fca5 > [ 14.816246] x9 : ffff000008c8a208 x8 : 000000000000000f > [ 14.816247] x7 : 0000000000000004 x6 : ffff8008fff58b9e > [ 14.816248] x5 : 0000000000000000 x4 : 0000000080000000 > [ 14.816249] x3 : 0000000000000000 x2 : 0000000080000000 > [ 14.816250] x1 : 0000000000120000 x0 : ffff0000095f56c0 > [ 14.816251] Call trace: > [ 14.816251] asm_nmi_enter+0x94/0x98 > [ 14.816251] el1_irq+0x8c/0x180 (IRQ C) > [ 14.816252] gic_handle_irq+0xbc/0x2e4 > [ 14.816252] el1_irq+0xcc/0x180 (IRQ B) > [ 14.816253] arch_timer_handler_virt+0x38/0x58 > [ 14.816253] handle_percpu_devid_irq+0x90/0x240 > [ 14.816253] generic_handle_irq+0x34/0x50 > [ 14.816254] __handle_domain_irq+0x68/0xc0 > [ 14.816254] gic_handle_irq+0xf8/0x2e4 > [ 14.816255] el1_irq+0xcc/0x180 (IRQ A) > [ 14.816255] arch_cpu_idle+0x34/0x1c8 > [ 14.816255] default_idle_call+0x24/0x44 > [ 14.816256] do_idle+0x1d0/0x2c8 > [ 14.816256] cpu_startup_entry+0x28/0x30 > [ 14.816256] rest_init+0xb8/0xc8 > [ 14.816257] start_kernel+0x4c8/0x4f4 > [ 14.816257] Code: 940587f1 d5384100 b9401001 36a7fd01 (d4210000) > [ 14.816258] Modules linked in: start_dp(O) smeth(O) > [ 15.103092] ---[ end trace 701753956cb14aa8 ]--- > [ 15.103093] Kernel panic - not syncing: Fatal exception in interrupt > [ 15.103099] SMP: stopping secondary CPUs > [ 15.103100] Kernel Offset: disabled > [ 15.103100] CPU features: 0x36,a2400218 > [ 15.103100] Memory Limit: none > > I look into this issue and find that it's caused by 'BUG_ON(in_nmi())' > in nmi_enter(). From the call trace, we can find three interrupts which > I mark as IRQ A, B and C. By adding some prints, I find the IRQ B also > calls nmi_enter(), but its priority is not GICD_INT_NMI_PRI and its irq > number is 1023. It enables irq by calling gic_arch_enable_irqs() in > gic_handle_irq(). At this moment, IRQ C preempts the IRQ B and it's > an NMI but current context is already in nmi. So that may be the problem. > > When handling spurious interrupts, we shouldn't enable irqs. That's > because for spurious interrupts we may enter nmi context in el1_irq() > because current PMR may be GIC_PRIO_IRQOFF. If we enable irqs at this > time, another NMI may happen and preempt this spurious interrupt > but the context is already in nmi. That causes a bug on if nested NMI > is not supported. Even for nested nmi, it's not a normal scenario. > > Though the issue is reported on our private tree, I think it also > exists on the latest tree for the reasons above. To fix this issue, > check spurious interrupts right after the read of ICC_IAR1_EL1 and > return directly for spurious interrupts. > > Fixes: 17ce302f3117 ("arm64: Fix interrupt tracing in the presence of NMIs") > Signed-off-by: He Ying I'm reckon the fixes tag should probably be either: Fixes: f32c926651dcd168 ("irqchip/gic-v3: Handle pseudo-NMIs") ... or: Fixes: 3f1f3234bc2db1c1 (" irqchip/gic-v3: Switch to PMR masking before calling IRQ handler") ... since the underlying issue is that gic_handle_irq() unmasks DAIF.I and permits unintended nesting, even if that doesn't trigger a BUG() at that point. Otherwise, this makes sense to me: Acked-by: Mark Rutland Mark. > --- > > v2: > - Move the check right after the read of ICC_IAR1_EL1 suggested by Marc. > > drivers/irqchip/irq-gic-v3.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c > index 94b89258d045..37a23aa6de37 100644 > --- a/drivers/irqchip/irq-gic-v3.c > +++ b/drivers/irqchip/irq-gic-v3.c > @@ -648,6 +648,10 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs > > irqnr = gic_read_iar(); > > + /* Check for special IDs first */ > + if ((irqnr >= 1020 && irqnr <= 1023)) > + return; > + > if (gic_supports_nmi() && > unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) { > gic_handle_nmi(irqnr, regs); > @@ -659,10 +663,6 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs > gic_arch_enable_irqs(); > } > > - /* Check for special IDs first */ > - if ((irqnr >= 1020 && irqnr <= 1023)) > - return; > - > if (static_branch_likely(&supports_deactivate_key)) > gic_write_eoir(irqnr); > else > -- > 2.17.1 > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel