From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DAF6C433B4 for ; Sun, 25 Apr 2021 05:32:34 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 887E0613BB for ; Sun, 25 Apr 2021 05:32:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 887E0613BB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:CC:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Xfmv1ctaw7lDoI8I9Ht4USfrHIURDiYaUXrE6IW3bxY=; b=Qajuk8+V6M8e0/EeR+0cjsoE8 BzPJkdREJdhSlepPuxZEfDNPVUdh5TdS5ispjT1VFHiIxArPE8hSeaclWvXCxQ/IRy9gyy2PCRG0U zDmn5rATZ9puxHex/rbhGQ7EhXy1PGCk9HmO7Uh+uVcAAqIONx9nDJl50zFghfNYhFteJw2NXfw5R WzSHVarEW04Dip+fJ/kXXQm3kZ8+Bogv8+cJEQ5bU1fa/EJvscczs+tKc3Cl3yY8OHbrY1eZ4fcw+ 1WWnMAWkOJbhy1ppNcWX8CLwHDXXS+D96MqsKodtqrUHRiIlbMvwhcZzODrk1YVoVVvdPpCmCxVWF llbBIH7Ig==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1laXLV-0054Vc-Hx; Sun, 25 Apr 2021 05:30:17 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1laXL7-0054UY-WE for linux-arm-kernel@desiato.infradead.org; Sun, 25 Apr 2021 05:29:55 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:CC:To: Subject:Sender:Reply-To:Content-ID:Content-Description; bh=cOXF9PTsPEDn9b2K6Injo+xbMadyzjNl2SV9MMzrufs=; b=2c50+DJAn8mWOu04xE2J9qAjj1 2wEny3CjFh5qwp6Q/6uF45C18iv0EWiRwT/NqDayWS14Dzx+QTlJ+FaNSBvlZ2werAMswYx2CUYkf HV4ncICVglEaRBbfqat5wmdS4R6pswBVEqWm8v0bN/eSWJAPeh3OnAT46FVE5Ro4xuN9r8fuNzNPW uWkiPnu+M9uJwcWNSjLZE/1xl0qUkNN/2/qJ6UPK/05Knm4wcSji7fgp9psDby1sw5hxA47XWkVTm yh5CsGCPk25+eEGtOa32BKkYN5IEbatZjhssmff/QtK8ix6NbsLN8Syfca94EWdufB8SLH6f8oZbW Hn1ZHE4A==; Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1laXL2-00FI8a-OF for linux-arm-kernel@lists.infradead.org; Sun, 25 Apr 2021 05:29:51 +0000 Received: from dggeml760-chm.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4FSc3b1tDhzRgwp; Sun, 25 Apr 2021 13:27:23 +0800 (CST) Received: from dggema764-chm.china.huawei.com (10.1.198.206) by dggeml760-chm.china.huawei.com (10.1.199.160) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Sun, 25 Apr 2021 13:29:37 +0800 Received: from [10.174.185.179] (10.174.185.179) by dggema764-chm.china.huawei.com (10.1.198.206) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2176.2; Sun, 25 Apr 2021 13:29:34 +0800 Subject: Re: [PATCHv2 09/11] arm64: entry: fix non-NMI kernel<->kernel transitions To: Mark Rutland CC: , , , , , , , , References: <20201130115950.22492-1-mark.rutland@arm.com> <20201130115950.22492-10-mark.rutland@arm.com> From: Zenghui Yu Message-ID: Date: Sun, 25 Apr 2021 13:29:31 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20201130115950.22492-10-mark.rutland@arm.com> Content-Language: en-US X-Originating-IP: [10.174.185.179] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggema764-chm.china.huawei.com (10.1.198.206) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210424_222949_231130_BA66DC73 X-CRM114-Status: GOOD ( 20.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Mark, On 2020/11/30 19:59, Mark Rutland wrote: > There are periods in kernel mode when RCU is not watching and/or the > scheduler tick is disabled, but we can still take exceptions such as > interrupts. The arm64 exception handlers do not account for this, and > it's possible that RCU is not watching while an exception handler runs. > > The x86/generic entry code handles this by ensuring that all (non-NMI) > kernel exception handlers call irqentry_enter() and irqentry_exit(), > which handle RCU, lockdep, and IRQ flag tracing. We can't yet move to > the generic entry code, and already hadnle the user<->kernel transitions > elsewhere, so we add new kernel<->kernel transition helpers alog the > lines of the generic entry code. > > Since we now track interrupts becoming masked when an exception is > taken, local_daif_inherit() is modified to track interrupts becoming > re-enabled when the original context is inherited. To balance the > entry/exit paths, each handler masks all DAIF exceptions before > exit_to_kernel_mode(). > > Signed-off-by: Mark Rutland > Cc: Catalin Marinas > Cc: James Morse > Cc: Will Deacon [...] > +/* > + * This is intended to match the logic in irqentry_enter(), handling the kernel > + * mode transitions only. > + */ > +static void noinstr enter_from_kernel_mode(struct pt_regs *regs) > +{ > + regs->exit_rcu = false; > + > + if (!IS_ENABLED(CONFIG_TINY_RCU) && is_idle_task(current)) { > + lockdep_hardirqs_off(CALLER_ADDR0); > + rcu_irq_enter(); > + trace_hardirqs_off_finish(); > + > + regs->exit_rcu = true; > + return; > + } > + > + lockdep_hardirqs_off(CALLER_ADDR0); > + rcu_irq_enter_check_tick(); > + trace_hardirqs_off_finish(); > +} Booting a lockdep-enabled kernel with "irqchip.gicv3_pseudo_nmi=1" would result in splats as below: | DEBUG_LOCKS_WARN_ON(!irqs_disabled()) | WARNING: CPU: 3 PID: 125 at kernel/locking/lockdep.c:4258 lockdep_hardirqs_off+0xd4/0xe8 | Modules linked in: | CPU: 3 PID: 125 Comm: modprobe Tainted: G W 5.12.0-rc8+ #463 | Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015 | pstate: 604003c5 (nZCv DAIF +PAN -UAO -TCO BTYPE=--) | pc : lockdep_hardirqs_off+0xd4/0xe8 | lr : lockdep_hardirqs_off+0xd4/0xe8 | sp : ffff80002a39bad0 | pmr_save: 000000e0 | x29: ffff80002a39bad0 x28: ffff0000de214bc0 | x27: ffff0000de1c0400 x26: 000000000049b328 | x25: 0000000000406f30 x24: ffff0000de1c00a0 | x23: 0000000020400005 x22: ffff8000105f747c | x21: 0000000096000044 x20: 0000000000498ef9 | x19: ffff80002a39bc88 x18: ffffffffffffffff | x17: 0000000000000000 x16: ffff800011c61eb0 | x15: ffff800011700a88 x14: 0720072007200720 | x13: 0720072007200720 x12: 0720072007200720 | x11: 0720072007200720 x10: 0720072007200720 | x9 : ffff80002a39bad0 x8 : ffff80002a39bad0 | x7 : ffff8000119f0800 x6 : c0000000ffff7fff | x5 : ffff8000119f07a8 x4 : 0000000000000001 | x3 : 9bcdab23f2432800 x2 : ffff800011730538 | x1 : 9bcdab23f2432800 x0 : 0000000000000000 | Call trace: | lockdep_hardirqs_off+0xd4/0xe8 | enter_from_kernel_mode.isra.5+0x7c/0xa8 | el1_abort+0x24/0x100 | el1_sync_handler+0x80/0xd0 | el1_sync+0x6c/0x100 | __arch_clear_user+0xc/0x90 | load_elf_binary+0x9fc/0x1450 | bprm_execve+0x404/0x880 | kernel_execve+0x180/0x188 | call_usermodehelper_exec_async+0xdc/0x158 | ret_from_fork+0x10/0x18 The code that triggers the splat is lockdep_hardirqs_off+0xd4/0xe8: | /* | * So we're supposed to get called after you mask local IRQs, but for | * some reason the hardware doesn't quite think you did a proper job. | */ | if (DEBUG_LOCKS_WARN_ON(!irqs_disabled())) | return; which looks like a false positive as DAIF are all masked on taking an synchronous exception and hardirqs are therefore disabled. With pseudo NMI used, irqs_disabled() takes the value of ICC_PMR_EL1 as the interrupt enable state, which is GIC_PRIO_IRQON (0xe0) in this case and doesn't help much. Not dig further though. Thanks, Zenghui _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel