From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92A65C433F5 for ; Sat, 25 Sep 2021 15:13:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 75A2B610F7 for ; Sat, 25 Sep 2021 15:13:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343745AbhIYPOr (ORCPT ); Sat, 25 Sep 2021 11:14:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55886 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234173AbhIYPOn (ORCPT ); Sat, 25 Sep 2021 11:14:43 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 638D0C061570 for ; Sat, 25 Sep 2021 08:13:08 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id s11so12904676pgr.11 for ; Sat, 25 Sep 2021 08:13:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=XEm0t7ryoVhJkdhQdSgea/DditBBxgJ9XTvsM+EhAlU=; b=BNWwsKUyLFJscWcAsXOpZORhYaoO2c/pVVryBCKJi5ZzRdP26aNOp32rEiVcfwprnK wNog7bcd7hL0dFmXCHDCEjJR/5gikbdmpnrSQ4tjhApl7b70+teMu+nAGrhCIR5dRMuu iduUVo+/zgoUQLFO4ESyAK7tCAI4Nae4z36HRP7luj1/SsrGzRi38EmZhOCngFm/gsh7 Cx9KMWnpP+USH803Os7dmUg5wIqX4M71JGP5SwQvF+k6obQ9+RAk5Lo8fuZLaBRAUvNm qNBXXpXOm3i5i+w27qNnMQij1G7yhqv7IGFpMqx8Wet9LaQqncmRbtg9WI4cfvhA+wsl pEcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XEm0t7ryoVhJkdhQdSgea/DditBBxgJ9XTvsM+EhAlU=; b=MtLkFOojtTA1dV7U7JrWS4rf+WO49ZKXpTrw59GoxA5P8Gad5UpNz+UXgFJ2lw0d0+ fSVIFAmaZDB3DzZMuWXGS6SCyNtULQ3owJUeKF8gM2sl7d/Js02H4G7AxE6cU7vlIbve H86Rp7uN+qAVE8rc1CzWcyR9GuCBTj2W71fHaYx0CJi1RKKH1vOrHXg2a6Aynco7TBQ3 f8mHsbdPpSjC2KEH7uxWmZA4cexPT6BUmc1jMIFCwANBRLFMd56b9XtP1CVmC6E0HaW+ Sc1C00XBwlb6f+RXEX962+SfUovyAMTr80WISsYPBo1EZYRnRSV0COzJAgUlgvIxrGIA EYTw== X-Gm-Message-State: AOAM533pOdQVfdsWsbncz722/nUkAWchE5lmAiFsgXUOtAtLvVxBZymJ Y9wwKNR0GThBTUNnYker6oB0UUVC9g== X-Google-Smtp-Source: ABdhPJzzNzdUnmRGWJE+VuLFDNiBzGvuoe+X1txlu8BHKPCyJPjRkor5c0KcwhhSdcZdKOvYUuaYMA== X-Received: by 2002:a62:ddd8:0:b0:435:4de8:2652 with SMTP id w207-20020a62ddd8000000b004354de82652mr14616389pff.53.1632582787762; Sat, 25 Sep 2021 08:13:07 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id r29sm12391351pfq.74.2021.09.25.08.13.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 25 Sep 2021 08:13:06 -0700 (PDT) Date: Sat, 25 Sep 2021 23:12:59 +0800 From: Pingfan Liu To: Mark Rutland Cc: Thomas Gleixner , "Paul E. McKenney" , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Marc Zyngier , Joey Gouly , Sami Tolvanen , Julien Thierry , Yuichi Ito , linux-kernel@vger.kernel.org, Sven Schnelle , Vasily Gorbik Subject: Re: [PATCHv2 0/5] arm64/irqentry: remove duplicate housekeeping of Message-ID: References: <20210924132837.45994-1-kernelfans@gmail.com> <20210924173615.GA42068@C02TD0UTHF1T.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210924173615.GA42068@C02TD0UTHF1T.local> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 24, 2021 at 06:36:15PM +0100, Mark Rutland wrote: > [Adding Paul for RCU, s390 folk for entry code RCU semantics] > > On Fri, Sep 24, 2021 at 09:28:32PM +0800, Pingfan Liu wrote: > > After introducing arm64/kernel/entry_common.c which is akin to > > kernel/entry/common.c , the housekeeping of rcu/trace are done twice as > > the following: > > enter_from_kernel_mode()->rcu_irq_enter(). > > And > > gic_handle_irq()->...->handle_domain_irq()->irq_enter()->rcu_irq_enter() > > > > Besides redundance, based on code analysis, the redundance also raise > > some mistake, e.g. rcu_data->dynticks_nmi_nesting inc 2, which causes > > rcu_is_cpu_rrupt_from_idle() unexpected. > > Hmmm... > > The fundamental questionss are: > > 1) Who is supposed to be responsible for doing the rcu entry/exit? > > 2) Is it supposed to matter if this happens multiple times? > > For (1), I'd generally expect that this is supposed to happen in the > arch/common entry code, since that itself (or the irqchip driver) could > depend on RCU, and if that's the case thatn handle_domain_irq() > shouldn't need to call rcu_irq_enter(). That would be consistent with > the way we handle all other exceptions. > In my humble opinion, it had better happen in arch/common entry code as you said. But for many arches which assures that before handle_domain_irq(), no data is involved in rcu updater, it can be done in handle_domain_irq(). And that is a cheap way to integrate with rcu system (at least for the time being). For the (2), it goes deeply into RCU core, hope guides from Paul and Thomas. But at least for the condition if ((user || rcu_is_cpu_rrupt_from_idle()) && rcu_nohz_full_cpu()) in rcu_pending(), it makes sense to tell the nested interrupt from a first level interrupt. Thanks, Pingfan > For (2) I don't know whether the level of nesting is suppoosed to > matter. I was under the impression it wasn't meant to matter in general, > so I'm a little surprised that rcu_is_cpu_rrupt_from_idle() depends on a > specific level of nesting. > > From a glance it looks like this would cause rcu_sched_clock_irq() to > skip setting TIF_NEED_RESCHED, and to not call invoke_rcu_core(), which > doesn't sound right, at least... > > Thomas, Paul, thoughts? > > AFAICT, s390 will have a similar flow on its IRQ handling path, so if > this is a real issue they'll be affected too. > > Thanks, > Mark. > > > Nmi also faces duplicate accounts. This series aims to address these > > duplicate issues. > > [1-2/5]: address nmi account duplicate > > [3-4/5]: address rcu housekeeping duplicate in irq > > [5/5]: as a natural result of [3-4/5], address a history issue. [1] > > > > > > History: > > v1 -> v2: > > change the subject as the motivation varies. > > add the fix for nmi account duplicate > > > > The subject of v1 is "[PATCH 1/3] kernel/irq: __handle_domain_irq() > > makes irq_enter/exit arch optional". [2] It is brought up to fix [1]. > > > > There have been some tries to enable crash-stop-NMI on arm64, one by me, > > the other by Yuichi's [4]. I hope after this series, they can advance, > > as Marc said in [3] "No additional NMI patches will make it until we > > have resolved the issues" > > > > [1] https://lore.kernel.org/linux-arm-kernel/87lfewnmdz.fsf@nanos.tec.linutronix.de/ > > [2] https://lore.kernel.org/linux-arm-kernel/1607912752-12481-1-git-send-email-kernelfans@gmail.com > > [3] https://lore.kernel.org/linux-arm-kernel/afd82be798cb55fd2f96940db7be78c0@kernel.org > > [4] https://lore.kernel.org/linux-arm-kernel/20201104080539.3205889-1-ito-yuichi@fujitsu.com > > > > Cc: Catalin Marinas > > Cc: Will Deacon > > Cc: Mark Rutland > > Cc: Marc Zyngier > > Cc: Joey Gouly > > Cc: Sami Tolvanen > > Cc: Julien Thierry > > Cc: Thomas Gleixner > > Cc: Yuichi Ito > > Cc: linux-kernel@vger.kernel.org > > To: linux-arm-kernel@lists.infradead.org > > > > > > Pingfan Liu (5): > > arm64/entry-common: push the judgement of nmi ahead > > irqchip/GICv3: expose handle_nmi() directly > > kernel/irq: make irq_{enter,exit}() in handle_domain_irq() arch > > optional > > irqchip/GICv3: let gic_handle_irq() utilize irqentry on arm64 > > irqchip/GICv3: make reschedule-ipi light weight > > > > arch/arm64/Kconfig | 1 + > > arch/arm64/include/asm/irq.h | 7 ++++ > > arch/arm64/kernel/entry-common.c | 45 +++++++++++++++------- > > arch/arm64/kernel/irq.c | 29 ++++++++++++++ > > drivers/irqchip/irq-gic-v3.c | 66 ++++++++++++++++++++------------ > > kernel/irq/Kconfig | 3 ++ > > kernel/irq/irqdesc.c | 4 ++ > > 7 files changed, 116 insertions(+), 39 deletions(-) > > > > -- > > 2.31.1 > > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A67C2C433F5 for ; Sat, 25 Sep 2021 15:15:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 72A4660EE9 for ; Sat, 25 Sep 2021 15:15:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 72A4660EE9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qGGiqA75bUU03YiPyahlocvXyGVhcWv+EV8LsWnJR60=; b=vBOZQYZes/FimD uOa5OY3lafe4ezMGogi5E9G0zOlOY7ONADrcwGwKEz4C6uKgsHhfha8gn4lutuU6ZvxbSOqXqQk2O FH/sjjyZD99zvfqXy7mPVDX7XBtuwLs1K885mNyWclawiwU1++mpT4zhV5zxW52yq/Ghk46HkR+oc dqeIbcfY9CELQCNGCuo1ofEc6QLjIngZssY9o3pAOSZWWRnAlMgdRX6FIpGDZBClcDKWWAMq5FZuf 54p/EMwCEcKD01EpAqPSmRuZKcXUh32YDjLkgYFsx76OXfoPj0X6X9we3Mmxx9rbyqd4dYPSvPdOB wJDNNIwR6k7pv/tQp1XA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mU9MX-00GnDk-61; Sat, 25 Sep 2021 15:13:13 +0000 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mU9MT-00GnDO-Ah for linux-arm-kernel@lists.infradead.org; Sat, 25 Sep 2021 15:13:11 +0000 Received: by mail-pg1-x52f.google.com with SMTP id t1so12970401pgv.3 for ; Sat, 25 Sep 2021 08:13:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=XEm0t7ryoVhJkdhQdSgea/DditBBxgJ9XTvsM+EhAlU=; b=BNWwsKUyLFJscWcAsXOpZORhYaoO2c/pVVryBCKJi5ZzRdP26aNOp32rEiVcfwprnK wNog7bcd7hL0dFmXCHDCEjJR/5gikbdmpnrSQ4tjhApl7b70+teMu+nAGrhCIR5dRMuu iduUVo+/zgoUQLFO4ESyAK7tCAI4Nae4z36HRP7luj1/SsrGzRi38EmZhOCngFm/gsh7 Cx9KMWnpP+USH803Os7dmUg5wIqX4M71JGP5SwQvF+k6obQ9+RAk5Lo8fuZLaBRAUvNm qNBXXpXOm3i5i+w27qNnMQij1G7yhqv7IGFpMqx8Wet9LaQqncmRbtg9WI4cfvhA+wsl pEcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=XEm0t7ryoVhJkdhQdSgea/DditBBxgJ9XTvsM+EhAlU=; b=CdAhzC6OlgiDjg73Ez6FZRy59Y93ka34Vw+dWf2GI6y0T+nL/qJQEQ3ko17AOtfB+/ xXx1K2D8sith026qaaqUZ+OqzxuQs1U9IACHcB5A/yxpXIBPDn2g7HyBxrm/8HZZKrr5 Wf8t6xm5IsNxFy5dU5Rulw2qM3wv2FcOm1KWD2PWT+9s1IAVVPsG0dzOoO1oWQd+EAID 77d41P0tMZQmjdAGBkGbRMuMwNutFPq/0EkdBN18FMYbYw2GjnWonzu4Is8t6+VtyeWC HTUrmQYKHvs14PubSFA/IzXcRx5r6OJ8rsoHrkxHqrDGBA57e3nt3Y7eWCdlHkRoGhxL 16sw== X-Gm-Message-State: AOAM533mvyUmG8pRXQO8fVVBWuqky76akvPUa5BIpgMTLFKGj16Um46B DqiZEFd79gL1+jqS6MpN7g== X-Google-Smtp-Source: ABdhPJzzNzdUnmRGWJE+VuLFDNiBzGvuoe+X1txlu8BHKPCyJPjRkor5c0KcwhhSdcZdKOvYUuaYMA== X-Received: by 2002:a62:ddd8:0:b0:435:4de8:2652 with SMTP id w207-20020a62ddd8000000b004354de82652mr14616389pff.53.1632582787762; Sat, 25 Sep 2021 08:13:07 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id r29sm12391351pfq.74.2021.09.25.08.13.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 25 Sep 2021 08:13:06 -0700 (PDT) Date: Sat, 25 Sep 2021 23:12:59 +0800 From: Pingfan Liu To: Mark Rutland Cc: Thomas Gleixner , "Paul E. McKenney" , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Will Deacon , Marc Zyngier , Joey Gouly , Sami Tolvanen , Julien Thierry , Yuichi Ito , linux-kernel@vger.kernel.org, Sven Schnelle , Vasily Gorbik Subject: Re: [PATCHv2 0/5] arm64/irqentry: remove duplicate housekeeping of Message-ID: References: <20210924132837.45994-1-kernelfans@gmail.com> <20210924173615.GA42068@C02TD0UTHF1T.local> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20210924173615.GA42068@C02TD0UTHF1T.local> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210925_081309_420963_FFA417CE X-CRM114-Status: GOOD ( 37.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Sep 24, 2021 at 06:36:15PM +0100, Mark Rutland wrote: > [Adding Paul for RCU, s390 folk for entry code RCU semantics] > > On Fri, Sep 24, 2021 at 09:28:32PM +0800, Pingfan Liu wrote: > > After introducing arm64/kernel/entry_common.c which is akin to > > kernel/entry/common.c , the housekeeping of rcu/trace are done twice as > > the following: > > enter_from_kernel_mode()->rcu_irq_enter(). > > And > > gic_handle_irq()->...->handle_domain_irq()->irq_enter()->rcu_irq_enter() > > > > Besides redundance, based on code analysis, the redundance also raise > > some mistake, e.g. rcu_data->dynticks_nmi_nesting inc 2, which causes > > rcu_is_cpu_rrupt_from_idle() unexpected. > > Hmmm... > > The fundamental questionss are: > > 1) Who is supposed to be responsible for doing the rcu entry/exit? > > 2) Is it supposed to matter if this happens multiple times? > > For (1), I'd generally expect that this is supposed to happen in the > arch/common entry code, since that itself (or the irqchip driver) could > depend on RCU, and if that's the case thatn handle_domain_irq() > shouldn't need to call rcu_irq_enter(). That would be consistent with > the way we handle all other exceptions. > In my humble opinion, it had better happen in arch/common entry code as you said. But for many arches which assures that before handle_domain_irq(), no data is involved in rcu updater, it can be done in handle_domain_irq(). And that is a cheap way to integrate with rcu system (at least for the time being). For the (2), it goes deeply into RCU core, hope guides from Paul and Thomas. But at least for the condition if ((user || rcu_is_cpu_rrupt_from_idle()) && rcu_nohz_full_cpu()) in rcu_pending(), it makes sense to tell the nested interrupt from a first level interrupt. Thanks, Pingfan > For (2) I don't know whether the level of nesting is suppoosed to > matter. I was under the impression it wasn't meant to matter in general, > so I'm a little surprised that rcu_is_cpu_rrupt_from_idle() depends on a > specific level of nesting. > > From a glance it looks like this would cause rcu_sched_clock_irq() to > skip setting TIF_NEED_RESCHED, and to not call invoke_rcu_core(), which > doesn't sound right, at least... > > Thomas, Paul, thoughts? > > AFAICT, s390 will have a similar flow on its IRQ handling path, so if > this is a real issue they'll be affected too. > > Thanks, > Mark. > > > Nmi also faces duplicate accounts. This series aims to address these > > duplicate issues. > > [1-2/5]: address nmi account duplicate > > [3-4/5]: address rcu housekeeping duplicate in irq > > [5/5]: as a natural result of [3-4/5], address a history issue. [1] > > > > > > History: > > v1 -> v2: > > change the subject as the motivation varies. > > add the fix for nmi account duplicate > > > > The subject of v1 is "[PATCH 1/3] kernel/irq: __handle_domain_irq() > > makes irq_enter/exit arch optional". [2] It is brought up to fix [1]. > > > > There have been some tries to enable crash-stop-NMI on arm64, one by me, > > the other by Yuichi's [4]. I hope after this series, they can advance, > > as Marc said in [3] "No additional NMI patches will make it until we > > have resolved the issues" > > > > [1] https://lore.kernel.org/linux-arm-kernel/87lfewnmdz.fsf@nanos.tec.linutronix.de/ > > [2] https://lore.kernel.org/linux-arm-kernel/1607912752-12481-1-git-send-email-kernelfans@gmail.com > > [3] https://lore.kernel.org/linux-arm-kernel/afd82be798cb55fd2f96940db7be78c0@kernel.org > > [4] https://lore.kernel.org/linux-arm-kernel/20201104080539.3205889-1-ito-yuichi@fujitsu.com > > > > Cc: Catalin Marinas > > Cc: Will Deacon > > Cc: Mark Rutland > > Cc: Marc Zyngier > > Cc: Joey Gouly > > Cc: Sami Tolvanen > > Cc: Julien Thierry > > Cc: Thomas Gleixner > > Cc: Yuichi Ito > > Cc: linux-kernel@vger.kernel.org > > To: linux-arm-kernel@lists.infradead.org > > > > > > Pingfan Liu (5): > > arm64/entry-common: push the judgement of nmi ahead > > irqchip/GICv3: expose handle_nmi() directly > > kernel/irq: make irq_{enter,exit}() in handle_domain_irq() arch > > optional > > irqchip/GICv3: let gic_handle_irq() utilize irqentry on arm64 > > irqchip/GICv3: make reschedule-ipi light weight > > > > arch/arm64/Kconfig | 1 + > > arch/arm64/include/asm/irq.h | 7 ++++ > > arch/arm64/kernel/entry-common.c | 45 +++++++++++++++------- > > arch/arm64/kernel/irq.c | 29 ++++++++++++++ > > drivers/irqchip/irq-gic-v3.c | 66 ++++++++++++++++++++------------ > > kernel/irq/Kconfig | 3 ++ > > kernel/irq/irqdesc.c | 4 ++ > > 7 files changed, 116 insertions(+), 39 deletions(-) > > > > -- > > 2.31.1 > > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel