From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11B5DC47082 for ; Tue, 8 Jun 2021 17:59:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBAA361375 for ; Tue, 8 Jun 2021 17:59:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234000AbhFHSA4 (ORCPT ); Tue, 8 Jun 2021 14:00:56 -0400 Received: from foss.arm.com ([217.140.110.172]:36724 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233983AbhFHSAv (ORCPT ); Tue, 8 Jun 2021 14:00:51 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A908FD6E; Tue, 8 Jun 2021 10:58:57 -0700 (PDT) Received: from lpieralisi (e121166-lin.cambridge.arm.com [10.1.196.255]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8B18A3F694; Tue, 8 Jun 2021 10:58:56 -0700 (PDT) Date: Tue, 8 Jun 2021 18:58:47 +0100 From: Lorenzo Pieralisi To: Marc Zyngier Cc: Valentin Schneider , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Thomas Gleixner , Vincenzo Frascino , Mark Rutland , sudeep.holla@arm.com Subject: Re: [RFC PATCH v2 00/10] irqchip/irq-gic: Optimize masking by leveraging EOImode=1 Message-ID: <20210608175840.GA15997@lpieralisi> References: <20210525173255.620606-1-valentin.schneider@arm.com> <87zgwgs9x0.wl-maz@kernel.org> <87tumhg9vm.mognet@arm.com> <87a6o0z86t.wl-maz@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87a6o0z86t.wl-maz@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [+Sudeep] On Tue, Jun 08, 2021 at 04:29:14PM +0100, Marc Zyngier wrote: > [+Mark, since we discussed about this on IRC] > > Hi Valentin, > > On Tue, 01 Jun 2021 11:25:01 +0100, > Valentin Schneider wrote: > > > > On 27/05/21 12:17, Marc Zyngier wrote: > > > On Tue, 25 May 2021 18:32:45 +0100, > > > Valentin Schneider wrote: > > >> I've tested this on my Ampere eMAG, which uncovered "fun" interactions with > > >> the MSI domains. Did the same trick as the Juno with the pl011. > > >> > > >> pNMIs cause said eMAG to freeze, but that's true even without my patches. I > > >> did try them out under QEMU+KVM and that looked fine, although that means I > > >> only got to test EOImode=0. I'll try to dig into this when I get some more > > >> cycles. > > > > > > That's interesting/worrying. As far as I remember, this machine uses > > > GIC500, which is a well known quantity. If pNMIs are causing issues, > > > that'd probably be a CPU interface problem. Can you elaborate on how > > > you tried to test that part? Just using the below benchmark? > > > > > > > Not even that, it would hang somewhere at boot. Julien suggested offline > > that it might be a problem with the secondaries' PMR initial value, but I > > really never got to do dig into it. > > I just hit a similar problem on an Altra box, which seems to be > related to using PSCI for idle. PSCI has no idea about priority > masking, and enters CPU suspend with interrupt masked at the PMR > level. Good luck waking up from that. Gah. If we can manage to understand which path in psci_cpu_suspend_enter() is causing this problem that'd be great too (it can be both, for different reasons): if (!psci_power_state_loses_context(state)) (1) ret = psci_ops.cpu_suspend(state, 0); else (2) ret = cpu_suspend(state, psci_suspend_finisher); I'd like to understand if the problem is on idle entry or exit (or both depending on the state we are entering). On (1) we would return from the call with the CPU state retained on (2) with CPU context restored (but it rebooted from reset - so the PMR value is gone). I am asking about (2) because I am trying to understand what the power controller does wrt PMR and wake-up IRQs (ie and whether the PMR plays a role in that). Reworded: trying to understand how the PMR behaviour is playing with the power controller wake-up capabilities. Thoughts appreciated. I am sorry that you had to debug this, thank you for that. Lorenzo > I've pushed a test branch at [1]. It'd be really good if you could > have a quick look and let me know if that helps in your case (it > certainly does on the box I have access to). > > Thanks, > > M. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=arm64/nmi-idle > > -- > Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E512DC47082 for ; Tue, 8 Jun 2021 18:00:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AC12D61376 for ; Tue, 8 Jun 2021 18:00:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC12D61376 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=irxrxJB5yFQ+H4jaQ49e7K5CICfG4S/+qth6RVnDGUk=; b=AvUq4NUKfqZR0u cy99qYf2Kz7PkPYRp9DdNYQdf4eWdVnPHFskV7keF4Mul9NVgDCL+8fv83N7p0axvAqtL0kiQq11/ B1espUD0+yW1FvMi6lB9XFyb3rCQrb6bvOQvjaA/m2JJ4Sbc410b0EwcKBKTF8tCuKt7tBQ+qtz4Q zfvmhJcgcAaf6RNj0xQqezoFNcJu0AuX4fq8RCh1OZBEuIUO95u14baBWKriVE6xT1YiYCv3wl0qP M1yzYfVJ18QosoS1V5Ak4XE7WVUdFky41iDVl120w1zvg0VOfvkDjaW51gjDJ4H9e6hYSF+I/jFxs zplMvKcpwpd0a3F5IDow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqg0I-009qBU-3m; Tue, 08 Jun 2021 17:59:06 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqg0E-009qAd-9A for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 17:59:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A908FD6E; Tue, 8 Jun 2021 10:58:57 -0700 (PDT) Received: from lpieralisi (e121166-lin.cambridge.arm.com [10.1.196.255]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8B18A3F694; Tue, 8 Jun 2021 10:58:56 -0700 (PDT) Date: Tue, 8 Jun 2021 18:58:47 +0100 From: Lorenzo Pieralisi To: Marc Zyngier Cc: Valentin Schneider , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Thomas Gleixner , Vincenzo Frascino , Mark Rutland , sudeep.holla@arm.com Subject: Re: [RFC PATCH v2 00/10] irqchip/irq-gic: Optimize masking by leveraging EOImode=1 Message-ID: <20210608175840.GA15997@lpieralisi> References: <20210525173255.620606-1-valentin.schneider@arm.com> <87zgwgs9x0.wl-maz@kernel.org> <87tumhg9vm.mognet@arm.com> <87a6o0z86t.wl-maz@kernel.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <87a6o0z86t.wl-maz@kernel.org> User-Agent: Mutt/1.9.4 (2018-02-28) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_105902_461689_AD53117F X-CRM114-Status: GOOD ( 33.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org [+Sudeep] On Tue, Jun 08, 2021 at 04:29:14PM +0100, Marc Zyngier wrote: > [+Mark, since we discussed about this on IRC] > > Hi Valentin, > > On Tue, 01 Jun 2021 11:25:01 +0100, > Valentin Schneider wrote: > > > > On 27/05/21 12:17, Marc Zyngier wrote: > > > On Tue, 25 May 2021 18:32:45 +0100, > > > Valentin Schneider wrote: > > >> I've tested this on my Ampere eMAG, which uncovered "fun" interactions with > > >> the MSI domains. Did the same trick as the Juno with the pl011. > > >> > > >> pNMIs cause said eMAG to freeze, but that's true even without my patches. I > > >> did try them out under QEMU+KVM and that looked fine, although that means I > > >> only got to test EOImode=0. I'll try to dig into this when I get some more > > >> cycles. > > > > > > That's interesting/worrying. As far as I remember, this machine uses > > > GIC500, which is a well known quantity. If pNMIs are causing issues, > > > that'd probably be a CPU interface problem. Can you elaborate on how > > > you tried to test that part? Just using the below benchmark? > > > > > > > Not even that, it would hang somewhere at boot. Julien suggested offline > > that it might be a problem with the secondaries' PMR initial value, but I > > really never got to do dig into it. > > I just hit a similar problem on an Altra box, which seems to be > related to using PSCI for idle. PSCI has no idea about priority > masking, and enters CPU suspend with interrupt masked at the PMR > level. Good luck waking up from that. Gah. If we can manage to understand which path in psci_cpu_suspend_enter() is causing this problem that'd be great too (it can be both, for different reasons): if (!psci_power_state_loses_context(state)) (1) ret = psci_ops.cpu_suspend(state, 0); else (2) ret = cpu_suspend(state, psci_suspend_finisher); I'd like to understand if the problem is on idle entry or exit (or both depending on the state we are entering). On (1) we would return from the call with the CPU state retained on (2) with CPU context restored (but it rebooted from reset - so the PMR value is gone). I am asking about (2) because I am trying to understand what the power controller does wrt PMR and wake-up IRQs (ie and whether the PMR plays a role in that). Reworded: trying to understand how the PMR behaviour is playing with the power controller wake-up capabilities. Thoughts appreciated. I am sorry that you had to debug this, thank you for that. Lorenzo > I've pushed a test branch at [1]. It'd be really good if you could > have a quick look and let me know if that helps in your case (it > certainly does on the box I have access to). > > Thanks, > > M. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=arm64/nmi-idle > > -- > Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel