From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_PATCH, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79C7AC433FE for ; Tue, 7 Sep 2021 16:54:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5D299610C9 for ; Tue, 7 Sep 2021 16:54:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1345262AbhIGQzF convert rfc822-to-8bit (ORCPT ); Tue, 7 Sep 2021 12:55:05 -0400 Received: from mail.kernel.org ([198.145.29.99]:33306 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345207AbhIGQzE (ORCPT ); Tue, 7 Sep 2021 12:55:04 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CED8061090; Tue, 7 Sep 2021 16:53:57 +0000 (UTC) Received: from 82-132-222-194.dab.02.net ([82.132.222.194] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mNeM7-009RKF-M2; Tue, 07 Sep 2021 17:53:55 +0100 Date: Tue, 07 Sep 2021 17:53:54 +0100 Message-ID: <87czpkconx.wl-maz@kernel.org> From: Marc Zyngier To: Linus Walleij Cc: Pali =?UTF-8?B?Um9ow6Fy?= , Lorenzo Pieralisi , Rob Herring , linux-pci , linux-kernel Subject: Re: pci-ftpci100: race condition in masking/unmasking interrupts In-Reply-To: References: <20210818114743.kksb7tydqjkww67h@pali> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-SA-Exim-Connect-IP: 82.132.222.194 X-SA-Exim-Rcpt-To: linus.walleij@linaro.org, pali@kernel.org, lorenzo.pieralisi@arm.com, robh@kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Linus, On Tue, 07 Sep 2021 12:22:37 +0100, Linus Walleij wrote: > > On Wed, Aug 18, 2021 at 1:47 PM Pali Rohár wrote: > > > I do not see any entry in MAINTAINERS file for pci-ftpci100.c driver, so > > I'm not sure to whom should I address this issue... > > It's me. > > > During pci-aardvark review, Marc pointed one issue which is currently > > available also in pci-ftpci100.c driver. > > > > When masking or unmasking interrupts there is read-modify-write sequence > > for FARADAY_PCI_CTRL2 register without any locking and is not atomic: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/controller/pci-ftpci100.c?h=v5.13#n270 > > > > So there is race condition when masking/unmasking more interrupts at the > > same time. > > I thought those operations were called in atomic context. > How did you fix it? They are. But that doesn't mean that you cannot have two CPUs dealing with two different interrupts at the same time (using disable_irq(), for example). When that happens, your interrupt masking becomes a bit soup. irq_ack() also gets in the way, as it does a RMW of the same register. If the underlying HW is strictly UP, you're safe. But even in this case, you could have some locking that gets elided at compile time. I also don't understand why you always clear the interrupt status every time you mask/unmask an interrupt. I came up with the following patchlet, which is completely untested (not even compile-tested). Thanks, M. diff --git a/drivers/pci/controller/pci-ftpci100.c b/drivers/pci/controller/pci-ftpci100.c index 88980a44461d..dd1697e61206 100644 --- a/drivers/pci/controller/pci-ftpci100.c +++ b/drivers/pci/controller/pci-ftpci100.c @@ -120,6 +120,7 @@ struct faraday_pci_variant { }; struct faraday_pci { + raw_spinlock_t lock struct device *dev; void __iomem *base; struct irq_domain *irqdomain; @@ -270,34 +271,41 @@ static struct pci_ops faraday_pci_ops = { static void faraday_pci_ack_irq(struct irq_data *d) { struct faraday_pci *p = irq_data_get_irq_chip_data(d); + unsigned long flags; unsigned int reg; + raw_spin_lock_irqsave(&p->lock, flags); faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, ®); reg &= ~(0xF << PCI_CTRL2_INTSTS_SHIFT); reg |= BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTSTS_SHIFT); faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg); + raw_spin_unlock_irqrestore(&p->lock, flags); } static void faraday_pci_mask_irq(struct irq_data *d) { struct faraday_pci *p = irq_data_get_irq_chip_data(d); + unsigned long flags; unsigned int reg; + raw_spin_lock_irqsave(&p->lock, flags); faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, ®); - reg &= ~((0xF << PCI_CTRL2_INTSTS_SHIFT) - | BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT)); + reg &= ~BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT); faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg); + raw_spin_unlock_irqrestore(&p->lock, flags); } static void faraday_pci_unmask_irq(struct irq_data *d) { struct faraday_pci *p = irq_data_get_irq_chip_data(d); + unsigned long flags; unsigned int reg; + raw_spin_lock_irqsave(&p->lock, flags); faraday_raw_pci_read_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, ®); - reg &= ~(0xF << PCI_CTRL2_INTSTS_SHIFT); reg |= BIT(irqd_to_hwirq(d) + PCI_CTRL2_INTMASK_SHIFT); faraday_raw_pci_write_config(p, 0, 0, FARADAY_PCI_CTRL2, 4, reg); + raw_spin_unlock_irqrestore(&p->lock, flags); } static void faraday_pci_irq_handler(struct irq_desc *desc) @@ -441,6 +449,8 @@ static int faraday_pci_probe(struct platform_device *pdev) host->sysdata = p; p->dev = dev; + raw_spin_lock_init(&p->lock); + /* Retrieve and enable optional clocks */ clk = devm_clk_get(dev, "PCLK"); if (IS_ERR(clk)) -- Without deviation from the norm, progress is not possible.