From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79EDFC43441 for ; Wed, 14 Nov 2018 09:54:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4333822360 for ; Wed, 14 Nov 2018 09:54:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4333822360 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-pci-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727598AbeKNT5N (ORCPT ); Wed, 14 Nov 2018 14:57:13 -0500 Received: from foss.arm.com ([217.140.101.70]:41086 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727558AbeKNT5N (ORCPT ); Wed, 14 Nov 2018 14:57:13 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 05A4D80D; Wed, 14 Nov 2018 01:54:41 -0800 (PST) Received: from big-swifty.misterjones.org (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A4C333F5BD; Wed, 14 Nov 2018 01:54:40 -0800 (PST) Date: Wed, 14 Nov 2018 09:54:39 +0000 Message-ID: <86a7mcdlwg.wl-marc.zyngier@arm.com> From: Marc Zyngier To: Gustavo Pimentel Cc: "linux-pci@vger.kernel.org" , Lorenzo Pieralisi , Bjorn Helgaas , Trent Piepho , Jingoo Han , "faiz_abbas@ti.com" , Joao Pinto , Vignesh R Subject: Re: [PATCH 0/3] PCI: designware: Fixing MSI handling flow In-Reply-To: References: <20181113225734.8026-1-marc.zyngier@arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/25.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: ARM Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Tue, 13 Nov 2018 23:16:33 +0000, Gustavo Pimentel wrote: > > On 13/11/2018 22:57, Marc Zyngier wrote: > > It recently came to light that the Designware PCIe driver is rather > > broken in the way it handles MSI[1]: > > > > - It masks interrupt by disabling them, meaning that MSIs generated > > during the masked window are simply lost. Oops. > > > > - Acking of the currently pending MSI is done outside of the interrupt > > flow, getting moved around randomly and ultimately breaking the > > driver. Not great. > > > > This series attempts to address this by switching to using the MASK > > register for masking interrupts (!), and move the ack into the > > appropriate callback, giving it a fixed place in the MSI handling > > flow. > > > > Note that this is only compile-tested on my arm64 laptop, as I'm > > travelling and do not have the required HW to test it anyway. I'd > > welcome both review and testing by the interested parties (dwc > > maintainer and users affected by existing bugs). > > As we spoke on the conference, as soon as I get back and I've the necessary > conditions I'll test the discussed modifications on my HW. I just realised (at 1am!) that the first patch is awfully buggy: - ENABLE and MASK have opposite polarities - There is nothing initialising the ENABLE and MASK registers I've stashed the following fix-up on top of the existing series: diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c index f06e67c60593..0fa9e8fdce66 100644 --- a/drivers/pci/controller/dwc/pcie-designware-host.c +++ b/drivers/pci/controller/dwc/pcie-designware-host.c @@ -166,7 +166,7 @@ static void dw_pci_bottom_mask(struct irq_data *data) pp->irq_status[ctrl] &= ~(1 << bit); dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4, - pp->irq_status[ctrl]); + ~pp->irq_status[ctrl]); } raw_spin_unlock_irqrestore(&pp->lock, flags); @@ -189,7 +189,7 @@ static void dw_pci_bottom_unmask(struct irq_data *data) pp->irq_status[ctrl] |= 1 << bit; dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + res, 4, - pp->irq_status[ctrl]); + ~pp->irq_status[ctrl]); } raw_spin_unlock_irqrestore(&pp->lock, flags); @@ -664,10 +664,15 @@ void dw_pcie_setup_rc(struct pcie_port *pp) num_ctrls = pp->num_vectors / MAX_MSI_IRQS_PER_CTRL; /* Initialize IRQ Status array */ - for (ctrl = 0; ctrl < num_ctrls; ctrl++) - dw_pcie_rd_own_conf(pp, PCIE_MSI_INTR0_ENABLE + + for (ctrl = 0; ctrl < num_ctrls; ctrl++) { + dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + (ctrl * MSI_REG_CTRL_BLOCK_SIZE), - 4, &pp->irq_status[ctrl]); + 4, ~0); + dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_ENABLE + + (ctrl * MSI_REG_CTRL_BLOCK_SIZE), + 4, ~0); + pp->irq_status[ctrl] = 0; + } /* Setup RC BARs */ dw_pcie_writel_dbi(pci, PCI_BASE_ADDRESS_0, 0x00000004); Please let me know if this helps. M. -- Jazz is not dead, it just smell funny.