From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6450FC43441 for ; Wed, 14 Nov 2018 22:44:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2D3BC2145D for ; Wed, 14 Nov 2018 22:44:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D3BC2145D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-pci-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728615AbeKOIuF (ORCPT ); Thu, 15 Nov 2018 03:50:05 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:52688 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728422AbeKOIuE (ORCPT ); Thu, 15 Nov 2018 03:50:04 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 636C380D; Wed, 14 Nov 2018 14:44:54 -0800 (PST) Received: from big-swifty.misterjones.org (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 11B553F5A0; Wed, 14 Nov 2018 14:44:54 -0800 (PST) Date: Wed, 14 Nov 2018 22:44:53 +0000 Message-ID: <86pnv746u2.wl-marc.zyngier@arm.com> From: Marc Zyngier To: Trent Piepho Cc: "jingoohan1@gmail.com" , "lorenzo.pieralisi@arm.com" , "gustavo.pimentel@synopsys.com" , "faiz_abbas@ti.com" , "Joao.Pinto@synopsys.com" , "linux-pci@vger.kernel.org" , "helgaas@google.com" , "vigneshr@ti.com" Subject: Re: [PATCH 0/3] PCI: designware: Fixing MSI handling flow In-Reply-To: <1542234336.30311.494.camel@impinj.com> References: <20181113225734.8026-1-marc.zyngier@arm.com> <86a7mcdlwg.wl-marc.zyngier@arm.com> <1542223166.30311.471.camel@impinj.com> <86sh0348tu.wl-marc.zyngier@arm.com> <1542234336.30311.494.camel@impinj.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/25.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: ARM Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Wed, 14 Nov 2018 22:25:37 +0000, Trent Piepho wrote: > > On Wed, 2018-11-14 at 22:01 +0000, Marc Zyngier wrote: > > On Wed, 14 Nov 2018 19:19:27 +0000, > > Trent Piepho wrote: > > > > > > On Wed, 2018-11-14 at 09:54 +0000, Marc Zyngier wrote: > > > > /* Initialize IRQ Status array */ > > > > - for (ctrl = 0; ctrl < num_ctrls; ctrl++) > > > > - dw_pcie_rd_own_conf(pp, PCIE_MSI_INTR0_ENABLE + > > > > + for (ctrl = 0; ctrl < num_ctrls; ctrl++) { > > > > + dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_MASK + > > > > (ctrl * MSI_REG_CTRL_BLOCK_SIZE), > > > > - 4, &pp->irq_status[ctrl]); > > > > + 4, ~0); > > > > + dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_ENABLE + > > > > + (ctrl * MSI_REG_CTRL_BLOCK_SIZE), > > > > + 4, ~0); > > > > + pp->irq_status[ctrl] = 0; > > > > + } > > > > > > > > > > I tested yesterday before this patch was sent and fixed this issue > > > another way. I pretty sure this would work as well, though it's not > > > clear to me it's more correct. > > > > Given that we don't have a spec or any form of useful documentation > > (except for the information that Gustavo gave us), I don't think > > *anything* we'll write here has a remote chance of being > > correct. We're simply poking in the dark. > > Should all MSIs start enabled, or should they start disabled and be > enabled via the irq_enable method of the irq_chip, seems like a Linux > design decision to me. Decide that, then try to figure out how to make > the hardware do what Linux expects it to do. > > Starting disabled seems like the right design to me. So here's my > attempt to make the driver do this. Works in my tests. I've not > tracked down all uses of irq_status outside the driver to determine how > it's supposed to work. > > From dfc015f9821f5105cbcf9686d360105ffbac4ffb Mon Sep 17 00:00:00 2001 > From: Trent Piepho > Date: Wed, 14 Nov 2018 14:12:47 -0800 > Subject: [PATCH] PCI: dwc: Allow enabling MSIs and start with disabled > > Add irq_enable callbacks to let MSIs be enabled. > > Previously the driver would leave any MSIs enabled when it initialized > that way. Rather than that, disable them all. > > Signed-off-by: Trent Piepho > --- > drivers/pci/controller/dwc/pcie-designware-host.c | 37 > ++++++++++++++++++++--- > 1 file changed, 33 insertions(+), 4 deletions(-) > > diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c > b/drivers/pci/controller/dwc/pcie-designware-host.c > index f06e67c60593..e7770fb1ced8 100644 > --- a/drivers/pci/controller/dwc/pcie-designware-host.c > +++ b/drivers/pci/controller/dwc/pcie-designware-host.c > @@ -61,11 +61,17 @@ static void dw_msi_unmask_irq(struct irq_data *d) > irq_chip_unmask_parent(d); > } > > +static void dw_msi_enable_irq(struct irq_data *d) > +{ > + irq_chip_enable_parent(d); > +} > + > static struct irq_chip dw_pcie_msi_irq_chip = { > .name = "PCI-MSI", > .irq_ack = dw_msi_ack_irq, > .irq_mask = dw_msi_mask_irq, > .irq_unmask = dw_msi_unmask_irq, > + .irq_enable = dw_msi_enable_irq, If you're doing that, please implement both enable *and* disable. > }; > > static struct msi_domain_info dw_pcie_msi_domain_info = { > @@ -215,6 +221,26 @@ static void dw_pci_bottom_ack(struct irq_data *d) > raw_spin_unlock_irqrestore(&pp->lock, flags); > } > > +static void dw_pci_bottom_enable(struct irq_data *data) > +{ > + struct pcie_port *pp = irq_data_get_irq_chip_data(data); > + unsigned int res, bit, ctrl; > + unsigned long flags; > + u32 enable; > + > + ctrl = data->hwirq / MAX_MSI_IRQS_PER_CTRL; > + res = ctrl * MSI_REG_CTRL_BLOCK_SIZE; > + bit = data->hwirq % MAX_MSI_IRQS_PER_CTRL; > + > + raw_spin_lock_irqsave(&pp->lock, flags); > + > + dw_pcie_rd_own_conf(pp, PCIE_MSI_INTR0_ENABLE + res, 4, > &enable); > + enable |= BIT(bit); > + dw_pcie_wr_own_conf(pp, PCIE_MSI_INTR0_ENABLE + res, 4, > enable); > + > + raw_spin_unlock_irqrestore(&pp->lock, flags); > +} How does it work for drivers that use the callbacks stuff? M. -- Jazz is not dead, it just smell funny.