From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E51CC433E2 for ; Wed, 24 Mar 2021 16:12:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E90C761A12 for ; Wed, 24 Mar 2021 16:12:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233311AbhCXQM0 (ORCPT ); Wed, 24 Mar 2021 12:12:26 -0400 Received: from mail.kernel.org ([198.145.29.99]:33484 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233296AbhCXQMA (ORCPT ); Wed, 24 Mar 2021 12:12:00 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 95F0061A01; Wed, 24 Mar 2021 16:11:59 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lP66v-003YPB-O4; Wed, 24 Mar 2021 16:11:57 +0000 Date: Wed, 24 Mar 2021 16:11:56 +0000 Message-ID: <8735wkjzub.wl-maz@kernel.org> From: Marc Zyngier To: Lorenzo Pieralisi Cc: Robin Murphy , Bjorn Helgaas , Frank Wunderlich , Thierry Reding , Thomas Gleixner , Rob Herring , Will Deacon , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Michael Kelley , Wei Liu , Thierry Reding , Jonathan Hunter , Ryder Lee , Marek Vasut , Yoshihiro Shimoda , Michal Simek , Paul Walmsley , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v2 12/15] PCI/MSI: Let PCI host bridges declare their reliance on MSI domains In-Reply-To: <20210324131938.GA16722@e121166-lin.cambridge.arm.com> References: <20210322184614.802565-1-maz@kernel.org> <20210322184614.802565-13-maz@kernel.org> <6a2eaa5d-1d83-159f-69e5-c9e0a00a7b50@arm.com> <87im5hkahr.wl-maz@kernel.org> <20210324131938.GA16722@e121166-lin.cambridge.arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: lorenzo.pieralisi@arm.com, robin.murphy@arm.com, bhelgaas@google.com, frank-w@public-files.de, treding@nvidia.com, tglx@linutronix.de, robh@kernel.org, will@kernel.org, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, mikelley@microsoft.com, wei.liu@kernel.org, thierry.reding@gmail.com, jonathanh@nvidia.com, ryder.lee@mediatek.com, marek.vasut+renesas@gmail.com, yoshihiro.shimoda.uh@renesas.com, michal.simek@xilinx.com, paul.walmsley@sifive.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-tegra@vger.kernel.org On Wed, 24 Mar 2021 13:19:38 +0000, Lorenzo Pieralisi wrote: > > On Tue, Mar 23, 2021 at 06:09:36PM +0000, Marc Zyngier wrote: > > Hi Robin, > > > > On Tue, 23 Mar 2021 11:45:02 +0000, > > Robin Murphy wrote: > > > > > > On 2021-03-22 18:46, Marc Zyngier wrote: > > > > The new 'no_msi' attribute solves the problem of advertising the lack > > > > of MSI capability for host bridges that know for sure that there will > > > > be no MSI for their end-points. > > > > > > > > However, there is a whole class of host bridges that cannot know > > > > whether MSIs will be provided or not, as they rely on other blocks > > > > to provide the MSI functionnality, using MSI domains. This is > > > > the case for example on systems that use the ARM GIC architecture. > > > > > > > > Introduce a new attribute ('msi_domain') indicating that implicit > > > > dependency, and use this property to set the NO_MSI flag when > > > > no MSI domain is found at probe time. > > > > > > > > Acked-by: Bjorn Helgaas > > > > Signed-off-by: Marc Zyngier > > > > --- > > > > drivers/pci/probe.c | 2 +- > > > > include/linux/pci.h | 1 + > > > > 2 files changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > > > > index 146bd85c037e..bac9f69a06a8 100644 > > > > --- a/drivers/pci/probe.c > > > > +++ b/drivers/pci/probe.c > > > > @@ -925,7 +925,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) > > > > device_enable_async_suspend(bus->bridge); > > > > pci_set_bus_of_node(bus); > > > > pci_set_bus_msi_domain(bus); > > > > - if (bridge->no_msi) > > > > + if (bridge->no_msi || (bridge->msi_domain && !bus->dev.msi_domain)) > > > > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > > > if (!parent) > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > > index 48605cca82ae..d322d00db432 100644 > > > > --- a/include/linux/pci.h > > > > +++ b/include/linux/pci.h > > > > @@ -551,6 +551,7 @@ struct pci_host_bridge { > > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > > unsigned int no_msi:1; /* Bridge has no MSI support */ > > > > + unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > > > > Aren't these really the same thing? Either way we're saying the bridge > > > itself doesn't handle MSIs, it's just in one case we're effectively > > > encoding a platform-specific assumption that an external domain won't > > > be provided. I can't help wondering whether that distinction is really > > > necessary... > > > > There is a subtle difference: no_msi indicates that there is no way > > *any* MSI can be dealt with whatsoever (maybe because the RC doesn't > > forward the corresponding TLPs?). msi_domain says "no MSI unless...". > > > > We could implement the former with the latter, but I have the feeling > > that's not totally bullet proof. Happy to revisit this if you think it > > really matters. > > IIUC msi_domain == 1 means: this host bridge needs an msi_domain to enable > MSIs, which in turn means that there are bridges that do _not_ require > an msi_domain to enable MSIs. I don't know how other arches handle the > msi_domain pointer but I am asking whether making: > > if (bridge->no_msi || !bus->dev.msi_domain)) > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > is a possibility (removing the need for the msi_domain flag). > > At least this looks more like an arch property than a host bridge > specific property (eg patch [13] pci_host_common_probe() may be used on > arches other than ARM where it is not necessary true that it requires an > msi_domain to enable MSIs). > > I agree that's complicated to untangle - just asking if there is way > to simplify it. I tried to simplify that in the past (see the original discussion at [1]), and tglx reported some breakages on systems that do not use MSI domains, which is why we ended up with an explicit flag. What I have done for now is to go with Robin's proposal of dropping 'no_msi' and rely on solely on 'msi_domain' to set PCI_BUS_FLAGS_NO_MSI when no domain is found. Note that if we indeed have a host bridge that uses pci_host_common_probe() that doesn't use MSI domains, we may indeed run into problems. I don't have a good way around that, unfortunately. Thanks, M. [1] https://lore.kernel.org/r/20201031140330.83768-1-linux@fw-web.de -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5113C433C1 for ; Wed, 24 Mar 2021 16:12:31 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 964BA61A07 for ; Wed, 24 Mar 2021 16:12:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 964BA61A07 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CrSHgp6KQsp3udiMK7XxY9Z1nHyIdS7vQVxrHy3tjVA=; b=NI65qnsqg+jwaPFQ7MNrjhgxn HbKVfeYB1RdgftyAQkb7QcvSC1l9fx8nT3/oX7ntsT8UMQYdYP8In6FCKXgL+yCJYNMQM2Xpq4qLd +tI7hRKelvkI+xlorGBkxILaMGpCkKeZgeZLCUg+vMlJmWTb8koCi61OIqZo773TM/dNlHOAJNOYr PHoj+aki7IzsRqEiYaMCPTYfEy54zjxjG3OjiMX1A4oKMgFAKZogX6nDU89SCUG1pJQI3fsXUSswv j+8807bmMniRsB7OXmHFzQyXbhlhPL0c/MaDnJIYmoEy/V0898iP2OQVXIlDa6CETT5P7rlASsDLN laGJUQAhA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lP67G-00HM40-LW; Wed, 24 Mar 2021 16:12:18 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lP671-00HM0N-1a; Wed, 24 Mar 2021 16:12:05 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 95F0061A01; Wed, 24 Mar 2021 16:11:59 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lP66v-003YPB-O4; Wed, 24 Mar 2021 16:11:57 +0000 Date: Wed, 24 Mar 2021 16:11:56 +0000 Message-ID: <8735wkjzub.wl-maz@kernel.org> From: Marc Zyngier To: Lorenzo Pieralisi Cc: Robin Murphy , Bjorn Helgaas , Frank Wunderlich , Thierry Reding , Thomas Gleixner , Rob Herring , Will Deacon , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Michael Kelley , Wei Liu , Thierry Reding , Jonathan Hunter , Ryder Lee , Marek Vasut , Yoshihiro Shimoda , Michal Simek , Paul Walmsley , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v2 12/15] PCI/MSI: Let PCI host bridges declare their reliance on MSI domains In-Reply-To: <20210324131938.GA16722@e121166-lin.cambridge.arm.com> References: <20210322184614.802565-1-maz@kernel.org> <20210322184614.802565-13-maz@kernel.org> <6a2eaa5d-1d83-159f-69e5-c9e0a00a7b50@arm.com> <87im5hkahr.wl-maz@kernel.org> <20210324131938.GA16722@e121166-lin.cambridge.arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: lorenzo.pieralisi@arm.com, robin.murphy@arm.com, bhelgaas@google.com, frank-w@public-files.de, treding@nvidia.com, tglx@linutronix.de, robh@kernel.org, will@kernel.org, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, mikelley@microsoft.com, wei.liu@kernel.org, thierry.reding@gmail.com, jonathanh@nvidia.com, ryder.lee@mediatek.com, marek.vasut+renesas@gmail.com, yoshihiro.shimoda.uh@renesas.com, michal.simek@xilinx.com, paul.walmsley@sifive.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210324_161203_563047_157F2DEB X-CRM114-Status: GOOD ( 48.44 ) X-BeenThere: linux-mediatek@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "Linux-mediatek" Errors-To: linux-mediatek-bounces+linux-mediatek=archiver.kernel.org@lists.infradead.org On Wed, 24 Mar 2021 13:19:38 +0000, Lorenzo Pieralisi wrote: > > On Tue, Mar 23, 2021 at 06:09:36PM +0000, Marc Zyngier wrote: > > Hi Robin, > > > > On Tue, 23 Mar 2021 11:45:02 +0000, > > Robin Murphy wrote: > > > > > > On 2021-03-22 18:46, Marc Zyngier wrote: > > > > The new 'no_msi' attribute solves the problem of advertising the lack > > > > of MSI capability for host bridges that know for sure that there will > > > > be no MSI for their end-points. > > > > > > > > However, there is a whole class of host bridges that cannot know > > > > whether MSIs will be provided or not, as they rely on other blocks > > > > to provide the MSI functionnality, using MSI domains. This is > > > > the case for example on systems that use the ARM GIC architecture. > > > > > > > > Introduce a new attribute ('msi_domain') indicating that implicit > > > > dependency, and use this property to set the NO_MSI flag when > > > > no MSI domain is found at probe time. > > > > > > > > Acked-by: Bjorn Helgaas > > > > Signed-off-by: Marc Zyngier > > > > --- > > > > drivers/pci/probe.c | 2 +- > > > > include/linux/pci.h | 1 + > > > > 2 files changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > > > > index 146bd85c037e..bac9f69a06a8 100644 > > > > --- a/drivers/pci/probe.c > > > > +++ b/drivers/pci/probe.c > > > > @@ -925,7 +925,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) > > > > device_enable_async_suspend(bus->bridge); > > > > pci_set_bus_of_node(bus); > > > > pci_set_bus_msi_domain(bus); > > > > - if (bridge->no_msi) > > > > + if (bridge->no_msi || (bridge->msi_domain && !bus->dev.msi_domain)) > > > > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > > > if (!parent) > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > > index 48605cca82ae..d322d00db432 100644 > > > > --- a/include/linux/pci.h > > > > +++ b/include/linux/pci.h > > > > @@ -551,6 +551,7 @@ struct pci_host_bridge { > > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > > unsigned int no_msi:1; /* Bridge has no MSI support */ > > > > + unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > > > > Aren't these really the same thing? Either way we're saying the bridge > > > itself doesn't handle MSIs, it's just in one case we're effectively > > > encoding a platform-specific assumption that an external domain won't > > > be provided. I can't help wondering whether that distinction is really > > > necessary... > > > > There is a subtle difference: no_msi indicates that there is no way > > *any* MSI can be dealt with whatsoever (maybe because the RC doesn't > > forward the corresponding TLPs?). msi_domain says "no MSI unless...". > > > > We could implement the former with the latter, but I have the feeling > > that's not totally bullet proof. Happy to revisit this if you think it > > really matters. > > IIUC msi_domain == 1 means: this host bridge needs an msi_domain to enable > MSIs, which in turn means that there are bridges that do _not_ require > an msi_domain to enable MSIs. I don't know how other arches handle the > msi_domain pointer but I am asking whether making: > > if (bridge->no_msi || !bus->dev.msi_domain)) > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > is a possibility (removing the need for the msi_domain flag). > > At least this looks more like an arch property than a host bridge > specific property (eg patch [13] pci_host_common_probe() may be used on > arches other than ARM where it is not necessary true that it requires an > msi_domain to enable MSIs). > > I agree that's complicated to untangle - just asking if there is way > to simplify it. I tried to simplify that in the past (see the original discussion at [1]), and tglx reported some breakages on systems that do not use MSI domains, which is why we ended up with an explicit flag. What I have done for now is to go with Robin's proposal of dropping 'no_msi' and rely on solely on 'msi_domain' to set PCI_BUS_FLAGS_NO_MSI when no domain is found. Note that if we indeed have a host bridge that uses pci_host_common_probe() that doesn't use MSI domains, we may indeed run into problems. I don't have a good way around that, unfortunately. Thanks, M. [1] https://lore.kernel.org/r/20201031140330.83768-1-linux@fw-web.de -- Without deviation from the norm, progress is not possible. _______________________________________________ Linux-mediatek mailing list Linux-mediatek@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-mediatek From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EBA5C433DB for ; Wed, 24 Mar 2021 16:14:07 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0BD8061878 for ; Wed, 24 Mar 2021 16:14:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0BD8061878 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cU9yOLt9D38Zx4/HOmBxOW1QrOhDppRk+UqSE7m2n/E=; b=Ju68GqErrX+htIV5ILQj+n4wJ O/UVCrn4VjvCulLgzUZ9Jx+PMsVUWXH9ATLf7L6eiP1CtdRopDJH4EVZ1Lz3qfD9CMlPe9emZcVGI L/MHScPavod/kxFaQXR1uvDl6Isdab1ErK16uIOaVZZy1Iw9NkfZFfIpu1NSJiidd4VN08nP13x0N Hja6Cq1oIEAuBvv5EQ5rn5hSQAlWOLgbbyQRC4XH6U3SvOfAeWgVqGhdhuCftnPXY+xkAyytTL+bl ZZL/tNKSxH2kqoKz/LlXgoEDAYTynDLUlnZhUcDPBdzJULO6dTsEFylfF/tU3yHxmeeTf4bHLTgf1 0Bj/vBN8g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lP678-00HM1h-5s; Wed, 24 Mar 2021 16:12:10 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lP671-00HM0N-1a; Wed, 24 Mar 2021 16:12:05 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 95F0061A01; Wed, 24 Mar 2021 16:11:59 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lP66v-003YPB-O4; Wed, 24 Mar 2021 16:11:57 +0000 Date: Wed, 24 Mar 2021 16:11:56 +0000 Message-ID: <8735wkjzub.wl-maz@kernel.org> From: Marc Zyngier To: Lorenzo Pieralisi Cc: Robin Murphy , Bjorn Helgaas , Frank Wunderlich , Thierry Reding , Thomas Gleixner , Rob Herring , Will Deacon , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Michael Kelley , Wei Liu , Thierry Reding , Jonathan Hunter , Ryder Lee , Marek Vasut , Yoshihiro Shimoda , Michal Simek , Paul Walmsley , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH v2 12/15] PCI/MSI: Let PCI host bridges declare their reliance on MSI domains In-Reply-To: <20210324131938.GA16722@e121166-lin.cambridge.arm.com> References: <20210322184614.802565-1-maz@kernel.org> <20210322184614.802565-13-maz@kernel.org> <6a2eaa5d-1d83-159f-69e5-c9e0a00a7b50@arm.com> <87im5hkahr.wl-maz@kernel.org> <20210324131938.GA16722@e121166-lin.cambridge.arm.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: lorenzo.pieralisi@arm.com, robin.murphy@arm.com, bhelgaas@google.com, frank-w@public-files.de, treding@nvidia.com, tglx@linutronix.de, robh@kernel.org, will@kernel.org, kys@microsoft.com, haiyangz@microsoft.com, sthemmin@microsoft.com, mikelley@microsoft.com, wei.liu@kernel.org, thierry.reding@gmail.com, jonathanh@nvidia.com, ryder.lee@mediatek.com, marek.vasut+renesas@gmail.com, yoshihiro.shimoda.uh@renesas.com, michal.simek@xilinx.com, paul.walmsley@sifive.com, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hyperv@vger.kernel.org, linux-tegra@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-renesas-soc@vger.kernel.org, kernel-team@android.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210324_161203_563047_157F2DEB X-CRM114-Status: GOOD ( 48.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 24 Mar 2021 13:19:38 +0000, Lorenzo Pieralisi wrote: > > On Tue, Mar 23, 2021 at 06:09:36PM +0000, Marc Zyngier wrote: > > Hi Robin, > > > > On Tue, 23 Mar 2021 11:45:02 +0000, > > Robin Murphy wrote: > > > > > > On 2021-03-22 18:46, Marc Zyngier wrote: > > > > The new 'no_msi' attribute solves the problem of advertising the lack > > > > of MSI capability for host bridges that know for sure that there will > > > > be no MSI for their end-points. > > > > > > > > However, there is a whole class of host bridges that cannot know > > > > whether MSIs will be provided or not, as they rely on other blocks > > > > to provide the MSI functionnality, using MSI domains. This is > > > > the case for example on systems that use the ARM GIC architecture. > > > > > > > > Introduce a new attribute ('msi_domain') indicating that implicit > > > > dependency, and use this property to set the NO_MSI flag when > > > > no MSI domain is found at probe time. > > > > > > > > Acked-by: Bjorn Helgaas > > > > Signed-off-by: Marc Zyngier > > > > --- > > > > drivers/pci/probe.c | 2 +- > > > > include/linux/pci.h | 1 + > > > > 2 files changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c > > > > index 146bd85c037e..bac9f69a06a8 100644 > > > > --- a/drivers/pci/probe.c > > > > +++ b/drivers/pci/probe.c > > > > @@ -925,7 +925,7 @@ static int pci_register_host_bridge(struct pci_host_bridge *bridge) > > > > device_enable_async_suspend(bus->bridge); > > > > pci_set_bus_of_node(bus); > > > > pci_set_bus_msi_domain(bus); > > > > - if (bridge->no_msi) > > > > + if (bridge->no_msi || (bridge->msi_domain && !bus->dev.msi_domain)) > > > > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > > > if (!parent) > > > > diff --git a/include/linux/pci.h b/include/linux/pci.h > > > > index 48605cca82ae..d322d00db432 100644 > > > > --- a/include/linux/pci.h > > > > +++ b/include/linux/pci.h > > > > @@ -551,6 +551,7 @@ struct pci_host_bridge { > > > > unsigned int preserve_config:1; /* Preserve FW resource setup */ > > > > unsigned int size_windows:1; /* Enable root bus sizing */ > > > > unsigned int no_msi:1; /* Bridge has no MSI support */ > > > > + unsigned int msi_domain:1; /* Bridge wants MSI domain */ > > > > > > Aren't these really the same thing? Either way we're saying the bridge > > > itself doesn't handle MSIs, it's just in one case we're effectively > > > encoding a platform-specific assumption that an external domain won't > > > be provided. I can't help wondering whether that distinction is really > > > necessary... > > > > There is a subtle difference: no_msi indicates that there is no way > > *any* MSI can be dealt with whatsoever (maybe because the RC doesn't > > forward the corresponding TLPs?). msi_domain says "no MSI unless...". > > > > We could implement the former with the latter, but I have the feeling > > that's not totally bullet proof. Happy to revisit this if you think it > > really matters. > > IIUC msi_domain == 1 means: this host bridge needs an msi_domain to enable > MSIs, which in turn means that there are bridges that do _not_ require > an msi_domain to enable MSIs. I don't know how other arches handle the > msi_domain pointer but I am asking whether making: > > if (bridge->no_msi || !bus->dev.msi_domain)) > bus->bus_flags |= PCI_BUS_FLAGS_NO_MSI; > > is a possibility (removing the need for the msi_domain flag). > > At least this looks more like an arch property than a host bridge > specific property (eg patch [13] pci_host_common_probe() may be used on > arches other than ARM where it is not necessary true that it requires an > msi_domain to enable MSIs). > > I agree that's complicated to untangle - just asking if there is way > to simplify it. I tried to simplify that in the past (see the original discussion at [1]), and tglx reported some breakages on systems that do not use MSI domains, which is why we ended up with an explicit flag. What I have done for now is to go with Robin's proposal of dropping 'no_msi' and rely on solely on 'msi_domain' to set PCI_BUS_FLAGS_NO_MSI when no domain is found. Note that if we indeed have a host bridge that uses pci_host_common_probe() that doesn't use MSI domains, we may indeed run into problems. I don't have a good way around that, unfortunately. Thanks, M. [1] https://lore.kernel.org/r/20201031140330.83768-1-linux@fw-web.de -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel