From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C92C6C433E3 for ; Wed, 15 Jul 2020 16:21:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B11212065E for ; Wed, 15 Jul 2020 16:21:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726768AbgGOQVN (ORCPT ); Wed, 15 Jul 2020 12:21:13 -0400 Received: from foss.arm.com ([217.140.110.172]:54384 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725792AbgGOQVN (ORCPT ); Wed, 15 Jul 2020 12:21:13 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F3E531B; Wed, 15 Jul 2020 09:21:12 -0700 (PDT) Received: from e121166-lin.cambridge.arm.com (e121166-lin.cambridge.arm.com [10.1.196.255]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 119153F718; Wed, 15 Jul 2020 09:21:10 -0700 (PDT) Date: Wed, 15 Jul 2020 17:21:08 +0100 From: Lorenzo Pieralisi To: Pali =?iso-8859-1?Q?Roh=E1r?= Cc: Thomas Petazzoni , Andrew Murray , Bjorn Helgaas , Marek =?iso-8859-1?Q?Beh=FAn?= , Remi Pommarel , Tomasz Maciej Nowak , Xogium , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3] PCI: aardvark: Don't touch PCIe registers if no card connected Message-ID: <20200715162108.GB3432@e121166-lin.cambridge.arm.com> References: <20200528143141.29956-1-pali@kernel.org> <20200702083036.12230-1-pali@kernel.org> <20200709113509.GB19638@e121166-lin.cambridge.arm.com> <20200709122208.rmfeuu6zgbwh3fr5@pali> <20200709144701.GA21760@e121166-lin.cambridge.arm.com> <20200709150959.wq6zfkcy4m6hvvpl@pali> <20200710091800.GA3419@e121166-lin.cambridge.arm.com> <20200713082747.e3q3ml3wpbszn4j7@pali> <20200713112325.GA25865@e121166-lin.cambridge.arm.com> <20200715121726.eh4xglkdbcqkh7td@pali> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200715121726.eh4xglkdbcqkh7td@pali> User-Agent: Mutt/1.9.4 (2018-02-28) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 15, 2020 at 02:17:26PM +0200, Pali Rohár wrote: > On Monday 13 July 2020 12:23:25 Lorenzo Pieralisi wrote: > > On Mon, Jul 13, 2020 at 10:27:47AM +0200, Pali Rohár wrote: > > > On Friday 10 July 2020 10:18:00 Lorenzo Pieralisi wrote: > > > > On Thu, Jul 09, 2020 at 05:09:59PM +0200, Pali Rohár wrote: > > > > > > I understand that but the bridge bus resource can be trimmed to just > > > > > > contain the root bus because that's the only one where there is a > > > > > > chance you can enumerate a device. > > > > > > > > > > It is possible to register only root bridge without endpoint? > > > > > > > > It is possible to register the root bridge with a trimmed IORESOURCE_BUS > > > > so that you don't enumerate anything other than the root port. > > > > > > Hello Lorenzo! I really do not know how to achieve it. From code it > > > looks like that pci/probe.c scans child buses unconditionally. > > > > > > pci-aardvark.c calls pci_host_probe() which calls functions > > > pci_scan_root_bus_bridge() which calls pci_scan_child_bus() which calls > > > pci_scan_child_bus_extend() which calls pci_scan_bridge_extend() (bridge > > > needs to be reconfigured) which then try to probe child bus via > > > pci_scan_child_bus_extend() because bridge is not card bus. > > > > > > In function pci_scan_bridge_extend() I do not see a way how to skip > > > probing for child buses which would avoid enumerating aardvark root > > > bridge when PCIe device is not connected. > > > > > > dmesg output contains: > > > > > > advk-pcie d0070000.pcie: link never came up > > > advk-pcie d0070000.pcie: PCI host bridge to bus 0000:00 > > > pci_bus 0000:00: root bus resource [bus 00-ff] > > > > This resource can be limited to the root bus number only before calling > > pci_host_probe() (ie see pci_parse_request_of_pci_ranges() and code in > > pci_scan_bridge_extend() that programs primary/secondary/subordinate > > busses) but I think that only papers over the issue, it does not fix it. > > I looked at the code in pci/probe.c again and I do not think it is > possible to avoid scanning devices. pci_scan_child_bus_extend() is > unconditionally calling pci_scan_slot() for devfn=0 as the first thing. > And this function unconditionally calls pci_scan_device() which is > directly trying to read vendor id from config register. > > So for me it looks like that kernel expects that can read vendor id and > device id from config register for device which is not connected. Not if it is connected to a bus that the root port does not decode, that's what I am saying. > And trying to read config register would cause those timeouts in > aardvark. The root port (which effectively works as PCI bridge from this standpoint) does not issue config cycles for busses that aren't within its decoded bus range, which in turn is determined by the firmware IORESOURCE_BUS resource. This issue is caused by devices that are connected downstream to the root port. Anyway - patch merged but I would be happy to keep this discussion going, somehow. If the LPC20 VFIO/IOMMU/PCI microconference is approved it can be a good venue for this to happen. Lorenzo From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C6F6C433E1 for ; Wed, 15 Jul 2020 16:22:33 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 177382065E for ; Wed, 15 Jul 2020 16:22:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="GFD+0D6/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 177382065E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=82slw8mhEKa6PPT18Npz8i+0svXye2h4lD6P44wAt7U=; b=GFD+0D6/8a1+eAw3n7oit2yvy XQ5SWWCYPmW0u2IMK2Ma0i7uTvP5v39R2AGHcBfpSjJZWkDmL/yAWxsvmUgvWLuloyKNuk/kUBM3z Ua1ICLnR0BsqYjoWtq9bKgsAa43saZegwxiJY5yND5a9QHAb5ewjMRzMafx7e2sHNl8B7ak2GmSws Td0IFWRx8c7ynxaSlvOU9kuBjMmvnnAn2qSPzDdUShQOAdccoAcngktrq3WtPQ+M9uFp8LYbHLROg bhcudHsP5Sxkfg7uCyZ1ekXTMYv7SbTBeSw4ZGBiB4GUwcEcgIZ8BnNQ2vS2Ig+GVRyklTKETsmCf Y3y2LWYWQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jvk9k-0002CB-F8; Wed, 15 Jul 2020 16:21:16 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jvk9i-0002Az-CZ for linux-arm-kernel@lists.infradead.org; Wed, 15 Jul 2020 16:21:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F3E531B; Wed, 15 Jul 2020 09:21:12 -0700 (PDT) Received: from e121166-lin.cambridge.arm.com (e121166-lin.cambridge.arm.com [10.1.196.255]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 119153F718; Wed, 15 Jul 2020 09:21:10 -0700 (PDT) Date: Wed, 15 Jul 2020 17:21:08 +0100 From: Lorenzo Pieralisi To: Pali =?iso-8859-1?Q?Roh=E1r?= Subject: Re: [PATCH v3] PCI: aardvark: Don't touch PCIe registers if no card connected Message-ID: <20200715162108.GB3432@e121166-lin.cambridge.arm.com> References: <20200528143141.29956-1-pali@kernel.org> <20200702083036.12230-1-pali@kernel.org> <20200709113509.GB19638@e121166-lin.cambridge.arm.com> <20200709122208.rmfeuu6zgbwh3fr5@pali> <20200709144701.GA21760@e121166-lin.cambridge.arm.com> <20200709150959.wq6zfkcy4m6hvvpl@pali> <20200710091800.GA3419@e121166-lin.cambridge.arm.com> <20200713082747.e3q3ml3wpbszn4j7@pali> <20200713112325.GA25865@e121166-lin.cambridge.arm.com> <20200715121726.eh4xglkdbcqkh7td@pali> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200715121726.eh4xglkdbcqkh7td@pali> User-Agent: Mutt/1.9.4 (2018-02-28) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200715_122114_551616_AD56EE49 X-CRM114-Status: GOOD ( 28.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tomasz Maciej Nowak , linux-pci@vger.kernel.org, Xogium , linux-kernel@vger.kernel.org, Marek =?iso-8859-1?Q?Beh=FAn?= , Remi Pommarel , Thomas Petazzoni , Bjorn Helgaas , linux-arm-kernel@lists.infradead.org, Andrew Murray Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Jul 15, 2020 at 02:17:26PM +0200, Pali Roh=E1r wrote: > On Monday 13 July 2020 12:23:25 Lorenzo Pieralisi wrote: > > On Mon, Jul 13, 2020 at 10:27:47AM +0200, Pali Roh=E1r wrote: > > > On Friday 10 July 2020 10:18:00 Lorenzo Pieralisi wrote: > > > > On Thu, Jul 09, 2020 at 05:09:59PM +0200, Pali Roh=E1r wrote: > > > > > > I understand that but the bridge bus resource can be trimmed to= just > > > > > > contain the root bus because that's the only one where there is= a > > > > > > chance you can enumerate a device. > > > > > = > > > > > It is possible to register only root bridge without endpoint? > > > > = > > > > It is possible to register the root bridge with a trimmed IORESOURC= E_BUS > > > > so that you don't enumerate anything other than the root port. > > > = > > > Hello Lorenzo! I really do not know how to achieve it. From code it > > > looks like that pci/probe.c scans child buses unconditionally. > > > = > > > pci-aardvark.c calls pci_host_probe() which calls functions > > > pci_scan_root_bus_bridge() which calls pci_scan_child_bus() which cal= ls > > > pci_scan_child_bus_extend() which calls pci_scan_bridge_extend() (bri= dge > > > needs to be reconfigured) which then try to probe child bus via > > > pci_scan_child_bus_extend() because bridge is not card bus. > > > = > > > In function pci_scan_bridge_extend() I do not see a way how to skip > > > probing for child buses which would avoid enumerating aardvark root > > > bridge when PCIe device is not connected. > > > = > > > dmesg output contains: > > > = > > > advk-pcie d0070000.pcie: link never came up > > > advk-pcie d0070000.pcie: PCI host bridge to bus 0000:00 > > > pci_bus 0000:00: root bus resource [bus 00-ff] > > = > > This resource can be limited to the root bus number only before calling > > pci_host_probe() (ie see pci_parse_request_of_pci_ranges() and code in > > pci_scan_bridge_extend() that programs primary/secondary/subordinate > > busses) but I think that only papers over the issue, it does not fix it. > = > I looked at the code in pci/probe.c again and I do not think it is > possible to avoid scanning devices. pci_scan_child_bus_extend() is > unconditionally calling pci_scan_slot() for devfn=3D0 as the first thing. > And this function unconditionally calls pci_scan_device() which is > directly trying to read vendor id from config register. > = > So for me it looks like that kernel expects that can read vendor id and > device id from config register for device which is not connected. Not if it is connected to a bus that the root port does not decode, that's what I am saying. > And trying to read config register would cause those timeouts in > aardvark. The root port (which effectively works as PCI bridge from this standpoint) does not issue config cycles for busses that aren't within its decoded bus range, which in turn is determined by the firmware IORESOURCE_BUS resource. This issue is caused by devices that are connected downstream to the root port. Anyway - patch merged but I would be happy to keep this discussion going, somehow. If the LPC20 VFIO/IOMMU/PCI microconference is approved it can be a good venue for this to happen. Lorenzo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel