From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752178AbcGTGZD (ORCPT ); Wed, 20 Jul 2016 02:25:03 -0400 Received: from mailout3.hostsharing.net ([176.9.242.54]:57129 "EHLO mailout3.hostsharing.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751195AbcGTGYo (ORCPT ); Wed, 20 Jul 2016 02:24:44 -0400 Date: Wed, 20 Jul 2016 08:24:50 +0200 From: Lukas Wunner To: "Rafael J. Wysocki" Cc: Marek Szyprowski , "linux-pm@vger.kernel.org" , Linux Kernel Mailing List , "open list:AMD IOMMU (AMD-VI)" , linux-samsung-soc@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , Joerg Roedel , Inki Dae , Kukjin Kim , Krzysztof Kozlowski , Bartlomiej Zolnierkiewicz , Ulf Hansson , Mark Brown , Greg Kroah-Hartman , Andreas Noever , linux-pci@vger.kernel.org Subject: Re: [PATCH v2 02/10] driver core: Functional dependencies tracking support Message-ID: <20160720062450.GA8066@wunner.de> References: <1466144820-6286-1-git-send-email-m.szyprowski@samsung.com> <20160617140738.GA1809@wunner.de> <1975118.iO8UCAENoH@vostro.rjw.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1975118.iO8UCAENoH@vostro.rjw.lan> User-Agent: Mutt/1.6.1 (2016-04-27) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 20, 2016 at 02:33:18AM +0200, Rafael J. Wysocki wrote: > On Friday, June 17, 2016 04:07:38 PM Lukas Wunner wrote: > > On Fri, Jun 17, 2016 at 02:54:56PM +0200, Rafael J. Wysocki wrote: > > > On Fri, Jun 17, 2016 at 12:36 PM, Lukas Wunner wrote: > > > > On Fri, Jun 17, 2016 at 08:26:52AM +0200, Marek Szyprowski wrote: > > > > > From: "Rafael J. Wysocki" > > > > We also have such a functional dependency for Thunderbolt on Macs: > > > > On resume from system sleep, the PCIe hotplug ports may not resume > > > > before the thunderbolt driver has reestablished the PCI tunnels. > > > > Currently this is enforced by quirk_apple_wait_for_thunderbolt() > > > > in drivers/pci/quirks.c. It would be good if we could represent > > > > this dependency using something like Rafael's approach instead of > > > > open coding it, however one detail in Rafael's patches is problematic: > > > > > > > > > New links are added by calling device_link_add() which may happen > > > > > either before the consumer device is probed or when probing it, in > > > > > which case the caller needs to ensure that the driver of the > > > > > supplier device is present and functional and the DEVICE_LINK_PROBE_TIME > > > > > flag should be passed to device_link_add() to reflect that. > > > > > > > > The thunderbolt driver cannot call device_link_add() before the > > > > PCIe hotplug ports are bound to a driver unless we amend portdrv > > > > to return -EPROBE_DEFER for Thunderbolt hotplug ports on Macs > > > > if the thunderbolt driver isn't loaded. > > > > > > > > It would therefore be beneficial if device_link_add() can be > > > > called even *after* the consumer is bound. > > > > > > I don't quite follow. > > > > > > Who's the provider and who's the consumer here? > > > > thunderbolt.ko is the supplier. > > But it binds to the children of the ports that are supposed to be its > consumers? > > Why is that even expected to work? No, the consumers are aunts (or uncles) of the supplier, if you will. :-) The consumers are the hotplug ports (named "Downstream Bridge 1 / 2" in the drawing below). The supplier is the NHI: (Root Port) ---- Upstream Bridge --+-- Downstream Bridge 0 ---- NHI +-- Downstream Bridge 1 -- +-- Downstream Bridge 2 -- ... We're calling pci_power_up() and pci_restore_state() from pci_pm_resume_noirq(). And that will fail for devices below the hotplug ports if the PCI tunnels haven't been re-established yet by the NHI. Currently we achieve that via quirk_apple_wait_for_thunderbolt() in drivers/pci/quirks.c. It would be more elegant if we could make this relationship explicit with "device links" and let the core handle it. Or am I mistaken and this particular use case is not what "device links" are intended for? Thanks, Lukas From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout3.hostsharing.net ([176.9.242.54]:57129 "EHLO mailout3.hostsharing.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751195AbcGTGYo (ORCPT ); Wed, 20 Jul 2016 02:24:44 -0400 Date: Wed, 20 Jul 2016 08:24:50 +0200 From: Lukas Wunner To: "Rafael J. Wysocki" Cc: Marek Szyprowski , "linux-pm@vger.kernel.org" , Linux Kernel Mailing List , "open list:AMD IOMMU (AMD-VI)" , linux-samsung-soc@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , Joerg Roedel , Inki Dae , Kukjin Kim , Krzysztof Kozlowski , Bartlomiej Zolnierkiewicz , Ulf Hansson , Mark Brown , Greg Kroah-Hartman , Andreas Noever , linux-pci@vger.kernel.org Subject: Re: [PATCH v2 02/10] driver core: Functional dependencies tracking support Message-ID: <20160720062450.GA8066@wunner.de> References: <1466144820-6286-1-git-send-email-m.szyprowski@samsung.com> <20160617140738.GA1809@wunner.de> <1975118.iO8UCAENoH@vostro.rjw.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <1975118.iO8UCAENoH@vostro.rjw.lan> Sender: linux-pci-owner@vger.kernel.org List-ID: On Wed, Jul 20, 2016 at 02:33:18AM +0200, Rafael J. Wysocki wrote: > On Friday, June 17, 2016 04:07:38 PM Lukas Wunner wrote: > > On Fri, Jun 17, 2016 at 02:54:56PM +0200, Rafael J. Wysocki wrote: > > > On Fri, Jun 17, 2016 at 12:36 PM, Lukas Wunner wrote: > > > > On Fri, Jun 17, 2016 at 08:26:52AM +0200, Marek Szyprowski wrote: > > > > > From: "Rafael J. Wysocki" > > > > We also have such a functional dependency for Thunderbolt on Macs: > > > > On resume from system sleep, the PCIe hotplug ports may not resume > > > > before the thunderbolt driver has reestablished the PCI tunnels. > > > > Currently this is enforced by quirk_apple_wait_for_thunderbolt() > > > > in drivers/pci/quirks.c. It would be good if we could represent > > > > this dependency using something like Rafael's approach instead of > > > > open coding it, however one detail in Rafael's patches is problematic: > > > > > > > > > New links are added by calling device_link_add() which may happen > > > > > either before the consumer device is probed or when probing it, in > > > > > which case the caller needs to ensure that the driver of the > > > > > supplier device is present and functional and the DEVICE_LINK_PROBE_TIME > > > > > flag should be passed to device_link_add() to reflect that. > > > > > > > > The thunderbolt driver cannot call device_link_add() before the > > > > PCIe hotplug ports are bound to a driver unless we amend portdrv > > > > to return -EPROBE_DEFER for Thunderbolt hotplug ports on Macs > > > > if the thunderbolt driver isn't loaded. > > > > > > > > It would therefore be beneficial if device_link_add() can be > > > > called even *after* the consumer is bound. > > > > > > I don't quite follow. > > > > > > Who's the provider and who's the consumer here? > > > > thunderbolt.ko is the supplier. > > But it binds to the children of the ports that are supposed to be its > consumers? > > Why is that even expected to work? No, the consumers are aunts (or uncles) of the supplier, if you will. :-) The consumers are the hotplug ports (named "Downstream Bridge 1 / 2" in the drawing below). The supplier is the NHI: (Root Port) ---- Upstream Bridge --+-- Downstream Bridge 0 ---- NHI +-- Downstream Bridge 1 -- +-- Downstream Bridge 2 -- ... We're calling pci_power_up() and pci_restore_state() from pci_pm_resume_noirq(). And that will fail for devices below the hotplug ports if the PCI tunnels haven't been re-established yet by the NHI. Currently we achieve that via quirk_apple_wait_for_thunderbolt() in drivers/pci/quirks.c. It would be more elegant if we could make this relationship explicit with "device links" and let the core handle it. Or am I mistaken and this particular use case is not what "device links" are intended for? Thanks, Lukas From mboxrd@z Thu Jan 1 00:00:00 1970 From: Lukas Wunner Subject: Re: [PATCH v2 02/10] driver core: Functional dependencies tracking support Date: Wed, 20 Jul 2016 08:24:50 +0200 Message-ID: <20160720062450.GA8066@wunner.de> References: <1466144820-6286-1-git-send-email-m.szyprowski@samsung.com> <20160617140738.GA1809@wunner.de> <1975118.iO8UCAENoH@vostro.rjw.lan> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <1975118.iO8UCAENoH-sKB8Sp2ER+y1GS7QM15AGw@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: "Rafael J. Wysocki" Cc: Krzysztof Kozlowski , linux-samsung-soc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, "linux-pm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , Greg Kroah-Hartman , Bartlomiej Zolnierkiewicz , Linux Kernel Mailing List , Inki Dae , "open list:AMD IOMMU (AMD-VI)" , Kukjin Kim , Mark Brown , linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Andreas Noever , Ulf Hansson , "linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org" List-Id: linux-pm@vger.kernel.org On Wed, Jul 20, 2016 at 02:33:18AM +0200, Rafael J. Wysocki wrote: > On Friday, June 17, 2016 04:07:38 PM Lukas Wunner wrote: > > On Fri, Jun 17, 2016 at 02:54:56PM +0200, Rafael J. Wysocki wrote: > > > On Fri, Jun 17, 2016 at 12:36 PM, Lukas Wunner wrote: > > > > On Fri, Jun 17, 2016 at 08:26:52AM +0200, Marek Szyprowski wrote: > > > > > From: "Rafael J. Wysocki" > > > > We also have such a functional dependency for Thunderbolt on Macs: > > > > On resume from system sleep, the PCIe hotplug ports may not resume > > > > before the thunderbolt driver has reestablished the PCI tunnels. > > > > Currently this is enforced by quirk_apple_wait_for_thunderbolt() > > > > in drivers/pci/quirks.c. It would be good if we could represent > > > > this dependency using something like Rafael's approach instead of > > > > open coding it, however one detail in Rafael's patches is problematic: > > > > > > > > > New links are added by calling device_link_add() which may happen > > > > > either before the consumer device is probed or when probing it, in > > > > > which case the caller needs to ensure that the driver of the > > > > > supplier device is present and functional and the DEVICE_LINK_PROBE_TIME > > > > > flag should be passed to device_link_add() to reflect that. > > > > > > > > The thunderbolt driver cannot call device_link_add() before the > > > > PCIe hotplug ports are bound to a driver unless we amend portdrv > > > > to return -EPROBE_DEFER for Thunderbolt hotplug ports on Macs > > > > if the thunderbolt driver isn't loaded. > > > > > > > > It would therefore be beneficial if device_link_add() can be > > > > called even *after* the consumer is bound. > > > > > > I don't quite follow. > > > > > > Who's the provider and who's the consumer here? > > > > thunderbolt.ko is the supplier. > > But it binds to the children of the ports that are supposed to be its > consumers? > > Why is that even expected to work? No, the consumers are aunts (or uncles) of the supplier, if you will. :-) The consumers are the hotplug ports (named "Downstream Bridge 1 / 2" in the drawing below). The supplier is the NHI: (Root Port) ---- Upstream Bridge --+-- Downstream Bridge 0 ---- NHI +-- Downstream Bridge 1 -- +-- Downstream Bridge 2 -- ... We're calling pci_power_up() and pci_restore_state() from pci_pm_resume_noirq(). And that will fail for devices below the hotplug ports if the PCI tunnels haven't been re-established yet by the NHI. Currently we achieve that via quirk_apple_wait_for_thunderbolt() in drivers/pci/quirks.c. It would be more elegant if we could make this relationship explicit with "device links" and let the core handle it. Or am I mistaken and this particular use case is not what "device links" are intended for? Thanks, Lukas From mboxrd@z Thu Jan 1 00:00:00 1970 From: lukas@wunner.de (Lukas Wunner) Date: Wed, 20 Jul 2016 08:24:50 +0200 Subject: [PATCH v2 02/10] driver core: Functional dependencies tracking support In-Reply-To: <1975118.iO8UCAENoH@vostro.rjw.lan> References: <1466144820-6286-1-git-send-email-m.szyprowski@samsung.com> <20160617140738.GA1809@wunner.de> <1975118.iO8UCAENoH@vostro.rjw.lan> Message-ID: <20160720062450.GA8066@wunner.de> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On Wed, Jul 20, 2016 at 02:33:18AM +0200, Rafael J. Wysocki wrote: > On Friday, June 17, 2016 04:07:38 PM Lukas Wunner wrote: > > On Fri, Jun 17, 2016 at 02:54:56PM +0200, Rafael J. Wysocki wrote: > > > On Fri, Jun 17, 2016 at 12:36 PM, Lukas Wunner wrote: > > > > On Fri, Jun 17, 2016 at 08:26:52AM +0200, Marek Szyprowski wrote: > > > > > From: "Rafael J. Wysocki" > > > > We also have such a functional dependency for Thunderbolt on Macs: > > > > On resume from system sleep, the PCIe hotplug ports may not resume > > > > before the thunderbolt driver has reestablished the PCI tunnels. > > > > Currently this is enforced by quirk_apple_wait_for_thunderbolt() > > > > in drivers/pci/quirks.c. It would be good if we could represent > > > > this dependency using something like Rafael's approach instead of > > > > open coding it, however one detail in Rafael's patches is problematic: > > > > > > > > > New links are added by calling device_link_add() which may happen > > > > > either before the consumer device is probed or when probing it, in > > > > > which case the caller needs to ensure that the driver of the > > > > > supplier device is present and functional and the DEVICE_LINK_PROBE_TIME > > > > > flag should be passed to device_link_add() to reflect that. > > > > > > > > The thunderbolt driver cannot call device_link_add() before the > > > > PCIe hotplug ports are bound to a driver unless we amend portdrv > > > > to return -EPROBE_DEFER for Thunderbolt hotplug ports on Macs > > > > if the thunderbolt driver isn't loaded. > > > > > > > > It would therefore be beneficial if device_link_add() can be > > > > called even *after* the consumer is bound. > > > > > > I don't quite follow. > > > > > > Who's the provider and who's the consumer here? > > > > thunderbolt.ko is the supplier. > > But it binds to the children of the ports that are supposed to be its > consumers? > > Why is that even expected to work? No, the consumers are aunts (or uncles) of the supplier, if you will. :-) The consumers are the hotplug ports (named "Downstream Bridge 1 / 2" in the drawing below). The supplier is the NHI: (Root Port) ---- Upstream Bridge --+-- Downstream Bridge 0 ---- NHI +-- Downstream Bridge 1 -- +-- Downstream Bridge 2 -- ... We're calling pci_power_up() and pci_restore_state() from pci_pm_resume_noirq(). And that will fail for devices below the hotplug ports if the PCI tunnels haven't been re-established yet by the NHI. Currently we achieve that via quirk_apple_wait_for_thunderbolt() in drivers/pci/quirks.c. It would be more elegant if we could make this relationship explicit with "device links" and let the core handle it. Or am I mistaken and this particular use case is not what "device links" are intended for? Thanks, Lukas