Re: Thunderbolt, direct-complete and long suspend/resume time of Suspend-to-idle

From: Mika Westerberg <mika.westerberg@linux.intel.com>
To: Kai-Heng Feng <kai.heng.feng@canonical.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>,
	"Shih-Yuan Lee (FourDollars)" <sylee@canonical.com>,
	Tiffany <tiffany.wang@canonical.com>,
	Linux PCI <linux-pci@vger.kernel.org>,
	Linux PM <linux-pm@vger.kernel.org>,
	open list <linux-kernel@vger.kernel.org>
Subject: Re: Thunderbolt, direct-complete and long suspend/resume time of Suspend-to-idle
Date: Wed, 11 Mar 2020 12:38:40 +0200	[thread overview]
Message-ID: <20200311103840.GB2540@lahna.fi.intel.com> (raw)
In-Reply-To: <02700895-048F-4EA1-9E18-4883E83AE210@canonical.com>

On Wed, Mar 11, 2020 at 01:39:51PM +0800, Kai-Heng Feng wrote:
> Hi,
> 
> I am currently investigating long suspend and resume time of suspend-to-idle.
> It's because Thunderbolt bridges need to wait for 1100ms [1] for runtime-resume on system suspend, and also for system resume.
> 
> I made a quick hack to the USB driver and xHCI driver to support direct-complete, but I failed to do so for the parent PCIe bridge as it always disables the direct-complete [2], since device_may_wakeup() returns true for the device:
> 
> 	/* Avoid direct_complete to let wakeup_path propagate. */
> 		if (device_may_wakeup(dev) || dev->power.wakeup_path)
> 			dev->power.direct_complete = false;

You need to be careful here because otherwise you end up situation where
the link is not properly trained and we tear down the whole tree of
devices which is worse than waiting bit more for resume.

> Once the direct-complete is disabled, system suspend/resume is used hence the delay in [1] is making the resume really slow. 
> So how do we make suspend-to-idle faster? I have some ideas but I am not sure if they are feasible:
> - Make PM core know the runtime_suspend() already use the same wakeup as suspend(), so it doesn't need to use device_may_wakeup() check to determine direct-complete.
> - Remove the DPM_FLAG_NEVER_SKIP flag in pcieport driver, and use pm_request_resume() in its complete() callback to prevent blocking the resume process.
> - Reduce the 1100ms delay. Maybe someone knows the values used in macOS and Windows...

Which system this is? ICL? I think it is the TBT root ports only that do
not support active link reporting. The PCIe spec is not entirely clear
about root ports since it explictly mentions only downstream ports so
one option would be to check for root port and that it supports gen 3
speeds and based on that wait for max say 2 * 100ms or something like
that.