From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0F11C433F8 for ; Mon, 27 Jul 2020 14:14:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B23DD21883 for ; Mon, 27 Jul 2020 14:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1595859284; bh=hqGjzG7kcr/MsbjRxsV9hAB3/B1VJxy+zFkGz8jLWdc=; h=Date:From:To:Cc:Subject:In-Reply-To:List-ID:From; b=GK1bfb8wgVyUSD7DNG0ZUdQ7xBxzYQCvrAFlJ8Z7EtnTcJzBijMkneddUeSCC8yFF VlsYBMoyRNMBpn+rIJUBWUZeaTgHMg+XJcZob+tnvMgdIs+iBI/QtV8HKT/wvCD//w KJKtc1CTnwAdZvOAhZ8DeEqXsq3Xr8F4V9MfmxJU= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730586AbgG0OOm (ORCPT ); Mon, 27 Jul 2020 10:14:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:40504 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730582AbgG0OOl (ORCPT ); Mon, 27 Jul 2020 10:14:41 -0400 Received: from localhost (mobile-166-175-191-139.mycingular.net [166.175.191.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 82D1F2173E; Mon, 27 Jul 2020 14:14:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1595859280; bh=hqGjzG7kcr/MsbjRxsV9hAB3/B1VJxy+zFkGz8jLWdc=; h=Date:From:To:Cc:Subject:In-Reply-To:From; b=gVWT8qiP8B2d3sTJWw8jQuwCKb3Ijd21gSw5+UBWrb5jM7RD83N2Xm6QSWdZUsPF4 eGVxErBhMwzWE7lKlM103qKgBzyjtjdGMAgnxaU3R+cp4PsPNU3lmozQjqz094GyZx 2MntVodiw2oVVzOD8/Rq62m6IKmhl5bzXm6+YpII= Date: Mon, 27 Jul 2020 09:14:38 -0500 From: Bjorn Helgaas To: James Ettle Cc: =?utf-8?B?5ZCz5piK5r6E?= Ricky , Rui Feng , Arnd Bergmann , Greg Kroah-Hartman , Len Brown , Puranjay Mohan , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Jacopo De Simoi Subject: Re: rtsx_pci not restoring ASPM state after suspend/resume Message-ID: <20200727141438.GA1743062@bjorn-Precision-5520> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org On Sat, Jul 25, 2020 at 09:27:11PM +0100, James Ettle wrote: > On Fri, 2020-07-24 at 18:13 -0500, Bjorn Helgaas wrote: > > > > Maybe we should simplify this a little bit more. James, if you don't > > touch ASPM config at all, either manually or via udev, does the ASPM > > configuration stay the same across suspend/resume? > > Yes, it stays the same. Explicitly: > > With the udev rule disabled, immediately following clean boot from > power-off (and no additional tinkering), ASPM is OFF to the best of my > knowledge: > > - link/l1_aspm in sysfs is 0 for PCI devices 0000:01:00.[01]; > - the processor sleeps no deeper than package C3. > > The situation above is the same following a suspend/resume cycle -- > both in terms of sysfs, and observed package C-state occupancy. > > [Tested on kernel 5.7.10, but the behaviour is the same as prior > kernels.] I don't know the connection between ASPM and package C-states, so I need to simplify this even more. All I want to do right now is verify that if we don't have any outside influences on the ASPM configuration (eg, no manual changes and no udev rules), it stays the same across suspend/resume. In https://bugzilla.kernel.org/show_bug.cgi?id=208117#c12, we saw that ASPM L0s was disabled before suspend but was enabled after resume. That should not happen. You're looking at the sysfs link/l1_aspm file, which tells us what the PCI core thinks the state is, but I'm not confident that's accurate, especially because the driver fiddles with the state behind the back of the PCI core. So let's read the ASPM state directly from the hardware like this: sudo lspci -vvs 00:1d.0 | egrep "^0|Lnk|L1|LTR|snoop" sudo lspci -vvs 01:00 | egrep "^0|Lnk|L1|LTR|snoop" Can you try that before and after suspend/resume?