From: Keith Busch <kbusch@kernel.org> To: "Rafael J. Wysocki" <rjw@rjwysocki.net> Cc: Mario Limonciello <Mario.Limonciello@dell.com>, Kai-Heng Feng <kai.heng.feng@canonical.com>, "Busch, Keith" <keith.busch@intel.com>, Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>, linux-nvme <linux-nvme@lists.infradead.org>, Linux PM <linux-pm@vger.kernel.org>, Linux Kernel Mailing List <linux-kernel@vger.kernel.org>, Rajat Jain <rajatja@google.com>, Linux PCI <linux-pci@vger.kernel.org>, Bjorn Helgaas <helgaas@kernel.org> Subject: Re: [PATCH] nvme-pci: Do not prevent PCI bus-level PM from being used Date: Wed, 7 Aug 2019 08:37:34 -0600 [thread overview] Message-ID: <20190807143733.GA25621@localhost.localdomain> (raw) In-Reply-To: <1893355.EP2830DdO9@kreacher> On Wed, Aug 07, 2019 at 02:53:44AM -0700, Rafael J. Wysocki wrote: > From: Rafael J. Wysocki <rafael.j.wysocki@intel.com> > > One of the modifications made by commit d916b1be94b6 ("nvme-pci: use > host managed power state for suspend") was adding a pci_save_state() > call to nvme_suspend() in order to prevent the PCI bus-level PM from > being applied to the suspended NVMe devices, but if ASPM is not > enabled for the target NVMe device, that causes its PCIe link to stay > up and the platform may not be able to get into its optimum low-power > state because of that. > > For example, if ASPM is disabled for the NVMe drive (PC401 NVMe SK > hynix 256GB) in my Dell XPS13 9380, leaving it in D0 during > suspend-to-idle prevents the SoC from reaching package idle states > deeper than PC3, which is way insufficient for system suspend. > > To address this shortcoming, make nvme_suspend() check if ASPM is > enabled for the target device and fall back to full device shutdown > and PCI bus-level PM if that is not the case. > > Fixes: d916b1be94b6 ("nvme-pci: use host managed power state for suspend") > Link: https://lore.kernel.org/linux-pm/2763495.NmdaWeg79L@kreacher/T/#t > Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com> Thanks for tracking down the cause. Sounds like your earlier assumption on ASPM's involvement was spot on. > +/* > + * pcie_aspm_enabled - Return the mask of enabled ASPM link states. > + * @pci_device: Target device. > + */ > +u32 pcie_aspm_enabled(struct pci_dev *pci_device) > +{ > + struct pci_dev *bridge = pci_device->bus->self; You may want use pci_upstream_bridge() instead, just in case someone calls this on a virtual function's pci_dev. > + u32 aspm_enabled; > + > + mutex_lock(&aspm_lock); > + aspm_enabled = bridge->link_state ? bridge->link_state->aspm_enabled : 0; > + mutex_unlock(&aspm_lock); > + > + return aspm_enabled; > +}
WARNING: multiple messages have this Message-ID (diff)
From: kbusch@kernel.org (Keith Busch) Subject: [PATCH] nvme-pci: Do not prevent PCI bus-level PM from being used Date: Wed, 7 Aug 2019 08:37:34 -0600 [thread overview] Message-ID: <20190807143733.GA25621@localhost.localdomain> (raw) In-Reply-To: <1893355.EP2830DdO9@kreacher> On Wed, Aug 07, 2019@02:53:44AM -0700, Rafael J. Wysocki wrote: > From: Rafael J. Wysocki <rafael.j.wysocki at intel.com> > > One of the modifications made by commit d916b1be94b6 ("nvme-pci: use > host managed power state for suspend") was adding a pci_save_state() > call to nvme_suspend() in order to prevent the PCI bus-level PM from > being applied to the suspended NVMe devices, but if ASPM is not > enabled for the target NVMe device, that causes its PCIe link to stay > up and the platform may not be able to get into its optimum low-power > state because of that. > > For example, if ASPM is disabled for the NVMe drive (PC401 NVMe SK > hynix 256GB) in my Dell XPS13 9380, leaving it in D0 during > suspend-to-idle prevents the SoC from reaching package idle states > deeper than PC3, which is way insufficient for system suspend. > > To address this shortcoming, make nvme_suspend() check if ASPM is > enabled for the target device and fall back to full device shutdown > and PCI bus-level PM if that is not the case. > > Fixes: d916b1be94b6 ("nvme-pci: use host managed power state for suspend") > Link: https://lore.kernel.org/linux-pm/2763495.NmdaWeg79L at kreacher/T/#t > Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki at intel.com> Thanks for tracking down the cause. Sounds like your earlier assumption on ASPM's involvement was spot on. > +/* > + * pcie_aspm_enabled - Return the mask of enabled ASPM link states. > + * @pci_device: Target device. > + */ > +u32 pcie_aspm_enabled(struct pci_dev *pci_device) > +{ > + struct pci_dev *bridge = pci_device->bus->self; You may want use pci_upstream_bridge() instead, just in case someone calls this on a virtual function's pci_dev. > + u32 aspm_enabled; > + > + mutex_lock(&aspm_lock); > + aspm_enabled = bridge->link_state ? bridge->link_state->aspm_enabled : 0; > + mutex_unlock(&aspm_lock); > + > + return aspm_enabled; > +}
next prev parent reply other threads:[~2019-08-07 14:40 UTC|newest] Thread overview: 150+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-07-25 9:51 [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems Rafael J. Wysocki 2019-07-25 9:51 ` Rafael J. Wysocki 2019-07-25 14:02 ` Kai-Heng Feng 2019-07-25 14:02 ` Kai-Heng Feng 2019-07-25 16:23 ` Mario.Limonciello 2019-07-25 16:23 ` Mario.Limonciello 2019-07-25 17:03 ` Rafael J. Wysocki 2019-07-25 17:03 ` Rafael J. Wysocki 2019-07-25 17:23 ` Mario.Limonciello 2019-07-25 17:23 ` Mario.Limonciello 2019-07-25 18:20 ` Kai-Heng Feng 2019-07-25 18:20 ` Kai-Heng Feng 2019-07-25 19:09 ` Mario.Limonciello 2019-07-25 19:09 ` Mario.Limonciello 2019-07-30 10:45 ` Rafael J. Wysocki 2019-07-30 10:45 ` Rafael J. Wysocki 2019-07-30 14:41 ` Keith Busch 2019-07-30 14:41 ` Keith Busch 2019-07-30 17:14 ` Mario.Limonciello 2019-07-30 17:14 ` Mario.Limonciello 2019-07-30 18:50 ` Kai-Heng Feng 2019-07-30 18:50 ` Kai-Heng Feng 2019-07-30 19:19 ` Keith Busch 2019-07-30 19:19 ` Keith Busch 2019-07-30 21:05 ` Mario.Limonciello 2019-07-30 21:05 ` Mario.Limonciello 2019-07-30 21:31 ` Keith Busch 2019-07-30 21:31 ` Keith Busch 2019-07-31 21:25 ` Rafael J. Wysocki 2019-07-31 21:25 ` Rafael J. Wysocki 2019-07-31 22:19 ` Keith Busch 2019-07-31 22:19 ` Keith Busch 2019-07-31 22:33 ` Rafael J. Wysocki 2019-07-31 22:33 ` Rafael J. Wysocki 2019-08-01 9:05 ` Kai-Heng Feng 2019-08-01 9:05 ` Kai-Heng Feng 2019-08-01 17:29 ` Rafael J. Wysocki 2019-08-01 17:29 ` Rafael J. Wysocki 2019-08-01 19:05 ` Mario.Limonciello 2019-08-01 19:05 ` Mario.Limonciello 2019-08-01 22:26 ` Rafael J. Wysocki 2019-08-01 22:26 ` Rafael J. Wysocki 2019-08-02 10:55 ` Kai-Heng Feng 2019-08-02 10:55 ` Kai-Heng Feng 2019-08-02 11:04 ` Rafael J. Wysocki 2019-08-02 11:04 ` Rafael J. Wysocki 2019-08-05 19:13 ` Kai-Heng Feng 2019-08-05 19:13 ` Kai-Heng Feng 2019-08-05 21:28 ` Rafael J. Wysocki 2019-08-05 21:28 ` Rafael J. Wysocki 2019-08-06 14:02 ` Mario.Limonciello 2019-08-06 14:02 ` Mario.Limonciello 2019-08-06 15:00 ` Rafael J. Wysocki 2019-08-06 15:00 ` Rafael J. Wysocki 2019-08-07 10:29 ` Rafael J. Wysocki 2019-08-07 10:29 ` Rafael J. Wysocki 2019-08-01 20:22 ` Keith Busch 2019-08-01 20:22 ` Keith Busch 2019-08-07 9:48 ` Rafael J. Wysocki 2019-08-07 9:48 ` Rafael J. Wysocki 2019-08-07 10:45 ` Christoph Hellwig 2019-08-07 10:45 ` Christoph Hellwig 2019-08-07 10:54 ` Rafael J. Wysocki 2019-08-07 10:54 ` Rafael J. Wysocki 2019-08-07 9:53 ` [PATCH] nvme-pci: Do not prevent PCI bus-level PM from being used Rafael J. Wysocki 2019-08-07 9:53 ` Rafael J. Wysocki 2019-08-07 10:14 ` Rafael J. Wysocki 2019-08-07 10:14 ` Rafael J. Wysocki 2019-08-07 10:43 ` Christoph Hellwig 2019-08-07 10:43 ` Christoph Hellwig 2019-08-07 14:37 ` Keith Busch [this message] 2019-08-07 14:37 ` Keith Busch 2019-08-08 8:36 ` [PATCH] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki 2019-08-08 8:36 ` Rafael J. Wysocki 2019-08-08 8:48 ` Christoph Hellwig 2019-08-08 8:48 ` Christoph Hellwig 2019-08-08 9:06 ` Rafael J. Wysocki 2019-08-08 9:06 ` Rafael J. Wysocki 2019-08-08 10:03 ` [PATCH v2 0/2] " Rafael J. Wysocki 2019-08-08 10:03 ` Rafael J. Wysocki 2019-08-08 10:06 ` [PATCH v2 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled_mask() Rafael J. Wysocki 2019-08-08 10:06 ` Rafael J. Wysocki 2019-08-08 13:15 ` Bjorn Helgaas 2019-08-08 13:15 ` Bjorn Helgaas 2019-08-08 14:48 ` Rafael J. Wysocki 2019-08-08 14:48 ` Rafael J. Wysocki 2019-08-08 10:10 ` [PATCH v2 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki 2019-08-08 10:10 ` Rafael J. Wysocki 2019-08-08 13:43 ` Bjorn Helgaas 2019-08-08 13:43 ` Bjorn Helgaas 2019-08-08 14:47 ` Rafael J. Wysocki 2019-08-08 14:47 ` Rafael J. Wysocki 2019-08-08 17:06 ` Rafael J. Wysocki 2019-08-08 17:06 ` Rafael J. Wysocki 2019-08-08 18:39 ` Bjorn Helgaas 2019-08-08 18:39 ` Bjorn Helgaas 2019-08-08 20:01 ` Keith Busch 2019-08-08 20:01 ` Keith Busch 2019-08-08 20:05 ` Mario.Limonciello 2019-08-08 20:05 ` Mario.Limonciello 2019-08-08 20:41 ` Rafael J. Wysocki 2019-08-08 20:41 ` Rafael J. Wysocki 2019-08-09 4:47 ` Bjorn Helgaas 2019-08-09 4:47 ` Bjorn Helgaas 2019-08-09 8:04 ` Rafael J. Wysocki 2019-08-09 8:04 ` Rafael J. Wysocki 2019-08-08 21:51 ` [PATCH v3 0/2] " Rafael J. Wysocki 2019-08-08 21:51 ` Rafael J. Wysocki 2019-08-08 21:55 ` [PATCH v3 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled() Rafael J. Wysocki 2019-08-08 21:55 ` Rafael J. Wysocki 2019-08-09 4:50 ` Bjorn Helgaas 2019-08-09 4:50 ` Bjorn Helgaas 2019-08-09 8:00 ` Rafael J. Wysocki 2019-08-09 8:00 ` Rafael J. Wysocki 2019-10-07 22:34 ` Bjorn Helgaas 2019-10-07 22:34 ` Bjorn Helgaas 2019-10-08 9:27 ` Rafael J. Wysocki 2019-10-08 9:27 ` Rafael J. Wysocki 2019-10-08 21:16 ` Bjorn Helgaas 2019-10-08 21:16 ` Bjorn Helgaas 2019-10-08 22:54 ` Rafael J. Wysocki 2019-10-08 22:54 ` Rafael J. Wysocki 2019-10-09 12:49 ` Bjorn Helgaas 2019-10-09 12:49 ` Bjorn Helgaas 2019-08-08 21:58 ` [PATCH v3 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki 2019-08-08 21:58 ` Rafael J. Wysocki 2019-08-08 22:13 ` [PATCH v3 0/2] " Keith Busch 2019-08-08 22:13 ` Keith Busch 2019-08-09 8:05 ` Rafael J. Wysocki 2019-08-09 8:05 ` Rafael J. Wysocki 2019-08-09 14:52 ` Keith Busch 2019-08-09 14:52 ` Keith Busch 2019-07-25 16:59 ` [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems Rafael J. Wysocki 2019-07-25 16:59 ` Rafael J. Wysocki 2019-07-25 14:52 ` Keith Busch 2019-07-25 14:52 ` Keith Busch 2019-07-25 19:48 ` Rafael J. Wysocki 2019-07-25 19:48 ` Rafael J. Wysocki 2019-07-25 19:52 ` Keith Busch 2019-07-25 19:52 ` Keith Busch 2019-07-25 20:02 ` Rafael J. Wysocki 2019-07-25 20:02 ` Rafael J. Wysocki 2019-07-26 14:02 ` Kai-Heng Feng 2019-07-26 14:02 ` Kai-Heng Feng 2019-07-27 12:55 ` Rafael J. Wysocki 2019-07-27 12:55 ` Rafael J. Wysocki 2019-07-29 15:51 ` Mario.Limonciello 2019-07-29 15:51 ` Mario.Limonciello 2019-07-29 22:05 ` Rafael J. Wysocki 2019-07-29 22:05 ` Rafael J. Wysocki
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190807143733.GA25621@localhost.localdomain \ --to=kbusch@kernel.org \ --cc=Mario.Limonciello@dell.com \ --cc=hch@lst.de \ --cc=helgaas@kernel.org \ --cc=kai.heng.feng@canonical.com \ --cc=keith.busch@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-nvme@lists.infradead.org \ --cc=linux-pci@vger.kernel.org \ --cc=linux-pm@vger.kernel.org \ --cc=rajatja@google.com \ --cc=rjw@rjwysocki.net \ --cc=sagi@grimberg.me \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.