From: Keith Busch <kbusch@kernel.org>
To: Kai Heng Feng <kai.heng.feng@canonical.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
Mario Limonciello <Mario.Limonciello@dell.com>,
Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@fb.com>,
Sagi Grimberg <sagi@grimberg.me>,
Linux PM <linux-pm@vger.kernel.org>,
Rafael Wysocki <rafael.j.wysocki@intel.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
linux-nvme <linux-nvme@lists.infradead.org>,
Keith Busch <keith.busch@intel.com>
Subject: Re: [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle
Date: Fri, 10 May 2019 09:36:58 -0600 [thread overview]
Message-ID: <20190510153658.GI9675@localhost.localdomain> (raw)
In-Reply-To: <54E4999C-DBE8-4FC1-8E82-17FEDFDA9CA6@canonical.com>
On Fri, May 10, 2019 at 11:15:05PM +0800, Kai Heng Feng wrote:
> Sorry, I should mention that I use a slightly modified drivers/nvme/host/pci.c:
>
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 3e4fb891a95a..ece428ce6876 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -18,6 +18,7 @@
> #include <linux/mutex.h>
> #include <linux/once.h>
> #include <linux/pci.h>
> +#include <linux/suspend.h>
> #include <linux/t10-pi.h>
> #include <linux/types.h>
> #include <linux/io-64-nonatomic-lo-hi.h>
> @@ -2833,6 +2834,11 @@ static int nvme_suspend(struct device *dev)
> struct pci_dev *pdev = to_pci_dev(dev);
> struct nvme_dev *ndev = pci_get_drvdata(pdev);
>
> + if (!pm_suspend_via_firmware()) {
> + nvme_set_power(&ndev->ctrl, ndev->ctrl.npss);
> + pci_save_state(pdev);
> + }
> +
> nvme_dev_disable(ndev, true);
This won't work because you're falling through to nvme_dev_disable after
setting the power, so the resume side would certainly fail.
This is what you'd want:
if (!pm_suspend_via_firmware()) {
pci_save_state(pdev);
return nvme_set_power(&ndev->ctrl, ndev->ctrl.npss);
}
> return 0;
> }
> @@ -2842,6 +2848,10 @@ static int nvme_resume(struct device *dev)
> struct pci_dev *pdev = to_pci_dev(dev);
> struct nvme_dev *ndev = pci_get_drvdata(pdev);
>
> + if (!pm_resume_via_firmware()) {
> + return nvme_set_power(&ndev->ctrl, 0);
> + }
> +
> nvme_reset_ctrl(&ndev->ctrl);
> return 0;
> }
>
> Does pci_save_state() here enough to prevent the device enter to D3?
Yes, saving the state during suspend will prevent pci pm from assuming
control over this device's power. It's a bit non-intuitive as Christoph
mentioned, so we'll need to make that clear in the comments. For
reference, here's the relevant part in pci-driver.c:
---
static int pci_pm_suspend_noirq(struct device *dev)
{
...
if (!pci_dev->state_saved) {
pci_save_state(pci_dev);
if (pci_power_manageable(pci_dev))
pci_prepare_to_sleep(pci_dev);
}
...
}
--
So by saving the state in our suspend callback, pci will skip
pci_prepare_to_sleep(), which is what's setting your device most likely
to a D3hot, undermining our nvme power state setting.
next prev parent reply other threads:[~2019-05-10 15:42 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-08 18:59 [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle Kai-Heng Feng
2019-05-08 19:15 ` Chaitanya Kulkarni
2019-05-08 19:16 ` Keith Busch
2019-05-08 19:30 ` Kai-Heng Feng
2019-05-08 19:38 ` Mario.Limonciello
2019-05-08 19:51 ` Christoph Hellwig
2019-05-08 20:28 ` Mario.Limonciello
2019-05-09 6:12 ` Christoph Hellwig
2019-05-09 6:48 ` Kai-Heng Feng
2019-05-09 6:52 ` Christoph Hellwig
2019-05-09 9:19 ` Rafael J. Wysocki
2019-05-09 9:25 ` Christoph Hellwig
2019-05-09 20:48 ` Rafael J. Wysocki
2019-05-09 9:07 ` Rafael J. Wysocki
2019-05-09 9:42 ` Kai-Heng Feng
2019-05-09 9:56 ` Christoph Hellwig
2019-05-09 10:28 ` Kai-Heng Feng
2019-05-09 10:31 ` Christoph Hellwig
2019-05-09 11:59 ` Kai-Heng Feng
2019-05-09 18:57 ` Mario.Limonciello
2019-05-09 19:28 ` Keith Busch
2019-05-09 20:54 ` Rafael J. Wysocki
2019-05-09 21:16 ` Keith Busch
2019-05-09 21:39 ` Rafael J. Wysocki
2019-05-09 21:37 ` Mario.Limonciello
2019-05-09 21:54 ` Keith Busch
2019-05-09 22:19 ` Mario.Limonciello
2019-05-10 6:05 ` Kai-Heng Feng
2019-05-10 8:23 ` Rafael J. Wysocki
2019-05-10 13:52 ` Keith Busch
2019-05-10 15:15 ` Kai Heng Feng
2019-05-10 15:36 ` Keith Busch [this message]
2019-05-10 14:02 ` Keith Busch
2019-05-10 15:18 ` Kai Heng Feng
2019-05-10 15:49 ` hch
2019-05-10 5:30 ` Christoph Hellwig
2019-05-10 13:51 ` Keith Busch
2019-05-09 16:20 ` Keith Busch
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190510153658.GI9675@localhost.localdomain \
--to=kbusch@kernel.org \
--cc=Mario.Limonciello@dell.com \
--cc=axboe@fb.com \
--cc=hch@lst.de \
--cc=kai.heng.feng@canonical.com \
--cc=keith.busch@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pm@vger.kernel.org \
--cc=rafael.j.wysocki@intel.com \
--cc=rafael@kernel.org \
--cc=sagi@grimberg.me \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).