linux-nvme.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Mario Limonciello <mario.limonciello@dell.com>
To: Keith Busch <kbusch@kernel.org>
Cc: Crag Wang <Crag.Wang@dell.com>, Sagi Grimberg <sagi@grimberg.me>,
	Mario Limonciello <mario.limonciello@dell.com>,
	sjg@google.com, LKML <linux-kernel@vger.kernel.org>,
	linux-nvme@lists.infradead.org, Jens Axboe <axboe@fb.com>,
	Ryan Hong <Ryan.Hong@Dell.com>,
	Jared Dominguez <jared.dominguez@dell.com>,
	Christoph Hellwig <hch@lst.de>
Subject: [PATCH] nvme-pci: Save PCI state before putting drive into deepest state
Date: Wed, 11 Sep 2019 18:42:33 -0500	[thread overview]
Message-ID: <1568245353-13787-1-git-send-email-mario.limonciello@dell.com> (raw)

The action of saving the PCI state will cause numerous PCI configuration
space reads which depending upon the vendor implementation may cause
the drive to exit the deepest NVMe state.

In these cases ASPM will typically resolve the PCIe link state and APST
may resolve the NVMe power state.  However it has also been observed
that this register access after quiesced will cause PC10 failure
on some device combinations.

To resolve this, move the PCI state saving to before SetFeatures has been
called.  This has been proven to resolve the issue across a 5000 sample
test on previously failing disk/system combinations.

Signed-off-by: Mario Limonciello <mario.limonciello@dell.com>
---
 drivers/nvme/host/pci.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 732d5b6..9b3fed4 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2894,6 +2894,13 @@ static int nvme_suspend(struct device *dev)
 	if (ret < 0)
 		goto unfreeze;
 
+	/*
+	 * A saved state prevents pci pm from generically controlling the
+	 * device's power. If we're using protocol specific settings, we don't
+	 * want pci interfering.
+	 */
+	pci_save_state(pdev);
+
 	ret = nvme_set_power_state(ctrl, ctrl->npss);
 	if (ret < 0)
 		goto unfreeze;
@@ -2908,12 +2915,6 @@ static int nvme_suspend(struct device *dev)
 		ret = 0;
 		goto unfreeze;
 	}
-	/*
-	 * A saved state prevents pci pm from generically controlling the
-	 * device's power. If we're using protocol specific settings, we don't
-	 * want pci interfering.
-	 */
-	pci_save_state(pdev);
 unfreeze:
 	nvme_unfreeze(ctrl);
 	return ret;
-- 
2.7.4


_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

             reply	other threads:[~2019-09-11 23:42 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-11 23:42 Mario Limonciello [this message]
2019-09-17 21:24 ` [PATCH] nvme-pci: Save PCI state before putting drive into deepest state Keith Busch
2019-09-17 21:35   ` Rafael J. Wysocki
2019-09-18 16:52     ` Mario.Limonciello
2019-09-18 21:27       ` Rafael J. Wysocki
2019-09-18 17:28 ` Keith Busch
2019-09-18 21:31 ` Rafael J. Wysocki
2019-09-18 21:42   ` Rafael J. Wysocki
2019-09-18 21:43   ` Mario.Limonciello
2019-09-18 21:57     ` Rafael J. Wysocki
2019-09-18 22:00       ` Mario.Limonciello
2019-09-18 22:03         ` Rafael J. Wysocki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1568245353-13787-1-git-send-email-mario.limonciello@dell.com \
    --to=mario.limonciello@dell.com \
    --cc=Crag.Wang@dell.com \
    --cc=Ryan.Hong@Dell.com \
    --cc=axboe@fb.com \
    --cc=hch@lst.de \
    --cc=jared.dominguez@dell.com \
    --cc=kbusch@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=sagi@grimberg.me \
    --cc=sjg@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).