Re: [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle - Kai-Heng Feng

From: Kai-Heng Feng <kai.heng.feng@canonical.com>
To: Christoph Hellwig <hch@lst.de>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
	Rafael Wysocki <rafael.j.wysocki@intel.com>,
	Mario Limonciello <Mario.Limonciello@dell.com>,
	Keith Busch <kbusch@kernel.org>,
	Keith Busch <keith.busch@intel.com>, Jens Axboe <axboe@fb.com>,
	Sagi Grimberg <sagi@grimberg.me>,
	linux-nvme <linux-nvme@lists.infradead.org>,
	Linux PM <linux-pm@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] nvme-pci: Use non-operational power state instead of D3 on Suspend-to-Idle
Date: Thu, 9 May 2019 19:59:55 +0800	[thread overview]
Message-ID: <AB325926-0D77-4851-8E8A-A10599756BF9@canonical.com> (raw)
In-Reply-To: <20190509103142.GA19550@lst.de>

at 18:31, Christoph Hellwig <hch@lst.de> wrote:

> On Thu, May 09, 2019 at 06:28:32PM +0800, Kai-Heng Feng wrote:
>> Based on my testing if queues (IRQ) are not disabled, NVMe controller
>> won’t be quiesced.
>> Symptoms can be high power drain or system freeze.
>>
>> I can check with vendors whether this also necessary under Windows.
>
> System freeze sounds odd.  And we had a patch from a person on the
> Cc list here that was handed to me through a few indirections that
> just skipps the suspend entirely for some cases, which seemd to
> work fine with the controllers in question.

That works fine for some devices, but for Toshiba NVMes this said scenario  
freezes the system, hence the new patch here.

And for all NVMes I tested this new suspend routine saves even more power  
than simply skipping suspend.

>
>>> Otherwise I think we should use a "no-op" suspend, just leaving the
>>> power management to the device, or a simple setting the device to the
>>> deepest power state for everything else, where everything else is
>>> suspend, or suspend to idle.
>>
>> I am not sure I get your idea. Does this “no-op” suspend happen in NVMe
>> driver or PM core?
>
> no-op means we don't want to do anything in nvme.  If that happens
> by not calling nvme or stubbing out the method for that particular
> case does not matter.

Ok, but we still need to figure out how to prevent the device device from  
tradition to D3.

>
>>> And of course than we have windows modern standby actually mandating
>>> runtime D3 in some case, and vague handwaving mentions of this being
>>> forced on the platforms, which I'm not entirely sure how they fit
>>> into the above picture.
>>
>> I was told that Windows doesn’t use runtime D3, APST is used exclusively.
>
> As far as I know the default power management modes in the Microsoft
> NVMe driver is explicit power management transitions, and in the Intel
> RST driver that is commonly used it is APST.  But both could still
> be comined with runtime D3 in theory, I'm just not sure if they are.
>
> Microsoft has been pushing for aggressive runtime D3 for a while, but
> I don't know if that includes NVMe devices.

Ok, I’ll check with vendors about this.

Kai-Heng

>
>> Kai-Heng
> ---end quoted text—