linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: <Mario.Limonciello@dell.com>
To: <kai.heng.feng@canonical.com>, <rafael@kernel.org>
Cc: <rjw@rjwysocki.net>, <keith.busch@intel.com>, <hch@lst.de>,
	<sagi@grimberg.me>, <linux-nvme@lists.infradead.org>,
	<linux-pm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<rajatja@google.com>
Subject: RE: [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems
Date: Thu, 25 Jul 2019 19:09:25 +0000	[thread overview]
Message-ID: <e7bbeedaa0df409a8348f4c2b7b1359e@AUSX13MPC105.AMER.DELL.COM> (raw)
In-Reply-To: <46857EDA-081E-4622-A8D7-D337127A38DE@canonical.com>

> -----Original Message-----
> From: Kai-Heng Feng <kai.heng.feng@canonical.com>
> Sent: Thursday, July 25, 2019 1:20 PM
> To: Rafael J. Wysocki
> Cc: Limonciello, Mario; Rafael J. Wysocki; Keith Busch; Christoph Hellwig; Sagi
> Grimberg; linux-nvme; Linux PM; Linux Kernel Mailing List; Rajat Jain
> Subject: Re: [Regression] Commit "nvme/pci: Use host managed power state for
> suspend" has problems
> 
> 
> [EXTERNAL EMAIL]
> 
> at 01:03, Rafael J. Wysocki <rafael@kernel.org> wrote:
> 
> > On Thu, Jul 25, 2019 at 6:24 PM <Mario.Limonciello@dell.com> wrote:
> >> +Rajat
> >>
> >>> -----Original Message-----
> >>> From: Kai-Heng Feng <kai.heng.feng@canonical.com>
> >>> Sent: Thursday, July 25, 2019 9:03 AM
> >>> To: Rafael J. Wysocki
> >>> Cc: Keith Busch; Christoph Hellwig; Sagi Grimberg; linux-
> >>> nvme@lists.infradead.org; Limonciello, Mario; Linux PM; LKML
> >>> Subject: Re: [Regression] Commit "nvme/pci: Use host managed power
> >>> state for
> >>> suspend" has problems
> >>>
> >>>
> >>> [EXTERNAL EMAIL]
> >>>
> >>> Hi Rafael,
> >>>
> >>> at 17:51, Rafael J. Wysocki <rjw@rjwysocki.net> wrote:
> >>>
> >>>> Hi Keith,
> >>>>
> >>>> Unfortunately,
> >>>>
> >>>> commit d916b1be94b6dc8d293abed2451f3062f6af7551
> >>>> Author: Keith Busch <keith.busch@intel.com>
> >>>> Date:   Thu May 23 09:27:35 2019 -0600
> >>>>
> >>>>     nvme-pci: use host managed power state for suspend
> >>>>
> >>>> doesn't universally improve things.  In fact, in some cases it makes
> >>>> things worse.
> >>>>
> >>>> For example, on the Dell XPS13 9380 I have here it prevents the
> >>>> processor
> >>>> package
> >>>> from reaching idle states deeper than PC2 in suspend-to-idle (which, of
> >>>> course, also
> >>>> prevents the SoC from reaching any kind of S0ix).
> >>>>
> >>>> That can be readily explained too.  Namely, with the commit above the
> >>>> NVMe device
> >>>> stays in D0 over suspend/resume, so the root port it is connected to
> >>>> also
> >>>> has to stay in
> >>>> D0 and that "blocks" package C-states deeper than PC2.
> >>>>
> >>>> In order for the root port to be able to go to D3, the device connected
> >>>> to it also needs
> >>>> to go into D3, so it looks like (at least on this particular machine,
> >>>> but
> >>>> maybe in
> >>>> general), both D3 and the NVMe-specific PM are needed.
> >>
> >> Well this is really unfortunate to hear.  I recall that with some disks
> >> we were
> >> seeing problems where NVME specific PM wasn't working when the disk was
> >> in D3.
> >>
> >> On your specific disk, it would be good to know if just removing the
> >> pci_save_state(pdev)
> >> call helps.
> >
> > Yes, it does help.
> >
> >> If so, :
> >> * that might be a better option to add as a parameter.
> >> * maybe we should double check all the disks one more time with that
> >> tweak.
> >
> > At this point it seems so.
> >
> >>>> I'm not sure what to do here, because evidently there are systems where
> >>>> that commit
> >>>> helps.  I was thinking about adding a module option allowing the user to
> >>>> override the
> >>>> default behavior which in turn should be compatible with 5.2 and earlier
> >>>> kernels.
> >>>
> >>> I just briefly tested s2i on XPS 9370, and the power meter shows a
> >>> 0.8~0.9W
> >>> power consumption so at least I don’t see the issue on XPS 9370.
> >>
> >> To me that confirms NVME is down, but it still seems higher than I would
> >> have
> >> expected.  We should be more typically in the order of ~0.3W I think.
> 
>  From what I’ve observed, ~0.8W s2idle is already better than Windows (~1W).
> 0.3W is what I see during S5.

Oh thanks for confirming, I'm probably mixing up with another system.

> 
> >
> > It may go to PC10, but not reach S0ix.
> >
> > Anyway, I run the s2idle tests under turbostat which then tells me
> > what has happened more precisely.
> 
> The XPS 9370 at my hand does reach to s0ix during s2idle:
> # cat /sys/kernel/debug/pmc_core/slp_s0_residency_usec
> 15998400
> 
> So I think keep the root port in D0 is not the culprit here.
> Maybe something is wrong on the ASPM settings?
> 
> Kai-Heng

I have a 9380 on hand and I'm also showing slps0 residency with the SSD in it
and this series (Hynix BC501).


  reply	other threads:[~2019-07-25 19:09 UTC|newest]

Thread overview: 75+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-25  9:51 Rafael J. Wysocki
2019-07-25 14:02 ` Kai-Heng Feng
2019-07-25 16:23   ` Mario.Limonciello
2019-07-25 17:03     ` Rafael J. Wysocki
2019-07-25 17:23       ` Mario.Limonciello
2019-07-25 18:20       ` Kai-Heng Feng
2019-07-25 19:09         ` Mario.Limonciello [this message]
2019-07-30 10:45       ` Rafael J. Wysocki
2019-07-30 14:41         ` Keith Busch
2019-07-30 17:14           ` Mario.Limonciello
2019-07-30 18:50             ` Kai-Heng Feng
2019-07-30 19:19               ` Keith Busch
2019-07-30 21:05                 ` Mario.Limonciello
2019-07-30 21:31                   ` Keith Busch
2019-07-31 21:25                     ` Rafael J. Wysocki
2019-07-31 22:19                       ` Keith Busch
2019-07-31 22:33                         ` Rafael J. Wysocki
2019-08-01  9:05                           ` Kai-Heng Feng
2019-08-01 17:29                             ` Rafael J. Wysocki
2019-08-01 19:05                               ` Mario.Limonciello
2019-08-01 22:26                                 ` Rafael J. Wysocki
2019-08-02 10:55                                   ` Kai-Heng Feng
2019-08-02 11:04                                     ` Rafael J. Wysocki
2019-08-05 19:13                                       ` Kai-Heng Feng
2019-08-05 21:28                                         ` Rafael J. Wysocki
2019-08-06 14:02                                           ` Mario.Limonciello
2019-08-06 15:00                                             ` Rafael J. Wysocki
2019-08-07 10:29                                               ` Rafael J. Wysocki
2019-08-01 20:22                             ` Keith Busch
2019-08-07  9:48                         ` Rafael J. Wysocki
2019-08-07 10:45                           ` Christoph Hellwig
2019-08-07 10:54                             ` Rafael J. Wysocki
2019-08-07  9:53                         ` [PATCH] nvme-pci: Do not prevent PCI bus-level PM from being used Rafael J. Wysocki
2019-08-07 10:14                           ` Rafael J. Wysocki
2019-08-07 10:43                           ` Christoph Hellwig
2019-08-07 14:37                           ` Keith Busch
2019-08-08  8:36                         ` [PATCH] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08  8:48                           ` Christoph Hellwig
2019-08-08  9:06                             ` Rafael J. Wysocki
2019-08-08 10:03                         ` [PATCH v2 0/2] " Rafael J. Wysocki
2019-08-08 10:06                           ` [PATCH v2 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled_mask() Rafael J. Wysocki
2019-08-08 13:15                             ` Bjorn Helgaas
2019-08-08 14:48                               ` Rafael J. Wysocki
2019-08-08 10:10                           ` [PATCH v2 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08 13:43                             ` Bjorn Helgaas
2019-08-08 14:47                               ` Rafael J. Wysocki
2019-08-08 17:06                                 ` Rafael J. Wysocki
2019-08-08 18:39                                 ` Bjorn Helgaas
2019-08-08 20:01                                   ` Keith Busch
2019-08-08 20:05                                   ` Mario.Limonciello
2019-08-08 20:41                                   ` Rafael J. Wysocki
2019-08-09  4:47                                     ` Bjorn Helgaas
2019-08-09  8:04                                       ` Rafael J. Wysocki
2019-08-08 21:51                         ` [PATCH v3 0/2] " Rafael J. Wysocki
2019-08-08 21:55                           ` [PATCH v3 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled() Rafael J. Wysocki
2019-08-09  4:50                             ` Bjorn Helgaas
2019-08-09  8:00                               ` Rafael J. Wysocki
2019-10-07 22:34                             ` Bjorn Helgaas
2019-10-08  9:27                               ` Rafael J. Wysocki
2019-10-08 21:16                                 ` Bjorn Helgaas
2019-10-08 22:54                                   ` Rafael J. Wysocki
2019-10-09 12:49                                     ` Bjorn Helgaas
2019-08-08 21:58                           ` [PATCH v3 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08 22:13                           ` [PATCH v3 0/2] " Keith Busch
2019-08-09  8:05                             ` Rafael J. Wysocki
2019-08-09 14:52                               ` Keith Busch
2019-07-25 16:59   ` [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems Rafael J. Wysocki
2019-07-25 14:52 ` Keith Busch
2019-07-25 19:48   ` Rafael J. Wysocki
2019-07-25 19:52     ` Keith Busch
2019-07-25 20:02       ` Rafael J. Wysocki
2019-07-26 14:02         ` Kai-Heng Feng
2019-07-27 12:55           ` Rafael J. Wysocki
2019-07-29 15:51             ` Mario.Limonciello
2019-07-29 22:05               ` Rafael J. Wysocki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e7bbeedaa0df409a8348f4c2b7b1359e@AUSX13MPC105.AMER.DELL.COM \
    --to=mario.limonciello@dell.com \
    --cc=hch@lst.de \
    --cc=kai.heng.feng@canonical.com \
    --cc=keith.busch@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=rafael@kernel.org \
    --cc=rajatja@google.com \
    --cc=rjw@rjwysocki.net \
    --cc=sagi@grimberg.me \
    --subject='RE: [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).