linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: <Mario.Limonciello@dell.com>
To: <rafael@kernel.org>, <kai.heng.feng@canonical.com>
Cc: <kbusch@kernel.org>, <keith.busch@intel.com>, <hch@lst.de>,
	<sagi@grimberg.me>, <linux-nvme@lists.infradead.org>,
	<linux-pm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<rajatja@google.com>
Subject: RE: [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems
Date: Tue, 6 Aug 2019 14:02:22 +0000	[thread overview]
Message-ID: <1d05e26a91f94e5b92eaf5854fa85289@AUSX13MPC105.AMER.DELL.COM> (raw)
In-Reply-To: <CAJZ5v0g_yAP=zgYDXtJWtwceQp4wWPWFghDDD0opdZ4zT-yo5Q@mail.gmail.com>



> -----Original Message-----
> From: Rafael J. Wysocki <rafael@kernel.org>
> Sent: Monday, August 5, 2019 4:29 PM
> To: Kai-Heng Feng
> Cc: Rafael J. Wysocki; Limonciello, Mario; Keith Busch; Keith Busch; Christoph
> Hellwig; Sagi Grimberg; linux-nvme; Linux PM; Linux Kernel Mailing List; Rajat Jain
> Subject: Re: [Regression] Commit "nvme/pci: Use host managed power state for
> suspend" has problems
> 
> 
> [EXTERNAL EMAIL]
> 
> On Mon, Aug 5, 2019 at 9:14 PM Kai-Heng Feng
> <kai.heng.feng@canonical.com> wrote:
> >
> > at 19:04, Rafael J. Wysocki <rafael@kernel.org> wrote:
> >
> > > On Fri, Aug 2, 2019 at 12:55 PM Kai-Heng Feng
> > > <kai.heng.feng@canonical.com> wrote:
> > >> at 06:26, Rafael J. Wysocki <rafael@kernel.org> wrote:
> > >>
> > >>> On Thu, Aug 1, 2019 at 9:05 PM <Mario.Limonciello@dell.com> wrote:
> > >>>>> -----Original Message-----
> > >>>>> From: Rafael J. Wysocki <rafael@kernel.org>
> > >>>>> Sent: Thursday, August 1, 2019 12:30 PM
> > >>>>> To: Kai-Heng Feng; Keith Busch; Limonciello, Mario
> > >>>>> Cc: Keith Busch; Christoph Hellwig; Sagi Grimberg; linux-nvme; Linux
> > >>>>> PM; Linux
> > >>>>> Kernel Mailing List; Rajat Jain
> > >>>>> Subject: Re: [Regression] Commit "nvme/pci: Use host managed power
> > >>>>> state for
> > >>>>> suspend" has problems
> > >>>>>
> > >>>>>
> > >>>>> [EXTERNAL EMAIL]
> > >>>>>
> > >>>>> On Thu, Aug 1, 2019 at 11:06 AM Kai-Heng Feng
> > >>>>> <kai.heng.feng@canonical.com> wrote:
> > >>>>>> at 06:33, Rafael J. Wysocki <rafael@kernel.org> wrote:
> > >>>>>>
> > >>>>>>> On Thu, Aug 1, 2019 at 12:22 AM Keith Busch <kbusch@kernel.org>
> > >>>>>>> wrote:
> > >>>>>>>> On Wed, Jul 31, 2019 at 11:25:51PM +0200, Rafael J. Wysocki wrote:
> > >>>>>>>>> A couple of remarks if you will.
> > >>>>>>>>>
> > >>>>>>>>> First, we don't know which case is the majority at this point.  For
> > >>>>>>>>> now, there is one example of each, but it may very well turn out
> > >>>>>>>>> that
> > >>>>>>>>> the SK Hynix BC501 above needs to be quirked.
> > >>>>>>>>>
> > >>>>>>>>> Second, the reference here really is 5.2, so if there are any
> > >>>>>>>>> systems
> > >>>>>>>>> that are not better off with 5.3-rc than they were with 5.2,
> > >>>>>>>>> well, we
> > >>>>>>>>> have not made progress.  However, if there are systems that are
> > >>>>>>>>> worse
> > >>>>>>>>> off with 5.3, that's bad.  In the face of the latest findings the
> > >>>>>>>>> only
> > >>>>>>>>> way to avoid that is to be backwards compatible with 5.2 and that's
> > >>>>>>>>> where my patch is going.  That cannot be achieved by quirking all
> > >>>>>>>>> cases that are reported as "bad", because there still may be
> > >>>>>>>>> unreported ones.
> > >>>>>>>>
> > >>>>>>>> I have to agree. I think your proposal may allow PCI D3cold,
> > >>>>>>>
> > >>>>>>> Yes, it may.
> > >>>>>>
> > >>>>>> Somehow the 9380 with Toshiba NVMe never hits SLP_S0 with or
> without
> > >>>>>> Rafael’s patch.
> > >>>>>> But the “real” s2idle power consumption does improve with the patch.
> > >>>>>
> > >>>>> Do you mean this patch:
> > >>>>>
> > >>>>> https://lore.kernel.org/linux-pm/70D536BE-8DC7-4CA2-84A9-
> > >>>>>
> AFB067BA520E@canonical.com/T/#m456aa5c69973a3b68f2cdd4713a1ce83be5145
> > >>>>> 8f
> > >>>>>
> > >>>>> or the $subject one without the above?
> > >>>>>
> > >>>>>> Can we use a DMI based quirk for this platform? It seems like a
> > >>>>>> platform
> > >>>>>> specific issue.
> > >>>>>
> > >>>>> We seem to see too many "platform-specific issues" here. :-)
> > >>>>>
> > >>>>> To me, the status quo (ie. what we have in 5.3-rc2) is not defensible.
> > >>>>> Something needs to be done to improve the situation.
> > >>>>
> > >>>> Rafael, would it be possible to try popping out PC401 from the 9380 and
> > >>>> into a 9360 to
> > >>>> confirm there actually being a platform impact or not?
> > >>>
> > >>> Not really, sorry.
> > >>>
> > >>>> I was hoping to have something useful from Hynix by now before
> > >>>> responding, but oh well.
> > >>>>
> > >>>> In terms of what is the majority, I do know that between folks at Dell,
> > >>>> Google, Compal,
> > >>>> Wistron, Canonical, Micron, Hynix, Toshiba, LiteOn, and Western Digital
> > >>>> we tested a wide
> > >>>> variety of SSDs with this patch series.  I would like to think that they
> > >>>> are representative of
> > >>>> what's being manufactured into machines now.
> > >>>
> > >>> Well, what about drives already in the field?  My concern is mostly
> > >>> about those ones.
> > >>>
> > >>>> Notably the LiteOn CL1 was tested with the HMB flushing support and
> > >>>> and Hynix PC401 was tested with older firmware though.
> > >>>>
> > >>>>>>>> In which case we do need to reintroduce the HMB handling.
> > >>>>>>>
> > >>>>>>> Right.
> > >>>>>>
> > >>>>>> The patch alone doesn’t break HMB Toshiba NVMe I tested. But I think
> > >>>>>> it’s
> > >>>>>> still safer to do proper HMB handling.
> > >>>>>
> > >>>>> Well, so can anyone please propose something specific?  Like an
> > >>>>> alternative patch?
> > >>>>
> > >>>> This was proposed a few days ago:
> > >>>> http://lists.infradead.org/pipermail/linux-nvme/2019-July/026056.html
> > >>>>
> > >>>> However we're still not sure why it is needed, and it will take some
> > >>>> time to get
> > >>>> a proper failure analysis from LiteOn  regarding the CL1.
> > >>>
> > >>> Thanks for the update, but IMO we still need to do something before
> > >>> final 5.3 while the investigation continues.
> > >>>
> > >>> Honestly, at this point I would vote for going back to the 5.2
> > >>> behavior at least by default and only running the new code on the
> > >>> drives known to require it (because they will block PC10 otherwise).
> > >>>
> > >>> Possibly (ideally) with an option for users who can't get beyond PC3
> > >>> to test whether or not the new code helps them.
> > >>
> > >> I just found out that the XPS 9380 at my hand never reaches SLP_S0 but
> > >> only
> > >> PC10.
> > >
> > > That's the case for me too.
> > >
> > >> This happens with or without putting the device to D3.
> > >
> > > On my system, though, it only can get to PC3 without putting the NVMe
> > > into D3 (as reported previously).
> >
> > I forgot to ask, what BIOS version does the system have?
> > I don’t see this issue on BIOS v1.5.0.
> 
> It is 1.5.0 here too.

All, regarding the need for the patch proposed by Rafael on PC401, I have some updates
to share from Hynix.
First off - the firmware changelog is misleading from 80006E00 to 80007E00.

The change was made in the firmware specifically because of a change in behavior from
Intel KBL to CFL and WHL.  On CFL/WHL the period of time that RefClk was turned on after L1.2
was larger than KBL platforms.  So this meant that Hynix couldn't lock up from CLKREQ#
to RefClk as quickly on CFL/WHL.  So there is a "larger" fixed delay introduced in their FW.

To those that don't know - XPS 9380 is a WHL platform.

Second - a hypothesis of what is happening with the patch proposed by Rafael is that the link
is only transitioning to L1.0 rather than L1.2.  This may satisfy the PMC but it shouldn't lead to
the lowest actual device power state.


  reply	other threads:[~2019-08-06 14:02 UTC|newest]

Thread overview: 75+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-25  9:51 Rafael J. Wysocki
2019-07-25 14:02 ` Kai-Heng Feng
2019-07-25 16:23   ` Mario.Limonciello
2019-07-25 17:03     ` Rafael J. Wysocki
2019-07-25 17:23       ` Mario.Limonciello
2019-07-25 18:20       ` Kai-Heng Feng
2019-07-25 19:09         ` Mario.Limonciello
2019-07-30 10:45       ` Rafael J. Wysocki
2019-07-30 14:41         ` Keith Busch
2019-07-30 17:14           ` Mario.Limonciello
2019-07-30 18:50             ` Kai-Heng Feng
2019-07-30 19:19               ` Keith Busch
2019-07-30 21:05                 ` Mario.Limonciello
2019-07-30 21:31                   ` Keith Busch
2019-07-31 21:25                     ` Rafael J. Wysocki
2019-07-31 22:19                       ` Keith Busch
2019-07-31 22:33                         ` Rafael J. Wysocki
2019-08-01  9:05                           ` Kai-Heng Feng
2019-08-01 17:29                             ` Rafael J. Wysocki
2019-08-01 19:05                               ` Mario.Limonciello
2019-08-01 22:26                                 ` Rafael J. Wysocki
2019-08-02 10:55                                   ` Kai-Heng Feng
2019-08-02 11:04                                     ` Rafael J. Wysocki
2019-08-05 19:13                                       ` Kai-Heng Feng
2019-08-05 21:28                                         ` Rafael J. Wysocki
2019-08-06 14:02                                           ` Mario.Limonciello [this message]
2019-08-06 15:00                                             ` Rafael J. Wysocki
2019-08-07 10:29                                               ` Rafael J. Wysocki
2019-08-01 20:22                             ` Keith Busch
2019-08-07  9:48                         ` Rafael J. Wysocki
2019-08-07 10:45                           ` Christoph Hellwig
2019-08-07 10:54                             ` Rafael J. Wysocki
2019-08-07  9:53                         ` [PATCH] nvme-pci: Do not prevent PCI bus-level PM from being used Rafael J. Wysocki
2019-08-07 10:14                           ` Rafael J. Wysocki
2019-08-07 10:43                           ` Christoph Hellwig
2019-08-07 14:37                           ` Keith Busch
2019-08-08  8:36                         ` [PATCH] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08  8:48                           ` Christoph Hellwig
2019-08-08  9:06                             ` Rafael J. Wysocki
2019-08-08 10:03                         ` [PATCH v2 0/2] " Rafael J. Wysocki
2019-08-08 10:06                           ` [PATCH v2 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled_mask() Rafael J. Wysocki
2019-08-08 13:15                             ` Bjorn Helgaas
2019-08-08 14:48                               ` Rafael J. Wysocki
2019-08-08 10:10                           ` [PATCH v2 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08 13:43                             ` Bjorn Helgaas
2019-08-08 14:47                               ` Rafael J. Wysocki
2019-08-08 17:06                                 ` Rafael J. Wysocki
2019-08-08 18:39                                 ` Bjorn Helgaas
2019-08-08 20:01                                   ` Keith Busch
2019-08-08 20:05                                   ` Mario.Limonciello
2019-08-08 20:41                                   ` Rafael J. Wysocki
2019-08-09  4:47                                     ` Bjorn Helgaas
2019-08-09  8:04                                       ` Rafael J. Wysocki
2019-08-08 21:51                         ` [PATCH v3 0/2] " Rafael J. Wysocki
2019-08-08 21:55                           ` [PATCH v3 1/2] PCI: PCIe: ASPM: Introduce pcie_aspm_enabled() Rafael J. Wysocki
2019-08-09  4:50                             ` Bjorn Helgaas
2019-08-09  8:00                               ` Rafael J. Wysocki
2019-10-07 22:34                             ` Bjorn Helgaas
2019-10-08  9:27                               ` Rafael J. Wysocki
2019-10-08 21:16                                 ` Bjorn Helgaas
2019-10-08 22:54                                   ` Rafael J. Wysocki
2019-10-09 12:49                                     ` Bjorn Helgaas
2019-08-08 21:58                           ` [PATCH v3 2/2] nvme-pci: Allow PCI bus-level PM to be used if ASPM is disabled Rafael J. Wysocki
2019-08-08 22:13                           ` [PATCH v3 0/2] " Keith Busch
2019-08-09  8:05                             ` Rafael J. Wysocki
2019-08-09 14:52                               ` Keith Busch
2019-07-25 16:59   ` [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems Rafael J. Wysocki
2019-07-25 14:52 ` Keith Busch
2019-07-25 19:48   ` Rafael J. Wysocki
2019-07-25 19:52     ` Keith Busch
2019-07-25 20:02       ` Rafael J. Wysocki
2019-07-26 14:02         ` Kai-Heng Feng
2019-07-27 12:55           ` Rafael J. Wysocki
2019-07-29 15:51             ` Mario.Limonciello
2019-07-29 22:05               ` Rafael J. Wysocki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1d05e26a91f94e5b92eaf5854fa85289@AUSX13MPC105.AMER.DELL.COM \
    --to=mario.limonciello@dell.com \
    --cc=hch@lst.de \
    --cc=kai.heng.feng@canonical.com \
    --cc=kbusch@kernel.org \
    --cc=keith.busch@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=rafael@kernel.org \
    --cc=rajatja@google.com \
    --cc=sagi@grimberg.me \
    --subject='RE: [Regression] Commit "nvme/pci: Use host managed power state for suspend" has problems' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).