Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

[Bjorn Helgaas]

[+cc linux-pci, Rafael, thread at
https://lore.kernel.org/linux-nvme/1618458725-17164-1-git-send-email-Prike.Liang@amd.com/#t]

On Thu, Apr 15, 2021 at 11:52:04AM +0800, Prike Liang wrote:
> The NVME device pluged in some AMD PCIE root port will resume timeout
> from s2idle which caused by NVME power CFG lost in the SMU FW restore.
> This issue can be workaround by using PCIe power set with simple
> suspend/resume process path instead of APST. In the onwards ASIC will
> try do the NVME shutdown save and restore in the BIOS and still need PCIe
> power setting to resume from RTD3 for s2idle.
> 
> In this preparation patch add a PCIe quirk for the AMD.

This needs to be cc'd to linux-pci (I did it for you this time).

Sorry, I can't make any sense out of the commit log.  Is this a Root
Port defect or an NVMe device defect?

Patch 2/2 only uses PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND in the nvme
driver, so AFAICT there is no reason for the PCI core to keep track of
the flag for you.

I see below that Christoph suggests it needs to be in the PCI core,
but the reason needs to be explained in the commit log.

I have not acked this patch.  Please don't merge it before clearing
these things up.

> Cc: <stable@vger.kernel.org> # 5.11+
> Signed-off-by: Prike Liang <Prike.Liang@amd.com>
> Signed-off-by: Shyam Sundar S K <Shyam-sundar.S-k@amd.com>
> [ck: split patches for nvme and pcie]
> Signed-off-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
> 
> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
> ---
> Changes in v2:
> Fix the patch format and check chip root complex DID instead of PCIe RP
> to avoid the storage device plugged in internal PCIe RP by USB adaptor.
> 
> Changes in v3:
> According to Christoph Hellwig do NVME PCIe related identify opt better 
> in PCIe quirk driver rather than in NVME module.
> 
> Changes in v4:
> Split the fix to PCIe and NVMe part and then call the pci_dev_put() put 
> the device reference count and finally refine the commit info.
> ---
> drivers/pci/quirks.c | 10 ++++++++++
>  include/linux/pci.h  |  2 ++
>  2 files changed, 12 insertions(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index 653660e3..f95c8b2 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -312,6 +312,16 @@ static void quirk_nopciamd(struct pci_dev *dev)
>  }
>  DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD,	PCI_DEVICE_ID_AMD_8151_0,	quirk_nopciamd);
>  
> +static void quirk_amd_nvme_fixup(struct pci_dev *dev)
> +{
> +	struct pci_dev *rdev;
> +
> +	dev->dev_flags |= PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND;
> +	pci_info(dev, "AMD simple suspend opt enabled\n");
> +
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_AMD, 0x1630, quirk_amd_nvme_fixup);
> +
>  /* Triton requires workarounds to be used by the drivers */
>  static void quirk_triton(struct pci_dev *dev)
>  {
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 53f4904..a6e1b1b 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -227,6 +227,8 @@ enum pci_dev_flags {
>  	PCI_DEV_FLAGS_NO_FLR_RESET = (__force pci_dev_flags_t) (1 << 10),
>  	/* Don't use Relaxed Ordering for TLPs directed at this device */
>  	PCI_DEV_FLAGS_NO_RELAXED_ORDERING = (__force pci_dev_flags_t) (1 << 11),
> +	/* AMD simple suspend opt quirk */
> +	PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND = (__force pci_dev_flags_t) (1 << 12),
>  };
>  
>  enum pci_irq_reroute_variant {
> -- 
> 2.7.4
> 
> 
> _______________________________________________
> Linux-nvme mailing list
> Linux-nvme@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

[Christoph Hellwig]

On Fri, Apr 30, 2021 at 12:50:49PM -0500, Bjorn Helgaas wrote:
> This needs to be cc'd to linux-pci (I did it for you this time).

I did ask for that before.

> Sorry, I can't make any sense out of the commit log.  Is this a Root
> Port defect or an NVMe device defect?

It is a root port quirk, although it appears to be intentional as Intel
is doing the same thing on some platforms.

> Patch 2/2 only uses PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND in the nvme
> driver, so AFAICT there is no reason for the PCI core to keep track of
> the flag for you.
> 
> I see below that Christoph suggests it needs to be in the PCI core,
> but the reason needs to be explained in the commit log.

As far as I can tell this has nothing to do with NVMe except for the
fact that right now it mostly hits NVMe as the nvme drivers is one of
the few drivers not always doing a full device shutdown when the
system goes into the S3 power state.  But various x86 platforms now
randomly power done the link in that case.

> 
> I have not acked this patch.  Please don't merge it before clearing
> these things up.

I would never merge PCI core changes that haven't been reviewd by the
maintainer.

Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

[Keith Busch]

On Mon, May 03, 2021 at 08:14:07AM +0100, Christoph Hellwig wrote:
> On Fri, Apr 30, 2021 at 12:50:49PM -0500, Bjorn Helgaas wrote:
> > Patch 2/2 only uses PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND in the nvme
> > driver, so AFAICT there is no reason for the PCI core to keep track of
> > the flag for you.
> > 
> > I see below that Christoph suggests it needs to be in the PCI core,
> > but the reason needs to be explained in the commit log.
> 
> As far as I can tell this has nothing to do with NVMe except for the
> fact that right now it mostly hits NVMe as the nvme drivers is one of
> the few drivers not always doing a full device shutdown when the
> system goes into the S3 power state.  But various x86 platforms now
> randomly power done the link in that case.

Right, and the v5 of this series uses a generic name for the PCI quirk
without mentioning "NVME".

Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

[Bjorn Helgaas]

On Mon, May 03, 2021 at 07:57:02AM -0700, Keith Busch wrote:
> On Mon, May 03, 2021 at 08:14:07AM +0100, Christoph Hellwig wrote:
> > On Fri, Apr 30, 2021 at 12:50:49PM -0500, Bjorn Helgaas wrote:
> > > Patch 2/2 only uses PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND in the nvme
> > > driver, so AFAICT there is no reason for the PCI core to keep track of
> > > the flag for you.
> > > 
> > > I see below that Christoph suggests it needs to be in the PCI core,
> > > but the reason needs to be explained in the commit log.
> > 
> > As far as I can tell this has nothing to do with NVMe except for the
> > fact that right now it mostly hits NVMe as the nvme drivers is one of
> > the few drivers not always doing a full device shutdown when the
> > system goes into the S3 power state.  But various x86 platforms now
> > randomly power done the link in that case.
> 
> Right, and the v5 of this series uses a generic name for the PCI quirk
> without mentioning "NVME".

It'd be nice if somebody would figure out how to cc: linux-pci on
these patches.

RE: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt

[Liang, Prike]

[AMD Public Use]

> From: Bjorn Helgaas <helgaas@kernel.org>
> Sent: Monday, May 3, 2021 11:50 PM
> To: Keith Busch <kbusch@kernel.org>
> Cc: Christoph Hellwig <hch@infradead.org>; Liang, Prike
> <Prike.Liang@amd.com>; linux-nvme@lists.infradead.org;
> Chaitanya.Kulkarni@wdc.com; gregkh@linuxfoundation.org;
> stable@vger.kernel.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; S-k, Shyam-sundar <Shyam-sundar.S-
> k@amd.com>; linux-pci@vger.kernel.org; Rafael J. Wysocki
> <rjw@rjwysocki.net>
> Subject: Re: [PATCH v4 1/2] PCI: add AMD PCIe quirk for nvme shutdown opt
>
> On Mon, May 03, 2021 at 07:57:02AM -0700, Keith Busch wrote:
> > On Mon, May 03, 2021 at 08:14:07AM +0100, Christoph Hellwig wrote:
> > > On Fri, Apr 30, 2021 at 12:50:49PM -0500, Bjorn Helgaas wrote:
> > > > Patch 2/2 only uses PCI_DEV_FLAGS_AMD_NVME_SIMPLE_SUSPEND in
> the
> > > > nvme driver, so AFAICT there is no reason for the PCI core to keep
> > > > track of the flag for you.
> > > >
> > > > I see below that Christoph suggests it needs to be in the PCI
> > > > core, but the reason needs to be explained in the commit log.
> > >
> > > As far as I can tell this has nothing to do with NVMe except for the
> > > fact that right now it mostly hits NVMe as the nvme drivers is one
> > > of the few drivers not always doing a full device shutdown when the
> > > system goes into the S3 power state.  But various x86 platforms now
> > > randomly power done the link in that case.
> >
> > Right, and the v5 of this series uses a generic name for the PCI quirk
> > without mentioning "NVME".
>
> It'd be nice if somebody would figure out how to cc: linux-pci on these
> patches.
Thank you for your review. Sorry miss that and now linux-pci has been added to the v5 patch.