Re: [PATCH] pci: add NVMe FLR quirk to the SM951 SSD

From: Bjorn Helgaas <helgaas@kernel.org>
To: Robert Straw <drbawb@fatalsyntax.com>
Cc: bhelgaas@google.com, linux-pci@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	Alex Williamson <alex.williamson@redhat.com>
Subject: Re: [PATCH] pci: add NVMe FLR quirk to the SM951 SSD
Date: Fri, 30 Apr 2021 15:51:05 -0500	[thread overview]
Message-ID: <20210430205105.GA683965@bjorn-Precision-5520> (raw)
In-Reply-To: <20210429230730.201266-1-drbawb@fatalsyntax.com>

[+cc Alex, author of ffb0863426eb]

Please make your subject line match ffb0863426eb ("PCI: Disable
Samsung SM961/PM961 NVMe before FLR")

On Thu, Apr 29, 2021 at 06:07:30PM -0500, Robert Straw wrote:
> The SM951/PM951, when used in conjunction with the vfio-pci driver and
> passed to a KVM guest, can exhibit the fatal state addressed by the
> existing `nvme_disable_and_flr` quirk. If the guest cleanly shuts down
> the SSD, and vfio-pci attempts an FLR to the device while it is in this 
> state, the nvme driver will fail when it attempts to bind to the device 
> after the FLR due to the frozen config area, e.g:
> 
>   nvme nvme2: frozen state error detected, reset controller
>   nvme nvme2: Removing after probe failure status: -12
> 
> By including this older model (Samsung 950 PRO) of the controller in the
> existing quirk: the device is able to be cleanly reset after being used
> by a KVM guest.

I don't see anything in the PCIe spec about software being required to
do something special before initiating an FLR, so I assume this is a
hardware defect in the Samsung 950 PRO?  Has Samsung published an
erratum or at least acknowledged it?

There's always the possibility that we are doing something wrong in
Linux *after* the FLR, e.g., not waiting long enough, not
reinitializing something correctly, etc.

> Signed-off-by: Robert Straw <drbawb@fatalsyntax.com>
> ---
>  drivers/pci/quirks.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
> index 653660e3b..e339ca238 100644
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -3920,6 +3920,7 @@ static const struct pci_dev_reset_methods pci_dev_reset_methods[] = {
>  		reset_ivb_igd },
>  	{ PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IVB_M2_VGA,
>  		reset_ivb_igd },
> +	{ PCI_VENDOR_ID_SAMSUNG, 0xa802, nvme_disable_and_flr },
>  	{ PCI_VENDOR_ID_SAMSUNG, 0xa804, nvme_disable_and_flr },
>  	{ PCI_VENDOR_ID_INTEL, 0x0953, delay_250ms_after_flr },
>  	{ PCI_VENDOR_ID_CHELSIO, PCI_ANY_ID,
> -- 
> 2.31.1
> 