Linux-PCI Archive on lore.kernel.org
 help / color / Atom feed
From: "Sean V Kelley" <sean.v.kelley@intel.com>
To: "Jonathan Cameron" <Jonathan.Cameron@Huawei.com>
Cc: bhelgaas@google.com, "Zhuo, Qiuxu" <qiuxu.zhuo@intel.com>,
	rjw@rjwysocki.net, "Luck, Tony" <tony.luck@intel.com>,
	sathyanarayanan.kuppuswamy@linux.intel.com,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, "Raj,
	Ashok" <ashok.raj@intel.com>
Subject: Re: [RFC PATCH 5/9] PCI/AER: Apply function level reset to RCiEP on fatal error
Date: Tue, 28 Jul 2020 10:42:33 -0700
Message-ID: <C21C050B-48B1-4429-B019-C81F3AB8E843@intel.com> (raw)
In-Reply-To: <20200728180235.00006ffe@Huawei.com>

On 28 Jul 2020, at 10:02, Jonathan Cameron wrote:

> On Tue, 28 Jul 2020 09:14:11 -0700
> Sean V Kelley <sean.v.kelley@intel.com> wrote:
>
>> On 28 Jul 2020, at 6:27, Zhuo, Qiuxu wrote:
>>
>>>> From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
>>>> Sent: Monday, July 27, 2020 7:17 PM
>>>> To: Kelley, Sean V <sean.v.kelley@intel.com>
>>>> Cc: bhelgaas@google.com; rjw@rjwysocki.net; ashok.raj@kernel.org;
>>>> Luck,
>>>> Tony <tony.luck@intel.com>;
>>>> sathyanarayanan.kuppuswamy@linux.intel.com;
>>>> linux-pci@vger.kernel.org;
>>>> linux-kernel@vger.kernel.org; Zhuo, Qiuxu <qiuxu.zhuo@intel.com>
>>>> Subject: Re: [RFC PATCH 5/9] PCI/AER: Apply function level reset to
>>>> RCiEP
>>>> on fatal error
>>>>
>>>> On Fri, 24 Jul 2020 10:22:19 -0700
>>>> Sean V Kelley <sean.v.kelley@intel.com> wrote:
>>>>
>>>>> From: Qiuxu Zhuo <qiuxu.zhuo@intel.com>
>>>>>
>>>>> Attempt to do function level reset for an RCiEP associated with an
>>>>> RCEC device on fatal error.
>>>>
>>>> I'd like to understand more on your reasoning for flr here.
>>>> Is it simply that it is all we can do, or is there some basis in a
>>>> spec
>>>> somewhere?
>>>>
>>>
>>> Yes. Though there isn't the link reset for the RCiEP here, I think 
>>> we
>>> should still be able to reset the RCiEP via FLR on fatal error, if 
>>> the
>>> RCiEP supports FLR.
>>>
>>> -Qiuxu
>>>
>>
>> Also see PCIe 5.0-1, Sec. 6.6.2 Function Level Reset (FLR)
>>
>> Implementation of FLR is optional (not required), but is strongly
>> recommended. For an example use case consider CXL. Function 0 DVSEC
>> instances control for the CXL functionality of the entire CXL device.
>> FLR may succeed in recovering from CXL.io domain errors.
>
> That feels a little bit of a weak argument in favour.  PCI spec lists 
> examples
> of use only for FLR and I can't see this matching any of them, but 
> then they
> are only examples, so we could argue it doesn't exclude this use. It's 
> not
> allowed to affect the link state, but I guess it 'might' recover from 
> some
> other type of error?
>
> I'd have read the statement in the CXL spec you are referring to as 
> matching
> with the first example in the PCIe spec which is about recovering from
> software errors.  For example, unexpected VM tear down.

 From my perspective, it can add value as the point is to address device 
functions and their associated software states. As the section in the 
spec goes on to state:

“The FLR mechanism enables software to quiesce and reset Endpoint 
hardware with Function-level granularity. Three example usage models 
illustrate the benefits of this feature:…”

Later changes in CXL 2.0 Section 9.8 (as of 0.9 draft) further look to 
extend FLR with an eFLR or now referred to as CXL Reset.

“All Functions in a CXL 2.0 (Single Logical Device) SLD that 
participate in CXL.cache or CXL.mem are required to support either FLR 
or CXL Reset. MLDs (Multiple Logical Devices), on the other hand, are 
required to support CXL Reset.”

In my mind the question is whether this change is too limited in scope 
with this patch series (RCiEP) and whether FLR should be considered in a 
broader, i.e., EP, as a ‘hammer’ so to speak.

Thanks,

Sean

>
> @Bjorn / All.  What's your view on using FLR as a reset to do when you 
> don't
> have any other hammers to use?
>
> Personally I don't have a particular problem with this, it just 
> doesn't fit
> with my mental model of what FLR is for (which may well need adjusting 
> :)
>
> Jonathan
>
>
>>
>> Thanks,
>>
>> Sean
>>
>>>>>
>>>>> Signed-off-by: Qiuxu Zhuo <qiuxu.zhuo@intel.com>
>>>>> ---
>>>>>  drivers/pci/pcie/err.c | 31 ++++++++++++++++++++++---------
>>>>>  1 file changed, 22 insertions(+), 9 deletions(-)
>>>>>
>>>>> diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c index
>>>>> 044df004f20b..9b3ec94bdf1d 100644
>>>>> --- a/drivers/pci/pcie/err.c
>>>>> +++ b/drivers/pci/pcie/err.c
>>>>> @@ -170,6 +170,17 @@ static void pci_walk_dev_affected(struct
>>>> pci_dev *dev, int (*cb)(struct pci_dev
>>>>>  }
>>>>>  }
>>>>>
>>>>> +static enum pci_channel_state flr_on_rciep(struct pci_dev *dev) {
>>>>> +if (!pcie_has_flr(dev))
>>>>> +return PCI_ERS_RESULT_NONE;
>>>>> +
>>>>> +if (pcie_flr(dev))
>>>>> +return PCI_ERS_RESULT_DISCONNECT;
>>>>> +
>>>>> +return PCI_ERS_RESULT_RECOVERED;
>>>>> +}
>>>>> +
>>>>>  pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
>>>>>  enum pci_channel_state state,
>>>>>  pci_ers_result_t (*reset_link)(struct pci_dev *pdev))
>>>> @@ -191,15
>>>>> +202,17 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
>>>>>  if (state == pci_channel_io_frozen) {
>>>>>  pci_walk_dev_affected(dev, report_frozen_detected,
>>>> &status);
>>>>>  if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_END) {
>>>>> -pci_warn(dev, "link reset not possible for RCiEP\n");
>>>>> -status = PCI_ERS_RESULT_NONE;
>>>>> -goto failed;
>>>>> -}
>>>>> -
>>>>> -status = reset_link(dev);
>>>>> -if (status != PCI_ERS_RESULT_RECOVERED) {
>>>>> -pci_warn(dev, "link reset failed\n");
>>>>> -goto failed;
>>>>> +status = flr_on_rciep(dev);
>>>>> +if (status != PCI_ERS_RESULT_RECOVERED) {
>>>>> +pci_warn(dev, "function level reset failed\n");
>>>>> +goto failed;
>>>>> +}
>>>>> +} else {
>>>>> +status = reset_link(dev);
>>>>> +if (status != PCI_ERS_RESULT_RECOVERED) {
>>>>> +pci_warn(dev, "link reset failed\n");
>>>>> +goto failed;
>>>>> +}
>>>>>  }
>>>>>  } else {
>>>>>  pci_walk_dev_affected(dev, report_normal_detected,
>>>> &status);
>>>>

  reply index

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-24 17:22 [RFC PATCH 0/9] Add RCEC handling to PCI/AER Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 1/9] pci_ids: Add class code and extended capability for RCEC Sean V Kelley
2020-07-27 10:00   ` Jonathan Cameron
2020-07-27 10:21     ` Jonathan Cameron
2020-07-27 15:22       ` Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 2/9] PCI: Extend Root Port Driver to support RCEC Sean V Kelley
2020-07-27 12:30   ` Jonathan Cameron
2020-07-27 15:05     ` Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 3/9] PCI/portdrv: Add pcie_walk_rcec() to walk RCiEPs associated with RCEC Sean V Kelley
2020-07-27 10:49   ` Jonathan Cameron
2020-07-27 15:21     ` Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 4/9] PCI/AER: Extend AER error handling to RCECs Sean V Kelley
2020-07-27 11:00   ` Jonathan Cameron
2020-07-27 14:58     ` Sean V Kelley
2020-07-27 14:04   ` Jonathan Cameron
2020-07-27 15:00     ` Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 5/9] PCI/AER: Apply function level reset to RCiEP on fatal error Sean V Kelley
2020-07-27 11:17   ` Jonathan Cameron
2020-07-28 13:27     ` Zhuo, Qiuxu
2020-07-28 16:14       ` Sean V Kelley
2020-07-28 17:02         ` Jonathan Cameron
2020-07-28 17:42           ` Sean V Kelley [this message]
2020-07-24 17:22 ` [RFC PATCH 6/9] PCI: Add 'rcec' field to pci_dev for associated RCiEPs Sean V Kelley
2020-07-27 11:23   ` Jonathan Cameron
2020-07-27 15:39     ` Sean V Kelley
2020-07-27 16:11     ` Jonathan Cameron
2020-07-27 16:28       ` Sean V Kelley
2020-07-24 17:22 ` [RFC PATCH 7/9] PCI/AER: Add RCEC AER handling Sean V Kelley
2020-07-27 12:22   ` Jonathan Cameron
2020-07-27 15:19     ` Sean V Kelley
2020-07-27 17:14       ` Jonathan Cameron
2020-07-24 17:22 ` [RFC PATCH 8/9] PCI/PME: Add RCEC PME handling Sean V Kelley
2020-08-04  8:35   ` Jay Fang
2020-08-04  9:47     ` Jonathan Cameron
2020-07-24 17:22 ` [RFC PATCH 9/9] PCI/AER: Add RCEC AER error injection support Sean V Kelley
2020-07-27 12:37 ` [RFC PATCH 0/9] Add RCEC handling to PCI/AER Jonathan Cameron
2020-07-27 14:56   ` Sean V Kelley

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=C21C050B-48B1-4429-B019-C81F3AB8E843@intel.com \
    --to=sean.v.kelley@intel.com \
    --cc=Jonathan.Cameron@Huawei.com \
    --cc=ashok.raj@intel.com \
    --cc=bhelgaas@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=qiuxu.zhuo@intel.com \
    --cc=rjw@rjwysocki.net \
    --cc=sathyanarayanan.kuppuswamy@linux.intel.com \
    --cc=tony.luck@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-PCI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \
		linux-pci@vger.kernel.org
	public-inbox-index linux-pci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git