All of lore.kernel.org
 help / color / mirror / Atom feed
From: Eric Badger <ebadger@purestorage.com>
To: Sathyanarayanan Kuppuswamy  <sathyanarayanan.kuppuswamy@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
	Russell Currey <ruscur@russell.cc>,
	Oliver OHalloran <oohall@gmail.com>,
	linux-pci@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
	linux-kernel@vger.kernel.org, ebadger@purestorage.com
Subject: Re: [PATCH v2] PCI/AER: Handle Multi UnCorrectable/Correctable errors properly
Date: Tue, 15 Mar 2022 12:52:55 -0700	[thread overview]
Message-ID: <20220315195255.GA1523195@ebps> (raw)
In-Reply-To: <2d4e8811-dce6-c891-e92d-e3746434685e@linux.intel.com>

On Tue, Mar 15, 2022 at 10:26:46AM -0700, Sathyanarayanan Kuppuswamy wrote:
> On 3/15/22 10:14 AM, Eric Badger wrote:
> > >   # Prep injection data for a correctable error.
> > >   $ cd /sys/kernel/debug/apei/einj
> > >   $ echo 0x00000040 > error_type
> > >   $ echo 0x4 > flags
> > >   $ echo 0x891000 > param4
> > > 
> > >   # Root Error Status is initially clear
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w
> > >   0000
> > > 
> > >   # Inject one error
> > >   $ echo 1 > error_inject
> > > 
> > >   # Interrupt received
> > >   pcieport <Dev ID>: AER: Root Error Status 0001
> > > 
> > >   # Inject another error (within 5 seconds)
> > >   $ echo 1 > error_inject
> > > 
> > >   # No interrupt received, but "multiple ERR_COR" is now set
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w
> > >   0003
> > > 
> > >   # Wait for a while, then clear ERR_COR. A new interrupt immediately
> > >     fires.
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w=0x1
> > >   pcieport <Dev ID>: AER: Root Error Status 0002
> > > 
> > > Currently, the above issue has been only reproduced in the ICL server
> > > platform.
> > > 
> > > [Eric: proposed reproducing steps]
> > Hmm, this differs from the procedure I described on v1, and I don't
> > think will work as described here.
> 
> I have attempted to modify the steps to reproduce it without returning
> IRQ_NONE for all cases (which will break the functionality). But I
> think I did not correct the last few steps.

Well, the thinking in always returning IRQ_NONE was so that only setpci
modified the register and we could clearly see how writes to the
register affect interrupt generation.

> How about replacing the last 3 steps with following?
> 
>  # Inject another error (within 5 seconds)
>  $ echo 1 > error_inject
> 
>  # You will get a new IRQ with only multiple ERR_COR bit set
>  pcieport <Dev ID>: AER: Root Error Status 0002

This seems accurate. Though it does muddy a detail that I think was
clearer in the original procedure: was the second interrupt triggered by
the second error, or by the write of 0x1 to Root Error Status?

Also, in terms of practically running the test, I find the mdelay() can
block other interrupts and can make running the test sort of confusing
("is it not printing because the interrupt didn't fire, or because it's
spinning and blocking my NIC driver?" :).

Cheers,
Eric

WARNING: multiple messages have this Message-ID (diff)
From: Eric Badger <ebadger@purestorage.com>
To: Sathyanarayanan Kuppuswamy <sathyanarayanan.kuppuswamy@linux.intel.com>
Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	ebadger@purestorage.com, Oliver OHalloran <oohall@gmail.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v2] PCI/AER: Handle Multi UnCorrectable/Correctable errors properly
Date: Tue, 15 Mar 2022 12:52:55 -0700	[thread overview]
Message-ID: <20220315195255.GA1523195@ebps> (raw)
In-Reply-To: <2d4e8811-dce6-c891-e92d-e3746434685e@linux.intel.com>

On Tue, Mar 15, 2022 at 10:26:46AM -0700, Sathyanarayanan Kuppuswamy wrote:
> On 3/15/22 10:14 AM, Eric Badger wrote:
> > >   # Prep injection data for a correctable error.
> > >   $ cd /sys/kernel/debug/apei/einj
> > >   $ echo 0x00000040 > error_type
> > >   $ echo 0x4 > flags
> > >   $ echo 0x891000 > param4
> > > 
> > >   # Root Error Status is initially clear
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w
> > >   0000
> > > 
> > >   # Inject one error
> > >   $ echo 1 > error_inject
> > > 
> > >   # Interrupt received
> > >   pcieport <Dev ID>: AER: Root Error Status 0001
> > > 
> > >   # Inject another error (within 5 seconds)
> > >   $ echo 1 > error_inject
> > > 
> > >   # No interrupt received, but "multiple ERR_COR" is now set
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w
> > >   0003
> > > 
> > >   # Wait for a while, then clear ERR_COR. A new interrupt immediately
> > >     fires.
> > >   $ setpci -s <Dev ID> ECAP0001+0x30.w=0x1
> > >   pcieport <Dev ID>: AER: Root Error Status 0002
> > > 
> > > Currently, the above issue has been only reproduced in the ICL server
> > > platform.
> > > 
> > > [Eric: proposed reproducing steps]
> > Hmm, this differs from the procedure I described on v1, and I don't
> > think will work as described here.
> 
> I have attempted to modify the steps to reproduce it without returning
> IRQ_NONE for all cases (which will break the functionality). But I
> think I did not correct the last few steps.

Well, the thinking in always returning IRQ_NONE was so that only setpci
modified the register and we could clearly see how writes to the
register affect interrupt generation.

> How about replacing the last 3 steps with following?
> 
>  # Inject another error (within 5 seconds)
>  $ echo 1 > error_inject
> 
>  # You will get a new IRQ with only multiple ERR_COR bit set
>  pcieport <Dev ID>: AER: Root Error Status 0002

This seems accurate. Though it does muddy a detail that I think was
clearer in the original procedure: was the second interrupt triggered by
the second error, or by the write of 0x1 to Root Error Status?

Also, in terms of practically running the test, I find the mdelay() can
block other interrupts and can make running the test sort of confusing
("is it not printing because the interrupt didn't fire, or because it's
spinning and blocking my NIC driver?" :).

Cheers,
Eric

  reply	other threads:[~2022-03-15 19:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-15  5:08 [PATCH v2] PCI/AER: Handle Multi UnCorrectable/Correctable errors properly Kuppuswamy Sathyanarayanan
2022-03-15  5:08 ` Kuppuswamy Sathyanarayanan
2022-03-15 17:14 ` Eric Badger
2022-03-15 17:14   ` Eric Badger
2022-03-15 17:26   ` Sathyanarayanan Kuppuswamy
2022-03-15 17:26     ` Sathyanarayanan Kuppuswamy
2022-03-15 19:52     ` Eric Badger [this message]
2022-03-15 19:52       ` Eric Badger
2022-03-15 21:29       ` Sathyanarayanan Kuppuswamy
2022-03-15 21:29         ` Sathyanarayanan Kuppuswamy
2022-03-16 16:27         ` Eric Badger
2022-03-16 16:27           ` Eric Badger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220315195255.GA1523195@ebps \
    --to=ebadger@purestorage.com \
    --cc=bhelgaas@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=oohall@gmail.com \
    --cc=ruscur@russell.cc \
    --cc=sathyanarayanan.kuppuswamy@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.