From: "Raj, Ashok" <ashok.raj@intel.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: "Raj, Ashok" <ashok.raj@linux.intel.com>,
Evan Green <evgreen@chromium.org>,
Mathias Nyman <mathias.nyman@linux.intel.com>,
x86@kernel.org, linux-pci <linux-pci@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Bjorn Helgaas <bhelgaas@google.com>,
"Ghorai, Sukumar" <sukumar.ghorai@intel.com>,
"Amara, Madhusudanarao" <madhusudanarao.amara@intel.com>,
"Nandamuri, Srikanth" <srikanth.nandamuri@intel.com>,
Ashok Raj <ashok.raj@intel.com>
Subject: Re: MSI interrupt for xhci still lost on 5.6-rc6 after cpu hotplug
Date: Tue, 5 May 2020 13:16:16 -0700 [thread overview]
Message-ID: <20200505201616.GA15481@otc-nc-03> (raw)
In-Reply-To: <878si6rx7f.fsf@nanos.tec.linutronix.de>
On Tue, May 05, 2020 at 09:36:04PM +0200, Thomas Gleixner wrote:
> Ashok,
>
>
> > Now the second question with Interrupt Remapping Support:
> >
> > intel_ir_set_affinity->intel_ir_reconfigure_irte()-> modify_irte()
> >
> > The flush of Interrupt Entry Cache (IEC) should ensure, if any interrupts
> > were in flight, they made it to the previous CPU, and any new interrupts
> > must be delivered to the new CPU.
> >
> > Question is do we need a check similar to the legacy MSI handling
> >
> > if (lapic_vector_set_in_irr())
> > handle interrupt?
> >
> > Is there a reason we don't check if the interrupt delivered to previous
> > CPU in intel_ir_set_affinity()? Or is the send_cleanup_vector() sends
> > an IPI to perform the cleanup?
> >
> > It appears that maybe send_cleanup_vector() sends IPI to the old cpu
> > and that somehow ensures the device interrupt handler actually getting
> > called? I lost my track somewhere down there :)
>
> The way it works is:
>
> 1) New vector on different CPU is allocated
>
> 2) New vector is installed
>
> 3) When the first interrupt on the new CPU arrives then the cleanup
> IPI is sent to the previous CPU which tears down the old vector
>
> So if the interrupt is sent to the original CPU just before #2 then this
> will be correctly handled on that CPU after the set affinity code
> reenables interrupts.
I'll have this test tried out.. but in msi_set_affinity() the check below
for lapic_vector_set_in_irr(cfg->vector), this is the new vector correct?
don't we want to check for old_cfg.vector instead?
msi_set_affinit ()
{
....
unlock_vector_lock();
/*
* Check whether the transition raced with a device interrupt and
* is pending in the local APICs IRR. It is safe to do this outside
* of vector lock as the irq_desc::lock of this interrupt is still
* held and interrupts are disabled: The check is not accessing the
* underlying vector store. It's just checking the local APIC's
* IRR.
*/
if (lapic_vector_set_in_irr(cfg->vector))
irq_data_get_irq_chip(irqd)->irq_retrigger(irqd);
>
> Can you try the patch below?
>
> Thanks,
>
> tglx
>
> 8<--------------
> drivers/pci/msi.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> --- a/drivers/pci/msi.c
> +++ b/drivers/pci/msi.c
> @@ -323,6 +323,7 @@ void __pci_write_msi_msg(struct msi_desc
> writel(msg->address_lo, base + PCI_MSIX_ENTRY_LOWER_ADDR);
> writel(msg->address_hi, base + PCI_MSIX_ENTRY_UPPER_ADDR);
> writel(msg->data, base + PCI_MSIX_ENTRY_DATA);
> + readl(base + PCI_MSIX_ENTRY_DATA);
> } else {
> int pos = dev->msi_cap;
> u16 msgctl;
> @@ -343,6 +344,7 @@ void __pci_write_msi_msg(struct msi_desc
> pci_write_config_word(dev, pos + PCI_MSI_DATA_32,
> msg->data);
> }
> + pci_read_config_word(dev, pos + PCI_MSI_FLAGS, &msgctl);
> }
>
> skip:
next prev parent reply other threads:[~2020-05-05 20:16 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-18 19:25 MSI interrupt for xhci still lost on 5.6-rc6 after cpu hotplug Mathias Nyman
2020-03-19 20:24 ` Evan Green
2020-03-20 8:07 ` Mathias Nyman
2020-03-20 9:52 ` Thomas Gleixner
2020-03-23 9:42 ` Mathias Nyman
2020-03-23 14:10 ` Thomas Gleixner
2020-03-23 20:32 ` Mathias Nyman
2020-03-24 0:24 ` Thomas Gleixner
2020-03-24 16:17 ` Evan Green
2020-03-24 19:03 ` Thomas Gleixner
2020-05-01 18:43 ` Raj, Ashok
2020-05-05 19:36 ` Thomas Gleixner
2020-05-05 20:16 ` Raj, Ashok [this message]
2020-05-05 21:47 ` Thomas Gleixner
2020-05-07 12:18 ` Raj, Ashok
2020-05-07 12:53 ` Thomas Gleixner
[not found] ` <20200507175715.GA22426@otc-nc-03>
2020-05-07 19:41 ` Thomas Gleixner
2020-03-25 17:12 ` Mathias Nyman
[not found] <20200508005528.GB61703@otc-nc-03>
2020-05-08 11:04 ` Thomas Gleixner
2020-05-08 16:09 ` Raj, Ashok
2020-05-08 16:49 ` Thomas Gleixner
2020-05-11 19:03 ` Raj, Ashok
2020-05-11 20:14 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200505201616.GA15481@otc-nc-03 \
--to=ashok.raj@intel.com \
--cc=ashok.raj@linux.intel.com \
--cc=bhelgaas@google.com \
--cc=evgreen@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=madhusudanarao.amara@intel.com \
--cc=mathias.nyman@linux.intel.com \
--cc=srikanth.nandamuri@intel.com \
--cc=sukumar.ghorai@intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).