From: Josef Johansson <josef@oderland.se>
To: Marc Zyngier <maz@kernel.org>
Cc: Bjorn Helgaas <helgaas@kernel.org>,
linux-pci@vger.kernel.org,
xen-devel <xen-devel@lists.xenproject.org>,
Thomas Gleixner <tglx@linutronix.de>,
Juergen Gross <jgross@suse.com>,
Boris Ostrovsky <boris.ostrovsky@oracle.com>,
Jason Andryuk <jandryuk@gmail.com>
Subject: Re: [PATCH v2] PCI/MSI: Re-add checks for skip masking MSI-X on Xen PV
Date: Thu, 21 Oct 2021 10:25:59 +0200 [thread overview]
Message-ID: <3434cb2d-4060-7969-d4c4-089c68190527@oderland.se> (raw)
In-Reply-To: <CAKf6xpt=ZYGyJXMwM7ccOWkx71R0O-QeLjkBF-LtdDcbSnzHsA@mail.gmail.com>
On 10/20/21 16:03, Jason Andryuk wrote:
> Hi, Marc,
>
> Adding Juergen and Boris since this involves Xen.
>
> On Wed, Oct 20, 2021 at 8:51 AM Marc Zyngier <maz@kernel.org> wrote:
>> On Tue, 19 Oct 2021 22:48:19 +0100,
>> Josef Johansson <josef@oderland.se> wrote:
>>> From: Josef Johansson <josef@oderland.se>
>>>
>>>
>>> PCI/MSI: Re-add checks for skip masking MSI-X on Xen PV
>>>
>>> commit fcacdfbef5a1 ("PCI/MSI: Provide a new set of mask and unmask
>>> functions") introduce functions pci_msi_update_mask() and
>>> pci_msix_write_vector_ctrl() that is missing checks for
>>> pci_msi_ignore_mask that exists in commit 446a98b19fd6 ("PCI/MSI: Use
>>> new mask/unmask functions"). Add them back since it is
>>> causing severe lockups in amdgpu drivers under Xen during boot.
>>>
>>> As explained in commit 1a519dc7a73c ("PCI/MSI: Skip masking MSI-X
>>> on Xen PV"), when running as Xen PV guest, masking MSI-X is a
>>> responsibility of the hypervisor.
>>>
>>> Fixes: fcacdfbef5a1 ("PCI/MSI: Provide a new set of mask and unmask
>>> functions")
>>> Suggested-by: Jason Andryuk <jandryuk@gmail.com>
>>> Signed-off-by: Josef Johansson <josef@oderland.se>
>>>
>> [...]
>>
>>> diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
>>> index 0099a00af361..355b791e382f 100644
>>> --- a/drivers/pci/msi.c
>>> +++ b/drivers/pci/msi.c
>>> @@ -148,6 +148,9 @@ static noinline void pci_msi_update_mask(struct msi_desc *desc, u32 clear, u32 s
>>> raw_spinlock_t *lock = &desc->dev->msi_lock;
>>> unsigned long flags;
>>>
>>> + if (pci_msi_ignore_mask || desc->msi_attrib.is_virtual)
>>> + return;
>>> +
>> I'd rather be consistent, and keep the check outside of
>> pci_msi_update_mask(), just like we do in __pci_msi_mask_desc().
>> Something like this instead:
>>
>> diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
>> index 0099a00af361..6c69eab304ce 100644
>> --- a/drivers/pci/msi.c
>> +++ b/drivers/pci/msi.c
>> @@ -420,7 +420,8 @@ static void __pci_restore_msi_state(struct pci_dev *dev)
>> arch_restore_msi_irqs(dev);
>>
>> pci_read_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, &control);
>> - pci_msi_update_mask(entry, 0, 0);
>> + if (!(pci_msi_ignore_mask || desc->msi_attrib.is_virtual))
>> + pci_msi_update_mask(entry, 0, 0);
>> control &= ~PCI_MSI_FLAGS_QSIZE;
>> control |= (entry->msi_attrib.multiple << 4) | PCI_MSI_FLAGS_ENABLE;
>> pci_write_config_word(dev, dev->msi_cap + PCI_MSI_FLAGS, control);
>>
>> But the commit message talks about MSI-X, and the above is MSI
>> only. Is Xen messing with the former, the latter, or both?
> My understanding is pci_msi_ignore_mask covers both MSI and MSI-X for Xen.
Please let me know if I should go ahead and try it out and send in a v3
of the patch.
I'm watching for further discussion right now, just to be clear.
>>> raw_spin_lock_irqsave(lock, flags);
>>> desc->msi_mask &= ~clear;
>>> desc->msi_mask |= set;
>>> @@ -181,6 +184,9 @@ static void pci_msix_write_vector_ctrl(struct msi_desc *desc, u32 ctrl)
>>> {
>>> void __iomem *desc_addr = pci_msix_desc_addr(desc);
>>>
>>> + if (pci_msi_ignore_mask || desc->msi_attrib.is_virtual)
>>> + return;
>>> +
>>> writel(ctrl, desc_addr + PCI_MSIX_ENTRY_VECTOR_CTRL);
>>> }
>> I have similar reservations for this one.
> The problem here is some of the changes in commit 446a98b19fd6
> ("PCI/MSI: Use new mask/unmask functions") bypass the checks in
> __pci_msi_mask_desc/__pci_msi_unmask_desc. I've wondered if it would
> be cleaner to push all the `if (pci_msi_ignore_mask)` checks down to
> the place of the writes. That keeps dropping the write local to the
> write and leaves the higher level code consistent between the regular
> and Xen PV cases. I don't know where checking
> desc->msi_attrib.is_virtual is appropriate.
>
> Regards,
> Jason
next prev parent reply other threads:[~2021-10-21 8:26 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-18 6:22 [PATCH] PCI/MSI: Re-add checks for skip masking MSI-X on Xen PV Josef Johansson
2021-10-19 19:57 ` Bjorn Helgaas
2021-10-19 20:15 ` Josef Johansson
2021-10-19 20:29 ` Bjorn Helgaas
2021-10-19 21:48 ` [PATCH v2] " Josef Johansson
2021-10-20 12:51 ` Marc Zyngier
2021-10-20 14:03 ` Jason Andryuk
2021-10-21 8:25 ` Josef Johansson [this message]
2021-10-24 18:55 ` Josef Johansson
2021-10-25 1:25 ` [PATCH] PCI/MSI: Fix masking MSI/MSI-X " Jason Andryuk
2021-10-25 7:44 ` David Woodhouse
2021-10-25 11:43 ` Roger Pau Monné
2021-10-25 11:53 ` David Woodhouse
2021-10-25 12:58 ` Roger Pau Monné
2021-10-25 13:02 ` David Woodhouse
2021-10-25 14:12 ` Roger Pau Monné
2021-10-25 12:31 ` Jason Andryuk
2021-10-25 12:27 ` Jason Andryuk
2021-10-25 16:46 ` Josef Johansson
2021-10-26 21:17 ` Josef Johansson
2021-10-27 8:45 ` Thomas Gleixner
2021-10-27 9:50 ` [PATCH] PCI/MSI: Move non-mask check back into low level accessors Thomas Gleixner
2021-10-27 9:54 ` Josef Johansson
2021-10-27 12:01 ` Josef Johansson
2021-10-27 15:29 ` Josef Johansson
2021-11-03 23:26 ` Thomas Gleixner
2021-11-03 23:27 ` [PATCH v2] " Thomas Gleixner
2021-11-09 14:53 ` Thomas Gleixner
2021-11-10 13:31 ` Josef Johansson
2021-11-10 16:05 ` Josef Johansson
2021-11-03 23:45 ` [PATCH] " Thomas Gleixner
2021-11-04 9:00 ` Josef Johansson
2021-11-04 17:12 ` Peter Zijlstra
2021-11-04 17:31 ` Vincent Guittot
2021-11-10 20:30 ` Josef Johansson
2021-11-10 23:13 ` Josef Johansson
2021-10-27 9:57 ` David Woodhouse
2021-10-25 1:25 ` [PATCH v2] PCI/MSI: Re-add checks for skip masking MSI-X on Xen PV Jason Andryuk
2021-10-25 19:21 ` Josef Johansson
2021-10-27 6:24 ` David Woodhouse
2021-10-27 8:13 ` Josef Johansson
2021-10-27 8:26 ` David Woodhouse
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3434cb2d-4060-7969-d4c4-089c68190527@oderland.se \
--to=josef@oderland.se \
--cc=boris.ostrovsky@oracle.com \
--cc=helgaas@kernel.org \
--cc=jandryuk@gmail.com \
--cc=jgross@suse.com \
--cc=linux-pci@vger.kernel.org \
--cc=maz@kernel.org \
--cc=tglx@linutronix.de \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).