linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "tip-bot2 for Thomas Gleixner" <tip-bot2@linutronix.de>
To: linux-tip-commits@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>,
	Marc Zyngier <maz@kernel.org>, Ashok Raj <ashok.raj@intel.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	stable@vger.kernel.org, x86@kernel.org,
	linux-kernel@vger.kernel.org
Subject: [tip: irq/core] PCI/MSI: Enable and mask MSI-X early
Date: Tue, 10 Aug 2021 09:07:47 -0000	[thread overview]
Message-ID: <162858646777.395.4058447464180026703.tip-bot2@tip-bot2> (raw)
In-Reply-To: <20210729222542.344136412@linutronix.de>

The following commit has been merged into the irq/core branch of tip:

Commit-ID:     438553958ba19296663c6d6583d208dfb6792830
Gitweb:        https://git.kernel.org/tip/438553958ba19296663c6d6583d208dfb6792830
Author:        Thomas Gleixner <tglx@linutronix.de>
AuthorDate:    Thu, 29 Jul 2021 23:51:40 +02:00
Committer:     Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Tue, 10 Aug 2021 10:59:20 +02:00

PCI/MSI: Enable and mask MSI-X early

The ordering of MSI-X enable in hardware is dysfunctional:

 1) MSI-X is disabled in the control register
 2) Various setup functions
 3) pci_msi_setup_msi_irqs() is invoked which ends up accessing
    the MSI-X table entries
 4) MSI-X is enabled and masked in the control register with the
    comment that enabling is required for some hardware to access
    the MSI-X table

Step #4 obviously contradicts #3. The history of this is an issue with the
NIU hardware. When #4 was introduced the table access actually happened in
msix_program_entries() which was invoked after enabling and masking MSI-X.

This was changed in commit d71d6432e105 ("PCI/MSI: Kill redundant call of
irq_set_msi_desc() for MSI-X interrupts") which removed the table write
from msix_program_entries().

Interestingly enough nobody noticed and either NIU still works or it did
not get any testing with a kernel 3.19 or later.

Nevertheless this is inconsistent and there is no reason why MSI-X can't be
enabled and masked in the control register early on, i.e. move step #4
above to step #1. This preserves the NIU workaround and has no side effects
on other hardware.

Fixes: d71d6432e105 ("PCI/MSI: Kill redundant call of irq_set_msi_desc() for MSI-X interrupts")
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Ashok Raj <ashok.raj@intel.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20210729222542.344136412@linutronix.de

---
 drivers/pci/msi.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c
index 9232255..5d39ed8 100644
--- a/drivers/pci/msi.c
+++ b/drivers/pci/msi.c
@@ -772,18 +772,25 @@ static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries,
 	u16 control;
 	void __iomem *base;
 
-	/* Ensure MSI-X is disabled while it is set up */
-	pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
+	/*
+	 * Some devices require MSI-X to be enabled before the MSI-X
+	 * registers can be accessed.  Mask all the vectors to prevent
+	 * interrupts coming in before they're fully set up.
+	 */
+	pci_msix_clear_and_set_ctrl(dev, 0, PCI_MSIX_FLAGS_MASKALL |
+				    PCI_MSIX_FLAGS_ENABLE);
 
 	pci_read_config_word(dev, dev->msix_cap + PCI_MSIX_FLAGS, &control);
 	/* Request & Map MSI-X table region */
 	base = msix_map_region(dev, msix_table_size(control));
-	if (!base)
-		return -ENOMEM;
+	if (!base) {
+		ret = -ENOMEM;
+		goto out_disable;
+	}
 
 	ret = msix_setup_entries(dev, base, entries, nvec, affd);
 	if (ret)
-		return ret;
+		goto out_disable;
 
 	ret = pci_msi_setup_msi_irqs(dev, nvec, PCI_CAP_ID_MSIX);
 	if (ret)
@@ -794,14 +801,6 @@ static int msix_capability_init(struct pci_dev *dev, struct msix_entry *entries,
 	if (ret)
 		goto out_free;
 
-	/*
-	 * Some devices require MSI-X to be enabled before we can touch the
-	 * MSI-X registers.  We need to mask all the vectors to prevent
-	 * interrupts coming in before they're fully set up.
-	 */
-	pci_msix_clear_and_set_ctrl(dev, 0,
-				PCI_MSIX_FLAGS_MASKALL | PCI_MSIX_FLAGS_ENABLE);
-
 	msix_program_entries(dev, entries);
 
 	ret = populate_msi_sysfs(dev);
@@ -836,6 +835,9 @@ out_avail:
 out_free:
 	free_msi_irqs(dev);
 
+out_disable:
+	pci_msix_clear_and_set_ctrl(dev, PCI_MSIX_FLAGS_ENABLE, 0);
+
 	return ret;
 }
 

  reply	other threads:[~2021-08-10  9:08 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-29 21:51 [patch V2 00/19] PCI/MSI, x86: Cure a couple of inconsistencies Thomas Gleixner
2021-07-29 21:51 ` [patch V2 01/19] PCI/MSI: Enable and mask MSI-X early Thomas Gleixner
2021-08-10  9:07   ` tip-bot2 for Thomas Gleixner [this message]
2021-07-29 21:51 ` [patch V2 02/19] PCI/MSI: Mask all unused MSI-X entries Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 03/19] PCI/MSI: Enforce that MSI-X table entry is masked for update Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 04/19] PCI/MSI: Enforce MSI[X] entry updates to be visible Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 05/19] PCI/MSI: Do not set invalid bits in MSI mask Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 06/19] PCI/MSI: Correct misleading comments Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 07/19] PCI/MSI: Use msi_mask_irq() in pci_msi_shutdown() Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 08/19] PCI/MSI: Protect msi_desc::masked for multi-MSI Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 09/19] genirq: Provide IRQCHIP_AFFINITY_PRE_STARTUP Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 10/19] x86/ioapic: Force affinity setup before startup Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 11/19] x86/msi: " Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 12/19] s390/pci: Do not mask MSI[-X] entries on teardown Thomas Gleixner
2021-08-03 12:48   ` Niklas Schnelle
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 13/19] PCI/MSI: Simplify msi_verify_entries() Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 14/19] PCI/MSI: Rename msi_desc::masked Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 15/19] PCI/MSI: Consolidate error handling in msi_capability_init() Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 16/19] PCI/MSI: Deobfuscate virtual MSI-X Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 17/19] PCI/MSI: Cleanup msi_mask() Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 18/19] PCI/MSI: Provide a new set of mask and unmask functions Thomas Gleixner
     [not found]   ` <87r1f6bpt7.wl-maz@kernel.org>
2021-08-09 18:56     ` Thomas Gleixner
2021-08-09 19:08       ` [patch V3 " Thomas Gleixner
2021-08-10  9:07         ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-07-29 21:51 ` [patch V2 19/19] PCI/MSI: Use new mask/unmask functions Thomas Gleixner
2021-08-10  9:07   ` [tip: irq/core] " tip-bot2 for Thomas Gleixner
2021-08-10  7:49 ` [patch V2 00/19] PCI/MSI, x86: Cure a couple of inconsistencies Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=162858646777.395.4058447464180026703.tip-bot2@tip-bot2 \
    --to=tip-bot2@linutronix.de \
    --cc=ashok.raj@intel.com \
    --cc=bhelgaas@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=stable@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).