linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: tip-bot for Thomas Gleixner <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: hpa@zytor.com, achirvasub@gmail.com, decui@microsoft.com,
	v-micos@microsoft.com, sixiao@microsoft.com,
	Jork.Loeser@microsoft.com, sthemmin@microsoft.com,
	alan@linux.intel.com, mikpelinux@gmail.com,
	haiyangz@microsoft.com, marc.zyngier@arm.com, mingo@kernel.org,
	bhelgaas@google.com, andriy.shevchenko@linux.intel.com,
	linux-kernel@vger.kernel.org, tglx@linutronix.de,
	saeedm@mellanox.com, macro@linux-mips.org,
	douly.fnst@cn.fujitsu.com, kys@microsoft.com,
	jopoulso@microsoft.com, pavel@ucw.cz
Subject: [tip:irq/urgent] genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI
Date: Fri, 29 Dec 2017 12:20:45 -0800	[thread overview]
Message-ID: <tip-bc976233a872c0f20f018fb1e89264a541584e25@git.kernel.org> (raw)
In-Reply-To: <alpine.DEB.2.20.1712291409460.1899@nanos>

Commit-ID:  bc976233a872c0f20f018fb1e89264a541584e25
Gitweb:     https://git.kernel.org/tip/bc976233a872c0f20f018fb1e89264a541584e25
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 29 Dec 2017 10:47:22 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Fri, 29 Dec 2017 21:13:05 +0100

genirq/msi, x86/vector: Prevent reservation mode for non maskable MSI

The new reservation mode for interrupts assigns a dummy vector when the
interrupt is allocated and assigns a real vector when the interrupt is
requested. The reservation mode prevents vector pressure when devices with
a large amount of queues/interrupts are initialized, but only a minimal
subset of those queues/interrupts is actually used.

This mode has an issue with MSI interrupts which cannot be masked. If the
driver is not careful or the hardware emits an interrupt before the device
irq is requestd by the driver then the interrupt ends up on the dummy
vector as a spurious interrupt which can cause malfunction of the device or
in the worst case a lockup of the machine.

Change the logic for the reservation mode so that the early activation of
MSI interrupts checks whether:

 - the device is a PCI/MSI device
 - the reservation mode of the underlying irqdomain is activated
 - PCI/MSI masking is globally enabled
 - the PCI/MSI device uses either MSI-X, which supports masking, or
   MSI with the maskbit supported.

If one of those conditions is false, then clear the reservation mode flag
in the irq data of the interrupt and invoke irq_domain_activate_irq() with
the reserve argument cleared. In the x86 vector code, clear the can_reserve
flag in the vector allocation data so a subsequent free_irq() won't create
the same situation again. The interrupt stays assigned to a real vector
until pci_disable_msi() is invoked and all allocations are undone.

Fixes: 4900be83602b ("x86/vector/msi: Switch to global reservation mode")
Reported-by: Alexandru Chirvasitu <achirvasub@gmail.com>
Reported-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Alexandru Chirvasitu <achirvasub@gmail.com>
Tested-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Dou Liyang <douly.fnst@cn.fujitsu.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Maciej W. Rozycki <macro@linux-mips.org>
Cc: Mikael Pettersson <mikpelinux@gmail.com>
Cc: Josh Poulson <jopoulso@microsoft.com>
Cc: Mihai Costache <v-micos@microsoft.com>
Cc: Stephen Hemminger <sthemmin@microsoft.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-pci@vger.kernel.org
Cc: Haiyang Zhang <haiyangz@microsoft.com>
Cc: Dexuan Cui <decui@microsoft.com>
Cc: Simon Xiao <sixiao@microsoft.com>
Cc: Saeed Mahameed <saeedm@mellanox.com>
Cc: Jork Loeser <Jork.Loeser@microsoft.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: devel@linuxdriverproject.org
Cc: KY Srinivasan <kys@microsoft.com>
Cc: Alan Cox <alan@linux.intel.com>
Cc: Sakari Ailus <sakari.ailus@intel.com>,
Cc: linux-media@vger.kernel.org
Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1712291406420.1899@nanos
Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1712291409460.1899@nanos
---
 arch/x86/kernel/apic/vector.c | 12 +++++++++++-
 kernel/irq/msi.c              | 37 +++++++++++++++++++++++++++++++++----
 2 files changed, 44 insertions(+), 5 deletions(-)

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 52c85c8..f8b03bb 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -369,8 +369,18 @@ static int activate_reserved(struct irq_data *irqd)
 	int ret;
 
 	ret = assign_irq_vector_any_locked(irqd);
-	if (!ret)
+	if (!ret) {
 		apicd->has_reserved = false;
+		/*
+		 * Core might have disabled reservation mode after
+		 * allocating the irq descriptor. Ideally this should
+		 * happen before allocation time, but that would require
+		 * completely convoluted ways of transporting that
+		 * information.
+		 */
+		if (!irqd_can_reserve(irqd))
+			apicd->can_reserve = false;
+	}
 	return ret;
 }
 
diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c
index 9ba9543..2f3c4f5 100644
--- a/kernel/irq/msi.c
+++ b/kernel/irq/msi.c
@@ -339,11 +339,38 @@ int msi_domain_populate_irqs(struct irq_domain *domain, struct device *dev,
 	return ret;
 }
 
-static bool msi_check_reservation_mode(struct msi_domain_info *info)
+/*
+ * Carefully check whether the device can use reservation mode. If
+ * reservation mode is enabled then the early activation will assign a
+ * dummy vector to the device. If the PCI/MSI device does not support
+ * masking of the entry then this can result in spurious interrupts when
+ * the device driver is not absolutely careful. But even then a malfunction
+ * of the hardware could result in a spurious interrupt on the dummy vector
+ * and render the device unusable. If the entry can be masked then the core
+ * logic will prevent the spurious interrupt and reservation mode can be
+ * used. For now reservation mode is restricted to PCI/MSI.
+ */
+static bool msi_check_reservation_mode(struct irq_domain *domain,
+				       struct msi_domain_info *info,
+				       struct device *dev)
 {
+	struct msi_desc *desc;
+
+	if (domain->bus_token != DOMAIN_BUS_PCI_MSI)
+		return false;
+
 	if (!(info->flags & MSI_FLAG_MUST_REACTIVATE))
 		return false;
-	return true;
+
+	if (IS_ENABLED(CONFIG_PCI_MSI) && pci_msi_ignore_mask)
+		return false;
+
+	/*
+	 * Checking the first MSI descriptor is sufficient. MSIX supports
+	 * masking and MSI does so when the maskbit is set.
+	 */
+	desc = first_msi_entry(dev);
+	return desc->msi_attrib.is_msix || desc->msi_attrib.maskbit;
 }
 
 /**
@@ -394,7 +421,7 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
 	if (ops->msi_finish)
 		ops->msi_finish(&arg, 0);
 
-	can_reserve = msi_check_reservation_mode(info);
+	can_reserve = msi_check_reservation_mode(domain, info, dev);
 
 	for_each_msi_entry(desc, dev) {
 		virq = desc->irq;
@@ -412,7 +439,9 @@ int msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev,
 			continue;
 
 		irq_data = irq_domain_get_irq_data(domain, desc->irq);
-		ret = irq_domain_activate_irq(irq_data, true);
+		if (!can_reserve)
+			irqd_clr_can_reserve(irq_data);
+		ret = irq_domain_activate_irq(irq_data, can_reserve);
 		if (ret)
 			goto cleanup;
 	}

  parent reply	other threads:[~2017-12-29 20:26 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-12-28 17:17 IRQ behaivour has been changed from v4.14 to v4.15-rc1 Shevchenko, Andriy
2017-12-28 17:21 ` Thomas Gleixner
2017-12-28 17:34   ` Andy Shevchenko
2017-12-28 17:44     ` Thomas Gleixner
2017-12-28 19:31       ` Andy Shevchenko
2017-12-28 19:36         ` Andy Shevchenko
2017-12-28 20:18         ` Thomas Gleixner
2017-12-28 21:03           ` Andy Shevchenko
2017-12-28 21:31             ` Thomas Gleixner
2017-12-28 21:59               ` Thomas Gleixner
2017-12-29 12:06                 ` Andy Shevchenko
2017-12-29 13:10                   ` Thomas Gleixner
2017-12-29 14:27                     ` Andy Shevchenko
2017-12-29 20:20                     ` tip-bot for Thomas Gleixner [this message]
2017-12-28 17:23 ` Andy Shevchenko
     [not found] <20171218082011.GA24638@arch-chirva.localdomain>
2017-12-18 10:11 ` PROBLEM: 4.15.0-rc3 APIC causes lockups on Core 2 Duo laptop Pavel Machek
2017-12-19  8:34   ` Alexandru Chirvasitu
2017-12-20  0:31     ` Thomas Gleixner
2017-12-20  3:58       ` Dou Liyang
2017-12-20 13:19         ` Alexandru Chirvasitu
2017-12-20 19:45           ` Alexandru Chirvasitu
2017-12-21  2:23             ` Alexandru Chirvasitu
2017-12-22 10:28               ` Dou Liyang
     [not found]                 ` <20171222142053.3cbhi2nhh24w7yoo@D-69-91-141-110.dhcp4.washington.edu>
2017-12-22 21:31                   ` Dexuan Cui
     [not found]                     ` <20171222222917.GA1138@arch-chirva.localdomain>
2017-12-23  1:35                       ` Dexuan Cui
2017-12-23  4:51                         ` Alexandru Chirvasitu
2017-12-23 13:32                         ` Thomas Gleixner
2017-12-23 20:01                           ` Alexandru Chirvasitu
2017-12-27  8:14                             ` Dou Liyang
2017-12-27 16:18                               ` Alexandru Chirvasitu
     [not found]                                 ` <20171227195007.GF1410@arch-chirva.localdomain>
2017-12-27 23:13                                   ` Alexandru Chirvasitu
2017-12-28  2:06                                 ` Dou Liyang
2017-12-28  2:51                                   ` Alexandru Chirvasitu
2017-12-28 10:23                                     ` Dou Liyang
2017-12-24  3:29                           ` Dou Liyang
2017-12-28 11:00           ` Thomas Gleixner
2017-12-28 14:21             ` Alexandru Chirvasitu
2017-12-28 14:48               ` Thomas Gleixner
2017-12-28 15:48                 ` Alexandru Chirvasitu
2017-12-28 16:05                   ` Alexandru Chirvasitu
2017-12-28 16:10                     ` Thomas Gleixner
2017-12-28 17:22                       ` Alexandru Chirvasitu
2017-12-28 17:29                         ` Thomas Gleixner
2017-12-28 17:50                           ` Alexandru Chirvasitu
2017-12-28 18:32                             ` Thomas Gleixner
2017-12-28 21:54                               ` Thomas Gleixner
2017-12-28 22:50                                 ` Alexandru Chirvasitu
2017-12-28 22:57                                   ` Thomas Gleixner
2017-12-28 23:19                                     ` Thomas Gleixner
2017-12-28 23:30                                       ` Alexandru Chirvasitu
2017-12-28 23:36                                         ` Thomas Gleixner
2017-12-28 23:59                                           ` Alexandru Chirvasitu
2017-12-29  8:07                                             ` Thomas Gleixner
2017-12-29 11:49                                               ` Alexandru Chirvasitu
2017-12-29 12:22                                                 ` Alexandru Chirvasitu
2017-12-29 13:09                                                 ` Thomas Gleixner
2017-12-29 14:06                                                   ` Alexandru Chirvasitu
2017-12-29  0:15                                         ` Bjorn Helgaas
2017-12-29  0:38                                           ` Alexandru Chirvasitu
2017-12-28 11:03           ` Thomas Gleixner
2017-12-28 19:01             ` Dexuan Cui
2017-12-28 20:14               ` Thomas Gleixner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tip-bc976233a872c0f20f018fb1e89264a541584e25@git.kernel.org \
    --to=tipbot@zytor.com \
    --cc=Jork.Loeser@microsoft.com \
    --cc=achirvasub@gmail.com \
    --cc=alan@linux.intel.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=bhelgaas@google.com \
    --cc=decui@microsoft.com \
    --cc=douly.fnst@cn.fujitsu.com \
    --cc=haiyangz@microsoft.com \
    --cc=hpa@zytor.com \
    --cc=jopoulso@microsoft.com \
    --cc=kys@microsoft.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=macro@linux-mips.org \
    --cc=marc.zyngier@arm.com \
    --cc=mikpelinux@gmail.com \
    --cc=mingo@kernel.org \
    --cc=pavel@ucw.cz \
    --cc=saeedm@mellanox.com \
    --cc=sixiao@microsoft.com \
    --cc=sthemmin@microsoft.com \
    --cc=tglx@linutronix.de \
    --cc=v-micos@microsoft.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).