[PATCH] genirq: Add default affinity mask command line option

[PATCH] genirq: Add default affinity mask command line option

[Thomas Gleixner]

From: Thomas Gleixner <tglx@linutronix.de>
Date: Fri May 25 16:59:47 2012 +0200
Subject: genirq: Add default affinity mask command line option

If we isolate CPUs, then we don't want random device interrupts on them. Even
w/o the user space irq balancer enabled we can end up with irqs on non boot
cpus and chasing newly requested interrupts is a tedious task.

Allow to restrict the default irq affinity mask.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 Documentation/kernel-parameters.txt |    9 +++++++++
 kernel/irq/irqdesc.c                |   21 +++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1682,6 +1682,15 @@ bytes respectively. Such letter suffixes
 	ip=		[IP_PNP]
 			See Documentation/filesystems/nfs/nfsroot.txt.
 
+	irqaffinity=	[SMP] Set the default irq affinity mask
+			Format:
+			<cpu number>,...,<cpu number>
+			or
+			<cpu number>-<cpu number>
+			(must be a positive range in ascending order)
+			or a mixture
+			<cpu number>,...,<cpu number>-<cpu number>
+
 	irqfixup	[HW]
 			When an interrupt is not handled search all handlers
 			for it. Intended to get systems with badly broken
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -24,10 +24,27 @@
 static struct lock_class_key irq_desc_lock_class;
 
 #if defined(CONFIG_SMP)
+static int __init irq_affinity_setup(char *str)
+{
+	zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+	cpulist_parse(str, irq_default_affinity);
+	/*
+	 * Set at least the boot cpu. We don't want to end up with
+	 * bugreports caused by random comandline masks
+	 */
+	cpumask_set_cpu(smp_processor_id(), irq_default_affinity);
+	return 1;
+}
+__setup("irqaffinity=", irq_affinity_setup);
+
 static void __init init_irq_default_affinity(void)
 {
-	alloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
-	cpumask_setall(irq_default_affinity);
+#ifdef CONFIG_CPUMASK_OFFSTACK
+	if (!irq_default_affinity)
+		zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+#endif
+	if (cpumask_empty(irq_default_affinity))
+		cpumask_setall(irq_default_affinity);
 }
 #else
 static void __init init_irq_default_affinity(void)

Re: [PATCH] genirq: Add default affinity mask command line option

[Rik van Riel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 02/03/2016 01:52 PM, Thomas Gleixner wrote:
> From: Thomas Gleixner <tglx@linutronix.de> Date: Fri May 25
> 16:59:47 2012 +0200 Subject: genirq: Add default affinity mask
> command line option
> 
> If we isolate CPUs, then we don't want random device interrupts on
> them. Even w/o the user space irq balancer enabled we can end up
> with irqs on non boot cpus and chasing newly requested interrupts
> is a tedious task.

The userspace irqbalance daemon has code nowadays to prevent
it from putting irqs on isolated CPUs.

> Allow to restrict the default irq affinity mask.

However, not placing them on there at boot time is a great
enhancement, too.

> --- a/kernel/irq/irqdesc.c +++ b/kernel/irq/irqdesc.c @@ -24,10
> +24,27 @@ static struct lock_class_key irq_desc_lock_class;
> 
> #if defined(CONFIG_SMP) +static int __init irq_affinity_setup(char
> *str) +{ +	zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT); +
> cpulist_parse(str, irq_default_affinity); +	/* +	 * Set at least
> the boot cpu. We don't want to end up with +	 * bugreports caused
> by random comandline masks +	 */ +
> cpumask_set_cpu(smp_processor_id(), irq_default_affinity); +	return
> 1; +} +__setup("irqaffinity=", irq_affinity_setup); + static void
> __init init_irq_default_affinity(void) { -
> alloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT); -
> cpumask_setall(irq_default_affinity); +#ifdef
> CONFIG_CPUMASK_OFFSTACK +	if (!irq_default_affinity) +
> zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT); +#endif +	if
> (cpumask_empty(irq_default_affinity)) +
> cpumask_setall(irq_default_affinity); }

Would it make sense to key off the isolated_cpus mask
if isolated_cpus= was specified, but no irqaffinity
mask was specified?

Is there any use case where we would want irqs on
isolated cpus by default, and could not bear the
cost of reassigning them from userspace after boot?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWslEWAAoJEM553pKExN6Dju0IAKDxqU2zS3rcC5jGfogfN9Q7
4XFOksNJ2VLfpDR/tSw6LTIkrDyl4BvhWv0WXcathfl3yVh1TEPsSWLUi1/ctL8U
SbxK1+zZvja9UXjPL/KDoJLeaQTd9C8nA+lhP+UUkR/A3g23WezFcJVB+a8CYaqg
pBQ4G1pz2xSfzraRe9HzzIb076QuieVUDllc+rdVxjBQRhmTBJ40MIxspTdRpNCY
wx7PadVP5UiY/gPX7Q+KrQNlybBs3x0dq4DenaOmpULeBa1sdpgeRJX8K01OcXgE
1wcKk8RYmzgw2SPNvkO97Gyl8Acj24+ceHerzMmV2aUiWS98LU3faISQgB2ZUwg=
=+ZYl
-----END PGP SIGNATURE-----

Re: [PATCH] genirq: Add default affinity mask command line option

[Thomas Gleixner]

On Wed, 3 Feb 2016, Rik van Riel wrote:
> On 02/03/2016 01:52 PM, Thomas Gleixner wrote:
> > From: Thomas Gleixner <tglx@linutronix.de> Date: Fri May 25
> > 16:59:47 2012 +0200 Subject: genirq: Add default affinity mask
> > command line option
> > 
> > If we isolate CPUs, then we don't want random device interrupts on
> > them. Even w/o the user space irq balancer enabled we can end up
> > with irqs on non boot cpus and chasing newly requested interrupts
> > is a tedious task.
> 
> The userspace irqbalance daemon has code nowadays to prevent
> it from putting irqs on isolated CPUs.

I know, but a lot of systems do not enable it at all.

> > Allow to restrict the default irq affinity mask.
> 
> However, not placing them on there at boot time is a great
> enhancement, too.

That's the main plan :)
 
> Would it make sense to key off the isolated_cpus mask
> if isolated_cpus= was specified, but no irqaffinity
> mask was specified?

That might be an add on.
 
> Is there any use case where we would want irqs on
> isolated cpus by default, and could not bear the
> cost of reassigning them from userspace after boot?

I don't think so. Those interrupts you want on an isolated cpu are obviously
hand selected.

Thanks,

	tglx

[tip:irq/core] genirq: Add default affinity mask command line option

[tip-bot for Thomas Gleixner]

Commit-ID:  35fdd6e4bf0868ed9b1d8662fc9fca766d6963d1
Gitweb:     http://git.kernel.org/tip/35fdd6e4bf0868ed9b1d8662fc9fca766d6963d1
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Wed, 3 Feb 2016 19:52:23 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 8 Feb 2016 11:45:22 +0100

genirq: Add default affinity mask command line option

If we isolate CPUs, then we don't want random device interrupts on them. Even
w/o the user space irq balancer enabled we can end up with irqs on non boot
cpus and chasing newly requested interrupts is a tedious task.

Allow to restrict the default irq affinity mask.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Chris Metcalf <cmetcalf@ezchip.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Link: http://lkml.kernel.org/r/alpine.DEB.2.11.1602031948190.25254@nanos
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 Documentation/kernel-parameters.txt |  9 +++++++++
 kernel/irq/irqdesc.c                | 21 +++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 551ecf0..87298f8 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1687,6 +1687,15 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 	ip=		[IP_PNP]
 			See Documentation/filesystems/nfs/nfsroot.txt.
 
+	irqaffinity=	[SMP] Set the default irq affinity mask
+			Format:
+			<cpu number>,...,<cpu number>
+			or
+			<cpu number>-<cpu number>
+			(must be a positive range in ascending order)
+			or a mixture
+			<cpu number>,...,<cpu number>-<cpu number>
+
 	irqfixup	[HW]
 			When an interrupt is not handled search all handlers
 			for it. Intended to get systems with badly broken
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 0409da0..0ccd028 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -24,10 +24,27 @@
 static struct lock_class_key irq_desc_lock_class;
 
 #if defined(CONFIG_SMP)
+static int __init irq_affinity_setup(char *str)
+{
+	zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+	cpulist_parse(str, irq_default_affinity);
+	/*
+	 * Set at least the boot cpu. We don't want to end up with
+	 * bugreports caused by random comandline masks
+	 */
+	cpumask_set_cpu(smp_processor_id(), irq_default_affinity);
+	return 1;
+}
+__setup("irqaffinity=", irq_affinity_setup);
+
 static void __init init_irq_default_affinity(void)
 {
-	alloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
-	cpumask_setall(irq_default_affinity);
+#ifdef CONFIG_CPUMASK_OFFSTACK
+	if (!irq_default_affinity)
+		zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+#endif
+	if (cpumask_empty(irq_default_affinity))
+		cpumask_setall(irq_default_affinity);
 }
 #else
 static void __init init_irq_default_affinity(void)

[tip:irq/core] genirq: Add default affinity mask command line option

[tip-bot for Thomas Gleixner]

Commit-ID:  fbf198030e0b027538c290300cfaf9e2efdce122
Gitweb:     http://git.kernel.org/tip/fbf198030e0b027538c290300cfaf9e2efdce122
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Wed, 3 Feb 2016 19:52:23 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 8 Feb 2016 15:03:42 +0100

genirq: Add default affinity mask command line option

If we isolate CPUs, then we don't want random device interrupts on them. Even
w/o the user space irq balancer enabled we can end up with irqs on non boot
cpus and chasing newly requested interrupts is a tedious task.

Allow to restrict the default irq affinity mask.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Chris Metcalf <cmetcalf@ezchip.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Link: http://lkml.kernel.org/r/alpine.DEB.2.11.1602031948190.25254@nanos
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 Documentation/kernel-parameters.txt |  9 +++++++++
 kernel/irq/irqdesc.c                | 21 +++++++++++++++++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 551ecf0..87298f8 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1687,6 +1687,15 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 	ip=		[IP_PNP]
 			See Documentation/filesystems/nfs/nfsroot.txt.
 
+	irqaffinity=	[SMP] Set the default irq affinity mask
+			Format:
+			<cpu number>,...,<cpu number>
+			or
+			<cpu number>-<cpu number>
+			(must be a positive range in ascending order)
+			or a mixture
+			<cpu number>,...,<cpu number>-<cpu number>
+
 	irqfixup	[HW]
 			When an interrupt is not handled search all handlers
 			for it. Intended to get systems with badly broken
diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c
index 0409da0..0ccd028 100644
--- a/kernel/irq/irqdesc.c
+++ b/kernel/irq/irqdesc.c
@@ -24,10 +24,27 @@
 static struct lock_class_key irq_desc_lock_class;
 
 #if defined(CONFIG_SMP)
+static int __init irq_affinity_setup(char *str)
+{
+	zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+	cpulist_parse(str, irq_default_affinity);
+	/*
+	 * Set at least the boot cpu. We don't want to end up with
+	 * bugreports caused by random comandline masks
+	 */
+	cpumask_set_cpu(smp_processor_id(), irq_default_affinity);
+	return 1;
+}
+__setup("irqaffinity=", irq_affinity_setup);
+
 static void __init init_irq_default_affinity(void)
 {
-	alloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
-	cpumask_setall(irq_default_affinity);
+#ifdef CONFIG_CPUMASK_OFFSTACK
+	if (!irq_default_affinity)
+		zalloc_cpumask_var(&irq_default_affinity, GFP_NOWAIT);
+#endif
+	if (cpumask_empty(irq_default_affinity))
+		cpumask_setall(irq_default_affinity);
 }
 #else
 static void __init init_irq_default_affinity(void)