[PATCH] Use NUMA node cpu mask in irq affinity

[PATCH] Use NUMA node cpu mask in irq affinity

[Prarit Bhargava]

The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used
to "pin" an irq to a set of cpus.  On boot this set is currently all cpus.
This can be incorrect as ACPI SRAT may tell us that a specific device or
bus is attached to a particular node and it's cpus.

When setting up the irq affinity we should take into account the NUMA node
cpu mask by and'ing it into the irq's affinity mask.

Signed-off-by: Prarit Bhargava <prarit@redhat.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
---
 kernel/irq/manage.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index a9a9dbe..2fb3469 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
 	}
 
 	cpumask_and(mask, cpu_online_mask, set);
+	if (desc->irq_data.node != -1)
+		cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
 	ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
 	switch (ret) {
 	case IRQ_SET_MASK_OK:
-- 
1.7.1

Re: [PATCH] Use NUMA node cpu mask in irq affinity

[Yinghai Lu]

On Wed, Feb 8, 2012 at 12:48 PM, Prarit Bhargava <prarit@redhat.com> wrote:
> The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used
> to "pin" an irq to a set of cpus.  On boot this set is currently all cpus.
> This can be incorrect as ACPI SRAT may tell us that a specific device or
> bus is attached to a particular node and it's cpus.
>
> When setting up the irq affinity we should take into account the NUMA node
> cpu mask by and'ing it into the irq's affinity mask.
>
> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
> Acked-by: Neil Horman <nhorman@tuxdriver.com>
> ---
>  kernel/irq/manage.c |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
>
> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
> index a9a9dbe..2fb3469 100644
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
>        }
>
>        cpumask_and(mask, cpu_online_mask, set);
> +       if (desc->irq_data.node != -1)
> +               cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
>        ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
>        switch (ret) {
>        case IRQ_SET_MASK_OK:
> --

How about all cpus on that node get offlined?

Yinghai

Re: [PATCH] Use NUMA node cpu mask in irq affinity

[Prarit Bhargava]

On 02/08/2012 06:51 PM, Yinghai Lu wrote:
> On Wed, Feb 8, 2012 at 12:48 PM, Prarit Bhargava <prarit@redhat.com> wrote:
>> The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used
>> to "pin" an irq to a set of cpus.  On boot this set is currently all cpus.
>> This can be incorrect as ACPI SRAT may tell us that a specific device or
>> bus is attached to a particular node and it's cpus.
>>
>> When setting up the irq affinity we should take into account the NUMA node
>> cpu mask by and'ing it into the irq's affinity mask.
>>
>> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
>> Acked-by: Neil Horman <nhorman@tuxdriver.com>
>> ---
>>  kernel/irq/manage.c |    2 ++
>>  1 files changed, 2 insertions(+), 0 deletions(-)
>>
>> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
>> index a9a9dbe..2fb3469 100644
>> --- a/kernel/irq/manage.c
>> +++ b/kernel/irq/manage.c
>> @@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
>>        }
>>
>>        cpumask_and(mask, cpu_online_mask, set);
>> +       if (desc->irq_data.node != -1)
>> +               cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));
>>        ret = chip->irq_set_affinity(&desc->irq_data, mask, false);
>>        switch (ret) {
>>        case IRQ_SET_MASK_OK:
>> --
> 
> How about all cpus on that node get offlined?

Good point.  I guess that also goes to tglx's comments wondering what happens if the mask ends up being zero.

I'll think about that and send out a new patch shortly.

P.
> 
> Yinghai

Re: [PATCH] Use NUMA node cpu mask in irq affinity

[Thomas Gleixner]

On Wed, 8 Feb 2012, Prarit Bhargava wrote:

Could you please CC the relevant maintainers ?

> The irq affinity files (/proc/irq/.../smp_affinity) contain a mask that is used
> to "pin" an irq to a set of cpus.  On boot this set is currently all cpus.
> This can be incorrect as ACPI SRAT may tell us that a specific device or
> bus is attached to a particular node and it's cpus.
> 
> When setting up the irq affinity we should take into account the NUMA node
> cpu mask by and'ing it into the irq's affinity mask.
> 
> Signed-off-by: Prarit Bhargava <prarit@redhat.com>
> Acked-by: Neil Horman <nhorman@tuxdriver.com>
> ---
>  kernel/irq/manage.c |    2 ++
>  1 files changed, 2 insertions(+), 0 deletions(-)
> 
> diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
> index a9a9dbe..2fb3469 100644
> --- a/kernel/irq/manage.c
> +++ b/kernel/irq/manage.c
> @@ -301,6 +301,8 @@ setup_affinity(unsigned int irq, struct irq_desc *desc, struct cpumask *mask)
>  	}
>  
>  	cpumask_and(mask, cpu_online_mask, set);
> +	if (desc->irq_data.node != -1)
> +		cpumask_and(mask, mask, cpumask_of_node(desc->irq_data.node));

What prevents mask from becoming empty if mask does not intersect with
the node mask?

Thanks,

	tglx