* [PATCH] sparseirq: Increase nr_irqs if needed
@ 2010-11-23 9:21 Yinghai Lu
[not found] ` <4CF75626.6000306@kernel.org>
0 siblings, 1 reply; 4+ messages in thread
From: Yinghai Lu @ 2010-11-23 9:21 UTC (permalink / raw)
To: Thomas Gleixner, Ingo Molnar, H. Peter Anvin; +Cc: linux-kernel
only do that when we need more.
For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064
120 + 8 * 128 + 120 * 16
systems could take 20 pcie, when intel 10g are used with
sriov and ixgbevf, every vf will need 3 irqs, and one device
have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.
with this patch, nr_irqs will increase 25% every time if nr_irqs is not
big enough.
Also later We could change nr_irqs initial value to
min(nr_irqs_gsi, 224*nr_cpu_ids).
for x86.
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
kernel/irq/irqdesc.c | 29 ++++++++++++++++++++++++++---
1 file changed, 26 insertions(+), 3 deletions(-)
Index: linux-2.6/kernel/irq/irqdesc.c
===================================================================
--- linux-2.6.orig/kernel/irq/irqdesc.c
+++ linux-2.6/kernel/irq/irqdesc.c
@@ -301,6 +301,27 @@ void irq_free_descs(unsigned int from, u
mutex_unlock(&sparse_irq_lock);
}
+static bool increase_nr_irqs(void)
+{
+#ifdef CONFIG_SPARSE_IRQ
+ int new;
+
+ if (nr_irqs == NR_IRQS)
+ return false;
+
+ new = nr_irqs;
+ new += nr_irqs >> 2;
+ if (new > NR_IRQS)
+ new = NR_IRQS;
+ if (new > nr_irqs) {
+ nr_irqs = new;
+ printk(KERN_INFO "nr_irqs increase to %d\n", nr_irqs);
+ return true;
+ }
+#endif
+ return false;
+}
+
/**
* irq_alloc_descs - allocate and initialize a range of irq descriptors
* @irq: Allocate for specific irq number if irq >= 0
@@ -320,14 +341,16 @@ irq_alloc_descs(int irq, unsigned int fr
mutex_lock(&sparse_irq_lock);
- start = bitmap_find_next_zero_area(allocated_irqs, nr_irqs, from, cnt, 0);
+ start = bitmap_find_next_zero_area(allocated_irqs, NR_IRQS, from, cnt, 0);
ret = -EEXIST;
if (irq >=0 && start != irq)
goto err;
ret = -ENOMEM;
- if (start >= nr_irqs)
- goto err;
+ while ((start + cnt) >= nr_irqs) {
+ if (!increase_nr_irqs())
+ goto err;
+ }
bitmap_set(allocated_irqs, start, cnt);
mutex_unlock(&sparse_irq_lock);
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS
[not found] ` <20101222125711.GI10809@elte.hu>
@ 2010-12-28 0:54 ` Yinghai Lu
2010-12-28 17:42 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 4+ messages in thread
From: Yinghai Lu @ 2010-12-28 0:54 UTC (permalink / raw)
To: Ingo Molnar, Thomas Gleixner, H. Peter Anvin; +Cc: linux-kernel
For x86_64 system:
when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064
120 + 8 * 128 + 120 * 16
systems could take 20 pcie, when intel 10g are used with
sriov and ixgbevf, every vf will need 3 irqs, and one device
have 64 vf. so will need 20 * 3 * 64 = 3840.
some 6 ports Intel 10gb may need more.
Just remove that function for x86, and let nr_irqs to NR_IRQS
because We already have radix-tree and bit_map for searching desc for irq.
Notes: long before same vresion cause one of Ingo's setup udev hang...
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
arch/x86/kernel/apic/io_apic.c | 22 ----------------------
1 file changed, 22 deletions(-)
Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/io_apic.c
+++ linux-2.6/arch/x86/kernel/apic/io_apic.c
@@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
return nr_irqs_gsi;
}
-#ifdef CONFIG_SPARSE_IRQ
-int __init arch_probe_nr_irqs(void)
-{
- int nr;
-
- if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
- nr_irqs = NR_VECTORS * nr_cpu_ids;
-
- nr = nr_irqs_gsi + 8 * nr_cpu_ids;
-#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
- /*
- * for MSI and HT dyn irq
- */
- nr += nr_irqs_gsi * 16;
-#endif
- if (nr < nr_irqs)
- nr_irqs = nr;
-
- return NR_IRQS_LEGACY;
-}
-#endif
-
static int __io_apic_set_pci_routing(struct device *dev, int irq,
struct io_apic_irq_attr *irq_attr)
{
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS
2010-12-28 0:54 ` [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS Yinghai Lu
@ 2010-12-28 17:42 ` Konrad Rzeszutek Wilk
2010-12-28 21:24 ` Yinghai Lu
0 siblings, 1 reply; 4+ messages in thread
From: Konrad Rzeszutek Wilk @ 2010-12-28 17:42 UTC (permalink / raw)
To: Yinghai Lu; +Cc: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, linux-kernel
On Mon, Dec 27, 2010 at 04:54:11PM -0800, Yinghai Lu wrote:
>
>
> For x86_64 system:
> when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064
> 120 + 8 * 128 + 120 * 16
> systems could take 20 pcie, when intel 10g are used with
> sriov and ixgbevf, every vf will need 3 irqs, and one device
> have 64 vf. so will need 20 * 3 * 64 = 3840.
> some 6 ports Intel 10gb may need more.
>
> Just remove that function for x86, and let nr_irqs to NR_IRQS
> because We already have radix-tree and bit_map for searching desc for irq.
Won't that make the arch_probe_nr_irqs be NR_IRQS_LEGACY?
I am looking at Linus's today tree, or is this patch based on some other tree?
>
> Notes: long before same vresion cause one of Ingo's setup udev hang...
>
> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>
> ---
> arch/x86/kernel/apic/io_apic.c | 22 ----------------------
> 1 file changed, 22 deletions(-)
>
> Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
> ===================================================================
> --- linux-2.6.orig/arch/x86/kernel/apic/io_apic.c
> +++ linux-2.6/arch/x86/kernel/apic/io_apic.c
> @@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
> return nr_irqs_gsi;
> }
>
> -#ifdef CONFIG_SPARSE_IRQ
> -int __init arch_probe_nr_irqs(void)
> -{
> - int nr;
> -
> - if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
> - nr_irqs = NR_VECTORS * nr_cpu_ids;
> -
> - nr = nr_irqs_gsi + 8 * nr_cpu_ids;
> -#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
> - /*
> - * for MSI and HT dyn irq
> - */
> - nr += nr_irqs_gsi * 16;
> -#endif
> - if (nr < nr_irqs)
> - nr_irqs = nr;
> -
> - return NR_IRQS_LEGACY;
> -}
> -#endif
> -
> static int __io_apic_set_pci_routing(struct device *dev, int irq,
> struct io_apic_irq_attr *irq_attr)
> {
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS
2010-12-28 17:42 ` Konrad Rzeszutek Wilk
@ 2010-12-28 21:24 ` Yinghai Lu
0 siblings, 0 replies; 4+ messages in thread
From: Yinghai Lu @ 2010-12-28 21:24 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Ingo Molnar, Thomas Gleixner, H. Peter Anvin, linux-kernel
On Tue, Dec 28, 2010 at 9:42 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Mon, Dec 27, 2010 at 04:54:11PM -0800, Yinghai Lu wrote:
>>
>>
>> For x86_64 system:
>> when we have 128 cpus with 5 ioapics, will have nr_irqs = 3064
>> 120 + 8 * 128 + 120 * 16
>> systems could take 20 pcie, when intel 10g are used with
>> sriov and ixgbevf, every vf will need 3 irqs, and one device
>> have 64 vf. so will need 20 * 3 * 64 = 3840.
>> some 6 ports Intel 10gb may need more.
>>
>> Just remove that function for x86, and let nr_irqs to NR_IRQS
>> because We already have radix-tree and bit_map for searching desc for irq.
>
> Won't that make the arch_probe_nr_irqs be NR_IRQS_LEGACY?
int nr_irqs = NR_IRQS;
and
int __init __weak arch_probe_nr_irqs(void)
{
return NR_IRQS_LEGACY;
}
int __init early_irq_init(void)
{
int i, initcnt, node = first_online_node;
struct irq_desc *desc;
init_irq_default_affinity();
/* Let arch update nr_irqs and return the nr of preallocated irqs */
initcnt = arch_probe_nr_irqs();
printk(KERN_INFO "NR_IRQS:%d nr_irqs:%d %d\n", NR_IRQS,
nr_irqs, initcnt);
that arch_probe_nr_irqs() is return for initcnt...
should change nr_irqs in the functions...
some confusing...
>
> I am looking at Linus's today tree, or is this patch based on some other tree?
>
>>
>> Notes: long before same vresion cause one of Ingo's setup udev hang...
>>
>> Signed-off-by: Yinghai Lu <yinghai@kernel.org>
>>
>> ---
>> arch/x86/kernel/apic/io_apic.c | 22 ----------------------
>> 1 file changed, 22 deletions(-)
>>
>> Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
>> ===================================================================
>> --- linux-2.6.orig/arch/x86/kernel/apic/io_apic.c
>> +++ linux-2.6/arch/x86/kernel/apic/io_apic.c
>> @@ -3637,28 +3637,6 @@ int get_nr_irqs_gsi(void)
>> return nr_irqs_gsi;
>> }
>>
>> -#ifdef CONFIG_SPARSE_IRQ
>> -int __init arch_probe_nr_irqs(void)
>> -{
>> - int nr;
>> -
>> - if (nr_irqs > (NR_VECTORS * nr_cpu_ids))
>> - nr_irqs = NR_VECTORS * nr_cpu_ids;
>> -
>> - nr = nr_irqs_gsi + 8 * nr_cpu_ids;
>> -#if defined(CONFIG_PCI_MSI) || defined(CONFIG_HT_IRQ)
>> - /*
>> - * for MSI and HT dyn irq
>> - */
>> - nr += nr_irqs_gsi * 16;
>> -#endif
>> - if (nr < nr_irqs)
>> - nr_irqs = nr;
>> -
>> - return NR_IRQS_LEGACY;
>> -}
>> -#endif
>> -
>> static int __io_apic_set_pci_routing(struct device *dev, int irq,
>> struct io_apic_irq_attr *irq_attr)
>> {
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2010-12-28 21:24 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-11-23 9:21 [PATCH] sparseirq: Increase nr_irqs if needed Yinghai Lu
[not found] ` <4CF75626.6000306@kernel.org>
[not found] ` <alpine.LFD.2.00.1012021019060.2653@localhost6.localdomain6>
[not found] ` <4CF7FA58.8070803@kernel.org>
[not found] ` <alpine.LFD.2.00.1012022112400.2653@localhost6.localdomain6>
[not found] ` <4CF7FF66.3010802@kernel.org>
[not found] ` <alpine.LFD.2.00.1012022125401.2653@localhost6.localdomain6>
[not found] ` <20101222125711.GI10809@elte.hu>
2010-12-28 0:54 ` [PATCH] x86, sparseirq: let nr_irqs equal to NR_IRQS Yinghai Lu
2010-12-28 17:42 ` Konrad Rzeszutek Wilk
2010-12-28 21:24 ` Yinghai Lu
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.