[BUG][2.6.0-test3-bk7] x86-64 UP_IOAPIC panic caused by cpumask_t conversion

[BUG][2.6.0-test3-bk7] x86-64 UP_IOAPIC panic caused by cpumask_t conversion

[Mikael Pettersson]

The box is UP with I/O-APIC, configured for !SMP, IO_APIC, !ACPI.
2.6.0-test3 worked fine, but 2.6.0-test3-bk7 panics and halts on
boot in arch/x86_64/kernel/io_apic.c:setup_ioapic_ids_from_mpc():

...
POSIX conformance testing by UNIFIX
ENABLING IO-APIC IRQs
BIOS bug, IO-APIC#0 ID 1 is already used!...
Kernel panic: Max APIC ID exceeded!

This happens because cpumask_t on UP implements "a set of CPUs"
as "a set is 0 or 1" (asm-generic/cpumask_up.h), while io_apic.c
actually wants "set of CPU (local APIC) and I/O-APIC IDs",
which obviously doesn't fit the "0 or 1" assumption.

As a quick-and-dirty test I changed linux/cpumask.h as follows

--- linux-2.6.0-test3-bk7/include/linux/cpumask.h.~1~	2003-08-19 23:48:50.000000000 +0200
+++ linux-2.6.0-test3-bk7/include/linux/cpumask.h	2003-08-20 00:07:17.000000000 +0200
@@ -21,7 +21,7 @@
 typedef unsigned long cpumask_t;
 #endif
 
-#ifdef CONFIG_SMP
+#if defined(CONFIG_SMP) || defined(CONFIG_X86_IO_APIC)
 #if NR_CPUS > BITS_PER_LONG
 #include <asm-generic/cpumask_array.h>
 #else

Since NR_CPUS==1 this makes UP_IOAPIC use cpumask_arith.h,
which is what the code used before the cpumask_t conversion.
With this change, the box boots Ok again.

(I believe this is the correct thing to do, except having
CONFIG_X86_IO_APIC in generic code isn't quite right.)

/Mikael

Re: [BUG][2.6.0-test3-bk7] x86-64 UP_IOAPIC panic caused by cpumask_t conversion

[Andi Kleen]

> Odd; I have a UP IO-APIC ia32 box here and it appears to do okay; there
> is a question of sparse APIC ID's and APIC ID space needing to be
> independent of NR_CPUS handled in the ia32 code that isn't handled in
> the x86_64 code. It was handled for ia32 by using a bitmap of size
> MAX_APICS (physid_mask_t) instead of cpumask_t for the things, which
> appears to eliminate various special cases for xAPIC's too.

Ok, then the physid_mask_t code needs to be ported over to x86-64.

-Andi

Re: [BUG][2.6.0-test3-bk7] x86-64 UP_IOAPIC panic caused by cpumask_t conversion

[William Lee Irwin III]

Mikael Pettersson <mikpe@csd.uu.se> writes:
> Nasty.
> But why does i386/UP work?

On Wed, Aug 20, 2003 at 01:39:10AM +0200, Andi Kleen wrote:
> > (I believe this is the correct thing to do, except having
> > CONFIG_X86_IO_APIC in generic code isn't quite right.)
> Better would be to undo the cpumask_t changes in io_apic.c
> and go back to unsigned long masks there again.
> Obviously a cpu mask is not the right data structure to manage APICs
> Another way would be to do whatever i386 does to avoid the problem.
> The IO-APIC code is unfortunately quite out of date/unsynced compared to i386,
> maybe it just needs some bug fix ported over. I will check that later.

Odd; I have a UP IO-APIC ia32 box here and it appears to do okay; there
is a question of sparse APIC ID's and APIC ID space needing to be
independent of NR_CPUS handled in the ia32 code that isn't handled in
the x86_64 code. It was handled for ia32 by using a bitmap of size
MAX_APICS (physid_mask_t) instead of cpumask_t for the things, which
appears to eliminate various special cases for xAPIC's too.


-- wli

Re: [BUG][2.6.0-test3-bk7] x86-64 UP_IOAPIC panic caused by cpumask_t conversion

[Andi Kleen]

Mikael Pettersson <mikpe@csd.uu.se> writes:

> --- linux-2.6.0-test3-bk7/include/linux/cpumask.h.~1~	2003-08-19 23:48:50.000000000 +0200
> +++ linux-2.6.0-test3-bk7/include/linux/cpumask.h	2003-08-20 00:07:17.000000000 +0200
> @@ -21,7 +21,7 @@
>  typedef unsigned long cpumask_t;
>  #endif
>  
> -#ifdef CONFIG_SMP
> +#if defined(CONFIG_SMP) || defined(CONFIG_X86_IO_APIC)
>  #if NR_CPUS > BITS_PER_LONG
>  #include <asm-generic/cpumask_array.h>
>  #else
>
> Since NR_CPUS==1 this makes UP_IOAPIC use cpumask_arith.h,
> which is what the code used before the cpumask_t conversion.
> With this change, the box boots Ok again.

Nasty.

But why does i386/UP work?

>
> (I believe this is the correct thing to do, except having
> CONFIG_X86_IO_APIC in generic code isn't quite right.)

Better would be to undo the cpumask_t changes in io_apic.c
and go back to unsigned long masks there again.

Obviously a cpu mask is not the right data structure to manage APICs

Another way would be to do whatever i386 does to avoid the problem.
The IO-APIC code is unfortunately quite out of date/unsynced compared to i386,
maybe it just needs some bug fix ported over. I will check that later.

-Andi