All of lore.kernel.org
 help / color / mirror / Atom feed
* x2apic boot failure on recent sandy bridge system
@ 2011-12-10  1:56 Berck E. Nash
  2011-12-10 20:17 ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Berck E. Nash @ 2011-12-10  1:56 UTC (permalink / raw)
  To: linux-kernel

[-- Attachment #1: Type: text/plain, Size: 317 bytes --]

X2APIC has never worked on this system, and continues to not work
3.2.0-rc5.  Simply hangs on boot with an error message that helpfully
indicates that x2apic is the problem.  The system is an ASRock P67
Extreme 6.  Log attached, let me know if I can do anything to help.
Please cc as I'm not subscribed.

Berck Nash


[-- Attachment #2: x2apic.log --]
[-- Type: text/plain, Size: 7536 bytes --]

[    0.000000] Linux version 3.2.0-rc5 (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #2 SMP Fri Dec 9 18:16:28 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3392.514 MHz processor.
[    0.000001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6785.02 BogoMIPS (lpj=3392514)
[    0.010594] pid_max: default: 32768 minimum: 301
[    0.015470] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023753] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031298] Mount-cache hash table entries: 256
[    0.035893] CPU: Physical Processor ID: 0
[    0.039900] CPU: Processor Core ID: 0
[    0.043557] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043558] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056570] mce: CPU supports 9 MCE banks
[    0.060577] CPU0: Thermal monitoring enabled (TM1)
[    0.065360] using mwait in idle threads.
[    0.069332] Freeing SMP alternatives: 20k freed
[    0.073857] ACPI: Core revision 20110623
[    0.085717] Kernel panic - not syncing: x2apic: enabled by BIOS but kernel init failed.
[    0.093697] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5 #2
[    0.099337] Call Trace:
[    0.101788]  [<ffffffff8137b7ce>] ? panic+0x95/0x18e
[    0.106755]  [<ffffffff81016259>] ? __ioapic_write_entry+0x16/0x32
[    0.112925]  [<ffffffff8152d5bf>] ? enable_IR_x2apic+0x19a/0x1d7
[    0.118913]  [<ffffffff8152f281>] ? default_setup_apic_routing+0xd/0x71
[    0.125506]  [<ffffffff8152c536>] ? native_smp_prepare_cpus+0x2f1/0x325
[    0.132102]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
[    0.137571]  [<ffffffff813841f4>] ? kernel_thread_helper+0x4/0x10
[    0.143650]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.149297]  [<ffffffff813841f0>] ? gs_change+0xb/0xb

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-10  1:56 x2apic boot failure on recent sandy bridge system Berck E. Nash
@ 2011-12-10 20:17 ` Yinghai Lu
  2011-12-10 22:58   ` Berck E. Nash
  2011-12-11  7:57   ` Ingo Molnar
  0 siblings, 2 replies; 26+ messages in thread
From: Yinghai Lu @ 2011-12-10 20:17 UTC (permalink / raw)
  To: Berck E. Nash; +Cc: linux-kernel, Ingo Molnar, Suresh Siddha

[-- Attachment #1: Type: text/plain, Size: 3101 bytes --]

On Fri, Dec 9, 2011 at 5:56 PM, Berck E. Nash <flyboy@gmail.com> wrote:
> X2APIC has never worked on this system, and continues to not work
> 3.2.0-rc5.  Simply hangs on boot with an error message that helpfully
> indicates that x2apic is the problem.  The system is an ASRock P67
> Extreme 6.  Log attached, let me know if I can do anything to help.
> Please cc as I'm not subscribed.

madt said cpu is x2apic.
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])

but bios does not provide DMAR table...

[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I
01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I
01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I
00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I
01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC
00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I
01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT
01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I
01072009 AMI. 00000004)

so kernel can not enable interrput remapping, then it will panic..

[    0.085717] Kernel panic - not syncing: x2apic: enabled by BIOS but
kernel init failed.
[    0.093697] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5 #2
[    0.099337] Call Trace:
[    0.101788]  [<ffffffff8137b7ce>] ? panic+0x95/0x18e
[    0.106755]  [<ffffffff81016259>] ? __ioapic_write_entry+0x16/0x32
[    0.112925]  [<ffffffff8152d5bf>] ? enable_IR_x2apic+0x19a/0x1d7
[    0.118913]  [<ffffffff8152f281>] ? default_setup_apic_routing+0xd/0x71
[    0.125506]  [<ffffffff8152c536>] ? native_smp_prepare_cpus+0x2f1/0x325
[    0.132102]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
[    0.137571]  [<ffffffff813841f4>] ? kernel_thread_helper+0x4/0x10
[    0.143650]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.149297]  [<ffffffff813841f0>] ? gs_change+0xb/0xb

your bios should not enable x2apic in this case.

please get one bios update.

also I have one old patch that could disable apic. but it is not get
merged and it may not be applied to current tree.

let us know if you can not get BIOS update or can not disable x2apic
in BIOS setup.

Thanks

Yinghai Lu

[-- Attachment #2: nox2apic.patch --]
[-- Type: text/x-patch, Size: 8285 bytes --]

Subject: [PATCH -v3] x86: Disable x2apic if nox2apic is specified

For
1. x2apic preenabled system
2. first kernel have x2apic enabled, and try to kexec second kernel with "nox2apic"

Will put back cpu with apic id < 255 into xapic mode, instead of panic.

-v2: use x2apic_disabled instead of nox2apic, Suggested by Thomas
     update x2apic_supported with x2apic_disabled, Suggested by Thomas

-v3: add checking for boot cpu apic id > 255. in that case will just panic
     --- pointed out by Suresh.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/apic.h      |    6 +++-
 arch/x86/include/asm/apicdef.h   |    1 
 arch/x86/include/asm/processor.h |    1 
 arch/x86/kernel/acpi/boot.c      |   10 ++++++-
 arch/x86/kernel/apic/apic.c      |   51 +++++++++++++++++++++++++++++++++------
 arch/x86/kernel/cpu/topology.c   |   21 ++++++++++++++++
 arch/x86/mm/srat.c               |    7 ++++-
 7 files changed, 85 insertions(+), 12 deletions(-)

Index: linux-2.6/arch/x86/include/asm/apic.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apic.h
+++ linux-2.6/arch/x86/include/asm/apic.h
@@ -175,6 +175,7 @@ static inline u64 native_x2apic_icr_read
 }
 
 extern int x2apic_phys;
+extern int x2apic_disabled;
 extern void check_x2apic(void);
 extern void enable_x2apic(void);
 extern void x2apic_icr_write(u32 low, u32 id);
@@ -182,7 +183,7 @@ static inline int x2apic_enabled(void)
 {
 	u64 msr;
 
-	if (!cpu_has_x2apic)
+	if (!cpu_has_x2apic || x2apic_disabled)
 		return 0;
 
 	rdmsrl(MSR_IA32_APICBASE, msr);
@@ -191,7 +192,7 @@ static inline int x2apic_enabled(void)
 	return 0;
 }
 
-#define x2apic_supported()	(cpu_has_x2apic)
+#define x2apic_supported()	(cpu_has_x2apic && !x2apic_disabled)
 static inline void x2apic_force_phys(void)
 {
 	x2apic_phys = 1;
@@ -213,6 +214,7 @@ static inline void x2apic_force_phys(voi
 
 #define	x2apic_preenabled 0
 #define	x2apic_supported()	0
+#define	x2apic_disabled	1
 #endif
 
 extern void enable_IR_x2apic(void);
Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -219,6 +219,8 @@ static int __init
 acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
 {
 	struct acpi_madt_local_x2apic *processor = NULL;
+	int apic_id;
+	u8 enabled;
 
 	processor = (struct acpi_madt_local_x2apic *)header;
 
@@ -227,6 +229,8 @@ acpi_parse_x2apic(struct acpi_subtable_h
 
 	acpi_table_print_madt_entry(header);
 
+	apic_id = processor->local_apic_id;
+	enabled = processor->lapic_flags & ACPI_MADT_ENABLED;
 #ifdef CONFIG_X86_X2APIC
 	/*
 	 * We need to register disabled CPU as well to permit
@@ -235,8 +239,10 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	 * to not preallocating memory for all NR_CPUS
 	 * when we use CPU hotplug.
 	 */
-	acpi_register_lapic(processor->local_apic_id,	/* APIC ID */
-			    processor->lapic_flags & ACPI_MADT_ENABLED);
+	if (x2apic_disabled && (apic_id >= 0xff) && enabled)
+		printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
+	else
+		acpi_register_lapic(apic_id, enabled);
 #else
 	printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
 #endif
Index: linux-2.6/arch/x86/kernel/apic/apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic.c
+++ linux-2.6/arch/x86/kernel/apic/apic.c
@@ -147,15 +147,14 @@ int x2apic_mode;
 #ifdef CONFIG_X86_X2APIC
 /* x2apic enabled before OS handover */
 static int x2apic_preenabled;
+int x2apic_disabled;
 static __init int setup_nox2apic(char *str)
 {
-	if (x2apic_enabled()) {
-		pr_warning("Bios already enabled x2apic, "
-			   "can't enforce nox2apic");
-		return 0;
-	}
+	if (x2apic_enabled())
+		pr_warning("Bios already enabled x2apic, will disable it");
+
+	x2apic_disabled = 1;
 
-	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
 	return 0;
 }
 early_param("nox2apic", setup_nox2apic);
@@ -1412,8 +1411,38 @@ void __init bsp_end_local_APIC_setup(voi
 }
 
 #ifdef CONFIG_X86_X2APIC
+
+static void disable_x2apic(void)
+{
+	int msr, msr2;
+
+	if (!cpu_has_x2apic)
+		return;
+
+	rdmsr(MSR_IA32_APICBASE, msr, msr2);
+	if (msr & X2APIC_ENABLE) {
+		u32 x2apic_id = x2apic_cpuid_initial_apicid();
+
+		if (x2apic_id > 255)
+			panic("Can not disable x2apic, id: %08x\n", x2apic_id);
+
+		pr_info("Disabling x2apic\n");
+		/*
+		 * Need to disable xapic and x2apic at the same time at first
+		 *  then enable xapic
+		 */
+		wrmsr(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE),
+			 0);
+		wrmsr(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE, 0);
+	}
+}
 void check_x2apic(void)
 {
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (x2apic_enabled()) {
 		pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
 		x2apic_preenabled = x2apic_mode = 1;
@@ -1424,6 +1453,11 @@ void enable_x2apic(void)
 {
 	int msr, msr2;
 
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (!x2apic_mode)
 		return;
 
@@ -1485,6 +1519,9 @@ void __init enable_IR_x2apic(void)
 	else
 		ret = enable_IR();
 
+	if (x2apic_disabled)
+		goto nox2apic;
+
 	if (!ret) {
 		/* IR is required if there is APIC ID > 255 even when running
 		 * under KVM
@@ -1519,7 +1556,7 @@ out:
 
 	if (x2apic_preenabled)
 		panic("x2apic: enabled by BIOS but kernel init failed.");
-	else if (cpu_has_x2apic)
+	else if (cpu_has_x2apic && !x2apic_disabled)
 		pr_info("Not enabling x2apic, Intr-remapping init failed.\n");
 }
 
Index: linux-2.6/arch/x86/mm/srat.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat.c
+++ linux-2.6/arch/x86/mm/srat.c
@@ -69,6 +69,12 @@ acpi_numa_x2apic_affinity_init(struct ac
 	if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0)
 		return;
 	pxm = pa->proximity_domain;
+	apic_id = pa->apic_id;
+	if (x2apic_disabled && (apic_id >= 0xff)) {
+		printk(KERN_INFO "SRAT: PXM %u -> X2APIC 0x%04x ignored\n",
+			 pxm, apic_id);
+		return;
+	}
 	node = setup_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
@@ -76,7 +82,6 @@ acpi_numa_x2apic_affinity_init(struct ac
 		return;
 	}
 
-	apic_id = pa->apic_id;
 	if (apic_id >= MAX_LOCAL_APIC) {
 		printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u skipped apicid that is too big\n", pxm, apic_id, node);
 		return;
Index: linux-2.6/arch/x86/include/asm/apicdef.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apicdef.h
+++ linux-2.6/arch/x86/include/asm/apicdef.h
@@ -142,6 +142,7 @@
 
 #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
 #define APIC_BASE_MSR	0x800
+#define XAPIC_ENABLE	(1UL << 11)
 #define X2APIC_ENABLE	(1UL << 10)
 
 #ifdef CONFIG_X86_32
Index: linux-2.6/arch/x86/include/asm/processor.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/processor.h
+++ linux-2.6/arch/x86/include/asm/processor.h
@@ -167,6 +167,7 @@ extern void init_scattered_cpuid_feature
 extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c);
 extern unsigned short num_cache_leaves;
 
+u32 x2apic_cpuid_initial_apicid(void);
 extern void detect_extended_topology(struct cpuinfo_x86 *c);
 extern void detect_ht(struct cpuinfo_x86 *c);
 
Index: linux-2.6/arch/x86/kernel/cpu/topology.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/topology.c
+++ linux-2.6/arch/x86/kernel/cpu/topology.c
@@ -21,6 +21,27 @@
 #define BITS_SHIFT_NEXT_LEVEL(eax)	((eax) & 0x1f)
 #define LEVEL_MAX_SIBLINGS(ebx)		((ebx) & 0xffff)
 
+u32 x2apic_cpuid_initial_apicid(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	if (boot_cpu_data.cpuid_level < 0xb)
+		return 0;
+
+	cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+	/*
+	 * check if the cpuid leaf 0xb is actually implemented.
+	 */
+	if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+		return 0;
+
+	/*
+	 * initial apic id, which also represents 32-bit extended x2apic id.
+	 */
+	return edx;
+}
+
 /*
  * Check for extended topology enumeration cpuid leaf 0xb and if it
  * exists, use it for populating initial_apicid and cpu topology

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-10 20:17 ` Yinghai Lu
@ 2011-12-10 22:58   ` Berck E. Nash
  2011-12-10 23:11     ` Yinghai Lu
  2011-12-11  7:57   ` Ingo Molnar
  1 sibling, 1 reply; 26+ messages in thread
From: Berck E. Nash @ 2011-12-10 22:58 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: linux-kernel, Ingo Molnar, Suresh Siddha

On 12/10/2011 01:17 PM, Yinghai Lu wrote:
> your bios should not enable x2apic in this case.

I am, in fact, able to disable x2apic in the BIOS and it works fine.  I
was just under the impression that x2apic should be available and should
work?  Is this wrong?  If so, then there's no kernel bug, and I just
misunderstood.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-10 22:58   ` Berck E. Nash
@ 2011-12-10 23:11     ` Yinghai Lu
  0 siblings, 0 replies; 26+ messages in thread
From: Yinghai Lu @ 2011-12-10 23:11 UTC (permalink / raw)
  To: Berck E. Nash; +Cc: linux-kernel, Ingo Molnar, Suresh Siddha

On Sat, Dec 10, 2011 at 2:58 PM, Berck E. Nash <flyboy@gmail.com> wrote:
> On 12/10/2011 01:17 PM, Yinghai Lu wrote:
>> your bios should not enable x2apic in this case.
>
> I am, in fact, able to disable x2apic in the BIOS and it works fine.  I
> was just under the impression that x2apic should be available and should
> work?  Is this wrong?  If so, then there's no kernel bug, and I just
> misunderstood.

it's not kernel bug.

BIOS need to provide DMAR table, so kernel could enable
intr-rempapping, and x2apic need intr-remap.

Thanks

Yinghai

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-10 20:17 ` Yinghai Lu
  2011-12-10 22:58   ` Berck E. Nash
@ 2011-12-11  7:57   ` Ingo Molnar
  2011-12-11 21:43     ` Yinghai Lu
  1 sibling, 1 reply; 26+ messages in thread
From: Ingo Molnar @ 2011-12-11  7:57 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Berck E. Nash, linux-kernel, Suresh Siddha


* Yinghai Lu <yinghai@kernel.org> wrote:

> so kernel can not enable interrput remapping, then it will panic..
> 
> [    0.085717] Kernel panic - not syncing: x2apic: enabled by BIOS but
> kernel init failed.
> [    0.093697] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5 #2
> [    0.099337] Call Trace:
> [    0.101788]  [<ffffffff8137b7ce>] ? panic+0x95/0x18e
> [    0.106755]  [<ffffffff81016259>] ? __ioapic_write_entry+0x16/0x32
> [    0.112925]  [<ffffffff8152d5bf>] ? enable_IR_x2apic+0x19a/0x1d7
> [    0.118913]  [<ffffffff8152f281>] ? default_setup_apic_routing+0xd/0x71
> [    0.125506]  [<ffffffff8152c536>] ? native_smp_prepare_cpus+0x2f1/0x325
> [    0.132102]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
> [    0.137571]  [<ffffffff813841f4>] ? kernel_thread_helper+0x4/0x10
> [    0.143650]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
> [    0.149297]  [<ffffffff813841f0>] ? gs_change+0xb/0xb
> 
> your bios should not enable x2apic in this case.
> 
> please get one bios update.
> 
> also I have one old patch that could disable apic. but it is 
> not get merged and it may not be applied to current tree.

What we want is not a nox2apic option but a good fall-back 
mechanism that is able to boot the system automatically and 
safely, without the user having to tweak *anything* ...

Tanks,

	Ingo

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-11  7:57   ` Ingo Molnar
@ 2011-12-11 21:43     ` Yinghai Lu
  2011-12-11 22:05       ` Berck E. Nash
  2011-12-12  7:23       ` Suresh Siddha
  0 siblings, 2 replies; 26+ messages in thread
From: Yinghai Lu @ 2011-12-11 21:43 UTC (permalink / raw)
  To: Ingo Molnar, Berck E. Nash; +Cc: linux-kernel, Suresh Siddha

[-- Attachment #1: Type: text/plain, Size: 418 bytes --]

On Sat, Dec 10, 2011 at 11:57 PM, Ingo Molnar <mingo@elte.hu> wrote:
>
> What we want is not a nox2apic option but a good fall-back
> mechanism that is able to boot the system automatically and
> safely, without the user having to tweak *anything* ...

Sure.

Attached -v4 should automatically switch to xapic mode.

Berck,
Can you please try -v4 patch on your setup when BIOS have x2apic enabled?

Thanks

Yinghai Lu

[-- Attachment #2: nox2apic_v4.patch --]
[-- Type: text/x-patch, Size: 10202 bytes --]

Subject: [PATCH -v4] x86: Disable x2apic if nox2apic is specified or intr-remap can not be enabled

For
1. x2apic preenabled system
2. first kernel have x2apic enabled, and try to kexec second kernel with "nox2apic"

Will put back cpu with apic id < 255 into xapic mode, instead of panic.

-v2: use x2apic_disabled instead of nox2apic, Suggested by Thomas
     update x2apic_supported with x2apic_disabled, Suggested by Thomas

-v3: add checking for boot cpu apic id > 255. in that case will just panic
     --- pointed out by Suresh.

-v4: according to Ingo, for x2apic pre-enabled system, if intr-remap can not
	be enabled, try to disable x2apic instead of panic and request to
	specify nox2apic for next boot.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/apic.h         |    6 +-
 arch/x86/include/asm/apicdef.h      |    1 
 arch/x86/include/asm/processor.h    |    1 
 arch/x86/kernel/acpi/boot.c         |   10 +++-
 arch/x86/kernel/apic/apic.c         |   74 ++++++++++++++++++++++++++----------
 arch/x86/kernel/apic/apic_flat_64.c |    7 ++-
 arch/x86/kernel/cpu/topology.c      |   21 ++++++++++
 arch/x86/mm/srat.c                  |    7 ++-
 8 files changed, 101 insertions(+), 26 deletions(-)

Index: linux-2.6/arch/x86/include/asm/apic.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apic.h
+++ linux-2.6/arch/x86/include/asm/apic.h
@@ -176,6 +176,7 @@ static inline u64 native_x2apic_icr_read
 }
 
 extern int x2apic_phys;
+extern int x2apic_disabled;
 extern void check_x2apic(void);
 extern void enable_x2apic(void);
 extern void x2apic_icr_write(u32 low, u32 id);
@@ -183,7 +184,7 @@ static inline int x2apic_enabled(void)
 {
 	u64 msr;
 
-	if (!cpu_has_x2apic)
+	if (!cpu_has_x2apic || x2apic_disabled)
 		return 0;
 
 	rdmsrl(MSR_IA32_APICBASE, msr);
@@ -192,7 +193,7 @@ static inline int x2apic_enabled(void)
 	return 0;
 }
 
-#define x2apic_supported()	(cpu_has_x2apic)
+#define x2apic_supported()	(cpu_has_x2apic && !x2apic_disabled)
 static inline void x2apic_force_phys(void)
 {
 	x2apic_phys = 1;
@@ -214,6 +215,7 @@ static inline void x2apic_force_phys(voi
 
 #define	x2apic_preenabled 0
 #define	x2apic_supported()	0
+#define	x2apic_disabled	1
 #endif
 
 extern void enable_IR_x2apic(void);
Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -219,6 +219,8 @@ static int __init
 acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
 {
 	struct acpi_madt_local_x2apic *processor = NULL;
+	int apic_id;
+	u8 enabled;
 
 	processor = (struct acpi_madt_local_x2apic *)header;
 
@@ -227,6 +229,8 @@ acpi_parse_x2apic(struct acpi_subtable_h
 
 	acpi_table_print_madt_entry(header);
 
+	apic_id = processor->local_apic_id;
+	enabled = processor->lapic_flags & ACPI_MADT_ENABLED;
 #ifdef CONFIG_X86_X2APIC
 	/*
 	 * We need to register disabled CPU as well to permit
@@ -235,8 +239,10 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	 * to not preallocating memory for all NR_CPUS
 	 * when we use CPU hotplug.
 	 */
-	acpi_register_lapic(processor->local_apic_id,	/* APIC ID */
-			    processor->lapic_flags & ACPI_MADT_ENABLED);
+	if (x2apic_disabled && (apic_id >= 0xff) && enabled)
+		printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
+	else
+		acpi_register_lapic(apic_id, enabled);
 #else
 	printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
 #endif
Index: linux-2.6/arch/x86/kernel/apic/apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic.c
+++ linux-2.6/arch/x86/kernel/apic/apic.c
@@ -147,15 +147,14 @@ int x2apic_mode;
 #ifdef CONFIG_X86_X2APIC
 /* x2apic enabled before OS handover */
 static int x2apic_preenabled;
+int x2apic_disabled;
 static __init int setup_nox2apic(char *str)
 {
-	if (x2apic_enabled()) {
-		pr_warning("Bios already enabled x2apic, "
-			   "can't enforce nox2apic");
-		return 0;
-	}
+	if (x2apic_enabled())
+		pr_warning("Bios already enabled x2apic, will disable it");
+
+	x2apic_disabled = 1;
 
-	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
 	return 0;
 }
 early_param("nox2apic", setup_nox2apic);
@@ -1431,8 +1430,38 @@ void __init bsp_end_local_APIC_setup(voi
 }
 
 #ifdef CONFIG_X86_X2APIC
+
+static void disable_x2apic(void)
+{
+	int msr, msr2;
+
+	if (!cpu_has_x2apic)
+		return;
+
+	rdmsr(MSR_IA32_APICBASE, msr, msr2);
+	if (msr & X2APIC_ENABLE) {
+		u32 x2apic_id = x2apic_cpuid_initial_apicid();
+
+		if (x2apic_id > 255)
+			panic("Can not disable x2apic, id: %08x\n", x2apic_id);
+
+		pr_info("Disabling x2apic\n");
+		/*
+		 * Need to disable xapic and x2apic at the same time at first
+		 *  then enable xapic
+		 */
+		wrmsr(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE),
+			 0);
+		wrmsr(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE, 0);
+	}
+}
 void check_x2apic(void)
 {
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (x2apic_enabled()) {
 		pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
 		x2apic_preenabled = x2apic_mode = 1;
@@ -1443,6 +1472,11 @@ void enable_x2apic(void)
 {
 	int msr, msr2;
 
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (!x2apic_mode)
 		return;
 
@@ -1493,7 +1527,7 @@ void __init enable_IR_x2apic(void)
 	ret = save_ioapic_entries();
 	if (ret) {
 		pr_info("Saving IO-APIC state failed: %d\n", ret);
-		goto out;
+		return;
 	}
 
 	local_irq_save(flags);
@@ -1505,13 +1539,22 @@ void __init enable_IR_x2apic(void)
 	else
 		ret = enable_IR();
 
+	if (x2apic_disabled)
+		goto nox2apic;
+
 	if (ret < 0) {
 		/* IR is required if there is APIC ID > 255 even when running
 		 * under KVM
 		 */
 		if (max_physical_apicid > 255 ||
-		    !hypervisor_x2apic_available())
+		    !hypervisor_x2apic_available()) {
+			if (x2apic_preenabled) {
+				disable_x2apic();
+				x2apic_disabled = 1;
+				x2apic_mode = 0;
+			}
 			goto nox2apic;
+		}
 		/*
 		 * without IR all CPUs can be addressed by IOAPIC/MSI
 		 * only in physical mode
@@ -1519,8 +1562,10 @@ void __init enable_IR_x2apic(void)
 		x2apic_force_phys();
 	}
 
-	if (ret == IRQ_REMAP_XAPIC_MODE)
+	if (ret == IRQ_REMAP_XAPIC_MODE) {
+		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
 		goto nox2apic;
+	}
 
 	x2apic_enabled = 1;
 
@@ -1535,17 +1580,6 @@ nox2apic:
 		restore_ioapic_entries();
 	legacy_pic->restore_mask();
 	local_irq_restore(flags);
-
-out:
-	if (x2apic_enabled || !x2apic_supported())
-		return;
-
-	if (x2apic_preenabled)
-		panic("x2apic: enabled by BIOS but kernel init failed.");
-	else if (ret == IRQ_REMAP_XAPIC_MODE)
-		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
-	else if (ret < 0)
-		pr_info("x2apic not enabled, IRQ remapping init failed\n");
 }
 
 #ifdef CONFIG_X86_64
Index: linux-2.6/arch/x86/mm/srat.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat.c
+++ linux-2.6/arch/x86/mm/srat.c
@@ -69,6 +69,12 @@ acpi_numa_x2apic_affinity_init(struct ac
 	if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0)
 		return;
 	pxm = pa->proximity_domain;
+	apic_id = pa->apic_id;
+	if (x2apic_disabled && (apic_id >= 0xff)) {
+		printk(KERN_INFO "SRAT: PXM %u -> X2APIC 0x%04x ignored\n",
+			 pxm, apic_id);
+		return;
+	}
 	node = setup_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
@@ -76,7 +82,6 @@ acpi_numa_x2apic_affinity_init(struct ac
 		return;
 	}
 
-	apic_id = pa->apic_id;
 	if (apic_id >= MAX_LOCAL_APIC) {
 		printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u skipped apicid that is too big\n", pxm, apic_id, node);
 		return;
Index: linux-2.6/arch/x86/include/asm/apicdef.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apicdef.h
+++ linux-2.6/arch/x86/include/asm/apicdef.h
@@ -144,6 +144,7 @@
 
 #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
 #define APIC_BASE_MSR	0x800
+#define XAPIC_ENABLE	(1UL << 11)
 #define X2APIC_ENABLE	(1UL << 10)
 
 #ifdef CONFIG_X86_32
Index: linux-2.6/arch/x86/include/asm/processor.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/processor.h
+++ linux-2.6/arch/x86/include/asm/processor.h
@@ -168,6 +168,7 @@ extern void init_scattered_cpuid_feature
 extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c);
 extern unsigned short num_cache_leaves;
 
+u32 x2apic_cpuid_initial_apicid(void);
 extern void detect_extended_topology(struct cpuinfo_x86 *c);
 extern void detect_ht(struct cpuinfo_x86 *c);
 
Index: linux-2.6/arch/x86/kernel/cpu/topology.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/topology.c
+++ linux-2.6/arch/x86/kernel/cpu/topology.c
@@ -21,6 +21,27 @@
 #define BITS_SHIFT_NEXT_LEVEL(eax)	((eax) & 0x1f)
 #define LEVEL_MAX_SIBLINGS(ebx)		((ebx) & 0xffff)
 
+u32 x2apic_cpuid_initial_apicid(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	if (boot_cpu_data.cpuid_level < 0xb)
+		return 0;
+
+	cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+	/*
+	 * check if the cpuid leaf 0xb is actually implemented.
+	 */
+	if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+		return 0;
+
+	/*
+	 * initial apic id, which also represents 32-bit extended x2apic id.
+	 */
+	return edx;
+}
+
 /*
  * Check for extended topology enumeration cpuid leaf 0xb and if it
  * exists, use it for populating initial_apicid and cpu topology
Index: linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic_flat_64.c
+++ linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
@@ -171,9 +171,14 @@ static int flat_phys_pkg_id(int initial_
 	return initial_apic_id >> index_msb;
 }
 
+static int flat_probe(void)
+{
+	return 1;
+}
+
 static struct apic apic_flat =  {
 	.name				= "flat",
-	.probe				= NULL,
+	.probe				= flat_probe,
 	.acpi_madt_oem_check		= flat_acpi_madt_oem_check,
 	.apic_id_registered		= flat_apic_id_registered,
 

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-11 21:43     ` Yinghai Lu
@ 2011-12-11 22:05       ` Berck E. Nash
  2011-12-12  3:40         ` Yinghai Lu
  2011-12-12  7:23       ` Suresh Siddha
  1 sibling, 1 reply; 26+ messages in thread
From: Berck E. Nash @ 2011-12-11 22:05 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel, Suresh Siddha

[-- Attachment #1: Type: text/plain, Size: 507 bytes --]

On 12/11/2011 02:43 PM, Yinghai Lu wrote:
> On Sat, Dec 10, 2011 at 11:57 PM, Ingo Molnar <mingo@elte.hu> wrote:
>>
>> What we want is not a nox2apic option but a good fall-back
>> mechanism that is able to boot the system automatically and
>> safely, without the user having to tweak *anything* ...
> 
> Attached -v4 should automatically switch to xapic mode.
> 
> Berck,
> Can you please try -v4 patch on your setup when BIOS have x2apic enabled?

No problem: It does not work as expected.  Log attached.

[-- Attachment #2: x2apic-enabled.log --]
[-- Type: text/plain, Size: 63946 bytes --]

[    0.000000] Linux version 3.2.0-rc5 (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #2 SMP Fri Dec 9 18:16:28 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 00092 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39a88 001D6 (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39c60 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ca0 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d10 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3392.136 MHz processor.
[    0.000002] Calibrating delay loop (skipped), value calculated using timer frequency.. 6784.27 BogoMIPS (lpj=3392136)
[    0.010596] pid_max: default: 32768 minimum: 301
[    0.015476] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023749] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031306] Mount-cache hash table entries: 256
[    0.035902] CPU: Physical Processor ID: 0
[    0.039908] CPU: Processor Core ID: 0
[    0.043566] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043566] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056571] mce: CPU supports 9 MCE banks
[    0.060578] CPU0: Thermal monitoring enabled (TM1)
[    0.065362] using mwait in idle threads.
[    0.069341] Freeing SMP alternatives: 20k freed
[    0.073867] ACPI: Core revision 20110623
[    0.085701] x2apic not enabled, IRQ remapping init failed
[    0.091507] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.107464] CPU0: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz stepping 07
[    0.215404] Performance Events: PEBS fmt1+, SandyBridge events, Intel PMU driver.
[    0.222922] PEBS disabled due to CPU errata.
[    0.227182] ... version:                3
[    0.231184] ... bit width:              48
[    0.235269] ... generic registers:      4
[    0.239263] ... value mask:             0000ffffffffffff
[    0.244561] ... max period:             000000007fffffff
[    0.249858] ... fixed-purpose events:   3
[    0.253849] ... event mask:             000000070000000f
[    0.259210] Booting Node   0, Processors  #1
[    0.354115]  #2
[    0.446836]  #3
[    0.539558]  #4
[    0.632319]  #5
[    0.725044]  #6
[    0.817770]  #7 Ok.
[    0.910387] Brought up 8 CPUs
[    0.913349] Total of 8 processors activated (54270.27 BogoMIPS).
[    0.924156] devtmpfs: initialized
[    0.927560] NET: Registered protocol family 16
[    0.932036] ACPI: bus type pci registered
[    0.936042] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.945316] PCI: not using MMCONFIG
[    0.948798] PCI: Using configuration type 1 for base access
[    0.954706] bio: create slab <bio-0> at 0
[    0.958746] ACPI: Added _OSI(Module Device)
[    0.962919] ACPI: Added _OSI(Processor Device)
[    0.967350] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.972034] ACPI: Added _OSI(Processor Aggregator Device)
[    0.978545] ACPI: Executed 1 blocks of module-level executable AML code
[    0.987537] ACPI: SSDT 00000000bf612898 006F4 (v01    AMI      IST 00000001 MSFT 03000001)
[    0.996015] ACPI: Dynamic OEM Table Load:
[    1.000044] ACPI: SSDT           (null) 006F4 (v01    AMI      IST 00000001 MSFT 03000001)
[    1.008342] ACPI: SSDT 00000000bf609d98 000E4 (v01    AMI      CST 00000001 MSFT 03000001)
[    1.016753] ACPI: Dynamic OEM Table Load:
[    1.020792] ACPI: SSDT           (null) 000E4 (v01    AMI      CST 00000001 MSFT 03000001)
[    1.029588] ACPI: Interpreter enabled
[    1.033237] ACPI: (supports S0 S5)
[    1.036668] ACPI: Using IOAPIC for interrupt routing
[    1.041631] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    1.050941] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[    1.073354] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
[    1.082047] HEST: Table not found.
[    1.085445] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    1.094656] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    1.100944] pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
[    1.107533] pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
[    1.114124] pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
[    1.121402] pci_root PNP0A08:00: host bridge window [mem 0x000c8000-0x000dffff]
[    1.128687] pci_root PNP0A08:00: host bridge window [mem 0xc0000000-0xffffffff]
[    1.135974] pci_root PNP0A08:00: ignoring host bridge window [mem 0x000c8000-0x000dffff] (conflicts with Video ROM [mem 0x000c0000-0x000ccdff])
[    1.151621] pci 0000:00:1c.0: PCI bridge to [bus 01-01]
[    1.158598] pci 0000:00:1c.4: PCI bridge to [bus 02-02]
[    1.165580] pci 0000:00:1c.6: PCI bridge to [bus 03-03]
[    1.172553] pci 0000:00:1c.7: PCI bridge to [bus 04-0d]
[    1.179159] pci 0000:04:00.0: PCI bridge to [bus 05-0d]
[    1.184762] pci 0000:05:01.0: PCI bridge to [bus 06-06]
[    1.191501] pci 0000:05:04.0: PCI bridge to [bus 07-08]
[    1.197426] pci 0000:07:00.0: PCI bridge to [bus 08-08]
[    1.203054] pci 0000:05:05.0: PCI bridge to [bus 09-09]
[    1.210449] pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
[    1.216033] pci 0000:05:07.0: PCI bridge to [bus 0b-0b]
[    1.221352] pci 0000:05:08.0: PCI bridge to [bus 0c-0c]
[    1.226664] pci 0000:05:09.0: PCI bridge to [bus 0d-0d]
[    1.232397]  pci0000:00: Requesting ACPI _OSC control (0x1d)
[    1.238160]  pci0000:00: ACPI _OSC control (0x1c) granted
[    1.245959] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
[    1.253214] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 *5 6 7 10 11 12 14 15)
[    1.260463] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)
[    1.267533] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 10 11 12 14 15)
[    1.274593] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    1.282033] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    1.289473] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 *10 11 12 14 15)
[    1.296725] ACPI: PCI Interrupt Link [LNKH] (IRQs *3 4 5 6 7 10 11 12 14 15)
[    1.303994] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none
[    1.312056] vgaarb: loaded
[    1.314755] vgaarb: bridge control possible 0000:01:00.0
[    1.320099] SCSI subsystem initialized
[    1.323877] PCI: Using ACPI for IRQ routing
[    1.337876] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    1.344191] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    1.352014] Switching to clocksource hpet
[    1.356060] pnp: PnP ACPI init
[    1.359120] ACPI: bus type pnp registered
[    1.363280] system 00:01: [mem 0xfed10000-0xfed19fff] has been reserved
[    1.369878] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    1.376472] system 00:01: [mem 0xfed90000-0xfed93fff] has been reserved
[    1.383066] system 00:01: [mem 0xfed20000-0xfed3ffff] has been reserved
[    1.389659] system 00:01: [mem 0xfee00000-0xfee0ffff] has been reserved
[    1.396307] system 00:02: [io  0x0290-0x029f] has been reserved
[    1.402598] system 00:07: [io  0x04d0-0x04d1] has been reserved
[    1.408850] system 00:0a: [io  0x0400-0x0453] has been reserved
[    1.414757] system 00:0a: [io  0x0458-0x047f] has been reserved
[    1.420657] system 00:0a: [io  0x1180-0x119f] has been reserved
[    1.426558] system 00:0a: [io  0x0500-0x057f] has been reserved
[    1.432462] system 00:0a: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    1.439055] system 00:0a: [mem 0xfec00000-0xfecfffff] could not be reserved
[    1.445994] system 00:0a: [mem 0xfed08000-0xfed08fff] has been reserved
[    1.452588] system 00:0a: [mem 0xff000000-0xffffffff] has been reserved
[    1.459227] system 00:0b: [io  0x0454-0x0457] has been reserved
[    1.465298] pnp: PnP ACPI: found 13 devices
[    1.469475] ACPI: ACPI bus type pnp unregistered
[    1.479903] pci 0000:00:1c.0: PCI bridge to [bus 01-01]
[    1.485119] pci 0000:00:1c.0:   bridge window [io  0xe000-0xefff]
[    1.491196] pci 0000:00:1c.0:   bridge window [mem 0xf8000000-0xfb0fffff]
[    1.497963] pci 0000:00:1c.0:   bridge window [mem 0xc0000000-0xcfffffff 64bit pref]
[    1.505683] pci 0000:00:1c.4: PCI bridge to [bus 02-02]
[    1.510896] pci 0000:00:1c.4:   bridge window [io  0xd000-0xdfff]
[    1.516975] pci 0000:00:1c.4:   bridge window [mem 0xfb700000-0xfb7fffff]
[    1.523745] pci 0000:00:1c.6: PCI bridge to [bus 03-03]
[    1.528961] pci 0000:00:1c.6:   bridge window [mem 0xfb600000-0xfb6fffff]
[    1.535732] pci 0000:05:01.0: PCI bridge to [bus 06-06]
[    1.540949] pci 0000:05:01.0:   bridge window [mem 0xfb400000-0xfb4fffff]
[    1.547729] pci 0000:07:00.0: PCI bridge to [bus 08-08]
[    1.552942] pci 0000:07:00.0:   bridge window [io  0xc000-0xcfff]
[    1.559021] pci 0000:07:00.0:   bridge window [mem 0xfb300000-0xfb3fffff]
[    1.565793] pci 0000:07:00.0:   bridge window [mem 0xd0000000-0xd3ffffff 64bit pref]
[    1.573517] pci 0000:05:04.0: PCI bridge to [bus 07-08]
[    1.578727] pci 0000:05:04.0:   bridge window [io  0xc000-0xcfff]
[    1.584807] pci 0000:05:04.0:   bridge window [mem 0xfb300000-0xfb3fffff]
[    1.591581] pci 0000:05:04.0:   bridge window [mem 0xd0000000-0xd3ffffff 64bit pref]
[    1.599310] pci 0000:05:05.0: PCI bridge to [bus 09-09]
[    1.604528] pci 0000:05:05.0:   bridge window [mem 0xfb200000-0xfb2fffff]
[    1.611310] pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
[    1.616519] pci 0000:05:06.0:   bridge window [io  0xb000-0xbfff]
[    1.622601] pci 0000:05:06.0:   bridge window [mem 0xd4000000-0xd40fffff 64bit pref]
[    1.630325] pci 0000:05:07.0: PCI bridge to [bus 0b-0b]
[    1.635540] pci 0000:05:07.0:   bridge window [io  0xa000-0xafff]
[    1.641619] pci 0000:05:07.0:   bridge window [mem 0xfb100000-0xfb1fffff]
[    1.648399] pci 0000:05:08.0: PCI bridge to [bus 0c-0c]
[    1.653625] pci 0000:05:09.0: PCI bridge to [bus 0d-0d]
[    1.658855] pci 0000:04:00.0: PCI bridge to [bus 05-0d]
[    1.664065] pci 0000:04:00.0:   bridge window [io  0xa000-0xcfff]
[    1.670145] pci 0000:04:00.0:   bridge window [mem 0xfb100000-0xfb4fffff]
[    1.676920] pci 0000:04:00.0:   bridge window [mem 0xd0000000-0xd40fffff 64bit pref]
[    1.684650] pci 0000:00:1c.7: PCI bridge to [bus 04-0d]
[    1.689862] pci 0000:00:1c.7:   bridge window [io  0xa000-0xcfff]
[    1.695940] pci 0000:00:1c.7:   bridge window [mem 0xfb100000-0xfb5fffff]
[    1.702705] pci 0000:00:1c.7:   bridge window [mem 0xd0000000-0xd40fffff 64bit pref]
[    1.710434] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.717125] pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.723816] pci 0000:00:1c.6: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    1.730505] pci 0000:00:1c.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    1.737194] pci 0000:04:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.743885] pci 0000:05:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.750573] pci 0000:05:04.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.757262] pci 0000:07:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.763956] pci 0000:05:05.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.770649] pci 0000:05:06.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.777337] pci 0000:05:07.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    1.784024] pci 0000:05:08.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.790713] pci 0000:05:09.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.797497] NET: Registered protocol family 2
[    1.801942] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    1.809621] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[    1.817908] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.824702] TCP: Hash tables configured (established 524288 bind 65536)
[    1.831294] TCP reno registered
[    1.834436] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    1.840533] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    1.847113] NET: Registered protocol family 1
[    2.097235] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    2.103654] Placing 64MB software IO TLB between ffff8800baee2000 - ffff8800beee2000
[    2.111369] software IO TLB at phys 0xbaee2000 - 0xbeee2000
[    2.117768] audit: initializing netlink socket (disabled)
[    2.123153] type=2000 audit(1323550917.488:1): initialized
[    2.147320] Btrfs loaded
[    2.149857] msgmni has been set to 15968
[    2.154126] NET: Registered protocol family 38
[    2.158578] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    2.165952] io scheduler noop registered
[    2.169885] io scheduler cfq registered (default)
[    2.176685] pcieport 0000:00:1c.0: Signaling PME through PCIe PME interrupt
[    2.183624] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    2.190150] pcieport 0000:00:1c.4: Signaling PME through PCIe PME interrupt
[    2.197089] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[    2.203615] pcieport 0000:00:1c.6: Signaling PME through PCIe PME interrupt
[    2.210552] pci 0000:03:00.0: Signaling PME through PCIe PME interrupt
[    2.217078] pcieport 0000:00:1c.7: Signaling PME through PCIe PME interrupt
[    2.224016] pcieport 0000:04:00.0: Signaling PME through PCIe PME interrupt
[    2.230957] pcieport 0000:05:01.0: Signaling PME through PCIe PME interrupt
[    2.237893] pci 0000:06:00.0: Signaling PME through PCIe PME interrupt
[    2.244401] pcieport 0000:05:04.0: Signaling PME through PCIe PME interrupt
[    2.251341] pci 0000:07:00.0: Signaling PME through PCIe PME interrupt
[    2.257848] pci 0000:08:00.0: Signaling PME through PCIe PME interrupt
[    2.264356] pci 0000:08:01.0: Signaling PME through PCIe PME interrupt
[    2.270863] pcieport 0000:05:05.0: Signaling PME through PCIe PME interrupt
[    2.277804] pci 0000:09:00.0: Signaling PME through PCIe PME interrupt
[    2.284309] pcieport 0000:05:06.0: Signaling PME through PCIe PME interrupt
[    2.291251] pci 0000:0a:00.0: Signaling PME through PCIe PME interrupt
[    2.297757] pcieport 0000:05:07.0: Signaling PME through PCIe PME interrupt
[    2.304698] pci 0000:0b:00.0: Signaling PME through PCIe PME interrupt
[    2.311204] pcieport 0000:05:08.0: Signaling PME through PCIe PME interrupt
[    2.318142] pcieport 0000:05:09.0: Signaling PME through PCIe PME interrupt
[    2.325119] vesafb: mode is 1920x1200x32, linelength=7680, pages=0
[    2.331279] vesafb: scrolling: redraw
[    2.334934] vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    2.341089] vesafb: framebuffer at 0xf9000000, mapped to 0xffffc90011800000, using 9024k, total 9024k
[    2.433124] Console: switching to colour frame buffer device 240x75
[    2.518826] fb0: VESA VGA frame buffer device
[    2.523463] input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0
[    2.532050] ACPI: Power Button [PWRB]
[    2.535918] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    2.543682] ACPI: Power Button [PWRF]
[    2.548246] ERST: Table is not found!
[    2.552137] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.579283] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    2.606208] 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    2.612194] Linux agpgart interface v0.103
[    2.616618] aic7xxx 0000:08:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    2.833341] scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
[    2.833342]         <Adaptec 2940 Ultra2 SCSI adapter>
[    2.833342]         aic7890/91: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs
[    2.833343] 
[    2.855033] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    2.872831] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode
[    2.881442] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst 
[    2.899980] scsi1 : ahci
[    2.902683] scsi2 : ahci
[    2.905377] scsi3 : ahci
[    2.908072] scsi4 : ahci
[    2.910769] scsi5 : ahci
[    2.913465] scsi6 : ahci
[    2.916207] ata1: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805100 irq 52
[    2.923969] ata2: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805180 irq 52
[    2.931727] ata3: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805200 irq 52
[    2.939487] ata4: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805280 irq 52
[    2.947249] ata5: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805300 irq 52
[    2.955011] ata6: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805380 irq 52
[    2.955026] ahci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    2.955155] ahci 0000:02:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    2.955157] ahci 0000:02:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    2.955342] scsi7 : ahci
[    2.955383] scsi8 : ahci
[    2.955412] ata7: SATA max UDMA/133 abar m2048@0xfb710000 port 0xfb710100 irq 53
[    2.955414] ata8: SATA max UDMA/133 abar m2048@0xfb710000 port 0xfb710180 irq 53
[    2.955431] ahci 0000:0b:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    2.955589] ahci 0000:0b:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    2.955591] ahci 0000:0b:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    2.955856] scsi9 : ahci
[    2.955921] scsi10 : ahci
[    2.955963] ata9: SATA max UDMA/133 abar m2048@0xfb110000 port 0xfb110100 irq 54
[    2.955969] ata10: SATA max UDMA/133 abar m2048@0xfb110000 port 0xfb110180 irq 54
[    2.956064] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    2.964172] serio: i8042 KBD port at 0x60,0x64 irq 1
[    2.964176] serio: i8042 AUX port at 0x60,0x64 irq 12
[    2.964279] mousedev: PS/2 mouse device common for all mice
[    2.964376] rtc_cmos 00:05: RTC can wake from S4
[    2.964497] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0
[    2.964545] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    3.075254] hpet1: lost 6 rtc interrupts
[    3.075294] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.07
[    3.075327] iTCO_wdt: Found a Cougar Point TCO device (Version=2, TCOBASE=0x0460)
[    3.075361] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
[    3.258753] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.258789] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    3.258816] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    3.258843] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    3.258866] ata4: SATA link down (SStatus 0 SControl 300)
[    3.258902] ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.264693] ata10: SATA link down (SStatus 0 SControl 330)
[    3.264722] ata9: SATA link down (SStatus 0 SControl 330)
[    3.286349] ata5.00: ATAPI: TSSTcorp CDDVDW SH-222AB, SB00, max UDMA/100
[    3.286373] device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com
[    3.286447] ata1.00: ATA-9: C300-CTFDDAC064MAG, 0006, max UDMA/100
[    3.286448] ata1.00: 125045424 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.286464] ata6.00: ATA-8: SAMSUNG HD403LJ, CT100-10, max UDMA7
[    3.286465] ata6.00: 781422768 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.486050] ata8: SATA link up 3.0 Gbps (SStatus 123 SControl 330)
[    3.545087] ata3.00: ATA-8: Hitachi HDS722020ALA330, JKAOA3MA, max UDMA/133
[    3.545088] ata3.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.545099] ata2.00: ATA-6: WDC WD360GD-00FLA2, 31.08F31, max UDMA/133
[    3.545100] ata2.00: 72303840 sectors, multi 16: LBA48 
[    3.696400] ata5.00: configured for UDMA/100
[    3.715558] ata8.00: ATA-8: WDC WD15EARS-00Z5B1, 80.00A80, max UDMA/133
[    3.715559] ata8.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    3.715570] ata1.00: configured for UDMA/100
[    3.715587] scsi: waiting for bus probes to complete ...
[    3.798560] ata2.00: configured for UDMA/133
[    3.798578] ata6.00: configured for UDMA/133
[    3.798723] cpuidle: using governor ladder
[    3.836635] ata3.00: configured for UDMA/133
[    3.836845] cpuidle: using governor menu
[    3.872878] TCP cubic registered
[    3.872881] Initializing XFRM netlink socket
[    3.925400] ata8.00: configured for UDMA/133
[    3.943282] Refined TSC clocksource calibration: 3392.293 MHz.
[    3.962633] NET: Registered protocol family 10
[    3.962689] Switching to clocksource tsc
[    3.999976] IPv6 over IPv4 tunneling driver
[    4.001189] scsi 0:0:1:0: Direct-Access     IBM-PCCO ST39102LC     !# B219 PQ: 0 ANSI: 2
[    4.001193] scsi0:A:1:0: Tagged Queuing enabled.  Depth 8
[    4.001196] scsi target0:0:1: Beginning Domain Validation
[    4.008216] scsi target0:0:1: wide asynchronous
[    4.013331] scsi target0:0:1: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 15)
[    4.017659] scsi target0:0:1: Domain Validation skipping write tests
[    4.017660] scsi target0:0:1: Ending Domain Validation
[    4.154146] ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 330)
[    4.154508] ata7.00: ATAPI: LITE-ON LTR-48246S, SS0E, max UDMA/33
[    4.155073] ata7.00: configured for UDMA/33
[    4.223618] NET: Registered protocol family 17
[    4.242771] NET: Registered protocol family 15
[    4.261643] Registering the dns_resolver key type
[    4.281254] registered taskstats version 1
[    4.306282] scsi: waiting for bus probes to complete ...
[    7.342755] sd 0:0:1:0: Attached scsi generic sg0 type 0
[  75.362808] rtc_cmos 00::s setting system clock to 2011-12-10 21:02:04 UTC (1323550924)
[    7.362852] scsi 1:0:0:0: Direct-Access   cATA      C300-CTFDDAC064M 0006 PQ o0N0SI: 5
[    7.362929] sd:i0:0:0: Attached scsi generic sg1 type 0
[    7.363017] sd 1:0:0:0: [sdb] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[    7.363034] sd 0:0:1:0: [sda] 17774160 512-byte logical blocks: (9.10 GB/8.47 GiB)
[    7.363056] sd 1:0:0:0: [sdb] Write Protect is off
[    7.363066] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.363070] scsi 2:0:0:0: Direct-Access     ATA      WDC WD360GD-00FL 31.0 PQ: 0 ANSI: 5
[    7.363119] sd 2:0:0:0: [sdc] 72303840 512-byte logical blocks: (37.0 GB/34.4 GiB)
[    7.363138] sd 2:0:0:0: [sdc] Write Protect is off
[    7.363149] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.363162] sd 2:0:0:0: Attached scsi generic sg2 type 0
[    7.363228] scsi 3:0:0:0: Direct-Access     ATA      Hitachi HDS72202 JKAO PQ: 0 ANSI: 5
[    7.363308] sd 3:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    7.363325] sd 3:0:0:0: [sdd] Write Protect is off
[    7.363329] sd 3:0:0:0: Attached scsi generic sg3 type 0
[    7.363332] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.363937] scsi 5:0:0:0: CD-ROM            TSSTcorp CDDVDW SH-222AB  SB00 PQ: 0 ANSI: 5
[    7.364007]  sdb: sdb1
[    7.365042] sd 0:0:1:0: [sda] Write Protect is off
[    7.366359] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
[    7.366360] cdrom: Uniform CD-ROM driver Revision: 3.20
[    7.366519] sr 5:0:0:0: Attached scsi generic sg4 type 5
[    7.366596] scsi 6:0:0:0: Direct-Access     ATA      SAMSUNG HD403LJ  CT10 PQ: 0 ANSI: 5
[    7.366634] sd 0:0:1:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    7.366657] sd 6:0:0:0: Attached scsi generic sg5 type 0
[    7.366689] sd 6:0:0:0: [sde] 781422768 512-byte logical blocks: (400 GB/372 GiB)
[    7.366780] sd 6:0:0:0: [sde] Write Protect is off
[    7.366812] sd 6:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.375567]  sde: sde1
[    7.378380]  sda: sda1 sda2
[    7.405721]  sdd: sdd1
[    7.409837]  sdc: sdc1 sdc2
[    7.637718] scsi 7:0:0:0: CD-ROM            LITE-ON  LTR-48246S       SS0E PQ: 0 ANSI: 5
[    8.116540] sr1: scsi3-mmc drive: 50x/48x writer cd/rw xa/form2 cdda tray
[    8.116634] sr 7:0:0:0: Attached scsi generic sg6 type 5
[    8.116693] scsi 8:0:0:0: Direct-Access     ATA      WDC WD15EARS-00Z 80.0 PQ: 0 ANSI: 5
[    8.116763] sd 8:0:0:0: Attached scsi generic sg7 type 0
[    8.116797] sd 8:0:0:0: [sdf] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
[    8.116817] sd 8:0:0:0: [sdf] Write Protect is off
[    8.116828] sd 8:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.167440]  sdf: sdf1
[    8.321540] sd 6:0:0:0: [sde] Attached SCSI disk
[    8.321582] sd 8:0:0:0: [sdf] Attached SCSI disk
[    8.321593] sd 3:0:0:0: [sdd] Attached SCSI disk
[    8.321607] sd 2:0:0:0: [sdc] Attached SCSI disk
[    8.321611] sd 1:0:0:0: [sdb] Attached SCSI disk
[    8.326436] sd 0:0:1:0: [sda] Attached SCSI disk
[    8.466980] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[    8.511139] VFS: Mounted root (ext4 filesystem) readonly on device 8:17.
[    8.534635] devtmpfs: mounted
[    8.553387] Freeing unused kernel memory: 672k freed
[    8.732921] udevd[154]: starting version 175
[    8.822243] wmi: Mapper loaded
[    8.823388] usbcore: registered new interface driver usbfs
[    8.823397] usbcore: registered new interface driver hub
[    8.823419] usbcore: registered new device driver usb
[    8.824046] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    8.824065] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    8.824088] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[    8.824092] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
[    8.824113] ehci_hcd 0000:00:1a.0: debug port 2
[    8.827977] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xfb807000
[    8.828491] xhci_hcd 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    8.828514] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    8.828519] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 2
[    8.828645] xhci_hcd 0000:03:00.0: irq 18, io mem 0xfb600000
[    8.828664] xhci_hcd 0000:03:00.0: Failed to enable MSI-X
[    8.828848] hub 2-0:1.0: USB hub found
[    8.828854] hub 2-0:1.0: 2 ports detected
[    8.828890] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    8.828893] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 3
[    8.828966] hub 3-0:1.0: USB hub found
[    8.828973] hub 3-0:1.0: 2 ports detected
[    8.829017] xhci_hcd 0000:06:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    8.829034] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    8.829038] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 4
[    8.829177] xhci_hcd 0000:06:00.0: irq 16, io mem 0xfb400000
[    8.829198] xhci_hcd 0000:06:00.0: Failed to enable MSI-X
[    8.829388] hub 4-0:1.0: USB hub found
[    8.829394] hub 4-0:1.0: 2 ports detected
[    8.829426] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    8.829428] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 5
[    8.829478] hub 5-0:1.0: USB hub found
[    8.829485] hub 5-0:1.0: 2 ports detected
[    8.829528] xhci_hcd 0000:09:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    8.829541] xhci_hcd 0000:09:00.0: xHCI Host Controller
[    8.829544] xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 6
[    8.829684] xhci_hcd 0000:09:00.0: irq 16, io mem 0xfb200000
[    8.829703] xhci_hcd 0000:09:00.0: Failed to enable MSI-X
[    8.829897] hub 6-0:1.0: USB hub found
[    8.829904] hub 6-0:1.0: 2 ports detected
[    8.829939] xhci_hcd 0000:09:00.0: xHCI Host Controller
[    8.829942] xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 7
[    8.830012] hub 7-0:1.0: USB hub found
[    8.830018] hub 7-0:1.0: 2 ports detected
[    8.833584] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    8.833597] r8169 0000:0a:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    8.837659] FDC 0 is a post-1991 82077
[    8.837755] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    8.837841] hub 1-0:1.0: USB hub found
[    8.837844] hub 1-0:1.0: 2 ports detected
[    8.837879] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    9.750978] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[    9.751002] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 8
[    9.751033] ehci_hcd 0000:00:1d.0: debug port 2
[    9.751236] r8169 0000:0a:00.0: eth0: RTL8168e/8111e at 0xffffc90000c78000, 00:25:22:9f:75:32, XID 0c200000 IRQ 58
[    9.751238] r8169 0000:0a:00.0: eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]
[    9.754903] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xfb806000
[    9.764079] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    9.764158] hub 8-0:1.0: USB hub found
[    9.764161] hub 8-0:1.0: 2 ports detected
[    9.764244] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    9.764323] snd_hda_intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
[    9.988441] usb 1-1: new high-speed USB device number 2 using ehci_hcd
[   10.102567] hub 1-1:1.0: USB hub found
[   10.102599] hub 1-1:1.0: 6 ports detected
[   10.204843] usb 8-1: new high-speed USB device number 2 using ehci_hcd
[   10.358061] hub 8-1:1.0: USB hub found
[   10.394517] hub 8-1:1.0: 8 ports detected
[   10.668343] Adding 4194300k swap on /dev/sdc1.  Priority:-1 extents:1 across:4194300k 
[   10.690592] usb 8-1.1: new low-speed USB device number 3 using ehci_hcd
[   10.755532] EXT4-fs (sdb1): re-mounted. Opts: (null)
[   10.882451] EXT4-fs (sdb1): re-mounted. Opts: discard
[   10.904964] usb 8-1.2: new high-speed USB device number 4 using ehci_hcd
[   11.011408] hub 8-1.2:1.0: USB hub found
[   11.031475] hub 8-1.2:1.0: 3 ports detected
[   11.115347] usb 8-1.7: new high-speed USB device number 5 using ehci_hcd
[   11.278551] w83627ehf: Found NCT6776F chip at 0x290
[   11.283153] fuse init (API version 7.17)
[   11.295526] input: Microsoft Microsoft IntelliMouse® Explorer as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.1/8-1.1:1.0/input/input2
[   11.295602] generic-usb 0003:045E:001E.0001: input,hidraw0: USB HID v1.00 Mouse [Microsoft Microsoft IntelliMouse® Explorer] on usb-0000:00:1d.0-1.1/input0
[   11.295612] usbcore: registered new interface driver usbhid
[   11.295613] usbhid: USB HID core driver
[   11.315746] usb 8-1.8: new low-speed USB device number 6 using ehci_hcd
[   11.335619] i2c /dev entries driver
[   11.338995] uhci_hcd: USB Universal Host Controller Interface driver
[   11.409334] input: USBPS2 as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.8/8-1.8:1.0/input/input3
[   11.409410] generic-usb 0003:0D3D:0001.0002: input,hidraw1: USB HID v1.00 Keyboard [USBPS2] on usb-0000:00:1d.0-1.8/input0
[   11.413724] input: USBPS2 as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.8/8-1.8:1.1/input/input4
[   11.413949] generic-usb 0003:0D3D:0001.0003: input,hidraw2: USB HID v1.00 Mouse [USBPS2] on usb-0000:00:1d.0-1.8/input1
[   11.465430] ipmi message handler version 39.2
[   11.476319] usb 8-1.2.2: new low-speed USB device number 7 using ehci_hcd
[   11.564770] input: Apple, Inc Apple Keyboard as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.2/8-1.2.2/8-1.2.2:1.0/input/input5
[   11.564849] apple 0003:05AC:0220.0004: input,hidraw3: USB HID v1.11 Keyboard [Apple, Inc Apple Keyboard] on usb-0000:00:1d.0-1.2.2/input0
[   11.567692] input: Apple, Inc Apple Keyboard as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.2/8-1.2.2/8-1.2.2:1.1/input/input6
[   11.567764] apple 0003:05AC:0220.0005: input,hidraw4: USB HID v1.11 Device [Apple, Inc Apple Keyboard] on usb-0000:00:1d.0-1.2.2/input1
[   14.819506] EXT4-fs (sdc2): mounted filesystem with ordered data mode. Opts: (null)
[   14.911581] EXT4-fs (sdd1): mounted filesystem with ordered data mode. Opts: (null)
[   14.951529] REISERFS (device sda1): found reiserfs format "3.6" with standard journal
[   14.984400] REISERFS (device sda1): using ordered data mode
[   15.007351] reiserfs: using flush barriers
[   15.031223] REISERFS (device sda1): journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
[   15.064626] REISERFS (device sda1): checking transaction log (sda1)
[   15.120781] REISERFS (device sda1): Using r5 hash to sort names
[   15.222910] EXT4-fs (sdf1): mounted filesystem with ordered data mode. Opts: (null)
[   15.529837] r8169 0000:0a:00.0: eth0: unable to load firmware patch rtl_nic/rtl8168e-2.fw (-2)
[   15.547095] r8169 0000:0a:00.0: eth0: link down
[   15.547100] r8169 0000:0a:00.0: eth0: link down
[   15.547406] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   17.979110] r8169 0000:0a:00.0: eth0: link up
[   18.015709] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   19.147049] RPC: Registered named UNIX socket transport module.
[   19.189908] RPC: Registered udp transport module.
[   19.231910] RPC: Registered tcp transport module.
[   19.273554] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   19.328020] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[   19.465495] tun: Universal TUN/TAP device driver, 1.6
[   19.492962] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[   19.524164] ADDRCONF(NETDEV_UP): tap0: link is not ready
[   19.535119] device eth0 entered promiscuous mode
[   19.538001] device tap0 entered promiscuous mode
[   19.538867] br0: port 1(eth0) entering forwarding state
[   19.538869] br0: port 1(eth0) entering forwarding state
[   24.775323] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   24.799166] NFSD: starting 90-second grace period
[   24.970507] [drm] Initialized drm 1.1.0 20060810
[   24.980078] MXM: GUID detected in BIOS
[   24.983836] nouveau 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   24.991494] [drm] nouveau 0000:01:00.0: Detected an NV50 generation card (0x084200a2)
[   24.999344] fb: conflicting fb hw usage nouveaufb vs VESA VGA - removing generic driver
[   25.007374] Console: switching to colour dummy device 80x25
[   25.015078] [drm] nouveau 0000:01:00.0: Attempting to load BIOS image from PRAMIN
[   25.127351] [drm] nouveau 0000:01:00.0: ... appears to be valid
[   25.133256] [drm] nouveau 0000:01:00.0: BIT BIOS found
[   25.138373] [drm] nouveau 0000:01:00.0: Bios version 60.84.51.00
[   25.144365] [drm] nouveau 0000:01:00.0: TMDS table version 2.0
[   25.150178] [drm] nouveau 0000:01:00.0: Found Display Configuration Block version 4.0
[   25.157984] [drm] nouveau 0000:01:00.0: Raw DCB entry 0: 02000300 00000028
[   25.164834] [drm] nouveau 0000:01:00.0: Raw DCB entry 1: 01000302 00020030
[   25.171689] [drm] nouveau 0000:01:00.0: Raw DCB entry 2: 04011310 00000028
[   25.178558] [drm] nouveau 0000:01:00.0: Raw DCB entry 3: 02011312 00020030
[   25.185410] [drm] nouveau 0000:01:00.0: Raw DCB entry 4: 010223f1 00c0c080
[   25.192264] [drm] nouveau 0000:01:00.0: DCB connector table: VHER 0x40 5 14 2
[   25.199374] [drm] nouveau 0000:01:00.0:   0: 0x00002030: type 0x30 idx 0 tag 0x08
[   25.206834] [drm] nouveau 0000:01:00.0:   1: 0x00001130: type 0x30 idx 1 tag 0x07
[   25.214296] [drm] nouveau 0000:01:00.0:   2: 0x00000210: type 0x10 idx 2 tag 0xff
[   25.221758] [drm] nouveau 0000:01:00.0:   3: 0x00000211: type 0x11 idx 3 tag 0xff
[   25.229221] [drm] nouveau 0000:01:00.0:   4: 0x00000213: type 0x13 idx 4 tag 0xff
[   25.236683] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 0 at offset 0xC1BE
[   25.267522] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 1 at offset 0xC50B
[   25.292007] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 2 at offset 0xCF58
[   25.299668] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 3 at offset 0xD04A
[   25.308398] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 4 at offset 0xD27D
[   25.316040] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table at offset 0xD2E2
[   25.343487] [drm] nouveau 0000:01:00.0: 0xD2E2: Condition still not met after 20ms, skipping following opcodes
[   25.372921] [drm] nouveau 0000:01:00.0: 1 available performance level(s)
[   25.379598] [drm] nouveau 0000:01:00.0: 0: core 580MHz shader 1508MHz memory 800MHz fanspeed 100%
[   25.388473] [drm] nouveau 0000:01:00.0: c: core 540MHz shader 1512MHz memory 702MHz voltage 1320mV
[   25.400006] [TTM] Zone  kernel: Available graphics memory: 4088300 kiB.
[   25.406608] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB.
[   25.413197] [TTM] Initializing pool allocator.
[   25.417653] [drm] nouveau 0000:01:00.0: Detected 256MiB VRAM
[   25.423316] mtrr: type mismatch for c0000000,10000000 old: write-back new: write-combining
[   25.431589] [drm] nouveau 0000:01:00.0: 512 MiB GART (aperture)
[   25.447822] [drm] nouveau 0000:01:00.0: DCB encoder 1 unknown
[   25.453566] [drm] nouveau 0000:01:00.0: TV-1 has no encoders, removing
[   25.475572] [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
[   25.482233] [drm] No driver support for vblank timestamp query.
[   25.488149] parse error at position 4 in video mode 'efifb'
[   25.493710] parse error at position 4 in video mode 'efifb'
[   25.718481] [drm] nouveau 0000:01:00.0: allocated 1920x1200 fb: 0x320000, bo ffff88023504a000
[   25.727092] fbcon: nouveaufb (fb0) is primary device
[   25.732118] Console: switching to colour frame buffer device 160x64
[   25.738373] fb0: nouveaufb frame buffer device
[   25.742807] drm: registered panic notifier
[   25.746946] [drm] Initialized nouveau 0.0.16 20090420 for 0000:01:00.0 on minor 0

Debian GNU/Linux wheezy/sid luna ttyS0

luna login: [   34.525172] br0: port 1(eth0) entering forwarding state
[   42.205032] EXT4-fs (sdb1): re-mounted. Opts: discard,commit=0
[   42.212123] EXT4-fs (sdc2): re-mounted. Opts: commit=0
[   42.218150] EXT4-fs (sdd1): re-mounted. Opts: commit=0
[   42.224391] EXT4-fs (sdf1): re-mounted. Opts: commit=0
[ 3983.686577] [drm] nouveau 0000:01:00.0: PFIFO_DMA_PUSHER - Ch 2 Get 0x002002e178 Put 0x002002e45c IbGet 0x00000caf IbPut 0x00000cc2 State 0x80002054 (err: INVALID_CMD) Push 0x00406040
[64195.493018] nfsd: last server has exited, flushing export cache
[64199.669682] br0: port 1(eth0) entering forwarding state
[64213.384189] device tap0 left promiscuous mode
[64213.388884] br0: port 2(tap0) entering disabled state
[64213.427643] device eth0 left promiscuous mode
[64213.432348] br0: port 1(eth0) entering disabled state
[64216.081808] EXT4-fs (sdb1): re-mounted. Opts: (null)
[64218.311281] kvm: exiting hardware virtualization
[64218.316392] sd 8:0:0:0: [sdf] Synchronizing SCSI cache
[64218.321893] sd 6:0:0:0: [sde] Synchronizing SCSI cache
[64218.327800] sd 3:0:0:0: [sdd] Synchronizing SCSI cache
[64218.333426] sd 2:0:0:0: [sdc] Synchronizing SCSI cache
[64218.339230] sd 1:0:0:0: [sdb] Synchronizing SCSI cache
[64219.160405] sd 0:0:1:0: [sda] Synchronizing SCSI cache
[64219.166745] xhci_hcd 0000:09:00.0: PCI INT A disabled
[64219.172156] xhci_hcd 0000:06:00.0: PCI INT A disabled
[64219.177669] xhci_hcd 0000:03:00.0: PCI INT A disabled
[64219.182903] ehci_hcd 0000:00:1d.0: PCI INT A disabled
[64219.188105] ehci_hcd 0000:00:1a.0: PCI INT A disabled
[    0.000000] Linux version 3.2.0-rc5+ (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #3 SMP Sun Dec 11 14:52:43 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3391.851 MHz processor.
[    0.000001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6783.70 BogoMIPS (lpj=3391851)
[    0.010596] pid_max: default: 32768 minimum: 301
[    0.015476] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023742] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031299] Mount-cache hash table entries: 256
[    0.035894] CPU: Physical Processor ID: 0
[    0.039901] CPU: Processor Core ID: 0
[    0.043557] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043558] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056562] mce: CPU supports 9 MCE banks
[    0.060570] CPU0: Thermal monitoring enabled (TM1)
[    0.065353] using mwait in idle threads.
[    0.069323] Freeing SMP alternatives: 20k freed
[    0.073852] ACPI: Core revision 20110623
[    0.085668] Disabling x2apic
[    0.088567] Switched APIC routing to flat.
[    0.092654] BUG: unable to handle kernel paging request at ffffffffff5fb020
[    0.099625] IP: [<ffffffff81018f7e>] native_apic_mem_read+0x2/0x9
[    0.105719] PGD 14cd067 PUD 14ce067 PMD 15be067 PTE 0
[    0.110818] Oops: 0000 [#1] SMP 
[    0.114068] CPU 0 
[    0.115898] Modules linked in:
[    0.119148] 
[    0.120635] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5+ #3 To Be Filled By O.E.M. To Be Filled By O.E.M./P67 Extreme6
[    0.131505] RIP: 0010:[<ffffffff81018f7e>]  [<ffffffff81018f7e>] native_apic_mem_read+0x2/0x9
[    0.140011] RSP: 0018:ffff880236499f18  EFLAGS: 00010246
[    0.145309] RAX: ffffffff814d8500 RBX: 000000000000cab8 RCX: 0000000000000061
[    0.152419] RDX: 00000000000000c0 RSI: 0000000000000046 RDI: 0000000000000020
[    0.159523] RBP: 000000000000cac0 R08: 0000000000000000 R09: 0000000000000000
[    0.166635] R10: 0000000000000001 R11: 0720072007200720 R12: 0000000000000008
[    0.173747] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[    0.180853] FS:  0000000000000000(0000) GS:ffff88023f400000(0000) knlGS:0000000000000000
[    0.188916] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.194645] CR2: ffffffffff5fb020 CR3: 00000000014cb000 CR4: 00000000000406f0
[    0.201755] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.208861] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.215971] Process swapper/0 (pid: 1, threadinfo ffff880236498000, task ffff880236497750)
[    0.224200] Stack:
[    0.226203]  ffffffff8152c571 ffffffff815b1f50 0000000000000000 0000000000000000
[    0.233643]  ffffffff81522abf 0000000000000000 ffffffff813842b4 0000000000000000
[    0.241112]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[    0.248604] Call Trace:
[    0.251043]  [<ffffffff8152c571>] ? native_smp_prepare_cpus+0x303/0x325
[    0.257636]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
[    0.263104]  [<ffffffff813842b4>] ? kernel_thread_helper+0x4/0x10
[    0.269173]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.274815]  [<ffffffff813842b0>] ? gs_change+0xb/0xb
[    0.279844] Code: b9 1b 00 00 00 0f 32 f6 c4 04 0f 95 c0 0f b6 c0 c3 83 3d f1 e6 5c 00 01 76 05 e9 00 c2 ff ff c3 89 ff 89 b7 00 b0 5f ff c3 89 ff <8b> 87 00 b0 5f ff c3 48 8b 07 25 ff 00 00 00 c3 48 8b 07 48 8b 
[    0.299789] RIP  [<ffffffff81018f7e>] native_apic_mem_read+0x2/0x9
[    0.305967]  RSP <ffff880236499f18>
[    0.309442] CR2:nel panic - not syncing: Attempted to kill init!
[    0.323252] Pid: 1, comm: swapper/0 Tainted: G      D      3.2.0-rc5+ #3
 [<ffffffff8137b2ef>] ? no_context+0x1ff/0x20e
[    0.353016]  [<ffffffff812477a4>] ? uart_console_write+0x39/0x47
[    0.358onsole_drivers+0x72/0x83
[    0.371065]  [<ffffffff8104eb5f>] ? up+0xb/0x34
[    0.375580]  [<ffffffff81037360>] ? console_un_thread_helper+0x4/0x10
[    0.410795]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.416431]  [<ffffffff813842b0>] [    0.000000] Linux version 3.2.0-rc5+ (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #3 SMP Sun Dec 11 14:52:43 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3392.371 MHz processor.
[    0.000002] Calibrating delay loop (skipped), value calculated using timer frequency.. 6784.74 BogoMIPS (lpj=3392371)
[    0.010596] pid_max: default: 32768 minimum: 301
[    0.015486] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023786] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031340] Mount-cache hash table entries: 256
[    0.035945] CPU: Physical Processor ID: 0
[    0.039961] CPU: Processor Core ID: 0
[    0.043618] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043619] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056632] mce: CPU supports 9 MCE banks
[    0.060639] CPU0: Thermal monitoring enabled (TM1)
[    0.065422] using mwait in idle threads.
[    0.069392] Freeing SMP alternatives: 20k freed
[    0.073920] ACPI: Core revision 20110623
[    0.085774] Disabling x2apic
[    0.088679] Switched APIC routing to flat.
[    0.092766] BUG: unable to handle kernel paging request at ffffffffff5fb020
[    0.099739] IP: [<ffffffff81018f7e>] native_apic_mem_read+0x2/0x9
[    0.105830] PGD 14cd067 PUD 14ce067 PMD 15be067 PTE 0
[    0.110929] Oops: 0000 [#1] SMP 
[    0.114180] CPU 0 
[    0.116011] Modules linked in:
[    0.119262] 
[    0.120749] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5+ #3 To Be Filled By O.E.M. To Be Filled By O.E.M./P67 Extreme6
[    0.131626] RIP: 0010:[<ffffffff81018f7e>]  [<ffffffff81018f7e>] native_apic_mem_read+0x2/0x9
[    0.140131] RSP: 0018:ffff880236499f18  EFLAGS: 00010246
[    0.145420] RAX: ffffffff814d8500 RBX: 000000000000cab8 RCX: 0000000000000061
[    0.152540] RDX: 00000000000000c0 RSI: 0000000000000046 RDI: 0000000000000020
[    0.159644] RBP: 000000000000cac0 R08: 0000000000000000 R09: 0000000000000000
[    0.166766] R10: 0000000000000001 R11: 0720072007200720 R12: 0000000000000008
[    0.173877] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[    0.180980] FS:  0000000000000000(0000) GS:ffff88023f400000(0000) knlGS:0000000000000000
[    0.189035] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[    0.194756] CR2: ffffffffff5fb020 CR3: 00000000014cb000 CR4: 00000000000406f0
[    0.201859] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.208965] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[    0.216075] Process swapper/0 (pid: 1, threadinfo ffff880236498000, task ffff880236497750)
[    0.224301] Stack:
[    0.226309]  ffffffff8152c571 ffffffff815b1f50 0000000000000000 0000000000000000
[    0.233740]  ffffffff81522abf 0000000000000000 ffffffff813842b4 0000000000000000
[    0.241171]  0000000000000000 0000000000000000 0000000000000000 0000000000000000
[    0.248614] Call Trace:
[    0.251052]  [<ffffffff8152c571>] ? native_smp_prepare_cpus+0x303/0x325
[    0.257644]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
[    0.263105]  [<ffffffff813842b4>] ? kernel_thread_helper+0x4/0x10
[    0.269174]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.274815]  [<ffffffff813842b0>] ? gs_change+0xb/0xb
[    0.279844] Code: b9 1b 00 00 00 0f 32 f6 c4 04 0f 95 c0 0f b6 c0 c3 83 3d f1 e6 5c 00 01 76 05 e9 00 c2 ff ff c3 89 ff 89 b7 00 b0 5f ff c3 89 ff <8b> 87 00 b0 5f ff c3 48 8b 07 25 ff 00 00 00 c3 48 8b 07 48 8b 
[    0.299773] RIP  [<ffffffff81018f7e>] native_apic_mem_read+0x2/336]  [<ffffffff8137b88e>] ? panic+0x95/0x18e
[    0.337287]  [<ffffffff81039498>] ? do_exit+0x8d/0x6c9
[    0.342405]  [<fff   0.352991]  [<ffffffff812477a4>] ? uart_console_write+0x39/0x47
[    0.358980]  [<ffffffff8101bc32>] ? do_page_fault+0x155/0ffffff81382b2f>] ? page_fault+0x1f/0x30
[    0.386668]  [<ffffffff81018f7e>] ? native_apic_mem_read+0x2/0x9
[    0.392658]  [

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-11 22:05       ` Berck E. Nash
@ 2011-12-12  3:40         ` Yinghai Lu
  2011-12-12 23:21           ` Berck E. Nash
  2011-12-12 23:42           ` H. Peter Anvin
  0 siblings, 2 replies; 26+ messages in thread
From: Yinghai Lu @ 2011-12-12  3:40 UTC (permalink / raw)
  To: Berck E. Nash; +Cc: Ingo Molnar, linux-kernel, Suresh Siddha

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]

On Sun, Dec 11, 2011 at 2:05 PM, Berck E. Nash <flyboy@gmail.com> wrote:
>
> No problem: It does not work as expected.  Log attached.

x2apic pre-enabled skip the mapping for lapic address. so need map it
after fall back.

please check attached -v5..

Thanks

Yinghai Lu

[-- Attachment #2: nox2apic_v5.patch --]
[-- Type: text/x-patch, Size: 10328 bytes --]

Subject: [PATCH -v5] x86: Disable x2apic if nox2apic is specified or intr-remap can not be enabled

For
1. x2apic preenabled system
2. first kernel have x2apic enabled, and try to kexec second kernel with "nox2apic"

Will put back cpu with apic id < 255 into xapic mode, instead of panic.

-v2: use x2apic_disabled instead of nox2apic, Suggested by Thomas
     update x2apic_supported with x2apic_disabled, Suggested by Thomas

-v3: add checking for boot cpu apic id > 255. in that case will just panic
     --- pointed out by Suresh.

-v4: according to Ingo, for x2apic pre-enabled system, if intr-remap can not
	be enabled, try to disable x2apic instead of panic and request to
	specify nox2apic for next boot.

-v5: fix the lapic mapping for falling path.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/apic.h         |    6 +-
 arch/x86/include/asm/apicdef.h      |    1 
 arch/x86/include/asm/processor.h    |    1 
 arch/x86/kernel/acpi/boot.c         |   10 +++-
 arch/x86/kernel/apic/apic.c         |   76 ++++++++++++++++++++++++++----------
 arch/x86/kernel/apic/apic_flat_64.c |    7 ++-
 arch/x86/kernel/cpu/topology.c      |   21 +++++++++
 arch/x86/mm/srat.c                  |    7 ++-
 8 files changed, 103 insertions(+), 26 deletions(-)

Index: linux-2.6/arch/x86/include/asm/apic.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apic.h
+++ linux-2.6/arch/x86/include/asm/apic.h
@@ -176,6 +176,7 @@ static inline u64 native_x2apic_icr_read
 }
 
 extern int x2apic_phys;
+extern int x2apic_disabled;
 extern void check_x2apic(void);
 extern void enable_x2apic(void);
 extern void x2apic_icr_write(u32 low, u32 id);
@@ -183,7 +184,7 @@ static inline int x2apic_enabled(void)
 {
 	u64 msr;
 
-	if (!cpu_has_x2apic)
+	if (!cpu_has_x2apic || x2apic_disabled)
 		return 0;
 
 	rdmsrl(MSR_IA32_APICBASE, msr);
@@ -192,7 +193,7 @@ static inline int x2apic_enabled(void)
 	return 0;
 }
 
-#define x2apic_supported()	(cpu_has_x2apic)
+#define x2apic_supported()	(cpu_has_x2apic && !x2apic_disabled)
 static inline void x2apic_force_phys(void)
 {
 	x2apic_phys = 1;
@@ -214,6 +215,7 @@ static inline void x2apic_force_phys(voi
 
 #define	x2apic_preenabled 0
 #define	x2apic_supported()	0
+#define	x2apic_disabled	1
 #endif
 
 extern void enable_IR_x2apic(void);
Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -219,6 +219,8 @@ static int __init
 acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
 {
 	struct acpi_madt_local_x2apic *processor = NULL;
+	int apic_id;
+	u8 enabled;
 
 	processor = (struct acpi_madt_local_x2apic *)header;
 
@@ -227,6 +229,8 @@ acpi_parse_x2apic(struct acpi_subtable_h
 
 	acpi_table_print_madt_entry(header);
 
+	apic_id = processor->local_apic_id;
+	enabled = processor->lapic_flags & ACPI_MADT_ENABLED;
 #ifdef CONFIG_X86_X2APIC
 	/*
 	 * We need to register disabled CPU as well to permit
@@ -235,8 +239,10 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	 * to not preallocating memory for all NR_CPUS
 	 * when we use CPU hotplug.
 	 */
-	acpi_register_lapic(processor->local_apic_id,	/* APIC ID */
-			    processor->lapic_flags & ACPI_MADT_ENABLED);
+	if (x2apic_disabled && (apic_id >= 0xff) && enabled)
+		printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
+	else
+		acpi_register_lapic(apic_id, enabled);
 #else
 	printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
 #endif
Index: linux-2.6/arch/x86/kernel/apic/apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic.c
+++ linux-2.6/arch/x86/kernel/apic/apic.c
@@ -147,15 +147,14 @@ int x2apic_mode;
 #ifdef CONFIG_X86_X2APIC
 /* x2apic enabled before OS handover */
 static int x2apic_preenabled;
+int x2apic_disabled;
 static __init int setup_nox2apic(char *str)
 {
-	if (x2apic_enabled()) {
-		pr_warning("Bios already enabled x2apic, "
-			   "can't enforce nox2apic");
-		return 0;
-	}
+	if (x2apic_enabled())
+		pr_warning("Bios already enabled x2apic, will disable it");
+
+	x2apic_disabled = 1;
 
-	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
 	return 0;
 }
 early_param("nox2apic", setup_nox2apic);
@@ -1431,8 +1430,38 @@ void __init bsp_end_local_APIC_setup(voi
 }
 
 #ifdef CONFIG_X86_X2APIC
+
+static void disable_x2apic(void)
+{
+	int msr, msr2;
+
+	if (!cpu_has_x2apic)
+		return;
+
+	rdmsr(MSR_IA32_APICBASE, msr, msr2);
+	if (msr & X2APIC_ENABLE) {
+		u32 x2apic_id = x2apic_cpuid_initial_apicid();
+
+		if (x2apic_id > 255)
+			panic("Can not disable x2apic, id: %08x\n", x2apic_id);
+
+		pr_info("Disabling x2apic\n");
+		/*
+		 * Need to disable xapic and x2apic at the same time at first
+		 *  then enable xapic
+		 */
+		wrmsr(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE),
+			 0);
+		wrmsr(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE, 0);
+	}
+}
 void check_x2apic(void)
 {
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (x2apic_enabled()) {
 		pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
 		x2apic_preenabled = x2apic_mode = 1;
@@ -1443,6 +1472,11 @@ void enable_x2apic(void)
 {
 	int msr, msr2;
 
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (!x2apic_mode)
 		return;
 
@@ -1493,7 +1527,7 @@ void __init enable_IR_x2apic(void)
 	ret = save_ioapic_entries();
 	if (ret) {
 		pr_info("Saving IO-APIC state failed: %d\n", ret);
-		goto out;
+		return;
 	}
 
 	local_irq_save(flags);
@@ -1505,13 +1539,24 @@ void __init enable_IR_x2apic(void)
 	else
 		ret = enable_IR();
 
+	if (x2apic_disabled)
+		goto nox2apic;
+
 	if (ret < 0) {
 		/* IR is required if there is APIC ID > 255 even when running
 		 * under KVM
 		 */
 		if (max_physical_apicid > 255 ||
-		    !hypervisor_x2apic_available())
+		    !hypervisor_x2apic_available()) {
+			if (x2apic_preenabled) {
+				disable_x2apic();
+				x2apic_disabled = 1;
+				x2apic_mode = 0;
+				/* need to map lapic address */
+				register_lapic_address(mp_lapic_addr);
+			}
 			goto nox2apic;
+		}
 		/*
 		 * without IR all CPUs can be addressed by IOAPIC/MSI
 		 * only in physical mode
@@ -1519,8 +1564,10 @@ void __init enable_IR_x2apic(void)
 		x2apic_force_phys();
 	}
 
-	if (ret == IRQ_REMAP_XAPIC_MODE)
+	if (ret == IRQ_REMAP_XAPIC_MODE) {
+		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
 		goto nox2apic;
+	}
 
 	x2apic_enabled = 1;
 
@@ -1535,17 +1582,6 @@ nox2apic:
 		restore_ioapic_entries();
 	legacy_pic->restore_mask();
 	local_irq_restore(flags);
-
-out:
-	if (x2apic_enabled || !x2apic_supported())
-		return;
-
-	if (x2apic_preenabled)
-		panic("x2apic: enabled by BIOS but kernel init failed.");
-	else if (ret == IRQ_REMAP_XAPIC_MODE)
-		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
-	else if (ret < 0)
-		pr_info("x2apic not enabled, IRQ remapping init failed\n");
 }
 
 #ifdef CONFIG_X86_64
Index: linux-2.6/arch/x86/mm/srat.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat.c
+++ linux-2.6/arch/x86/mm/srat.c
@@ -69,6 +69,12 @@ acpi_numa_x2apic_affinity_init(struct ac
 	if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0)
 		return;
 	pxm = pa->proximity_domain;
+	apic_id = pa->apic_id;
+	if (x2apic_disabled && (apic_id >= 0xff)) {
+		printk(KERN_INFO "SRAT: PXM %u -> X2APIC 0x%04x ignored\n",
+			 pxm, apic_id);
+		return;
+	}
 	node = setup_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
@@ -76,7 +82,6 @@ acpi_numa_x2apic_affinity_init(struct ac
 		return;
 	}
 
-	apic_id = pa->apic_id;
 	if (apic_id >= MAX_LOCAL_APIC) {
 		printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u skipped apicid that is too big\n", pxm, apic_id, node);
 		return;
Index: linux-2.6/arch/x86/include/asm/apicdef.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apicdef.h
+++ linux-2.6/arch/x86/include/asm/apicdef.h
@@ -144,6 +144,7 @@
 
 #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
 #define APIC_BASE_MSR	0x800
+#define XAPIC_ENABLE	(1UL << 11)
 #define X2APIC_ENABLE	(1UL << 10)
 
 #ifdef CONFIG_X86_32
Index: linux-2.6/arch/x86/include/asm/processor.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/processor.h
+++ linux-2.6/arch/x86/include/asm/processor.h
@@ -168,6 +168,7 @@ extern void init_scattered_cpuid_feature
 extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c);
 extern unsigned short num_cache_leaves;
 
+u32 x2apic_cpuid_initial_apicid(void);
 extern void detect_extended_topology(struct cpuinfo_x86 *c);
 extern void detect_ht(struct cpuinfo_x86 *c);
 
Index: linux-2.6/arch/x86/kernel/cpu/topology.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/topology.c
+++ linux-2.6/arch/x86/kernel/cpu/topology.c
@@ -21,6 +21,27 @@
 #define BITS_SHIFT_NEXT_LEVEL(eax)	((eax) & 0x1f)
 #define LEVEL_MAX_SIBLINGS(ebx)		((ebx) & 0xffff)
 
+u32 x2apic_cpuid_initial_apicid(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	if (boot_cpu_data.cpuid_level < 0xb)
+		return 0;
+
+	cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+	/*
+	 * check if the cpuid leaf 0xb is actually implemented.
+	 */
+	if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+		return 0;
+
+	/*
+	 * initial apic id, which also represents 32-bit extended x2apic id.
+	 */
+	return edx;
+}
+
 /*
  * Check for extended topology enumeration cpuid leaf 0xb and if it
  * exists, use it for populating initial_apicid and cpu topology
Index: linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic_flat_64.c
+++ linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
@@ -171,9 +171,14 @@ static int flat_phys_pkg_id(int initial_
 	return initial_apic_id >> index_msb;
 }
 
+static int flat_probe(void)
+{
+	return 1;
+}
+
 static struct apic apic_flat =  {
 	.name				= "flat",
-	.probe				= NULL,
+	.probe				= flat_probe,
 	.acpi_madt_oem_check		= flat_acpi_madt_oem_check,
 	.apic_id_registered		= flat_apic_id_registered,
 

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-11 21:43     ` Yinghai Lu
  2011-12-11 22:05       ` Berck E. Nash
@ 2011-12-12  7:23       ` Suresh Siddha
  1 sibling, 0 replies; 26+ messages in thread
From: Suresh Siddha @ 2011-12-12  7:23 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Ingo Molnar, Berck E. Nash, linux-kernel

On Sun, 2011-12-11 at 13:43 -0800, Yinghai Lu wrote:
> -v4: according to Ingo, for x2apic pre-enabled system, if intr-remap can not
>         be enabled, try to disable x2apic instead of panic and request to
>         specify nox2apic for next boot. 

If bios has not exported any DMAR tables but enabled x2apic pre-boot, it
is tricky for the OS to disable x2apic and continue with xapic for
number of reasons. For example, if chipset has already enabled
interrupt-remapping (which is typically recommended to be enabled before
x2apic) but didn't provide DMAR tables, then there is no way OS can
disable interrupt-remapping in this case.

So even though the cpu might be back in xapic mode, but external
interrupts won't work.

thanks,
suresh


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-12  3:40         ` Yinghai Lu
@ 2011-12-12 23:21           ` Berck E. Nash
  2011-12-13  0:25             ` Suresh Siddha
  2011-12-12 23:42           ` H. Peter Anvin
  1 sibling, 1 reply; 26+ messages in thread
From: Berck E. Nash @ 2011-12-12 23:21 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Ingo Molnar, linux-kernel, Suresh Siddha

[-- Attachment #1: Type: text/plain, Size: 230 bytes --]

On 12/11/2011 08:40 PM, Yinghai Lu wrote:
> x2apic pre-enabled skip the mapping for lapic address. so need map it
> after fall back.
> 
> please check attached -v5..

It appears to work correctly as far as I can tell.

Berck Nash

[-- Attachment #2: x2apic-v5.log --]
[-- Type: text/plain, Size: 43034 bytes --]

[    0.000000] Linux version 3.2.0-rc5+ (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #4 SMP Mon Dec 12 16:03:52 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3391.832 MHz processor.
[    0.000001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6783.66 BogoMIPS (lpj=3391832)
[    0.010595] pid_max: default: 32768 minimum: 301
[    0.015473] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023750] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031306] Mount-cache hash table entries: 256
[    0.035901] CPU: Physical Processor ID: 0
[    0.039900] CPU: Processor Core ID: 0
[    0.043558] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043558] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056562] mce: CPU supports 9 MCE banks
[    0.060571] CPU0: Thermal monitoring enabled (TM1)
[    0.065354] using mwait in idle threads.
[    0.069332] Freeing SMP alternatives: 20k freed
[    0.073857] ACPI: Core revision 20110623
[    0.085677] Disabling x2apic
[    0.088578] Switched APIC routing to flat.
[    0.093036] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
[    0.108819] CPU0: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz stepping 07
[    0.216400] Performance Events: PEBS fmt1+, SandyBridge events, Intel PMU driver.
[    0.223907] PEBS disabled due to CPU errata.
[    0.228168] ... version:                3
[    0.232168] ... bit width:              48
[    0.236255] ... generic registers:      4
[    0.240247] ... value mask:             0000ffffffffffff
[    0.245537] ... max period:             000000007fffffff
[    0.250834] ... fixed-purpose events:   3
[    0.254828] ... event mask:             000000070000000f
[    0.260194] Booting Node   0, Processors  #1
[    0.274900] Disabling x2apic
[    0.357120]  #2
[    0.369179] Disabling x2apic
[    0.451829]  #3
[    0.463901] Disabling x2apic
[    0.546545]  #4
[    0.558655] Disabling x2apic
[    0.641286]  #5
[    0.653348] Disabling x2apic
[    0.735919]  #6
[    0.747976] Disabling x2apic
[    0.830722]  #7 Ok.
[    0.843117] Disabling x2apic
[    0.925323] Brought up 8 CPUs
[    0.928281] Total of 8 processors activated (54269.62 BogoMIPS).
[    0.939081] devtmpfs: initialized
[    0.942490] NET: Registered protocol family 16
[    0.946971] ACPI: bus type pci registered
[    0.950974] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.960248] PCI: not using MMCONFIG
[    0.963730] PCI: Using configuration type 1 for base access
[    0.969631] bio: create slab <bio-0> at 0
[    0.973668] ACPI: Added _OSI(Module Device)
[    0.977841] ACPI: Added _OSI(Processor Device)
[    0.982267] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.986951] ACPI: Added _OSI(Processor Aggregator Device)
[    0.993486] ACPI: Executed 1 blocks of module-level executable AML code
[    1.002505] ACPI: Interpreter enabled
[    1.006154] ACPI: (supports S0 S5)
[    1.009593] ACPI: Using IOAPIC for interrupt routing
[    1.014556] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    1.023868] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[    1.046241] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
[    1.055070] HEST: Table not found.
[    1.058464] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    1.067676] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    1.073965] pci_root PNP0A08:00: host bridge window [io  0x0000-0x0cf7]
[    1.080553] pci_root PNP0A08:00: host bridge window [io  0x0d00-0xffff]
[    1.087145] pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
[    1.094431] pci_root PNP0A08:00: host bridge window [mem 0x000c8000-0x000dffff]
[    1.101715] pci_root PNP0A08:00: host bridge window [mem 0xc0000000-0xffffffff]
[    1.108994] pci_root PNP0A08:00: ignoring host bridge window [mem 0x000c8000-0x000dffff] (conflicts with Video ROM [mem 0x000c0000-0x000ccdff])
[    1.124701] pci 0000:00:1c.0: PCI bridge to [bus 01-01]
[    1.131678] pci 0000:00:1c.4: PCI bridge to [bus 02-02]
[    1.138660] pci 0000:00:1c.6: PCI bridge to [bus 03-03]
[    1.145633] pci 0000:00:1c.7: PCI bridge to [bus 04-0d]
[    1.152241] pci 0000:04:00.0: PCI bridge to [bus 05-0d]
[    1.157843] pci 0000:05:01.0: PCI bridge to [bus 06-06]
[    1.164581] pci 0000:05:04.0: PCI bridge to [bus 07-08]
[    1.170495] pci 0000:07:00.0: PCI bridge to [bus 08-08]
[    1.176116] pci 0000:05:05.0: PCI bridge to [bus 09-09]
[    1.183529] pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
[    1.189113] pci 0000:05:07.0: PCI bridge to [bus 0b-0b]
[    1.194433] pci 0000:05:08.0: PCI bridge to [bus 0c-0c]
[    1.199748] pci 0000:05:09.0: PCI bridge to [bus 0d-0d]
[    1.205480]  pci0000:00: Requesting ACPI _OSC control (0x1d)
[    1.211250]  pci0000:00: ACPI _OSC control (0x1c) granted
[    1.219050] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
[    1.226311] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 *5 6 7 10 11 12 14 15)
[    1.233562] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15)
[    1.240622] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 10 11 12 14 15)
[    1.247683] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    1.255131] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0
[    1.262573] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 *10 11 12 14 15)
[    1.269823] ACPI: PCI Interrupt Link [LNKH] (IRQs *3 4 5 6 7 10 11 12 14 15)
[    1.277093] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=io+mem,locks=none
[    1.285153] vgaarb: loaded
[    1.287854] vgaarb: bridge control possible 0000:01:00.0
[    1.293207] SCSI subsystem initialized
[    1.296983] PCI: Using ACPI for IRQ routing
[    1.311026] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    1.317338] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    1.325154] Switching to clocksource hpet
[    1.329219] pnp: PnP ACPI init
[    1.332272] ACPI: bus type pnp registered
[    1.336442] system 00:01: [mem 0xfed10000-0xfed19fff] has been reserved
[    1.343036] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
[    1.349629] system 00:01: [mem 0xfed90000-0xfed93fff] has been reserved
[    1.356224] system 00:01: [mem 0xfed20000-0xfed3ffff] has been reserved
[    1.362817] system 00:01: [mem 0xfee00000-0xfee0ffff] has been reserved
[    1.369465] system 00:02: [io  0x0290-0x029f] has been reserved
[    1.375755] system 00:07: [io  0x04d0-0x04d1] has been reserved
[    1.382008] system 00:0a: [io  0x0400-0x0453] has been reserved
[    1.387913] system 00:0a: [io  0x0458-0x047f] has been reserved
[    1.393817] system 00:0a: [io  0x1180-0x119f] has been reserved
[    1.399717] system 00:0a: [io  0x0500-0x057f] has been reserved
[    1.405619] system 00:0a: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    1.412213] system 00:0a: [mem 0xfec00000-0xfecfffff] could not be reserved
[    1.419153] system 00:0a: [mem 0xfed08000-0xfed08fff] has been reserved
[    1.425746] system 00:0a: [mem 0xff000000-0xffffffff] has been reserved
[    1.432383] system 00:0b: [io  0x0454-0x0457] has been reserved
[    1.438459] pnp: PnP ACPI: found 13 devices
[    1.442633] ACPI: ACPI bus type pnp unregistered
[    1.453068] pci 0000:00:1c.0: PCI bridge to [bus 01-01]
[    1.458277] pci 0000:00:1c.0:   bridge window [io  0xe000-0xefff]
[    1.464354] pci 0000:00:1c.0:   bridge window [mem 0xf8000000-0xfb0fffff]
[    1.471120] pci 0000:00:1c.0:   bridge window [mem 0xc0000000-0xcfffffff 64bit pref]
[    1.478840] pci 0000:00:1c.4: PCI bridge to [bus 02-02]
[    1.484056] pci 0000:00:1c.4:   bridge window [io  0xd000-0xdfff]
[    1.490133] pci 0000:00:1c.4:   bridge window [mem 0xfb700000-0xfb7fffff]
[    1.496905] pci 0000:00:1c.6: PCI bridge to [bus 03-03]
[    1.502121] pci 0000:00:1c.6:   bridge window [mem 0xfb600000-0xfb6fffff]
[    1.508900] pci 0000:05:01.0: PCI bridge to [bus 06-06]
[    1.514117] pci 0000:05:01.0:   bridge window [mem 0xfb400000-0xfb4fffff]
[    1.520899] pci 0000:07:00.0: PCI bridge to [bus 08-08]
[    1.526109] pci 0000:07:00.0:   bridge window [io  0xc000-0xcfff]
[    1.532189] pci 0000:07:00.0:   bridge window [mem 0xfb300000-0xfb3fffff]
[    1.538963] pci 0000:07:00.0:   bridge window [mem 0xd0000000-0xd3ffffff 64bit pref]
[    1.546693] pci 0000:05:04.0: PCI bridge to [bus 07-08]
[    1.551903] pci 0000:05:04.0:   bridge window [io  0xc000-0xcfff]
[    1.557983] pci 0000:05:04.0:   bridge window [mem 0xfb300000-0xfb3fffff]
[    1.564756] pci 0000:05:04.0:   bridge window [mem 0xd0000000-0xd3ffffff 64bit pref]
[    1.572477] pci 0000:05:05.0: PCI bridge to [bus 09-09]
[    1.577696] pci 0000:05:05.0:   bridge window [mem 0xfb200000-0xfb2fffff]
[    1.584475] pci 0000:05:06.0: PCI bridge to [bus 0a-0a]
[    1.589684] pci 0000:05:06.0:   bridge window [io  0xb000-0xbfff]
[    1.595768] pci 0000:05:06.0:   bridge window [mem 0xd4000000-0xd40fffff 64bit pref]
[    1.603494] pci 0000:05:07.0: PCI bridge to [bus 0b-0b]
[    1.608706] pci 0000:05:07.0:   bridge window [io  0xa000-0xafff]
[    1.614786] pci 0000:05:07.0:   bridge window [mem 0xfb100000-0xfb1fffff]
[    1.621567] pci 0000:05:08.0: PCI bridge to [bus 0c-0c]
[    1.626793] pci 0000:05:09.0: PCI bridge to [bus 0d-0d]
[    1.632022] pci 0000:04:00.0: PCI bridge to [bus 05-0d]
[    1.637234] pci 0000:04:00.0:   bridge window [io  0xa000-0xcfff]
[    1.643312] pci 0000:04:00.0:   bridge window [mem 0xfb100000-0xfb4fffff]
[    1.650085] pci 0000:04:00.0:   bridge window [mem 0xd0000000-0xd40fffff 64bit pref]
[    1.657806] pci 0000:00:1c.7: PCI bridge to [bus 04-0d]
[    1.663020] pci 0000:00:1c.7:   bridge window [io  0xa000-0xcfff]
[    1.669097] pci 0000:00:1c.7:   bridge window [mem 0xfb100000-0xfb5fffff]
[    1.675862] pci 0000:00:1c.7:   bridge window [mem 0xd0000000-0xd40fffff 64bit pref]
[    1.683592] pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.690284] pci 0000:00:1c.4: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.696974] pci 0000:00:1c.6: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    1.703663] pci 0000:00:1c.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19
[    1.710351] pci 0000:04:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.717044] pci 0000:05:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.723732] pci 0000:05:04.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.730420] pci 0000:07:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.737111] pci 0000:05:05.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.743807] pci 0000:05:06.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    1.750493] pci 0000:05:07.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    1.757184] pci 0000:05:08.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    1.763874] pci 0000:05:09.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    1.770651] NET: Registered protocol family 2
[    1.775098] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    1.782782] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[    1.791062] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.797859] TCP: Hash tables configured (established 524288 bind 65536)
[    1.804454] TCP reno registered
[    1.807596] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    1.813694] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    1.820270] NET: Registered protocol family 1
[    2.070372] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    2.076794] Placing 64MB software IO TLB between ffff8800baee2000 - ffff8800beee2000
[    2.084510] software IO TLB at phys 0xbaee2000 - 0xbeee2000
[    2.090867] audit: initializing netlink socket (disabled)
[    2.096258] type=2000 audit(1323706554.485:1): initialized
[    2.120413] Btrfs loaded
[    2.122951] msgmni has been set to 15968
[    2.127217] NET: Registered protocol family 38
[    2.131667] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    2.139040] io scheduler noop registered
[    2.142975] io scheduler cfq registered (default)
[    2.149778] pcieport 0000:00:1c.0: Signaling PME through PCIe PME interrupt
[    2.156724] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    2.163247] pcieport 0000:00:1c.4: Signaling PME through PCIe PME interrupt
[    2.170185] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[    2.176711] pcieport 0000:00:1c.6: Signaling PME through PCIe PME interrupt
[    2.183650] pci 0000:03:00.0: Signaling PME through PCIe PME interrupt
[    2.190177] pcieport 0000:00:1c.7: Signaling PME through PCIe PME interrupt
[    2.197114] pcieport 0000:04:00.0: Signaling PME through PCIe PME interrupt
[    2.204054] pcieport 0000:05:01.0: Signaling PME through PCIe PME interrupt
[    2.210994] pci 0000:06:00.0: Signaling PME through PCIe PME interrupt
[    2.217501] pcieport 0000:05:04.0: Signaling PME through PCIe PME interrupt
[    2.224440] pci 0000:07:00.0: Signaling PME through PCIe PME interrupt
[    2.230947] pci 0000:08:00.0: Signaling PME through PCIe PME interrupt
[    2.237455] pci 0000:08:01.0: Signaling PME through PCIe PME interrupt
[    2.243962] pcieport 0000:05:05.0: Signaling PME through PCIe PME interrupt
[    2.250900] pci 0000:09:00.0: Signaling PME through PCIe PME interrupt
[    2.257409] pcieport 0000:05:06.0: Signaling PME through PCIe PME interrupt
[    2.264348] pci 0000:0a:00.0: Signaling PME through PCIe PME interrupt
[    2.270854] pcieport 0000:05:07.0: Signaling PME through PCIe PME interrupt
[    2.277795] pci 0000:0b:00.0: Signaling PME through PCIe PME interrupt
[    2.284301] pcieport 0000:05:08.0: Signaling PME through PCIe PME interrupt
[    2.291242] pcieport 0000:05:09.0: Signaling PME through PCIe PME interrupt
[    2.298216] vesafb: mode is 1920x1200x32, linelength=7680, pages=0
[    2.304378] vesafb: scrolling: redraw
[    2.308035] vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    2.314192] vesafb: framebuffer at 0xf9000000, mapped to 0xffffc90011800000, using 9024k, total 9024k
[    2.406602] Console: switching to colour frame buffer device 240x75
[    2.492666] fb0: VESA VGA frame buffer device
[    2.497302] input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0
[    2.505891] ACPI: Power Button [PWRB]
[    2.509769] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
[    2.517530] ACPI: Power Button [PWRF]
[    2.521514] ERST: Table is not found!
[    2.525398] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    2.552551] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    2.579481] 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    2.585481] Linux agpgart interface v0.103
[    2.589903] aic7xxx 0000:08:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    2.806691] scsi0 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
[    2.806692]         <Adaptec 2940 Ultra2 SCSI adapter>
[    2.806692]         aic7890/91: Ultra2 Wide Channel A, SCSI Id=7, 32/253 SCBs
[    2.806693] 
[    2.828395] ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
[    2.845968] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode
[    2.854582] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst 
[    2.873121] scsi1 : ahci
[    2.875823] scsi2 : ahci
[    2.878518] scsi3 : ahci
[    2.881213] scsi4 : ahci
[    2.883911] scsi5 : ahci
[    2.886606] scsi6 : ahci
[    2.889348] ata1: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805100 irq 52
[    2.897109] ata2: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805180 irq 52
[    2.904868] ata3: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805200 irq 52
[    2.912629] ata4: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805280 irq 52
[    2.920391] ata5: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805300 irq 52
[    2.928162] ata6: SATA max UDMA/133 abar m2048@0xfb805000 port 0xfb805380 irq 52
[    2.928176] ahci 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    2.928292] ahci 0000:02:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    2.928294] ahci 0000:02:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    2.928475] scsi7 : ahci
[    2.928517] scsi8 : ahci
[    2.928546] ata7: SATA max UDMA/133 abar m2048@0xfb710000 port 0xfb710100 irq 53
[    2.928548] ata8: SATA max UDMA/133 abar m2048@0xfb710000 port 0xfb710180 irq 53
[    2.928563] ahci 0000:0b:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    2.928715] ahci 0000:0b:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    2.928717] ahci 0000:0b:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    2.928973] scsi9 : ahci
[    2.929032] scsi10 : ahci
[    2.929076] ata9: SATA max UDMA/133 abar m2048@0xfb110000 port 0xfb110100 irq 54
[    2.929080] ata10: SATA max UDMA/133 abar m2048@0xfb110000 port 0xfb110180 irq 54
[    2.929172] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    2.931854] serio: i8042 KBD port at 0x60,0x64 irq 1
[    2.931857] serio: i8042 AUX port at 0x60,0x64 irq 12
[    2.931922] mousedev: PS/2 mouse device common for all mice
[    2.931991] rtc_cmos 00:05: RTC can wake from S4
[    2.932121] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0
[    2.932161] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    2.936054] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.07
[    2.936081] iTCO_wdt: Found a Cougar Point TCO device (Version=2, TCOBASE=0x0460)
[    2.936110] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
[    3.088660] scsi 0:0:1:0: Direct-Access     IBM-PCCO ST39102LC     !# B219 PQ: 0 ANSI: 2
[    3.088663] scsi0:A:1:0: Tagged Queuing enabled.  Depth 8
[    3.088666] scsi target0:0:1: Beginning Domain Validation
[    3.095482] scsi target0:0:1: wide asynchronous
[    3.100454] scsi target0:0:1: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 15)
[    3.104744] scsi target0:0:1: Domain Validation skipping write tests
[    3.104745] scsi target0:0:1: Ending Domain Validation
[    3.231842] ata5: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.231858] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    3.231871] ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    3.231883] ata6: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    3.231892] ata4: SATA link down (SStatus 0 SControl 300)
[    3.231908] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    3.232121] ata1.00: ATA-9: C300-CTFDDAC064MAG, 0006, max UDMA/100
[    3.232122] ata1.00: 125045424 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.232339] ata5.00: ATAPI: TSSTcorp CDDVDW SH-222AB, SB00, max UDMA/100
[    3.232361] ata1.00: configured for UDMA/100
[    3.232367] scsi: waiting for bus probes to complete ...
[    3.232840] ata10: SATA link down (SStatus 0 SControl 330)
[    3.232869] ata9: SATA link down (SStatus 0 SControl 330)
[    3.233124] ata5.00: configured for UDMA/100
[    3.233235] ata3.00: ATA-8: Hitachi HDS722020ALA330, JKAOA3MA, max UDMA/133
[    3.233236] ata3.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.233831] ata6.00: ATA-8: SAMSUNG HD403LJ, CT100-10, max UDMA7
[    3.233832] ata6.00: 781422768 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    3.234742] ata3.00: configured for UDMA/133
[    3.235209] ata2.00: ATA-6: WDC WD360GD-00FLA2, 31.08F31, max UDMA/133
[    3.235210] ata2.00: 72303840 sectors, multi 16: LBA48 
[    3.235790] ata6.00: configured for UDMA/133
[    3.239581] ata2.00: configured for UDMA/133
[    3.863825] Refined TSC clocksource calibration: 3392.293 MHz.
[    3.883785] Switching to clocksource tsc
[    3.883808] device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com
[    3.924596] cpuidle: using governor ladder
[    3.942713] cpuidle: using governor menu
[    3.960537] TCP cubic registered
[    3.977230] Initializing XFRM netlink socket
[    3.995231] NET: Registered protocol family 10
[    4.013725] IPv6 over IPv4 tunneling driver
[    4.031740] NET: Registered protocol family 17
[    4.049685] NET: Registered protocol family 15
[    4.067631] Registering the dns_resolver key type
[    4.067692] registered taskstats version 1
[    4.074089] scsi: waiting for bus probes to complete ...
[    4.127317] ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 330)
[    4.148438] ata7.00: ATAPI: LITE-ON LTR-48246S, SS0E, max UDMA/33
[    4.169640] ata7.00: configured for UDMA/33
[    4.269892] ata8: SATA link up 3.0 Gbps (SStatus 123 SControl 330)
[    4.317621] ata8.00: ATA-8: WDC WD15EARS-00Z5B1, 80.00A80, max UDMA/133
[    4.338454] ata8.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    4.364135] ata8.00: configured for UDMA/133
[    6.450325] sd 0:0:1:0: Attached scsi generic sg0 type 0
[    6.470550] rtc_cmos 00:05: setting system clock to 2011-12-12 16:16:00 UTC (1323706560)
[    6.470616] scsi 1:0:0:0: Direct-Access     ATA      C300-CTFDDAC064M 0006 PQ: 0 ANSI: 5
[    6.470735] sd 1:0:0:0: Attached scsi generic sg1 type 0
[    6.470786] sd 1:0:0:0: [sdb] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[    6.470819] sd 0:0:1:0: [sda] 17774160 512-byte logical blocks: (9.10 GB/8.47 GiB)
[    6.470821] sd 1:0:0:0: [sdb] Write Protect is off
[    6.470829] scsi 2:0:0:0: Direct-Access     ATA      WDC WD360GD-00FL 31.0 PQ: 0 ANSI: 5
[    6.470838] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    6.470923] sd 2:0:0:0: Attached scsi generic sg2 type 0
[    6.470933] sd 2:0:0:0: [sdc] 72303840 512-byte logical blocks: (37.0 GB/34.4 GiB)
[    6.470966] sd 2:0:0:0: [sdc] Write Protect is off
[    6.470984] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    6.471041] scsi 3:0:0:0: Direct-Access     ATA      Hitachi HDS72202 JKAO PQ: 0 ANSI: 5
[    6.471171] sd 3:0:0:0: Attached scsi generic sg3 type 0
[    6.471182] sd 3:0:0:0: [sdd] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    6.471276] sd 3:0:0:0: [sdd] Write Protect is off
[    6.471308] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    6.471757] scsi 5:0:0:0: CD-ROM            TSSTcorp CDDVDW SH-222AB  SB00 PQ: 0 ANSI: 5
[    6.471886]  sdb: sdb1
[    6.472817] sd 0:0:1:0: [sda] Write Protect is off
[    6.474148] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
[    6.474149] cdrom: Uniform CD-ROM driver Revision: 3.20
[    6.474246] sr 5:0:0:0: Attached scsi generic sg4 type 5
[    6.474333] scsi 6:0:0:0: Direct-Access     ATA      SAMSUNG HD403LJ  CT10 PQ: 0 ANSI: 5
[    6.474394] sd 6:0:0:0: [sde] 781422768 512-byte logical blocks: (400 GB/372 GiB)
[    6.474396] sd 6:0:0:0: Attached scsi generic sg5 type 0
[    6.474403] sd 0:0:1:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[    6.474433] sd 6:0:0:0: [sde] Write Protect is off
[    6.474446] sd 6:0:0:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    6.487776]  sda: sda1 sda2
[    6.518245]  sdd: sdd1
[    6.519296]  sdc: sdc1 sdc2
[    6.523418]  sde: sde1
[    6.671006] scsi 7:0:0:0: CD-ROM            LITE-ON  LTR-48246S       SS0E PQ: 0 ANSI: 5
[    7.149013] sr1: scsi3-mmc drive: 50x/48x writer cd/rw xa/form2 cdda tray
[    7.262652] sd 1:0:0:0: [sdb] Attached SCSI disk
[    7.262676] sr 7:0:0:0: Attached scsi generic sg6 type 5
[    7.262678] sd 2:0:0:0: [sdc] Attached SCSI disk
[    7.262706] sd 3:0:0:0: [sdd] Attached SCSI disk
[    7.262707] sd 6:0:0:0: [sde] Attached SCSI disk
[    7.262730] scsi 8:0:0:0: Direct-Access     ATA      WDC WD15EARS-00Z 80.0 PQ: 0 ANSI: 5
[    7.262780] sd 8:0:0:0: [sdf] 2930277168 512-byte logical blocks: (1.50 TB/1.36 TiB)
[    7.262791] sd 8:0:0:0: Attached scsi generic sg7 type 0
[    7.262808] sd 8:0:0:0: [sdf] Write Protect is off
[    7.262819] sd 8:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    7.493477] sd 0:0:1:0: [sda] Attached SCSI disk
[    7.522216]  sdf: sdf1
[    7.541237] sd 8:0:0:0: [sdf] Attached SCSI disk
[    7.577353] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[    7.601524] VFS: Mounted root (ext4 filesystem) readonly on device 8:17.
[    7.625428] devtmpfs: mounted
[    7.644587] Freeing unused kernel memory: 672k freed
[    7.785833] udevd[154]: starting version 175
[    7.845758] usbcore: registered new interface driver usbfs
[    7.845766] usbcore: registered new interface driver hub
[    7.846050] wmi: Mapper loaded
[    7.846198] usbcore: registered new device driver usb
[    7.847013] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    7.847029] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    7.847052] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[    7.847057] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
[    7.847078] ehci_hcd 0000:00:1a.0: debug port 2
[    7.850785] xhci_hcd 0000:03:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
[    7.850809] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    7.850813] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 2
[    7.850950] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xfb807000
[    7.850958] xhci_hcd 0000:03:00.0: irq 18, io mem 0xfb600000
[    7.850983] xhci_hcd 0000:03:00.0: Failed to enable MSI-X
[    7.851182] hub 2-0:1.0: USB hub found
[    7.851189] hub 2-0:1.0: 2 ports detected
[    7.851227] xhci_hcd 0000:03:00.0: xHCI Host Controller
[    7.851230] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 3
[    7.851290] hub 3-0:1.0: USB hub found
[    7.851296] hub 3-0:1.0: 2 ports detected
[    7.851334] xhci_hcd 0000:06:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    7.851351] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    7.851355] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 4
[    7.851519] xhci_hcd 0000:06:00.0: irq 16, io mem 0xfb400000
[    7.851543] xhci_hcd 0000:06:00.0: Failed to enable MSI-X
[    7.851766] hub 4-0:1.0: USB hub found
[    7.851773] hub 4-0:1.0: 2 ports detected
[    7.851807] xhci_hcd 0000:06:00.0: xHCI Host Controller
[    7.851809] xhci_hcd 0000:06:00.0: new USB bus registered, assigned bus number 5
[    7.851874] hub 5-0:1.0: USB hub found
[    7.851882] hub 5-0:1.0: 2 ports detected
[    7.851929] xhci_hcd 0000:09:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    7.851947] xhci_hcd 0000:09:00.0: xHCI Host Controller
[    7.851950] xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 6
[    7.852114] xhci_hcd 0000:09:00.0: irq 16, io mem 0xfb200000
[    7.852136] xhci_hcd 0000:09:00.0: Failed to enable MSI-X
[    7.852338] hub 6-0:1.0: USB hub found
[    7.852345] hub 6-0:1.0: 2 ports detected
[    7.852381] xhci_hcd 0000:09:00.0: xHCI Host Controller
[    7.852383] xhci_hcd 0000:09:00.0: new USB bus registered, assigned bus number 7
[    7.852450] hub 7-0:1.0: USB hub found
[    7.852457] hub 7-0:1.0: 2 ports detected
[    7.854577] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    7.854591] r8169 0000:0a:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[    7.860623] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    7.860794] hub 1-0:1.0: USB hub found
[    7.860797] hub 1-0:1.0: 2 ports detected
[    7.860844] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[    8.780620] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[    8.780636] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 8
[    8.780673] ehci_hcd 0000:00:1d.0: debug port 2
[    8.780842] r8169 0000:0a:00.0: eth0: RTL8168e/8111e at 0xffffc90000c7c000, 00:25:22:9f:75:32, XID 0c200000 IRQ 58
[    8.780844] r8169 0000:0a:00.0: eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]
[    8.781150] FDC 0 is a post-1991 82077
[    8.784578] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xfb806000
[    8.793916] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    8.794027] hub 8-0:1.0: USB hub found
[    8.794030] hub 8-0:1.0: 2 ports detected
[    8.794071] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[    8.794183] snd_hda_intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
[    9.002324] usb 1-1: new high-speed USB device number 2 using ehci_hcd
[    9.158894] hub 1-1:1.0: USB hub found
[    9.159127] hub 1-1:1.0: 6 ports detected
[    9.261605] usb 8-1: new high-speed USB device number 2 using ehci_hcd
[    9.396840] hub 8-1:1.0: USB hub found
[    9.416019] hub 8-1:1.0: 8 ports detected
[    9.708458] usb 8-1.1: new low-speed USB device number 3 using ehci_hcd
[    9.874980] usb 8-1.2: new high-speed USB device number 4 using ehci_hcd
[    9.961398] Adding 4194300k swap on /dev/sdc1.  Priority:-1 extents:1 across:4194300k 
[    9.989762] hub 8-1.2:1.0: USB hub found
[    9.989893] hub 8-1.2:1.0: 3 ports detected
[   10.040985] EXT4-fs (sdb1): re-mounted. Opts: (null)
[   10.063474] usb 8-1.7: new high-speed USB device number 5 using ehci_hcd
[   10.152833] EXT4-fs (sdb1): re-mounted. Opts: discard
[   10.250898] usb 8-1.8: new low-speed USB device number 6 using ehci_hcd
[   10.374521] input: Microsoft Microsoft IntelliMouse® Explorer as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.1/8-1.1:1.0/input/input2
[   10.404124] generic-usb 0003:045E:001E.0001: input,hidraw0: USB HID v1.00 Mouse [Microsoft Microsoft IntelliMouse® Explorer] on usb-0000:00:1d.0-1.1/input0
[   10.438346] input: USBPS2 as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.8/8-1.8:1.0/input/input3
[   10.465450] generic-usb 0003:0D3D:0001.0002: input,hidraw1: USB HID v1.00 Keyboard [USBPS2] on usb-0000:00:1d.0-1.8/input0
[   10.494849] usb 8-1.2.2: new low-speed USB device number 7 using ehci_hcd
[   10.498767] input: USBPS2 as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.8/8-1.8:1.1/input/input4
[   10.498848] generic-usb 0003:0D3D:0001.0003: input,hidraw2: USB HID v1.00 Mouse [USBPS2] on usb-0000:00:1d.0-1.8/input1
[   10.498874] usbcore: registered new interface driver usbhid
[   10.498875] usbhid: USB HID core driver
[   10.587522] input: Apple, Inc Apple Keyboard as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.2/8-1.2.2/8-1.2.2:1.0/input/input5
[   10.587601] apple 0003:05AC:0220.0004: input,hidraw3: USB HID v1.11 Keyboard [Apple, Inc Apple Keyboard] on usb-0000:00:1d.0-1.2.2/input0
[   10.590424] input: Apple, Inc Apple Keyboard as /devices/pci0000:00/0000:00:1d.0/usb8/8-1/8-1.2/8-1.2.2/8-1.2.2:1.1/input/input6
[   10.590494] apple 0003:05AC:0220.0005: input,hidraw4: USB HID v1.11 Device [Apple, Inc Apple Keyboard] on usb-0000:00:1d.0-1.2.2/input1
[   10.827130] w83627ehf: Found NCT6776F chip at 0x290
[   10.830908] fuse init (API version 7.17)
[   10.838360] i2c /dev entries driver
[   10.841633] uhci_hcd: USB Universal Host Controller Interface driver
[   10.843321] ipmi message handler version 39.2
[   11.801833] EXT4-fs (sdc2): mounted filesystem with ordered data mode. Opts: (null)
[   11.884535] EXT4-fs (sdd1): mounted filesystem with ordered data mode. Opts: (null)
[   11.923124] REISERFS (device sda1): found reiserfs format "3.6" with standard journal
[   11.948425] REISERFS (device sda1): using ordered data mode
[   11.971028] reiserfs: using flush barriers
[   11.996849] REISERFS (device sda1): journal params: device sda1, size 8192, journal first block 18, max trans len 1024, max batch 900, max commit age 30, max trans age 30
[   12.029721] REISERFS (device sda1): checking transaction log (sda1)
[   12.086397] REISERFS (device sda1): Using r5 hash to sort names
[   12.176635] EXT4-fs (sdf1): mounted filesystem with ordered data mode. Opts: (null)
[   12.387833] r8169 0000:0a:00.0: eth0: unable to load firmware patch rtl_nic/rtl8168e-2.fw (-2)
[   12.421246] r8169 0000:0a:00.0: eth0: link down
[   12.421582] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   14.802259] r8169 0000:0a:00.0: eth0: link up
[   14.820869] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   17.168658] RPC: Registered named UNIX socket transport module.
[   17.168659] RPC: Registered udp transport module.
[   17.168660] RPC: Registered tcp transport module.
[   17.168661] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   17.258509] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[   17.344335] tun: Universal TUN/TAP device driver, 1.6
[   17.365785] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[   17.366927] ADDRCONF(NETDEV_UP): tap0: link is not ready
[   17.391764] device eth0 entered promiscuous mode
[   17.392979] device tap0 entered promiscuous mode
[   17.393708] br0: port 1(eth0) entering forwarding state
[   17.393711] br0: port 1(eth0) entering forwarding state
[   23.531880] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   23.532342] NFSD: starting 90-second grace period
[   23.723041] [drm] Initialized drm 1.1.0 20060810
[   23.732738] MXM: GUID detected in BIOS
[   23.736495] nouveau 0000:01:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[   23.744123] [drm] nouveau 0000:01:00.0: Detected an NV50 generation card (0x084200a2)
[   23.751930] fb: conflicting fb hw usage nouveaufb vs VESA VGA - removing generic driver
[   23.759941] Console: switching to colour dummy device 80x25
[   23.767661] [drm] nouveau 0000:01:00.0: Attempting to load BIOS image from PRAMIN
[   23.880210] [drm] nouveau 0000:01:00.0: ... appears to be valid
[   23.886112] [drm] nouveau 0000:01:00.0: BIT BIOS found
[   23.891237] [drm] nouveau 0000:01:00.0: Bios version 60.84.51.00
[   23.897226] [drm] nouveau 0000:01:00.0: TMDS table version 2.0
[   23.903041] [drm] nouveau 0000:01:00.0: Found Display Configuration Block version 4.0
[   23.910845] [drm] nouveau 0000:01:00.0: Raw DCB entry 0: 02000300 00000028
[   23.917698] [drm] nouveau 0000:01:00.0: Raw DCB entry 1: 01000302 00020030
[   23.924552] [drm] nouveau 0000:01:00.0: Raw DCB entry 2: 04011310 00000028
[   23.931405] [drm] nouveau 0000:01:00.0: Raw DCB entry 3: 02011312 00020030
[   23.938258] [drm] nouveau 0000:01:00.0: Raw DCB entry 4: 010223f1 00c0c080
[   23.945112] [drm] nouveau 0000:01:00.0: DCB connector table: VHER 0x40 5 14 2
[   23.952223] [drm] nouveau 0000:01:00.0:   0: 0x00002030: type 0x30 idx 0 tag 0x08
[   23.959688] [drm] nouveau 0000:01:00.0:   1: 0x00001130: type 0x30 idx 1 tag 0x07
[   23.967141] [drm] nouveau 0000:01:00.0:   2: 0x00000210: type 0x10 idx 2 tag 0xff
[   23.974597] [drm] nouveau 0000:01:00.0:   3: 0x00000211: type 0x11 idx 3 tag 0xff
[   23.982056] [drm] nouveau 0000:01:00.0:   4: 0x00000213: type 0x13 idx 4 tag 0xff
[   23.989517] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 0 at offset 0xC1BE
[   24.020376] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 1 at offset 0xC50B
[   24.044840] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 2 at offset 0xCF58
[   24.052481] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 3 at offset 0xD04A
[   24.061201] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table 4 at offset 0xD27D
[   24.068831] [drm] nouveau 0000:01:00.0: Parsing VBIOS init table at offset 0xD2E2
[   24.096235] [drm] nouveau 0000:01:00.0: 0xD2E2: Condition still not met after 20ms, skipping following opcodes
[   24.125550] [drm] nouveau 0000:01:00.0: 1 available performance level(s)
[   24.132228] [drm] nouveau 0000:01:00.0: 0: core 580MHz shader 1508MHz memory 800MHz fanspeed 100%
[   24.141103] [drm] nouveau 0000:01:00.0: c: core 540MHz shader 1512MHz memory 702MHz voltage 1320mV
[   24.151394] [TTM] Zone  kernel: Available graphics memory: 4088300 kiB.
[   24.157986] [TTM] Zone   dma32: Available graphics memory: 2097152 kiB.
[   24.164582] [TTM] Initializing pool allocator.
[   24.169022] [drm] nouveau 0000:01:00.0: Detected 256MiB VRAM
[   24.174668] mtrr: type mismatch for c0000000,10000000 old: write-back new: write-combining
[   24.182938] [drm] nouveau 0000:01:00.0: 512 MiB GART (aperture)
[   24.199293] [drm] nouveau 0000:01:00.0: DCB encoder 1 unknown
[   24.205026] [drm] nouveau 0000:01:00.0: TV-1 has no encoders, removing
[   24.226756] [drm] Supports vblank timestamp caching Rev 1 (10.10.2010).
[   24.233364] [drm] No driver support for vblank timestamp query.
[   24.239267] parse error at position 4 in video mode 'efifb'
[   24.244829] parse error at position 4 in video mode 'efifb'
[   24.469199] [drm] nouveau 0000:01:00.0: allocated 1920x1200 fb: 0x320000, bo ffff88023535a400
[   24.477808] fbcon: nouveaufb (fb0) is primary device
[   24.482814] Console: switching to colour frame buffer device 160x64
[   24.489058] fb0: nouveaufb frame buffer device
[   24.493490] drm: registered panic notifier
[   24.497580] [drm] Initialized nouveau 0.0.16 20090420 for 0000:01:00.0 on minor 0

Debian GNU/Linux wheezy/sid luna ttyS0

luna login: [   32.363396] br0: port 1(eth0) entering forwarding state
[   37.019052] EXT4-fs (sdb1): re-mounted. Opts: discard,commit=0
[   37.026396] EXT4-fs (sdc2): re-mounted. Opts: commit=0
[   37.032583] EXT4-fs (sdd1): re-mounted. Opts: commit=0
[   37.038728] EXT4-fs (sdf1): re-mounted. Opts: commit=0

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-12  3:40         ` Yinghai Lu
  2011-12-12 23:21           ` Berck E. Nash
@ 2011-12-12 23:42           ` H. Peter Anvin
  2011-12-13  1:25             ` Yinghai Lu
  1 sibling, 1 reply; 26+ messages in thread
From: H. Peter Anvin @ 2011-12-12 23:42 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, Suresh Siddha

On 12/11/2011 07:40 PM, Yinghai Lu wrote:
> -v2: use x2apic_disabled instead of nox2apic, Suggested by Thomas
>      update x2apic_supported with x2apic_disabled, Suggested by Thomas

This is ugly.  Typically we have no- for boolean options.  If we have a
multivalue option it should be option=value, i.e. x2apic=disabled.  We
can allow x2apic_supported as a backwards compatibility option.

	-hpa


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-12 23:21           ` Berck E. Nash
@ 2011-12-13  0:25             ` Suresh Siddha
  2011-12-13  1:29               ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Suresh Siddha @ 2011-12-13  0:25 UTC (permalink / raw)
  To: Berck E. Nash; +Cc: Yinghai Lu, Ingo Molnar, linux-kernel, H. Peter Anvin

On Mon, 2011-12-12 at 15:21 -0800, Berck E. Nash wrote:
> On 12/11/2011 08:40 PM, Yinghai Lu wrote:
> > x2apic pre-enabled skip the mapping for lapic address. so need map it
> > after fall back.
> > 
> > please check attached -v5..
> 
> It appears to work correctly as far as I can tell.

Ok. In this case, it looks like bios didn't enable interrupt-remapping
(part of vt-d) and just enabled x2apic with out touching or exporting
vt-d HW.

Yinghai, In other scenarios where OS can't recover, mostly likely we
will fail in check_timer() when we fall back to xapic. Can we add a
printk in this case to convey the potential reason (that I mentioned in
the previous e-mail), so that the user can get more accurate information
for the check timer failure.

thanks,
suresh


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-12 23:42           ` H. Peter Anvin
@ 2011-12-13  1:25             ` Yinghai Lu
  0 siblings, 0 replies; 26+ messages in thread
From: Yinghai Lu @ 2011-12-13  1:25 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, Suresh Siddha

On Mon, Dec 12, 2011 at 3:42 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 12/11/2011 07:40 PM, Yinghai Lu wrote:
>> -v2: use x2apic_disabled instead of nox2apic, Suggested by Thomas
>>      update x2apic_supported with x2apic_disabled, Suggested by Thomas
>
> This is ugly.  Typically we have no- for boolean options.  If we have a
> multivalue option it should be option=value, i.e. x2apic=disabled.  We
> can allow x2apic_supported as a backwards compatibility option.

x2apic_disabled is variable.

for command line, still use nox2apic for that purpose.

Thanks

Yinghai

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-13  0:25             ` Suresh Siddha
@ 2011-12-13  1:29               ` Yinghai Lu
  2011-12-13  1:41                 ` Suresh Siddha
  0 siblings, 1 reply; 26+ messages in thread
From: Yinghai Lu @ 2011-12-13  1:29 UTC (permalink / raw)
  To: Suresh Siddha; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, H. Peter Anvin

On Mon, Dec 12, 2011 at 4:25 PM, Suresh Siddha
<suresh.b.siddha@intel.com> wrote:
> On Mon, 2011-12-12 at 15:21 -0800, Berck E. Nash wrote:
>> On 12/11/2011 08:40 PM, Yinghai Lu wrote:
>> > x2apic pre-enabled skip the mapping for lapic address. so need map it
>> > after fall back.
>> >
>> > please check attached -v5..
>>
>> It appears to work correctly as far as I can tell.
>
> Ok. In this case, it looks like bios didn't enable interrupt-remapping
> (part of vt-d) and just enabled x2apic with out touching or exporting
> vt-d HW.
>
> Yinghai, In other scenarios where OS can't recover, mostly likely we
> will fail in check_timer() when we fall back to xapic. Can we add a
> printk in this case to convey the potential reason (that I mentioned in
> the previous e-mail), so that the user can get more accurate information
> for the check timer failure.

agree. in that case, will have problem.

but i don't think bios buy can go that far to enable intr-remapping
without providing DMAR.

will check if can add print out in check_timer() when
x2apic_preenabled == 1 and x2apic == 0...

Thanks

Yinghai Lu

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-13  1:29               ` Yinghai Lu
@ 2011-12-13  1:41                 ` Suresh Siddha
  2011-12-13  4:32                   ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Suresh Siddha @ 2011-12-13  1:41 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, H. Peter Anvin

On Mon, 2011-12-12 at 17:29 -0800, Yinghai Lu wrote:
> but i don't think bios buy can go that far to enable intr-remapping
> without providing DMAR.

We never know. And also It is not just bios. kexec'd kernel with out the
interrupt-remapping support will also run into this issue and we should
handle this also gracefully.

thanks,
suresh


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-13  1:41                 ` Suresh Siddha
@ 2011-12-13  4:32                   ` Yinghai Lu
  2011-12-13  5:13                     ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Yinghai Lu @ 2011-12-13  4:32 UTC (permalink / raw)
  To: Suresh Siddha; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, H. Peter Anvin

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

On Mon, Dec 12, 2011 at 5:41 PM, Suresh Siddha
<suresh.b.siddha@intel.com> wrote:
> On Mon, 2011-12-12 at 17:29 -0800, Yinghai Lu wrote:
>> but i don't think bios buy can go that far to enable intr-remapping
>> without providing DMAR.
>
> We never know. And also It is not just bios. kexec'd kernel with out the
> interrupt-remapping support will also run into this issue and we should
> handle this also gracefully.

please check attached -v6 with print out as you requested.

Thanks

Yinghai

[-- Attachment #2: nox2apic_v6.patch --]
[-- Type: text/x-patch, Size: 11313 bytes --]

Subject: [PATCH -v6] x86: Disable x2apic if nox2apic is specified or intr-remap can not be enabled

For
1. x2apic preenabled system
2. first kernel have x2apic enabled, and try to kexec second kernel with "nox2apic"

Will put back cpu with apic id < 255 into xapic mode, instead of panic.

-v2: use variable x2apic_disabled instead of variable nox2apic, Suggested by Thomas
     update x2apic_supported with x2apic_disabled, Suggested by Thomas

-v3: add checking for boot cpu apic id > 255. in that case will just panic
     --- pointed out by Suresh.

-v4: according to Ingo, for x2apic pre-enabled system, if intr-remap can not
	be enabled, try to disable x2apic instead of panic and request to
	specify nox2apic for next boot.

-v5: fix the lapic mapping for falling path.

-v6: add warning when check_timer fails in strange case when BIOS enable x2apic
     and intr_remapping but does not provide dmar table. requested by Suresh.

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/apic.h         |    6 +-
 arch/x86/include/asm/apicdef.h      |    1 
 arch/x86/include/asm/processor.h    |    1 
 arch/x86/kernel/acpi/boot.c         |   10 +++-
 arch/x86/kernel/apic/apic.c         |   76 ++++++++++++++++++++++++++----------
 arch/x86/kernel/apic/apic_flat_64.c |    7 ++-
 arch/x86/kernel/apic/io_apic.c      |    4 +
 arch/x86/kernel/cpu/topology.c      |   21 +++++++++
 arch/x86/mm/srat.c                  |    7 ++-
 9 files changed, 107 insertions(+), 26 deletions(-)

Index: linux-2.6/arch/x86/include/asm/apic.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apic.h
+++ linux-2.6/arch/x86/include/asm/apic.h
@@ -176,6 +176,7 @@ static inline u64 native_x2apic_icr_read
 }
 
 extern int x2apic_phys;
+extern int x2apic_disabled;
 extern void check_x2apic(void);
 extern void enable_x2apic(void);
 extern void x2apic_icr_write(u32 low, u32 id);
@@ -183,7 +184,7 @@ static inline int x2apic_enabled(void)
 {
 	u64 msr;
 
-	if (!cpu_has_x2apic)
+	if (!cpu_has_x2apic || x2apic_disabled)
 		return 0;
 
 	rdmsrl(MSR_IA32_APICBASE, msr);
@@ -192,7 +193,7 @@ static inline int x2apic_enabled(void)
 	return 0;
 }
 
-#define x2apic_supported()	(cpu_has_x2apic)
+#define x2apic_supported()	(cpu_has_x2apic && !x2apic_disabled)
 static inline void x2apic_force_phys(void)
 {
 	x2apic_phys = 1;
@@ -214,6 +215,7 @@ static inline void x2apic_force_phys(voi
 
 #define	x2apic_preenabled 0
 #define	x2apic_supported()	0
+#define	x2apic_disabled	1
 #endif
 
 extern void enable_IR_x2apic(void);
Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -219,6 +219,8 @@ static int __init
 acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
 {
 	struct acpi_madt_local_x2apic *processor = NULL;
+	int apic_id;
+	u8 enabled;
 
 	processor = (struct acpi_madt_local_x2apic *)header;
 
@@ -227,6 +229,8 @@ acpi_parse_x2apic(struct acpi_subtable_h
 
 	acpi_table_print_madt_entry(header);
 
+	apic_id = processor->local_apic_id;
+	enabled = processor->lapic_flags & ACPI_MADT_ENABLED;
 #ifdef CONFIG_X86_X2APIC
 	/*
 	 * We need to register disabled CPU as well to permit
@@ -235,8 +239,10 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	 * to not preallocating memory for all NR_CPUS
 	 * when we use CPU hotplug.
 	 */
-	acpi_register_lapic(processor->local_apic_id,	/* APIC ID */
-			    processor->lapic_flags & ACPI_MADT_ENABLED);
+	if (x2apic_disabled && (apic_id >= 0xff) && enabled)
+		printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
+	else
+		acpi_register_lapic(apic_id, enabled);
 #else
 	printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
 #endif
Index: linux-2.6/arch/x86/kernel/apic/apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic.c
+++ linux-2.6/arch/x86/kernel/apic/apic.c
@@ -147,15 +147,14 @@ int x2apic_mode;
 #ifdef CONFIG_X86_X2APIC
 /* x2apic enabled before OS handover */
 static int x2apic_preenabled;
+int x2apic_disabled;
 static __init int setup_nox2apic(char *str)
 {
-	if (x2apic_enabled()) {
-		pr_warning("Bios already enabled x2apic, "
-			   "can't enforce nox2apic");
-		return 0;
-	}
+	if (x2apic_enabled())
+		pr_warning("Bios already enabled x2apic, will disable it");
+
+	x2apic_disabled = 1;
 
-	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
 	return 0;
 }
 early_param("nox2apic", setup_nox2apic);
@@ -1431,8 +1430,38 @@ void __init bsp_end_local_APIC_setup(voi
 }
 
 #ifdef CONFIG_X86_X2APIC
+
+static void disable_x2apic(void)
+{
+	int msr, msr2;
+
+	if (!cpu_has_x2apic)
+		return;
+
+	rdmsr(MSR_IA32_APICBASE, msr, msr2);
+	if (msr & X2APIC_ENABLE) {
+		u32 x2apic_id = x2apic_cpuid_initial_apicid();
+
+		if (x2apic_id > 255)
+			panic("Can not disable x2apic, id: %08x\n", x2apic_id);
+
+		pr_info("Disabling x2apic\n");
+		/*
+		 * Need to disable xapic and x2apic at the same time at first
+		 *  then enable xapic
+		 */
+		wrmsr(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE),
+			 0);
+		wrmsr(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE, 0);
+	}
+}
 void check_x2apic(void)
 {
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (x2apic_enabled()) {
 		pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
 		x2apic_preenabled = x2apic_mode = 1;
@@ -1443,6 +1472,11 @@ void enable_x2apic(void)
 {
 	int msr, msr2;
 
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (!x2apic_mode)
 		return;
 
@@ -1493,7 +1527,7 @@ void __init enable_IR_x2apic(void)
 	ret = save_ioapic_entries();
 	if (ret) {
 		pr_info("Saving IO-APIC state failed: %d\n", ret);
-		goto out;
+		return;
 	}
 
 	local_irq_save(flags);
@@ -1505,13 +1539,24 @@ void __init enable_IR_x2apic(void)
 	else
 		ret = enable_IR();
 
+	if (x2apic_disabled)
+		goto nox2apic;
+
 	if (ret < 0) {
 		/* IR is required if there is APIC ID > 255 even when running
 		 * under KVM
 		 */
 		if (max_physical_apicid > 255 ||
-		    !hypervisor_x2apic_available())
+		    !hypervisor_x2apic_available()) {
+			if (x2apic_preenabled) {
+				disable_x2apic();
+				x2apic_disabled = 1;
+				x2apic_mode = 0;
+				/* need to map lapic address */
+				register_lapic_address(mp_lapic_addr);
+			}
 			goto nox2apic;
+		}
 		/*
 		 * without IR all CPUs can be addressed by IOAPIC/MSI
 		 * only in physical mode
@@ -1519,8 +1564,10 @@ void __init enable_IR_x2apic(void)
 		x2apic_force_phys();
 	}
 
-	if (ret == IRQ_REMAP_XAPIC_MODE)
+	if (ret == IRQ_REMAP_XAPIC_MODE) {
+		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
 		goto nox2apic;
+	}
 
 	x2apic_enabled = 1;
 
@@ -1535,17 +1582,6 @@ nox2apic:
 		restore_ioapic_entries();
 	legacy_pic->restore_mask();
 	local_irq_restore(flags);
-
-out:
-	if (x2apic_enabled || !x2apic_supported())
-		return;
-
-	if (x2apic_preenabled)
-		panic("x2apic: enabled by BIOS but kernel init failed.");
-	else if (ret == IRQ_REMAP_XAPIC_MODE)
-		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
-	else if (ret < 0)
-		pr_info("x2apic not enabled, IRQ remapping init failed\n");
 }
 
 #ifdef CONFIG_X86_64
Index: linux-2.6/arch/x86/mm/srat.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat.c
+++ linux-2.6/arch/x86/mm/srat.c
@@ -69,6 +69,12 @@ acpi_numa_x2apic_affinity_init(struct ac
 	if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0)
 		return;
 	pxm = pa->proximity_domain;
+	apic_id = pa->apic_id;
+	if (x2apic_disabled && (apic_id >= 0xff)) {
+		printk(KERN_INFO "SRAT: PXM %u -> X2APIC 0x%04x ignored\n",
+			 pxm, apic_id);
+		return;
+	}
 	node = setup_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
@@ -76,7 +82,6 @@ acpi_numa_x2apic_affinity_init(struct ac
 		return;
 	}
 
-	apic_id = pa->apic_id;
 	if (apic_id >= MAX_LOCAL_APIC) {
 		printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u skipped apicid that is too big\n", pxm, apic_id, node);
 		return;
Index: linux-2.6/arch/x86/include/asm/apicdef.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apicdef.h
+++ linux-2.6/arch/x86/include/asm/apicdef.h
@@ -144,6 +144,7 @@
 
 #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
 #define APIC_BASE_MSR	0x800
+#define XAPIC_ENABLE	(1UL << 11)
 #define X2APIC_ENABLE	(1UL << 10)
 
 #ifdef CONFIG_X86_32
Index: linux-2.6/arch/x86/include/asm/processor.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/processor.h
+++ linux-2.6/arch/x86/include/asm/processor.h
@@ -168,6 +168,7 @@ extern void init_scattered_cpuid_feature
 extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c);
 extern unsigned short num_cache_leaves;
 
+u32 x2apic_cpuid_initial_apicid(void);
 extern void detect_extended_topology(struct cpuinfo_x86 *c);
 extern void detect_ht(struct cpuinfo_x86 *c);
 
Index: linux-2.6/arch/x86/kernel/cpu/topology.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/topology.c
+++ linux-2.6/arch/x86/kernel/cpu/topology.c
@@ -21,6 +21,27 @@
 #define BITS_SHIFT_NEXT_LEVEL(eax)	((eax) & 0x1f)
 #define LEVEL_MAX_SIBLINGS(ebx)		((ebx) & 0xffff)
 
+u32 x2apic_cpuid_initial_apicid(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	if (boot_cpu_data.cpuid_level < 0xb)
+		return 0;
+
+	cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+	/*
+	 * check if the cpuid leaf 0xb is actually implemented.
+	 */
+	if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+		return 0;
+
+	/*
+	 * initial apic id, which also represents 32-bit extended x2apic id.
+	 */
+	return edx;
+}
+
 /*
  * Check for extended topology enumeration cpuid leaf 0xb and if it
  * exists, use it for populating initial_apicid and cpu topology
Index: linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic_flat_64.c
+++ linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
@@ -171,9 +171,14 @@ static int flat_phys_pkg_id(int initial_
 	return initial_apic_id >> index_msb;
 }
 
+static int flat_probe(void)
+{
+	return 1;
+}
+
 static struct apic apic_flat =  {
 	.name				= "flat",
-	.probe				= NULL,
+	.probe				= flat_probe,
 	.acpi_madt_oem_check		= flat_acpi_madt_oem_check,
 	.apic_id_registered		= flat_apic_id_registered,
 
Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/io_apic.c
+++ linux-2.6/arch/x86/kernel/apic/io_apic.c
@@ -2947,6 +2947,10 @@ static inline void __init check_timer(vo
 	}
 	local_irq_disable();
 	apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n");
+	if (x2apic_preenabled && !x2apic_mode)
+		pr_info("Your system has x2apic pre-enabled, but kernel can not enable intr-remapping.\n"
+			"So kernel try to disable x2apic, but interrupt still have problem.\n"
+			"You could disable x2apic from BIOS setup\n");
 	panic("IO-APIC + timer doesn't work!  Boot with apic=debug and send a "
 		"report.  Then try booting with the 'noapic' option.\n");
 out:

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-13  4:32                   ` Yinghai Lu
@ 2011-12-13  5:13                     ` Yinghai Lu
  2011-12-16 13:59                       ` Berck E. Nash
  0 siblings, 1 reply; 26+ messages in thread
From: Yinghai Lu @ 2011-12-13  5:13 UTC (permalink / raw)
  To: Suresh Siddha; +Cc: Berck E. Nash, Ingo Molnar, linux-kernel, H. Peter Anvin

[-- Attachment #1: Type: text/plain, Size: 327 bytes --]

On Mon, Dec 12, 2011 at 8:32 PM, Yinghai Lu <yinghai@kernel.org> wrote:
>> We never know. And also It is not just bios. kexec'd kernel with out the
>> interrupt-remapping support will also run into this issue and we should
>> handle this also gracefully.

sorry,  sent that -v6 too fast.

please check -v7.

Thanks

Yinghai Lu

[-- Attachment #2: nox2apic_v7.patch --]
[-- Type: text/x-patch, Size: 11454 bytes --]

Subject: [PATCH -v7] x86: Disable x2apic if nox2apic is specified or intr-remap can not be enabled

For
1. x2apic preenabled system
2. first kernel have x2apic enabled, and try to kexec second kernel with "nox2apic"

Will put back cpu with apic id < 255 into xapic mode, instead of panic.

-v2: use variable x2apic_disabled instead of variable nox2apic, Suggested by Thomas
     update x2apic_supported with x2apic_disabled, Suggested by Thomas

-v3: add checking for boot cpu apic id > 255. in that case will just panic
     --- pointed out by Suresh.

-v4: according to Ingo, for x2apic pre-enabled system, if intr-remap can not
	be enabled, try to disable x2apic instead of panic and request to
	specify nox2apic for next boot.

-v5: fix the lapic mapping for falling path.

-v6: add warning when check_timer fails in strange case when BIOS enable x2apic
     and intr_remapping but does not provide dmar table. requested by Suresh.

-v7: fix compiling with x2apic_preenabled

Signed-off-by: Yinghai Lu <yinghai@kernel.org>

---
 arch/x86/include/asm/apic.h         |    7 ++-
 arch/x86/include/asm/apicdef.h      |    1 
 arch/x86/include/asm/processor.h    |    1 
 arch/x86/kernel/acpi/boot.c         |   10 +++-
 arch/x86/kernel/apic/apic.c         |   78 ++++++++++++++++++++++++++----------
 arch/x86/kernel/apic/apic_flat_64.c |    7 ++-
 arch/x86/kernel/apic/io_apic.c      |    4 +
 arch/x86/kernel/cpu/topology.c      |   21 +++++++++
 arch/x86/mm/srat.c                  |    7 ++-
 9 files changed, 109 insertions(+), 27 deletions(-)

Index: linux-2.6/arch/x86/include/asm/apic.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apic.h
+++ linux-2.6/arch/x86/include/asm/apic.h
@@ -176,6 +176,8 @@ static inline u64 native_x2apic_icr_read
 }
 
 extern int x2apic_phys;
+extern int x2apic_preenabled;
+extern int x2apic_disabled;
 extern void check_x2apic(void);
 extern void enable_x2apic(void);
 extern void x2apic_icr_write(u32 low, u32 id);
@@ -183,7 +185,7 @@ static inline int x2apic_enabled(void)
 {
 	u64 msr;
 
-	if (!cpu_has_x2apic)
+	if (!cpu_has_x2apic || x2apic_disabled)
 		return 0;
 
 	rdmsrl(MSR_IA32_APICBASE, msr);
@@ -192,7 +194,7 @@ static inline int x2apic_enabled(void)
 	return 0;
 }
 
-#define x2apic_supported()	(cpu_has_x2apic)
+#define x2apic_supported()	(cpu_has_x2apic && !x2apic_disabled)
 static inline void x2apic_force_phys(void)
 {
 	x2apic_phys = 1;
@@ -214,6 +216,7 @@ static inline void x2apic_force_phys(voi
 
 #define	x2apic_preenabled 0
 #define	x2apic_supported()	0
+#define	x2apic_disabled	1
 #endif
 
 extern void enable_IR_x2apic(void);
Index: linux-2.6/arch/x86/kernel/acpi/boot.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/acpi/boot.c
+++ linux-2.6/arch/x86/kernel/acpi/boot.c
@@ -219,6 +219,8 @@ static int __init
 acpi_parse_x2apic(struct acpi_subtable_header *header, const unsigned long end)
 {
 	struct acpi_madt_local_x2apic *processor = NULL;
+	int apic_id;
+	u8 enabled;
 
 	processor = (struct acpi_madt_local_x2apic *)header;
 
@@ -227,6 +229,8 @@ acpi_parse_x2apic(struct acpi_subtable_h
 
 	acpi_table_print_madt_entry(header);
 
+	apic_id = processor->local_apic_id;
+	enabled = processor->lapic_flags & ACPI_MADT_ENABLED;
 #ifdef CONFIG_X86_X2APIC
 	/*
 	 * We need to register disabled CPU as well to permit
@@ -235,8 +239,10 @@ acpi_parse_x2apic(struct acpi_subtable_h
 	 * to not preallocating memory for all NR_CPUS
 	 * when we use CPU hotplug.
 	 */
-	acpi_register_lapic(processor->local_apic_id,	/* APIC ID */
-			    processor->lapic_flags & ACPI_MADT_ENABLED);
+	if (x2apic_disabled && (apic_id >= 0xff) && enabled)
+		printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
+	else
+		acpi_register_lapic(apic_id, enabled);
 #else
 	printk(KERN_WARNING PREFIX "x2apic entry ignored\n");
 #endif
Index: linux-2.6/arch/x86/kernel/apic/apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic.c
+++ linux-2.6/arch/x86/kernel/apic/apic.c
@@ -146,16 +146,15 @@ __setup("apicpmtimer", setup_apicpmtimer
 int x2apic_mode;
 #ifdef CONFIG_X86_X2APIC
 /* x2apic enabled before OS handover */
-static int x2apic_preenabled;
+int x2apic_preenabled;
+int x2apic_disabled;
 static __init int setup_nox2apic(char *str)
 {
-	if (x2apic_enabled()) {
-		pr_warning("Bios already enabled x2apic, "
-			   "can't enforce nox2apic");
-		return 0;
-	}
+	if (x2apic_enabled())
+		pr_warning("Bios already enabled x2apic, will disable it");
+
+	x2apic_disabled = 1;
 
-	setup_clear_cpu_cap(X86_FEATURE_X2APIC);
 	return 0;
 }
 early_param("nox2apic", setup_nox2apic);
@@ -1431,8 +1430,38 @@ void __init bsp_end_local_APIC_setup(voi
 }
 
 #ifdef CONFIG_X86_X2APIC
+
+static void disable_x2apic(void)
+{
+	int msr, msr2;
+
+	if (!cpu_has_x2apic)
+		return;
+
+	rdmsr(MSR_IA32_APICBASE, msr, msr2);
+	if (msr & X2APIC_ENABLE) {
+		u32 x2apic_id = x2apic_cpuid_initial_apicid();
+
+		if (x2apic_id > 255)
+			panic("Can not disable x2apic, id: %08x\n", x2apic_id);
+
+		pr_info("Disabling x2apic\n");
+		/*
+		 * Need to disable xapic and x2apic at the same time at first
+		 *  then enable xapic
+		 */
+		wrmsr(MSR_IA32_APICBASE, msr & ~(X2APIC_ENABLE | XAPIC_ENABLE),
+			 0);
+		wrmsr(MSR_IA32_APICBASE, msr & ~X2APIC_ENABLE, 0);
+	}
+}
 void check_x2apic(void)
 {
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (x2apic_enabled()) {
 		pr_info("x2apic enabled by BIOS, switching to x2apic ops\n");
 		x2apic_preenabled = x2apic_mode = 1;
@@ -1443,6 +1472,11 @@ void enable_x2apic(void)
 {
 	int msr, msr2;
 
+	if (x2apic_disabled) {
+		disable_x2apic();
+		return;
+	}
+
 	if (!x2apic_mode)
 		return;
 
@@ -1486,7 +1520,7 @@ void __init enable_IR_x2apic(void)
 	ret = save_ioapic_entries();
 	if (ret) {
 		pr_info("Saving IO-APIC state failed: %d\n", ret);
-		goto out;
+		return;
 	}
 
 	local_irq_save(flags);
@@ -1498,13 +1532,24 @@ void __init enable_IR_x2apic(void)
 	else
 		ret = enable_IR();
 
+	if (x2apic_disabled)
+		goto nox2apic;
+
 	if (ret < 0) {
 		/* IR is required if there is APIC ID > 255 even when running
 		 * under KVM
 		 */
 		if (max_physical_apicid > 255 ||
-		    !hypervisor_x2apic_available())
+		    !hypervisor_x2apic_available()) {
+			if (x2apic_preenabled) {
+				disable_x2apic();
+				x2apic_disabled = 1;
+				x2apic_mode = 0;
+				/* need to map lapic address */
+				register_lapic_address(mp_lapic_addr);
+			}
 			goto nox2apic;
+		}
 		/*
 		 * without IR all CPUs can be addressed by IOAPIC/MSI
 		 * only in physical mode
@@ -1512,8 +1557,10 @@ void __init enable_IR_x2apic(void)
 		x2apic_force_phys();
 	}
 
-	if (ret == IRQ_REMAP_XAPIC_MODE)
+	if (ret == IRQ_REMAP_XAPIC_MODE) {
+		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
 		goto nox2apic;
+	}
 
 	x2apic_enabled = 1;
 
@@ -1528,17 +1575,6 @@ nox2apic:
 		restore_ioapic_entries();
 	legacy_pic->restore_mask();
 	local_irq_restore(flags);
-
-out:
-	if (x2apic_enabled || !x2apic_supported())
-		return;
-
-	if (x2apic_preenabled)
-		panic("x2apic: enabled by BIOS but kernel init failed.");
-	else if (ret == IRQ_REMAP_XAPIC_MODE)
-		pr_info("x2apic not enabled, IRQ remapping is in xapic mode\n");
-	else if (ret < 0)
-		pr_info("x2apic not enabled, IRQ remapping init failed\n");
 }
 
 #ifdef CONFIG_X86_64
Index: linux-2.6/arch/x86/mm/srat.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat.c
+++ linux-2.6/arch/x86/mm/srat.c
@@ -69,6 +69,12 @@ acpi_numa_x2apic_affinity_init(struct ac
 	if ((pa->flags & ACPI_SRAT_CPU_ENABLED) == 0)
 		return;
 	pxm = pa->proximity_domain;
+	apic_id = pa->apic_id;
+	if (x2apic_disabled && (apic_id >= 0xff)) {
+		printk(KERN_INFO "SRAT: PXM %u -> X2APIC 0x%04x ignored\n",
+			 pxm, apic_id);
+		return;
+	}
 	node = setup_node(pxm);
 	if (node < 0) {
 		printk(KERN_ERR "SRAT: Too many proximity domains %x\n", pxm);
@@ -76,7 +82,6 @@ acpi_numa_x2apic_affinity_init(struct ac
 		return;
 	}
 
-	apic_id = pa->apic_id;
 	if (apic_id >= MAX_LOCAL_APIC) {
 		printk(KERN_INFO "SRAT: PXM %u -> APIC 0x%04x -> Node %u skipped apicid that is too big\n", pxm, apic_id, node);
 		return;
Index: linux-2.6/arch/x86/include/asm/apicdef.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/apicdef.h
+++ linux-2.6/arch/x86/include/asm/apicdef.h
@@ -144,6 +144,7 @@
 
 #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
 #define APIC_BASE_MSR	0x800
+#define XAPIC_ENABLE	(1UL << 11)
 #define X2APIC_ENABLE	(1UL << 10)
 
 #ifdef CONFIG_X86_32
Index: linux-2.6/arch/x86/include/asm/processor.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/processor.h
+++ linux-2.6/arch/x86/include/asm/processor.h
@@ -168,6 +168,7 @@ extern void init_scattered_cpuid_feature
 extern unsigned int init_intel_cacheinfo(struct cpuinfo_x86 *c);
 extern unsigned short num_cache_leaves;
 
+u32 x2apic_cpuid_initial_apicid(void);
 extern void detect_extended_topology(struct cpuinfo_x86 *c);
 extern void detect_ht(struct cpuinfo_x86 *c);
 
Index: linux-2.6/arch/x86/kernel/cpu/topology.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/cpu/topology.c
+++ linux-2.6/arch/x86/kernel/cpu/topology.c
@@ -21,6 +21,27 @@
 #define BITS_SHIFT_NEXT_LEVEL(eax)	((eax) & 0x1f)
 #define LEVEL_MAX_SIBLINGS(ebx)		((ebx) & 0xffff)
 
+u32 x2apic_cpuid_initial_apicid(void)
+{
+	unsigned int eax, ebx, ecx, edx;
+
+	if (boot_cpu_data.cpuid_level < 0xb)
+		return 0;
+
+	cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+	/*
+	 * check if the cpuid leaf 0xb is actually implemented.
+	 */
+	if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+		return 0;
+
+	/*
+	 * initial apic id, which also represents 32-bit extended x2apic id.
+	 */
+	return edx;
+}
+
 /*
  * Check for extended topology enumeration cpuid leaf 0xb and if it
  * exists, use it for populating initial_apicid and cpu topology
Index: linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/apic_flat_64.c
+++ linux-2.6/arch/x86/kernel/apic/apic_flat_64.c
@@ -171,9 +171,14 @@ static int flat_phys_pkg_id(int initial_
 	return initial_apic_id >> index_msb;
 }
 
+static int flat_probe(void)
+{
+	return 1;
+}
+
 static struct apic apic_flat =  {
 	.name				= "flat",
-	.probe				= NULL,
+	.probe				= flat_probe,
 	.acpi_madt_oem_check		= flat_acpi_madt_oem_check,
 	.apic_id_registered		= flat_apic_id_registered,
 
Index: linux-2.6/arch/x86/kernel/apic/io_apic.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/apic/io_apic.c
+++ linux-2.6/arch/x86/kernel/apic/io_apic.c
@@ -2948,6 +2948,10 @@ static inline void __init check_timer(vo
 	}
 	local_irq_disable();
 	apic_printk(APIC_QUIET, KERN_INFO "..... failed :(.\n");
+	if (x2apic_preenabled && !x2apic_mode)
+		pr_info("Your system has x2apic pre-enabled, but kernel can not enable intr-remapping.\n"
+			"So kernel try to disable x2apic, but interrupt still have problem.\n"
+			"You could disable x2apic from BIOS setup\n");
 	panic("IO-APIC + timer doesn't work!  Boot with apic=debug and send a "
 		"report.  Then try booting with the 'noapic' option.\n");
 out:

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-13  5:13                     ` Yinghai Lu
@ 2011-12-16 13:59                       ` Berck E. Nash
  2011-12-18  8:47                         ` Ingo Molnar
  0 siblings, 1 reply; 26+ messages in thread
From: Berck E. Nash @ 2011-12-16 13:59 UTC (permalink / raw)
  To: Yinghai Lu; +Cc: Suresh Siddha, Ingo Molnar, linux-kernel, H. Peter Anvin

On 12/12/2011 10:13 PM, Yinghai Lu wrote:
> please check -v7.

Works just fine for me.

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2011-12-16 13:59                       ` Berck E. Nash
@ 2011-12-18  8:47                         ` Ingo Molnar
  0 siblings, 0 replies; 26+ messages in thread
From: Ingo Molnar @ 2011-12-18  8:47 UTC (permalink / raw)
  To: Berck E. Nash; +Cc: Yinghai Lu, Suresh Siddha, linux-kernel, H. Peter Anvin


* Berck E. Nash <flyboy@gmail.com> wrote:

> On 12/12/2011 10:13 PM, Yinghai Lu wrote:
> > please check -v7.
> 
> Works just fine for me.

That's interesting.

Now if this could be made completely automatic (no extra 
parameter passing), then we have a usable fallback mode - which 
is preferred to crashing the box.

Thanks,

	Ingo

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2012-10-20  0:02                   ` Suresh Siddha
@ 2012-10-20  0:41                     ` Yinghai Lu
  0 siblings, 0 replies; 26+ messages in thread
From: Yinghai Lu @ 2012-10-20  0:41 UTC (permalink / raw)
  To: Suresh Siddha
  Cc: H. Peter Anvin, rrl125, fa.linux.kernel, Berck E. Nash, linux-kernel

On Fri, Oct 19, 2012 at 5:02 PM, Suresh Siddha
<suresh.b.siddha@intel.com> wrote:
> On Fri, 2012-10-19 at 16:36 -0700, H. Peter Anvin wrote:
>> On 10/19/2012 04:32 PM, Yinghai Lu wrote:
>> > On Fri, Oct 19, 2012 at 4:03 PM, Suresh Siddha
>> > <suresh.b.siddha@intel.com> wrote:
>> >> On Fri, 2012-10-19 at 13:42 -0700, rrl125@gmail.com wrote:
>> >>> Any update? The messages just seem to have stopped months ago. A
>> >>> fallback would be nice, I have been booting the kernel with noa2xpic
>> >>> for since kernel 3.2, and currently I am working with 3.6.2.
>> >>>
>> >>> If needed I can try to attempt modifying the patch to include
>> >>> fallback, but I am probably not the best person to do it.
>> >>>
>> >>
>> >> Are you referring to this commit that made into the mainline tree
>> >> already?
>> >>
>> >> commit fb209bd891645bb87b9618b724f0b4928e0df3de
>> >> Author: Yinghai Lu <yinghai@kernel.org>
>> >> Date:   Wed Dec 21 17:45:17 2011 -0800
>> >>
>> >>      x86, x2apic: Fallback to xapic when BIOS doesn't setup interrupt-remapping
>> >
>> > I think his system has DMAR table and cpu support x2apic, So kernel
>> > will switch to x2apic,
>> >
>> > but somehow BIOS SMI handler has problem with x2apic. should be thinkpad W520?
>> >
>>
>> Right, StinkPad W520 needs a quirk.
>
> yes. Yinghai, if you remember you had a T420 that didn't show the
> problem. And someone in the bugzilla with T420 had the problem. And
> their dmidecode is
> https://launchpadlibrarian.net/109393850/dmidecode.txt
>
> What is the difference with your system? Bios I think is the same.

1.37 was working too, i updated it to 1.39 and it is still working.

>
> Can you see what we should check for in the dmi tables to black list
> these systems? Can you post your T420's dmidecode to see the difference.
>
> https://bugzilla.kernel.org/show_bug.cgi?id=43054

post dmesg and dmidecode to that bug report.

Thanks

Yinghai

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2012-10-19 23:36                 ` H. Peter Anvin
@ 2012-10-20  0:02                   ` Suresh Siddha
  2012-10-20  0:41                     ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Suresh Siddha @ 2012-10-20  0:02 UTC (permalink / raw)
  To: H. Peter Anvin
  Cc: Yinghai Lu, rrl125, fa.linux.kernel, Berck E. Nash, linux-kernel

On Fri, 2012-10-19 at 16:36 -0700, H. Peter Anvin wrote:
> On 10/19/2012 04:32 PM, Yinghai Lu wrote:
> > On Fri, Oct 19, 2012 at 4:03 PM, Suresh Siddha
> > <suresh.b.siddha@intel.com> wrote:
> >> On Fri, 2012-10-19 at 13:42 -0700, rrl125@gmail.com wrote:
> >>> Any update? The messages just seem to have stopped months ago. A
> >>> fallback would be nice, I have been booting the kernel with noa2xpic
> >>> for since kernel 3.2, and currently I am working with 3.6.2.
> >>>
> >>> If needed I can try to attempt modifying the patch to include
> >>> fallback, but I am probably not the best person to do it.
> >>>
> >>
> >> Are you referring to this commit that made into the mainline tree
> >> already?
> >>
> >> commit fb209bd891645bb87b9618b724f0b4928e0df3de
> >> Author: Yinghai Lu <yinghai@kernel.org>
> >> Date:   Wed Dec 21 17:45:17 2011 -0800
> >>
> >>      x86, x2apic: Fallback to xapic when BIOS doesn't setup interrupt-remapping
> >
> > I think his system has DMAR table and cpu support x2apic, So kernel
> > will switch to x2apic,
> >
> > but somehow BIOS SMI handler has problem with x2apic. should be thinkpad W520?
> >
> 
> Right, StinkPad W520 needs a quirk.

yes. Yinghai, if you remember you had a T420 that didn't show the
problem. And someone in the bugzilla with T420 had the problem. And
their dmidecode is
https://launchpadlibrarian.net/109393850/dmidecode.txt

What is the difference with your system? Bios I think is the same.

Can you see what we should check for in the dmi tables to black list
these systems? Can you post your T420's dmidecode to see the difference.

Bugs I have on this are:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/776999
https://bugzilla.kernel.org/show_bug.cgi?id=43054

thanks,
suresh


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2012-10-19 23:32               ` Yinghai Lu
@ 2012-10-19 23:36                 ` H. Peter Anvin
  2012-10-20  0:02                   ` Suresh Siddha
  0 siblings, 1 reply; 26+ messages in thread
From: H. Peter Anvin @ 2012-10-19 23:36 UTC (permalink / raw)
  To: Yinghai Lu
  Cc: Suresh Siddha, rrl125, fa.linux.kernel, Berck E. Nash, linux-kernel

On 10/19/2012 04:32 PM, Yinghai Lu wrote:
> On Fri, Oct 19, 2012 at 4:03 PM, Suresh Siddha
> <suresh.b.siddha@intel.com> wrote:
>> On Fri, 2012-10-19 at 13:42 -0700, rrl125@gmail.com wrote:
>>> Any update? The messages just seem to have stopped months ago. A
>>> fallback would be nice, I have been booting the kernel with noa2xpic
>>> for since kernel 3.2, and currently I am working with 3.6.2.
>>>
>>> If needed I can try to attempt modifying the patch to include
>>> fallback, but I am probably not the best person to do it.
>>>
>>
>> Are you referring to this commit that made into the mainline tree
>> already?
>>
>> commit fb209bd891645bb87b9618b724f0b4928e0df3de
>> Author: Yinghai Lu <yinghai@kernel.org>
>> Date:   Wed Dec 21 17:45:17 2011 -0800
>>
>>      x86, x2apic: Fallback to xapic when BIOS doesn't setup interrupt-remapping
>
> I think his system has DMAR table and cpu support x2apic, So kernel
> will switch to x2apic,
>
> but somehow BIOS SMI handler has problem with x2apic. should be thinkpad W520?
>

Right, StinkPad W520 needs a quirk.


-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2012-10-19 23:03             ` Suresh Siddha
@ 2012-10-19 23:32               ` Yinghai Lu
  2012-10-19 23:36                 ` H. Peter Anvin
  0 siblings, 1 reply; 26+ messages in thread
From: Yinghai Lu @ 2012-10-19 23:32 UTC (permalink / raw)
  To: Suresh Siddha
  Cc: rrl125, fa.linux.kernel, Berck E. Nash, linux-kernel, H. Peter Anvin

On Fri, Oct 19, 2012 at 4:03 PM, Suresh Siddha
<suresh.b.siddha@intel.com> wrote:
> On Fri, 2012-10-19 at 13:42 -0700, rrl125@gmail.com wrote:
>> Any update? The messages just seem to have stopped months ago. A
>> fallback would be nice, I have been booting the kernel with noa2xpic
>> for since kernel 3.2, and currently I am working with 3.6.2.
>>
>> If needed I can try to attempt modifying the patch to include
>> fallback, but I am probably not the best person to do it.
>>
>
> Are you referring to this commit that made into the mainline tree
> already?
>
> commit fb209bd891645bb87b9618b724f0b4928e0df3de
> Author: Yinghai Lu <yinghai@kernel.org>
> Date:   Wed Dec 21 17:45:17 2011 -0800
>
>     x86, x2apic: Fallback to xapic when BIOS doesn't setup interrupt-remapping

I think his system has DMAR table and cpu support x2apic, So kernel
will switch to x2apic,

but somehow BIOS SMI handler has problem with x2apic. should be thinkpad W520?

^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
  2012-10-19 20:42           ` rrl125
@ 2012-10-19 23:03             ` Suresh Siddha
  2012-10-19 23:32               ` Yinghai Lu
  0 siblings, 1 reply; 26+ messages in thread
From: Suresh Siddha @ 2012-10-19 23:03 UTC (permalink / raw)
  To: rrl125
  Cc: fa.linux.kernel, Berck E. Nash, Yinghai Lu, linux-kernel, H. Peter Anvin

On Fri, 2012-10-19 at 13:42 -0700, rrl125@gmail.com wrote:
> Any update? The messages just seem to have stopped months ago. A
> fallback would be nice, I have been booting the kernel with noa2xpic
> for since kernel 3.2, and currently I am working with 3.6.2.
> 
> If needed I can try to attempt modifying the patch to include
> fallback, but I am probably not the best person to do it.
> 

Are you referring to this commit that made into the mainline tree
already?

commit fb209bd891645bb87b9618b724f0b4928e0df3de
Author: Yinghai Lu <yinghai@kernel.org>
Date:   Wed Dec 21 17:45:17 2011 -0800

    x86, x2apic: Fallback to xapic when BIOS doesn't setup interrupt-remapping
    
    On some of the recent Intel SNB platforms, by default bios is pre-enabling
    x2apic mode in the cpu with out setting up interrupt-remapping.
    This case was resulting in the kernel to panic as the cpu is already in
    x2apic mode but the OS was not able to enable interrupt-remapping (which
    is a pre-req for using x2apic capability).
    
    On these platforms all the apic-ids are < 255 and the kernel can fallback to
    xapic mode if the bios has not enabled interrupt-remapping (which is
    mostly the case if the bios has not exported interrupt-remapping tables to the
    OS).
    
    Reported-by: Berck E. Nash <flyboy@gmail.com>
    Signed-off-by: Yinghai Lu <yinghai@kernel.org>
    Link: http://lkml.kernel.org/r/20111222014632.600418637@sbsiddha-desk.sc.intel.com
    Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
    Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: x2apic boot failure on recent sandy bridge system
       [not found]         ` <fa.eH+BpYDdp4xIWE4V+wJu6hFrVxM@ifi.uio.no>
@ 2012-10-19 20:42           ` rrl125
  2012-10-19 23:03             ` Suresh Siddha
  0 siblings, 1 reply; 26+ messages in thread
From: rrl125 @ 2012-10-19 20:42 UTC (permalink / raw)
  To: fa.linux.kernel
  Cc: Berck E. Nash, Yinghai Lu, Suresh Siddha, linux-kernel, H. Peter Anvin

Any update? The messages just seem to have stopped months ago. A fallback would be nice, I have been booting the kernel with noa2xpic for since kernel 3.2, and currently I am working with 3.6.2.

If needed I can try to attempt modifying the patch to include fallback, but I am probably not the best person to do it.

On Sunday, December 18, 2011 3:49:27 AM UTC-5, Ingo Molnar wrote:
> * Berck E. Nash <flyboy@gmail.com> wrote:
> 
> 
> 
> > On 12/12/2011 10:13 PM, Yinghai Lu wrote:
> 
> > > please check -v7.
> 
> > 
> 
> > Works just fine for me.
> 
> 
> 
> That's interesting.
> 
> 
> 
> Now if this could be made completely automatic (no extra 
> 
> parameter passing), then we have a usable fallback mode - which 
> 
> is preferred to crashing the box.
> 
> 
> 
> Thanks,
> 
> 
> 
> 	Ingo
> 
> --
> 
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> 
> the body of a message to majordomo@vger.kernel.org
> 
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> Please read the FAQ at  http://www.tux.org/lkml/


^ permalink raw reply	[flat|nested] 26+ messages in thread

* x2apic boot failure on recent sandy bridge system
@ 2011-12-10  1:46 Berck E. Nash
  0 siblings, 0 replies; 26+ messages in thread
From: Berck E. Nash @ 2011-12-10  1:46 UTC (permalink / raw)
  To: linux-ide

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

X2APIC has never worked on this system, and continues to not work
3.2.0-rc5.  Simply hangs on boot with an error message that helpfully
indicates that x2apic is the problem.  The system is an ASRock P67
Extreme 6.  Log attached, let me know if I can do anything to help.
Please cc as I'm not subscribed.

Berck Nash

[-- Attachment #2: x2apic.log --]
[-- Type: text/plain, Size: 7536 bytes --]

[    0.000000] Linux version 3.2.0-rc5 (berck@luna) (gcc version 4.6.2 (Debian 4.6.2-6) ) #2 SMP Fri Dec 9 18:16:28 MST 2011
[    0.000000] Command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009d400 (usable)
[    0.000000]  BIOS-e820: 000000000009d400 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000beee2000 (usable)
[    0.000000]  BIOS-e820: 00000000beee2000 - 00000000bef3a000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bef3a000 - 00000000bf607000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf607000 - 00000000bf608000 (usable)
[    0.000000]  BIOS-e820: 00000000bf608000 - 00000000bf609000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000bf609000 - 00000000bf613000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf613000 - 00000000bf638000 (reserved)
[    0.000000]  BIOS-e820: 00000000bf638000 - 00000000bf67b000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000bf67b000 - 00000000bf800000 (usable)
[    0.000000]  BIOS-e820: 00000000fed1c000 - 00000000fed40000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 000000023f800000 (usable)
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI 2.6 present.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x23f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] last_pfn = 0xbf800 max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000bf800000
[    0.000000] init_memory_mapping: 0000000100000000-000000023f800000
[    0.000000] ACPI: RSDP 00000000000f0450 00024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 00000000bef32068 00054 (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 00000000bef398f8 000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 00000000bef32150 077A6 (v02 ALASKA    A M I 00000000 INTL 20051117)
[    0.000000] ACPI: FACS 00000000bf60af80 00040
[    0.000000] ACPI: APIC 00000000bef399f0 000D8 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: SSDT 00000000bef39ac8 001DE (v01 AMICPU     PROC 00000001 MSFT 03000001)
[    0.000000] ACPI: MCFG 00000000bef39ca8 0003C (v01 ALASKA    A M I 01072009 MSFT 00000097)
[    0.000000] ACPI: AAFT 00000000bef39ce8 0006F (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 00000000bef39d58 00038 (v01 ALASKA    A M I 01072009 AMI. 00000004)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-000000023f800000
[    0.000000] Initmem setup node 0 0000000000000000-000000023f800000
[    0.000000]   NODE_DATA [000000023f7fb000 - 000000023f7fffff]
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x0023f800
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[5] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009d
[    0.000000]     0: 0x00000100 -> 0x000beee2
[    0.000000]     0: 0x000bf607 -> 0x000bf608
[    0.000000]     0: 0x000bf67b -> 0x000bf800
[    0.000000]     0: 0x00100000 -> 0x0023f800
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: X2APIC (apic_id[0x00] uid[0x01] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x02] uid[0x02] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x04] uid[0x03] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x06] uid[0x04] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x01] uid[0x05] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x03] uid[0x06] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x05] uid[0x07] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x07] uid[0x08] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] SMP: Allowing 8 CPUs, 0 hotplug CPUs
[    0.000000] Allocating PCI resources starting at bf800000 (gap: bf800000:3f51c000)
[    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
[    0.000000] PERCPU: Embedded 25 pages/cpu @ffff88023f400000 s70976 r8192 d23232 u262144
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2058767
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,1)/vmlinuz root=/dev/sdb1 ro video=efifb console=ttyS0,115200n8 console=tty0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 8175908k/9428992k available (3610k kernel code, 1065004k absent, 188080k reserved, 1567k data, 672k init)
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:744 16
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] console [ttyS0] enabled
[    0.000000] Fast TSC calibration using PIT
[    0.001000] Detected 3392.514 MHz processor.
[    0.000001] Calibrating delay loop (skipped), value calculated using timer frequency.. 6785.02 BogoMIPS (lpj=3392514)
[    0.010594] pid_max: default: 32768 minimum: 301
[    0.015470] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.023753] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.031298] Mount-cache hash table entries: 256
[    0.035893] CPU: Physical Processor ID: 0
[    0.039900] CPU: Processor Core ID: 0
[    0.043557] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.043558] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.056570] mce: CPU supports 9 MCE banks
[    0.060577] CPU0: Thermal monitoring enabled (TM1)
[    0.065360] using mwait in idle threads.
[    0.069332] Freeing SMP alternatives: 20k freed
[    0.073857] ACPI: Core revision 20110623
[    0.085717] Kernel panic - not syncing: x2apic: enabled by BIOS but kernel init failed.
[    0.093697] Pid: 1, comm: swapper/0 Not tainted 3.2.0-rc5 #2
[    0.099337] Call Trace:
[    0.101788]  [<ffffffff8137b7ce>] ? panic+0x95/0x18e
[    0.106755]  [<ffffffff81016259>] ? __ioapic_write_entry+0x16/0x32
[    0.112925]  [<ffffffff8152d5bf>] ? enable_IR_x2apic+0x19a/0x1d7
[    0.118913]  [<ffffffff8152f281>] ? default_setup_apic_routing+0xd/0x71
[    0.125506]  [<ffffffff8152c536>] ? native_smp_prepare_cpus+0x2f1/0x325
[    0.132102]  [<ffffffff81522abf>] ? kernel_init+0x46/0x10f
[    0.137571]  [<ffffffff813841f4>] ? kernel_thread_helper+0x4/0x10
[    0.143650]  [<ffffffff81522a79>] ? start_kernel+0x341/0x341
[    0.149297]  [<ffffffff813841f0>] ? gs_change+0xb/0xb

^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2012-10-20  0:41 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-10  1:56 x2apic boot failure on recent sandy bridge system Berck E. Nash
2011-12-10 20:17 ` Yinghai Lu
2011-12-10 22:58   ` Berck E. Nash
2011-12-10 23:11     ` Yinghai Lu
2011-12-11  7:57   ` Ingo Molnar
2011-12-11 21:43     ` Yinghai Lu
2011-12-11 22:05       ` Berck E. Nash
2011-12-12  3:40         ` Yinghai Lu
2011-12-12 23:21           ` Berck E. Nash
2011-12-13  0:25             ` Suresh Siddha
2011-12-13  1:29               ` Yinghai Lu
2011-12-13  1:41                 ` Suresh Siddha
2011-12-13  4:32                   ` Yinghai Lu
2011-12-13  5:13                     ` Yinghai Lu
2011-12-16 13:59                       ` Berck E. Nash
2011-12-18  8:47                         ` Ingo Molnar
2011-12-12 23:42           ` H. Peter Anvin
2011-12-13  1:25             ` Yinghai Lu
2011-12-12  7:23       ` Suresh Siddha
     [not found] <fa.1zfuXwqYK6u81uBoo3IdpawcpWw@ifi.uio.no>
     [not found] ` <fa.p3cpkKIGUNv0/iot3o7KT9bb98M@ifi.uio.no>
     [not found]   ` <fa.iPHW7usT23k+cCA19L3eXEDu91Q@ifi.uio.no>
     [not found]     ` <fa.7eOK+XcsKUWU4ZXhBj2K6znxrvc@ifi.uio.no>
     [not found]       ` <fa.VGxfKOh1Z07X1TXm5PxMIlV4xu8@ifi.uio.no>
     [not found]         ` <fa.eH+BpYDdp4xIWE4V+wJu6hFrVxM@ifi.uio.no>
2012-10-19 20:42           ` rrl125
2012-10-19 23:03             ` Suresh Siddha
2012-10-19 23:32               ` Yinghai Lu
2012-10-19 23:36                 ` H. Peter Anvin
2012-10-20  0:02                   ` Suresh Siddha
2012-10-20  0:41                     ` Yinghai Lu
  -- strict thread matches above, loose matches on Subject: below --
2011-12-10  1:46 Berck E. Nash

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.