* Inconsistent load average on tickless kernels
@ 2012-02-06 6:51 Aman Gupta
2012-02-23 15:46 ` Lesław Kopeć
0 siblings, 1 reply; 15+ messages in thread
From: Aman Gupta @ 2012-02-06 6:51 UTC (permalink / raw)
To: linux-kernel, Peter Zijlstra, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder,
Lesław Kopeć
[-- Attachment #1: Type: text/plain, Size: 2362 bytes --]
I have an LVS/DR cluster of 10 machines that receive similar traffic
via a round-robin strategy. These machines run Debian Lenny with
2.6.26, and consistently have a 15-minute load average between 4-12
depending on the time of day.
Upgrading any one of these machines to a newer kernel compiled with
NO_HZ=y causes the reported load average to drop significantly. Here,
fe3 and fe12 are running 3.2.4:
fe3: 0.48 0.53 0.55
fe4: 6.73 5.59 5.11
fe5: 5.93 5.29 5.60
fe6: 6.20 5.79 6.08
fe7: 8.32 5.65 5.05
fe8: 6.34 5.85 5.93
fe9: 5.80 5.46 5.53
fe10: 5.49 4.91 5.03
fe11: 6.60 6.11 6.10
fe12: 0.39 0.54 0.46
The newly reported load average is much lower than the other machines
performing equivalent work, and does not match cpu usage numbers
reported by vmstat.
Originally, I attempted to upgrade to 2.6.32 (used by lenny-backports
and squeeze). In 2.6.32, load averages are misreported even when using
NO_HZ=n. This bug was fixed in 2.6.34-rc4 (74f5187ac8: sched: Cure
load average vs NO_HZ woes).
The NO_HZ=y case was supposed to be fixed in 2.6.37-rc5 (0f004f5a69:
sched: Cure more NO_HZ load average woes), with the commit stating
"behaviour between CONFIG_NO_HZ=[yn] should be equivalent". In my
environment, however, kernels after this patch was introduced still
misreport load averages when compiled with NO_HZ=y.
Here's a list of the kernel versions I've tried (more details in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=620297):
Correct Load Average
2.6.26.25
2.6.32.55-620297patch (CONFIG_NO_HZ=n)
Incorrect Load Average
2.6.32-bpo.5-amd64
2.6.32.55
2.6.32.55-620297patch
2.6.32.55-620297patch (nohz=off)
2.6.37-rc5-cure-more
2.6.39.4
3.2.2
3.2.4
Also worth nothing is that fe12 is much newer than fe3, which should
help rule out hardware as a cause of this bug.
fe3: Dell PowerEdge 2950 w/ Xeon(R) CPU L5420 @ 2.50GHz
fe12: Dell PowerEdge R710 w/ Xeon(R) CPU E5645 @ 2.40GHz
I've attached dmesg output from fe12 booting up on 3.2.4. I am happy
to provide any other information that would be useful, and would
appreciate any advice on patches to try or ways to narrow the bug down
further.
Aman
[-- Attachment #2: dmesg-fe12-3.2.4.txt --]
[-- Type: text/plain, Size: 59632 bytes --]
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 3.2.4 (root@fe3) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) #1 SMP Sat Feb 4 08:24:56 PST 2012
[ 0.000000] Command line: root=/dev/mapper/fe12-root ro console=ttyS1,115200 rootdelay=30
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] BIOS-e820: 0000000000100000 - 00000000cf379000 (usable)
[ 0.000000] BIOS-e820: 00000000cf379000 - 00000000cf38f000 (reserved)
[ 0.000000] BIOS-e820: 00000000cf38f000 - 00000000cf3ce000 (ACPI data)
[ 0.000000] BIOS-e820: 00000000cf3ce000 - 00000000d0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fe000000 - 0000000100000000 (reserved)
[ 0.000000] BIOS-e820: 0000000100000000 - 0000000630000000 (usable)
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] DMI 2.6 present.
[ 0.000000] DMI: Dell Inc. PowerEdge R710/00NH4P, BIOS 3.0.0 01/31/2011
[ 0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[ 0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
[ 0.000000] No AGP bridge found
[ 0.000000] last_pfn = 0x630000 max_arch_pfn = 0x400000000
[ 0.000000] MTRR default type: uncachable
[ 0.000000] MTRR fixed ranges enabled:
[ 0.000000] 00000-9FFFF write-back
[ 0.000000] A0000-BFFFF uncachable
[ 0.000000] C0000-D3FFF write-protect
[ 0.000000] D4000-EBFFF uncachable
[ 0.000000] EC000-FFFFF write-protect
[ 0.000000] MTRR variable ranges enabled:
[ 0.000000] 0 base 0000000000 mask FF80000000 write-back
[ 0.000000] 1 base 0080000000 mask FFC0000000 write-back
[ 0.000000] 2 base 00C0000000 mask FFF0000000 write-back
[ 0.000000] 3 base 0100000000 mask FF00000000 write-back
[ 0.000000] 4 base 0200000000 mask FE00000000 write-back
[ 0.000000] 5 base 0400000000 mask FE00000000 write-back
[ 0.000000] 6 base 0600000000 mask FFE0000000 write-back
[ 0.000000] 7 base 0620000000 mask FFF0000000 write-back
[ 0.000000] 8 disabled
[ 0.000000] 9 disabled
[ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[ 0.000000] e820 update range: 00000000d0000000 - 0000000100000000 (usable) ==> (reserved)
[ 0.000000] last_pfn = 0xcf379 max_arch_pfn = 0x400000000
[ 0.000000] found SMP MP-table at [ffff8800000fe710] fe710
[ 0.000000] initial memory mapped : 0 - 20000000
[ 0.000000] Base memory trampoline at [ffff880000099000] 99000 size 20480
[ 0.000000] Using GB pages for direct mapping
[ 0.000000] init_memory_mapping: 0000000000000000-00000000cf379000
[ 0.000000] 0000000000 - 00c0000000 page 1G
[ 0.000000] 00c0000000 - 00cf200000 page 2M
[ 0.000000] 00cf200000 - 00cf379000 page 4k
[ 0.000000] kernel direct mapping tables up to cf379000 @ 1fffd000-20000000
[ 0.000000] init_memory_mapping: 0000000100000000-0000000630000000
[ 0.000000] 0100000000 - 0600000000 page 1G
[ 0.000000] 0600000000 - 0630000000 page 2M
[ 0.000000] kernel direct mapping tables up to 630000000 @ cf377000-cf379000
[ 0.000000] RAMDISK: 369e9000 - 37ff0000
[ 0.000000] ACPI: RSDP 00000000000f11a0 00024 (v02 DELL )
[ 0.000000] ACPI: XSDT 00000000000f12a4 0009C (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: FACP 00000000cf3b3f9c 000F4 (v03 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: DSDT 00000000cf38f000 03D72 (v01 DELL PE_SC3 00000001 INTL 20050624)
[ 0.000000] ACPI: FACS 00000000cf3b6000 00040
[ 0.000000] ACPI: APIC 00000000cf3b3478 0015E (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: SPCR 00000000cf3b35d8 00050 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: HPET 00000000cf3b362c 00038 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: DMAR 00000000cf3b3668 001C0 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: MCFG 00000000cf3b38c4 0003C (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: WD__ 00000000cf3b3904 00134 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: SLIC 00000000cf3b3a3c 00024 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: ERST 00000000cf392ef4 00270 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: HEST 00000000cf393164 003A8 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: BERT 00000000cf392d74 00030 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: EINJ 00000000cf392da4 00150 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: SRAT 00000000cf3b3bc0 00370 (v01 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: TCPA 00000000cf3b3f34 00064 (v02 DELL PE_SC3 00000001 DELL 00000001)
[ 0.000000] ACPI: SSDT 00000000cf3b7000 03300 (v01 INTEL PPM RCM 80000001 INTL 20061109)
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] SRAT: PXM 1 -> APIC 0x20 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x00 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 0x22 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x02 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 0x24 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x04 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 0x30 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x10 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 0x32 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x12 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 0x34 -> Node 0
[ 0.000000] SRAT: PXM 2 -> APIC 0x14 -> Node 1
[ 0.000000] SRAT: Node 1 PXM 2 0-d0000000
[ 0.000000] SRAT: Node 1 PXM 2 100000000-330000000
[ 0.000000] SRAT: Node 0 PXM 1 330000000-630000000
[ 0.000000] NUMA: Node 1 [0,d0000000) + [100000000,330000000) -> [0,330000000)
[ 0.000000] Initmem setup node 0 0000000330000000-0000000630000000
[ 0.000000] NODE_DATA [000000062fffb000 - 000000062fffffff]
[ 0.000000] Initmem setup node 1 0000000000000000-0000000330000000
[ 0.000000] NODE_DATA [000000032fffb000 - 000000032fffffff]
[ 0.000000] [ffffea0000000000-ffffea000cbfffff] PMD -> [ffff880323e00000-ffff88032fdfffff] on node 1
[ 0.000000] [ffffea000cc00000-ffffea0018bfffff] PMD -> [ffff880623600000-ffff88062f5fffff] on node 0
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000010 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x00630000
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[4] active PFN ranges
[ 0.000000] 1: 0x00000010 -> 0x000000a0
[ 0.000000] 1: 0x00000100 -> 0x000cf379
[ 0.000000] 1: 0x00100000 -> 0x00330000
[ 0.000000] 0: 0x00330000 -> 0x00630000
[ 0.000000] On node 0 totalpages: 3145728
[ 0.000000] Normal zone: 49152 pages used for memmap
[ 0.000000] Normal zone: 3096576 pages, LIFO batch:31
[ 0.000000] On node 1 totalpages: 3142409
[ 0.000000] DMA zone: 64 pages used for memmap
[ 0.000000] DMA zone: 7 pages reserved
[ 0.000000] DMA zone: 3913 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 16320 pages used for memmap
[ 0.000000] DMA32 zone: 828345 pages, LIFO batch:31
[ 0.000000] Normal zone: 35840 pages used for memmap
[ 0.000000] Normal zone: 2257920 pages, LIFO batch:31
[ 0.000000] ACPI: PM-Timer IO Port: 0x808
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x20] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x22] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x02] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x24] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x04] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x30] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x10] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x32] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x12] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x34] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x14] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0xff] disabled)
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[ 0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: IOAPIC (id[0x01] address[0xfec80000] gsi_base[32])
[ 0.000000] IOAPIC[1]: apic_id 1, version 32, address 0xfec80000, GSI 32-55
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: IRQ0 used by override.
[ 0.000000] ACPI: IRQ2 used by override.
[ 0.000000] ACPI: IRQ9 used by override.
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a301 base: 0xfed00000
[ 0.000000] SMP: Allowing 32 CPUs, 20 hotplug CPUs
[ 0.000000] nr_irqs_gsi: 72
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
[ 0.000000] PM: Registered nosave memory: 00000000cf379000 - 00000000cf38f000
[ 0.000000] PM: Registered nosave memory: 00000000cf38f000 - 00000000cf3ce000
[ 0.000000] PM: Registered nosave memory: 00000000cf3ce000 - 00000000d0000000
[ 0.000000] PM: Registered nosave memory: 00000000d0000000 - 00000000e0000000
[ 0.000000] PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
[ 0.000000] PM: Registered nosave memory: 00000000f0000000 - 00000000fe000000
[ 0.000000] PM: Registered nosave memory: 00000000fe000000 - 0000000100000000
[ 0.000000] Allocating PCI resources starting at d0000000 (gap: d0000000:10000000)
[ 0.000000] Booting paravirtualized kernel on bare hardware
[ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:32 nr_node_ids:2
[ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff880323c00000 s81600 r8192 d20800 u131072
[ 0.000000] pcpu-alloc: s81600 r8192 d20800 u131072 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 00 02 04 06 08 10 12 14 16 18 20 22 24 26 28 30
[ 0.000000] pcpu-alloc: [1] 01 03 05 07 09 11 13 15 17 19 21 23 25 27 29 31
[ 0.000000] Built 2 zonelists in Zone order, mobility grouping on. Total pages: 6186754
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: root=/dev/mapper/fe12-root ro console=ttyS1,115200 rootdelay=30
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Calgary: detecting Calgary via BIOS EBDA area
[ 0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[ 0.000000] Memory: 24658712k/25952256k available (3326k kernel code, 799708k absent, 493836k reserved, 3342k data, 564k init)
[ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=2
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] NR_IRQS:33024 nr_irqs:1480 16
[ 0.000000] Extended CMOS year: 2000
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS1] enabled
[ 0.000000] hpet clockevent registered
[ 0.000000] Fast TSC calibration using PIT
[ 0.004000] Detected 2394.162 MHz processor.
[ 0.000009] Calibrating delay loop (skipped), value calculated using timer frequency.. 4788.32 BogoMIPS (lpj=9576648)
[ 0.010597] pid_max: default: 32768 minimum: 301
[ 0.015255] Security Framework initialized
[ 0.019339] SELinux: Disabled at boot.
[ 0.025343] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes)
[ 0.039491] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[ 0.049780] Mount-cache hash table entries: 256
[ 0.054550] Initializing cgroup subsys cpuacct
[ 0.058991] Initializing cgroup subsys devices
[ 0.063421] Initializing cgroup subsys freezer
[ 0.067849] Initializing cgroup subsys net_cls
[ 0.072314] CPU: Physical Processor ID: 1
[ 0.076310] CPU: Processor Core ID: 0
[ 0.079963] mce: CPU supports 9 MCE banks
[ 0.083968] CPU0: Thermal monitoring enabled (TM1)
[ 0.088749] using mwait in idle threads.
[ 0.093446] ACPI: Core revision 20110623
[ 0.098789] Switched APIC routing to physical flat.
[ 0.104190] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.149787] CPU0: Intel(R) Xeon(R) CPU E5645 @ 2.40GHz stepping 02
[ 0.264012] Performance Events: PEBS fmt1+, Westmere events, Intel PMU driver.
[ 0.271234] ... version: 3
[ 0.275226] ... bit width: 48
[ 0.279301] ... generic registers: 4
[ 0.283292] ... value mask: 0000ffffffffffff
[ 0.288579] ... max period: 000000007fffffff
[ 0.293867] ... fixed-purpose events: 3
[ 0.297858] ... event mask: 000000070000000f
[ 0.303386] Booting Node 1, Processors #1
[ 0.307474] smpboot cpu 1: start_ip = 99000
[ 0.415944] Ok.
[ 0.417771] Booting Node 0, Processors #2
[ 0.421859] smpboot cpu 2: start_ip = 99000
[ 0.531736] Ok.
[ 0.533563] Booting Node 1, Processors #3
[ 0.537651] smpboot cpu 3: start_ip = 99000
[ 0.647528] Ok.
[ 0.649355] Booting Node 0, Processors #4
[ 0.653443] smpboot cpu 4: start_ip = 99000
[ 0.763364] Ok.
[ 0.765193] Booting Node 1, Processors #5
[ 0.769281] smpboot cpu 5: start_ip = 99000
[ 0.879061] Ok.
[ 0.880888] Booting Node 0, Processors #6
[ 0.884976] smpboot cpu 6: start_ip = 99000
[ 0.994901] Ok.
[ 0.996730] Booting Node 1, Processors #7
[ 1.000817] smpboot cpu 7: start_ip = 99000
[ 1.110692] Ok.
[ 1.112517] Booting Node 0, Processors #8
[ 1.116604] smpboot cpu 8: start_ip = 99000
[ 1.226522] Ok.
[ 1.228349] Booting Node 1, Processors #9
[ 1.232435] smpboot cpu 9: start_ip = 99000
[ 1.342314] Ok.
[ 1.344141] Booting Node 0, Processors #10
[ 1.348316] smpboot cpu 10: start_ip = 99000
[ 1.458094] Ok.
[ 1.459922] Booting Node 1, Processors #11
[ 1.464095] smpboot cpu 11: start_ip = 99000
[ 1.573858] Brought up 12 CPUs
[ 1.576898] Total of 12 processors activated (57456.12 BogoMIPS).
[ 1.600611] devtmpfs: initialized
[ 1.608513] print_constraints: dummy:
[ 1.612321] NET: Registered protocol family 16
[ 1.616839] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[ 1.624380] ACPI: bus type pci registered
[ 1.628432] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[ 1.637706] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
[ 1.661077] PCI: Using configuration type 1 for base access
[ 1.667876] bio: create slab <bio-0> at 0
[ 1.671973] ACPI: Added _OSI(Module Device)
[ 1.676141] ACPI: Added _OSI(Processor Device)
[ 1.680565] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 1.685249] ACPI: Added _OSI(Processor Aggregator Device)
[ 1.691066] ACPI: EC: Look up EC in DSDT
[ 1.691791] [Firmware Bug]: ACPI: BIOS _OSI(Linux) query ignored
[ 1.699216] ACPI: Interpreter enabled
[ 1.702866] ACPI: (supports S0 S4 S5)
[ 1.706548] ACPI: Using IOAPIC for interrupt routing
[ 1.714540] ACPI: No dock devices found.
[ 1.718451] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 1.727736] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 1.734056] pci_root PNP0A08:00: host bridge window [io 0x0000-0x0cf7]
[ 1.740646] pci_root PNP0A08:00: host bridge window [io 0x0d00-0xffff]
[ 1.747235] pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff]
[ 1.754514] pci_root PNP0A08:00: host bridge window [mem 0xd0000000-0xfdffffff]
[ 1.761794] pci_root PNP0A08:00: host bridge window [mem 0xfed40000-0xfed44fff]
[ 1.769088] pci 0000:00:00.0: [8086:3406] type 0 class 0x000600
[ 1.769139] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[ 1.769142] pci 0000:00:00.0: PME# disabled
[ 1.769165] pci 0000:00:01.0: [8086:3408] type 1 class 0x000604
[ 1.769215] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[ 1.769218] pci 0000:00:01.0: PME# disabled
[ 1.769238] pci 0000:00:03.0: [8086:340a] type 1 class 0x000604
[ 1.769287] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[ 1.769290] pci 0000:00:03.0: PME# disabled
[ 1.769309] pci 0000:00:04.0: [8086:340b] type 1 class 0x000604
[ 1.769359] pci 0000:00:04.0: PME# supported from D0 D3hot D3cold
[ 1.769362] pci 0000:00:04.0: PME# disabled
[ 1.769385] pci 0000:00:05.0: [8086:340c] type 1 class 0x000604
[ 1.769435] pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
[ 1.769438] pci 0000:00:05.0: PME# disabled
[ 1.769457] pci 0000:00:06.0: [8086:340d] type 1 class 0x000604
[ 1.769507] pci 0000:00:06.0: PME# supported from D0 D3hot D3cold
[ 1.769510] pci 0000:00:06.0: PME# disabled
[ 1.769528] pci 0000:00:07.0: [8086:340e] type 1 class 0x000604
[ 1.769579] pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
[ 1.769581] pci 0000:00:07.0: PME# disabled
[ 1.769601] pci 0000:00:09.0: [8086:3410] type 1 class 0x000604
[ 1.769651] pci 0000:00:09.0: PME# supported from D0 D3hot D3cold
[ 1.769654] pci 0000:00:09.0: PME# disabled
[ 1.769676] pci 0000:00:14.0: [8086:342e] type 0 class 0x000800
[ 1.769739] pci 0000:00:14.1: [8086:3422] type 0 class 0x000800
[ 1.769797] pci 0000:00:14.2: [8086:3423] type 0 class 0x000800
[ 1.769864] pci 0000:00:1a.0: [8086:2937] type 0 class 0x000c03
[ 1.769903] pci 0000:00:1a.0: reg 20: [io 0xec40-0xec5f]
[ 1.769951] pci 0000:00:1a.1: [8086:2938] type 0 class 0x000c03
[ 1.769990] pci 0000:00:1a.1: reg 20: [io 0xec60-0xec7f]
[ 1.770046] pci 0000:00:1a.7: [8086:293c] type 0 class 0x000c03
[ 1.770066] pci 0000:00:1a.7: reg 10: [mem 0xdf0fe000-0xdf0fe3ff]
[ 1.770149] pci 0000:00:1a.7: PME# supported from D0 D3hot D3cold
[ 1.770154] pci 0000:00:1a.7: PME# disabled
[ 1.770173] pci 0000:00:1d.0: [8086:2934] type 0 class 0x000c03
[ 1.770212] pci 0000:00:1d.0: reg 20: [io 0xec80-0xec9f]
[ 1.770259] pci 0000:00:1d.1: [8086:2935] type 0 class 0x000c03
[ 1.770298] pci 0000:00:1d.1: reg 20: [io 0xeca0-0xecbf]
[ 1.770355] pci 0000:00:1d.7: [8086:293a] type 0 class 0x000c03
[ 1.770374] pci 0000:00:1d.7: reg 10: [mem 0xdf0ff000-0xdf0ff3ff]
[ 1.770458] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
[ 1.770462] pci 0000:00:1d.7: PME# disabled
[ 1.770481] pci 0000:00:1e.0: [8086:244e] type 1 class 0x000604
[ 1.770538] pci 0000:00:1f.0: [8086:2918] type 0 class 0x000601
[ 1.770614] pci 0000:00:1f.0: ICH7 LPC Generic IO decode 1 PIO at 0c00 (mask 007f)
[ 1.778154] pci 0000:00:1f.0: ICH7 LPC Generic IO decode 2 PIO at 0ca0 (mask 000f)
[ 1.785694] pci 0000:00:1f.0: ICH7 LPC Generic IO decode 3 PIO at 00e0 (mask 000f)
[ 1.793316] pci 0000:01:00.0: [14e4:1639] type 0 class 0x000200
[ 1.793332] pci 0000:01:00.0: reg 10: [mem 0xd6000000-0xd7ffffff 64bit]
[ 1.793410] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 1.793414] pci 0000:01:00.0: PME# disabled
[ 1.793442] pci 0000:01:00.1: [14e4:1639] type 0 class 0x000200
[ 1.793457] pci 0000:01:00.1: reg 10: [mem 0xd8000000-0xd9ffffff 64bit]
[ 1.793535] pci 0000:01:00.1: PME# supported from D0 D3hot D3cold
[ 1.793538] pci 0000:01:00.1: PME# disabled
[ 1.793560] pci 0000:00:01.0: PCI bridge to [bus 01-01]
[ 1.798767] pci 0000:00:01.0: bridge window [mem 0xd6000000-0xd9ffffff]
[ 1.798816] pci 0000:02:00.0: [14e4:1639] type 0 class 0x000200
[ 1.798831] pci 0000:02:00.0: reg 10: [mem 0xda000000-0xdbffffff 64bit]
[ 1.798909] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[ 1.798912] pci 0000:02:00.0: PME# disabled
[ 1.798940] pci 0000:02:00.1: [14e4:1639] type 0 class 0x000200
[ 1.798955] pci 0000:02:00.1: reg 10: [mem 0xdc000000-0xddffffff 64bit]
[ 1.799032] pci 0000:02:00.1: PME# supported from D0 D3hot D3cold
[ 1.799036] pci 0000:02:00.1: PME# disabled
[ 1.799056] pci 0000:00:03.0: PCI bridge to [bus 02-02]
[ 1.804263] pci 0000:00:03.0: bridge window [mem 0xda000000-0xddffffff]
[ 1.804305] pci 0000:03:00.0: [1000:0060] type 0 class 0x000104
[ 1.804320] pci 0000:03:00.0: reg 10: [mem 0xdf180000-0xdf1bffff 64bit]
[ 1.804328] pci 0000:03:00.0: reg 18: [io 0xfc00-0xfcff]
[ 1.804340] pci 0000:03:00.0: reg 1c: [mem 0xdf1c0000-0xdf1fffff 64bit]
[ 1.804355] pci 0000:03:00.0: reg 30: [mem 0xdf100000-0xdf107fff pref]
[ 1.804396] pci 0000:03:00.0: supports D1
[ 1.804414] pci 0000:00:04.0: PCI bridge to [bus 03-03]
[ 1.809618] pci 0000:00:04.0: bridge window [io 0xf000-0xffff]
[ 1.809621] pci 0000:00:04.0: bridge window [mem 0xdf100000-0xdf1fffff]
[ 1.809655] pci 0000:00:05.0: PCI bridge to [bus 04-04]
[ 1.814893] pci 0000:00:06.0: PCI bridge to [bus 05-05]
[ 1.820129] pci 0000:00:07.0: PCI bridge to [bus 06-06]
[ 1.825366] pci 0000:00:09.0: PCI bridge to [bus 07-07]
[ 1.830604] pci 0000:08:03.0: [102b:0532] type 0 class 0x000300
[ 1.830619] pci 0000:08:03.0: reg 10: [mem 0xd5800000-0xd5ffffff pref]
[ 1.830628] pci 0000:08:03.0: reg 14: [mem 0xde7fc000-0xde7fffff]
[ 1.830637] pci 0000:08:03.0: reg 18: [mem 0xde800000-0xdeffffff]
[ 1.830668] pci 0000:08:03.0: reg 30: [mem 0x00000000-0x0000ffff pref]
[ 1.830729] pci 0000:00:1e.0: PCI bridge to [bus 08-08] (subtractive decode)
[ 1.837753] pci 0000:00:1e.0: bridge window [mem 0xde000000-0xdeffffff]
[ 1.837758] pci 0000:00:1e.0: bridge window [mem 0xd5800000-0xd5ffffff 64bit pref]
[ 1.837763] pci 0000:00:1e.0: bridge window [io 0x0000-0x0cf7] (subtractive decode)
[ 1.837765] pci 0000:00:1e.0: bridge window [io 0x0d00-0xffff] (subtractive decode)
[ 1.837767] pci 0000:00:1e.0: bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
[ 1.837770] pci 0000:00:1e.0: bridge window [mem 0xd0000000-0xfdffffff] (subtractive decode)
[ 1.837772] pci 0000:00:1e.0: bridge window [mem 0xfed40000-0xfed44fff] (subtractive decode)
[ 1.837797] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[ 1.838172] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX1._PRT]
[ 1.838236] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX3._PRT]
[ 1.838299] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX4._PRT]
[ 1.838362] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX5._PRT]
[ 1.838425] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX6._PRT]
[ 1.838487] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX7._PRT]
[ 1.838552] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX9._PRT]
[ 1.838619] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.COMP._PRT]
[ 1.838751] pci0000:00: Requesting ACPI _OSC control (0x1d)
[ 1.844579] pci0000:00: ACPI _OSC control (0x1d) granted
[ 1.852312] ACPI: PCI Interrupt Link [LK00] (IRQs 3 4 5 6 7 10 11 14 *15)
[ 1.859222] ACPI: PCI Interrupt Link [LK01] (IRQs 3 4 5 6 7 10 11 *14 15)
[ 1.866128] ACPI: PCI Interrupt Link [LK02] (IRQs 3 4 5 6 7 10 *11 14 15)
[ 1.873031] ACPI: PCI Interrupt Link [LK03] (IRQs 3 4 5 6 7 *10 11 14 15)
[ 1.879935] ACPI: PCI Interrupt Link [LK04] (IRQs 3 4 *5 6 7 10 11 14 15)
[ 1.886838] ACPI: PCI Interrupt Link [LK05] (IRQs 3 4 5 *6 7 10 11 14 15)
[ 1.893742] ACPI: PCI Interrupt Link [LK06] (IRQs 3 4 5 6 7 10 11 14 15) *0, disabled.
[ 1.901791] ACPI: PCI Interrupt Link [LK07] (IRQs 3 4 5 6 7 10 11 *14 15)
[ 1.908746] vgaarb: device added: PCI:0000:08:03.0,decodes=io+mem,owns=io+mem,locks=none
[ 1.916807] vgaarb: loaded
[ 1.919500] vgaarb: bridge control possible 0000:08:03.0
[ 1.924818] PCI: Using ACPI for IRQ routing
[ 1.934620] PCI: Discovered peer bus fe
[ 1.938463] pci 0000:fe:00.0: [8086:2c70] type 0 class 0x000600
[ 1.938481] pci 0000:fe:00.1: [8086:2d81] type 0 class 0x000600
[ 1.938500] pci 0000:fe:02.0: [8086:2d90] type 0 class 0x000600
[ 1.938516] pci 0000:fe:02.1: [8086:2d91] type 0 class 0x000600
[ 1.938532] pci 0000:fe:02.2: [8086:2d92] type 0 class 0x000600
[ 1.938548] pci 0000:fe:02.3: [8086:2d93] type 0 class 0x000600
[ 1.938564] pci 0000:fe:02.4: [8086:2d94] type 0 class 0x000600
[ 1.938580] pci 0000:fe:02.5: [8086:2d95] type 0 class 0x000600
[ 1.938598] pci 0000:fe:03.0: [8086:2d98] type 0 class 0x000600
[ 1.938614] pci 0000:fe:03.1: [8086:2d99] type 0 class 0x000600
[ 1.938630] pci 0000:fe:03.2: [8086:2d9a] type 0 class 0x000600
[ 1.938646] pci 0000:fe:03.4: [8086:2d9c] type 0 class 0x000600
[ 1.938664] pci 0000:fe:04.0: [8086:2da0] type 0 class 0x000600
[ 1.938680] pci 0000:fe:04.1: [8086:2da1] type 0 class 0x000600
[ 1.938695] pci 0000:fe:04.2: [8086:2da2] type 0 class 0x000600
[ 1.938711] pci 0000:fe:04.3: [8086:2da3] type 0 class 0x000600
[ 1.938730] pci 0000:fe:05.0: [8086:2da8] type 0 class 0x000600
[ 1.938746] pci 0000:fe:05.1: [8086:2da9] type 0 class 0x000600
[ 1.938762] pci 0000:fe:05.2: [8086:2daa] type 0 class 0x000600
[ 1.938778] pci 0000:fe:05.3: [8086:2dab] type 0 class 0x000600
[ 1.938796] pci 0000:fe:06.0: [8086:2db0] type 0 class 0x000600
[ 1.938812] pci 0000:fe:06.1: [8086:2db1] type 0 class 0x000600
[ 1.938828] pci 0000:fe:06.2: [8086:2db2] type 0 class 0x000600
[ 1.938844] pci 0000:fe:06.3: [8086:2db3] type 0 class 0x000600
[ 1.939322] PCI: Discovered peer bus ff
[ 1.943166] pci 0000:ff:00.0: [8086:2c70] type 0 class 0x000600
[ 1.943185] pci 0000:ff:00.1: [8086:2d81] type 0 class 0x000600
[ 1.943207] pci 0000:ff:02.0: [8086:2d90] type 0 class 0x000600
[ 1.943225] pci 0000:ff:02.1: [8086:2d91] type 0 class 0x000600
[ 1.943244] pci 0000:ff:02.2: [8086:2d92] type 0 class 0x000600
[ 1.943262] pci 0000:ff:02.3: [8086:2d93] type 0 class 0x000600
[ 1.943281] pci 0000:ff:02.4: [8086:2d94] type 0 class 0x000600
[ 1.943299] pci 0000:ff:02.5: [8086:2d95] type 0 class 0x000600
[ 1.943320] pci 0000:ff:03.0: [8086:2d98] type 0 class 0x000600
[ 1.943338] pci 0000:ff:03.1: [8086:2d99] type 0 class 0x000600
[ 1.943357] pci 0000:ff:03.2: [8086:2d9a] type 0 class 0x000600
[ 1.943375] pci 0000:ff:03.4: [8086:2d9c] type 0 class 0x000600
[ 1.943395] pci 0000:ff:04.0: [8086:2da0] type 0 class 0x000600
[ 1.943416] pci 0000:ff:04.1: [8086:2da1] type 0 class 0x000600
[ 1.943435] pci 0000:ff:04.2: [8086:2da2] type 0 class 0x000600
[ 1.943453] pci 0000:ff:04.3: [8086:2da3] type 0 class 0x000600
[ 1.943475] pci 0000:ff:05.0: [8086:2da8] type 0 class 0x000600
[ 1.943493] pci 0000:ff:05.1: [8086:2da9] type 0 class 0x000600
[ 1.943512] pci 0000:ff:05.2: [8086:2daa] type 0 class 0x000600
[ 1.943530] pci 0000:ff:05.3: [8086:2dab] type 0 class 0x000600
[ 1.943551] pci 0000:ff:06.0: [8086:2db0] type 0 class 0x000600
[ 1.943569] pci 0000:ff:06.1: [8086:2db1] type 0 class 0x000600
[ 1.943587] pci 0000:ff:06.2: [8086:2db2] type 0 class 0x000600
[ 1.943606] pci 0000:ff:06.3: [8086:2db3] type 0 class 0x000600
[ 1.944089] PCI: pci_cache_line_size set to 64 bytes
[ 1.944213] reserve RAM buffer: 00000000cf379000 - 00000000cfffffff
[ 1.944313] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
[ 1.949478] hpet0: 4 comparators, 64-bit 14.318180 MHz counter
[ 1.957311] Switching to clocksource hpet
[ 1.963501] pnp: PnP ACPI init
[ 1.966565] ACPI: bus type pnp registered
[ 1.970731] pnp 00:00: [bus 00-ff]
[ 1.970735] pnp 00:00: [io 0x0cf8-0x0cff]
[ 1.970737] pnp 00:00: [io 0x0000-0x0cf7 window]
[ 1.970740] pnp 00:00: [io 0x0d00-0xffff window]
[ 1.970743] pnp 00:00: [mem 0x000a0000-0x000bffff window]
[ 1.970746] pnp 00:00: [mem 0xd0000000-0xfdffffff window]
[ 1.970749] pnp 00:00: [mem 0xfed40000-0xfed44fff window]
[ 1.970813] pnp 00:00: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active)
[ 1.970872] pnp 00:01: [io 0x0080-0x009f]
[ 1.970875] pnp 00:01: [io 0x0000-0x001f]
[ 1.970878] pnp 00:01: [io 0x00c0-0x00df]
[ 1.970880] pnp 00:01: [dma 4]
[ 1.970918] pnp 00:01: Plug and Play ACPI device, IDs PNP0200 (active)
[ 1.970945] pnp 00:02: [io 0x00f0-0x00ff]
[ 1.970956] pnp 00:02: [irq 13]
[ 1.970990] pnp 00:02: Plug and Play ACPI device, IDs PNP0c04 (active)
[ 1.971018] pnp 00:03: [io 0x0061]
[ 1.971124] system 00:03: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 1.971153] pnp 00:04: [io 0x0070-0x007f]
[ 1.971160] pnp 00:04: [irq 8]
[ 1.971198] pnp 00:04: Plug and Play ACPI device, IDs PNP0b00 (active)
[ 1.971403] pnp 00:05: [io 0x03f8-0x03ff]
[ 1.971410] pnp 00:05: [irq 4]
[ 1.971538] pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
[ 1.971737] pnp 00:06: [io 0x02f8-0x02ff]
[ 1.971744] pnp 00:06: [irq 3]
[ 1.971875] pnp 00:06: Plug and Play ACPI device, IDs PNP0501 (active)
[ 1.972716] pnp 00:07: [io 0x0800-0x087f]
[ 1.972719] pnp 00:07: [io 0x00e0-0x00e7]
[ 1.972722] pnp 00:07: [io 0x0880-0x08ff]
[ 1.972725] pnp 00:07: [io 0x0900-0x091f]
[ 1.972727] pnp 00:07: [io 0x0920-0x0923]
[ 1.972729] pnp 00:07: [io 0x0924]
[ 1.972731] pnp 00:07: [io 0x0c00-0x0c7f]
[ 1.972734] pnp 00:07: [io 0x0ca0-0x0ca7]
[ 1.972736] pnp 00:07: [io 0x0ca9-0x0cab]
[ 1.972739] pnp 00:07: [io 0x0cad-0x0caf]
[ 1.972741] pnp 00:07: [io 0x0060]
[ 1.972743] pnp 00:07: [io 0x0064]
[ 1.972830] system 00:07: [io 0x0800-0x087f] has been reserved
[ 1.978734] system 00:07: [io 0x0880-0x08ff] has been reserved
[ 1.984636] system 00:07: [io 0x0900-0x091f] has been reserved
[ 1.990537] system 00:07: [io 0x0920-0x0923] has been reserved
[ 1.996438] system 00:07: [io 0x0924] has been reserved
[ 2.001732] system 00:07: [io 0x0c00-0x0c7f] has been reserved
[ 2.007632] system 00:07: [io 0x0ca0-0x0ca7] has been reserved
[ 2.013533] system 00:07: [io 0x0ca9-0x0cab] has been reserved
[ 2.019433] system 00:07: [io 0x0cad-0x0caf] has been reserved
[ 2.025334] system 00:07: Plug and Play ACPI device, IDs PNP0c01 (active)
[ 2.025398] pnp 00:08: [io 0x0ca8]
[ 2.025400] pnp 00:08: [io 0x0cac]
[ 2.025403] pnp 00:08: [irq 0 disabled]
[ 2.025489] system 00:08: [io 0x0ca8] has been reserved
[ 2.030788] system 00:08: [io 0x0cac] has been reserved
[ 2.036083] system 00:08: Plug and Play ACPI device, IDs IPI0001 PNP0c01 (active)
[ 2.036755] pnp 00:09: [mem 0xe0000000-0xefffffff]
[ 2.036840] system 00:09: [mem 0xe0000000-0xefffffff] has been reserved
[ 2.043437] system 00:09: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 2.043500] pnp 00:0a: [mem 0xfed00000-0xfed003ff]
[ 2.043544] pnp 00:0a: Plug and Play ACPI device, IDs PNP0103 (active)
[ 2.043564] pnp 00:0b: [mem 0xfed90000-0xfed91fff]
[ 2.043649] system 00:0b: [mem 0xfed90000-0xfed91fff] has been reserved
[ 2.050245] system 00:0b: Plug and Play ACPI device, IDs PNP0c02 (active)
[ 2.050467] pnp: PnP ACPI: found 12 devices
[ 2.054637] ACPI: ACPI bus type pnp unregistered
[ 2.066130] PCI: max bus depth: 1 pci_try_num: 2
[ 2.066187] pci 0000:00:01.0: PCI bridge to [bus 01-01]
[ 2.071399] pci 0000:00:01.0: bridge window [mem 0xd6000000-0xd9ffffff]
[ 2.078168] pci 0000:00:03.0: PCI bridge to [bus 02-02]
[ 2.083376] pci 0000:00:03.0: bridge window [mem 0xda000000-0xddffffff]
[ 2.090146] pci 0000:00:04.0: PCI bridge to [bus 03-03]
[ 2.095354] pci 0000:00:04.0: bridge window [io 0xf000-0xffff]
[ 2.101429] pci 0000:00:04.0: bridge window [mem 0xdf100000-0xdf1fffff]
[ 2.108198] pci 0000:00:05.0: PCI bridge to [bus 04-04]
[ 2.113412] pci 0000:00:06.0: PCI bridge to [bus 05-05]
[ 2.118626] pci 0000:00:07.0: PCI bridge to [bus 06-06]
[ 2.123841] pci 0000:00:09.0: PCI bridge to [bus 07-07]
[ 2.129058] pci 0000:08:03.0: BAR 6: assigned [mem 0xde000000-0xde00ffff pref]
[ 2.136257] pci 0000:00:1e.0: PCI bridge to [bus 08-08]
[ 2.141467] pci 0000:00:1e.0: bridge window [mem 0xde000000-0xdeffffff]
[ 2.148234] pci 0000:00:1e.0: bridge window [mem 0xd5800000-0xd5ffffff 64bit pref]
[ 2.155973] pci 0000:00:01.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.162653] pci 0000:00:01.0: setting latency timer to 64
[ 2.162661] pci 0000:00:03.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.169340] pci 0000:00:03.0: setting latency timer to 64
[ 2.169347] pci 0000:00:04.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.176026] pci 0000:00:04.0: setting latency timer to 64
[ 2.176033] pci 0000:00:05.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.182712] pci 0000:00:05.0: setting latency timer to 64
[ 2.182718] pci 0000:00:06.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.189398] pci 0000:00:06.0: setting latency timer to 64
[ 2.189405] pci 0000:00:07.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.196083] pci 0000:00:07.0: setting latency timer to 64
[ 2.196090] pci 0000:00:09.0: PCI INT A -> GSI 53 (level, low) -> IRQ 53
[ 2.202769] pci 0000:00:09.0: setting latency timer to 64
[ 2.202777] pci 0000:00:1e.0: setting latency timer to 64
[ 2.202782] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7]
[ 2.202785] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff]
[ 2.202789] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[ 2.202792] pci_bus 0000:00: resource 7 [mem 0xd0000000-0xfdffffff]
[ 2.202795] pci_bus 0000:00: resource 8 [mem 0xfed40000-0xfed44fff]
[ 2.202798] pci_bus 0000:01: resource 1 [mem 0xd6000000-0xd9ffffff]
[ 2.202801] pci_bus 0000:02: resource 1 [mem 0xda000000-0xddffffff]
[ 2.202805] pci_bus 0000:03: resource 0 [io 0xf000-0xffff]
[ 2.202808] pci_bus 0000:03: resource 1 [mem 0xdf100000-0xdf1fffff]
[ 2.202811] pci_bus 0000:08: resource 1 [mem 0xde000000-0xdeffffff]
[ 2.202815] pci_bus 0000:08: resource 2 [mem 0xd5800000-0xd5ffffff 64bit pref]
[ 2.202818] pci_bus 0000:08: resource 4 [io 0x0000-0x0cf7]
[ 2.202821] pci_bus 0000:08: resource 5 [io 0x0d00-0xffff]
[ 2.202823] pci_bus 0000:08: resource 6 [mem 0x000a0000-0x000bffff]
[ 2.202826] pci_bus 0000:08: resource 7 [mem 0xd0000000-0xfdffffff]
[ 2.202829] pci_bus 0000:08: resource 8 [mem 0xfed40000-0xfed44fff]
[ 2.202833] pci_bus 0000:fe: resource 0 [io 0x0000-0xffff]
[ 2.202836] pci_bus 0000:fe: resource 1 [mem 0x00000000-0xffffffffff]
[ 2.202840] pci_bus 0000:ff: resource 0 [io 0x0000-0xffff]
[ 2.202843] pci_bus 0000:ff: resource 1 [mem 0x00000000-0xffffffffff]
[ 2.202919] NET: Registered protocol family 2
[ 2.207765] IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
[ 2.216771] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[ 2.226270] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 2.233178] TCP: Hash tables configured (established 524288 bind 65536)
[ 2.239770] TCP reno registered
[ 2.242958] UDP hash table entries: 16384 (order: 7, 524288 bytes)
[ 2.249315] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes)
[ 2.256225] NET: Registered protocol family 1
[ 2.276780] pci 0000:08:03.0: Boot video device
[ 2.276877] PCI: CLS 64 bytes, default 64
[ 2.276933] Unpacking initramfs...
[ 2.930247] Freeing initrd memory: 22556k freed
[ 2.939236] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 2.945670] Placing 64MB software IO TLB between ffff8800cb377000 - ffff8800cf377000
[ 2.953385] software IO TLB at phys 0xcb377000 - 0xcf377000
[ 2.959827] audit: initializing netlink socket (disabled)
[ 2.965217] type=2000 audit(1328507169.580:1): initialized
[ 3.045060] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 3.053042] VFS: Disk quotas dquot_6.5.2
[ 3.057014] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 3.063543] msgmni has been set to 32768
[ 3.067669] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[ 3.075039] io scheduler noop registered
[ 3.078944] io scheduler deadline registered
[ 3.083228] io scheduler cfq registered (default)
[ 3.088069] pcieport 0000:00:01.0: setting latency timer to 64
[ 3.088104] pcieport 0000:00:01.0: irq 72 for MSI/MSI-X
[ 3.088168] pcieport 0000:00:03.0: setting latency timer to 64
[ 3.088198] pcieport 0000:00:03.0: irq 73 for MSI/MSI-X
[ 3.088256] pcieport 0000:00:04.0: setting latency timer to 64
[ 3.088285] pcieport 0000:00:04.0: irq 74 for MSI/MSI-X
[ 3.088344] pcieport 0000:00:05.0: setting latency timer to 64
[ 3.088375] pcieport 0000:00:05.0: irq 75 for MSI/MSI-X
[ 3.088433] pcieport 0000:00:06.0: setting latency timer to 64
[ 3.088462] pcieport 0000:00:06.0: irq 76 for MSI/MSI-X
[ 3.088520] pcieport 0000:00:07.0: setting latency timer to 64
[ 3.088549] pcieport 0000:00:07.0: irq 77 for MSI/MSI-X
[ 3.088612] pcieport 0000:00:09.0: setting latency timer to 64
[ 3.088641] pcieport 0000:00:09.0: irq 78 for MSI/MSI-X
[ 3.088723] aer 0000:00:01.0:pcie02: service driver aer loaded
[ 3.088743] aer 0000:00:03.0:pcie02: service driver aer loaded
[ 3.088769] aer 0000:00:04.0:pcie02: service driver aer loaded
[ 3.088788] aer 0000:00:05.0:pcie02: service driver aer loaded
[ 3.088808] aer 0000:00:06.0:pcie02: service driver aer loaded
[ 3.088827] aer 0000:00:07.0:pcie02: service driver aer loaded
[ 3.088846] aer 0000:00:09.0:pcie02: service driver aer loaded
[ 3.088862] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[ 3.095801] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 3.102304] pci 0000:01:00.1: Signaling PME through PCIe PME interrupt
[ 3.108809] pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded
[ 3.108816] pcieport 0000:00:03.0: Signaling PME through PCIe PME interrupt
[ 3.115753] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[ 3.122256] pci 0000:02:00.1: Signaling PME through PCIe PME interrupt
[ 3.128761] pcie_pme 0000:00:03.0:pcie01: service driver pcie_pme loaded
[ 3.128769] pcieport 0000:00:04.0: Signaling PME through PCIe PME interrupt
[ 3.135704] pci 0000:03:00.0: Signaling PME through PCIe PME interrupt
[ 3.142208] pcie_pme 0000:00:04.0:pcie01: service driver pcie_pme loaded
[ 3.142215] pcieport 0000:00:05.0: Signaling PME through PCIe PME interrupt
[ 3.149152] pcie_pme 0000:00:05.0:pcie01: service driver pcie_pme loaded
[ 3.149159] pcieport 0000:00:06.0: Signaling PME through PCIe PME interrupt
[ 3.156097] pcie_pme 0000:00:06.0:pcie01: service driver pcie_pme loaded
[ 3.156104] pcieport 0000:00:07.0: Signaling PME through PCIe PME interrupt
[ 3.163042] pcie_pme 0000:00:07.0:pcie01: service driver pcie_pme loaded
[ 3.163050] pcieport 0000:00:09.0: Signaling PME through PCIe PME interrupt
[ 3.169987] pcie_pme 0000:00:09.0:pcie01: service driver pcie_pme loaded
[ 3.170109] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 3.196694] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 3.279309] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[ 3.475087] 00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 3.538916] 00:06: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[ 3.614618] Linux agpgart interface v0.103
[ 3.618894] i8042: PNP: No PS/2 controller found. Probing ports directly.
[ 3.628346] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 3.633311] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 3.638491] mousedev: PS/2 mouse device common for all mice
[ 3.644096] rtc_cmos 00:04: RTC can wake from S4
[ 3.648808] rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0
[ 3.654920] rtc0: alarms up to one day, y3k, 242 bytes nvram, hpet irqs
[ 3.661524] cpuidle: using governor ladder
[ 3.666002] cpuidle: using governor menu
[ 3.670201] TCP cubic registered
[ 3.673600] NET: Registered protocol family 10
[ 3.678818] Mobile IPv6
[ 3.681253] NET: Registered protocol family 17
[ 3.685687] Registering the dns_resolver key type
[ 3.690744] PM: Hibernation image not present or could not be loaded.
[ 3.690757] registered taskstats version 1
[ 3.695450] rtc_cmos 00:04: setting system clock to 2012-02-06 05:46:11 UTC (1328507171)
[ 3.703558] Initializing network drop monitor service
[ 3.710123] Freeing unused kernel memory: 564k freed
[ 3.715210] Write protecting the kernel read-only data: 6144k
[ 3.724058] Freeing unused kernel memory: 752k freed
[ 3.730832] Freeing unused kernel memory: 380k freed
[ 3.775479] udevd (107): /proc/107/oom_adj is deprecated, please use /proc/107/oom_score_adj instead.
[ 3.836510] bnx2: Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.1.11 (July 20, 2011)
[ 3.844997] bnx2 0000:01:00.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
[ 3.845343] SCSI subsystem initialized
[ 3.846579] megasas: 00.00.06.12-rc1 Wed. Oct. 5 17:00:00 PDT 2011
[ 3.846596] megasas: 0x1000:0x0060:0x1028:0x1f0c: bus 3:slot 0:func 0
[ 3.846616] megaraid_sas 0000:03:00.0: PCI INT A -> GSI 33 (level, low) -> IRQ 33
[ 3.846621] megaraid_sas 0000:03:00.0: setting latency timer to 64
[ 3.855824] usbcore: registered new interface driver usbfs
[ 3.855846] usbcore: registered new interface driver hub
[ 3.855882] usbcore: registered new device driver usb
[ 3.856593] uhci_hcd: USB Universal Host Controller Interface driver
[ 3.856631] uhci_hcd 0000:00:1a.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17
[ 3.856639] uhci_hcd 0000:00:1a.0: setting latency timer to 64
[ 3.856642] uhci_hcd 0000:00:1a.0: UHCI Host Controller
[ 3.856664] uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1
[ 3.856708] uhci_hcd 0000:00:1a.0: irq 17, io base 0x0000ec40
[ 3.856747] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.856749] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.856751] usb usb1: Product: UHCI Host Controller
[ 3.856753] usb usb1: Manufacturer: Linux 3.2.4 uhci_hcd
[ 3.856755] usb usb1: SerialNumber: 0000:00:1a.0
[ 3.856773] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 3.856776] Warning! ehci_hcd should always be loaded before uhci_hcd and ohci_hcd, not after
[ 3.856861] hub 1-0:1.0: USB hub found
[ 3.856865] hub 1-0:1.0: 2 ports detected
[ 3.856975] uhci_hcd 0000:00:1a.1: PCI INT B -> GSI 18 (level, low) -> IRQ 18
[ 3.856982] uhci_hcd 0000:00:1a.1: setting latency timer to 64
[ 3.856986] uhci_hcd 0000:00:1a.1: UHCI Host Controller
[ 3.856993] uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 2
[ 3.857029] uhci_hcd 0000:00:1a.1: irq 18, io base 0x0000ec60
[ 3.857061] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.857064] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.857066] usb usb2: Product: UHCI Host Controller
[ 3.857068] usb usb2: Manufacturer: Linux 3.2.4 uhci_hcd
[ 3.857069] usb usb2: SerialNumber: 0000:00:1a.1
[ 3.857157] hub 2-0:1.0: USB hub found
[ 3.857161] hub 2-0:1.0: 2 ports detected
[ 3.857244] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[ 3.857250] uhci_hcd 0000:00:1d.0: setting latency timer to 64
[ 3.857253] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[ 3.857260] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 3
[ 3.857299] uhci_hcd 0000:00:1d.0: irq 21, io base 0x0000ec80
[ 3.857333] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.857336] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.857338] usb usb3: Product: UHCI Host Controller
[ 3.857340] usb usb3: Manufacturer: Linux 3.2.4 uhci_hcd
[ 3.857341] usb usb3: SerialNumber: 0000:00:1d.0
[ 3.857432] hub 3-0:1.0: USB hub found
[ 3.857436] hub 3-0:1.0: 2 ports detected
[ 3.857525] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 20 (level, low) -> IRQ 20
[ 3.857532] uhci_hcd 0000:00:1d.1: setting latency timer to 64
[ 3.857535] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[ 3.857541] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 4
[ 3.857581] uhci_hcd 0000:00:1d.1: irq 20, io base 0x0000eca0
[ 3.857612] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.857614] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.857616] usb usb4: Product: UHCI Host Controller
[ 3.857618] usb usb4: Manufacturer: Linux 3.2.4 uhci_hcd
[ 3.857620] usb usb4: SerialNumber: 0000:00:1d.1
[ 3.857725] hub 4-0:1.0: USB hub found
[ 3.857728] hub 4-0:1.0: 2 ports detected
[ 3.857832] ehci_hcd 0000:00:1a.7: PCI INT C -> GSI 19 (level, low) -> IRQ 19
[ 4.166843] megasas: FW now in Ready state
[ 4.166857] ehci_hcd 0000:00:1a.7: setting latency timer to 64
[ 4.166861] ehci_hcd 0000:00:1a.7: EHCI Host Controller
[ 4.166874] Refined TSC clocksource calibration: 2393.999 MHz.
[ 4.166879] Switching to clocksource tsc
[ 4.166882] ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 5
[ 4.193219] usb 2-1: new full-speed USB device number 2 using uhci_hcd
[ 4.193222] bnx2 0000:01:00.0: setting latency timer to 64
[ 4.193243] ehci_hcd 0000:00:1a.7: debug port 1
[ 4.194138] bnx2 0000:01:00.0: eth0: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem d6000000, IRQ 36, node addr 78:2b:cb:1d:03:7a
[ 4.194175] bnx2 0000:01:00.1: PCI INT B -> GSI 48 (level, low) -> IRQ 48
[ 4.194181] bnx2 0000:01:00.1: setting latency timer to 64
[ 4.195084] bnx2 0000:01:00.1: eth1: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem d8000000, IRQ 48, node addr 78:2b:cb:1d:03:7c
[ 4.195116] bnx2 0000:02:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
[ 4.195122] bnx2 0000:02:00.0: setting latency timer to 64
[ 4.196062] bnx2 0000:02:00.0: eth2: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem da000000, IRQ 32, node addr 78:2b:cb:1d:03:7e
[ 4.196092] bnx2 0000:02:00.1: PCI INT B -> GSI 42 (level, low) -> IRQ 42
[ 4.196098] bnx2 0000:02:00.1: setting latency timer to 64
[ 4.197067] bnx2 0000:02:00.1: eth3: Broadcom NetXtreme II BCM5709 1000Base-T (C0) PCI Express found at mem dc000000, IRQ 42, node addr 78:2b:cb:1d:03:80
[ 4.197137] ehci_hcd 0000:00:1a.7: cache line size of 64 is not supported
[ 4.197155] ehci_hcd 0000:00:1a.7: irq 19, io mem 0xdf0fe000
[ 4.209353] ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00
[ 4.209374] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
[ 4.209377] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 4.209380] usb usb5: Product: EHCI Host Controller
[ 4.209382] usb usb5: Manufacturer: Linux 3.2.4 ehci_hcd
[ 4.209384] usb usb5: SerialNumber: 0000:00:1a.7
[ 4.209501] hub 5-0:1.0: USB hub found
[ 4.209506] hub 5-0:1.0: 4 ports detected
[ 4.209610] ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[ 4.241296] megasas_init_mfi: fw_support_ieee=0
[ 4.241298] megasas: INIT adapter done
[ 4.313195] scsi0 : LSI SAS based MegaRAID driver
[ 4.314270] scsi 0:0:0:0: Direct-Access FUJITSU MBA3147RC D306 PQ: 0 ANSI: 5
[ 4.314972] scsi 0:0:1:0: Direct-Access SEAGATE ST3300657SS-H EH02 PQ: 0 ANSI: 5
[ 4.363115] ehci_hcd 0000:00:1d.7: setting latency timer to 64
[ 4.363120] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[ 4.368361] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 6
[ 4.375817] ehci_hcd 0000:00:1d.7: debug port 1
[ 4.384218] ehci_hcd 0000:00:1d.7: cache line size of 64 is not supported
[ 4.384226] ehci_hcd 0000:00:1d.7: irq 21, io mem 0xdf0ff000
[ 4.417777] scsi 0:0:32:0: Enclosure DP BACKPLANE 1.07 PQ: 0 ANSI: 5
[ 4.429009] hub 2-0:1.0: unable to enumerate USB device on port 1
[ 4.440953] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
[ 4.442262] scsi 0:2:0:0: Direct-Access DELL PERC 6/i 1.22 PQ: 0 ANSI: 5
[ 4.454754] usb usb6: New USB device found, idVendor=1d6b, idProduct=0002
[ 4.461520] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 4.463645] sd 0:2:0:0: [sda] 285474816 512-byte logical blocks: (146 GB/136 GiB)
[ 4.463726] sd 0:2:0:0: [sda] Write Protect is off
[ 4.463729] sd 0:2:0:0: [sda] Mode Sense: 1f 00 10 08
[ 4.463781] sd 0:2:0:0: [sda] Write cache: disabled, read cache: enabled, supports DPO and FUA
[ 4.464316] sda: sda1 sda2
[ 4.492286] usb usb6: Product: EHCI Host Controller
[ 4.492563] sd 0:2:0:0: [sda] Attached SCSI disk
[ 4.501770] usb usb6: Manufacturer: Linux 3.2.4 ehci_hcd
[ 4.507062] usb usb6: SerialNumber: 0000:00:1d.7
[ 4.511778] hub 6-0:1.0: USB hub found
[ 4.515519] hub 6-0:1.0: 4 ports detected
[ 4.654928] ses 0:0:32:0: Attached Enclosure device
[ 4.672559] usb 5-3: new high-speed USB device number 2 using ehci_hcd
[ 4.808646] usb 5-3: New USB device found, idVendor=0424, idProduct=2514
[ 4.815324] usb 5-3: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[ 4.822657] hub 5-3:1.0: USB hub found
[ 4.826487] hub 5-3:1.0: 2 ports detected
[ 5.235589] usb 3-2: new full-speed USB device number 2 using uhci_hcd
[ 5.406665] usb 3-2: New USB device found, idVendor=0624, idProduct=0248
[ 5.413350] usb 3-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 5.420463] usb 3-2: Product: USB Composite Device-0
[ 5.425410] usb 3-2: Manufacturer: Avocent
[ 5.429489] usb 3-2: SerialNumber: 20080519
[ 5.452802] input: Avocent USB Composite Device-0 as /devices/pci0000:00/0000:00:1d.0/usb3/3-2/3-2:1.0/input/input0
[ 5.463268] generic-usb 0003:0624:0248.0001: input,hidraw0: USB HID v1.00 Keyboard [Avocent USB Composite Device-0] on usb-0000:00:1d.0-2/input0
[ 5.479700] input: Avocent USB Composite Device-0 as /devices/pci0000:00/0000:00:1d.0/usb3/3-2/3-2:1.1/input/input1
[ 5.490185] generic-usb 0003:0624:0248.0002: input,hidraw1: USB HID v1.00 Mouse [Avocent USB Composite Device-0] on usb-0000:00:1d.0-2/input1
[ 5.502856] usbcore: registered new interface driver usbhid
[ 5.508407] usbhid: USB HID core driver
[ 34.697445] device-mapper: uevent: version 1.0.3
[ 34.702152] device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com
[ 34.758381] PM: Starting manual resume from disk
[ 34.763106] PM: Hibernation image partition 254:1 present
[ 34.763109] PM: Looking for hibernation image.
[ 34.768272] PM: Image not found (code -22)
[ 34.768276] PM: Hibernation image not present or could not be loaded.
[ 34.793890] kjournald starting. Commit interval 5 seconds
[ 34.793929] EXT3-fs (dm-0): mounted filesystem with ordered data mode
[ 35.597021] udevd version 125 started
[ 35.800449] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input2
[ 35.808097] ACPI: Power Button [PWRF]
[ 35.849871] Monitor-Mwait will be used to enter C-1 state
[ 35.908621] Monitor-Mwait will be used to enter C-3 state
[ 35.908644] ACPI: acpi_idle registered with cpuidle
[ 35.931331] dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
[ 36.109588] input: PC Speaker as /devices/platform/pcspkr/input/input3
[ 37.139584] EXT3-fs (dm-0): using internal journal
[ 38.277428] Fusion MPT base driver 3.04.20
[ 38.281513] Copyright (c) 1999-2008 LSI Corporation
[ 38.298307] Fusion MPT misc device (ioctl) driver 3.04.20
[ 38.303871] mptctl: Registered with Fusion MPT base driver
[ 38.309343] mptctl: /dev/mptctl @ (major,minor=10,220)
[ 38.704680] Adding 1048572k swap on /dev/mapper/fe12-swap. Priority:-1 extents:1 across:1048572k
[ 39.207877] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 218.384839] bonding: Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
[ 218.395734] bonding: bond1 is being created...
[ 218.401411] bonding: bond1: setting mode to active-backup (1).
[ 218.407296] bonding: bond1: Setting MII monitoring interval to 100.
[ 218.413624] bonding: bond1: adding ARP target 173.203.140.193.
[ 218.419492] bonding: bond1: Setting ARP monitoring interval to 200.
[ 218.425744] bonding: bond1: ARP monitoring cannot be used with MII monitoring. bond1 Disabling MII monitoring.
[ 218.439763] ADDRCONF(NETDEV_UP): bond1: link is not ready
[ 218.495728] bnx2 0000:01:00.1: irq 79 for MSI/MSI-X
[ 218.495740] bnx2 0000:01:00.1: irq 80 for MSI/MSI-X
[ 218.495749] bnx2 0000:01:00.1: irq 81 for MSI/MSI-X
[ 218.495758] bnx2 0000:01:00.1: irq 82 for MSI/MSI-X
[ 218.495767] bnx2 0000:01:00.1: irq 83 for MSI/MSI-X
[ 218.495808] bnx2 0000:01:00.1: irq 84 for MSI/MSI-X
[ 218.495816] bnx2 0000:01:00.1: irq 85 for MSI/MSI-X
[ 218.495824] bnx2 0000:01:00.1: irq 86 for MSI/MSI-X
[ 218.561751] bnx2 0000:01:00.1: eth1: using MSIX
[ 218.568099] bonding: bond1: making interface eth1 the new active one.
[ 218.574545] bonding: bond1: first active interface up!
[ 218.579678] bonding: bond1: enslaving eth1 as an active interface with an up link.
[ 218.588930] ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
[ 218.617861] bnx2 0000:02:00.1: irq 87 for MSI/MSI-X
[ 218.617871] bnx2 0000:02:00.1: irq 88 for MSI/MSI-X
[ 218.617879] bnx2 0000:02:00.1: irq 89 for MSI/MSI-X
[ 218.617888] bnx2 0000:02:00.1: irq 90 for MSI/MSI-X
[ 218.617896] bnx2 0000:02:00.1: irq 91 for MSI/MSI-X
[ 218.617904] bnx2 0000:02:00.1: irq 92 for MSI/MSI-X
[ 218.617912] bnx2 0000:02:00.1: irq 93 for MSI/MSI-X
[ 218.617920] bnx2 0000:02:00.1: irq 94 for MSI/MSI-X
[ 218.681413] bnx2 0000:02:00.1: eth3: using MSIX
[ 218.687817] bonding: bond1: enslaving eth3 as a backup interface with an up link.
[ 219.035049] bonding: bond1: link status definitely down for interface eth1, disabling it
[ 219.043117] bonding: bond1: making interface eth3 the new active one.
[ 219.446337] bonding: bond1: link status definitely down for interface eth3, disabling it
[ 219.454411] bonding: bond1: now running without any active interface !
[ 221.745808] bnx2 0000:01:00.1: eth1: NIC Copper Link is Up, 1000 Mbps full duplex
[ 221.854188] bonding: bond1: link status definitely up for interface eth1.
[ 221.860956] bonding: bond1: making interface eth1 the new active one.
[ 221.867434] bonding: bond1: first active interface up!
[ 221.946233] bnx2 0000:02:00.1: eth3: NIC Copper Link is Up, 1000 Mbps full duplex
[ 222.269458] bonding: bond1: link status definitely up for interface eth3.
[ 228.858097] bond1: no IPv6 routers present
[ 250.828944] bonding: bond0: setting mode to active-backup (1).
[ 250.834834] bonding: bond0: Setting MII monitoring interval to 100.
[ 250.841208] bonding: bond0: adding ARP target 172.17.0.254.
[ 250.846863] bonding: bond0: Setting ARP monitoring interval to 200.
[ 250.853166] bonding: bond0: ARP monitoring cannot be used with MII monitoring. bond0 Disabling MII monitoring.
[ 250.867158] ADDRCONF(NETDEV_UP): bond0: link is not ready
[ 250.900258] bnx2 0000:01:00.0: irq 95 for MSI/MSI-X
[ 250.900268] bnx2 0000:01:00.0: irq 96 for MSI/MSI-X
[ 250.900276] bnx2 0000:01:00.0: irq 97 for MSI/MSI-X
[ 250.900286] bnx2 0000:01:00.0: irq 98 for MSI/MSI-X
[ 250.900294] bnx2 0000:01:00.0: irq 99 for MSI/MSI-X
[ 250.900303] bnx2 0000:01:00.0: irq 100 for MSI/MSI-X
[ 250.900311] bnx2 0000:01:00.0: irq 101 for MSI/MSI-X
[ 250.900320] bnx2 0000:01:00.0: irq 102 for MSI/MSI-X
[ 250.965758] bnx2 0000:01:00.0: eth0: using MSIX
[ 250.972100] bonding: bond0: making interface eth0 the new active one.
[ 250.978564] bonding: bond0: first active interface up!
[ 250.983748] bonding: bond0: enslaving eth0 as an active interface with an up link.
[ 250.995335] ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
[ 251.026798] bnx2 0000:02:00.0: irq 103 for MSI/MSI-X
[ 251.026807] bnx2 0000:02:00.0: irq 104 for MSI/MSI-X
[ 251.026816] bnx2 0000:02:00.0: irq 105 for MSI/MSI-X
[ 251.026824] bnx2 0000:02:00.0: irq 106 for MSI/MSI-X
[ 251.026832] bnx2 0000:02:00.0: irq 107 for MSI/MSI-X
[ 251.026840] bnx2 0000:02:00.0: irq 108 for MSI/MSI-X
[ 251.026848] bnx2 0000:02:00.0: irq 109 for MSI/MSI-X
[ 251.026856] bnx2 0000:02:00.0: irq 110 for MSI/MSI-X
[ 251.089519] bnx2 0000:02:00.0: eth2: using MSIX
[ 251.095854] bonding: bond0: enslaving eth2 as a backup interface with an up link.
[ 251.463134] bonding: bond0: link status definitely down for interface eth0, disabling it
[ 251.471203] bonding: bond0: making interface eth2 the new active one.
[ 251.874425] bonding: bond0: link status definitely down for interface eth2, disabling it
[ 251.882509] bonding: bond0: now running without any active interface !
[ 254.154499] bnx2 0000:01:00.0: eth0: NIC Copper Link is Up, 1000 Mbps full duplex
[ 254.237937] bnx2 0000:02:00.0: eth2: NIC Copper Link is Up, 1000 Mbps full duplex
[ 254.282279] bonding: bond0: link status definitely up for interface eth0.
[ 254.289099] bonding: bond0: making interface eth0 the new active one.
[ 254.295626] bonding: bond0: first active interface up!
[ 254.497893] bonding: bond0: link status definitely up for interface eth2.
[ 257.175651] ipmi message handler version 39.2
[ 257.198538] IPMI System Interface driver.
[ 257.202631] ipmi_si: probing via SMBIOS
[ 257.206508] ipmi_si: SMBIOS: io 0xca8 regsize 1 spacing 4 irq 0
[ 257.212471] ipmi_si: Adding SMBIOS-specified kcs state machine
[ 257.218350] ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca8, slave address 0x20, irq 0
[ 257.449812] ipmi_si ipmi_si.0: Found new BMC (man_id: 0x0002a2, prod_id: 0x0100, dev_id: 0x20)
[ 257.458471] ipmi_si ipmi_si.0: IPMI kcs interface initialized
[ 257.480858] ipmi device interface
[ 261.350006] megaraid_sas 0000:03:00.0: vpd r/w failed. This is likely a firmware bug on this device. Contact the card vendor for a firmware update.
[ 261.409982] bond0: no IPv6 routers present
[ 271.064164] hrtimer: interrupt took 117467 ns
[ 391.688175] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-02-06 6:51 Inconsistent load average on tickless kernels Aman Gupta
@ 2012-02-23 15:46 ` Lesław Kopeć
2012-02-29 12:06 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Lesław Kopeć @ 2012-02-23 15:46 UTC (permalink / raw)
To: Aman Gupta
Cc: linux-kernel, Peter Zijlstra, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
[-- Attachment #1.1: Type: text/plain, Size: 2783 bytes --]
On 02/06/2012 07:51 AM, Aman Gupta wrote:
> I have an LVS/DR cluster of 10 machines that receive similar traffic
> via a round-robin strategy. These machines run Debian Lenny with
> 2.6.26, and consistently have a 15-minute load average between 4-12
> depending on the time of day.
>
> Upgrading any one of these machines to a newer kernel compiled with
> NO_HZ=y causes the reported load average to drop significantly. [...]
I can confirm Aman's results on kernels 2.6.32 and higher on a similar
setup. I did a test on a cluster of diskless PHP workers. Servers were
running on identical hardware and software platform. The workload should
have been the same. However load average was reporting different values
depending on which kernel the host was running.
I have tested the following vanilla kernels:
* 2.6.32.55-*
* 2.6.32.55-*-74f5187ac8 (2.6.32.55 with patch 74f5187ac8)
* 2.6.32.55-*-0f004f5a69 (2.6.32.55 with patch 74f5187ac8 and 0f004f5a69)
* 2.6.37-rc5-*-0f004f5a69 (2.6.37 at commit 0f004f5a69)
* 2.6.37-rc5-*-pre-0f004f5a69 (2.6.37 at commit 6313e3c217)
Each kernel was compiled with CONFIG_NO_HZ enabled (no-hz variant) and
disabled (hz variant). Here's a snapshot of load 15 on each kernel:
no-hz hz
2.6.32.55-* 0.59 0.57
2.6.32.55-*-74f5187ac8 3.56 11.79
2.6.32.55-*-0f004f5a69 0.61 11.76
2.6.37-rc5-*-0f004f5a69 0.67 11.65
2.6.37-rc5-*-pre-0f004f5a69 3.97 12.05
I've also uploaded load average [1] and CPU utilization [2] charts for a
visual comparison.
My observations are:
1. On tickless kernels load is very low where no or both patches
(74f5187ac8 and 0f004f5a69) are applied.
2. Kernels that have only patch 74f5187ac8 applied have the smallest
difference between hz and no-hz variants. Still no-hz kernels are
returning values lower than their hz siblings.
3. Non-tickless kernels seem to be reporting correct load values.
Overall trend and values are matching CPU utilization. Only exception is
2.6.32.55-hz which reports the same values as 2.6.32.55-no-hz.
4. If x processes are using all available cycles load is correctly
incremented by x. This behavior is consistent on all kernels.
Steps to reproduce: run a bunch of CPU bound processes that will not use
all available cycles. The biggest difference between expected and
measured load is around 30% CPU utilization in my case.
Has there been any other patches that correct load calculation? Maybe
I'm testing it in a wrong way? I'd appreciate any suggestions. I'd be
happy to test new patches. Sadly, I cannot propose any fixes as kernel
sources are still a mystery to me.
[1] http://img841.imageshack.us/img841/2204/kernelload.png
[2] http://img854.imageshack.us/img854/8194/kernelcpu.png
--
Lesław Kopeć
[-- Attachment #1.2: dmesg-2.6.32.55.log --]
[-- Type: text/plain, Size: 82738 bytes --]
[ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Linux version 2.6.32.55-no-hz (root@playground) (gcc version 4.4.5 (Debian 4.4.5-8) ) #7 SMP Fri Feb 3 11:59:46 CET 2012
[ 0.000000] Command line: BOOT_IMAGE=vmlinuz-2.6.32.55-no-hz ro initrd=initrd.img-2.6.32.55-no-hz quiet root=/dev/nfs ip=dhcp boot=nfs nfsroot=192.168.0.2:/srv/ltsp/php-kernel-test/,nocto,retrans=10
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Centaur CentaurHauls
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 0000000000097c00 (usable)
[ 0.000000] BIOS-e820: 0000000000097c00 - 00000000000a0000 (reserved)
[ 0.000000] BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
[ 0.000000] BIOS-e820: 0000000000100000 - 00000000e762f000 (usable)
[ 0.000000] BIOS-e820: 00000000e762f000 - 00000000e763c000 (ACPI data)
[ 0.000000] BIOS-e820: 00000000e763c000 - 00000000e763d000 (usable)
[ 0.000000] BIOS-e820: 00000000e763d000 - 00000000ec000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fec00000 - 00000000fee10000 (reserved)
[ 0.000000] BIOS-e820: 00000000ff800000 - 0000000100000000 (reserved)
[ 0.000000] BIOS-e820: 0000000100000000 - 0000000317fff000 (usable)
[ 0.000000] DMI 2.7 present.
[ 0.000000] last_pfn = 0x317fff max_arch_pfn = 0x400000000
[ 0.000000] MTRR default type: write-back
[ 0.000000] MTRR fixed ranges enabled:
[ 0.000000] 00000-9FFFF write-back
[ 0.000000] A0000-BFFFF uncachable
[ 0.000000] C0000-FFFFF write-protect
[ 0.000000] MTRR variable ranges enabled:
[ 0.000000] 0 base 00E8000000 mask FFF8000000 uncachable
[ 0.000000] 1 base 00F0000000 mask FFF0000000 uncachable
[ 0.000000] 2 disabled
[ 0.000000] 3 disabled
[ 0.000000] 4 disabled
[ 0.000000] 5 disabled
[ 0.000000] 6 disabled
[ 0.000000] 7 disabled
[ 0.000000] 8 disabled
[ 0.000000] 9 disabled
[ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[ 0.000000] last_pfn = 0xe763d max_arch_pfn = 0x400000000
[ 0.000000] initial memory mapped : 0 - 20000000
[ 0.000000] Using GB pages for direct mapping
[ 0.000000] init_memory_mapping: 0000000000000000-00000000e763d000
[ 0.000000] 0000000000 - 00c0000000 page 1G
[ 0.000000] 00c0000000 - 00e7600000 page 2M
[ 0.000000] 00e7600000 - 00e763d000 page 4k
[ 0.000000] kernel direct mapping tables up to e763d000 @ 8000-b000
[ 0.000000] init_memory_mapping: 0000000100000000-0000000317fff000
[ 0.000000] 0100000000 - 0300000000 page 1G
[ 0.000000] 0300000000 - 0317e00000 page 2M
[ 0.000000] 0317e00000 - 0317fff000 page 4k
[ 0.000000] kernel direct mapping tables up to 317fff000 @ a000-d000
[ 0.000000] RAMDISK: 7fac9000 - 7ffffef0
[ 0.000000] ACPI: RSDP 00000000000f4f00 00024 (v02 HP )
[ 0.000000] ACPI: XSDT 00000000e7630080 000AC (v01 HP ProLiant 00000002 Ò? 0000162E)
[ 0.000000] ACPI: FACP 00000000e7630180 000F4 (v03 HP ProLiant 00000002 Ò? 0000162E)
[ 0.000000] ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20090903/tbfadt-607)
[ 0.000000] ACPI Warning: Invalid length for Pm2ControlBlock: 32, using default 8 (20090903/tbfadt-607)
[ 0.000000] ACPI: DSDT 00000000e7630280 0216E (v01 HP DSDT 00000001 INTL 20030228)
[ 0.000000] ACPI: FACS 00000000e762f100 00040
[ 0.000000] ACPI: SPCR 00000000e762f140 00050 (v01 HP SPCRRBSU 00000001 Ò? 0000162E)
[ 0.000000] ACPI: MCFG 00000000e762f1c0 0003C (v01 HP ProLiant 00000001 00000000)
[ 0.000000] ACPI: HPET 00000000e762f200 00038 (v01 HP ProLiant 00000002 Ò? 0000162E)
[ 0.000000] ACPI: FFFF 00000000e762f240 00064 (v02 HP ProLiant 00000002 Ò? 0000162E)
[ 0.000000] ACPI: SPMI 00000000e762f2c0 00040 (v05 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: ERST 00000000e762f300 001D0 (v01 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: APIC 00000000e762f500 0015E (v01 HP ProLiant 00000002 00000000)
[ 0.000000] ACPI: SRAT 00000000e762f680 00570 (v01 HP Proliant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: FFFF 00000000e762fc00 00176 (v01 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: BERT 00000000e762fd80 00030 (v01 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: HEST 00000000e762fdc0 000BC (v01 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: DMAR 00000000e762fe80 00106 (v01 HP ProLiant 00000001 Ò? 0000162E)
[ 0.000000] ACPI: SSDT 00000000e7632400 00125 (v03 HP CRSPCI0 00000002 HP 00000001)
[ 0.000000] ACPI: SSDT 00000000e7632540 003BB (v01 HP pcc 00000001 INTL 20090625)
[ 0.000000] ACPI: SSDT 00000000e7632900 00377 (v01 HP pmab 00000001 INTL 20090625)
[ 0.000000] ACPI: SSDT 00000000e7632c80 04104 (v01 INTEL PPM RCM 00000001 INTL 20061109)
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] SRAT: PXM 0 -> APIC 0 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 1 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 2 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 3 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 4 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 5 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 16 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 17 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 18 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 19 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 20 -> Node 0
[ 0.000000] SRAT: PXM 0 -> APIC 21 -> Node 0
[ 0.000000] SRAT: PXM 1 -> APIC 32 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 33 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 34 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 35 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 36 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 37 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 48 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 49 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 50 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 51 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 52 -> Node 1
[ 0.000000] SRAT: PXM 1 -> APIC 53 -> Node 1
[ 0.000000] SRAT: Node 0 PXM 0 0-e8000000
[ 0.000000] SRAT: Node 0 PXM 0 100000000-198000000
[ 0.000000] SRAT: Node 1 PXM 1 198000000-318000000
[ 0.000000] NUMA: Allocated memnodemap from c000 - c100
[ 0.000000] NUMA: Using 27 for the hash shift.
[ 0.000000] Bootmem setup node 0 0000000000000000-0000000198000000
[ 0.000000] NODE_DATA [000000000000c100 - 00000000000140ff]
[ 0.000000] bootmap [0000000000015000 - 0000000000047fff] pages 33
[ 0.000000] (9 early reservations) ==> bootmem [0000000000 - 0198000000]
[ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
[ 0.000000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000]
[ 0.000000] #2 [0001000000 - 00016c6414] TEXT DATA BSS ==> [0001000000 - 00016c6414]
[ 0.000000] #3 [007fac9000 - 007ffffef0] RAMDISK ==> [007fac9000 - 007ffffef0]
[ 0.000000] #4 [0000097800 - 0000100000] BIOS reserved ==> [0000097800 - 0000100000]
[ 0.000000] #5 [00016c7000 - 00016c7180] BRK ==> [00016c7000 - 00016c7180]
[ 0.000000] #6 [0000008000 - 000000a000] PGTABLE ==> [0000008000 - 000000a000]
[ 0.000000] #7 [000000a000 - 000000c000] PGTABLE ==> [000000a000 - 000000c000]
[ 0.000000] #8 [000000c000 - 000000c100] MEMNODEMAP ==> [000000c000 - 000000c100]
[ 0.000000] Bootmem setup node 1 0000000198000000-0000000317fff000
[ 0.000000] NODE_DATA [0000000198000000 - 0000000198007fff]
[ 0.000000] bootmap [0000000198008000 - 0000000198037fff] pages 30
[ 0.000000] (9 early reservations) ==> bootmem [0198000000 - 0317fff000]
[ 0.000000] #0 [0000000000 - 0000001000] BIOS data page
[ 0.000000] #1 [0000006000 - 0000008000] TRAMPOLINE
[ 0.000000] #2 [0001000000 - 00016c6414] TEXT DATA BSS
[ 0.000000] #3 [007fac9000 - 007ffffef0] RAMDISK
[ 0.000000] #4 [0000097800 - 0000100000] BIOS reserved
[ 0.000000] #5 [00016c7000 - 00016c7180] BRK
[ 0.000000] #6 [0000008000 - 000000a000] PGTABLE
[ 0.000000] #7 [000000a000 - 000000c000] PGTABLE
[ 0.000000] #8 [000000c000 - 000000c100] MEMNODEMAP
[ 0.000000] found SMP MP-table at [ffff8800000f4f80] f4f80
[ 0.000000] [ffffea0005940000-ffffea00059fffff] potential offnode page_structs
[ 0.000000] [ffffea0000000000-ffffea00059fffff] PMD -> [ffff880001800000-ffff880006dfffff] on node 0
[ 0.000000] [ffffea0005a00000-ffffea000adfffff] PMD -> [ffff880198200000-ffff88019d5fffff] on node 1
[ 0.000000] Zone PFN ranges:
[ 0.000000] DMA 0x00000000 -> 0x00001000
[ 0.000000] DMA32 0x00001000 -> 0x00100000
[ 0.000000] Normal 0x00100000 -> 0x00317fff
[ 0.000000] Movable zone start PFN for each node
[ 0.000000] early_node_map[5] active PFN ranges
[ 0.000000] 0: 0x00000000 -> 0x00000097
[ 0.000000] 0: 0x00000100 -> 0x000e762f
[ 0.000000] 0: 0x000e763c -> 0x000e763d
[ 0.000000] 0: 0x00100000 -> 0x00198000
[ 0.000000] 1: 0x00198000 -> 0x00317fff
[ 0.000000] On node 0 totalpages: 1570247
[ 0.000000] DMA zone: 56 pages used for memmap
[ 0.000000] DMA zone: 111 pages reserved
[ 0.000000] DMA zone: 3824 pages, LIFO batch:0
[ 0.000000] DMA32 zone: 14280 pages used for memmap
[ 0.000000] DMA32 zone: 929384 pages, LIFO batch:31
[ 0.000000] Normal zone: 8512 pages used for memmap
[ 0.000000] Normal zone: 614080 pages, LIFO batch:31
[ 0.000000] On node 1 totalpages: 1572863
[ 0.000000] Normal zone: 21504 pages used for memmap
[ 0.000000] Normal zone: 1551359 pages, LIFO batch:31
[ 0.000000] ACPI: PM-Timer IO Port: 0x908
[ 0.000000] ACPI: Local APIC address 0xfee00000
[ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x20] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x10] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x30] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x24] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x14] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x34] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x04] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x22] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x12] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x32] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x02] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x26] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x16] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x36] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x06] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x11] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x31] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x25] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x15] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x35] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x05] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x23] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x13] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x33] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x03] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x27] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x17] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x37] disabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x07] disabled)
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
[ 0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec80000] gsi_base[24])
[ 0.000000] IOAPIC[1]: apic_id 0, version 32, address 0xfec80000, GSI 24-47
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: IRQ0 used by override.
[ 0.000000] ACPI: IRQ2 used by override.
[ 0.000000] ACPI: IRQ9 used by override.
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[ 0.000000] SMP: Allowing 32 CPUs, 8 hotplug CPUs
[ 0.000000] nr_irqs_gsi: 48
[ 0.000000] PM: Registered nosave memory: 0000000000097000 - 0000000000098000
[ 0.000000] PM: Registered nosave memory: 0000000000098000 - 00000000000a0000
[ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
[ 0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
[ 0.000000] PM: Registered nosave memory: 00000000e762f000 - 00000000e763c000
[ 0.000000] PM: Registered nosave memory: 00000000e763d000 - 00000000ec000000
[ 0.000000] PM: Registered nosave memory: 00000000ec000000 - 00000000fec00000
[ 0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fee10000
[ 0.000000] PM: Registered nosave memory: 00000000fee10000 - 00000000ff800000
[ 0.000000] PM: Registered nosave memory: 00000000ff800000 - 0000000100000000
[ 0.000000] Allocating PCI resources starting at ec000000 (gap: ec000000:12c00000)
[ 0.000000] Booting paravirtualized kernel on bare hardware
[ 0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:32 nr_node_ids:2
[ 0.000000] PERCPU: Embedded 30 pages/cpu @ffff880007000000 s90328 r8192 d24360 u131072
[ 0.000000] pcpu-alloc: s90328 r8192 d24360 u131072 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 00 02 04 06 08 10 12 14 16 18 20 22 25 27 29 31
[ 0.000000] pcpu-alloc: [1] 01 03 05 07 09 11 13 15 17 19 21 23 24 26 28 30
[ 0.000000] Built 2 zonelists in Zone order, mobility grouping on. Total pages: 3098647
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: BOOT_IMAGE=vmlinuz-2.6.32.55-no-hz ro initrd=initrd.img-2.6.32.55-no-hz quiet root=/dev/nfs ip=dhcp boot=nfs nfsroot=192.168.0.2:/srv/ltsp/php-kernel-test/,nocto,retrans=10
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Initializing CPU#0
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Calgary: detecting Calgary via BIOS EBDA area
[ 0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[ 0.000000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.000000] Placing 64MB software IO TLB between ffff8800071fe000 - ffff88000b1fe000
[ 0.000000] software IO TLB at phys 0x71fe000 - 0xb1fe000
[ 0.000000] Memory: 12315712k/12976124k available (3061k kernel code, 403684k absent, 256728k reserved, 2005k data, 588k init)
[ 0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=2
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] NR_IRQS:4352 nr_irqs:1072
[ 0.000000] Extended CMOS year: 2000
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [tty0] enabled
[ 0.000000] hpet clockevent registered
[ 0.000000] Fast TSC calibration using PIT
[ 0.004000] Detected 2266.240 MHz processor.
[ 0.000009] Calibrating delay loop (skipped), value calculated using timer frequency.. 4532.48 BogoMIPS (lpj=9064960)
[ 0.000027] Security Framework initialized
[ 0.000031] SELinux: Disabled at boot.
[ 0.001124] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[ 0.004531] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[ 0.006029] Mount-cache hash table entries: 256
[ 0.006200] Initializing cgroup subsys ns
[ 0.006203] Initializing cgroup subsys cpuacct
[ 0.006210] Initializing cgroup subsys devices
[ 0.006212] Initializing cgroup subsys freezer
[ 0.006213] Initializing cgroup subsys net_cls
[ 0.006233] CPU: Physical Processor ID: 1
[ 0.006234] CPU: Processor Core ID: 0
[ 0.006237] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.006238] CPU: L2 cache: 256K
[ 0.006239] CPU: L3 cache: 12288K
[ 0.006242] CPU 0/0x20 -> Node 1
[ 0.006245] mce: CPU supports 9 MCE banks
[ 0.006256] CPU0: Thermal monitoring enabled (TM1)
[ 0.006260] CPU 0 MCA banks CMCI:2 CMCI:3 CMCI:5 CMCI:6 CMCI:8
[ 0.006267] using mwait in idle threads.
[ 0.006269] Performance Events: Westmere events, Intel PMU driver.
[ 0.006273] ... version: 3
[ 0.006274] ... bit width: 48
[ 0.006275] ... generic registers: 4
[ 0.006276] ... value mask: 0000ffffffffffff
[ 0.006278] ... max period: 000000007fffffff
[ 0.006279] ... fixed-purpose events: 3
[ 0.006280] ... event mask: 000000070000000f
[ 0.007047] ACPI: Core revision 20090903
[ 0.015353] DMAR: Host address width 39
[ 0.015355] DMAR: DRHD base: 0x000000efffe000 flags: 0x1
[ 0.015363] IOMMU efffe000: ver 1:0 cap c90780106f0462 ecap f0207e
[ 0.015365] DMAR: RMRR base: 0x000000e77fc000 end: 0x000000e77fdfff
[ 0.015368] DMAR: RMRR base: 0x000000e77f5000 end: 0x000000e77fafff
[ 0.015370] DMAR: ATSR flags: 0x0
[ 0.015377] IOAPIC id 8 under DRHD base 0xefffe000
[ 0.015379] IOAPIC id 0 under DRHD base 0xefffe000
[ 0.015603] Enabled Interrupt-remapping
[ 0.015607] Setting APIC routing to physical flat
[ 0.015878] alloc irq_desc for 48 on node 1
[ 0.015880] alloc kstat_irqs on node 1
[ 0.015906] alloc irq_2_iommu on node 0
[ 0.015914] alloc irq_2_iommu on node 0
[ 0.015921] alloc irq_2_iommu on node 0
[ 0.015928] alloc irq_2_iommu on node 0
[ 0.015935] alloc irq_2_iommu on node 0
[ 0.015942] alloc irq_2_iommu on node 0
[ 0.015948] alloc irq_2_iommu on node 0
[ 0.015955] alloc irq_2_iommu on node 0
[ 0.015962] alloc irq_2_iommu on node 0
[ 0.015969] alloc irq_2_iommu on node 0
[ 0.015976] alloc irq_2_iommu on node 0
[ 0.015983] alloc irq_2_iommu on node 0
[ 0.015990] alloc irq_2_iommu on node 0
[ 0.015997] alloc irq_2_iommu on node 0
[ 0.016004] alloc irq_2_iommu on node 0
[ 0.016145] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.055754] CPU0: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.160623] Booting processor 1 APIC 0x10 ip 0x6000
[ 0.171254] Initializing CPU#1
[ 0.248321] CPU: Physical Processor ID: 0
[ 0.248322] CPU: Processor Core ID: 8
[ 0.248325] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.248327] CPU: L2 cache: 256K
[ 0.248328] CPU: L3 cache: 12288K
[ 0.248331] CPU 1/0x10 -> Node 0
[ 0.248345] CPU1: Thermal monitoring enabled (TM1)
[ 0.248349] CPU 1 MCA banks CMCI:2 CMCI:3 CMCI:5 CMCI:6 CMCI:8
[ 0.248399] CPU1: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.248405] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
[ 0.268434] Booting processor 2 APIC 0x30 ip 0x6000
[ 0.279875] Initializing CPU#2
[ 0.356103] CPU: Physical Processor ID: 1
[ 0.356104] CPU: Processor Core ID: 8
[ 0.356106] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.356107] CPU: L2 cache: 256K
[ 0.356108] CPU: L3 cache: 12288K
[ 0.356110] CPU 2/0x30 -> Node 1
[ 0.356120] CPU2: Thermal monitoring enabled (TM1)
[ 0.356123] CPU 2 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.356211] CPU2: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.356218] checking TSC synchronization [CPU#0 -> CPU#2]: passed.
[ 0.376288] Booting processor 3 APIC 0x0 ip 0x6000
[ 0.386920] Initializing CPU#3
[ 0.463888] CPU: Physical Processor ID: 0
[ 0.463889] CPU: Processor Core ID: 0
[ 0.463892] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.463894] CPU: L2 cache: 256K
[ 0.463896] CPU: L3 cache: 12288K
[ 0.463898] CPU 3/0x0 -> Node 0
[ 0.463911] CPU3: Thermal monitoring enabled (TM1)
[ 0.463915] CPU 3 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.463951] CPU3: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.463957] checking TSC synchronization [CPU#0 -> CPU#3]: passed.
[ 0.483986] Booting processor 4 APIC 0x24 ip 0x6000
[ 0.494556] Initializing CPU#4
[ 0.571670] CPU: Physical Processor ID: 1
[ 0.571671] CPU: Processor Core ID: 2
[ 0.571673] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.571675] CPU: L2 cache: 256K
[ 0.571676] CPU: L3 cache: 12288K
[ 0.571678] CPU 4/0x24 -> Node 1
[ 0.571688] CPU4: Thermal monitoring enabled (TM1)
[ 0.571690] CPU 4 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.571770] CPU4: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.571776] checking TSC synchronization [CPU#0 -> CPU#4]: passed.
[ 0.591842] Booting processor 5 APIC 0x14 ip 0x6000
[ 0.602473] Initializing CPU#5
[ 0.679456] CPU: Physical Processor ID: 0
[ 0.679457] CPU: Processor Core ID: 10
[ 0.679460] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.679462] CPU: L2 cache: 256K
[ 0.679464] CPU: L3 cache: 12288K
[ 0.679466] CPU 5/0x14 -> Node 0
[ 0.679479] CPU5: Thermal monitoring enabled (TM1)
[ 0.679483] CPU 5 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.679592] CPU5: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.679598] checking TSC synchronization [CPU#0 -> CPU#5]: passed.
[ 0.699631] Booting processor 6 APIC 0x34 ip 0x6000
[ 0.710140] Initializing CPU#6
[ 0.787238] CPU: Physical Processor ID: 1
[ 0.787239] CPU: Processor Core ID: 10
[ 0.787241] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.787242] CPU: L2 cache: 256K
[ 0.787243] CPU: L3 cache: 12288K
[ 0.787245] CPU 6/0x34 -> Node 1
[ 0.787255] CPU6: Thermal monitoring enabled (TM1)
[ 0.787258] CPU 6 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.787308] CPU6: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.787314] checking TSC synchronization [CPU#0 -> CPU#6]: passed.
[ 0.807384] Booting processor 7 APIC 0x4 ip 0x6000
[ 0.818553] Initializing CPU#7
[ 0.895023] CPU: Physical Processor ID: 0
[ 0.895025] CPU: Processor Core ID: 2
[ 0.895028] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 0.895030] CPU: L2 cache: 256K
[ 0.895031] CPU: L3 cache: 12288K
[ 0.895034] CPU 7/0x4 -> Node 0
[ 0.895047] CPU7: Thermal monitoring enabled (TM1)
[ 0.895050] CPU 7 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 0.895077] CPU7: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 0.895083] checking TSC synchronization [CPU#0 -> CPU#7]: passed.
[ 0.915117] Booting processor 8 APIC 0x22 ip 0x6000
[ 0.925687] Initializing CPU#8
[ 1.002806] CPU: Physical Processor ID: 1
[ 1.002807] CPU: Processor Core ID: 1
[ 1.002809] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.002810] CPU: L2 cache: 256K
[ 1.002811] CPU: L3 cache: 12288K
[ 1.002813] CPU 8/0x22 -> Node 1
[ 1.002823] CPU8: Thermal monitoring enabled (TM1)
[ 1.002826] CPU 8 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 1.002931] CPU8: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.002938] checking TSC synchronization [CPU#0 -> CPU#8]: passed.
[ 1.023009] Booting processor 9 APIC 0x12 ip 0x6000
[ 1.033640] Initializing CPU#9
[ 1.110591] CPU: Physical Processor ID: 0
[ 1.110592] CPU: Processor Core ID: 9
[ 1.110595] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.110597] CPU: L2 cache: 256K
[ 1.110598] CPU: L3 cache: 12288K
[ 1.110601] CPU 9/0x12 -> Node 0
[ 1.110614] CPU9: Thermal monitoring enabled (TM1)
[ 1.110618] CPU 9 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 1.110631] CPU9: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.110637] checking TSC synchronization [CPU#0 -> CPU#9]: passed.
[ 1.130669] Booting processor 10 APIC 0x32 ip 0x6000
[ 1.141178] Initializing CPU#10
[ 1.218373] CPU: Physical Processor ID: 1
[ 1.218374] CPU: Processor Core ID: 9
[ 1.218376] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.218378] CPU: L2 cache: 256K
[ 1.218378] CPU: L3 cache: 12288K
[ 1.218380] CPU 10/0x32 -> Node 1
[ 1.218390] CPU10: Thermal monitoring enabled (TM1)
[ 1.218393] CPU 10 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 1.218481] CPU10: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.218488] checking TSC synchronization [CPU#0 -> CPU#10]: passed.
[ 1.238566] Booting processor 11 APIC 0x2 ip 0x6000
[ 1.249197] Initializing CPU#11
[ 1.326158] CPU: Physical Processor ID: 0
[ 1.326160] CPU: Processor Core ID: 1
[ 1.326163] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.326165] CPU: L2 cache: 256K
[ 1.326166] CPU: L3 cache: 12288K
[ 1.326169] CPU 11/0x2 -> Node 0
[ 1.326182] CPU11: Thermal monitoring enabled (TM1)
[ 1.326185] CPU 11 MCA banks CMCI:2 CMCI:3 CMCI:5 SHD:6 SHD:8
[ 1.326242] CPU11: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.326248] checking TSC synchronization [CPU#0 -> CPU#11]: passed.
[ 1.346279] Booting processor 12 APIC 0x21 ip 0x6000
[ 1.357518] Initializing CPU#12
[ 1.433941] CPU: Physical Processor ID: 1
[ 1.433942] CPU: Processor Core ID: 0
[ 1.433945] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.433947] CPU: L2 cache: 256K
[ 1.433948] CPU: L3 cache: 12288K
[ 1.433950] CPU 12/0x21 -> Node 1
[ 1.433961] CPU12: Thermal monitoring enabled (TM1)
[ 1.433964] CPU 12 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.433997] CPU12: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.434004] checking TSC synchronization [CPU#0 -> CPU#12]: passed.
[ 1.454077] Booting processor 13 APIC 0x11 ip 0x6000
[ 1.464708] Initializing CPU#13
[ 1.541726] CPU: Physical Processor ID: 0
[ 1.541727] CPU: Processor Core ID: 8
[ 1.541730] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.541732] CPU: L2 cache: 256K
[ 1.541733] CPU: L3 cache: 12288K
[ 1.541736] CPU 13/0x11 -> Node 0
[ 1.541749] CPU13: Thermal monitoring enabled (TM1)
[ 1.541752] CPU 13 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.541820] CPU13: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.541827] checking TSC synchronization [CPU#0 -> CPU#13]: passed.
[ 1.561861] Booting processor 14 APIC 0x31 ip 0x6000
[ 1.572431] Initializing CPU#14
[ 1.649508] CPU: Physical Processor ID: 1
[ 1.649509] CPU: Processor Core ID: 8
[ 1.649511] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.649512] CPU: L2 cache: 256K
[ 1.649513] CPU: L3 cache: 12288K
[ 1.649515] CPU 14/0x31 -> Node 1
[ 1.649525] CPU14: Thermal monitoring enabled (TM1)
[ 1.649528] CPU 14 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.649546] CPU14: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.649552] checking TSC synchronization [CPU#0 -> CPU#14]: passed.
[ 1.669630] Booting processor 15 APIC 0x1 ip 0x6000
[ 1.680261] Initializing CPU#15
[ 1.757293] CPU: Physical Processor ID: 0
[ 1.757295] CPU: Processor Core ID: 0
[ 1.757297] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.757299] CPU: L2 cache: 256K
[ 1.757300] CPU: L3 cache: 12288K
[ 1.757303] CPU 15/0x1 -> Node 0
[ 1.757316] CPU15: Thermal monitoring enabled (TM1)
[ 1.757320] CPU 15 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.757365] CPU15: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.757371] checking TSC synchronization [CPU#0 -> CPU#15]: passed.
[ 1.777404] Booting processor 16 APIC 0x25 ip 0x6000
[ 1.787974] Initializing CPU#16
[ 1.865075] CPU: Physical Processor ID: 1
[ 1.865076] CPU: Processor Core ID: 2
[ 1.865078] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.865080] CPU: L2 cache: 256K
[ 1.865081] CPU: L3 cache: 12288K
[ 1.865082] CPU 16/0x25 -> Node 1
[ 1.865093] CPU16: Thermal monitoring enabled (TM1)
[ 1.865095] CPU 16 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.865103] CPU16: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.865110] checking TSC synchronization [CPU#0 -> CPU#16]: passed.
[ 1.885191] Booting processor 17 APIC 0x15 ip 0x6000
[ 1.896667] Initializing CPU#17
[ 1.972861] CPU: Physical Processor ID: 0
[ 1.972862] CPU: Processor Core ID: 10
[ 1.972865] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 1.972867] CPU: L2 cache: 256K
[ 1.972868] CPU: L3 cache: 12288K
[ 1.972871] CPU 17/0x15 -> Node 0
[ 1.972884] CPU17: Thermal monitoring enabled (TM1)
[ 1.972887] CPU 17 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 1.972976] CPU17: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 1.972982] checking TSC synchronization [CPU#0 -> CPU#17]: passed.
[ 1.993020] Booting processor 18 APIC 0x35 ip 0x6000
[ 2.002630] Initializing CPU#18
[ 2.080643] CPU: Physical Processor ID: 1
[ 2.080644] CPU: Processor Core ID: 10
[ 2.080646] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.080647] CPU: L2 cache: 256K
[ 2.080648] CPU: L3 cache: 12288K
[ 2.080650] CPU 18/0x35 -> Node 1
[ 2.080660] CPU18: Thermal monitoring enabled (TM1)
[ 2.080662] CPU 18 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.080706] CPU18: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.080713] checking TSC synchronization [CPU#0 -> CPU#18]: passed.
[ 2.100784] Booting processor 19 APIC 0x5 ip 0x6000
[ 2.112471] Initializing CPU#19
[ 2.188428] CPU: Physical Processor ID: 0
[ 2.188430] CPU: Processor Core ID: 2
[ 2.188432] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.188434] CPU: L2 cache: 256K
[ 2.188436] CPU: L3 cache: 12288K
[ 2.188438] CPU 19/0x5 -> Node 0
[ 2.188451] CPU19: Thermal monitoring enabled (TM1)
[ 2.188455] CPU 19 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.188526] CPU19: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.188532] checking TSC synchronization [CPU#0 -> CPU#19]: passed.
[ 2.208565] Booting processor 20 APIC 0x23 ip 0x6000
[ 2.219136] Initializing CPU#20
[ 2.296211] CPU: Physical Processor ID: 1
[ 2.296211] CPU: Processor Core ID: 1
[ 2.296213] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.296215] CPU: L2 cache: 256K
[ 2.296216] CPU: L3 cache: 12288K
[ 2.296217] CPU 20/0x23 -> Node 1
[ 2.296228] CPU20: Thermal monitoring enabled (TM1)
[ 2.296230] CPU 20 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.296257] CPU20: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.296264] checking TSC synchronization [CPU#0 -> CPU#20]: passed.
[ 2.316337] Booting processor 21 APIC 0x13 ip 0x6000
[ 2.326968] Initializing CPU#21
[ 2.403996] CPU: Physical Processor ID: 0
[ 2.403997] CPU: Processor Core ID: 9
[ 2.404000] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.404002] CPU: L2 cache: 256K
[ 2.404003] CPU: L3 cache: 12288K
[ 2.404006] CPU 21/0x13 -> Node 0
[ 2.404019] CPU21: Thermal monitoring enabled (TM1)
[ 2.404022] CPU 21 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.404113] CPU21: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.404119] checking TSC synchronization [CPU#0 -> CPU#21]: passed.
[ 2.424158] Booting processor 22 APIC 0x33 ip 0x6000
[ 2.435628] Initializing CPU#22
[ 2.511778] CPU: Physical Processor ID: 1
[ 2.511779] CPU: Processor Core ID: 9
[ 2.511781] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.511782] CPU: L2 cache: 256K
[ 2.511783] CPU: L3 cache: 12288K
[ 2.511785] CPU 22/0x33 -> Node 1
[ 2.511795] CPU22: Thermal monitoring enabled (TM1)
[ 2.511798] CPU 22 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.511862] CPU22: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.511870] checking TSC synchronization [CPU#0 -> CPU#22]: passed.
[ 2.531942] Booting processor 23 APIC 0x3 ip 0x6000
[ 2.542573] Initializing CPU#23
[ 2.619563] CPU: Physical Processor ID: 0
[ 2.619565] CPU: Processor Core ID: 1
[ 2.619567] CPU: L1 I cache: 32K, L1 D cache: 32K
[ 2.619569] CPU: L2 cache: 256K
[ 2.619571] CPU: L3 cache: 12288K
[ 2.619573] CPU 23/0x3 -> Node 0
[ 2.619586] CPU23: Thermal monitoring enabled (TM1)
[ 2.619589] CPU 23 MCA banks SHD:2 SHD:3 SHD:5 SHD:6 SHD:8
[ 2.619693] CPU23: Intel(R) Xeon(R) CPU L5640 @ 2.27GHz stepping 02
[ 2.619699] checking TSC synchronization [CPU#0 -> CPU#23]: passed.
[ 2.639687] Brought up 24 CPUs
[ 2.639690] Total of 24 processors activated (108769.55 BogoMIPS).
[ 2.661682] CPU0 attaching sched-domain:
[ 2.661686] domain 0: span 0,12 level SIBLING
[ 2.661688] groups: 0 (cpu_power = 589) 12 (cpu_power = 589)
[ 2.661694] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661695] groups: 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178)
[ 2.661704] domain 2: span 0-23 level NODE
[ 2.661706] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661715] CPU1 attaching sched-domain:
[ 2.661716] domain 0: span 1,13 level SIBLING
[ 2.661718] groups: 1 (cpu_power = 589) 13 (cpu_power = 589)
[ 2.661723] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661724] groups: 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178)
[ 2.661733] domain 2: span 0-23 level NODE
[ 2.661734] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.661742] CPU2 attaching sched-domain:
[ 2.661744] domain 0: span 2,14 level SIBLING
[ 2.661745] groups: 2 (cpu_power = 589) 14 (cpu_power = 589)
[ 2.661750] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661752] groups: 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178)
[ 2.661760] domain 2: span 0-23 level NODE
[ 2.661762] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661769] CPU3 attaching sched-domain:
[ 2.661771] domain 0: span 3,15 level SIBLING
[ 2.661772] groups: 3 (cpu_power = 589) 15 (cpu_power = 589)
[ 2.661777] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661779] groups: 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178)
[ 2.661787] domain 2: span 0-23 level NODE
[ 2.661789] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.661796] CPU4 attaching sched-domain:
[ 2.661798] domain 0: span 4,16 level SIBLING
[ 2.661799] groups: 4 (cpu_power = 589) 16 (cpu_power = 589)
[ 2.661804] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661805] groups: 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178)
[ 2.661814] domain 2: span 0-23 level NODE
[ 2.661815] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661823] CPU5 attaching sched-domain:
[ 2.661824] domain 0: span 5,17 level SIBLING
[ 2.661826] groups: 5 (cpu_power = 589) 17 (cpu_power = 589)
[ 2.661830] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661832] groups: 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178)
[ 2.661841] domain 2: span 0-23 level NODE
[ 2.661842] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.661850] CPU6 attaching sched-domain:
[ 2.661851] domain 0: span 6,18 level SIBLING
[ 2.661853] groups: 6 (cpu_power = 589) 18 (cpu_power = 589)
[ 2.661857] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661859] groups: 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178)
[ 2.661868] domain 2: span 0-23 level NODE
[ 2.661869] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661876] CPU7 attaching sched-domain:
[ 2.661878] domain 0: span 7,19 level SIBLING
[ 2.661879] groups: 7 (cpu_power = 589) 19 (cpu_power = 589)
[ 2.661884] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661886] groups: 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178)
[ 2.661894] domain 2: span 0-23 level NODE
[ 2.661896] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.661903] CPU8 attaching sched-domain:
[ 2.661905] domain 0: span 8,20 level SIBLING
[ 2.661906] groups: 8 (cpu_power = 589) 20 (cpu_power = 589)
[ 2.661911] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661912] groups: 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178)
[ 2.661921] domain 2: span 0-23 level NODE
[ 2.661922] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661930] CPU9 attaching sched-domain:
[ 2.661931] domain 0: span 9,21 level SIBLING
[ 2.661933] groups: 9 (cpu_power = 589) 21 (cpu_power = 589)
[ 2.661937] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661939] groups: 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178)
[ 2.661948] domain 2: span 0-23 level NODE
[ 2.661949] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.661957] CPU10 attaching sched-domain:
[ 2.661958] domain 0: span 10,22 level SIBLING
[ 2.661960] groups: 10 (cpu_power = 589) 22 (cpu_power = 589)
[ 2.661964] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.661966] groups: 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178)
[ 2.661975] domain 2: span 0-23 level NODE
[ 2.661976] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.661983] CPU11 attaching sched-domain:
[ 2.661985] domain 0: span 11,23 level SIBLING
[ 2.661986] groups: 11 (cpu_power = 589) 23 (cpu_power = 589)
[ 2.661991] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.661993] groups: 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178)
[ 2.662001] domain 2: span 0-23 level NODE
[ 2.662003] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662010] CPU12 attaching sched-domain:
[ 2.662011] domain 0: span 0,12 level SIBLING
[ 2.662013] groups: 12 (cpu_power = 589) 0 (cpu_power = 589)
[ 2.662018] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662019] groups: 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178)
[ 2.662028] domain 2: span 0-23 level NODE
[ 2.662029] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662037] CPU13 attaching sched-domain:
[ 2.662038] domain 0: span 1,13 level SIBLING
[ 2.662040] groups: 13 (cpu_power = 589) 1 (cpu_power = 589)
[ 2.662044] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662046] groups: 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178)
[ 2.662054] domain 2: span 0-23 level NODE
[ 2.662056] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662064] CPU14 attaching sched-domain:
[ 2.662065] domain 0: span 2,14 level SIBLING
[ 2.662066] groups: 14 (cpu_power = 589) 2 (cpu_power = 589)
[ 2.662071] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662073] groups: 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178)
[ 2.662081] domain 2: span 0-23 level NODE
[ 2.662083] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662090] CPU15 attaching sched-domain:
[ 2.662092] domain 0: span 3,15 level SIBLING
[ 2.662093] groups: 15 (cpu_power = 589) 3 (cpu_power = 589)
[ 2.662098] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662099] groups: 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178)
[ 2.662108] domain 2: span 0-23 level NODE
[ 2.662109] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662117] CPU16 attaching sched-domain:
[ 2.662118] domain 0: span 4,16 level SIBLING
[ 2.662120] groups: 16 (cpu_power = 589) 4 (cpu_power = 589)
[ 2.662125] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662126] groups: 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178)
[ 2.662135] domain 2: span 0-23 level NODE
[ 2.662136] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662144] CPU17 attaching sched-domain:
[ 2.662145] domain 0: span 5,17 level SIBLING
[ 2.662147] groups: 17 (cpu_power = 589) 5 (cpu_power = 589)
[ 2.662151] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662153] groups: 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178)
[ 2.662162] domain 2: span 0-23 level NODE
[ 2.662163] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662171] CPU18 attaching sched-domain:
[ 2.662172] domain 0: span 6,18 level SIBLING
[ 2.662174] groups: 18 (cpu_power = 589) 6 (cpu_power = 589)
[ 2.662178] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662180] groups: 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178)
[ 2.662189] domain 2: span 0-23 level NODE
[ 2.662190] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662197] CPU19 attaching sched-domain:
[ 2.662199] domain 0: span 7,19 level SIBLING
[ 2.662200] groups: 19 (cpu_power = 589) 7 (cpu_power = 589)
[ 2.662205] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662207] groups: 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178)
[ 2.662215] domain 2: span 0-23 level NODE
[ 2.662217] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662224] CPU20 attaching sched-domain:
[ 2.662226] domain 0: span 8,20 level SIBLING
[ 2.662227] groups: 20 (cpu_power = 589) 8 (cpu_power = 589)
[ 2.662232] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662233] groups: 8,20 (cpu_power = 1178) 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178)
[ 2.662242] domain 2: span 0-23 level NODE
[ 2.662243] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662251] CPU21 attaching sched-domain:
[ 2.662252] domain 0: span 9,21 level SIBLING
[ 2.662254] groups: 21 (cpu_power = 589) 9 (cpu_power = 589)
[ 2.662259] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662260] groups: 9,21 (cpu_power = 1178) 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178)
[ 2.662269] domain 2: span 0-23 level NODE
[ 2.662270] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.662278] CPU22 attaching sched-domain:
[ 2.662279] domain 0: span 10,22 level SIBLING
[ 2.662281] groups: 22 (cpu_power = 589) 10 (cpu_power = 589)
[ 2.662285] domain 1: span 0,2,4,6,8,10,12,14,16,18,20,22 level MC
[ 2.662287] groups: 10,22 (cpu_power = 1178) 0,12 (cpu_power = 1178) 2,14 (cpu_power = 1178) 4,16 (cpu_power = 1178) 6,18 (cpu_power = 1178) 8,20 (cpu_power = 1178)
[ 2.662296] domain 2: span 0-23 level NODE
[ 2.662297] groups: 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068) 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068)
[ 2.662304] CPU23 attaching sched-domain:
[ 2.662306] domain 0: span 11,23 level SIBLING
[ 2.662307] groups: 23 (cpu_power = 589) 11 (cpu_power = 589)
[ 2.662312] domain 1: span 1,3,5,7,9,11,13,15,17,19,21,23 level MC
[ 2.662314] groups: 11,23 (cpu_power = 1178) 1,13 (cpu_power = 1178) 3,15 (cpu_power = 1178) 5,17 (cpu_power = 1178) 7,19 (cpu_power = 1178) 9,21 (cpu_power = 1178)
[ 2.662322] domain 2: span 0-23 level NODE
[ 2.662324] groups: 1,3,5,7,9,11,13,15,17,19,21,23 (cpu_power = 7068) 0,2,4,6,8,10,12,14,16,18,20,22 (cpu_power = 7068)
[ 2.663778] devtmpfs: initialized
[ 2.667629] regulator: core version 0.5
[ 2.667702] NET: Registered protocol family 16
[ 2.667793] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[ 2.667796] ACPI: bus type pci registered
[ 2.667870] PCI: MCFG configuration 0: base e8000000 segment 0 buses 0 - 63
[ 2.667872] PCI: MCFG area at e8000000 reserved in E820
[ 2.669279] PCI: Using MMCONFIG at e8000000 - ebffffff
[ 2.669281] PCI: Using configuration type 1 for base access
[ 2.670298] bio: create slab <bio-0> at 0
[ 2.671192] ACPI: EC: Look up EC in DSDT
[ 2.673125] ACPI: Interpreter enabled
[ 2.673127] ACPI: (supports S0 S4 S5)
[ 2.673140] ACPI: Using IOAPIC for interrupt routing
[ 2.676243] ACPI: No dock devices found.
[ 2.676262] ACPI: PCI Root Bridge [PCI0] (0000:00)
[ 2.676339] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[ 2.676342] pci 0000:00:00.0: PME# disabled
[ 2.676410] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[ 2.676413] pci 0000:00:01.0: PME# disabled
[ 2.676478] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
[ 2.676480] pci 0000:00:02.0: PME# disabled
[ 2.676544] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[ 2.676547] pci 0000:00:03.0: PME# disabled
[ 2.676613] pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
[ 2.676616] pci 0000:00:07.0: PME# disabled
[ 2.676680] pci 0000:00:08.0: PME# supported from D0 D3hot D3cold
[ 2.676683] pci 0000:00:08.0: PME# disabled
[ 2.676747] pci 0000:00:09.0: PME# supported from D0 D3hot D3cold
[ 2.676749] pci 0000:00:09.0: PME# disabled
[ 2.676813] pci 0000:00:0a.0: PME# supported from D0 D3hot D3cold
[ 2.676816] pci 0000:00:0a.0: PME# disabled
[ 2.677598] pci 0000:00:1d.0: reg 20 io port: [0x1000-0x101f]
[ 2.677689] pci 0000:00:1d.1: reg 20 io port: [0x1020-0x103f]
[ 2.677803] pci 0000:00:1d.2: reg 20 io port: [0x1040-0x105f]
[ 2.677913] pci 0000:00:1d.3: reg 20 io port: [0x1060-0x107f]
[ 2.678005] pci 0000:00:1d.7: reg 10 32bit mmio: [0xfbcf0000-0xfbcf03ff]
[ 2.678070] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
[ 2.678074] pci 0000:00:1d.7: PME# disabled
[ 2.678303] pci 0000:02:00.0: reg 10 32bit mmio: [0xfbfe0000-0xfbffffff]
[ 2.678308] pci 0000:02:00.0: reg 14 32bit mmio: [0xfbfc0000-0xfbfdffff]
[ 2.678314] pci 0000:02:00.0: reg 18 io port: [0x4000-0x401f]
[ 2.678319] pci 0000:02:00.0: reg 1c 32bit mmio: [0xfbfb0000-0xfbfb3fff]
[ 2.678333] pci 0000:02:00.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[ 2.678374] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[ 2.678377] pci 0000:02:00.0: PME# disabled
[ 2.678413] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.678424] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.678460] pci 0000:02:00.1: reg 10 32bit mmio: [0xfbf80000-0xfbf9ffff]
[ 2.678465] pci 0000:02:00.1: reg 14 32bit mmio: [0xfbf60000-0xfbf7ffff]
[ 2.678471] pci 0000:02:00.1: reg 18 io port: [0x4020-0x403f]
[ 2.678476] pci 0000:02:00.1: reg 1c 32bit mmio: [0xfbf50000-0xfbf53fff]
[ 2.678489] pci 0000:02:00.1: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[ 2.678531] pci 0000:02:00.1: PME# supported from D0 D3hot D3cold
[ 2.678534] pci 0000:02:00.1: PME# disabled
[ 2.678560] pci 0000:02:00.1: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.678571] pci 0000:02:00.1: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.678631] pci 0000:00:03.0: bridge io port: [0x4000-0x4fff]
[ 2.678634] pci 0000:00:03.0: bridge 32bit mmio: [0xfbf00000-0xfbffffff]
[ 2.678761] pci 0000:01:03.0: reg 10 32bit mmio pref: [0xf0000000-0xf7ffffff]
[ 2.678768] pci 0000:01:03.0: reg 14 io port: [0x3000-0x30ff]
[ 2.678774] pci 0000:01:03.0: reg 18 32bit mmio: [0xfbef0000-0xfbefffff]
[ 2.678794] pci 0000:01:03.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[ 2.678823] pci 0000:01:03.0: supports D1 D2
[ 2.678861] pci 0000:01:04.0: reg 10 io port: [0x2800-0x28ff]
[ 2.678868] pci 0000:01:04.0: reg 14 32bit mmio: [0xfbee0000-0xfbee01ff]
[ 2.678918] pci 0000:01:04.0: PME# supported from D0 D3hot D3cold
[ 2.678921] pci 0000:01:04.0: PME# disabled
[ 2.678958] pci 0000:01:04.2: reg 10 io port: [0x3400-0x34ff]
[ 2.678965] pci 0000:01:04.2: reg 14 32bit mmio: [0xfbed0000-0xfbed07ff]
[ 2.678971] pci 0000:01:04.2: reg 18 32bit mmio: [0xfbec0000-0xfbec3fff]
[ 2.678978] pci 0000:01:04.2: reg 1c 32bit mmio: [0xfbe00000-0xfbe7ffff]
[ 2.678995] pci 0000:01:04.2: reg 30 32bit mmio pref: [0x000000-0x00ffff]
[ 2.679026] pci 0000:01:04.2: PME# supported from D0 D3hot D3cold
[ 2.679030] pci 0000:01:04.2: PME# disabled
[ 2.679088] pci 0000:01:04.4: reg 20 io port: [0x3800-0x381f]
[ 2.679129] pci 0000:01:04.4: PME# supported from D0 D3hot D3cold
[ 2.679133] pci 0000:01:04.4: PME# disabled
[ 2.679166] pci 0000:01:04.6: reg 10 32bit mmio: [0xfbdf0000-0xfbdf00ff]
[ 2.679221] pci 0000:01:04.6: PME# supported from D0 D3hot D3cold
[ 2.679225] pci 0000:01:04.6: PME# disabled
[ 2.679269] pci 0000:00:1e.0: transparent bridge
[ 2.679272] pci 0000:00:1e.0: bridge io port: [0x2000-0x3fff]
[ 2.679275] pci 0000:00:1e.0: bridge 32bit mmio: [0xfbd00000-0xfbefffff]
[ 2.679280] pci 0000:00:1e.0: bridge 64bit mmio pref: [0xf0000000-0xf7ffffff]
[ 2.679306] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[ 2.679464] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.IP2P._PRT]
[ 2.679519] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT01._PRT]
[ 2.679574] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT02._PRT]
[ 2.679627] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT03._PRT]
[ 2.679689] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT07._PRT]
[ 2.679770] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT08._PRT]
[ 2.679823] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT09._PRT]
[ 2.679876] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PT0A._PRT]
[ 2.683139] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 7 *10 11)
[ 2.683211] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 7 *10 11)
[ 2.683281] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 7 10 11) *0, disabled.
[ 2.683352] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 7 10 11) *0, disabled.
[ 2.683422] ACPI: PCI Interrupt Link [LNKE] (IRQs 5 7 10 *11)
[ 2.683503] ACPI: PCI Interrupt Link [LNKF] (IRQs 5 *7 10 11)
[ 2.683578] ACPI: PCI Interrupt Link [LNKG] (IRQs *5 7 10 11)
[ 2.683649] ACPI: PCI Interrupt Link [LNKH] (IRQs 5 7 10 11) *3
[ 2.683735] vgaarb: device added: PCI:0000:01:03.0,decodes=io+mem,owns=io+mem,locks=none
[ 2.683738] vgaarb: loaded
[ 2.683790] PCI: Using ACPI for IRQ routing
[ 2.683991] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
[ 2.683994] hpet0: 4 comparators, 64-bit 14.318180 MHz counter
[ 2.686010] Switching to clocksource tsc
[ 2.688386] pnp: PnP ACPI init
[ 2.688413] ACPI: bus type pnp registered
[ 2.689782] pnp: PnP ACPI: found 10 devices
[ 2.689784] ACPI: ACPI bus type pnp unregistered
[ 2.689796] system 00:01: ioport range 0x408-0x40f has been reserved
[ 2.689800] system 00:01: ioport range 0x4d0-0x4d1 has been reserved
[ 2.689803] system 00:01: ioport range 0x700-0x71f has been reserved
[ 2.689806] system 00:01: ioport range 0x880-0x8ff has been reserved
[ 2.689809] system 00:01: ioport range 0x900-0x97f has been reserved
[ 2.689812] system 00:01: ioport range 0xc80-0xc83 has been reserved
[ 2.689815] system 00:01: ioport range 0xcd4-0xcd7 has been reserved
[ 2.689819] system 00:01: ioport range 0xf50-0xf58 has been reserved
[ 2.689822] system 00:01: ioport range 0xca0-0xca1 has been reserved
[ 2.689829] system 00:01: ioport range 0xca4-0xca5 has been reserved
[ 2.689832] system 00:01: ioport range 0x3f8-0x3ff has been reserved
[ 2.689836] system 00:01: iomem range 0xe8000000-0xebffffff has been reserved
[ 2.689840] system 00:01: iomem range 0xfe000000-0xfebfffff has been reserved
[ 2.689843] system 00:01: iomem range 0xefffe000-0xefffffff has been reserved
[ 2.695031] pci 0000:00:01.0: PCI bridge, secondary bus 0000:06
[ 2.695033] pci 0000:00:01.0: IO window: disabled
[ 2.695038] pci 0000:00:01.0: MEM window: disabled
[ 2.695041] pci 0000:00:01.0: PREFETCH window: disabled
[ 2.695047] pci 0000:00:02.0: PCI bridge, secondary bus 0000:07
[ 2.695049] pci 0000:00:02.0: IO window: disabled
[ 2.695053] pci 0000:00:02.0: MEM window: disabled
[ 2.695056] pci 0000:00:02.0: PREFETCH window: disabled
[ 2.695070] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695080] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695089] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695098] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695107] pci 0000:02:00.1: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695116] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695126] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695135] pci 0000:02:00.1: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695144] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695152] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695161] pci 0000:02:00.1: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695171] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695180] pci 0000:02:00.0: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695193] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695202] pci 0000:02:00.0: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695214] pci 0000:02:00.1: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695223] pci 0000:02:00.1: reg 184 64bit mmio: [0x000000-0x003fff]
[ 2.695227] pci 0000:02:00.1: BAR 7: can't allocate mem resource [0xfc000000-0xfbffffff]
[ 2.695236] pci 0000:02:00.1: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695245] pci 0000:02:00.1: reg 190 64bit mmio: [0x000000-0x003fff]
[ 2.695248] pci 0000:02:00.1: BAR 10: can't allocate mem resource [0xfc000000-0xfbffffff]
[ 2.695251] pci 0000:00:03.0: PCI bridge, secondary bus 0000:02
[ 2.695255] pci 0000:00:03.0: IO window: 0x4000-0x4fff
[ 2.695259] pci 0000:00:03.0: MEM window: 0xfbf00000-0xfbffffff
[ 2.695264] pci 0000:00:03.0: PREFETCH window: 0xec000000-0xec0fffff
[ 2.695269] pci 0000:00:07.0: PCI bridge, secondary bus 0000:03
[ 2.695271] pci 0000:00:07.0: IO window: disabled
[ 2.695275] pci 0000:00:07.0: MEM window: disabled
[ 2.695279] pci 0000:00:07.0: PREFETCH window: disabled
[ 2.695286] pci 0000:00:08.0: PCI bridge, secondary bus 0000:08
[ 2.695288] pci 0000:00:08.0: IO window: disabled
[ 2.695292] pci 0000:00:08.0: MEM window: disabled
[ 2.695296] pci 0000:00:08.0: PREFETCH window: disabled
[ 2.695301] pci 0000:00:09.0: PCI bridge, secondary bus 0000:09
[ 2.695303] pci 0000:00:09.0: IO window: disabled
[ 2.695307] pci 0000:00:09.0: MEM window: disabled
[ 2.695310] pci 0000:00:09.0: PREFETCH window: disabled
[ 2.695316] pci 0000:00:0a.0: PCI bridge, secondary bus 0000:0a
[ 2.695318] pci 0000:00:0a.0: IO window: disabled
[ 2.695322] pci 0000:00:0a.0: MEM window: disabled
[ 2.695326] pci 0000:00:0a.0: PREFETCH window: disabled
[ 2.695334] pci 0000:00:1e.0: PCI bridge, secondary bus 0000:01
[ 2.695337] pci 0000:00:1e.0: IO window: 0x2000-0x3fff
[ 2.695342] pci 0000:00:1e.0: MEM window: 0xfbd00000-0xfbefffff
[ 2.695347] pci 0000:00:1e.0: PREFETCH window: 0x000000f0000000-0x000000f7ffffff
[ 2.695361] pci 0000:00:01.0: setting latency timer to 64
[ 2.695369] pci 0000:00:02.0: setting latency timer to 64
[ 2.695378] pci 0000:00:03.0: setting latency timer to 64
[ 2.695386] pci 0000:00:07.0: setting latency timer to 64
[ 2.695394] pci 0000:00:08.0: setting latency timer to 64
[ 2.695402] pci 0000:00:09.0: setting latency timer to 64
[ 2.695410] pci 0000:00:0a.0: setting latency timer to 64
[ 2.695417] pci 0000:00:1e.0: setting latency timer to 64
[ 2.695421] pci_bus 0000:00: resource 0 io: [0x00-0xffff]
[ 2.695424] pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
[ 2.695427] pci_bus 0000:02: resource 0 io: [0x4000-0x4fff]
[ 2.695430] pci_bus 0000:02: resource 1 mem: [0xfbf00000-0xfbffffff]
[ 2.695433] pci_bus 0000:02: resource 2 pref mem [0xec000000-0xec0fffff]
[ 2.695436] pci_bus 0000:01: resource 0 io: [0x2000-0x3fff]
[ 2.695439] pci_bus 0000:01: resource 1 mem: [0xfbd00000-0xfbefffff]
[ 2.695442] pci_bus 0000:01: resource 2 pref mem [0xf0000000-0xf7ffffff]
[ 2.695444] pci_bus 0000:01: resource 3 io: [0x00-0xffff]
[ 2.695447] pci_bus 0000:01: resource 4 mem: [0x000000-0xffffffffffffffff]
[ 2.695489] NET: Registered protocol family 2
[ 2.696105] IP route cache hash table entries: 524288 (order: 10, 4194304 bytes)
[ 2.697721] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[ 2.699771] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[ 2.699992] TCP: Hash tables configured (established 524288 bind 65536)
[ 2.699995] TCP reno registered
[ 2.700253] NET: Registered protocol family 1
[ 2.714514] pci 0000:01:03.0: Boot video device
[ 2.715202] Unpacking initramfs...
[ 2.859301] Freeing initrd memory: 5339k freed
[ 2.862221] audit: initializing netlink socket (disabled)
[ 2.862233] type=2000 audit(1328888374.380:1): initialized
[ 2.867148] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 2.869136] VFS: Disk quotas dquot_6.5.2
[ 2.869195] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 2.869309] msgmni has been set to 24064
[ 2.869962] alg: No test for stdrng (krng)
[ 2.870017] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[ 2.870020] io scheduler noop registered
[ 2.870022] io scheduler anticipatory registered
[ 2.870024] io scheduler deadline registered
[ 2.870092] io scheduler cfq registered (default)
[ 2.870257] alloc irq_desc for 49 on node -1
[ 2.870259] alloc kstat_irqs on node -1
[ 2.870264] alloc irq_2_iommu on node -1
[ 2.870275] pcieport 0000:00:01.0: irq 49 for MSI/MSI-X
[ 2.870281] pcieport 0000:00:01.0: setting latency timer to 64
[ 2.870391] alloc irq_desc for 50 on node -1
[ 2.870394] alloc kstat_irqs on node -1
[ 2.870397] alloc irq_2_iommu on node -1
[ 2.870405] pcieport 0000:00:02.0: irq 50 for MSI/MSI-X
[ 2.870411] pcieport 0000:00:02.0: setting latency timer to 64
[ 2.870517] alloc irq_desc for 51 on node -1
[ 2.870520] alloc kstat_irqs on node -1
[ 2.870522] alloc irq_2_iommu on node -1
[ 2.870530] pcieport 0000:00:03.0: irq 51 for MSI/MSI-X
[ 2.870536] pcieport 0000:00:03.0: setting latency timer to 64
[ 2.870642] alloc irq_desc for 52 on node -1
[ 2.870645] alloc kstat_irqs on node -1
[ 2.870648] alloc irq_2_iommu on node -1
[ 2.870656] pcieport 0000:00:07.0: irq 52 for MSI/MSI-X
[ 2.870662] pcieport 0000:00:07.0: setting latency timer to 64
[ 2.870771] alloc irq_desc for 53 on node -1
[ 2.870773] alloc kstat_irqs on node -1
[ 2.870776] alloc irq_2_iommu on node -1
[ 2.870784] pcieport 0000:00:08.0: irq 53 for MSI/MSI-X
[ 2.870790] pcieport 0000:00:08.0: setting latency timer to 64
[ 2.870896] alloc irq_desc for 54 on node -1
[ 2.870898] alloc kstat_irqs on node -1
[ 2.870901] alloc irq_2_iommu on node -1
[ 2.870909] pcieport 0000:00:09.0: irq 54 for MSI/MSI-X
[ 2.870915] pcieport 0000:00:09.0: setting latency timer to 64
[ 2.871024] alloc irq_desc for 55 on node -1
[ 2.871026] alloc kstat_irqs on node -1
[ 2.871029] alloc irq_2_iommu on node -1
[ 2.871037] pcieport 0000:00:0a.0: irq 55 for MSI/MSI-X
[ 2.871043] pcieport 0000:00:0a.0: setting latency timer to 64
[ 2.871131] aer 0000:00:01.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871140] aer 0000:00:02.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871147] aer 0000:00:03.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871153] aer 0000:00:07.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871159] aer 0000:00:08.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871165] aer 0000:00:09.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.871171] aer 0000:00:0a.0:pcie02: AER service couldn't init device: no _OSC support
[ 2.872655] Linux agpgart interface v0.103
[ 2.872704] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 2.872776] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[ 2.873179] input: Macintosh mouse button emulation as /devices/virtual/input/input0
[ 2.873236] PNP: PS/2 Controller [PNP0303:KBD,PNP0f0e:PS2M] at 0x60,0x64 irq 1,12
[ 2.874675] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 2.874682] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 2.874772] mice: PS/2 mouse device common for all mice
[ 2.874810] rtc_cmos 00:09: RTC can wake from S4
[ 2.874850] rtc_cmos 00:09: rtc core: registered rtc_cmos as rtc0
[ 2.874876] rtc0: alarms up to one year, y3k, 114 bytes nvram, hpet irqs
[ 2.874885] cpuidle: using governor ladder
[ 2.874887] cpuidle: using governor menu
[ 2.874892] No iBFT detected.
[ 2.875297] TCP cubic registered
[ 2.875464] NET: Registered protocol family 10
[ 2.876489] Mobile IPv6
[ 2.876493] NET: Registered protocol family 17
[ 2.876573] PM: Resume from disk failed.
[ 2.876581] registered taskstats version 1
[ 2.877279] rtc_cmos 00:09: setting system clock to 2012-02-10 15:39:35 UTC (1328888375)
[ 2.877331] Initalizing network drop monitor service
[ 2.877395] Freeing unused kernel memory: 588k freed
[ 2.877533] Write protecting the kernel read-only data: 4176k
[ 2.922425] nbd: registered device at major 43
[ 2.926358] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 2.952603] thermal LNXTHERM:01: registered as thermal_zone0
[ 2.952611] ACPI: Thermal Zone [THM0] (8 C)
[ 3.082913] dca service started, version 1.12.1
[ 3.086362] Intel(R) Gigabit Ethernet Network Driver - version 1.3.16-k2
[ 3.086364] Copyright (c) 2007-2009 Intel Corporation.
[ 3.086400] alloc irq_desc for 24 on node -1
[ 3.086402] alloc kstat_irqs on node -1
[ 3.086412] alloc irq_2_iommu on node -1
[ 3.086421] igb 0000:02:00.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
[ 3.086444] igb 0000:02:00.0: setting latency timer to 64
[ 3.086579] alloc irq_desc for 56 on node -1
[ 3.086581] alloc kstat_irqs on node -1
[ 3.086585] alloc irq_2_iommu on node -1
[ 3.086592] igb 0000:02:00.0: irq 56 for MSI/MSI-X
[ 3.086594] alloc irq_desc for 57 on node -1
[ 3.086596] alloc kstat_irqs on node -1
[ 3.086598] alloc irq_2_iommu on node -1
[ 3.086604] igb 0000:02:00.0: irq 57 for MSI/MSI-X
[ 3.086606] alloc irq_desc for 58 on node -1
[ 3.086608] alloc kstat_irqs on node -1
[ 3.086611] alloc irq_2_iommu on node -1
[ 3.086615] igb 0000:02:00.0: irq 58 for MSI/MSI-X
[ 3.086618] alloc irq_desc for 59 on node -1
[ 3.086620] alloc kstat_irqs on node -1
[ 3.086623] alloc irq_2_iommu on node -1
[ 3.086628] igb 0000:02:00.0: irq 59 for MSI/MSI-X
[ 3.086630] alloc irq_desc for 60 on node -1
[ 3.086632] alloc kstat_irqs on node -1
[ 3.086634] alloc irq_2_iommu on node -1
[ 3.086639] igb 0000:02:00.0: irq 60 for MSI/MSI-X
[ 3.086641] alloc irq_desc for 61 on node -1
[ 3.086643] alloc kstat_irqs on node -1
[ 3.086646] alloc irq_2_iommu on node -1
[ 3.086651] igb 0000:02:00.0: irq 61 for MSI/MSI-X
[ 3.086653] alloc irq_desc for 62 on node -1
[ 3.086655] alloc kstat_irqs on node -1
[ 3.086658] alloc irq_2_iommu on node -1
[ 3.086663] igb 0000:02:00.0: irq 62 for MSI/MSI-X
[ 3.086665] alloc irq_desc for 63 on node -1
[ 3.086666] alloc kstat_irqs on node -1
[ 3.086669] alloc irq_2_iommu on node -1
[ 3.086674] igb 0000:02:00.0: irq 63 for MSI/MSI-X
[ 3.086677] alloc irq_desc for 64 on node -1
[ 3.086678] alloc kstat_irqs on node -1
[ 3.086681] alloc irq_2_iommu on node -1
[ 3.086686] igb 0000:02:00.0: irq 64 for MSI/MSI-X
[ 3.118117] usbcore: registered new interface driver usbfs
[ 3.118139] usbcore: registered new interface driver hub
[ 3.118181] usbcore: registered new device driver usb
[ 3.119252] uhci_hcd: USB Universal Host Controller Interface driver
[ 3.119295] alloc irq_desc for 20 on node -1
[ 3.119297] alloc kstat_irqs on node -1
[ 3.119301] alloc irq_2_iommu on node -1
[ 3.119307] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[ 3.119314] uhci_hcd 0000:00:1d.0: setting latency timer to 64
[ 3.119317] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[ 3.119350] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 1
[ 3.119381] uhci_hcd 0000:00:1d.0: irq 20, io base 0x00001000
[ 3.119412] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.119413] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.119415] usb usb1: Product: UHCI Host Controller
[ 3.119416] usb usb1: Manufacturer: Linux 2.6.32.55-no-hz uhci_hcd
[ 3.119418] usb usb1: SerialNumber: 0000:00:1d.0
[ 3.119482] usb usb1: configuration #1 chosen from 1 choice
[ 3.119500] hub 1-0:1.0: USB hub found
[ 3.119504] hub 1-0:1.0: 2 ports detected
[ 3.119544] alloc irq_desc for 23 on node -1
[ 3.119545] alloc kstat_irqs on node -1
[ 3.119548] alloc irq_2_iommu on node -1
[ 3.119552] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 23 (level, low) -> IRQ 23
[ 3.119557] uhci_hcd 0000:00:1d.1: setting latency timer to 64
[ 3.119559] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[ 3.119565] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 2
[ 3.119591] uhci_hcd 0000:00:1d.1: irq 23, io base 0x00001020
[ 3.119614] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.119616] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.119618] usb usb2: Product: UHCI Host Controller
[ 3.119619] usb usb2: Manufacturer: Linux 2.6.32.55-no-hz uhci_hcd
[ 3.119620] usb usb2: SerialNumber: 0000:00:1d.1
[ 3.119659] usb usb2: configuration #1 chosen from 1 choice
[ 3.119677] hub 2-0:1.0: USB hub found
[ 3.119682] hub 2-0:1.0: 2 ports detected
[ 3.119714] alloc irq_desc for 22 on node -1
[ 3.119715] alloc kstat_irqs on node -1
[ 3.119718] alloc irq_2_iommu on node -1
[ 3.119722] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 22 (level, low) -> IRQ 22
[ 3.119727] uhci_hcd 0000:00:1d.2: setting latency timer to 64
[ 3.119729] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[ 3.119735] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 3
[ 3.119761] uhci_hcd 0000:00:1d.2: irq 22, io base 0x00001040
[ 3.119784] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.119786] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.119787] usb usb3: Product: UHCI Host Controller
[ 3.119789] usb usb3: Manufacturer: Linux 2.6.32.55-no-hz uhci_hcd
[ 3.119790] usb usb3: SerialNumber: 0000:00:1d.2
[ 3.119828] usb usb3: configuration #1 chosen from 1 choice
[ 3.119847] hub 3-0:1.0: USB hub found
[ 3.119852] hub 3-0:1.0: 2 ports detected
[ 3.119884] uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 23 (level, low) -> IRQ 23
[ 3.119889] uhci_hcd 0000:00:1d.3: setting latency timer to 64
[ 3.119891] uhci_hcd 0000:00:1d.3: UHCI Host Controller
[ 3.119896] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 4
[ 3.119917] uhci_hcd 0000:00:1d.3: irq 23, io base 0x00001060
[ 3.119941] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.119943] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.119944] usb usb4: Product: UHCI Host Controller
[ 3.119945] usb usb4: Manufacturer: Linux 2.6.32.55-no-hz uhci_hcd
[ 3.119947] usb usb4: SerialNumber: 0000:00:1d.3
[ 3.119985] usb usb4: configuration #1 chosen from 1 choice
[ 3.120003] hub 4-0:1.0: USB hub found
[ 3.120007] hub 4-0:1.0: 2 ports detected
[ 3.120054] uhci_hcd 0000:01:04.4: PCI INT B -> GSI 22 (level, low) -> IRQ 22
[ 3.120060] uhci_hcd 0000:01:04.4: UHCI Host Controller
[ 3.120064] uhci_hcd 0000:01:04.4: new USB bus registered, assigned bus number 5
[ 3.120075] uhci_hcd 0000:01:04.4: port count misdetected? forcing to 2 ports
[ 3.120708] uhci_hcd 0000:01:04.4: irq 22, io base 0x00003800
[ 3.120787] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
[ 3.120789] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.120791] usb usb5: Product: UHCI Host Controller
[ 3.120792] usb usb5: Manufacturer: Linux 2.6.32.55-no-hz uhci_hcd
[ 3.120794] usb usb5: SerialNumber: 0000:01:04.4
[ 3.120833] usb usb5: configuration #1 chosen from 1 choice
[ 3.120853] hub 5-0:1.0: USB hub found
[ 3.120857] hub 5-0:1.0: 2 ports detected
[ 3.121238] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 3.121240] Warning! ehci_hcd should always be loaded before uhci_hcd and ohci_hcd, not after
[ 3.121258] ehci_hcd 0000:00:1d.7: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[ 3.121294] ehci_hcd 0000:00:1d.7: setting latency timer to 64
[ 3.121298] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[ 3.121322] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 6
[ 3.121359] ehci_hcd 0000:00:1d.7: debug port 1
[ 3.125231] ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
[ 3.125241] ehci_hcd 0000:00:1d.7: irq 20, io mem 0xfbcf0000
[ 3.135291] igb 0000:02:00.0: Intel(R) Gigabit Ethernet Network Connection
[ 3.135296] igb 0000:02:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 02:00:00:28:00:1c
[ 3.135374] igb 0000:02:00.0: eth0: PBA No: ffffff-0ff
[ 3.135377] igb 0000:02:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[ 3.135398] alloc irq_desc for 34 on node -1
[ 3.135403] alloc kstat_irqs on node -1
[ 3.135408] alloc irq_2_iommu on node -1
[ 3.135417] igb 0000:02:00.1: PCI INT B -> GSI 34 (level, low) -> IRQ 34
[ 3.135438] igb 0000:02:00.1: setting latency timer to 64
[ 3.135554] alloc irq_desc for 65 on node -1
[ 3.135556] alloc kstat_irqs on node -1
[ 3.135560] alloc irq_2_iommu on node -1
[ 3.135566] igb 0000:02:00.1: irq 65 for MSI/MSI-X
[ 3.135568] alloc irq_desc for 66 on node -1
[ 3.135570] alloc kstat_irqs on node -1
[ 3.135573] alloc irq_2_iommu on node -1
[ 3.135578] igb 0000:02:00.1: irq 66 for MSI/MSI-X
[ 3.135580] alloc irq_desc for 67 on node -1
[ 3.135582] alloc kstat_irqs on node -1
[ 3.135585] alloc irq_2_iommu on node -1
[ 3.135589] igb 0000:02:00.1: irq 67 for MSI/MSI-X
[ 3.135592] alloc irq_desc for 68 on node -1
[ 3.135594] alloc kstat_irqs on node -1
[ 3.135596] alloc irq_2_iommu on node -1
[ 3.135601] igb 0000:02:00.1: irq 68 for MSI/MSI-X
[ 3.135603] alloc irq_desc for 69 on node -1
[ 3.135605] alloc kstat_irqs on node -1
[ 3.135608] alloc irq_2_iommu on node -1
[ 3.135612] igb 0000:02:00.1: irq 69 for MSI/MSI-X
[ 3.135615] alloc irq_desc for 70 on node -1
[ 3.135616] alloc kstat_irqs on node -1
[ 3.135619] alloc irq_2_iommu on node -1
[ 3.135624] igb 0000:02:00.1: irq 70 for MSI/MSI-X
[ 3.135626] alloc irq_desc for 71 on node -1
[ 3.135628] alloc kstat_irqs on node -1
[ 3.135631] alloc irq_2_iommu on node -1
[ 3.135635] igb 0000:02:00.1: irq 71 for MSI/MSI-X
[ 3.135637] alloc irq_desc for 72 on node -1
[ 3.135639] alloc kstat_irqs on node -1
[ 3.135642] alloc irq_2_iommu on node -1
[ 3.135647] igb 0000:02:00.1: irq 72 for MSI/MSI-X
[ 3.135649] alloc irq_desc for 73 on node -1
[ 3.135650] alloc kstat_irqs on node -1
[ 3.135653] alloc irq_2_iommu on node -1
[ 3.135658] igb 0000:02:00.1: irq 73 for MSI/MSI-X
[ 3.137888] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
[ 3.137903] usb usb6: New USB device found, idVendor=1d6b, idProduct=0002
[ 3.137905] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 3.137907] usb usb6: Product: EHCI Host Controller
[ 3.137908] usb usb6: Manufacturer: Linux 2.6.32.55-no-hz ehci_hcd
[ 3.137910] usb usb6: SerialNumber: 0000:00:1d.7
[ 3.137970] usb usb6: configuration #1 chosen from 1 choice
[ 3.137990] hub 6-0:1.0: USB hub found
[ 3.137994] hub 6-0:1.0: 8 ports detected
[ 3.183187] igb 0000:02:00.1: Intel(R) Gigabit Ethernet Network Connection
[ 3.183192] igb 0000:02:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 02:00:00:28:00:1e
[ 3.183269] igb 0000:02:00.1: eth1: PBA No: ffffff-0ff
[ 3.183272] igb 0000:02:00.1: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[ 3.275461] Ethernet Channel Bonding Driver: v3.5.0 (November 4, 2008)
[ 3.275463] bonding: Warning: either miimon or arp_interval and arp_ip_target module parameters must be specified, otherwise bonding will not detect link failures! see bonding.txt for details.
[ 3.276104] bonding: bond0 is being created...
[ 3.276106] bonding: cannot add bond bond0; already exists
[ 3.276153] bonding: Bond creation failed.
[ 3.276187] bonding: bond0: setting mode to active-backup (1).
[ 3.276204] bonding: bond0: Setting MII monitoring interval to 100.
[ 3.276223] bonding: bond0: Setting up delay to 200.
[ 3.276239] bonding: bond0: Setting down delay to 200.
[ 3.278596] ADDRCONF(NETDEV_UP): bond0: link is not ready
[ 3.337767] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[ 3.338659] bonding: bond0: enslaving eth0 as a backup interface with an up link.
[ 3.369706] igb: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[ 3.370842] bonding: bond0: enslaving eth1 as a backup interface with an up link.
[ 3.372504] bonding: bond0: Setting eth0 as primary slave.
[ 3.373260] bonding: bond0: link status definitely up for interface eth0.
[ 3.373264] bonding: bond0: making interface eth0 the new active one.
[ 3.373386] bonding: bond0: first active interface up!
[ 3.375041] ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready
[ 3.390784] RPC: Registered udp transport module.
[ 3.390787] RPC: Registered tcp transport module.
[ 3.390789] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 3.398765] Slow work thread pool: Starting up
[ 3.398828] Slow work thread pool: Ready
[ 3.398872] FS-Cache: Loaded
[ 3.414282] FS-Cache: Netfs 'nfs' registered for caching
[ 3.429096] usb 5-1: new full speed USB device using uhci_hcd and address 2
[ 3.572820] bonding: bond0: link status definitely up for interface eth1.
[ 3.575887] usb 5-1: New USB device found, idVendor=03f0, idProduct=1027
[ 3.575891] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.575894] usb 5-1: Product: Virtual Keyboard
[ 3.575896] usb 5-1: Manufacturer: HP
[ 3.575980] usb 5-1: configuration #1 chosen from 1 choice
[ 3.594596] usbcore: registered new interface driver hiddev
[ 3.598899] input: HP Virtual Keyboard as /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb5/5-1/5-1:1.0/input/input1
[ 3.598945] generic-usb 0003:03F0:1027.0001: input,hidraw0: USB HID v1.01 Keyboard [HP Virtual Keyboard] on usb-0000:01:04.4-1/input0
[ 3.605307] input: HP Virtual Keyboard as /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb5/5-1/5-1:1.1/input/input2
[ 3.605366] generic-usb 0003:03F0:1027.0002: input,hidraw1: USB HID v1.01 Mouse [HP Virtual Keyboard] on usb-0000:01:04.4-1/input1
[ 3.605387] usbcore: registered new interface driver usbhid
[ 3.605390] usbhid: v2.6:USB HID core driver
[ 4.996009] udevd version 125 started
[ 5.253716] Monitor-Mwait will be used to enter C-1 state
[ 5.261699] Monitor-Mwait will be used to enter C-3 state
[ 5.289647] Monitor-Mwait will be used to enter C-3 state
[ 5.289785] processor LNXCPU:00: registered as cooling_device0
[ 5.323366] processor LNXCPU:01: registered as cooling_device1
[ 5.323438] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3
[ 5.323445] ACPI: Power Button [PWRF]
[ 5.353925] processor LNXCPU:02: registered as cooling_device2
[ 5.365415] power_meter ACPI000D:00: Found ACPI power meter.
[ 5.365465] power_meter ACPI000D:00: Ignoring unsafe software power cap!
[ 5.390824] input: PC Speaker as /devices/platform/pcspkr/input/input4
[ 5.398817] processor LNXCPU:03: registered as cooling_device3
[ 5.427181] processor LNXCPU:04: registered as cooling_device4
[ 5.454591] hpilo 0000:01:04.2: PCI INT B -> GSI 22 (level, low) -> IRQ 22
[ 5.454820] processor LNXCPU:05: registered as cooling_device5
[ 5.456317] Error: Driver 'pcspkr' is already registered, aborting...
[ 5.463642] ipmi message handler version 39.2
[ 5.464675] processor LNXCPU:08: registered as cooling_device6
[ 5.466966] IPMI System Interface driver.
[ 5.466971] ipmi_si: Trying SMBIOS-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0
[ 5.473531] processor LNXCPU:09: registered as cooling_device7
[ 5.474029] processor LNXCPU:0a: registered as cooling_device8
[ 5.474520] processor LNXCPU:0b: registered as cooling_device9
[ 5.474998] processor LNXCPU:0c: registered as cooling_device10
[ 5.475470] processor LNXCPU:0d: registered as cooling_device11
[ 5.477198] processor LNXCPU:10: registered as cooling_device12
[ 5.477790] processor LNXCPU:11: registered as cooling_device13
[ 5.478258] processor LNXCPU:12: registered as cooling_device14
[ 5.478792] processor LNXCPU:13: registered as cooling_device15
[ 5.479271] processor LNXCPU:14: registered as cooling_device16
[ 5.479790] processor LNXCPU:15: registered as cooling_device17
[ 5.480309] processor LNXCPU:18: registered as cooling_device18
[ 5.480914] processor LNXCPU:19: registered as cooling_device19
[ 5.481373] processor LNXCPU:1a: registered as cooling_device20
[ 5.481930] processor LNXCPU:1b: registered as cooling_device21
[ 5.482615] processor LNXCPU:1c: registered as cooling_device22
[ 5.483080] processor LNXCPU:1d: registered as cooling_device23
[ 5.628633] ipmi: Found new BMC (man_id: 0x00000b, prod_id: 0x2020, dev_id: 0x11)
[ 5.628643] IPMI kcs interface initialized
[ 5.628659] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
[ 5.628662] ipmi_si: duplicate interface
[ 5.669725] alloc irq_desc for 21 on node -1
[ 5.669729] alloc kstat_irqs on node -1
[ 5.669735] alloc irq_2_iommu on node -1
[ 5.669744] ipmi_si 0000:01:04.6: PCI INT A -> GSI 21 (level, low) -> IRQ 21
[ 5.669749] ipmi_si: Trying PCI-specified kcs state machine at mem address 0xfbdf0000, slave address 0x0, irq 21
[ 5.817412] IRQ 21/ipmi_si: IRQF_DISABLED is not guaranteed on shared IRQs
[ 5.817433] Using irq 21
[ 6.003849] ipmi: interfacing existing BMC (man_id: 0x00000b, prod_id: 0x2020, dev_id: 0x11)
[ 6.003861] IPMI kcs interface initialized
[ 7.577657] ipmi device interface
[ 13.961305] bond0: no IPv6 routers present
[ 21.961240] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 22.263077] warning: `ntpd' uses 32-bit capabilities (legacy support in use)
[ 33.606423] svc: failed to register lockdv1 RPC service (errno 97).
[ 100.888177] IPVS: Registered protocols (TCP, UDP, AH, ESP)
[ 100.889103] IPVS: Connection hash table configured (size=1048576, memory=16384Kbytes)
[ 100.891831] IPVS: ipvs loaded.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-02-23 15:46 ` Lesław Kopeć
@ 2012-02-29 12:06 ` Peter Zijlstra
2012-02-29 16:24 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-02-29 12:06 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
On Thu, 2012-02-23 at 16:46 +0100, Lesław Kopeć wrote:
> Each kernel was compiled with CONFIG_NO_HZ enabled (no-hz variant) and
> disabled (hz variant). Here's a snapshot of load 15 on each kernel:
> no-hz hz
> 2.6.32.55-* 0.59 0.57
> 2.6.32.55-*-74f5187ac8 3.56 11.79
> 2.6.32.55-*-0f004f5a69 0.61 11.76
> 2.6.37-rc5-*-0f004f5a69 0.67 11.65
> 2.6.37-rc5-*-pre-0f004f5a69 3.97 12.05
Missing here is a kernel build with CONFIG_NO_HZ but booted with
nohz=off; this would be an interesting data point because it includes
all the funny code but still ticks are the right frequency.
> My observations are:
>
> 1. On tickless kernels load is very low where no or both patches
> (74f5187ac8 and 0f004f5a69) are applied.
>
> 2. Kernels that have only patch 74f5187ac8 applied have the smallest
> difference between hz and no-hz variants. Still no-hz kernels are
> returning values lower than their hz siblings.
>
> 3. Non-tickless kernels seem to be reporting correct load values.
> Overall trend and values are matching CPU utilization. Only exception is
> 2.6.32.55-hz which reports the same values as 2.6.32.55-no-hz.
>
> 4. If x processes are using all available cycles load is correctly
> incremented by x. This behavior is consistent on all kernels.
Yay! at least we get something right.. Also, I think we actually will go
down to load 0 if the machine is idle, we used to get that wrong for
nohz too.
> Steps to reproduce: run a bunch of CPU bound processes that will not use
> all available cycles. The biggest difference between expected and
> measured load is around 30% CPU utilization in my case.
Hrmm, this suggests we age too hard with nohz code.. in your test case
is there significant idle time? That is, suppose you run each cpu at 30%
what is the period of you load? Running 3s out of 10s is significantly
different from running .3ms out of 1ms.
> Has there been any other patches that correct load calculation? Maybe
> I'm testing it in a wrong way? I'd appreciate any suggestions. I'd be
> happy to test new patches. Sadly, I cannot propose any fixes as kernel
> sources are still a mystery to me.
Darned load-tracking stuff.. I went over it again but couldn't spot
anything obviously broken. I suspect the tail magic of
calc_global_nohz() is busted, just not seeing it atm.
Will go brew myself a fresh pot of tea and stare more.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-02-29 12:06 ` Peter Zijlstra
@ 2012-02-29 16:24 ` Peter Zijlstra
2012-02-29 17:03 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-02-29 16:24 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
On Wed, 2012-02-29 at 13:06 +0100, Peter Zijlstra wrote:
>
> > Steps to reproduce: run a bunch of CPU bound processes that will not use
> > all available cycles. The biggest difference between expected and
> > measured load is around 30% CPU utilization in my case.
>
> Hrmm, this suggests we age too hard with nohz code.. in your test case
> is there significant idle time? That is, suppose you run each cpu at 30%
> what is the period of you load? Running 3s out of 10s is significantly
> different from running .3ms out of 1ms.
I can indeed see some weirdness, but not only downwards, I can manage to
get a load of 1 with two 20% burners (0.1 ms period). Still need to try
with bigger periods.
> > Has there been any other patches that correct load calculation? Maybe
> > I'm testing it in a wrong way? I'd appreciate any suggestions. I'd be
> > happy to test new patches. Sadly, I cannot propose any fixes as kernel
> > sources are still a mystery to me.
>
> Darned load-tracking stuff.. I went over it again but couldn't spot
> anything obviously broken. I suspect the tail magic of
> calc_global_nohz() is busted, just not seeing it atm.
>
> Will go brew myself a fresh pot of tea and stare more.
The only thing I could find is that on nohz we can confuse the per-rq
sample period, does the below make a difference?
---
kernel/sched/core.c | 9 +--------
kernel/sched/sched.h | 1 -
2 files changed, 1 insertions(+), 9 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d7c4322..370c578 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2372,15 +2372,13 @@ static void calc_load_account_active(struct rq *this_rq)
{
long delta;
- if (time_before(jiffies, this_rq->calc_load_update))
+ if (time_before(jiffies, calc_load_update))
return;
delta = calc_load_fold_active(this_rq);
delta += calc_load_fold_idle();
if (delta)
atomic_long_add(delta, &calc_load_tasks);
-
- this_rq->calc_load_update += LOAD_FREQ;
}
/*
@@ -5329,10 +5327,6 @@ migration_call(struct notifier_block *nfb, unsigned long action, void *hcpu)
switch (action & ~CPU_TASKS_FROZEN) {
- case CPU_UP_PREPARE:
- rq->calc_load_update = calc_load_update;
- break;
-
case CPU_ONLINE:
/* Update our root-domain */
raw_spin_lock_irqsave(&rq->lock, flags);
@@ -6879,7 +6873,6 @@ void __init sched_init(void)
raw_spin_lock_init(&rq->lock);
rq->nr_running = 0;
rq->calc_load_active = 0;
- rq->calc_load_update = jiffies + LOAD_FREQ;
init_cfs_rq(&rq->cfs);
init_rt_rq(&rq->rt, rq);
#ifdef CONFIG_FAIR_GROUP_SCHED
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 8a2c768..59b5a33 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -441,7 +441,6 @@ struct rq {
#endif
/* calc_load related fields */
- unsigned long calc_load_update;
long calc_load_active;
#ifdef CONFIG_SCHED_HRTICK
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-02-29 16:24 ` Peter Zijlstra
@ 2012-02-29 17:03 ` Peter Zijlstra
2012-03-05 19:57 ` Lesław Kopeć
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-02-29 17:03 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
On Wed, 2012-02-29 at 17:24 +0100, Peter Zijlstra wrote:
>
> The only thing I could find is that on nohz we can confuse the per-rq
> sample period, does the below make a difference?
Uhm, something like so that is..
---
kernel/sched/core.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d7c4322..44f61df 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2380,7 +2380,8 @@ static void calc_load_account_active(struct rq *this_rq)
if (delta)
atomic_long_add(delta, &calc_load_tasks);
- this_rq->calc_load_update += LOAD_FREQ;
+ while (!time_before(jiffies, this_rq->calc_load_update))
+ this_rq->calc_load_update += LOAD_FREQ;
}
/*
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-02-29 17:03 ` Peter Zijlstra
@ 2012-03-05 19:57 ` Lesław Kopeć
2012-03-05 22:45 ` Aman Gupta
0 siblings, 1 reply; 15+ messages in thread
From: Lesław Kopeć @ 2012-03-05 19:57 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
[-- Attachment #1: Type: text/plain, Size: 3045 bytes --]
On 29.02.2012 13:06, Peter Zijlstra wrote:
> Missing here is a kernel build with CONFIG_NO_HZ but booted with
> nohz=off; this would be an interesting data point because it includes
> all the funny code but still ticks are the right frequency.
You've asked for it and you got it. I have rebooted some servers with
nohz=off parameter set on kernels complied with CONFIG_NO_HZ=y. They're
the ones listed below with 'off' suffix.
On 29.02.2012 17:24, Peter Zijlstra wrote:
> Hrmm, this suggests we age too hard with nohz code.. in your test case
> is there significant idle time? That is, suppose you run each cpu at 30%
> what is the period of you load? Running 3s out of 10s is significantly
> different from running .3ms out of 1ms.
It's definitely more similar to the second case - very frequent, but
short bursts of activity. A single process does a tiny bit of
computation mixed with a fair amount of network activity on each
request. There are 80 such processes which are responsible for majority
of system load.
On 29.02.2012 18:03, Peter Zijlstra wrote:
>> The only thing I could find is that on nohz we can confuse the per-rq
>> sample period, does the below make a difference?
>
> Uhm, something like so that is..
>
> ---
> kernel/sched/core.c | 3 ++-
> 1 files changed, 2 insertions(+), 1 deletions(-)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index d7c4322..44f61df 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2380,7 +2380,8 @@ static void calc_load_account_active(struct rq *this_rq)
> if (delta)
> atomic_long_add(delta, &calc_load_tasks);
>
> - this_rq->calc_load_update += LOAD_FREQ;
> + while (!time_before(jiffies, this_rq->calc_load_update))
> + this_rq->calc_load_update += LOAD_FREQ;
> }
>
> /*
>
I have compiled another batch of kernels with this patch applied
(they're the ones with 'patch0' suffix). The only difference was the
patch had to go to kernel/sched.c, but that's what you get when not
using the latest sources. Anyway, here are the results accompanied by a
pretty picture [1]:
std off patch0
2.6.32.55-no-hz 0.76 0.91 -
2.6.32.55-no-hz-74f5187ac8 6.41 9.40 4.93
2.6.32.55-no-hz-0f004f5a69 0.78 0.92 0.90
2.6.37-rc5-no-hz-0f004f5a69 0.95 0.92 0.86
2.6.37-rc5-no-hz-pre-0f004f5a69 9.16 10.47 8.02
It seems that the patch didn't help much on kernels with 0f004f5a69
applied. The ones with just 74f5187ac8 are reporting a more plausible
values, but slightly lower than the ones without patch0. Am I right to
assume that the correct load values are the ones produced by kernels
complied with CONFIG_NO_HZ=n? Should they be the baseline?
I can run additional tests if you have other leads to follow. Is there a
particular kernel version I should focus on? If not I will continue
to use the current bundle. I'm also planning to give the latest stable
release a spin.
[1] http://img835.imageshack.us/img835/2204/kernelload.png
--
Lesław Kopeć
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-03-05 19:57 ` Lesław Kopeć
@ 2012-03-05 22:45 ` Aman Gupta
2012-03-05 23:25 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Aman Gupta @ 2012-03-05 22:45 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Peter Zijlstra, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder
On Mon, Mar 5, 2012 at 11:57 AM, Lesław Kopeć
<leslaw.kopec@nasza-klasa.pl> wrote:
> On 29.02.2012 13:06, Peter Zijlstra wrote:
>
>> Missing here is a kernel build with CONFIG_NO_HZ but booted with
>> nohz=off; this would be an interesting data point because it includes
>> all the funny code but still ticks are the right frequency.
>
> You've asked for it and you got it. I have rebooted some servers with
> nohz=off parameter set on kernels complied with CONFIG_NO_HZ=y. They're
> the ones listed below with 'off' suffix.
>
> On 29.02.2012 17:24, Peter Zijlstra wrote:
>
>> Hrmm, this suggests we age too hard with nohz code.. in your test case
>> is there significant idle time? That is, suppose you run each cpu at 30%
>> what is the period of you load? Running 3s out of 10s is significantly
>> different from running .3ms out of 1ms.
>
> It's definitely more similar to the second case - very frequent, but
> short bursts of activity. A single process does a tiny bit of
> computation mixed with a fair amount of network activity on each
> request. There are 80 such processes which are responsible for majority
> of system load.
>
> On 29.02.2012 18:03, Peter Zijlstra wrote:
>
>>> The only thing I could find is that on nohz we can confuse the per-rq
>>> sample period, does the below make a difference?
>>
>> Uhm, something like so that is..
>>
>> ---
>> kernel/sched/core.c | 3 ++-
>> 1 files changed, 2 insertions(+), 1 deletions(-)
>>
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index d7c4322..44f61df 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -2380,7 +2380,8 @@ static void calc_load_account_active(struct rq *this_rq)
>> if (delta)
>> atomic_long_add(delta, &calc_load_tasks);
>>
>> - this_rq->calc_load_update += LOAD_FREQ;
>> + while (!time_before(jiffies, this_rq->calc_load_update))
>> + this_rq->calc_load_update += LOAD_FREQ;
>> }
>>
>> /*
>>
>
> I have compiled another batch of kernels with this patch applied
> (they're the ones with 'patch0' suffix). The only difference was the
> patch had to go to kernel/sched.c, but that's what you get when not
> using the latest sources. Anyway, here are the results accompanied by a
> pretty picture [1]:
>
> std off patch0
> 2.6.32.55-no-hz 0.76 0.91 -
> 2.6.32.55-no-hz-74f5187ac8 6.41 9.40 4.93
> 2.6.32.55-no-hz-0f004f5a69 0.78 0.92 0.90
> 2.6.37-rc5-no-hz-0f004f5a69 0.95 0.92 0.86
> 2.6.37-rc5-no-hz-pre-0f004f5a69 9.16 10.47 8.02
>
> It seems that the patch didn't help much on kernels with 0f004f5a69
> applied. The ones with just 74f5187ac8 are reporting a more plausible
> values, but slightly lower than the ones without patch0. Am I right to
> assume that the correct load values are the ones produced by kernels
> complied with CONFIG_NO_HZ=n? Should they be the baseline?
>
> I can run additional tests if you have other leads to follow. Is there a
> particular kernel version I should focus on? If not I will continue
> to use the current bundle. I'm also planning to give the latest stable
> release a spin.
I can confirm these results on 3.2.8. Booting with nohz=off makes no
difference. Applying the patch to kernel/sched.c made no noticeable
difference either.
Aman
>
>
> [1] http://img835.imageshack.us/img835/2204/kernelload.png
>
> --
> Lesław Kopeć
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-03-05 22:45 ` Aman Gupta
@ 2012-03-05 23:25 ` Peter Zijlstra
2012-03-05 23:32 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-03-05 23:25 UTC (permalink / raw)
To: Aman Gupta
Cc: Lesław Kopeć,
linux-kernel, Chase Douglas, Damien Wyart, Kyle McMartin,
Venkatesh Pallipadi, Jonathan Nieder, Doug Smythies
Doug actually spotted the problem and reported it off-list.
The below patch appears to sort the issue, but I haven't been able to
test the very long idle path simply because x86 doesn't go idle that
long.
I tried writing hpet64 support so we could idle that long, killed all
kinds of stupid kernel threads (watchdogs mostly) that keep waking up
and got a brick..
Clearly I need to try again... but I thought I'd at least share this
stuff.
---
Subject: sched: Fix nohz load accounting -- again!
From: Peter Zijlstra <peterz@infradead.org>
Date: Thu, 01 Mar 2012 15:04:46 +0100
Various people reported nohz load tracking still being wrecked, but Doug
spotted the actual problem. We fold the nohz remainder in too soon,
causing us to loose samples and under-account.
So instead of playing catch-up up-front, always do a single load-fold
with whatever state we encounter and only then fold the nohz remainder
and play catch-up.
Reported-by: Doug Smythies <dsmythies@telus.net>
Reported-by: Les�=82aw Kope=C4=87 <leslaw.kopec@nasza-klasa.pl>
Reported-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
---
kernel/sched/core.c | 53 +++++++++++++++++++++++++--------------------------
1 files changed, 26 insertions(+), 27 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b83e8d0..6ffde97 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2266,13 +2266,10 @@ calc_load_n(unsigned long load, unsigned long exp,
* Once we've updated the global active value, we need to apply the exponential
* weights adjusted to the number of cycles missed.
*/
-static void calc_global_nohz(unsigned long ticks)
+static void calc_global_nohz(void)
{
long delta, active, n;
- if (time_before(jiffies, calc_load_update))
- return;
-
/*
* If we crossed a calc_load_update boundary, make sure to fold
* any pending idle changes, the respective CPUs might have
@@ -2284,31 +2281,25 @@ static void calc_global_nohz(unsigned long ticks)
atomic_long_add(delta, &calc_load_tasks);
/*
- * If we were idle for multiple load cycles, apply them.
+ * It could be the one fold was all it took, we done!
*/
- if (ticks >= LOAD_FREQ) {
- n = ticks / LOAD_FREQ;
+ if (time_before(jiffies, calc_load_update + 10))
+ return;
- active = atomic_long_read(&calc_load_tasks);
- active = active > 0 ? active * FIXED_1 : 0;
+ /*
+ * Catch-up, fold however many we are behind still
+ */
+ delta = jiffies - calc_load_update - 10;
+ n = 1 + (delta / LOAD_FREQ);
- avenrun[0] = calc_load_n(avenrun[0], EXP_1, active, n);
- avenrun[1] = calc_load_n(avenrun[1], EXP_5, active, n);
- avenrun[2] = calc_load_n(avenrun[2], EXP_15, active, n);
+ active = atomic_long_read(&calc_load_tasks);
+ active = active > 0 ? active * FIXED_1 : 0;
- calc_load_update += n * LOAD_FREQ;
- }
+ avenrun[0] = calc_load_n(avenrun[0], EXP_1, active, n);
+ avenrun[1] = calc_load_n(avenrun[1], EXP_5, active, n);
+ avenrun[2] = calc_load_n(avenrun[2], EXP_15, active, n);
- /*
- * Its possible the remainder of the above division also crosses
- * a LOAD_FREQ period, the regular check in calc_global_load()
- * which comes after this will take care of that.
- *
- * Consider us being 11 ticks before a cycle completion, and us
- * sleeping for 4*LOAD_FREQ + 22 ticks, then the above code will
- * age us 4 cycles, and the test in calc_global_load() will
- * pick up the final one.
- */
+ calc_load_update += n * LOAD_FREQ;
}
#else
void calc_load_account_idle(struct rq *this_rq)
@@ -2320,7 +2311,7 @@ static inline long calc_load_fold_idle(void)
return 0;
}
-static void calc_global_nohz(unsigned long ticks)
+static void calc_global_nohz(void)
{
}
#endif
@@ -2348,8 +2339,6 @@ void calc_global_load(unsigned long ticks)
{
long active;
- calc_global_nohz(ticks);
-
if (time_before(jiffies, calc_load_update + 10))
return;
@@ -2361,6 +2350,16 @@ void calc_global_load(unsigned long ticks)
avenrun[2] = calc_load(avenrun[2], EXP_15, active);
calc_load_update += LOAD_FREQ;
+
+ /*
+ * Account one period with whatever state we found before
+ * folding in the nohz state and ageing the entire idle period.
+ *
+ * This avoids loosing a sample when we go idle between
+ * calc_load_account_active() (10 ticks ago) and now and thus
+ * under-accounting.
+ */
+ calc_global_nohz();
}
/*
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-03-05 23:25 ` Peter Zijlstra
@ 2012-03-05 23:32 ` Peter Zijlstra
2012-03-05 23:33 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-03-05 23:32 UTC (permalink / raw)
To: Aman Gupta
Cc: Lesław Kopeć,
linux-kernel, Chase Douglas, Damien Wyart, Kyle McMartin,
Venkatesh Pallipadi, Jonathan Nieder, Doug Smythies,
Thomas Gleixner
On Tue, 2012-03-06 at 00:25 +0100, Peter Zijlstra wrote:
> I tried writing hpet64 support so we could idle that long, killed all
> kinds of stupid kernel threads (watchdogs mostly) that keep waking up
> and got a brick..
Just in case someone wants to have a go at fixing this mess.. :-)
I _think_ the below was the latest, but it was 2am on friday night or
something, so recollections are somewhat hazy.
---
diff --git a/arch/x86/kernel/hpet.c b/arch/x86/kernel/hpet.c
index ad0de0c..fd2aab0 100644
--- a/arch/x86/kernel/hpet.c
+++ b/arch/x86/kernel/hpet.c
@@ -17,6 +17,7 @@
#include <asm/time.h>
#define HPET_MASK CLOCKSOURCE_MASK(32)
+#define HPET64_MASK CLOCKSOURCE_MASK(64)
/* FSEC = 10^-15
NSEC = 10^-9 */
@@ -43,6 +44,12 @@ static unsigned long hpet_num_timers;
#endif
static void __iomem *hpet_virt_address;
+#ifdef X86_64
+static int hpet64;
+#else
+ #define hpet64 (0)
+#endif
+
struct hpet_dev {
struct clock_event_device evt;
unsigned int num;
@@ -67,6 +74,26 @@ static inline void hpet_writel(unsigned int d, unsigned int a)
writel(d, hpet_virt_address + a);
}
+inline u64 hpet_read(unsigned int a)
+{
+ u64 ret;
+
+ if (hpet64)
+ ret = readq(hpet_virt_address + a);
+ else
+ ret = readl(hpet_virt_address + a);
+
+ return ret;
+}
+
+static inline void hpet_write(u64 d, unsigned int a)
+{
+ if (hpet64)
+ writeq(d, hpet_virt_address + a);
+ else
+ writel(d, hpet_virt_address + a);
+}
+
#ifdef CONFIG_X86_64
#include <asm/pgtable.h>
#endif
@@ -91,6 +118,10 @@ static inline void hpet_clear_mapping(void)
static int boot_hpet_disable;
int hpet_force_user;
static int hpet_verbose;
+#ifdef X86_64
+static int hpet_force_64;
+static int hpet_force_32;
+#endif
static int __init hpet_setup(char *str)
{
@@ -101,6 +132,12 @@ static int __init hpet_setup(char *str)
hpet_force_user = 1;
if (!strncmp("verbose", str, 7))
hpet_verbose = 1;
+#ifdef X86_64
+ if(!strncmp("force64", str, 7))
+ hpet_force_64 = 1;
+ if(!strncmp("force32", str, 7))
+ hpet_force_32 = 1;
+#endif
}
return 1;
}
@@ -249,8 +286,11 @@ static void hpet_stop_counter(void)
static void hpet_reset_counter(void)
{
- hpet_writel(0, HPET_COUNTER);
- hpet_writel(0, HPET_COUNTER + 4);
+ if (!hpet64) {
+ hpet_writel(0, HPET_COUNTER);
+ hpet_writel(0, HPET_COUNTER + 4);
+ } else
+ hpet_write(0, HPET_COUNTER);
}
static void hpet_start_counter(void)
@@ -298,7 +338,8 @@ static void hpet_legacy_clockevent_register(void)
*/
hpet_clockevent.cpumask = cpumask_of(smp_processor_id());
clockevents_config_and_register(&hpet_clockevent, hpet_freq,
- HPET_MIN_PROG_DELTA, 0x7FFFFFFF);
+ HPET_MIN_PROG_DELTA,
+ hpet64 ? 0x7FFFFFFFFFFFFFFF : 0x7FFFFFFF);
global_clock_event = &hpet_clockevent;
printk(KERN_DEBUG "hpet clockevent registered\n");
}
@@ -308,23 +349,25 @@ static int hpet_setup_msi_irq(unsigned int irq);
static void hpet_set_mode(enum clock_event_mode mode,
struct clock_event_device *evt, int timer)
{
- unsigned int cfg, cmp, now;
- uint64_t delta;
+ uint64_t delta, cmp, now;
+ unsigned int cfg;
switch (mode) {
case CLOCK_EVT_MODE_PERIODIC:
hpet_stop_counter();
delta = ((uint64_t)(NSEC_PER_SEC/HZ)) * evt->mult;
delta >>= evt->shift;
- now = hpet_readl(HPET_COUNTER);
+ now = hpet_read(HPET_COUNTER);
cmp = now + (unsigned int) delta;
cfg = hpet_readl(HPET_Tn_CFG(timer));
/* Make sure we use edge triggered interrupts */
cfg &= ~HPET_TN_LEVEL;
cfg |= HPET_TN_ENABLE | HPET_TN_PERIODIC |
- HPET_TN_SETVAL | HPET_TN_32BIT;
+ HPET_TN_SETVAL;
+ if (!hpet64)
+ cfg |= HPET_TN_32BIT;
hpet_writel(cfg, HPET_Tn_CFG(timer));
- hpet_writel(cmp, HPET_Tn_CMP(timer));
+ hpet_write(cmp, HPET_Tn_CMP(timer));
udelay(1);
/*
* HPET on AMD 81xx needs a second write (with HPET_TN_SETVAL
@@ -333,7 +376,7 @@ static void hpet_set_mode(enum clock_event_mode mode,
* (See AMD-8111 HyperTransport I/O Hub Data Sheet,
* Publication # 24674)
*/
- hpet_writel((unsigned int) delta, HPET_Tn_CMP(timer));
+ hpet_write(delta, HPET_Tn_CMP(timer));
hpet_start_counter();
hpet_print_config();
break;
@@ -341,7 +384,9 @@ static void hpet_set_mode(enum clock_event_mode mode,
case CLOCK_EVT_MODE_ONESHOT:
cfg = hpet_readl(HPET_Tn_CFG(timer));
cfg &= ~HPET_TN_PERIODIC;
- cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
+ cfg |= HPET_TN_ENABLE;
+ if (!hpet64)
+ cfg |= HPET_TN_32BIT;
hpet_writel(cfg, HPET_Tn_CFG(timer));
break;
@@ -370,12 +415,12 @@ static void hpet_set_mode(enum clock_event_mode mode,
static int hpet_next_event(unsigned long delta,
struct clock_event_device *evt, int timer)
{
- u32 cnt;
- s32 res;
+ u64 cnt;
+ s64 res;
- cnt = hpet_readl(HPET_COUNTER);
- cnt += (u32) delta;
- hpet_writel(cnt, HPET_Tn_CMP(timer));
+ cnt = hpet_read(HPET_COUNTER);
+ cnt += delta;
+ hpet_write(cnt, HPET_Tn_CMP(timer));
/*
* HPETs are a complete disaster. The compare register is
@@ -399,7 +444,7 @@ static int hpet_next_event(unsigned long delta,
* the event. The minimum programming delta for the generic
* clockevents code is set to 1.5 * HPET_MIN_CYCLES.
*/
- res = (s32)(cnt - hpet_readl(HPET_COUNTER));
+ res = (s64)(cnt - hpet_read(HPET_COUNTER));
return res < HPET_MIN_CYCLES ? -ETIME : 0;
}
@@ -739,7 +784,7 @@ static int hpet_cpuhp_notify(struct notifier_block *n,
*/
static cycle_t read_hpet(struct clocksource *cs)
{
- return (cycle_t)hpet_readl(HPET_COUNTER);
+ return (cycle_t)hpet_read(HPET_COUNTER);
}
static struct clocksource clocksource_hpet = {
@@ -763,7 +808,7 @@ static int hpet_clocksource_register(void)
hpet_restart_counter();
/* Verify whether hpet counter works */
- t1 = hpet_readl(HPET_COUNTER);
+ t1 = hpet_read(HPET_COUNTER);
rdtscll(start);
/*
@@ -777,7 +822,7 @@ static int hpet_clocksource_register(void)
rdtscll(now);
} while ((now - start) < 200000UL);
- if (t1 == hpet_readl(HPET_COUNTER)) {
+ if (t1 == hpet_read(HPET_COUNTER)) {
printk(KERN_WARNING
"HPET counter not counting. HPET disabled\n");
return -ENODEV;
@@ -847,6 +892,13 @@ int __init hpet_enable(void)
id = hpet_readl(HPET_ID);
hpet_print_config();
+#ifdef X86_64
+ if (((id & HPET_ID_64BIT) || hpet_force_64) && !hpet_force_32) {
+ hpet64 = 1;
+ clocksource_hpet.mask = HPET64_MASK;
+ }
+#endif
+
#ifdef CONFIG_HPET_EMULATE_RTC
/*
* The legacy routing mode needs at least two channels, tick timer
@@ -962,9 +1014,9 @@ static unsigned long hpet_rtc_flags;
static int hpet_prev_update_sec;
static struct rtc_time hpet_alarm_time;
static unsigned long hpet_pie_count;
-static u32 hpet_t1_cmp;
-static u32 hpet_default_delta;
-static u32 hpet_pie_delta;
+static u64 hpet_t1_cmp;
+static u64 hpet_default_delta;
+static u64 hpet_pie_delta;
static unsigned long hpet_pie_limit;
static rtc_irq_handler irq_handler;
@@ -972,9 +1024,9 @@ static rtc_irq_handler irq_handler;
/*
* Check that the hpet counter c1 is ahead of the c2
*/
-static inline int hpet_cnt_ahead(u32 c1, u32 c2)
+static inline int hpet_cnt_ahead(u64 c1, u64 c2)
{
- return (s32)(c2 - c1) < 0;
+ return (s64)(c2 - c1) < 0;
}
/*
@@ -1015,7 +1067,8 @@ EXPORT_SYMBOL_GPL(hpet_unregister_irq_handler);
*/
int hpet_rtc_timer_init(void)
{
- unsigned int cfg, cnt, delta;
+ unsigned int cfg;
+ u64 cnt, delta;
unsigned long flags;
if (!is_hpet_enabled())
@@ -1036,13 +1089,15 @@ int hpet_rtc_timer_init(void)
local_irq_save(flags);
- cnt = delta + hpet_readl(HPET_COUNTER);
- hpet_writel(cnt, HPET_T1_CMP);
+ cnt = delta + hpet_read(HPET_COUNTER);
+ hpet_write(cnt, HPET_T1_CMP);
hpet_t1_cmp = cnt;
cfg = hpet_readl(HPET_T1_CFG);
cfg &= ~HPET_TN_PERIODIC;
- cfg |= HPET_TN_ENABLE | HPET_TN_32BIT;
+ cfg |= HPET_TN_ENABLE;
+ if (!hpet64)
+ cfg |= HPET_TN_32BIT;
hpet_writel(cfg, HPET_T1_CFG);
local_irq_restore(flags);
@@ -1155,9 +1210,9 @@ static void hpet_rtc_timer_reinit(void)
*/
do {
hpet_t1_cmp += delta;
- hpet_writel(hpet_t1_cmp, HPET_T1_CMP);
+ hpet_write(hpet_t1_cmp, HPET_T1_CMP);
lost_ints++;
- } while (!hpet_cnt_ahead(hpet_t1_cmp, hpet_readl(HPET_COUNTER)));
+ } while (!hpet_cnt_ahead(hpet_t1_cmp, hpet_read(HPET_COUNTER)));
if (lost_ints) {
if (hpet_rtc_flags & RTC_PIE)
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-03-05 23:32 ` Peter Zijlstra
@ 2012-03-05 23:33 ` Peter Zijlstra
2012-04-17 12:52 ` Lesław Kopeć
0 siblings, 1 reply; 15+ messages in thread
From: Peter Zijlstra @ 2012-03-05 23:33 UTC (permalink / raw)
To: Aman Gupta
Cc: Lesław Kopeć,
linux-kernel, Chase Douglas, Damien Wyart, Kyle McMartin,
Venkatesh Pallipadi, Jonathan Nieder, Doug Smythies,
Thomas Gleixner
On Tue, 2012-03-06 at 00:32 +0100, Peter Zijlstra wrote:
> +#ifdef X86_64
Ah see it wasn't, this patch wants s/\<X86_64\>/CONFIG_&/g done on it.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-03-05 23:33 ` Peter Zijlstra
@ 2012-04-17 12:52 ` Lesław Kopeć
2012-04-17 15:30 ` Jonathan Nieder
0 siblings, 1 reply; 15+ messages in thread
From: Lesław Kopeć @ 2012-04-17 12:52 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Jonathan Nieder,
Doug Smythies, Thomas Gleixner
[-- Attachment #1: Type: text/plain, Size: 1424 bytes --]
Hello!
I've finally finished testing patch c308b56b53. Apologies for the delay
in reporting back. This time I've compared kernels from 3.2 and 2.6.32
branches. Here's a snapshot of load 15 on different versions:
2.6.32.55-hz-0f004f5a69 12.78
2.6.32.55-no-hz-74f5187ac8 4.42
2.6.32.55-no-hz-0f004f5a69 0.49
2.6.32.55-no-hz-c308b56b53 4.37
3.2.12-hz 12.85
3.2.12-no-hz 0.65
3.2.12-no-hz-c308b56b53 7.25
3.2.12-no-hz-c308b56b53 nohz=off 10.59
A whole day trend is available on a comparison chart [1].
Just to make things clear 2.6.32 kernels were patched as follows:
* 74f5187ac8 - just 74f5187ac8
* 0f004f5a69 - 74f5187ac8 + 0f004f5a69
* c308b56b53 - 74f5187ac8 + 0f004f5a69 + c308b56b53
For kernel 3.2.12 patch c308b56b53 seems almost perfect. For low CPU
utilization the load value is slightly lower for NO_HZ version than it
is for HZ one. However the difference is small and the overall trend
relates to CPU usage quite closely. This is definitely the best match so
far. Thanks!
Looking at results for 2.6.32.55 branch it seems that we're back at
74f5187ac8 patch - the values are almost the same. The difference
between NO_HZ and HZ versions is noticeable. At the risk of sounding
like an ungrateful bastard - will there be further attempts at fixing
this bug for 2.6.32 kernels?
[1] http://img535.imageshack.us/img535/2204/kernelload.png
--
Lesław Kopeć
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-04-17 12:52 ` Lesław Kopeć
@ 2012-04-17 15:30 ` Jonathan Nieder
2012-04-23 16:20 ` Lesław Kopeć
0 siblings, 1 reply; 15+ messages in thread
From: Jonathan Nieder @ 2012-04-17 15:30 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Peter Zijlstra, Aman Gupta, linux-kernel, Chase Douglas,
Damien Wyart, Kyle McMartin, Venkatesh Pallipadi, Doug Smythies,
Thomas Gleixner
Lesław Kopeć wrote:
> I've finally finished testing patch c308b56b53. Apologies for the delay
> in reporting back. This time I've compared kernels from 3.2 and 2.6.32
> branches. Here's a snapshot of load 15 on different versions:
[...]
> Just to make things clear 2.6.32 kernels were patched as follows:
> * 74f5187ac8 - just 74f5187ac8
> * 0f004f5a69 - 74f5187ac8 + 0f004f5a69
> * c308b56b53 - 74f5187ac8 + 0f004f5a69 + c308b56b53
>
> For kernel 3.2.12 patch c308b56b53 seems almost perfect. For low CPU
> utilization the load value is slightly lower for NO_HZ version than it
> is for HZ one. However the difference is small and the overall trend
> relates to CPU usage quite closely.
Nice.
[...]
> Looking at results for 2.6.32.55 branch it seems that we're back at
> 74f5187ac8 patch - the values are almost the same. The difference
> between NO_HZ and HZ versions is noticeable.
How does 3.0.y + c308b56b53 do? (I ask because the usual flow of
fixes is mainline -> 3.3.y -> 3.2.y -> 3.0.y -> 2.6.32.y with the
first three steps happening pretty quickly, so it we can get this
working on 3.0.y then that would be progress. Also because, like
2.6.32.y, 3.0.y is longterm maintained, so it might be useful in the
meantime.)
Curious,
Jonathan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-04-17 15:30 ` Jonathan Nieder
@ 2012-04-23 16:20 ` Lesław Kopeć
2012-04-23 17:57 ` Jonathan Nieder
0 siblings, 1 reply; 15+ messages in thread
From: Lesław Kopeć @ 2012-04-23 16:20 UTC (permalink / raw)
To: Jonathan Nieder
Cc: Peter Zijlstra, Aman Gupta, linux-kernel, Chase Douglas,
Damien Wyart, Kyle McMartin, Venkatesh Pallipadi, Doug Smythies,
Thomas Gleixner
[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]
On 04/17/2012 05:30 PM, Jonathan Nieder wrote:
>> Looking at results for 2.6.32.55 branch it seems that we're back at
>> 74f5187ac8 patch - the values are almost the same. The difference
>> between NO_HZ and HZ versions is noticeable.
>
> How does 3.0.y + c308b56b53 do? (I ask because the usual flow of
> fixes is mainline -> 3.3.y -> 3.2.y -> 3.0.y -> 2.6.32.y with the
> first three steps happening pretty quickly, so it we can get this
> working on 3.0.y then that would be progress. Also because, like
> 2.6.32.y, 3.0.y is longterm maintained, so it might be useful in the
> meantime.)
It seems that load reported by 3.0.28 with and without c308b56b53 patch
is almost identical to 3.2.12. This might be more clearly visible on a
comparison chart. [1]
2.6.32.55-hz-0f004f5a69 9.88
2.6.32.55-no-hz-74f5187ac8 2.48
2.6.32.55-no-hz-c308b56b53 2.22
3.0.28-hz 10.66
3.0.28-no-hz 0.60
3.0.28-no-hz-c308b56b53 4.09
3.0.28-no-hz-c308b56b53 nohz=off 6.78
3.2.12-hz 10.16
3.2.12-no-hz 0.66
3.2.12-no-hz-c308b56b53 4.36
What's worth noting is that I haven't seen any nasty side effects of the
latest patch on all kernel versions that I've tested. Hope that helps.
[1] http://img855.imageshack.us/img855/2204/kernelload.png
--
Lesław Kopeć
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-04-23 16:20 ` Lesław Kopeć
@ 2012-04-23 17:57 ` Jonathan Nieder
2012-04-23 20:21 ` Peter Zijlstra
0 siblings, 1 reply; 15+ messages in thread
From: Jonathan Nieder @ 2012-04-23 17:57 UTC (permalink / raw)
To: Lesław Kopeć
Cc: Peter Zijlstra, Aman Gupta, linux-kernel, Chase Douglas,
Damien Wyart, Kyle McMartin, Venkatesh Pallipadi, Doug Smythies,
Thomas Gleixner
Lesław Kopeć wrote:
> 2.6.32.55-hz-0f004f5a69 9.88
> 2.6.32.55-no-hz-74f5187ac8 2.48
> 2.6.32.55-no-hz-c308b56b53 2.22
> 3.0.28-hz 10.66
> 3.0.28-no-hz 0.60
> 3.0.28-no-hz-c308b56b53 4.09
> 3.0.28-no-hz-c308b56b53 nohz=off 6.78
> 3.2.12-hz 10.16
> 3.2.12-no-hz 0.66
> 3.2.12-no-hz-c308b56b53 4.36
>
> What's worth noting is that I haven't seen any nasty side effects of the
> latest patch on all kernel versions that I've tested. Hope that helps.
Ok.
Peter, do you think it would make sense to add c308b56b53987 to 3.0.y
and newer stable kernels?
Thanks,
Jonathan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Inconsistent load average on tickless kernels
2012-04-23 17:57 ` Jonathan Nieder
@ 2012-04-23 20:21 ` Peter Zijlstra
0 siblings, 0 replies; 15+ messages in thread
From: Peter Zijlstra @ 2012-04-23 20:21 UTC (permalink / raw)
To: Jonathan Nieder
Cc: Lesław Kopeć,
Aman Gupta, linux-kernel, Chase Douglas, Damien Wyart,
Kyle McMartin, Venkatesh Pallipadi, Doug Smythies,
Thomas Gleixner
On Mon, 2012-04-23 at 12:57 -0500, Jonathan Nieder wrote:
> Lesław Kopeć wrote:
>
> > 2.6.32.55-hz-0f004f5a69 9.88
> > 2.6.32.55-no-hz-74f5187ac8 2.48
> > 2.6.32.55-no-hz-c308b56b53 2.22
> > 3.0.28-hz 10.66
> > 3.0.28-no-hz 0.60
> > 3.0.28-no-hz-c308b56b53 4.09
> > 3.0.28-no-hz-c308b56b53 nohz=off 6.78
> > 3.2.12-hz 10.16
> > 3.2.12-no-hz 0.66
> > 3.2.12-no-hz-c308b56b53 4.36
> >
> > What's worth noting is that I haven't seen any nasty side effects of the
> > latest patch on all kernel versions that I've tested. Hope that helps.
>
> Ok.
>
> Peter, do you think it would make sense to add c308b56b53987 to 3.0.y
> and newer stable kernels?
If anybody is still maintaining those and cares enough to do the work,
sure whatever.. ;)
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2012-04-23 20:21 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-02-06 6:51 Inconsistent load average on tickless kernels Aman Gupta
2012-02-23 15:46 ` Lesław Kopeć
2012-02-29 12:06 ` Peter Zijlstra
2012-02-29 16:24 ` Peter Zijlstra
2012-02-29 17:03 ` Peter Zijlstra
2012-03-05 19:57 ` Lesław Kopeć
2012-03-05 22:45 ` Aman Gupta
2012-03-05 23:25 ` Peter Zijlstra
2012-03-05 23:32 ` Peter Zijlstra
2012-03-05 23:33 ` Peter Zijlstra
2012-04-17 12:52 ` Lesław Kopeć
2012-04-17 15:30 ` Jonathan Nieder
2012-04-23 16:20 ` Lesław Kopeć
2012-04-23 17:57 ` Jonathan Nieder
2012-04-23 20:21 ` Peter Zijlstra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).