linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Clocksource tsc unstable (delta = -8589361717 ns) in current git
@ 2010-10-26 11:20 Markus Trippelsdorf
  2010-10-26 13:18 ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-26 11:20 UTC (permalink / raw)
  To: linux-kernel; +Cc: borislav.petkov, hpa

[-- Attachment #1: Type: text/plain, Size: 800 bytes --]

On my AMD system the following kernel messages are sometimes printed
after several hours of uptime:
Clocksource tsc unstable (delta = -8589361717 ns)
Switching to clocksource hpet

This happened several times already:
# grep "Clocksource" kernel.*
kernel.log:Oct 24 18:31:17 arch kernel: Clocksource tsc unstable (delta = -8591703466 ns)
kernel.log:Oct 26 12:54:38 arch kernel: Clocksource tsc unstable (delta = -8589361717 ns)
kernel.log.1:Oct 22 21:43:51 arch kernel: Clocksource tsc unstable (delta = -25772316650 ns)

The computer is nearly unusable afterwards (second long delays, that
seem to grow larger and larger) and only a prompt reboot can fix the
problem.
I'm running the latest git kernel. Version 2.6.36 does not show these
symptoms.

Dmesg and kernel config are attached.

-- 
Markus

[-- Attachment #2: dmesg --]
[-- Type: text/plain, Size: 27436 bytes --]

Initializing cgroup subsys cpu
Linux version 2.6.36-05618-gb18cae4-dirty (markus@arch.trippelsdorf.de) (gcc version 4.5.1 (GCC) ) #41 SMP PREEMPT Tue Oct 26 06:51:47 CEST 2010
Command line: root=/dev/sdb2 fbcon=rotate:3 quiet kexec_jump_back_entry=0x23d64482b7e72bb7
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000100 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e6000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dfe90000 (usable)
 BIOS-e820: 00000000dfe90000 - 00000000dfea8000 (ACPI data)
 BIOS-e820: 00000000dfea8000 - 00000000dfed0000 (ACPI NVS)
 BIOS-e820: 00000000dfed0000 - 00000000dff00000 (reserved)
 BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
NX (Execute Disable) protection: active
DMI present.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x120000 max_arch_pfn = 0x400000000
MTRR default type: uncachable
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-EFFFF uncachable
  F0000-FFFFF write-protect
MTRR variable ranges enabled:
  0 base 000000000000 mask FFFF80000000 write-back
  1 base 000080000000 mask FFFFC0000000 write-back
  2 base 0000C0000000 mask FFFFE0000000 write-back
  3 base 0000F0000000 mask FFFFF8000000 write-combining
  4 disabled
  5 disabled
  6 disabled
  7 disabled
TOM2: 0000000120000000 aka 4608M
x86 PAT enabled: cpu 0, old 0x7010600070106, new 0x7010600070106
last_pfn = 0xdfe90 max_arch_pfn = 0x400000000
initial memory mapped : 0 - 20000000
Using GB pages for direct mapping
init_memory_mapping: 0000000000000000-00000000dfe90000
 0000000000 - 00c0000000 page 1G
 00c0000000 - 00dfe00000 page 2M
 00dfe00000 - 00dfe90000 page 4k
kernel direct mapping tables up to dfe90000 @ 1fffd000-20000000
init_memory_mapping: 0000000100000000-0000000120000000
 0100000000 - 0120000000 page 2M
kernel direct mapping tables up to 120000000 @ dfe8e000-dfe90000
ACPI: RSDP 00000000000fb880 00024 (v02 ACPIAM)
ACPI: XSDT 00000000dfe90100 0005C (v01 041910 XSDT2028 20100419 MSFT 00000097)
ACPI: FACP 00000000dfe90290 000F4 (v03 041910 FACP2028 20100419 MSFT 00000097)
ACPI Warning: Optional field Pm2ControlBlock has zero address or length: 0x0000000000000000/0x1 (20100702/tbfadt-557)
ACPI: DSDT 00000000dfe90450 0E6FE (v01  A1152 A1152000 00000000 INTL 20060113)
ACPI: FACS 00000000dfea8000 00040
ACPI: APIC 00000000dfe90390 0007C (v01 041910 APIC2028 20100419 MSFT 00000097)
ACPI: MCFG 00000000dfe90410 0003C (v01 041910 OEMMCFG  20100419 MSFT 00000097)
ACPI: OEMB 00000000dfea8040 00072 (v01 041910 OEMB2028 20100419 MSFT 00000097)
ACPI: SRAT 00000000dfe9f450 000E8 (v03 AMD    FAM_F_10 00000002 AMD  00000001)
ACPI: HPET 00000000dfe9f540 00038 (v01 041910 OEMHPET  20100419 MSFT 00000097)
ACPI: SSDT 00000000dfe9f580 0088C (v01 A M I  POWERNOW 00000001 AMD  00000001)
ACPI: Local APIC address 0xfee00000
 [ffffea0000000000-ffffea0003ffffff] PMD -> [ffff8800d0000000-ffff8800d39fffff] on node 0
Zone PFN ranges:
  DMA      0x00000010 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00120000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000010 -> 0x0000009f
    0: 0x00000100 -> 0x000dfe90
    0: 0x00100000 -> 0x00120000
On node 0 totalpages: 1048095
  DMA zone: 56 pages used for memmap
  DMA zone: 2 pages reserved
  DMA zone: 3925 pages, LIFO batch:0
  DMA32 zone: 14280 pages used for memmap
  DMA32 zone: 898760 pages, LIFO batch:31
  Normal zone: 1792 pages used for memmap
  Normal zone: 129280 pages, LIFO batch:31
ACPI: PM-Timer IO Port: 0x808
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
ACPI: LAPIC (acpi_id[0x05] lapic_id[0x84] disabled)
ACPI: LAPIC (acpi_id[0x06] lapic_id[0x85] disabled)
ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8300 base: 0xfed00000
SMP: Allowing 4 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 40
Allocating PCI resources starting at dff00000 (gap: dff00000:20000000)
converting mcount calls to 0f 1f 44 00 00
setup_percpu: NR_CPUS:4 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 23 pages/cpu @ffff8800dfc00000 s73472 r0 d20736 u524288
pcpu-alloc: s73472 r0 d20736 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1031965
Kernel command line: root=/dev/sdb2 fbcon=rotate:3 quiet kexec_jump_back_entry=0x23d64482b7e72bb7
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Memory: 4053396k/4718592k available (4465k kernel code, 526212k absent, 138984k reserved, 1739k data, 420k init)
SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Preemptable hierarchical RCU implementation.
	RCU-based detection of stalled CPUs is disabled.
	Verbose stalled-CPUs detection is disabled.
NR_IRQS:384
Extended CMOS year: 2000
Console: colour VGA+ 80x25
console [tty0] enabled
hpet clockevent registered
Fast TSC calibration using PIT
Detected 3211.176 MHz processor.
Calibrating delay loop (skipped), value calculated using timer frequency.. 6422.35 BogoMIPS (lpj=3211176)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 256
Initializing cgroup subsys blkio
tseg: 0000000000
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 6 MCE banks
using C1E aware idle routine
Performance Events: AMD PMU driver.
... version:                0
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             00007fffffffffff
... fixed-purpose events:   0
... event mask:             000000000000000f
Freeing SMP alternatives: 16k freed
ACPI: Core revision 20100702
Setting APIC routing to flat
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: AMD Phenom(tm) II X4 955 Processor stepping 02
System has AMD C1E enabled
Switch to broadcast mode on CPU0
MCE: In-kernel MCE decoding enabled.
Booting Node   0, Processors  #1
Switch to broadcast mode on CPU1
 #2
Switch to broadcast mode on CPU2
 #3 Ok.
Brought up 4 CPUs
Total of 4 processors activated (25687.49 BogoMIPS).
Switch to broadcast mode on CPU3
NET: Registered protocol family 16
node 0 link 0: io port [1000, ffffff]
TOM: 00000000e0000000 aka 3584M
Fam 10h mmconf [e0000000, efffffff]
node 0 link 0: mmio [a0000, bffff]
node 0 link 0: mmio [e0000000, efffffff] ==> none
node 0 link 0: mmio [f0000000, fbcfffff]
node 0 link 0: mmio [fbd00000, fbefffff]
node 0 link 0: mmio [fbf00000, ffefffff]
TOM2: 0000000120000000 aka 4608M
bus: [00, 07] on node 0 link 0
bus: 00 index 0 [io  0x0000-0xffff]
bus: 00 index 1 [mem 0x000a0000-0x000bffff]
bus: 00 index 2 [mem 0xf0000000-0xffffffff]
bus: 00 index 3 [mem 0x120000000-0xfcffffffff]
ACPI: bus type pci registered
PCI: Using configuration type 1 for base access
PCI: Using configuration type 1 for extended access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Executed 3 blocks of module-level executable AML code
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using IOAPIC for interrupt routing
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0x000d0000-0x000dffff]
pci_root PNP0A03:00: host bridge window [mem 0xdff00000-0xdfffffff]
pci_root PNP0A03:00: host bridge window [mem 0xf0000000-0xfebfffff]
pci 0000:00:11.0: reg 10: [io  0xc000-0xc007]
pci 0000:00:11.0: reg 14: [io  0xb000-0xb003]
pci 0000:00:11.0: reg 18: [io  0xa000-0xa007]
pci 0000:00:11.0: reg 1c: [io  0x9000-0x9003]
pci 0000:00:11.0: reg 20: [io  0x8000-0x800f]
pci 0000:00:11.0: reg 24: [mem 0xfbcffc00-0xfbcfffff]
pci 0000:00:12.0: reg 10: [mem 0xfbcfd000-0xfbcfdfff]
pci 0000:00:12.1: reg 10: [mem 0xfbcfe000-0xfbcfefff]
pci 0000:00:12.2: reg 10: [mem 0xfbcff800-0xfbcff8ff]
pci 0000:00:12.2: supports D1 D2
pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
pci 0000:00:12.2: PME# disabled
pci 0000:00:13.0: reg 10: [mem 0xfbcfb000-0xfbcfbfff]
pci 0000:00:13.1: reg 10: [mem 0xfbcfc000-0xfbcfcfff]
pci 0000:00:13.2: reg 10: [mem 0xfbcff400-0xfbcff4ff]
pci 0000:00:13.2: supports D1 D2
pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
pci 0000:00:13.2: PME# disabled
pci 0000:00:14.1: reg 10: [io  0x0000-0x0007]
pci 0000:00:14.1: reg 14: [io  0x0000-0x0003]
pci 0000:00:14.1: reg 18: [io  0x0000-0x0007]
pci 0000:00:14.1: reg 1c: [io  0x0000-0x0003]
pci 0000:00:14.1: reg 20: [io  0xff00-0xff0f]
pci 0000:00:14.5: reg 10: [mem 0xfbcfa000-0xfbcfafff]
pci 0000:01:05.0: reg 10: [mem 0xf0000000-0xf7ffffff pref]
pci 0000:01:05.0: reg 14: [io  0xd000-0xd0ff]
pci 0000:01:05.0: reg 18: [mem 0xfbee0000-0xfbeeffff]
pci 0000:01:05.0: reg 24: [mem 0xfbd00000-0xfbdfffff]
pci 0000:01:05.0: supports D1 D2
pci 0000:01:05.1: reg 10: [mem 0xfbefc000-0xfbefffff]
pci 0000:01:05.1: supports D1 D2
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  0xd000-0xdfff]
pci 0000:00:01.0:   bridge window [mem 0xfbd00000-0xfbefffff]
pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
pci 0000:02:05.0: reg 10: [io  0xe800-0xe8ff]
pci 0000:02:05.0: reg 14: [mem 0xfbfffc00-0xfbfffcff]
pci 0000:02:05.0: reg 30: [mem 0xfbfc0000-0xfbfdffff pref]
pci 0000:02:05.0: supports D1 D2
pci 0000:02:05.0: PME# supported from D1 D2 D3hot D3cold
pci 0000:02:05.0: PME# disabled
pci 0000:00:14.4: PCI bridge to [bus 02-02] (subtractive decode)
pci 0000:00:14.4:   bridge window [io  0xe000-0xefff]
pci 0000:00:14.4:   bridge window [mem 0xfbf00000-0xfbffffff]
pci 0000:00:14.4:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
pci 0000:00:14.4:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
pci 0000:00:14.4:   bridge window [io  0x0d00-0xffff] (subtractive decode)
pci 0000:00:14.4:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
pci 0000:00:14.4:   bridge window [mem 0x000d0000-0x000dffff] (subtractive decode)
pci 0000:00:14.4:   bridge window [mem 0xdff00000-0xdfffffff] (subtractive decode)
pci 0000:00:14.4:   bridge window [mem 0xf0000000-0xfebfffff] (subtractive decode)
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0PC._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKC] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKD] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs 4 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKH] (IRQs 4 7 10 11 12 14 15) *0, disabled.
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
Advanced Linux Sound Architecture Driver Version 1.0.23.
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
reserve RAM buffer: 000000000009fc00 - 000000000009ffff 
reserve RAM buffer: 00000000dfe90000 - 00000000dfffffff 
hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
hpet0: 4 comparators, 32-bit 14.318180 MHz counter
Switching to clocksource tsc
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 12 devices
ACPI: ACPI bus type pnp unregistered
system 00:07: [mem 0xfec00000-0xfec00fff] could not be reserved
system 00:07: [mem 0xfee00000-0xfee00fff] has been reserved
system 00:08: [io  0x04d0-0x04d1] has been reserved
system 00:08: [io  0x040b] has been reserved
system 00:08: [io  0x04d6] has been reserved
system 00:08: [io  0x0c00-0x0c01] has been reserved
system 00:08: [io  0x0c14] has been reserved
system 00:08: [io  0x0c50-0x0c51] has been reserved
system 00:08: [io  0x0c52] has been reserved
system 00:08: [io  0x0c6c] has been reserved
system 00:08: [io  0x0c6f] has been reserved
system 00:08: [io  0x0cd0-0x0cd1] has been reserved
system 00:08: [io  0x0cd2-0x0cd3] has been reserved
system 00:08: [io  0x0cd4-0x0cd5] has been reserved
system 00:08: [io  0x0cd6-0x0cd7] has been reserved
system 00:08: [io  0x0cd8-0x0cdf] has been reserved
system 00:08: [io  0x0b00-0x0b3f] has been reserved
system 00:08: [io  0x0800-0x089f] has been reserved
system 00:08: [io  0x0b00-0x0b0f] has been reserved
system 00:08: [io  0x0b20-0x0b3f] has been reserved
system 00:08: [io  0x0900-0x090f] has been reserved
system 00:08: [io  0x0910-0x091f] has been reserved
system 00:08: [io  0xfe00-0xfefe] has been reserved
system 00:08: [mem 0xdff00000-0xdfffffff] has been reserved
system 00:08: [mem 0xffb80000-0xffbfffff] has been reserved
system 00:08: [mem 0xfec10000-0xfec1001f] has been reserved
system 00:09: [io  0x0230-0x023f] has been reserved
system 00:09: [io  0x0290-0x029f] has been reserved
system 00:09: [io  0x0f40-0x0f4f] has been reserved
system 00:09: [io  0x0a30-0x0a3f] has been reserved
system 00:0a: [mem 0xe0000000-0xefffffff] has been reserved
system 00:0b: [mem 0x00000000-0x0009ffff] could not be reserved
system 00:0b: [mem 0x000c0000-0x000cffff] has been reserved
system 00:0b: [mem 0x000e0000-0x000fffff] could not be reserved
system 00:0b: [mem 0x00100000-0xdfefffff] could not be reserved
system 00:0b: [mem 0xfec00000-0xffffffff] could not be reserved
pci 0000:00:01.0: PCI bridge to [bus 01-01]
pci 0000:00:01.0:   bridge window [io  0xd000-0xdfff]
pci 0000:00:01.0:   bridge window [mem 0xfbd00000-0xfbefffff]
pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
pci 0000:00:14.4: PCI bridge to [bus 02-02]
pci 0000:00:14.4:   bridge window [io  0xe000-0xefff]
pci 0000:00:14.4:   bridge window [mem 0xfbf00000-0xfbffffff]
pci 0000:00:14.4:   bridge window [mem pref disabled]
pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000dffff]
pci_bus 0000:00: resource 8 [mem 0xdff00000-0xdfffffff]
pci_bus 0000:00: resource 9 [mem 0xf0000000-0xfebfffff]
pci_bus 0000:01: resource 0 [io  0xd000-0xdfff]
pci_bus 0000:01: resource 1 [mem 0xfbd00000-0xfbefffff]
pci_bus 0000:01: resource 2 [mem 0xf0000000-0xf7ffffff 64bit pref]
pci_bus 0000:02: resource 0 [io  0xe000-0xefff]
pci_bus 0000:02: resource 1 [mem 0xfbf00000-0xfbffffff]
pci_bus 0000:02: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:02: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:02: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:02: resource 7 [mem 0x000d0000-0x000dffff]
pci_bus 0000:02: resource 8 [mem 0xdff00000-0xdfffffff]
pci_bus 0000:02: resource 9 [mem 0xf0000000-0xfebfffff]
NET: Registered protocol family 2
IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
UDP hash table entries: 2048 (order: 4, 65536 bytes)
UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
NET: Registered protocol family 1
pci 0000:00:01.0: MSI quirk detected; subordinate MSI disabled
pci 0000:01:05.0: Boot video device
PCI: CLS mismatch (64 != 32), using 64 bytes
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Placing 64MB software IO TLB between ffff8800db48b000 - ffff8800df48b000
software IO TLB at phys 0xdb48b000 - 0xdf48b000
kvm: Nested Virtualization enabled
kvm: Nested Paging enabled
msgmni has been set to 7916
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler cfq registered (default)
input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
ACPI: Power Button [PWRB]
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
ACPI: Power Button [PWRF]
ACPI: acpi_idle registered with cpuidle
ACPI: processor limited to max C-state 1
Real Time Clock Driver v1.12b
Linux agpgart interface v0.103
[drm] Initialized drm 1.1.0 20060810
[drm] radeon defaulting to kernel modesetting.
[drm] radeon kernel modesetting enabled.
radeon 0000:01:05.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
radeon 0000:01:05.0: setting latency timer to 64
[drm] initializing kernel modesetting (RS780 0x1002:0x9614).
[drm] register mmio base: 0xFBEE0000
[drm] register mmio size: 65536
ATOM BIOS: 113
radeon 0000:01:05.0: VRAM: 128M 0xC0000000 - 0xC7FFFFFF (128M used)
radeon 0000:01:05.0: GTT: 512M 0xA0000000 - 0xBFFFFFFF
[drm] Detected VRAM RAM=128M, BAR=128M
[drm] RAM width 32bits DDR
[TTM] Zone  kernel: Available graphics memory: 2026706 kiB.
[TTM] Initializing pool allocator.
[drm] radeon: 128M of VRAM memory ready
[drm] radeon: 512M of GTT memory ready.
[drm] radeon: irq initialized.
[drm] GART: num cpu pages 131072, num gpu pages 131072
[drm] Loading RS780 Microcode
[drm] ring test succeeded in 0 usecs
[drm] radeon: ib pool ready.
[drm] ib test succeeded in 0 usecs
[drm] Enabling audio support
[drm] Radeon Display Connectors
[drm] Connector 0:
[drm]   VGA
[drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
[drm]   Encoders:
[drm]     CRT1: INTERNAL_KLDSCP_DAC1
[drm] Connector 1:
[drm]   DVI-D
[drm]   HPD3
[drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
[drm]   Encoders:
[drm]     DFP3: INTERNAL_KLDSCP_LVTMA
[drm] radeon: power management initialized
[drm] fb mappable at 0xF0141000
[drm] vram apper at 0xF0000000
[drm] size 7258112
[drm] fb depth is 24
[drm]    pitch is 6912
Console: switching to colour frame buffer device 131x105
fb0: radeondrmfb frame buffer device
drm: registered panic notifier
[drm] Initialized radeon 2.6.0 20080528 for 0000:01:05.0 on minor 0
loop: module loaded
ahci 0000:00:11.0: version 3.0
ahci 0000:00:11.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
ahci 0000:00:11.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA mode
ahci 0000:00:11.0: flags: 64bit ncq sntf ilck pm led clo pmp pio slum part ccc 
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
ata1: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffd00 irq 22
ata2: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffd80 irq 22
ata3: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffe00 irq 22
ata4: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffe80 irq 22
pata_atiixp 0000:00:14.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16
scsi4 : pata_atiixp
scsi5 : pata_atiixp
ata5: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xff00 irq 14
ata6: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xff08 irq 15
PPP generic driver version 2.4.2
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
r8169 0000:02:05.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
r8169 0000:02:05.0: (unregistered net_device): no PCI Express capability
r8169 0000:02:05.0: eth0: RTL8110s at 0xffffc90000452c00, 00:08:54:36:f2:2f, XID 04000000 IRQ 20
usbmon: debugfs is not available
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci_hcd 0000:00:12.2: PCI INT B -> GSI 17 (level, low) -> IRQ 17
ehci_hcd 0000:00:12.2: EHCI Host Controller
ehci_hcd 0000:00:12.2: new USB bus registered, assigned bus number 1
ehci_hcd 0000:00:12.2: applying AMD SB600/SB700 USB freeze workaround
ehci_hcd 0000:00:12.2: debug port 1
ehci_hcd 0000:00:12.2: irq 17, io mem 0xfbcff800
ehci_hcd 0000:00:12.2: USB 2.0 started, EHCI 1.00
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 6 ports detected
ehci_hcd 0000:00:13.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
ehci_hcd 0000:00:13.2: EHCI Host Controller
ehci_hcd 0000:00:13.2: new USB bus registered, assigned bus number 2
ehci_hcd 0000:00:13.2: applying AMD SB600/SB700 USB freeze workaround
ehci_hcd 0000:00:13.2: debug port 1
ehci_hcd 0000:00:13.2: irq 19, io mem 0xfbcff400
ehci_hcd 0000:00:13.2: USB 2.0 started, EHCI 1.00
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 6 ports detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci_hcd 0000:00:12.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
ohci_hcd 0000:00:12.0: OHCI Host Controller
ohci_hcd 0000:00:12.0: new USB bus registered, assigned bus number 3
ohci_hcd 0000:00:12.0: irq 16, io mem 0xfbcfd000
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 3 ports detected
ohci_hcd 0000:00:12.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16
ohci_hcd 0000:00:12.1: OHCI Host Controller
ohci_hcd 0000:00:12.1: new USB bus registered, assigned bus number 4
ohci_hcd 0000:00:12.1: irq 16, io mem 0xfbcfe000
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 3 ports detected
ohci_hcd 0000:00:13.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
ohci_hcd 0000:00:13.0: OHCI Host Controller
ohci_hcd 0000:00:13.0: new USB bus registered, assigned bus number 5
ohci_hcd 0000:00:13.0: irq 18, io mem 0xfbcfb000
ata5.00: ATAPI: HL-DT-STDVD-RAM GH22NP20, 1.03, max UDMA/66
ata5.00: configured for UDMA/66
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 3 ports detected
ohci_hcd 0000:00:13.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
ohci_hcd 0000:00:13.1: OHCI Host Controller
ohci_hcd 0000:00:13.1: new USB bus registered, assigned bus number 6
ohci_hcd 0000:00:13.1: irq 18, io mem 0xfbcfc000
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 3 ports detected
ohci_hcd 0000:00:14.5: PCI INT C -> GSI 18 (level, low) -> IRQ 18
ohci_hcd 0000:00:14.5: OHCI Host Controller
ohci_hcd 0000:00:14.5: new USB bus registered, assigned bus number 7
ohci_hcd 0000:00:14.5: irq 18, io mem 0xfbcfa000
hub 7-0:1.0: USB hub found
hub 7-0:1.0: 2 ports detected
Initializing USB Mass Storage driver...
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usbcore: registered new interface driver usbserial
USB Serial support registered for generic
ata4: SATA link down (SStatus 0 SControl 300)
ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: ATA-8: OCZ-VERTEX, 1.6, max UDMA/133
ata3.00: 62533296 sectors, multi 1: LBA48 NCQ (depth 31/32), AA
ata2: SATA link down (SStatus 0 SControl 300)
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata3.00: configured for UDMA/133
ata1.00: ATA-7: SAMSUNG HD103UJ, 1AA01118, max UDMA7
ata1.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access     ATA      SAMSUNG HD103UJ  1AA0 PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: Attached scsi generic sg0 type 0
scsi 2:0:0:0: Direct-Access     ATA      OCZ-VERTEX       1.6  PQ: 0 ANSI: 5
sd 2:0:0:0: [sdb] 62533296 512-byte logical blocks: (32.0 GB/29.8 GiB)
sd 2:0:0:0: Attached scsi generic sg1 type 0
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
scsi 4:0:0:0: CD-ROM            HL-DT-ST DVD-RAM GH22NP20 1.03 PQ: 0 ANSI: 5
 sdb: sdb1 sdb2
sd 2:0:0:0: [sdb] Attached SCSI disk
sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 4:0:0:0: Attached scsi CD-ROM sr0
sr 4:0:0:0: Attached scsi generic sg2 type 5
 sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
usb 4-1: new full speed USB device using ohci_hcd and address 2
usb 4-2: new full speed USB device using ohci_hcd and address 3
usb 4-3: new low speed USB device using ohci_hcd and address 4
usbcore: registered new interface driver usbserial_generic
usbserial: USB Serial Driver core
USB Serial support registered for GSM modem (1-port)
usbcore: registered new interface driver option
option: v0.7.2:USB Driver for GSM modems
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
i2c /dev entries driver
cpuidle: using governor ladder
cpuidle: using governor menu
input: C-Media USB Headphone Set   as /devices/pci0000:00/0000:00:12.1/usb4/4-1/4-1:1.3/input/input2
generic-usb 0003:0D8C:000C.0001: input,hidraw0: USB HID v1.00 Device [C-Media USB Headphone Set  ] on usb-0000:00:12.1-1/input3
input: Logitech USB Receiver as /devices/pci0000:00/0000:00:12.1/usb4/4-2/4-2:1.0/input/input3
generic-usb 0003:046D:C52B.0002: input,hidraw1: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:12.1-2/input0
input: Logitech USB Receiver as /devices/pci0000:00/0000:00:12.1/usb4/4-2/4-2:1.1/input/input4
generic-usb 0003:046D:C52B.0003: input,hiddev0,hidraw2: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:12.1-2/input1
generic-usb 0003:046D:C52B.0004: hiddev0,hidraw3: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:12.1-2/input2
input: HID 046a:0011 as /devices/pci0000:00/0000:00:12.1/usb4/4-3/4-3:1.0/input/input5
generic-usb 0003:046A:0011.0005: input,hidraw4: USB HID v1.10 Keyboard [HID 046a:0011] on usb-0000:00:12.1-3/input0
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
usbcore: registered new interface driver snd-usb-audio
ALSA device list:
  #0: C-Media USB Headphone Set   at usb-0000:00:12.1-1, full speed
Netfilter messages via NETLINK v0.30.
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ctnetlink v0.93: registering with nfnetlink.
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
NET: Registered protocol family 17
powernow-k8: Found 1 AMD Phenom(tm) II X4 955 Processor (4 cpu cores) (version 2.20.00)
powernow-k8:    0 : pstate 0 (3200 MHz)
powernow-k8:    1 : pstate 1 (2500 MHz)
powernow-k8:    2 : pstate 2 (2100 MHz)
powernow-k8:    3 : pstate 3 (800 MHz)
EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) readonly on device 8:18.
Freeing unused kernel memory: 420k freed
udev[831]: starting version 163
EXT4-fs (sdb2): re-mounted. Opts: (null)
EXT4-fs (sdb2): re-mounted. Opts: (null)
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
Adding 200808k swap on /dev/sda3.  Priority:-1 extents:1 across:200808k 
r8169 0000:02:05.0: eth0: link up

[-- Attachment #3: config --]
[-- Type: text/plain, Size: 46610 bytes --]

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.36
# Tue Oct 26 06:50:32 2010
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11"
# CONFIG_KTIME_SCALAR is not set
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_GZIP=y
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_LZO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
# CONFIG_TASKSTATS is not set
# CONFIG_AUDIT is not set
CONFIG_HAVE_GENERIC_HARDIRQS=y

#
# IRQ subsystem
#
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS_NO__DO_IRQ=y
# CONFIG_GENERIC_HARDIRQS_NO_DEPRECATED is not set
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
# CONFIG_AUTO_IRQ_AFFINITY is not set
# CONFIG_IRQ_PER_CPU is not set
# CONFIG_HARDIRQS_SW_RESEND is not set
# CONFIG_SPARSE_IRQ is not set

#
# RCU Subsystem
#
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=16
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
# CONFIG_CGROUP_FREEZER is not set
# CONFIG_CGROUP_DEVICE is not set
# CONFIG_CPUSETS is not set
# CONFIG_CGROUP_CPUACCT is not set
# CONFIG_RESOURCE_COUNTERS is not set
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
# CONFIG_RT_GROUP_SCHED is not set
CONFIG_BLK_CGROUP=y
# CONFIG_DEBUG_BLK_CGROUP is not set
CONFIG_SCHED_AUTOGROUP=y
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_NAMESPACES is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_EMBEDDED=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
# CONFIG_ELF_CORE is not set
# CONFIG_PCSPKR_PLATFORM is not set
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_HAVE_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_PERF_COUNTERS is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
# CONFIG_SLUB_DEBUG is not set
# CONFIG_COMPAT_BRK is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
CONFIG_HAVE_OPROFILE=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y

#
# GCOV-based kernel profiling
#
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
# CONFIG_MODULES is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_BLK_DEV_BSG=y
# CONFIG_BLK_DEV_INTEGRITY is not set
# CONFIG_BLK_DEV_THROTTLING is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
CONFIG_CFQ_GROUP_IOSCHED=y
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set
# CONFIG_INLINE_SPIN_UNLOCK is not set
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQ is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set
# CONFIG_INLINE_READ_UNLOCK is not set
# CONFIG_INLINE_READ_UNLOCK_BH is not set
# CONFIG_INLINE_READ_UNLOCK_IRQ is not set
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set
# CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set
# CONFIG_INLINE_WRITE_UNLOCK is not set
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQ is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set
CONFIG_MUTEX_SPIN_ON_OWNER=y
# CONFIG_FREEZER is not set

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
# CONFIG_X86_MPPARSE is not set
# CONFIG_X86_EXTENDED_PLATFORM is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
CONFIG_NO_BOOTMEM=y
# CONFIG_MEMTEST is not set
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_PROCESSOR_SELECT=y
# CONFIG_CPU_SUP_INTEL is not set
CONFIG_CPU_SUP_AMD=y
# CONFIG_CPU_SUP_CENTAUR is not set
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
# CONFIG_GART_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
# CONFIG_AMD_IOMMU is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
# CONFIG_IOMMU_API is not set
CONFIG_NR_CPUS=4
# CONFIG_SCHED_SMT is not set
CONFIG_SCHED_MC=y
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_INTEL is not set
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
# CONFIG_X86_MCE_INJECT is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
# CONFIG_NUMA is not set
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_MEMBLOCK=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_X86_RESERVE_LOW=64
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=2
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
# CONFIG_EFI is not set
CONFIG_SECCOMP=y
# CONFIG_CC_STACKPROTECTOR is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x1000000
# CONFIG_HOTPLUG_CPU is not set
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management and ACPI options
#
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
# CONFIG_SUSPEND is not set
# CONFIG_HIBERNATION is not set
# CONFIG_PM_RUNTIME is not set
# CONFIG_PM_OPP is not set
CONFIG_ACPI=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROCFS_POWER is not set
# CONFIG_ACPI_POWER_METER is not set
# CONFIG_ACPI_SYSFS_POWER is not set
# CONFIG_ACPI_EC_DEBUGFS is not set
# CONFIG_ACPI_PROC_EVENT is not set
# CONFIG_ACPI_AC is not set
# CONFIG_ACPI_BATTERY is not set
CONFIG_ACPI_BUTTON=y
# CONFIG_ACPI_FAN is not set
# CONFIG_ACPI_DOCK is not set
CONFIG_ACPI_PROCESSOR=y
# CONFIG_ACPI_PROCESSOR_AGGREGATOR is not set
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
# CONFIG_ACPI_SBS is not set
# CONFIG_ACPI_HED is not set
# CONFIG_ACPI_APEI is not set
# CONFIG_SFI is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=y
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_GOV_CONSERVATIVE is not set

#
# CPUFreq processor drivers
#
# CONFIG_X86_PCC_CPUFREQ is not set
# CONFIG_X86_ACPI_CPUFREQ is not set
CONFIG_X86_POWERNOW_K8=y
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set

#
# shared options
#
# CONFIG_X86_SPEEDSTEP_LIB is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y

#
# Memory power savings
#
# CONFIG_I7300_IDLE is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_MMCONFIG is not set
CONFIG_PCI_DOMAINS=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
# CONFIG_DMAR is not set
# CONFIG_INTR_REMAP is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_STUB is not set
CONFIG_HT_IRQ=y
# CONFIG_PCI_IOV is not set
CONFIG_PCI_IOAPIC=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y
# CONFIG_PCCARD is not set
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_HAVE_AOUT is not set
CONFIG_BINFMT_MISC=y
# CONFIG_IA32_EMULATION is not set
# CONFIG_COMPAT_FOR_U64_ALIGNMENT is not set
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
# CONFIG_INET_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
# CONFIG_NETFILTER_ADVANCED is not set

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=y
CONFIG_NETFILTER_NETLINK_LOG=y
CONFIG_NF_CONNTRACK=y
CONFIG_NF_CONNTRACK_FTP=y
CONFIG_NF_CONNTRACK_IRC=y
CONFIG_NF_CONNTRACK_SIP=y
CONFIG_NF_CT_NETLINK=y
CONFIG_NETFILTER_XTABLES=y

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=y

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_NFLOG=y
CONFIG_NETFILTER_XT_TARGET_TCPMSS=y

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_MATCH_STATE=y
# CONFIG_IP_VS is not set

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=y
CONFIG_NF_CONNTRACK_IPV4=y
CONFIG_NF_CONNTRACK_PROC_COMPAT=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_ULOG=y
CONFIG_NF_NAT=y
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NF_NAT_FTP=y
CONFIG_NF_NAT_IRC=y
# CONFIG_NF_NAT_TFTP is not set
# CONFIG_NF_NAT_AMANDA is not set
# CONFIG_NF_NAT_PPTP is not set
# CONFIG_NF_NAT_H323 is not set
CONFIG_NF_NAT_SIP=y
CONFIG_IP_NF_MANGLE=y
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_RDS is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
CONFIG_STP=y
CONFIG_BRIDGE=y
CONFIG_BRIDGE_IGMP_SNOOPING=y
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
CONFIG_LLC=y
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_PHONET is not set
# CONFIG_IEEE802154 is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set
CONFIG_RPS=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
# CONFIG_WIRELESS is not set
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set
# CONFIG_CEPH_LIB is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
# CONFIG_DEVTMPFS is not set
CONFIG_STANDALONE=y
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE="radeon/R600_rlc.bin"
CONFIG_EXTRA_FIRMWARE_DIR="firmware"
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set

#
# DRBD disabled because PROC_FS, INET or CONNECTOR not selected
#
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
# CONFIG_BLK_DEV_RAM is not set
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_BLK_DEV_HD is not set
# CONFIG_BLK_DEV_RBD is not set
# CONFIG_MISC_DEVICES is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
# CONFIG_ATA_ACPI is not set
# CONFIG_SATA_PMP is not set

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_SIL24 is not set
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_SX4 is not set
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
# CONFIG_ATA_PIIX is not set
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_SVW is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set

#
# PATA SFF controllers with BMDMA
#
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
CONFIG_PATA_ATIIXP=y
# CONFIG_PATA_ATP867X is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RDC is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SCH is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_TOSHIBA is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set

#
# PIO-only SFF controllers
#
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_PLATFORM is not set
# CONFIG_PATA_RZ1000 is not set

#
# Generic fallback / legacy drivers
#
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_LEGACY is not set
# CONFIG_MD is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_FIREWIRE_NOSY is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
CONFIG_MII=y
# CONFIG_PHYLIB is not set
# CONFIG_NET_ETHERNET is not set
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_E1000E is not set
# CONFIG_IP1000 is not set
# CONFIG_IGB is not set
# CONFIG_IGBVF is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_R8169=y
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_CNIC is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
# CONFIG_ATL1E is not set
# CONFIG_ATL1C is not set
# CONFIG_JME is not set
# CONFIG_STMMAC_ETH is not set
# CONFIG_PCH_GBE is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set
# CONFIG_WLAN is not set

#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
# CONFIG_USB_IPHETH is not set
# CONFIG_WAN is not set

#
# CAIF transport drivers
#
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=y
# CONFIG_PPP_MULTILINK is not set
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=y
CONFIG_PPP_SYNC_TTY=y
# CONFIG_PPP_DEFLATE is not set
# CONFIG_PPP_BSDCOMP is not set
# CONFIG_PPP_MPPE is not set
# CONFIG_PPPOE is not set
# CONFIG_SLIP is not set
CONFIG_SLHC=y
# CONFIG_NET_FC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_VMXNET3 is not set
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_POLLDEV is not set
# CONFIG_INPUT_SPARSEKMAP is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_INPUT_MOUSE=y
# CONFIG_MOUSE_PS2 is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_MOUSE_SYNAPTICS_I2C is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_SERIO_ALTERA_PS2 is not set
# CONFIG_SERIO_PS2MULT is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
# CONFIG_DEVKMEM is not set
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_N_GSM is not set
# CONFIG_NOZOMI is not set

#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
CONFIG_FIX_EARLYCON_MEM=y

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_MFD_HSU is not set
# CONFIG_SERIAL_JSM is not set
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
# CONFIG_LEGACY_PTYS is not set
# CONFIG_TTY_PRINTK is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_RAMOOPS is not set
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
# CONFIG_I2C_MUX is not set
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=y

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_ISCH is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set

#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_SIMTEC is not set
# CONFIG_I2C_XILINX is not set

#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set

#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_SPI is not set

#
# PPS support
#
# CONFIG_PPS is not set
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_TEST_POWER is not set
# CONFIG_BATTERY_DS2760 is not set
# CONFIG_BATTERY_DS2782 is not set
# CONFIG_BATTERY_BQ27x00 is not set
# CONFIG_BATTERY_MAX17040 is not set
CONFIG_HWMON=y
# CONFIG_HWMON_VID is not set
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_AD7414 is not set
# CONFIG_SENSORS_AD7418 is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ADT7411 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
# CONFIG_SENSORS_ADT7475 is not set
# CONFIG_SENSORS_ASC7621 is not set
# CONFIG_SENSORS_K8TEMP is not set
CONFIG_SENSORS_K10TEMP=y
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
# CONFIG_SENSORS_F75375S is not set
# CONFIG_SENSORS_FSCHMD is not set
# CONFIG_SENSORS_G760A is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_CORETEMP is not set
# CONFIG_SENSORS_PKGTEMP is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_JC42 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM73 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
# CONFIG_SENSORS_LTC4215 is not set
# CONFIG_SENSORS_LTC4245 is not set
# CONFIG_SENSORS_LTC4261 is not set
# CONFIG_SENSORS_LM95241 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMM665 is not set
# CONFIG_SENSORS_DME1737 is not set
# CONFIG_SENSORS_EMC1403 is not set
# CONFIG_SENSORS_EMC2103 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_ADS7828 is not set
# CONFIG_SENSORS_AMC6821 is not set
# CONFIG_SENSORS_THMC50 is not set
# CONFIG_SENSORS_TMP102 is not set
# CONFIG_SENSORS_TMP401 is not set
# CONFIG_SENSORS_TMP421 is not set
# CONFIG_SENSORS_VIA_CPUTEMP is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83L786NG is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_LIS3_I2C is not set
# CONFIG_SENSORS_APPLESMC is not set

#
# ACPI drivers
#
CONFIG_SENSORS_ATK0110=y
# CONFIG_SENSORS_LIS3LV02D is not set
CONFIG_THERMAL=y
CONFIG_THERMAL_HWMON=y
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set
# CONFIG_MFD_SUPPORT is not set
# CONFIG_REGULATOR is not set
# CONFIG_MEDIA_SUPPORT is not set

#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_VGA_ARB is not set
# CONFIG_VGA_SWITCHEROO is not set
CONFIG_DRM=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_TTM=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=y
CONFIG_DRM_RADEON_KMS=y
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_VGASTATE is not set
CONFIG_VIDEO_OUTPUT_CONTROL=y
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
# CONFIG_FB_DDC is not set
# CONFIG_FB_BOOT_VESA_SUPPORT is not set
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not set
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_VESA is not set
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
# CONFIG_LOGO is not set
CONFIG_SOUND=y
CONFIG_SOUND_OSS_CORE=y
CONFIG_SOUND_OSS_CORE_PRECLAIM=y
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_HWDEP=y
CONFIG_SND_RAWMIDI=y
CONFIG_SND_SEQUENCER=y
CONFIG_SND_SEQ_DUMMY=y
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=y
CONFIG_SND_PCM_OSS=y
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_RTCTIMER=y
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_SUPPORT_OLD_API=y
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_RAWMIDI_SEQ=y
# CONFIG_SND_OPL3_LIB_SEQ is not set
# CONFIG_SND_OPL4_LIB_SEQ is not set
# CONFIG_SND_SBAWE_SEQ is not set
# CONFIG_SND_EMU10K1_SEQ is not set
# CONFIG_SND_DRIVERS is not set
# CONFIG_SND_PCI is not set
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=y
# CONFIG_SND_USB_UA101 is not set
# CONFIG_SND_USB_USX2Y is not set
# CONFIG_SND_USB_CAIAQ is not set
# CONFIG_SND_USB_US122L is not set
# CONFIG_SND_SOC is not set
# CONFIG_SOUND_PRIME is not set
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_HIDRAW=y

#
# USB Input Devices
#
CONFIG_USB_HID=y
# CONFIG_HID_PID is not set
CONFIG_USB_HIDDEV=y

#
# Special HID drivers
#
# CONFIG_HID_3M_PCT is not set
# CONFIG_HID_A4TECH is not set
# CONFIG_HID_ACRUX_FF is not set
# CONFIG_HID_APPLE is not set
# CONFIG_HID_BELKIN is not set
# CONFIG_HID_CANDO is not set
# CONFIG_HID_CHERRY is not set
# CONFIG_HID_CHICONY is not set
# CONFIG_HID_PRODIKEYS is not set
# CONFIG_HID_CYPRESS is not set
# CONFIG_HID_DRAGONRISE is not set
# CONFIG_HID_EGALAX is not set
# CONFIG_HID_EZKEY is not set
# CONFIG_HID_KYE is not set
# CONFIG_HID_UCLOGIC is not set
# CONFIG_HID_WALTOP is not set
# CONFIG_HID_GYRATION is not set
# CONFIG_HID_TWINHAN is not set
# CONFIG_HID_KENSINGTON is not set
# CONFIG_HID_LOGITECH is not set
# CONFIG_HID_MICROSOFT is not set
# CONFIG_HID_MOSART is not set
# CONFIG_HID_MONTEREY is not set
# CONFIG_HID_NTRIG is not set
# CONFIG_HID_ORTEK is not set
# CONFIG_HID_PANTHERLORD is not set
# CONFIG_HID_PETALYNX is not set
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_QUANTA is not set
# CONFIG_HID_ROCCAT is not set
# CONFIG_HID_ROCCAT_KONE is not set
# CONFIG_HID_ROCCAT_PYRA is not set
# CONFIG_HID_SAMSUNG is not set
# CONFIG_HID_SONY is not set
# CONFIG_HID_STANTUM is not set
# CONFIG_HID_SUNPLUS is not set
# CONFIG_HID_GREENASIA is not set
# CONFIG_HID_SMARTJOYPLUS is not set
# CONFIG_HID_TOPSEED is not set
# CONFIG_HID_THRUSTMASTER is not set
# CONFIG_HID_ZEROPLUS is not set
# CONFIG_HID_ZYDACRON is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set

#
# Miscellaneous USB options
#
# CONFIG_USB_DEVICEFS is not set
# CONFIG_USB_DEVICE_CLASS is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
CONFIG_USB_MON=y
# CONFIG_USB_WUSB is not set
# CONFIG_USB_WUSB_CBAF is not set

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
# CONFIG_USB_XHCI_HCD is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_ISP1760_HCD is not set
# CONFIG_USB_ISP1362_HCD is not set
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
# CONFIG_USB_UHCI_HCD is not set
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_WHCI_HCD is not set
# CONFIG_USB_HWA_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#

#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_UAS is not set
# CONFIG_USB_LIBUSUAL is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB port drivers
#
CONFIG_USB_SERIAL=y
# CONFIG_USB_SERIAL_CONSOLE is not set
# CONFIG_USB_EZUSB is not set
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRCABLE is not set
# CONFIG_USB_SERIAL_ARK3116 is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_CH341 is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP210X is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_FUNSOFT is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_IUU is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MOTOROLA is not set
# CONFIG_USB_SERIAL_NAVMAN is not set
# CONFIG_USB_SERIAL_PL2303 is not set
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QCAUX is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
# CONFIG_USB_SERIAL_HP4X is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_SAMBA is not set
# CONFIG_USB_SERIAL_SIEMENS_MPI is not set
# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
# CONFIG_USB_SERIAL_SYMBOL is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
CONFIG_USB_SERIAL_WWAN=y
CONFIG_USB_SERIAL_OPTION=y
# CONFIG_USB_SERIAL_OMNINET is not set
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_VIVOPAY_SERIAL is not set
# CONFIG_USB_SERIAL_ZIO is not set
# CONFIG_USB_SERIAL_SSU100 is not set
# CONFIG_USB_SERIAL_DEBUG is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_YUREX is not set
# CONFIG_USB_GADGET is not set

#
# OTG and related infrastructure
#
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_DECODE_MCE=y
# CONFIG_EDAC_MCE_INJ is not set
# CONFIG_EDAC_MM_EDAC is not set
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_STAGING is not set
# CONFIG_X86_PLATFORM_DEVICES is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set
# CONFIG_ISCSI_IBFT_FIND is not set

#
# File systems
#
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=y
# CONFIG_EXT4_USE_FOR_EXT23 is not set
# CONFIG_EXT4_FS_XATTR is not set
# CONFIG_EXT4_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_FS_POSIX_ACL is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_CONFIGFS_FS is not set
# CONFIG_MISC_FILESYSTEMS is not set
# CONFIG_NETWORK_FILESYSTEMS is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=y
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y
# CONFIG_DLM is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_WARN_DEPRECATED is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_FRAME_WARN=1024
# CONFIG_MAGIC_SYSRQ is not set
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_HEADERS_CHECK is not set
# CONFIG_DEBUG_KERNEL is not set
# CONFIG_HARDLOCKUP_DETECTOR is not set
# CONFIG_SLUB_STATS is not set
CONFIG_BKL=y
# CONFIG_SPARSE_RCU_POINTER is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_MEMORY_INIT is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
# CONFIG_FRAME_POINTER is not set
# CONFIG_RCU_CPU_STALL_DETECTOR is not set
# CONFIG_SYSCTL_SYSCALL_CHECK is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_HAVE_ARCH_KMEMCHECK=y
CONFIG_STRICT_DEVMEM=y
# CONFIG_X86_VERBOSE_BOOTUP is not set
# CONFIG_EARLY_PRINTK is not set
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_OPTIMIZE_INLINING is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
# CONFIG_CRYPTO is not set
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_APIC_ARCHITECTURE=y
CONFIG_KVM_MMIO=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
# CONFIG_KVM_INTEL is not set
CONFIG_KVM_AMD=y
CONFIG_VHOST_NET=y
# CONFIG_VIRTIO_PCI is not set
# CONFIG_VIRTIO_BALLOON is not set
# CONFIG_BINARY_PRINTF is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
# CONFIG_CRC_T10DIF is not set
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_NLATTR=y

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 11:20 Clocksource tsc unstable (delta = -8589361717 ns) in current git Markus Trippelsdorf
@ 2010-10-26 13:18 ` Borislav Petkov
  2010-10-26 13:58   ` Markus Trippelsdorf
  0 siblings, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-10-26 13:18 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: linux-kernel, hpa, Thomas Gleixner, Ingo Molnar, Andreas Herrmann

On Tue, Oct 26, 2010 at 07:20:52AM -0400, Markus Trippelsdorf wrote:
> On my AMD system the following kernel messages are sometimes printed
> after several hours of uptime:
> Clocksource tsc unstable (delta = -8589361717 ns)
> Switching to clocksource hpet

This is strange, tsc is actually stable on your F10h revC.

> This happened several times already:
> # grep "Clocksource" kernel.*
> kernel.log:Oct 24 18:31:17 arch kernel: Clocksource tsc unstable (delta = -8591703466 ns)
> kernel.log:Oct 26 12:54:38 arch kernel: Clocksource tsc unstable (delta = -8589361717 ns)
> kernel.log.1:Oct 22 21:43:51 arch kernel: Clocksource tsc unstable (delta = -25772316650 ns)

Do you have any warnings/oopses before that, can we have the whole dmesg
with the above messages?

> The computer is nearly unusable afterwards (second long delays, that
> seem to grow larger and larger) and only a prompt reboot can fix the
> problem.
> I'm running the latest git kernel. Version 2.6.36 does not show these
> symptoms.
> 
> Dmesg and kernel config are attached.
> 
> -- 
> Markus

> Initializing cgroup subsys cpu
> Linux version 2.6.36-05618-gb18cae4-dirty (markus@arch.trippelsdorf.de) (gcc version 4.5.1 (GCC) ) #41 SMP PREEMPT Tue Oct 26 06:51:47 CEST 2010

 ...-dirty means you have some changes ontop of b18cae4. What are those?

> Command line: root=/dev/sdb2 fbcon=rotate:3 quiet kexec_jump_back_entry=0x23d64482b7e72bb7
> BIOS-provided physical RAM map:

otherwise, I don't see anything strange in your dmesg. Unless tglx has a
better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
much but I don't know, if the issue appears every several hours it could
still be tedious.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 13:18 ` Borislav Petkov
@ 2010-10-26 13:58   ` Markus Trippelsdorf
  2010-10-26 14:05     ` Markus Trippelsdorf
                       ` (3 more replies)
  0 siblings, 4 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-26 13:58 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-kernel, hpa, Thomas Gleixner, Ingo Molnar, Andreas Herrmann

On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> On Tue, Oct 26, 2010 at 07:20:52AM -0400, Markus Trippelsdorf wrote:
> > On my AMD system the following kernel messages are sometimes printed
> > after several hours of uptime:
> > Clocksource tsc unstable (delta = -8589361717 ns)
> > Switching to clocksource hpet
> 
> This is strange, tsc is actually stable on your F10h revC.

Yes. constant_tsc is among the CPU flags.

> > This happened several times already:
> > # grep "Clocksource" kernel.*
> > kernel.log:Oct 24 18:31:17 arch kernel: Clocksource tsc unstable (delta = -8591703466 ns)
> > kernel.log:Oct 26 12:54:38 arch kernel: Clocksource tsc unstable (delta = -8589361717 ns)
> > kernel.log.1:Oct 22 21:43:51 arch kernel: Clocksource tsc unstable (delta = -25772316650 ns)
> 
> Do you have any warnings/oopses before that, can we have the whole dmesg
> with the above messages?

No warning/oopses. The whole dmesg is attached.

> 
> > The computer is nearly unusable afterwards (second long delays, that
> > seem to grow larger and larger) and only a prompt reboot can fix the
> > problem.
> > I'm running the latest git kernel. Version 2.6.36 does not show these
> > symptoms.
> > 
> > Dmesg and kernel config are attached.
> > 
> > -- 
> > Markus
> 
> > Initializing cgroup subsys cpu
> > Linux version 2.6.36-05618-gb18cae4-dirty (markus@arch.trippelsdorf.de) (gcc version 4.5.1 (GCC) ) #41 SMP PREEMPT Tue Oct 26 06:51:47 CEST 2010
> 
>  ...-dirty means you have some changes ontop of b18cae4. What are those?

The SCHED_AUTOGROUP patches from Mike Galbraith.

> > Command line: root=/dev/sdb2 fbcon=rotate:3 quiet kexec_jump_back_entry=0x23d64482b7e72bb7
> > BIOS-provided physical RAM map:
> 
> otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> much but I don't know, if the issue appears every several hours it could
> still be tedious.

That would be a several week long process, as the issue appears ~ every
2 days here on this machine running 24/7.

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 13:58   ` Markus Trippelsdorf
@ 2010-10-26 14:05     ` Markus Trippelsdorf
  2010-10-26 14:38     ` Peter Zijlstra
                       ` (2 subsequent siblings)
  3 siblings, 0 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-26 14:05 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: linux-kernel, hpa, Thomas Gleixner, Ingo Molnar, Andreas Herrmann

[-- Attachment #1: Type: text/plain, Size: 1133 bytes --]

On Tue, Oct 26, 2010 at 03:58:08PM +0200, Markus Trippelsdorf wrote:
> On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > On Tue, Oct 26, 2010 at 07:20:52AM -0400, Markus Trippelsdorf wrote:
> > > On my AMD system the following kernel messages are sometimes printed
> > > after several hours of uptime:
> > > Clocksource tsc unstable (delta = -8589361717 ns)
> > > Switching to clocksource hpet
> > 
> > This is strange, tsc is actually stable on your F10h revC.
> 
> Yes. constant_tsc is among the CPU flags.
> 
> > > This happened several times already:
> > > # grep "Clocksource" kernel.*
> > > kernel.log:Oct 24 18:31:17 arch kernel: Clocksource tsc unstable (delta = -8591703466 ns)
> > > kernel.log:Oct 26 12:54:38 arch kernel: Clocksource tsc unstable (delta = -8589361717 ns)
> > > kernel.log.1:Oct 22 21:43:51 arch kernel: Clocksource tsc unstable (delta = -25772316650 ns)
> > 
> > Do you have any warnings/oopses before that, can we have the whole dmesg
> > with the above messages?
> 
> No warning/oopses. The whole dmesg is attached.

This time I will hopefully hit the right key in mutt...
-- 
Markus

[-- Attachment #2: kernel.log --]
[-- Type: text/plain, Size: 44724 bytes --]

Oct 24 10:19:05 arch kernel: Initializing cgroup subsys cpu
Oct 24 10:19:05 arch kernel: Linux version 2.6.36-03963-g35da7a3-dirty (markus@arch.trippelsdorf.de) (gcc version 4.5.1 (GCC) ) #29 SMP PREEMPT Sun Oct 24 03:26:33 CEST 2010
Oct 24 10:19:05 arch kernel: Command line: root=/dev/sdb2 fbcon=rotate:3 quiet
Oct 24 10:19:05 arch kernel: BIOS-provided physical RAM map:
Oct 24 10:19:05 arch kernel: BIOS-e820: 0000000000000100 - 000000000009fc00 (usable)
Oct 24 10:19:05 arch kernel: BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
Oct 24 10:19:05 arch kernel: BIOS-e820: 00000000000e6000 - 0000000000100000 (reserved)
Oct 24 10:19:05 arch kernel: BIOS-e820: 0000000000100000 - 00000000dfe90000 (usable)
Oct 24 10:19:05 arch kernel: BIOS-e820: 00000000dfe90000 - 00000000dfea8000 (ACPI data)
Oct 24 10:19:05 arch kernel: BIOS-e820: 00000000dfea8000 - 00000000dfed0000 (ACPI NVS)
Oct 24 10:19:05 arch kernel: BIOS-e820: 00000000dfed0000 - 00000000dff00000 (reserved)
Oct 24 10:19:05 arch kernel: BIOS-e820: 00000000fff00000 - 0000000100000000 (reserved)
Oct 24 10:19:05 arch kernel: BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
Oct 24 10:19:05 arch kernel: NX (Execute Disable) protection: active
Oct 24 10:19:05 arch kernel: DMI present.
Oct 24 10:19:05 arch kernel: e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
Oct 24 10:19:05 arch kernel: e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
Oct 24 10:19:05 arch kernel: last_pfn = 0x120000 max_arch_pfn = 0x400000000
Oct 24 10:19:05 arch kernel: MTRR default type: uncachable
Oct 24 10:19:05 arch kernel: MTRR fixed ranges enabled:
Oct 24 10:19:05 arch kernel: 00000-9FFFF write-back
Oct 24 10:19:05 arch kernel: A0000-EFFFF uncachable
Oct 24 10:19:05 arch kernel: F0000-FFFFF write-protect
Oct 24 10:19:05 arch kernel: MTRR variable ranges enabled:
Oct 24 10:19:05 arch kernel: 0 base 000000000000 mask FFFF80000000 write-back
Oct 24 10:19:05 arch kernel: 1 base 000080000000 mask FFFFC0000000 write-back
Oct 24 10:19:05 arch kernel: 2 base 0000C0000000 mask FFFFE0000000 write-back
Oct 24 10:19:05 arch kernel: 3 base 0000F0000000 mask FFFFF8000000 write-combining
Oct 24 10:19:05 arch kernel: 4 disabled
Oct 24 10:19:05 arch kernel: 5 disabled
Oct 24 10:19:05 arch kernel: 6 disabled
Oct 24 10:19:05 arch kernel: 7 disabled
Oct 24 10:19:05 arch kernel: TOM2: 0000000120000000 aka 4608M
Oct 24 10:19:05 arch kernel: x86 PAT enabled: cpu 0, old 0x7010600070106, new 0x7010600070106
Oct 24 10:19:05 arch kernel: last_pfn = 0xdfe90 max_arch_pfn = 0x400000000
Oct 24 10:19:05 arch kernel: initial memory mapped : 0 - 20000000
Oct 24 10:19:05 arch kernel: Using GB pages for direct mapping
Oct 24 10:19:05 arch kernel: init_memory_mapping: 0000000000000000-00000000dfe90000
Oct 24 10:19:05 arch kernel: 0000000000 - 00c0000000 page 1G
Oct 24 10:19:05 arch kernel: 00c0000000 - 00dfe00000 page 2M
Oct 24 10:19:05 arch kernel: 00dfe00000 - 00dfe90000 page 4k
Oct 24 10:19:05 arch kernel: kernel direct mapping tables up to dfe90000 @ 1fffd000-20000000
Oct 24 10:19:05 arch kernel: init_memory_mapping: 0000000100000000-0000000120000000
Oct 24 10:19:05 arch kernel: 0100000000 - 0120000000 page 2M
Oct 24 10:19:05 arch kernel: kernel direct mapping tables up to 120000000 @ dfe8e000-dfe90000
Oct 24 10:19:05 arch kernel: ACPI: RSDP 00000000000fb880 00024 (v02 ACPIAM)
Oct 24 10:19:05 arch kernel: ACPI: XSDT 00000000dfe90100 0005C (v01 041910 XSDT2028 20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI: FACP 00000000dfe90290 000F4 (v03 041910 FACP2028 20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI Warning: Optional field Pm2ControlBlock has zero address or length: 0x0000000000000000/0x1 (20100702/tbfadt-557)
Oct 24 10:19:05 arch kernel: ACPI: DSDT 00000000dfe90450 0E6FE (v01  A1152 A1152000 00000000 INTL 20060113)
Oct 24 10:19:05 arch kernel: ACPI: FACS 00000000dfea8000 00040
Oct 24 10:19:05 arch kernel: ACPI: APIC 00000000dfe90390 0007C (v01 041910 APIC2028 20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI: MCFG 00000000dfe90410 0003C (v01 041910 OEMMCFG  20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI: OEMB 00000000dfea8040 00072 (v01 041910 OEMB2028 20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI: SRAT 00000000dfe9f450 000E8 (v03 AMD    FAM_F_10 00000002 AMD  00000001)
Oct 24 10:19:05 arch kernel: ACPI: HPET 00000000dfe9f540 00038 (v01 041910 OEMHPET  20100419 MSFT 00000097)
Oct 24 10:19:05 arch kernel: ACPI: SSDT 00000000dfe9f580 0088C (v01 A M I  POWERNOW 00000001 AMD  00000001)
Oct 24 10:19:05 arch kernel: ACPI: Local APIC address 0xfee00000
Oct 24 10:19:05 arch kernel: [ffffea0000000000-ffffea0003ffffff] PMD -> [ffff8800d0000000-ffff8800d39fffff] on node 0
Oct 24 10:19:05 arch kernel: Zone PFN ranges:
Oct 24 10:19:05 arch kernel: DMA      0x00000010 -> 0x00001000
Oct 24 10:19:05 arch kernel: DMA32    0x00001000 -> 0x00100000
Oct 24 10:19:05 arch kernel: Normal   0x00100000 -> 0x00120000
Oct 24 10:19:05 arch kernel: Movable zone start PFN for each node
Oct 24 10:19:05 arch kernel: early_node_map[3] active PFN ranges
Oct 24 10:19:05 arch kernel: 0: 0x00000010 -> 0x0000009f
Oct 24 10:19:05 arch kernel: 0: 0x00000100 -> 0x000dfe90
Oct 24 10:19:05 arch kernel: 0: 0x00100000 -> 0x00120000
Oct 24 10:19:05 arch kernel: On node 0 totalpages: 1048095
Oct 24 10:19:05 arch kernel: DMA zone: 56 pages used for memmap
Oct 24 10:19:05 arch kernel: DMA zone: 2 pages reserved
Oct 24 10:19:05 arch kernel: DMA zone: 3925 pages, LIFO batch:0
Oct 24 10:19:05 arch kernel: DMA32 zone: 14280 pages used for memmap
Oct 24 10:19:05 arch kernel: DMA32 zone: 898760 pages, LIFO batch:31
Oct 24 10:19:05 arch kernel: Normal zone: 1792 pages used for memmap
Oct 24 10:19:05 arch kernel: Normal zone: 129280 pages, LIFO batch:31
Oct 24 10:19:05 arch kernel: ACPI: PM-Timer IO Port: 0x808
Oct 24 10:19:05 arch kernel: ACPI: Local APIC address 0xfee00000
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x02] enabled)
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x84] disabled)
Oct 24 10:19:05 arch kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x85] disabled)
Oct 24 10:19:05 arch kernel: ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
Oct 24 10:19:05 arch kernel: IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
Oct 24 10:19:05 arch kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Oct 24 10:19:05 arch kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
Oct 24 10:19:05 arch kernel: ACPI: IRQ0 used by override.
Oct 24 10:19:05 arch kernel: ACPI: IRQ2 used by override.
Oct 24 10:19:05 arch kernel: ACPI: IRQ9 used by override.
Oct 24 10:19:05 arch kernel: Using ACPI (MADT) for SMP configuration information
Oct 24 10:19:05 arch kernel: ACPI: HPET id: 0x8300 base: 0xfed00000
Oct 24 10:19:05 arch kernel: SMP: Allowing 4 CPUs, 0 hotplug CPUs
Oct 24 10:19:05 arch kernel: nr_irqs_gsi: 40
Oct 24 10:19:05 arch kernel: Allocating PCI resources starting at dff00000 (gap: dff00000:20000000)
Oct 24 10:19:05 arch kernel: converting mcount calls to 0f 1f 44 00 00
Oct 24 10:19:05 arch kernel: setup_percpu: NR_CPUS:4 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
Oct 24 10:19:05 arch kernel: PERCPU: Embedded 24 pages/cpu @ffff8800dfc00000 s74176 r0 d24128 u524288
Oct 24 10:19:05 arch kernel: pcpu-alloc: s74176 r0 d24128 u524288 alloc=1*2097152
Oct 24 10:19:05 arch kernel: pcpu-alloc: [0] 0 1 2 3 
Oct 24 10:19:05 arch kernel: Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1031965
Oct 24 10:19:05 arch kernel: Kernel command line: root=/dev/sdb2 fbcon=rotate:3 quiet
Oct 24 10:19:05 arch kernel: PID hash table entries: 4096 (order: 3, 32768 bytes)
Oct 24 10:19:05 arch kernel: Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Oct 24 10:19:05 arch kernel: Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Oct 24 10:19:05 arch kernel: Memory: 4053416k/4718592k available (4432k kernel code, 526212k absent, 138964k reserved, 1733k data, 424k init)
Oct 24 10:19:05 arch kernel: SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Oct 24 10:19:05 arch kernel: Preemptable hierarchical RCU implementation.
Oct 24 10:19:05 arch kernel: 	RCU-based detection of stalled CPUs is disabled.
Oct 24 10:19:05 arch kernel: 	Verbose stalled-CPUs detection is disabled.
Oct 24 10:19:05 arch kernel: NR_IRQS:384
Oct 24 10:19:05 arch kernel: Extended CMOS year: 2000
Oct 24 10:19:05 arch kernel: Console: colour VGA+ 80x25
Oct 24 10:19:05 arch kernel: console [tty0] enabled
Oct 24 10:19:05 arch kernel: hpet clockevent registered
Oct 24 10:19:05 arch kernel: Fast TSC calibration using PIT
Oct 24 10:19:05 arch kernel: Detected 3211.206 MHz processor.
Oct 24 10:19:05 arch kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 6422.41 BogoMIPS (lpj=3211206)
Oct 24 10:19:05 arch kernel: pid_max: default: 32768 minimum: 301
Oct 24 10:19:05 arch kernel: Mount-cache hash table entries: 256
Oct 24 10:19:05 arch kernel: Initializing cgroup subsys blkio
Oct 24 10:19:05 arch kernel: tseg: 0000000000
Oct 24 10:19:05 arch kernel: CPU: Physical Processor ID: 0
Oct 24 10:19:05 arch kernel: CPU: Processor Core ID: 0
Oct 24 10:19:05 arch kernel: mce: CPU supports 6 MCE banks
Oct 24 10:19:05 arch kernel: using C1E aware idle routine
Oct 24 10:19:05 arch kernel: Performance Events: AMD PMU driver.
Oct 24 10:19:05 arch kernel: ... version:                0
Oct 24 10:19:05 arch kernel: ... bit width:              48
Oct 24 10:19:05 arch kernel: ... generic registers:      4
Oct 24 10:19:05 arch kernel: ... value mask:             0000ffffffffffff
Oct 24 10:19:05 arch kernel: ... max period:             00007fffffffffff
Oct 24 10:19:05 arch kernel: ... fixed-purpose events:   0
Oct 24 10:19:05 arch kernel: ... event mask:             000000000000000f
Oct 24 10:19:05 arch kernel: Freeing SMP alternatives: 16k freed
Oct 24 10:19:05 arch kernel: ACPI: Core revision 20100702
Oct 24 10:19:05 arch kernel: Setting APIC routing to flat
Oct 24 10:19:05 arch kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Oct 24 10:19:05 arch kernel: CPU0: AMD Phenom(tm) II X4 955 Processor stepping 02
Oct 24 10:19:05 arch kernel: System has AMD C1E enabled
Oct 24 10:19:05 arch kernel: Switch to broadcast mode on CPU0
Oct 24 10:19:05 arch kernel: Booting Node   0, Processors  #1
Oct 24 10:19:05 arch kernel: Switch to broadcast mode on CPU1
Oct 24 10:19:05 arch kernel: #2
Oct 24 10:19:05 arch kernel: Switch to broadcast mode on CPU2
Oct 24 10:19:05 arch kernel: #3 Ok.
Oct 24 10:19:05 arch kernel: Brought up 4 CPUs
Oct 24 10:19:05 arch kernel: Total of 4 processors activated (25687.49 BogoMIPS).
Oct 24 10:19:05 arch kernel: Switch to broadcast mode on CPU3
Oct 24 10:19:05 arch kernel: NET: Registered protocol family 16
Oct 24 10:19:05 arch kernel: node 0 link 0: io port [1000, ffffff]
Oct 24 10:19:05 arch kernel: TOM: 00000000e0000000 aka 3584M
Oct 24 10:19:05 arch kernel: Fam 10h mmconf [e0000000, efffffff]
Oct 24 10:19:05 arch kernel: node 0 link 0: mmio [a0000, bffff]
Oct 24 10:19:05 arch kernel: node 0 link 0: mmio [e0000000, efffffff] ==> none
Oct 24 10:19:05 arch kernel: node 0 link 0: mmio [f0000000, fbcfffff]
Oct 24 10:19:05 arch kernel: node 0 link 0: mmio [fbd00000, fbefffff]
Oct 24 10:19:05 arch kernel: node 0 link 0: mmio [fbf00000, ffefffff]
Oct 24 10:19:05 arch kernel: TOM2: 0000000120000000 aka 4608M
Oct 24 10:19:05 arch kernel: bus: [00, 07] on node 0 link 0
Oct 24 10:19:05 arch kernel: bus: 00 index 0 [io  0x0000-0xffff]
Oct 24 10:19:05 arch kernel: bus: 00 index 1 [mem 0x000a0000-0x000bffff]
Oct 24 10:19:05 arch kernel: bus: 00 index 2 [mem 0xf0000000-0xffffffff]
Oct 24 10:19:05 arch kernel: bus: 00 index 3 [mem 0x120000000-0xfcffffffff]
Oct 24 10:19:05 arch kernel: ACPI: bus type pci registered
Oct 24 10:19:05 arch kernel: PCI: Using configuration type 1 for base access
Oct 24 10:19:05 arch kernel: PCI: Using configuration type 1 for extended access
Oct 24 10:19:05 arch kernel: bio: create slab <bio-0> at 0
Oct 24 10:19:05 arch kernel: ACPI: EC: Look up EC in DSDT
Oct 24 10:19:05 arch kernel: ACPI: Executed 3 blocks of module-level executable AML code
Oct 24 10:19:05 arch kernel: ACPI: Interpreter enabled
Oct 24 10:19:05 arch kernel: ACPI: (supports S0 S5)
Oct 24 10:19:05 arch kernel: ACPI: Using IOAPIC for interrupt routing
Oct 24 10:19:05 arch kernel: PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Oct 24 10:19:05 arch kernel: ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [mem 0x000d0000-0x000dffff]
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [mem 0xdff00000-0xdfffffff]
Oct 24 10:19:05 arch kernel: pci_root PNP0A03:00: host bridge window [mem 0xf0000000-0xfebfffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 10: [io  0xc000-0xc007]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 14: [io  0xb000-0xb003]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 18: [io  0xa000-0xa007]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 1c: [io  0x9000-0x9003]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 20: [io  0x8000-0x800f]
Oct 24 10:19:05 arch kernel: pci 0000:00:11.0: reg 24: [mem 0xfbcffc00-0xfbcfffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:12.0: reg 10: [mem 0xfbcfd000-0xfbcfdfff]
Oct 24 10:19:05 arch kernel: pci 0000:00:12.1: reg 10: [mem 0xfbcfe000-0xfbcfefff]
Oct 24 10:19:05 arch kernel: pci 0000:00:12.2: reg 10: [mem 0xfbcff800-0xfbcff8ff]
Oct 24 10:19:05 arch kernel: pci 0000:00:12.2: supports D1 D2
Oct 24 10:19:05 arch kernel: pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
Oct 24 10:19:05 arch kernel: pci 0000:00:12.2: PME# disabled
Oct 24 10:19:05 arch kernel: pci 0000:00:13.0: reg 10: [mem 0xfbcfb000-0xfbcfbfff]
Oct 24 10:19:05 arch kernel: pci 0000:00:13.1: reg 10: [mem 0xfbcfc000-0xfbcfcfff]
Oct 24 10:19:05 arch kernel: pci 0000:00:13.2: reg 10: [mem 0xfbcff400-0xfbcff4ff]
Oct 24 10:19:05 arch kernel: pci 0000:00:13.2: supports D1 D2
Oct 24 10:19:05 arch kernel: pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
Oct 24 10:19:05 arch kernel: pci 0000:00:13.2: PME# disabled
Oct 24 10:19:05 arch kernel: pci 0000:00:14.1: reg 10: [io  0x0000-0x0007]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.1: reg 14: [io  0x0000-0x0003]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.1: reg 18: [io  0x0000-0x0007]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.1: reg 1c: [io  0x0000-0x0003]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.1: reg 20: [io  0xff00-0xff0f]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.5: reg 10: [mem 0xfbcfa000-0xfbcfafff]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: reg 10: [mem 0xf0000000-0xf7ffffff pref]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: reg 14: [io  0xd000-0xd0ff]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: reg 18: [mem 0xfbee0000-0xfbeeffff]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: reg 24: [mem 0xfbd00000-0xfbdfffff]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: supports D1 D2
Oct 24 10:19:05 arch kernel: pci 0000:01:05.1: reg 10: [mem 0xfbefc000-0xfbefffff]
Oct 24 10:19:05 arch kernel: pci 0000:01:05.1: supports D1 D2
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [io  0xd000-0xdfff]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [mem 0xfbd00000-0xfbefffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: reg 10: [io  0xe800-0xe8ff]
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: reg 14: [mem 0xfbfffc00-0xfbfffcff]
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: reg 30: [mem 0xfbfc0000-0xfbfdffff pref]
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: supports D1 D2
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: PME# supported from D1 D2 D3hot D3cold
Oct 24 10:19:05 arch kernel: pci 0000:02:05.0: PME# disabled
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4: PCI bridge to [bus 02-02] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [io  0xe000-0xefff]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0xfbf00000-0xfbffffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [io  0x0d00-0xffff] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0x000d0000-0x000dffff] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0xdff00000-0xdfffffff] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0xf0000000-0xfebfffff] (subtractive decode)
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: on NUMA node 0
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0PC._PRT]
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKE] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKF] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKG] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: ACPI: PCI Interrupt Link [LNKH] (IRQs 4 7 10 11 12 14 15) *0, disabled.
Oct 24 10:19:05 arch kernel: SCSI subsystem initialized
Oct 24 10:19:05 arch kernel: libata version 3.00 loaded.
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver usbfs
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver hub
Oct 24 10:19:05 arch kernel: usbcore: registered new device driver usb
Oct 24 10:19:05 arch kernel: Advanced Linux Sound Architecture Driver Version 1.0.23.
Oct 24 10:19:05 arch kernel: PCI: Using ACPI for IRQ routing
Oct 24 10:19:05 arch kernel: PCI: pci_cache_line_size set to 64 bytes
Oct 24 10:19:05 arch kernel: reserve RAM buffer: 000000000009fc00 - 000000000009ffff 
Oct 24 10:19:05 arch kernel: reserve RAM buffer: 00000000dfe90000 - 00000000dfffffff 
Oct 24 10:19:05 arch kernel: hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0
Oct 24 10:19:05 arch kernel: hpet0: 4 comparators, 32-bit 14.318180 MHz counter
Oct 24 10:19:05 arch kernel: Switching to clocksource tsc
Oct 24 10:19:05 arch kernel: pnp: PnP ACPI init
Oct 24 10:19:05 arch kernel: ACPI: bus type pnp registered
Oct 24 10:19:05 arch kernel: pnp: PnP ACPI: found 12 devices
Oct 24 10:19:05 arch kernel: ACPI: ACPI bus type pnp unregistered
Oct 24 10:19:05 arch kernel: system 00:07: [mem 0xfec00000-0xfec00fff] could not be reserved
Oct 24 10:19:05 arch kernel: system 00:07: [mem 0xfee00000-0xfee00fff] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x04d0-0x04d1] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x040b] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x04d6] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c00-0x0c01] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c14] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c50-0x0c51] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c52] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c6c] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0c6f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0cd0-0x0cd1] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0cd2-0x0cd3] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0cd4-0x0cd5] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0cd6-0x0cd7] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0cd8-0x0cdf] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0b00-0x0b3f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0800-0x089f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0b00-0x0b0f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0b20-0x0b3f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0900-0x090f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0x0910-0x091f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [io  0xfe00-0xfefe] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [mem 0xdff00000-0xdfffffff] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [mem 0xffb80000-0xffbfffff] has been reserved
Oct 24 10:19:05 arch kernel: system 00:08: [mem 0xfec10000-0xfec1001f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:09: [io  0x0230-0x023f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:09: [io  0x0290-0x029f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:09: [io  0x0f40-0x0f4f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:09: [io  0x0a30-0x0a3f] has been reserved
Oct 24 10:19:05 arch kernel: system 00:0a: [mem 0xe0000000-0xefffffff] has been reserved
Oct 24 10:19:05 arch kernel: system 00:0b: [mem 0x00000000-0x0009ffff] could not be reserved
Oct 24 10:19:05 arch kernel: system 00:0b: [mem 0x000c0000-0x000cffff] has been reserved
Oct 24 10:19:05 arch kernel: system 00:0b: [mem 0x000e0000-0x000fffff] could not be reserved
Oct 24 10:19:05 arch kernel: system 00:0b: [mem 0x00100000-0xdfefffff] could not be reserved
Oct 24 10:19:05 arch kernel: system 00:0b: [mem 0xfec00000-0xffffffff] could not be reserved
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [io  0xd000-0xdfff]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [mem 0xfbd00000-0xfbefffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0:   bridge window [mem 0xf0000000-0xf7ffffff 64bit pref]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4: PCI bridge to [bus 02-02]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [io  0xe000-0xefff]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem 0xfbf00000-0xfbffffff]
Oct 24 10:19:05 arch kernel: pci 0000:00:14.4:   bridge window [mem pref disabled]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000dffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 8 [mem 0xdff00000-0xdfffffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:00: resource 9 [mem 0xf0000000-0xfebfffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:01: resource 0 [io  0xd000-0xdfff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:01: resource 1 [mem 0xfbd00000-0xfbefffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:01: resource 2 [mem 0xf0000000-0xf7ffffff 64bit pref]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 0 [io  0xe000-0xefff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 1 [mem 0xfbf00000-0xfbffffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 4 [io  0x0000-0x0cf7]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 5 [io  0x0d00-0xffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 6 [mem 0x000a0000-0x000bffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 7 [mem 0x000d0000-0x000dffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 8 [mem 0xdff00000-0xdfffffff]
Oct 24 10:19:05 arch kernel: pci_bus 0000:02: resource 9 [mem 0xf0000000-0xfebfffff]
Oct 24 10:19:05 arch kernel: NET: Registered protocol family 2
Oct 24 10:19:05 arch kernel: IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)
Oct 24 10:19:05 arch kernel: TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
Oct 24 10:19:05 arch kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
Oct 24 10:19:05 arch kernel: TCP: Hash tables configured (established 262144 bind 65536)
Oct 24 10:19:05 arch kernel: TCP reno registered
Oct 24 10:19:05 arch kernel: UDP hash table entries: 2048 (order: 4, 65536 bytes)
Oct 24 10:19:05 arch kernel: UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
Oct 24 10:19:05 arch kernel: NET: Registered protocol family 1
Oct 24 10:19:05 arch kernel: pci 0000:00:01.0: MSI quirk detected; subordinate MSI disabled
Oct 24 10:19:05 arch kernel: pci 0000:01:05.0: Boot video device
Oct 24 10:19:05 arch kernel: PCI: CLS mismatch (64 != 32), using 64 bytes
Oct 24 10:19:05 arch kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Oct 24 10:19:05 arch kernel: Placing 64MB software IO TLB between ffff8800db48b000 - ffff8800df48b000
Oct 24 10:19:05 arch kernel: software IO TLB at phys 0xdb48b000 - 0xdf48b000
Oct 24 10:19:05 arch kernel: kvm: Nested Virtualization enabled
Oct 24 10:19:05 arch kernel: kvm: Nested Paging enabled
Oct 24 10:19:05 arch kernel: msgmni has been set to 7916
Oct 24 10:19:05 arch kernel: Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
Oct 24 10:19:05 arch kernel: io scheduler noop registered
Oct 24 10:19:05 arch kernel: io scheduler cfq registered (default)
Oct 24 10:19:05 arch kernel: input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
Oct 24 10:19:05 arch kernel: ACPI: Power Button [PWRB]
Oct 24 10:19:05 arch kernel: input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1
Oct 24 10:19:05 arch kernel: ACPI: Power Button [PWRF]
Oct 24 10:19:05 arch kernel: ACPI: acpi_idle registered with cpuidle
Oct 24 10:19:05 arch kernel: ACPI: processor limited to max C-state 1
Oct 24 10:19:05 arch kernel: Real Time Clock Driver v1.12b
Oct 24 10:19:05 arch kernel: Linux agpgart interface v0.103
Oct 24 10:19:05 arch kernel: [drm] Initialized drm 1.1.0 20060810
Oct 24 10:19:05 arch kernel: [drm] radeon defaulting to kernel modesetting.
Oct 24 10:19:05 arch kernel: [drm] radeon kernel modesetting enabled.
Oct 24 10:19:05 arch kernel: radeon 0000:01:05.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
Oct 24 10:19:05 arch kernel: radeon 0000:01:05.0: setting latency timer to 64
Oct 24 10:19:05 arch kernel: [drm] initializing kernel modesetting (RS780 0x1002:0x9614).
Oct 24 10:19:05 arch kernel: [drm] register mmio base: 0xFBEE0000
Oct 24 10:19:05 arch kernel: [drm] register mmio size: 65536
Oct 24 10:19:05 arch kernel: ATOM BIOS: 113
Oct 24 10:19:05 arch kernel: radeon 0000:01:05.0: VRAM: 128M 0xC0000000 - 0xC7FFFFFF (128M used)
Oct 24 10:19:05 arch kernel: radeon 0000:01:05.0: GTT: 512M 0xA0000000 - 0xBFFFFFFF
Oct 24 10:19:05 arch kernel: [drm] Detected VRAM RAM=128M, BAR=128M
Oct 24 10:19:05 arch kernel: [drm] RAM width 32bits DDR
Oct 24 10:19:05 arch kernel: [TTM] Zone  kernel: Available graphics memory: 2026716 kiB.
Oct 24 10:19:05 arch kernel: [TTM] Initializing pool allocator.
Oct 24 10:19:05 arch kernel: [drm] radeon: 128M of VRAM memory ready
Oct 24 10:19:05 arch kernel: [drm] radeon: 512M of GTT memory ready.
Oct 24 10:19:05 arch kernel: [drm] radeon: irq initialized.
Oct 24 10:19:05 arch kernel: [drm] GART: num cpu pages 131072, num gpu pages 131072
Oct 24 10:19:05 arch kernel: [drm] Loading RS780 Microcode
Oct 24 10:19:05 arch kernel: [drm] ring test succeeded in 1 usecs
Oct 24 10:19:05 arch kernel: [drm] radeon: ib pool ready.
Oct 24 10:19:05 arch kernel: [drm] ib test succeeded in 0 usecs
Oct 24 10:19:05 arch kernel: [drm] Enabling audio support
Oct 24 10:19:05 arch kernel: [drm] Radeon Display Connectors
Oct 24 10:19:05 arch kernel: [drm] Connector 0:
Oct 24 10:19:05 arch kernel: [drm]   VGA
Oct 24 10:19:05 arch kernel: [drm]   DDC: 0x7e40 0x7e40 0x7e44 0x7e44 0x7e48 0x7e48 0x7e4c 0x7e4c
Oct 24 10:19:05 arch kernel: [drm]   Encoders:
Oct 24 10:19:05 arch kernel: [drm]     CRT1: INTERNAL_KLDSCP_DAC1
Oct 24 10:19:05 arch kernel: [drm] Connector 1:
Oct 24 10:19:05 arch kernel: [drm]   DVI-D
Oct 24 10:19:05 arch kernel: [drm]   HPD3
Oct 24 10:19:05 arch kernel: [drm]   DDC: 0x7e50 0x7e50 0x7e54 0x7e54 0x7e58 0x7e58 0x7e5c 0x7e5c
Oct 24 10:19:05 arch kernel: [drm]   Encoders:
Oct 24 10:19:05 arch kernel: [drm]     DFP3: INTERNAL_KLDSCP_LVTMA
Oct 24 10:19:05 arch kernel: [drm] radeon: power management initialized
Oct 24 10:19:05 arch kernel: [drm] fb mappable at 0xF0141000
Oct 24 10:19:05 arch kernel: [drm] vram apper at 0xF0000000
Oct 24 10:19:05 arch kernel: [drm] size 7258112
Oct 24 10:19:05 arch kernel: [drm] fb depth is 24
Oct 24 10:19:05 arch kernel: [drm]    pitch is 6912
Oct 24 10:19:05 arch kernel: Console: switching to colour frame buffer device 131x105
Oct 24 10:19:05 arch kernel: fb0: radeondrmfb frame buffer device
Oct 24 10:19:05 arch kernel: drm: registered panic notifier
Oct 24 10:19:05 arch kernel: [drm] Initialized radeon 2.6.0 20080528 for 0000:01:05.0 on minor 0
Oct 24 10:19:05 arch kernel: loop: module loaded
Oct 24 10:19:05 arch kernel: ahci 0000:00:11.0: version 3.0
Oct 24 10:19:05 arch kernel: ahci 0000:00:11.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
Oct 24 10:19:05 arch kernel: ahci 0000:00:11.0: AHCI 0001.0100 32 slots 4 ports 3 Gbps 0xf impl SATA mode
Oct 24 10:19:05 arch kernel: ahci 0000:00:11.0: flags: 64bit ncq sntf ilck pm led clo pmp pio slum part ccc 
Oct 24 10:19:05 arch kernel: scsi0 : ahci
Oct 24 10:19:05 arch kernel: scsi1 : ahci
Oct 24 10:19:05 arch kernel: scsi2 : ahci
Oct 24 10:19:05 arch kernel: scsi3 : ahci
Oct 24 10:19:05 arch kernel: ata1: SATA max UDMA/133 irq_stat 0x00400000, PHY RDY changed
Oct 24 10:19:05 arch kernel: ata2: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffd80 irq 22
Oct 24 10:19:05 arch kernel: ata3: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffe00 irq 22
Oct 24 10:19:05 arch kernel: ata4: SATA max UDMA/133 abar m1024@0xfbcffc00 port 0xfbcffe80 irq 22
Oct 24 10:19:05 arch kernel: pata_atiixp 0000:00:14.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 24 10:19:05 arch kernel: scsi4 : pata_atiixp
Oct 24 10:19:05 arch kernel: scsi5 : pata_atiixp
Oct 24 10:19:05 arch kernel: ata5: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xff00 irq 14
Oct 24 10:19:05 arch kernel: ata6: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xff08 irq 15
Oct 24 10:19:05 arch kernel: PPP generic driver version 2.4.2
Oct 24 10:19:05 arch kernel: tun: Universal TUN/TAP device driver, 1.6
Oct 24 10:19:05 arch kernel: tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
Oct 24 10:19:05 arch kernel: r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
Oct 24 10:19:05 arch kernel: r8169 0000:02:05.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
Oct 24 10:19:05 arch kernel: r8169 0000:02:05.0: (unregistered net_device): no PCI Express capability
Oct 24 10:19:05 arch kernel: r8169 0000:02:05.0: eth0: RTL8110s at 0xffffc90000452c00, 00:08:54:36:f2:2f, XID 04000000 IRQ 20
Oct 24 10:19:05 arch kernel: usbmon: debugfs is not available
Oct 24 10:19:05 arch kernel: ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: PCI INT B -> GSI 17 (level, low) -> IRQ 17
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: EHCI Host Controller
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: new USB bus registered, assigned bus number 1
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: applying AMD SB600/SB700 USB freeze workaround
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: debug port 1
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: irq 17, io mem 0xfbcff800
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:12.2: USB 2.0 started, EHCI 1.00
Oct 24 10:19:05 arch kernel: hub 1-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 1-0:1.0: 6 ports detected
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: EHCI Host Controller
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: new USB bus registered, assigned bus number 2
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: applying AMD SB600/SB700 USB freeze workaround
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: debug port 1
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: irq 19, io mem 0xfbcff400
Oct 24 10:19:05 arch kernel: ehci_hcd 0000:00:13.2: USB 2.0 started, EHCI 1.00
Oct 24 10:19:05 arch kernel: hub 2-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 2-0:1.0: 6 ports detected
Oct 24 10:19:05 arch kernel: ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.0: OHCI Host Controller
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.0: new USB bus registered, assigned bus number 3
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.0: irq 16, io mem 0xfbcfd000
Oct 24 10:19:05 arch kernel: hub 3-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 3-0:1.0: 3 ports detected
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.1: PCI INT A -> GSI 16 (level, low) -> IRQ 16
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.1: OHCI Host Controller
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.1: new USB bus registered, assigned bus number 4
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:12.1: irq 16, io mem 0xfbcfe000
Oct 24 10:19:05 arch kernel: hub 4-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 4-0:1.0: 3 ports detected
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.0: OHCI Host Controller
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.0: new USB bus registered, assigned bus number 5
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.0: irq 18, io mem 0xfbcfb000
Oct 24 10:19:05 arch kernel: ata5.00: ATAPI: HL-DT-STDVD-RAM GH22NP20, 1.03, max UDMA/66
Oct 24 10:19:05 arch kernel: ata5.00: configured for UDMA/66
Oct 24 10:19:05 arch kernel: hub 5-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 5-0:1.0: 3 ports detected
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.1: OHCI Host Controller
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.1: new USB bus registered, assigned bus number 6
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:13.1: irq 18, io mem 0xfbcfc000
Oct 24 10:19:05 arch kernel: hub 6-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 6-0:1.0: 3 ports detected
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:14.5: PCI INT C -> GSI 18 (level, low) -> IRQ 18
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:14.5: OHCI Host Controller
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:14.5: new USB bus registered, assigned bus number 7
Oct 24 10:19:05 arch kernel: ohci_hcd 0000:00:14.5: irq 18, io mem 0xfbcfa000
Oct 24 10:19:05 arch kernel: hub 7-0:1.0: USB hub found
Oct 24 10:19:05 arch kernel: hub 7-0:1.0: 2 ports detected
Oct 24 10:19:05 arch kernel: Initializing USB Mass Storage driver...
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver usb-storage
Oct 24 10:19:05 arch kernel: USB Mass Storage support registered.
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver usbserial
Oct 24 10:19:05 arch kernel: USB Serial support registered for generic
Oct 24 10:19:05 arch kernel: ata4: SATA link down (SStatus 0 SControl 300)
Oct 24 10:19:05 arch kernel: ata2: SATA link down (SStatus 0 SControl 300)
Oct 24 10:19:05 arch kernel: ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Oct 24 10:19:05 arch kernel: ata3.00: ATA-8: OCZ-VERTEX, 1.6, max UDMA/133
Oct 24 10:19:05 arch kernel: ata3.00: 62533296 sectors, multi 1: LBA48 NCQ (depth 31/32), AA
Oct 24 10:19:05 arch kernel: ata3.00: configured for UDMA/133
Oct 24 10:19:05 arch kernel: usb 4-1: new full speed USB device using ohci_hcd and address 2
Oct 24 10:19:05 arch kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Oct 24 10:19:05 arch kernel: ata1.00: ATA-7: SAMSUNG HD103UJ, 1AA01118, max UDMA7
Oct 24 10:19:05 arch kernel: ata1.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
Oct 24 10:19:05 arch kernel: ata1.00: configured for UDMA/133
Oct 24 10:19:05 arch kernel: scsi 0:0:0:0: Direct-Access     ATA      SAMSUNG HD103UJ  1AA0 PQ: 0 ANSI: 5
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB)
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: [sda] Write Protect is off
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: Attached scsi generic sg0 type 0
Oct 24 10:19:05 arch kernel: scsi 2:0:0:0: Direct-Access     ATA      OCZ-VERTEX       1.6  PQ: 0 ANSI: 5
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: [sdb] 62533296 512-byte logical blocks: (32.0 GB/29.8 GiB)
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: Attached scsi generic sg1 type 0
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: [sdb] Write Protect is off
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 10:19:05 arch kernel: scsi 4:0:0:0: CD-ROM            HL-DT-ST DVD-RAM GH22NP20 1.03 PQ: 0 ANSI: 5
Oct 24 10:19:05 arch kernel: sdb: sdb1 sdb2
Oct 24 10:19:05 arch kernel: sd 2:0:0:0: [sdb] Attached SCSI disk
Oct 24 10:19:05 arch kernel: sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
Oct 24 10:19:05 arch kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 24 10:19:05 arch kernel: sr 4:0:0:0: Attached scsi CD-ROM sr0
Oct 24 10:19:05 arch kernel: sr 4:0:0:0: Attached scsi generic sg2 type 5
Oct 24 10:19:05 arch kernel: sda: sda1 sda2 sda3
Oct 24 10:19:05 arch kernel: sd 0:0:0:0: [sda] Attached SCSI disk
Oct 24 10:19:05 arch kernel: usb 4-2: new full speed USB device using ohci_hcd and address 3
Oct 24 10:19:05 arch kernel: usb 4-3: new low speed USB device using ohci_hcd and address 4
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver usbserial_generic
Oct 24 10:19:05 arch kernel: usbserial: USB Serial Driver core
Oct 24 10:19:05 arch kernel: USB Serial support registered for GSM modem (1-port)
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver option
Oct 24 10:19:05 arch kernel: option: v0.7.2:USB Driver for GSM modems
Oct 24 10:19:05 arch kernel: PNP: No PS/2 controller found. Probing ports directly.
Oct 24 10:19:05 arch kernel: serio: i8042 KBD port at 0x60,0x64 irq 1
Oct 24 10:19:05 arch kernel: serio: i8042 AUX port at 0x60,0x64 irq 12
Oct 24 10:19:05 arch kernel: mice: PS/2 mouse device common for all mice
Oct 24 10:19:05 arch kernel: i2c /dev entries driver
Oct 24 10:19:05 arch kernel: cpuidle: using governor ladder
Oct 24 10:19:05 arch kernel: cpuidle: using governor menu
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver hiddev
Oct 24 10:19:05 arch kernel: input: C-Media USB Headphone Set   as /devices/pci0000:00/0000:00:12.1/usb4/4-1/4-1:1.3/input/input2
Oct 24 10:19:05 arch kernel: generic-usb 0003:0D8C:000C.0001: input,hidraw0: USB HID v1.00 Device [C-Media USB Headphone Set  ] on usb-0000:00:12.1-1/input3
Oct 24 10:19:05 arch kernel: input: Logitech USB Receiver as /devices/pci0000:00/0000:00:12.1/usb4/4-2/4-2:1.0/input/input3
Oct 24 10:19:05 arch kernel: generic-usb 0003:046D:C52B.0002: input,hidraw1: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:12.1-2/input0
Oct 24 10:19:05 arch kernel: input: Logitech USB Receiver as /devices/pci0000:00/0000:00:12.1/usb4/4-2/4-2:1.1/input/input4
Oct 24 10:19:05 arch kernel: generic-usb 0003:046D:C52B.0003: input,hiddev0,hidraw2: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:12.1-2/input1
Oct 24 10:19:05 arch kernel: generic-usb 0003:046D:C52B.0004: hiddev0,hidraw3: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:12.1-2/input2
Oct 24 10:19:05 arch kernel: input: HID 046a:0011 as /devices/pci0000:00/0000:00:12.1/usb4/4-3/4-3:1.0/input/input5
Oct 24 10:19:05 arch kernel: generic-usb 0003:046A:0011.0005: input,hidraw4: USB HID v1.10 Keyboard [HID 046a:0011] on usb-0000:00:12.1-3/input0
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver usbhid
Oct 24 10:19:05 arch kernel: usbhid: USB HID core driver
Oct 24 10:19:05 arch kernel: usbcore: registered new interface driver snd-usb-audio
Oct 24 10:19:05 arch kernel: ALSA device list:
Oct 24 10:19:05 arch kernel: #0: C-Media USB Headphone Set   at usb-0000:00:12.1-1, full speed
Oct 24 10:19:05 arch kernel: Netfilter messages via NETLINK v0.30.
Oct 24 10:19:05 arch kernel: nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Oct 24 10:19:05 arch kernel: ctnetlink v0.93: registering with nfnetlink.
Oct 24 10:19:05 arch kernel: ip_tables: (C) 2000-2006 Netfilter Core Team
Oct 24 10:19:05 arch kernel: TCP cubic registered
Oct 24 10:19:05 arch kernel: NET: Registered protocol family 17
Oct 24 10:19:05 arch kernel: powernow-k8: Found 1 AMD Phenom(tm) II X4 955 Processor (4 cpu cores) (version 2.20.00)
Oct 24 10:19:05 arch kernel: powernow-k8:    0 : pstate 0 (3200 MHz)
Oct 24 10:19:05 arch kernel: powernow-k8:    1 : pstate 1 (2500 MHz)
Oct 24 10:19:05 arch kernel: powernow-k8:    2 : pstate 2 (2100 MHz)
Oct 24 10:19:05 arch kernel: powernow-k8:    3 : pstate 3 (800 MHz)
Oct 24 10:19:05 arch kernel: EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null)
Oct 24 10:19:05 arch kernel: VFS: Mounted root (ext4 filesystem) readonly on device 8:18.
Oct 24 10:19:05 arch kernel: Freeing unused kernel memory: 424k freed
Oct 24 10:19:05 arch kernel: udev[751]: starting version 163
Oct 24 10:19:05 arch kernel: EXT4-fs (sdb2): re-mounted. Opts: (null)
Oct 24 10:19:05 arch kernel: EXT4-fs (sdb2): re-mounted. Opts: (null)
Oct 24 10:19:05 arch kernel: EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
Oct 24 10:19:05 arch kernel: EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
Oct 24 10:19:05 arch kernel: Adding 200808k swap on /dev/sda3.  Priority:-1 extents:1 across:200808k 
Oct 24 10:19:05 arch kernel: r8169 0000:02:05.0: eth0: link up
Oct 24 16:56:39 arch kernel: usb 5-1: new full speed USB device using ohci_hcd and address 2
Oct 24 16:56:40 arch kernel: input: Burr-Brown from TI               USB Audio CODEC  as /devices/pci0000:00/0000:00:13.0/usb5/5-1/5-1:1.3/input/input6
Oct 24 16:56:40 arch kernel: generic-usb 0003:08BB:2902.0006: input,hidraw5: USB HID v1.00 Device [Burr-Brown from TI               USB Audio CODEC ] on usb-0000:00:13.0-1/input3
Oct 24 16:56:43 arch kernel: usb 5-1: USB disconnect, address 2
Oct 24 16:57:05 arch kernel: usb 5-2: new full speed USB device using ohci_hcd and address 3
Oct 24 16:57:05 arch kernel: input: Burr-Brown from TI               USB Audio CODEC  as /devices/pci0000:00/0000:00:13.0/usb5/5-2/5-2:1.3/input/input7
Oct 24 16:57:05 arch kernel: generic-usb 0003:08BB:2902.0007: input,hidraw5: USB HID v1.00 Device [Burr-Brown from TI               USB Audio CODEC ] on usb-0000:00:13.0-2/input3
Oct 24 18:16:06 arch kernel: usb 5-2: USB disconnect, address 3
Oct 24 18:31:17 arch kernel: Clocksource tsc unstable (delta = -8591703466 ns)
Oct 24 18:31:17 arch kernel: Switching to clocksource hpet

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 13:58   ` Markus Trippelsdorf
  2010-10-26 14:05     ` Markus Trippelsdorf
@ 2010-10-26 14:38     ` Peter Zijlstra
  2010-10-26 15:46       ` Thomas Gleixner
  2010-10-26 15:48     ` Thomas Gleixner
  2010-10-27  3:24     ` Clocksource tsc unstable (delta = -8589361717 ns) in current git Mike Galbraith
  3 siblings, 1 reply; 46+ messages in thread
From: Peter Zijlstra @ 2010-10-26 14:38 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, linux-kernel, hpa, Thomas Gleixner, Ingo Molnar,
	Andreas Herrmann

On Tue, 2010-10-26 at 15:58 +0200, Markus Trippelsdorf wrote:
> > 
> > This is strange, tsc is actually stable on your F10h revC.
> 
> Yes. constant_tsc is among the CPU flags. 

constant_tsc only means the TSC is not affected by cpu freq changes.

It has zero relevance to cross-cpu sync of the TSC. The clocksource
stuff requires your TSC to be synchronous over all your CPUs.

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 14:38     ` Peter Zijlstra
@ 2010-10-26 15:46       ` Thomas Gleixner
  2010-10-26 16:41         ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Thomas Gleixner @ 2010-10-26 15:46 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Markus Trippelsdorf, Borislav Petkov, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann

On Tue, 26 Oct 2010, Peter Zijlstra wrote:

> On Tue, 2010-10-26 at 15:58 +0200, Markus Trippelsdorf wrote:
> > > 
> > > This is strange, tsc is actually stable on your F10h revC.
> > 
> > Yes. constant_tsc is among the CPU flags. 
>  
> constant_tsc only means the TSC is not affected by cpu freq changes.
> 
> It has zero relevance to cross-cpu sync of the TSC. The clocksource
> stuff requires your TSC to be synchronous over all your CPUs.

That and not affected by C-States either.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 13:58   ` Markus Trippelsdorf
  2010-10-26 14:05     ` Markus Trippelsdorf
  2010-10-26 14:38     ` Peter Zijlstra
@ 2010-10-26 15:48     ` Thomas Gleixner
  2010-10-26 17:56       ` Markus Trippelsdorf
  2010-10-26 19:18       ` john stultz
  2010-10-27  3:24     ` Clocksource tsc unstable (delta = -8589361717 ns) in current git Mike Galbraith
  3 siblings, 2 replies; 46+ messages in thread
From: Thomas Gleixner @ 2010-10-26 15:48 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, linux-kernel, hpa, Ingo Molnar,
	Andreas Herrmann, John Stultz

On Tue, 26 Oct 2010, Markus Trippelsdorf wrote:
> On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> > better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> > much but I don't know, if the issue appears every several hours it could
> > still be tedious.
> 
> That would be a several week long process, as the issue appears ~ every
> 2 days here on this machine running 24/7.

There is only a single commit in that area post 2.6.36:

      8af3c153baf95374eff20a37f00c59a295b52756 

But I have a hard time how this should make this happen. John ?

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 15:46       ` Thomas Gleixner
@ 2010-10-26 16:41         ` Borislav Petkov
  0 siblings, 0 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-10-26 16:41 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Peter Zijlstra, Markus Trippelsdorf, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann

From: Thomas Gleixner <tglx@linutronix.de>
Date: Tue, Oct 26, 2010 at 05:46:00PM +0200

> On Tue, 26 Oct 2010, Peter Zijlstra wrote:
> 
> > On Tue, 2010-10-26 at 15:58 +0200, Markus Trippelsdorf wrote:
> > > > 
> > > > This is strange, tsc is actually stable on your F10h revC.
> > > 
> > > Yes. constant_tsc is among the CPU flags. 
> >  
> > constant_tsc only means the TSC is not affected by cpu freq changes.
> > 
> > It has zero relevance to cross-cpu sync of the TSC. The clocksource
> > stuff requires your TSC to be synchronous over all your CPUs.
> 
> That and not affected by C-States either.

Well, F10h TSC is P- and C-state invariant - I've never seen any systems
not passing the TSC sync check at boot so far. And if there are still
systems where this is not the case or you see the TSC not behaving, make
sure to tell me about it.

Besides, the TSC calibration with PIT seems to function ok, and the
"Detected " line shows reasonable core frequency. FWIW, I have a similar
configuration here and will run latest -git on it too to check.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 15:48     ` Thomas Gleixner
@ 2010-10-26 17:56       ` Markus Trippelsdorf
  2010-10-26 18:19         ` Borislav Petkov
  2010-10-26 19:18       ` john stultz
  1 sibling, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-26 17:56 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Borislav Petkov, linux-kernel, hpa, Ingo Molnar,
	Andreas Herrmann, John Stultz

On Tue, Oct 26, 2010 at 05:48:41PM +0200, Thomas Gleixner wrote:
> On Tue, 26 Oct 2010, Markus Trippelsdorf wrote:
> > On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > > otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> > > better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> > > much but I don't know, if the issue appears every several hours it could
> > > still be tedious.
> > 
> > That would be a several week long process, as the issue appears ~ every
> > 2 days here on this machine running 24/7.
> 
> There is only a single commit in that area post 2.6.36:
> 
>       8af3c153baf95374eff20a37f00c59a295b52756 

There is also: acf01734b1747b1ec4be6f159aff579ea5f7f8e2
but I'm not sure if this commit is really relevant.

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 17:56       ` Markus Trippelsdorf
@ 2010-10-26 18:19         ` Borislav Petkov
  0 siblings, 0 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-10-26 18:19 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Thomas Gleixner, linux-kernel, hpa, Ingo Molnar, Herrmann3,
	Andreas, John Stultz

On Tue, Oct 26, 2010 at 01:56:39PM -0400, Markus Trippelsdorf wrote:
> There is also: acf01734b1747b1ec4be6f159aff579ea5f7f8e2 but I'm not
> sure if this commit is really relevant.

This shouldn't be at fault since your detected core frequency printed
after the TSC PIT calibration looks fine:

Oct 24 10:19:05 arch kernel: Fast TSC calibration using PIT
Oct 24 10:19:05 arch kernel: Detected 3211.206 MHz processor.

The changeset above removes the recalibration which was only required by
very early F10h's (and yours is not).

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 15:48     ` Thomas Gleixner
  2010-10-26 17:56       ` Markus Trippelsdorf
@ 2010-10-26 19:18       ` john stultz
  2010-10-27 14:26         ` Markus Trippelsdorf
  1 sibling, 1 reply; 46+ messages in thread
From: john stultz @ 2010-10-26 19:18 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Markus Trippelsdorf, Borislav Petkov, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann

On Tue, 2010-10-26 at 17:48 +0200, Thomas Gleixner wrote:
> On Tue, 26 Oct 2010, Markus Trippelsdorf wrote:
> > On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > > otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> > > better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> > > much but I don't know, if the issue appears every several hours it could
> > > still be tedious.
> > 
> > That would be a several week long process, as the issue appears ~ every
> > 2 days here on this machine running 24/7.
> 
> There is only a single commit in that area post 2.6.36:
> 
>       8af3c153baf95374eff20a37f00c59a295b52756 
> 
> But I have a hard time how this should make this happen. John ?

Yea, that one doesn't look connected to me.

There have been a few cases that I've seen where we can get false
positives for bad TSCs due to the watchdog clocksource having problems
(or the clocksource watchdog thread getting delayed for such a long time
the watchdog clocksource wraps and we then can't validly compare the two
- although this would be hard to trigger with non-rt kernels).

>From your dmesg, I'd guess the hpet is the watchdog clocksource, so I'd
be on the lookout for hpet related changes. 

Maybe does reverting 995bd3bb5c78f3ff71339803c0b8337ed36d64fb hide the
issue?

thanks
-john




^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 13:58   ` Markus Trippelsdorf
                       ` (2 preceding siblings ...)
  2010-10-26 15:48     ` Thomas Gleixner
@ 2010-10-27  3:24     ` Mike Galbraith
  3 siblings, 0 replies; 46+ messages in thread
From: Mike Galbraith @ 2010-10-27  3:24 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, linux-kernel, hpa, Thomas Gleixner, Ingo Molnar,
	Andreas Herrmann

On Tue, 2010-10-26 at 15:58 +0200, Markus Trippelsdorf wrote:

> > > Initializing cgroup subsys cpu
> > > Linux version 2.6.36-05618-gb18cae4-dirty (markus@arch.trippelsdorf.de) (gcc version 4.5.1 (GCC) ) #41 SMP PREEMPT Tue Oct 26 06:51:47 CEST 2010
> > 
> >  ...-dirty means you have some changes ontop of b18cae4. What are those?
> 
> The SCHED_AUTOGROUP patches from Mike Galbraith.

Turns out that patch is still racy btw.  Very hard to trigger, but you
should remove it for safety and test result integrity anyway.

	-Mike


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: Clocksource tsc unstable (delta = -8589361717 ns) in current git
  2010-10-26 19:18       ` john stultz
@ 2010-10-27 14:26         ` Markus Trippelsdorf
  2010-10-27 18:26           ` [bisected] Clocksource tsc unstable git markus
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-27 14:26 UTC (permalink / raw)
  To: john stultz
  Cc: Thomas Gleixner, Borislav Petkov, linux-kernel, hpa, Ingo Molnar,
	Andreas Herrmann

On Tue, Oct 26, 2010 at 12:18:56PM -0700, john stultz wrote:
> On Tue, 2010-10-26 at 17:48 +0200, Thomas Gleixner wrote:
> > On Tue, 26 Oct 2010, Markus Trippelsdorf wrote:
> > > On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > > > otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> > > > better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> > > > much but I don't know, if the issue appears every several hours it could
> > > > still be tedious.
> > > 
> > > That would be a several week long process, as the issue appears ~ every
> > > 2 days here on this machine running 24/7.
> > 
> > There is only a single commit in that area post 2.6.36:
> > 
> >       8af3c153baf95374eff20a37f00c59a295b52756 
> > 
> > But I have a hard time how this should make this happen. John ?
> 
> Yea, that one doesn't look connected to me.
> 
> There have been a few cases that I've seen where we can get false
> positives for bad TSCs due to the watchdog clocksource having problems
> (or the clocksource watchdog thread getting delayed for such a long time
> the watchdog clocksource wraps and we then can't validly compare the two
> - although this would be hard to trigger with non-rt kernels).

I think the remark above points in the right direction, because the
mysterious slowness happened again today (and this time on a vanilla git
kernel). But this time no "Clocksource tsc unstable" message appeared in
the kernel log. My system recovered by itself and after 2 minutes of
slowness it was usable again. (The symptoms were the same as desribed in
my first mail: Second long delays when typing and switching from X to
the console took many seconds.)

Is there anything I can do to pinpoint the cause of this slowdown should
it happen again? (Maybe a perf timechart?)

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 14:26         ` Markus Trippelsdorf
@ 2010-10-27 18:26           ` markus
  2010-10-27 19:36             ` Peter Zijlstra
  2010-10-29  8:18             ` Borislav Petkov
  0 siblings, 2 replies; 46+ messages in thread
From: markus @ 2010-10-27 18:26 UTC (permalink / raw)
  To: john stultz
  Cc: Thomas Gleixner, Borislav Petkov, linux-kernel, hpa, Ingo Molnar,
	Andreas Herrmann, heiko.carstens, a.p.zijlstra, avi, mtosatti

On Wed, Oct 27, 2010 at 04:26:22PM +0200, Markus Trippelsdorf wrote:
> On Tue, Oct 26, 2010 at 12:18:56PM -0700, john stultz wrote:
> > On Tue, 2010-10-26 at 17:48 +0200, Thomas Gleixner wrote:
> > > On Tue, 26 Oct 2010, Markus Trippelsdorf wrote:
> > > > On Tue, Oct 26, 2010 at 03:18:43PM +0200, Borislav Petkov wrote:
> > > > > otherwise, I don't see anything strange in your dmesg. Unless tglx has a
> > > > > better idea, I'd ask you to bisect it. 5618 changesets shouldn't be that
> > > > > much but I don't know, if the issue appears every several hours it could
> > > > > still be tedious.
> > > > 
> > > > That would be a several week long process, as the issue appears ~ every
> > > > 2 days here on this machine running 24/7.
> > > 
> > > There is only a single commit in that area post 2.6.36:
> > > 
> > >       8af3c153baf95374eff20a37f00c59a295b52756 
> > > 
> > > But I have a hard time how this should make this happen. John ?
> > 
> > Yea, that one doesn't look connected to me.
> > 
> > There have been a few cases that I've seen where we can get false
> > positives for bad TSCs due to the watchdog clocksource having problems
> > (or the clocksource watchdog thread getting delayed for such a long time
> > the watchdog clocksource wraps and we then can't validly compare the two
> > - although this would be hard to trigger with non-rt kernels).
> 
> I think the remark above points in the right direction, because the
> mysterious slowness happened again today (and this time on a vanilla git
> kernel). But this time no "Clocksource tsc unstable" message appeared in
> the kernel log. My system recovered by itself and after 2 minutes of
> slowness it was usable again. (The symptoms were the same as desribed in
> my first mail: Second long delays when typing and switching from X to
> the console took many seconds.)
> 
> Is there anything I can do to pinpoint the cause of this slowdown should
> it happen again? (Maybe a perf timechart?)

During my search for a reliable testcase I found out that kvm guests also
showed random hangs (that didn't happen in v2.6.36) here. These hangs last a
few seconds each, during which the mouse is not movable at all.

So I ran git-bisect with this testcase and the result of the bisection is:

34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date:   Wed Sep 22 13:53:15 2010 +0200

    sched: Create special class for stop/migrate work

    In order to separate the stop/migrate work thread from the SCHED_FIFO
    implementation, create a special class for it that is of higher priority than
    SCHED_FIFO itself.

    This currently solves a problem where cpu-hotplug consumes so much cpu-time
    that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
    timer pending on the now dead cpu.

    It is also required for when we add the planned deadline scheduling class above
    SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.

    Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
    Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
    LKML-Reference: <1285165776.2275.1022.camel@laptop>
    Signed-off-by: Ingo Molnar <mingo@elte.hu>

Reverting the commit solves the kvm hang issue.
(If this issue is related to my original tsc problem is of course open for
debate, but I have a strong hunch it is.)
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 18:26           ` [bisected] Clocksource tsc unstable git markus
@ 2010-10-27 19:36             ` Peter Zijlstra
  2010-10-27 19:42               ` Markus Trippelsdorf
                                 ` (2 more replies)
  2010-10-29  8:18             ` Borislav Petkov
  1 sibling, 3 replies; 46+ messages in thread
From: Peter Zijlstra @ 2010-10-27 19:36 UTC (permalink / raw)
  To: markus
  Cc: john stultz, Thomas Gleixner, Borislav Petkov, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann, heiko.carstens, avi, mtosatti

On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> 
> 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Date:   Wed Sep 22 13:53:15 2010 +0200
> 
>     sched: Create special class for stop/migrate work
> 
>     In order to separate the stop/migrate work thread from the SCHED_FIFO
>     implementation, create a special class for it that is of higher priority than
>     SCHED_FIFO itself.
> 
>     This currently solves a problem where cpu-hotplug consumes so much cpu-time
>     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
>     timer pending on the now dead cpu.
> 
>     It is also required for when we add the planned deadline scheduling class above
>     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> 
>     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
>     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
>     LKML-Reference: <1285165776.2275.1022.camel@laptop>
>     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> 
> Reverting the commit solves the kvm hang issue.
> (If this issue is related to my original tsc problem is of course open for
> debate, but I have a strong hunch it is.) 

Too weird,.. what does the hang look like?

Can you generate a sysrq-t dump? The thing I'm looking for is the
migration/# thread being runnable but not being current.

How can I reproduce this?

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 19:36             ` Peter Zijlstra
@ 2010-10-27 19:42               ` Markus Trippelsdorf
  2010-10-27 19:58               ` Marcelo Tosatti
  2010-11-09 12:58               ` Heiko Carstens
  2 siblings, 0 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-27 19:42 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: john stultz, Thomas Gleixner, Borislav Petkov, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann, heiko.carstens, avi, mtosatti

On Wed, Oct 27, 2010 at 09:36:55PM +0200, Peter Zijlstra wrote:
> On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> > 
> > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Date:   Wed Sep 22 13:53:15 2010 +0200
> > 
> >     sched: Create special class for stop/migrate work
> > 
> >     In order to separate the stop/migrate work thread from the SCHED_FIFO
> >     implementation, create a special class for it that is of higher priority than
> >     SCHED_FIFO itself.
> > 
> >     This currently solves a problem where cpu-hotplug consumes so much cpu-time
> >     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
> >     timer pending on the now dead cpu.
> > 
> >     It is also required for when we add the planned deadline scheduling class above
> >     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> > 
> >     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> >     LKML-Reference: <1285165776.2275.1022.camel@laptop>
> >     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> > 
> > Reverting the commit solves the kvm hang issue.
> > (If this issue is related to my original tsc problem is of course open for
> > debate, but I have a strong hunch it is.) 
> 
> Too weird,.. what does the hang look like?
> 
> Can you generate a sysrq-t dump? The thing I'm looking for is the
> migration/# thread being runnable but not being current.
> 
> How can I reproduce this?

It's quite easy to reproduce on my setup. I simply run:

qemu-kvm -net nic,vlan=0,model=virtio -net user -drive file=ubuntu,if=virtio,boot=on -smp 2 -m 512

and then move the mousepointer in circles in the qemu window. It will stop and
freeze after a few seconds.
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 19:36             ` Peter Zijlstra
  2010-10-27 19:42               ` Markus Trippelsdorf
@ 2010-10-27 19:58               ` Marcelo Tosatti
  2010-11-09 12:58               ` Heiko Carstens
  2 siblings, 0 replies; 46+ messages in thread
From: Marcelo Tosatti @ 2010-10-27 19:58 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: markus, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, heiko.carstens,
	avi

[-- Attachment #1: Type: text/plain, Size: 4132 bytes --]

On Wed, Oct 27, 2010 at 09:36:55PM +0200, Peter Zijlstra wrote:
> On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> > 
> > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Date:   Wed Sep 22 13:53:15 2010 +0200
> > 
> >     sched: Create special class for stop/migrate work
> > 
> >     In order to separate the stop/migrate work thread from the SCHED_FIFO
> >     implementation, create a special class for it that is of higher priority than
> >     SCHED_FIFO itself.
> > 
> >     This currently solves a problem where cpu-hotplug consumes so much cpu-time
> >     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
> >     timer pending on the now dead cpu.
> > 
> >     It is also required for when we add the planned deadline scheduling class above
> >     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> > 
> >     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> >     LKML-Reference: <1285165776.2275.1022.camel@laptop>
> >     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> > 
> > Reverting the commit solves the kvm hang issue.
> > (If this issue is related to my original tsc problem is of course open for
> > debate, but I have a strong hunch it is.) 
> 
> Too weird,.. what does the hang look like?
> 
> Can you generate a sysrq-t dump? The thing I'm looking for is the
> migration/# thread being runnable but not being current.
> 
> How can I reproduce this?

I can reproduce it reliably here (requires kvm-autotest setup, its
difficult to reproduce manually).

INFO: task qemu:1872 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this
message.
qemu          D ffff88007f44f1c0     0  1872      1 0x00000084
ffff88006522dbb8 0000000000000086 ffff8800ffffffff ffff88006522dfd8
0000000000013580 ffff880078dd0000 ffff880078dd0380 ffff880078dd0378
ffff88006522c000 ffff88006522dfd8 0000000000013580 0000000000013580
Call Trace:
[<ffffffff8143c919>] ? _raw_spin_unlock_irq+0x12/0x3c
[<ffffffff8143b111>] schedule_timeout+0x27/0xc0
[<ffffffff8143fc50>] ? sub_preempt_count+0xe/0xab
[<ffffffff810414e0>] ? get_parent_ip+0x11/0x41
[<ffffffff810414e0>] ? get_parent_ip+0x11/0x41
[<ffffffff8143fcd9>] ? sub_preempt_count+0x97/0xab
[<ffffffff8143a6e8>] wait_for_common+0xa3/0x110
[<ffffffff81046f02>] ? default_wake_function+0x0/0x14
[<ffffffff81039e92>] ? synchronize_sched_expedited_cpu_stop+0x0/0x10
[<ffffffff8143a80d>] wait_for_completion+0x1d/0x1f
[<ffffffff81091d90>] __stop_cpus+0xf5/0x112
[<ffffffff81091ded>] try_stop_cpus+0x40/0x59
[<ffffffff81039e92>] ? synchronize_sched_expedited_cpu_stop+0x0/0x10
[<ffffffff81040f50>] synchronize_sched_expedited+0x7e/0x98
[<ffffffff81040ed2>] ? synchronize_sched_expedited+0x0/0x98
[<ffffffff8106b707>] __synchronize_srcu+0x31/0x70
[<ffffffff8106b75b>] synchronize_srcu_expedited+0x15/0x17
[<ffffffffa0206b5c>] kvm_vm_ioctl_get_dirty_log+0x104/0x19b [kvm]
[<ffffffff8120c3e4>] ? rb_insert_color+0x68/0xe5
[<ffffffffa01f6e0b>] kvm_vm_ioctl+0x239/0x353 [kvm]
[<ffffffff8143c8fc>] ? _raw_spin_unlock_irqrestore+0x37/0x42
[<ffffffff8106affc>] ? __hrtimer_start_range_ns+0x329/0x33b
[<ffffffff8143c6ae>] ? _raw_spin_lock_irqsave+0x2a/0x46
[<ffffffff81117563>] do_vfs_ioctl+0x46a/0x4b9
[<ffffffff8143c8fc>] ? _raw_spin_unlock_irqrestore+0x37/0x42
[<ffffffff81117608>] sys_ioctl+0x56/0x79
[<ffffffff8143d48a>] ? do_device_not_available+0xe/0x10
[<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b


 cpu#0, 2660.055 MHz

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime
sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------
 R    kworker/0:0     4   3424836.020475 183681301   120 3424836.020475   3423901.420588   2685682.072677
     migration/0     6       124.991395    297339     0       124.991395 0.001706         0.000000

Full log attached. 



[-- Attachment #2: nehalem-dmesg.txt --]
[-- Type: text/plain, Size: 123799 bytes --]

 [<ffffffff810e1928>] ? handle_mm_fault+0x8f4/0x921
 [<ffffffff8128a1c7>] tty_ioctl+0x914/0x97c
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 console-kit-d S ffff880077111700     0  1105      1 0x00000080
 ffff880078295c88 0000000000000082 0000000000000000 000280da003fffff
 ffff880078295fd8 ffff880077111700 ffff880078295fd8 ffff880077111a80
 ffff880077111a78 ffff880078294000 ffff880078294000 0000000000013580
 Call Trace:
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff81291f99>] vt_event_wait+0xc7/0x12d
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81439581>] ? schedule+0x6aa/0x6c4
 [<ffffffff81292030>] vt_waitactive+0x31/0x4e
 [<ffffffff81293523>] vt_ioctl+0x11b0/0x18c3
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff810e9492>] ? page_add_new_anon_rmap+0x60/0x72
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810e1928>] ? handle_mm_fault+0x8f4/0x921
 [<ffffffff8128a1c7>] tty_ioctl+0x914/0x97c
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff8120f6a7>] ? call_rwsem_wake+0x17/0x30
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 console-kit-d S ffff880077112e00     0  1106      1 0x00000080
 ffff880078297c88 0000000000000082 0000000000002937 000280da77112e38
 ffff880078297fd8 ffff880077112e00 ffff880078297fd8 ffff880077113180
 ffff880077113178 ffff880078296000 ffff880078296000 0000000000013580
 Call Trace:
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff81291f99>] vt_event_wait+0xc7/0x12d
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81439581>] ? schedule+0x6aa/0x6c4
 [<ffffffff81292030>] vt_waitactive+0x31/0x4e
 [<ffffffff81293523>] vt_ioctl+0x11b0/0x18c3
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff810e9492>] ? page_add_new_anon_rmap+0x60/0x72
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810e1928>] ? handle_mm_fault+0x8f4/0x921
 [<ffffffff8128a1c7>] tty_ioctl+0x914/0x97c
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 console-kit-d S ffff880077114500     0  1107      1 0x00000080
 ffff880077895c88 0000000000000082 0000000000000000 000280da003fffff
 ffff880077895fd8 ffff880077114500 ffff880077895fd8 ffff880077114880
 ffff880077114878 ffff880077894000 ffff880077894000 0000000000013580
 Call Trace:
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff81291f99>] vt_event_wait+0xc7/0x12d
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81439581>] ? schedule+0x6aa/0x6c4
 [<ffffffff81292030>] vt_waitactive+0x31/0x4e
 [<ffffffff81293523>] vt_ioctl+0x11b0/0x18c3
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff810e9492>] ? page_add_new_anon_rmap+0x60/0x72
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810e1928>] ? handle_mm_fault+0x8f4/0x921
 [<ffffffff8128a1c7>] tty_ioctl+0x914/0x97c
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 console-kit-d S ffff880077115c00     0  1108      1 0x00000080
 ffff880077897c88 0000000000000082 0000000000000000 000280da003fffff
 ffff880077897fd8 ffff880077115c00 ffff880077897fd8 ffff880077115f80
 ffff880077115f78 ffff880077896000 ffff880077896000 0000000000013580
 Call Trace:
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff81291f99>] vt_event_wait+0xc7/0x12d
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81292030>] vt_waitactive+0x31/0x4e
 [<ffffffff81293523>] vt_ioctl+0x11b0/0x18c3
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff810e9492>] ? page_add_new_anon_rmap+0x60/0x72
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810e1928>] ? handle_mm_fault+0x8f4/0x921
 [<ffffffff8128a1c7>] tty_ioctl+0x914/0x97c
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 console-kit-d S ffff880078291ce8     0  1110      1 0x00000080
 ffff880078291cd8 0000000000000082 ffff880078291c78 ffffffff81036a1f
 ffff880078291fd8 ffff880077954500 ffff880078291fd8 ffff880077954880
 ffff880077954878 ffff880078290000 ffff880078290000 0000000000013580
 Call Trace:
 [<ffffffff81036a1f>] ? native_flush_tlb_others+0xd6/0xe5
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff81110649>] pipe_wait+0x63/0x7f
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81110dee>] pipe_read+0x336/0x3b7
 [<ffffffff810e1905>] ? handle_mm_fault+0x8d1/0x921
 [<ffffffff81108aa4>] do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff81109c96>] ? fget_light+0xa5/0xb0
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 hald-runner   S 0000000000000000     0  1046   1042 0x00000080
 ffff880078e33a58 0000000000000082 ffff88007f413580 ffff880000000000
 ffff880078e33fd8 ffff880078ad0000 ffff880078e33fd8 ffff880078ad0380
 ffff880078ad0378 ffff880078e32000 ffff880078e32000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81380b38>] ? sock_aio_write+0x0/0xd4
 [<ffffffff81380bf8>] ? sock_aio_write+0xc0/0xd4
 [<ffffffff81108892>] ? do_sync_readv_writev+0xc1/0x100
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81108776>] ? fsnotify_modify+0x66/0x6e
 [<ffffffff81109646>] ? do_readv_writev+0x110/0x129
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 hald-addon-in S 0000000000000000     0  1138   1046 0x00000080
 ffff88007810fa58 0000000000000082 00ff880000000000 ffff880000000000
 ffff88007810ffd8 ffff880078354500 ffff88007810ffd8 ffff880078354880
 ffff880078354878 ffff88007810e000 ffff88007810e000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff81387619>] ? __alloc_skb+0x83/0x141
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81108892>] ? do_sync_readv_writev+0xc1/0x100
 [<ffffffff811078d1>] ? nameidata_to_filp+0x3f/0x50
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81109646>] ? do_readv_writev+0x110/0x129
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 hald-addon-st S 0000000000000000     0  1155   1046 0x00000080
 ffff8800788b1a58 0000000000000086 ffff8800788b19b8 ffffffff8143ae48
 ffff8800788b1fd8 ffff880077951700 ffff8800788b1fd8 ffff880077951a80
 ffff880077951a78 ffff8800788b0000 ffff8800788b0000 0000000000013580
 Call Trace:
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff812cd878>] ? scsi_execute+0x137/0x148
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff812cd92e>] ? scsi_execute_req+0xa5/0xb7
 [<ffffffff812c7bc1>] ? ioctl_internal_command.clone.0+0x6b/0x163
 [<ffffffff812da25d>] ? sr_drive_status+0x5f/0xa8
 [<ffffffff812c7d05>] ? scsi_set_medium_removal+0x4c/0x71
 [<ffffffff81207bba>] ? kobject_put+0x47/0x4b
 [<ffffffff812b40a8>] ? put_device+0x17/0x19
 [<ffffffff812c5ddc>] ? scsi_device_put+0x42/0x47
 [<ffffffff8111c33f>] ? iput+0x35/0x249
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff810a1952>] ? rcu_start_gp+0x20f/0x218
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 hald-addon-ac S 0000000000000000     0  1156   1046 0x00000080
 ffff880077a3fb08 0000000000000086 ffff880077a3fa78 ffffea0000000000
 ffff880077a3ffd8 ffff880077955c00 ffff880077a3ffd8 ffff880077955f80
 ffff880077955f78 ffff880077a3e000 ffff880077a3e000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8106706b>] ? prepare_to_wait+0x76/0x81
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff8140fbc1>] unix_stream_recvmsg+0x2f1/0x56b
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] sock_aio_read+0xc7/0xdb
 [<ffffffff81108aa4>] do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff811090f4>] vfs_read+0xbe/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 NetworkManage S ffff88007f44f1c0     0  1170      1 0x00000080
 ffff88007813ba58 0000000000000082 ffff88007813b9b8 ffffffffffffffff
 ffff88007813bfd8 ffff880078352e00 ffff88007813bfd8 ffff880078353180
 ffff880078353178 ffff88007813a000 ffff88007813a000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8137ec1c>] ? __sys_recvmsg+0x184/0x223
 [<ffffffff8137efed>] ? fput_light+0x12/0x14
 [<ffffffff8137f53c>] ? sys_sendto+0xef/0x105
 [<ffffffff81109c96>] ? fget_light+0xa5/0xb0
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 NetworkManage S 0000000000000000     0  1282      1 0x00000080
 ffff8800778c7cd8 0000000000000082 ffff8800778c7c78 ffff880000000000
 ffff8800778c7fd8 ffff880077b1ae00 ffff8800778c7fd8 ffff880077b1b180
 ffff880077b1b178 ffff8800778c6000 ffff8800778c6000 0000000000013580
 Call Trace:
 [<ffffffff81110649>] pipe_wait+0x63/0x7f
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81110dee>] pipe_read+0x336/0x3b7
 [<ffffffff81108aa4>] do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff81109c96>] ? fget_light+0xa5/0xb0
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 modem-manager S 0000000000000000     0  1177      1 0x00000080
 ffff8800795afa58 0000000000000082 ffff88007f77c700 ffff880000000000
 ffff8800795affd8 ffff880078a22e00 ffff8800795affd8 ffff880078a23180
 ffff880078a23178 ffff8800795ae000 ffff8800795ae000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8110cc00>] ? vfs_fstatat+0x37/0x62
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sshd          S 0000000000000000     0  1187      1 0x00000080
 ffff880078047928 0000000000000086 ffff88007c5e8f08 0000000000000000
 ffff880078047fd8 ffff880078351700 ffff880078047fd8 ffff880078351a80
 ffff880078351a78 ffff880078046000 ffff880078046000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810ca021>] ? zone_watermark_ok+0x29/0xba
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81205cfd>] ? cpumask_any_but+0x29/0x36
 [<ffffffff81036b24>] ? flush_tlb_page+0x73/0xb2
 [<ffffffff81035a49>] ? ptep_set_access_flags+0x44/0x52
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810dfc45>] ? do_wp_page+0x63b/0x6e4
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81118900>] ? d_free+0x37/0x4d
 [<ffffffff81118a68>] ? d_kill+0x60/0x68
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 wpa_supplican S 0000000000000000     0  1221      1 0x00000080
 ffff880078af3928 0000000000000086 ffff880078af3888 ffffffff00000000
 ffff880078af3fd8 ffff880078a25c00 ffff880078af3fd8 ffff880078a25f80
 ffff880078a25f78 ffff880078af2000 ffff880078af2000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8140e482>] ? scm_recv+0x3f/0xca
 [<ffffffff81385b0d>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8140fe1b>] ? unix_stream_recvmsg+0x54b/0x56b
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff8110116c>] ? unlock_page_cgroup+0x2a/0x35
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff810c5885>] ? unlock_page+0x17/0x19
 [<ffffffff810df32a>] ? __do_fault+0x414/0x44a
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 abrtd         S 0000000000000000     0  1222      1 0x00000080
 ffff880078ac9a58 0000000000000086 ffff880078ac99b8 ffffffff00000000
 ffff880078ac9fd8 ffff8800781fae00 ffff880078ac9fd8 ffff8800781fb180
 ffff8800781fb178 ffff880078ac8000 ffff880078ac8000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111b8d2>] ? touch_atime+0x87/0x12a
 [<ffffffff810c5202>] ? file_accessed+0x1c/0x1e
 [<ffffffff810c6fcc>] ? generic_file_aio_read+0x535/0x58d
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff810a1952>] ? rcu_start_gp+0x20f/0x218
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gpm           S 0000000000000000     0  1241      1 0x00000080
 ffff880077267928 0000000000000082 ffff880077267888 ffffffff00000000
 ffff880077267fd8 ffff8800781f9700 ffff880077267fd8 ffff8800781f9a80
 ffff8800781f9a78 ffff880077266000 ffff880077266000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8110174d>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff81103e3b>] ? __mem_cgroup_try_charge+0x13d/0x456
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff810c5ecd>] ? filemap_fault+0xb4/0x31c
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff810c5885>] ? unlock_page+0x17/0x19
 [<ffffffff810df32a>] ? __do_fault+0x414/0x44a
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8143b30f>] ? tty_unlock+0x43/0x45
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 ksmtuned      S 0000000000000000     0  1251      1 0x00000080
 ffff880077ba3e58 0000000000000086 ffffffff8104538d ffff880000000000
 ffff880077ba3fd8 ffff880078cd8000 ffff880077ba3fd8 ffff880078cd8380
 ffff880078cd8378 ffff880077ba2000 ffff880077ba2000 0000000000013580
 Call Trace:
 [<ffffffff8104538d>] ? enqueue_task_fair+0x2e6/0x339
 [<ffffffff8104f221>] do_wait+0x1d9/0x24d
 [<ffffffff8104f333>] sys_wait4+0x9e/0xc1
 [<ffffffff8104d90d>] ? child_wait_callback+0x0/0x58
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 crond         S ffff88007f50f1c0     0  1260      1 0x00000080
 ffff880079b65e68 0000000000000086 ffff880079b65dc8 ffffffffffffffff
 ffff880079b65fd8 ffff88007a301700 ffff880079b65fd8 ffff88007a301a80
 ffff88007a301a78 ffff880079b64000 ffff880079b64000 0000000000013580
 Call Trace:
 [<ffffffff81439ff5>] do_nanosleep+0x75/0xb7
 [<ffffffff8106ac5c>] hrtimer_nanosleep+0xb0/0x11d
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff8106ad22>] sys_nanosleep+0x59/0x69
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 atd           S 0000000000000000     0  1271      1 0x00000080
 ffff880077b0fe68 0000000000000082 ffff880077b0fdc8 ffffffff00000000
 ffff880077b0ffd8 ffff880078df4500 ffff880077b0ffd8 ffff880078df4880
 ffff880078df4878 ffff880077b0e000 ffff880077b0e000 0000000000013580
 Call Trace:
 [<ffffffff81439ff5>] do_nanosleep+0x75/0xb7
 [<ffffffff8106ac5c>] hrtimer_nanosleep+0xb0/0x11d
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff8106ad22>] sys_nanosleep+0x59/0x69
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 dhclient      S 0000000000000000     0  1281   1170 0x00000080
 ffff88007723b928 0000000000000082 ffff88007723b888 ffffffff00000000
 ffff88007723bfd8 ffff8800778b9700 ffff88007723bfd8 ffff8800778b9a80
 ffff8800778b9a78 ffff88007723a000 ffff88007723a000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c53ba>] ? find_get_pages_tag+0xbf/0xeb
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce9b>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff81205cfd>] ? cpumask_any_but+0x29/0x36
 [<ffffffff81036b24>] ? flush_tlb_page+0x73/0xb2
 [<ffffffff81035a49>] ? ptep_set_access_flags+0x44/0x52
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff810de880>] ? spin_unlock+0xe/0x10
 [<ffffffff810dfc45>] ? do_wp_page+0x63b/0x6e4
 [<ffffffff810cae09>] ? __pagevec_free+0x95/0xad
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S 0000000000000000     0  1285      1 0x00000080
 ffff880078a5da58 0000000000000086 ffff880078a5da68 0000000000000000
 ffff880078a5dfd8 ffff880077298000 ffff880078a5dfd8 ffff880077298380
 ffff880077298378 ffff880078a5c000 ffff880078a5c000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S 0000000000000000     0  1477      1 0x00000080
 ffff880079785c38 0000000000000086 20244c8d48fffffe 358d482800000000
 ffff880079785fd8 ffff880078cdc500 ffff880079785fd8 ffff880078cdc880
 ffff880078cdc878 ffff880079784000 ffff880079784000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81046dd0>] ? wake_up_state+0x10/0x12
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff811118a3>] ? path_put+0x22/0x26
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S 0000000000000000     0  1478      1 0x00000080
 ffff880079787c38 0000000000000086 0000000000000002 ffff880000000000
 ffff880079787fd8 ffff880078cdae00 ffff880079787fd8 ffff880078cdb180
 ffff880078cdb178 ffff880079786000 ffff880079786000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81046dd0>] ? wake_up_state+0x10/0x12
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff811118a3>] ? path_put+0x22/0x26
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S 0000000000000000     0  1479      1 0x00000080
 ffff880079759c38 0000000000000086 08344c834b002fc2 8349067400000000
 ffff880079759fd8 ffff880078355c00 ffff880079759fd8 ffff880078355f80
 ffff880078355f78 ffff880079758000 ffff880079758000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff811118a3>] ? path_put+0x22/0x26
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S 0000000000000000     0  1480      1 0x00000080
 ffff88007975bc38 0000000000000086 358d48002f855b3d 48ea894c00000000
 ffff88007975bfd8 ffff880072f79700 ffff88007975bfd8 ffff880072f79a80
 ffff880072f79a78 ffff88007975a000 ffff88007975a000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff811118a3>] ? path_put+0x22/0x26
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 libvirtd      S ffff880078e5aa00     0  1481      1 0x00000080
 ffff88007975dc38 0000000000000086 4808738948d86753 1874c08500000001
 ffff88007975dfd8 ffff880072f7c500 ffff88007975dfd8 ffff880072f7c880
 ffff880072f7c878 ffff88007975c000 ffff88007975c000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff811118a3>] ? path_put+0x22/0x26
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gdm-binary    S ffff88007f4cf1c0     0  1304      1 0x00000080
 ffff880072d3da58 0000000000000082 ffff880072d3d9b8 ffffffffffffffff
 ffff880072d3dfd8 ffff88007729ae00 ffff880072d3dfd8 ffff88007729b180
 ffff88007729b178 ffff880072d3c000 ffff880072d3c000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] ? sock_aio_read+0xc7/0xdb
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 mingetty      S 7fffffffffffffff     0  1305      1 0x00000080
 ffff880072d3fd08 0000000000000082 0000000000000000 ffffffffffffffff
 ffff880072d3ffd8 ffff88007729c500 ffff880072d3ffd8 ffff88007729c880
 ffff88007729c878 ffff880072d3e000 ffff880072d3e000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8104c46a>] ? release_console_sem+0x197/0x1c7
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 mingetty      S 7fffffffffffffff     0  1306      1 0x00000080
 ffff880078a71d08 0000000000000082 0000000000000000 ffffffffffffffff
 ffff880078a71fd8 ffff880077299700 ffff880078a71fd8 ffff880077299a80
 ffff880077299a78 ffff880078a70000 ffff880078a70000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8104c46a>] ? release_console_sem+0x197/0x1c7
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 mingetty      S 7fffffffffffffff     0  1307      1 0x00000080
 ffff880078a73d08 0000000000000086 ffff880078a73c78 ffffffff8103c26a
 ffff880078a73fd8 ffff88007a304500 ffff880078a73fd8 ffff88007a304880
 ffff88007a304878 ffff880078a72000 ffff880078a72000 0000000000013580
 Call Trace:
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8104c46a>] ? release_console_sem+0x197/0x1c7
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 mingetty      S 7fffffffffffffff     0  1308      1 0x00000080
 ffff880078a75d08 0000000000000086 ffff880078a75c78 ffffffff8103c21f
 ffff880078a75fd8 ffff880077b1c500 ffff880078a75fd8 ffff880077b1c880
 ffff880077b1c878 ffff880078a74000 ffff880078a74000 0000000000013580
 Call Trace:
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8104c46a>] ? release_console_sem+0x197/0x1c7
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 mingetty      S 7fffffffffffffff     0  1309      1 0x00000080
 ffff880078a77d08 0000000000000086 ffff880078a77c78 ffffffff8103c26a
 ffff880078a77fd8 ffff8800781fdc00 ffff880078a77fd8 ffff8800781fdf80
 ffff8800781fdf78 ffff880078a76000 ffff880078a76000 0000000000013580
 Call Trace:
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8104c46a>] ? release_console_sem+0x197/0x1c7
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gdm-simple-sl S ffff88007f44f1c0     0  1352   1304 0x00000080
 ffff8800789ada58 0000000000000082 ffff88007f553580 ffff8800ffffffff
 ffff8800789adfd8 ffff880079365c00 ffff8800789adfd8 ffff880079365f80
 ffff880079365f78 ffff8800789ac000 ffff8800789ac000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81108892>] ? do_sync_readv_writev+0xc1/0x100
 [<ffffffff810ce5c0>] ? release_pages+0x179/0x18b
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81109646>] ? do_readv_writev+0x110/0x129
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff810e519c>] ? remove_vma+0x69/0x71
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 Xorg          S ffff88007f5cf1c0     0  1353   1352 0x00400080
 ffff880079a8b928 0000000000000082 ffff88007f4d35f0 ffff8800ffffffff
 ffff880079a8bfd8 ffff880078cd9700 ffff880079a8bfd8 ffff880078cd9a80
 ffff880078cd9a78 ffff880079a8a000 ffff880079a8a000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81117105>] ? zero_fd_set+0x1a/0x1e
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff81108776>] ? fsnotify_modify+0x66/0x6e
 [<ffffffff81069d14>] ? __remove_hrtimer+0x7d/0x8a
 [<ffffffff81069d76>] ? remove_hrtimer+0x55/0x61
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 dnsmasq       S 0000000000000000     0  1368      1 0x00000080
 ffff880072f53928 0000000000000086 ffff88005e040780 ffffffff00000000
 ffff880072f53fd8 ffff880079688000 ffff880072f53fd8 ffff880079688380
 ffff880079688378 ffff880072f52000 ffff880072f52000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff811117d9>] ? path_to_nameidata+0x1b/0x3e
 [<ffffffff81112607>] ? link_path_walk+0x55e/0x563
 [<ffffffff81111744>] ? mntput+0x1d/0x1f
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff8117d1dd>] ? spin_unlock+0xe/0x10
 [<ffffffff8117d94a>] ? ext4_getattr+0x49/0x6b
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff8110ca1a>] ? cp_new_stat+0xe3/0xef
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sendmail      S 0000000000000000     0  1423      1 0x00000080
 ffff880072da7928 0000000000000086 ffff880072da7888 ffffffff00000000
 ffff880072da7fd8 ffff880072f78000 ffff880072da7fd8 ffff880072f78380
 ffff880072f78378 ffff880072da6000 ffff880072da6000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81153593>] ? proc_alloc_inode+0x20/0x93
 [<ffffffff8143b40e>] ? common_interrupt+0xe/0x13
 [<ffffffff8143b40e>] ? common_interrupt+0xe/0x13
 [<ffffffff81122757>] ? seq_open+0x3e/0xb1
 [<ffffffff8112274a>] ? seq_open+0x31/0xb1
 [<ffffffff8120c24f>] ? put_dec+0x20/0x3c
 [<ffffffff8120c3b3>] ? number+0x148/0x248
 [<ffffffff8112280a>] ? __seq_open_private+0x40/0x60
 [<ffffffff81118948>] ? spin_unlock+0xe/0x10
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff811537ed>] ? spin_unlock+0xe/0x10
 [<ffffffff81153fac>] ? proc_reg_open+0x1b9/0x1cd
 [<ffffffff8120ddd2>] ? vsnprintf+0x84/0x42a
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff810fbc63>] ? virt_to_head_page+0xe/0x31
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff810a1952>] ? rcu_start_gp+0x20f/0x218
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sendmail      S 0000000000000000     0  1425      1 0x00000080
 ffff880072f99f68 0000000000000082 ffff880072f7ae00 ffff880000000000
 ffff880072f99fd8 ffff880072f7ae00 ffff880072f99fd8 ffff880072f7b180
 ffff880072f7b178 ffff880072f98000 ffff880072f98000 0000000000013580
 Call Trace:
 [<ffffffff8105a834>] sys_pause+0x1e/0x27
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 dbus-launch   S 0000000000000000     0  1443      1 0x00000080
 ffff880078361928 0000000000000082 000000000ed3c8e8 ffff880078df0038
 ffff880078361fd8 ffff880078df0000 ffff880078361fd8 ffff880078df0380
 ffff880078df0378 ffff880078360000 ffff880078360000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8140e482>] ? scm_recv+0x3f/0xca
 [<ffffffff81385b0d>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8140fe1b>] ? unix_stream_recvmsg+0x54b/0x56b
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 dbus-daemon   S 0000000000000000     0  1444      1 0x00000080
 ffff8800771e3a58 0000000000000082 ffff88007f453580 ffff88007f453580
 ffff8800771e3fd8 ffff880078bc9700 ffff8800771e3fd8 ffff880078bc9a80
 ffff880078bc9a78 ffff8800771e2000 ffff8800771e2000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gnome-session S ffff88007f44f1c0     0  1445   1352 0x00000080
 ffff880072f03a58 0000000000000082 ffff880072f039d8 ffffffffffffffff
 ffff880072f03fd8 ffff880072f7dc00 ffff880072f03fd8 ffff880072f7df80
 ffff880072f7df78 ffff880072f02000 ffff880072f02000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gnome-session S 0000000000000000     0  1469   1352 0x00000080
 ffff880079615cd8 0000000000000082 ffff880079615c78 ffffffff00000000
 ffff880079615fd8 ffff880078cddc00 ffff880079615fd8 ffff880078cddf80
 ffff880078cddf78 ffff880079614000 ffff880079614000 0000000000013580
 Call Trace:
 [<ffffffff81110649>] pipe_wait+0x63/0x7f
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81110dee>] pipe_read+0x336/0x3b7
 [<ffffffff810e1905>] ? handle_mm_fault+0x8d1/0x921
 [<ffffffff81108aa4>] do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff81109c96>] ? fget_light+0xa5/0xb0
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 devkit-power- S 0000000000000000     0  1448      1 0x00000080
 ffff8800789bfa58 0000000000000086 ffff8800789bf9b8 ffffffff81046dbe
 ffff8800789bffd8 ffff880078bcdc00 ffff8800789bffd8 ffff880078bcdf80
 ffff880078bcdf78 ffff8800789be000 ffff8800789be000 0000000000013580
 Call Trace:
 [<ffffffff81046dbe>] ? default_wake_function+0x12/0x14
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] ? sock_aio_read+0xc7/0xdb
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gconfd-2      S ffff880079206c80     0  1452      1 0x00000080
 ffff880072d91a58 0000000000000082 ffff880072d919b8 ffffffff8143ae48
 ffff880072d91fd8 ffff880078350000 ffff880072d91fd8 ffff880078350380
 ffff880078350378 ffff880072d90000 ffff880072d90000 0000000000013580
 Call Trace:
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gnome-setting S 0000000000000000     0  1470      1 0x00000080
 ffff880072ff7a58 0000000000000082 ffff880072ff79b8 ffffffff00000000
 ffff880072ff7fd8 ffff880078bcc500 ffff880072ff7fd8 ffff880078bcc880
 ffff880078bcc878 ffff880072ff6000 ffff880072ff6000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gnome-setting S 0000000000000000     0  1475      1 0x00000080
 ffff880079745cd8 0000000000000082 ffff880079745c78 ffffffff00000000
 ffff880079745fd8 ffff880077808000 ffff880079745fd8 ffff880077808380
 ffff880077808378 ffff880079744000 ffff880079744000 0000000000013580
 Call Trace:
 [<ffffffff81110649>] pipe_wait+0x63/0x7f
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff81110dee>] pipe_read+0x336/0x3b7
 [<ffffffff810e1905>] ? handle_mm_fault+0x8d1/0x921
 [<ffffffff81108aa4>] do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff81109c96>] ? fget_light+0xa5/0xb0
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gvfsd         S 0000000000000000     0  1483      1 0x00000080
 ffff880079727a58 0000000000000082 ffff88007f413580 ffff880000000000
 ffff880079727fd8 ffff8800778bae00 ffff880079727fd8 ffff8800778bb180
 ffff8800778bb178 ffff880079726000 ffff880079726000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81380b38>] ? sock_aio_write+0x0/0xd4
 [<ffffffff81380bf8>] ? sock_aio_write+0xc0/0xd4
 [<ffffffff81108892>] ? do_sync_readv_writev+0xc1/0x100
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81108776>] ? fsnotify_modify+0x66/0x6e
 [<ffffffff81109646>] ? do_readv_writev+0x110/0x129
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 metacity      S 0000000000000000     0  1484   1445 0x00000080
 ffff880072cb9a58 0000000000000082 ffff88007f453580 ffff88007f453580
 ffff880072cb9fd8 ffff8800781fc500 ffff880072cb9fd8 ffff8800781fc880
 ffff8800781fc878 ffff880072cb8000 ffff880072cb8000 0000000000013580
 Call Trace:
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 plymouth-log- S ffff88007a355080     0  1485   1445 0x00000080
 ffff88007977ba58 0000000000000082 ffff88007977b9b8 ffffffff8143ae48
 ffff88007977bfd8 ffff880077950000 ffff88007977bfd8 ffff880077950380
 ffff880077950378 ffff88007977a000 ffff88007977a000 0000000000013580
 Call Trace:
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] ? sock_aio_read+0xc7/0xdb
 [<ffffffff81111744>] ? mntput+0x1d/0x1f
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gnome-power-m S 0000000000000000     0  1486   1445 0x00000080
 ffff88007905ba58 0000000000000086 ffff88007905b9b8 ffffffff00000000
 ffff88007905bfd8 ffff880079689700 ffff88007905bfd8 ffff880079689a80
 ffff880079689a78 ffff88007905a000 ffff88007905a000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gdm-simple-gr S ffff88007f50f1c0     0  1487   1445 0x00000080
 ffff880072e11a58 0000000000000086 ffff880072e119b8 ffffffffffffffff
 ffff880072e11fd8 ffff880078a20000 ffff880072e11fd8 ffff880078a20380
 ffff880078a20378 ffff880072e10000 ffff880072e10000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 polkit-gnome- S 0000000000000000     0  1488   1445 0x00000080
 ffff880072e53a58 0000000000000086 ffff880072e53a68 0000000000000000
 ffff880072e53fd8 ffff880078bc8000 ffff880072e53fd8 ffff880078bc8380
 ffff880078bc8378 ffff880072e52000 ffff880072e52000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] ? sock_aio_read+0xc7/0xdb
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 polkitd       S 0000000000000000     0  1492      1 0x00000080
 ffff8800773dfa58 0000000000000082 ffff8800773df9b8 ffffffff00000000
 ffff8800773dffd8 ffff8800778bc500 ffff8800773dffd8 ffff8800778bc880
 ffff8800778bc878 ffff8800773de000 ffff8800773de000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff81382105>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8137d7bf>] ? __sock_recvmsg_nosec+0x6a/0x73
 [<ffffffff8137ea04>] ? __sock_recvmsg+0x4e/0x59
 [<ffffffff81380cd3>] ? sock_aio_read+0xc7/0xdb
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 notification- S ffff88007f50f1c0     0  1494      1 0x00000080
 ffff8800773eba58 0000000000000082 ffff8800773eb9b8 ffffffffffffffff
 ffff8800773ebfd8 ffff880079484500 ffff8800773ebfd8 ffff880079484880
 ffff880079484878 ffff8800773ea000 ffff8800773ea000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81108aa4>] ? do_sync_read+0xcb/0x108
 [<ffffffff811cff6e>] ? fsnotify_perm+0x4a/0x50
 [<ffffffff811cffd5>] ? security_file_permission+0x2e/0x33
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 pulseaudio    S ffff88007f50f1c0     0  1501      1 0x00000080
 ffff880069161a08 0000000000000086 ffff880069161a58 ffffffffffffffff
 ffff880069161fd8 ffff8800773b8000 ffff880069161fd8 ffff8800773b8380
 ffff8800773b8378 ffff880069160000 ffff880069160000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff810e5126>] ? tlb_finish_mmu+0x5b/0x68
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 pulseaudio    S 0000000000000000     0  1509      1 0x00000080
 ffff88006932fa08 0000000000000086 ffff88006932fa18 0000000000000000
 ffff88006932ffd8 ffff8800773e5c00 ffff88006932ffd8 ffff8800773e5f80
 ffff8800773e5f78 ffff88006932e000 ffff88006932e000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8103e24b>] ? select_task_rq_fair+0x658/0x65d
 [<ffffffff8103e2cb>] ? __enqueue_entity+0x7b/0x7d
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffff81046d91>] ? try_to_wake_up+0x23a/0x255
 [<ffffffff81046dd0>] ? wake_up_state+0x10/0x12
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffffa00ddaf5>] ? snd_card_file_remove+0x10d/0x11e [snd]
 [<ffffffff810773f9>] ? do_futex+0xba/0x860
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 pulseaudio    S 0000000000000000     0  1510      1 0x00000080
 ffff88006933ba08 0000000000000086 ffff88006933ba18 0000000000000000
 ffff88006933bfd8 ffff880069219700 ffff88006933bfd8 ffff880069219a80
 ffff880069219a78 ffff88006933a000 ffff88006933a000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8103e24b>] ? select_task_rq_fair+0x658/0x65d
 [<ffffffff8103e2cb>] ? __enqueue_entity+0x7b/0x7d
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffff81046d91>] ? try_to_wake_up+0x23a/0x255
 [<ffffffff81046dd0>] ? wake_up_state+0x10/0x12
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffff810773f9>] ? do_futex+0xba/0x860
 [<ffffffffa00ddaf5>] ? snd_card_file_remove+0x10d/0x11e [snd]
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 pulseaudio    S 0000000000000000     0  1511      1 0x00000080
 ffff88006933da08 0000000000000086 ffff88006933da18 0000000000000000
 ffff88006933dfd8 ffff88006921ae00 ffff88006933dfd8 ffff88006921b180
 ffff88006921b178 ffff88006933c000 ffff88006933c000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8103e24b>] ? select_task_rq_fair+0x658/0x65d
 [<ffffffff8103e2cb>] ? __enqueue_entity+0x7b/0x7d
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffff81046d91>] ? try_to_wake_up+0x23a/0x255
 [<ffffffff81046dd0>] ? wake_up_state+0x10/0x12
 [<ffffffff8143ae0d>] ? _raw_spin_unlock+0x23/0x2e
 [<ffffffff81075441>] ? spin_unlock+0xe/0x10
 [<ffffffff81075b90>] ? futex_wake+0xe3/0xf5
 [<ffffffff810773f9>] ? do_futex+0xba/0x860
 [<ffffffffa00ddaf5>] ? snd_card_file_remove+0x10d/0x11e [snd]
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 rtkit-daemon  S 0000000000000000     0  1503      1 0x00000080
 ffff8800691c3a58 0000000000000082 ffff88007f493580 ffff88007f493580
 ffff8800691c3fd8 ffff8800773bc500 ffff8800691c3fd8 ffff8800773bc880
 ffff8800773bc878 ffff8800691c2000 ffff8800691c2000 0000000000013580
 Call Trace:
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81117882>] ? __pollwait+0xc3/0xcc
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81380b38>] ? sock_aio_write+0x0/0xd4
 [<ffffffff81380bf8>] ? sock_aio_write+0xc0/0xd4
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81108892>] ? do_sync_readv_writev+0xc1/0x100
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81108776>] ? fsnotify_modify+0x66/0x6e
 [<ffffffff81109646>] ? do_readv_writev+0x110/0x129
 [<ffffffff811ce6f6>] ? cap_safe_nice+0x41/0x5e
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 rtkit-daemon  S 0000000000000000     0  1504      1 0x00000080
 ffff8800773b5a58 0000000000000082 ffff8800773b59b8 ffffffff00000000
 ffff8800773b5fd8 ffff8800773bdc00 ffff8800773b5fd8 ffff8800773bdf80
 ffff8800773bdf78 ffff8800773b4000 ffff8800773b4000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff81069e33>] ? enqueue_hrtimer+0xb1/0xc1
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8106ab6a>] ? __hrtimer_start_range_ns+0x310/0x322
 [<ffffffff81040984>] ? enqueue_pushable_task+0x6c/0x74
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c26a>] ? resched_task+0x3b/0x72
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff81045c9e>] ? task_woken_rt+0x23/0x72
 [<ffffffff810455a0>] ? ttwu_post_activation+0x92/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffff81046d91>] ? try_to_wake_up+0x23a/0x255
 [<ffffffff81046dbe>] ? default_wake_function+0x12/0x14
 [<ffffffff811178dd>] ? pollwake+0x52/0x54
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 rtkit-daemon  S 0000000000000000     0  1505      1 0x00000080
 ffff8800691e3a58 0000000000000082 ffff8800691e39b8 ffffffff00000000
 ffff8800691e3fd8 ffff8800773b9700 ffff8800691e3fd8 ffff8800773b9a80
 ffff8800773b9a78 ffff8800691e2000 ffff8800691e2000 0000000000013580
 Call Trace:
 [<ffffffff81439f24>] schedule_hrtimeout_range_clock+0xde/0x111
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff8137ef7f>] ? sock_sendmsg+0xa8/0xc1
 [<ffffffff810d06b1>] ? page_evictable+0x17/0x7a
 [<ffffffff810ce80a>] ? lru_cache_add_lru+0x41/0x43
 [<ffffffff810e9492>] ? page_add_new_anon_rmap+0x60/0x72
 [<ffffffff8143b40e>] ? common_interrupt+0xe/0x13
 [<ffffffff8143b40e>] ? common_interrupt+0xe/0x13
 [<ffffffff8113aee7>] ? eventfd_read+0x0/0x63
 [<ffffffff8143aac1>] ? _raw_spin_lock_irq+0x1/0x35
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff810101b6>] ? read_tsc+0x9/0x1b
 [<ffffffff8106f097>] ? timekeeping_get_ns+0x1b/0x3d
 [<ffffffff8106f8e3>] ? ktime_get_ts+0x85/0x8e
 [<ffffffff8111716f>] ? poll_select_set_timeout+0x66/0x81
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 gdm-session-w S 0000000000000000     0  1506   1352 0x00000080
 ffff880069203a58 0000000000000082 ffff8800692039b8 ffffffff81046dbe
 ffff880069203fd8 ffff8800773e4500 ffff880069203fd8 ffff8800773e4880
 ffff8800773e4878 ffff880069202000 ffff880069202000 0000000000013580
 Call Trace:
 [<ffffffff81046dbe>] ? default_wake_function+0x12/0x14
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff810c5ecd>] ? filemap_fault+0xb4/0x31c
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff810c5885>] ? unlock_page+0x17/0x19
 [<ffffffff810df32a>] ? __do_fault+0x414/0x44a
 [<ffffffff810e148b>] ? handle_mm_fault+0x457/0x921
 [<ffffffff8111beae>] ? alloc_inode+0x30/0x7b
 [<ffffffff81110e82>] ? kzalloc.clone.0+0x13/0x15
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff8105a86b>] ? spin_unlock_irq+0xe/0x10
 [<ffffffff8105af59>] ? do_sigaction+0x171/0x189
 [<ffffffff8111837f>] sys_poll+0x55/0xbf
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sshd          S ffff88007f44f1c0     0  1593   1187 0x00000080
 ffff880069065928 0000000000000086 ffff8800690658d8 ffffffffffffffff
 ffff880069065fd8 ffff88007729dc00 ffff880069065fd8 ffff88007729df80
 ffff88007729df78 ffff880069064000 ffff880069064000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8103c1ed>] ? test_tsk_need_resched+0x17/0x21
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffff81046d91>] ? try_to_wake_up+0x23a/0x255
 [<ffffffff81046dbe>] ? default_wake_function+0x12/0x14
 [<ffffffff81039df2>] ? __wake_up_common+0x4e/0x84
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff8103d062>] ? __wake_up+0x44/0x4d
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          R  running task        0  1596   1593 0x00000080
 ffff880069275d88 0000000000000014 00000000ffffffff ffff880069275c98
 ffff880069275d08 0000000000000002 ffff880069275c88 ffffffff8106b43e
 ffff880069275c98 0000000000000002 ffff880069275ca8 ffffffff8104bfac
 Call Trace:
 [<ffffffff8100c9e4>] ? dump_trace+0x307/0x337
 [<ffffffff8100d4e0>] show_trace_log_lvl+0x54/0x5d
 [<ffffffff8100c4b0>] show_stack_log_lvl+0x13e/0x14d
 [<ffffffff8100d48a>] show_stack+0x1c/0x1e
 [<ffffffff810421a4>] sched_show_task+0xb1/0xb6
 [<ffffffff81042205>] show_state_filter+0x5c/0xba
 [<ffffffff8129e6be>] sysrq_handle_showstate+0x10/0x12
 [<ffffffff8129ea96>] __handle_sysrq+0xbf/0x15c
 [<ffffffff8129eb33>] ? write_sysrq_trigger+0x0/0x3b
 [<ffffffff8129eb66>] write_sysrq_trigger+0x33/0x3b
 [<ffffffff81153c6e>] proc_reg_write+0x92/0xb1
 [<ffffffff81108ef6>] vfs_write+0xae/0x10a
 [<ffffffff81109012>] sys_write+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 udevd         S ffff8800771f5080     0  1631    170 0x00000080
 ffff880077383a08 0000000000000082 0000000000002e8c ffff8800793edc00
 ffff880077383fd8 ffff8800792c2e00 ffff880077383fd8 ffff8800792c3180
 ffff8800792c3178 ffff880077382000 ffff880077382000 0000000000013580
 Call Trace:
 [<ffffffff8103e2cb>] ? __enqueue_entity+0x7b/0x7d
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8140e320>] ? scm_destroy_cred+0x2e/0x3b
 [<ffffffff8140e343>] ? scm_destroy+0x16/0x2f
 [<ffffffff8140f4d8>] ? unix_dgram_sendmsg+0x3d3/0x418
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff8137ef7f>] ? sock_sendmsg+0xa8/0xc1
 [<ffffffff8137ed04>] ? move_addr_to_kernel+0x49/0x4e
 [<ffffffff81388652>] ? copy_from_user+0xe/0x10
 [<ffffffff81388a23>] ? verify_iovec+0x57/0x98
 [<ffffffff8137f78a>] ? sys_sendmsg+0x222/0x241
 [<ffffffff8137f00f>] ? sockfd_lookup_light+0x20/0x58
 [<ffffffff8137f53c>] ? sys_sendto+0xef/0x105
 [<ffffffff81069d76>] ? remove_hrtimer+0x55/0x61
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 kvm-irqfd-cle S 0000000000000000     0  1634      2 0x00000080
 ffff880065d61e60 0000000000000046 ffff88007f593580 ffff880000000000
 ffff880065d61fd8 ffff8800792c5c00 ffff880065d61fd8 ffff8800792c5f80
 ffff8800792c5f78 ffff880065d60000 ffff880065d60000 0000000000013580
 Call Trace:
 [<ffffffff81062484>] rescuer_thread+0x1a5/0x1aa
 [<ffffffff810622df>] ? rescuer_thread+0x0/0x1aa
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 screen        S 0000000000000000     0  1638      1 0x00000080
 ffff880069223928 0000000000000086 ffff8800692238d8 ffffffff812cc6e8
 ffff880069223fd8 ffff88006921c500 ffff880069223fd8 ffff88006921c880
 ffff88006921c878 ffff880069222000 ffff880069222000 0000000000013580
 Call Trace:
 [<ffffffff812cc6e8>] ? scsi_request_fn+0x3ab/0x469
 [<ffffffff8106224a>] ? flush_work+0x1f/0x39
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810c574c>] ? __lock_page+0x66/0x6d
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff8110116c>] ? unlock_page_cgroup+0x2a/0x35
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff810c5885>] ? unlock_page+0x17/0x19
 [<ffffffff810df32a>] ? __do_fault+0x414/0x44a
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  1639   1638 0x00000080
 ffff880065e47d08 0000000000000082 ffff880065e47c78 ffffffff00000000
 ffff880065e47fd8 ffff880069218000 ffff880065e47fd8 ffff880069218380
 ffff880069218378 ffff880065e46000 ffff880065e46000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8128970e>] ? tty_wakeup+0x5b/0x60
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 kworker/u:0   S 0000000000000000     0  1924      2 0x00000080
 ffff88001c6afe80 0000000000000046 ffff88001c6afe00 0000000000000000
 ffff88001c6affd8 ffff880079362e00 ffff88001c6affd8 ffff880079363180
 ffff880079363178 ffff88001c6ae000 ffff88001c6ae000 0000000000013580
 Call Trace:
 [<ffffffff81063162>] worker_thread+0x255/0x25a
 [<ffffffff81062f0d>] ? worker_thread+0x0/0x25a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 kworker/u:1   S ffff88007f44f1c0     0  2062      2 0x00000080
 ffff880018f7fe80 0000000000000046 ffff88007a0e64f8 ffff8800ffffffff
 ffff880018f7ffd8 ffff8800692a2e00 ffff880018f7ffd8 ffff8800692a3180
 ffff8800692a3178 ffff880018f7e000 ffff880018f7e000 0000000000013580
 Call Trace:
 [<ffffffff81063162>] worker_thread+0x255/0x25a
 [<ffffffff81062f0d>] ? worker_thread+0x0/0x25a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 sshd          S 0000000000000000     0  2144   1187 0x00000080
 ffff88001badf928 0000000000000086 ffff88001badf8c8 ffffffff00000000
 ffff88001badffd8 ffff8800778bdc00 ffff88001badffd8 ffff8800778bdf80
 ffff8800778bdf78 ffff88001bade000 ffff88001bade000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  2147   2144 0x00000080
 ffff8800225e7d08 0000000000000086 ffff8800225e7c68 0000000000000000
 ffff8800225e7fd8 ffff880079482e00 ffff8800225e7fd8 ffff880079483180
 ffff880079483178 ffff8800225e6000 ffff8800225e6000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8128970e>] ? tty_wakeup+0x5b/0x60
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 udevd         S ffff880078e5e580     0  2235    170 0x00000080
 ffff8800691bfa08 0000000000000082 0000003bb1a12475 ffff88007f453580
 ffff8800691bffd8 ffff88005bc6ae00 ffff8800691bffd8 ffff88005bc6b180
 ffff88005bc6b178 ffff8800691be000 ffff8800691be000 0000000000013580
 Call Trace:
 [<ffffffff81044bc4>] ? check_preempt_wakeup+0x111/0x18c
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff8103c21f>] ? check_preempt_curr+0x28/0x38
 [<ffffffff81045582>] ? ttwu_post_activation+0x74/0x10d
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81067158>] ? add_wait_queue+0x44/0x4c
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81118156>] do_sys_poll+0x35f/0x3ea
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8140e320>] ? scm_destroy_cred+0x2e/0x3b
 [<ffffffff8140e343>] ? scm_destroy+0x16/0x2f
 [<ffffffff8140f4d8>] ? unix_dgram_sendmsg+0x3d3/0x418
 [<ffffffff810c55f9>] ? find_get_page+0x7d/0x90
 [<ffffffff810c5ecd>] ? filemap_fault+0xb4/0x31c
 [<ffffffff810c5827>] ? wake_up_page+0x25/0x2a
 [<ffffffff810c5885>] ? unlock_page+0x17/0x19
 [<ffffffff810df32a>] ? __do_fault+0x414/0x44a
 [<ffffffff810e148b>] ? handle_mm_fault+0x457/0x921
 [<ffffffff8137f78a>] ? sys_sendmsg+0x222/0x241
 [<ffffffff8143e244>] ? do_page_fault+0x33d/0x377
 [<ffffffff81069d76>] ? remove_hrtimer+0x55/0x61
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff811182a1>] sys_ppoll+0xc0/0x149
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sshd          S 0000000000000000     0  2247   1187 0x00000080
 ffff880072f11928 0000000000000086 ffff88007974e2f0 ffff880000000000
 ffff880072f11fd8 ffff880079480000 ffff880072f11fd8 ffff880079480380
 ffff880079480378 ffff880072f10000 ffff880072f10000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  2249   2247 0x00000080
 ffff880022587d08 0000000000000086 ffff880022587c78 ffffffff00000000
 ffff880022587fd8 ffff880079481700 ffff880022587fd8 ffff880079481a80
 ffff880079481a78 ffff880022586000 ffff880022586000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8128970e>] ? tty_wakeup+0x5b/0x60
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sshd          S ffff88007f44f1c0     0  2466   1187 0x00000080
 ffff88005dc7d928 0000000000000086 ffff88005dc7d8d8 ffffffffffffffff
 ffff88005dc7dfd8 ffff880072c62e00 ffff88005dc7dfd8 ffff880072c63180
 ffff880072c63178 ffff88005dc7c000 ffff88005dc7c000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  2469   2466 0x00000080
 ffff88005e0ebe58 0000000000000086 ffff88005e0ebdb8 ffffffff00000000
 ffff88005e0ebfd8 ffff880078ad5c00 ffff88005e0ebfd8 ffff880078ad5f80
 ffff880078ad5f78 ffff88005e0ea000 ffff88005e0ea000 0000000000013580
 Call Trace:
 [<ffffffff8104f221>] do_wait+0x1d9/0x24d
 [<ffffffff8104f333>] sys_wait4+0x9e/0xc1
 [<ffffffff8104d90d>] ? child_wait_callback+0x0/0x58
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 sshd          S 0000000000000000     0  2502   1187 0x00000080
 ffff88005e147928 0000000000000082 ffff88005e1478d8 ffffffff00000000
 ffff88005e147fd8 ffff8800778b8000 ffff88005e147fd8 ffff8800778b8380
 ffff8800778b8378 ffff88005e146000 ffff88005e146000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S ffff880079361700     0  2505   2502 0x00000080
 ffff880065fa3e58 0000000000000086 ffff880065fa3db8 ffffffff812879fb
 ffff880065fa3fd8 ffff880079361700 ffff880065fa3fd8 ffff880079361a80
 ffff880079361a78 ffff880065fa2000 ffff880065fa2000 0000000000013580
 Call Trace:
 [<ffffffff812879fb>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8104f221>] do_wait+0x1d9/0x24d
 [<ffffffff8104f333>] sys_wait4+0x9e/0xc1
 [<ffffffff8104d90d>] ? child_wait_callback+0x0/0x58
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 kworker/0:2   S 0000000000000000     0  2607      2 0x00000080
 ffff880065c99e80 0000000000000046 ffff880065c99e00 0000000000000286
 ffff880065c99fd8 ffff8800692a1700 ffff880065c99fd8 ffff8800692a1a80
 ffff8800692a1a78 ffff880065c98000 ffff880065c98000 0000000000013580
 Call Trace:
 [<ffffffff81063162>] worker_thread+0x255/0x25a
 [<ffffffff81062f0d>] ? worker_thread+0x0/0x25a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 sshd          S 0000000000000000     0  2679   1187 0x00000080
 ffff880077395928 0000000000000086 ffff8800773958d8 ffffffff00000000
 ffff880077395fd8 ffff880014810000 ffff880077395fd8 ffff880014810380
 ffff880014810378 ffff880077394000 ffff880077394000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  2681   2679 0x00000080
 ffff88005a27fd08 0000000000000086 ffff88005a27fc78 ffffffff00000000
 ffff88005a27ffd8 ffff88005e1d1700 ffff88005a27ffd8 ffff88005e1d1a80
 ffff88005e1d1a78 ffff88005a27e000 ffff88005a27e000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8128970e>] ? tty_wakeup+0x5b/0x60
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 kworker/1:2   S ffff88007f44f1c0     0  2729      2 0x00000080
 ffff88005a327e80 0000000000000046 ffff88002f488564 ffff8800ffffffff
 ffff88005a327fd8 ffff88005e192e00 ffff88005a327fd8 ffff88005e193180
 ffff88005e193178 ffff88005a326000 ffff88005a326000 0000000000013580
 Call Trace:
 [<ffffffff81063162>] worker_thread+0x255/0x25a
 [<ffffffff81062f0d>] ? worker_thread+0x0/0x25a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 sshd          S 0000000000000000     0  2732   1187 0x00000080
 ffff88002241d928 0000000000000086 ffff88002241d8d8 ffffffff00000000
 ffff88002241dfd8 ffff88005e1d5c00 ffff88002241dfd8 ffff88005e1d5f80
 ffff88005e1d5f78 ffff88002241c000 ffff88002241c000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff813ce580>] ? tcp_sendmsg+0x786/0x879
 [<ffffffff813ea747>] ? inet_sendmsg+0x66/0x6f
 [<ffffffff8137e9aa>] ? __sock_sendmsg+0x6b/0x77
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8110899c>] ? do_sync_write+0xcb/0x108
 [<ffffffff811358f1>] ? fsnotify+0x211/0x230
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 bash          S 0000000000000000     0  2734   2732 0x00000080
 ffff88007a279d08 0000000000000086 ffff88007a279c78 0000000000000000
 ffff88007a279fd8 ffff88005e1d0000 ffff88007a279fd8 ffff88005e1d0380
 ffff88005e1d0378 ffff88007a278000 ffff88007a278000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8105894b>] ? del_timer_sync+0x19/0x25
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff8128caa3>] n_tty_read+0x4dc/0x823
 [<ffffffff8128970e>] ? tty_wakeup+0x5b/0x60
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81288014>] tty_read+0x86/0xca
 [<ffffffff811090e1>] vfs_read+0xab/0x107
 [<ffffffff811091fd>] sys_read+0x4a/0x6e
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 vim           S 0000000000000000     0  3201   2505 0x00000080
 ffff88005e143928 0000000000000086 0000000000011210 0000000000000000
 ffff88005e143fd8 ffff880072c60000 ffff88005e143fd8 ffff880072c60380
 ffff880072c60378 ffff88005e142000 ffff88005e142000 0000000000013580
 Call Trace:
 [<ffffffff81439e88>] schedule_hrtimeout_range_clock+0x42/0x111
 [<ffffffff810622ad>] ? flush_delayed_work+0x49/0x4e
 [<ffffffff81290136>] ? tty_flush_to_ldisc+0x15/0x17
 [<ffffffff8128bc3b>] ? input_available_p+0x17/0x54
 [<ffffffff81439f6a>] schedule_hrtimeout_range+0x13/0x15
 [<ffffffff811171d2>] poll_schedule_timeout+0x48/0x64
 [<ffffffff81117781>] do_select+0x4f9/0x537
 [<ffffffff811177bf>] ? __pollwait+0x0/0xcc
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff8111788b>] ? pollwake+0x0/0x54
 [<ffffffff810087eb>] ? __switch_to+0x215/0x227
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff81042868>] ? finish_task_switch+0x86/0xa5
 [<ffffffff8103cdc7>] ? rcu_read_unlock+0xe/0x10
 [<ffffffff8103ce24>] ? cpuacct_charge+0x5b/0x64
 [<ffffffff8103da13>] ? __dequeue_entity+0x33/0x38
 [<ffffffff810087eb>] ? __switch_to+0x215/0x227
 [<ffffffff8143ae7d>] ? _raw_spin_unlock_irq+0x2a/0x35
 [<ffffffff81117af5>] core_sys_select+0x185/0x21b
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff81066e75>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff81066f6d>] ? finish_wait+0x66/0x6e
 [<ffffffff8143aef7>] ? _raw_read_unlock+0x24/0x2f
 [<ffffffff811b6fc2>] ? jbd2_log_wait_commit+0x15c/0x183
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffff8143afee>] ? _raw_write_unlock+0x28/0x33
 [<ffffffff81117dcf>] sys_select+0x96/0xbe
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 flush-253:1   S ffff88007f44f1c0     0  3329      2 0x00000080
 ffff88005beb1dd0 0000000000000046 0000000000000000 ffff8800ffffffff
 ffff88005beb1fd8 ffff88005be1c500 ffff88005beb1fd8 ffff88005be1c880
 ffff88005be1c878 ffff88005beb0000 ffff88005beb0000 0000000000013580
 Call Trace:
 [<ffffffff814397f6>] schedule_timeout+0x9c/0xc0
 [<ffffffff810589d9>] ? process_timeout+0x0/0x10
 [<ffffffff811271ef>] bdi_writeback_thread+0x1bc/0x24a
 [<ffffffff81127033>] ? bdi_writeback_thread+0x0/0x24a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 flush-8:0     S 0000000000000000     0  3335      2 0x00000080
 ffff880079b59e50 0000000000000046 000000010058bd20 ffff880000000000
 ffff880079b59fd8 ffff88005be18000 ffff880079b59fd8 ffff88005be18380
 ffff88005be18378 ffff880079b58000 ffff880079b58000 0000000000013580
 Call Trace:
 [<ffffffff811271f6>] bdi_writeback_thread+0x1c3/0x24a
 [<ffffffff81127033>] ? bdi_writeback_thread+0x0/0x24a
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 sh            S 0000000000000000     0  3360   2469 0x00000080
 ffff88005a24fe58 0000000000000086 ffffffff8104538d ffff880000000000
 ffff88005a24ffd8 ffff880014814500 ffff88005a24ffd8 ffff880014814880
 ffff880014814878 ffff88005a24e000 ffff88005a24e000 0000000000013580
 Call Trace:
 [<ffffffff8104538d>] ? enqueue_task_fair+0x2e6/0x339
 [<ffffffff8104f221>] do_wait+0x1d9/0x24d
 [<ffffffff8104f333>] sys_wait4+0x9e/0xc1
 [<ffffffff8104d90d>] ? child_wait_callback+0x0/0x58
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 qemu-system-x D 7fffffffffffffff     0  3361   3360 0x00000080
 ffff8800224b9bb8 0000000000000086 ffff88005e190000 ffff880014815c00
 ffff8800224b9fd8 ffff880014815c00 ffff8800224b9fd8 ffff880014815f80
 ffff880014815f78 ffff8800224b8000 ffff8800224b8000 0000000000013580
 Call Trace:
 [<ffffffff81439781>] schedule_timeout+0x27/0xc0
 [<ffffffff8103c3d8>] ? need_resched+0x23/0x2d
 [<ffffffff81439581>] ? schedule+0x6aa/0x6c4
 [<ffffffff81438d40>] wait_for_common+0xa3/0x110
 [<ffffffff81046dac>] ? default_wake_function+0x0/0x14
 [<ffffffff81039f4c>] ? synchronize_sched_expedited_cpu_stop+0x0/0x10
 [<ffffffff81438e65>] wait_for_completion+0x1d/0x1f
 [<ffffffff810916fe>] __stop_cpus+0xf6/0x113
 [<ffffffff8109175b>] try_stop_cpus+0x40/0x59
 [<ffffffff81039f4c>] ? synchronize_sched_expedited_cpu_stop+0x0/0x10
 [<ffffffff81040e45>] synchronize_sched_expedited+0x7e/0x97
 [<ffffffff81040dc7>] ? synchronize_sched_expedited+0x0/0x97
 [<ffffffff8106b273>] __synchronize_srcu+0x31/0x70
 [<ffffffff8106b2c7>] synchronize_srcu_expedited+0x15/0x17
 [<ffffffffa03319d9>] kvm_vm_ioctl_get_dirty_log+0x104/0x19b [kvm]
 [<ffffffffa0321d2d>] kvm_vm_ioctl+0x239/0x353 [kvm]
 [<ffffffff8143ae48>] ? _raw_spin_unlock_irqrestore+0x30/0x3b
 [<ffffffff8106ab6a>] ? __hrtimer_start_range_ns+0x310/0x322
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff8106f94a>] ? ktime_get+0x5e/0x98
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff8143ba4a>] ? do_device_not_available+0xe/0x10
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 qemu-system-x S 0000000000000000     0  3370   3360 0x00000080
 ffff88007a011c38 0000000000000086 ffff88007a011bc8 ffffffff00000000
 ffff88007a011fd8 ffff88005e195c00 ffff88007a011fd8 ffff88005e195f80
 ffff88005e195f78 ffff88007a010000 ffff88007a010000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffffa034327c>] ? apic_update_ppr+0x32/0x7f [kvm]
 [<ffffffffa00ef20b>] ? __vmx_load_host_state+0x111/0x11c [kvm_intel]
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffffa032151c>] ? kvm_vcpu_ioctl+0x4aa/0x4ea [kvm]
 [<ffffffff81116523>] ? do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff810c4696>] ? fire_user_return_notifiers+0x3f/0x69
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 qemu-system-x S ffffffffa031fbf9     0  3371   3360 0x00000080
 ffff88007906fcc8 0000000000000086 0000000000000000 ffffffffffffffff
 ffff88007906ffd8 ffff88005e191700 ffff88007906ffd8 ffff88005e191a80
 ffff88005e191a78 ffff88007906e000 ffff88007906e000 0000000000013580
 Call Trace:
 [<ffffffffa031fd57>] kvm_vcpu_block+0x93/0xae [kvm]
 [<ffffffff81066dfb>] ? autoremove_wake_function+0x0/0x39
 [<ffffffffa00f1764>] ? vmx_handle_exit+0x232/0x264 [kvm_intel]
 [<ffffffffa032eeee>] kvm_arch_vcpu_ioctl_run+0xa0f/0xc71 [kvm]
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffffa00f344b>] ? vmx_vcpu_load+0x33/0x172 [kvm_intel]
 [<ffffffffa0321184>] kvm_vcpu_ioctl+0x112/0x4ea [kvm]
 [<ffffffff81116523>] do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 qemu-system-x R  running task        0  3372   3360 0x00000880
 ffff88006929bc78 ffffffff00000000 ffff880017b80000 ffff880017b80000
 0000000000000000 ffffffffff5fc380 ffff88006929bca8 ffffffffa03375f6
 ffff880017b80000 ffffffffff5fc380 000000000000000b 000000008000030e
 Call Trace:
 [<ffffffffa03375f6>] kvm_mmu_page_fault+0x4b/0x77 [kvm]
 [<ffffffffa00f13d7>] handle_exception+0x171/0x2cc [kvm_intel]
 [<ffffffffa034327c>] ? apic_update_ppr+0x32/0x7f [kvm]
 [<ffffffffa00f186c>] ? vmx_vcpu_run+0x96/0x1ac [kvm_intel]
 [<ffffffffa032ee09>] ? kvm_arch_vcpu_ioctl_run+0x92a/0xc71 [kvm]
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffffa00f344b>] ? vmx_vcpu_load+0x33/0x172 [kvm_intel]
 [<ffffffffa0321184>] ? kvm_vcpu_ioctl+0x112/0x4ea [kvm]
 [<ffffffff81116523>] ? do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff811165c8>] ? sys_ioctl+0x56/0x79
 [<ffffffff81009cf2>] ? system_call_fastpath+0x16/0x1b
 qemu-system-x S ffff88007f50f1c0     0  3373   3360 0x00000080
 ffff880077977c38 0000000000000086 ffff880077977bc8 ffffffffffffffff
 ffff880077977fd8 ffff88005e190000 ffff880077977fd8 ffff88005e190380
 ffff88005e190378 ffff880077976000 ffff880077976000 0000000000013580
 Call Trace:
 [<ffffffff81075681>] futex_wait_queue_me+0x87/0x9f
 [<ffffffff810757d1>] futex_wait+0x138/0x2c3
 [<ffffffffa034327c>] ? apic_update_ppr+0x32/0x7f [kvm]
 [<ffffffff8103ce8b>] ? task_rq_unlock+0x11/0x13
 [<ffffffffa00ef20b>] ? __vmx_load_host_state+0x111/0x11c [kvm_intel]
 [<ffffffff810773d9>] do_futex+0x9a/0x860
 [<ffffffffa032151c>] ? kvm_vcpu_ioctl+0x4aa/0x4ea [kvm]
 [<ffffffff81116523>] ? do_vfs_ioctl+0x46a/0x4b9
 [<ffffffff81077cc6>] sys_futex+0x127/0x141
 [<ffffffff810c4696>] ? fire_user_return_notifiers+0x3f/0x69
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 kvm-pit-wq    S 0000000000000000     0  3362      2 0x00000080
 ffff88005e0ede60 0000000000000046 ffff88007f5d3580 ffff880000000000
 ffff88005e0edfd8 ffff88005be19700 ffff88005e0edfd8 ffff88005be19a80
 ffff88005be19a78 ffff88005e0ec000 ffff88005e0ec000 0000000000013580
 Call Trace:
 [<ffffffff81062484>] rescuer_thread+0x1a5/0x1aa
 [<ffffffff810622df>] ? rescuer_thread+0x0/0x1aa
 [<ffffffff81066902>] kthread+0x82/0x8a
 [<ffffffff8100aae4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81066880>] ? kthread+0x0/0x8a
 [<ffffffff8100aae0>] ? kernel_thread_helper+0x0/0x10
 sleep         S 0000000000000000     0  3417   1251 0x00000080
 ffff88005bd49e68 0000000000000086 ffff88005bd49dc8 ffffffff00000000
 ffff88005bd49fd8 ffff88005be1dc00 ffff88005bd49fd8 ffff88005be1df80
 ffff88005be1df78 ffff88005bd48000 ffff88005bd48000 0000000000013580
 Call Trace:
 [<ffffffff81439ff5>] do_nanosleep+0x75/0xb7
 [<ffffffff8106ac5c>] hrtimer_nanosleep+0xb0/0x11d
 [<ffffffff81069eea>] ? hrtimer_wakeup+0x0/0x26
 [<ffffffff8106abaa>] ? hrtimer_start_range_ns+0x14/0x16
 [<ffffffff81098867>] ? audit_syscall_entry+0x11e/0x14a
 [<ffffffff8106ad22>] sys_nanosleep+0x59/0x69
 [<ffffffff81009cf2>] system_call_fastpath+0x16/0x1b
 Sched Debug Version: v0.09, 2.6.36+ #34
 now at 6127331.929107 msecs
  .jiffies                                 : 4300794627
  .sysctl_sched_latency                    : 24.000000
  .sysctl_sched_min_granularity            : 3.000000
  .sysctl_sched_wakeup_granularity         : 4.000000
  .sysctl_sched_child_runs_first           : 0
  .sysctl_sched_features                   : 31855
  .sysctl_sched_tunable_scaling            : 1 (logaritmic)

 cpu#0, 2660.055 MHz
  .nr_running                    : 2
  .load                          : 1024
  .nr_switches                   : 367585120
  .nr_load_updates               : 2823701
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794717
  .curr->pid                     : 4
  .clock                         : 6112404.277207
  .cpu_load[0]                   : 1024
  .cpu_load[1]                   : 842
  .cpu_load[2]                   : 643
  .cpu_load[3]                   : 531
  .cpu_load[4]                   : 478
  .yld_count                     : 96
  .sched_switch                  : 0
  .sched_count                   : 367593934
  .sched_goidle                  : 183487125
  .avg_idle                      : 14139
  .ttwu_count                    : 183828830
  .ttwu_local                    : 60288
  .bkl_count                     : 0

 cfs_rq[0]:
  .exec_clock                    : 3427951.406584
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 3424836.020475
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : 0.000000
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 0

 rt_rq[0]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------
 R    kworker/0:0     4   3424836.020475 183681301   120   3424836.020475   3423901.420588   2685682.072677
     migration/0     6       124.991395    297339     0       124.991395         0.001706         0.000000

 cpu#1, 2660.055 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 8237096
  .nr_load_updates               : 523891
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794619
  .curr->pid                     : 0
  .clock                         : 6112403.389317
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 224
  .sched_switch                  : 0
  .sched_count                   : 8257278
  .sched_goidle                  : 3723734
  .avg_idle                      : 915358
  .ttwu_count                    : 4202826
  .ttwu_local                    : 3355584
  .bkl_count                     : 0

 cfs_rq[1]:
  .exec_clock                    : 341879.002271
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 329830.097078
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -3095005.952663
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 3

 rt_rq[1]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------
 Rqemu-system-x86  3371    329830.097078     79054   120    329830.097078     15536.739524    262721.038000

 cpu#2, 2660.055 MHz
  .nr_running                    : 1
  .load                          : 1024
  .nr_switches                   : 1549643
  .nr_load_updates               : 653413
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794619
  .curr->pid                     : 1596
  .clock                         : 6112393.374517
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 219
  .sched_switch                  : 0
  .sched_count                   : 1773538
  .sched_goidle                  : 731639
  .avg_idle                      : 1000000
  .ttwu_count                    : 777483
  .ttwu_local                    : 188952
  .bkl_count                     : 0

 cfs_rq[2]:
  .exec_clock                    : 523765.802874
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 521576.144535
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -2903259.919699
  .nr_running                    : 1
  .load                          : 1024
  .nr_spread_over                : 0

 rt_rq[2]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------
 R           bash  1596    521565.344293       192   120    521565.344293        37.790398   5542690.671971

 cpu#3, 2660.055 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 2260658
  .nr_load_updates               : 464910
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794619
  .curr->pid                     : 0
  .clock                         : 6111076.698090
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 375
  .sched_switch                  : 0
  .sched_count                   : 2474753
  .sched_goidle                  : 1114816
  .avg_idle                      : 947021
  .ttwu_count                    : 1130888
  .ttwu_local                    : 526961
  .bkl_count                     : 0

 cfs_rq[3]:
  .exec_clock                    : 358477.223045
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 357104.014391
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -3067732.078847
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 0

 rt_rq[3]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------

 cpu#4, 2660.055 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 1059754
  .nr_load_updates               : 245783
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794621
  .curr->pid                     : 0
  .clock                         : 6112197.718863
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 866
  .sched_switch                  : 0
  .sched_count                   : 1073459
  .sched_goidle                  : 493357
  .avg_idle                      : 1000000
  .ttwu_count                    : 535445
  .ttwu_local                    : 192916
  .bkl_count                     : 0

 cfs_rq[4]:
  .exec_clock                    : 54383.952007
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 53882.247178
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -3370953.874914
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 2

 rt_rq[4]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------

 cpu#5, 2660.055 MHz
  .nr_running                    : 1
  .load                          : 1024
  .nr_switches                   : 5375905
  .nr_load_updates               : 927931
  .nr_uninterruptible            : 1
  .next_balance                  : 4300.794720
  .curr->pid                     : 3372
  .clock                         : 6112404.339340
  .cpu_load[0]                   : 1024
  .cpu_load[1]                   : 1024
  .cpu_load[2]                   : 1024
  .cpu_load[3]                   : 1024
  .cpu_load[4]                   : 1024
  .yld_count                     : 75
  .sched_switch                  : 0
  .sched_count                   : 5385091
  .sched_goidle                  : 2515995
  .avg_idle                      : 1000000
  .ttwu_count                    : 2810228
  .ttwu_local                    : 1909712
  .bkl_count                     : 0

 cfs_rq[5]:
  .exec_clock                    : 766998.263190
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 759364.561098
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -2665471.575489
  .nr_running                    : 1
  .load                          : 1024
  .nr_spread_over                : 5

 rt_rq[5]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------
 Rqemu-system-x86  3372    759364.561098     23286   120    759364.561098     49699.461327    228580.154535

 cpu#6, 2660.055 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 1221131
  .nr_load_updates               : 492591
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794619
  .curr->pid                     : 0
  .clock                         : 6111761.992260
  .cpu_load[0]                   : 1024
  .cpu_load[1]                   : 512
  .cpu_load[2]                   : 256
  .cpu_load[3]                   : 128
  .cpu_load[4]                   : 64
  .yld_count                     : 882
  .sched_switch                  : 0
  .sched_count                   : 1261732
  .sched_goidle                  : 592047
  .avg_idle                      : 1000000
  .ttwu_count                    : 615417
  .ttwu_local                    : 267776
  .bkl_count                     : 0

 cfs_rq[6]:
  .exec_clock                    : 125402.413319
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 138155.462923
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -3286680.702708
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 3

 rt_rq[6]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------

 cpu#7, 2660.055 MHz
  .nr_running                    : 0
  .load                          : 0
  .nr_switches                   : 2293330
  .nr_load_updates               : 547519
  .nr_uninterruptible            : 0
  .next_balance                  : 4300.794619
  .curr->pid                     : 0
  .clock                         : 6111627.759502
  .cpu_load[0]                   : 0
  .cpu_load[1]                   : 0
  .cpu_load[2]                   : 0
  .cpu_load[3]                   : 0
  .cpu_load[4]                   : 0
  .yld_count                     : 194
  .sched_switch                  : 0
  .sched_count                   : 2306912
  .sched_goidle                  : 1136351
  .avg_idle                      : 1000000
  .ttwu_count                    : 1147579
  .ttwu_local                    : 789619
  .bkl_count                     : 0

 cfs_rq[7]:
  .exec_clock                    : 45096.875353
  .MIN_vruntime                  : 0.000001
  .min_vruntime                  : 50922.066321
  .max_vruntime                  : 0.000001
  .spread                        : 0.000000
  .spread0                       : -3373914.113755
  .nr_running                    : 0
  .load                          : 0
  .nr_spread_over                : 2

 rt_rq[7]:
  .rt_nr_running                 : 0
  .rt_throttled                  : 0
  .rt_time                       : 0.000000
  .rt_runtime                    : 950.000000

 runnable tasks:
            task   PID         tree-key  switches  prio     exec-runtime         sum-exec        sum-sleep
 ----------------------------------------------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 18:26           ` [bisected] Clocksource tsc unstable git markus
  2010-10-27 19:36             ` Peter Zijlstra
@ 2010-10-29  8:18             ` Borislav Petkov
  2010-10-29  8:30               ` Markus Trippelsdorf
  1 sibling, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-10-29  8:18 UTC (permalink / raw)
  To: markus
  Cc: john stultz, Thomas Gleixner, Borislav Petkov, linux-kernel, hpa,
	Ingo Molnar, Andreas Herrmann, heiko.carstens, a.p.zijlstra, avi,
	mtosatti

On Wed, Oct 27, 2010 at 08:26:08PM +0200, markus@trippelsdorf.de wrote:
> During my search for a reliable testcase I found out that kvm guests also
> showed random hangs (that didn't happen in v2.6.36) here. These hangs last a
> few seconds each, during which the mouse is not movable at all.
> 
> So I ran git-bisect with this testcase and the result of the bisection is:
> 
> 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Date:   Wed Sep 22 13:53:15 2010 +0200
>

...

>
> Reverting the commit solves the kvm hang issue. (If this issue is
> related to my original tsc problem is of course open for debate, but I
> have a strong hunch it is.)

Well, what do you know, the temporary "slowdown" triggered on my setup
too. After waiting a minute maybe, the machine recovered and guess what
appeared in dmesg:

[44729.650859] Clocksource tsc unstable (delta = -8615263009 ns)
[44729.656229] Switching to clocksource hpet

I've reverted the changeset above and will run without it to check
whether it makes any difference.

-- 
Regards/Gruss,
    Boris.

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29  8:18             ` Borislav Petkov
@ 2010-10-29  8:30               ` Markus Trippelsdorf
  2010-10-29 10:27                 ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-29  8:30 UTC (permalink / raw)
  To: Borislav Petkov, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, heiko.carstens,
	a.p.zijlstra, avi, mtosatti

On Fri, Oct 29, 2010 at 10:18:31AM +0200, Borislav Petkov wrote:
> On Wed, Oct 27, 2010 at 08:26:08PM +0200, markus@trippelsdorf.de wrote:
> > During my search for a reliable testcase I found out that kvm guests also
> > showed random hangs (that didn't happen in v2.6.36) here. These hangs last a
> > few seconds each, during which the mouse is not movable at all.
> > 
> > So I ran git-bisect with this testcase and the result of the bisection is:
> > 
> > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Date:   Wed Sep 22 13:53:15 2010 +0200
> >
> 
> ...
> 
> >
> > Reverting the commit solves the kvm hang issue. (If this issue is
> > related to my original tsc problem is of course open for debate, but I
> > have a strong hunch it is.)
> 
> Well, what do you know, the temporary "slowdown" triggered on my setup
> too. After waiting a minute maybe, the machine recovered and guess what
> appeared in dmesg:
> 
> [44729.650859] Clocksource tsc unstable (delta = -8615263009 ns)
> [44729.656229] Switching to clocksource hpet
> 
> I've reverted the changeset above and will run without it to check
> whether it makes any difference.

After further testing I've found out that the tsc- and the kvm-issue are
unrelated. IOW I get these "slowdowns" even with 34f971f6f7988be4
reverted. 

I guess John was right when he wrote:
> From your dmesg, I'd guess the hpet is the watchdog clocksource, so I
> be on the lookout for hpet related changes.
>
> Maybe does reverting 995bd3bb5c78f3ff71339803c0b8337ed36d64fb hide the
> issue?  

Because with 995bd3bb5c78f reverted I never had a single "slowdown" here.
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29  8:30               ` Markus Trippelsdorf
@ 2010-10-29 10:27                 ` Borislav Petkov
  2010-10-29 11:34                   ` Markus Trippelsdorf
  2010-10-29 12:14                   ` Thomas Gleixner
  0 siblings, 2 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-10-29 10:27 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, john stultz, Thomas Gleixner, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Oct 29, 2010 at 04:30:13AM -0400, Markus Trippelsdorf wrote:
> > Well, what do you know, the temporary "slowdown" triggered on my setup
> > too. After waiting a minute maybe, the machine recovered and guess what
> > appeared in dmesg:
> > 
> > [44729.650859] Clocksource tsc unstable (delta = -8615263009 ns)
> > [44729.656229] Switching to clocksource hpet
> > 
> > I've reverted the changeset above and will run without it to check
> > whether it makes any difference.
> 
> After further testing I've found out that the tsc- and the kvm-issue are
> unrelated. IOW I get these "slowdowns" even with 34f971f6f7988be4
> reverted.

hmm, I'll run with 34f971f6f798 reverted for a while just to confirm
your observations...

> I guess John was right when he wrote:
> > From your dmesg, I'd guess the hpet is the watchdog clocksource, so I
> > be on the lookout for hpet related changes.
> >
> > Maybe does reverting 995bd3bb5c78f3ff71339803c0b8337ed36d64fb hide the
> > issue?  
> 
> Because with 995bd3bb5c78f reverted I never had a single "slowdown" here.

That's strange, when we tested this one everything seemed fine so I
guess this is one of those bugs which appear later, just as if out of
nowhere.

Thomas, could it be that SMI fires in between the HPET write and
subsequent read:

	hpet_writel(cnt, HPET_Tn_CMP(timer));

	...

	res = (s32)(cnt - hpet_readl(HPET_COUNTER));

causing the -ETIME and thus a wait for HPET wraparound? My machine at
least does SMI-initiated C1E so it could very well be the problem.

Markus, can you verify this on your system by compiling x86info from
git://git.choralone.org/git/x86info and doing

./lsmsr Int

as root?

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 10:27                 ` Borislav Petkov
@ 2010-10-29 11:34                   ` Markus Trippelsdorf
  2010-10-29 11:39                     ` Borislav Petkov
  2010-10-29 12:14                   ` Thomas Gleixner
  1 sibling, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-29 11:34 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Borislav Petkov, john stultz, Thomas Gleixner, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

[-- Attachment #1: Type: text/plain, Size: 4002 bytes --]

On Fri, Oct 29, 2010 at 12:27:13PM +0200, Borislav Petkov wrote:
> On Fri, Oct 29, 2010 at 04:30:13AM -0400, Markus Trippelsdorf wrote:
> > > Well, what do you know, the temporary "slowdown" triggered on my setup
> > > too. After waiting a minute maybe, the machine recovered and guess what
> > > appeared in dmesg:
> > > 
> > > [44729.650859] Clocksource tsc unstable (delta = -8615263009 ns)
> > > [44729.656229] Switching to clocksource hpet
> > > 
> > > I've reverted the changeset above and will run without it to check
> > > whether it makes any difference.
> > 
> > After further testing I've found out that the tsc- and the kvm-issue are
> > unrelated. IOW I get these "slowdowns" even with 34f971f6f7988be4
> > reverted.
> 
> hmm, I'll run with 34f971f6f798 reverted for a while just to confirm
> your observations...
> 
> > I guess John was right when he wrote:
> > > From your dmesg, I'd guess the hpet is the watchdog clocksource, so I
> > > be on the lookout for hpet related changes.
> > >
> > > Maybe does reverting 995bd3bb5c78f3ff71339803c0b8337ed36d64fb hide the
> > > issue?  
> > 
> > Because with 995bd3bb5c78f reverted I never had a single "slowdown" here.
> 
> That's strange, when we tested this one everything seemed fine so I
> guess this is one of those bugs which appear later, just as if out of
> nowhere.
> 
> Thomas, could it be that SMI fires in between the HPET write and
> subsequent read:
> 
> 	hpet_writel(cnt, HPET_Tn_CMP(timer));
> 
> 	...
> 
> 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
> 
> causing the -ETIME and thus a wait for HPET wraparound? My machine at
> least does SMI-initiated C1E so it could very well be the problem.
> 
> Markus, can you verify this on your system by compiling x86info from
> git://git.choralone.org/git/x86info and doing
> 
> ./lsmsr Int


x86info v1.28beta.  Dave Jones 2001-2010
Feedback to <davej@redhat.com>.

Found 4 CPUs
--------------------------------------------------------------------------
CPU #1
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
--------------------------------------------------------------------------
CPU #2
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
--------------------------------------------------------------------------
CPU #3
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
--------------------------------------------------------------------------
CPU #4
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
--------------------------------------------------------------------------
 running at an estimated 3.20GHz

x86info --all output is attached
-- 
Markus

[-- Attachment #2: x86info_all --]
[-- Type: text/plain, Size: 35072 bytes --]

x86info v1.28beta.  Dave Jones 2001-2010
Feedback to <davej@redhat.com>.

Found 4 CPUs
MP Table:
#	APIC ID	Version	State		Family	Model	Step	Flags
#	 0	 0x10	 BSP, usable	 16	 4	 2	 0x178bfbff
#	 1	 0x10	 AP, usable	 16	 4	 2	 0x178bfbff
#	 2	 0x10	 AP, usable	 16	 4	 2	 0x178bfbff
#	 3	 0x10	 AP, usable	 16	 4	 2	 0x178bfbff

--------------------------------------------------------------------------
CPU #1
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Number of reporting banks : 6

MCG_CTL:
 Data cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Data cache data parity enabled
  Data cache main tag parity enabled
  Data cache snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
 Instruction cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Instruction cache data parity enabled
  IC main tag parity enabled
  IC snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
  Predecode array parity enabled
  Target selector parity enabled
  Read data error enabled
 Bus unit check enabled
  External L2 tag parity error enabled
  L2 partial tag parity error enabled
  System ECC TLB reload error enabled
  L2 ECC TLB reload error enabled
  L2 ECC K7 deallocate enabled
  L2 ECC probe deallocate enabled
  System datareaderror reporting enabled
 Load/Store unit check enabled
  Read data error enable (loads) enabled
  Read data error enable (stores) enabled

           31       23       15       7 
Bank: 0 (0x400)
MC0CTL:    00000000 00000000 00000000 11111111
MC0STATUS: 00000000 00000000 00000000 00000000
MC0ADDR:   11111111 11111111 11111111 11111111
MC0MISC:   00000000 00000000 00000000 00000000

Bank: 1 (0x404)
MC1CTL:    11111111 11111111 11111111 11111111
MC1STATUS: 00000000 00000000 00000000 00000000
MC1ADDR:   11111111 11111111 11111111 11111111
MC1MISC:   00000000 00000000 00000000 00000000

Bank: 2 (0x408)
MC2CTL:    00000000 00000000 00001111 11111111
MC2STATUS: 00000000 00000000 00000000 00000000
MC2ADDR:   11111111 11111111 11111111 11111111
MC2MISC:   00000000 00000000 00000000 00000000

Bank: 3 (0x40c)
MC3CTL:    00000000 00000000 00000000 00000011
MC3STATUS: 00000000 00000000 00000000 00000000
MC3ADDR:   11111111 11111111 11111111 11111111
MC3MISC:   00000000 00000000 00000000 00000000

Bank: 4 (0x410)
MC4CTL:    00111111 11111111 11111111 11111111
MC4STATUS: 00000000 00000000 00000000 00000000
MC4ADDR:   11111111 11111111 11111111 11111111
MC4MISC:   11111111 11111111 11111111 11111111

Bank: 5 (0x414)
MC5CTL:    00000000 00000000 00000000 00000001
MC5STATUS: 00000000 00000000 00000000 00000000
MC5ADDR:   11111111 11111111 11111111 11111111
MC5MISC:   00000000 00000000 00001111 11111111

Microcode patch level: 0x1000086

PowerNOW! Technology information
Available features:
	Temperature sensing diode present.
	Thermal Trip
	Thermal Monitoring
	Software Thermal Control
	100MHz multiplier control
	Hardware P-state control
	invariant TSC

Pstate-0: fid=10, did=0, vid=10 (3200MHz) (current)
Pstate-1: fid=9, did=0, vid=18 (2500MHz)
Pstate-2: fid=5, did=0, vid=20 (2100MHz)
Pstate-3: fid=0, did=1, vid=2e (800MHz)

Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
eax in: 0x00000000, eax = 00000005 ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x00000001, eax = 00100f42 ebx = 00040800 ecx = 00802009 edx = 178bfbff
eax in: 0x00000002, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000005, eax = 00000040 ebx = 00000040 ecx = 00000003 edx = 00000000

eax in: 0x80000000, eax = 8000001b ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x80000001, eax = 00100f42 ebx = 10001b76 ecx = 000037ff edx = efd3fbff
eax in: 0x80000002, eax = 20444d41 ebx = 6e656850 ecx = 74286d6f edx = 4920296d
eax in: 0x80000003, eax = 34582049 ebx = 35353920 ecx = 6f725020 edx = 73736563
eax in: 0x80000004, eax = 0000726f ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000005, eax = ff30ff10 ebx = ff30ff20 ecx = 40020140 edx = 40020140
eax in: 0x80000006, eax = 20800000 ebx = 42004200 ecx = 02008140 edx = 0030b140
eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000001f9
eax in: 0x80000008, eax = 00003030 ebx = 00000000 ecx = 00002003 edx = 00000000
eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000a, eax = 00000001 ebx = 00000040 ecx = 00000000 edx = 0000000f
eax in: 0x8000000b, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000c, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000d, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000e, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000f, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000010, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000011, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000012, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000013, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000014, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000015, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000016, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000017, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000018, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000019, eax = f0300000 ebx = 60100000 ecx = 00000000 edx = 00000000
eax in: 0x8000001a, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000001b, eax = 0000001f ebx = 00000000 ecx = 00000000 edx = 00000000

L1 Data TLB (1G):           Fully associative. 48 entries.
L1 Instruction TLB (1G):    Disabled. 0 entries.
L1 Data TLB (2M/4M):        Fully associative. 48 entries.
L1 Instruction TLB (2M/4M): Fully associative. 16 entries.
L1 Data TLB (4K):           Fully associative. 48 entries.
L1 Instruction TLB (4K):    Fully associative. 32 entries.
L1 Data cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L1 Instruction cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L2 Data TLB (1G):           8-way associative. 16 entries.
L2 Instruction TLB (1G):    Disabled. 0 entries.
L2 Data TLB (2M/4M):        2-way associative. 128 entries.
L2 Instruction TLB (2M/4M): Disabled. 0 entries.
L2 Data TLB (4K):           4-way associative. 512 entries.
L2 Instruction TLB (4K):    4-way associative. 512 entries.
L2 cache:
	Size: 512Kb	16-way associative. 
	lines per tag=1	line size=64 bytes.
L3 (shared) cache:
	Size: 6144Kb	48-way associative. 
	lines per tag=1	line size=64 bytes.

Feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht sse3 mwait cmpxchg16b popcnt
Extended feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 nx mmxext mmx fxsr ffxsr page1gb rdtscp lm 3dnowext 3dnow lahf/sahf CmpLegacy svm ExtApicSpace LockMovCr0 abm sse4a misalignsse 3dnowPref osvw ibs skinit wdt

Long NOPs supported: yes
Connector type: Socket AM2r2

MTRR registers:
MTRRcap (0xfe): 0x0000000000000508
MTRRphysBase0 (0x200): 0x0000000000000006
MTRRphysMask0 (0x201): 0x0000ffff80000800
MTRRphysBase1 (0x202): 0x0000000080000006
MTRRphysMask1 (0x203): 0x0000ffffc0000800
MTRRphysBase2 (0x204): 0x00000000c0000006
MTRRphysMask2 (0x205): 0x0000ffffe0000800
MTRRphysBase3 (0x206): 0x00000000f0000001
MTRRphysMask3 (0x207): 0x0000fffff8000800
MTRRphysBase4 (0x208): 0x0000000000000000
MTRRphysMask4 (0x209): 0x0000000000000000
MTRRphysBase5 (0x20a): 0x0000000000000000
MTRRphysMask5 (0x20b): 0x0000000000000000
MTRRphysBase6 (0x20c): 0x0000000000000000
MTRRphysMask6 (0x20d): 0x0000000000000000
MTRRphysBase7 (0x20e): 0x0000000000000000
MTRRphysMask7 (0x20f): 0x0000000000000000
MTRRfix64K_00000 (0x250): 0x0606060606060606
MTRRfix16K_80000 (0x258): 0x0606060606060606
MTRRfix16K_A0000 (0x259): 0x0000000000000000
MTRRfix4K_C8000 (0x269): 0x0000000000000000
MTRRfix4K_D0000 0x26a: 0x0000000000000000
MTRRfix4K_D8000 0x26b: 0x0000000000000000
MTRRfix4K_E0000 0x26c: 0x0000000000000000
MTRRfix4K_E8000 0x26d: 0x0000000000000000
MTRRfix4K_F0000 0x26e: 0x0505050505050505
MTRRfix4K_F8000 0x26f: 0x0505050505050505
MTRRdefType (0x2ff): 0x0000000000000c00


3.20GHz processor (estimate).

--------------------------------------------------------------------------
CPU #2
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Number of reporting banks : 6

MCG_CTL:
 Data cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Data cache data parity enabled
  Data cache main tag parity enabled
  Data cache snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
 Instruction cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Instruction cache data parity enabled
  IC main tag parity enabled
  IC snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
  Predecode array parity enabled
  Target selector parity enabled
  Read data error enabled
 Bus unit check enabled
  External L2 tag parity error enabled
  L2 partial tag parity error enabled
  System ECC TLB reload error enabled
  L2 ECC TLB reload error enabled
  L2 ECC K7 deallocate enabled
  L2 ECC probe deallocate enabled
  System datareaderror reporting enabled
 Load/Store unit check enabled
  Read data error enable (loads) enabled
  Read data error enable (stores) enabled

           31       23       15       7 
Bank: 0 (0x400)
MC0CTL:    00000000 00000000 00000000 11111111
MC0STATUS: 00000000 00000000 00000000 00000000
MC0ADDR:   11111111 11111111 11111111 11111111
MC0MISC:   00000000 00000000 00000000 00000000

Bank: 1 (0x404)
MC1CTL:    11111111 11111111 11111111 11111111
MC1STATUS: 00000000 00000000 00000000 00000000
MC1ADDR:   11111111 11111111 11111111 11111111
MC1MISC:   00000000 00000000 00000000 00000000

Bank: 2 (0x408)
MC2CTL:    00000000 00000000 00001111 11111111
MC2STATUS: 00000000 00000000 00000000 00000000
MC2ADDR:   11111111 11111111 11111111 11111111
MC2MISC:   00000000 00000000 00000000 00000000

Bank: 3 (0x40c)
MC3CTL:    00000000 00000000 00000000 00000011
MC3STATUS: 00000000 00000000 00000000 00000000
MC3ADDR:   11111111 11111111 11111111 11111111
MC3MISC:   00000000 00000000 00000000 00000000

Bank: 4 (0x410)
MC4CTL:    00000000 00000000 00000000 00000000
MC4STATUS: 00000000 00000000 00000000 00000000
MC4ADDR:   00000000 00000000 00000000 00000000
MC4MISC:   11111111 11111111 11111111 11111111

Bank: 5 (0x414)
MC5CTL:    00000000 00000000 00000000 00000001
MC5STATUS: 00000000 00000000 00000000 00000000
MC5ADDR:   11111111 11111111 11111111 11111111
MC5MISC:   00000000 00000000 00001111 11111111

Microcode patch level: 0x1000086

PowerNOW! Technology information
Available features:
	Temperature sensing diode present.
	Thermal Trip
	Thermal Monitoring
	Software Thermal Control
	100MHz multiplier control
	Hardware P-state control
	invariant TSC

Pstate-0: fid=10, did=0, vid=10 (3200MHz)
Pstate-1: fid=9, did=0, vid=18 (2500MHz)
Pstate-2: fid=5, did=0, vid=20 (2100MHz)
Pstate-3: fid=0, did=1, vid=2e (800MHz) (current)

Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
eax in: 0x00000000, eax = 00000005 ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x00000001, eax = 00100f42 ebx = 01040800 ecx = 00802009 edx = 178bfbff
eax in: 0x00000002, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000005, eax = 00000040 ebx = 00000040 ecx = 00000003 edx = 00000000

eax in: 0x80000000, eax = 8000001b ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x80000001, eax = 00100f42 ebx = 10001b76 ecx = 000037ff edx = efd3fbff
eax in: 0x80000002, eax = 20444d41 ebx = 6e656850 ecx = 74286d6f edx = 4920296d
eax in: 0x80000003, eax = 34582049 ebx = 35353920 ecx = 6f725020 edx = 73736563
eax in: 0x80000004, eax = 0000726f ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000005, eax = ff30ff10 ebx = ff30ff20 ecx = 40020140 edx = 40020140
eax in: 0x80000006, eax = 20800000 ebx = 42004200 ecx = 02008140 edx = 0030b140
eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000001f9
eax in: 0x80000008, eax = 00003030 ebx = 00000000 ecx = 00002003 edx = 00000000
eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000a, eax = 00000001 ebx = 00000040 ecx = 00000000 edx = 0000000f
eax in: 0x8000000b, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000c, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000d, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000e, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000f, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000010, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000011, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000012, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000013, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000014, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000015, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000016, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000017, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000018, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000019, eax = f0300000 ebx = 60100000 ecx = 00000000 edx = 00000000
eax in: 0x8000001a, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000001b, eax = 0000001f ebx = 00000000 ecx = 00000000 edx = 00000000

L1 Data TLB (1G):           Fully associative. 48 entries.
L1 Instruction TLB (1G):    Disabled. 0 entries.
L1 Data TLB (2M/4M):        Fully associative. 48 entries.
L1 Instruction TLB (2M/4M): Fully associative. 16 entries.
L1 Data TLB (4K):           Fully associative. 48 entries.
L1 Instruction TLB (4K):    Fully associative. 32 entries.
L1 Data cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L1 Instruction cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L2 Data TLB (1G):           8-way associative. 16 entries.
L2 Instruction TLB (1G):    Disabled. 0 entries.
L2 Data TLB (2M/4M):        2-way associative. 128 entries.
L2 Instruction TLB (2M/4M): Disabled. 0 entries.
L2 Data TLB (4K):           4-way associative. 512 entries.
L2 Instruction TLB (4K):    4-way associative. 512 entries.
L2 cache:
	Size: 512Kb	16-way associative. 
	lines per tag=1	line size=64 bytes.
L3 (shared) cache:
	Size: 6144Kb	48-way associative. 
	lines per tag=1	line size=64 bytes.

Feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht sse3 mwait cmpxchg16b popcnt
Extended feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 nx mmxext mmx fxsr ffxsr page1gb rdtscp lm 3dnowext 3dnow lahf/sahf CmpLegacy svm ExtApicSpace LockMovCr0 abm sse4a misalignsse 3dnowPref osvw ibs skinit wdt

Long NOPs supported: yes
Connector type: Socket AM2r2

MTRR registers:
MTRRcap (0xfe): 0x0000000000000508
MTRRphysBase0 (0x200): 0x0000000000000006
MTRRphysMask0 (0x201): 0x0000ffff80000800
MTRRphysBase1 (0x202): 0x0000000080000006
MTRRphysMask1 (0x203): 0x0000ffffc0000800
MTRRphysBase2 (0x204): 0x00000000c0000006
MTRRphysMask2 (0x205): 0x0000ffffe0000800
MTRRphysBase3 (0x206): 0x00000000f0000001
MTRRphysMask3 (0x207): 0x0000fffff8000800
MTRRphysBase4 (0x208): 0x0000000000000000
MTRRphysMask4 (0x209): 0x0000000000000000
MTRRphysBase5 (0x20a): 0x0000000000000000
MTRRphysMask5 (0x20b): 0x0000000000000000
MTRRphysBase6 (0x20c): 0x0000000000000000
MTRRphysMask6 (0x20d): 0x0000000000000000
MTRRphysBase7 (0x20e): 0x0000000000000000
MTRRphysMask7 (0x20f): 0x0000000000000000
MTRRfix64K_00000 (0x250): 0x0606060606060606
MTRRfix16K_80000 (0x258): 0x0606060606060606
MTRRfix16K_A0000 (0x259): 0x0000000000000000
MTRRfix4K_C8000 (0x269): 0x0000000000000000
MTRRfix4K_D0000 0x26a: 0x0000000000000000
MTRRfix4K_D8000 0x26b: 0x0000000000000000
MTRRfix4K_E0000 0x26c: 0x0000000000000000
MTRRfix4K_E8000 0x26d: 0x0000000000000000
MTRRfix4K_F0000 0x26e: 0x0505050505050505
MTRRfix4K_F8000 0x26f: 0x0505050505050505
MTRRdefType (0x2ff): 0x0000000000000c00


3.20GHz processor (estimate).

--------------------------------------------------------------------------
CPU #3
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Number of reporting banks : 6

MCG_CTL:
 Data cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Data cache data parity enabled
  Data cache main tag parity enabled
  Data cache snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
 Instruction cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Instruction cache data parity enabled
  IC main tag parity enabled
  IC snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
  Predecode array parity enabled
  Target selector parity enabled
  Read data error enabled
 Bus unit check enabled
  External L2 tag parity error enabled
  L2 partial tag parity error enabled
  System ECC TLB reload error enabled
  L2 ECC TLB reload error enabled
  L2 ECC K7 deallocate enabled
  L2 ECC probe deallocate enabled
  System datareaderror reporting enabled
 Load/Store unit check enabled
  Read data error enable (loads) enabled
  Read data error enable (stores) enabled

           31       23       15       7 
Bank: 0 (0x400)
MC0CTL:    00000000 00000000 00000000 11111111
MC0STATUS: 00000000 00000000 00000000 00000000
MC0ADDR:   11111111 11111111 11111111 11111111
MC0MISC:   00000000 00000000 00000000 00000000

Bank: 1 (0x404)
MC1CTL:    11111111 11111111 11111111 11111111
MC1STATUS: 00000000 00000000 00000000 00000000
MC1ADDR:   11111111 11111111 11111111 11111111
MC1MISC:   00000000 00000000 00000000 00000000

Bank: 2 (0x408)
MC2CTL:    00000000 00000000 00001111 11111111
MC2STATUS: 00000000 00000000 00000000 00000000
MC2ADDR:   11111111 11111111 11111111 11111111
MC2MISC:   00000000 00000000 00000000 00000000

Bank: 3 (0x40c)
MC3CTL:    00000000 00000000 00000000 00000011
MC3STATUS: 00000000 00000000 00000000 00000000
MC3ADDR:   11111111 11111111 11111111 11111111
MC3MISC:   00000000 00000000 00000000 00000000

Bank: 4 (0x410)
MC4CTL:    00000000 00000000 00000000 00000000
MC4STATUS: 00000000 00000000 00000000 00000000
MC4ADDR:   00000000 00000000 00000000 00000000
MC4MISC:   11111111 11111111 11111111 11111111

Bank: 5 (0x414)
MC5CTL:    00000000 00000000 00000000 00000001
MC5STATUS: 00000000 00000000 00000000 00000000
MC5ADDR:   00111111 11111111 11111111 11111111
MC5MISC:   00000000 00000000 00001111 11111111

Microcode patch level: 0x1000086

PowerNOW! Technology information
Available features:
	Temperature sensing diode present.
	Thermal Trip
	Thermal Monitoring
	Software Thermal Control
	100MHz multiplier control
	Hardware P-state control
	invariant TSC

Pstate-0: fid=10, did=0, vid=10 (3200MHz)
Pstate-1: fid=9, did=0, vid=18 (2500MHz)
Pstate-2: fid=5, did=0, vid=20 (2100MHz)
Pstate-3: fid=0, did=1, vid=2e (800MHz) (current)

Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
eax in: 0x00000000, eax = 00000005 ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x00000001, eax = 00100f42 ebx = 02040800 ecx = 00802009 edx = 178bfbff
eax in: 0x00000002, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000005, eax = 00000040 ebx = 00000040 ecx = 00000003 edx = 00000000

eax in: 0x80000000, eax = 8000001b ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x80000001, eax = 00100f42 ebx = 10001b76 ecx = 000037ff edx = efd3fbff
eax in: 0x80000002, eax = 20444d41 ebx = 6e656850 ecx = 74286d6f edx = 4920296d
eax in: 0x80000003, eax = 34582049 ebx = 35353920 ecx = 6f725020 edx = 73736563
eax in: 0x80000004, eax = 0000726f ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000005, eax = ff30ff10 ebx = ff30ff20 ecx = 40020140 edx = 40020140
eax in: 0x80000006, eax = 20800000 ebx = 42004200 ecx = 02008140 edx = 0030b140
eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000001f9
eax in: 0x80000008, eax = 00003030 ebx = 00000000 ecx = 00002003 edx = 00000000
eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000a, eax = 00000001 ebx = 00000040 ecx = 00000000 edx = 0000000f
eax in: 0x8000000b, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000c, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000d, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000e, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000f, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000010, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000011, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000012, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000013, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000014, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000015, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000016, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000017, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000018, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000019, eax = f0300000 ebx = 60100000 ecx = 00000000 edx = 00000000
eax in: 0x8000001a, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000001b, eax = 0000001f ebx = 00000000 ecx = 00000000 edx = 00000000

L1 Data TLB (1G):           Fully associative. 48 entries.
L1 Instruction TLB (1G):    Disabled. 0 entries.
L1 Data TLB (2M/4M):        Fully associative. 48 entries.
L1 Instruction TLB (2M/4M): Fully associative. 16 entries.
L1 Data TLB (4K):           Fully associative. 48 entries.
L1 Instruction TLB (4K):    Fully associative. 32 entries.
L1 Data cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L1 Instruction cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L2 Data TLB (1G):           8-way associative. 16 entries.
L2 Instruction TLB (1G):    Disabled. 0 entries.
L2 Data TLB (2M/4M):        2-way associative. 128 entries.
L2 Instruction TLB (2M/4M): Disabled. 0 entries.
L2 Data TLB (4K):           4-way associative. 512 entries.
L2 Instruction TLB (4K):    4-way associative. 512 entries.
L2 cache:
	Size: 512Kb	16-way associative. 
	lines per tag=1	line size=64 bytes.
L3 (shared) cache:
	Size: 6144Kb	48-way associative. 
	lines per tag=1	line size=64 bytes.

Feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht sse3 mwait cmpxchg16b popcnt
Extended feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 nx mmxext mmx fxsr ffxsr page1gb rdtscp lm 3dnowext 3dnow lahf/sahf CmpLegacy svm ExtApicSpace LockMovCr0 abm sse4a misalignsse 3dnowPref osvw ibs skinit wdt

Long NOPs supported: yes
Connector type: Socket AM2r2

MTRR registers:
MTRRcap (0xfe): 0x0000000000000508
MTRRphysBase0 (0x200): 0x0000000000000006
MTRRphysMask0 (0x201): 0x0000ffff80000800
MTRRphysBase1 (0x202): 0x0000000080000006
MTRRphysMask1 (0x203): 0x0000ffffc0000800
MTRRphysBase2 (0x204): 0x00000000c0000006
MTRRphysMask2 (0x205): 0x0000ffffe0000800
MTRRphysBase3 (0x206): 0x00000000f0000001
MTRRphysMask3 (0x207): 0x0000fffff8000800
MTRRphysBase4 (0x208): 0x0000000000000000
MTRRphysMask4 (0x209): 0x0000000000000000
MTRRphysBase5 (0x20a): 0x0000000000000000
MTRRphysMask5 (0x20b): 0x0000000000000000
MTRRphysBase6 (0x20c): 0x0000000000000000
MTRRphysMask6 (0x20d): 0x0000000000000000
MTRRphysBase7 (0x20e): 0x0000000000000000
MTRRphysMask7 (0x20f): 0x0000000000000000
MTRRfix64K_00000 (0x250): 0x0606060606060606
MTRRfix16K_80000 (0x258): 0x0606060606060606
MTRRfix16K_A0000 (0x259): 0x0000000000000000
MTRRfix4K_C8000 (0x269): 0x0000000000000000
MTRRfix4K_D0000 0x26a: 0x0000000000000000
MTRRfix4K_D8000 0x26b: 0x0000000000000000
MTRRfix4K_E0000 0x26c: 0x0000000000000000
MTRRfix4K_E8000 0x26d: 0x0000000000000000
MTRRfix4K_F0000 0x26e: 0x0505050505050505
MTRRfix4K_F8000 0x26f: 0x0505050505050505
MTRRdefType (0x2ff): 0x0000000000000c00


3.20GHz processor (estimate).

--------------------------------------------------------------------------
CPU #4
EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
Processor name string: AMD Phenom(tm) II X4 955 Processor
Number of reporting banks : 6

MCG_CTL:
 Data cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Data cache data parity enabled
  Data cache main tag parity enabled
  Data cache snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
 Instruction cache check enabled
  ECC 1 bit error reporting enabled
  ECC multi bit error reporting enabled
  Instruction cache data parity enabled
  IC main tag parity enabled
  IC snoop tag parity enabled
  L1 TLB parity enabled
  L2 TLB parity enabled
  Predecode array parity enabled
  Target selector parity enabled
  Read data error enabled
 Bus unit check enabled
  External L2 tag parity error enabled
  L2 partial tag parity error enabled
  System ECC TLB reload error enabled
  L2 ECC TLB reload error enabled
  L2 ECC K7 deallocate enabled
  L2 ECC probe deallocate enabled
  System datareaderror reporting enabled
 Load/Store unit check enabled
  Read data error enable (loads) enabled
  Read data error enable (stores) enabled

           31       23       15       7 
Bank: 0 (0x400)
MC0CTL:    00000000 00000000 00000000 11111111
MC0STATUS: 00000000 00000000 00000000 00000000
MC0ADDR:   11111111 11111111 11111111 11111111
MC0MISC:   00000000 00000000 00000000 00000000

Bank: 1 (0x404)
MC1CTL:    11111111 11111111 11111111 11111111
MC1STATUS: 00000000 00000000 00000000 00000000
MC1ADDR:   00000000 00000000 00000000 00000000
MC1MISC:   00000000 00000000 00000000 00000000

Bank: 2 (0x408)
MC2CTL:    00000000 00000000 00001111 11111111
MC2STATUS: 00000000 00000000 00000000 00000000
MC2ADDR:   11111111 11111111 11111111 11111111
MC2MISC:   00000000 00000000 00000000 00000000

Bank: 3 (0x40c)
MC3CTL:    00000000 00000000 00000000 00000011
MC3STATUS: 00000000 00000000 00000000 00000000
MC3ADDR:   11111111 11111111 11111111 11111111
MC3MISC:   00000000 00000000 00000000 00000000

Bank: 4 (0x410)
MC4CTL:    00000000 00000000 00000000 00000000
MC4STATUS: 00000000 00000000 00000000 00000000
MC4ADDR:   00000000 00000000 00000000 00000000
MC4MISC:   11111111 11111111 11111111 11111111

Bank: 5 (0x414)
MC5CTL:    00000000 00000000 00000000 00000001
MC5STATUS: 00000000 00000000 00000000 00000000
MC5ADDR:   11111111 11111111 11111111 11111111
MC5MISC:   00000000 00000000 00001111 11111111

Microcode patch level: 0x1000086

PowerNOW! Technology information
Available features:
	Temperature sensing diode present.
	Thermal Trip
	Thermal Monitoring
	Software Thermal Control
	100MHz multiplier control
	Hardware P-state control
	invariant TSC

Pstate-0: fid=10, did=0, vid=10 (3200MHz)
Pstate-1: fid=9, did=0, vid=18 (2500MHz)
Pstate-2: fid=5, did=0, vid=20 (2100MHz)
Pstate-3: fid=0, did=1, vid=2e (800MHz) (current)

Monitor/Mwait: min/max line size 64/64, ecx bit 0 support, enumeration extension
SVM: revision 1, 64 ASIDs, np, lbrVirt, SVMLock, NRIPSave
Address Size: 48 bits virtual, 48 bits physical
The physical package has 4 of 4 possible cores implemented.
eax in: 0x00000000, eax = 00000005 ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x00000001, eax = 00100f42 ebx = 03040800 ecx = 00802009 edx = 178bfbff
eax in: 0x00000002, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000004, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x00000005, eax = 00000040 ebx = 00000040 ecx = 00000003 edx = 00000000

eax in: 0x80000000, eax = 8000001b ebx = 68747541 ecx = 444d4163 edx = 69746e65
eax in: 0x80000001, eax = 00100f42 ebx = 10001b76 ecx = 000037ff edx = efd3fbff
eax in: 0x80000002, eax = 20444d41 ebx = 6e656850 ecx = 74286d6f edx = 4920296d
eax in: 0x80000003, eax = 34582049 ebx = 35353920 ecx = 6f725020 edx = 73736563
eax in: 0x80000004, eax = 0000726f ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000005, eax = ff30ff10 ebx = ff30ff20 ecx = 40020140 edx = 40020140
eax in: 0x80000006, eax = 20800000 ebx = 42004200 ecx = 02008140 edx = 0030b140
eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 000001f9
eax in: 0x80000008, eax = 00003030 ebx = 00000000 ecx = 00002003 edx = 00000000
eax in: 0x80000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000a, eax = 00000001 ebx = 00000040 ecx = 00000000 edx = 0000000f
eax in: 0x8000000b, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000c, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000d, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000e, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000000f, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000010, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000011, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000012, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000013, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000014, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000015, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000016, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000017, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000018, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x80000019, eax = f0300000 ebx = 60100000 ecx = 00000000 edx = 00000000
eax in: 0x8000001a, eax = 00000003 ebx = 00000000 ecx = 00000000 edx = 00000000
eax in: 0x8000001b, eax = 0000001f ebx = 00000000 ecx = 00000000 edx = 00000000

L1 Data TLB (1G):           Fully associative. 48 entries.
L1 Instruction TLB (1G):    Disabled. 0 entries.
L1 Data TLB (2M/4M):        Fully associative. 48 entries.
L1 Instruction TLB (2M/4M): Fully associative. 16 entries.
L1 Data TLB (4K):           Fully associative. 48 entries.
L1 Instruction TLB (4K):    Fully associative. 32 entries.
L1 Data cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L1 Instruction cache:
	Size: 64Kb	2-way associative. 
	lines per tag=1	line size=64 bytes.
L2 Data TLB (1G):           8-way associative. 16 entries.
L2 Instruction TLB (1G):    Disabled. 0 entries.
L2 Data TLB (2M/4M):        2-way associative. 128 entries.
L2 Instruction TLB (2M/4M): Disabled. 0 entries.
L2 Data TLB (4K):           4-way associative. 512 entries.
L2 Instruction TLB (4K):    4-way associative. 512 entries.
L2 cache:
	Size: 512Kb	16-way associative. 
	lines per tag=1	line size=64 bytes.
L3 (shared) cache:
	Size: 6144Kb	48-way associative. 
	lines per tag=1	line size=64 bytes.

Feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh mmx fxsr sse sse2 ht sse3 mwait cmpxchg16b popcnt
Extended feature flags:
 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 nx mmxext mmx fxsr ffxsr page1gb rdtscp lm 3dnowext 3dnow lahf/sahf CmpLegacy svm ExtApicSpace LockMovCr0 abm sse4a misalignsse 3dnowPref osvw ibs skinit wdt

Long NOPs supported: yes
Connector type: Socket AM2r2

MTRR registers:
MTRRcap (0xfe): 0x0000000000000508
MTRRphysBase0 (0x200): 0x0000000000000006
MTRRphysMask0 (0x201): 0x0000ffff80000800
MTRRphysBase1 (0x202): 0x0000000080000006
MTRRphysMask1 (0x203): 0x0000ffffc0000800
MTRRphysBase2 (0x204): 0x00000000c0000006
MTRRphysMask2 (0x205): 0x0000ffffe0000800
MTRRphysBase3 (0x206): 0x00000000f0000001
MTRRphysMask3 (0x207): 0x0000fffff8000800
MTRRphysBase4 (0x208): 0x0000000000000000
MTRRphysMask4 (0x209): 0x0000000000000000
MTRRphysBase5 (0x20a): 0x0000000000000000
MTRRphysMask5 (0x20b): 0x0000000000000000
MTRRphysBase6 (0x20c): 0x0000000000000000
MTRRphysMask6 (0x20d): 0x0000000000000000
MTRRphysBase7 (0x20e): 0x0000000000000000
MTRRphysMask7 (0x20f): 0x0000000000000000
MTRRfix64K_00000 (0x250): 0x0606060606060606
MTRRfix16K_80000 (0x258): 0x0606060606060606
MTRRfix16K_A0000 (0x259): 0x0000000000000000
MTRRfix4K_C8000 (0x269): 0x0000000000000000
MTRRfix4K_D0000 0x26a: 0x0000000000000000
MTRRfix4K_D8000 0x26b: 0x0000000000000000
MTRRfix4K_E0000 0x26c: 0x0000000000000000
MTRRfix4K_E8000 0x26d: 0x0000000000000000
MTRRfix4K_F0000 0x26e: 0x0505050505050505
MTRRfix4K_F8000 0x26f: 0x0505050505050505
MTRRdefType (0x2ff): 0x0000000000000c00


3.20GHz processor (estimate).

--------------------------------------------------------------------------
 running at an estimated 3.20GHz

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 11:34                   ` Markus Trippelsdorf
@ 2010-10-29 11:39                     ` Borislav Petkov
  2010-10-29 11:54                       ` Markus Trippelsdorf
  0 siblings, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-10-29 11:39 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, john stultz, Thomas Gleixner, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Oct 29, 2010 at 07:34:33AM -0400, Markus Trippelsdorf wrote:
> > Markus, can you verify this on your system by compiling x86info from
> > git://git.choralone.org/git/x86info and doing
> > 
> > ./lsmsr Int
> 
> 
> x86info v1.28beta.  Dave Jones 2001-2010
> Feedback to <davej@redhat.com>.
> 
> Found 4 CPUs
> --------------------------------------------------------------------------
> CPU #1
> EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
> CPU Model: Quad-Core Opteron/Phenom II (RB-C2)

well, x86info doesn't help - there's a lsmsr tool which is being built
too but no need, I can infer the info from your revision: your CPU is
RB-C2 which does SMI initiated C1E.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 11:39                     ` Borislav Petkov
@ 2010-10-29 11:54                       ` Markus Trippelsdorf
  2010-10-29 13:13                         ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-29 11:54 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Borislav Petkov, john stultz, Thomas Gleixner, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Oct 29, 2010 at 01:39:34PM +0200, Borislav Petkov wrote:
> On Fri, Oct 29, 2010 at 07:34:33AM -0400, Markus Trippelsdorf wrote:
> > > Markus, can you verify this on your system by compiling x86info from
> > > git://git.choralone.org/git/x86info and doing
> > > 
> > > ./lsmsr Int
> > 
> > 
> > x86info v1.28beta.  Dave Jones 2001-2010
> > Feedback to <davej@redhat.com>.
> > 
> > Found 4 CPUs
> > --------------------------------------------------------------------------
> > CPU #1
> > EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
> > CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
> 
> well, x86info doesn't help - there's a lsmsr tool which is being built
> too but no need, I can infer the info from your revision: your CPU is
> RB-C2 which does SMI initiated C1E.

Sorry. Here is the correct output:

IntPendingMessage    = 0x0000000008c200b0

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 10:27                 ` Borislav Petkov
  2010-10-29 11:34                   ` Markus Trippelsdorf
@ 2010-10-29 12:14                   ` Thomas Gleixner
  2010-10-29 17:00                     ` Borislav Petkov
  1 sibling, 1 reply; 46+ messages in thread
From: Thomas Gleixner @ 2010-10-29 12:14 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Markus Trippelsdorf, Borislav Petkov, john stultz, linux-kernel,
	hpa, Ingo Molnar, Herrmann3, Andreas, heiko.carstens,
	a.p.zijlstra, avi, mtosatti

On Fri, 29 Oct 2010, Borislav Petkov wrote:
> On Fri, Oct 29, 2010 at 04:30:13AM -0400, Markus Trippelsdorf wrote:
> That's strange, when we tested this one everything seemed fine so I
> guess this is one of those bugs which appear later, just as if out of
> nowhere.
> 
> Thomas, could it be that SMI fires in between the HPET write and
> subsequent read:
> 
> 	hpet_writel(cnt, HPET_Tn_CMP(timer));
> 
> 	...
> 
> 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
> 
> causing the -ETIME and thus a wait for HPET wraparound? My machine at
> least does SMI-initiated C1E so it could very well be the problem.

Well, even if a SMI fires, then the counter will be ahead of cnt and
we get -ETIME. The upper layer of clockevents/timers will then
reprogram HPET. So that's not an issue.

The only problem which might hit us is when our assumption of 8 HPET
cycles being enough to transfer the new match value into the real
match register is wrong.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 11:54                       ` Markus Trippelsdorf
@ 2010-10-29 13:13                         ` Borislav Petkov
  0 siblings, 0 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-10-29 13:13 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, john stultz, Thomas Gleixner, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Oct 29, 2010 at 07:54:38AM -0400, Markus Trippelsdorf wrote:
> On Fri, Oct 29, 2010 at 01:39:34PM +0200, Borislav Petkov wrote:
> > On Fri, Oct 29, 2010 at 07:34:33AM -0400, Markus Trippelsdorf wrote:
> > > > Markus, can you verify this on your system by compiling x86info from
> > > > git://git.choralone.org/git/x86info and doing
> > > > 
> > > > ./lsmsr Int
> > > 
> > > 
> > > x86info v1.28beta.  Dave Jones 2001-2010
> > > Feedback to <davej@redhat.com>.
> > > 
> > > Found 4 CPUs
> > > --------------------------------------------------------------------------
> > > CPU #1
> > > EFamily: 1 EModel: 0 Family: 15 Model: 4 Stepping: 2
> > > CPU Model: Quad-Core Opteron/Phenom II (RB-C2)
> > 
> > well, x86info doesn't help - there's a lsmsr tool which is being built
> > too but no need, I can infer the info from your revision: your CPU is
> > RB-C2 which does SMI initiated C1E.
> 
> Sorry. Here is the correct output:
> 
> IntPendingMessage    = 0x0000000008c200b0

Yep, bit 27 is set, SmiOnCmpHalt.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 12:14                   ` Thomas Gleixner
@ 2010-10-29 17:00                     ` Borislav Petkov
  2010-10-29 17:26                       ` Markus Trippelsdorf
  2010-10-29 17:34                       ` Thomas Gleixner
  0 siblings, 2 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-10-29 17:00 UTC (permalink / raw)
  To: Thomas Gleixner, Markus Trippelsdorf
  Cc: Borislav Petkov, john stultz, linux-kernel, hpa, Ingo Molnar,
	Herrmann3, Andreas, heiko.carstens, a.p.zijlstra, avi, mtosatti

On Fri, Oct 29, 2010 at 08:14:12AM -0400, Thomas Gleixner wrote:
> On Fri, 29 Oct 2010, Borislav Petkov wrote:
> > On Fri, Oct 29, 2010 at 04:30:13AM -0400, Markus Trippelsdorf wrote:
> > That's strange, when we tested this one everything seemed fine so I
> > guess this is one of those bugs which appear later, just as if out of
> > nowhere.
> > 
> > Thomas, could it be that SMI fires in between the HPET write and
> > subsequent read:
> > 
> > 	hpet_writel(cnt, HPET_Tn_CMP(timer));
> > 
> > 	...
> > 
> > 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
> > 
> > causing the -ETIME and thus a wait for HPET wraparound? My machine at
> > least does SMI-initiated C1E so it could very well be the problem.
> 
> Well, even if a SMI fires, then the counter will be ahead of cnt and
> we get -ETIME. The upper layer of clockevents/timers will then
> reprogram HPET. So that's not an issue.

Ah, I missed the upper layer, good.

> The only problem which might hit us is when our assumption of 8 HPET
> cycles being enough to transfer the new match value into the real
> match register is wrong.

So Markus, can you try with 995bd3bb5c78, but by increasing the value
to, say 16 (I don't know what's a good value here, let's double the old
one). Simply change the line

	return res < 8 ? -ETIME : 0;

to

	return res < 16 ? -ETIME : 0;


in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
machine here when I get around to it.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 17:00                     ` Borislav Petkov
@ 2010-10-29 17:26                       ` Markus Trippelsdorf
  2010-11-01 18:05                         ` Thomas Gleixner
  2010-10-29 17:34                       ` Thomas Gleixner
  1 sibling, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-10-29 17:26 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Oct 29, 2010 at 07:00:40PM +0200, Borislav Petkov wrote:
> On Fri, Oct 29, 2010 at 08:14:12AM -0400, Thomas Gleixner wrote:
> > On Fri, 29 Oct 2010, Borislav Petkov wrote:
> > > On Fri, Oct 29, 2010 at 04:30:13AM -0400, Markus Trippelsdorf wrote:
> > > That's strange, when we tested this one everything seemed fine so I
> > > guess this is one of those bugs which appear later, just as if out of
> > > nowhere.
> > > 
> > > Thomas, could it be that SMI fires in between the HPET write and
> > > subsequent read:
> > > 
> > > 	hpet_writel(cnt, HPET_Tn_CMP(timer));
> > > 
> > > 	...
> > > 
> > > 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
> > > 
> > > causing the -ETIME and thus a wait for HPET wraparound? My machine at
> > > least does SMI-initiated C1E so it could very well be the problem.
> > 
> > Well, even if a SMI fires, then the counter will be ahead of cnt and
> > we get -ETIME. The upper layer of clockevents/timers will then
> > reprogram HPET. So that's not an issue.
> 
> Ah, I missed the upper layer, good.
> 
> > The only problem which might hit us is when our assumption of 8 HPET
> > cycles being enough to transfer the new match value into the real
> > match register is wrong.
> 
> So Markus, can you try with 995bd3bb5c78, but by increasing the value
> to, say 16 (I don't know what's a good value here, let's double the old
> one). Simply change the line
> 
> 	return res < 8 ? -ETIME : 0;
> 
> to
> 
> 	return res < 16 ? -ETIME : 0;
> 
> 
> in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
> machine here when I get around to it.

Yes, I already did this, although I'm running
return res < 12 ? -ETIME : 0; 
at the moment.

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 17:00                     ` Borislav Petkov
  2010-10-29 17:26                       ` Markus Trippelsdorf
@ 2010-10-29 17:34                       ` Thomas Gleixner
  1 sibling, 0 replies; 46+ messages in thread
From: Thomas Gleixner @ 2010-10-29 17:34 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Markus Trippelsdorf, Borislav Petkov, john stultz, linux-kernel,
	hpa, Ingo Molnar, Herrmann3, Andreas, heiko.carstens,
	a.p.zijlstra, avi, mtosatti

Boris,

On Fri, 29 Oct 2010, Borislav Petkov wrote:
> So Markus, can you try with 995bd3bb5c78, but by increasing the value
> to, say 16 (I don't know what's a good value here, let's double the old
> one). Simply change the line
> 
> 	return res < 8 ? -ETIME : 0;
> 
> to
> 
> 	return res < 16 ? -ETIME : 0;
> 
> 
> in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
> machine here when I get around to it.

can you please check with your hardware folks? If we can't get some
affirmative info about that then we'd be forced to revert that commit
(if it turns out to be the culprit one). That'd be a pity as it would
bring back the full IO access penalty to everyone.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-29 17:26                       ` Markus Trippelsdorf
@ 2010-11-01 18:05                         ` Thomas Gleixner
  2010-11-01 18:45                           ` Markus Trippelsdorf
  0 siblings, 1 reply; 46+ messages in thread
From: Thomas Gleixner @ 2010-11-01 18:05 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, 29 Oct 2010, Markus Trippelsdorf wrote:
> On Fri, Oct 29, 2010 at 07:00:40PM +0200, Borislav Petkov wrote:
> > So Markus, can you try with 995bd3bb5c78, but by increasing the value
> > to, say 16 (I don't know what's a good value here, let's double the old
> > one). Simply change the line
> > 
> > 	return res < 8 ? -ETIME : 0;
> > 
> > to
> > 
> > 	return res < 16 ? -ETIME : 0;
> > 
> > 
> > in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
> > machine here when I get around to it.
> 
> Yes, I already did this, although I'm running
> return res < 12 ? -ETIME : 0; 
> at the moment.

Any update on this ?

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-01 18:05                         ` Thomas Gleixner
@ 2010-11-01 18:45                           ` Markus Trippelsdorf
  2010-11-02 15:26                             ` Thomas Gleixner
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-01 18:45 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Borislav Petkov, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Mon, Nov 01, 2010 at 07:05:17PM +0100, Thomas Gleixner wrote:
> On Fri, 29 Oct 2010, Markus Trippelsdorf wrote:
> > On Fri, Oct 29, 2010 at 07:00:40PM +0200, Borislav Petkov wrote:
> > > So Markus, can you try with 995bd3bb5c78, but by increasing the value
> > > to, say 16 (I don't know what's a good value here, let's double the old
> > > one). Simply change the line
> > > 
> > > 	return res < 8 ? -ETIME : 0;
> > > 
> > > to
> > > 
> > > 	return res < 16 ? -ETIME : 0;
> > > 
> > > 
> > > in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
> > > machine here when I get around to it.
> > 
> > Yes, I already did this, although I'm running
> > return res < 12 ? -ETIME : 0; 
> > at the moment.
> 
> Any update on this ?

Yes, it runs stable thus far. No "slowdowns", no problems.
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-01 18:45                           ` Markus Trippelsdorf
@ 2010-11-02 15:26                             ` Thomas Gleixner
  2010-11-05 16:09                               ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Thomas Gleixner @ 2010-11-02 15:26 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Mon, 1 Nov 2010, Markus Trippelsdorf wrote:

> On Mon, Nov 01, 2010 at 07:05:17PM +0100, Thomas Gleixner wrote:
> > On Fri, 29 Oct 2010, Markus Trippelsdorf wrote:
> > > On Fri, Oct 29, 2010 at 07:00:40PM +0200, Borislav Petkov wrote:
> > > > So Markus, can you try with 995bd3bb5c78, but by increasing the value
> > > > to, say 16 (I don't know what's a good value here, let's double the old
> > > > one). Simply change the line
> > > > 
> > > > 	return res < 8 ? -ETIME : 0;
> > > > 
> > > > to
> > > > 
> > > > 	return res < 16 ? -ETIME : 0;
> > > > 
> > > > 
> > > > in <arch/x86/kernel/hpet.c:hpet_next_event()>. I'll do that too on the
> > > > machine here when I get around to it.
> > > 
> > > Yes, I already did this, although I'm running
> > > return res < 12 ? -ETIME : 0; 
> > > at the moment.
> > 
> > Any update on this ?
> 
> Yes, it runs stable thus far. No "slowdowns", no problems.

Ok, I'm waiting for Boris to get us the confirmation from HW folks.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-02 15:26                             ` Thomas Gleixner
@ 2010-11-05 16:09                               ` Borislav Petkov
  2010-11-05 16:42                                 ` Markus Trippelsdorf
  0 siblings, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-11-05 16:09 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Markus Trippelsdorf, Borislav Petkov, Borislav Petkov,
	john stultz, linux-kernel, hpa, Ingo Molnar, Herrmann3, Andreas,
	heiko.carstens, a.p.zijlstra, avi, mtosatti

On Tue, Nov 02, 2010 at 11:26:54AM -0400, Thomas Gleixner wrote:
> Ok, I'm waiting for Boris to get us the confirmation from HW folks.

Ok, I don't have much but it could be something. We could use the
minimum tick u16 value in the ACPI HPET table, offset 53:

Main Counter Minimum   2 53 Unit: Clock tick
Clock_tick in Periodic      The minimum clock ticks can be set without lost
Mode                        interrupts while the counter is programmed to operate in
*Note#3                     periodic mode

...

* Note #3: This field is written by BIOS and may be chipset and/or
platform dependent. This indicates the minimum value that must be used
for any counter programmed in periodic mode to avoid lost interrupts.
For any counter x that has been configured for periodic mode, the number
can be programmed in any Tx_Compare Register must be greater than P,
where P = (Minimum Period) / (Main counter period) in order to avoid
lost interrupts.

However, we don't know (yet) whether this can be used in the
non-periodic mode too. My gut feeling says yes but I wouldn't trust it.

We went and collected that value a bunch of systems and it looks like it
would need more massaging (read: capping) since some BIOSen simply write
crap in it. It ranges from

- 0x37ee on old nVidia and Intel boards (this is definitely crap, I
can't imagine a minimum ticks value of 14318 for a HPET but who knows)

- 0x1000 on a HP machine (also fishy)

- 0x10, 0x14 on current SBxxx boards

- 0x80 on newer Intel boards

and it looks like 0x80 is also the default value which we can use as a
capping value. Btw, I've been running the following on the machine in
question for a while now and no lockups so far.

Opinions?

---
 arch/x86/include/asm/hpet.h |    1 +
 arch/x86/kernel/acpi/boot.c |   12 +++++++++---
 arch/x86/kernel/hpet.c      |    3 ++-
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/hpet.h b/arch/x86/include/asm/hpet.h
index 2c392d6..01d9480 100644
--- a/arch/x86/include/asm/hpet.h
+++ b/arch/x86/include/asm/hpet.h
@@ -68,6 +68,7 @@ extern unsigned long force_hpet_address;
 extern u8 hpet_blockid;
 extern int hpet_force_user;
 extern u8 hpet_msi_disable;
+extern u16 hpet_min_tick;
 extern int is_hpet_enabled(void);
 extern int hpet_enable(void);
 extern void hpet_disable(void);
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 71232b9..37b93e8 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -772,9 +772,6 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
 		hpet_address >>= 32;
 	}
 #endif
-	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
-	       hpet_tbl->id, hpet_address);
-
 	/*
 	 * Allocate and initialize the HPET firmware resource for adding into
 	 * the resource tree during the lateinit timeframe.
@@ -790,6 +787,15 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
 	hpet_res->start = hpet_address;
 	hpet_res->end = hpet_address + (1 * 1024) - 1;
 
+	hpet_min_tick = hpet_tbl->minimum_tick;
+	if (hpet_min_tick > 0x80) {
+		printk(KERN_WARNING PREFIX "Limiting HPET read delay to 0x80.\n");
+		hpet_min_tick = 0x80;
+	}
+
+	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx min tick: %d\n",
+		hpet_tbl->id, hpet_address, hpet_min_tick);
+
 	return 0;
 }
 
diff --git a/arch/x86/kernel/hpet.c b/arch/x86/kernel/hpet.c
index e49c3b9..a0b790a 100644
--- a/arch/x86/kernel/hpet.c
+++ b/arch/x86/kernel/hpet.c
@@ -40,6 +40,7 @@ u8					hpet_msi_disable;
 static unsigned long			hpet_num_timers;
 #endif
 static void __iomem			*hpet_virt_address;
+u16					hpet_min_tick = 0x80;
 
 struct hpet_dev {
 	struct clock_event_device	evt;
@@ -408,7 +409,7 @@ static int hpet_next_event(unsigned long delta,
 	 */
 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
 
-	return res < 8 ? -ETIME : 0;
+	return res < hpet_min_tick ? -ETIME : 0;
 }
 
 static void hpet_legacy_set_mode(enum clock_event_mode mode,
-- 
1.7.3.1.50.g1e633




-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply related	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-05 16:09                               ` Borislav Petkov
@ 2010-11-05 16:42                                 ` Markus Trippelsdorf
  2010-11-05 17:45                                   ` Markus Trippelsdorf
  2010-11-05 21:27                                   ` Markus Trippelsdorf
  0 siblings, 2 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-05 16:42 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Nov 05, 2010 at 05:09:19PM +0100, Borislav Petkov wrote:
> On Tue, Nov 02, 2010 at 11:26:54AM -0400, Thomas Gleixner wrote:
> > Ok, I'm waiting for Boris to get us the confirmation from HW folks.
> 
> Ok, I don't have much but it could be something. We could use the
> minimum tick u16 value in the ACPI HPET table, offset 53:
> 
> Main Counter Minimum   2 53 Unit: Clock tick
> Clock_tick in Periodic      The minimum clock ticks can be set without lost
> Mode                        interrupts while the counter is programmed to operate in
> *Note#3                     periodic mode
> 
> ...
> 
> * Note #3: This field is written by BIOS and may be chipset and/or
> platform dependent. This indicates the minimum value that must be used
> for any counter programmed in periodic mode to avoid lost interrupts.
> For any counter x that has been configured for periodic mode, the number
> can be programmed in any Tx_Compare Register must be greater than P,
> where P = (Minimum Period) / (Main counter period) in order to avoid
> lost interrupts.
> 
> However, we don't know (yet) whether this can be used in the
> non-periodic mode too. My gut feeling says yes but I wouldn't trust it.
> 
> We went and collected that value a bunch of systems and it looks like it
> would need more massaging (read: capping) since some BIOSen simply write
> crap in it. It ranges from
> 
> - 0x37ee on old nVidia and Intel boards (this is definitely crap, I
> can't imagine a minimum ticks value of 14318 for a HPET but who knows)
> 
> - 0x1000 on a HP machine (also fishy)
> 
> - 0x10, 0x14 on current SBxxx boards
> 
> - 0x80 on newer Intel boards

"min tick: 20" on my machine. Given that 12 is running stable here, maybe
the value is a bit too large?

> and it looks like 0x80 is also the default value which we can use as a
> capping value. Btw, I've been running the following on the machine in
> question for a while now and no lockups so far.
> 
> Opinions?
> 
> ---
>  arch/x86/include/asm/hpet.h |    1 +
>  arch/x86/kernel/acpi/boot.c |   12 +++++++++---
>  arch/x86/kernel/hpet.c      |    3 ++-
>  3 files changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/arch/x86/include/asm/hpet.h b/arch/x86/include/asm/hpet.h
> index 2c392d6..01d9480 100644
> --- a/arch/x86/include/asm/hpet.h
> +++ b/arch/x86/include/asm/hpet.h
> @@ -68,6 +68,7 @@ extern unsigned long force_hpet_address;
>  extern u8 hpet_blockid;
>  extern int hpet_force_user;
>  extern u8 hpet_msi_disable;
> +extern u16 hpet_min_tick;
>  extern int is_hpet_enabled(void);
>  extern int hpet_enable(void);
>  extern void hpet_disable(void);
> diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
> index 71232b9..37b93e8 100644
> --- a/arch/x86/kernel/acpi/boot.c
> +++ b/arch/x86/kernel/acpi/boot.c
> @@ -772,9 +772,6 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
>  		hpet_address >>= 32;
>  	}
>  #endif
> -	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
> -	       hpet_tbl->id, hpet_address);
> -
>  	/*
>  	 * Allocate and initialize the HPET firmware resource for adding into
>  	 * the resource tree during the lateinit timeframe.
> @@ -790,6 +787,15 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
>  	hpet_res->start = hpet_address;
>  	hpet_res->end = hpet_address + (1 * 1024) - 1;
>  
> +	hpet_min_tick = hpet_tbl->minimum_tick;
> +	if (hpet_min_tick > 0x80) {
> +		printk(KERN_WARNING PREFIX "Limiting HPET read delay to 0x80.\n");
> +		hpet_min_tick = 0x80;
> +	}
> +
> +	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx min tick: %d\n",
> +		hpet_tbl->id, hpet_address, hpet_min_tick);
> +
>  	return 0;
>  }
>  
> diff --git a/arch/x86/kernel/hpet.c b/arch/x86/kernel/hpet.c
> index e49c3b9..a0b790a 100644
> --- a/arch/x86/kernel/hpet.c
> +++ b/arch/x86/kernel/hpet.c
> @@ -40,6 +40,7 @@ u8					hpet_msi_disable;
>  static unsigned long			hpet_num_timers;
>  #endif
>  static void __iomem			*hpet_virt_address;
> +u16					hpet_min_tick = 0x80;

Wouldn't:

+u16                                  hpet_min_tick;

make more sense, if you want to use the value detected in boot.c?

>  struct hpet_dev {
>  	struct clock_event_device	evt;
> @@ -408,7 +409,7 @@ static int hpet_next_event(unsigned long delta,
>  	 */
>  	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
>  
> -	return res < 8 ? -ETIME : 0;
> +	return res < hpet_min_tick ? -ETIME : 0;
>  }
>  
>  static void hpet_legacy_set_mode(enum clock_event_mode mode,

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-05 16:42                                 ` Markus Trippelsdorf
@ 2010-11-05 17:45                                   ` Markus Trippelsdorf
  2010-11-05 21:27                                   ` Markus Trippelsdorf
  1 sibling, 0 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-05 17:45 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Nov 05, 2010 at 05:42:25PM +0100, Markus Trippelsdorf wrote:
> On Fri, Nov 05, 2010 at 05:09:19PM +0100, Borislav Petkov wrote:
> > On Tue, Nov 02, 2010 at 11:26:54AM -0400, Thomas Gleixner wrote:
> > > Ok, I'm waiting for Boris to get us the confirmation from HW folks.
> > 
> > +++ b/arch/x86/kernel/hpet.c
> > @@ -40,6 +40,7 @@ u8					hpet_msi_disable;
> >  static unsigned long			hpet_num_timers;
> >  #endif
> >  static void __iomem			*hpet_virt_address;
> > +u16					hpet_min_tick = 0x80;
> 
> Wouldn't:
> 
> +u16                                  hpet_min_tick;
> 
> make more sense, if you want to use the value detected in boot.c?

Please scratch that.
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-05 16:42                                 ` Markus Trippelsdorf
  2010-11-05 17:45                                   ` Markus Trippelsdorf
@ 2010-11-05 21:27                                   ` Markus Trippelsdorf
  2010-11-05 21:32                                     ` Borislav Petkov
  1 sibling, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-05 21:27 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Fri, Nov 05, 2010 at 05:42:25PM +0100, Markus Trippelsdorf wrote:
> On Fri, Nov 05, 2010 at 05:09:19PM +0100, Borislav Petkov wrote:
> > On Tue, Nov 02, 2010 at 11:26:54AM -0400, Thomas Gleixner wrote:
> > > Ok, I'm waiting for Boris to get us the confirmation from HW folks.
> > 
> > However, we don't know (yet) whether this can be used in the
> > non-periodic mode too. My gut feeling says yes but I wouldn't trust it.
> > 
> > We went and collected that value a bunch of systems and it looks like it
> > would need more massaging (read: capping) since some BIOSen simply write
> > crap in it. It ranges from
> > 
> > - 0x37ee on old nVidia and Intel boards (this is definitely crap, I
> > can't imagine a minimum ticks value of 14318 for a HPET but who knows)
> > 
> > - 0x1000 on a HP machine (also fishy)
> > 
> > - 0x10, 0x14 on current SBxxx boards
> > 
> > - 0x80 on newer Intel boards
> 
> "min tick: 20" on my machine. Given that 12 is running stable here, maybe
> the value is a bit too large?

Err, 20==0x14 so it's not that much difference and it's running fine here
also.

Feel free to add:
    Reported-and-tested-by: Markus Trippelsdorf <markus@trippelsdorf.de>

Thanks.

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-05 21:27                                   ` Markus Trippelsdorf
@ 2010-11-05 21:32                                     ` Borislav Petkov
  2010-11-09 14:02                                       ` Thomas Gleixner
  0 siblings, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-11-05 21:32 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Borislav Petkov, Thomas Gleixner, Borislav Petkov, john stultz,
	linux-kernel, hpa, Ingo Molnar, Herrmann3, Andreas,
	heiko.carstens, a.p.zijlstra, avi, mtosatti

On Fri, Nov 05, 2010 at 05:27:49PM -0400, Markus Trippelsdorf wrote:
> Err, 20==0x14 so it's not that much difference and it's running fine here
> also.
> 
> Feel free to add:
>     Reported-and-tested-by: Markus Trippelsdorf <markus@trippelsdorf.de>

Thanks Markus,

actually your board is not what concerns me, 20 ticks is still ok, more
or less, but there are other machines which contain absurd values in
there like 0x37ee or 0x1000 (a Broadcom chipset). We'll need to give a
change like that a good run before we can be absolutely sure it doesn't
break any machines.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-10-27 19:36             ` Peter Zijlstra
  2010-10-27 19:42               ` Markus Trippelsdorf
  2010-10-27 19:58               ` Marcelo Tosatti
@ 2010-11-09 12:58               ` Heiko Carstens
  2010-11-09 13:07                 ` Markus Trippelsdorf
                                   ` (2 more replies)
  2 siblings, 3 replies; 46+ messages in thread
From: Heiko Carstens @ 2010-11-09 12:58 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: markus, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Wed, Oct 27, 2010 at 09:36:55PM +0200, Peter Zijlstra wrote:
> On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> > 
> > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Date:   Wed Sep 22 13:53:15 2010 +0200
> > 
> >     sched: Create special class for stop/migrate work
> > 
> >     In order to separate the stop/migrate work thread from the SCHED_FIFO
> >     implementation, create a special class for it that is of higher priority than
> >     SCHED_FIFO itself.
> > 
> >     This currently solves a problem where cpu-hotplug consumes so much cpu-time
> >     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
> >     timer pending on the now dead cpu.
> > 
> >     It is also required for when we add the planned deadline scheduling class above
> >     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> > 
> >     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> >     LKML-Reference: <1285165776.2275.1022.camel@laptop>
> >     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> > 
> > Reverting the commit solves the kvm hang issue.
> > (If this issue is related to my original tsc problem is of course open for
> > debate, but I have a strong hunch it is.) 
> 
> Too weird,.. what does the hang look like?
> 
> Can you generate a sysrq-t dump? The thing I'm looking for is the
> migration/# thread being runnable but not being current.
> 
> How can I reproduce this?

I think there is a bug in pick_next_task_stop() in sched_stopclass.c:

If a stop-task scheduling class task (well... the migration thread ;) sets
its state to TASK_INTERRUPTIBLE and then gets preempted it will never
scheduled again, because pick_next_task_stop() ignores all tasks with a
state != TASK_RUNNING:

static struct task_struct *pick_next_task_stop(struct rq *rq)
{
	struct task_struct *stop = rq->stop;

	if (stop && stop->state == TASK_RUNNING)
		return stop;

	return NULL;
}

At least I have two dumps of machines where all cpus but one are running
the migration thread. Only the last one runs usual user space processes but
has a preempted migration thread:

>> trace 0x7e50cd40
================================================================
STACK TRACE FOR TASK: 0x7e50cd40 (migration/9)

 STACK:
 0 schedule+1204 [0x55f754]
 1 preempt_schedule+102 [0x56031e]
 2 _raw_spin_unlock_irq+118 [0x563fa6]
 3 cpu_stopper_thread+144 [0x1ad854]
 4 kthread+166 [0x1685aa]
 5 kernel_thread_starter+6 [0x106bea]
================================================================

struct task_struct {
        state = 0x1  /* TASK_INTERRUPTIBLE */
...

I would guess something like the below would probably fix it.
Does that make any sense or did I miss something obvious?

diff --git a/kernel/sched_stoptask.c b/kernel/sched_stoptask.c
index 45bddc0..d5cf344 100644
--- a/kernel/sched_stoptask.c
+++ b/kernel/sched_stoptask.c
@@ -26,7 +26,8 @@ static struct task_struct *pick_next_task_stop(struct rq *rq)
 {
 	struct task_struct *stop = rq->stop;
 
-	if (stop && stop->state == TASK_RUNNING)
+	/* got preempted if on_rq == 1 -- ignore */
+	if (stop && ((stop->state == TASK_RUNNING) || stop->se.on_rq))
 		return stop;
 
 	return NULL;

^ permalink raw reply related	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 12:58               ` Heiko Carstens
@ 2010-11-09 13:07                 ` Markus Trippelsdorf
  2010-11-09 13:21                 ` Markus Trippelsdorf
  2010-11-09 13:33                 ` Peter Zijlstra
  2 siblings, 0 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-09 13:07 UTC (permalink / raw)
  To: Heiko Carstens
  Cc: Peter Zijlstra, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Tue, Nov 09, 2010 at 01:58:42PM +0100, Heiko Carstens wrote:
> On Wed, Oct 27, 2010 at 09:36:55PM +0200, Peter Zijlstra wrote:
> > On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> > > 
> > > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > > Date:   Wed Sep 22 13:53:15 2010 +0200
> > > 
> > >     sched: Create special class for stop/migrate work
> > > 
> > >     In order to separate the stop/migrate work thread from the SCHED_FIFO
> > >     implementation, create a special class for it that is of higher priority than
> > >     SCHED_FIFO itself.
> > > 
> > >     This currently solves a problem where cpu-hotplug consumes so much cpu-time
> > >     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
> > >     timer pending on the now dead cpu.
> > > 
> > >     It is also required for when we add the planned deadline scheduling class above
> > >     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> > > 
> > >     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> > >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > >     LKML-Reference: <1285165776.2275.1022.camel@laptop>
> > >     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> > > 
> > > Reverting the commit solves the kvm hang issue.
> > > (If this issue is related to my original tsc problem is of course open for
> > > debate, but I have a strong hunch it is.) 
> > 
> > Too weird,.. what does the hang look like?
> > 
> > Can you generate a sysrq-t dump? The thing I'm looking for is the
> > migration/# thread being runnable but not being current.
> > 
> > How can I reproduce this?
> 
> I think there is a bug in pick_next_task_stop() in sched_stopclass.c:
> 
> If a stop-task scheduling class task (well... the migration thread ;) sets
> its state to TASK_INTERRUPTIBLE and then gets preempted it will never
> scheduled again, because pick_next_task_stop() ignores all tasks with a
> state != TASK_RUNNING:
> ...
> 
> I would guess something like the below would probably fix it.
> Does that make any sense or did I miss something obvious?

You missed the fact that Peter already has a patch that fixes the
problem, but never bothered to post it in this thread.

http://article.gmane.org/gmane.linux.kernel/1058018

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 12:58               ` Heiko Carstens
  2010-11-09 13:07                 ` Markus Trippelsdorf
@ 2010-11-09 13:21                 ` Markus Trippelsdorf
  2010-11-09 13:28                   ` Heiko Carstens
  2010-11-09 13:45                   ` Peter Zijlstra
  2010-11-09 13:33                 ` Peter Zijlstra
  2 siblings, 2 replies; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-09 13:21 UTC (permalink / raw)
  To: Heiko Carstens
  Cc: Peter Zijlstra, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Tue, Nov 09, 2010 at 01:58:42PM +0100, Heiko Carstens wrote:
> On Wed, Oct 27, 2010 at 09:36:55PM +0200, Peter Zijlstra wrote:
> > On Wed, 2010-10-27 at 20:26 +0200, markus@trippelsdorf.de wrote:
> > > 
> > > 34f971f6f7988be4d014eec3e3526bee6d007ffa is the first bad commit
> > > commit 34f971f6f7988be4d014eec3e3526bee6d007ffa
> > > Author: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > > Date:   Wed Sep 22 13:53:15 2010 +0200
> > > 
> > >     sched: Create special class for stop/migrate work
> > > 
> > >     In order to separate the stop/migrate work thread from the SCHED_FIFO
> > >     implementation, create a special class for it that is of higher priority than
> > >     SCHED_FIFO itself.
> > > 
> > >     This currently solves a problem where cpu-hotplug consumes so much cpu-time
> > >     that the SCHED_FIFO class gets throttled, but has the bandwidth replenishment
> > >     timer pending on the now dead cpu.
> > > 
> > >     It is also required for when we add the planned deadline scheduling class above
> > >     SCHED_FIFO, as the stop/migrate thread still needs to transcent those tasks.
> > > 
> > >     Tested-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> > >     Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > >     LKML-Reference: <1285165776.2275.1022.camel@laptop>
> > >     Signed-off-by: Ingo Molnar <mingo@elte.hu>
> > > 
> > > Reverting the commit solves the kvm hang issue.
> > > (If this issue is related to my original tsc problem is of course open for
> > > debate, but I have a strong hunch it is.) 
> > 
> > Too weird,.. what does the hang look like?
> > 
> > Can you generate a sysrq-t dump? The thing I'm looking for is the
> > migration/# thread being runnable but not being current.
> > 
> > How can I reproduce this?
> 
> I think there is a bug in pick_next_task_stop() in sched_stopclass.c:
> 
> If a stop-task scheduling class task (well... the migration thread ;) sets
> its state to TASK_INTERRUPTIBLE and then gets preempted it will never
> scheduled again, because pick_next_task_stop() ignores all tasks with a
> state != TASK_RUNNING:
> ...
> 
> I would guess something like the below would probably fix it.
> Does that make any sense or did I miss something obvious?

You've missed the fact that Peter already has a patch that fixes the
problem, but he never bothered to post it in this thread.

http://article.gmane.org/gmane.linux.kernel/1058018

-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 13:21                 ` Markus Trippelsdorf
@ 2010-11-09 13:28                   ` Heiko Carstens
  2010-11-09 13:45                   ` Peter Zijlstra
  1 sibling, 0 replies; 46+ messages in thread
From: Heiko Carstens @ 2010-11-09 13:28 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Peter Zijlstra, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Tue, Nov 09, 2010 at 02:21:38PM +0100, Markus Trippelsdorf wrote:
> > I would guess something like the below would probably fix it.
> > Does that make any sense or did I miss something obvious?
> 
> You've missed the fact that Peter already has a patch that fixes the
> problem, but he never bothered to post it in this thread.
> 
> http://article.gmane.org/gmane.linux.kernel/1058018

Ah, great... wasted my time for something that's already fixed. ;)

Thanks for the pointer!

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 12:58               ` Heiko Carstens
  2010-11-09 13:07                 ` Markus Trippelsdorf
  2010-11-09 13:21                 ` Markus Trippelsdorf
@ 2010-11-09 13:33                 ` Peter Zijlstra
  2 siblings, 0 replies; 46+ messages in thread
From: Peter Zijlstra @ 2010-11-09 13:33 UTC (permalink / raw)
  To: Heiko Carstens
  Cc: markus, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Tue, 2010-11-09 at 13:58 +0100, Heiko Carstens wrote:
> Does that make any sense or did I miss something obvious?

I'd write it as a PREEMPT_ACTIVE check, but yes I think you're right.

That said, I also have the below patch queued which should fix up the
wakeup-preemption.


---
Subject: sched: Fixup cross sched_class wakeup preemption
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Sun Oct 31 12:37:04 CET 2010

Instead of dealing with sched classes inside each check_preempt_curr()
implementation, pull out this logic into the generic wakeup preemption
path.

This fixes a hang in KVM (and others) where we are waiting for the
stop machine thread to run..

Tested-by: Marcelo Tosatti <mtosatti@redhat.com>
Tested-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <1288891946.2039.31.camel@laptop>
---
 kernel/sched.c          |   38 +++++++++++++++++++++++++++-----------
 kernel/sched_fair.c     |    6 ------
 kernel/sched_stoptask.c |    2 +-
 3 files changed, 28 insertions(+), 18 deletions(-)

Index: linux-2.6/kernel/sched.c
===================================================================
--- linux-2.6.orig/kernel/sched.c
+++ linux-2.6/kernel/sched.c
@@ -560,18 +560,8 @@ struct rq {
 
 static DEFINE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
 
-static inline
-void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
-{
-	rq->curr->sched_class->check_preempt_curr(rq, p, flags);
 
-	/*
-	 * A queue event has occurred, and we're going to schedule.  In
-	 * this case, we can save a useless back to back clock update.
-	 */
-	if (test_tsk_need_resched(p))
-		rq->skip_clock_update = 1;
-}
+static void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags);
 
 static inline int cpu_of(struct rq *rq)
 {
@@ -9402,4 +9392,30 @@ void synchronize_sched_expedited(void)
 }
 EXPORT_SYMBOL_GPL(synchronize_sched_expedited);
 
+static void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
+{
+	const struct sched_class *class;
+
+	if (p->sched_class == rq->curr->sched_class) {
+		rq->curr->sched_class->check_preempt_curr(rq, p, flags);
+	} else {
+		for_each_class(class) {
+			if (class == rq->curr->sched_class)
+				break;
+			if (class == p->sched_class) {
+				resched_task(rq->curr);
+				break;
+			}
+		}
+	}
+
+	/*
+	 * A queue event has occurred, and we're going to schedule.  In
+	 * this case, we can save a useless back to back clock update.
+	 */
+	if (test_tsk_need_resched(rq->curr))
+		rq->skip_clock_update = 1;
+}
+
+
 #endif /* #else #ifndef CONFIG_SMP */
Index: linux-2.6/kernel/sched_fair.c
===================================================================
--- linux-2.6.orig/kernel/sched_fair.c
+++ linux-2.6/kernel/sched_fair.c
@@ -1654,12 +1654,6 @@ static void check_preempt_wakeup(struct 
 	struct cfs_rq *cfs_rq = task_cfs_rq(curr);
 	int scale = cfs_rq->nr_running >= sched_nr_latency;
 
-	if (unlikely(rt_prio(p->prio)))
-		goto preempt;
-
-	if (unlikely(p->sched_class != &fair_sched_class))
-		return;
-
 	if (unlikely(se == pse))
 		return;
 
Index: linux-2.6/kernel/sched_stoptask.c
===================================================================
--- linux-2.6.orig/kernel/sched_stoptask.c
+++ linux-2.6/kernel/sched_stoptask.c
@@ -19,7 +19,7 @@ select_task_rq_stop(struct rq *rq, struc
 static void
 check_preempt_curr_stop(struct rq *rq, struct task_struct *p, int flags)
 {
-	resched_task(rq->curr); /* we preempt everything */
+	/* we're never preempted */
 }
 
 static struct task_struct *pick_next_task_stop(struct rq *rq)


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 13:21                 ` Markus Trippelsdorf
  2010-11-09 13:28                   ` Heiko Carstens
@ 2010-11-09 13:45                   ` Peter Zijlstra
  1 sibling, 0 replies; 46+ messages in thread
From: Peter Zijlstra @ 2010-11-09 13:45 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Heiko Carstens, john stultz, Thomas Gleixner, Borislav Petkov,
	linux-kernel, hpa, Ingo Molnar, Andreas Herrmann, avi, mtosatti

On Tue, 2010-11-09 at 14:21 +0100, Markus Trippelsdorf wrote:
> 
> You've missed the fact that Peter already has a patch that fixes the
> problem, but he never bothered to post it in this thread.
> 
Right, forgot about that, sorry.

Still Heiko has a good point, I've queued the below after I realized
that PREEMPT_ACTIVE would still require two tests and the on_rq bit is
actually sufficient.

---
Subject: sched: Fix runnable condition for stoptask
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
Date: Tue Nov 09 14:36:52 CET 2010

Heiko reported that the TASK_RUNNING check is not sufficient for
CONFIG_PREEMPT=y since we can get preempted with !TASK_RUNNING.

He suggested adding a ->se.on_rq test to the existing TASK_RUNNING
one, however TASK_RUNNING will always have ->se.on_rq, so we might as
well reduce that to a single test.

Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <new-submission>
---
 kernel/sched_stoptask.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: linux-2.6/kernel/sched_stoptask.c
===================================================================
--- linux-2.6.orig/kernel/sched_stoptask.c
+++ linux-2.6/kernel/sched_stoptask.c
@@ -26,7 +26,7 @@ static struct task_struct *pick_next_tas
 {
 	struct task_struct *stop = rq->stop;
 
-	if (stop && stop->state == TASK_RUNNING)
+	if (stop && stop->se.on_rq)
 		return stop;
 
 	return NULL;


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-05 21:32                                     ` Borislav Petkov
@ 2010-11-09 14:02                                       ` Thomas Gleixner
  2010-11-09 14:39                                         ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Thomas Gleixner @ 2010-11-09 14:02 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Markus Trippelsdorf, Borislav Petkov, john stultz, linux-kernel,
	hpa, Ingo Molnar, Herrmann3, Andreas, heiko.carstens,
	a.p.zijlstra, avi, mtosatti

On Fri, 5 Nov 2010, Borislav Petkov wrote:
> On Fri, Nov 05, 2010 at 05:27:49PM -0400, Markus Trippelsdorf wrote:
> > Err, 20==0x14 so it's not that much difference and it's running fine here
> > also.
> > 
> > Feel free to add:
> >     Reported-and-tested-by: Markus Trippelsdorf <markus@trippelsdorf.de>
> 
> Thanks Markus,
> 
> actually your board is not what concerns me, 20 ticks is still ok, more
> or less, but there are other machines which contain absurd values in
> there like 0x37ee or 0x1000 (a Broadcom chipset). We'll need to give a
> change like that a good run before we can be absolutely sure it doesn't
> break any machines.

If the ACPI entry is known to be flaky, shouldn't we simply err out on
the safe side and use 128 ticks in any case, which is not a really big
deal.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 14:02                                       ` Thomas Gleixner
@ 2010-11-09 14:39                                         ` Borislav Petkov
  2010-11-17 23:21                                           ` Markus Trippelsdorf
  0 siblings, 1 reply; 46+ messages in thread
From: Borislav Petkov @ 2010-11-09 14:39 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Markus Trippelsdorf, Borislav Petkov, john stultz, linux-kernel,
	hpa, Ingo Molnar, Herrmann3, Andreas, heiko.carstens,
	a.p.zijlstra, avi, mtosatti

On Tue, Nov 09, 2010 at 09:02:13AM -0500, Thomas Gleixner wrote:
> > actually your board is not what concerns me, 20 ticks is still ok, more
> > or less, but there are other machines which contain absurd values in
> > there like 0x37ee or 0x1000 (a Broadcom chipset). We'll need to give a
> > change like that a good run before we can be absolutely sure it doesn't
> > break any machines.
> 
> If the ACPI entry is known to be flaky, shouldn't we simply err out on
> the safe side and use 128 ticks in any case, which is not a really big
> deal.

Yep, this is what my proposed fix does. I set it by default to 0x80 and
the hpet detection code in acpi_parse_hpet() overrides it if it is less
than that (and obviously a sensible value written by the BIOS).

Otherwise it issues a warning. Come to think of it, we shouldn't be
issuing a warning because this'll scream on a very high number of
systems, IMHO, especially older boards. Instead, we should issue it in
dmesg during boot just in case.

The other concern I have is whether min tick of 128 would work for _all_
possible HPET implementations - I don't know whether there are some very
b0rked incarnations which delay HPET accesses to more than 128 cycles.
Kinda hard to say.

So, to be more specific, here's what I have in mind:

--
From: Borislav Petkov <borislav.petkov@amd.com>
Date: Thu, 4 Nov 2010 00:01:49 +0100
Subject: [PATCH] HPET: Fix HPET readout for small deltas

Some HPET implementations might require longer delay when accessing the
HPET than what 995bd3bb5c78f3ff71339803c0b8337ed36d64fb established (8
cycles). Generally, the proper value should be programmed by BIOS and
written into the ACPI HPET table as the main counter minimum tick in
periodic mode (offset 53).

We assume that value as the minimum value a delta can be in order to
reprogram the HPET successfully. For BIOSen which contain crap, we fall
back to a default value of 128 cycles which should be sensible on all
more or less sane HPET implementations.

LKML-Reference: <20101026112052.GA1672@arch.trippelsdorf.de>
Not-Yet-Signed-off-by: Borislav Petkov <borislav.petkov@amd.com>
---
 arch/x86/include/asm/hpet.h |    1 +
 arch/x86/kernel/acpi/boot.c |   10 +++++++---
 arch/x86/kernel/hpet.c      |    3 ++-
 3 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/hpet.h b/arch/x86/include/asm/hpet.h
index 2c392d6..01d9480 100644
--- a/arch/x86/include/asm/hpet.h
+++ b/arch/x86/include/asm/hpet.h
@@ -68,6 +68,7 @@ extern unsigned long force_hpet_address;
 extern u8 hpet_blockid;
 extern int hpet_force_user;
 extern u8 hpet_msi_disable;
+extern u16 hpet_min_tick;
 extern int is_hpet_enabled(void);
 extern int hpet_enable(void);
 extern void hpet_disable(void);
diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c
index 71232b9..91a46bc 100644
--- a/arch/x86/kernel/acpi/boot.c
+++ b/arch/x86/kernel/acpi/boot.c
@@ -772,9 +772,6 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
 		hpet_address >>= 32;
 	}
 #endif
-	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
-	       hpet_tbl->id, hpet_address);
-
 	/*
 	 * Allocate and initialize the HPET firmware resource for adding into
 	 * the resource tree during the lateinit timeframe.
@@ -790,6 +787,13 @@ static int __init acpi_parse_hpet(struct acpi_table_header *table)
 	hpet_res->start = hpet_address;
 	hpet_res->end = hpet_address + (1 * 1024) - 1;
 
+	/* Accept only sensible values written by BIOS */
+	if (hpet_tbl->minimum_tick < hpet_min_tick)
+		hpet_min_tick = hpet_tbl->minimum_tick;
+
+	printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx min tick: %d\n",
+		hpet_tbl->id, hpet_address, hpet_min_tick);
+
 	return 0;
 }
 
diff --git a/arch/x86/kernel/hpet.c b/arch/x86/kernel/hpet.c
index ae03cab..a0b790a 100644
--- a/arch/x86/kernel/hpet.c
+++ b/arch/x86/kernel/hpet.c
@@ -40,6 +40,7 @@ u8					hpet_msi_disable;
 static unsigned long			hpet_num_timers;
 #endif
 static void __iomem			*hpet_virt_address;
+u16					hpet_min_tick = 0x80;
 
 struct hpet_dev {
 	struct clock_event_device	evt;
@@ -408,7 +409,7 @@ static int hpet_next_event(unsigned long delta,
 	 */
 	res = (s32)(cnt - hpet_readl(HPET_COUNTER));
 
-	return res < 8 ? -ETIME : 0;
+	return res < hpet_min_tick ? -ETIME : 0;
 }
 
 static void hpet_legacy_set_mode(enum clock_event_mode mode,
-- 
1.7.3.1.50.g1e633




-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply related	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-09 14:39                                         ` Borislav Petkov
@ 2010-11-17 23:21                                           ` Markus Trippelsdorf
  2010-11-24 11:25                                             ` Borislav Petkov
  0 siblings, 1 reply; 46+ messages in thread
From: Markus Trippelsdorf @ 2010-11-17 23:21 UTC (permalink / raw)
  To: Borislav Petkov
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On 2010.11.09 at 15:39 +0100, Borislav Petkov wrote:
> On Tue, Nov 09, 2010 at 09:02:13AM -0500, Thomas Gleixner wrote:
> > > actually your board is not what concerns me, 20 ticks is still ok, more
> > > or less, but there are other machines which contain absurd values in
> > > there like 0x37ee or 0x1000 (a Broadcom chipset). We'll need to give a
> > > change like that a good run before we can be absolutely sure it doesn't
> > > break any machines.
> > 
> > If the ACPI entry is known to be flaky, shouldn't we simply err out on
> > the safe side and use 128 ticks in any case, which is not a really big
> > deal.
> 
> Yep, this is what my proposed fix does. I set it by default to 0x80 and
> the hpet detection code in acpi_parse_hpet() overrides it if it is less
> than that (and obviously a sensible value written by the BIOS).
> 
> Otherwise it issues a warning. Come to think of it, we shouldn't be
> issuing a warning because this'll scream on a very high number of
> systems, IMHO, especially older boards. Instead, we should issue it in
> dmesg during boot just in case.
> 
> The other concern I have is whether min tick of 128 would work for _all_
> possible HPET implementations - I don't know whether there are some very
> b0rked incarnations which delay HPET accesses to more than 128 cycles.
> Kinda hard to say.
> 
> So, to be more specific, here's what I have in mind:

Any update on this issue?

Borislav's patch solves the mysterious "slowdown" problem and is running
without problems for the last two weeks here. 

(And rc2 is already out. So maybe it's time to push this to Linus so
that more people have a chance to test it?)
-- 
Markus

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [bisected] Clocksource tsc unstable git
  2010-11-17 23:21                                           ` Markus Trippelsdorf
@ 2010-11-24 11:25                                             ` Borislav Petkov
  0 siblings, 0 replies; 46+ messages in thread
From: Borislav Petkov @ 2010-11-24 11:25 UTC (permalink / raw)
  To: Markus Trippelsdorf
  Cc: Thomas Gleixner, Borislav Petkov, john stultz, linux-kernel, hpa,
	Ingo Molnar, Herrmann3, Andreas, heiko.carstens, a.p.zijlstra,
	avi, mtosatti

On Thu, Nov 18, 2010 at 12:21:20AM +0100, Markus Trippelsdorf wrote:
> Any update on this issue?
> 
> Borislav's patch solves the mysterious "slowdown" problem and is running
> without problems for the last two weeks here. 
> 
> (And rc2 is already out. So maybe it's time to push this to Linus so
> that more people have a chance to test it?)

Yep, we definitely need some kind of fix for this. It just triggered on
me on -rc3 (system became sluggish, everything advanced only after a
keypress along with the already familiar clocksource unstable message in
dmesg).

Thomas, want me to rebase the patch ontop of tip:master?

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
General Managers: Alberto Bozzo, Andrew Bowd
Registration: Dornach, Gemeinde Aschheim, Landkreis Muenchen
Registergericht Muenchen, HRB Nr. 43632

^ permalink raw reply	[flat|nested] 46+ messages in thread

end of thread, other threads:[~2010-11-24 11:25 UTC | newest]

Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-10-26 11:20 Clocksource tsc unstable (delta = -8589361717 ns) in current git Markus Trippelsdorf
2010-10-26 13:18 ` Borislav Petkov
2010-10-26 13:58   ` Markus Trippelsdorf
2010-10-26 14:05     ` Markus Trippelsdorf
2010-10-26 14:38     ` Peter Zijlstra
2010-10-26 15:46       ` Thomas Gleixner
2010-10-26 16:41         ` Borislav Petkov
2010-10-26 15:48     ` Thomas Gleixner
2010-10-26 17:56       ` Markus Trippelsdorf
2010-10-26 18:19         ` Borislav Petkov
2010-10-26 19:18       ` john stultz
2010-10-27 14:26         ` Markus Trippelsdorf
2010-10-27 18:26           ` [bisected] Clocksource tsc unstable git markus
2010-10-27 19:36             ` Peter Zijlstra
2010-10-27 19:42               ` Markus Trippelsdorf
2010-10-27 19:58               ` Marcelo Tosatti
2010-11-09 12:58               ` Heiko Carstens
2010-11-09 13:07                 ` Markus Trippelsdorf
2010-11-09 13:21                 ` Markus Trippelsdorf
2010-11-09 13:28                   ` Heiko Carstens
2010-11-09 13:45                   ` Peter Zijlstra
2010-11-09 13:33                 ` Peter Zijlstra
2010-10-29  8:18             ` Borislav Petkov
2010-10-29  8:30               ` Markus Trippelsdorf
2010-10-29 10:27                 ` Borislav Petkov
2010-10-29 11:34                   ` Markus Trippelsdorf
2010-10-29 11:39                     ` Borislav Petkov
2010-10-29 11:54                       ` Markus Trippelsdorf
2010-10-29 13:13                         ` Borislav Petkov
2010-10-29 12:14                   ` Thomas Gleixner
2010-10-29 17:00                     ` Borislav Petkov
2010-10-29 17:26                       ` Markus Trippelsdorf
2010-11-01 18:05                         ` Thomas Gleixner
2010-11-01 18:45                           ` Markus Trippelsdorf
2010-11-02 15:26                             ` Thomas Gleixner
2010-11-05 16:09                               ` Borislav Petkov
2010-11-05 16:42                                 ` Markus Trippelsdorf
2010-11-05 17:45                                   ` Markus Trippelsdorf
2010-11-05 21:27                                   ` Markus Trippelsdorf
2010-11-05 21:32                                     ` Borislav Petkov
2010-11-09 14:02                                       ` Thomas Gleixner
2010-11-09 14:39                                         ` Borislav Petkov
2010-11-17 23:21                                           ` Markus Trippelsdorf
2010-11-24 11:25                                             ` Borislav Petkov
2010-10-29 17:34                       ` Thomas Gleixner
2010-10-27  3:24     ` Clocksource tsc unstable (delta = -8589361717 ns) in current git Mike Galbraith

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).