* XFS kernal crash
@ 2014-09-23 12:53 Thomas Stagge
0 siblings, 0 replies; only message in thread
From: Thomas Stagge @ 2014-09-23 12:53 UTC (permalink / raw)
To: xfs
[-- Attachment #1.1: Type: text/plain, Size: 12520 bytes --]
Hello,
we have in the last few weeks always problems with the XFS file system.
XFS: possible memory allocation deadlock in kmem_alloc (mode: 0x250)
After a certain time there is a kernel crash which looks as follows:
September 23 10:17:57 hostname kernel: [1282081.010814] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:17:59 hostname kernel: [1282082.913437] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:01 hostname kernel: [1282084.908582] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:03 hostname kernel: [1282086.691791] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:05 hostname kernel: [1282088.691319] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:07 hostname kernel: [1282090.690182] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:09 hostname kernel: [1282092.673355] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:11 hostname kernel: [1282094.596504] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:13 hostname kernel: [1282096.595971] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:15 hostname kernel: [1282098.595301] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:17 hostname kernel: [1282100.594232] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:19 hostname kernel: [1282102.433262] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:21 hostname kernel: [1282104.412432] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:23 hostname kernel: [1282106.411598] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:25 hostname kernel: [1282108.310740] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:27 hostname kernel: [1282110.289892] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:29 hostname kernel: [1282112.289106] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:31 hostname kernel: [1282114.288247] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:33 hostname kernel: [1282116.215953] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:35 hostname kernel: [1282118.198488] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:37 hostname kernel: [1282120.193686] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:39 hostname kernel: [1282122.288716] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:41 hostname kernel: [1282124.215970] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:43 hostname kernel: [1282126.307031] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:45 hostname kernel: [1282128.306190] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:47 hostname kernel: [1282130.405310] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:49 hostname kernel: [1282132.076611] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:51 hostname kernel: [1282134.027790] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:53 hostname kernel: [1282136.122911] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:55 hostname kernel: [1282138.218032] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:57 hostname kernel: [1282140.145365] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:18:59 hostname kernel: [1282142.140528] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:19:01 hostname kernel: [1282144.091708] XFS: possible
memory allocation deadlock in kmem_alloc (mode: 0x250)
September 23 10:19:01 hostname kernel: [1282144.223485] INFO: task
kswapd0: 87 blocked for more than 120 seconds.
September 23 10:19:01 hostname kernel: [1282144.224156] Not tainted
3.13.0-34-generic # 60-Ubuntu ~ precise1
September 23 10:19:01 hostname kernel: [1282144.224837] "echo 0> / proc
/ sys / kernel / hung_task_timeout_secs" disables this message.
September 23 10:19:01 hostname kernel: [1282144.225597] kswapd0 D
ffff88031662a020 0 87 2 0x00000000
September 23 10:19:01 hostname kernel: [1282144.225609] ffff88019500d8f8
0000000000000002 ffff880199d94440 ffff88019500dfd8
September 23 10:19:01 hostname kernel: [1282144.225621] 0000000000014440
0000000000014440 ffff880316cd2fe0 ffff880316f3afe0
September 23 10:19:01 hostname kernel: [1282144.225629] ffff88019500d970
ffff880314119128 ffff88031411912c 00000000ffffffff
September 23 10:19:01 hostname kernel: [1282144.225638] Call Trace:
September 23 10:19:01 hostname kernel: [1282144.225658]
[<ffffffff8175b8b9>] schedule + 0x29 / 0x70
September 23 10:19:01 hostname kernel: [1282144.225669]
[<ffffffff8175bbde>] schedule_preempt_disabled + 0xe / 0x10
September 23 10:19:01 hostname kernel: [1282144.225680]
[<ffffffff8175da14>] __mutex_lock_slowpath + 0x114 / 0x1b0
September 23 10:19:01 hostname kernel: [1282144.225771]
[<ffffffffa0218291>]? xfs_perag_get_tag + 0x41 / 0xf0 [xfs]
September 23 10:19:01 hostname kernel: [1282144.225781]
[<ffffffff8175dad3>] mutex_lock + 0x23 / 0x37
September 23 10:19:01 hostname kernel: [1282144.225832]
[<ffffffffa01c7540>] xfs_reclaim_inodes_ag + 0x2e0 / 0x3a0 [xfs]
September 23 10:19:01 hostname kernel: [1282144.225843]
[<ffffffff8101d0b9>]? sched_clock + 0x9 / 0x10
September 23 10:19:01 hostname kernel: [1282144.225852]
[<ffffffff81042c9b>]? native_smp_send_reschedule + 0x4b / 0x60
September 23 10:19:01 hostname kernel: [1282144.225862]
[<ffffffff8109cf7d>]? ttwu_queue + 0x8D / 0xD0
September 23 10:19:01 hostname kernel: [1282144.225871]
[<ffffffff8109f860>]? try_to_wake_up + 0x190 / 0x210
September 23 10:19:01 hostname kernel: [1282144.225879]
[<ffffffff8109f947>]? wake_up_process + 0x27 / 0x50
uname -a
hostname 3.13.0-34-generic # 60-Ubuntu SMP Wed precise1 ~ August 13
15:55:33 UTC 2014 x86_64 x86_64 x86_64 GNU / Linux
xfs_repair -V
xfs_repair Version 3.1.7
7 CPUs -> cat /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 16
model : 4
model name : Quad-Core AMD Opteron(tm) Processor 2376
stepping : 2
microcode : 0x1000086
cpu MHz : 800.000
cache size : 512 KB
physical id : 0
siblings : 4
core id : 0
cpu cores : 4
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext
fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl
nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm
extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit
wdt hw_pstate npt lbrv svm_lock nrip_save
bogomips : 4600.18
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 48 bits physical, 48 bits virtual
power management: ts ttp tm stc 100mhzsteps hwpstate
cat /proc/meminfo
MemTotal: 12304884 kB
MemFree: 151816 kB
Buffers: 1076 kB
Cached: 10328496 kB
SwapCached: 0 kB
Active: 2658232 kB
Inactive: 7766540 kB
Active(anon): 42644 kB
Inactive(anon): 53636 kB
Active(file): 2615588 kB
Inactive(file): 7712904 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 4878268 kB
SwapFree: 4878268 kB
Dirty: 1220 kB
Writeback: 0 kB
AnonPages: 95728 kB
Mapped: 32208 kB
Shmem: 352 kB
Slab: 1412904 kB
SReclaimable: 1188188 kB
SUnreclaim: 224716 kB
KernelStack: 3512 kB
PageTables: 28816 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 11030708 kB
Committed_AS: 949792 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 310312 kB
VmallocChunk: 34353030140 kB
HardwareCorrupted: 0 kB
AnonHugePages: 12288 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 75520 kB
DirectMap2M: 3069952 kB
DirectMap1G: 9437184 kB
cat /proc/mounts
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,relatime,size=6108796k,nr_inodes=1527199,mode=755 0 0
devpts /dev/pts devpts
rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,noexec,relatime,size=1230492k,mode=755 0 0
/dev/disk/by-uuid/d6821e50-e82b-448d-bc03-9c286ef59c8f / xfs
rw,relatime,attr2,inode64,noquota 0 0
none /sys/fs/fuse/connections fusectl rw,relatime 0 0
none /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/kernel/security securityfs rw,relatime 0 0
none /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0
none /run/shm tmpfs rw,nosuid,nodev,relatime 0 0
/dev/sdc1 /extern/data0 xfs rw,relatime,attr2,inode64,noquota 0 0
rpc_pipefs /run/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
cat /proc/partitions
major minor #blocks name
11 0 1048575 sr0
8 0 245175336 sda
8 1 97654784 sda1
8 2 4882432 sda2
8 16 245175336 sdb
8 17 97654784 sdb1
8 18 4882432 sdb2
9 2 97589120 md2
9 1 4878272 md1
8 32 27345461248 sdc
8 33 27345461214 sdc1
xfs_info /dev/md2
Metadaten =/dev/disk/by-uuid/d6821e50-e82b-448d-bc03-9c286ef59c8f
isize=256 agcount=4, agsize=6099320 blks
= sectsz=512 attr=2
Daten = bsize=4096 Blöcke=24397280, imaxpct=25
= sunit=0 swidth=0 blks
Benennung =Version 2 bsize=4096 ascii-ci=0
Protokoll =Intern bsize=4096 Blöcke=11912, Version=2
= sectsz=512 sunit=0 blks, lazy-count=1
Echtzeit =keine extsz=4096 Blöcke=0, rtextents=0
xfs_info /dev/md2
Metadaten =/dev/disk/by-uuid/d6821e50-e82b-448d-bc03-9c286ef59c8f
isize=256 agcount=4, agsize=6099320 blks
= sectsz=512 attr=2
Daten = bsize=4096 Blöcke=24397280, imaxpct=25
= sunit=0 swidth=0 blks
Benennung =Version 2 bsize=4096 ascii-ci=0
Protokoll =Intern bsize=4096 Blöcke=11912, Version=2
= sectsz=512 sunit=0 blks, lazy-count=1
Echtzeit =keine extsz=4096 Blöcke=0, rtextents=0
xfs_info /dev/sdc1
Metadaten =/dev/sdc1 isize=256 agcount=26,
agsize=268435455 blks
= sectsz=512 attr=2
Daten = bsize=4096 Blöcke=6836365303, imaxpct=5
= sunit=0 swidth=0 blks
Benennung =Version 2 bsize=4096 ascii-ci=0
Protokoll =Intern bsize=4096 Blöcke=32768, Version=2
= sectsz=512 sunit=0 blks, lazy-count=0
Echtzeit =keine extsz=4096 Blöcke=0, rtextents=0
Can you help us?
Best regards,
--
Mit freundlichen Grüssen,
Thomas Stagge
--------------------------------------
Thomas Stagge, Dipl.Ing.(FH)
System Administrator
OPTIMARE Systems GmbH
Am Luneort 15a
27572 Bremerhaven, Germany
T: +49 471 48361-27
F: +49 471 48361-11
M: +49 172 8698575
thomas.stagge@optimare.de
Internet: http://www.optimare.de
CEO: Dipl.-Ing. Hans J. Stahl; Sitz der Gesellschaft: Bremerhaven; Registergericht: Amtsgericht Bremen HRB 28618 HB
[-- Attachment #1.2: Type: text/html, Size: 58857 bytes --]
[-- Attachment #2: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2014-09-23 12:54 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-23 12:53 XFS kernal crash Thomas Stagge
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.