If you run LTP Memory CGROUP Controller functional test on linux-2.6.36-git7, the following Backtrace, OOMKill & rcu_sched_state detected stall jiffies are created. The machine is not reachable thereafter. Ways to reproduce this problem: 1) Build and boot kernel 2.6.36-git7 on Power7 machine with attached config file, 2) Fetch, build and install LTP: git clone git://ltp.git.sourceforge.net/gitroot/ltp/ltp cd ltp ./configure make make install 3) Create a LTP runtest file /opt/ltp/runtest/memcg_function_test with the following entry: memcg_function memcg_function_test.sh <> cd /opt/ltp ./runltp -f memcg_function_test Regards-- Subrata Memory Cgroup Controller Function test results: ========================================================== memcg_test_1 invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_test_1 cpuset=/ mems_allowed=0 Call Trace: [c00000002065b200] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000002065b2b0] [c000000000156d8c] .dump_header+0xac/0x220 [c00000002065b390] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000002065b470] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000002065b520] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000002065b610] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000002065b760] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000002065b810] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c00000002065b900] [c00000000017dc4c] .__get_user_pages+0x1bc/0x500 [c00000002065ba00] [c000000000180b2c] .__mlock_vma_pages_range+0xdc/0x240 [c00000002065bb50] [c0000000001814b8] .mlock_vma_pages_range+0xb8/0xc0 [c00000002065bbe0] [c000000000184af8] .mmap_region+0x358/0x5c0 [c00000002065bcf0] [c0000000001851f4] .SyS_mmap_pgoff+0x94/0x1a0 [c00000002065bdb0] [c00000000000cb54] .sys_mmap+0x74/0x90 [c00000002065be30] [c000000000008564] syscall_exit+0x0/0x40 Task in /0 killed as a result of limit of /0 memory: usage 0kB, limit 0kB, failcnt 2548 memory+swap: usage 0kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 5 CPU 1: hi: 6, btch: 1 usd: 0 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 3 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 3 CPU 6: hi: 6, btch: 1 usd: 4 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 0 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 1 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 5 CPU 15: hi: 6, btch: 1 usd: 3 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 0 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 1 CPU 20: hi: 6, btch: 1 usd: 5 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 5 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:390 inactive_anon:1073 isolated_anon:0 active_file:1694 inactive_file:1438 isolated_file:0 unevictable:0 dirty:23 writeback:0 unstable:0 free:54998 slab_reclaimable:580 slab_unreclaimable:2929 mapped:236 shmem:1 pagetables:174 bounce:0 Node 0 DMA free:3519872kB min:8128kB low:10112kB high:12160kB active_anon:24960kB inactive_anon:68672kB active_file:108416kB inactive_file:92032kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:1472kB writeback:0kB mapped:15104kB shmem:64kB slab_reclaimable:37120kB slab_unreclaimable:187456kB kernel_stack:4112kB pagetables:11136kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 162*64kB 91*128kB 480*256kB 433*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3520512kB 4258 total pagecache pages 1143 pages in swap cache Swap cache stats: add 64700, delete 63557, find 39392/46116 Free swap = 3974720kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3183 pages shared 6337 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [16827] 0 16827 61 25 8 0 0 memcg_test_1 [16828] 0 16828 60 24 0 0 0 memcg_test_1 Memory cgroup out of memory: Kill process 16827 (memcg_test_1) score 1000 or sacrifice child Killed process 16827 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16828 (memcg_test_1) score 1000 or sacrifice child Killed process 16828 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16829 (memcg_test_1) score 1000 or sacrifice child Killed process 16829 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16830 (memcg_test_1) score 1000 or sacrifice child Killed process 16830 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16831 (memcg_test_1) score 1000 or sacrifice child Killed process 16831 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16832 (memcg_test_1) score 1000 or sacrifice child Killed process 16832 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16833 (memcg_test_1) score 1000 or sacrifice child Killed process 16833 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16834 (memcg_test_1) score 1000 or sacrifice child Killed process 16834 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16835 (memcg_test_1) score 1000 or sacrifice child Killed process 16835 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16836 (memcg_test_1) score 1000 or sacrifice child Killed process 16836 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16837 (memcg_test_1) score 1000 or sacrifice child Killed process 16837 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16838 (memcg_test_1) score 1000 or sacrifice child Killed process 16838 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16839 (memcg_test_1) score 1000 or sacrifice child Killed process 16839 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16840 (memcg_test_1) score 1000 or sacrifice child Killed process 16840 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16841 (memcg_test_1) score 1000 or sacrifice child Killed process 16841 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16842 (memcg_test_1) score 1000 or sacrifice child Killed process 16842 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16843 (memcg_test_1) score 1000 or sacrifice child Killed process 16843 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16844 (memcg_test_1) score 1000 or sacrifice child Killed process 16844 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16845 (memcg_test_1) score 1000 or sacrifice child Killed process 16845 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16846 (memcg_test_1) score 1000 or sacrifice child Killed process 16846 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16847 (memcg_test_1) score 1000 or sacrifice child Killed process 16847 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16848 (memcg_test_1) score 1000 or sacrifice child Killed process 16848 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16849 (memcg_test_1) score 1000 or sacrifice child Killed process 16849 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16850 (memcg_test_1) score 1000 or sacrifice child Killed process 16850 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16851 (memcg_test_1) score 1000 or sacrifice child Killed process 16851 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16852 (memcg_test_1) score 1000 or sacrifice child Killed process 16852 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16853 (memcg_test_1) score 1000 or sacrifice child Killed process 16853 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16854 (memcg_test_1) score 1000 or sacrifice child Killed process 16854 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16855 (memcg_test_1) score 1000 or sacrifice child Killed process 16855 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16856 (memcg_test_1) score 1000 or sacrifice child Killed process 16856 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16857 (memcg_test_1) score 1000 or sacrifice child Killed process 16857 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16858 (memcg_test_1) score 1000 or sacrifice child Killed process 16858 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16859 (memcg_test_1) score 1000 or sacrifice child Killed process 16859 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16860 (memcg_test_1) score 1000 or sacrifice child Killed process 16860 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16861 (memcg_test_1) score 1000 or sacrifice child Killed process 16861 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16862 (memcg_test_1) score 1000 or sacrifice child Killed process 16862 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB Memory cgroup out of memory: Kill process 16863 (memcg_test_1) score 1000 or sacrifice child Killed process 16863 (memcg_test_1) total-vm:3904kB, anon-rss:576kB, file-rss:1024kB oom_kill_process: 36 callbacks suppressed swapoff invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 swapoff cpuset=/ mems_allowed=0 Call Trace: [c00000005f49b560] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000005f49b610] [c000000000156d8c] .dump_header+0xac/0x220 [c00000005f49b6f0] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000005f49b7d0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000005f49b880] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000005f49b970] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000005f49bac0] [c0000000001b88f4] .mem_cgroup_try_charge_swapin+0xc4/0x130 [c00000005f49bb60] [c000000000193bf0] .unuse_mm+0x340/0x520 [c00000005f49bca0] [c0000000001941ec] .SyS_swapoff+0x41c/0xcd0 [c00000005f49be30] [c000000000008564] syscall_exit+0x0/0x40 Task in /0 killed as a result of limit of /0 memory: usage 40960kB, limit 40960kB, failcnt 72 memory+swap: usage 105152kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 1 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 1 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 3 CPU 9: hi: 6, btch: 1 usd: 1 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 3 CPU 12: hi: 6, btch: 1 usd: 1 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 1 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 3 CPU 17: hi: 6, btch: 1 usd: 5 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 5 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 5 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:648 inactive_anon:1686 isolated_anon:0 active_file:1697 inactive_file:1439 isolated_file:0 unevictable:0 dirty:0 writeback:0 unstable:0 free:54033 slab_reclaimable:580 slab_unreclaimable:2921 mapped:235 shmem:7 pagetables:179 bounce:0 Node 0 DMA free:3458112kB min:8128kB low:10112kB high:12160kB active_anon:41472kB inactive_anon:107904kB active_file:108608kB inactive_file:92096kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:0kB writeback:0kB mapped:15040kB shmem:448kB slab_reclaimable:37120kB slab_unreclaimable:186944kB kernel_stack:4000kB pagetables:11456kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 123*64kB 83*128kB 246*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3457600kB 4504 total pagecache pages 1321 pages in swap cache Swap cache stats: add 66495, delete 65174, find 39409/46134 Free swap = -170496kB Total swap = 0kB 65536 pages RAM 1556 pages reserved 3229 pages shared 7268 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [16897] 0 16897 1614 27 9 0 0 sleep [16914] 0 16914 1643 65 0 0 0 memcg_test_4.sh [16916] 0 16916 1660 346 11 0 0 memcg_test_4 [16920] 0 16920 1620 40 8 0 0 swapoff Memory cgroup out of memory: Kill process 16916 (memcg_test_4) score 1000 or sacrifice child Killed process 16916 (memcg_test_4) total-vm:106240kB, anon-rss:20800kB, file-rss:1344kB swapoff invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 swapoff cpuset=/ mems_allowed=0 Call Trace: [c00000005f49b560] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000005f49b610] [c000000000156d8c] .dump_header+0xac/0x220 [c00000005f49b6f0] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000005f49b7d0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000005f49b880] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000005f49b970] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000005f49bac0] [c0000000001b88f4] .mem_cgroup_try_charge_swapin+0xc4/0x130 [c00000005f49bb60] [c000000000193bf0] .unuse_mm+0x340/0x520 [c00000005f49bca0] [c0000000001941ec] .SyS_swapoff+0x41c/0xcd0 [c00000005f49be30] [c000000000008564] syscall_exit+0x0/0x40 Task in /0 killed as a result of limit of /0 memory: usage 40960kB, limit 40960kB, failcnt 78 memory+swap: usage 105152kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 1 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 1 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 3 CPU 9: hi: 6, btch: 1 usd: 1 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 3 CPU 12: hi: 6, btch: 1 usd: 1 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 1 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 3 CPU 17: hi: 6, btch: 1 usd: 5 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 5 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 5 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:648 inactive_anon:1686 isolated_anon:0 active_file:1697 inactive_file:1439 isolated_file:0 unevictable:0 dirty:0 writeback:0 unstable:0 free:54033 slab_reclaimable:580 slab_unreclaimable:2921 mapped:235 shmem:7 pagetables:179 bounce:0 Node 0 DMA free:3458112kB min:8128kB low:10112kB high:12160kB active_anon:41472kB inactive_anon:107904kB active_file:108608kB inactive_file:92096kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:0kB writeback:0kB mapped:15040kB shmem:448kB slab_reclaimable:37120kB slab_unreclaimable:186944kB kernel_stack:4000kB pagetables:11456kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 123*64kB 83*128kB 246*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3457600kB 4504 total pagecache pages 1321 pages in swap cache Swap cache stats: add 66495, delete 65174, find 39409/46134 Free swap = -170496kB Total swap = 0kB 65536 pages RAM 1556 pages reserved 3229 pages shared 7268 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [16897] 0 16897 1614 27 9 0 0 sleep [16914] 0 16914 1643 65 10 0 0 memcg_test_4.sh [16920] 0 16920 1620 40 8 0 0 swapoff Memory cgroup out of memory: Kill process 16920 (swapoff) score 1000 or sacrifice child Killed process 16920 (swapoff) total-vm:103680kB, anon-rss:640kB, file-rss:1920kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c00000007a86f480] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000007a86f530] [c000000000156d8c] .dump_header+0xac/0x220 [c00000007a86f610] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000007a86f6f0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000007a86f7a0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000007a86f890] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000007a86f9e0] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000007a86fa90] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c00000007a86fb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c00000007a86fe30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /11 killed as a result of limit of /11 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 5 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 3 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 0 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 1 CPU 11: hi: 6, btch: 1 usd: 0 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 5 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 2 CPU 17: hi: 6, btch: 1 usd: 5 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 5 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:441 inactive_anon:1392 isolated_anon:0 active_file:1719 inactive_file:1441 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:54632 slab_reclaimable:580 slab_unreclaimable:2924 mapped:231 shmem:16 pagetables:176 bounce:0 Node 0 DMA free:3496448kB min:8128kB low:10112kB high:12160kB active_anon:28224kB inactive_anon:89088kB active_file:110016kB inactive_file:92224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:14784kB shmem:1024kB slab_reclaimable:37120kB slab_unreclaimable:187136kB kernel_stack:3968kB pagetables:11264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 143*64kB 90*128kB 388*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3496128kB 4182 total pagecache pages 1024 pages in swap cache Swap cache stats: add 66504, delete 65480, find 39419/46147 Free swap = 4008640kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3166 pages shared 6677 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17264] 0 17264 62 33 2 0 0 memcg_process Memory cgroup out of memory: Kill process 17264 (memcg_process) score 1000 or sacrifice child Killed process 17264 (memcg_process) total-vm:3968kB, anon-rss:640kB, file-rss:1472kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000eee4b130] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c0000000eee4b1e0] [c000000000156d8c] .dump_header+0xac/0x220 [c0000000eee4b2c0] [c000000000157488] .oom_kill_process+0x108/0x260 [c0000000eee4b3a0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000eee4b450] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000eee4b540] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c0000000eee4b690] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000eee4b740] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c0000000eee4b7f0] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c0000000eee4b910] [c00000000016be90] .shmem_fault+0x60/0xa0 [c0000000eee4b990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c0000000eee4ba90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c0000000eee4bb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000eee4be30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /12 killed as a result of limit of /12 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 5 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 0 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 0 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 3 CPU 14: hi: 6, btch: 1 usd: 5 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 3 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 0 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:434 inactive_anon:1392 isolated_anon:0 active_file:1720 inactive_file:1441 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:54630 slab_reclaimable:580 slab_unreclaimable:2924 mapped:231 shmem:16 pagetables:172 bounce:0 Node 0 DMA free:3496320kB min:8128kB low:10112kB high:12160kB active_anon:27776kB inactive_anon:89088kB active_file:110080kB inactive_file:92224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:14784kB shmem:1024kB slab_reclaimable:37120kB slab_unreclaimable:187136kB kernel_stack:3952kB pagetables:11008kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 141*64kB 84*128kB 390*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3495744kB 4183 total pagecache pages 1024 pages in swap cache Swap cache stats: add 66504, delete 65480, find 39419/46147 Free swap = 4008640kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3177 pages shared 6676 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17280] 0 17280 62 34 16 0 0 memcg_process Memory cgroup out of memory: Kill process 17280 (memcg_process) score 1000 or sacrifice child Killed process 17280 (memcg_process) total-vm:3968kB, anon-rss:640kB, file-rss:1536kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000079ab0c0] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c0000000079ab170] [c000000000156d8c] .dump_header+0xac/0x220 [c0000000079ab250] [c000000000157488] .oom_kill_process+0x108/0x260 [c0000000079ab330] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000079ab3e0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000079ab4d0] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c0000000079ab620] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000079ab6d0] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c0000000079ab780] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c0000000079ab8a0] [c00000000016be90] .shmem_fault+0x60/0xa0 [c0000000079ab920] [c00000000038a0a4] .shm_fault+0x34/0x50 [c0000000079ab990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c0000000079aba90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c0000000079abb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000079abe30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /13 killed as a result of limit of /13 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 4 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 0 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 3 CPU 14: hi: 6, btch: 1 usd: 0 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 1 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 2 CPU 21: hi: 6, btch: 1 usd: 0 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:434 inactive_anon:1392 isolated_anon:0 active_file:1720 inactive_file:1441 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:54622 slab_reclaimable:580 slab_unreclaimable:2924 mapped:231 shmem:16 pagetables:176 bounce:0 Node 0 DMA free:3495808kB min:8128kB low:10112kB high:12160kB active_anon:27776kB inactive_anon:89088kB active_file:110080kB inactive_file:92224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:14784kB shmem:1024kB slab_reclaimable:37120kB slab_unreclaimable:187136kB kernel_stack:3952kB pagetables:11264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 142*64kB 82*128kB 389*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3495296kB 4183 total pagecache pages 1024 pages in swap cache Swap cache stats: add 66504, delete 65480, find 39419/46147 Free swap = 4008640kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3178 pages shared 6685 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17296] 0 17296 62 33 13 0 0 memcg_process Memory cgroup out of memory: Kill process 17296 (memcg_process) score 1000 or sacrifice child Killed process 17296 (memcg_process) total-vm:3968kB, anon-rss:576kB, file-rss:1536kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c00000007d46b200] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000007d46b2b0] [c000000000156d8c] .dump_header+0xac/0x220 [c00000007d46b390] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000007d46b470] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000007d46b520] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000007d46b610] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000007d46b760] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000007d46b810] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c00000007d46b900] [c00000000017dc4c] .__get_user_pages+0x1bc/0x500 [c00000007d46ba00] [c000000000180b2c] .__mlock_vma_pages_range+0xdc/0x240 [c00000007d46bb50] [c0000000001814b8] .mlock_vma_pages_range+0xb8/0xc0 [c00000007d46bbe0] [c000000000184af8] .mmap_region+0x358/0x5c0 [c00000007d46bcf0] [c0000000001851f4] .SyS_mmap_pgoff+0x94/0x1a0 [c00000007d46bdb0] [c00000000000cb54] .sys_mmap+0x74/0x90 [c00000007d46be30] [c000000000008564] syscall_exit+0x0/0x40 Task in /14 killed as a result of limit of /14 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 4 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 3 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 0 CPU 13: hi: 6, btch: 1 usd: 0 CPU 14: hi: 6, btch: 1 usd: 3 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 3 CPU 17: hi: 6, btch: 1 usd: 2 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 0 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:443 inactive_anon:1394 isolated_anon:0 active_file:1721 inactive_file:1440 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:54625 slab_reclaimable:580 slab_unreclaimable:2924 mapped:231 shmem:18 pagetables:176 bounce:0 Node 0 DMA free:3496000kB min:8128kB low:10112kB high:12160kB active_anon:28352kB inactive_anon:89216kB active_file:110144kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:14784kB shmem:1152kB slab_reclaimable:37120kB slab_unreclaimable:187136kB kernel_stack:3936kB pagetables:11264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 143*64kB 84*128kB 390*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3495872kB 4185 total pagecache pages 1024 pages in swap cache Swap cache stats: add 66504, delete 65480, find 39419/46147 Free swap = 4008640kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3174 pages shared 6679 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17312] 0 17312 62 33 16 0 0 memcg_process Memory cgroup out of memory: Kill process 17312 (memcg_process) score 1000 or sacrifice child Killed process 17312 (memcg_process) total-vm:3968kB, anon-rss:640kB, file-rss:1472kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000079bf2e0] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c0000000079bf390] [c000000000156d8c] .dump_header+0xac/0x220 [c0000000079bf470] [c000000000157488] .oom_kill_process+0x108/0x260 [c0000000079bf550] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000079bf600] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000079bf6f0] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c0000000079bf840] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000079bf8f0] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c0000000079bf9e0] [c00000000017dc4c] .__get_user_pages+0x1bc/0x500 [c0000000079bfae0] [c000000000180b2c] .__mlock_vma_pages_range+0xdc/0x240 [c0000000079bfc30] [c000000000180e50] .mlock_fixup+0x1c0/0x250 [c0000000079bfcf0] [c0000000001811fc] .do_mlock+0x10c/0x160 [c0000000079bfda0] [c00000000018139c] .SyS_mlock+0xbc/0x120 [c0000000079bfe30] [c000000000008564] syscall_exit+0x0/0x40 Task in /15 killed as a result of limit of /15 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 3 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 2 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 2 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 3 CPU 13: hi: 6, btch: 1 usd: 3 CPU 14: hi: 6, btch: 1 usd: 3 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 0 CPU 17: hi: 6, btch: 1 usd: 5 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 5 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:414 inactive_anon:1394 isolated_anon:0 active_file:1721 inactive_file:1440 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:54657 slab_reclaimable:580 slab_unreclaimable:2924 mapped:220 shmem:18 pagetables:167 bounce:0 Node 0 DMA free:3498048kB min:8128kB low:10112kB high:12160kB active_anon:26496kB inactive_anon:89216kB active_file:110144kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:14080kB shmem:1152kB slab_reclaimable:37120kB slab_unreclaimable:187136kB kernel_stack:3936kB pagetables:10688kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 154*64kB 84*128kB 391*256kB 434*512kB 306*1024kB 291*2048kB 184*4096kB 96*8192kB 43*16384kB = 3496832kB 4186 total pagecache pages 1024 pages in swap cache Swap cache stats: add 66504, delete 65480, find 39419/46147 Free swap = 4008640kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3174 pages shared 6663 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17325] 0 17325 62 33 12 0 0 memcg_process Memory cgroup out of memory: Kill process 17325 (memcg_process) score 1000 or sacrifice child Killed process 17325 (memcg_process) total-vm:3968kB, anon-rss:640kB, file-rss:1472kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c00000005f4a7480] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000005f4a7530] [c000000000156d8c] .dump_header+0xac/0x220 [c00000005f4a7610] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000005f4a76f0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000005f4a77a0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000005f4a7890] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000005f4a79e0] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000005f4a7a90] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c00000005f4a7b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c00000005f4a7e30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /16 killed as a result of limit of /16 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 0 CPU 3: hi: 6, btch: 1 usd: 1 CPU 4: hi: 6, btch: 1 usd: 5 CPU 5: hi: 6, btch: 1 usd: 0 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 0 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 3 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 3 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 4 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:634 inactive_anon:765 isolated_anon:0 active_file:1711 inactive_file:1440 isolated_file:0 unevictable:0 dirty:8 writeback:0 unstable:0 free:55076 slab_reclaimable:578 slab_unreclaimable:2924 mapped:236 shmem:38 pagetables:180 bounce:0 Node 0 DMA free:3524864kB min:8128kB low:10112kB high:12160kB active_anon:40576kB inactive_anon:48960kB active_file:109504kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:512kB writeback:0kB mapped:15104kB shmem:2432kB slab_reclaimable:36992kB slab_unreclaimable:187136kB kernel_stack:3936kB pagetables:11520kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 163*64kB 84*128kB 302*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3524288kB 3171 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 0kB Total swap = 0kB 65536 pages RAM 1556 pages reserved 3242 pages shared 6288 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17342] 0 17342 62 33 17 0 0 memcg_process Memory cgroup out of memory: Kill process 17342 (memcg_process) score 1000 or sacrifice child Killed process 17342 (memcg_process) total-vm:3968kB, anon-rss:640kB, file-rss:1472kB Adding 4095936k swap on /dev/sda6. Priority:-1 extents:1 across:4095936k memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c00000007a863130] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000007a8631e0] [c000000000156d8c] .dump_header+0xac/0x220 [c00000007a8632c0] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000007a8633a0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000007a863450] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000007a863540] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000007a863690] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000007a863740] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c00000007a8637f0] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c00000007a863910] [c00000000016be90] .shmem_fault+0x60/0xa0 [c00000007a863990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c00000007a863a90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c00000007a863b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c00000007a863e30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /17 killed as a result of limit of /17 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 3 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 0 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 0 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 4 CPU 14: hi: 6, btch: 1 usd: 3 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 3 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 2 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:575 inactive_anon:765 isolated_anon:0 active_file:1712 inactive_file:1440 isolated_file:0 unevictable:0 dirty:12 writeback:0 unstable:0 free:55149 slab_reclaimable:578 slab_unreclaimable:2924 mapped:231 shmem:38 pagetables:168 bounce:0 Node 0 DMA free:3529536kB min:8128kB low:10112kB high:12160kB active_anon:36800kB inactive_anon:48960kB active_file:109568kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:768kB writeback:0kB mapped:14784kB shmem:2432kB slab_reclaimable:36992kB slab_unreclaimable:187136kB kernel_stack:3920kB pagetables:10752kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 168*64kB 94*128kB 311*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3528192kB 3172 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 0kB Total swap = 0kB 65536 pages RAM 1556 pages reserved 3156 pages shared 6217 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17358] 0 17358 62 33 16 0 0 memcg_process Memory cgroup out of memory: Kill process 17358 (memcg_process) score 1000 or sacrifice child Killed process 17358 (memcg_process) total-vm:3968kB, anon-rss:576kB, file-rss:1536kB Adding 4095936k swap on /dev/sda6. Priority:-1 extents:1 across:4095936k memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000206130c0] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c000000020613170] [c000000000156d8c] .dump_header+0xac/0x220 [c000000020613250] [c000000000157488] .oom_kill_process+0x108/0x260 [c000000020613330] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000206133e0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000206134d0] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c000000020613620] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000206136d0] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c000000020613780] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c0000000206138a0] [c00000000016be90] .shmem_fault+0x60/0xa0 [c000000020613920] [c00000000038a0a4] .shm_fault+0x34/0x50 [c000000020613990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c000000020613a90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c000000020613b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c000000020613e30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /18 killed as a result of limit of /18 memory: usage 64kB, limit 64kB, failcnt 8 memory+swap: usage 64kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 1 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 0 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 2 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 0 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 3 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 1 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 1 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 1 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:571 inactive_anon:765 isolated_anon:0 active_file:1712 inactive_file:1440 isolated_file:0 unevictable:0 dirty:12 writeback:0 unstable:0 free:55133 slab_reclaimable:578 slab_unreclaimable:2924 mapped:231 shmem:38 pagetables:175 bounce:0 Node 0 DMA free:3528512kB min:8128kB low:10112kB high:12160kB active_anon:36544kB inactive_anon:48960kB active_file:109568kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:768kB writeback:0kB mapped:14784kB shmem:2432kB slab_reclaimable:36992kB slab_unreclaimable:187136kB kernel_stack:3920kB pagetables:11200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 167*64kB 95*128kB 312*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3528512kB 3172 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 0kB Total swap = 0kB 65536 pages RAM 1556 pages reserved 3160 pages shared 6210 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17375] 0 17375 62 33 0 0 0 memcg_process Memory cgroup out of memory: Kill process 17375 (memcg_process) score 1000 or sacrifice child Killed process 17375 (memcg_process) total-vm:3968kB, anon-rss:576kB, file-rss:1536kB Adding 4095936k swap on /dev/sda6. Priority:-1 extents:1 across:4095936k memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000f9887480] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c0000000f9887530] [c000000000156d8c] .dump_header+0xac/0x220 [c0000000f9887610] [c000000000157488] .oom_kill_process+0x108/0x260 [c0000000f98876f0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000f98877a0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000f9887890] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c0000000f98879e0] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000f9887a90] [c00000000017d854] .handle_mm_fault+0x8f4/0xb30 [c0000000f9887b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000f9887e30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /19 killed as a result of limit of /19 memory: usage 0kB, limit 0kB, failcnt 7 memory+swap: usage 0kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 5 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 4 CPU 4: hi: 6, btch: 1 usd: 2 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 5 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 2 CPU 15: hi: 6, btch: 1 usd: 5 CPU 16: hi: 6, btch: 1 usd: 3 CPU 17: hi: 6, btch: 1 usd: 4 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 0 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 0 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:585 inactive_anon:767 isolated_anon:0 active_file:1713 inactive_file:1440 isolated_file:0 unevictable:0 dirty:12 writeback:0 unstable:0 free:55128 slab_reclaimable:578 slab_unreclaimable:2924 mapped:231 shmem:40 pagetables:176 bounce:0 Node 0 DMA free:3528192kB min:8128kB low:10112kB high:12160kB active_anon:37440kB inactive_anon:49088kB active_file:109632kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:768kB writeback:0kB mapped:14784kB shmem:2560kB slab_reclaimable:36992kB slab_unreclaimable:187136kB kernel_stack:3904kB pagetables:11264kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 166*64kB 95*128kB 312*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3528448kB 3175 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 4095936kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3153 pages shared 6205 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17391] 0 17391 61 32 16 0 0 memcg_process Memory cgroup out of memory: Kill process 17391 (memcg_process) score 1000 or sacrifice child Killed process 17391 (memcg_process) total-vm:3904kB, anon-rss:576kB, file-rss:1472kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c00000003fdd3130] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c00000003fdd31e0] [c000000000156d8c] .dump_header+0xac/0x220 [c00000003fdd32c0] [c000000000157488] .oom_kill_process+0x108/0x260 [c00000003fdd33a0] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c00000003fdd3450] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c00000003fdd3540] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c00000003fdd3690] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c00000003fdd3740] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c00000003fdd37f0] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c00000003fdd3910] [c00000000016be90] .shmem_fault+0x60/0xa0 [c00000003fdd3990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c00000003fdd3a90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c00000003fdd3b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c00000003fdd3e30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /20 killed as a result of limit of /20 memory: usage 0kB, limit 0kB, failcnt 7 memory+swap: usage 0kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 4 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 2 CPU 5: hi: 6, btch: 1 usd: 3 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 3 CPU 9: hi: 6, btch: 1 usd: 5 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 0 CPU 13: hi: 6, btch: 1 usd: 5 CPU 14: hi: 6, btch: 1 usd: 5 CPU 15: hi: 6, btch: 1 usd: 0 CPU 16: hi: 6, btch: 1 usd: 5 CPU 17: hi: 6, btch: 1 usd: 2 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 0 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 5 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:574 inactive_anon:767 isolated_anon:0 active_file:1713 inactive_file:1440 isolated_file:0 unevictable:0 dirty:12 writeback:0 unstable:0 free:55136 slab_reclaimable:578 slab_unreclaimable:2924 mapped:231 shmem:40 pagetables:171 bounce:0 Node 0 DMA free:3528704kB min:8128kB low:10112kB high:12160kB active_anon:36736kB inactive_anon:49088kB active_file:109632kB inactive_file:92160kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:768kB writeback:0kB mapped:14784kB shmem:2560kB slab_reclaimable:36992kB slab_unreclaimable:187136kB kernel_stack:3904kB pagetables:10944kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 156*64kB 95*128kB 312*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3527808kB 3175 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 4095936kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3153 pages shared 6223 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17404] 0 17404 61 32 8 0 0 memcg_process Memory cgroup out of memory: Kill process 17404 (memcg_process) score 1000 or sacrifice child Killed process 17404 (memcg_process) total-vm:3904kB, anon-rss:576kB, file-rss:1472kB memcg_process invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 memcg_process cpuset=/ mems_allowed=0 Call Trace: [c0000000079db0c0] [c000000000014c64] .show_stack+0x74/0x1c0 (unreliable) [c0000000079db170] [c000000000156d8c] .dump_header+0xac/0x220 [c0000000079db250] [c000000000157488] .oom_kill_process+0x108/0x260 [c0000000079db330] [c000000000157ca4] .mem_cgroup_out_of_memory+0xb4/0xf0 [c0000000079db3e0] [c0000000001b7ab0] .mem_cgroup_handle_oom+0x220/0x230 [c0000000079db4d0] [c0000000001b814c] .__mem_cgroup_try_charge+0x68c/0x6b0 [c0000000079db620] [c0000000001b8a60] .mem_cgroup_charge_common+0x60/0xb0 [c0000000079db6d0] [c0000000001b8c3c] .mem_cgroup_cache_charge+0x18c/0x1f0 [c0000000079db780] [c00000000016ba74] .shmem_getpage+0x974/0xd30 [c0000000079db8a0] [c00000000016be90] .shmem_fault+0x60/0xa0 [c0000000079db920] [c00000000038a0a4] .shm_fault+0x34/0x50 [c0000000079db990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c0000000079dba90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c0000000079dbb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000079dbe30] [c00000000000522c] handle_page_fault+0x20/0x74 Task in /21 killed as a result of limit of /21 memory: usage 0kB, limit 0kB, failcnt 7 memory+swap: usage 0kB, limit 9007199254740991kB, failcnt 0 Mem-Info: Node 0 DMA per-cpu: CPU 0: hi: 6, btch: 1 usd: 0 CPU 1: hi: 6, btch: 1 usd: 5 CPU 2: hi: 6, btch: 1 usd: 5 CPU 3: hi: 6, btch: 1 usd: 5 CPU 4: hi: 6, btch: 1 usd: 0 CPU 5: hi: 6, btch: 1 usd: 5 CPU 6: hi: 6, btch: 1 usd: 5 CPU 7: hi: 6, btch: 1 usd: 5 CPU 8: hi: 6, btch: 1 usd: 5 CPU 9: hi: 6, btch: 1 usd: 3 CPU 10: hi: 6, btch: 1 usd: 5 CPU 11: hi: 6, btch: 1 usd: 5 CPU 12: hi: 6, btch: 1 usd: 0 CPU 13: hi: 6, btch: 1 usd: 3 CPU 14: hi: 6, btch: 1 usd: 5 CPU 15: hi: 6, btch: 1 usd: 4 CPU 16: hi: 6, btch: 1 usd: 0 CPU 17: hi: 6, btch: 1 usd: 5 CPU 18: hi: 6, btch: 1 usd: 5 CPU 19: hi: 6, btch: 1 usd: 5 CPU 20: hi: 6, btch: 1 usd: 4 CPU 21: hi: 6, btch: 1 usd: 5 CPU 22: hi: 6, btch: 1 usd: 5 CPU 23: hi: 6, btch: 1 usd: 5 active_anon:586 inactive_anon:767 isolated_anon:0 active_file:1713 inactive_file:1441 isolated_file:0 unevictable:0 dirty:12 writeback:0 unstable:0 free:55111 slab_reclaimable:578 slab_unreclaimable:2925 mapped:231 shmem:40 pagetables:175 bounce:0 Node 0 DMA free:3527104kB min:8128kB low:10112kB high:12160kB active_anon:37504kB inactive_anon:49088kB active_file:109632kB inactive_file:92224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4190720kB mlocked:0kB dirty:768kB writeback:0kB mapped:14784kB shmem:2560kB slab_reclaimable:36992kB slab_unreclaimable:187200kB kernel_stack:3904kB pagetables:11200kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 Node 0 DMA: 148*64kB 90*128kB 312*256kB 439*512kB 312*1024kB 297*2048kB 187*4096kB 98*8192kB 43*16384kB = 3526656kB 3176 total pagecache pages 0 pages in swap cache Swap cache stats: add 66844, delete 66844, find 39419/46147 Free swap = 4095936kB Total swap = 4095936kB 65536 pages RAM 1556 pages reserved 3154 pages shared 6240 pages non-shared [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [17417] 0 17417 61 32 12 0 0 memcg_process Memory cgroup out of memory: Kill process 17417 (memcg_process) score 1000 or sacrifice child Killed process 17417 (memcg_process) total-vm:3904kB, anon-rss:576kB, file-rss:1472kB -- 0:conmux-control -- time-stamp -- Oct/26/10 0:05:55 -- -- 0:conmux-control -- time-stamp -- Oct/26/10 1:01:02 -- INFO: rcu_sched_state detected stall on CPU 8 (t=6000 jiffies) INFO: task rcu_torture_fak:88 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rcu_torture_f D 0000000000000000 0 88 2 0x00008000 Call Trace: [c0000000fcab3690] [c0000000fcab3740] 0xc0000000fcab3740 (unreliable) [c0000000fcab3860] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000fcab38f0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000fcab3a60] [c0000000006ec0d4] .schedule_timeout+0x234/0x330 [c0000000fcab3bb0] [c0000000006ebce4] .wait_for_common+0x154/0x1b0 [c0000000fcab3c80] [c00000000011c800] .synchronize_sched+0x60/0x80 [c0000000fcab3d70] [c000000000117bc0] .synchronize_rcu+0x10/0x30 [c0000000fcab3de0] [c000000000118ed8] .rcu_torture_fakewriter+0xc8/0x190 [c0000000fcab3e90] [c0000000000b6b0c] .kthread+0xbc/0xd0 [c0000000fcab3f90] [c000000000032164] .kernel_thread+0x54/0x70 no locks held by rcu_torture_fak/88. INFO: task rcu_torture_fak:89 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rcu_torture_f D 0000000000000000 0 89 2 0x00008000 Call Trace: [c0000000fcab7690] [c0000000fcab7740] 0xc0000000fcab7740 (unreliable) [c0000000fcab7860] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000fcab78f0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000fcab7a60] [c0000000006ec0d4] .schedule_timeout+0x234/0x330 [c0000000fcab7bb0] [c0000000006ebce4] .wait_for_common+0x154/0x1b0 [c0000000fcab7c80] [c00000000011c800] .synchronize_sched+0x60/0x80 [c0000000fcab7d70] [c000000000117bc0] .synchronize_rcu+0x10/0x30 [c0000000fcab7de0] [c000000000118ed8] .rcu_torture_fakewriter+0xc8/0x190 [c0000000fcab7e90] [c0000000000b6b0c] .kthread+0xbc/0xd0 [c0000000fcab7f90] [c000000000032164] .kernel_thread+0x54/0x70 no locks held by rcu_torture_fak/89. INFO: task rcu_torture_fak:90 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rcu_torture_f D 0000000000000000 0 90 2 0x00008000 Call Trace: [c0000000fcabb690] [c0000000fcabb740] 0xc0000000fcabb740 (unreliable) [c0000000fcabb860] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000fcabb8f0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000fcabba60] [c0000000006ec0d4] .schedule_timeout+0x234/0x330 [c0000000fcabbbb0] [c0000000006ebce4] .wait_for_common+0x154/0x1b0 [c0000000fcabbc80] [c00000000011c800] .synchronize_sched+0x60/0x80 [c0000000fcabbd70] [c000000000117bc0] .synchronize_rcu+0x10/0x30 [c0000000fcabbde0] [c000000000118ed8] .rcu_torture_fakewriter+0xc8/0x190 [c0000000fcabbe90] [c0000000000b6b0c] .kthread+0xbc/0xd0 [c0000000fcabbf90] [c000000000032164] .kernel_thread+0x54/0x70 no locks held by rcu_torture_fak/90. INFO: task rcu_torture_fak:91 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. rcu_torture_f D 0000000000000000 0 91 2 0x00008000 Call Trace: [c0000000fcabf690] [c0000000fcabf740] 0xc0000000fcabf740 (unreliable) [c0000000fcabf860] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000fcabf8f0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000fcabfa60] [c0000000006ec0d4] .schedule_timeout+0x234/0x330 [c0000000fcabfbb0] [c0000000006ebce4] .wait_for_common+0x154/0x1b0 [c0000000fcabfc80] [c00000000011c800] .synchronize_sched+0x60/0x80 [c0000000fcabfd70] [c000000000117bc0] .synchronize_rcu+0x10/0x30 [c0000000fcabfde0] [c000000000118ed8] .rcu_torture_fakewriter+0xc8/0x190 [c0000000fcabfe90] [c0000000000b6b0c] .kthread+0xbc/0xd0 [c0000000fcabff90] [c000000000032164] .kernel_thread+0x54/0x70 no locks held by rcu_torture_fak/91. INFO: task kjournald:571 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kjournald D 0000000000000000 0 571 2 0x00008000 Call Trace: [c0000000093c3550] [c0000000093c3600] 0xc0000000093c3600 (unreliable) [c0000000093c3720] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000093c37b0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000093c3920] [c0000000006eba00] .io_schedule+0x90/0x110 [c0000000093c39b0] [c0000000001f7c98] .sync_buffer+0x68/0x80 [c0000000093c3a20] [c0000000006ec4d4] .__wait_on_bit+0xb4/0x130 [c0000000093c3ad0] [c0000000006ec5c0] .out_of_line_wait_on_bit+0x70/0x90 [c0000000093c3bb0] [c0000000001f7ae0] .__wait_on_buffer+0x30/0x50 [c0000000093c3c30] [c0000000002c8aec] .journal_commit_transaction+0x82c/0x1670 [c0000000093c3db0] [c0000000002cd6bc] .kjournald+0x10c/0x2e0 [c0000000093c3e90] [c0000000000b6b0c] .kthread+0xbc/0xd0 [c0000000093c3f90] [c000000000032164] .kernel_thread+0x54/0x70 no locks held by kjournald/571. INFO: task master:1451 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. master D 00000fffa4b92434 0 1451 1 0x00008080 Call Trace: [c000000006d2b430] [c000000006d2b4e0] 0xc000000006d2b4e0 (unreliable) [c000000006d2b600] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c000000006d2b690] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c000000006d2b800] [c0000000006eba00] .io_schedule+0x90/0x110 [c000000006d2b890] [c0000000001537d4] .sync_page+0x74/0xa0 [c000000006d2b910] [c0000000006ec2d4] .__wait_on_bit_lock+0xd4/0x190 [c000000006d2b9d0] [c000000000153714] .__lock_page+0x54/0x70 [c000000006d2ba90] [c00000000017da74] .handle_mm_fault+0xb14/0xb30 [c000000006d2bb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c000000006d2be30] [c00000000000522c] handle_page_fault+0x20/0x74 1 lock held by master/1451: #0: (&mm->mmap_sem){......}, at: [] .do_page_fault+0x1dc/0x780 INFO: task abrtd:1462 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. abrtd D 00000fff8d8873c0 0 1462 1 0x00008080 Call Trace: [c0000000fb7a2b40] [c0000000fb7a2bf0] 0xc0000000fb7a2bf0 (unreliable) [c0000000fb7a2d10] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000fb7a2da0] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000fb7a2f10] [c0000000006eba00] .io_schedule+0x90/0x110 [c0000000fb7a2fa0] [c0000000001537d4] .sync_page+0x74/0xa0 [c0000000fb7a3020] [c0000000006ec2d4] .__wait_on_bit_lock+0xd4/0x190 [c0000000fb7a30e0] [c000000000153714] .__lock_page+0x54/0x70 [c0000000fb7a31a0] [c0000000001539f8] .find_lock_page+0xb8/0xe0 [c0000000fb7a3240] [c000000000154514] .filemap_fault+0x1e4/0x490 [c0000000fb7a3310] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c0000000fb7a3410] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c0000000fb7a3500] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000fb7a37b0] [c00000000000522c] handle_page_fault+0x20/0x74 --- Exception: 301 at .__strncpy_from_user+0x14/0x3c LR = .getname+0x114/0x260 [c0000000fb7a3aa0] [c0000000001ced70] .getname+0xe0/0x260 (unreliable) [c0000000fb7a3b40] [c0000000001d1b10] .user_path_at+0x30/0xb0 [c0000000fb7a3c90] [c0000000001c5824] .vfs_fstatat+0x44/0xb0 [c0000000fb7a3d30] [c0000000001c5ac4] .SyS_newstat+0x24/0x50 [c0000000fb7a3e30] [c000000000008564] syscall_exit+0x0/0x40 1 lock held by abrtd/1462: #0: (&mm->mmap_sem){......}, at: [] .do_page_fault+0x1dc/0x780 INFO: task crond:1470 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. crond D 00000000550575a8 0 1470 1 0x00008080 Call Trace: [c0000000070031c0] [c000000007003270] 0xc000000007003270 (unreliable) [c000000007003390] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c000000007003420] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c000000007003590] [c0000000006eba00] .io_schedule+0x90/0x110 [c000000007003620] [c0000000001537d4] .sync_page+0x74/0xa0 [c0000000070036a0] [c0000000006ec2d4] .__wait_on_bit_lock+0xd4/0x190 [c000000007003760] [c000000000153714] .__lock_page+0x54/0x70 [c000000007003820] [c0000000001539f8] .find_lock_page+0xb8/0xe0 [c0000000070038c0] [c000000000154514] .filemap_fault+0x1e4/0x490 [c000000007003990] [c00000000017a4ac] .__do_fault+0xac/0x6d0 [c000000007003a90] [c00000000017d0ac] .handle_mm_fault+0x14c/0xb30 [c000000007003b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c000000007003e30] [c00000000000522c] handle_page_fault+0x20/0x74 1 lock held by crond/1470: #0: (&mm->mmap_sem){......}, at: [] .do_page_fault+0x1dc/0x780 INFO: task memcg_process_s:17690 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. memcg_process D 0000000010000ac8 0 17690 17665 0x00008080 Call Trace: [c0000000079ae800] [c0000000079ae8b0] 0xc0000000079ae8b0 (unreliable) [c0000000079ae9d0] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c0000000079aea60] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c0000000079aebd0] [c0000000006eba00] .io_schedule+0x90/0x110 [c0000000079aec60] [c0000000003df57c] .get_request_wait+0x15c/0x230 [c0000000079aed70] [c0000000003df9f0] .__make_request+0x90/0x680 [c0000000079aee30] [c0000000003dd400] .generic_make_request+0x340/0x520 [c0000000079aef40] [c0000000003dd674] .submit_bio+0x94/0x130 [c0000000079af010] [c00000000018feb0] .swap_writepage+0xb0/0x130 [c0000000079af0a0] [c00000000016652c] .pageout+0x1bc/0x420 [c0000000079af180] [c00000000016777c] .shrink_page_list+0x38c/0x640 [c0000000079af2b0] [c000000000167e60] .shrink_inactive_list+0x130/0x4d0 [c0000000079af3c0] [c0000000001688e0] .shrink_zone+0x300/0x500 [c0000000079af4e0] [c000000000169da0] .do_try_to_free_pages+0x110/0x4e0 [c0000000079af5d0] [c00000000016a4ec] .try_to_free_pages+0xbc/0x1a0 [c0000000079af6e0] [c00000000015d820] .__alloc_pages_nodemask+0x540/0x940 [c0000000079af860] [c00000000019eb68] .alloc_page_vma+0x98/0x1d0 [c0000000079af900] [c000000000190480] .read_swap_cache_async+0x160/0x1e0 [c0000000079af9d0] [c000000000190594] .swapin_readahead+0x94/0xe0 [c0000000079afa90] [c00000000017d5c4] .handle_mm_fault+0x664/0xb30 [c0000000079afb80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c0000000079afe30] [c00000000000522c] handle_page_fault+0x20/0x74 1 lock held by memcg_process_s/17690: #0: (&mm->mmap_sem){......}, at: [] .do_page_fault+0x1dc/0x780 INFO: task memcg_process_s:17692 blocked for more than 120 seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. memcg_process D 0000000010000ac8 0 17692 17665 0x00008080 Call Trace: [c00000007d402800] [c00000007d4028b0] 0xc00000007d4028b0 (unreliable) [c00000007d4029d0] [c000000000015c50] .__switch_to+0xe0/0x1a0 [c00000007d402a60] [c0000000006eb1f4] .schedule+0x3f4/0xb70 [c00000007d402bd0] [c0000000006eba00] .io_schedule+0x90/0x110 [c00000007d402c60] [c0000000003df57c] .get_request_wait+0x15c/0x230 [c00000007d402d70] [c0000000003df9f0] .__make_request+0x90/0x680 [c00000007d402e30] [c0000000003dd400] .generic_make_request+0x340/0x520 [c00000007d402f40] [c0000000003dd674] .submit_bio+0x94/0x130 [c00000007d403010] [c00000000018feb0] .swap_writepage+0xb0/0x130 [c00000007d4030a0] [c00000000016652c] .pageout+0x1bc/0x420 [c00000007d403180] [c00000000016777c] .shrink_page_list+0x38c/0x640 [c00000007d4032b0] [c000000000167e60] .shrink_inactive_list+0x130/0x4d0 [c00000007d4033c0] [c0000000001688e0] .shrink_zone+0x300/0x500 [c00000007d4034e0] [c000000000169da0] .do_try_to_free_pages+0x110/0x4e0 [c00000007d4035d0] [c00000000016a4ec] .try_to_free_pages+0xbc/0x1a0 [c00000007d4036e0] [c00000000015d820] .__alloc_pages_nodemask+0x540/0x940 [c00000007d403860] [c00000000019eb68] .alloc_page_vma+0x98/0x1d0 [c00000007d403900] [c000000000190480] .read_swap_cache_async+0x160/0x1e0 [c00000007d4039d0] [c000000000190594] .swapin_readahead+0x94/0xe0 [c00000007d403a90] [c00000000017d5c4] .handle_mm_fault+0x664/0xb30 [c00000007d403b80] [c0000000006f1c84] .do_page_fault+0x344/0x780 [c00000007d403e30] [c00000000000522c] handle_page_fault+0x20/0x74 1 lock held by memcg_process_s/17692: #0: (&mm->mmap_sem){......}, at: [] .do_page_fault+0x1dc/0x780 INFO: rcu_sched_state detected stall on CPU 8 (t=24030 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:06:01 -- INFO: rcu_sched_state detected stall on CPU 8 (t=42060 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:09:02 -- INFO: rcu_sched_state detected stall on CPU 8 (t=60090 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:12:02 -- INFO: rcu_sched_state detected stall on CPU 8 (t=78120 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:15:02 -- INFO: rcu_sched_state detected stall on CPU 8 (t=96150 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:18:03 -- INFO: rcu_sched_state detected stall on CPU 8 (t=114180 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:21:03 -- INFO: rcu_sched_state detected stall on CPU 8 (t=132210 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:24:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=150240 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:27:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=168270 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:30:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=186300 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:33:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=204330 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:36:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=222360 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:39:04 -- INFO: rcu_sched_state detected stall on CPU 8 (t=240390 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:42:06 -- INFO: rcu_sched_state detected stall on CPU 8 (t=258420 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:45:06 -- INFO: rcu_sched_state detected stall on CPU 8 (t=276450 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:48:06 -- INFO: rcu_sched_state detected stall on CPU 8 (t=294480 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:51:06 -- INFO: rcu_sched_state detected stall on CPU 8 (t=312510 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 1:54:06 -- -- 0:conmux-control -- time-stamp -- Oct/26/10 2:04:14 -- INFO: rcu_sched_state detected stall on CPU 0 (t=6000 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:06:13 -- INFO: rcu_sched_state detected stall on CPU 0 (t=24030 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:09:13 -- INFO: rcu_sched_state detected stall on CPU 0 (t=42060 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:12:14 -- INFO: rcu_sched_state detected stall on CPU 0 (t=60090 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:15:15 -- INFO: rcu_sched_state detected stall on CPU 0 (t=78120 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:18:15 -- INFO: rcu_sched_state detected stall on CPU 0 (t=96150 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:21:15 -- -- 0:conmux-control -- time-stamp -- Oct/26/10 2:25:53 -- INFO: rcu_sched_state detected stall on CPU 20 (t=6000 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:27:53 -- INFO: rcu_sched_state detected stall on CPU 20 (t=24030 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:30:53 -- INFO: rcu_sched_state detected stall on CPU 20 (t=42060 jiffies) -- 0:conmux-control -- time-stamp -- Oct/26/10 2:33:53 -- INFO: rcu_sched_state detected stall on CPU 20 (t=60090 jiffies) ==========================================================