From: Trevor Cordes <trevor@tecnopolis.ca>
To: Michal Hocko <mhocko@kernel.org>
Cc: Mel Gorman <mgorman@techsingularity.net>,
linux-kernel@vger.kernel.org,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Minchan Kim <minchan@kernel.org>, Rik van Riel <riel@surriel.com>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Subject: Re: mm, vmscan: commit makes PAE kernel crash nightly (bisected)
Date: Thu, 26 Jan 2017 17:18:58 -0600 [thread overview]
Message-ID: <20170126171858.5e1b484a@pog.tecnopolis.ca> (raw)
In-Reply-To: <20170124125458.GH6867@dhcp22.suse.cz>
[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]
On 2017-01-24 Michal Hocko wrote:
> On Sun 22-01-17 18:45:59, Trevor Cordes wrote:
> [...]
> > Also, completely separate from your patch I ran mhocko's 4.9 tree
> > with mem=2G to see if lower ram amount would help, but it didn't.
> > Even with 2G the system oom and hung same as usual. So far the
> > only thing that helps at all was the cgroup_disable=memory option,
> > which makes the problem disappear completely for me.
>
> OK, can we reduce the problem space slightly more and could you boot
> with kmem accounting enabled? cgroup.memory=nokmem,nosocket
I ran for 30 hours with cgroup.memory=nokmem,nosocket using vanilla
4.9.0+ and it oom'd during a big rdiff-backup at 9am. My script was
able to reboot it before it hung. Only one oom occurred before the
reboot, which is a bit odd, usually there is 5-50. See attached
messages log (oom6).
So, still, only cgroup_disable=memory mitigates this bug (so far). If
you need me to test cgroup.memory=nokmem,nosocket with your since-4.9
branch specifically, let me know and I'll add it to the to-test list.
On 2017-01-25 Michal Hocko wrote:
> On Wed 25-01-17 04:02:46, Trevor Cordes wrote:
> > OK, I patched & compiled mhocko's git tree from the other day
> > 4.9.0+. (To confirm, weird, but mhocko's git tree I'm using from a
> > couple of weeks ago shows the newest commit (git log) is
> > 69973b830859bc6529a7a0468ba0d80ee5117826 "Linux 4.9"? Let me know
> > if I'm doing something wrong, see below.)
>
> My fault. I should have noted that you should use since-4.9 branch.
OK, I got it now, I'm retesting the runs I did (with/without the
various patches) on your git tree and will re-report the (correct)
results. Will take a few days. Thanks!
[-- Attachment #2: oom6 --]
[-- Type: application/octet-stream, Size: 23915 bytes --]
Jan 26 08:59:46 firewallfsi kernel: [104057.183579] rdiff-backup invoked oom-killer: gfp_mask=0x2420848(GFP_NOFS|__GFP_NOFAIL|__GFP_HARDWALL|__GFP_MOVABLE), nodemask=0, order=0, oom_score_adj=0
Jan 26 08:59:46 firewallfsi kernel: [104057.186272] rdiff-backup cpuset=/ mems_allowed=0
Jan 26 08:59:46 firewallfsi kernel: [104057.187565] CPU: 3 PID: 18243 Comm: rdiff-backup Not tainted 4.9.0+ #2
Jan 26 08:59:46 firewallfsi kernel: [104057.188821] Hardware name: Intel Corporation S1200BTL/S1200BTL, BIOS S1200BT.86B.02.00.0035.030220120927 03/02/2012
Jan 26 08:59:46 firewallfsi kernel: [104057.190083] de14fab0 ceb604e7 de14fbe8 eeb95a00 de14fae0 ce9e17a6 de14fac0 cef63bad
Jan 26 08:59:46 firewallfsi kernel: [104057.191343] de14fae0 ceb6625f de14fae4 ea682c00 f6ba2600 eeb95a00 cf168bce de14fbe8
Jan 26 08:59:46 firewallfsi kernel: [104057.192588] de14fb24 ce97aff7 ce876d8a de14fb10 ce97ac6b 00000006 00000000 0000000c
Jan 26 08:59:46 firewallfsi kernel: [104057.193821] Call Trace:
Jan 26 08:59:46 firewallfsi kernel: [104057.195025] [<ceb604e7>] dump_stack+0x58/0x81
Jan 26 08:59:46 firewallfsi kernel: [104057.196213] [<ce9e17a6>] dump_header+0x64/0x1a6
Jan 26 08:59:46 firewallfsi kernel: [104057.197381] [<cef63bad>] ? _raw_spin_unlock_irqrestore+0xd/0x10
Jan 26 08:59:46 firewallfsi kernel: [104057.198536] [<ceb6625f>] ? ___ratelimit+0x9f/0x100
Jan 26 08:59:46 firewallfsi kernel: [104057.199672] [<ce97aff7>] oom_kill_process+0x207/0x3d0
Jan 26 08:59:46 firewallfsi kernel: [104057.200790] [<ce876d8a>] ? has_capability_noaudit+0x1a/0x30
Jan 26 08:59:46 firewallfsi kernel: [104057.201893] [<ce97ac6b>] ? oom_badness.part.13+0xcb/0x140
Jan 26 08:59:46 firewallfsi kernel: [104057.202979] [<ce97b4d8>] out_of_memory+0xf8/0x2a0
Jan 26 08:59:46 firewallfsi kernel: [104057.204045] [<ce980016>] __alloc_pages_nodemask+0xc46/0xd10
Jan 26 08:59:46 firewallfsi kernel: [104057.205096] [<ce9768c2>] ? find_get_entry+0x22/0x160
Jan 26 08:59:46 firewallfsi kernel: [104057.206131] [<ceb32010>] ? generic_make_request+0xd0/0x1b0
Jan 26 08:59:46 firewallfsi kernel: [104057.207148] [<ce97727e>] pagecache_get_page+0xbe/0x2d0
Jan 26 08:59:46 firewallfsi kernel: [104057.208148] [<cea19044>] __getblk_gfp+0x104/0x360
Jan 26 08:59:46 firewallfsi kernel: [104057.209127] [<cea19a3a>] ? submit_bh_wbc+0x14a/0x1f0
Jan 26 08:59:46 firewallfsi kernel: [104057.210097] [<cea1a6bb>] __breadahead+0x2b/0x70
Jan 26 08:59:46 firewallfsi kernel: [104057.211039] [<cea690ee>] __ext4_get_inode_loc+0x40e/0x440
Jan 26 08:59:46 firewallfsi kernel: [104057.211966] [<ce9feaf9>] ? inode_init_always+0x119/0x1c0
Jan 26 08:59:46 firewallfsi kernel: [104057.212874] [<cea6c07b>] ext4_iget+0x7b/0xaf0
Jan 26 08:59:46 firewallfsi kernel: [104057.213762] [<ce9fd683>] ? __d_alloc+0x23/0x190
Jan 26 08:59:46 firewallfsi kernel: [104057.214632] [<cea6cb1f>] ext4_iget_normal+0x2f/0x40
Jan 26 08:59:46 firewallfsi kernel: [104057.215483] [<cea779c5>] ext4_lookup+0xb5/0x240
Jan 26 08:59:46 firewallfsi kernel: [104057.216315] [<ce9ef257>] ? legitimize_path.isra.33+0x27/0x60
Jan 26 08:59:46 firewallfsi kernel: [104057.217132] [<ce9ef3fe>] ? unlazy_walk+0x16e/0x1a0
Jan 26 08:59:46 firewallfsi kernel: [104057.217929] [<ce9f01cc>] lookup_slow+0x7c/0x130
Jan 26 08:59:46 firewallfsi kernel: [104057.218710] [<ce9f0da4>] walk_component+0x1e4/0x300
Jan 26 08:59:46 firewallfsi kernel: [104057.219475] [<ce9efdde>] ? path_init+0x19e/0x330
Jan 26 08:59:46 firewallfsi kernel: [104057.220221] [<ce9ee69f>] ? terminate_walk+0x8f/0x100
Jan 26 08:59:46 firewallfsi kernel: [104057.220949] [<ce9f1f83>] path_lookupat+0x53/0xe0
Jan 26 08:59:46 firewallfsi kernel: [104057.221657] [<ce9f4127>] filename_lookup+0x97/0x190
Jan 26 08:59:46 firewallfsi kernel: [104057.222348] [<ce9cf1ad>] ? kmem_cache_alloc+0x15d/0x1c0
Jan 26 08:59:46 firewallfsi kernel: [104057.223021] [<ce9f3d6a>] ? getname_flags+0x3a/0x1a0
Jan 26 08:59:46 firewallfsi kernel: [104057.223676] [<ce9f42f6>] user_path_at_empty+0x36/0x40
Jan 26 08:59:46 firewallfsi kernel: [104057.224311] [<ce9e9a10>] vfs_fstatat+0x60/0xb0
Jan 26 08:59:46 firewallfsi kernel: [104057.224928] [<ce9ea40d>] SyS_lstat64+0x2d/0x50
Jan 26 08:59:46 firewallfsi kernel: [104057.225524] [<ce912e8e>] ? __audit_syscall_exit+0x1ce/0x260
Jan 26 08:59:46 firewallfsi kernel: [104057.226105] [<ce8035a6>] ? syscall_slow_exit_work+0xd6/0xe0
Jan 26 08:59:46 firewallfsi kernel: [104057.226667] [<ce80377a>] do_fast_syscall_32+0x8a/0x150
Jan 26 08:59:46 firewallfsi kernel: [104057.227212] [<cef640ca>] sysenter_past_esp+0x47/0x75
Jan 26 08:59:46 firewallfsi kernel: [104057.227745] Mem-Info:
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] active_anon:126164 inactive_anon:65161 isolated_anon:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] active_file:193794 inactive_file:602556 isolated_file:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] unevictable:0 dirty:198 writeback:0 unstable:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] slab_reclaimable:97990 slab_unreclaimable:9475
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] mapped:24098 shmem:324 pagetables:1512 bounce:0
Jan 26 08:59:46 firewallfsi kernel: [104057.228569] free:109080 free_pcp:1076 free_cma:0
Jan 26 08:59:46 firewallfsi kernel: [104057.233530] Node 0 active_anon:504656kB inactive_anon:260644kB active_file:775176kB inactive_file:2410224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:96412kB dirty:852kB writeback:0kB shmem:1296kB writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no
Jan 26 08:59:46 firewallfsi kernel: [104057.236152] DMA free:3180kB min:68kB low:84kB high:100kB active_anon:0kB inactive_anon:0kB active_file:5536kB inactive_file:0kB unevictable:0kB writepending:0kB present:15976kB managed:15900kB mlocked:0kB slab_reclaimable:7016kB slab_unreclaimable:24kB kernel_stack:8kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 778 4734 4734
Jan 26 08:59:46 firewallfsi kernel: [104057.240078] Normal free:3504kB min:3532kB low:4412kB high:5292kB active_anon:260kB inactive_anon:0kB active_file:348276kB inactive_file:100kB unevictable:0kB writepending:380kB present:892920kB managed:817180kB mlocked:0kB slab_reclaimable:384944kB slab_unreclaimable:37876kB kernel_stack:2552kB pagetables:0kB bounce:0kB free_pcp:2000kB local_pcp:276kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 0 31652 31652
Jan 26 08:59:46 firewallfsi kernel: [104057.244247] HighMem free:429636kB min:512kB low:5000kB high:9488kB active_anon:504396kB inactive_anon:260644kB active_file:421364kB inactive_file:2410124kB unevictable:0kB writepending:500kB present:4051548kB managed:4051548kB mlocked:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:6048kB bounce:0kB free_pcp:2300kB local_pcp:308kB free_cma:0kB
Jan 26 08:59:46 firewallfsi kernel: lowmem_reserve[]: 0 0 0 0
Jan 26 08:59:46 firewallfsi kernel: [104057.248837] DMA: 5*4kB (UE) 21*8kB (UME) 17*16kB (UME) 17*32kB (UE) 8*64kB (UME) 3*128kB (UE) 3*256kB (ME) 1*512kB (U) 0*1024kB 0*2048kB 0*4096kB = 3180kB
Jan 26 08:59:46 firewallfsi kernel: Normal: 432*4kB (UMEH) 24*8kB (UH) 13*16kB (H) 11*32kB (H) 8*64kB (H) 4*128kB (H) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3504kB
Jan 26 08:59:46 firewallfsi kernel: HighMem: 7*4kB (U) 205*8kB (UM) 58*16kB (UM) 21*32kB (UM) 12*64kB (UM) 11*128kB (UM) 3*256kB (UM) 31*512kB (M) 4*1024kB (M) 1*2048kB (M) 98*4096kB (M) = 429636kB
Jan 26 08:59:46 firewallfsi kernel: [104057.256399] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Jan 26 08:59:46 firewallfsi kernel: [104057.257716] 796796 total pagecache pages
Jan 26 08:59:46 firewallfsi kernel: [104057.259050] 99 pages in swap cache
Jan 26 08:59:46 firewallfsi kernel: [104057.260364] Swap cache stats: add 1286, delete 1187, find 150/171
Jan 26 08:59:46 firewallfsi kernel: [104057.261639] Free swap = 33779604kB
Jan 26 08:59:46 firewallfsi kernel: [104057.262912] Total swap = 33784572kB
Jan 26 08:59:46 firewallfsi kernel: [104057.264179] 1240111 pages RAM
Jan 26 08:59:46 firewallfsi kernel: [104057.265440] 1012887 pages HighMem/MovableOnly
Jan 26 08:59:46 firewallfsi kernel: [104057.266706] 18954 pages reserved
Jan 26 08:59:46 firewallfsi kernel: [104057.267969] 0 pages hwpoisoned
Jan 26 08:59:46 firewallfsi kernel: [104057.269225] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Jan 26 08:59:46 firewallfsi kernel: [104057.270519] [ 588] 0 588 3279 1647 10 3 29 0 systemd-journal
Jan 26 08:59:46 firewallfsi kernel: [104057.271779] [ 624] 0 624 3592 832 8 3 344 -1000 systemd-udevd
Jan 26 08:59:46 firewallfsi kernel: [104057.273037] [ 728] 0 728 8633 1065 10 3 0 0 rsyslogd
Jan 26 08:59:46 firewallfsi kernel: [104057.274298] [ 733] 0 733 1704 625 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.275559] [ 736] 0 736 1033 628 5 3 0 0 irqbalance
Jan 26 08:59:46 firewallfsi kernel: [104057.276816] [ 737] 288 737 16729 1733 19 3 0 0 milter-greylist
Jan 26 08:59:46 firewallfsi kernel: [104057.278074] [ 739] 0 739 1704 616 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.279326] [ 750] 0 750 1704 644 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.280573] [ 751] 0 751 1704 676 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.281764] [ 767] 0 767 1005 714 6 3 0 0 systemd-logind
Jan 26 08:59:46 firewallfsi kernel: [104057.282949] [ 768] 0 768 1472 995 6 3 0 0 smartd
Jan 26 08:59:46 firewallfsi kernel: [104057.284124] [ 771] 0 771 1704 675 7 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.285288] [ 772] 0 772 1704 657 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.286442] [ 777] 0 777 800 486 6 3 0 0 mdadm
Jan 26 08:59:46 firewallfsi kernel: [104057.287590] [ 779] 0 779 1704 627 7 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.288729] [ 790] 81 790 1700 1036 7 3 0 -900 dbus-daemon
Jan 26 08:59:46 firewallfsi kernel: [104057.289848] [ 810] 0 810 3266 2194 10 3 0 0 restarter
Jan 26 08:59:46 firewallfsi kernel: [104057.290942] [ 811] 0 811 3234 2159 9 3 0 0 dynamic-ip-upda
Jan 26 08:59:46 firewallfsi kernel: [104057.291965] [ 812] 0 812 3165 2073 10 3 0 0 mailwarnings
Jan 26 08:59:46 firewallfsi kernel: [104057.292949] [ 814] 0 814 1736 717 6 3 0 0 tickle-pog
Jan 26 08:59:46 firewallfsi kernel: [104057.293907] [ 815] 0 815 1819 804 6 3 0 0 watch-services
Jan 26 08:59:46 firewallfsi kernel: [104057.294839] [ 816] 0 816 2708 1666 9 3 0 0 udp-sgr
Jan 26 08:59:46 firewallfsi kernel: [104057.295754] [ 817] 0 817 3695 1987 10 3 0 0 fetchmail
Jan 26 08:59:46 firewallfsi kernel: [104057.296646] [ 845] 0 845 2499 370 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.297512] [ 846] 0 846 2518 912 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.298343] [ 847] 0 847 2499 126 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.299150] [ 848] 0 848 2518 912 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.299942] [ 849] 0 849 2499 126 9 3 0 0 saslauthd
Jan 26 08:59:46 firewallfsi kernel: [104057.300691] [ 888] 0 888 2769 731 9 3 0 -1000 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.301407] [ 930] 0 930 1704 634 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.302105] [ 931] 0 931 3309 2240 10 3 0 0 watch-ip
Jan 26 08:59:46 firewallfsi kernel: [104057.302765] [ 932] 0 932 583 403 5 3 0 0 acpid
Jan 26 08:59:46 firewallfsi kernel: [104057.303380] [ 937] 0 937 7287 1292 11 3 0 0 apcupsd
Jan 26 08:59:46 firewallfsi kernel: [104057.303975] [ 973] 0 973 860 491 5 3 0 0 atd
Jan 26 08:59:46 firewallfsi kernel: [104057.304544] [ 1036] 0 1036 7049 2372 16 3 0 0 nmbd
Jan 26 08:59:46 firewallfsi kernel: [104057.305081] [ 1037] 0 1037 6840 2278 16 3 0 0 nmbd
Jan 26 08:59:46 firewallfsi kernel: [104057.305593] [ 1052] 25 1052 82757 48632 115 3 0 0 named
Jan 26 08:59:46 firewallfsi kernel: [104057.306084] [ 1065] 27 1065 1707 767 6 3 0 0 mysqld_safe
Jan 26 08:59:46 firewallfsi kernel: [104057.306568] [ 1235] 27 1235 126038 14727 64 3 0 0 mysqld
Jan 26 08:59:46 firewallfsi kernel: [104057.307046] [ 1239] 0 1239 12052 6480 27 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.307528] [ 1357] 0 1357 1116 484 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.307996] [ 1358] 0 1358 1116 545 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.308466] [ 1359] 0 1359 1116 546 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.308926] [ 1360] 0 1360 1116 487 5 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.309382] [ 1361] 0 1361 1116 519 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.309836] [ 1362] 0 1362 1116 477 6 3 0 0 agetty
Jan 26 08:59:46 firewallfsi kernel: [104057.310281] [ 1578] 48 1578 49258 4298 45 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.310724] [ 1579] 48 1579 16216 4543 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311144] [ 1582] 48 1582 16220 4299 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311561] [ 1599] 48 1599 16386 4796 29 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.311966] [ 1604] 48 1604 16386 4558 29 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.312366] [ 1754] 0 1754 1704 632 6 3 0 0 sh
Jan 26 08:59:46 firewallfsi kernel: [104057.312773] [ 1755] 0 1755 2741 1664 9 3 0 0 udp-sgs
Jan 26 08:59:46 firewallfsi kernel: [104057.313167] [ 1759] 0 1759 9561 3551 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.313551] [ 1760] 0 1760 9197 1115 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.313931] [ 1761] 0 1761 9446 1192 22 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.314311] [ 1850] 0 1850 5116 2511 13 3 0 0 dhclient
Jan 26 08:59:46 firewallfsi kernel: [104057.314698] [ 1927] 0 1927 594 420 5 3 0 0 pptpd
Jan 26 08:59:46 firewallfsi kernel: [104057.315080] [ 1929] 0 1929 954 672 6 3 0 0 dovecot
Jan 26 08:59:46 firewallfsi kernel: [104057.315467] [ 1930] 97 1930 904 548 5 3 0 0 anvil
Jan 26 08:59:46 firewallfsi kernel: [104057.315853] [ 1931] 0 1931 937 581 5 3 0 0 log
Jan 26 08:59:46 firewallfsi kernel: [104057.316238] [ 1938] 0 1938 1899 739 7 3 0 0 crond
Jan 26 08:59:46 firewallfsi kernel: [104057.316629] [ 1939] 177 1939 6832 4716 17 3 0 0 dhcpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317013] [ 1941] 38 1941 1536 1054 6 3 0 0 ntpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317393] [ 1944] 48 1944 16216 4289 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.317783] [ 1947] 48 1947 16220 4571 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.318161] [ 1948] 48 1948 16220 4500 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.318539] [ 1960] 302 1960 1446 1049 7 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.318913] [ 1962] 301 1962 1647 1133 8 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.319282] [ 1965] 303 1965 1394 921 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.319652] [ 2090] 0 2090 4181 1978 11 3 5 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.320019] [ 2092] 0 2092 1584 1140 6 3 0 0 systemd
Jan 26 08:59:46 firewallfsi kernel: [104057.320390] [ 2093] 0 2093 6809 333 11 3 62 0 (sd-pam)
Jan 26 08:59:46 firewallfsi kernel: [104057.320771] [ 2096] 0 2096 4214 1103 11 3 1 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.321149] [ 2097] 0 2097 1239 989 6 3 0 0 tcsh
Jan 26 08:59:46 firewallfsi kernel: [104057.321536] [ 2410] 0 2410 1142 800 5 3 0 0 config
Jan 26 08:59:46 firewallfsi kernel: [104057.321916] [ 2619] 304 2619 1399 906 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.322295] [ 6076] 0 6076 1307 923 6 3 0 0 reboot-when-oom
Jan 26 08:59:46 firewallfsi kernel: [104057.322685] [10382] 48 10382 16216 4285 28 3 0 0 /usr/sbin/httpd
Jan 26 08:59:46 firewallfsi kernel: [104057.323076] [12318] 273 12318 2208 1470 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.323475] [12319] 301 12319 1672 1079 7 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.323863] [10772] 273 10772 2209 1439 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.324243] [10776] 301 10776 1396 1008 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.324630] [11099] 0 11099 11587 4409 26 3 0 0 smbd
Jan 26 08:59:46 firewallfsi kernel: [104057.325012] [12143] 0 12143 589 167 5 3 0 0 tail
Jan 26 08:59:46 firewallfsi kernel: [104057.325392] [18786] 276 18786 118337 104981 225 3 0 0 clamd
Jan 26 08:59:46 firewallfsi kernel: [104057.325779] [18801] 290 18801 13143 1160 15 3 0 0 clamav-milter
Jan 26 08:59:46 firewallfsi kernel: [104057.326164] [18822] 0 18822 3830 1760 11 3 0 0 sendmail
Jan 26 08:59:46 firewallfsi kernel: [104057.326550] [18838] 51 18838 3501 781 10 3 0 0 sendmail
Jan 26 08:59:46 firewallfsi kernel: [104057.326935] [18953] 23 18953 5440 723 13 3 0 0 squid
Jan 26 08:59:46 firewallfsi kernel: [104057.327324] [18955] 23 18955 9136 5994 20 3 0 0 squid
Jan 26 08:59:46 firewallfsi kernel: [104057.327720] [18956] 23 18956 1179 411 6 3 0 0 unlinkd
Jan 26 08:59:46 firewallfsi kernel: [104057.328116] [13386] 0 13386 1396 169 6 3 0 0 sleep
Jan 26 08:59:46 firewallfsi kernel: [104057.328520] [13604] 273 13604 2209 1435 8 3 0 0 imap-login
Jan 26 08:59:46 firewallfsi kernel: [104057.328918] [13608] 302 13608 1419 1042 6 3 0 0 imap
Jan 26 08:59:46 firewallfsi kernel: [104057.329316] [18231] 0 18231 4181 1967 11 3 0 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.329725] [18233] 299 18233 1603 1165 7 3 0 0 systemd
Jan 26 08:59:46 firewallfsi kernel: [104057.330148] [18234] 299 18234 6809 364 11 3 32 0 (sd-pam)
Jan 26 08:59:46 firewallfsi kernel: [104057.330564] [18237] 299 18237 4214 1127 11 3 0 0 sshd
Jan 26 08:59:46 firewallfsi kernel: [104057.330988] [18238] 299 18238 879 564 5 3 0 0 bash
Jan 26 08:59:46 firewallfsi kernel: [104057.331394] [18243] 299 18243 7292 6386 18 3 0 0 rdiff-backup
Jan 26 08:59:46 firewallfsi kernel: [104057.331810] [18244] 299 18244 619 220 4 3 0 0 pv
Jan 26 08:59:46 firewallfsi kernel: [104057.332238] [18905] 0 18905 1396 164 6 3 0 0 sleep
Jan 26 08:59:46 firewallfsi kernel: [104057.332661] Out of memory: Kill process 18786 (clamd) score 10 or sacrifice child
Jan 26 08:59:46 firewallfsi kernel: [104057.333107] Killed process 18786 (clamd) total-vm:473348kB, anon-rss:403644kB, file-rss:16280kB, shmem-rss:0kB
Jan 26 08:59:46 firewallfsi kernel: [104057.366106] audit: type=1131 audit(1485442786.940:3251): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Jan 26 08:59:47 firewallfsi kernel: [104057.493007] audit: type=1130 audit(1485442787.067:3252): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 26 08:59:47 firewallfsi kernel: [104057.495502] audit: type=1131 audit(1485442787.070:3253): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Jan 26 08:59:47 firewallfsi kernel: [104057.497128] audit: type=1130 audit(1485442787.071:3254): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=clamd@scan comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
next prev parent reply other threads:[~2017-01-26 23:19 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-11 10:32 mm, vmscan: commit makes PAE kernel crash nightly (bisected) Trevor Cordes
2017-01-11 12:11 ` Mel Gorman
2017-01-11 12:14 ` Mel Gorman
2017-01-11 22:52 ` Trevor Cordes
2017-01-12 9:36 ` Michal Hocko
2017-01-15 6:27 ` Trevor Cordes
2017-01-16 11:09 ` Mel Gorman
2017-01-17 13:52 ` Michal Hocko
2017-01-17 14:21 ` Mel Gorman
2017-01-17 14:54 ` Michal Hocko
2017-01-18 7:25 ` Trevor Cordes
2017-01-18 17:48 ` Mel Gorman
2017-01-18 18:07 ` Mel Gorman
2017-01-19 9:48 ` Trevor Cordes
2017-01-19 11:37 ` Michal Hocko
2017-01-20 6:35 ` Trevor Cordes
2017-01-20 11:02 ` Mel Gorman
2017-01-20 15:55 ` Mel Gorman
2017-01-23 0:45 ` Trevor Cordes
2017-01-23 10:48 ` Mel Gorman
2017-01-23 11:04 ` Mel Gorman
2017-01-25 9:46 ` Michal Hocko
2017-01-24 12:59 ` Michal Hocko
2017-01-25 10:02 ` Trevor Cordes
2017-01-25 12:04 ` Michal Hocko
2017-01-29 22:50 ` Trevor Cordes
2017-01-30 7:51 ` Michal Hocko
2017-02-01 9:29 ` Trevor Cordes
2017-02-01 10:14 ` Michal Hocko
2017-02-04 0:36 ` Trevor Cordes
2017-02-04 20:05 ` Rik van Riel
2017-02-05 10:03 ` Michal Hocko
2017-02-05 22:53 ` Trevor Cordes
2017-01-30 9:10 ` Mel Gorman
2017-01-24 12:54 ` Michal Hocko
2017-01-26 23:18 ` Trevor Cordes [this message]
2017-01-27 7:36 ` Michal Hocko
2017-01-24 12:51 ` Michal Hocko
2017-01-18 6:52 ` Trevor Cordes
2017-01-17 13:45 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170126171858.5e1b484a@pog.tecnopolis.ca \
--to=trevor@tecnopolis.ca \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@kernel.org \
--cc=minchan@kernel.org \
--cc=riel@surriel.com \
--cc=srikar@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.