From: Michal Hocko <mhocko@kernel.org> To: Gerhard Wiesinger <lists@wiesinger.com> Cc: Vlastimil Babka <vbabka@suse.cz>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds <torvalds@linux-foundation.org> Subject: Re: Still OOM problems with 4.9er kernels Date: Mon, 12 Dec 2016 09:24:29 +0100 [thread overview] Message-ID: <20161212082429.GA18163@dhcp22.suse.cz> (raw) In-Reply-To: <5e7490ea-4e59-7965-bc4d-171f9d60e439@wiesinger.com> On Sat 10-12-16 14:50:34, Gerhard Wiesinger wrote: [...] > IMHO: The OOM killer should NOT kick in even on the highest workloads if > there is swap available. This is not so simple. Take a heavy swap trashing situation as an example. You still have a lot of swap space and an anonymous memory which can be swapped out but if the workload simply keeps refaulting the swapped out memory all the time then you can barely make any further progress and end up in the swap IO hell. Invoking the OOM killer in such a situation would be a relief to make the system usable again. > https://www.spinics.net/lists/linux-mm/msg113665.html > > Yeah, but I do think that "oom when you have 156MB free and 7GB > reclaimable, and haven't even tried swapping" counts as obviously > wrong. No question about this part of course. > So Linus also thinks that trying swapping is a must have. And there > always was enough swap available in my cases. Then it should swap > out/swapin all the time (which worked well in kernel 2.4/2.6 times). > > Another topic: Why does the kernel prefer to swap in/swap out instead of use > cache pages/buffers (see vmstat 1 output below)? In the vast majority cases it is quite contrary. We heavily bias page cache reclaim in favor of the anonymous memory. Have a look at get_scan_count function which determines the balance. I would need to see /proc/vmstat collected during this time period to tell you more about why the particular balance was used. [...] > With kernel 4.7./4.8 it was really reaproduceable at every dnf update. With > 4.9rc8 it has been much much better. So something must have changed, too. that is good to hear but I it would be much better to collect reclaim related data so that we can analyse what is actually going on here. > As far as I understood it the order is 2^order kB pagesize. I don't think it > makes a difference when swap is not used which order the memory allocation > request is. > > BTW: What were the commit that introduced the regression anf fixed it in > 4.9? I cannot really answer this without actually understanding what is going on here and for that I do not have the relevant data. -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org> To: Gerhard Wiesinger <lists@wiesinger.com> Cc: Vlastimil Babka <vbabka@suse.cz>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds <torvalds@linux-foundation.org> Subject: Re: Still OOM problems with 4.9er kernels Date: Mon, 12 Dec 2016 09:24:29 +0100 [thread overview] Message-ID: <20161212082429.GA18163@dhcp22.suse.cz> (raw) In-Reply-To: <5e7490ea-4e59-7965-bc4d-171f9d60e439@wiesinger.com> On Sat 10-12-16 14:50:34, Gerhard Wiesinger wrote: [...] > IMHO: The OOM killer should NOT kick in even on the highest workloads if > there is swap available. This is not so simple. Take a heavy swap trashing situation as an example. You still have a lot of swap space and an anonymous memory which can be swapped out but if the workload simply keeps refaulting the swapped out memory all the time then you can barely make any further progress and end up in the swap IO hell. Invoking the OOM killer in such a situation would be a relief to make the system usable again. > https://www.spinics.net/lists/linux-mm/msg113665.html > > Yeah, but I do think that "oom when you have 156MB free and 7GB > reclaimable, and haven't even tried swapping" counts as obviously > wrong. No question about this part of course. > So Linus also thinks that trying swapping is a must have. And there > always was enough swap available in my cases. Then it should swap > out/swapin all the time (which worked well in kernel 2.4/2.6 times). > > Another topic: Why does the kernel prefer to swap in/swap out instead of use > cache pages/buffers (see vmstat 1 output below)? In the vast majority cases it is quite contrary. We heavily bias page cache reclaim in favor of the anonymous memory. Have a look at get_scan_count function which determines the balance. I would need to see /proc/vmstat collected during this time period to tell you more about why the particular balance was used. [...] > With kernel 4.7./4.8 it was really reaproduceable at every dnf update. With > 4.9rc8 it has been much much better. So something must have changed, too. that is good to hear but I it would be much better to collect reclaim related data so that we can analyse what is actually going on here. > As far as I understood it the order is 2^order kB pagesize. I don't think it > makes a difference when swap is not used which order the memory allocation > request is. > > BTW: What were the commit that introduced the regression anf fixed it in > 4.9? I cannot really answer this without actually understanding what is going on here and for that I do not have the relevant data. -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-12-12 8:24 UTC|newest] Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-11-30 7:10 Still OOM problems with 4.9er kernels Gerhard Wiesinger 2016-11-30 7:20 ` Gerhard Wiesinger 2016-12-09 7:06 ` Gerhard Wiesinger 2016-12-09 13:40 ` Michal Hocko 2016-12-09 13:40 ` Michal Hocko 2016-12-09 15:52 ` Gerhard Wiesinger 2016-12-09 15:52 ` Gerhard Wiesinger 2016-12-09 15:58 ` Gerhard Wiesinger 2016-12-09 15:58 ` Gerhard Wiesinger 2016-12-09 16:09 ` Michal Hocko 2016-12-09 16:09 ` Michal Hocko 2016-12-09 16:58 ` Gerhard Wiesinger 2016-12-09 17:30 ` Michal Hocko 2016-12-09 17:30 ` Michal Hocko 2016-12-09 18:01 ` Gerhard Wiesinger 2016-12-09 18:01 ` Gerhard Wiesinger 2016-12-09 21:42 ` Vlastimil Babka 2016-12-09 21:42 ` Vlastimil Babka 2016-12-10 13:50 ` Gerhard Wiesinger 2016-12-10 13:50 ` Gerhard Wiesinger 2016-12-12 8:24 ` Michal Hocko [this message] 2016-12-12 8:24 ` Michal Hocko 2016-12-23 2:55 ` Minchan Kim 2016-12-23 2:55 ` Minchan Kim 2017-01-01 17:20 ` Gerhard Wiesinger 2017-01-01 17:20 ` Gerhard Wiesinger 2017-01-04 8:40 ` Gerhard Wiesinger 2017-01-04 9:11 ` Michal Hocko 2017-01-04 9:11 ` Michal Hocko 2017-02-26 8:40 ` Still OOM problems with 4.9er/4.10er kernels Gerhard Wiesinger 2017-02-27 8:27 ` Michal Hocko 2017-02-27 8:27 ` Michal Hocko 2017-02-28 6:06 ` Gerhard Wiesinger 2017-02-28 6:06 ` Gerhard Wiesinger 2017-02-28 8:14 ` Michal Hocko 2017-02-28 8:14 ` Michal Hocko 2017-02-27 9:02 ` Minchan Kim 2017-02-27 9:02 ` Minchan Kim 2017-02-27 9:44 ` Michal Hocko 2017-02-27 9:44 ` Michal Hocko 2017-02-28 5:17 ` Minchan Kim 2017-02-28 5:17 ` Minchan Kim 2017-02-28 8:12 ` Michal Hocko 2017-02-28 8:12 ` Michal Hocko 2017-03-02 7:17 ` Minchan Kim 2017-03-02 7:17 ` Minchan Kim 2017-03-16 6:38 ` Gerhard Wiesinger 2017-03-16 6:38 ` Gerhard Wiesinger 2017-03-16 8:27 ` Michal Hocko 2017-03-16 8:27 ` Michal Hocko 2017-03-16 8:47 ` lkml 2017-03-16 8:47 ` lkml 2017-03-16 9:08 ` Michal Hocko 2017-03-16 9:08 ` Michal Hocko 2017-03-16 9:23 ` lkml 2017-03-16 9:23 ` lkml 2017-03-16 9:39 ` Michal Hocko 2017-03-16 9:39 ` Michal Hocko 2017-03-17 16:37 ` Gerhard Wiesinger 2017-03-17 16:37 ` Gerhard Wiesinger 2017-03-17 17:13 ` Michal Hocko 2017-03-17 17:13 ` Michal Hocko 2017-03-17 20:08 ` Gerhard Wiesinger 2017-03-17 20:08 ` Gerhard Wiesinger 2017-03-19 8:17 ` Gerhard Wiesinger 2017-03-19 8:17 ` Gerhard Wiesinger 2017-03-20 1:54 ` Tetsuo Handa 2017-03-20 1:54 ` Tetsuo Handa 2017-03-19 15:18 ` Michal Hocko 2017-03-19 15:18 ` Michal Hocko 2017-03-19 16:02 ` Gerhard Wiesinger 2017-03-19 16:02 ` Gerhard Wiesinger 2017-03-20 3:05 ` Mike Galbraith 2017-03-20 3:05 ` Mike Galbraith 2017-03-21 5:59 ` Gerhard Wiesinger 2017-03-21 5:59 ` Gerhard Wiesinger 2017-03-21 7:13 ` Mike Galbraith 2017-03-21 7:13 ` Mike Galbraith 2017-03-23 7:16 ` Gerhard Wiesinger 2017-03-23 7:16 ` Gerhard Wiesinger 2017-03-23 8:38 ` Mike Galbraith 2017-03-23 8:38 ` Mike Galbraith 2017-03-23 14:46 ` Tetsuo Handa 2017-03-23 14:46 ` Tetsuo Handa 2017-03-26 8:36 ` Gerhard Wiesinger 2017-03-26 8:36 ` Gerhard Wiesinger 2016-12-09 16:03 ` Still OOM problems with 4.9er kernels Gerhard Wiesinger 2016-12-09 16:03 ` Gerhard Wiesinger
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20161212082429.GA18163@dhcp22.suse.cz \ --to=mhocko@kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=lists@wiesinger.com \ --cc=torvalds@linux-foundation.org \ --cc=vbabka@suse.cz \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.