From: Gerhard Wiesinger <lists@wiesinger.com>
To: Michal Hocko <mhocko@kernel.org>, lkml@pengaru.com
Cc: Minchan Kim <minchan@kernel.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: Still OOM problems with 4.9er/4.10er kernels
Date: Fri, 17 Mar 2017 17:37:48 +0100 [thread overview]
Message-ID: <a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com> (raw)
In-Reply-To: <20170316093931.GH30501@dhcp22.suse.cz>
On 16.03.2017 10:39, Michal Hocko wrote:
> On Thu 16-03-17 02:23:18, lkml@pengaru.com wrote:
>> On Thu, Mar 16, 2017 at 10:08:44AM +0100, Michal Hocko wrote:
>>> On Thu 16-03-17 01:47:33, lkml@pengaru.com wrote:
>>> [...]
>>>> While on the topic of understanding allocation stalls, Philip Freeman recently
>>>> mailed linux-kernel with a similar report, and in his case there are plenty of
>>>> page cache pages. It was also a GFP_HIGHUSER_MOVABLE 0-order allocation.
>>> care to point me to the report?
>> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06360.html
> Thanks. It is gone from my lkml mailbox. Could you CC me (and linux-mm) please?
>
>>>
>>>> I'm no MM expert, but it appears a bit broken for such a low-order allocation
>>>> to stall on the order of 10 seconds when there's plenty of reclaimable pages,
>>>> in addition to mostly unused and abundant swap space on SSD.
>>> yes this might indeed signal a problem.
>> Well maybe I missed something obvious that a better informed eye will catch.
> Nothing really obvious. There is indeed a lot of anonymous memory to
> swap out. Almost no pages on file LRU lists (active_file:759
> inactive_file:749) but 158783 total pagecache pages so we have to have a
> lot of pages in the swap cache. I would probably have to see more data
> to make a full picture.
>
Why does the kernel prefer to swapin/out and not use
a.) the free memory?
b.) the buffer/cache?
There is ~100M memory available but kernel swaps all the time ...
Any ideas?
Kernel: 4.9.14-200.fc25.x86_64
top - 17:33:43 up 28 min, 3 users, load average: 3.58, 1.67, 0.89
Tasks: 145 total, 4 running, 141 sleeping, 0 stopped, 0 zombie
%Cpu(s): 19.1 us, 56.2 sy, 0.0 ni, 4.3 id, 13.4 wa, 2.0 hi, 0.3 si,
4.7 st
KiB Mem : 230076 total, 61508 free, 123472 used, 45096 buff/cache
procs -----------memory---------- ---swap-- -----io---- -system--
------cpu-----
r b swpd free buff cache si so bi bo in cs us sy
id wa st
3 5 303916 60372 328 43864 27828 200 41420 236 6984 11138 11
47 6 23 14
5 4 292852 52904 756 58584 19600 448 48780 540 8088 10528 18
61 1 7 13
3 3 288792 49052 1152 65924 4856 576 9824 1100 4324 5720 7
18 2 64 8
2 2 283676 54160 716 67604 6332 344 31740 964 3879 5055 12 34
10 37 7
3 3 286852 66712 216 53136 28064 4832 56532 4920 9175 12625 10
55 12 14 10
2 0 299680 62428 196 53316 36312 13164 54728 13212 16820 25283
7 56 18 12 7
1 1 300756 63220 624 58160 17944 1260 24528 1304 5804 9302 3
22 38 34 3
Thnx.
Ciao,
Gerhard
WARNING: multiple messages have this Message-ID (diff)
From: Gerhard Wiesinger <lists@wiesinger.com>
To: Michal Hocko <mhocko@kernel.org>, lkml@pengaru.com
Cc: Minchan Kim <minchan@kernel.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: Still OOM problems with 4.9er/4.10er kernels
Date: Fri, 17 Mar 2017 17:37:48 +0100 [thread overview]
Message-ID: <a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com> (raw)
In-Reply-To: <20170316093931.GH30501@dhcp22.suse.cz>
On 16.03.2017 10:39, Michal Hocko wrote:
> On Thu 16-03-17 02:23:18, lkml@pengaru.com wrote:
>> On Thu, Mar 16, 2017 at 10:08:44AM +0100, Michal Hocko wrote:
>>> On Thu 16-03-17 01:47:33, lkml@pengaru.com wrote:
>>> [...]
>>>> While on the topic of understanding allocation stalls, Philip Freeman recently
>>>> mailed linux-kernel with a similar report, and in his case there are plenty of
>>>> page cache pages. It was also a GFP_HIGHUSER_MOVABLE 0-order allocation.
>>> care to point me to the report?
>> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06360.html
> Thanks. It is gone from my lkml mailbox. Could you CC me (and linux-mm) please?
>
>>>
>>>> I'm no MM expert, but it appears a bit broken for such a low-order allocation
>>>> to stall on the order of 10 seconds when there's plenty of reclaimable pages,
>>>> in addition to mostly unused and abundant swap space on SSD.
>>> yes this might indeed signal a problem.
>> Well maybe I missed something obvious that a better informed eye will catch.
> Nothing really obvious. There is indeed a lot of anonymous memory to
> swap out. Almost no pages on file LRU lists (active_file:759
> inactive_file:749) but 158783 total pagecache pages so we have to have a
> lot of pages in the swap cache. I would probably have to see more data
> to make a full picture.
>
Why does the kernel prefer to swapin/out and not use
a.) the free memory?
b.) the buffer/cache?
There is ~100M memory available but kernel swaps all the time ...
Any ideas?
Kernel: 4.9.14-200.fc25.x86_64
top - 17:33:43 up 28 min, 3 users, load average: 3.58, 1.67, 0.89
Tasks: 145 total, 4 running, 141 sleeping, 0 stopped, 0 zombie
%Cpu(s): 19.1 us, 56.2 sy, 0.0 ni, 4.3 id, 13.4 wa, 2.0 hi, 0.3 si,
4.7 st
KiB Mem : 230076 total, 61508 free, 123472 used, 45096 buff/cache
procs -----------memory---------- ---swap-- -----io---- -system--
------cpu-----
r b swpd free buff cache si so bi bo in cs us sy
id wa st
3 5 303916 60372 328 43864 27828 200 41420 236 6984 11138 11
47 6 23 14
5 4 292852 52904 756 58584 19600 448 48780 540 8088 10528 18
61 1 7 13
3 3 288792 49052 1152 65924 4856 576 9824 1100 4324 5720 7
18 2 64 8
2 2 283676 54160 716 67604 6332 344 31740 964 3879 5055 12 34
10 37 7
3 3 286852 66712 216 53136 28064 4832 56532 4920 9175 12625 10
55 12 14 10
2 0 299680 62428 196 53316 36312 13164 54728 13212 16820 25283
7 56 18 12 7
1 1 300756 63220 624 58160 17944 1260 24528 1304 5804 9302 3
22 38 34 3
Thnx.
Ciao,
Gerhard
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-03-17 16:47 UTC|newest]
Thread overview: 88+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-30 7:10 Still OOM problems with 4.9er kernels Gerhard Wiesinger
2016-11-30 7:20 ` Gerhard Wiesinger
2016-12-09 7:06 ` Gerhard Wiesinger
2016-12-09 13:40 ` Michal Hocko
2016-12-09 13:40 ` Michal Hocko
2016-12-09 15:52 ` Gerhard Wiesinger
2016-12-09 15:52 ` Gerhard Wiesinger
2016-12-09 15:58 ` Gerhard Wiesinger
2016-12-09 15:58 ` Gerhard Wiesinger
2016-12-09 16:09 ` Michal Hocko
2016-12-09 16:09 ` Michal Hocko
2016-12-09 16:58 ` Gerhard Wiesinger
2016-12-09 17:30 ` Michal Hocko
2016-12-09 17:30 ` Michal Hocko
2016-12-09 18:01 ` Gerhard Wiesinger
2016-12-09 18:01 ` Gerhard Wiesinger
2016-12-09 21:42 ` Vlastimil Babka
2016-12-09 21:42 ` Vlastimil Babka
2016-12-10 13:50 ` Gerhard Wiesinger
2016-12-10 13:50 ` Gerhard Wiesinger
2016-12-12 8:24 ` Michal Hocko
2016-12-12 8:24 ` Michal Hocko
2016-12-23 2:55 ` Minchan Kim
2016-12-23 2:55 ` Minchan Kim
2017-01-01 17:20 ` Gerhard Wiesinger
2017-01-01 17:20 ` Gerhard Wiesinger
2017-01-04 8:40 ` Gerhard Wiesinger
2017-01-04 9:11 ` Michal Hocko
2017-01-04 9:11 ` Michal Hocko
2017-02-26 8:40 ` Still OOM problems with 4.9er/4.10er kernels Gerhard Wiesinger
2017-02-27 8:27 ` Michal Hocko
2017-02-27 8:27 ` Michal Hocko
2017-02-28 6:06 ` Gerhard Wiesinger
2017-02-28 6:06 ` Gerhard Wiesinger
2017-02-28 8:14 ` Michal Hocko
2017-02-28 8:14 ` Michal Hocko
2017-02-27 9:02 ` Minchan Kim
2017-02-27 9:02 ` Minchan Kim
2017-02-27 9:44 ` Michal Hocko
2017-02-27 9:44 ` Michal Hocko
2017-02-28 5:17 ` Minchan Kim
2017-02-28 5:17 ` Minchan Kim
2017-02-28 8:12 ` Michal Hocko
2017-02-28 8:12 ` Michal Hocko
2017-03-02 7:17 ` Minchan Kim
2017-03-02 7:17 ` Minchan Kim
2017-03-16 6:38 ` Gerhard Wiesinger
2017-03-16 6:38 ` Gerhard Wiesinger
2017-03-16 8:27 ` Michal Hocko
2017-03-16 8:27 ` Michal Hocko
2017-03-16 8:47 ` lkml
2017-03-16 8:47 ` lkml
2017-03-16 9:08 ` Michal Hocko
2017-03-16 9:08 ` Michal Hocko
2017-03-16 9:23 ` lkml
2017-03-16 9:23 ` lkml
2017-03-16 9:39 ` Michal Hocko
2017-03-16 9:39 ` Michal Hocko
2017-03-17 16:37 ` Gerhard Wiesinger [this message]
2017-03-17 16:37 ` Gerhard Wiesinger
2017-03-17 17:13 ` Michal Hocko
2017-03-17 17:13 ` Michal Hocko
2017-03-17 20:08 ` Gerhard Wiesinger
2017-03-17 20:08 ` Gerhard Wiesinger
2017-03-19 8:17 ` Gerhard Wiesinger
2017-03-19 8:17 ` Gerhard Wiesinger
2017-03-20 1:54 ` Tetsuo Handa
2017-03-20 1:54 ` Tetsuo Handa
2017-03-19 15:18 ` Michal Hocko
2017-03-19 15:18 ` Michal Hocko
2017-03-19 16:02 ` Gerhard Wiesinger
2017-03-19 16:02 ` Gerhard Wiesinger
2017-03-20 3:05 ` Mike Galbraith
2017-03-20 3:05 ` Mike Galbraith
2017-03-21 5:59 ` Gerhard Wiesinger
2017-03-21 5:59 ` Gerhard Wiesinger
2017-03-21 7:13 ` Mike Galbraith
2017-03-21 7:13 ` Mike Galbraith
2017-03-23 7:16 ` Gerhard Wiesinger
2017-03-23 7:16 ` Gerhard Wiesinger
2017-03-23 8:38 ` Mike Galbraith
2017-03-23 8:38 ` Mike Galbraith
2017-03-23 14:46 ` Tetsuo Handa
2017-03-23 14:46 ` Tetsuo Handa
2017-03-26 8:36 ` Gerhard Wiesinger
2017-03-26 8:36 ` Gerhard Wiesinger
2016-12-09 16:03 ` Still OOM problems with 4.9er kernels Gerhard Wiesinger
2016-12-09 16:03 ` Gerhard Wiesinger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com \
--to=lists@wiesinger.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lkml@pengaru.com \
--cc=mhocko@kernel.org \
--cc=minchan@kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.