All of lore.kernel.org
 help / color / mirror / Atom feed
From: Gerhard Wiesinger <lists@wiesinger.com>
To: Michal Hocko <mhocko@kernel.org>, lkml@pengaru.com
Cc: Minchan Kim <minchan@kernel.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: Still OOM problems with 4.9er/4.10er kernels
Date: Fri, 17 Mar 2017 17:37:48 +0100	[thread overview]
Message-ID: <a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com> (raw)
In-Reply-To: <20170316093931.GH30501@dhcp22.suse.cz>

On 16.03.2017 10:39, Michal Hocko wrote:
> On Thu 16-03-17 02:23:18, lkml@pengaru.com wrote:
>> On Thu, Mar 16, 2017 at 10:08:44AM +0100, Michal Hocko wrote:
>>> On Thu 16-03-17 01:47:33, lkml@pengaru.com wrote:
>>> [...]
>>>> While on the topic of understanding allocation stalls, Philip Freeman recently
>>>> mailed linux-kernel with a similar report, and in his case there are plenty of
>>>> page cache pages.  It was also a GFP_HIGHUSER_MOVABLE 0-order allocation.
>>> care to point me to the report?
>> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06360.html
> Thanks. It is gone from my lkml mailbox. Could you CC me (and linux-mm) please?
>   
>>>   
>>>> I'm no MM expert, but it appears a bit broken for such a low-order allocation
>>>> to stall on the order of 10 seconds when there's plenty of reclaimable pages,
>>>> in addition to mostly unused and abundant swap space on SSD.
>>> yes this might indeed signal a problem.
>> Well maybe I missed something obvious that a better informed eye will catch.
> Nothing really obvious. There is indeed a lot of anonymous memory to
> swap out. Almost no pages on file LRU lists (active_file:759
> inactive_file:749) but 158783 total pagecache pages so we have to have a
> lot of pages in the swap cache. I would probably have to see more data
> to make a full picture.
>

Why does the kernel prefer to swapin/out and not use

a.) the free memory?

b.) the buffer/cache?

There is ~100M memory available but kernel swaps all the time ...

Any ideas?

Kernel: 4.9.14-200.fc25.x86_64

top - 17:33:43 up 28 min,  3 users,  load average: 3.58, 1.67, 0.89
Tasks: 145 total,   4 running, 141 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.1 us, 56.2 sy,  0.0 ni,  4.3 id, 13.4 wa, 2.0 hi,  0.3 si,  
4.7 st
KiB Mem :   230076 total,    61508 free,   123472 used,    45096 buff/cache

procs -----------memory---------- ---swap-- -----io---- -system-- 
------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo in   cs us sy 
id wa st
  3  5 303916  60372    328  43864 27828  200 41420   236 6984 11138 11 
47  6 23 14
  5  4 292852  52904    756  58584 19600  448 48780   540 8088 10528 18 
61  1  7 13
  3  3 288792  49052   1152  65924 4856  576  9824  1100 4324 5720  7 
18  2 64  8
  2  2 283676  54160    716  67604 6332  344 31740   964 3879 5055 12 34 
10 37  7
  3  3 286852  66712    216  53136 28064 4832 56532  4920 9175 12625 10 
55 12 14 10
  2  0 299680  62428    196  53316 36312 13164 54728 13212 16820 25283  
7 56 18 12  7
  1  1 300756  63220    624  58160 17944 1260 24528  1304 5804 9302  3 
22 38 34  3

Thnx.


Ciao,

Gerhard

WARNING: multiple messages have this Message-ID (diff)
From: Gerhard Wiesinger <lists@wiesinger.com>
To: Michal Hocko <mhocko@kernel.org>, lkml@pengaru.com
Cc: Minchan Kim <minchan@kernel.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: Still OOM problems with 4.9er/4.10er kernels
Date: Fri, 17 Mar 2017 17:37:48 +0100	[thread overview]
Message-ID: <a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com> (raw)
In-Reply-To: <20170316093931.GH30501@dhcp22.suse.cz>

On 16.03.2017 10:39, Michal Hocko wrote:
> On Thu 16-03-17 02:23:18, lkml@pengaru.com wrote:
>> On Thu, Mar 16, 2017 at 10:08:44AM +0100, Michal Hocko wrote:
>>> On Thu 16-03-17 01:47:33, lkml@pengaru.com wrote:
>>> [...]
>>>> While on the topic of understanding allocation stalls, Philip Freeman recently
>>>> mailed linux-kernel with a similar report, and in his case there are plenty of
>>>> page cache pages.  It was also a GFP_HIGHUSER_MOVABLE 0-order allocation.
>>> care to point me to the report?
>> http://lkml.iu.edu/hypermail/linux/kernel/1703.1/06360.html
> Thanks. It is gone from my lkml mailbox. Could you CC me (and linux-mm) please?
>   
>>>   
>>>> I'm no MM expert, but it appears a bit broken for such a low-order allocation
>>>> to stall on the order of 10 seconds when there's plenty of reclaimable pages,
>>>> in addition to mostly unused and abundant swap space on SSD.
>>> yes this might indeed signal a problem.
>> Well maybe I missed something obvious that a better informed eye will catch.
> Nothing really obvious. There is indeed a lot of anonymous memory to
> swap out. Almost no pages on file LRU lists (active_file:759
> inactive_file:749) but 158783 total pagecache pages so we have to have a
> lot of pages in the swap cache. I would probably have to see more data
> to make a full picture.
>

Why does the kernel prefer to swapin/out and not use

a.) the free memory?

b.) the buffer/cache?

There is ~100M memory available but kernel swaps all the time ...

Any ideas?

Kernel: 4.9.14-200.fc25.x86_64

top - 17:33:43 up 28 min,  3 users,  load average: 3.58, 1.67, 0.89
Tasks: 145 total,   4 running, 141 sleeping,   0 stopped,   0 zombie
%Cpu(s): 19.1 us, 56.2 sy,  0.0 ni,  4.3 id, 13.4 wa, 2.0 hi,  0.3 si,  
4.7 st
KiB Mem :   230076 total,    61508 free,   123472 used,    45096 buff/cache

procs -----------memory---------- ---swap-- -----io---- -system-- 
------cpu-----
  r  b   swpd   free   buff  cache   si   so    bi    bo in   cs us sy 
id wa st
  3  5 303916  60372    328  43864 27828  200 41420   236 6984 11138 11 
47  6 23 14
  5  4 292852  52904    756  58584 19600  448 48780   540 8088 10528 18 
61  1  7 13
  3  3 288792  49052   1152  65924 4856  576  9824  1100 4324 5720  7 
18  2 64  8
  2  2 283676  54160    716  67604 6332  344 31740   964 3879 5055 12 34 
10 37  7
  3  3 286852  66712    216  53136 28064 4832 56532  4920 9175 12625 10 
55 12 14 10
  2  0 299680  62428    196  53316 36312 13164 54728 13212 16820 25283  
7 56 18 12  7
  1  1 300756  63220    624  58160 17944 1260 24528  1304 5804 9302  3 
22 38 34  3

Thnx.


Ciao,

Gerhard

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-03-17 16:47 UTC|newest]

Thread overview: 88+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-11-30  7:10 Still OOM problems with 4.9er kernels Gerhard Wiesinger
2016-11-30  7:20 ` Gerhard Wiesinger
2016-12-09  7:06   ` Gerhard Wiesinger
2016-12-09 13:40     ` Michal Hocko
2016-12-09 13:40       ` Michal Hocko
2016-12-09 15:52       ` Gerhard Wiesinger
2016-12-09 15:52         ` Gerhard Wiesinger
2016-12-09 15:58         ` Gerhard Wiesinger
2016-12-09 15:58           ` Gerhard Wiesinger
2016-12-09 16:09         ` Michal Hocko
2016-12-09 16:09           ` Michal Hocko
2016-12-09 16:58           ` Gerhard Wiesinger
2016-12-09 17:30             ` Michal Hocko
2016-12-09 17:30               ` Michal Hocko
2016-12-09 18:01               ` Gerhard Wiesinger
2016-12-09 18:01                 ` Gerhard Wiesinger
2016-12-09 21:42                 ` Vlastimil Babka
2016-12-09 21:42                   ` Vlastimil Babka
2016-12-10 13:50                   ` Gerhard Wiesinger
2016-12-10 13:50                     ` Gerhard Wiesinger
2016-12-12  8:24                     ` Michal Hocko
2016-12-12  8:24                       ` Michal Hocko
2016-12-23  2:55         ` Minchan Kim
2016-12-23  2:55           ` Minchan Kim
2017-01-01 17:20           ` Gerhard Wiesinger
2017-01-01 17:20             ` Gerhard Wiesinger
2017-01-04  8:40           ` Gerhard Wiesinger
2017-01-04  9:11             ` Michal Hocko
2017-01-04  9:11               ` Michal Hocko
2017-02-26  8:40               ` Still OOM problems with 4.9er/4.10er kernels Gerhard Wiesinger
2017-02-27  8:27                 ` Michal Hocko
2017-02-27  8:27                   ` Michal Hocko
2017-02-28  6:06                   ` Gerhard Wiesinger
2017-02-28  6:06                     ` Gerhard Wiesinger
2017-02-28  8:14                     ` Michal Hocko
2017-02-28  8:14                       ` Michal Hocko
2017-02-27  9:02                 ` Minchan Kim
2017-02-27  9:02                   ` Minchan Kim
2017-02-27  9:44                   ` Michal Hocko
2017-02-27  9:44                     ` Michal Hocko
2017-02-28  5:17                     ` Minchan Kim
2017-02-28  5:17                       ` Minchan Kim
2017-02-28  8:12                       ` Michal Hocko
2017-02-28  8:12                         ` Michal Hocko
2017-03-02  7:17                         ` Minchan Kim
2017-03-02  7:17                           ` Minchan Kim
2017-03-16  6:38                           ` Gerhard Wiesinger
2017-03-16  6:38                             ` Gerhard Wiesinger
2017-03-16  8:27                             ` Michal Hocko
2017-03-16  8:27                               ` Michal Hocko
2017-03-16  8:47                               ` lkml
2017-03-16  8:47                                 ` lkml
2017-03-16  9:08                                 ` Michal Hocko
2017-03-16  9:08                                   ` Michal Hocko
2017-03-16  9:23                                   ` lkml
2017-03-16  9:23                                     ` lkml
2017-03-16  9:39                                     ` Michal Hocko
2017-03-16  9:39                                       ` Michal Hocko
2017-03-17 16:37                                       ` Gerhard Wiesinger [this message]
2017-03-17 16:37                                         ` Gerhard Wiesinger
2017-03-17 17:13                                         ` Michal Hocko
2017-03-17 17:13                                           ` Michal Hocko
2017-03-17 20:08                                           ` Gerhard Wiesinger
2017-03-17 20:08                                             ` Gerhard Wiesinger
2017-03-19  8:17                                             ` Gerhard Wiesinger
2017-03-19  8:17                                               ` Gerhard Wiesinger
2017-03-20  1:54                                               ` Tetsuo Handa
2017-03-20  1:54                                                 ` Tetsuo Handa
2017-03-19 15:18                                             ` Michal Hocko
2017-03-19 15:18                                               ` Michal Hocko
2017-03-19 16:02                                               ` Gerhard Wiesinger
2017-03-19 16:02                                                 ` Gerhard Wiesinger
2017-03-20  3:05                                                 ` Mike Galbraith
2017-03-20  3:05                                                   ` Mike Galbraith
2017-03-21  5:59                                                   ` Gerhard Wiesinger
2017-03-21  5:59                                                     ` Gerhard Wiesinger
2017-03-21  7:13                                                     ` Mike Galbraith
2017-03-21  7:13                                                       ` Mike Galbraith
2017-03-23  7:16                                                       ` Gerhard Wiesinger
2017-03-23  7:16                                                         ` Gerhard Wiesinger
2017-03-23  8:38                                                         ` Mike Galbraith
2017-03-23  8:38                                                           ` Mike Galbraith
2017-03-23 14:46                                                           ` Tetsuo Handa
2017-03-23 14:46                                                             ` Tetsuo Handa
2017-03-26  8:36                                                           ` Gerhard Wiesinger
2017-03-26  8:36                                                             ` Gerhard Wiesinger
2016-12-09 16:03       ` Still OOM problems with 4.9er kernels Gerhard Wiesinger
2016-12-09 16:03         ` Gerhard Wiesinger

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a65e4b73-5c97-d915-c79e-7df0771db823@wiesinger.com \
    --to=lists@wiesinger.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lkml@pengaru.com \
    --cc=mhocko@kernel.org \
    --cc=minchan@kernel.org \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.