OOM killer kicks in after minutes or never

OOM killer kicks in after minutes or never

[Marcin Szewczyk]

Hi,

In 2010 I noticed that viewing many GIFs in a row using gpicview renders
my Linux unresponsive. There is very little I can do in such a
situation. Rarely after some minutes the OOM killer kicks in and saves
the day. Nevertheless, usually I end up using Alt+SysRq+B.

This is the second computer I can observe this problem on. First was
Asus EeePC 1000 with Atom N270 and now I have Lenovo S210 with Celeron
1037U.

What happens is gpicview exhausting whole available memory in such a
pattern that userspace becomes unresponsive. I cannot switch to another
terminal either. I have written a tool that allocates memory in a very
similar way using GDK -- https://github.com/wodny/crasher.

I have also uploaded some logs to the repository -- top, iostat (showing
a lot of reads during an episode), dmesg.

I suppose the OS starts to oscillate between freeing memory, cleaning
caches and buffers, and loading some new data (see iostat logs).

Currently I am using Debian Jessie with the following kernel:
3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 GNU/Linux

I can observe the most impressive effects on my physical machine
(logs/ph-*). On a VM (logs/vm-*) usually the OOM killer kills the
process after a short time (5-120 seconds).

Possible factors differentiating cases of recovering in seconds from
recoveries after minutes (or never):
- another memory-consuming process running (e.g. Firefox),
- physical machine or a VM (see dmesg logs),
- chipset and associated kernel functions (see dmesg logs).

Things that seem irrelevant (after testing):
- running the application in Xorg or a TTY,
- LUKS encryption of the root filesystem,
- vm.oom_kill_allocating_task setting.

What can I do to diagnose the problem further?


(Sorry if a duplicate appears)

-- 
Marcin Szewczyk                       http://wodny.org
mailto:Marcin.Szewczyk@wodny.borg  <- remove b / usuń b
xmpp:wodny@ubuntu.pl                  xmpp:wodny@jabster.pl

Re: OOM killer kicks in after minutes or never

[Vlastimil Babka]

+CC so this doesn't get lost

On 21.12.2015 13:35, Marcin Szewczyk wrote:
> Hi,
> 
> In 2010 I noticed that viewing many GIFs in a row using gpicview renders
> my Linux unresponsive. There is very little I can do in such a
> situation. Rarely after some minutes the OOM killer kicks in and saves
> the day. Nevertheless, usually I end up using Alt+SysRq+B.
> 
> This is the second computer I can observe this problem on. First was
> Asus EeePC 1000 with Atom N270 and now I have Lenovo S210 with Celeron
> 1037U.
> 
> What happens is gpicview exhausting whole available memory in such a
> pattern that userspace becomes unresponsive. I cannot switch to another
> terminal either. I have written a tool that allocates memory in a very
> similar way using GDK -- https://github.com/wodny/crasher.
> 
> I have also uploaded some logs to the repository -- top, iostat (showing
> a lot of reads during an episode), dmesg.
> 
> I suppose the OS starts to oscillate between freeing memory, cleaning
> caches and buffers, and loading some new data (see iostat logs).
> 
> Currently I am using Debian Jessie with the following kernel:
> 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 GNU/Linux
> 
> I can observe the most impressive effects on my physical machine
> (logs/ph-*). On a VM (logs/vm-*) usually the OOM killer kills the
> process after a short time (5-120 seconds).
> 
> Possible factors differentiating cases of recovering in seconds from
> recoveries after minutes (or never):
> - another memory-consuming process running (e.g. Firefox),
> - physical machine or a VM (see dmesg logs),
> - chipset and associated kernel functions (see dmesg logs).
> 
> Things that seem irrelevant (after testing):
> - running the application in Xorg or a TTY,
> - LUKS encryption of the root filesystem,
> - vm.oom_kill_allocating_task setting.
> 
> What can I do to diagnose the problem further?
> 
> 
> (Sorry if a duplicate appears)
> 

Re: OOM killer kicks in after minutes or never

[Vlastimil Babka]

+CC so this doesn't get lost

On 21.12.2015 13:35, Marcin Szewczyk wrote:
> Hi,
> 
> In 2010 I noticed that viewing many GIFs in a row using gpicview renders
> my Linux unresponsive. There is very little I can do in such a
> situation. Rarely after some minutes the OOM killer kicks in and saves
> the day. Nevertheless, usually I end up using Alt+SysRq+B.
> 
> This is the second computer I can observe this problem on. First was
> Asus EeePC 1000 with Atom N270 and now I have Lenovo S210 with Celeron
> 1037U.
> 
> What happens is gpicview exhausting whole available memory in such a
> pattern that userspace becomes unresponsive. I cannot switch to another
> terminal either. I have written a tool that allocates memory in a very
> similar way using GDK -- https://github.com/wodny/crasher.
> 
> I have also uploaded some logs to the repository -- top, iostat (showing
> a lot of reads during an episode), dmesg.
> 
> I suppose the OS starts to oscillate between freeing memory, cleaning
> caches and buffers, and loading some new data (see iostat logs).
> 
> Currently I am using Debian Jessie with the following kernel:
> 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 GNU/Linux
> 
> I can observe the most impressive effects on my physical machine
> (logs/ph-*). On a VM (logs/vm-*) usually the OOM killer kills the
> process after a short time (5-120 seconds).
> 
> Possible factors differentiating cases of recovering in seconds from
> recoveries after minutes (or never):
> - another memory-consuming process running (e.g. Firefox),
> - physical machine or a VM (see dmesg logs),
> - chipset and associated kernel functions (see dmesg logs).
> 
> Things that seem irrelevant (after testing):
> - running the application in Xorg or a TTY,
> - LUKS encryption of the root filesystem,
> - vm.oom_kill_allocating_task setting.
> 
> What can I do to diagnose the problem further?
> 
> 
> (Sorry if a duplicate appears)
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: OOM killer kicks in after minutes or never

[Marcin Szewczyk]

On Thu, Dec 24, 2015 at 07:27:14AM +0100, Vlastimil Babka wrote:
> +CC so this doesn't get lost
> 
> On 21.12.2015 13:35, Marcin Szewczyk wrote:
> > In 2010 I noticed that viewing many GIFs in a row using gpicview renders
> > my Linux unresponsive. There is very little I can do in such a
> > situation. Rarely after some minutes the OOM killer kicks in and saves
> > the day. Nevertheless, usually I end up using Alt+SysRq+B.

Hi,

I thought that due to high throughput of the linux-kernel mailing list
my email will not get a reply if it didn't happen in a day so I allowed
myself to write another email to linux-mm as well as it was suggested to
me on #debian-kernel.

The email is here:
Subject: Exhausting memory makes the system unresponsive but doesn't
  invoke OOM killer
Message-ID: <20151223143109.GC3519@orkisz>
http://marc.info/?t=145088116000002&r=1&w=2

I have also updated the description in the repository:
https://github.com/wodny/crasher

Contrary to my original suspicion the OOM killer doesn't need much time
to clean up, it just isn't invoked. Johannes Weiner explained the
probable cause to me in a response in the linux-mm thread.


-- 
Marcin Szewczyk                       http://wodny.org
mailto:Marcin.Szewczyk@wodny.borg  <- remove b / usuń b
xmpp:wodny@ubuntu.pl                  xmpp:wodny@jabster.pl

Re: OOM killer kicks in after minutes or never

[Marcin Szewczyk]

On Thu, Dec 24, 2015 at 07:27:14AM +0100, Vlastimil Babka wrote:
> +CC so this doesn't get lost
> 
> On 21.12.2015 13:35, Marcin Szewczyk wrote:
> > In 2010 I noticed that viewing many GIFs in a row using gpicview renders
> > my Linux unresponsive. There is very little I can do in such a
> > situation. Rarely after some minutes the OOM killer kicks in and saves
> > the day. Nevertheless, usually I end up using Alt+SysRq+B.

Hi,

I thought that due to high throughput of the linux-kernel mailing list
my email will not get a reply if it didn't happen in a day so I allowed
myself to write another email to linux-mm as well as it was suggested to
me on #debian-kernel.

The email is here:
Subject: Exhausting memory makes the system unresponsive but doesn't
  invoke OOM killer
Message-ID: <20151223143109.GC3519@orkisz>
http://marc.info/?t=145088116000002&r=1&w=2

I have also updated the description in the repository:
https://github.com/wodny/crasher

Contrary to my original suspicion the OOM killer doesn't need much time
to clean up, it just isn't invoked. Johannes Weiner explained the
probable cause to me in a response in the linux-mm thread.


-- 
Marcin Szewczyk                       http://wodny.org
mailto:Marcin.Szewczyk@wodny.borg  <- remove b / usuA? b
xmpp:wodny@ubuntu.pl                  xmpp:wodny@jabster.pl

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

OOM killer kicks in after minutes or never

[Marcin Szewczyk]

Hi,

In 2010 I noticed that viewing many GIFs in a row using gpicview renders
my Linux unresponsive. There is very little I can do in such a
situation. Rarely after some minutes the OOM killer kicks in and saves
the day. Nevertheless, usually I end up using Alt+SysRq+B.

This is the second computer I can observe this problem on. First was
Asus EeePC 1000 with Atom N270 and now I have Lenovo S210 with Celeron
1037U.

What happens is gpicview exhausting whole available memory in such a
pattern that userspace becomes unresponsive. I cannot switch to another
terminal either. I have written a tool that allocates memory in a very
similar way using GDK -- https://github.com/wodny/crasher.

I have also uploaded some logs to the repository -- top, iostat (showing
a lot of reads during an episode), dmesg.

I suppose the OS starts to oscillate between freeing memory, cleaning
caches and buffers, and loading some new data (see iostat logs).

Currently I am using Debian Jessie with the following kernel:
3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09) x86_64 GNU/Linux

I can observe the most impressive effects on my physical machine
(logs/ph-*). On a VM (logs/vm-*) usually the OOM killer kills the
process after a short time (5-120 seconds).

Possible factors differentiating cases of recovering in seconds from
recoveries after minutes (or never):
- another memory-consuming process running (e.g. Firefox),
- physical machine or a VM (see dmesg logs),
- chipset and associated kernel functions (see dmesg logs).

Things that seem irrelevant (after testing):
- running the application in Xorg or a TTY,
- LUKS encryption of the root filesystem,
- vm.oom_kill_allocating_task setting.

What can I do to diagnose the problem further?

-- 
Marcin Szewczyk                       http://wodny.org
mailto:Marcin.Szewczyk@wodny.borg  <- remove b / usuń b
xmpp:wodny@ubuntu.pl                  xmpp:wodny@jabster.pl