linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Kernel call chain search tool?
@ 2002-09-25  4:36 Dan Kegel
  2002-09-26 15:15 ` Alan Cox
  0 siblings, 1 reply; 5+ messages in thread
From: Dan Kegel @ 2002-09-25  4:36 UTC (permalink / raw)
  To: linux-kernel

<prelude>
I have a large multithreaded program that has a habit of using too
much memory, and as a safeguard, I want to kill it before it makes
the system unstable.  The OOM killer often guesses wrong, and RLIMIT_AS
kills too soon because of the address space used up by the many thread
stacks.
So I'd like an RLIMIT_RSS that just kills the fat process.

There have been a couple patches to implement RLIMIT_RSS, e.g.
Peter Chubb's
http://marc.theaimsgroup.com/?l=linux-kernel&m=97892951101598&w=2
and the two from Rik, all of which are too complex for my needs
(and which only swap out instead of kill), so I guess I have to roll my
own.
</prelude>

Rik's patch checks rss in handle_mm_fault(); Peter's
checked it in do_swap_page() and do_anonymous_page().
As a kernel newbie, I don't have a feel for how
those calls relate to each other.  Is there a tool 
somewhere that will take a set of function names and 
list all the kernel call chains that start in one of
the functions and end in another?

- Dan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Kernel call chain search tool?
  2002-09-25  4:36 Kernel call chain search tool? Dan Kegel
@ 2002-09-26 15:15 ` Alan Cox
  2002-09-26 16:17   ` Dan Kegel
  0 siblings, 1 reply; 5+ messages in thread
From: Alan Cox @ 2002-09-26 15:15 UTC (permalink / raw)
  To: Dan Kegel; +Cc: linux-kernel

On Wed, 2002-09-25 at 05:36, Dan Kegel wrote:
> <prelude>
> I have a large multithreaded program that has a habit of using too
> much memory, and as a safeguard, I want to kill it before it makes
> the system unstable.  The OOM killer often guesses wrong, and RLIMIT_AS
> kills too soon because of the address space used up by the many thread
> stacks.
> So I'd like an RLIMIT_RSS that just kills the fat process.

The RSS limit isnt a "kill" limit in Unix. its a residency limit. Its
preventing the obese process from getting more than a certain amount of
RAM as opposed to swap


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Kernel call chain search tool?
  2002-09-26 15:15 ` Alan Cox
@ 2002-09-26 16:17   ` Dan Kegel
  2002-09-26 19:07     ` Rob Landley
  0 siblings, 1 reply; 5+ messages in thread
From: Dan Kegel @ 2002-09-26 16:17 UTC (permalink / raw)
  To: Alan Cox; +Cc: linux-kernel

Alan Cox wrote:
> 
> On Wed, 2002-09-25 at 05:36, Dan Kegel wrote:
> > <prelude>
> > I have a large multithreaded program that has a habit of using too
> > much memory, and as a safeguard, I want to kill it before it makes
> > the system unstable.  The OOM killer often guesses wrong, and RLIMIT_AS
> > kills too soon because of the address space used up by the many thread
> > stacks.
> > So I'd like an RLIMIT_RSS that just kills the fat process.
> 
> The RSS limit isnt a "kill" limit in Unix. its a residency limit. Its
> preventing the obese process from getting more than a certain amount of
> RAM as opposed to swap

Yeah.  RLIMIT_RSS seemed like something I could hijack for the
purpose, though.  And the code change was really small
( http://marc.theaimsgroup.com/?l=linux-kernel&m=103299570928378 ).

If only the darn program didn't have so many threads, RLIMIT_AS
or the no-overcommit patch would be perfect.  I unfortunately can't 
get rid of the threads, so I'm stuck trying to figure out some way
to kill the right program when the system gets low on memory.

Maybe I should look at giving the OOM killer hints?
- Dan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Kernel call chain search tool?
  2002-09-26 16:17   ` Dan Kegel
@ 2002-09-26 19:07     ` Rob Landley
  2002-09-27  0:39       ` OOM killer hints (was: Re: Kernel call chain search tool?) Dan Kegel
  0 siblings, 1 reply; 5+ messages in thread
From: Rob Landley @ 2002-09-26 19:07 UTC (permalink / raw)
  To: Dan Kegel, Alan Cox; +Cc: linux-kernel

On Thursday 26 September 2002 12:17 pm, Dan Kegel wrote:

> If only the darn program didn't have so many threads, RLIMIT_AS
> or the no-overcommit patch would be perfect.  I unfortunately can't
> get rid of the threads, so I'm stuck trying to figure out some way
> to kill the right program when the system gets low on memory.
>
> Maybe I should look at giving the OOM killer hints?

The OOM killer should certainly know about threads and thread groups.  If you 
kill one thread, you generally have to kill the whole group because there's 
no way of knowing if that thread was holding a futex or otherwise custodian 
of critical data and thus you just threw the program into la-la land.

> - Dan

Rob

^ permalink raw reply	[flat|nested] 5+ messages in thread

* OOM killer hints (was: Re: Kernel call chain search tool?)
  2002-09-26 19:07     ` Rob Landley
@ 2002-09-27  0:39       ` Dan Kegel
  0 siblings, 0 replies; 5+ messages in thread
From: Dan Kegel @ 2002-09-27  0:39 UTC (permalink / raw)
  To: Rob Landley; +Cc: Alan Cox, linux-kernel

Rob Landley wrote:
> 
> On Thursday 26 September 2002 12:17 pm, Dan Kegel wrote:
> 
> > If only the darn program didn't have so many threads, RLIMIT_AS
> > or the no-overcommit patch would be perfect.  I unfortunately can't
> > get rid of the threads, so I'm stuck trying to figure out some way
> > to kill the right program when the system gets low on memory.
> >
> > Maybe I should look at giving the OOM killer hints?
> 
> The OOM killer should certainly know about threads and thread groups.  If you
> kill one thread, you generally have to kill the whole group because there's
> no way of knowing if that thread was holding a futex or otherwise custodian
> of critical data and thus you just threw the program into la-la land.

The OOM killer gets that part right; it kills all threads that share the
same mm.   Where it screws up is in picking the process to kill.
This is understandable, since it's a tough problem.

Hey, how about this: I could teach the OOM killer to look at
RLIMIT_RSS.  Processes which were at or nearly at their RLIMIT_RSS
would be killed first.  That would be more generally useful than
my hacky little patch, and it would be even tinier.  Like this, say:

--- oom_kill.c.orig	Thu Sep 26 17:31:12 2002
+++ oom_kill.c	Thu Sep 26 17:36:44 2002
@@ -86,6 +86,15 @@
		points *= 2;

	/*
+	 * Processes at or near their RSS or AS limits are probably causing
+	 * trouble, so double their badness points.
+	 */
+	if (((3 * p->mm->rss) / 4) >= (p->rlim[RLIMIT_RSS].rlim_max >>
PAGE_SHIFT))
+		points *= 2;
+	if (((3 * p->mm->total_vm) / 4) >= (p->rlim[RLIMIT_AS].rlim_max >>
PAGE_SHIFT))
+		points *= 2;
+
+	/*
	 * Superuser processes are usually more important, so we make it
	 * less likely that we kill those.
	 */

How's that look?
- Dan

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2002-09-27  0:24 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-09-25  4:36 Kernel call chain search tool? Dan Kegel
2002-09-26 15:15 ` Alan Cox
2002-09-26 16:17   ` Dan Kegel
2002-09-26 19:07     ` Rob Landley
2002-09-27  0:39       ` OOM killer hints (was: Re: Kernel call chain search tool?) Dan Kegel

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).