Re: User space out of memory approach

[Thomas Gleixner <tglx@linutronix.de>]

On Tue, 2005-01-25 at 17:13 -0400, Mauricio Lin wrote:
> Hi Andrea,
> 
> Your OOM Killer patch was tested and a strange behaviour was found.
> Basically as normal user we started some applications as openoffice,
> mozilla and emacs.
> And as a root (in another tty) we started a simple program that uses
> malloc in a forever loop as below:
> 
> int main (void)
> {
>   int * mem;
>   for (;;)
>         mem = (int *) malloc(sizeof(int));
>   return 0;
> }
> 
> 
> Using the original OOM Killer, malloc is the first killed application
> and the sytem is restored in a useful state. After applying your patch
> and accomplish the same experiment, the OOM Killer it does not kill
> malloc program and it enters in a kind of forever loop as below:
> 
> 1) out_of_memory is invoked;
> 2) select_bad_process is invoked;

Which process is selected ?

> 3) the following condition is fullfied;
> if ((unlikely(test_tsk_thread_flag(p, TIF_MEMDIE)) || (p->flags &
> PF_EXITING)) &&
> 			    !(p->flags & PF_DEAD))
> 				return ERR_PTR(-1UL);

???

Can you please show the kernel messages ?

tglx