Re: [PATCH memcg v4] memcg: prohibit unconditional exceeding the limit of dying tasks

[Michal Hocko <mhocko@suse.com>]

On Wed 20-10-21 11:07:02, Vasily Averin wrote:
[...]
I haven't read through the changelog and only focused on the patch this
time.

[...]
> @@ -1810,11 +1810,21 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int
>  		mem_cgroup_oom_notify(memcg);
>  
>  	mem_cgroup_unmark_under_oom(memcg);
> -	if (mem_cgroup_out_of_memory(memcg, mask, order))
> +	if (mem_cgroup_out_of_memory(memcg, mask, order)) {
>  		ret = OOM_SUCCESS;
> -	else
> +	} else {
>  		ret = OOM_FAILED;
> -
> +		/*
> +		 * In some rare cases mem_cgroup_out_of_memory() can return false.
> +		 * If it was called from #PF it forces handle_mm_fault()
> +		 * return VM_FAULT_OOM and executes pagefault_out_of_memory().
> +		 * memcg_in_oom is set here to notify pagefault_out_of_memory()
> +		 * that it was a memcg-related failure and not allow to run
> +		 * global OOM.
> +		 */
> +		if (current->in_user_fault)
> +			current->memcg_in_oom = (struct mem_cgroup *)ret;
> +	}
>  	if (locked)
>  		mem_cgroup_oom_unlock(memcg);
>  
> @@ -1848,6 +1858,15 @@ bool mem_cgroup_oom_synchronize(bool handle)
>  	if (!memcg)
>  		return false;
>  
> +	/* OOM is memcg, however out_of_memory() found no victim */
> +	if (memcg == (struct mem_cgroup *)OOM_FAILED) {
> +		/*
> +		 * Should be called from pagefault_out_of_memory() only,
> +		 * where it is used to prevent false global OOM.
> +		 */
> +		current->memcg_in_oom = NULL;
> +		return true;
> +	}
>  	if (!handle)
>  		goto cleanup;

I have to say I am not a great fan of this but this belongs to a
separate patch on top of all the previous ones.

[...]
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 831340e7ad8b..1deef8c7a71b 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -1137,6 +1137,9 @@ void pagefault_out_of_memory(void)
>  	if (mem_cgroup_oom_synchronize(true))
>  		return;
>  
> +	if (fatal_signal_pending(current))
> +		return;
> +

This belongs to its own patch as well.

All that being said I would go with pagefault_out_of_memory as the first
patch because it is necessary to handle charge failures. Then go with a
patch to remove charge forcing when OOM killer succeeds but the retry
still fails and finally go with one that tries to handle oom failures.
-- 
Michal Hocko
SUSE Labs