From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756237AbaFRUpA (ORCPT ); Wed, 18 Jun 2014 16:45:00 -0400 Received: from zene.cmpxchg.org ([85.214.230.12]:37516 "EHLO zene.cmpxchg.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754965AbaFRUk5 (ORCPT ); Wed, 18 Jun 2014 16:40:57 -0400 From: Johannes Weiner To: Andrew Morton Cc: Michal Hocko , Hugh Dickins , Tejun Heo , Vladimir Davydov , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [patch 02/13] mm: memcontrol: rearrange charging fast path Date: Wed, 18 Jun 2014 16:40:34 -0400 Message-Id: <1403124045-24361-3-git-send-email-hannes@cmpxchg.org> X-Mailer: git-send-email 2.0.0 In-Reply-To: <1403124045-24361-1-git-send-email-hannes@cmpxchg.org> References: <1403124045-24361-1-git-send-email-hannes@cmpxchg.org> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The charging path currently starts out with OOM condition checks when OOM is the rarest possible case. Rearrange this code to run OOM/task dying checks only after trying the percpu charge and the res_counter charge and bail out before entering reclaim. Attempting a charge does not hurt an (oom-)killed task as much as every charge attempt having to check OOM conditions. Also, only check __GFP_NOFAIL when the charge would actually fail. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- mm/memcontrol.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 94531df14d37..e946f7439b16 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2575,22 +2575,6 @@ static int mem_cgroup_try_charge(struct mem_cgroup *memcg, if (mem_cgroup_is_root(memcg)) goto done; - /* - * Unlike in global OOM situations, memcg is not in a physical - * memory shortage. Allow dying and OOM-killed tasks to - * bypass the last charges so that they can exit quickly and - * free their memory. - */ - if (unlikely(test_thread_flag(TIF_MEMDIE) || - fatal_signal_pending(current) || - current->flags & PF_EXITING)) - goto bypass; - - if (unlikely(task_in_memcg_oom(current))) - goto nomem; - - if (gfp_mask & __GFP_NOFAIL) - oom = false; retry: if (consume_stock(memcg, nr_pages)) goto done; @@ -2612,6 +2596,20 @@ retry: goto retry; } + /* + * Unlike in global OOM situations, memcg is not in a physical + * memory shortage. Allow dying and OOM-killed tasks to + * bypass the last charges so that they can exit quickly and + * free their memory. + */ + if (unlikely(test_thread_flag(TIF_MEMDIE) || + fatal_signal_pending(current) || + current->flags & PF_EXITING)) + goto bypass; + + if (unlikely(task_in_memcg_oom(current))) + goto nomem; + if (!(gfp_mask & __GFP_WAIT)) goto nomem; @@ -2640,6 +2638,9 @@ retry: if (mem_cgroup_wait_acct_move(mem_over_limit)) goto retry; + if (gfp_mask & __GFP_NOFAIL) + goto bypass; + if (fatal_signal_pending(current)) goto bypass; -- 2.0.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Johannes Weiner Subject: [patch 02/13] mm: memcontrol: rearrange charging fast path Date: Wed, 18 Jun 2014 16:40:34 -0400 Message-ID: <1403124045-24361-3-git-send-email-hannes@cmpxchg.org> References: <1403124045-24361-1-git-send-email-hannes@cmpxchg.org> Return-path: DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cmpxchg.org; s=zene; h=References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From; bh=5cIpCRs8UlBceLTRefai5CyqZDwhUd0UF8vpZr68mYo=; b=cB7K4EQHdlor2q8ZgcFh0tyC78K4o+dKFh8Mp5FO7+w9FdwnhXTOWSEMWuUH/IeQJkcLb7eSSra6gXZBtcmgn+W4OzWGs92Murwm1P8ZmcWCRS4ADX4IB3vxHggBJepxGEtDielFurkv+H4vRCTSrKccwRbZ+0a4+XQeQbex4ng=; In-Reply-To: <1403124045-24361-1-git-send-email-hannes@cmpxchg.org> Sender: owner-linux-mm@kvack.org List-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Andrew Morton Cc: Michal Hocko , Hugh Dickins , Tejun Heo , Vladimir Davydov , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org The charging path currently starts out with OOM condition checks when OOM is the rarest possible case. Rearrange this code to run OOM/task dying checks only after trying the percpu charge and the res_counter charge and bail out before entering reclaim. Attempting a charge does not hurt an (oom-)killed task as much as every charge attempt having to check OOM conditions. Also, only check __GFP_NOFAIL when the charge would actually fail. Signed-off-by: Johannes Weiner Acked-by: Michal Hocko --- mm/memcontrol.c | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 94531df14d37..e946f7439b16 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2575,22 +2575,6 @@ static int mem_cgroup_try_charge(struct mem_cgroup *memcg, if (mem_cgroup_is_root(memcg)) goto done; - /* - * Unlike in global OOM situations, memcg is not in a physical - * memory shortage. Allow dying and OOM-killed tasks to - * bypass the last charges so that they can exit quickly and - * free their memory. - */ - if (unlikely(test_thread_flag(TIF_MEMDIE) || - fatal_signal_pending(current) || - current->flags & PF_EXITING)) - goto bypass; - - if (unlikely(task_in_memcg_oom(current))) - goto nomem; - - if (gfp_mask & __GFP_NOFAIL) - oom = false; retry: if (consume_stock(memcg, nr_pages)) goto done; @@ -2612,6 +2596,20 @@ retry: goto retry; } + /* + * Unlike in global OOM situations, memcg is not in a physical + * memory shortage. Allow dying and OOM-killed tasks to + * bypass the last charges so that they can exit quickly and + * free their memory. + */ + if (unlikely(test_thread_flag(TIF_MEMDIE) || + fatal_signal_pending(current) || + current->flags & PF_EXITING)) + goto bypass; + + if (unlikely(task_in_memcg_oom(current))) + goto nomem; + if (!(gfp_mask & __GFP_WAIT)) goto nomem; @@ -2640,6 +2638,9 @@ retry: if (mem_cgroup_wait_acct_move(mem_over_limit)) goto retry; + if (gfp_mask & __GFP_NOFAIL) + goto bypass; + if (fatal_signal_pending(current)) goto bypass; -- 2.0.0 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org