From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F301C433B4 for ; Wed, 14 Apr 2021 13:08:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C7DF2611AD for ; Wed, 14 Apr 2021 13:08:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7DF2611AD Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 42F736B00A0; Wed, 14 Apr 2021 09:08:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E0218D0001; Wed, 14 Apr 2021 09:08:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 281626B00A2; Wed, 14 Apr 2021 09:08:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0147.hostedemail.com [216.40.44.147]) by kanga.kvack.org (Postfix) with ESMTP id 0D0826B00A0 for ; Wed, 14 Apr 2021 09:08:22 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B664918013697 for ; Wed, 14 Apr 2021 13:08:21 +0000 (UTC) X-FDA: 78031001202.18.BC396F9 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf06.hostedemail.com (Postfix) with ESMTP id E2F81C0007EA for ; Wed, 14 Apr 2021 13:08:22 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1618405699; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MB7Ot9cl8Zex6E9GFGi3H9hwXIbZ5RJWUQJFa9P2goI=; b=Zbjg9ovG+d9/9vGV8RaTM7HKCscUH32eZ6VaEugqu4QcDn5B1Udm2ck1qlsW5RLkunawcD iLyy34jFd1Myy0LuNutj9Pkj3jPjj1aSPYJRMZLeuNh1Lz0CqiuAwjyjwOnmu01hUC/lQY ke+wLmHPhEQ2B+lVdaOzGCbmYCh4VjQ= Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id B14ACAFC1; Wed, 14 Apr 2021 13:08:19 +0000 (UTC) Date: Wed, 14 Apr 2021 15:08:19 +0200 From: Michal Hocko To: Feng Tang Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Andrea Arcangeli , David Rientjes , Mel Gorman , Mike Kravetz , Randy Dunlap , Vlastimil Babka , Dave Hansen , Ben Widawsky , Andi Kleen , Dan Williams Subject: Re: [PATCH v4 08/13] mm/mempolicy: Create a page allocator for policy Message-ID: References: <1615952410-36895-1-git-send-email-feng.tang@intel.com> <1615952410-36895-9-git-send-email-feng.tang@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1615952410-36895-9-git-send-email-feng.tang@intel.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E2F81C0007EA X-Stat-Signature: kmwo1egi8ssnesa6nctafyaofww9z5cf Received-SPF: none (suse.com>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mx2.suse.de; client-ip=195.135.220.15 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618405702-683658 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 17-03-21 11:40:05, Feng Tang wrote: > From: Ben Widawsky > > Add a helper function which takes care of handling multiple preferred > nodes. It will be called by future patches that need to handle this, > specifically VMA based page allocation, and task based page allocation. > Huge pages don't quite fit the same pattern because they use different > underlying page allocation functions. This consumes the previous > interleave policy specific allocation function to make a one stop shop > for policy based allocation. > > With this, MPOL_PREFERRED_MANY's semantic is more like MPOL_PREFERRED > that it will first try the preferred node/nodes, and fallback to all > other nodes when first try fails. Thanks to Michal Hocko for suggestions > on this. > > For now, only interleaved policy will be used so there should be no > functional change yet. However, if bisection points to issues in the > next few commits, it was likely the fault of this patch. I am not sure this is helping much. Let's see in later patches but I would keep them separate and rather create a dedicated function for the new policy allocation mode. > Similar functionality is offered via policy_node() and > policy_nodemask(). By themselves however, neither can achieve this > fallback style of sets of nodes. > > [ Feng: for the first try, add NOWARN flag, and skip the direct reclaim > to speedup allocation in some case ] > > Link: https://lore.kernel.org/r/20200630212517.308045-9-ben.widawsky@intel.com > Signed-off-by: Ben Widawsky > Signed-off-by: Feng Tang > --- > mm/mempolicy.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++------------ > 1 file changed, 52 insertions(+), 13 deletions(-) > > diff --git a/mm/mempolicy.c b/mm/mempolicy.c > index d945f29..d21105b 100644 > --- a/mm/mempolicy.c > +++ b/mm/mempolicy.c > @@ -2187,22 +2187,60 @@ bool mempolicy_nodemask_intersects(struct task_struct *tsk, > return ret; > } > > -/* Allocate a page in interleaved policy. > - Own path because it needs to do special accounting. */ > -static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, > - unsigned nid) > +/* Handle page allocation for all but interleaved policies */ > +static struct page *alloc_pages_policy(struct mempolicy *pol, gfp_t gfp, > + unsigned int order, int preferred_nid) > { > struct page *page; > + gfp_t gfp_mask = gfp; > > - page = __alloc_pages(gfp, order, nid); > - /* skip NUMA_INTERLEAVE_HIT counter update if numa stats is disabled */ > - if (!static_branch_likely(&vm_numa_stat_key)) > + if (pol->mode == MPOL_INTERLEAVE) { > + page = __alloc_pages(gfp, order, preferred_nid); > + /* skip NUMA_INTERLEAVE_HIT counter update if numa stats is disabled */ > + if (!static_branch_likely(&vm_numa_stat_key)) > + return page; > + if (page && page_to_nid(page) == preferred_nid) { > + preempt_disable(); > + __inc_numa_state(page_zone(page), NUMA_INTERLEAVE_HIT); > + preempt_enable(); > + } > return page; > - if (page && page_to_nid(page) == nid) { > - preempt_disable(); > - __inc_numa_state(page_zone(page), NUMA_INTERLEAVE_HIT); > - preempt_enable(); > } > + > + VM_BUG_ON(preferred_nid != NUMA_NO_NODE); > + > + preferred_nid = numa_node_id(); > + > + /* > + * There is a two pass approach implemented here for > + * MPOL_PREFERRED_MANY. In the first pass we try the preferred nodes > + * but allow the allocation to fail. The below table explains how > + * this is achieved. > + * > + * | Policy | preferred nid | nodemask | > + * |-------------------------------|---------------|------------| > + * | MPOL_DEFAULT | local | NULL | > + * | MPOL_PREFERRED | best | NULL | > + * | MPOL_INTERLEAVE | ERR | ERR | > + * | MPOL_BIND | local | pol->nodes | > + * | MPOL_PREFERRED_MANY | best | pol->nodes | > + * | MPOL_PREFERRED_MANY (round 2) | local | NULL | > + * +-------------------------------+---------------+------------+ > + */ > + if (pol->mode == MPOL_PREFERRED_MANY) { > + gfp_mask |= __GFP_NOWARN; > + > + /* Skip direct reclaim, as there will be a second try */ > + gfp_mask &= ~__GFP_DIRECT_RECLAIM; > + } > + > + page = __alloc_pages_nodemask(gfp_mask, order, > + policy_node(gfp, pol, preferred_nid), > + policy_nodemask(gfp, pol)); > + > + if (unlikely(!page && pol->mode == MPOL_PREFERRED_MANY)) > + page = __alloc_pages_nodemask(gfp, order, preferred_nid, NULL); > + > return page; > } > > @@ -2244,8 +2282,8 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, > unsigned nid; > > nid = interleave_nid(pol, vma, addr, PAGE_SHIFT + order); > + page = alloc_pages_policy(pol, gfp, order, nid); > mpol_cond_put(pol); > - page = alloc_page_interleave(gfp, order, nid); > goto out; > } > > @@ -2329,7 +2367,8 @@ struct page *alloc_pages_current(gfp_t gfp, unsigned order) > * nor system default_policy > */ > if (pol->mode == MPOL_INTERLEAVE) > - page = alloc_page_interleave(gfp, order, interleave_nodes(pol)); > + page = alloc_pages_policy(pol, gfp, order, > + interleave_nodes(pol)); > else > page = __alloc_pages_nodemask(gfp, order, > policy_node(gfp, pol, numa_node_id()), > -- > 2.7.4 -- Michal Hocko SUSE Labs