From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5B12C433DF for ; Fri, 19 Jun 2020 16:25:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70E32217A0 for ; Fri, 19 Jun 2020 16:25:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70E32217A0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 794638D00ED; Fri, 19 Jun 2020 12:24:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D0178D00EC; Fri, 19 Jun 2020 12:24:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 544E88D00EB; Fri, 19 Jun 2020 12:24:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id 36B6D8D00E7 for ; Fri, 19 Jun 2020 12:24:48 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EDB13181AC9C6 for ; Fri, 19 Jun 2020 16:24:47 +0000 (UTC) X-FDA: 76946485014.10.money64_300bb8d26e1a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id B0F5884A0D for ; Fri, 19 Jun 2020 16:24:47 +0000 (UTC) X-HE-Tag: money64_300bb8d26e1a X-Filterd-Recvd-Size: 7835 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 19 Jun 2020 16:24:46 +0000 (UTC) IronPort-SDR: wTW5he6lOcRUxwlBFXOOnzabyjl1X1ml2L1EnT6YB104ijf4nfKp4m77EcSNfYSofzgE5X848I mEPpGW1viaww== X-IronPort-AV: E=McAfee;i="6000,8403,9657"; a="140535481" X-IronPort-AV: E=Sophos;i="5.75,256,1589266800"; d="scan'208";a="140535481" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jun 2020 09:24:16 -0700 IronPort-SDR: 25xBWZLEf/AQGLSl7I3eg46zuhUdkM//uzgNksUXMmDd+f5TrVMZoiI3dOTILUqZ+EKRzcwXV+ BULhhm7G9Rwg== X-IronPort-AV: E=Sophos;i="5.75,255,1589266800"; d="scan'208";a="264366311" Received: from sjiang-mobl2.ccr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.131.131]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jun 2020 09:24:16 -0700 From: Ben Widawsky To: linux-mm Subject: [PATCH 06/18] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes Date: Fri, 19 Jun 2020 09:24:02 -0700 Message-Id: <20200619162414.1052234-7-ben.widawsky@intel.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200619162414.1052234-1-ben.widawsky@intel.com> References: <20200619162414.1052234-1-ben.widawsky@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: B0F5884A0D X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen MPOL_PREFERRED honors only a single node set in the nodemask. Add the bare define for a new mode which will allow more than one. The patch does all the plumbing without actually adding the new policy type. v2: Plumb most MPOL_PREFERRED_MANY without exposing UAPI (Ben) Fixes for checkpatch (Ben) Cc: Andrew Morton Signed-off-by: Dave Hansen Co-developed-by: Ben Widawsky Signed-off-by: Ben Widawsky --- mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 5 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index e0b576838e57..6c7301cefeb6 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -31,6 +31,9 @@ * but useful to set in a VMA when you have a non default * process policy. * + * preferred many Try a set of nodes first before normal fallback. This = is + * similar to preferred without the special case. + * * default Allocate on the local node first, or when on a VMA * use the process policy. This is what Linux always did * in a NUMA aware kernel and still does by, ahem, default. @@ -105,6 +108,8 @@ =20 #include "internal.h" =20 +#define MPOL_PREFERRED_MANY MPOL_MAX + /* Internal flags */ #define MPOL_MF_DISCONTIG_OK (MPOL_MF_INTERNAL << 0) /* Skip checks for = continuous vmas */ #define MPOL_MF_INVERT (MPOL_MF_INTERNAL << 1) /* Invert check for node= mask */ @@ -175,7 +180,7 @@ struct mempolicy *get_task_policy(struct task_struct = *p) static const struct mempolicy_operations { int (*create)(struct mempolicy *pol, const nodemask_t *nodes); void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes); -} mpol_ops[MPOL_MAX]; +} mpol_ops[MPOL_MAX + 1]; =20 static inline int mpol_store_user_nodemask(const struct mempolicy *pol) { @@ -415,7 +420,7 @@ void mpol_rebind_mm(struct mm_struct *mm, nodemask_t = *new) mmap_write_unlock(mm); } =20 -static const struct mempolicy_operations mpol_ops[MPOL_MAX] =3D { +static const struct mempolicy_operations mpol_ops[MPOL_MAX + 1] =3D { [MPOL_DEFAULT] =3D { .rebind =3D mpol_rebind_default, }, @@ -432,6 +437,10 @@ static const struct mempolicy_operations mpol_ops[MP= OL_MAX] =3D { .rebind =3D mpol_rebind_nodemask, }, /* MPOL_LOCAL is converted to MPOL_PREFERRED on policy creation */ + [MPOL_PREFERRED_MANY] =3D { + .create =3D NULL, + .rebind =3D NULL, + }, }; =20 static int migrate_page_add(struct page *page, struct list_head *pagelis= t, @@ -915,6 +924,9 @@ static void get_policy_nodemask(struct mempolicy *p, = nodemask_t *nodes) case MPOL_INTERLEAVE: *nodes =3D p->v.nodes; break; + case MPOL_PREFERRED_MANY: + *nodes =3D p->v.preferred_nodes; + break; case MPOL_PREFERRED: if (!(p->flags & MPOL_F_LOCAL)) *nodes =3D p->v.preferred_nodes; @@ -1910,7 +1922,9 @@ static nodemask_t *policy_nodemask(gfp_t gfp, struc= t mempolicy *policy) static int policy_node(gfp_t gfp, struct mempolicy *policy, int nd) { - if (policy->mode =3D=3D MPOL_PREFERRED && !(policy->flags & MPOL_F_LOCA= L)) { + if ((policy->mode =3D=3D MPOL_PREFERRED || + policy->mode =3D=3D MPOL_PREFERRED_MANY) && + !(policy->flags & MPOL_F_LOCAL)) { nd =3D first_node(policy->v.preferred_nodes); } else { /* @@ -1953,6 +1967,7 @@ unsigned int mempolicy_slab_node(void) return node; =20 switch (policy->mode) { + case MPOL_PREFERRED_MANY: case MPOL_PREFERRED: /* * handled MPOL_F_LOCAL above @@ -2087,6 +2102,9 @@ bool init_nodemask_of_mempolicy(nodemask_t *mask) task_lock(current); mempolicy =3D current->mempolicy; switch (mempolicy->mode) { + case MPOL_PREFERRED_MANY: + *mask =3D mempolicy->v.preferred_nodes; + break; case MPOL_PREFERRED: if (mempolicy->flags & MPOL_F_LOCAL) nid =3D numa_node_id(); @@ -2141,6 +2159,9 @@ bool mempolicy_nodemask_intersects(struct task_stru= ct *tsk, * nodes in mask. */ break; + case MPOL_PREFERRED_MANY: + ret =3D nodes_intersects(mempolicy->v.preferred_nodes, *mask); + break; case MPOL_BIND: case MPOL_INTERLEAVE: ret =3D nodes_intersects(mempolicy->v.nodes, *mask); @@ -2225,8 +2246,9 @@ alloc_pages_vma(gfp_t gfp, int order, struct vm_are= a_struct *vma, * node and don't fall back to other nodes, as the cost of * remote accesses would likely offset THP benefits. * - * If the policy is interleave, or does not allow the current - * node in its nodemask, we allocate the standard way. + * If the policy is interleave or multiple preferred nodes, or + * does not allow the current node in its nodemask, we allocate + * the standard way. */ if (pol->mode =3D=3D MPOL_PREFERRED && !(pol->flags & MPOL_F_LOCAL)) hpage_node =3D first_node(pol->v.preferred_nodes); @@ -2364,6 +2386,9 @@ bool __mpol_equal(struct mempolicy *a, struct mempo= licy *b) case MPOL_BIND: case MPOL_INTERLEAVE: return !!nodes_equal(a->v.nodes, b->v.nodes); + case MPOL_PREFERRED_MANY: + return !!nodes_equal(a->v.preferred_nodes, + b->v.preferred_nodes); case MPOL_PREFERRED: /* a's ->flags is the same as b's */ if (a->flags & MPOL_F_LOCAL) @@ -2532,6 +2557,8 @@ int mpol_misplaced(struct page *page, struct vm_are= a_struct *vma, unsigned long polnid =3D zone_to_nid(z->zone); break; =20 + /* case MPOL_PREFERRED_MANY: */ + default: BUG(); } @@ -2883,6 +2910,7 @@ static const char * const policy_modes[] =3D [MPOL_BIND] =3D "bind", [MPOL_INTERLEAVE] =3D "interleave", [MPOL_LOCAL] =3D "local", + [MPOL_PREFERRED_MANY] =3D "prefer (many)", }; =20 =20 @@ -2962,6 +2990,7 @@ int mpol_parse_str(char *str, struct mempolicy **mp= ol) if (!nodelist) err =3D 0; goto out; + case MPOL_PREFERRED_MANY: case MPOL_BIND: /* * Insist on a nodelist @@ -3044,6 +3073,9 @@ void mpol_to_str(char *buffer, int maxlen, struct m= empolicy *pol) switch (mode) { case MPOL_DEFAULT: break; + case MPOL_PREFERRED_MANY: + WARN_ON(flags & MPOL_F_LOCAL); + fallthrough; case MPOL_PREFERRED: if (flags & MPOL_F_LOCAL) mode =3D MPOL_LOCAL; --=20 2.27.0