From: Feng Tang <feng.tang@intel.com>
To: David Rientjes <rientjes@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Michal Hocko <mhocko@suse.com>, Tejun Heo <tj@kernel.org>,
Zefan Li <lizefan.x@bytedance.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Mel Gorman <mgorman@techsingularity.net>,
Vlastimil Babka <vbabka@suse.cz>,
linux-mm@kvack.org, cgroups@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH v3] mm/page_alloc: detect allocation forbidden by cpuset and bail out early
Date: Wed, 15 Sep 2021 13:32:47 +0800 [thread overview]
Message-ID: <20210915053247.GG56674@shbuild999.sh.intel.com> (raw)
In-Reply-To: <3bd87d8a-d09e-ac7-1d1d-25ad1b9d5ed9@google.com>
On Tue, Sep 14, 2021 at 05:30:03PM -0700, David Rientjes wrote:
> On Tue, 14 Sep 2021, Feng Tang wrote:
>
> > diff --git a/include/linux/cpuset.h b/include/linux/cpuset.h
> > index d2b9c41..d58e047 100644
> > --- a/include/linux/cpuset.h
> > +++ b/include/linux/cpuset.h
> > @@ -34,6 +34,8 @@
> > */
> > extern struct static_key_false cpusets_pre_enable_key;
> > extern struct static_key_false cpusets_enabled_key;
> > +extern struct static_key_false cpusets_insane_config_key;
> > +
> > static inline bool cpusets_enabled(void)
> > {
> > return static_branch_unlikely(&cpusets_enabled_key);
> > @@ -51,6 +53,19 @@ static inline void cpuset_dec(void)
> > static_branch_dec_cpuslocked(&cpusets_pre_enable_key);
> > }
> >
> > +/*
> > + * This will get enabled whenever a cpuset configuration is considered
> > + * unsupportable in general. E.g. movable only node which cannot satisfy
> > + * any non movable allocations (see update_nodemask). Page allocator
> > + * needs to make additional checks for those configurations and this
> > + * check is meant to guard those checks without any overhead for sane
> > + * configurations.
> > + */
> > +static inline bool cpusets_insane_config(void)
> > +{
> > + return static_branch_unlikely(&cpusets_insane_config_key);
> > +}
> > +
> > extern int cpuset_init(void);
> > extern void cpuset_init_smp(void);
> > extern void cpuset_force_rebuild(void);
> > @@ -167,6 +182,8 @@ static inline void set_mems_allowed(nodemask_t nodemask)
> >
> > static inline bool cpusets_enabled(void) { return false; }
> >
> > +static inline bool cpusets_insane_config(void) { return false; }
> > +
> > static inline int cpuset_init(void) { return 0; }
> > static inline void cpuset_init_smp(void) {}
> >
> > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
> > index 6a1d79d..a455333 100644
> > --- a/include/linux/mmzone.h
> > +++ b/include/linux/mmzone.h
> > @@ -1220,6 +1220,22 @@ static inline struct zoneref *first_zones_zonelist(struct zonelist *zonelist,
> > #define for_each_zone_zonelist(zone, z, zlist, highidx) \
> > for_each_zone_zonelist_nodemask(zone, z, zlist, highidx, NULL)
> >
> > +/* Whether the 'nodes' are all movable nodes */
> > +static inline bool movable_only_nodes(nodemask_t *nodes)
> > +{
> > + struct zonelist *zonelist;
> > + struct zoneref *z;
> > +
> > + if (nodes_empty(*nodes))
> > + return false;
> > +
> > + zonelist =
> > + &NODE_DATA(first_node(*nodes))->node_zonelists[ZONELIST_FALLBACK];
> > + z = first_zones_zonelist(zonelist, ZONE_NORMAL, nodes);
> > + return (!z->zone) ? true : false;
> > +}
> > +
> > +
> > #ifdef CONFIG_SPARSEMEM
> > #include <asm/sparsemem.h>
> > #endif
> > diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
> > index df1ccf4..7fa633e 100644
> > --- a/kernel/cgroup/cpuset.c
> > +++ b/kernel/cgroup/cpuset.c
> > @@ -69,6 +69,13 @@
> > DEFINE_STATIC_KEY_FALSE(cpusets_pre_enable_key);
> > DEFINE_STATIC_KEY_FALSE(cpusets_enabled_key);
> >
> > +/*
> > + * There could be abnormal cpuset configurations for cpu or memory
> > + * node binding, add this key to provide a quick low-cost judgement
> > + * of the situation.
> > + */
> > +DEFINE_STATIC_KEY_FALSE(cpusets_insane_config_key);
> > +
> > /* See "Frequency meter" comments, below. */
> >
> > struct fmeter {
> > @@ -1868,6 +1875,14 @@ static int update_nodemask(struct cpuset *cs, struct cpuset *trialcs,
> > if (retval < 0)
> > goto done;
> >
> > + if (!cpusets_insane_config() &&
> > + movable_only_nodes(&trialcs->mems_allowed)) {
> > + static_branch_enable(&cpusets_insane_config_key);
> > + pr_info("Unsupported (movable nodes only) cpuset configuration detected (nmask=%*pbl)! "
> > + "Cpuset allocations might fail even with a lot of memory available.\n",
> > + nodemask_pr_args(&trialcs->mems_allowed));
> > + }
> > +
> > spin_lock_irq(&callback_lock);
> > cs->mems_allowed = trialcs->mems_allowed;
> > spin_unlock_irq(&callback_lock);
>
> Is this the only time that the state of the nodemask may change?
>
> I'm wondering about a single node nodemask, for example, where all
> ZONE_NORMAL memory is hot-removed.
Thanks for the reminding! Yes, memory hot remove can change the
cpuset's effective nodemask, we may need to add similar check inside
cpuset_hotplug_update_tasks() which is called by cpuset_hotplug_workfn(),
something like below?
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 7fa633e..d5f6776 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -3186,6 +3186,14 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)
cpus_updated = !cpumask_equal(&new_cpus, cs->effective_cpus);
mems_updated = !nodes_equal(new_mems, cs->effective_mems);
+ if (mems_updated && !cpusets_insane_config() &&
+ movable_only_nodes(new_mems)) {
+ static_branch_enable(&cpusets_insane_config_key);
+ pr_info("Unsupported (movable nodes only) cpuset configuration detected (nmask=%*pbl) after memory hotplug."
+ "Cpuset allocations might fail even with a lot of memory available.\n",
+ nodemask_pr_args(new_mems);
+ }
+
if (is_in_v2_mode())
hotplug_update_tasks(cs, &new_cpus, &new_mems,
cpus_updated, mems_updated);
Thanks,
Feng
next prev parent reply other threads:[~2021-09-15 5:33 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-14 3:40 [PATCH v3] mm/page_alloc: detect allocation forbidden by cpuset and bail out early Feng Tang
2021-09-14 8:01 ` Vlastimil Babka
2021-09-14 8:17 ` Michal Hocko
2021-09-14 8:50 ` Michal Hocko
2021-09-24 6:10 ` Feng Tang
2021-09-24 7:17 ` Michal Hocko
2021-09-15 0:30 ` David Rientjes
2021-09-15 5:32 ` Feng Tang [this message]
2021-09-15 11:30 ` Michal Hocko
2021-09-16 8:11 ` Feng Tang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210915053247.GG56674@shbuild999.sh.intel.com \
--to=feng.tang@intel.com \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lizefan.x@bytedance.com \
--cc=mgorman@techsingularity.net \
--cc=mhocko@suse.com \
--cc=rientjes@google.com \
--cc=tj@kernel.org \
--cc=vbabka@suse.cz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).