[PATCH v3 0/2] cpuset: Allow v2 behavior in v1 cgroup

[PATCH v3 0/2] cpuset: Allow v2 behavior in v1 cgroup

[Waiman Long]

v2->v3:
 - Change the generic CGRP_ROOT_V2_MODE flag to a cpuset specific
   CGRP_ROOT_CPUSET_V2_MODE flag.

v1->v2:
 - Drop the kernel command line option and use cgroupfs mount option
   instead to enable v2 controller behavior in v1 cgroup.

v1: https://lkml.org/lkml/2017/8/15/570

This patchset adds a new cgroupfs mount option "cpuset_v2_mode"
that enables the cpuset controller to use v2 behavior in a v1 cgroup.

Patch 1 adds the v1 cgroupfs mount option "cpuset_v2_mode" which set
a flag in the cgroup_root flags.

Patch 2 modifies the cpuset controller to check the cgroup_root flag
and switch to v2 behavior even in a v1 cgroup.

Waiman Long (2):
  cgroup: Add mount flag to enable cpuset to use v2 behavior in v1
    cgroup
  cpuset: Allow v2 behavior in v1 cgroup

 include/linux/cgroup-defs.h |  5 +++++
 kernel/cgroup/cgroup-v1.c   |  6 ++++++
 kernel/cgroup/cpuset.c      | 33 ++++++++++++++++++++-------------
 3 files changed, 31 insertions(+), 13 deletions(-)

-- 
1.8.3.1

[PATCH v3 1/2] cgroup: Add mount flag to enable cpuset to use v2 behavior in v1 cgroup

[Waiman Long]

A new mount option "cpuset_v2_mode" is added to the v1 cgroupfs
filesystem to enable cpuset controller to use v2 behavior in a v1
cgroup. This mount option applies only to cpuset controller and have
no effect on other controllers.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 include/linux/cgroup-defs.h | 5 +++++
 kernel/cgroup/cgroup-v1.c   | 6 ++++++
 2 files changed, 11 insertions(+)

diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 09f4c7d..c344e77 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -74,6 +74,11 @@ enum {
 	 * aren't writeable from inside the namespace.
 	 */
 	CGRP_ROOT_NS_DELEGATE	= (1 << 3),
+
+	/*
+	 * Enable cpuset controller in v1 cgroup to use v2 behavior.
+	 */
+	CGRP_ROOT_CPUSET_V2_MODE = (1 << 4),
 };
 
 /* cftype->flags */
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 7bf4b15..ce7426b 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -846,6 +846,8 @@ static int cgroup1_show_options(struct seq_file *seq, struct kernfs_root *kf_roo
 		seq_puts(seq, ",noprefix");
 	if (root->flags & CGRP_ROOT_XATTR)
 		seq_puts(seq, ",xattr");
+	if (root->flags & CGRP_ROOT_CPUSET_V2_MODE)
+		seq_puts(seq, ",cpuset_v2_mode");
 
 	spin_lock(&release_agent_path_lock);
 	if (strlen(root->release_agent_path))
@@ -900,6 +902,10 @@ static int parse_cgroupfs_options(char *data, struct cgroup_sb_opts *opts)
 			opts->cpuset_clone_children = true;
 			continue;
 		}
+		if (!strcmp(token, "cpuset_v2_mode")) {
+			opts->flags |= CGRP_ROOT_CPUSET_V2_MODE;
+			continue;
+		}
 		if (!strcmp(token, "xattr")) {
 			opts->flags |= CGRP_ROOT_XATTR;
 			continue;
-- 
1.8.3.1

[PATCH v3 2/2] cpuset: Allow v2 behavior in v1 cgroup

[Waiman Long]

Cpuset v2 has some useful behaviors that are not present in v1 because
of backward compatibility concern. One of that is the restoration of
the original cpu and memory node mask after a hot removal and addition
event sequence.

This patch makes the cpuset controller to check the
CGRP_ROOT_CPUSET_V2_MODE flag and use the v2 behavior if it is set.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 kernel/cgroup/cpuset.c | 33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 9ed6a05..060d575 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -300,6 +300,16 @@ static inline int is_spread_slab(const struct cpuset *cs)
 static DECLARE_WAIT_QUEUE_HEAD(cpuset_attach_wq);
 
 /*
+ * Cgroup v2 behavior is used when on default hierarchy or the
+ * cgroup_v2_mode flag is set.
+ */
+static inline bool is_in_v2_mode(void)
+{
+	return cgroup_subsys_on_dfl(cpuset_cgrp_subsys) ||
+	      (cpuset_cgrp_subsys.root->flags & CGRP_ROOT_CPUSET_V2_MODE);
+}
+
+/*
  * This is ugly, but preserves the userspace API for existing cpuset
  * users. If someone tries to mount the "cpuset" filesystem, we
  * silently switch it to mount "cgroup" instead
@@ -489,8 +499,7 @@ static int validate_change(struct cpuset *cur, struct cpuset *trial)
 
 	/* On legacy hiearchy, we must be a subset of our parent cpuset. */
 	ret = -EACCES;
-	if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
-	    !is_cpuset_subset(trial, par))
+	if (!is_in_v2_mode() && !is_cpuset_subset(trial, par))
 		goto out;
 
 	/*
@@ -903,8 +912,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some CPUs.
 		 */
-		if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
-		    cpumask_empty(new_cpus))
+		if (is_in_v2_mode() && cpumask_empty(new_cpus))
 			cpumask_copy(new_cpus, parent->effective_cpus);
 
 		/* Skip the whole subtree if the cpumask remains the same. */
@@ -921,7 +929,7 @@ static void update_cpumasks_hier(struct cpuset *cs, struct cpumask *new_cpus)
 		cpumask_copy(cp->effective_cpus, new_cpus);
 		spin_unlock_irq(&callback_lock);
 
-		WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+		WARN_ON(!is_in_v2_mode() &&
 			!cpumask_equal(cp->cpus_allowed, cp->effective_cpus));
 
 		update_tasks_cpumask(cp);
@@ -1157,8 +1165,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
 		 * If it becomes empty, inherit the effective mask of the
 		 * parent, which is guaranteed to have some MEMs.
 		 */
-		if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
-		    nodes_empty(*new_mems))
+		if (is_in_v2_mode() && nodes_empty(*new_mems))
 			*new_mems = parent->effective_mems;
 
 		/* Skip the whole subtree if the nodemask remains the same. */
@@ -1175,7 +1182,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)
 		cp->effective_mems = *new_mems;
 		spin_unlock_irq(&callback_lock);
 
-		WARN_ON(!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+		WARN_ON(!is_in_v2_mode() &&
 			!nodes_equal(cp->mems_allowed, cp->effective_mems));
 
 		update_tasks_nodemask(cp);
@@ -1467,7 +1474,7 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
 
 	/* allow moving tasks into an empty cpuset if on default hierarchy */
 	ret = -ENOSPC;
-	if (!cgroup_subsys_on_dfl(cpuset_cgrp_subsys) &&
+	if (!is_in_v2_mode() &&
 	    (cpumask_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed)))
 		goto out_unlock;
 
@@ -1985,7 +1992,7 @@ static int cpuset_css_online(struct cgroup_subsys_state *css)
 	cpuset_inc();
 
 	spin_lock_irq(&callback_lock);
-	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
+	if (is_in_v2_mode()) {
 		cpumask_copy(cs->effective_cpus, parent->effective_cpus);
 		cs->effective_mems = parent->effective_mems;
 	}
@@ -2062,7 +2069,7 @@ static void cpuset_bind(struct cgroup_subsys_state *root_css)
 	mutex_lock(&cpuset_mutex);
 	spin_lock_irq(&callback_lock);
 
-	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys)) {
+	if (is_in_v2_mode()) {
 		cpumask_copy(top_cpuset.cpus_allowed, cpu_possible_mask);
 		top_cpuset.mems_allowed = node_possible_map;
 	} else {
@@ -2256,7 +2263,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs)
 	cpus_updated = !cpumask_equal(&new_cpus, cs->effective_cpus);
 	mems_updated = !nodes_equal(new_mems, cs->effective_mems);
 
-	if (cgroup_subsys_on_dfl(cpuset_cgrp_subsys))
+	if (is_in_v2_mode())
 		hotplug_update_tasks(cs, &new_cpus, &new_mems,
 				     cpus_updated, mems_updated);
 	else
@@ -2287,7 +2294,7 @@ static void cpuset_hotplug_workfn(struct work_struct *work)
 	static cpumask_t new_cpus;
 	static nodemask_t new_mems;
 	bool cpus_updated, mems_updated;
-	bool on_dfl = cgroup_subsys_on_dfl(cpuset_cgrp_subsys);
+	bool on_dfl = is_in_v2_mode();
 
 	mutex_lock(&cpuset_mutex);
 
-- 
1.8.3.1

Re: [PATCH v3 0/2] cpuset: Allow v2 behavior in v1 cgroup

[Tejun Heo]

On Thu, Aug 17, 2017 at 03:33:08PM -0400, Waiman Long wrote:
> v2->v3:
>  - Change the generic CGRP_ROOT_V2_MODE flag to a cpuset specific
>    CGRP_ROOT_CPUSET_V2_MODE flag.
> 
> v1->v2:
>  - Drop the kernel command line option and use cgroupfs mount option
>    instead to enable v2 controller behavior in v1 cgroup.
> 
> v1: https://lkml.org/lkml/2017/8/15/570

Waiman, patches look good to me but can you please take care of the
kbuild warnings on 32bit and maybe merge the two patches into one?

Thanks.

-- 
tejun

Re: [PATCH v3 0/2] cpuset: Allow v2 behavior in v1 cgroup

[Waiman Long]

On 08/18/2017 07:29 AM, Tejun Heo wrote:
> On Thu, Aug 17, 2017 at 03:33:08PM -0400, Waiman Long wrote:
>> v2->v3:
>>  - Change the generic CGRP_ROOT_V2_MODE flag to a cpuset specific
>>    CGRP_ROOT_CPUSET_V2_MODE flag.
>>
>> v1->v2:
>>  - Drop the kernel command line option and use cgroupfs mount option
>>    instead to enable v2 controller behavior in v1 cgroup.
>>
>> v1: https://lkml.org/lkml/2017/8/15/570
> Waiman, patches look good to me but can you please take care of the
> kbuild warnings on 32bit and maybe merge the two patches into one?

The kbuild warning is in the __init code of the v1 patch for checking
the boot time argument. The __init code was gone in the v3 patch. So if
there is no further kbuild warning after a couple of days, I think it
should be OK.

Cheers,
Longman

Re: [PATCH v3 2/2] cpuset: Allow v2 behavior in v1 cgroup

[Tejun Heo]

On Thu, Aug 17, 2017 at 03:33:10PM -0400, Waiman Long wrote:
> Cpuset v2 has some useful behaviors that are not present in v1 because
> of backward compatibility concern. One of that is the restoration of
> the original cpu and memory node mask after a hot removal and addition
> event sequence.
> 
> This patch makes the cpuset controller to check the
> CGRP_ROOT_CPUSET_V2_MODE flag and use the v2 behavior if it is set.
> 
> Signed-off-by: Waiman Long <longman@redhat.com>

Applied 1-2 to cgroup/for-4.14.

Thanks!

-- 
tejun