[PATCH 0/2] bind rcu offload (nohz_full/isolation) into cpuset

[PATCH 0/2] bind rcu offload (nohz_full/isolation) into cpuset

[Paul Gortmaker]

One of the earlier pre-mainline RCU nocb patchsets had a temporary sysfs
knob in /sys/devices/system/cpu/cpu*/hotplug/nocb for testing[1].

That not-for-merge commit from Frederic said:

  This is only intended for those who want to test this patchset. The
  real interfaces will be cpuset/isolation and rcutorture.

We've had rcutorture as the one and only mainline user of nocb toggle
for a while now[2], and so I thought I'd take a crack at what Frederic
had in mind for cpuset with some code vs. asking 100 random questions.

Note that I intentionally didn't Cc any cgroup/cpuset people (yet),
since at this point this is only my guess on what things were to look
like based on a single sentence fragment.  So this is really early
"Not-for-Merge", but truly just RFC -- to start a conversation.

It won't be really useful until we adjust tick/housekeeping in addition
to nocb, but I think we can develop the interface in parallel to that?
And maybe use this to expand testing at the same time if it is layered
on top of those future work/patchsets?  I don't know...

We'll also have to look at corner cases - like whether we want to treat
the root cpuset differently; whether we want to sync boot arg values
with the cpuset's initial isol flag value, whether we un-isolate cores
when an isolation cpuset is rmdir/removed, etc etc.

But as a proof of concept, it "works" as can be seen in the 2nd commit.

Paul.
--

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>

[1] https://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git/commit/?h=rcu/nocb&id=6abe8408307e
    part of https://lwn.net/Articles/820544/
            https://lwn.net/Articles/832031/   <------ v2
            https://lwn.net/Articles/835039/   <------ v3
            https://lwn.net/Articles/837128/   <------ v4

[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d97b078182406


Paul Gortmaker (2):
  sched: isolation: cpu isolation handles for cpuset
  cpuset: add binding to CPU isolation

 include/linux/sched/isolation.h |  4 ++++
 kernel/cgroup/cpuset.c          | 42 +++++++++++++++++++++++++++++++++++++++++
 kernel/sched/isolation.c        | 22 +++++++++++++++++++++
 3 files changed, 68 insertions(+)

-- 
2.15.0

[PATCH 1/2] sched: isolation: cpu isolation handles for cpuset

[Paul Gortmaker]

Assuming we want to drive isolation from cpuset and not something
like /sys/devices/system/cpu/cpu*/hotplug/isolation then we'll
need some kind of handle for cpuset to drive it from.

These would also serve as a collection point for all the isolation
related operations - current and future.  While only RCU nocb toggle
is currently deployed, I've left some guesses at what is probably to
come in the future.

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
[PG: RFC code - not for merge]
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
 include/linux/sched/isolation.h |  4 ++++
 kernel/sched/isolation.c        | 22 ++++++++++++++++++++++
 2 files changed, 26 insertions(+)

diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index cc9f393e2a70..3ab9c667c441 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -25,6 +25,8 @@ extern bool housekeeping_enabled(enum hk_flags flags);
 extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags);
 extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags);
 extern void __init housekeeping_init(void);
+extern void isolate_cpu(int cpu);
+extern void deisolate_cpu(int cpu);
 
 #else
 
@@ -46,6 +48,8 @@ static inline bool housekeeping_enabled(enum hk_flags flags)
 static inline void housekeeping_affine(struct task_struct *t,
 				       enum hk_flags flags) { }
 static inline void housekeeping_init(void) { }
+static void isolate_cpu(int cpu) { }
+static void deisolate_cpu(int cpu) { }
 #endif /* CONFIG_CPU_ISOLATION */
 
 static inline bool housekeeping_cpu(int cpu, enum hk_flags flags)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 7f06eaf12818..57b105d42632 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -63,6 +63,28 @@ bool housekeeping_test_cpu(int cpu, enum hk_flags flags)
 }
 EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
 
+void isolate_cpu(int cpu)
+{
+	pr_info("Isolating core %d\n", cpu);
+	if (rcu_nocb_cpu_offload(cpu))
+		pr_warn("RCU; unable to nocb offload CPU %d\n", cpu);
+#if 0	/* TODO */
+	housekeeping_clear_cpu(cpu);
+	tick_nohz_full_add_cpus_to(cpumask_of(cpu));
+#endif
+}
+
+void deisolate_cpu(int cpu)
+{
+	pr_info("Deisolating core %d\n", cpu);
+#if 0	/* TODO */
+	tick_nohz_full_clear_cpus_from(cpumask_of(cpu));
+	housekeeping_add_cpu(cpu);
+#endif
+	if (rcu_nocb_cpu_deoffload(cpu))
+		pr_warn("RCU: unable to nocb reload CPU %d\n", cpu);
+}
+
 void __init housekeeping_init(void)
 {
 	if (!housekeeping_flags)
-- 
2.15.0

[PATCH 2/2] cpuset: add binding to CPU isolation

[Paul Gortmaker]

The idea is to put one or a group of cores into a set and then use
that set to control the core's isolation setting.

Currently the isolation only toggles RCU nocb, and note that a
limitation exists where you can only RCU toggle if you use a boot
arg to enable nocb offload at boot initially.  So the 1st offload
request of the subset in the cpuset are no-ops.

An example probably describes things the best:

root@hackbox:/sys/fs/cgroup/cpuset# cat /proc/cmdline
BOOT_IMAGE=/boot/bzImage rcu_nocbs=1-11 root=/dev/sda3 console=ttyS0,115200 ro
root@hackbox:/sys/fs/cgroup/cpuset# dmesg|grep Offload
[    0.154015] rcu:     Offload RCU callbacks from CPUs: 1-11.
root@hackbox:/sys/fs/cgroup/cpuset# mkdir cores4-6
root@hackbox:/sys/fs/cgroup/cpuset# cd cores4-6/
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg -c > /dev/null
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# echo 4-6 > cpuset.cpus
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# echo 1 > cpuset.cpu_isolation
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg
[ 1021.217641] Isolating core 4
[ 1021.217650] Isolating core 5
[ 1021.217651] Isolating core 6
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg -c > /dev/null
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# echo 0 > cpuset.cpu_isolation
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg
[ 1050.589700] Deisolating core 4
[ 1050.589778] rcu: De-offloading 4
[ 1050.590001] Deisolating core 5
[ 1050.590065] rcu: De-offloading 5
[ 1050.590284] Deisolating core 6
[ 1050.590366] rcu: De-offloading 6
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg -c > /dev/null
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# echo 1 > cpuset.cpu_isolation
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6# dmesg
[ 1071.584802] Isolating core 4
[ 1071.584886] rcu: Offloading 4
[ 1071.585130] Isolating core 5
[ 1071.585203] rcu: Offloading 5
[ 1071.585448] Isolating core 6
[ 1071.585459] rcu: Offloading 6
root@hackbox:/sys/fs/cgroup/cpuset/cores4-6#

Cc: Ingo Molnar <mingo@redhat.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Paul E. McKenney <paulmck@kernel.org>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Valentin Schneider <valentin.schneider@arm.com>
[PG: RFC code - not for merge]
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
---
 kernel/cgroup/cpuset.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index adb5190c4429..4420bffcc232 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -209,6 +209,7 @@ static inline struct cpuset *parent_cs(struct cpuset *cs)
 typedef enum {
 	CS_ONLINE,
 	CS_CPU_EXCLUSIVE,
+	CS_CPU_ISOLATED,
 	CS_MEM_EXCLUSIVE,
 	CS_MEM_HARDWALL,
 	CS_MEMORY_MIGRATE,
@@ -228,6 +229,11 @@ static inline int is_cpu_exclusive(const struct cpuset *cs)
 	return test_bit(CS_CPU_EXCLUSIVE, &cs->flags);
 }
 
+static inline int is_cpu_isolated(const struct cpuset *cs)
+{
+	return test_bit(CS_CPU_ISOLATED, &cs->flags);
+}
+
 static inline int is_mem_exclusive(const struct cpuset *cs)
 {
 	return test_bit(CS_MEM_EXCLUSIVE, &cs->flags);
@@ -1866,6 +1872,23 @@ static int update_relax_domain_level(struct cpuset *cs, s64 val)
 	return 0;
 }
 
+/**
+ * update_isol_state - update isolated state for allowed cores
+ * @cs: the cpuset in which each core might need to be changed
+ *
+ */
+static void update_isol_state(struct cpuset *cs)
+{
+	struct cpumask *mask = cs->cpus_allowed;
+	int cpu, isolate = is_cpu_isolated(cs);
+
+	for_each_cpu(cpu, mask)
+		if (isolate)
+			isolate_cpu(cpu);
+		else
+			deisolate_cpu(cpu);
+}
+
 /**
  * update_tasks_flags - update the spread flags of tasks in the cpuset.
  * @cs: the cpuset in which each task's spread flags needs to be changed
@@ -1900,6 +1923,7 @@ static int update_flag(cpuset_flagbits_t bit, struct cpuset *cs,
 	struct cpuset *trialcs;
 	int balance_flag_changed;
 	int spread_flag_changed;
+	int isol_flag_changed;
 	int err;
 
 	trialcs = alloc_trial_cpuset(cs);
@@ -1921,6 +1945,8 @@ static int update_flag(cpuset_flagbits_t bit, struct cpuset *cs,
 	spread_flag_changed = ((is_spread_slab(cs) != is_spread_slab(trialcs))
 			|| (is_spread_page(cs) != is_spread_page(trialcs)));
 
+	isol_flag_changed = (is_cpu_isolated(cs) != is_cpu_isolated(trialcs));
+
 	spin_lock_irq(&callback_lock);
 	cs->flags = trialcs->flags;
 	spin_unlock_irq(&callback_lock);
@@ -1930,6 +1956,9 @@ static int update_flag(cpuset_flagbits_t bit, struct cpuset *cs,
 
 	if (spread_flag_changed)
 		update_tasks_flags(cs);
+
+	if (isol_flag_changed)
+		update_isol_state(cs);
 out:
 	free_cpuset(trialcs);
 	return err;
@@ -2264,6 +2293,7 @@ typedef enum {
 	FILE_EFFECTIVE_MEMLIST,
 	FILE_SUBPARTS_CPULIST,
 	FILE_CPU_EXCLUSIVE,
+	FILE_CPU_ISOLATED,
 	FILE_MEM_EXCLUSIVE,
 	FILE_MEM_HARDWALL,
 	FILE_SCHED_LOAD_BALANCE,
@@ -2293,6 +2323,9 @@ static int cpuset_write_u64(struct cgroup_subsys_state *css, struct cftype *cft,
 	case FILE_CPU_EXCLUSIVE:
 		retval = update_flag(CS_CPU_EXCLUSIVE, cs, val);
 		break;
+	case FILE_CPU_ISOLATED:
+		retval = update_flag(CS_CPU_ISOLATED, cs, val);
+		break;
 	case FILE_MEM_EXCLUSIVE:
 		retval = update_flag(CS_MEM_EXCLUSIVE, cs, val);
 		break;
@@ -2465,6 +2498,8 @@ static u64 cpuset_read_u64(struct cgroup_subsys_state *css, struct cftype *cft)
 	switch (type) {
 	case FILE_CPU_EXCLUSIVE:
 		return is_cpu_exclusive(cs);
+	case FILE_CPU_ISOLATED:
+		return is_cpu_isolated(cs);
 	case FILE_MEM_EXCLUSIVE:
 		return is_mem_exclusive(cs);
 	case FILE_MEM_HARDWALL:
@@ -2595,6 +2630,13 @@ static struct cftype legacy_files[] = {
 		.private = FILE_CPU_EXCLUSIVE,
 	},
 
+	{
+		.name = "cpu_isolation",
+		.read_u64 = cpuset_read_u64,
+		.write_u64 = cpuset_write_u64,
+		.private = FILE_CPU_ISOLATED,
+	},
+
 	{
 		.name = "mem_exclusive",
 		.read_u64 = cpuset_read_u64,
-- 
2.15.0

Re: [PATCH 0/2] bind rcu offload (nohz_full/isolation) into cpuset

[Frederic Weisbecker]

On Wed, Oct 27, 2021 at 04:43:17PM -0400, Paul Gortmaker wrote:
> One of the earlier pre-mainline RCU nocb patchsets had a temporary sysfs
> knob in /sys/devices/system/cpu/cpu*/hotplug/nocb for testing[1].
> 
> That not-for-merge commit from Frederic said:
> 
>   This is only intended for those who want to test this patchset. The
>   real interfaces will be cpuset/isolation and rcutorture.
> 
> We've had rcutorture as the one and only mainline user of nocb toggle
> for a while now[2], and so I thought I'd take a crack at what Frederic
> had in mind for cpuset with some code vs. asking 100 random questions.
> 
> Note that I intentionally didn't Cc any cgroup/cpuset people (yet),
> since at this point this is only my guess on what things were to look
> like based on a single sentence fragment.  So this is really early
> "Not-for-Merge", but truly just RFC -- to start a conversation.
> 
> It won't be really useful until we adjust tick/housekeeping in addition
> to nocb, but I think we can develop the interface in parallel to that?
> And maybe use this to expand testing at the same time if it is layered
> on top of those future work/patchsets?  I don't know...
> 
> We'll also have to look at corner cases - like whether we want to treat
> the root cpuset differently; whether we want to sync boot arg values
> with the cpuset's initial isol flag value, whether we un-isolate cores
> when an isolation cpuset is rmdir/removed, etc etc.
> 
> But as a proof of concept, it "works" as can be seen in the 2nd commit.

I'm working on the same thing :o)

With quite a rework of housekeeping core behind (WIP):

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
	isolation/split

It's not yet ready either and I'm glad you posted this, it shows I'm not
the only one interested in it.

One thing about cpuset: I arranged to implement it only on cgroup v2 and
exclusively mutable on root partition (which doesn't mean only _the_ root
partition but also those whose cpuset.cpus.partition == "root". This way
I make sure the set of cpus is exclusive. I didn't want to bother with
intersecting cpusets with different nocb values.

Thanks.

> Paul.
> --
> 
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Cc: Paul E. McKenney <paulmck@kernel.org>
> Cc: Frederic Weisbecker <frederic@kernel.org>
> Cc: Valentin Schneider <valentin.schneider@arm.com>
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git/commit/?h=rcu/nocb&id=6abe8408307e
>     part of https://lwn.net/Articles/820544/
>             https://lwn.net/Articles/832031/   <------ v2
>             https://lwn.net/Articles/835039/   <------ v3
>             https://lwn.net/Articles/837128/   <------ v4
> 
> [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d97b078182406
> 
> 
> Paul Gortmaker (2):
>   sched: isolation: cpu isolation handles for cpuset
>   cpuset: add binding to CPU isolation
> 
>  include/linux/sched/isolation.h |  4 ++++
>  kernel/cgroup/cpuset.c          | 42 +++++++++++++++++++++++++++++++++++++++++
>  kernel/sched/isolation.c        | 22 +++++++++++++++++++++
>  3 files changed, 68 insertions(+)
> 
> -- 
> 2.15.0
> 

Re: [PATCH 1/2] sched: isolation: cpu isolation handles for cpuset

[Frederic Weisbecker]

On Wed, Oct 27, 2021 at 04:43:18PM -0400, Paul Gortmaker wrote:
> Assuming we want to drive isolation from cpuset and not something
> like /sys/devices/system/cpu/cpu*/hotplug/isolation then we'll
> need some kind of handle for cpuset to drive it from.
> 
> These would also serve as a collection point for all the isolation
> related operations - current and future.  While only RCU nocb toggle
> is currently deployed, I've left some guesses at what is probably to
> come in the future.
> 
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Cc: Paul E. McKenney <paulmck@kernel.org>
> Cc: Frederic Weisbecker <frederic@kernel.org>
> Cc: Valentin Schneider <valentin.schneider@arm.com>
> [PG: RFC code - not for merge]
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
> ---
>  include/linux/sched/isolation.h |  4 ++++
>  kernel/sched/isolation.c        | 22 ++++++++++++++++++++++
>  2 files changed, 26 insertions(+)
> 
> diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
> index cc9f393e2a70..3ab9c667c441 100644
> --- a/include/linux/sched/isolation.h
> +++ b/include/linux/sched/isolation.h
> @@ -25,6 +25,8 @@ extern bool housekeeping_enabled(enum hk_flags flags);
>  extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags);
>  extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags);
>  extern void __init housekeeping_init(void);
> +extern void isolate_cpu(int cpu);
> +extern void deisolate_cpu(int cpu);

I first read "desolate_cpu()". That gives me ideas to rename nohz_full.

>  
>  #else
>  
> @@ -46,6 +48,8 @@ static inline bool housekeeping_enabled(enum hk_flags flags)
>  static inline void housekeeping_affine(struct task_struct *t,
>  				       enum hk_flags flags) { }
>  static inline void housekeeping_init(void) { }
> +static void isolate_cpu(int cpu) { }
> +static void deisolate_cpu(int cpu) { }
>  #endif /* CONFIG_CPU_ISOLATION */
>  
>  static inline bool housekeeping_cpu(int cpu, enum hk_flags flags)
> diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
> index 7f06eaf12818..57b105d42632 100644
> --- a/kernel/sched/isolation.c
> +++ b/kernel/sched/isolation.c
> @@ -63,6 +63,28 @@ bool housekeeping_test_cpu(int cpu, enum hk_flags flags)
>  }
>  EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
>  
> +void isolate_cpu(int cpu)
> +{
> +	pr_info("Isolating core %d\n", cpu);
> +	if (rcu_nocb_cpu_offload(cpu))
> +		pr_warn("RCU; unable to nocb offload CPU %d\n", cpu);
> +#if 0	/* TODO */
> +	housekeeping_clear_cpu(cpu);
> +	tick_nohz_full_add_cpus_to(cpumask_of(cpu));
> +#endif
> +}
> +
> +void deisolate_cpu(int cpu)
> +{
> +	pr_info("Deisolating core %d\n", cpu);
> +#if 0	/* TODO */
> +	tick_nohz_full_clear_cpus_from(cpumask_of(cpu));
> +	housekeeping_add_cpu(cpu);
> +#endif
> +	if (rcu_nocb_cpu_deoffload(cpu))
> +		pr_warn("RCU: unable to nocb reload CPU %d\n", cpu);
> +}

In my series I'm eventually splitting housekeeping cpumasks so that we
can modify HK_FLAG_RCU without modifying the others.

Thanks.

> +
>  void __init housekeeping_init(void)
>  {
>  	if (!housekeeping_flags)
> -- 
> 2.15.0
>