* [PATCH 1/4] RCU: move kthread_prio bounds-check to a separate function
2022-01-11 23:32 [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Alison Chaiken
@ 2022-01-11 23:32 ` Alison Chaiken
2022-01-11 23:32 ` [PATCH 2/4] RCU: make priority of grace-period thread consistent Alison Chaiken
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Alison Chaiken @ 2022-01-11 23:32 UTC (permalink / raw)
To: tglx
Cc: peterz, paulmck, valentin.schneider, frederic, linux-kernel,
glenn, alison, Alison Chaiken
Move the bounds-check of the kthread_prio cmdline parameter to a new
function in order to faciliate a different callsite.
Signed-off-by: Alison Chaiken <achaiken@aurora.tech>
---
kernel/rcu/tree.c | 39 +++++++++++++++++++++++----------------
1 file changed, 23 insertions(+), 16 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 9a04550cc54b..15cabb288a1c 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4404,26 +4404,10 @@ static int rcu_pm_notify(struct notifier_block *self,
static int __init rcu_spawn_gp_kthread(void)
{
unsigned long flags;
- int kthread_prio_in = kthread_prio;
struct rcu_node *rnp;
struct sched_param sp;
struct task_struct *t;
- /* Force priority into range. */
- if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 2
- && IS_BUILTIN(CONFIG_RCU_TORTURE_TEST))
- kthread_prio = 2;
- else if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 1)
- kthread_prio = 1;
- else if (kthread_prio < 0)
- kthread_prio = 0;
- else if (kthread_prio > 99)
- kthread_prio = 99;
-
- if (kthread_prio != kthread_prio_in)
- pr_alert("rcu_spawn_gp_kthread(): Limited prio to %d from %d\n",
- kthread_prio, kthread_prio_in);
-
rcu_scheduler_fully_active = 1;
t = kthread_create(rcu_gp_kthread, NULL, "%s", rcu_state.name);
if (WARN_ONCE(IS_ERR(t), "%s: Could not start grace-period kthread, OOM is now expected behavior\n", __func__))
@@ -4548,6 +4532,28 @@ static void __init rcu_init_one(void)
}
}
+/*
+ * Force priority from the kernel command-line into range.
+ */
+static void __init sanitize_kthread_prio(void)
+{
+ int kthread_prio_in = kthread_prio;
+
+ if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 2
+ && IS_BUILTIN(CONFIG_RCU_TORTURE_TEST))
+ kthread_prio = 2;
+ else if (IS_ENABLED(CONFIG_RCU_BOOST) && kthread_prio < 1)
+ kthread_prio = 1;
+ else if (kthread_prio < 0)
+ kthread_prio = 0;
+ else if (kthread_prio > 99)
+ kthread_prio = 99;
+
+ if (kthread_prio != kthread_prio_in)
+ pr_alert("%s: Limited prio to %d from %d\n",
+ __func__, kthread_prio, kthread_prio_in);
+}
+
/*
* Compute the rcu_node tree geometry from kernel parameters. This cannot
* replace the definitions in tree.h because those are needed to size
@@ -4708,6 +4714,7 @@ void __init rcu_init(void)
kfree_rcu_batch_init();
rcu_bootup_announce();
+ sanitize_kthread_prio();
rcu_init_geometry();
rcu_init_one();
if (dump_tree)
--
2.32.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 2/4] RCU: make priority of grace-period thread consistent
2022-01-11 23:32 [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Alison Chaiken
2022-01-11 23:32 ` [PATCH 1/4] RCU: move kthread_prio bounds-check to a separate function Alison Chaiken
@ 2022-01-11 23:32 ` Alison Chaiken
2022-01-11 23:32 ` [PATCH 3/4] RCU: elevate priority of offloaded callback threads Alison Chaiken
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Alison Chaiken @ 2022-01-11 23:32 UTC (permalink / raw)
To: tglx
Cc: peterz, paulmck, valentin.schneider, frederic, linux-kernel,
glenn, alison, Alison Chaiken
The priority of RCU grace period threads is set to kthread_prio when
they are launched from rcu_spawn_gp_kthread(). The same is not true
of rcu_spawn_one_nocb_kthread(). Accordingly, add priority elevation
to rcu_spawn_one_nocb_kthread().
Signed-off-by: Alison Chaiken <achaiken@aurora.tech>
---
kernel/rcu/tree_nocb.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 8fdf44f8523f..08b5c0042979 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1183,7 +1183,9 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
struct rcu_data *rdp_gp;
struct task_struct *t;
+ struct sched_param sp;
+ sp.sched_priority = kthread_prio;
/*
* If this isn't a no-CBs CPU or if it already has an rcuo kthread,
* then nothing to do.
@@ -1199,6 +1201,8 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
if (WARN_ONCE(IS_ERR(t), "%s: Could not start rcuo GP kthread, OOM is now expected behavior\n", __func__))
return;
WRITE_ONCE(rdp_gp->nocb_gp_kthread, t);
+ if (kthread_prio)
+ sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
}
/* Spawn the kthread for this CPU. */
--
2.32.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH 3/4] RCU: elevate priority of offloaded callback threads
2022-01-11 23:32 [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Alison Chaiken
2022-01-11 23:32 ` [PATCH 1/4] RCU: move kthread_prio bounds-check to a separate function Alison Chaiken
2022-01-11 23:32 ` [PATCH 2/4] RCU: make priority of grace-period thread consistent Alison Chaiken
@ 2022-01-11 23:32 ` Alison Chaiken
2022-01-11 23:32 ` [PATCH 4/4] RCU: update documentation regarding kthread_prio cmdline parameter Alison Chaiken
2022-01-12 1:03 ` [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Alison Chaiken @ 2022-01-11 23:32 UTC (permalink / raw)
To: tglx
Cc: peterz, paulmck, valentin.schneider, frederic, linux-kernel,
glenn, alison, Alison Chaiken
When CONFIG_PREEMPT_RT=y, the rcutree.kthread_prio command-line
parameter signals initialization code to boost the priority of rcuc
callbacks to the designated value. With the additional
CONFIG_RCU_NOCB_CPU=y configuration and an additional rcu_nocbs
command-line parameter, the callbacks on the listed cores are
offloaded to new rcuop threads that are not pinned to the cores whose
post-grace-period work is performed. While the rcuop threads perform
the same function as the rcuc threads they offload, the kthread_prio
parameter only boosts the priority of the rcuc threads. Fix this
inconsistency by elevating rcuop threads to the same priority as rcuc
ones.
Signed-off-by: Alison Chaiken <achaiken@aurora.tech>
---
kernel/rcu/tree.c | 2 +-
kernel/rcu/tree_nocb.h | 3 +++
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 15cabb288a1c..ba9eaa9a72b0 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -153,7 +153,7 @@ static void sync_sched_exp_online_cleanup(int cpu);
static void check_cb_ovld_locked(struct rcu_data *rdp, struct rcu_node *rnp);
static bool rcu_rdp_is_offloaded(struct rcu_data *rdp);
-/* rcuc/rcub kthread realtime priority */
+/* rcuc/rcub/rcuop kthread realtime priority */
static int kthread_prio = IS_ENABLED(CONFIG_RCU_BOOST) ? 1 : 0;
module_param(kthread_prio, int, 0444);
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 08b5c0042979..3343e42685dc 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1210,6 +1210,9 @@ static void rcu_spawn_one_nocb_kthread(int cpu)
"rcuo%c/%d", rcu_state.abbr, cpu);
if (WARN_ONCE(IS_ERR(t), "%s: Could not start rcuo CB kthread, OOM is now expected behavior\n", __func__))
return;
+
+ if (kthread_prio)
+ sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
WRITE_ONCE(rdp->nocb_cb_kthread, t);
WRITE_ONCE(rdp->nocb_gp_kthread, rdp_gp->nocb_gp_kthread);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 4/4] RCU: update documentation regarding kthread_prio cmdline parameter
2022-01-11 23:32 [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Alison Chaiken
` (2 preceding siblings ...)
2022-01-11 23:32 ` [PATCH 3/4] RCU: elevate priority of offloaded callback threads Alison Chaiken
@ 2022-01-11 23:32 ` Alison Chaiken
2022-01-12 1:03 ` [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Alison Chaiken @ 2022-01-11 23:32 UTC (permalink / raw)
To: tglx
Cc: peterz, paulmck, valentin.schneider, frederic, linux-kernel,
glenn, alison, Alison Chaiken
Inform readers that the priority of RCU no-callback threads will also
be boosted.
Signed-off-by: Alison Chaiken <achaiken@aurora.tech>
---
Documentation/admin-guide/kernel-parameters.txt | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 1396fd2d9031..b844978e1bdf 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4439,6 +4439,8 @@
(the least-favored priority). Otherwise, when
RCU_BOOST is not set, valid values are 0-99 and
the default is zero (non-realtime operation).
+ When RCU_NOCB_CPU is set, also adjust the
+ priority of NOCB callback kthreads.
rcutree.rcu_nocb_gp_stride= [KNL]
Set the number of NOCB callback kthreads in
--
2.32.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* Re: [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently
2022-01-11 23:32 [PATCH 0/5] handle kthread_prio kernel cmdline parameter consistently Alison Chaiken
` (3 preceding siblings ...)
2022-01-11 23:32 ` [PATCH 4/4] RCU: update documentation regarding kthread_prio cmdline parameter Alison Chaiken
@ 2022-01-12 1:03 ` Paul E. McKenney
4 siblings, 0 replies; 6+ messages in thread
From: Paul E. McKenney @ 2022-01-12 1:03 UTC (permalink / raw)
To: Alison Chaiken
Cc: tglx, peterz, valentin.schneider, frederic, linux-kernel, glenn, alison
On Tue, Jan 11, 2022 at 03:32:49PM -0800, Alison Chaiken wrote:
> When CONFIG_PREEMPT_RT=y, RCU_SOFTIRQ work is moved to dedicated rcuc
> per-core threads. The rcutree.kthread_prio kernel cmdline parameter
> controls the priority of those threads.
>
> A provided kthread_prio kernel cmdline parameter also elevates the
> priority of rcuog threads, but only if they are launched from
> rcu_spawn_gp_kthread(), not if they are launched from
> rcu_spawn_one_nocb_kthread(). Fix this inconsistency.
>
> When CONFIG_RCU_NOCB_CPU=y, an rcu_nocbs command-line parameter
> offloads the work of rcuc on designated cores to new rcuop threads
> that are not pinned to the cores whose expired grace-period timer
> callbacks they run. While rcuop threads have the same function as
> rcuc threads, their priority is not controlled by the kthread_prio
> parameter. Add this feature and update the documentation accordingly.
>
> Alison Chaiken (4):
> RCU: move kthread_prio bounds-check to a separate function
> RCU: make priority of grace-period thread consistent
> RCU: elevate priority of offloaded callback threads
> RCU: update documentation regarding kthread_prio cmdline parameter
I have queued these for review and testing, thank you!
2/4 appears to have been produced against a pre-merge-window mainline,
so I had to adjust it a bit to queue it on -rcu. Please check to make
sure that I did not mess anything up.
Thanx, Paul
^ permalink raw reply [flat|nested] 6+ messages in thread