* [GIT PULL] Housekeeping subsystem v6
@ 2017-10-27 2:42 Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
` (11 more replies)
0 siblings, 12 replies; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Ingo,
Please pull the core/isolation-v7 branch that can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
core/isolation-v7
HEAD: 69dc2180d1f17ff1f861c7231f4a07423c30842a
Changes since last take:
* Move kernel/housekeeping.c to kernel/sched/isolation.c
and linux/housekeeping.h to linux/sched/isolation.h
* Fix tilegx build error reported by Kbuild test robot
* Rebase against latest tip:sched/core
Thanks,
Frederic
---
Frederic Weisbecker (12):
housekeeping: Move housekeeping related code to its own file
watchdog: Use housekeeping_cpumask() instead of ad-hoc version
housekeeping: Provide a dynamic off-case to housekeeping_any_cpu()
housekeeping: Make housekeeping cpumask private
housekeeping: Use its own static key
housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu
housekeeping: Move it under its own config, independent from NO_HZ
housekeeping: Introduce housekeeping flags
housekeeping: Handle the nohz_full= parameter
housekeeping: Move isolcpus to housekeeping
housekeeping: Add basic isolcpus flags
housekeeping: Document isolcpus flags
Documentation/admin-guide/kernel-parameters.txt | 33 +++--
drivers/base/cpu.c | 11 +-
drivers/net/ethernet/tile/tilegx.c | 6 +-
include/linux/sched.h | 2 -
include/linux/sched/isolation.h | 51 ++++++++
include/linux/tick.h | 39 +-----
init/Kconfig | 7 ++
init/main.c | 2 +
kernel/cgroup/cpuset.c | 15 +--
kernel/rcu/tree_plugin.h | 3 +-
kernel/rcu/update.c | 3 +-
kernel/sched/Makefile | 1 +
kernel/sched/core.c | 25 +---
kernel/sched/fair.c | 3 +-
kernel/sched/isolation.c | 155 ++++++++++++++++++++++++
kernel/sched/topology.c | 24 +---
kernel/time/tick-sched.c | 31 +----
kernel/watchdog.c | 13 +-
18 files changed, 282 insertions(+), 142 deletions(-)
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH 01/12] housekeeping: Move housekeeping related code to its own file
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:01 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 02/12] watchdog: Use housekeeping_cpumask() instead of ad-hoc version Frederic Weisbecker
` (10 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
The housekeeping code is currently tied to the nohz code. As we are
planning to make housekeeping independant from it, start with moving
the relevant code to its own file.
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
drivers/net/ethernet/tile/tilegx.c | 2 +-
include/linux/sched/isolation.h | 56 ++++++++++++++++++++++++++++++++++++++
include/linux/tick.h | 37 -------------------------
init/main.c | 2 ++
kernel/rcu/tree_plugin.h | 1 +
kernel/rcu/update.c | 1 +
kernel/sched/Makefile | 1 +
kernel/sched/core.c | 1 +
kernel/sched/fair.c | 1 +
kernel/sched/isolation.c | 33 ++++++++++++++++++++++
kernel/time/tick-sched.c | 18 ------------
kernel/watchdog.c | 1 +
12 files changed, 98 insertions(+), 56 deletions(-)
create mode 100644 include/linux/sched/isolation.h
create mode 100644 kernel/sched/isolation.c
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index c00102b..27a3272 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -40,7 +40,7 @@
#include <linux/tcp.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
-#include <linux/tick.h>
+#include <linux/sched/isolation.h>
#include <asm/checksum.h>
#include <asm/homecache.h>
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
new file mode 100644
index 0000000..b7cfbc4
--- /dev/null
+++ b/include/linux/sched/isolation.h
@@ -0,0 +1,56 @@
+#ifndef _LINUX_SCHED_ISOLATION_H
+#define _LINUX_SCHED_ISOLATION_H
+
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/tick.h>
+
+#ifdef CONFIG_NO_HZ_FULL
+extern cpumask_var_t housekeeping_mask;
+
+static inline int housekeeping_any_cpu(void)
+{
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+}
+
+extern void __init housekeeping_init(void);
+
+#else
+
+static inline int housekeeping_any_cpu(void)
+{
+ return smp_processor_id();
+}
+
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
+
+static inline const struct cpumask *housekeeping_cpumask(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+#endif
+ return cpu_possible_mask;
+}
+
+static inline bool is_housekeeping_cpu(int cpu)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+#endif
+ return true;
+}
+
+static inline void housekeeping_affine(struct task_struct *t)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+
+#endif
+}
+
+#endif /* _LINUX_SCHED_ISOLATION_H */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index fe01e68..68afc09 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
extern cpumask_var_t tick_nohz_full_mask;
-extern cpumask_var_t housekeeping_mask;
static inline bool tick_nohz_full_enabled(void)
{
@@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
cpumask_or(mask, mask, tick_nohz_full_mask);
}
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void tick_nohz_dep_set(enum tick_dep_bits bit);
extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
@@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
#else
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
@@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
#endif
-static inline const struct cpumask *housekeeping_cpumask(void)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
- return cpu_possible_mask;
-}
-
-static inline bool is_housekeeping_cpu(int cpu)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
-#endif
- return true;
-}
-
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
static inline void tick_nohz_task_switch(void)
{
if (tick_nohz_full_enabled())
diff --git a/init/main.c b/init/main.c
index 0ee9c686..4610c99 100644
--- a/init/main.c
+++ b/init/main.c
@@ -46,6 +46,7 @@
#include <linux/cgroup.h>
#include <linux/efi.h>
#include <linux/tick.h>
+#include <linux/sched/isolation.h>
#include <linux/interrupt.h>
#include <linux/taskstats_kern.h>
#include <linux/delayacct.h>
@@ -606,6 +607,7 @@ asmlinkage __visible void __init start_kernel(void)
early_irq_init();
init_IRQ();
tick_init();
+ housekeeping_init();
rcu_init_nohz();
init_timers();
hrtimers_init();
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e012b9b..c7632f5 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -29,6 +29,7 @@
#include <linux/oom.h>
#include <linux/sched/debug.h>
#include <linux/smpboot.h>
+#include <linux/sched/isolation.h>
#include <uapi/linux/sched/types.h>
#include "../time/tick-internal.h"
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 5033b66..79abeb0 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -51,6 +51,7 @@
#include <linux/kthread.h>
#include <linux/tick.h>
#include <linux/rcupdate_wait.h>
+#include <linux/sched/isolation.h>
#define CREATE_TRACE_POINTS
diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile
index 78f5493..871d43d 100644
--- a/kernel/sched/Makefile
+++ b/kernel/sched/Makefile
@@ -26,3 +26,4 @@ obj-$(CONFIG_CGROUP_CPUACCT) += cpuacct.o
obj-$(CONFIG_CPU_FREQ) += cpufreq.o
obj-$(CONFIG_CPU_FREQ_GOV_SCHEDUTIL) += cpufreq_schedutil.o
obj-$(CONFIG_MEMBARRIER) += membarrier.o
+obj-$(CONFIG_NO_HZ_FULL) += isolation.o
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2288a14..ad188ac 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -26,6 +26,7 @@
#include <linux/profile.h>
#include <linux/security.h>
#include <linux/syscalls.h>
+#include <linux/sched/isolation.h>
#include <asm/switch_to.h>
#include <asm/tlb.h>
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 56f343b..591481d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -32,6 +32,7 @@
#include <linux/mempolicy.h>
#include <linux/migrate.h>
#include <linux/task_work.h>
+#include <linux/sched/isolation.h>
#include <trace/events/sched.h>
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
new file mode 100644
index 0000000..3589252
--- /dev/null
+++ b/kernel/sched/isolation.c
@@ -0,0 +1,33 @@
+/*
+ * Housekeeping management. Manage the targets for routine code that can run on
+ * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
+ *
+ * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ *
+ */
+
+#include <linux/sched/isolation.h>
+#include <linux/tick.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+
+cpumask_var_t housekeeping_mask;
+
+void __init housekeeping_init(void)
+{
+ if (!tick_nohz_full_enabled())
+ return;
+
+ if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
+ WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
+ cpumask_clear(tick_nohz_full_mask);
+ tick_nohz_full_running = false;
+ return;
+ }
+
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, tick_nohz_full_mask);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 7b258c5..27d7d52 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -166,7 +166,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
#ifdef CONFIG_NO_HZ_FULL
cpumask_var_t tick_nohz_full_mask;
-cpumask_var_t housekeeping_mask;
bool tick_nohz_full_running;
static atomic_t tick_dep_mask;
@@ -438,13 +437,6 @@ void __init tick_nohz_init(void)
return;
}
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
/*
* Full dynticks uses irq work to drive the tick rescheduling on safe
* locking contexts. But then we need irq work to raise its own
@@ -453,7 +445,6 @@ void __init tick_nohz_init(void)
if (!arch_irq_work_has_interrupt()) {
pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
cpumask_clear(tick_nohz_full_mask);
- cpumask_copy(housekeeping_mask, cpu_possible_mask);
tick_nohz_full_running = false;
return;
}
@@ -466,9 +457,6 @@ void __init tick_nohz_init(void)
cpumask_clear_cpu(cpu, tick_nohz_full_mask);
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
for_each_cpu(cpu, tick_nohz_full_mask)
context_tracking_cpu_set(cpu);
@@ -478,12 +466,6 @@ void __init tick_nohz_init(void)
WARN_ON(ret < 0);
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
-
- /*
- * We need at least one CPU to handle housekeeping work such
- * as timekeeping, unbound timers, workqueues, ...
- */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
#endif
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 6bcb854..3c44dba 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -24,6 +24,7 @@
#include <linux/workqueue.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
+#include <linux/sched/isolation.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 02/12] watchdog: Use housekeeping_cpumask() instead of ad-hoc version
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:02 ` [tip:sched/core] sched/isolation, " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 03/12] housekeeping: Provide a dynamic off-case to housekeeping_any_cpu() Frederic Weisbecker
` (9 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
While trying to disable the watchog on nohz_full CPUs, the watchdog
implements an ad-hoc version of housekeeping_cpumask(). Lets replace
those re-invented lines.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
kernel/watchdog.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 3c44dba..562652c 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -774,15 +774,10 @@ int proc_watchdog_cpumask(struct ctl_table *table, int write,
void __init lockup_detector_init(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled()) {
+ if (tick_nohz_full_enabled())
pr_info("Disabling watchdog on nohz_full cores by default\n");
- cpumask_copy(&watchdog_cpumask, housekeeping_mask);
- } else
- cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
-#else
- cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
-#endif
+
+ cpumask_copy(&watchdog_cpumask, housekeeping_cpumask());
if (!watchdog_nmi_probe())
nmi_watchdog_available = true;
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 03/12] housekeeping: Provide a dynamic off-case to housekeeping_any_cpu()
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 02/12] watchdog: Use housekeeping_cpumask() instead of ad-hoc version Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:02 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 04/12] housekeeping: Make housekeeping cpumask private Frederic Weisbecker
` (8 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
housekeeping_any_cpu() doesn't handle correctly the case where
CONFIG_NO_HZ_FULL=y and no CPU is in nohz_full mode. So far no caller
needs this but let's prepare to avoid any future surprise.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 21 ++++++++-------------
1 file changed, 8 insertions(+), 13 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index b7cfbc4..040df04 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -7,24 +7,19 @@
#ifdef CONFIG_NO_HZ_FULL
extern cpumask_var_t housekeeping_mask;
-
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void __init housekeeping_init(void);
-
#else
-
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
-
static inline void housekeeping_init(void) { }
#endif /* CONFIG_NO_HZ_FULL */
+static inline int housekeeping_any_cpu(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+#endif
+ return smp_processor_id();
+}
static inline const struct cpumask *housekeeping_cpumask(void)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 04/12] housekeeping: Make housekeeping cpumask private
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (2 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 03/12] housekeeping: Provide a dynamic off-case to housekeeping_any_cpu() Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: Make the " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 05/12] housekeeping: Use its own static key Frederic Weisbecker
` (7 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Nobody needs to access this detail. housekeeping_cpumask() already
takes care about it.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 31 ++++++++++---------------------
kernel/sched/isolation.c | 36 +++++++++++++++++++++++++++++++++++-
2 files changed, 45 insertions(+), 22 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 040df04..ed935ff 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -6,46 +6,35 @@
#include <linux/tick.h>
#ifdef CONFIG_NO_HZ_FULL
-extern cpumask_var_t housekeeping_mask;
+extern int housekeeping_any_cpu(void);
+extern const struct cpumask *housekeeping_cpumask(void);
+extern void housekeeping_affine(struct task_struct *t);
+extern bool housekeeping_test_cpu(int cpu);
extern void __init housekeeping_init(void);
+
#else
-static inline void housekeeping_init(void) { }
-#endif /* CONFIG_NO_HZ_FULL */
static inline int housekeeping_any_cpu(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-#endif
return smp_processor_id();
}
static inline const struct cpumask *housekeeping_cpumask(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
return cpu_possible_mask;
}
+static inline void housekeeping_affine(struct task_struct *t) { }
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
static inline bool is_housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
+ return housekeeping_test_cpu(cpu);
#endif
return true;
}
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
#endif /* _LINUX_SCHED_ISOLATION_H */
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 3589252..1644509 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -11,7 +11,41 @@
#include <linux/init.h>
#include <linux/kernel.h>
-cpumask_var_t housekeeping_mask;
+static cpumask_var_t housekeeping_mask;
+
+int housekeeping_any_cpu(void)
+{
+ if (tick_nohz_full_enabled())
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+
+ return smp_processor_id();
+}
+EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
+
+const struct cpumask *housekeeping_cpumask(void)
+{
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+
+ return cpu_possible_mask;
+}
+EXPORT_SYMBOL_GPL(housekeeping_cpumask);
+
+void housekeeping_affine(struct task_struct *t)
+{
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+}
+EXPORT_SYMBOL_GPL(housekeeping_affine);
+
+bool housekeeping_test_cpu(int cpu)
+{
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
void __init housekeeping_init(void)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 05/12] housekeeping: Use its own static key
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (3 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 04/12] housekeeping: Make housekeeping cpumask private Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 06/12] housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu Frederic Weisbecker
` (6 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Housekeeping code still depends on nohz_full static key. Since we want
to decouple housekeeping from nohz, let's create a housekeeping own static
key. It's mostly relevant for calls to is_housekeeping_cpu() from the
scheduler.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 3 ++-
kernel/sched/isolation.c | 13 +++++++++----
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index ed935ff..194c586 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -6,6 +6,7 @@
#include <linux/tick.h>
#ifdef CONFIG_NO_HZ_FULL
+DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
extern int housekeeping_any_cpu(void);
extern const struct cpumask *housekeeping_cpumask(void);
extern void housekeeping_affine(struct task_struct *t);
@@ -31,7 +32,7 @@ static inline void housekeeping_init(void) { }
static inline bool is_housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_test_cpu(cpu);
#endif
return true;
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 1644509..bb8ba19 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -10,12 +10,15 @@
#include <linux/tick.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/static_key.h>
+DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
+EXPORT_SYMBOL_GPL(housekeeping_overriden);
static cpumask_var_t housekeeping_mask;
int housekeeping_any_cpu(void)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return cpumask_any_and(housekeeping_mask, cpu_online_mask);
return smp_processor_id();
@@ -24,7 +27,7 @@ EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
const struct cpumask *housekeeping_cpumask(void)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_mask;
return cpu_possible_mask;
@@ -33,14 +36,14 @@ EXPORT_SYMBOL_GPL(housekeeping_cpumask);
void housekeeping_affine(struct task_struct *t)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
set_cpus_allowed_ptr(t, housekeeping_mask);
}
EXPORT_SYMBOL_GPL(housekeeping_affine);
bool housekeeping_test_cpu(int cpu)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return cpumask_test_cpu(cpu, housekeeping_mask);
return true;
@@ -62,6 +65,8 @@ void __init housekeeping_init(void)
cpumask_andnot(housekeeping_mask,
cpu_possible_mask, tick_nohz_full_mask);
+ static_branch_enable(&housekeeping_overriden);
+
/* We need at least one CPU to handle housekeeping work */
WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 06/12] housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (4 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 05/12] housekeeping: Use its own static key Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: Rename is_housekeeping_cpu() to housekeeping_cpu() tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 07/12] housekeeping: Move it under its own config, independent from NO_HZ Frederic Weisbecker
` (5 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
To keep a proper housekeeping namespace.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 2 +-
kernel/sched/core.c | 6 +++---
kernel/sched/fair.c | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 194c586..ad0f5d9 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -29,7 +29,7 @@ static inline void housekeeping_affine(struct task_struct *t) { }
static inline void housekeeping_init(void) { }
#endif /* CONFIG_NO_HZ_FULL */
-static inline bool is_housekeeping_cpu(int cpu)
+static inline bool housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
if (static_branch_unlikely(&housekeeping_overriden))
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ad188ac..d0fb448d 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -527,7 +527,7 @@ int get_nohz_timer_target(void)
int i, cpu = smp_processor_id();
struct sched_domain *sd;
- if (!idle_cpu(cpu) && is_housekeeping_cpu(cpu))
+ if (!idle_cpu(cpu) && housekeeping_cpu(cpu))
return cpu;
rcu_read_lock();
@@ -536,14 +536,14 @@ int get_nohz_timer_target(void)
if (cpu == i)
continue;
- if (!idle_cpu(i) && is_housekeeping_cpu(i)) {
+ if (!idle_cpu(i) && housekeeping_cpu(i)) {
cpu = i;
goto unlock;
}
}
}
- if (!is_housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu))
cpu = housekeeping_any_cpu();
unlock:
rcu_read_unlock();
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 591481d..cdece8f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9027,7 +9027,7 @@ void nohz_balance_enter_idle(int cpu)
return;
/* Spare idle load balancing on CPUs that don't want to be disturbed: */
- if (!is_housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu))
return;
if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 07/12] housekeeping: Move it under its own config, independent from NO_HZ
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (5 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 06/12] housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:04 ` [tip:sched/core] sched/isolation: Split out new CONFIG_CPU_ISOLATION=y config from CONFIG_NO_HZ_FULL tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 08/12] housekeeping: Introduce housekeeping flags Frederic Weisbecker
` (4 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Split housekeeping config from CONFIG_NO_HZ_FULL. This way we finally
separate the isolation code from nohz.
Although a dependency to CONFIG_NO_HZ_FULL remains for now until the
housekeeping code doesn't deal anymore with nohz internals directly.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 6 +++---
init/Kconfig | 8 ++++++++
kernel/sched/Makefile | 2 +-
3 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index ad0f5d9..93ac2367 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -5,7 +5,7 @@
#include <linux/init.h>
#include <linux/tick.h>
-#ifdef CONFIG_NO_HZ_FULL
+#ifdef CONFIG_CPU_ISOLATION
DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
extern int housekeeping_any_cpu(void);
extern const struct cpumask *housekeeping_cpumask(void);
@@ -27,11 +27,11 @@ static inline const struct cpumask *housekeeping_cpumask(void)
static inline void housekeeping_affine(struct task_struct *t) { }
static inline void housekeeping_init(void) { }
-#endif /* CONFIG_NO_HZ_FULL */
+#endif /* CONFIG_CPU_ISOLATION */
static inline bool housekeeping_cpu(int cpu)
{
-#ifdef CONFIG_NO_HZ_FULL
+#ifdef CONFIG_CPU_ISOLATION
if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_test_cpu(cpu);
#endif
diff --git a/init/Kconfig b/init/Kconfig
index 78cb246..6f52e6f 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -472,6 +472,14 @@ config TASK_IO_ACCOUNTING
endmenu # "CPU/Task time and stats accounting"
+config CPU_ISOLATION
+ bool "CPU isolation"
+ depends on NO_HZ_FULL
+ help
+ Make sure that CPUs running critical tasks are not disturbed by
+ any source of "noise" such as unbound workqueues, timers, kthreads...
+ Unbound jobs get offloaded to housekeeping CPUs.
+
source "kernel/rcu/Kconfig"
config BUILD_BIN2C
diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile
index 871d43d..dbe6130 100644
--- a/kernel/sched/Makefile
+++ b/kernel/sched/Makefile
@@ -26,4 +26,4 @@ obj-$(CONFIG_CGROUP_CPUACCT) += cpuacct.o
obj-$(CONFIG_CPU_FREQ) += cpufreq.o
obj-$(CONFIG_CPU_FREQ_GOV_SCHEDUTIL) += cpufreq_schedutil.o
obj-$(CONFIG_MEMBARRIER) += membarrier.o
-obj-$(CONFIG_NO_HZ_FULL) += isolation.o
+obj-$(CONFIG_CPU_ISOLATION) += isolation.o
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 08/12] housekeeping: Introduce housekeeping flags
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (6 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 07/12] housekeeping: Move it under its own config, independent from NO_HZ Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:04 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 09/12] housekeeping: Handle the nohz_full= parameter Frederic Weisbecker
` (3 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Before we implement isolcpus under housekeeping, we need the isolation
features to be more finegrained. For example some people want nohz_full
without the full scheduler isolation, others want full scheduler
isolation without nohz_full.
So let's cut all these isolation features piecewise, at the risk of
overcutting it right now. We can still merge some flags later if they
always make sense together.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
drivers/net/ethernet/tile/tilegx.c | 4 ++--
include/linux/sched/isolation.h | 26 +++++++++++++++++---------
kernel/rcu/tree_plugin.h | 2 +-
kernel/rcu/update.c | 2 +-
kernel/sched/core.c | 8 ++++----
kernel/sched/fair.c | 2 +-
kernel/sched/isolation.c | 26 +++++++++++++++-----------
kernel/watchdog.c | 3 ++-
8 files changed, 43 insertions(+), 30 deletions(-)
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index 27a3272..b3e5816 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -2270,8 +2270,8 @@ static int __init tile_net_init_module(void)
tile_net_dev_init(name, mac);
if (!network_cpus_init())
- cpumask_and(&network_cpus_map, housekeeping_cpumask(),
- cpu_online_mask);
+ cpumask_and(&network_cpus_map,
+ housekeeping_cpumask(HK_FLAG_MISC), cpu_online_mask);
return 0;
}
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 93ac2367..9bb753e 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -5,35 +5,43 @@
#include <linux/init.h>
#include <linux/tick.h>
+enum hk_flags {
+ HK_FLAG_TIMER = 1,
+ HK_FLAG_RCU = (1 << 1),
+ HK_FLAG_MISC = (1 << 2),
+ HK_FLAG_SCHED = (1 << 3),
+};
+
#ifdef CONFIG_CPU_ISOLATION
DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
-extern int housekeeping_any_cpu(void);
-extern const struct cpumask *housekeeping_cpumask(void);
-extern void housekeeping_affine(struct task_struct *t);
-extern bool housekeeping_test_cpu(int cpu);
+extern int housekeeping_any_cpu(enum hk_flags flags);
+extern const struct cpumask *housekeeping_cpumask(enum hk_flags flags);
+extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags);
+extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags);
extern void __init housekeeping_init(void);
#else
-static inline int housekeeping_any_cpu(void)
+static inline int housekeeping_any_cpu(enum hk_flags flags)
{
return smp_processor_id();
}
-static inline const struct cpumask *housekeeping_cpumask(void)
+static inline const struct cpumask *housekeeping_cpumask(enum hk_flags flags)
{
return cpu_possible_mask;
}
-static inline void housekeeping_affine(struct task_struct *t) { }
+static inline void housekeeping_affine(struct task_struct *t,
+ enum hk_flags flags) { }
static inline void housekeeping_init(void) { }
#endif /* CONFIG_CPU_ISOLATION */
-static inline bool housekeeping_cpu(int cpu)
+static inline bool housekeeping_cpu(int cpu, enum hk_flags flags)
{
#ifdef CONFIG_CPU_ISOLATION
if (static_branch_unlikely(&housekeeping_overriden))
- return housekeeping_test_cpu(cpu);
+ return housekeeping_test_cpu(cpu, flags);
#endif
return true;
}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index c7632f5..34125d2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -2584,7 +2584,7 @@ static void rcu_bind_gp_kthread(void)
if (!tick_nohz_full_enabled())
return;
- housekeeping_affine(current);
+ housekeeping_affine(current, HK_FLAG_RCU);
}
/* Record the current task on dyntick-idle entry. */
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 79abeb0..e3e60ef 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -719,7 +719,7 @@ static int __noreturn rcu_tasks_kthread(void *arg)
LIST_HEAD(rcu_tasks_holdouts);
/* Run on housekeeping CPUs by default. Sysadm can move if desired. */
- housekeeping_affine(current);
+ housekeeping_affine(current, HK_FLAG_RCU);
/*
* Each pass through the following loop makes one check for
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d0fb448d..2210c02 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -527,7 +527,7 @@ int get_nohz_timer_target(void)
int i, cpu = smp_processor_id();
struct sched_domain *sd;
- if (!idle_cpu(cpu) && housekeeping_cpu(cpu))
+ if (!idle_cpu(cpu) && housekeeping_cpu(cpu, HK_FLAG_TIMER))
return cpu;
rcu_read_lock();
@@ -536,15 +536,15 @@ int get_nohz_timer_target(void)
if (cpu == i)
continue;
- if (!idle_cpu(i) && housekeeping_cpu(i)) {
+ if (!idle_cpu(i) && housekeeping_cpu(i, HK_FLAG_TIMER)) {
cpu = i;
goto unlock;
}
}
}
- if (!housekeeping_cpu(cpu))
- cpu = housekeeping_any_cpu();
+ if (!housekeeping_cpu(cpu, HK_FLAG_TIMER))
+ cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
unlock:
rcu_read_unlock();
return cpu;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index cdece8f..f755de8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9027,7 +9027,7 @@ void nohz_balance_enter_idle(int cpu)
return;
/* Spare idle load balancing on CPUs that don't want to be disturbed: */
- if (!housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu, HK_FLAG_SCHED))
return;
if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index bb8ba19..37a138a 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -15,37 +15,39 @@
DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
EXPORT_SYMBOL_GPL(housekeeping_overriden);
static cpumask_var_t housekeeping_mask;
+static unsigned int housekeeping_flags;
-int housekeeping_any_cpu(void)
+int housekeeping_any_cpu(enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-
+ if (housekeeping_flags & flags)
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
return smp_processor_id();
}
EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
-const struct cpumask *housekeeping_cpumask(void)
+const struct cpumask *housekeeping_cpumask(enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return housekeeping_mask;
-
+ if (housekeeping_flags & flags)
+ return housekeeping_mask;
return cpu_possible_mask;
}
EXPORT_SYMBOL_GPL(housekeeping_cpumask);
-void housekeeping_affine(struct task_struct *t)
+void housekeeping_affine(struct task_struct *t, enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- set_cpus_allowed_ptr(t, housekeeping_mask);
+ if (housekeeping_flags & flags)
+ set_cpus_allowed_ptr(t, housekeeping_mask);
}
EXPORT_SYMBOL_GPL(housekeeping_affine);
-bool housekeeping_test_cpu(int cpu)
+bool housekeeping_test_cpu(int cpu, enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return cpumask_test_cpu(cpu, housekeeping_mask);
-
+ if (housekeeping_flags & flags)
+ return cpumask_test_cpu(cpu, housekeeping_mask);
return true;
}
EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
@@ -65,6 +67,8 @@ void __init housekeeping_init(void)
cpumask_andnot(housekeeping_mask,
cpu_possible_mask, tick_nohz_full_mask);
+ housekeeping_flags = HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+
static_branch_enable(&housekeeping_overriden);
/* We need at least one CPU to handle housekeeping work */
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 562652c..e9e2ebb 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -777,7 +777,8 @@ void __init lockup_detector_init(void)
if (tick_nohz_full_enabled())
pr_info("Disabling watchdog on nohz_full cores by default\n");
- cpumask_copy(&watchdog_cpumask, housekeeping_cpumask());
+ cpumask_copy(&watchdog_cpumask,
+ housekeeping_cpumask(HK_FLAG_TIMER));
if (!watchdog_nmi_probe())
nmi_watchdog_available = true;
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 09/12] housekeeping: Handle the nohz_full= parameter
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (7 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 08/12] housekeeping: Introduce housekeeping flags Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 10/12] housekeeping: Move isolcpus to housekeeping Frederic Weisbecker
` (2 subsequent siblings)
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
We want to centralize the isolation management from the housekeeping
subsystem. Therefore we need to handle the nohz_full= parameter from
there.
Since nohz_full= so far has involved unbound timers, watchdog, RCU
and tilegx NAPI isolation, we keep that default behaviour.
nohz_full= is deemed to be deprecated in the future. We want to control
the isolation features from the isolcpus= parameter.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Wanpeng Li <kernellwp@gmail.com>
---
include/linux/sched/isolation.h | 1 +
include/linux/tick.h | 2 ++
init/Kconfig | 1 -
kernel/sched/isolation.c | 44 +++++++++++++++++++++++++++++------------
kernel/time/tick-sched.c | 13 +++---------
5 files changed, 37 insertions(+), 24 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 9bb753e..e53cfa9 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -10,6 +10,7 @@ enum hk_flags {
HK_FLAG_RCU = (1 << 1),
HK_FLAG_MISC = (1 << 2),
HK_FLAG_SCHED = (1 << 3),
+ HK_FLAG_TICK = (1 << 4),
};
#ifdef CONFIG_CPU_ISOLATION
diff --git a/include/linux/tick.h b/include/linux/tick.h
index 68afc09..e2a163a 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -228,6 +228,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
+extern void __init tick_nohz_full_setup(cpumask_var_t cpumask);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
@@ -248,6 +249,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
+static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { }
#endif
static inline void tick_nohz_task_switch(void)
diff --git a/init/Kconfig b/init/Kconfig
index 6f52e6f..f8564df5 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -474,7 +474,6 @@ endmenu # "CPU/Task time and stats accounting"
config CPU_ISOLATION
bool "CPU isolation"
- depends on NO_HZ_FULL
help
Make sure that CPUs running critical tasks are not disturbed by
any source of "noise" such as unbound workqueues, timers, kthreads...
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 37a138a..1f61e44 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -54,23 +54,41 @@ EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
void __init housekeeping_init(void)
{
- if (!tick_nohz_full_enabled())
+ if (!housekeeping_flags)
return;
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
- housekeeping_flags = HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
-
static_branch_enable(&housekeeping_overriden);
/* We need at least one CPU to handle housekeeping work */
WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
+
+#ifdef CONFIG_NO_HZ_FULL
+static int __init housekeeping_nohz_full_setup(char *str)
+{
+ cpumask_var_t non_housekeeping_mask;
+
+ alloc_bootmem_cpumask_var(&non_housekeeping_mask);
+ if (cpulist_parse(str, non_housekeeping_mask) < 0) {
+ pr_warn("Housekeeping: Incorrect nohz_full cpumask\n");
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
+ }
+
+ alloc_bootmem_cpumask_var(&housekeeping_mask);
+ cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask);
+
+ if (cpumask_empty(housekeeping_mask))
+ cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+
+ housekeeping_flags = HK_FLAG_TICK | HK_FLAG_TIMER |
+ HK_FLAG_RCU | HK_FLAG_MISC;
+
+ tick_nohz_full_setup(non_housekeeping_mask);
+
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+
+ return 1;
+}
+__setup("nohz_full=", housekeeping_nohz_full_setup);
+#endif
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 27d7d52..69f3dbe 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -385,20 +385,13 @@ void __tick_nohz_task_switch(void)
local_irq_restore(flags);
}
-/* Parse the boot-time nohz CPU list from the kernel parameters. */
-static int __init tick_nohz_full_setup(char *str)
+/* Get the boot-time nohz CPU list from the kernel parameters. */
+void __init tick_nohz_full_setup(cpumask_var_t cpumask)
{
alloc_bootmem_cpumask_var(&tick_nohz_full_mask);
- if (cpulist_parse(str, tick_nohz_full_mask) < 0) {
- pr_warn("NO_HZ: Incorrect nohz_full cpumask\n");
- free_bootmem_cpumask_var(tick_nohz_full_mask);
- return 1;
- }
+ cpumask_copy(tick_nohz_full_mask, cpumask);
tick_nohz_full_running = true;
-
- return 1;
}
-__setup("nohz_full=", tick_nohz_full_setup);
static int tick_nohz_cpu_down(unsigned int cpu)
{
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 10/12] housekeeping: Move isolcpus to housekeeping
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (8 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 09/12] housekeeping: Handle the nohz_full= parameter Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: Move isolcpus= handling to the housekeeping code tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 11/12] housekeeping: Add basic isolcpus flags Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 12/12] housekeeping: Document " Frederic Weisbecker
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
We want to centralize the isolation features on the housekeeping
subsystem and scheduler domain isolation is a significant part of it.
No intended behaviour change, we just reuse the housekeeping cpumask
and core code.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
drivers/base/cpu.c | 11 +++++++-
include/linux/sched.h | 2 --
include/linux/sched/isolation.h | 1 +
kernel/cgroup/cpuset.c | 15 ++++------
kernel/sched/core.c | 16 +----------
kernel/sched/isolation.c | 61 +++++++++++++++++++++++++++++++++--------
kernel/sched/topology.c | 24 ++++------------
7 files changed, 72 insertions(+), 58 deletions(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 321cd7b..a73ab95 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -18,6 +18,7 @@
#include <linux/cpufeature.h>
#include <linux/tick.h>
#include <linux/pm_qos.h>
+#include <linux/sched/isolation.h>
#include "base.h"
@@ -271,8 +272,16 @@ static ssize_t print_cpus_isolated(struct device *dev,
struct device_attribute *attr, char *buf)
{
int n = 0, len = PAGE_SIZE-2;
+ cpumask_var_t isolated;
- n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(cpu_isolated_map));
+ if (!alloc_cpumask_var(&isolated, GFP_KERNEL))
+ return -ENOMEM;
+
+ cpumask_andnot(isolated, cpu_possible_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
+ n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(isolated));
+
+ free_cpumask_var(isolated);
return n;
}
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 0f897df..1b0cc0d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -165,8 +165,6 @@ struct task_group;
/* Task command name length: */
#define TASK_COMM_LEN 16
-extern cpumask_var_t cpu_isolated_map;
-
extern void scheduler_tick(void);
#define MAX_SCHEDULE_TIMEOUT LONG_MAX
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index e53cfa9..d849431 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -11,6 +11,7 @@ enum hk_flags {
HK_FLAG_MISC = (1 << 2),
HK_FLAG_SCHED = (1 << 3),
HK_FLAG_TICK = (1 << 4),
+ HK_FLAG_DOMAIN = (1 << 5),
};
#ifdef CONFIG_CPU_ISOLATION
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 4657e29..f7efa7b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -57,7 +57,7 @@
#include <linux/backing-dev.h>
#include <linux/sort.h>
#include <linux/oom.h>
-
+#include <linux/sched/isolation.h>
#include <linux/uaccess.h>
#include <linux/atomic.h>
#include <linux/mutex.h>
@@ -656,7 +656,6 @@ static int generate_sched_domains(cpumask_var_t **domains,
int csn; /* how many cpuset ptrs in csa so far */
int i, j, k; /* indices for partition finding loops */
cpumask_var_t *doms; /* resulting partition; i.e. sched domains */
- cpumask_var_t non_isolated_cpus; /* load balanced CPUs */
struct sched_domain_attr *dattr; /* attributes for custom domains */
int ndoms = 0; /* number of sched domains in result */
int nslot; /* next empty doms[] struct cpumask slot */
@@ -666,10 +665,6 @@ static int generate_sched_domains(cpumask_var_t **domains,
dattr = NULL;
csa = NULL;
- if (!alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL))
- goto done;
- cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
-
/* Special case for the 99% of systems with one, full, sched domain */
if (is_sched_load_balance(&top_cpuset)) {
ndoms = 1;
@@ -683,7 +678,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
update_domain_attr_tree(dattr, &top_cpuset);
}
cpumask_and(doms[0], top_cpuset.effective_cpus,
- non_isolated_cpus);
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
goto done;
}
@@ -707,7 +702,8 @@ static int generate_sched_domains(cpumask_var_t **domains,
*/
if (!cpumask_empty(cp->cpus_allowed) &&
!(is_sched_load_balance(cp) &&
- cpumask_intersects(cp->cpus_allowed, non_isolated_cpus)))
+ cpumask_intersects(cp->cpus_allowed,
+ housekeeping_cpumask(HK_FLAG_DOMAIN))))
continue;
if (is_sched_load_balance(cp))
@@ -789,7 +785,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
if (apn == b->pn) {
cpumask_or(dp, dp, b->effective_cpus);
- cpumask_and(dp, dp, non_isolated_cpus);
+ cpumask_and(dp, dp, housekeeping_cpumask(HK_FLAG_DOMAIN));
if (dattr)
update_domain_attr_tree(dattr + nslot, b);
@@ -802,7 +798,6 @@ static int generate_sched_domains(cpumask_var_t **domains,
BUG_ON(nslot != ndoms);
done:
- free_cpumask_var(non_isolated_cpus);
kfree(csa);
/*
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2210c02..1a55c84 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -84,9 +84,6 @@ __read_mostly int scheduler_running;
*/
int sysctl_sched_rt_runtime = 950000;
-/* CPUs with isolated domains */
-cpumask_var_t cpu_isolated_map;
-
/*
* __task_rq_lock - lock the rq @p resides on.
*/
@@ -5735,10 +5732,6 @@ static inline void sched_init_smt(void) { }
void __init sched_init_smp(void)
{
- cpumask_var_t non_isolated_cpus;
-
- alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL);
-
sched_init_numa();
/*
@@ -5748,16 +5741,12 @@ void __init sched_init_smp(void)
*/
mutex_lock(&sched_domains_mutex);
sched_init_domains(cpu_active_mask);
- cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
- if (cpumask_empty(non_isolated_cpus))
- cpumask_set_cpu(smp_processor_id(), non_isolated_cpus);
mutex_unlock(&sched_domains_mutex);
/* Move init over to a non-isolated CPU */
- if (set_cpus_allowed_ptr(current, non_isolated_cpus) < 0)
+ if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) < 0)
BUG();
sched_init_granularity();
- free_cpumask_var(non_isolated_cpus);
init_sched_rt_class();
init_sched_dl_class();
@@ -5961,9 +5950,6 @@ void __init sched_init(void)
calc_load_update = jiffies + LOAD_FREQ;
#ifdef CONFIG_SMP
- /* May be allocated at isolcpus cmdline parse time */
- if (cpu_isolated_map == NULL)
- zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
idle_thread_set_boot_cpu();
set_cpu_rq_start_time(smp_processor_id());
#endif
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 1f61e44..8f666bc 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -63,32 +63,69 @@ void __init housekeeping_init(void)
WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
-#ifdef CONFIG_NO_HZ_FULL
-static int __init housekeeping_nohz_full_setup(char *str)
+static int __init housekeeping_setup(char *str, enum hk_flags flags)
{
cpumask_var_t non_housekeeping_mask;
+ int err;
alloc_bootmem_cpumask_var(&non_housekeeping_mask);
- if (cpulist_parse(str, non_housekeeping_mask) < 0) {
- pr_warn("Housekeeping: Incorrect nohz_full cpumask\n");
+ err = cpulist_parse(str, non_housekeeping_mask);
+ if (err < 0 || cpumask_last(non_housekeeping_mask) >= nr_cpu_ids) {
+ pr_warn("Housekeeping: nohz_full= or isolcpus= incorrect CPU range\n");
free_bootmem_cpumask_var(non_housekeeping_mask);
return 0;
}
- alloc_bootmem_cpumask_var(&housekeeping_mask);
- cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask);
+ if (!housekeeping_flags) {
+ alloc_bootmem_cpumask_var(&housekeeping_mask);
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, non_housekeeping_mask);
+ if (cpumask_empty(housekeeping_mask))
+ cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+ } else {
+ cpumask_var_t tmp;
- if (cpumask_empty(housekeeping_mask))
- cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+ alloc_bootmem_cpumask_var(&tmp);
+ cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
+ if (!cpumask_equal(tmp, housekeeping_mask)) {
+ pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
+ free_bootmem_cpumask_var(tmp);
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
+ }
+ free_bootmem_cpumask_var(tmp);
+ }
- housekeeping_flags = HK_FLAG_TICK | HK_FLAG_TIMER |
- HK_FLAG_RCU | HK_FLAG_MISC;
+ if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
+ if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
+ tick_nohz_full_setup(non_housekeeping_mask);
+ } else {
+ pr_warn("Housekeeping: nohz unsupported."
+ " Build with CONFIG_NO_HZ_FULL\n");
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
+ }
+ }
- tick_nohz_full_setup(non_housekeeping_mask);
+ housekeeping_flags |= flags;
free_bootmem_cpumask_var(non_housekeeping_mask);
return 1;
}
+
+static int __init housekeeping_nohz_full_setup(char *str)
+{
+ unsigned int flags;
+
+ flags = HK_FLAG_TICK | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+
+ return housekeeping_setup(str, flags);
+}
__setup("nohz_full=", housekeeping_nohz_full_setup);
-#endif
+
+static int __init housekeeping_isolcpus_setup(char *str)
+{
+ return housekeeping_setup(str, HK_FLAG_DOMAIN);
+}
+__setup("isolcpus=", housekeeping_isolcpus_setup);
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index e3d31b0..2e6b912 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -3,6 +3,7 @@
*/
#include <linux/sched.h>
#include <linux/mutex.h>
+#include <linux/sched/isolation.h>
#include "sched.h"
@@ -469,21 +470,6 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu)
update_top_cache_domain(cpu);
}
-/* Setup the mask of CPUs configured for isolated domains */
-static int __init isolated_cpu_setup(char *str)
-{
- int ret;
-
- alloc_bootmem_cpumask_var(&cpu_isolated_map);
- ret = cpulist_parse(str, cpu_isolated_map);
- if (ret || cpumask_last(cpu_isolated_map) >= nr_cpu_ids) {
- pr_err("sched: Error, all isolcpus= values must be between 0 and %u - ignoring them.\n", nr_cpu_ids-1);
- return 0;
- }
- return 1;
-}
-__setup("isolcpus=", isolated_cpu_setup);
-
struct s_data {
struct sched_domain ** __percpu sd;
struct root_domain *rd;
@@ -1792,7 +1778,7 @@ int sched_init_domains(const struct cpumask *cpu_map)
doms_cur = alloc_sched_domains(ndoms_cur);
if (!doms_cur)
doms_cur = &fallback_doms;
- cpumask_andnot(doms_cur[0], cpu_map, cpu_isolated_map);
+ cpumask_and(doms_cur[0], cpu_map, housekeeping_cpumask(HK_FLAG_DOMAIN));
err = build_sched_domains(doms_cur[0], NULL);
register_sched_domain_sysctl();
@@ -1875,7 +1861,8 @@ void partition_sched_domains(int ndoms_new, cpumask_var_t doms_new[],
doms_new = alloc_sched_domains(1);
if (doms_new) {
n = 1;
- cpumask_andnot(doms_new[0], cpu_active_mask, cpu_isolated_map);
+ cpumask_and(doms_new[0], cpu_active_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
}
} else {
n = ndoms_new;
@@ -1898,7 +1885,8 @@ void partition_sched_domains(int ndoms_new, cpumask_var_t doms_new[],
if (!doms_new) {
n = 0;
doms_new = &fallback_doms;
- cpumask_andnot(doms_new[0], cpu_active_mask, cpu_isolated_map);
+ cpumask_and(doms_new[0], cpu_active_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
}
/* Build new domains: */
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 11/12] housekeeping: Add basic isolcpus flags
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (9 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 10/12] housekeeping: Move isolcpus to housekeeping Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 12/12] housekeeping: Document " Frederic Weisbecker
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Add flags to control nohz and domain isolations from "isolcpus=", in
order to centralize the isolation features to a common interface. Domain
isolation remains the default so not to break the existing isolcpus
boot paramater behaviour.
Further flags in the future may include 0hz (1hz tick offload) and timers,
workqueue, RCU, kthread, watchdog, likely all merged together in a
common flag ("async"?). In any case, this will have to be modifiable by
cpusets.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
kernel/sched/isolation.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 8f666bc..b71b436 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -11,6 +11,7 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/static_key.h>
+#include <linux/ctype.h>
DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
EXPORT_SYMBOL_GPL(housekeeping_overriden);
@@ -126,6 +127,29 @@ __setup("nohz_full=", housekeeping_nohz_full_setup);
static int __init housekeeping_isolcpus_setup(char *str)
{
- return housekeeping_setup(str, HK_FLAG_DOMAIN);
+ unsigned int flags = 0;
+
+ while (isalpha(*str)) {
+ if (!strncmp(str, "nohz,", 5)) {
+ str += 5;
+ flags |= HK_FLAG_TICK;
+ continue;
+ }
+
+ if (!strncmp(str, "domain,", 7)) {
+ str += 7;
+ flags |= HK_FLAG_DOMAIN;
+ continue;
+ }
+
+ pr_warn("isolcpus: Error, unknown flag\n");
+ return 0;
+ }
+
+ /* Default behaviour for isolcpus without flags */
+ if (!flags)
+ flags |= HK_FLAG_DOMAIN;
+
+ return housekeeping_setup(str, flags);
}
__setup("isolcpus=", housekeeping_isolcpus_setup);
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 12/12] housekeeping: Document isolcpus flags
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
` (10 preceding siblings ...)
2017-10-27 2:42 ` [PATCH 11/12] housekeeping: Add basic isolcpus flags Frederic Weisbecker
@ 2017-10-27 2:42 ` Frederic Weisbecker
2017-10-27 12:06 ` [tip:sched/core] sched/isolation: Document the isolcpus= flags tip-bot for Frederic Weisbecker
11 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 2:42 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Linus Torvalds,
Chris Metcalf, Thomas Gleixner, Luiz Capitulino,
Christoph Lameter, Paul E . McKenney, Mike Galbraith,
Rik van Riel, Wanpeng Li
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 35 +++++++++++++++----------
1 file changed, 21 insertions(+), 14 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 6b99c8b..5853618 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1727,20 +1727,27 @@
isapnp= [ISAPNP]
Format: <RDP>,<reset>,<pci_scan>,<verbosity>
- isolcpus= [KNL,SMP] Isolate CPUs from the general scheduler.
- The argument is a cpu list, as described above.
-
- This option can be used to specify one or more CPUs
- to isolate from the general SMP balancing and scheduling
- algorithms. You can move a process onto or off an
- "isolated" CPU via the CPU affinity syscalls or cpuset.
- <cpu number> begins at 0 and the maximum value is
- "number of CPUs in system - 1".
-
- This option is the preferred way to isolate CPUs. The
- alternative -- manually setting the CPU mask of all
- tasks in the system -- can cause problems and
- suboptimal load balancer performance.
+ isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
+ Format: [flag-list,]<cpu-list>
+
+ Specify one or more CPUs to isolate from disturbances
+ specified in the flag list (default: domain):
+
+ nohz
+ Disable the tick when a single task runs.
+ domain
+ Isolate from the general SMP balancing and scheduling
+ algorithms. This option is the preferred way to isolate
+ CPUs from tasks. The alternative -- manually setting the
+ CPU mask of all tasks in the system, can cause problems
+ and suboptimal load balancer performance. You can move a
+ process onto or off an "isolated" CPU via the CPU
+ affinity syscalls or cpuset. <cpu number> begins at 0
+ and the maximum value is "number of CPUs in system - 1".
+
+ The format of <cpu-list> is described above.
+
+
iucv= [HW,NET]
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Move housekeeping related code to its own file
2017-10-27 2:42 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
@ 2017-10-27 12:01 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:01 UTC (permalink / raw)
To: linux-tip-commits
Cc: torvalds, peterz, linux-kernel, efault, kernellwp, paulmck, hpa,
tglx, riel, mingo, cl, cmetcalf, lcapitulino, frederic
Commit-ID: 7863406143d8bbbbda07a61285c5f4c217908dfd
Gitweb: https://git.kernel.org/tip/7863406143d8bbbbda07a61285c5f4c217908dfd
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:28 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:24 +0200
sched/isolation: Move housekeeping related code to its own file
The housekeeping code is currently tied to the NOHZ code. As we are
planning to make housekeeping independent from it, start with moving
the relevant code to its own file.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-2-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
drivers/net/ethernet/tile/tilegx.c | 2 +-
include/linux/sched/isolation.h | 56 ++++++++++++++++++++++++++++++++++++++
include/linux/tick.h | 37 -------------------------
init/main.c | 2 ++
kernel/rcu/tree_plugin.h | 1 +
kernel/rcu/update.c | 1 +
kernel/sched/Makefile | 1 +
kernel/sched/core.c | 1 +
kernel/sched/fair.c | 1 +
kernel/sched/isolation.c | 33 ++++++++++++++++++++++
kernel/time/tick-sched.c | 18 ------------
kernel/watchdog.c | 1 +
12 files changed, 98 insertions(+), 56 deletions(-)
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index c00102b..27a3272 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -40,7 +40,7 @@
#include <linux/tcp.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
-#include <linux/tick.h>
+#include <linux/sched/isolation.h>
#include <asm/checksum.h>
#include <asm/homecache.h>
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
new file mode 100644
index 0000000..b7cfbc4
--- /dev/null
+++ b/include/linux/sched/isolation.h
@@ -0,0 +1,56 @@
+#ifndef _LINUX_SCHED_ISOLATION_H
+#define _LINUX_SCHED_ISOLATION_H
+
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/tick.h>
+
+#ifdef CONFIG_NO_HZ_FULL
+extern cpumask_var_t housekeeping_mask;
+
+static inline int housekeeping_any_cpu(void)
+{
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+}
+
+extern void __init housekeeping_init(void);
+
+#else
+
+static inline int housekeeping_any_cpu(void)
+{
+ return smp_processor_id();
+}
+
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
+
+static inline const struct cpumask *housekeeping_cpumask(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+#endif
+ return cpu_possible_mask;
+}
+
+static inline bool is_housekeeping_cpu(int cpu)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+#endif
+ return true;
+}
+
+static inline void housekeeping_affine(struct task_struct *t)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+
+#endif
+}
+
+#endif /* _LINUX_SCHED_ISOLATION_H */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index fe01e68..68afc09 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
extern cpumask_var_t tick_nohz_full_mask;
-extern cpumask_var_t housekeeping_mask;
static inline bool tick_nohz_full_enabled(void)
{
@@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
cpumask_or(mask, mask, tick_nohz_full_mask);
}
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void tick_nohz_dep_set(enum tick_dep_bits bit);
extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
@@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
#else
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
@@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
#endif
-static inline const struct cpumask *housekeeping_cpumask(void)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
- return cpu_possible_mask;
-}
-
-static inline bool is_housekeeping_cpu(int cpu)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
-#endif
- return true;
-}
-
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
static inline void tick_nohz_task_switch(void)
{
if (tick_nohz_full_enabled())
diff --git a/init/main.c b/init/main.c
index 0ee9c686..4610c99 100644
--- a/init/main.c
+++ b/init/main.c
@@ -46,6 +46,7 @@
#include <linux/cgroup.h>
#include <linux/efi.h>
#include <linux/tick.h>
+#include <linux/sched/isolation.h>
#include <linux/interrupt.h>
#include <linux/taskstats_kern.h>
#include <linux/delayacct.h>
@@ -606,6 +607,7 @@ asmlinkage __visible void __init start_kernel(void)
early_irq_init();
init_IRQ();
tick_init();
+ housekeeping_init();
rcu_init_nohz();
init_timers();
hrtimers_init();
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e012b9b..c7632f5 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -29,6 +29,7 @@
#include <linux/oom.h>
#include <linux/sched/debug.h>
#include <linux/smpboot.h>
+#include <linux/sched/isolation.h>
#include <uapi/linux/sched/types.h>
#include "../time/tick-internal.h"
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 5033b66..79abeb0 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -51,6 +51,7 @@
#include <linux/kthread.h>
#include <linux/tick.h>
#include <linux/rcupdate_wait.h>
+#include <linux/sched/isolation.h>
#define CREATE_TRACE_POINTS
diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile
index 78f5493..871d43d 100644
--- a/kernel/sched/Makefile
+++ b/kernel/sched/Makefile
@@ -26,3 +26,4 @@ obj-$(CONFIG_CGROUP_CPUACCT) += cpuacct.o
obj-$(CONFIG_CPU_FREQ) += cpufreq.o
obj-$(CONFIG_CPU_FREQ_GOV_SCHEDUTIL) += cpufreq_schedutil.o
obj-$(CONFIG_MEMBARRIER) += membarrier.o
+obj-$(CONFIG_NO_HZ_FULL) += isolation.o
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2288a14..ad188ac 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -26,6 +26,7 @@
#include <linux/profile.h>
#include <linux/security.h>
#include <linux/syscalls.h>
+#include <linux/sched/isolation.h>
#include <asm/switch_to.h>
#include <asm/tlb.h>
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 56f343b..591481d 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -32,6 +32,7 @@
#include <linux/mempolicy.h>
#include <linux/migrate.h>
#include <linux/task_work.h>
+#include <linux/sched/isolation.h>
#include <trace/events/sched.h>
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
new file mode 100644
index 0000000..3589252
--- /dev/null
+++ b/kernel/sched/isolation.c
@@ -0,0 +1,33 @@
+/*
+ * Housekeeping management. Manage the targets for routine code that can run on
+ * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
+ *
+ * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ *
+ */
+
+#include <linux/sched/isolation.h>
+#include <linux/tick.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+
+cpumask_var_t housekeeping_mask;
+
+void __init housekeeping_init(void)
+{
+ if (!tick_nohz_full_enabled())
+ return;
+
+ if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
+ WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
+ cpumask_clear(tick_nohz_full_mask);
+ tick_nohz_full_running = false;
+ return;
+ }
+
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, tick_nohz_full_mask);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 7b258c5..27d7d52 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -166,7 +166,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
#ifdef CONFIG_NO_HZ_FULL
cpumask_var_t tick_nohz_full_mask;
-cpumask_var_t housekeeping_mask;
bool tick_nohz_full_running;
static atomic_t tick_dep_mask;
@@ -438,13 +437,6 @@ void __init tick_nohz_init(void)
return;
}
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
/*
* Full dynticks uses irq work to drive the tick rescheduling on safe
* locking contexts. But then we need irq work to raise its own
@@ -453,7 +445,6 @@ void __init tick_nohz_init(void)
if (!arch_irq_work_has_interrupt()) {
pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
cpumask_clear(tick_nohz_full_mask);
- cpumask_copy(housekeeping_mask, cpu_possible_mask);
tick_nohz_full_running = false;
return;
}
@@ -466,9 +457,6 @@ void __init tick_nohz_init(void)
cpumask_clear_cpu(cpu, tick_nohz_full_mask);
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
for_each_cpu(cpu, tick_nohz_full_mask)
context_tracking_cpu_set(cpu);
@@ -478,12 +466,6 @@ void __init tick_nohz_init(void)
WARN_ON(ret < 0);
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
-
- /*
- * We need at least one CPU to handle housekeeping work such
- * as timekeeping, unbound timers, workqueues, ...
- */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
#endif
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 6bcb854..3c44dba 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -24,6 +24,7 @@
#include <linux/workqueue.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
+#include <linux/sched/isolation.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation, watchdog: Use housekeeping_cpumask() instead of ad-hoc version
2017-10-27 2:42 ` [PATCH 02/12] watchdog: Use housekeeping_cpumask() instead of ad-hoc version Frederic Weisbecker
@ 2017-10-27 12:02 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: hpa, paulmck, kernellwp, cl, linux-kernel, efault, torvalds,
riel, lcapitulino, mingo, cmetcalf, peterz, tglx, frederic
Commit-ID: 13316b31fdaaa45f06793eb7992588359ba6ab9f
Gitweb: https://git.kernel.org/tip/13316b31fdaaa45f06793eb7992588359ba6ab9f
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:29 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:25 +0200
sched/isolation, watchdog: Use housekeeping_cpumask() instead of ad-hoc version
While trying to disable the watchog on nohz_full CPUs, the watchdog
implements an ad-hoc version of housekeeping_cpumask(). Lets replace
those re-invented lines.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-3-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
kernel/watchdog.c | 11 +++--------
1 file changed, 3 insertions(+), 8 deletions(-)
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 3c44dba..562652c 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -774,15 +774,10 @@ int proc_watchdog_cpumask(struct ctl_table *table, int write,
void __init lockup_detector_init(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled()) {
+ if (tick_nohz_full_enabled())
pr_info("Disabling watchdog on nohz_full cores by default\n");
- cpumask_copy(&watchdog_cpumask, housekeeping_mask);
- } else
- cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
-#else
- cpumask_copy(&watchdog_cpumask, cpu_possible_mask);
-#endif
+
+ cpumask_copy(&watchdog_cpumask, housekeeping_cpumask());
if (!watchdog_nmi_probe())
nmi_watchdog_available = true;
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Provide a dynamic off-case to housekeeping_any_cpu()
2017-10-27 2:42 ` [PATCH 03/12] housekeeping: Provide a dynamic off-case to housekeeping_any_cpu() Frederic Weisbecker
@ 2017-10-27 12:02 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:02 UTC (permalink / raw)
To: linux-tip-commits
Cc: paulmck, peterz, kernellwp, cmetcalf, riel, tglx, torvalds, cl,
efault, linux-kernel, mingo, frederic, lcapitulino, hpa
Commit-ID: 9f0ca2d97ef0b5e966be2cfef26c7c094ec14e41
Gitweb: https://git.kernel.org/tip/9f0ca2d97ef0b5e966be2cfef26c7c094ec14e41
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:30 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:26 +0200
sched/isolation: Provide a dynamic off-case to housekeeping_any_cpu()
housekeeping_any_cpu() doesn't handle correctly the case where
CONFIG_NO_HZ_FULL=y and no CPU is in nohz_full mode. So far no caller
needs this but let's prepare to avoid any future surprise.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-4-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 17 ++++++-----------
1 file changed, 6 insertions(+), 11 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index b7cfbc4..040df04 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -7,25 +7,20 @@
#ifdef CONFIG_NO_HZ_FULL
extern cpumask_var_t housekeeping_mask;
-
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void __init housekeeping_init(void);
-
#else
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
static inline int housekeeping_any_cpu(void)
{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+#endif
return smp_processor_id();
}
-static inline void housekeeping_init(void) { }
-#endif /* CONFIG_NO_HZ_FULL */
-
-
static inline const struct cpumask *housekeeping_cpumask(void)
{
#ifdef CONFIG_NO_HZ_FULL
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Make the housekeeping cpumask private
2017-10-27 2:42 ` [PATCH 04/12] housekeeping: Make housekeeping cpumask private Frederic Weisbecker
@ 2017-10-27 12:03 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:03 UTC (permalink / raw)
To: linux-tip-commits
Cc: kernellwp, torvalds, cmetcalf, cl, mingo, riel, efault,
linux-kernel, hpa, tglx, paulmck, frederic, lcapitulino, peterz
Commit-ID: 7e56a1cf4b28f5739526877b8dbad623fae2e4e7
Gitweb: https://git.kernel.org/tip/7e56a1cf4b28f5739526877b8dbad623fae2e4e7
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:31 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:26 +0200
sched/isolation: Make the housekeeping cpumask private
Nobody needs to access this detail. housekeeping_cpumask() already
takes care of it.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-5-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 31 ++++++++++---------------------
kernel/sched/isolation.c | 36 +++++++++++++++++++++++++++++++++++-
2 files changed, 45 insertions(+), 22 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 040df04..ed935ff 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -6,46 +6,35 @@
#include <linux/tick.h>
#ifdef CONFIG_NO_HZ_FULL
-extern cpumask_var_t housekeeping_mask;
+extern int housekeeping_any_cpu(void);
+extern const struct cpumask *housekeeping_cpumask(void);
+extern void housekeeping_affine(struct task_struct *t);
+extern bool housekeeping_test_cpu(int cpu);
extern void __init housekeeping_init(void);
+
#else
-static inline void housekeeping_init(void) { }
-#endif /* CONFIG_NO_HZ_FULL */
static inline int housekeeping_any_cpu(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-#endif
return smp_processor_id();
}
static inline const struct cpumask *housekeeping_cpumask(void)
{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
return cpu_possible_mask;
}
+static inline void housekeeping_affine(struct task_struct *t) { }
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
static inline bool is_housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
+ return housekeeping_test_cpu(cpu);
#endif
return true;
}
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
#endif /* _LINUX_SCHED_ISOLATION_H */
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 3589252..1644509 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -11,7 +11,41 @@
#include <linux/init.h>
#include <linux/kernel.h>
-cpumask_var_t housekeeping_mask;
+static cpumask_var_t housekeeping_mask;
+
+int housekeeping_any_cpu(void)
+{
+ if (tick_nohz_full_enabled())
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+
+ return smp_processor_id();
+}
+EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
+
+const struct cpumask *housekeeping_cpumask(void)
+{
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+
+ return cpu_possible_mask;
+}
+EXPORT_SYMBOL_GPL(housekeeping_cpumask);
+
+void housekeeping_affine(struct task_struct *t)
+{
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+}
+EXPORT_SYMBOL_GPL(housekeeping_affine);
+
+bool housekeeping_test_cpu(int cpu)
+{
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+
+ return true;
+}
+EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
void __init housekeeping_init(void)
{
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Use its own static key
2017-10-27 2:42 ` [PATCH 05/12] housekeeping: Use its own static key Frederic Weisbecker
@ 2017-10-27 12:03 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:03 UTC (permalink / raw)
To: linux-tip-commits
Cc: kernellwp, cl, peterz, frederic, cmetcalf, tglx, torvalds, riel,
lcapitulino, hpa, mingo, paulmck, efault, linux-kernel
Commit-ID: e179f5a04ba46ee5c5439480c2bfd68c358168b7
Gitweb: https://git.kernel.org/tip/e179f5a04ba46ee5c5439480c2bfd68c358168b7
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:32 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:27 +0200
sched/isolation: Use its own static key
Housekeeping code still depends on the nohz_full static key. Since we want
to decouple housekeeping from NOHZ, let's create a housekeeping specific
static key.
It's mostly relevant for calls to is_housekeeping_cpu() from the scheduler.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-6-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 3 ++-
kernel/sched/isolation.c | 13 +++++++++----
2 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index ed935ff..194c586 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -6,6 +6,7 @@
#include <linux/tick.h>
#ifdef CONFIG_NO_HZ_FULL
+DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
extern int housekeeping_any_cpu(void);
extern const struct cpumask *housekeeping_cpumask(void);
extern void housekeeping_affine(struct task_struct *t);
@@ -31,7 +32,7 @@ static inline void housekeeping_init(void) { }
static inline bool is_housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_test_cpu(cpu);
#endif
return true;
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 1644509..bb8ba19 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -10,12 +10,15 @@
#include <linux/tick.h>
#include <linux/init.h>
#include <linux/kernel.h>
+#include <linux/static_key.h>
+DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
+EXPORT_SYMBOL_GPL(housekeeping_overriden);
static cpumask_var_t housekeeping_mask;
int housekeeping_any_cpu(void)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return cpumask_any_and(housekeeping_mask, cpu_online_mask);
return smp_processor_id();
@@ -24,7 +27,7 @@ EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
const struct cpumask *housekeeping_cpumask(void)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_mask;
return cpu_possible_mask;
@@ -33,14 +36,14 @@ EXPORT_SYMBOL_GPL(housekeeping_cpumask);
void housekeeping_affine(struct task_struct *t)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
set_cpus_allowed_ptr(t, housekeeping_mask);
}
EXPORT_SYMBOL_GPL(housekeeping_affine);
bool housekeeping_test_cpu(int cpu)
{
- if (tick_nohz_full_enabled())
+ if (static_branch_unlikely(&housekeeping_overriden))
return cpumask_test_cpu(cpu, housekeeping_mask);
return true;
@@ -62,6 +65,8 @@ void __init housekeeping_init(void)
cpumask_andnot(housekeeping_mask,
cpu_possible_mask, tick_nohz_full_mask);
+ static_branch_enable(&housekeeping_overriden);
+
/* We need at least one CPU to handle housekeeping work */
WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Rename is_housekeeping_cpu() to housekeeping_cpu()
2017-10-27 2:42 ` [PATCH 06/12] housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu Frederic Weisbecker
@ 2017-10-27 12:03 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:03 UTC (permalink / raw)
To: linux-tip-commits
Cc: tglx, linux-kernel, cl, frederic, hpa, paulmck, efault, mingo,
kernellwp, riel, lcapitulino, peterz, cmetcalf, torvalds
Commit-ID: 204c083a009378dfa751175b5fcddc75988bab6c
Gitweb: https://git.kernel.org/tip/204c083a009378dfa751175b5fcddc75988bab6c
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:33 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:28 +0200
sched/isolation: Rename is_housekeeping_cpu() to housekeeping_cpu()
Fit it into the housekeeping_*() namespace.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-7-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 2 +-
kernel/sched/core.c | 6 +++---
kernel/sched/fair.c | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 194c586..ad0f5d9 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -29,7 +29,7 @@ static inline void housekeeping_affine(struct task_struct *t) { }
static inline void housekeeping_init(void) { }
#endif /* CONFIG_NO_HZ_FULL */
-static inline bool is_housekeeping_cpu(int cpu)
+static inline bool housekeeping_cpu(int cpu)
{
#ifdef CONFIG_NO_HZ_FULL
if (static_branch_unlikely(&housekeeping_overriden))
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index ad188ac..d0fb448d 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -527,7 +527,7 @@ int get_nohz_timer_target(void)
int i, cpu = smp_processor_id();
struct sched_domain *sd;
- if (!idle_cpu(cpu) && is_housekeeping_cpu(cpu))
+ if (!idle_cpu(cpu) && housekeeping_cpu(cpu))
return cpu;
rcu_read_lock();
@@ -536,14 +536,14 @@ int get_nohz_timer_target(void)
if (cpu == i)
continue;
- if (!idle_cpu(i) && is_housekeeping_cpu(i)) {
+ if (!idle_cpu(i) && housekeeping_cpu(i)) {
cpu = i;
goto unlock;
}
}
}
- if (!is_housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu))
cpu = housekeeping_any_cpu();
unlock:
rcu_read_unlock();
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 591481d..cdece8f 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9027,7 +9027,7 @@ void nohz_balance_enter_idle(int cpu)
return;
/* Spare idle load balancing on CPUs that don't want to be disturbed: */
- if (!is_housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu))
return;
if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Split out new CONFIG_CPU_ISOLATION=y config from CONFIG_NO_HZ_FULL
2017-10-27 2:42 ` [PATCH 07/12] housekeeping: Move it under its own config, independent from NO_HZ Frederic Weisbecker
@ 2017-10-27 12:04 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:04 UTC (permalink / raw)
To: linux-tip-commits
Cc: lcapitulino, cl, peterz, riel, linux-kernel, kernellwp, mingo,
torvalds, tglx, hpa, paulmck, efault, cmetcalf, frederic
Commit-ID: 5c4991e24c69737bd41fc2737b1e3980abbf73f9
Gitweb: https://git.kernel.org/tip/5c4991e24c69737bd41fc2737b1e3980abbf73f9
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:34 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:28 +0200
sched/isolation: Split out new CONFIG_CPU_ISOLATION=y config from CONFIG_NO_HZ_FULL
Split the housekeeping config from CONFIG_NO_HZ_FULL. This way we finally
separate the isolation code from NOHZ.
Although a dependency to CONFIG_NO_HZ_FULL remains for now, while the
housekeeping code still deals with NOHZ internals.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-8-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 6 +++---
init/Kconfig | 8 ++++++++
kernel/sched/Makefile | 2 +-
3 files changed, 12 insertions(+), 4 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index ad0f5d9..93ac2367 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -5,7 +5,7 @@
#include <linux/init.h>
#include <linux/tick.h>
-#ifdef CONFIG_NO_HZ_FULL
+#ifdef CONFIG_CPU_ISOLATION
DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
extern int housekeeping_any_cpu(void);
extern const struct cpumask *housekeeping_cpumask(void);
@@ -27,11 +27,11 @@ static inline const struct cpumask *housekeeping_cpumask(void)
static inline void housekeeping_affine(struct task_struct *t) { }
static inline void housekeeping_init(void) { }
-#endif /* CONFIG_NO_HZ_FULL */
+#endif /* CONFIG_CPU_ISOLATION */
static inline bool housekeeping_cpu(int cpu)
{
-#ifdef CONFIG_NO_HZ_FULL
+#ifdef CONFIG_CPU_ISOLATION
if (static_branch_unlikely(&housekeeping_overriden))
return housekeeping_test_cpu(cpu);
#endif
diff --git a/init/Kconfig b/init/Kconfig
index 78cb246..6f52e6f 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -472,6 +472,14 @@ config TASK_IO_ACCOUNTING
endmenu # "CPU/Task time and stats accounting"
+config CPU_ISOLATION
+ bool "CPU isolation"
+ depends on NO_HZ_FULL
+ help
+ Make sure that CPUs running critical tasks are not disturbed by
+ any source of "noise" such as unbound workqueues, timers, kthreads...
+ Unbound jobs get offloaded to housekeeping CPUs.
+
source "kernel/rcu/Kconfig"
config BUILD_BIN2C
diff --git a/kernel/sched/Makefile b/kernel/sched/Makefile
index 871d43d..dbe6130 100644
--- a/kernel/sched/Makefile
+++ b/kernel/sched/Makefile
@@ -26,4 +26,4 @@ obj-$(CONFIG_CGROUP_CPUACCT) += cpuacct.o
obj-$(CONFIG_CPU_FREQ) += cpufreq.o
obj-$(CONFIG_CPU_FREQ_GOV_SCHEDUTIL) += cpufreq_schedutil.o
obj-$(CONFIG_MEMBARRIER) += membarrier.o
-obj-$(CONFIG_NO_HZ_FULL) += isolation.o
+obj-$(CONFIG_CPU_ISOLATION) += isolation.o
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Introduce housekeeping flags
2017-10-27 2:42 ` [PATCH 08/12] housekeeping: Introduce housekeeping flags Frederic Weisbecker
@ 2017-10-27 12:04 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:04 UTC (permalink / raw)
To: linux-tip-commits
Cc: riel, lcapitulino, linux-kernel, mingo, cl, hpa, kernellwp,
torvalds, peterz, efault, tglx, paulmck, cmetcalf, frederic
Commit-ID: de201559df872f83d0c08fb4effe3efd28e6cbc8
Gitweb: https://git.kernel.org/tip/de201559df872f83d0c08fb4effe3efd28e6cbc8
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:35 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:29 +0200
sched/isolation: Introduce housekeeping flags
Before we implement isolcpus under housekeeping, we need the isolation
features to be more finegrained. For example some people want NOHZ_FULL
without the full scheduler isolation, others want full scheduler
isolation without NOHZ_FULL.
So let's cut all these isolation features piecewise, at the risk of
overcutting it right now. We can still merge some flags later if they
always make sense together.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-9-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
drivers/net/ethernet/tile/tilegx.c | 4 ++--
include/linux/sched/isolation.h | 26 +++++++++++++++++---------
kernel/rcu/tree_plugin.h | 2 +-
kernel/rcu/update.c | 2 +-
kernel/sched/core.c | 8 ++++----
kernel/sched/fair.c | 2 +-
kernel/sched/isolation.c | 26 +++++++++++++++-----------
kernel/watchdog.c | 3 ++-
8 files changed, 43 insertions(+), 30 deletions(-)
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index 27a3272..b3e5816 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -2270,8 +2270,8 @@ static int __init tile_net_init_module(void)
tile_net_dev_init(name, mac);
if (!network_cpus_init())
- cpumask_and(&network_cpus_map, housekeeping_cpumask(),
- cpu_online_mask);
+ cpumask_and(&network_cpus_map,
+ housekeeping_cpumask(HK_FLAG_MISC), cpu_online_mask);
return 0;
}
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 93ac2367..9bb753e 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -5,35 +5,43 @@
#include <linux/init.h>
#include <linux/tick.h>
+enum hk_flags {
+ HK_FLAG_TIMER = 1,
+ HK_FLAG_RCU = (1 << 1),
+ HK_FLAG_MISC = (1 << 2),
+ HK_FLAG_SCHED = (1 << 3),
+};
+
#ifdef CONFIG_CPU_ISOLATION
DECLARE_STATIC_KEY_FALSE(housekeeping_overriden);
-extern int housekeeping_any_cpu(void);
-extern const struct cpumask *housekeeping_cpumask(void);
-extern void housekeeping_affine(struct task_struct *t);
-extern bool housekeeping_test_cpu(int cpu);
+extern int housekeeping_any_cpu(enum hk_flags flags);
+extern const struct cpumask *housekeeping_cpumask(enum hk_flags flags);
+extern void housekeeping_affine(struct task_struct *t, enum hk_flags flags);
+extern bool housekeeping_test_cpu(int cpu, enum hk_flags flags);
extern void __init housekeeping_init(void);
#else
-static inline int housekeeping_any_cpu(void)
+static inline int housekeeping_any_cpu(enum hk_flags flags)
{
return smp_processor_id();
}
-static inline const struct cpumask *housekeeping_cpumask(void)
+static inline const struct cpumask *housekeeping_cpumask(enum hk_flags flags)
{
return cpu_possible_mask;
}
-static inline void housekeeping_affine(struct task_struct *t) { }
+static inline void housekeeping_affine(struct task_struct *t,
+ enum hk_flags flags) { }
static inline void housekeeping_init(void) { }
#endif /* CONFIG_CPU_ISOLATION */
-static inline bool housekeeping_cpu(int cpu)
+static inline bool housekeeping_cpu(int cpu, enum hk_flags flags)
{
#ifdef CONFIG_CPU_ISOLATION
if (static_branch_unlikely(&housekeeping_overriden))
- return housekeeping_test_cpu(cpu);
+ return housekeeping_test_cpu(cpu, flags);
#endif
return true;
}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index c7632f5..34125d2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -2584,7 +2584,7 @@ static void rcu_bind_gp_kthread(void)
if (!tick_nohz_full_enabled())
return;
- housekeeping_affine(current);
+ housekeeping_affine(current, HK_FLAG_RCU);
}
/* Record the current task on dyntick-idle entry. */
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 79abeb0..e3e60ef 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -719,7 +719,7 @@ static int __noreturn rcu_tasks_kthread(void *arg)
LIST_HEAD(rcu_tasks_holdouts);
/* Run on housekeeping CPUs by default. Sysadm can move if desired. */
- housekeeping_affine(current);
+ housekeeping_affine(current, HK_FLAG_RCU);
/*
* Each pass through the following loop makes one check for
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d0fb448d..2210c02 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -527,7 +527,7 @@ int get_nohz_timer_target(void)
int i, cpu = smp_processor_id();
struct sched_domain *sd;
- if (!idle_cpu(cpu) && housekeeping_cpu(cpu))
+ if (!idle_cpu(cpu) && housekeeping_cpu(cpu, HK_FLAG_TIMER))
return cpu;
rcu_read_lock();
@@ -536,15 +536,15 @@ int get_nohz_timer_target(void)
if (cpu == i)
continue;
- if (!idle_cpu(i) && housekeeping_cpu(i)) {
+ if (!idle_cpu(i) && housekeeping_cpu(i, HK_FLAG_TIMER)) {
cpu = i;
goto unlock;
}
}
}
- if (!housekeeping_cpu(cpu))
- cpu = housekeeping_any_cpu();
+ if (!housekeeping_cpu(cpu, HK_FLAG_TIMER))
+ cpu = housekeeping_any_cpu(HK_FLAG_TIMER);
unlock:
rcu_read_unlock();
return cpu;
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index cdece8f..f755de8 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9027,7 +9027,7 @@ void nohz_balance_enter_idle(int cpu)
return;
/* Spare idle load balancing on CPUs that don't want to be disturbed: */
- if (!housekeeping_cpu(cpu))
+ if (!housekeeping_cpu(cpu, HK_FLAG_SCHED))
return;
if (test_bit(NOHZ_TICK_STOPPED, nohz_flags(cpu)))
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index bb8ba19..37a138a 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -15,37 +15,39 @@
DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
EXPORT_SYMBOL_GPL(housekeeping_overriden);
static cpumask_var_t housekeeping_mask;
+static unsigned int housekeeping_flags;
-int housekeeping_any_cpu(void)
+int housekeeping_any_cpu(enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-
+ if (housekeeping_flags & flags)
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
return smp_processor_id();
}
EXPORT_SYMBOL_GPL(housekeeping_any_cpu);
-const struct cpumask *housekeeping_cpumask(void)
+const struct cpumask *housekeeping_cpumask(enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return housekeeping_mask;
-
+ if (housekeeping_flags & flags)
+ return housekeeping_mask;
return cpu_possible_mask;
}
EXPORT_SYMBOL_GPL(housekeeping_cpumask);
-void housekeeping_affine(struct task_struct *t)
+void housekeeping_affine(struct task_struct *t, enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- set_cpus_allowed_ptr(t, housekeeping_mask);
+ if (housekeeping_flags & flags)
+ set_cpus_allowed_ptr(t, housekeeping_mask);
}
EXPORT_SYMBOL_GPL(housekeeping_affine);
-bool housekeeping_test_cpu(int cpu)
+bool housekeeping_test_cpu(int cpu, enum hk_flags flags)
{
if (static_branch_unlikely(&housekeeping_overriden))
- return cpumask_test_cpu(cpu, housekeeping_mask);
-
+ if (housekeeping_flags & flags)
+ return cpumask_test_cpu(cpu, housekeeping_mask);
return true;
}
EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
@@ -65,6 +67,8 @@ void __init housekeeping_init(void)
cpumask_andnot(housekeeping_mask,
cpu_possible_mask, tick_nohz_full_mask);
+ housekeeping_flags = HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+
static_branch_enable(&housekeeping_overriden);
/* We need at least one CPU to handle housekeeping work */
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 562652c..e9e2ebb 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -777,7 +777,8 @@ void __init lockup_detector_init(void)
if (tick_nohz_full_enabled())
pr_info("Disabling watchdog on nohz_full cores by default\n");
- cpumask_copy(&watchdog_cpumask, housekeeping_cpumask());
+ cpumask_copy(&watchdog_cpumask,
+ housekeeping_cpumask(HK_FLAG_TIMER));
if (!watchdog_nmi_probe())
nmi_watchdog_available = true;
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Handle the nohz_full= parameter
2017-10-27 2:42 ` [PATCH 09/12] housekeeping: Handle the nohz_full= parameter Frederic Weisbecker
@ 2017-10-27 12:05 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:05 UTC (permalink / raw)
To: linux-tip-commits
Cc: riel, lcapitulino, frederic, cmetcalf, tglx, cl, mingo,
linux-kernel, kernellwp, paulmck, peterz, torvalds, hpa, efault
Commit-ID: 6f1982fedd59856bcc42a9b521be4c3ffd2f60a7
Gitweb: https://git.kernel.org/tip/6f1982fedd59856bcc42a9b521be4c3ffd2f60a7
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:36 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:30 +0200
sched/isolation: Handle the nohz_full= parameter
We want to centralize the isolation management, done by the housekeeping
subsystem. Therefore we need to handle the nohz_full= parameter from
there.
Since nohz_full= so far has involved unbound timers, watchdog, RCU
and tilegx NAPI isolation, we keep that default behaviour.
nohz_full= will be deprecated in the future. We want to control
the isolation features from the isolcpus= parameter.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-10-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
include/linux/sched/isolation.h | 1 +
include/linux/tick.h | 2 ++
init/Kconfig | 1 -
kernel/sched/isolation.c | 42 +++++++++++++++++++++++++++++------------
kernel/time/tick-sched.c | 13 +++----------
5 files changed, 36 insertions(+), 23 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index 9bb753e..e53cfa9 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -10,6 +10,7 @@ enum hk_flags {
HK_FLAG_RCU = (1 << 1),
HK_FLAG_MISC = (1 << 2),
HK_FLAG_SCHED = (1 << 3),
+ HK_FLAG_TICK = (1 << 4),
};
#ifdef CONFIG_CPU_ISOLATION
diff --git a/include/linux/tick.h b/include/linux/tick.h
index 68afc09..e2a163a 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -228,6 +228,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
+extern void __init tick_nohz_full_setup(cpumask_var_t cpumask);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
@@ -248,6 +249,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
+static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { }
#endif
static inline void tick_nohz_task_switch(void)
diff --git a/init/Kconfig b/init/Kconfig
index 6f52e6f..f8564df 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -474,7 +474,6 @@ endmenu # "CPU/Task time and stats accounting"
config CPU_ISOLATION
bool "CPU isolation"
- depends on NO_HZ_FULL
help
Make sure that CPUs running critical tasks are not disturbed by
any source of "noise" such as unbound workqueues, timers, kthreads...
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 37a138a..1f61e44 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -54,23 +54,41 @@ EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
void __init housekeeping_init(void)
{
- if (!tick_nohz_full_enabled())
+ if (!housekeeping_flags)
return;
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
+ static_branch_enable(&housekeeping_overriden);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
+
+#ifdef CONFIG_NO_HZ_FULL
+static int __init housekeeping_nohz_full_setup(char *str)
+{
+ cpumask_var_t non_housekeeping_mask;
+
+ alloc_bootmem_cpumask_var(&non_housekeeping_mask);
+ if (cpulist_parse(str, non_housekeeping_mask) < 0) {
+ pr_warn("Housekeeping: Incorrect nohz_full cpumask\n");
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
+ alloc_bootmem_cpumask_var(&housekeeping_mask);
+ cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask);
- housekeeping_flags = HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+ if (cpumask_empty(housekeeping_mask))
+ cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
- static_branch_enable(&housekeeping_overriden);
+ housekeeping_flags = HK_FLAG_TICK | HK_FLAG_TIMER |
+ HK_FLAG_RCU | HK_FLAG_MISC;
- /* We need at least one CPU to handle housekeeping work */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+ tick_nohz_full_setup(non_housekeeping_mask);
+
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+
+ return 1;
}
+__setup("nohz_full=", housekeeping_nohz_full_setup);
+#endif
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 27d7d52..69f3dbe 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -385,20 +385,13 @@ out:
local_irq_restore(flags);
}
-/* Parse the boot-time nohz CPU list from the kernel parameters. */
-static int __init tick_nohz_full_setup(char *str)
+/* Get the boot-time nohz CPU list from the kernel parameters. */
+void __init tick_nohz_full_setup(cpumask_var_t cpumask)
{
alloc_bootmem_cpumask_var(&tick_nohz_full_mask);
- if (cpulist_parse(str, tick_nohz_full_mask) < 0) {
- pr_warn("NO_HZ: Incorrect nohz_full cpumask\n");
- free_bootmem_cpumask_var(tick_nohz_full_mask);
- return 1;
- }
+ cpumask_copy(tick_nohz_full_mask, cpumask);
tick_nohz_full_running = true;
-
- return 1;
}
-__setup("nohz_full=", tick_nohz_full_setup);
static int tick_nohz_cpu_down(unsigned int cpu)
{
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Move isolcpus= handling to the housekeeping code
2017-10-27 2:42 ` [PATCH 10/12] housekeeping: Move isolcpus to housekeeping Frederic Weisbecker
@ 2017-10-27 12:05 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:05 UTC (permalink / raw)
To: linux-tip-commits
Cc: cl, efault, lcapitulino, mingo, torvalds, riel, peterz, tglx,
linux-kernel, kernellwp, hpa, cmetcalf, frederic, paulmck
Commit-ID: edb9382175c3ebdced8ffdb3e0f20052ad9fdbe9
Gitweb: https://git.kernel.org/tip/edb9382175c3ebdced8ffdb3e0f20052ad9fdbe9
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:37 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:30 +0200
sched/isolation: Move isolcpus= handling to the housekeeping code
We want to centralize the isolation features, to be done by the housekeeping
subsystem and scheduler domain isolation is a significant part of it.
No intended behaviour change, we just reuse the housekeeping cpumask
and core code.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-11-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
drivers/base/cpu.c | 11 ++++++-
include/linux/sched.h | 2 --
include/linux/sched/isolation.h | 1 +
kernel/cgroup/cpuset.c | 15 ++++------
kernel/sched/core.c | 16 +----------
kernel/sched/isolation.c | 63 ++++++++++++++++++++++++++++++++---------
kernel/sched/topology.c | 24 ++++------------
7 files changed, 73 insertions(+), 59 deletions(-)
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 321cd7b..a73ab95 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -18,6 +18,7 @@
#include <linux/cpufeature.h>
#include <linux/tick.h>
#include <linux/pm_qos.h>
+#include <linux/sched/isolation.h>
#include "base.h"
@@ -271,8 +272,16 @@ static ssize_t print_cpus_isolated(struct device *dev,
struct device_attribute *attr, char *buf)
{
int n = 0, len = PAGE_SIZE-2;
+ cpumask_var_t isolated;
- n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(cpu_isolated_map));
+ if (!alloc_cpumask_var(&isolated, GFP_KERNEL))
+ return -ENOMEM;
+
+ cpumask_andnot(isolated, cpu_possible_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
+ n = scnprintf(buf, len, "%*pbl\n", cpumask_pr_args(isolated));
+
+ free_cpumask_var(isolated);
return n;
}
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 0f897df..1b0cc0d 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -165,8 +165,6 @@ struct task_group;
/* Task command name length: */
#define TASK_COMM_LEN 16
-extern cpumask_var_t cpu_isolated_map;
-
extern void scheduler_tick(void);
#define MAX_SCHEDULE_TIMEOUT LONG_MAX
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index e53cfa9..d849431 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -11,6 +11,7 @@ enum hk_flags {
HK_FLAG_MISC = (1 << 2),
HK_FLAG_SCHED = (1 << 3),
HK_FLAG_TICK = (1 << 4),
+ HK_FLAG_DOMAIN = (1 << 5),
};
#ifdef CONFIG_CPU_ISOLATION
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 4657e29..f7efa7b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -57,7 +57,7 @@
#include <linux/backing-dev.h>
#include <linux/sort.h>
#include <linux/oom.h>
-
+#include <linux/sched/isolation.h>
#include <linux/uaccess.h>
#include <linux/atomic.h>
#include <linux/mutex.h>
@@ -656,7 +656,6 @@ static int generate_sched_domains(cpumask_var_t **domains,
int csn; /* how many cpuset ptrs in csa so far */
int i, j, k; /* indices for partition finding loops */
cpumask_var_t *doms; /* resulting partition; i.e. sched domains */
- cpumask_var_t non_isolated_cpus; /* load balanced CPUs */
struct sched_domain_attr *dattr; /* attributes for custom domains */
int ndoms = 0; /* number of sched domains in result */
int nslot; /* next empty doms[] struct cpumask slot */
@@ -666,10 +665,6 @@ static int generate_sched_domains(cpumask_var_t **domains,
dattr = NULL;
csa = NULL;
- if (!alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL))
- goto done;
- cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
-
/* Special case for the 99% of systems with one, full, sched domain */
if (is_sched_load_balance(&top_cpuset)) {
ndoms = 1;
@@ -683,7 +678,7 @@ static int generate_sched_domains(cpumask_var_t **domains,
update_domain_attr_tree(dattr, &top_cpuset);
}
cpumask_and(doms[0], top_cpuset.effective_cpus,
- non_isolated_cpus);
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
goto done;
}
@@ -707,7 +702,8 @@ static int generate_sched_domains(cpumask_var_t **domains,
*/
if (!cpumask_empty(cp->cpus_allowed) &&
!(is_sched_load_balance(cp) &&
- cpumask_intersects(cp->cpus_allowed, non_isolated_cpus)))
+ cpumask_intersects(cp->cpus_allowed,
+ housekeeping_cpumask(HK_FLAG_DOMAIN))))
continue;
if (is_sched_load_balance(cp))
@@ -789,7 +785,7 @@ restart:
if (apn == b->pn) {
cpumask_or(dp, dp, b->effective_cpus);
- cpumask_and(dp, dp, non_isolated_cpus);
+ cpumask_and(dp, dp, housekeeping_cpumask(HK_FLAG_DOMAIN));
if (dattr)
update_domain_attr_tree(dattr + nslot, b);
@@ -802,7 +798,6 @@ restart:
BUG_ON(nslot != ndoms);
done:
- free_cpumask_var(non_isolated_cpus);
kfree(csa);
/*
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2210c02..1a55c84 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -84,9 +84,6 @@ __read_mostly int scheduler_running;
*/
int sysctl_sched_rt_runtime = 950000;
-/* CPUs with isolated domains */
-cpumask_var_t cpu_isolated_map;
-
/*
* __task_rq_lock - lock the rq @p resides on.
*/
@@ -5735,10 +5732,6 @@ static inline void sched_init_smt(void) { }
void __init sched_init_smp(void)
{
- cpumask_var_t non_isolated_cpus;
-
- alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL);
-
sched_init_numa();
/*
@@ -5748,16 +5741,12 @@ void __init sched_init_smp(void)
*/
mutex_lock(&sched_domains_mutex);
sched_init_domains(cpu_active_mask);
- cpumask_andnot(non_isolated_cpus, cpu_possible_mask, cpu_isolated_map);
- if (cpumask_empty(non_isolated_cpus))
- cpumask_set_cpu(smp_processor_id(), non_isolated_cpus);
mutex_unlock(&sched_domains_mutex);
/* Move init over to a non-isolated CPU */
- if (set_cpus_allowed_ptr(current, non_isolated_cpus) < 0)
+ if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) < 0)
BUG();
sched_init_granularity();
- free_cpumask_var(non_isolated_cpus);
init_sched_rt_class();
init_sched_dl_class();
@@ -5961,9 +5950,6 @@ void __init sched_init(void)
calc_load_update = jiffies + LOAD_FREQ;
#ifdef CONFIG_SMP
- /* May be allocated at isolcpus cmdline parse time */
- if (cpu_isolated_map == NULL)
- zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
idle_thread_set_boot_cpu();
set_cpu_rq_start_time(smp_processor_id());
#endif
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 1f61e44..8f666bc 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -63,32 +63,69 @@ void __init housekeeping_init(void)
WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
-#ifdef CONFIG_NO_HZ_FULL
-static int __init housekeeping_nohz_full_setup(char *str)
+static int __init housekeeping_setup(char *str, enum hk_flags flags)
{
cpumask_var_t non_housekeeping_mask;
+ int err;
alloc_bootmem_cpumask_var(&non_housekeeping_mask);
- if (cpulist_parse(str, non_housekeeping_mask) < 0) {
- pr_warn("Housekeeping: Incorrect nohz_full cpumask\n");
+ err = cpulist_parse(str, non_housekeeping_mask);
+ if (err < 0 || cpumask_last(non_housekeeping_mask) >= nr_cpu_ids) {
+ pr_warn("Housekeeping: nohz_full= or isolcpus= incorrect CPU range\n");
free_bootmem_cpumask_var(non_housekeeping_mask);
return 0;
}
- alloc_bootmem_cpumask_var(&housekeeping_mask);
- cpumask_andnot(housekeeping_mask, cpu_possible_mask, non_housekeeping_mask);
-
- if (cpumask_empty(housekeeping_mask))
- cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+ if (!housekeeping_flags) {
+ alloc_bootmem_cpumask_var(&housekeeping_mask);
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, non_housekeeping_mask);
+ if (cpumask_empty(housekeeping_mask))
+ cpumask_set_cpu(smp_processor_id(), housekeeping_mask);
+ } else {
+ cpumask_var_t tmp;
+
+ alloc_bootmem_cpumask_var(&tmp);
+ cpumask_andnot(tmp, cpu_possible_mask, non_housekeeping_mask);
+ if (!cpumask_equal(tmp, housekeeping_mask)) {
+ pr_warn("Housekeeping: nohz_full= must match isolcpus=\n");
+ free_bootmem_cpumask_var(tmp);
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
+ }
+ free_bootmem_cpumask_var(tmp);
+ }
- housekeeping_flags = HK_FLAG_TICK | HK_FLAG_TIMER |
- HK_FLAG_RCU | HK_FLAG_MISC;
+ if ((flags & HK_FLAG_TICK) && !(housekeeping_flags & HK_FLAG_TICK)) {
+ if (IS_ENABLED(CONFIG_NO_HZ_FULL)) {
+ tick_nohz_full_setup(non_housekeeping_mask);
+ } else {
+ pr_warn("Housekeeping: nohz unsupported."
+ " Build with CONFIG_NO_HZ_FULL\n");
+ free_bootmem_cpumask_var(non_housekeeping_mask);
+ return 0;
+ }
+ }
- tick_nohz_full_setup(non_housekeeping_mask);
+ housekeeping_flags |= flags;
free_bootmem_cpumask_var(non_housekeeping_mask);
return 1;
}
+
+static int __init housekeeping_nohz_full_setup(char *str)
+{
+ unsigned int flags;
+
+ flags = HK_FLAG_TICK | HK_FLAG_TIMER | HK_FLAG_RCU | HK_FLAG_MISC;
+
+ return housekeeping_setup(str, flags);
+}
__setup("nohz_full=", housekeeping_nohz_full_setup);
-#endif
+
+static int __init housekeeping_isolcpus_setup(char *str)
+{
+ return housekeeping_setup(str, HK_FLAG_DOMAIN);
+}
+__setup("isolcpus=", housekeeping_isolcpus_setup);
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index e3d31b0..2e6b912 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -3,6 +3,7 @@
*/
#include <linux/sched.h>
#include <linux/mutex.h>
+#include <linux/sched/isolation.h>
#include "sched.h"
@@ -469,21 +470,6 @@ cpu_attach_domain(struct sched_domain *sd, struct root_domain *rd, int cpu)
update_top_cache_domain(cpu);
}
-/* Setup the mask of CPUs configured for isolated domains */
-static int __init isolated_cpu_setup(char *str)
-{
- int ret;
-
- alloc_bootmem_cpumask_var(&cpu_isolated_map);
- ret = cpulist_parse(str, cpu_isolated_map);
- if (ret || cpumask_last(cpu_isolated_map) >= nr_cpu_ids) {
- pr_err("sched: Error, all isolcpus= values must be between 0 and %u - ignoring them.\n", nr_cpu_ids-1);
- return 0;
- }
- return 1;
-}
-__setup("isolcpus=", isolated_cpu_setup);
-
struct s_data {
struct sched_domain ** __percpu sd;
struct root_domain *rd;
@@ -1792,7 +1778,7 @@ int sched_init_domains(const struct cpumask *cpu_map)
doms_cur = alloc_sched_domains(ndoms_cur);
if (!doms_cur)
doms_cur = &fallback_doms;
- cpumask_andnot(doms_cur[0], cpu_map, cpu_isolated_map);
+ cpumask_and(doms_cur[0], cpu_map, housekeeping_cpumask(HK_FLAG_DOMAIN));
err = build_sched_domains(doms_cur[0], NULL);
register_sched_domain_sysctl();
@@ -1875,7 +1861,8 @@ void partition_sched_domains(int ndoms_new, cpumask_var_t doms_new[],
doms_new = alloc_sched_domains(1);
if (doms_new) {
n = 1;
- cpumask_andnot(doms_new[0], cpu_active_mask, cpu_isolated_map);
+ cpumask_and(doms_new[0], cpu_active_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
}
} else {
n = ndoms_new;
@@ -1898,7 +1885,8 @@ match1:
if (!doms_new) {
n = 0;
doms_new = &fallback_doms;
- cpumask_andnot(doms_new[0], cpu_active_mask, cpu_isolated_map);
+ cpumask_and(doms_new[0], cpu_active_mask,
+ housekeeping_cpumask(HK_FLAG_DOMAIN));
}
/* Build new domains: */
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Add basic isolcpus flags
2017-10-27 2:42 ` [PATCH 11/12] housekeeping: Add basic isolcpus flags Frederic Weisbecker
@ 2017-10-27 12:05 ` tip-bot for Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:05 UTC (permalink / raw)
To: linux-tip-commits
Cc: mingo, lcapitulino, kernellwp, frederic, peterz, tglx, cl,
torvalds, cmetcalf, paulmck, efault, linux-kernel, hpa, riel
Commit-ID: 150dfee95feb913876ced5cc559a342384e8ea97
Gitweb: https://git.kernel.org/tip/150dfee95feb913876ced5cc559a342384e8ea97
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:38 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:31 +0200
sched/isolation: Add basic isolcpus flags
Add flags to control NOHZ and domain isolation from "isolcpus=", in
order to centralize the isolation features to a common interface. Domain
isolation remains the default so not to break the existing isolcpus
boot paramater behaviour.
Further flags in the future may include 0hz (1hz tick offload) and timers,
workqueue, RCU, kthread, watchdog, likely all merged together in a
common flag ("async"?). In any case, this will have to be modifiable by
cpusets.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-12-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
kernel/sched/isolation.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 8f666bc..b71b436 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -11,6 +11,7 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/static_key.h>
+#include <linux/ctype.h>
DEFINE_STATIC_KEY_FALSE(housekeeping_overriden);
EXPORT_SYMBOL_GPL(housekeeping_overriden);
@@ -126,6 +127,29 @@ __setup("nohz_full=", housekeeping_nohz_full_setup);
static int __init housekeeping_isolcpus_setup(char *str)
{
- return housekeeping_setup(str, HK_FLAG_DOMAIN);
+ unsigned int flags = 0;
+
+ while (isalpha(*str)) {
+ if (!strncmp(str, "nohz,", 5)) {
+ str += 5;
+ flags |= HK_FLAG_TICK;
+ continue;
+ }
+
+ if (!strncmp(str, "domain,", 7)) {
+ str += 7;
+ flags |= HK_FLAG_DOMAIN;
+ continue;
+ }
+
+ pr_warn("isolcpus: Error, unknown flag\n");
+ return 0;
+ }
+
+ /* Default behaviour for isolcpus without flags */
+ if (!flags)
+ flags |= HK_FLAG_DOMAIN;
+
+ return housekeeping_setup(str, flags);
}
__setup("isolcpus=", housekeeping_isolcpus_setup);
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 2:42 ` [PATCH 12/12] housekeeping: Document " Frederic Weisbecker
@ 2017-10-27 12:06 ` tip-bot for Frederic Weisbecker
2017-10-27 13:58 ` Peter Zijlstra
0 siblings, 1 reply; 42+ messages in thread
From: tip-bot for Frederic Weisbecker @ 2017-10-27 12:06 UTC (permalink / raw)
To: linux-tip-commits
Cc: peterz, hpa, riel, tglx, linux-kernel, cmetcalf, torvalds,
kernellwp, paulmck, lcapitulino, mingo, cl, frederic, efault
Commit-ID: 98a06af93784a6ae541e9fc6292ef26177832543
Gitweb: https://git.kernel.org/tip/98a06af93784a6ae541e9fc6292ef26177832543
Author: Frederic Weisbecker <frederic@kernel.org>
AuthorDate: Fri, 27 Oct 2017 04:42:39 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 27 Oct 2017 09:55:32 +0200
sched/isolation: Document the isolcpus= flags
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Link: http://lkml.kernel.org/r/1509072159-31808-13-git-send-email-frederic@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
Documentation/admin-guide/kernel-parameters.txt | 33 +++++++++++++++----------
1 file changed, 20 insertions(+), 13 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 6b99c8b..5853618 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1727,20 +1727,27 @@
isapnp= [ISAPNP]
Format: <RDP>,<reset>,<pci_scan>,<verbosity>
- isolcpus= [KNL,SMP] Isolate CPUs from the general scheduler.
- The argument is a cpu list, as described above.
+ isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
+ Format: [flag-list,]<cpu-list>
+
+ Specify one or more CPUs to isolate from disturbances
+ specified in the flag list (default: domain):
+
+ nohz
+ Disable the tick when a single task runs.
+ domain
+ Isolate from the general SMP balancing and scheduling
+ algorithms. This option is the preferred way to isolate
+ CPUs from tasks. The alternative -- manually setting the
+ CPU mask of all tasks in the system, can cause problems
+ and suboptimal load balancer performance. You can move a
+ process onto or off an "isolated" CPU via the CPU
+ affinity syscalls or cpuset. <cpu number> begins at 0
+ and the maximum value is "number of CPUs in system - 1".
+
+ The format of <cpu-list> is described above.
+
- This option can be used to specify one or more CPUs
- to isolate from the general SMP balancing and scheduling
- algorithms. You can move a process onto or off an
- "isolated" CPU via the CPU affinity syscalls or cpuset.
- <cpu number> begins at 0 and the maximum value is
- "number of CPUs in system - 1".
-
- This option is the preferred way to isolate CPUs. The
- alternative -- manually setting the CPU mask of all
- tasks in the system -- can cause problems and
- suboptimal load balancer performance.
iucv= [HW,NET]
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 12:06 ` [tip:sched/core] sched/isolation: Document the isolcpus= flags tip-bot for Frederic Weisbecker
@ 2017-10-27 13:58 ` Peter Zijlstra
2017-10-27 14:36 ` Frederic Weisbecker
` (2 more replies)
0 siblings, 3 replies; 42+ messages in thread
From: Peter Zijlstra @ 2017-10-27 13:58 UTC (permalink / raw)
To: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, cl, mingo,
efault, frederic, kernellwp, paulmck, lcapitulino
Cc: linux-tip-commits
On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic Weisbecker wrote:
> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
> + Format: [flag-list,]<cpu-list>
> +
> + Specify one or more CPUs to isolate from disturbances
> + specified in the flag list (default: domain):
> +
> + nohz
> + Disable the tick when a single task runs.
> + domain
> + Isolate from the general SMP balancing and scheduling
> + algorithms. This option is the preferred way to isolate
> + CPUs from tasks.
I _strongly_ object to this statement, isolcpus is _not_ the preferred
way, cpusets are.
And yes, while cpusets suffers some problems, we _should_ really fix
those and not promote this piece of shit isolcpus crap.
> The alternative -- manually setting the
> + CPU mask of all tasks in the system, can cause problems
> + and suboptimal load balancer performance. You can move a
> + process onto or off an "isolated" CPU via the CPU
> + affinity syscalls or cpuset. <cpu number> begins at 0
> + and the maximum value is "number of CPUs in system - 1".
> +
> + The format of <cpu-list> is described above.
> +
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 13:58 ` Peter Zijlstra
@ 2017-10-27 14:36 ` Frederic Weisbecker
2017-10-27 17:06 ` Ingo Molnar
2017-10-27 14:38 ` Mike Galbraith
2017-10-30 15:48 ` Christopher Lameter
2 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 14:36 UTC (permalink / raw)
To: Peter Zijlstra
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, cl, mingo,
efault, kernellwp, paulmck, lcapitulino, linux-tip-commits
2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
> On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic Weisbecker
> wrote:
>> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
>> + Format: [flag-list,]<cpu-list>
>> +
>> + Specify one or more CPUs to isolate from disturbances
>> + specified in the flag list (default: domain):
>> +
>> + nohz
>> + Disable the tick when a single task runs.
>> + domain
>> + Isolate from the general SMP balancing and scheduling
>> + algorithms. This option is the preferred way to isolate
>> + CPUs from tasks.
>
> I _strongly_ object to this statement, isolcpus is _not_ the preferred
> way, cpusets are.
>
> And yes, while cpusets suffers some problems, we _should_ really fix
> those and not promote this piece of shit isolcpus crap.
I definitely agree with that so your position is a relief :-) This
patch only indented the existing parameter documentation so fixing its
content was beyond its scope. I'll send a patch to correct the text.
Thanks.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 13:58 ` Peter Zijlstra
2017-10-27 14:36 ` Frederic Weisbecker
@ 2017-10-27 14:38 ` Mike Galbraith
2017-10-30 15:48 ` Christopher Lameter
2 siblings, 0 replies; 42+ messages in thread
From: Mike Galbraith @ 2017-10-27 14:38 UTC (permalink / raw)
To: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, mingo, frederic, kernellwp, paulmck, lcapitulino
Cc: linux-tip-commits
On Fri, 2017-10-27 at 15:58 +0200, Peter Zijlstra wrote:
> On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic Weisbecker wrote:
> > + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
> > + Format: [flag-list,]<cpu-list>
> > +
> > + Specify one or more CPUs to isolate from disturbances
> > + specified in the flag list (default: domain):
> > +
> > + nohz
> > + Disable the tick when a single task runs.
> > + domain
> > + Isolate from the general SMP balancing and scheduling
> > + algorithms. This option is the preferred way to isolate
> > + CPUs from tasks.
>
> I _strongly_ object to this statement, isolcpus is _not_ the preferred
> way, cpusets are.
>
> And yes, while cpusets suffers some problems, we _should_ really fix
> those and not promote this piece of shit isolcpus crap.
+1, isolcpus is the dinosaur that just won't frickn' die.
-Mike
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 14:36 ` Frederic Weisbecker
@ 2017-10-27 17:06 ` Ingo Molnar
2017-10-27 17:33 ` Frederic Weisbecker
0 siblings, 1 reply; 42+ messages in thread
From: Ingo Molnar @ 2017-10-27 17:06 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, efault, kernellwp, paulmck, lcapitulino,
linux-tip-commits
* Frederic Weisbecker <frederic@kernel.org> wrote:
> 2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
> > On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic Weisbecker
> > wrote:
> >> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
> >> + Format: [flag-list,]<cpu-list>
> >> +
> >> + Specify one or more CPUs to isolate from disturbances
> >> + specified in the flag list (default: domain):
> >> +
> >> + nohz
> >> + Disable the tick when a single task runs.
> >> + domain
> >> + Isolate from the general SMP balancing and scheduling
> >> + algorithms. This option is the preferred way to isolate
> >> + CPUs from tasks.
> >
> > I _strongly_ object to this statement, isolcpus is _not_ the preferred
> > way, cpusets are.
> >
> > And yes, while cpusets suffers some problems, we _should_ really fix
> > those and not promote this piece of shit isolcpus crap.
>
> I definitely agree with that so your position is a relief :-) This
> patch only indented the existing parameter documentation so fixing its
> content was beyond its scope. I'll send a patch to correct the text.
Since it was the last commit in tip:sched/core that was pushed out just hours ago
I zapped that last commit, please send an updated patch which we can apply and get
a clean series.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 17:06 ` Ingo Molnar
@ 2017-10-27 17:33 ` Frederic Weisbecker
2017-10-27 18:21 ` Ingo Molnar
0 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 17:33 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, efault, kernellwp, paulmck, lcapitulino,
linux-tip-commits
2017-10-27 19:06 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
>
> * Frederic Weisbecker <frederic@kernel.org> wrote:
>
>> 2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
>> > On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic
>> > Weisbecker
>> > wrote:
>> >> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
>> >> + Format: [flag-list,]<cpu-list>
>> >> +
>> >> + Specify one or more CPUs to isolate from disturbances
>> >> + specified in the flag list (default: domain):
>> >> +
>> >> + nohz
>> >> + Disable the tick when a single task runs.
>> >> + domain
>> >> + Isolate from the general SMP balancing and scheduling
>> >> + algorithms. This option is the preferred way to isolate
>> >> + CPUs from tasks.
>> >
>> > I _strongly_ object to this statement, isolcpus is _not_ the preferred
>> > way, cpusets are.
>> >
>> > And yes, while cpusets suffers some problems, we _should_ really fix
>> > those and not promote this piece of shit isolcpus crap.
>>
>> I definitely agree with that so your position is a relief :-) This
>> patch only indented the existing parameter documentation so fixing its
>> content was beyond its scope. I'll send a patch to correct the text.
>
> Since it was the last commit in tip:sched/core that was pushed out just
> hours ago
> I zapped that last commit, please send an updated patch which we can apply
> and get
> a clean series.
>
> Thanks,
Note the issue was there before that patch. But nevermind I'll resend
an updated version of the patch.
Thanks.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 17:33 ` Frederic Weisbecker
@ 2017-10-27 18:21 ` Ingo Molnar
2017-10-27 18:39 ` Frederic Weisbecker
0 siblings, 1 reply; 42+ messages in thread
From: Ingo Molnar @ 2017-10-27 18:21 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, efault, kernellwp, paulmck, lcapitulino,
linux-tip-commits
* Frederic Weisbecker <frederic@kernel.org> wrote:
> 2017-10-27 19:06 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
> >
> > * Frederic Weisbecker <frederic@kernel.org> wrote:
> >
> >> 2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
> >> > On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic
> >> > Weisbecker
> >> > wrote:
> >> >> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
> >> >> + Format: [flag-list,]<cpu-list>
> >> >> +
> >> >> + Specify one or more CPUs to isolate from disturbances
> >> >> + specified in the flag list (default: domain):
> >> >> +
> >> >> + nohz
> >> >> + Disable the tick when a single task runs.
> >> >> + domain
> >> >> + Isolate from the general SMP balancing and scheduling
> >> >> + algorithms. This option is the preferred way to isolate
> >> >> + CPUs from tasks.
> >> >
> >> > I _strongly_ object to this statement, isolcpus is _not_ the preferred
> >> > way, cpusets are.
> >> >
> >> > And yes, while cpusets suffers some problems, we _should_ really fix
> >> > those and not promote this piece of shit isolcpus crap.
> >>
> >> I definitely agree with that so your position is a relief :-) This
> >> patch only indented the existing parameter documentation so fixing its
> >> content was beyond its scope. I'll send a patch to correct the text.
> >
> > Since it was the last commit in tip:sched/core that was pushed out just
> > hours ago
> > I zapped that last commit, please send an updated patch which we can apply
> > and get
> > a clean series.
> >
> > Thanks,
>
> Note the issue was there before that patch. But nevermind I'll resend
> an updated version of the patch.
Yeah - so we get a single 'improve documentation' commit.
Thanks,
Ingo
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 18:21 ` Ingo Molnar
@ 2017-10-27 18:39 ` Frederic Weisbecker
2017-10-27 19:04 ` Ingo Molnar
0 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-27 18:39 UTC (permalink / raw)
To: Ingo Molnar
Cc: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, efault, kernellwp, paulmck, lcapitulino,
linux-tip-commits
2017-10-27 20:21 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
>
> * Frederic Weisbecker <frederic@kernel.org> wrote:
>
>> 2017-10-27 19:06 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
>> >
>> > * Frederic Weisbecker <frederic@kernel.org> wrote:
>> >
>> >> 2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
>> >> > On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic
>> >> > Weisbecker
>> >> > wrote:
>> >> >> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
>> >> >> + Format: [flag-list,]<cpu-list>
>> >> >> +
>> >> >> + Specify one or more CPUs to isolate from disturbances
>> >> >> + specified in the flag list (default: domain):
>> >> >> +
>> >> >> + nohz
>> >> >> + Disable the tick when a single task runs.
>> >> >> + domain
>> >> >> + Isolate from the general SMP balancing and scheduling
>> >> >> + algorithms. This option is the preferred way to isolate
>> >> >> + CPUs from tasks.
>> >> >
>> >> > I _strongly_ object to this statement, isolcpus is _not_ the
>> >> > preferred
>> >> > way, cpusets are.
>> >> >
>> >> > And yes, while cpusets suffers some problems, we _should_ really fix
>> >> > those and not promote this piece of shit isolcpus crap.
>> >>
>> >> I definitely agree with that so your position is a relief :-) This
>> >> patch only indented the existing parameter documentation so fixing its
>> >> content was beyond its scope. I'll send a patch to correct the text.
>> >
>> > Since it was the last commit in tip:sched/core that was pushed out just
>> > hours ago
>> > I zapped that last commit, please send an updated patch which we can
>> > apply
>> > and get
>> > a clean series.
>> >
>> > Thanks,
>>
>> Note the issue was there before that patch. But nevermind I'll resend
>> an updated version of the patch.
>
> Yeah - so we get a single 'improve documentation' commit.
Ah ok I see. No problem I'll resend.
Thanks.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 18:39 ` Frederic Weisbecker
@ 2017-10-27 19:04 ` Ingo Molnar
0 siblings, 0 replies; 42+ messages in thread
From: Ingo Molnar @ 2017-10-27 19:04 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Peter Zijlstra, cmetcalf, linux-kernel, tglx, torvalds, hpa,
riel, cl, efault, kernellwp, paulmck, lcapitulino,
linux-tip-commits
* Frederic Weisbecker <frederic@kernel.org> wrote:
> 2017-10-27 20:21 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
> >
> > * Frederic Weisbecker <frederic@kernel.org> wrote:
> >
> >> 2017-10-27 19:06 UTC+02:00, Ingo Molnar <mingo@kernel.org>:
> >> >
> >> > * Frederic Weisbecker <frederic@kernel.org> wrote:
> >> >
> >> >> 2017-10-27 15:58 UTC+02:00, Peter Zijlstra <peterz@infradead.org>:
> >> >> > On Fri, Oct 27, 2017 at 05:06:25AM -0700, tip-bot for Frederic
> >> >> > Weisbecker
> >> >> > wrote:
> >> >> >> + isolcpus= [KNL,SMP] Isolate a given set of CPUs from disturbance.
> >> >> >> + Format: [flag-list,]<cpu-list>
> >> >> >> +
> >> >> >> + Specify one or more CPUs to isolate from disturbances
> >> >> >> + specified in the flag list (default: domain):
> >> >> >> +
> >> >> >> + nohz
> >> >> >> + Disable the tick when a single task runs.
> >> >> >> + domain
> >> >> >> + Isolate from the general SMP balancing and scheduling
> >> >> >> + algorithms. This option is the preferred way to isolate
> >> >> >> + CPUs from tasks.
> >> >> >
> >> >> > I _strongly_ object to this statement, isolcpus is _not_ the
> >> >> > preferred
> >> >> > way, cpusets are.
> >> >> >
> >> >> > And yes, while cpusets suffers some problems, we _should_ really fix
> >> >> > those and not promote this piece of shit isolcpus crap.
> >> >>
> >> >> I definitely agree with that so your position is a relief :-) This
> >> >> patch only indented the existing parameter documentation so fixing its
> >> >> content was beyond its scope. I'll send a patch to correct the text.
> >> >
> >> > Since it was the last commit in tip:sched/core that was pushed out just
> >> > hours ago
> >> > I zapped that last commit, please send an updated patch which we can
> >> > apply
> >> > and get
> >> > a clean series.
> >> >
> >> > Thanks,
> >>
> >> Note the issue was there before that patch. But nevermind I'll resend
> >> an updated version of the patch.
> >
> > Yeah - so we get a single 'improve documentation' commit.
>
> Ah ok I see. No problem I'll resend.
Thanks!
Ingo
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-27 13:58 ` Peter Zijlstra
2017-10-27 14:36 ` Frederic Weisbecker
2017-10-27 14:38 ` Mike Galbraith
@ 2017-10-30 15:48 ` Christopher Lameter
2017-10-30 16:11 ` Peter Zijlstra
2 siblings, 1 reply; 42+ messages in thread
From: Christopher Lameter @ 2017-10-30 15:48 UTC (permalink / raw)
To: Peter Zijlstra
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, mingo, efault,
frederic, kernellwp, paulmck, lcapitulino, linux-tip-commits
On Fri, 27 Oct 2017, Peter Zijlstra wrote:
> I _strongly_ object to this statement, isolcpus is _not_ the preferred
> way, cpusets are.
>
> And yes, while cpusets suffers some problems, we _should_ really fix
> those and not promote this piece of shit isolcpus crap.
Well low level control at the processor level is important and this allows
controlling activities on a processor that is supposed to be dedicated to
certain activities without OS interaction.
isolcpus is the *right* approach here because you are micromanaging the OS
and are putting dedicated pieces of software on each core.
A cgroup suggests that threads would be scheduled over multiple cores
which is *not* what you want. cgroup has to do something with containers
etc which is inherently more noisy and needed if you want to do different
things with your processing resources.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-30 15:48 ` Christopher Lameter
@ 2017-10-30 16:11 ` Peter Zijlstra
2017-10-30 16:30 ` Christopher Lameter
2017-10-30 16:56 ` Mike Galbraith
0 siblings, 2 replies; 42+ messages in thread
From: Peter Zijlstra @ 2017-10-30 16:11 UTC (permalink / raw)
To: Christopher Lameter
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, mingo, efault,
frederic, kernellwp, paulmck, lcapitulino, linux-tip-commits
On Mon, Oct 30, 2017 at 10:48:04AM -0500, Christopher Lameter wrote:
> On Fri, 27 Oct 2017, Peter Zijlstra wrote:
>
> > I _strongly_ object to this statement, isolcpus is _not_ the preferred
> > way, cpusets are.
> >
> > And yes, while cpusets suffers some problems, we _should_ really fix
> > those and not promote this piece of shit isolcpus crap.
>
> Well low level control at the processor level is important and this allows
> controlling activities on a processor that is supposed to be dedicated to
> certain activities without OS interaction.
>
> isolcpus is the *right* approach here because you are micromanaging the OS
> and are putting dedicated pieces of software on each core.
That is what you want, and cpusets should allow for that just fine.
> A cgroup suggests that threads would be scheduled over multiple cores
> which is *not* what you want.
No, that suggestion is false. cpusets should allow you to isolate
individual CPUs just fine.
> cgroup has to do something with containers
Sod containers. That's just modern group think. cpusets existed long
before all that wankery and it should very well retain the original use
cases.
That said, I know there's problems with cpusets, and those should be
fixed. But that doesn't mean isolcpus is anything other than a vile
hack.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-30 16:11 ` Peter Zijlstra
@ 2017-10-30 16:30 ` Christopher Lameter
2017-10-30 16:58 ` Peter Zijlstra
2017-10-30 16:56 ` Mike Galbraith
1 sibling, 1 reply; 42+ messages in thread
From: Christopher Lameter @ 2017-10-30 16:30 UTC (permalink / raw)
To: Peter Zijlstra
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, mingo, efault,
frederic, kernellwp, paulmck, lcapitulino, linux-tip-commits
On Mon, 30 Oct 2017, Peter Zijlstra wrote:
> > isolcpus is the *right* approach here because you are micromanaging the OS
> > and are putting dedicated pieces of software on each core.
>
> That is what you want, and cpusets should allow for that just fine.
Well yes a cpuset of one processor I guess.
> > A cgroup suggests that threads would be scheduled over multiple cores
> > which is *not* what you want.
>
> No, that suggestion is false. cpusets should allow you to isolate
> individual CPUs just fine.
>
> > cgroup has to do something with containers
>
> Sod containers. That's just modern group think. cpusets existed long
> before all that wankery and it should very well retain the original use
> cases.
Historically cpusets were not used for cpu isolation. They were used to
restrict applications threads to sets of cpus for performance reasons. And
we are here dealing with individual processors.
> That said, I know there's problems with cpusets, and those should be
> fixed. But that doesn't mean isolcpus is anything other than a vile
> hack.
Controlling the way an individual processor works would be best done with
some kind of configuration in sysfs. I.e. /sys/cpu/<xx>/no_sched or so.
In lieu of that I think isolcpus is definitely better than a cpuset and it
is the only way that traditionally processor handling of the OS has been
restricted.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-30 16:11 ` Peter Zijlstra
2017-10-30 16:30 ` Christopher Lameter
@ 2017-10-30 16:56 ` Mike Galbraith
1 sibling, 0 replies; 42+ messages in thread
From: Mike Galbraith @ 2017-10-30 16:56 UTC (permalink / raw)
To: Peter Zijlstra, Christopher Lameter
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, mingo,
frederic, kernellwp, paulmck, lcapitulino, linux-tip-commits
On Mon, 2017-10-30 at 17:11 +0100, Peter Zijlstra wrote:
> On Mon, Oct 30, 2017 at 10:48:04AM -0500, Christopher Lameter wrote:
> > On Fri, 27 Oct 2017, Peter Zijlstra wrote:
> >
> > > I _strongly_ object to this statement, isolcpus is _not_ the preferred
> > > way, cpusets are.
> > >
> > > And yes, while cpusets suffers some problems, we _should_ really fix
> > > those and not promote this piece of shit isolcpus crap.
> >
> > Well low level control at the processor level is important and this allows
> > controlling activities on a processor that is supposed to be dedicated to
> > certain activities without OS interaction.
> >
> > isolcpus is the *right* approach here because you are micromanaging the OS
> > and are putting dedicated pieces of software on each core.
>
> That is what you want, and cpusets should allow for that just fine.
>
> > A cgroup suggests that threads would be scheduled over multiple cores
> > which is *not* what you want.
>
> No, that suggestion is false. cpusets should allow you to isolate
> individual CPUs just fine.
It does. I do RT jitter testing with it regularly. If it didn't work,
my 8 socket box would let me know instantly, by completely sucking :)
-Mike
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [tip:sched/core] sched/isolation: Document the isolcpus= flags
2017-10-30 16:30 ` Christopher Lameter
@ 2017-10-30 16:58 ` Peter Zijlstra
0 siblings, 0 replies; 42+ messages in thread
From: Peter Zijlstra @ 2017-10-30 16:58 UTC (permalink / raw)
To: Christopher Lameter
Cc: cmetcalf, linux-kernel, tglx, torvalds, hpa, riel, mingo, efault,
frederic, kernellwp, paulmck, lcapitulino, linux-tip-commits
On Mon, Oct 30, 2017 at 11:30:59AM -0500, Christopher Lameter wrote:
> Historically cpusets were not used for cpu isolation. They were used to
> restrict applications threads to sets of cpus for performance reasons. And
> we are here dealing with individual processors.
The HPC workloads very much disabled load-balancing across most CPUs. If
you disable "sched_load_balance" the thing creates NULL sched_domains,
the exact thing isolcpus ends up doing.
This is something cpusets have done for a _long_ time, if not from the
very start.
Yes, you can also create smaller sched_domains which is useful for other
cases and you can even mix the lot, by creating a small set of
load-balanced CPUs for the system tasks while giving a bunch of
unbalanced CPUs to your application.
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [PATCH 01/12] housekeeping: Move housekeeping related code to its own file
2017-10-24 13:06 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
@ 2017-10-24 13:35 ` Paul E. McKenney
0 siblings, 0 replies; 42+ messages in thread
From: Paul E. McKenney @ 2017-10-24 13:35 UTC (permalink / raw)
To: Frederic Weisbecker
Cc: Ingo Molnar, LKML, Peter Zijlstra, Chris Metcalf,
Thomas Gleixner, Luiz Capitulino, Christoph Lameter,
Mike Galbraith, Rik van Riel, Wanpeng Li
On Tue, Oct 24, 2017 at 03:06:50PM +0200, Frederic Weisbecker wrote:
> The housekeeping code is currently tied to the nohz code. As we are
> planning to make housekeeping independant from it, start with moving
> the relevant code to its own file.
>
> Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> Cc: Chris Metcalf <cmetcalf@mellanox.com>
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Mike Galbraith <efault@gmx.de>
> Cc: Ingo Molnar <mingo@kernel.org>
> Cc: Christoph Lameter <cl@linux.com>
> Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
>From an RCU perspective:
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Wanpeng Li <kernellwp@gmail.com>
> Cc: Luiz Capitulino <lcapitulino@redhat.com>
> ---
> drivers/net/ethernet/tile/tilegx.c | 2 +-
> include/linux/housekeeping.h | 56 ++++++++++++++++++++++++++++++++++++++
> include/linux/tick.h | 37 -------------------------
> init/main.c | 2 ++
> kernel/Makefile | 1 +
> kernel/housekeeping.c | 33 ++++++++++++++++++++++
> kernel/rcu/tree_plugin.h | 1 +
> kernel/rcu/update.c | 1 +
> kernel/sched/core.c | 1 +
> kernel/sched/fair.c | 1 +
> kernel/time/tick-sched.c | 18 ------------
> kernel/watchdog.c | 1 +
> 12 files changed, 98 insertions(+), 56 deletions(-)
> create mode 100644 include/linux/housekeeping.h
> create mode 100644 kernel/housekeeping.c
>
> diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
> index c00102b..8c7ef12 100644
> --- a/drivers/net/ethernet/tile/tilegx.c
> +++ b/drivers/net/ethernet/tile/tilegx.c
> @@ -40,7 +40,7 @@
> #include <linux/tcp.h>
> #include <linux/net_tstamp.h>
> #include <linux/ptp_clock_kernel.h>
> -#include <linux/tick.h>
> +#include <linux/housekeeping.h>
>
> #include <asm/checksum.h>
> #include <asm/homecache.h>
> diff --git a/include/linux/housekeeping.h b/include/linux/housekeeping.h
> new file mode 100644
> index 0000000..3d6a8e6
> --- /dev/null
> +++ b/include/linux/housekeeping.h
> @@ -0,0 +1,56 @@
> +#ifndef _LINUX_HOUSEKEEPING_H
> +#define _LINUX_HOUSEKEEPING_H
> +
> +#include <linux/cpumask.h>
> +#include <linux/init.h>
> +#include <linux/tick.h>
> +
> +#ifdef CONFIG_NO_HZ_FULL
> +extern cpumask_var_t housekeeping_mask;
> +
> +static inline int housekeeping_any_cpu(void)
> +{
> + return cpumask_any_and(housekeeping_mask, cpu_online_mask);
> +}
> +
> +extern void __init housekeeping_init(void);
> +
> +#else
> +
> +static inline int housekeeping_any_cpu(void)
> +{
> + return smp_processor_id();
> +}
> +
> +static inline void housekeeping_init(void) { }
> +#endif /* CONFIG_NO_HZ_FULL */
> +
> +
> +static inline const struct cpumask *housekeeping_cpumask(void)
> +{
> +#ifdef CONFIG_NO_HZ_FULL
> + if (tick_nohz_full_enabled())
> + return housekeeping_mask;
> +#endif
> + return cpu_possible_mask;
> +}
> +
> +static inline bool is_housekeeping_cpu(int cpu)
> +{
> +#ifdef CONFIG_NO_HZ_FULL
> + if (tick_nohz_full_enabled())
> + return cpumask_test_cpu(cpu, housekeeping_mask);
> +#endif
> + return true;
> +}
> +
> +static inline void housekeeping_affine(struct task_struct *t)
> +{
> +#ifdef CONFIG_NO_HZ_FULL
> + if (tick_nohz_full_enabled())
> + set_cpus_allowed_ptr(t, housekeeping_mask);
> +
> +#endif
> +}
> +
> +#endif /* _LINUX_HOUSEKEEPING_H */
> diff --git a/include/linux/tick.h b/include/linux/tick.h
> index fe01e68..68afc09 100644
> --- a/include/linux/tick.h
> +++ b/include/linux/tick.h
> @@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
> #ifdef CONFIG_NO_HZ_FULL
> extern bool tick_nohz_full_running;
> extern cpumask_var_t tick_nohz_full_mask;
> -extern cpumask_var_t housekeeping_mask;
>
> static inline bool tick_nohz_full_enabled(void)
> {
> @@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
> cpumask_or(mask, mask, tick_nohz_full_mask);
> }
>
> -static inline int housekeeping_any_cpu(void)
> -{
> - return cpumask_any_and(housekeeping_mask, cpu_online_mask);
> -}
> -
> extern void tick_nohz_dep_set(enum tick_dep_bits bit);
> extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
> extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
> @@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
> extern void tick_nohz_full_kick_cpu(int cpu);
> extern void __tick_nohz_task_switch(void);
> #else
> -static inline int housekeeping_any_cpu(void)
> -{
> - return smp_processor_id();
> -}
> static inline bool tick_nohz_full_enabled(void) { return false; }
> static inline bool tick_nohz_full_cpu(int cpu) { return false; }
> static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
> @@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
> static inline void __tick_nohz_task_switch(void) { }
> #endif
>
> -static inline const struct cpumask *housekeeping_cpumask(void)
> -{
> -#ifdef CONFIG_NO_HZ_FULL
> - if (tick_nohz_full_enabled())
> - return housekeeping_mask;
> -#endif
> - return cpu_possible_mask;
> -}
> -
> -static inline bool is_housekeeping_cpu(int cpu)
> -{
> -#ifdef CONFIG_NO_HZ_FULL
> - if (tick_nohz_full_enabled())
> - return cpumask_test_cpu(cpu, housekeeping_mask);
> -#endif
> - return true;
> -}
> -
> -static inline void housekeeping_affine(struct task_struct *t)
> -{
> -#ifdef CONFIG_NO_HZ_FULL
> - if (tick_nohz_full_enabled())
> - set_cpus_allowed_ptr(t, housekeeping_mask);
> -
> -#endif
> -}
> -
> static inline void tick_nohz_task_switch(void)
> {
> if (tick_nohz_full_enabled())
> diff --git a/init/main.c b/init/main.c
> index 0ee9c686..bf138b9 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -46,6 +46,7 @@
> #include <linux/cgroup.h>
> #include <linux/efi.h>
> #include <linux/tick.h>
> +#include <linux/housekeeping.h>
> #include <linux/interrupt.h>
> #include <linux/taskstats_kern.h>
> #include <linux/delayacct.h>
> @@ -606,6 +607,7 @@ asmlinkage __visible void __init start_kernel(void)
> early_irq_init();
> init_IRQ();
> tick_init();
> + housekeeping_init();
> rcu_init_nohz();
> init_timers();
> hrtimers_init();
> diff --git a/kernel/Makefile b/kernel/Makefile
> index ed470aa..c63f893 100644
> --- a/kernel/Makefile
> +++ b/kernel/Makefile
> @@ -109,6 +109,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
> obj-$(CONFIG_JUMP_LABEL) += jump_label.o
> obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o
> obj-$(CONFIG_TORTURE_TEST) += torture.o
> +obj-$(CONFIG_NO_HZ_FULL) += housekeeping.o
>
> obj-$(CONFIG_HAS_IOMEM) += memremap.o
>
> diff --git a/kernel/housekeeping.c b/kernel/housekeeping.c
> new file mode 100644
> index 0000000..b41c52e
> --- /dev/null
> +++ b/kernel/housekeeping.c
> @@ -0,0 +1,33 @@
> +/*
> + * Housekeeping management. Manage the targets for routine code that can run on
> + * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
> + *
> + * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
> + *
> + */
> +
> +#include <linux/housekeeping.h>
> +#include <linux/tick.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +
> +cpumask_var_t housekeeping_mask;
> +
> +void __init housekeeping_init(void)
> +{
> + if (!tick_nohz_full_enabled())
> + return;
> +
> + if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
> + WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
> + cpumask_clear(tick_nohz_full_mask);
> + tick_nohz_full_running = false;
> + return;
> + }
> +
> + cpumask_andnot(housekeeping_mask,
> + cpu_possible_mask, tick_nohz_full_mask);
> +
> + /* We need at least one CPU to handle housekeeping work */
> + WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
> +}
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index e012b9b..387e0a2 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -29,6 +29,7 @@
> #include <linux/oom.h>
> #include <linux/sched/debug.h>
> #include <linux/smpboot.h>
> +#include <linux/housekeeping.h>
> #include <uapi/linux/sched/types.h>
> #include "../time/tick-internal.h"
>
> diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
> index 5033b66..1c003e2 100644
> --- a/kernel/rcu/update.c
> +++ b/kernel/rcu/update.c
> @@ -51,6 +51,7 @@
> #include <linux/kthread.h>
> #include <linux/tick.h>
> #include <linux/rcupdate_wait.h>
> +#include <linux/housekeeping.h>
>
> #define CREATE_TRACE_POINTS
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index d17c5da..4cc01a7 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -26,6 +26,7 @@
> #include <linux/profile.h>
> #include <linux/security.h>
> #include <linux/syscalls.h>
> +#include <linux/housekeeping.h>
>
> #include <asm/switch_to.h>
> #include <asm/tlb.h>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d3f3094..c76fab6 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -32,6 +32,7 @@
> #include <linux/mempolicy.h>
> #include <linux/migrate.h>
> #include <linux/task_work.h>
> +#include <linux/housekeeping.h>
>
> #include <trace/events/sched.h>
>
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index c7a899c..9d29dee 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -165,7 +165,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
>
> #ifdef CONFIG_NO_HZ_FULL
> cpumask_var_t tick_nohz_full_mask;
> -cpumask_var_t housekeeping_mask;
> bool tick_nohz_full_running;
> static atomic_t tick_dep_mask;
>
> @@ -437,13 +436,6 @@ void __init tick_nohz_init(void)
> return;
> }
>
> - if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
> - WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
> - cpumask_clear(tick_nohz_full_mask);
> - tick_nohz_full_running = false;
> - return;
> - }
> -
> /*
> * Full dynticks uses irq work to drive the tick rescheduling on safe
> * locking contexts. But then we need irq work to raise its own
> @@ -452,7 +444,6 @@ void __init tick_nohz_init(void)
> if (!arch_irq_work_has_interrupt()) {
> pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
> cpumask_clear(tick_nohz_full_mask);
> - cpumask_copy(housekeeping_mask, cpu_possible_mask);
> tick_nohz_full_running = false;
> return;
> }
> @@ -465,9 +456,6 @@ void __init tick_nohz_init(void)
> cpumask_clear_cpu(cpu, tick_nohz_full_mask);
> }
>
> - cpumask_andnot(housekeeping_mask,
> - cpu_possible_mask, tick_nohz_full_mask);
> -
> for_each_cpu(cpu, tick_nohz_full_mask)
> context_tracking_cpu_set(cpu);
>
> @@ -477,12 +465,6 @@ void __init tick_nohz_init(void)
> WARN_ON(ret < 0);
> pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
> cpumask_pr_args(tick_nohz_full_mask));
> -
> - /*
> - * We need at least one CPU to handle housekeeping work such
> - * as timekeeping, unbound timers, workqueues, ...
> - */
> - WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
> }
> #endif
>
> diff --git a/kernel/watchdog.c b/kernel/watchdog.c
> index 6bcb854..d71eee0 100644
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -24,6 +24,7 @@
> #include <linux/workqueue.h>
> #include <linux/sched/clock.h>
> #include <linux/sched/debug.h>
> +#include <linux/housekeeping.h>
>
> #include <asm/irq_regs.h>
> #include <linux/kvm_para.h>
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH 01/12] housekeeping: Move housekeeping related code to its own file
2017-10-24 13:06 [GIT PULL] Introduce housekeeping subsystem v5 Frederic Weisbecker
@ 2017-10-24 13:06 ` Frederic Weisbecker
2017-10-24 13:35 ` Paul E. McKenney
0 siblings, 1 reply; 42+ messages in thread
From: Frederic Weisbecker @ 2017-10-24 13:06 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Chris Metcalf,
Thomas Gleixner, Luiz Capitulino, Christoph Lameter,
Paul E . McKenney, Mike Galbraith, Rik van Riel, Wanpeng Li
The housekeeping code is currently tied to the nohz code. As we are
planning to make housekeeping independant from it, start with moving
the relevant code to its own file.
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
drivers/net/ethernet/tile/tilegx.c | 2 +-
include/linux/housekeeping.h | 56 ++++++++++++++++++++++++++++++++++++++
include/linux/tick.h | 37 -------------------------
init/main.c | 2 ++
kernel/Makefile | 1 +
kernel/housekeeping.c | 33 ++++++++++++++++++++++
kernel/rcu/tree_plugin.h | 1 +
kernel/rcu/update.c | 1 +
kernel/sched/core.c | 1 +
kernel/sched/fair.c | 1 +
kernel/time/tick-sched.c | 18 ------------
kernel/watchdog.c | 1 +
12 files changed, 98 insertions(+), 56 deletions(-)
create mode 100644 include/linux/housekeeping.h
create mode 100644 kernel/housekeeping.c
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index c00102b..8c7ef12 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -40,7 +40,7 @@
#include <linux/tcp.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
-#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <asm/checksum.h>
#include <asm/homecache.h>
diff --git a/include/linux/housekeeping.h b/include/linux/housekeeping.h
new file mode 100644
index 0000000..3d6a8e6
--- /dev/null
+++ b/include/linux/housekeeping.h
@@ -0,0 +1,56 @@
+#ifndef _LINUX_HOUSEKEEPING_H
+#define _LINUX_HOUSEKEEPING_H
+
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/tick.h>
+
+#ifdef CONFIG_NO_HZ_FULL
+extern cpumask_var_t housekeeping_mask;
+
+static inline int housekeeping_any_cpu(void)
+{
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+}
+
+extern void __init housekeeping_init(void);
+
+#else
+
+static inline int housekeeping_any_cpu(void)
+{
+ return smp_processor_id();
+}
+
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
+
+static inline const struct cpumask *housekeeping_cpumask(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+#endif
+ return cpu_possible_mask;
+}
+
+static inline bool is_housekeeping_cpu(int cpu)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+#endif
+ return true;
+}
+
+static inline void housekeeping_affine(struct task_struct *t)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+
+#endif
+}
+
+#endif /* _LINUX_HOUSEKEEPING_H */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index fe01e68..68afc09 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
extern cpumask_var_t tick_nohz_full_mask;
-extern cpumask_var_t housekeeping_mask;
static inline bool tick_nohz_full_enabled(void)
{
@@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
cpumask_or(mask, mask, tick_nohz_full_mask);
}
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void tick_nohz_dep_set(enum tick_dep_bits bit);
extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
@@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
#else
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
@@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
#endif
-static inline const struct cpumask *housekeeping_cpumask(void)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
- return cpu_possible_mask;
-}
-
-static inline bool is_housekeeping_cpu(int cpu)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
-#endif
- return true;
-}
-
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
static inline void tick_nohz_task_switch(void)
{
if (tick_nohz_full_enabled())
diff --git a/init/main.c b/init/main.c
index 0ee9c686..bf138b9 100644
--- a/init/main.c
+++ b/init/main.c
@@ -46,6 +46,7 @@
#include <linux/cgroup.h>
#include <linux/efi.h>
#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <linux/interrupt.h>
#include <linux/taskstats_kern.h>
#include <linux/delayacct.h>
@@ -606,6 +607,7 @@ asmlinkage __visible void __init start_kernel(void)
early_irq_init();
init_IRQ();
tick_init();
+ housekeeping_init();
rcu_init_nohz();
init_timers();
hrtimers_init();
diff --git a/kernel/Makefile b/kernel/Makefile
index ed470aa..c63f893 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -109,6 +109,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
obj-$(CONFIG_JUMP_LABEL) += jump_label.o
obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o
obj-$(CONFIG_TORTURE_TEST) += torture.o
+obj-$(CONFIG_NO_HZ_FULL) += housekeeping.o
obj-$(CONFIG_HAS_IOMEM) += memremap.o
diff --git a/kernel/housekeeping.c b/kernel/housekeeping.c
new file mode 100644
index 0000000..b41c52e
--- /dev/null
+++ b/kernel/housekeeping.c
@@ -0,0 +1,33 @@
+/*
+ * Housekeeping management. Manage the targets for routine code that can run on
+ * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
+ *
+ * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ *
+ */
+
+#include <linux/housekeeping.h>
+#include <linux/tick.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+
+cpumask_var_t housekeeping_mask;
+
+void __init housekeeping_init(void)
+{
+ if (!tick_nohz_full_enabled())
+ return;
+
+ if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
+ WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
+ cpumask_clear(tick_nohz_full_mask);
+ tick_nohz_full_running = false;
+ return;
+ }
+
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, tick_nohz_full_mask);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e012b9b..387e0a2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -29,6 +29,7 @@
#include <linux/oom.h>
#include <linux/sched/debug.h>
#include <linux/smpboot.h>
+#include <linux/housekeeping.h>
#include <uapi/linux/sched/types.h>
#include "../time/tick-internal.h"
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 5033b66..1c003e2 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -51,6 +51,7 @@
#include <linux/kthread.h>
#include <linux/tick.h>
#include <linux/rcupdate_wait.h>
+#include <linux/housekeeping.h>
#define CREATE_TRACE_POINTS
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index d17c5da..4cc01a7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -26,6 +26,7 @@
#include <linux/profile.h>
#include <linux/security.h>
#include <linux/syscalls.h>
+#include <linux/housekeeping.h>
#include <asm/switch_to.h>
#include <asm/tlb.h>
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d3f3094..c76fab6 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -32,6 +32,7 @@
#include <linux/mempolicy.h>
#include <linux/migrate.h>
#include <linux/task_work.h>
+#include <linux/housekeeping.h>
#include <trace/events/sched.h>
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index c7a899c..9d29dee 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -165,7 +165,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
#ifdef CONFIG_NO_HZ_FULL
cpumask_var_t tick_nohz_full_mask;
-cpumask_var_t housekeeping_mask;
bool tick_nohz_full_running;
static atomic_t tick_dep_mask;
@@ -437,13 +436,6 @@ void __init tick_nohz_init(void)
return;
}
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
/*
* Full dynticks uses irq work to drive the tick rescheduling on safe
* locking contexts. But then we need irq work to raise its own
@@ -452,7 +444,6 @@ void __init tick_nohz_init(void)
if (!arch_irq_work_has_interrupt()) {
pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
cpumask_clear(tick_nohz_full_mask);
- cpumask_copy(housekeeping_mask, cpu_possible_mask);
tick_nohz_full_running = false;
return;
}
@@ -465,9 +456,6 @@ void __init tick_nohz_init(void)
cpumask_clear_cpu(cpu, tick_nohz_full_mask);
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
for_each_cpu(cpu, tick_nohz_full_mask)
context_tracking_cpu_set(cpu);
@@ -477,12 +465,6 @@ void __init tick_nohz_init(void)
WARN_ON(ret < 0);
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
-
- /*
- * We need at least one CPU to handle housekeeping work such
- * as timekeeping, unbound timers, workqueues, ...
- */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
#endif
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 6bcb854..d71eee0 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -24,6 +24,7 @@
#include <linux/workqueue.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
+#include <linux/housekeeping.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 01/12] housekeeping: Move housekeeping related code to its own file
2017-09-18 13:53 [GIT PULL] Introduce housekeeping subsystem v4 Frederic Weisbecker
@ 2017-09-18 13:53 ` Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: Frederic Weisbecker @ 2017-09-18 13:53 UTC (permalink / raw)
To: Ingo Molnar
Cc: LKML, Frederic Weisbecker, Peter Zijlstra, Chris Metcalf,
Thomas Gleixner, Luiz Capitulino, Christoph Lameter,
Paul E . McKenney, Mike Galbraith, Rik van Riel, Wanpeng Li
The housekeeping code is currently tied to the nohz code. As we are
planning to make housekeeping independant from it, start with moving
the relevant code to its own file.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
drivers/net/ethernet/tile/tilegx.c | 2 +-
include/linux/housekeeping.h | 56 ++++++++++++++++++++++++++++++++++++++
include/linux/tick.h | 37 -------------------------
init/main.c | 2 ++
kernel/Makefile | 1 +
kernel/housekeeping.c | 33 ++++++++++++++++++++++
kernel/rcu/tree_plugin.h | 1 +
kernel/rcu/update.c | 1 +
kernel/sched/core.c | 1 +
kernel/sched/fair.c | 1 +
kernel/time/tick-sched.c | 18 ------------
kernel/watchdog.c | 1 +
12 files changed, 98 insertions(+), 56 deletions(-)
create mode 100644 include/linux/housekeeping.h
create mode 100644 kernel/housekeeping.c
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index c00102b..8c7ef12 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -40,7 +40,7 @@
#include <linux/tcp.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
-#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <asm/checksum.h>
#include <asm/homecache.h>
diff --git a/include/linux/housekeeping.h b/include/linux/housekeeping.h
new file mode 100644
index 0000000..3d6a8e6
--- /dev/null
+++ b/include/linux/housekeeping.h
@@ -0,0 +1,56 @@
+#ifndef _LINUX_HOUSEKEEPING_H
+#define _LINUX_HOUSEKEEPING_H
+
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/tick.h>
+
+#ifdef CONFIG_NO_HZ_FULL
+extern cpumask_var_t housekeeping_mask;
+
+static inline int housekeeping_any_cpu(void)
+{
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+}
+
+extern void __init housekeeping_init(void);
+
+#else
+
+static inline int housekeeping_any_cpu(void)
+{
+ return smp_processor_id();
+}
+
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
+
+static inline const struct cpumask *housekeeping_cpumask(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+#endif
+ return cpu_possible_mask;
+}
+
+static inline bool is_housekeeping_cpu(int cpu)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+#endif
+ return true;
+}
+
+static inline void housekeeping_affine(struct task_struct *t)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+
+#endif
+}
+
+#endif /* _LINUX_HOUSEKEEPING_H */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index fe01e68..68afc09 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
extern cpumask_var_t tick_nohz_full_mask;
-extern cpumask_var_t housekeeping_mask;
static inline bool tick_nohz_full_enabled(void)
{
@@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
cpumask_or(mask, mask, tick_nohz_full_mask);
}
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void tick_nohz_dep_set(enum tick_dep_bits bit);
extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
@@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
#else
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
@@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
#endif
-static inline const struct cpumask *housekeeping_cpumask(void)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
- return cpu_possible_mask;
-}
-
-static inline bool is_housekeeping_cpu(int cpu)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
-#endif
- return true;
-}
-
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
static inline void tick_nohz_task_switch(void)
{
if (tick_nohz_full_enabled())
diff --git a/init/main.c b/init/main.c
index 0ee9c686..bf138b9 100644
--- a/init/main.c
+++ b/init/main.c
@@ -46,6 +46,7 @@
#include <linux/cgroup.h>
#include <linux/efi.h>
#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <linux/interrupt.h>
#include <linux/taskstats_kern.h>
#include <linux/delayacct.h>
@@ -606,6 +607,7 @@ asmlinkage __visible void __init start_kernel(void)
early_irq_init();
init_IRQ();
tick_init();
+ housekeeping_init();
rcu_init_nohz();
init_timers();
hrtimers_init();
diff --git a/kernel/Makefile b/kernel/Makefile
index ed470aa..c63f893 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -109,6 +109,7 @@ obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
obj-$(CONFIG_JUMP_LABEL) += jump_label.o
obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o
obj-$(CONFIG_TORTURE_TEST) += torture.o
+obj-$(CONFIG_NO_HZ_FULL) += housekeeping.o
obj-$(CONFIG_HAS_IOMEM) += memremap.o
diff --git a/kernel/housekeeping.c b/kernel/housekeeping.c
new file mode 100644
index 0000000..b41c52e
--- /dev/null
+++ b/kernel/housekeeping.c
@@ -0,0 +1,33 @@
+/*
+ * Housekeeping management. Manage the targets for routine code that can run on
+ * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
+ *
+ * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ *
+ */
+
+#include <linux/housekeeping.h>
+#include <linux/tick.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+
+cpumask_var_t housekeeping_mask;
+
+void __init housekeeping_init(void)
+{
+ if (!tick_nohz_full_enabled())
+ return;
+
+ if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
+ WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
+ cpumask_clear(tick_nohz_full_mask);
+ tick_nohz_full_running = false;
+ return;
+ }
+
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, tick_nohz_full_mask);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index e012b9b..387e0a2 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -29,6 +29,7 @@
#include <linux/oom.h>
#include <linux/sched/debug.h>
#include <linux/smpboot.h>
+#include <linux/housekeeping.h>
#include <uapi/linux/sched/types.h>
#include "../time/tick-internal.h"
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 5033b66..1c003e2 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -51,6 +51,7 @@
#include <linux/kthread.h>
#include <linux/tick.h>
#include <linux/rcupdate_wait.h>
+#include <linux/housekeeping.h>
#define CREATE_TRACE_POINTS
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 18a6966..b9769d1 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -26,6 +26,7 @@
#include <linux/profile.h>
#include <linux/security.h>
#include <linux/syscalls.h>
+#include <linux/housekeeping.h>
#include <asm/switch_to.h>
#include <asm/tlb.h>
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 70ba32e..3af0630 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -32,6 +32,7 @@
#include <linux/mempolicy.h>
#include <linux/migrate.h>
#include <linux/task_work.h>
+#include <linux/housekeeping.h>
#include <trace/events/sched.h>
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index c7a899c..9d29dee 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -165,7 +165,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
#ifdef CONFIG_NO_HZ_FULL
cpumask_var_t tick_nohz_full_mask;
-cpumask_var_t housekeeping_mask;
bool tick_nohz_full_running;
static atomic_t tick_dep_mask;
@@ -437,13 +436,6 @@ void __init tick_nohz_init(void)
return;
}
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
/*
* Full dynticks uses irq work to drive the tick rescheduling on safe
* locking contexts. But then we need irq work to raise its own
@@ -452,7 +444,6 @@ void __init tick_nohz_init(void)
if (!arch_irq_work_has_interrupt()) {
pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
cpumask_clear(tick_nohz_full_mask);
- cpumask_copy(housekeeping_mask, cpu_possible_mask);
tick_nohz_full_running = false;
return;
}
@@ -465,9 +456,6 @@ void __init tick_nohz_init(void)
cpumask_clear_cpu(cpu, tick_nohz_full_mask);
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
for_each_cpu(cpu, tick_nohz_full_mask)
context_tracking_cpu_set(cpu);
@@ -477,12 +465,6 @@ void __init tick_nohz_init(void)
WARN_ON(ret < 0);
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
-
- /*
- * We need at least one CPU to handle housekeeping work such
- * as timekeeping, unbound timers, workqueues, ...
- */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
#endif
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index f5d5202..3cc5596 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -24,6 +24,7 @@
#include <linux/workqueue.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
+#include <linux/housekeeping.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH 01/12] housekeeping: Move housekeeping related code to its own file
2017-09-01 16:41 [PATCH 00/12] Introduce housekeeping subsystem v3 Frederic Weisbecker
@ 2017-09-01 16:41 ` Frederic Weisbecker
0 siblings, 0 replies; 42+ messages in thread
From: Frederic Weisbecker @ 2017-09-01 16:41 UTC (permalink / raw)
To: LKML
Cc: Frederic Weisbecker, Peter Zijlstra, Chris Metcalf,
Thomas Gleixner, Luiz Capitulino, Christoph Lameter,
Paul E . McKenney, Ingo Molnar, Mike Galbraith, Rik van Riel,
Wanpeng Li
The housekeeping code is currently tied to the nohz code. As we are
planning to make housekeeping independant from it, start with moving
the relevant code to its own file.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
---
drivers/net/ethernet/tile/tilegx.c | 2 +-
include/linux/housekeeping.h | 56 ++++++++++++++++++++++++++++++++++++++
include/linux/tick.h | 37 -------------------------
init/main.c | 2 ++
kernel/Makefile | 1 +
kernel/housekeeping.c | 32 ++++++++++++++++++++++
kernel/rcu/tree_plugin.h | 1 +
kernel/rcu/update.c | 1 +
kernel/sched/core.c | 1 +
kernel/sched/fair.c | 1 +
kernel/time/tick-sched.c | 18 ------------
kernel/watchdog.c | 1 +
12 files changed, 97 insertions(+), 56 deletions(-)
create mode 100644 include/linux/housekeeping.h
create mode 100644 kernel/housekeeping.c
diff --git a/drivers/net/ethernet/tile/tilegx.c b/drivers/net/ethernet/tile/tilegx.c
index aec9538..eb74e09 100644
--- a/drivers/net/ethernet/tile/tilegx.c
+++ b/drivers/net/ethernet/tile/tilegx.c
@@ -40,7 +40,7 @@
#include <linux/tcp.h>
#include <linux/net_tstamp.h>
#include <linux/ptp_clock_kernel.h>
-#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <asm/checksum.h>
#include <asm/homecache.h>
diff --git a/include/linux/housekeeping.h b/include/linux/housekeeping.h
new file mode 100644
index 0000000..3d6a8e6
--- /dev/null
+++ b/include/linux/housekeeping.h
@@ -0,0 +1,56 @@
+#ifndef _LINUX_HOUSEKEEPING_H
+#define _LINUX_HOUSEKEEPING_H
+
+#include <linux/cpumask.h>
+#include <linux/init.h>
+#include <linux/tick.h>
+
+#ifdef CONFIG_NO_HZ_FULL
+extern cpumask_var_t housekeeping_mask;
+
+static inline int housekeeping_any_cpu(void)
+{
+ return cpumask_any_and(housekeeping_mask, cpu_online_mask);
+}
+
+extern void __init housekeeping_init(void);
+
+#else
+
+static inline int housekeeping_any_cpu(void)
+{
+ return smp_processor_id();
+}
+
+static inline void housekeeping_init(void) { }
+#endif /* CONFIG_NO_HZ_FULL */
+
+
+static inline const struct cpumask *housekeeping_cpumask(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return housekeeping_mask;
+#endif
+ return cpu_possible_mask;
+}
+
+static inline bool is_housekeeping_cpu(int cpu)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ return cpumask_test_cpu(cpu, housekeeping_mask);
+#endif
+ return true;
+}
+
+static inline void housekeeping_affine(struct task_struct *t)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_enabled())
+ set_cpus_allowed_ptr(t, housekeeping_mask);
+
+#endif
+}
+
+#endif /* _LINUX_HOUSEKEEPING_H */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index fe01e68..68afc09 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -137,7 +137,6 @@ static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
extern cpumask_var_t tick_nohz_full_mask;
-extern cpumask_var_t housekeeping_mask;
static inline bool tick_nohz_full_enabled(void)
{
@@ -161,11 +160,6 @@ static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask)
cpumask_or(mask, mask, tick_nohz_full_mask);
}
-static inline int housekeeping_any_cpu(void)
-{
- return cpumask_any_and(housekeeping_mask, cpu_online_mask);
-}
-
extern void tick_nohz_dep_set(enum tick_dep_bits bit);
extern void tick_nohz_dep_clear(enum tick_dep_bits bit);
extern void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit);
@@ -235,10 +229,6 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
#else
-static inline int housekeeping_any_cpu(void)
-{
- return smp_processor_id();
-}
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
static inline void tick_nohz_full_add_cpus_to(struct cpumask *mask) { }
@@ -260,33 +250,6 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
#endif
-static inline const struct cpumask *housekeeping_cpumask(void)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return housekeeping_mask;
-#endif
- return cpu_possible_mask;
-}
-
-static inline bool is_housekeeping_cpu(int cpu)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- return cpumask_test_cpu(cpu, housekeeping_mask);
-#endif
- return true;
-}
-
-static inline void housekeeping_affine(struct task_struct *t)
-{
-#ifdef CONFIG_NO_HZ_FULL
- if (tick_nohz_full_enabled())
- set_cpus_allowed_ptr(t, housekeeping_mask);
-
-#endif
-}
-
static inline void tick_nohz_task_switch(void)
{
if (tick_nohz_full_enabled())
diff --git a/init/main.c b/init/main.c
index 881d624..4047c85 100644
--- a/init/main.c
+++ b/init/main.c
@@ -46,6 +46,7 @@
#include <linux/cgroup.h>
#include <linux/efi.h>
#include <linux/tick.h>
+#include <linux/housekeeping.h>
#include <linux/interrupt.h>
#include <linux/taskstats_kern.h>
#include <linux/delayacct.h>
@@ -604,6 +605,7 @@ asmlinkage __visible void __init start_kernel(void)
early_irq_init();
init_IRQ();
tick_init();
+ housekeeping_init();
rcu_init_nohz();
init_timers();
hrtimers_init();
diff --git a/kernel/Makefile b/kernel/Makefile
index 4cb8e8b..8a85c4b 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -109,6 +109,7 @@ obj-$(CONFIG_JUMP_LABEL) += jump_label.o
obj-$(CONFIG_CONTEXT_TRACKING) += context_tracking.o
obj-$(CONFIG_TORTURE_TEST) += torture.o
obj-$(CONFIG_MEMBARRIER) += membarrier.o
+obj-$(CONFIG_NO_HZ_FULL) += housekeeping.o
obj-$(CONFIG_HAS_IOMEM) += memremap.o
diff --git a/kernel/housekeeping.c b/kernel/housekeeping.c
new file mode 100644
index 0000000..6d8afd5
--- /dev/null
+++ b/kernel/housekeeping.c
@@ -0,0 +1,32 @@
+/*
+ * Housekeeping management. Manage the targets for routine code that can run on
+ * any CPU: unbound workqueues, timers, kthreads and any offloadable work.
+ *
+ * Copyright (C) 2017 Red Hat, Inc., Frederic Weisbecker
+ *
+ */
+
+#include <linux/housekeeping.h>
+#include <linux/tick.h>
+#include <linux/init.h>
+
+cpumask_var_t housekeeping_mask;
+
+void __init housekeeping_init(void)
+{
+ if (!tick_nohz_full_enabled())
+ return;
+
+ if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
+ WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
+ cpumask_clear(tick_nohz_full_mask);
+ tick_nohz_full_running = false;
+ return;
+ }
+
+ cpumask_andnot(housekeeping_mask,
+ cpu_possible_mask, tick_nohz_full_mask);
+
+ /* We need at least one CPU to handle housekeeping work */
+ WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
+}
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 908b309..c66d162 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -29,6 +29,7 @@
#include <linux/oom.h>
#include <linux/sched/debug.h>
#include <linux/smpboot.h>
+#include <linux/housekeeping.h>
#include <uapi/linux/sched/types.h>
#include "../time/tick-internal.h"
diff --git a/kernel/rcu/update.c b/kernel/rcu/update.c
index 00e77c4..bfe973d 100644
--- a/kernel/rcu/update.c
+++ b/kernel/rcu/update.c
@@ -51,6 +51,7 @@
#include <linux/kthread.h>
#include <linux/tick.h>
#include <linux/rcupdate_wait.h>
+#include <linux/housekeeping.h>
#define CREATE_TRACE_POINTS
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index f9f9948..536d6a5 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -26,6 +26,7 @@
#include <linux/profile.h>
#include <linux/security.h>
#include <linux/syscalls.h>
+#include <linux/housekeeping.h>
#include <asm/switch_to.h>
#include <asm/tlb.h>
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 8d58687..5455e98 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -32,6 +32,7 @@
#include <linux/mempolicy.h>
#include <linux/migrate.h>
#include <linux/task_work.h>
+#include <linux/housekeeping.h>
#include <trace/events/sched.h>
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index c7a899c..9d29dee 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -165,7 +165,6 @@ static void tick_sched_handle(struct tick_sched *ts, struct pt_regs *regs)
#ifdef CONFIG_NO_HZ_FULL
cpumask_var_t tick_nohz_full_mask;
-cpumask_var_t housekeeping_mask;
bool tick_nohz_full_running;
static atomic_t tick_dep_mask;
@@ -437,13 +436,6 @@ void __init tick_nohz_init(void)
return;
}
- if (!alloc_cpumask_var(&housekeeping_mask, GFP_KERNEL)) {
- WARN(1, "NO_HZ: Can't allocate not-full dynticks cpumask\n");
- cpumask_clear(tick_nohz_full_mask);
- tick_nohz_full_running = false;
- return;
- }
-
/*
* Full dynticks uses irq work to drive the tick rescheduling on safe
* locking contexts. But then we need irq work to raise its own
@@ -452,7 +444,6 @@ void __init tick_nohz_init(void)
if (!arch_irq_work_has_interrupt()) {
pr_warn("NO_HZ: Can't run full dynticks because arch doesn't support irq work self-IPIs\n");
cpumask_clear(tick_nohz_full_mask);
- cpumask_copy(housekeeping_mask, cpu_possible_mask);
tick_nohz_full_running = false;
return;
}
@@ -465,9 +456,6 @@ void __init tick_nohz_init(void)
cpumask_clear_cpu(cpu, tick_nohz_full_mask);
}
- cpumask_andnot(housekeeping_mask,
- cpu_possible_mask, tick_nohz_full_mask);
-
for_each_cpu(cpu, tick_nohz_full_mask)
context_tracking_cpu_set(cpu);
@@ -477,12 +465,6 @@ void __init tick_nohz_init(void)
WARN_ON(ret < 0);
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
-
- /*
- * We need at least one CPU to handle housekeeping work such
- * as timekeeping, unbound timers, workqueues, ...
- */
- WARN_ON_ONCE(cpumask_empty(housekeeping_mask));
}
#endif
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 06d3389..7a9df162 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -24,6 +24,7 @@
#include <linux/workqueue.h>
#include <linux/sched/clock.h>
#include <linux/sched/debug.h>
+#include <linux/housekeeping.h>
#include <asm/irq_regs.h>
#include <linux/kvm_para.h>
--
2.7.4
^ permalink raw reply related [flat|nested] 42+ messages in thread
end of thread, other threads:[~2017-10-30 16:58 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-27 2:42 [GIT PULL] Housekeeping subsystem v6 Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
2017-10-27 12:01 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 02/12] watchdog: Use housekeeping_cpumask() instead of ad-hoc version Frederic Weisbecker
2017-10-27 12:02 ` [tip:sched/core] sched/isolation, " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 03/12] housekeeping: Provide a dynamic off-case to housekeeping_any_cpu() Frederic Weisbecker
2017-10-27 12:02 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 04/12] housekeeping: Make housekeeping cpumask private Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: Make the " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 05/12] housekeeping: Use its own static key Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 06/12] housekeeping: Rename is_housekeeping_cpu to housekeeping_cpu Frederic Weisbecker
2017-10-27 12:03 ` [tip:sched/core] sched/isolation: Rename is_housekeeping_cpu() to housekeeping_cpu() tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 07/12] housekeeping: Move it under its own config, independent from NO_HZ Frederic Weisbecker
2017-10-27 12:04 ` [tip:sched/core] sched/isolation: Split out new CONFIG_CPU_ISOLATION=y config from CONFIG_NO_HZ_FULL tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 08/12] housekeeping: Introduce housekeeping flags Frederic Weisbecker
2017-10-27 12:04 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 09/12] housekeeping: Handle the nohz_full= parameter Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 10/12] housekeeping: Move isolcpus to housekeeping Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: Move isolcpus= handling to the housekeeping code tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 11/12] housekeeping: Add basic isolcpus flags Frederic Weisbecker
2017-10-27 12:05 ` [tip:sched/core] sched/isolation: " tip-bot for Frederic Weisbecker
2017-10-27 2:42 ` [PATCH 12/12] housekeeping: Document " Frederic Weisbecker
2017-10-27 12:06 ` [tip:sched/core] sched/isolation: Document the isolcpus= flags tip-bot for Frederic Weisbecker
2017-10-27 13:58 ` Peter Zijlstra
2017-10-27 14:36 ` Frederic Weisbecker
2017-10-27 17:06 ` Ingo Molnar
2017-10-27 17:33 ` Frederic Weisbecker
2017-10-27 18:21 ` Ingo Molnar
2017-10-27 18:39 ` Frederic Weisbecker
2017-10-27 19:04 ` Ingo Molnar
2017-10-27 14:38 ` Mike Galbraith
2017-10-30 15:48 ` Christopher Lameter
2017-10-30 16:11 ` Peter Zijlstra
2017-10-30 16:30 ` Christopher Lameter
2017-10-30 16:58 ` Peter Zijlstra
2017-10-30 16:56 ` Mike Galbraith
-- strict thread matches above, loose matches on Subject: below --
2017-10-24 13:06 [GIT PULL] Introduce housekeeping subsystem v5 Frederic Weisbecker
2017-10-24 13:06 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
2017-10-24 13:35 ` Paul E. McKenney
2017-09-18 13:53 [GIT PULL] Introduce housekeeping subsystem v4 Frederic Weisbecker
2017-09-18 13:53 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
2017-09-01 16:41 [PATCH 00/12] Introduce housekeeping subsystem v3 Frederic Weisbecker
2017-09-01 16:41 ` [PATCH 01/12] housekeeping: Move housekeeping related code to its own file Frederic Weisbecker
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).