All of lore.kernel.org
 help / color / mirror / Atom feed
* [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework
@ 2016-02-26 18:43 Thomas Gleixner
  2016-02-26 18:43 ` [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner
                   ` (19 more replies)
  0 siblings, 20 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

Hi folks!

the following series contains the core infrastructure of our ongoing cpu
hotplug refactoring work. It's a follow up to the work I've done in 2013

	https://lwn.net/Articles/535764/

What's wrong with the current cpu hotplug infrastructure?

 - Asymmetry

   The hotplug notifier mechanism is asymmetric versus the bringup and
   teardown. This is mostly caused by the notifier mechanism.

 - Largely undocumented dependencies

   While some notifiers use explicitely defined notifier priorities, we have
   quite some notifiers which use numerical priorities to express
   dependencies without any documentation why.

 - Control processor driven

   Most of the bringup/teardown of a cpu is driven by a control
   processor. While it is understandable, that preperatory steps, like idle
   thread creation, memory allocation for and initialization of essential
   facilities needs to be done before a cpu can boot, there is no reason why
   everything else must run on a control processor. Todays bringup looks like
   this:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
   bring the rest up


 - All or nothing approach

   There is no way to do partial bringups. That's something which is really
   desired because we waste e.g. at boot substantial amount of time just busy
   waiting that the cpu comes to life. That's stupid as we could very well do
   preparatory steps and the initial IPI for other cpus and then go back and
   do the necessary low level synchronization with the freshly booted cpu.

 - Minimal debuggability

   Due to the notifier based design, it's impossible to switch between two
   stages of the bringup/teardown back and forth in order to test the
   correctness. So in many hotplug notifiers the cancel mechanisms are either
   not existant or completely untested.

 - Notifier [un]registering is tidious

   To [un]register notifiers we need to protect against hotplug at every
   callsite. There is no mechanism that bringup/teardown callbacks are issued
   on the online cpus, so every caller needs to do it itself. That also
   includes error rollback.


What's the new design?

   The base of the new design is a symmetric state machine, where both the
   control processor and the booting/dying cpu execute a well defined set of
   states. Each state is symmetric in the end, except for some well defined
   exceptions, and the bringup/teardown can be stopped and reversed at almost
   all states.

   So the bringup of a cpu will look like this in the future:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
				bring itself up

   The synchronization step does not require the control cpu to wait. That
   mechanism can be done asynchronously via a worker or some other mechanism.

   The teardown can be made very similar, so that the dying cpu cleans up and
   brings itself down. Cleanups which need to be done after the cpu is gone,
   can be scheduled asynchronously as well.

There is a long way to this, as we need to refactor the notion when a cpu is
available. Today we set the cpu online right after it comes out of the low
level bringup, which is not really correct.

The proper mechanism is to set it to available, i.e. cpu local threads, like
softirqd, hotplug thread etc. can be scheduled on that cpu, and once it
finished all booting steps, it's set to online, so general workloads can be
scheduled on it. The reverse happens on teardown. First thing to do is to
forbid scheduling of general workloads, then teardown all the per cpu
resources and finally shut it off completely.


The following patch series implements the basic infrastructure for this at the
core level. This includes the following:

 - Basic state machine implementation with well defined states, so ordering
   and prioritization can be expressed.

 - Interfaces to [un]register state callbacks

   This invokes the bringup/teardown callback on all online cpus with the
   proper protection in place and [un]installs the callbacks in the state
   machine array.

   For callbacks which have no particular ordering requirement we have a
   dynamic state space, so that drivers don't have to register an explicit
   hotplug state.

   If a callback fails, the code automatically does a rollback to the previous
   state.


 - Sysfs interface to drive the state machine to a particular step.

   This is only partially functional today. Full functionality and therefor
   testability will be achieved once we converted all existing hotplug
   notifiers over to the new scheme.


 - Run all CPU_ONLINE/DOWN_PREPARE notifiers on the booting/dying processor:

    Control CPU  	   	Booting CPU

    do preparatory steps
    kick cpu into life

				do low level init

    sync with booting cpu	sync with control cpu
    wait for boot     		
				bring itself up

    				Signal completion to control cpu
				
In a previous step of this work we've done a full tree mechanical conversion
of all hotplug notifiers to the new scheme. The balance is a net removal of
about 4000 lines of code.

This is not included in this post, as we decided to take a different
approach. Instead of mechanically converting everything over, we will do a
proper overhaul of the usage sites one by one so they nicely fit into the
symmetric callback scheme.

I decided to do that after I looked at the ugliness of some of the converted
sites and figured out that their hotplug mechanism is completely buggered
anyway. So there is no point to do a mechanical conversion first as we need to
go through the usage sites one by one again in order to achieve a full
symmetric and testable behaviour.

The lot is also available at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.hotplug

Thanks,

	tglx

---
 arch/ia64/kernel/smpboot.c        |    2 
 arch/metag/kernel/smp.c           |    2 
 arch/sparc/kernel/smp_32.c        |    2 
 arch/sparc/kernel/smp_64.c        |    2 
 arch/tile/kernel/smpboot.c        |    2 
 arch/x86/kernel/process.c         |   12 
 arch/x86/xen/smp.c                |    2 
 b/arch/alpha/kernel/smp.c         |    2 
 b/arch/arc/kernel/smp.c           |    2 
 b/arch/arm/kernel/smp.c           |    2 
 b/arch/arm64/kernel/smp.c         |    2 
 b/arch/blackfin/mach-common/smp.c |    2 
 b/arch/hexagon/kernel/smp.c       |    2 
 b/arch/m32r/kernel/smpboot.c      |    2 
 b/arch/mips/kernel/smp.c          |    2 
 b/arch/mn10300/kernel/smp.c       |    2 
 b/arch/parisc/kernel/smp.c        |    2 
 b/arch/powerpc/kernel/smp.c       |    2 
 b/arch/s390/kernel/smp.c          |    2 
 b/arch/sh/kernel/smp.c            |    2 
 b/arch/x86/kernel/smpboot.c       |    2 
 b/arch/xtensa/kernel/smp.c        |    2 
 b/include/linux/cpuhotplug.h      |   93 +++
 b/include/trace/events/cpuhp.h    |   66 ++
 include/linux/cpu.h               |   27 
 include/linux/notifier.h          |    2 
 include/linux/rcupdate.h          |    4 
 init/main.c                       |   16 
 kernel/cpu.c                      | 1099 +++++++++++++++++++++++++++++++++-----
 kernel/rcu/tree.c                 |   26 
 kernel/sched/core.c               |   10 
 kernel/sched/idle.c               |   24 
 kernel/smp.c                      |    1 
 kernel/smpboot.c                  |    6 
 kernel/smpboot.h                  |    6 
 lib/Kconfig.debug                 |   13 
 36 files changed, 1217 insertions(+), 230 deletions(-)







   

   

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 01/20] idle: Move x86ism out of generic code
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-02-27 20:29   ` Brian Gerst
  2016-02-26 18:43 ` [patch 02/20] cpu/hotplug: Restructure FROZEN state handling Thomas Gleixner
                   ` (18 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: idle--Move-x86ism-out-of-generic-code.patch --]
[-- Type: text/plain, Size: 1800 bytes --]

We have an arch specific callback here already.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/process.c |   12 ++++++++++++
 kernel/sched/idle.c       |   15 ---------------
 2 files changed, 12 insertions(+), 15 deletions(-)

Index: b/arch/x86/kernel/process.c
===================================================================
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -271,6 +271,18 @@ void exit_idle(void)
 }
 #endif
 
+void arch_cpu_idle_prepare(void)
+{
+	/*
+	 * If we're the non-boot CPU, nothing set the stack canary up
+	 * for us. The boot CPU already has it initialized but no harm
+	 * in doing it again. This is a good place for updating it, as
+	 * we wont ever return from this function (so the invalid
+	 * canaries already on the stack wont ever trigger).
+	 */
+	boot_init_stack_canary();
+}
+
 void arch_cpu_idle_enter(void)
 {
 	local_touch_nmi();
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -275,21 +275,6 @@ static void cpu_idle_loop(void)
 
 void cpu_startup_entry(enum cpuhp_state state)
 {
-	/*
-	 * This #ifdef needs to die, but it's too late in the cycle to
-	 * make this generic (arm and sh have never invoked the canary
-	 * init for the non boot cpus!). Will be fixed in 3.11
-	 */
-#ifdef CONFIG_X86
-	/*
-	 * If we're the non-boot CPU, nothing set the stack canary up
-	 * for us. The boot CPU already has it initialized but no harm
-	 * in doing it again. This is a good place for updating it, as
-	 * we wont ever return from this function (so the invalid
-	 * canaries already on the stack wont ever trigger).
-	 */
-	boot_init_stack_canary();
-#endif
 	arch_cpu_idle_prepare();
 	cpu_idle_loop();
 }

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 02/20] cpu/hotplug: Restructure FROZEN state handling
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
  2016-02-26 18:43 ` [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:51   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 03/20] cpu/hotplug: Restructure cpu_up code Thomas Gleixner
                   ` (17 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-Restructure-FROZEN-state-handling.patch --]
[-- Type: text/plain, Size: 6488 bytes --]

There are only a few callbacks which really care about FROZEN
vs. !FROZEN. No need to have extra states for this.

Publish the frozen state in an extra variable which is updated under
the hotplug lock and let the users interested deal with it w/o
imposing that extra state checks on everyone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h |    2 +
 kernel/cpu.c        |   69 +++++++++++++++++++++-------------------------------
 2 files changed, 31 insertions(+), 40 deletions(-)

--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -118,6 +118,7 @@ enum {
 
 
 #ifdef CONFIG_SMP
+extern bool cpuhp_tasks_frozen;
 /* Need to know about CPUs going up/down? */
 #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
 #define cpu_notifier(fn, pri) {					\
@@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
 #define cpu_notifier_register_done	cpu_maps_update_done
 
 #else	/* CONFIG_SMP */
+#define cpuhp_tasks_frozen	0
 
 #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
 #define __cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -29,6 +29,8 @@
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
+bool cpuhp_tasks_frozen;
+EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
 
 /*
  * The following two APIs (cpu_maps_update_begin/done) must be used when
@@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notif
 	return raw_notifier_chain_register(&cpu_chain, nb);
 }
 
-static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
+static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
 			int *nr_calls)
 {
+	unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
+	void *hcpu = (void *)(long)cpu;
+
 	int ret;
 
-	ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
+	ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
 					nr_calls);
 
 	return notifier_to_errno(ret);
 }
 
-static int cpu_notify(unsigned long val, void *v)
+static int cpu_notify(unsigned long val, unsigned int cpu)
 {
-	return __cpu_notify(val, v, -1, NULL);
+	return __cpu_notify(val, cpu, -1, NULL);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
 
-static void cpu_notify_nofail(unsigned long val, void *v)
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
 {
-	BUG_ON(cpu_notify(val, v));
+	BUG_ON(cpu_notify(val, cpu));
 }
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -311,27 +316,21 @@ static inline void check_for_tasks(int d
 	read_unlock(&tasklist_lock);
 }
 
-struct take_cpu_down_param {
-	unsigned long mod;
-	void *hcpu;
-};
-
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
-	struct take_cpu_down_param *param = _param;
-	int err;
+	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
 	err = __cpu_disable();
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING | param->mod, param->hcpu);
+	cpu_notify(CPU_DYING, cpu);
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
-	stop_machine_park((long)param->hcpu);
+	stop_machine_park(cpu);
 	return 0;
 }
 
@@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
 static int _cpu_down(unsigned int cpu, int tasks_frozen)
 {
 	int err, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
-	struct take_cpu_down_param tcd_param = {
-		.mod = mod,
-		.hcpu = hcpu,
-	};
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, i
 
 	cpu_hotplug_begin();
 
-	err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
 	if (err) {
 		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
 		pr_warn("%s: attempt to take down CPU %u failed\n",
 			__func__, cpu);
 		goto out_release;
@@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, i
 	/*
 	 * So now all preempt/rcu users must observe !cpu_active().
 	 */
-	err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+	err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
 	if (err) {
 		/* CPU didn't die: tell everyone.  Can't complain. */
-		cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
+		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
 		goto out_release;
 	}
@@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, i
 
 	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD | mod, hcpu);
+	cpu_notify_nofail(CPU_DEAD, cpu);
 
 	check_for_tasks(cpu);
 
 out_release:
 	cpu_hotplug_done();
 	if (!err)
-		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+		cpu_notify_nofail(CPU_POST_DEAD, cpu);
 	return err;
 }
 
@@ -485,10 +480,8 @@ void smpboot_thread_init(void)
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
-	int ret, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
 	struct task_struct *idle;
+	int ret, nr_calls = 0;
 
 	cpu_hotplug_begin();
 
@@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int
 	if (ret)
 		goto out;
 
-	ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
 	if (ret) {
 		nr_calls--;
 		pr_warn("%s: attempt to bring up CPU %u failed\n",
@@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int
 	BUG_ON(!cpu_online(cpu));
 
 	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE | mod, hcpu);
+	cpu_notify(CPU_ONLINE, cpu);
 
 out_notify:
 	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
 out:
 	cpu_hotplug_done();
 
@@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
  */
 void notify_cpu_starting(unsigned int cpu)
 {
-	unsigned long val = CPU_STARTING;
-
-#ifdef CONFIG_PM_SLEEP_SMP
-	if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
-		val = CPU_STARTING_FROZEN;
-#endif /* CONFIG_PM_SLEEP_SMP */
-	cpu_notify(val, (void *)(long)cpu);
+	cpu_notify(CPU_STARTING, cpu);
 }
 
 #endif /* CONFIG_SMP */

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 03/20] cpu/hotplug: Restructure cpu_up code
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
  2016-02-26 18:43 ` [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner
  2016-02-26 18:43 ` [patch 02/20] cpu/hotplug: Restructure FROZEN state handling Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 04/20] cpu/hotplug: Split out cpu down functions Thomas Gleixner
                   ` (16 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-Restructure-cpu_down-code.patch --]
[-- Type: text/plain, Size: 2435 bytes --]

Split out into separate functions, so we can convert it to a state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   69 ++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 22 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val,
 	return __cpu_notify(val, cpu, -1, NULL);
 }
 
+/* Notifier wrappers for transitioning to state machine */
+static int notify_prepare(unsigned int cpu)
+{
+	int nr_calls = 0;
+	int ret;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
+	if (ret) {
+		nr_calls--;
+		printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
+				__func__, cpu);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	}
+	return ret;
+}
+
+static int notify_online(unsigned int cpu)
+{
+	cpu_notify(CPU_ONLINE, cpu);
+	return 0;
+}
+
+static int bringup_cpu(unsigned int cpu)
+{
+	struct task_struct *idle = idle_thread_get(cpu);
+	int ret;
+
+	/* Arch-specific enabling code. */
+	ret = __cpu_up(cpu, idle);
+	if (ret) {
+		cpu_notify(CPU_UP_CANCELED, cpu);
+		return ret;
+	}
+	BUG_ON(!cpu_online(cpu));
+	return 0;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 
 static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
@@ -481,7 +518,7 @@ void smpboot_thread_init(void)
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
 	struct task_struct *idle;
-	int ret, nr_calls = 0;
+	int ret;
 
 	cpu_hotplug_begin();
 
@@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int
 		goto out;
 	}
 
+	cpuhp_tasks_frozen = tasks_frozen;
+
 	ret = smpboot_create_threads(cpu);
 	if (ret)
 		goto out;
 
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
-	if (ret) {
-		nr_calls--;
-		pr_warn("%s: attempt to bring up CPU %u failed\n",
-			__func__, cpu);
-		goto out_notify;
-	}
-
-	/* Arch-specific enabling code. */
-	ret = __cpu_up(cpu, idle);
-
-	if (ret != 0)
-		goto out_notify;
-	BUG_ON(!cpu_online(cpu));
+	ret = notify_prepare(cpu);
+	if (ret)
+		goto out;
 
-	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE, cpu);
+	ret = bringup_cpu(cpu);
+	if (ret)
+		goto out;
 
-out_notify:
-	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	notify_online(cpu);
 out:
 	cpu_hotplug_done();
 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 04/20] cpu/hotplug: Split out cpu down functions
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (2 preceding siblings ...)
  2016-02-26 18:43 ` [patch 03/20] cpu/hotplug: Restructure cpu_up code Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 05/20] cpu/hotplug: Add tracepoints Thomas Gleixner
                   ` (15 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-hotplug-Split-out-cpu-down-functions.patch --]
[-- Type: text/plain, Size: 3061 bytes --]

Split cpu_down in separate functions in preparation for state machine
conversion.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   83 +++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 53 insertions(+), 30 deletions(-)

--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
-	BUG_ON(cpu_notify(val, cpu));
-}
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
 
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int d
 	read_unlock(&tasklist_lock);
 }
 
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+	BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+	int err, nr_calls = 0;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+	if (err) {
+		nr_calls--;
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+		pr_warn("%s: attempt to take down CPU %u failed\n",
+				__func__, cpu);
+	}
+	return err;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
 	return 0;
 }
 
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
 {
-	int err, nr_calls = 0;
-
-	if (num_online_cpus() == 1)
-		return -EBUSY;
-
-	if (!cpu_online(cpu))
-		return -EINVAL;
-
-	cpu_hotplug_begin();
-
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
-	if (err) {
-		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
-		pr_warn("%s: attempt to take down CPU %u failed\n",
-			__func__, cpu);
-		goto out_release;
-	}
+	int err;
 
 	/*
 	 * By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, i
 		/* CPU didn't die: tell everyone.  Can't complain. */
 		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
-		goto out_release;
+		return err;
 	}
 	BUG_ON(cpu_online(cpu));
 
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, i
 	/* This actually kills the CPU. */
 	__cpu_die(cpu);
 
-	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD, cpu);
+	return 0;
+}
 
+static int notify_dead(unsigned int cpu)
+{
+	cpu_notify_nofail(CPU_DEAD, cpu);
 	check_for_tasks(cpu);
+	return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+	int err;
+
+	if (num_online_cpus() == 1)
+		return -EBUSY;
+
+	if (!cpu_online(cpu))
+		return -EINVAL;
+
+	cpu_hotplug_begin();
+
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = notify_down_prepare(cpu);
+	if (err)
+		goto out_release;
+	err = takedown_cpu(cpu);
+	if (err)
+		goto out_release;
+
+	notify_dead(cpu);
 
 out_release:
 	cpu_hotplug_done();

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 05/20] cpu/hotplug: Add tracepoints
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (3 preceding siblings ...)
  2016-02-26 18:43 ` [patch 04/20] cpu/hotplug: Split out cpu down functions Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 06/20] cpu/hotplug: Convert to a state machine for the control processor Thomas Gleixner
                   ` (14 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Add-tracepoints.patch --]
[-- Type: text/plain, Size: 1779 bytes --]

We want to trace the hotplug machinery. Add tracepoints to track the
invocation of callbacks and their result.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/trace/events/cpuhp.h |   66 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

Index: b/include/trace/events/cpuhp.h
===================================================================
--- /dev/null
+++ b/include/trace/events/cpuhp.h
@@ -0,0 +1,66 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM cpuhp
+
+#if !defined(_TRACE_CPUHP_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_CPUHP_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(cpuhp_enter,
+
+	TP_PROTO(unsigned int cpu,
+		 int target,
+		 int idx,
+		 int (*fun)(unsigned int)),
+
+	TP_ARGS(cpu, target, idx, fun),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		target		)
+		__field( int,		idx		)
+		__field( void *,	fun		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->target	= target;
+		__entry->idx	= idx;
+		__entry->fun	= fun;
+	),
+
+	TP_printk("cpu: %04u target: %3d step: %3d (%pf)",
+		  __entry->cpu, __entry->target, __entry->idx, __entry->fun)
+);
+
+TRACE_EVENT(cpuhp_exit,
+
+	TP_PROTO(unsigned int cpu,
+		 int state,
+		 int idx,
+		 int ret),
+
+	TP_ARGS(cpu, state, idx, ret),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		state		)
+		__field( int,		idx		)
+		__field( int,		ret		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->state	= state;
+		__entry->idx	= idx;
+		__entry->ret	= ret;
+	),
+
+	TP_printk(" cpu: %04u  state: %3d step: %3d ret: %d",
+		  __entry->cpu, __entry->state, __entry->idx,  __entry->ret)
+);
+
+#endif
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 06/20] cpu/hotplug: Convert to a state machine for the control processor
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (4 preceding siblings ...)
  2016-02-26 18:43 ` [patch 05/20] cpu/hotplug: Add tracepoints Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:53   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine Thomas Gleixner
                   ` (13 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-hotplug-Convert-to-a-state-machine-for-the-contr.patch --]
[-- Type: text/plain, Size: 9363 bytes --]

Move the split out steps into a callback array and let the cpu_up/down
code iterate through the array functions. For now most of the
callbacks are asymmetric to resemble the current hotplug maze.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    9 -
 include/linux/cpuhotplug.h |   13 ++
 init/main.c                |   15 ---
 kernel/cpu.c               |  204 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 195 insertions(+), 46 deletions(-)
 create mode 100644 include/linux/cpuhotplug.h

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -16,6 +16,7 @@
 #include <linux/node.h>
 #include <linux/compiler.h>
 #include <linux/cpumask.h>
+#include <linux/cpuhotplug.h>
 
 struct device;
 struct device_node;
@@ -27,6 +28,9 @@ struct cpu {
 	struct device dev;
 };
 
+extern void boot_cpu_init(void);
+extern void boot_cpu_state_init(void);
+
 extern int register_cpu(struct cpu *cpu, int num);
 extern struct device *get_cpu_device(unsigned cpu);
 extern bool cpu_is_hotpluggable(unsigned cpu);
@@ -267,11 +271,6 @@ static inline int disable_nonboot_cpus(v
 static inline void enable_nonboot_cpus(void) {}
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
-enum cpuhp_state {
-	CPUHP_OFFLINE,
-	CPUHP_ONLINE,
-};
-
 void cpu_startup_entry(enum cpuhp_state state);
 
 void cpu_idle_poll_ctrl(bool enable);
Index: b/include/linux/cpuhotplug.h
===================================================================
--- /dev/null
+++ b/include/linux/cpuhotplug.h
@@ -0,0 +1,13 @@
+#ifndef __CPUHOTPLUG_H
+#define __CPUHOTPLUG_H
+
+enum cpuhp_state {
+	CPUHP_OFFLINE,
+	CPUHP_CREATE_THREADS,
+	CPUHP_NOTIFY_PREPARE,
+	CPUHP_BRINGUP_CPU,
+	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE,
+};
+
+#endif
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -452,20 +452,6 @@ void __init parse_early_param(void)
 	done = 1;
 }
 
-/*
- *	Activate the first processor.
- */
-
-static void __init boot_cpu_init(void)
-{
-	int cpu = smp_processor_id();
-	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
-	set_cpu_online(cpu, true);
-	set_cpu_active(cpu, true);
-	set_cpu_present(cpu, true);
-	set_cpu_possible(cpu, true);
-}
-
 void __init __weak smp_setup_processor_id(void)
 {
 }
@@ -530,6 +516,7 @@ asmlinkage __visible void __init start_k
 	setup_command_line(command_line);
 	setup_nr_cpu_ids();
 	setup_per_cpu_areas();
+	boot_cpu_state_init();
 	smp_prepare_boot_cpu();	/* arch-specific boot-cpu hooks */
 
 	build_all_zonelists(NULL, NULL);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,10 +22,64 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+
 #include <trace/events/power.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/cpuhp.h>
 
 #include "smpboot.h"
 
+/**
+ * cpuhp_cpu_state - Per cpu hotplug state storage
+ * @state:	The current cpu state
+ * @target:	The target state
+ */
+struct cpuhp_cpu_state {
+	enum cpuhp_state	state;
+	enum cpuhp_state	target;
+};
+
+static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
+
+/**
+ * cpuhp_step - Hotplug state machine step
+ * @name:	Name of the step
+ * @startup:	Startup function of the step
+ * @teardown:	Teardown function of the step
+ * @skip_onerr:	Do not invoke the functions on error rollback
+ *		Will go away once the notifiers	are gone
+ */
+struct cpuhp_step {
+	const char	*name;
+	int		(*startup)(unsigned int cpu);
+	int		(*teardown)(unsigned int cpu);
+	bool		skip_onerr;
+};
+
+static struct cpuhp_step cpuhp_bp_states[];
+
+/**
+ * cpuhp_invoke_callback _ Invoke the callbacks for a given state
+ * @cpu:	The cpu for which the callback should be invoked
+ * @step:	The step in the state machine
+ * @cb:		The callback function to invoke
+ *
+ * Called from cpu hotplug and from the state register machinery
+ */
+static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state step,
+				 int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int ret = 0;
+
+	if (cb) {
+		trace_cpuhp_enter(cpu, st->target, step, cb);
+		ret = cb(cpu);
+		trace_cpuhp_exit(cpu, st->state, step, ret);
+	}
+	return ret;
+}
+
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
@@ -454,10 +508,29 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+#else
+#define notify_down_prepare	NULL
+#define takedown_cpu		NULL
+#define notify_dead		NULL
+#endif
+
+#ifdef CONFIG_HOTPLUG_CPU
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 {
-	int err;
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int prev_state, ret = 0;
+	bool hasdied = false;
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -469,20 +542,25 @@ static int __ref _cpu_down(unsigned int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	err = notify_down_prepare(cpu);
-	if (err)
-		goto out_release;
-	err = takedown_cpu(cpu);
-	if (err)
-		goto out_release;
-
-	notify_dead(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_OFFLINE;
+	for (; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st);
+			break;
+		}
+	}
+	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
-out_release:
 	cpu_hotplug_done();
-	if (!err)
+	/* This post dead nonsense must die */
+	if (!ret && hasdied)
 		cpu_notify_nofail(CPU_POST_DEAD, cpu);
-	return err;
+	return ret;
 }
 
 int cpu_down(unsigned int cpu)
@@ -537,11 +615,22 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int ret;
+	int prev_state, ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -550,6 +639,7 @@ static int _cpu_up(unsigned int cpu, int
 		goto out;
 	}
 
+	/* Let it fail before we try to bring the cpu up */
 	idle = idle_thread_get(cpu);
 	if (IS_ERR(idle)) {
 		ret = PTR_ERR(idle);
@@ -558,22 +648,22 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	ret = smpboot_create_threads(cpu);
-	if (ret)
-		goto out;
-
-	ret = notify_prepare(cpu);
-	if (ret)
-		goto out;
-
-	ret = bringup_cpu(cpu);
-	if (ret)
-		goto out;
-
-	notify_online(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_ONLINE;
+	while (st->state < st->target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_bp_states + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st);
+			break;
+		}
+	}
 out:
 	cpu_hotplug_done();
-
 	return ret;
 }
 
@@ -767,6 +857,44 @@ void notify_cpu_starting(unsigned int cp
 
 #endif /* CONFIG_SMP */
 
+/* Boot processor state steps */
+static struct cpuhp_step cpuhp_bp_states[] = {
+	[CPUHP_OFFLINE] = {
+		.name			= "offline",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+#ifdef CONFIG_SMP
+	[CPUHP_CREATE_THREADS]= {
+		.name			= "threads:create",
+		.startup		= smpboot_create_threads,
+		.teardown		= NULL,
+	},
+	[CPUHP_NOTIFY_PREPARE] = {
+		.name			= "notify:prepare",
+		.startup		= notify_prepare,
+		.teardown		= notify_dead,
+		.skip_onerr		= true,
+	},
+	[CPUHP_BRINGUP_CPU] = {
+		.name			= "cpu:bringup",
+		.startup		= bringup_cpu,
+		.teardown		= takedown_cpu,
+		.skip_onerr		= true,
+	},
+	[CPUHP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
+#endif
+	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.
@@ -826,3 +954,25 @@ void init_cpu_online(const struct cpumas
 {
 	cpumask_copy(&__cpu_online_mask, src);
 }
+
+/*
+ * Activate the first processor.
+ */
+void __init boot_cpu_init(void)
+{
+	int cpu = smp_processor_id();
+
+	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
+	set_cpu_online(cpu, true);
+	set_cpu_active(cpu, true);
+	set_cpu_present(cpu, true);
+	set_cpu_possible(cpu, true);
+}
+
+/*
+ * Must be called _AFTER_ setting up the per_cpu areas
+ */
+void __init boot_cpu_state_init(void)
+{
+	per_cpu_ptr(&cpuhp_state, smp_processor_id())->state = CPUHP_ONLINE;
+}

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (5 preceding siblings ...)
  2016-02-26 18:43 ` [patch 06/20] cpu/hotplug: Convert to a state machine for the control processor Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:53   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down Thomas Gleixner
                   ` (12 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-hotplug-Convert-the-hotplugged-processor-work-to.patch --]
[-- Type: text/plain, Size: 5104 bytes --]

Move the functions which need to run on the hotplugged processor into
a state machine array and let the code iterate through these functions.

In a later state, this will grow synchronization points between the
control processor and the hotplugged processor, so we can move the
various architecture implementations of the synchronizations to the
core.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    4 ++
 kernel/cpu.c               |   81 ++++++++++++++++++++++++++++++++++++---------
 2 files changed, 70 insertions(+), 15 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,10 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_OFFLINE,
+	CPUHP_AP_NOTIFY_STARTING,
+	CPUHP_AP_ONLINE,
+	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE,
 };
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -57,6 +57,7 @@ struct cpuhp_step {
 };
 
 static struct cpuhp_step cpuhp_bp_states[];
+static struct cpuhp_step cpuhp_ap_states[];
 
 /**
  * cpuhp_invoke_callback _ Invoke the callbacks for a given state
@@ -304,6 +305,12 @@ static int notify_online(unsigned int cp
 	return 0;
 }
 
+static int notify_starting(unsigned int cpu)
+{
+	cpu_notify(CPU_STARTING, cpu);
+	return 0;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -421,9 +428,17 @@ static int notify_down_prepare(unsigned
 	return err;
 }
 
+static int notify_dying(unsigned int cpu)
+{
+	cpu_notify(CPU_DYING, cpu);
+	return 0;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_OFFLINE);
 	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
@@ -431,7 +446,12 @@ static int take_cpu_down(void *_param)
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING, cpu);
+	/* Invoke the former CPU_DYING callbacks */
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = cpuhp_ap_states + st->state;
+
+		cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
@@ -512,6 +532,7 @@ static int notify_dead(unsigned int cpu)
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
 #define notify_dead		NULL
+#define notify_dying		NULL
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -615,6 +636,28 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+/**
+ * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
+ * @cpu: cpu that just started
+ *
+ * This function calls the cpu_chain notifiers with CPU_STARTING.
+ * It must be called by the arch code on the new cpu, before the new cpu
+ * enables interrupts and before the "boot" cpu returns from __cpu_up().
+ */
+void notify_cpu_starting(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE);
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_ap_states + st->state;
+		cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -842,19 +885,6 @@ core_initcall(cpu_hotplug_pm_sync_init);
 
 #endif /* CONFIG_PM_SLEEP_SMP */
 
-/**
- * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
- * @cpu: cpu that just started
- *
- * This function calls the cpu_chain notifiers with CPU_STARTING.
- * It must be called by the arch code on the new cpu, before the new cpu
- * enables interrupts and before the "boot" cpu returns from __cpu_up().
- */
-void notify_cpu_starting(unsigned int cpu)
-{
-	cpu_notify(CPU_STARTING, cpu);
-}
-
 #endif /* CONFIG_SMP */
 
 /* Boot processor state steps */
@@ -879,8 +909,12 @@ static struct cpuhp_step cpuhp_bp_states
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
+		.teardown		= NULL,
+	},
+	[CPUHP_TEARDOWN_CPU] = {
+		.name			= "cpu:teardown",
+		.startup		= NULL,
 		.teardown		= takedown_cpu,
-		.skip_onerr		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
@@ -889,6 +923,23 @@ static struct cpuhp_step cpuhp_bp_states
 	},
 #endif
 	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
+/* Application processor state steps */
+static struct cpuhp_step cpuhp_ap_states[] = {
+#ifdef CONFIG_SMP
+	[CPUHP_AP_NOTIFY_STARTING] = {
+		.name			= "notify:starting",
+		.startup		= notify_starting,
+		.teardown		= notify_dying,
+		.skip_onerr		= true,
+	},
+#endif
+	[CPUHP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (6 preceding siblings ...)
  2016-02-26 18:43 ` [patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:54   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 09/20] cpu/hotplug: Add sysfs state interface Thomas Gleixner
                   ` (11 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Hand-in-target-state-to-_cpu_up-down.patch --]
[-- Type: text/plain, Size: 3302 bytes --]

We want to be able to bringup/teardown the cpu to a particular state. Add a
target argument to _cpu_up/down.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -547,7 +547,8 @@ static void undo_cpu_down(unsigned int c
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
+			   enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int prev_state, ret = 0;
@@ -564,7 +565,7 @@ static int __ref _cpu_down(unsigned int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_OFFLINE;
+	st->target = target;
 	for (; st->state > st->target; st->state--) {
 		struct cpuhp_step *step = cpuhp_bp_states + st->state;
 
@@ -584,7 +585,7 @@ static int __ref _cpu_down(unsigned int
 	return ret;
 }
 
-int cpu_down(unsigned int cpu)
+static int do_cpu_down(unsigned int cpu, enum cpuhp_state target)
 {
 	int err;
 
@@ -595,12 +596,16 @@ int cpu_down(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_down(cpu, 0);
+	err = _cpu_down(cpu, 0, target);
 
 out:
 	cpu_maps_update_done();
 	return err;
 }
+int cpu_down(unsigned int cpu)
+{
+	return do_cpu_down(cpu, CPUHP_OFFLINE);
+}
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
@@ -669,7 +674,7 @@ static void undo_cpu_up(unsigned int cpu
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int _cpu_up(unsigned int cpu, int tasks_frozen)
+static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
@@ -692,7 +697,7 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_ONLINE;
+	st->target = target;
 	while (st->state < st->target) {
 		struct cpuhp_step *step;
 
@@ -710,7 +715,7 @@ static int _cpu_up(unsigned int cpu, int
 	return ret;
 }
 
-int cpu_up(unsigned int cpu)
+static int do_cpu_up(unsigned int cpu, enum cpuhp_state target)
 {
 	int err = 0;
 
@@ -734,12 +739,16 @@ int cpu_up(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_up(cpu, 0);
-
+	err = _cpu_up(cpu, 0, target);
 out:
 	cpu_maps_update_done();
 	return err;
 }
+
+int cpu_up(unsigned int cpu)
+{
+	return do_cpu_up(cpu, CPUHP_ONLINE);
+}
 EXPORT_SYMBOL_GPL(cpu_up);
 
 #ifdef CONFIG_PM_SLEEP_SMP
@@ -762,7 +771,7 @@ int disable_nonboot_cpus(void)
 		if (cpu == first_cpu)
 			continue;
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, true);
-		error = _cpu_down(cpu, 1);
+		error = _cpu_down(cpu, 1, CPUHP_OFFLINE);
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, false);
 		if (!error)
 			cpumask_set_cpu(cpu, frozen_cpus);
@@ -812,7 +821,7 @@ void enable_nonboot_cpus(void)
 
 	for_each_cpu(cpu, frozen_cpus) {
 		trace_suspend_resume(TPS("CPU_ON"), cpu, true);
-		error = _cpu_up(cpu, 1);
+		error = _cpu_up(cpu, 1, CPUHP_ONLINE);
 		trace_suspend_resume(TPS("CPU_ON"), cpu, false);
 		if (!error) {
 			pr_info("CPU%d is up\n", cpu);

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 09/20] cpu/hotplug: Add sysfs state interface
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (7 preceding siblings ...)
  2016-02-26 18:43 ` [patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:54   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 10/20] cpu/hotplug: Make target state writeable Thomas Gleixner
                   ` (10 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Add-sysfs-state-interface.patch --]
[-- Type: text/plain, Size: 3062 bytes --]

Add a sysfs interface so we can actually see in which state the cpus are in.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   98 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -56,6 +56,7 @@ struct cpuhp_step {
 	bool		skip_onerr;
 };
 
+static DEFINE_MUTEX(cpuhp_state_mutex);
 static struct cpuhp_step cpuhp_bp_states[];
 static struct cpuhp_step cpuhp_ap_states[];
 
@@ -955,6 +956,103 @@ static struct cpuhp_step cpuhp_ap_states
 	},
 };
 
+static bool cpuhp_is_ap_state(enum cpuhp_state state)
+{
+	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+}
+
+static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
+{
+	struct cpuhp_step *sp;
+
+	sp = cpuhp_is_ap_state(state) ? cpuhp_ap_states : cpuhp_bp_states;
+	return sp + state;
+}
+
+#if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
+static ssize_t show_cpuhp_state(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->state);
+}
+static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
+
+static ssize_t show_cpuhp_target(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->target);
+}
+static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+
+static struct attribute *cpuhp_cpu_attrs[] = {
+	&dev_attr_state.attr,
+	&dev_attr_target.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_attr_group = {
+	.attrs = cpuhp_cpu_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static ssize_t show_cpuhp_states(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	ssize_t cur, res = 0;
+	int i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = 0; i <= CPUHP_ONLINE; i++) {
+		struct cpuhp_step *sp = cpuhp_get_step(i);
+
+		if (sp->name) {
+			cur = sprintf(buf, "%3d: %s\n", i, sp->name);
+			buf += cur;
+			res += cur;
+		}
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	return res;
+}
+static DEVICE_ATTR(states, 0444, show_cpuhp_states, NULL);
+
+static struct attribute *cpuhp_cpu_root_attrs[] = {
+	&dev_attr_states.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_root_attr_group = {
+	.attrs = cpuhp_cpu_root_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static int __init cpuhp_sysfs_init(void)
+{
+	int cpu, ret;
+
+	ret = sysfs_create_group(&cpu_subsys.dev_root->kobj,
+				 &cpuhp_cpu_root_attr_group);
+	if (ret)
+		return ret;
+
+	for_each_possible_cpu(cpu) {
+		struct device *dev = get_cpu_device(cpu);
+
+		ret = sysfs_create_group(&dev->kobj, &cpuhp_cpu_attr_group);
+		if (ret)
+			return ret;
+	}
+	return 0;
+}
+device_initcall(cpuhp_sysfs_init);
+#endif
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (8 preceding siblings ...)
  2016-02-26 18:43 ` [patch 09/20] cpu/hotplug: Add sysfs state interface Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-02-26 23:46   ` Rafael J. Wysocki
  2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 11/20] cpu/hotplug: Implement setup/removal interface Thomas Gleixner
                   ` (9 subsequent siblings)
  19 siblings, 2 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Make-target-state-writeable.patch --]
[-- Type: text/plain, Size: 5263 bytes --]

Make it possible to write a target state to the per cpu state file, so we can
switch between states.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c      |   66 +++++++++++++++++++++++++++++++++++++++++++++++-------
 lib/Kconfig.debug |   13 ++++++++++
 2 files changed, 71 insertions(+), 8 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -48,12 +48,14 @@ static DEFINE_PER_CPU(struct cpuhp_cpu_s
  * @teardown:	Teardown function of the step
  * @skip_onerr:	Do not invoke the functions on error rollback
  *		Will go away once the notifiers	are gone
+ * @cant_stop:	Bringup/teardown can't be stopped at this step
  */
 struct cpuhp_step {
 	const char	*name;
 	int		(*startup)(unsigned int cpu);
 	int		(*teardown)(unsigned int cpu);
 	bool		skip_onerr;
+	bool		cant_stop;
 };
 
 static DEFINE_MUTEX(cpuhp_state_mutex);
@@ -558,7 +560,7 @@ static int __ref _cpu_down(unsigned int
 	if (num_online_cpus() == 1)
 		return -EBUSY;
 
-	if (!cpu_online(cpu))
+	if (!cpu_present(cpu))
 		return -EINVAL;
 
 	cpu_hotplug_begin();
@@ -683,16 +685,25 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpu_hotplug_begin();
 
-	if (cpu_online(cpu) || !cpu_present(cpu)) {
+	if (!cpu_present(cpu)) {
 		ret = -EINVAL;
 		goto out;
 	}
 
-	/* Let it fail before we try to bring the cpu up */
-	idle = idle_thread_get(cpu);
-	if (IS_ERR(idle)) {
-		ret = PTR_ERR(idle);
+	/*
+	 * The caller of do_cpu_up might have raced with another
+	 * caller. Ignore it for now.
+	 */
+	if (st->state >= target)
 		goto out;
+
+	if (st->state == CPUHP_OFFLINE) {
+		/* Let it fail before we try to bring the cpu up */
+		idle = idle_thread_get(cpu);
+		if (IS_ERR(idle)) {
+			ret = PTR_ERR(idle);
+			goto out;
+		}
 	}
 
 	cpuhp_tasks_frozen = tasks_frozen;
@@ -909,27 +920,32 @@ static struct cpuhp_step cpuhp_bp_states
 		.name			= "threads:create",
 		.startup		= smpboot_create_threads,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_PREPARE] = {
 		.name			= "notify:prepare",
 		.startup		= notify_prepare,
 		.teardown		= notify_dead,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_TEARDOWN_CPU] = {
 		.name			= "cpu:teardown",
 		.startup		= NULL,
 		.teardown		= takedown_cpu,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
 		.teardown		= notify_down_prepare,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -947,6 +963,7 @@ static struct cpuhp_step cpuhp_ap_states
 		.startup		= notify_starting,
 		.teardown		= notify_dying,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -979,6 +996,39 @@ static ssize_t show_cpuhp_state(struct d
 }
 static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
 
+static ssize_t write_cpuhp_target(struct device *dev,
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+	struct cpuhp_step *sp;
+	int target, ret;
+
+	ret = kstrtoint(buf, 10, &target);
+	if (ret)
+		return ret;
+
+#ifdef CONFIG_CPU_HOTPLUG_STATE_CONTROL
+	if (target < CPUHP_OFFLINE || target > CPUHP_ONLINE)
+		return -EINVAL;
+#else
+	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
+		return -EINVAL;
+#endif
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(target);
+	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
+	mutex_unlock(&cpuhp_state_mutex);
+	if (ret)
+		return ret;
+
+	if (st->state < target)
+		ret = do_cpu_up(dev->id, target);
+	else
+		ret = do_cpu_down(dev->id, target);
+	return ret ? ret : count;
+}
+
 static ssize_t show_cpuhp_target(struct device *dev,
 				 struct device_attribute *attr, char *buf)
 {
@@ -986,7 +1036,7 @@ static ssize_t show_cpuhp_target(struct
 
 	return sprintf(buf, "%d\n", st->target);
 }
-static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+static DEVICE_ATTR(target, 0644, show_cpuhp_target, write_cpuhp_target);
 
 static struct attribute *cpuhp_cpu_attrs[] = {
 	&dev_attr_state.attr,
@@ -1007,7 +1057,7 @@ static ssize_t show_cpuhp_states(struct
 	int i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = 0; i <= CPUHP_ONLINE; i++) {
+	for (i = CPUHP_OFFLINE; i <= CPUHP_ONLINE; i++) {
 		struct cpuhp_step *sp = cpuhp_get_step(i);
 
 		if (sp->name) {
Index: b/lib/Kconfig.debug
===================================================================
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1442,6 +1442,19 @@ config DEBUG_BLOCK_EXT_DEVT
 
 	  Say N if you are unsure.
 
+config CPU_HOTPLUG_STATE_CONTROL
+	bool "Enable CPU hotplug state control"
+	depends on DEBUG_KERNEL
+	depends on HOTPLUG_CPU
+	default n
+	help
+	  Allows to write steps between "offline" and "online" to the CPUs
+	  sysfs target file so states can be stepped granular. This is a debug
+	  option for now as the hotplug machinery cannot be stopped and
+	  restarted at arbitrary points yet.
+
+	  Say N if your are unsure.
+
 config NOTIFIER_ERROR_INJECTION
 	tristate "Notifier error injection"
 	depends on DEBUG_KERNEL

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 11/20] cpu/hotplug: Implement setup/removal interface
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (9 preceding siblings ...)
  2016-02-26 18:43 ` [patch 10/20] cpu/hotplug: Make target state writeable Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core Thomas Gleixner
                   ` (8 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: cpu-hotplug-Implement-setup-removal-interface.patch --]
[-- Type: text/plain, Size: 9949 bytes --]

Implement function which allow to setup/remove hotplug state callbacks.

The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.

The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.

This includes rollback to the previous state in case of failure.

A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them

For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.

[ Dynamic registering implemented by Sebastian Siewior ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   67 +++++++++++++
 kernel/cpu.c               |  224 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 291 insertions(+)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE_DYN,
+	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
+int __cpuhp_setup_state(enum cpuhp_state state,	const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback (will be used in debug output)
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+				    const char *name,
+				    int (*startup)(unsigned int cpu),
+				    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ *			       callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback.
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+					    const char *name,
+					    int (*startup)(unsigned int cpu),
+					    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state:	The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ *				teardown
+ * @state:	The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, false);
+}
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states
 	},
 };
 
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+	if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+		return -EINVAL;
+	return 0;
+}
+
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
 	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step
 	return sp + state;
 }
 
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+				  const char *name,
+				  int (*startup)(unsigned int cpu),
+				  int (*teardown)(unsigned int cpu))
+{
+	/* (Un)Install the callbacks for further cpu hotplug operations */
+	struct cpuhp_step *sp;
+
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(state);
+	sp->startup = startup;
+	sp->teardown = teardown;
+	sp->name = name;
+	mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+	return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+	int (*cb)(unsigned int cpu) = __cb;
+
+	BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+			    int (*cb)(unsigned int), bool bringup)
+{
+	int ret;
+
+	if (!cb)
+		return 0;
+
+	/*
+	 * This invokes the callback directly for now. In a later step we
+	 * convert that to use cpuhp_invoke_callback().
+	 */
+	if (cpuhp_is_ap_state(state)) {
+		/*
+		 * Note, that a function called on the AP is not
+		 * allowed to fail.
+		 */
+		if (cpu_online(cpu))
+			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+		return 0;
+	}
+
+	/*
+	 * The non AP bound callbacks can fail on bringup. On teardown
+	 * e.g. module removal we crash for now.
+	 */
+	ret = cb(cpu);
+	BUG_ON(ret && !bringup);
+	return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+				   int (*teardown)(unsigned int cpu))
+{
+	int cpu;
+
+	if (!teardown)
+		return;
+
+	/* Roll back the already executed steps on the other cpus */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpu >= failedcpu)
+			break;
+
+		/* Did we invoke the startup call on that cpu ? */
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+	enum cpuhp_state i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+		if (cpuhp_bp_states[i].name)
+			continue;
+
+		cpuhp_bp_states[i].name = "Reserved";
+		mutex_unlock(&cpuhp_state_mutex);
+		return i;
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	WARN(1, "No more dynamic states available for CPU hotplug\n");
+	return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state:	The state to setup
+ * @invoke:	If true, the startup function is invoked for cpus where
+ *		cpu state >= @state
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+			const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu))
+{
+	int cpu, ret = 0;
+	int dyn_state = 0;
+
+	if (cpuhp_cb_check(state) || !name)
+		return -EINVAL;
+
+	get_online_cpus();
+
+	/* currently assignments for the ONLINE state are possible */
+	if (state == CPUHP_ONLINE_DYN) {
+		dyn_state = 1;
+		ret = cpuhp_reserve_state(state);
+		if (ret < 0)
+			goto out;
+		state = ret;
+	}
+
+	cpuhp_store_callbacks(state, name, startup, teardown);
+
+	if (!invoke || !startup)
+		goto out;
+
+	/*
+	 * Try to call the startup callback for each present cpu
+	 * depending on the hotplug state of the cpu.
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate < state)
+			continue;
+
+		ret = cpuhp_issue_call(cpu, state, startup, true);
+		if (ret) {
+			cpuhp_rollback_install(cpu, state, teardown);
+			cpuhp_store_callbacks(state, NULL, NULL, NULL);
+			goto out;
+		}
+	}
+out:
+	put_online_cpus();
+	if (!ret && dyn_state)
+		return state;
+	return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state:	The state to remove
+ * @invoke:	If true, the teardown function is invoked for cpus where
+ *		cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+	int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+	int cpu;
+
+	BUG_ON(cpuhp_cb_check(state));
+
+	get_online_cpus();
+
+	if (!invoke || !teardown)
+		goto remove;
+
+	/*
+	 * Call the teardown callback for each present cpu depending
+	 * on the hotplug state of the cpu. This function is not
+	 * allowed to fail currently!
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+remove:
+	cpuhp_store_callbacks(state, NULL, NULL, NULL);
+	put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
 #if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
 static ssize_t show_cpuhp_state(struct device *dev,
 				struct device_attribute *attr, char *buf)

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (10 preceding siblings ...)
  2016-02-26 18:43 ` [patch 11/20] cpu/hotplug: Implement setup/removal interface Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine Thomas Gleixner
                   ` (7 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Move-scheduler-cpu_online-notifier-to-hotplug-core.patch --]
[-- Type: text/plain, Size: 2431 bytes --]

Move the scheduler cpu online notifier part to the hotplug core. This is
anyway the highest priority callback and we need that functionality right now
for the next changes.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    1 +
 kernel/cpu.c               |   18 ++++++++++++++++++
 kernel/sched/core.c        |   10 ----------
 3 files changed, 19 insertions(+), 10 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,6 +10,7 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
+	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -666,6 +666,19 @@ void notify_cpu_starting(unsigned int cp
 	}
 }
 
+/*
+ * Called from the idle task. We need to set active here, so we can kick off
+ * the stopper thread.
+ */
+static int cpuhp_set_cpu_active(unsigned int cpu)
+{
+	/* The cpu is marked online, set it active now */
+	set_cpu_active(cpu, true);
+	/* Unpark the stopper thread */
+	stop_machine_unpark(cpu);
+	return 0;
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -941,6 +954,11 @@ static struct cpuhp_step cpuhp_bp_states
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
+	[CPUHP_CPU_SET_ACTIVE] = {
+		.name			= "cpu:active",
+		.startup		= cpuhp_set_cpu_active,
+		.teardown		= NULL,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
Index: b/kernel/sched/core.c
===================================================================
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5692,16 +5692,6 @@ static int sched_cpu_active(struct notif
 		set_cpu_rq_start_time();
 		return NOTIFY_OK;
 
-	case CPU_ONLINE:
-		/*
-		 * At this point a starting CPU has marked itself as online via
-		 * set_cpu_online(). But it might not yet have marked itself
-		 * as active, which is essential from here on.
-		 */
-		set_cpu_active(cpu, true);
-		stop_machine_unpark(cpu);
-		return NOTIFY_OK;
-
 	case CPU_DOWN_FAILED:
 		set_cpu_active(cpu, true);
 		return NOTIFY_OK;

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (11 preceding siblings ...)
  2016-02-26 18:43 ` [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:56   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 14/20] cpu/hotplug: Split out the state walk into functions Thomas Gleixner
                   ` (6 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Handle-smpboot-threads-in-state-machine.patch --]
[-- Type: text/plain, Size: 5177 bytes --]

Handle the smpboot threads in the state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    7 +------
 include/linux/cpuhotplug.h |    1 +
 init/main.c                |    1 -
 kernel/cpu.c               |   39 +++++----------------------------------
 kernel/smpboot.c           |    6 ++++--
 kernel/smpboot.h           |    4 ++--
 6 files changed, 13 insertions(+), 45 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -78,7 +78,7 @@ enum {
 	/* migration should happen before other stuff but after perf */
 	CPU_PRI_PERF		= 20,
 	CPU_PRI_MIGRATION	= 10,
-	CPU_PRI_SMPBOOT		= 9,
+
 	/* bring up workqueues before normal notifiers and down after */
 	CPU_PRI_WORKQUEUE_UP	= 5,
 	CPU_PRI_WORKQUEUE_DOWN	= -5,
@@ -172,7 +172,6 @@ static inline void __unregister_cpu_noti
 }
 #endif
 
-void smpboot_thread_init(void);
 int cpu_up(unsigned int cpu);
 void notify_cpu_starting(unsigned int cpu);
 extern void cpu_maps_update_begin(void);
@@ -221,10 +220,6 @@ static inline void cpu_notifier_register
 {
 }
 
-static inline void smpboot_thread_init(void)
-{
-}
-
 #endif /* CONFIG_SMP */
 extern struct bus_type cpu_subsys;
 
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,6 +11,7 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
+	CPUHP_SMPBOOT_THREADS,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -388,7 +388,6 @@ static noinline void __init_refok rest_i
 	int pid;
 
 	rcu_scheduler_starting();
-	smpboot_thread_init();
 	/*
 	 * We need to spawn init first so that it obtains pid 1, however
 	 * the init task will end up wanting to create kthreads, which, if
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -481,8 +481,6 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
-	smpboot_park_threads(cpu);
-
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -612,38 +610,6 @@ int cpu_down(unsigned int cpu)
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
-/*
- * Unpark per-CPU smpboot kthreads at CPU-online time.
- */
-static int smpboot_thread_call(struct notifier_block *nfb,
-			       unsigned long action, void *hcpu)
-{
-	int cpu = (long)hcpu;
-
-	switch (action & ~CPU_TASKS_FROZEN) {
-
-	case CPU_DOWN_FAILED:
-	case CPU_ONLINE:
-		smpboot_unpark_threads(cpu);
-		break;
-
-	default:
-		break;
-	}
-
-	return NOTIFY_OK;
-}
-
-static struct notifier_block smpboot_thread_notifier = {
-	.notifier_call = smpboot_thread_call,
-	.priority = CPU_PRI_SMPBOOT,
-};
-
-void smpboot_thread_init(void)
-{
-	register_cpu_notifier(&smpboot_thread_notifier);
-}
-
 /**
  * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
  * @cpu: cpu that just started
@@ -959,6 +925,11 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
+	[CPUHP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
Index: b/kernel/smpboot.c
===================================================================
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -226,7 +226,7 @@ static void smpboot_unpark_thread(struct
 		kthread_unpark(tsk);
 }
 
-void smpboot_unpark_threads(unsigned int cpu)
+int smpboot_unpark_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -235,6 +235,7 @@ void smpboot_unpark_threads(unsigned int
 		if (cpumask_test_cpu(cpu, cur->cpumask))
 			smpboot_unpark_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
@@ -245,7 +246,7 @@ static void smpboot_park_thread(struct s
 		kthread_park(tsk);
 }
 
-void smpboot_park_threads(unsigned int cpu)
+int smpboot_park_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -253,6 +254,7 @@ void smpboot_park_threads(unsigned int c
 	list_for_each_entry_reverse(cur, &hotplug_threads, list)
 		smpboot_park_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -14,7 +14,7 @@ static inline void idle_threads_init(voi
 #endif
 
 int smpboot_create_threads(unsigned int cpu);
-void smpboot_park_threads(unsigned int cpu);
-void smpboot_unpark_threads(unsigned int cpu);
+int smpboot_park_threads(unsigned int cpu);
+int smpboot_unpark_threads(unsigned int cpu);
 
 #endif

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 14/20] cpu/hotplug: Split out the state walk into functions
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (12 preceding siblings ...)
  2016-02-26 18:43 ` [patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:56   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 15/20] cpu/hotplug: Create hotplug threads Thomas Gleixner
                   ` (5 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Create-state-walk-functions.patch --]
[-- Type: text/plain, Size: 4493 bytes --]

We need that for running callbacks on the AP and the BP.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |  111 ++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 68 insertions(+), 43 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+			  struct cpuhp_step *steps)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+				struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+			struct cpuhp_step *steps)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+			      struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = steps + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
-
 void unregister_cpu_notifier(struct notifier_block *nb)
 {
 	cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state++; st->state < st->target; st->state++) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->startup);
-	}
-}
 
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
-	for (; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
-		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_down(cpu, st);
-			break;
-		}
-	}
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
 	cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned
 	return 0;
 }
 
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state--; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->teardown);
-	}
-}
-
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int prev_state, ret = 0;
+	int ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	prev_state = st->state;
 	st->target = target;
-	while (st->state < st->target) {
-		struct cpuhp_step *step;
-
-		st->state++;
-		step = cpuhp_bp_states + st->state;
-		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_up(cpu, st);
-			break;
-		}
-	}
+	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
 	return ret;

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 15/20] cpu/hotplug: Create hotplug threads
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (13 preceding siblings ...)
  2016-02-26 18:43 ` [patch 14/20] cpu/hotplug: Split out the state walk into functions Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu Thomas Gleixner
                   ` (4 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Create-hotplug-threads.patch --]
[-- Type: text/plain, Size: 6098 bytes --]

In order to let the hotplugged cpu take care of the setup/teardown, we need a
seperate hotplug thread.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c     |  145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/smp.c     |    1 
 kernel/smpboot.h |    2 
 3 files changed, 147 insertions(+), 1 deletion(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,6 +22,7 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+#include <linux/smpboot.h>
 
 #include <trace/events/power.h>
 #define CREATE_TRACE_POINTS
@@ -33,10 +34,24 @@
  * cpuhp_cpu_state - Per cpu hotplug state storage
  * @state:	The current cpu state
  * @target:	The target state
+ * @thread:	Pointer to the hotplug thread
+ * @should_run:	Thread should execute
+ * @cb_stat:	The state for a single callback (install/uninstall)
+ * @cb:		Single callback function (install/uninstall)
+ * @result:	Result of the operation
+ * @done:	Signal completion to the issuer of the task
  */
 struct cpuhp_cpu_state {
 	enum cpuhp_state	state;
 	enum cpuhp_state	target;
+#ifdef CONFIG_SMP
+	struct task_struct	*thread;
+	bool			should_run;
+	enum cpuhp_state	cb_state;
+	int			(*cb)(unsigned int cpu);
+	int			result;
+	struct completion	done;
+#endif
 };
 
 static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
@@ -394,6 +409,134 @@ static int cpuhp_up_callbacks(unsigned i
 	return ret;
 }
 
+/*
+ * The cpu hotplug threads manage the bringup and teardown of the cpus
+ */
+static void cpuhp_create(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	init_completion(&st->done);
+}
+
+static int cpuhp_should_run(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	return st->should_run;
+}
+
+/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
+static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+
+	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
+}
+
+/* Execute the online startup callbacks. Used to be CPU_ONLINE */
+static int cpuhp_ap_online(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	return cpuhp_up_callbacks(cpu, st, cpuhp_ap_states, st->target);
+}
+
+/*
+ * Execute teardown/startup callbacks on the plugged cpu. Also used to invoke
+ * callbacks when a state gets [un]installed at runtime.
+ */
+static void cpuhp_thread_fun(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	int ret = 0;
+
+	/*
+	 * Paired with the mb() in cpuhp_kick_ap_work and
+	 * cpuhp_invoke_ap_callback, so the work set is consistent visible.
+	 */
+	smp_mb();
+	if (!st->should_run)
+		return;
+
+	st->should_run = false;
+
+	/* Single callback invocation for [un]install ? */
+	if (st->cb) {
+		if (st->cb_state < CPUHP_AP_ONLINE) {
+			local_irq_disable();
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+			local_irq_enable();
+		} else {
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+		}
+	} else {
+		/* Regular hotplug work */
+		if (st->state < st->target)
+			ret = cpuhp_ap_online(cpu, st);
+		else if (st->state > st->target)
+			ret = cpuhp_ap_offline(cpu, st);
+	}
+	st->result = ret;
+	complete(&st->done);
+}
+
+/* Invoke a single callback on a remote cpu */
+static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
+				    int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	if (!cpu_online(cpu))
+		return 0;
+
+	st->cb_state = state;
+	st->cb = cb;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
+/* Regular hotplug invocation of the AP hotplug thread */
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	st->result = 0;
+	st->cb = NULL;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	trace_cpuhp_exit(cpu, st->state, state, st->result);
+	return st->result;
+}
+
+static struct smp_hotplug_thread cpuhp_threads = {
+	.store			= &cpuhp_state.thread,
+	.create			= &cpuhp_create,
+	.thread_should_run	= cpuhp_should_run,
+	.thread_fn		= cpuhp_thread_fun,
+	.thread_comm		= "cpuhp/%u",
+	.selfparking		= true,
+};
+
+void __init cpuhp_threads_init(void)
+{
+	BUG_ON(smpboot_register_percpu_thread(&cpuhp_threads));
+	kthread_unpark(this_cpu_read(cpuhp_state.thread));
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -997,7 +1140,7 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/smp.c
===================================================================
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -569,6 +569,7 @@ void __init smp_init(void)
 	unsigned int cpu;
 
 	idle_threads_init();
+	cpuhp_threads_init();
 
 	/* FIXME: This should be done in userspace --RR */
 	for_each_present_cpu(cpu) {
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -17,4 +17,6 @@ int smpboot_create_threads(unsigned int
 int smpboot_park_threads(unsigned int cpu);
 int smpboot_unpark_threads(unsigned int cpu);
 
+void __init cpuhp_threads_init(void);
+
 #endif

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (14 preceding siblings ...)
  2016-02-26 18:43 ` [patch 15/20] cpu/hotplug: Create hotplug threads Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 17/20] arch/hotplug: Call into idle with a proper state Thomas Gleixner
                   ` (3 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Move-online-calls-to-hotplugged-cpu.patch --]
[-- Type: text/plain, Size: 8671 bytes --]

Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   10 +--
 kernel/cpu.c               |  144 ++++++++++++++++++++++++++++++---------------
 2 files changed, 102 insertions(+), 52 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_SMPBOOT_THREADS,
-	CPUHP_NOTIFY_ONLINE,
-	CPUHP_ONLINE_DYN,
-	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
+	CPUHP_KICK_AP_THREAD,
+	CPUHP_BP_ONLINE,
+	CPUHP_AP_SMPBOOT_THREADS,
+	CPUHP_AP_NOTIFY_ONLINE,
+	CPUHP_AP_ONLINE_DYN,
+	CPUHP_AP_ONLINE_DYN_END		= CPUHP_AP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int
 /* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
 static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
-	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+	enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
 
 	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
 }
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned in
 			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
 		}
 	} else {
+		/* Cannot happen .... */
+		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
 		/* Regular hotplug work */
 		if (st->state < st->target)
 			ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int
 }
 
 /* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
-	enum cpuhp_state state = st->state;
-
-	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
 	st->result = 0;
 	st->cb = NULL;
 	/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned i
 	smp_mb();
 	st->should_run = true;
 	wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	__cpuhp_kick_ap_work(st);
 	wait_for_completion(&st->done);
 	trace_cpuhp_exit(cpu, st->state, state, st->result);
 	return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
+	/* Park the hotplug thread */
+	kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+
+		/*
+		 * We might have stopped still in the range of the AP hotplug
+		 * thread. Nothing to do anymore.
+		 */
+		if (st->state >= CPUHP_KICK_AP_THREAD)
+			goto out;
+	}
+	/*
+	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * to do the further cleanups.
+	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
 	cpu_hotplug_done();
 	/* This post dead nonsense must die */
 	if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cp
  */
 static int cpuhp_set_cpu_active(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread */
+	/* Unpark the stopper thread and the hotplug thread */
 	stop_machine_unpark(cpu);
+	kthread_unpark(st->thread);
 	return 0;
 }
 
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread once more.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+	}
+
+	/*
+	 * Try to reach the target state. We max out on the BP at
+	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * responsible for bringing it up to the target state.
+	 */
+	target = min((int)target, CPUHP_KICK_AP_THREAD);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
-	[CPUHP_SMPBOOT_THREADS] = {
-		.name			= "smpboot:threads",
-		.startup		= smpboot_unpark_threads,
-		.teardown		= smpboot_park_threads,
-	},
-	[CPUHP_NOTIFY_ONLINE] = {
-		.name			= "notify:online",
-		.startup		= notify_online,
-		.teardown		= notify_down_prepare,
-		.cant_stop		= true,
+	[CPUHP_KICK_AP_THREAD] = {
+		.name			= "cpuhp:kickthread",
+		.startup		= cpuhp_kick_ap_work,
+		.teardown		= cpuhp_kick_ap_work,
 	},
 #endif
-	[CPUHP_ONLINE] = {
+	[CPUHP_BP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states
 		.skip_onerr		= true,
 		.cant_stop		= true,
 	},
+	[CPUHP_AP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
+	[CPUHP_AP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
 #endif
 	[CPUHP_ONLINE] = {
 		.name			= "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+		return true;
+	return state > CPUHP_BP_ONLINE;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum
 	return cpuhp_get_step(state)->teardown;
 }
 
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
-	int (*cb)(unsigned int cpu) = __cb;
-
-	BUG_ON(cb(smp_processor_id()));
-}
-
 /*
  * Call the startup/teardown function for a step either on the AP or
  * on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enu
 
 	if (!cb)
 		return 0;
-
-	/*
-	 * This invokes the callback directly for now. In a later step we
-	 * convert that to use cpuhp_invoke_callback().
-	 */
-	if (cpuhp_is_ap_state(state)) {
-		/*
-		 * Note, that a function called on the AP is not
-		 * allowed to fail.
-		 */
-		if (cpu_online(cpu))
-			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
-		return 0;
-	}
-
 	/*
 	 * The non AP bound callbacks can fail on bringup. On teardown
 	 * e.g. module removal we crash for now.
 	 */
-	ret = cb(cpu);
+#ifdef CONFIG_SMP
+	if (cpuhp_is_ap_state(state))
+		ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+	else
+		ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+	ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
 	BUG_ON(ret && !bringup);
 	return ret;
 }
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuh
 	enum cpuhp_state i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
-		if (cpuhp_bp_states[i].name)
+	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+		if (cpuhp_ap_states[i].name)
 			continue;
 
-		cpuhp_bp_states[i].name = "Reserved";
+		cpuhp_ap_states[i].name = "Reserved";
 		mutex_unlock(&cpuhp_state_mutex);
 		return i;
 	}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state
 	get_online_cpus();
 
 	/* currently assignments for the ONLINE state are possible */
-	if (state == CPUHP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN) {
 		dyn_state = 1;
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 17/20] arch/hotplug: Call into idle with a proper state
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (15 preceding siblings ...)
  2016-02-26 18:43 ` [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner
                   ` (2 subsequent siblings)
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: arch-hotplug--Call-into-idle-with-a-proper-state.patch --]
[-- Type: text/plain, Size: 9949 bytes --]

Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/alpha/kernel/smp.c         |    2 +-
 arch/arc/kernel/smp.c           |    2 +-
 arch/arm/kernel/smp.c           |    2 +-
 arch/arm64/kernel/smp.c         |    2 +-
 arch/blackfin/mach-common/smp.c |    2 +-
 arch/hexagon/kernel/smp.c       |    2 +-
 arch/ia64/kernel/smpboot.c      |    2 +-
 arch/m32r/kernel/smpboot.c      |    2 +-
 arch/metag/kernel/smp.c         |    2 +-
 arch/mips/kernel/smp.c          |    2 +-
 arch/mn10300/kernel/smp.c       |    2 +-
 arch/parisc/kernel/smp.c        |    2 +-
 arch/powerpc/kernel/smp.c       |    2 +-
 arch/s390/kernel/smp.c          |    2 +-
 arch/sh/kernel/smp.c            |    2 +-
 arch/sparc/kernel/smp_32.c      |    2 +-
 arch/sparc/kernel/smp_64.c      |    2 +-
 arch/tile/kernel/smpboot.c      |    2 +-
 arch/x86/kernel/smpboot.c       |    2 +-
 arch/x86/xen/smp.c              |    2 +-
 arch/xtensa/kernel/smp.c        |    2 +-
 include/linux/cpuhotplug.h      |    1 +
 22 files changed, 22 insertions(+), 21 deletions(-)

Index: b/arch/alpha/kernel/smp.c
===================================================================
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
 	      cpuid, current, current->active_mm));
 
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Wait until hwrpb->txrdy is clear for cpu.  Return -1 on timeout.  */
Index: b/arch/arc/kernel/smp.c
===================================================================
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
 
 	local_irq_enable();
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /*
Index: b/arch/arm/kernel/smp.c
===================================================================
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/arm64/kernel/smp.c
===================================================================
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
Index: b/arch/blackfin/mach-common/smp.c
===================================================================
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
 
 	/* We are done with local CPU inits, unblock the boot CPU. */
 	set_cpu_online(cpu, true);
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_prepare_boot_cpu(void)
Index: b/arch/hexagon/kernel/smp.c
===================================================================
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 
Index: b/arch/ia64/kernel/smpboot.c
===================================================================
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
 	preempt_disable();
 	smp_callin();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/m32r/kernel/smpboot.c
===================================================================
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
 	 */
 	local_flush_tlb_all();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/metag/kernel/smp.c
===================================================================
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/mips/kernel/smp.c
===================================================================
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
 	WARN_ON_ONCE(!irqs_disabled());
 	mp_ops->smp_finish();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void stop_this_cpu(void *dummy)
Index: b/arch/mn10300/kernel/smp.c
===================================================================
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
 #ifdef CONFIG_GENERIC_CLOCKEVENTS
 	init_clockevents();
 #endif
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/parisc/kernel/smp.c
===================================================================
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
 
 	local_irq_enable();  /* Interrupts have been off until now */
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* NOTREACHED */
 	panic("smp_callin() AAAAaaaaahhhh....\n");
Index: b/arch/powerpc/kernel/smp.c
===================================================================
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	BUG();
 }
Index: b/arch/s390/kernel/smp.c
===================================================================
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cp
 	set_cpu_online(smp_processor_id(), true);
 	inc_irq_stat(CPU_RST);
 	local_irq_enable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Upping and downing of CPUs */
Index: b/arch/sh/kernel/smp.c
===================================================================
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
 	set_cpu_online(cpu, true);
 	per_cpu(cpu_state, cpu) = CPU_ONLINE;
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 extern struct {
Index: b/arch/sparc/kernel/smp_32.c
===================================================================
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *
 	local_irq_enable();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* We should never reach here! */
 	BUG();
Index: b/arch/sparc/kernel/smp_64.c
===================================================================
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void cpu_panic(void)
Index: b/arch/tile/kernel/smpboot.c
===================================================================
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
 	/* Set up tile-timer clock-event device on this cpu */
 	setup_tile_timer();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 int __cpu_up(unsigned int cpu, struct task_struct *tidle)
Index: b/arch/x86/kernel/smpboot.c
===================================================================
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void
 	x86_cpuinit.setup_percpu_clockev();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_store_boot_cpu_info(void)
Index: b/arch/x86/xen/smp.c
===================================================================
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_an
 		xen_pvh_secondary_vcpu_init(cpu);
 #endif
 	cpu_bringup();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void xen_smp_intr_free(unsigned int cpu)
Index: b/arch/xtensa/kernel/smp.c
===================================================================
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
 
 	complete(&cpu_running);
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void mx_cpu_start(void *p)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
 	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_KICK_AP_THREAD,
 	CPUHP_BP_ONLINE,
+	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
 	CPUHP_AP_ONLINE_DYN,

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (16 preceding siblings ...)
  2016-02-26 18:43 ` [patch 17/20] arch/hotplug: Call into idle with a proper state Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-03-02 17:28   ` [patch 18/20] " Richard Cochran
  2016-02-26 18:43 ` [patch 19/20] cpu/hotplug: Make wait for dead cpu completion based Thomas Gleixner
  2016-02-26 18:43 ` [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner
  19 siblings, 2 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Let-upcoming-cpu-bring-itself-fully-up.patch --]
[-- Type: text/plain, Size: 6415 bytes --]

Let the upcoming cpu kick the hotplug thread and let itself complete the
bringup. That way the controll side can just wait for the completion or later
when we made the hotplug machinery async not care at all.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    9 ++++--
 kernel/cpu.c               |   66 +++++++++++++++++++++++++--------------------
 kernel/sched/idle.c        |    2 +
 3 files changed, 45 insertions(+), 32 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,9 +10,6 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
-	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_KICK_AP_THREAD,
-	CPUHP_BP_ONLINE,
 	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
@@ -86,4 +83,10 @@ static inline void cpuhp_remove_state_no
 	__cpuhp_remove_state(state, false);
 }
 
+#ifdef CONFIG_SMP
+void cpuhp_online_idle(enum cpuhp_state state);
+#else
+static inline void cpuhp_online_idle(enum cpuhp_state state) { }
+#endif
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,6 +329,14 @@ static int notify_starting(unsigned int
 	return 0;
 }
 
+static int bringup_wait_for_ap(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -340,8 +348,9 @@ static int bringup_cpu(unsigned int cpu)
 		cpu_notify(CPU_UP_CANCELED, cpu);
 		return ret;
 	}
+	ret = bringup_wait_for_ap(cpu);
 	BUG_ON(!cpu_online(cpu));
-	return 0;
+	return ret;
 }
 
 /*
@@ -470,7 +479,7 @@ static void cpuhp_thread_fun(unsigned in
 		}
 	} else {
 		/* Cannot happen .... */
-		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+		BUG_ON(st->state < CPUHP_AP_ONLINE_IDLE);
 
 		/* Regular hotplug work */
 		if (st->state < st->target)
@@ -780,7 +789,7 @@ static int __ref _cpu_down(unsigned int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_TEARDOWN_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -793,11 +802,11 @@ static int __ref _cpu_down(unsigned int
 		 * We might have stopped still in the range of the AP hotplug
 		 * thread. Nothing to do anymore.
 		 */
-		if (st->state >= CPUHP_KICK_AP_THREAD)
+		if (st->state > CPUHP_TEARDOWN_CPU)
 			goto out;
 	}
 	/*
-	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * The AP brought itself down to CPUHP_TEARDOWN_CPU. So we need
 	 * to do the further cleanups.
 	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
@@ -859,18 +868,32 @@ void notify_cpu_starting(unsigned int cp
 
 /*
  * Called from the idle task. We need to set active here, so we can kick off
- * the stopper thread.
+ * the stopper thread and unpark the smpboot threads. If the target state is
+ * beyond CPUHP_AP_ONLINE_IDLE we kick cpuhp thread and let it bring up the
+ * cpu further.
  */
-static int cpuhp_set_cpu_active(unsigned int cpu)
+void cpuhp_online_idle(enum cpuhp_state state)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	unsigned int cpu = smp_processor_id();
+
+	/* Happens for the boot cpu */
+	if (state != CPUHP_AP_ONLINE_IDLE)
+		return;
+
+	st->state = CPUHP_AP_ONLINE_IDLE;
 
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread and the hotplug thread */
+	/* Unpark the stopper thread and the hotplug thread of this cpu */
 	stop_machine_unpark(cpu);
 	kthread_unpark(st->thread);
-	return 0;
+
+	/* Should we go further up ? */
+	if (st->target > CPUHP_AP_ONLINE_IDLE)
+		__cpuhp_kick_ap_work(st);
+	else
+		complete(&st->done);
 }
 
 /* Requires cpu_add_remove_lock to be held */
@@ -910,7 +933,7 @@ static int _cpu_up(unsigned int cpu, int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread once more.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_BRINGUP_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -922,10 +945,10 @@ static int _cpu_up(unsigned int cpu, int
 
 	/*
 	 * Try to reach the target state. We max out on the BP at
-	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * CPUHP_BRINGUP_CPU. After that the AP hotplug thread is
 	 * responsible for bringing it up to the target state.
 	 */
-	target = min((int)target, CPUHP_KICK_AP_THREAD);
+	target = min((int)target, CPUHP_BRINGUP_CPU);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1146,22 +1169,7 @@ static struct cpuhp_step cpuhp_bp_states
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
-	[CPUHP_CPU_SET_ACTIVE] = {
-		.name			= "cpu:active",
-		.startup		= cpuhp_set_cpu_active,
-		.teardown		= NULL,
-	},
-	[CPUHP_KICK_AP_THREAD] = {
-		.name			= "cpuhp:kickthread",
-		.startup		= cpuhp_kick_ap_work,
-		.teardown		= cpuhp_kick_ap_work,
-	},
 #endif
-	[CPUHP_BP_ONLINE] = {
-		.name			= "online",
-		.startup		= NULL,
-		.teardown		= NULL,
-	},
 };
 
 /* Application processor state steps */
@@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp
 {
 	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
 		return true;
-	return state > CPUHP_BP_ONLINE;
+	return state > CPUHP_BRINGUP_CPU;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -4,6 +4,7 @@
 #include <linux/sched.h>
 #include <linux/cpu.h>
 #include <linux/cpuidle.h>
+#include <linux/cpuhotplug.h>
 #include <linux/tick.h>
 #include <linux/mm.h>
 #include <linux/stackprotector.h>
@@ -276,5 +277,6 @@ static void cpu_idle_loop(void)
 void cpu_startup_entry(enum cpuhp_state state)
 {
 	arch_cpu_idle_prepare();
+	cpuhp_online_idle(state);
 	cpu_idle_loop();
 }

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 19/20] cpu/hotplug: Make wait for dead cpu completion based
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (17 preceding siblings ...)
  2016-02-26 18:43 ` [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-02-26 18:43 ` [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner
  19 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: hotplug--Make-wait-for-dead-cpu-completion-based.patch --]
[-- Type: text/plain, Size: 3225 bytes --]

Kill the busy spinning on the control side and just wait for the hotplugged
cpu to tell that it reached the dead state.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    5 +++--
 include/linux/cpuhotplug.h |    1 +
 kernel/cpu.c               |   16 ++++++++++++----
 kernel/sched/idle.c        |    5 +----
 4 files changed, 17 insertions(+), 10 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -276,14 +276,15 @@ void arch_cpu_idle_enter(void);
 void arch_cpu_idle_exit(void);
 void arch_cpu_idle_dead(void);
 
-DECLARE_PER_CPU(bool, cpu_dead_idle);
-
 int cpu_report_state(int cpu);
 int cpu_check_up_prepare(int cpu);
 void cpu_set_state_online(int cpu);
 #ifdef CONFIG_HOTPLUG_CPU
 bool cpu_wait_death(unsigned int cpu, int seconds);
 bool cpu_report_death(void);
+void cpuhp_report_idle_dead(void);
+#else
+static inline void cpuhp_report_idle_dead(void) { }
 #endif /* #ifdef CONFIG_HOTPLUG_CPU */
 
 #endif /* _LINUX_CPU_H_ */
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,7 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -688,6 +688,7 @@ static int take_cpu_down(void *_param)
 
 static int takedown_cpu(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int err;
 
 	/*
@@ -733,10 +734,8 @@ static int takedown_cpu(unsigned int cpu
 	 *
 	 * Wait for the stop thread to go away.
 	 */
-	while (!per_cpu(cpu_dead_idle, cpu))
-		cpu_relax();
-	smp_mb(); /* Read from cpu_dead_idle before __cpu_die(). */
-	per_cpu(cpu_dead_idle, cpu) = false;
+	wait_for_completion(&st->done);
+	BUG_ON(st->state != CPUHP_AP_IDLE_DEAD);
 
 	/* Interrupts are moved away from the dying cpu, reenable alloc/free */
 	irq_unlock_sparse();
@@ -756,6 +755,15 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+void cpuhp_report_idle_dead(void)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	BUG_ON(st->state != CPUHP_AP_OFFLINE);
+	st->state = CPUHP_AP_IDLE_DEAD;
+	complete(&st->done);
+}
+
 #else
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -194,8 +194,6 @@ static void cpuidle_idle_call(void)
 	rcu_idle_exit();
 }
 
-DEFINE_PER_CPU(bool, cpu_dead_idle);
-
 /*
  * Generic idle loop implementation
  *
@@ -224,8 +222,7 @@ static void cpu_idle_loop(void)
 			if (cpu_is_offline(smp_processor_id())) {
 				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
 					       (void *)(long)smp_processor_id());
-				smp_mb(); /* all activity before dead. */
-				this_cpu_write(cpu_dead_idle, true);
+				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}
 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
                   ` (18 preceding siblings ...)
  2016-02-26 18:43 ` [patch 19/20] cpu/hotplug: Make wait for dead cpu completion based Thomas Gleixner
@ 2016-02-26 18:43 ` Thomas Gleixner
  2016-02-27  2:14   ` Paul E. McKenney
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  19 siblings, 2 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-26 18:43 UTC (permalink / raw)
  To: LKML
  Cc: Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

[-- Attachment #1: rcu--Make-CPU_DYING_IDLE-an-explicit-call.patch --]
[-- Type: text/plain, Size: 3849 bytes --]

Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
invoked at the proper place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h      |    4 +---
 include/linux/notifier.h |    2 ++
 include/linux/rcupdate.h |    4 +---
 kernel/cpu.c             |    1 +
 kernel/rcu/tree.c        |   26 +++++++++++++++-----------
 kernel/sched/idle.c      |    2 --
 6 files changed, 20 insertions(+), 19 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -101,9 +101,7 @@ enum {
 					* Called on the new cpu, just before
 					* enabling interrupts. Must not sleep,
 					* must not fail */
-#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
-					* idle loop. */
-#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
+#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
 					* perhaps due to preemption. */
 
 /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
Index: b/include/linux/notifier.h
===================================================================
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -47,6 +47,8 @@
  * runtime initialization.
  */
 
+struct notifier_block;
+
 typedef	int (*notifier_fn_t)(struct notifier_block *nb,
 			unsigned long action, void *data);
 
Index: b/include/linux/rcupdate.h
===================================================================
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -332,9 +332,7 @@ void rcu_init(void);
 void rcu_sched_qs(void);
 void rcu_bh_qs(void);
 void rcu_check_callbacks(int user);
-struct notifier_block;
-int rcu_cpu_notify(struct notifier_block *self,
-		   unsigned long action, void *hcpu);
+void rcu_report_dead(unsigned int cpu);
 
 #ifndef CONFIG_TINY_RCU
 void rcu_end_inkernel_boot(void);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
 	BUG_ON(st->state != CPUHP_AP_OFFLINE);
 	st->state = CPUHP_AP_IDLE_DEAD;
 	complete(&st->done);
+	rcu_report_dead(smp_processor_id());
 }
 
 #else
Index: b/kernel/rcu/tree.c
===================================================================
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
 		rcu_init_percpu_data(cpu, rsp);
 }
 
+#ifdef CONFIG_HOTPLUG_CPU
+void rcu_report_dead(unsigned int cpu)
+{
+	struct rcu_state *rsp;
+
+	/* QS for any half-done expedited RCU-sched GP. */
+	preempt_disable();
+	rcu_report_exp_rdp(&rcu_sched_state,
+			   this_cpu_ptr(rcu_sched_state.rda), true);
+	preempt_enable();
+	for_each_rcu_flavor(rsp)
+		rcu_cleanup_dying_idle_cpu(cpu, rsp);
+}
+#endif
+
 /*
  * Handle CPU online/offline notification events.
  */
@@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dying_cpu(rsp);
 		break;
-	case CPU_DYING_IDLE:
-		/* QS for any half-done expedited RCU-sched GP. */
-		preempt_disable();
-		rcu_report_exp_rdp(&rcu_sched_state,
-				   this_cpu_ptr(rcu_sched_state.rda), true);
-		preempt_enable();
-
-		for_each_rcu_flavor(rsp) {
-			rcu_cleanup_dying_idle_cpu(cpu, rsp);
-		}
-		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
 			rmb();
 
 			if (cpu_is_offline(smp_processor_id())) {
-				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
-					       (void *)(long)smp_processor_id());
 				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-26 18:43 ` [patch 10/20] cpu/hotplug: Make target state writeable Thomas Gleixner
@ 2016-02-26 23:46   ` Rafael J. Wysocki
  2016-02-27  7:39     ` Thomas Gleixner
  2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  1 sibling, 1 reply; 69+ messages in thread
From: Rafael J. Wysocki @ 2016-02-26 23:46 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

Hi Thomas,

On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> Make it possible to write a target state to the per cpu state file, so we can
> switch between states.

One thing that potentially may be problematic here is that any kind of
"offline" operations needs to be carried out under device_hotplug_lock,
because there are cases in which devices (including CPUs) are taken
offline in groups and if one offline fails, the whole operation has to
be rolled back.

So if you put a CPU into one of the intermediate states manually and
something like the above happens in parallel with it, they may not
play well together IMO.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-26 18:43 ` [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner
@ 2016-02-27  2:14   ` Paul E. McKenney
  2016-02-27  2:23     ` Paul E. McKenney
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  1 sibling, 1 reply; 69+ messages in thread
From: Paul E. McKenney @ 2016-02-27  2:14 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Fri, Feb 26, 2016 at 06:43:44PM -0000, Thomas Gleixner wrote:
> Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
> invoked at the proper place.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

A question below...

> ---
>  include/linux/cpu.h      |    4 +---
>  include/linux/notifier.h |    2 ++
>  include/linux/rcupdate.h |    4 +---
>  kernel/cpu.c             |    1 +
>  kernel/rcu/tree.c        |   26 +++++++++++++++-----------
>  kernel/sched/idle.c      |    2 --
>  6 files changed, 20 insertions(+), 19 deletions(-)
> 
> Index: b/include/linux/cpu.h
> ===================================================================
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -101,9 +101,7 @@ enum {
>  					* Called on the new cpu, just before
>  					* enabling interrupts. Must not sleep,
>  					* must not fail */
> -#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
> -					* idle loop. */
> -#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
> +#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
>  					* perhaps due to preemption. */
> 
>  /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
> Index: b/include/linux/notifier.h
> ===================================================================
> --- a/include/linux/notifier.h
> +++ b/include/linux/notifier.h
> @@ -47,6 +47,8 @@
>   * runtime initialization.
>   */
> 
> +struct notifier_block;
> +
>  typedef	int (*notifier_fn_t)(struct notifier_block *nb,
>  			unsigned long action, void *data);
> 
> Index: b/include/linux/rcupdate.h
> ===================================================================
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -332,9 +332,7 @@ void rcu_init(void);
>  void rcu_sched_qs(void);
>  void rcu_bh_qs(void);
>  void rcu_check_callbacks(int user);
> -struct notifier_block;
> -int rcu_cpu_notify(struct notifier_block *self,
> -		   unsigned long action, void *hcpu);
> +void rcu_report_dead(unsigned int cpu);
> 
>  #ifndef CONFIG_TINY_RCU
>  void rcu_end_inkernel_boot(void);
> Index: b/kernel/cpu.c
> ===================================================================
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
>  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
>  	st->state = CPUHP_AP_IDLE_DEAD;
>  	complete(&st->done);

What prevents the other CPU from killing this CPU at this point, so
that this CPU does not tell RCU that it is dead?

I agree that the odds should be low, but there are all manner of things
that might delay a CPU for just a little bit too long...

Or am I missing something subtle here?

							Thanx, Paul

> +	rcu_report_dead(smp_processor_id());
>  }
> 
>  #else
> Index: b/kernel/rcu/tree.c
> ===================================================================
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
>  		rcu_init_percpu_data(cpu, rsp);
>  }
> 
> +#ifdef CONFIG_HOTPLUG_CPU
> +void rcu_report_dead(unsigned int cpu)
> +{
> +	struct rcu_state *rsp;
> +
> +	/* QS for any half-done expedited RCU-sched GP. */
> +	preempt_disable();
> +	rcu_report_exp_rdp(&rcu_sched_state,
> +			   this_cpu_ptr(rcu_sched_state.rda), true);
> +	preempt_enable();
> +	for_each_rcu_flavor(rsp)
> +		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> +}
> +#endif
> +
>  /*
>   * Handle CPU online/offline notification events.
>   */
> @@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
>  		for_each_rcu_flavor(rsp)
>  			rcu_cleanup_dying_cpu(rsp);
>  		break;
> -	case CPU_DYING_IDLE:
> -		/* QS for any half-done expedited RCU-sched GP. */
> -		preempt_disable();
> -		rcu_report_exp_rdp(&rcu_sched_state,
> -				   this_cpu_ptr(rcu_sched_state.rda), true);
> -		preempt_enable();
> -
> -		for_each_rcu_flavor(rsp) {
> -			rcu_cleanup_dying_idle_cpu(cpu, rsp);
> -		}
> -		break;
>  	case CPU_DEAD:
>  	case CPU_DEAD_FROZEN:
>  	case CPU_UP_CANCELED:
> Index: b/kernel/sched/idle.c
> ===================================================================
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
>  			rmb();
> 
>  			if (cpu_is_offline(smp_processor_id())) {
> -				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
> -					       (void *)(long)smp_processor_id());
>  				cpuhp_report_idle_dead();
>  				arch_cpu_idle_dead();
>  			}
> 
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-27  2:14   ` Paul E. McKenney
@ 2016-02-27  2:23     ` Paul E. McKenney
  2016-02-27  7:47       ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Paul E. McKenney @ 2016-02-27  2:23 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Fri, Feb 26, 2016 at 06:14:29PM -0800, Paul E. McKenney wrote:
> On Fri, Feb 26, 2016 at 06:43:44PM -0000, Thomas Gleixner wrote:
> > Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
> > invoked at the proper place.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> 
> A question below...
> 
> > ---
> >  include/linux/cpu.h      |    4 +---
> >  include/linux/notifier.h |    2 ++
> >  include/linux/rcupdate.h |    4 +---
> >  kernel/cpu.c             |    1 +
> >  kernel/rcu/tree.c        |   26 +++++++++++++++-----------
> >  kernel/sched/idle.c      |    2 --
> >  6 files changed, 20 insertions(+), 19 deletions(-)
> > 
> > Index: b/include/linux/cpu.h
> > ===================================================================
> > --- a/include/linux/cpu.h
> > +++ b/include/linux/cpu.h
> > @@ -101,9 +101,7 @@ enum {
> >  					* Called on the new cpu, just before
> >  					* enabling interrupts. Must not sleep,
> >  					* must not fail */
> > -#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
> > -					* idle loop. */
> > -#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
> > +#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
> >  					* perhaps due to preemption. */
> > 
> >  /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
> > Index: b/include/linux/notifier.h
> > ===================================================================
> > --- a/include/linux/notifier.h
> > +++ b/include/linux/notifier.h
> > @@ -47,6 +47,8 @@
> >   * runtime initialization.
> >   */
> > 
> > +struct notifier_block;
> > +
> >  typedef	int (*notifier_fn_t)(struct notifier_block *nb,
> >  			unsigned long action, void *data);
> > 
> > Index: b/include/linux/rcupdate.h
> > ===================================================================
> > --- a/include/linux/rcupdate.h
> > +++ b/include/linux/rcupdate.h
> > @@ -332,9 +332,7 @@ void rcu_init(void);
> >  void rcu_sched_qs(void);
> >  void rcu_bh_qs(void);
> >  void rcu_check_callbacks(int user);
> > -struct notifier_block;
> > -int rcu_cpu_notify(struct notifier_block *self,
> > -		   unsigned long action, void *hcpu);
> > +void rcu_report_dead(unsigned int cpu);
> > 
> >  #ifndef CONFIG_TINY_RCU
> >  void rcu_end_inkernel_boot(void);
> > Index: b/kernel/cpu.c
> > ===================================================================
> > --- a/kernel/cpu.c
> > +++ b/kernel/cpu.c
> > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> >  	st->state = CPUHP_AP_IDLE_DEAD;
> >  	complete(&st->done);
> 
> What prevents the other CPU from killing this CPU at this point, so
> that this CPU does not tell RCU that it is dead?
> 
> I agree that the odds should be low, but there are all manner of things
> that might delay a CPU for just a little bit too long...
> 
> Or am I missing something subtle here?

Just in case I am not missing anything...

One approach is to go back to the spinning, but to do rcu_report_dead()
just before kicking the other CPU.  This would also fix some issues with
use of RCU of the offline path, so would definitely be better than my
earlier approach of notifying RCU from within the idle loop.

This assumes that all the offline paths have been consolidated into
this path.  (Yes, I was too lazy and cowardly to consolidate them all
last I touched this code, but perhaps that has happened elsewise?)

							Thanx, Paul

> > +	rcu_report_dead(smp_processor_id());
> >  }
> > 
> >  #else
> > Index: b/kernel/rcu/tree.c
> > ===================================================================
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
> >  		rcu_init_percpu_data(cpu, rsp);
> >  }
> > 
> > +#ifdef CONFIG_HOTPLUG_CPU
> > +void rcu_report_dead(unsigned int cpu)
> > +{
> > +	struct rcu_state *rsp;
> > +
> > +	/* QS for any half-done expedited RCU-sched GP. */
> > +	preempt_disable();
> > +	rcu_report_exp_rdp(&rcu_sched_state,
> > +			   this_cpu_ptr(rcu_sched_state.rda), true);
> > +	preempt_enable();
> > +	for_each_rcu_flavor(rsp)
> > +		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> > +}
> > +#endif
> > +
> >  /*
> >   * Handle CPU online/offline notification events.
> >   */
> > @@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
> >  		for_each_rcu_flavor(rsp)
> >  			rcu_cleanup_dying_cpu(rsp);
> >  		break;
> > -	case CPU_DYING_IDLE:
> > -		/* QS for any half-done expedited RCU-sched GP. */
> > -		preempt_disable();
> > -		rcu_report_exp_rdp(&rcu_sched_state,
> > -				   this_cpu_ptr(rcu_sched_state.rda), true);
> > -		preempt_enable();
> > -
> > -		for_each_rcu_flavor(rsp) {
> > -			rcu_cleanup_dying_idle_cpu(cpu, rsp);
> > -		}
> > -		break;
> >  	case CPU_DEAD:
> >  	case CPU_DEAD_FROZEN:
> >  	case CPU_UP_CANCELED:
> > Index: b/kernel/sched/idle.c
> > ===================================================================
> > --- a/kernel/sched/idle.c
> > +++ b/kernel/sched/idle.c
> > @@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
> >  			rmb();
> > 
> >  			if (cpu_is_offline(smp_processor_id())) {
> > -				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
> > -					       (void *)(long)smp_processor_id());
> >  				cpuhp_report_idle_dead();
> >  				arch_cpu_idle_dead();
> >  			}
> > 
> > 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-26 23:46   ` Rafael J. Wysocki
@ 2016-02-27  7:39     ` Thomas Gleixner
  2016-02-27 14:43       ` Rafael J. Wysocki
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-27  7:39 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

Rafael,

On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> > Make it possible to write a target state to the per cpu state file, so we can
> > switch between states.
> 
> One thing that potentially may be problematic here is that any kind of
> "offline" operations needs to be carried out under device_hotplug_lock,
> because there are cases in which devices (including CPUs) are taken
> offline in groups and if one offline fails, the whole operation has to
> be rolled back.
>
> So if you put a CPU into one of the intermediate states manually and
> something like the above happens in parallel with it, they may not
> play well together IMO.

I don't see how that is related. device_hotplug_lock is completely independent
of cpu hotplug today, unless I'm missing some magic connection here.

Physical CPU hotplug is a different story, but that's about bringing the cpus
into the system or taking them out. Sure, if you want to take one or more cpus
physically out, you have to bring them offline first. If you plug them in then
it's not necessarily related to actually bringing them online. That's a
different set of operations.

We surely need to look into that aspect, but I don't see a reason why e.g. a
device hotplug operation should be in any way related to the intermediate
state of a particular cpu. If that's the case, then there is something really
wrong.

I'm aware that we have a gazillion of silly assumptions all over the place and
some of them are wrong today and just do not explode in our face simply
because it's extremly hard to trigger. That's one reason why we need to go
through all the cpu notifier related sites and inspect them deeply.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-27  2:23     ` Paul E. McKenney
@ 2016-02-27  7:47       ` Thomas Gleixner
  2016-02-27 11:05         ` Paul E. McKenney
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-27  7:47 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > --- a/kernel/cpu.c
> > > +++ b/kernel/cpu.c
> > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > >  	complete(&st->done);
> > 
> > What prevents the other CPU from killing this CPU at this point, so
> > that this CPU does not tell RCU that it is dead?
> >
> > I agree that the odds should be low, but there are all manner of things
> > that might delay a CPU for just a little bit too long...
> > 
> > Or am I missing something subtle here?

No. The reason why I moved the rcu call past the complete is, that otherwise
complete() complains about rcu being dead already. Hmm, but you are right. In
theory the other side could allow physical removal before it actually told rcu
that it's gone.

> Just in case I am not missing anything...
> 
> One approach is to go back to the spinning, but to do rcu_report_dead()
> just before kicking the other CPU.  This would also fix some issues with
> use of RCU of the offline path, so would definitely be better than my
> earlier approach of notifying RCU from within the idle loop.
> 
> This assumes that all the offline paths have been consolidated into
> this path.  (Yes, I was too lazy and cowardly to consolidate them all
> last I touched this code, but perhaps that has happened elsewise?)

The question is whether the rcu dead notification has to happen
instantaniously and needs to be done on the dead cpu. If we can avoid both,
then there is a very simple solution.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-27  7:47       ` Thomas Gleixner
@ 2016-02-27 11:05         ` Paul E. McKenney
  2016-02-27 11:30           ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Paul E. McKenney @ 2016-02-27 11:05 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > --- a/kernel/cpu.c
> > > > +++ b/kernel/cpu.c
> > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > >  	complete(&st->done);
> > > 
> > > What prevents the other CPU from killing this CPU at this point, so
> > > that this CPU does not tell RCU that it is dead?
> > >
> > > I agree that the odds should be low, but there are all manner of things
> > > that might delay a CPU for just a little bit too long...
> > > 
> > > Or am I missing something subtle here?
> 
> No. The reason why I moved the rcu call past the complete is, that otherwise
> complete() complains about rcu being dead already. Hmm, but you are right. In
> theory the other side could allow physical removal before it actually told rcu
> that it's gone.

There is one case where this is OK, and that is where the outgoing CPU
puts itself to sleep (or whatever) without help from the other CPU.

> > Just in case I am not missing anything...
> > 
> > One approach is to go back to the spinning, but to do rcu_report_dead()
> > just before kicking the other CPU.  This would also fix some issues with
> > use of RCU of the offline path, so would definitely be better than my
> > earlier approach of notifying RCU from within the idle loop.
> > 
> > This assumes that all the offline paths have been consolidated into
> > this path.  (Yes, I was too lazy and cowardly to consolidate them all
> > last I touched this code, but perhaps that has happened elsewise?)
> 
> The question is whether the rcu dead notification has to happen
> instantaniously and needs to be done on the dead cpu. If we can avoid both,
> then there is a very simple solution.

Hmmm...

The rcu_cleanup_dying_idle_cpu() can be invoked from the surviving CPU,
-as- -long- -as- nothing in the intervening code path waits for a grace
period.  The wakeup path itself had better not wait for a grace period,
of course.  The concern would be that the task running on the surviving
CPU might be waiting for a grace period before sleeping -- which used to
be possible due to the CPU-hotplug notifiers that it might be executing
before getting to RCU's CPU-hotplug notifiers.

The rcu_report_exp_rdp() is considerably more scary.  At first glance,
it looks OK, but I will need to stare at it for a bit.

Of course, if the task running on the surviving CPU can be waiting for a
grace period, one of two problems can happen:

o	RCU times out the outgoing CPU before it has really left.
	This is the current state, and needs to change.  The dying idle
	stuff was half of the needed change, the other half being on
	the incoming side.

o	Deadlock -- the outgoing CPU won't respond to RCU, so the
	task running on the surviving CPU never wakes up from its
	wait on a grace period.

So if rcu_report_exp_rdp() turns out to be OK -and- if the outgoing
task never waits on a grace period during the CPU-shutdown process,
this might work.

Of course, my ability to test this sufficiently viciously is currently
blocked by the lost-wakeup problem I am currently chasing.  (Hey, at
least I finally get ftrace output!  Completely baffling output, but so
it goes...)  :-/

							Thanx, Paul

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-27 11:05         ` Paul E. McKenney
@ 2016-02-27 11:30           ` Thomas Gleixner
  2016-02-27 16:33             ` Paul E. McKenney
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-27 11:30 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Sat, 27 Feb 2016, Paul E. McKenney wrote:
> On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> > On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > > --- a/kernel/cpu.c
> > > > > +++ b/kernel/cpu.c
> > > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > > >  	complete(&st->done);
> > > > 
> > > > What prevents the other CPU from killing this CPU at this point, so
> > > > that this CPU does not tell RCU that it is dead?
> > > >
> > > > I agree that the odds should be low, but there are all manner of things
> > > > that might delay a CPU for just a little bit too long...
> > > > 
> > > > Or am I missing something subtle here?
> > 
> > No. The reason why I moved the rcu call past the complete is, that otherwise
> > complete() complains about rcu being dead already. Hmm, but you are right. In
> > theory the other side could allow physical removal before it actually told rcu
> > that it's gone.
> 
> There is one case where this is OK, and that is where the outgoing CPU
> puts itself to sleep (or whatever) without help from the other CPU.

That's the case. It's the last call before the outgoing CPU goes into
arch_cpu_idle_dead(). There is no involvement of the controlling CPU at this
point. It just wants to know, that the outgoing one is dead finally.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-27  7:39     ` Thomas Gleixner
@ 2016-02-27 14:43       ` Rafael J. Wysocki
  2016-02-28 14:49         ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Rafael J. Wysocki @ 2016-02-27 14:43 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

On Saturday, February 27, 2016 08:39:42 AM Thomas Gleixner wrote:
> Rafael,
> 
> On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> > On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> > > Make it possible to write a target state to the per cpu state file, so we can
> > > switch between states.
> > 
> > One thing that potentially may be problematic here is that any kind of
> > "offline" operations needs to be carried out under device_hotplug_lock,
> > because there are cases in which devices (including CPUs) are taken
> > offline in groups and if one offline fails, the whole operation has to
> > be rolled back.
> >
> > So if you put a CPU into one of the intermediate states manually and
> > something like the above happens in parallel with it, they may not
> > play well together IMO.
> 
> I don't see how that is related. device_hotplug_lock is completely independent
> of cpu hotplug today, unless I'm missing some magic connection here.

Well, there is a magic connection which is my point.  That's mostly about
physical hot-remove.

> Physical CPU hotplug is a different story, but that's about bringing the cpus
> into the system or taking them out. Sure, if you want to take one or more cpus
> physically out, you have to bring them offline first. If you plug them in then
> it's not necessarily related to actually bringing them online. That's a
> different set of operations.

So that's mostly about the hot-remove part.  Namely, devices may need to go
away together (like in one package), so we need to offline them together first.

That's because generally offline may fail, for example for memory, and now
if a CPU is bundled to a set of memory that cannot be taken offline, it
can't be hot-removed too.  If offline fails for one component, we roll back,
but if it is successful for all of them, we can eject the whole bundle and
that's where the problem resides.

Say we've taken all of them offline and now we are ready to eject.  If an
online from sysfs (or any other place) comes in at this point, we'll be
ejecting a CPU that's potentially doing something which is not awesome.

That's why we have device_hotplug_lock and some ugly code related to it.

It extends to parents and children somewhat because of device objects
representing packages (we want those to be "offline" only if all their
children are offline) and that's why the lock is held around offline from
sysfs too.

I'm not entirely happy with this for quite obvious reasons, but it gets
the job done ATM.

> We surely need to look into that aspect, but I don't see a reason why e.g. a
> device hotplug operation should be in any way related to the intermediate
> state of a particular cpu. If that's the case, then there is something really
> wrong.

If that state is different from complete offline, we should not try to eject
(the package containing) that CPU.

> I'm aware that we have a gazillion of silly assumptions all over the place and
> some of them are wrong today and just do not explode in our face simply
> because it's extremly hard to trigger. That's one reason why we need to go
> through all the cpu notifier related sites and inspect them deeply.

Agreed, but the particular concern I'm talking about is not in that category IMO.

Thanks,
Rafael

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-27 11:30           ` Thomas Gleixner
@ 2016-02-27 16:33             ` Paul E. McKenney
  0 siblings, 0 replies; 69+ messages in thread
From: Paul E. McKenney @ 2016-02-27 16:33 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Rafael Wysocki, Arjan van de Ven,
	Rik van Riel, Srivatsa S. Bhat, Sebastian Siewior, Paul Turner

On Sat, Feb 27, 2016 at 12:30:33PM +0100, Thomas Gleixner wrote:
> On Sat, 27 Feb 2016, Paul E. McKenney wrote:
> > On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> > > On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > > > --- a/kernel/cpu.c
> > > > > > +++ b/kernel/cpu.c
> > > > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > > > >  	complete(&st->done);
> > > > > 
> > > > > What prevents the other CPU from killing this CPU at this point, so
> > > > > that this CPU does not tell RCU that it is dead?
> > > > >
> > > > > I agree that the odds should be low, but there are all manner of things
> > > > > that might delay a CPU for just a little bit too long...
> > > > > 
> > > > > Or am I missing something subtle here?
> > > 
> > > No. The reason why I moved the rcu call past the complete is, that otherwise
> > > complete() complains about rcu being dead already. Hmm, but you are right. In
> > > theory the other side could allow physical removal before it actually told rcu
> > > that it's gone.
> > 
> > There is one case where this is OK, and that is where the outgoing CPU
> > puts itself to sleep (or whatever) without help from the other CPU.
> 
> That's the case. It's the last call before the outgoing CPU goes into
> arch_cpu_idle_dead(). There is no involvement of the controlling CPU at this
> point. It just wants to know, that the outgoing one is dead finally.

Ah, so you have gotten rid of all the things like arm's and xtensa's
platform_cpu_kill(), where the surviving CPU does things like stopping
the outgoing CPU's clock?  That would make things simpler!

							Thanx, Paul

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 01/20] idle: Move x86ism out of generic code
  2016-02-26 18:43 ` [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner
@ 2016-02-27 20:29   ` Brian Gerst
  2016-02-29 19:35     ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Brian Gerst @ 2016-02-27 20:29 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

On Fri, Feb 26, 2016 at 1:43 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> We have an arch specific callback here already.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/kernel/process.c |   12 ++++++++++++
>  kernel/sched/idle.c       |   15 ---------------
>  2 files changed, 12 insertions(+), 15 deletions(-)
>
> Index: b/arch/x86/kernel/process.c
> ===================================================================
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -271,6 +271,18 @@ void exit_idle(void)
>  }
>  #endif
>
> +void arch_cpu_idle_prepare(void)
> +{
> +       /*
> +        * If we're the non-boot CPU, nothing set the stack canary up
> +        * for us. The boot CPU already has it initialized but no harm
> +        * in doing it again. This is a good place for updating it, as
> +        * we wont ever return from this function (so the invalid
> +        * canaries already on the stack wont ever trigger).
> +        */
> +       boot_init_stack_canary();
> +}
> +
>  void arch_cpu_idle_enter(void)
>  {
>         local_touch_nmi();
> Index: b/kernel/sched/idle.c
> ===================================================================
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -275,21 +275,6 @@ static void cpu_idle_loop(void)
>
>  void cpu_startup_entry(enum cpuhp_state state)
>  {
> -       /*
> -        * This #ifdef needs to die, but it's too late in the cycle to
> -        * make this generic (arm and sh have never invoked the canary
> -        * init for the non boot cpus!). Will be fixed in 3.11
> -        */
> -#ifdef CONFIG_X86
> -       /*
> -        * If we're the non-boot CPU, nothing set the stack canary up
> -        * for us. The boot CPU already has it initialized but no harm
> -        * in doing it again. This is a good place for updating it, as
> -        * we wont ever return from this function (so the invalid
> -        * canaries already on the stack wont ever trigger).
> -        */
> -       boot_init_stack_canary();
> -#endif
>         arch_cpu_idle_prepare();
>         cpu_idle_loop();
>  }

Does this actually work with stack protector enabled?
boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
not.

--
Brian Gerst

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-27 14:43       ` Rafael J. Wysocki
@ 2016-02-28 14:49         ` Thomas Gleixner
  2016-02-29 15:49           ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-28 14:49 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

Rafael,

On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> Say we've taken all of them offline and now we are ready to eject.  If an
> online from sysfs (or any other place) comes in at this point, we'll be
> ejecting a CPU that's potentially doing something which is not awesome.
> 
> That's why we have device_hotplug_lock and some ugly code related to it.
> 
> It extends to parents and children somewhat because of device objects
> representing packages (we want those to be "offline" only if all their
> children are offline) and that's why the lock is held around offline from
> sysfs too.
> 
> I'm not entirely happy with this for quite obvious reasons, but it gets
> the job done ATM.

Understood. I'll fix that thing up so that won't happen and I put it on the
list of things to look at deeper.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-28 14:49         ` Thomas Gleixner
@ 2016-02-29 15:49           ` Thomas Gleixner
  2016-03-01  1:53             ` Rafael J. Wysocki
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-29 15:49 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

On Sun, 28 Feb 2016, Thomas Gleixner wrote:
> 
> Understood. I'll fix that thing up so that won't happen and I put it on the
> list of things to look at deeper.

The below delta patch should do what you need, right?

Thanks,

	tglx

8<-------------

--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1461,6 +1461,11 @@ static ssize_t write_cpuhp_target(struct
 	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
 		return -EINVAL;
 #endif
+
+	ret = lock_device_hotplug_sysfs();
+	if (ret)
+		return ret;
+
 	mutex_lock(&cpuhp_state_mutex);
 	sp = cpuhp_get_step(target);
 	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
@@ -1472,6 +1477,8 @@ static ssize_t write_cpuhp_target(struct
 		ret = do_cpu_up(dev->id, target);
 	else
 		ret = do_cpu_down(dev->id, target);
+
+	unlock_device_hotplug();
 	return ret ? ret : count;
 }
 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 01/20] idle: Move x86ism out of generic code
  2016-02-27 20:29   ` Brian Gerst
@ 2016-02-29 19:35     ` Thomas Gleixner
  2016-02-29 19:48       ` Will Deacon
  0 siblings, 1 reply; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-29 19:35 UTC (permalink / raw)
  To: Brian Gerst
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner, Russell King, Catalin Marinas,
	Will Deacon, Ralf Baechle, Yoshinori Sato

On Sat, 27 Feb 2016, Brian Gerst wrote:
> >         arch_cpu_idle_prepare();
> >         cpu_idle_loop();
> >  }
> 
> Does this actually work with stack protector enabled?
> boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> not.

Stupid me. No it does of course not. I could have sworn that I tested that,
but obvioulsy not.

I drop that patch, but actually the real question is whether we can drop that
'#ifdef x86' around that boot_init_stack_canary() invocation.

AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
boot cpu. I can't see why that would be an issue on those architectures and
why it would be a problem if the boot cpu calls it again here.

CC'ed the relevant maintainers. Is there any issue with the patch below?

Thanks,

	tglx

8<------------------

--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -276,12 +276,6 @@ static void cpu_idle_loop(void)
 void cpu_startup_entry(enum cpuhp_state state)
 {
 	/*
-	 * This #ifdef needs to die, but it's too late in the cycle to
-	 * make this generic (arm and sh have never invoked the canary
-	 * init for the non boot cpus!). Will be fixed in 3.11
-	 */
-#ifdef CONFIG_X86
-	/*
 	 * If we're the non-boot CPU, nothing set the stack canary up
 	 * for us. The boot CPU already has it initialized but no harm
 	 * in doing it again. This is a good place for updating it, as
@@ -289,7 +283,7 @@ void cpu_startup_entry(enum cpuhp_state
 	 * canaries already on the stack wont ever trigger).
 	 */
 	boot_init_stack_canary();
-#endif
+
 	arch_cpu_idle_prepare();
 	cpu_idle_loop();
 }

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 01/20] idle: Move x86ism out of generic code
  2016-02-29 19:35     ` Thomas Gleixner
@ 2016-02-29 19:48       ` Will Deacon
  2016-02-29 20:06         ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Will Deacon @ 2016-02-29 19:48 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: Brian Gerst, LKML, Linus Torvalds, Andrew Morton, Ingo Molnar,
	Peter Zijlstra, Peter Anvin, Oleg Nesterov, linux-arch,
	Tejun Heo, Steven Rostedt, Rusty Russell, Paul McKenney,
	Rafael Wysocki, Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner, Russell King, Catalin Marinas,
	Ralf Baechle, Yoshinori Sato

On Mon, Feb 29, 2016 at 08:35:41PM +0100, Thomas Gleixner wrote:
> On Sat, 27 Feb 2016, Brian Gerst wrote:
> > >         arch_cpu_idle_prepare();
> > >         cpu_idle_loop();
> > >  }
> > 
> > Does this actually work with stack protector enabled?
> > boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> > not.
> 
> Stupid me. No it does of course not. I could have sworn that I tested that,
> but obvioulsy not.
> 
> I drop that patch, but actually the real question is whether we can drop that
> '#ifdef x86' around that boot_init_stack_canary() invocation.
> 
> AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
> boot cpu. I can't see why that would be an issue on those architectures and
> why it would be a problem if the boot cpu calls it again here.
> 
> CC'ed the relevant maintainers. Is there any issue with the patch below?

On arm[64], the canary is unfortunately global, so I don't think it would
be safe to update it live like this without effectively stopping the
machine and forcing everybody into idle.

Will

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 01/20] idle: Move x86ism out of generic code
  2016-02-29 19:48       ` Will Deacon
@ 2016-02-29 20:06         ` Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-02-29 20:06 UTC (permalink / raw)
  To: Will Deacon
  Cc: Brian Gerst, LKML, Linus Torvalds, Andrew Morton, Ingo Molnar,
	Peter Zijlstra, Peter Anvin, Oleg Nesterov, linux-arch,
	Tejun Heo, Steven Rostedt, Rusty Russell, Paul McKenney,
	Rafael Wysocki, Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner, Russell King, Catalin Marinas,
	Ralf Baechle, Yoshinori Sato

On Mon, 29 Feb 2016, Will Deacon wrote:

> On Mon, Feb 29, 2016 at 08:35:41PM +0100, Thomas Gleixner wrote:
> > On Sat, 27 Feb 2016, Brian Gerst wrote:
> > > >         arch_cpu_idle_prepare();
> > > >         cpu_idle_loop();
> > > >  }
> > > 
> > > Does this actually work with stack protector enabled?
> > > boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> > > not.
> > 
> > Stupid me. No it does of course not. I could have sworn that I tested that,
> > but obvioulsy not.
> > 
> > I drop that patch, but actually the real question is whether we can drop that
> > '#ifdef x86' around that boot_init_stack_canary() invocation.
> > 
> > AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
> > boot cpu. I can't see why that would be an issue on those architectures and
> > why it would be a problem if the boot cpu calls it again here.
> > 
> > CC'ed the relevant maintainers. Is there any issue with the patch below?
> 
> On arm[64], the canary is unfortunately global, so I don't think it would
> be safe to update it live like this without effectively stopping the
> machine and forcing everybody into idle.

Thanks for clarification.

       Thomas

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 10/20] cpu/hotplug: Make target state writeable
  2016-02-29 15:49           ` Thomas Gleixner
@ 2016-03-01  1:53             ` Rafael J. Wysocki
  0 siblings, 0 replies; 69+ messages in thread
From: Rafael J. Wysocki @ 2016-03-01  1:53 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

On Monday, February 29, 2016 04:49:24 PM Thomas Gleixner wrote:
> On Sun, 28 Feb 2016, Thomas Gleixner wrote:
> > 
> > Understood. I'll fix that thing up so that won't happen and I put it on the
> > list of things to look at deeper.
> 
> The below delta patch should do what you need, right?

Yup, thanks!

Rafael


> 8<-------------
> 
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1461,6 +1461,11 @@ static ssize_t write_cpuhp_target(struct
>  	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
>  		return -EINVAL;
>  #endif
> +
> +	ret = lock_device_hotplug_sysfs();
> +	if (ret)
> +		return ret;
> +
>  	mutex_lock(&cpuhp_state_mutex);
>  	sp = cpuhp_get_step(target);
>  	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
> @@ -1472,6 +1477,8 @@ static ssize_t write_cpuhp_target(struct
>  		ret = do_cpu_up(dev->id, target);
>  	else
>  		ret = do_cpu_down(dev->id, target);
> +
> +	unlock_device_hotplug();
>  	return ret ? ret : count;
>  }

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling
  2016-02-26 18:43 ` [patch 02/20] cpu/hotplug: Restructure FROZEN state handling Thomas Gleixner
@ 2016-03-01 19:51   ` tip-bot for Thomas Gleixner
  2016-03-02 22:35     ` Srivatsa S. Bhat
  2016-03-02 23:43     ` Srivatsa S. Bhat
  0 siblings, 2 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:51 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: linux-kernel, rostedt, tj, akpm, rusty, riel, bigeasy, oleg,
	torvalds, srivatsa, hpa, peterz, mingo, paulmck, tglx, arjan,
	rafael.j.wysocki, pjt

Commit-ID:  090e77c391dd983c8945b8e2e16d09f378d2e334
Gitweb:     http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100

cpu/hotplug: Restructure FROZEN state handling

There are only a few callbacks which really care about FROZEN
vs. !FROZEN. No need to have extra states for this.

Publish the frozen state in an extra variable which is updated under
the hotplug lock and let the users interested deal with it w/o
imposing that extra state checks on everyone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h |  2 ++
 kernel/cpu.c        | 69 ++++++++++++++++++++++-------------------------------
 2 files changed, 31 insertions(+), 40 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index d2ca8c3..f2fb549 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -118,6 +118,7 @@ enum {
 
 
 #ifdef CONFIG_SMP
+extern bool cpuhp_tasks_frozen;
 /* Need to know about CPUs going up/down? */
 #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
 #define cpu_notifier(fn, pri) {					\
@@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
 #define cpu_notifier_register_done	cpu_maps_update_done
 
 #else	/* CONFIG_SMP */
+#define cpuhp_tasks_frozen	0
 
 #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
 #define __cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 5b9d396..41a6cb8 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -29,6 +29,8 @@
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
+bool cpuhp_tasks_frozen;
+EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
 
 /*
  * The following two APIs (cpu_maps_update_begin/done) must be used when
@@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notifier_block *nb)
 	return raw_notifier_chain_register(&cpu_chain, nb);
 }
 
-static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
+static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
 			int *nr_calls)
 {
+	unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
+	void *hcpu = (void *)(long)cpu;
+
 	int ret;
 
-	ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
+	ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
 					nr_calls);
 
 	return notifier_to_errno(ret);
 }
 
-static int cpu_notify(unsigned long val, void *v)
+static int cpu_notify(unsigned long val, unsigned int cpu)
 {
-	return __cpu_notify(val, v, -1, NULL);
+	return __cpu_notify(val, cpu, -1, NULL);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
 
-static void cpu_notify_nofail(unsigned long val, void *v)
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
 {
-	BUG_ON(cpu_notify(val, v));
+	BUG_ON(cpu_notify(val, cpu));
 }
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -311,27 +316,21 @@ static inline void check_for_tasks(int dead_cpu)
 	read_unlock(&tasklist_lock);
 }
 
-struct take_cpu_down_param {
-	unsigned long mod;
-	void *hcpu;
-};
-
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
-	struct take_cpu_down_param *param = _param;
-	int err;
+	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
 	err = __cpu_disable();
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING | param->mod, param->hcpu);
+	cpu_notify(CPU_DYING, cpu);
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
-	stop_machine_park((long)param->hcpu);
+	stop_machine_park(cpu);
 	return 0;
 }
 
@@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
 static int _cpu_down(unsigned int cpu, int tasks_frozen)
 {
 	int err, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
-	struct take_cpu_down_param tcd_param = {
-		.mod = mod,
-		.hcpu = hcpu,
-	};
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
 
 	cpu_hotplug_begin();
 
-	err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
 	if (err) {
 		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
 		pr_warn("%s: attempt to take down CPU %u failed\n",
 			__func__, cpu);
 		goto out_release;
@@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
 	/*
 	 * So now all preempt/rcu users must observe !cpu_active().
 	 */
-	err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+	err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
 	if (err) {
 		/* CPU didn't die: tell everyone.  Can't complain. */
-		cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
+		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
 		goto out_release;
 	}
@@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
 
 	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD | mod, hcpu);
+	cpu_notify_nofail(CPU_DEAD, cpu);
 
 	check_for_tasks(cpu);
 
 out_release:
 	cpu_hotplug_done();
 	if (!err)
-		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+		cpu_notify_nofail(CPU_POST_DEAD, cpu);
 	return err;
 }
 
@@ -485,10 +480,8 @@ void smpboot_thread_init(void)
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
-	int ret, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
 	struct task_struct *idle;
+	int ret, nr_calls = 0;
 
 	cpu_hotplug_begin();
 
@@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 	if (ret)
 		goto out;
 
-	ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
 	if (ret) {
 		nr_calls--;
 		pr_warn("%s: attempt to bring up CPU %u failed\n",
@@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 	BUG_ON(!cpu_online(cpu));
 
 	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE | mod, hcpu);
+	cpu_notify(CPU_ONLINE, cpu);
 
 out_notify:
 	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
 out:
 	cpu_hotplug_done();
 
@@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
  */
 void notify_cpu_starting(unsigned int cpu)
 {
-	unsigned long val = CPU_STARTING;
-
-#ifdef CONFIG_PM_SLEEP_SMP
-	if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
-		val = CPU_STARTING_FROZEN;
-#endif /* CONFIG_PM_SLEEP_SMP */
-	cpu_notify(val, (void *)(long)cpu);
+	cpu_notify(CPU_STARTING, cpu);
 }
 
 #endif /* CONFIG_SMP */

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code
  2016-02-26 18:43 ` [patch 03/20] cpu/hotplug: Restructure cpu_up code Thomas Gleixner
@ 2016-03-01 19:52   ` tip-bot for Thomas Gleixner
  2016-03-02 22:36     ` Srivatsa S. Bhat
  0 siblings, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:52 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: hpa, rusty, mingo, akpm, peterz, srivatsa, torvalds, oleg,
	paulmck, tglx, linux-kernel, tj, pjt, rafael.j.wysocki, bigeasy,
	arjan, rostedt, riel

Commit-ID:  ba997462435f48ad1501320e9da8770fd40c59b1
Gitweb:     http://git.kernel.org/tip/ba997462435f48ad1501320e9da8770fd40c59b1
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:24 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100

cpu/hotplug: Restructure cpu_up code

Split out into separate functions, so we can convert it to a state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.429389195@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c | 69 +++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 22 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 41a6cb8..15a4136 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val, unsigned int cpu)
 	return __cpu_notify(val, cpu, -1, NULL);
 }
 
+/* Notifier wrappers for transitioning to state machine */
+static int notify_prepare(unsigned int cpu)
+{
+	int nr_calls = 0;
+	int ret;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
+	if (ret) {
+		nr_calls--;
+		printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
+				__func__, cpu);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	}
+	return ret;
+}
+
+static int notify_online(unsigned int cpu)
+{
+	cpu_notify(CPU_ONLINE, cpu);
+	return 0;
+}
+
+static int bringup_cpu(unsigned int cpu)
+{
+	struct task_struct *idle = idle_thread_get(cpu);
+	int ret;
+
+	/* Arch-specific enabling code. */
+	ret = __cpu_up(cpu, idle);
+	if (ret) {
+		cpu_notify(CPU_UP_CANCELED, cpu);
+		return ret;
+	}
+	BUG_ON(!cpu_online(cpu));
+	return 0;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 
 static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
@@ -481,7 +518,7 @@ void smpboot_thread_init(void)
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
 	struct task_struct *idle;
-	int ret, nr_calls = 0;
+	int ret;
 
 	cpu_hotplug_begin();
 
@@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 		goto out;
 	}
 
+	cpuhp_tasks_frozen = tasks_frozen;
+
 	ret = smpboot_create_threads(cpu);
 	if (ret)
 		goto out;
 
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
-	if (ret) {
-		nr_calls--;
-		pr_warn("%s: attempt to bring up CPU %u failed\n",
-			__func__, cpu);
-		goto out_notify;
-	}
-
-	/* Arch-specific enabling code. */
-	ret = __cpu_up(cpu, idle);
-
-	if (ret != 0)
-		goto out_notify;
-	BUG_ON(!cpu_online(cpu));
+	ret = notify_prepare(cpu);
+	if (ret)
+		goto out;
 
-	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE, cpu);
+	ret = bringup_cpu(cpu);
+	if (ret)
+		goto out;
 
-out_notify:
-	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	notify_online(cpu);
 out:
 	cpu_hotplug_done();
 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Split out cpu down functions
  2016-02-26 18:43 ` [patch 04/20] cpu/hotplug: Split out cpu down functions Thomas Gleixner
@ 2016-03-01 19:52   ` tip-bot for Thomas Gleixner
  2016-03-02 22:37     ` Srivatsa S. Bhat
  0 siblings, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:52 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: mingo, oleg, rafael.j.wysocki, bigeasy, hpa, rostedt, torvalds,
	pjt, akpm, tglx, linux-kernel, srivatsa, paulmck, arjan, riel,
	rusty, tj, peterz

Commit-ID:  984581728eb4b2e10baed3d606f85a119795b207
Gitweb:     http://git.kernel.org/tip/984581728eb4b2e10baed3d606f85a119795b207
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:25 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:53 +0100

cpu/hotplug: Split out cpu down functions

Split cpu_down in separate functions in preparation for state machine
conversion.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.511796562@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c | 83 ++++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 53 insertions(+), 30 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 15a4136..0b5d259 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
-	BUG_ON(cpu_notify(val, cpu));
-}
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
 
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int dead_cpu)
 	read_unlock(&tasklist_lock);
 }
 
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+	BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+	int err, nr_calls = 0;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+	if (err) {
+		nr_calls--;
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+		pr_warn("%s: attempt to take down CPU %u failed\n",
+				__func__, cpu);
+	}
+	return err;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
 	return 0;
 }
 
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
 {
-	int err, nr_calls = 0;
-
-	if (num_online_cpus() == 1)
-		return -EBUSY;
-
-	if (!cpu_online(cpu))
-		return -EINVAL;
-
-	cpu_hotplug_begin();
-
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
-	if (err) {
-		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
-		pr_warn("%s: attempt to take down CPU %u failed\n",
-			__func__, cpu);
-		goto out_release;
-	}
+	int err;
 
 	/*
 	 * By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
 		/* CPU didn't die: tell everyone.  Can't complain. */
 		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
-		goto out_release;
+		return err;
 	}
 	BUG_ON(cpu_online(cpu));
 
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
 	/* This actually kills the CPU. */
 	__cpu_die(cpu);
 
-	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD, cpu);
+	return 0;
+}
 
+static int notify_dead(unsigned int cpu)
+{
+	cpu_notify_nofail(CPU_DEAD, cpu);
 	check_for_tasks(cpu);
+	return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+	int err;
+
+	if (num_online_cpus() == 1)
+		return -EBUSY;
+
+	if (!cpu_online(cpu))
+		return -EINVAL;
+
+	cpu_hotplug_begin();
+
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = notify_down_prepare(cpu);
+	if (err)
+		goto out_release;
+	err = takedown_cpu(cpu);
+	if (err)
+		goto out_release;
+
+	notify_dead(cpu);
 
 out_release:
 	cpu_hotplug_done();

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Add tracepoints
  2016-02-26 18:43 ` [patch 05/20] cpu/hotplug: Add tracepoints Thomas Gleixner
@ 2016-03-01 19:52   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:52 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: akpm, torvalds, rusty, linux-kernel, rafael.j.wysocki, pjt,
	srivatsa, mingo, oleg, tglx, tj, bigeasy, paulmck, peterz, hpa,
	rostedt, arjan, riel

Commit-ID:  5ba9ac8e2c45ab165e5b4a246f4821d319656e9d
Gitweb:     http://git.kernel.org/tip/5ba9ac8e2c45ab165e5b4a246f4821d319656e9d
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:27 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:54 +0100

cpu/hotplug: Add tracepoints

We want to trace the hotplug machinery. Add tracepoints to track the
invocation of callbacks and their result.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.593563875@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/trace/events/cpuhp.h | 66 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

diff --git a/include/trace/events/cpuhp.h b/include/trace/events/cpuhp.h
new file mode 100644
index 0000000..a72bd93
--- /dev/null
+++ b/include/trace/events/cpuhp.h
@@ -0,0 +1,66 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM cpuhp
+
+#if !defined(_TRACE_CPUHP_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_CPUHP_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(cpuhp_enter,
+
+	TP_PROTO(unsigned int cpu,
+		 int target,
+		 int idx,
+		 int (*fun)(unsigned int)),
+
+	TP_ARGS(cpu, target, idx, fun),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		target		)
+		__field( int,		idx		)
+		__field( void *,	fun		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->target	= target;
+		__entry->idx	= idx;
+		__entry->fun	= fun;
+	),
+
+	TP_printk("cpu: %04u target: %3d step: %3d (%pf)",
+		  __entry->cpu, __entry->target, __entry->idx, __entry->fun)
+);
+
+TRACE_EVENT(cpuhp_exit,
+
+	TP_PROTO(unsigned int cpu,
+		 int state,
+		 int idx,
+		 int ret),
+
+	TP_ARGS(cpu, state, idx, ret),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		state		)
+		__field( int,		idx		)
+		__field( int,		ret		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->state	= state;
+		__entry->idx	= idx;
+		__entry->ret	= ret;
+	),
+
+	TP_printk(" cpu: %04u  state: %3d step: %3d ret: %d",
+		  __entry->cpu, __entry->state, __entry->idx,  __entry->ret)
+);
+
+#endif
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Convert to a state machine for the control processor
  2016-02-26 18:43 ` [patch 06/20] cpu/hotplug: Convert to a state machine for the control processor Thomas Gleixner
@ 2016-03-01 19:53   ` tip-bot for Thomas Gleixner
  2016-03-02 11:23     ` Peter Zijlstra
  0 siblings, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:53 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: rostedt, rusty, bigeasy, oleg, tglx, pjt, riel, torvalds,
	linux-kernel, tj, srivatsa, akpm, arjan, hpa, paulmck, mingo,
	peterz, rafael.j.wysocki

Commit-ID:  cff7d378d3fdbb53db9b6e2578b14855f401cd41
Gitweb:     http://git.kernel.org/tip/cff7d378d3fdbb53db9b6e2578b14855f401cd41
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:28 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:54 +0100

cpu/hotplug: Convert to a state machine for the control processor

Move the split out steps into a callback array and let the cpu_up/down
code iterate through the array functions. For now most of the
callbacks are asymmetric to resemble the current hotplug maze.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.671816690@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |   9 +-
 include/linux/cpuhotplug.h |  13 +++
 init/main.c                |  15 +---
 kernel/cpu.c               | 202 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 194 insertions(+), 45 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index f2fb549..78989f2 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -16,6 +16,7 @@
 #include <linux/node.h>
 #include <linux/compiler.h>
 #include <linux/cpumask.h>
+#include <linux/cpuhotplug.h>
 
 struct device;
 struct device_node;
@@ -27,6 +28,9 @@ struct cpu {
 	struct device dev;
 };
 
+extern void boot_cpu_init(void);
+extern void boot_cpu_state_init(void);
+
 extern int register_cpu(struct cpu *cpu, int num);
 extern struct device *get_cpu_device(unsigned cpu);
 extern bool cpu_is_hotpluggable(unsigned cpu);
@@ -267,11 +271,6 @@ static inline int disable_nonboot_cpus(void) { return 0; }
 static inline void enable_nonboot_cpus(void) {}
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
-enum cpuhp_state {
-	CPUHP_OFFLINE,
-	CPUHP_ONLINE,
-};
-
 void cpu_startup_entry(enum cpuhp_state state);
 
 void cpu_idle_poll_ctrl(bool enable);
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
new file mode 100644
index 0000000..d55c9e6
--- /dev/null
+++ b/include/linux/cpuhotplug.h
@@ -0,0 +1,13 @@
+#ifndef __CPUHOTPLUG_H
+#define __CPUHOTPLUG_H
+
+enum cpuhp_state {
+	CPUHP_OFFLINE,
+	CPUHP_CREATE_THREADS,
+	CPUHP_NOTIFY_PREPARE,
+	CPUHP_BRINGUP_CPU,
+	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE,
+};
+
+#endif
diff --git a/init/main.c b/init/main.c
index 58c9e37..c2ea723 100644
--- a/init/main.c
+++ b/init/main.c
@@ -452,20 +452,6 @@ void __init parse_early_param(void)
 	done = 1;
 }
 
-/*
- *	Activate the first processor.
- */
-
-static void __init boot_cpu_init(void)
-{
-	int cpu = smp_processor_id();
-	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
-	set_cpu_online(cpu, true);
-	set_cpu_active(cpu, true);
-	set_cpu_present(cpu, true);
-	set_cpu_possible(cpu, true);
-}
-
 void __init __weak smp_setup_processor_id(void)
 {
 }
@@ -530,6 +516,7 @@ asmlinkage __visible void __init start_kernel(void)
 	setup_command_line(command_line);
 	setup_nr_cpu_ids();
 	setup_per_cpu_areas();
+	boot_cpu_state_init();
 	smp_prepare_boot_cpu();	/* arch-specific boot-cpu hooks */
 
 	build_all_zonelists(NULL, NULL);
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 0b5d259..3018519 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,10 +22,64 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+
 #include <trace/events/power.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/cpuhp.h>
 
 #include "smpboot.h"
 
+/**
+ * cpuhp_cpu_state - Per cpu hotplug state storage
+ * @state:	The current cpu state
+ * @target:	The target state
+ */
+struct cpuhp_cpu_state {
+	enum cpuhp_state	state;
+	enum cpuhp_state	target;
+};
+
+static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
+
+/**
+ * cpuhp_step - Hotplug state machine step
+ * @name:	Name of the step
+ * @startup:	Startup function of the step
+ * @teardown:	Teardown function of the step
+ * @skip_onerr:	Do not invoke the functions on error rollback
+ *		Will go away once the notifiers	are gone
+ */
+struct cpuhp_step {
+	const char	*name;
+	int		(*startup)(unsigned int cpu);
+	int		(*teardown)(unsigned int cpu);
+	bool		skip_onerr;
+};
+
+static struct cpuhp_step cpuhp_bp_states[];
+
+/**
+ * cpuhp_invoke_callback _ Invoke the callbacks for a given state
+ * @cpu:	The cpu for which the callback should be invoked
+ * @step:	The step in the state machine
+ * @cb:		The callback function to invoke
+ *
+ * Called from cpu hotplug and from the state register machinery
+ */
+static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state step,
+				 int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int ret = 0;
+
+	if (cb) {
+		trace_cpuhp_enter(cpu, st->target, step, cb);
+		ret = cb(cpu);
+		trace_cpuhp_exit(cpu, st->state, step, ret);
+	}
+	return ret;
+}
+
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
@@ -454,10 +508,29 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+#else
+#define notify_down_prepare	NULL
+#define takedown_cpu		NULL
+#define notify_dead		NULL
+#endif
+
+#ifdef CONFIG_HOTPLUG_CPU
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 {
-	int err;
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int prev_state, ret = 0;
+	bool hasdied = false;
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -469,20 +542,25 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	err = notify_down_prepare(cpu);
-	if (err)
-		goto out_release;
-	err = takedown_cpu(cpu);
-	if (err)
-		goto out_release;
+	prev_state = st->state;
+	st->target = CPUHP_OFFLINE;
+	for (; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
 
-	notify_dead(cpu);
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st);
+			break;
+		}
+	}
+	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
-out_release:
 	cpu_hotplug_done();
-	if (!err)
+	/* This post dead nonsense must die */
+	if (!ret && hasdied)
 		cpu_notify_nofail(CPU_POST_DEAD, cpu);
-	return err;
+	return ret;
 }
 
 int cpu_down(unsigned int cpu)
@@ -537,11 +615,22 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int ret;
+	int prev_state, ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -550,6 +639,7 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 		goto out;
 	}
 
+	/* Let it fail before we try to bring the cpu up */
 	idle = idle_thread_get(cpu);
 	if (IS_ERR(idle)) {
 		ret = PTR_ERR(idle);
@@ -558,22 +648,22 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	ret = smpboot_create_threads(cpu);
-	if (ret)
-		goto out;
-
-	ret = notify_prepare(cpu);
-	if (ret)
-		goto out;
-
-	ret = bringup_cpu(cpu);
-	if (ret)
-		goto out;
-
-	notify_online(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_ONLINE;
+	while (st->state < st->target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_bp_states + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st);
+			break;
+		}
+	}
 out:
 	cpu_hotplug_done();
-
 	return ret;
 }
 
@@ -767,6 +857,44 @@ void notify_cpu_starting(unsigned int cpu)
 
 #endif /* CONFIG_SMP */
 
+/* Boot processor state steps */
+static struct cpuhp_step cpuhp_bp_states[] = {
+	[CPUHP_OFFLINE] = {
+		.name			= "offline",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+#ifdef CONFIG_SMP
+	[CPUHP_CREATE_THREADS]= {
+		.name			= "threads:create",
+		.startup		= smpboot_create_threads,
+		.teardown		= NULL,
+	},
+	[CPUHP_NOTIFY_PREPARE] = {
+		.name			= "notify:prepare",
+		.startup		= notify_prepare,
+		.teardown		= notify_dead,
+		.skip_onerr		= true,
+	},
+	[CPUHP_BRINGUP_CPU] = {
+		.name			= "cpu:bringup",
+		.startup		= bringup_cpu,
+		.teardown		= takedown_cpu,
+		.skip_onerr		= true,
+	},
+	[CPUHP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
+#endif
+	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.
@@ -826,3 +954,25 @@ void init_cpu_online(const struct cpumask *src)
 {
 	cpumask_copy(&__cpu_online_mask, src);
 }
+
+/*
+ * Activate the first processor.
+ */
+void __init boot_cpu_init(void)
+{
+	int cpu = smp_processor_id();
+
+	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
+	set_cpu_online(cpu, true);
+	set_cpu_active(cpu, true);
+	set_cpu_present(cpu, true);
+	set_cpu_possible(cpu, true);
+}
+
+/*
+ * Must be called _AFTER_ setting up the per_cpu areas
+ */
+void __init boot_cpu_state_init(void)
+{
+	per_cpu_ptr(&cpuhp_state, smp_processor_id())->state = CPUHP_ONLINE;
+}

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Convert the hotplugged cpu work to a state machine
  2016-02-26 18:43 ` [patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine Thomas Gleixner
@ 2016-03-01 19:53   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:53 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: tj, rostedt, arjan, peterz, mingo, pjt, paulmck, tglx, bigeasy,
	oleg, riel, linux-kernel, rusty, torvalds, srivatsa, hpa, akpm,
	rafael.j.wysocki

Commit-ID:  4baa0afc6719cbf36a1e08551484a641926b3fd1
Gitweb:     http://git.kernel.org/tip/4baa0afc6719cbf36a1e08551484a641926b3fd1
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:29 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:54 +0100

cpu/hotplug: Convert the hotplugged cpu work to a state machine

Move the functions which need to run on the hotplugged processor into
a state machine array and let the code iterate through these functions.

In a later state, this will grow synchronization points between the
control processor and the hotplugged processor, so we can move the
various architecture implementations of the synchronizations to the
core.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.770651526@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |  4 +++
 kernel/cpu.c               | 81 +++++++++++++++++++++++++++++++++++++---------
 2 files changed, 70 insertions(+), 15 deletions(-)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index d55c9e6..d9303cc 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,10 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_OFFLINE,
+	CPUHP_AP_NOTIFY_STARTING,
+	CPUHP_AP_ONLINE,
+	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE,
 };
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3018519..797723e 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -57,6 +57,7 @@ struct cpuhp_step {
 };
 
 static struct cpuhp_step cpuhp_bp_states[];
+static struct cpuhp_step cpuhp_ap_states[];
 
 /**
  * cpuhp_invoke_callback _ Invoke the callbacks for a given state
@@ -304,6 +305,12 @@ static int notify_online(unsigned int cpu)
 	return 0;
 }
 
+static int notify_starting(unsigned int cpu)
+{
+	cpu_notify(CPU_STARTING, cpu);
+	return 0;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -421,9 +428,17 @@ static int notify_down_prepare(unsigned int cpu)
 	return err;
 }
 
+static int notify_dying(unsigned int cpu)
+{
+	cpu_notify(CPU_DYING, cpu);
+	return 0;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_OFFLINE);
 	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
@@ -431,7 +446,12 @@ static int take_cpu_down(void *_param)
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING, cpu);
+	/* Invoke the former CPU_DYING callbacks */
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = cpuhp_ap_states + st->state;
+
+		cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
@@ -512,6 +532,7 @@ static int notify_dead(unsigned int cpu)
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
 #define notify_dead		NULL
+#define notify_dying		NULL
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -615,6 +636,28 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+/**
+ * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
+ * @cpu: cpu that just started
+ *
+ * This function calls the cpu_chain notifiers with CPU_STARTING.
+ * It must be called by the arch code on the new cpu, before the new cpu
+ * enables interrupts and before the "boot" cpu returns from __cpu_up().
+ */
+void notify_cpu_starting(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE);
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_ap_states + st->state;
+		cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -842,19 +885,6 @@ core_initcall(cpu_hotplug_pm_sync_init);
 
 #endif /* CONFIG_PM_SLEEP_SMP */
 
-/**
- * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
- * @cpu: cpu that just started
- *
- * This function calls the cpu_chain notifiers with CPU_STARTING.
- * It must be called by the arch code on the new cpu, before the new cpu
- * enables interrupts and before the "boot" cpu returns from __cpu_up().
- */
-void notify_cpu_starting(unsigned int cpu)
-{
-	cpu_notify(CPU_STARTING, cpu);
-}
-
 #endif /* CONFIG_SMP */
 
 /* Boot processor state steps */
@@ -879,8 +909,12 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
+		.teardown		= NULL,
+	},
+	[CPUHP_TEARDOWN_CPU] = {
+		.name			= "cpu:teardown",
+		.startup		= NULL,
 		.teardown		= takedown_cpu,
-		.skip_onerr		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
@@ -895,6 +929,23 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 	},
 };
 
+/* Application processor state steps */
+static struct cpuhp_step cpuhp_ap_states[] = {
+#ifdef CONFIG_SMP
+	[CPUHP_AP_NOTIFY_STARTING] = {
+		.name			= "notify:starting",
+		.startup		= notify_starting,
+		.teardown		= notify_dying,
+		.skip_onerr		= true,
+	},
+#endif
+	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Hand in target state to _cpu_up/down
  2016-02-26 18:43 ` [patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down Thomas Gleixner
@ 2016-03-01 19:54   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:54 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: oleg, tglx, mingo, riel, paulmck, akpm, hpa, bigeasy,
	rafael.j.wysocki, pjt, peterz, tj, srivatsa, torvalds, rusty,
	arjan, linux-kernel, rostedt

Commit-ID:  af1f40457da6f79b6a1d7e1198b32f671572abd2
Gitweb:     http://git.kernel.org/tip/af1f40457da6f79b6a1d7e1198b32f671572abd2
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:30 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:54 +0100

cpu/hotplug: Hand in target state to _cpu_up/down

We want to be able to bringup/teardown the cpu to a particular state. Add a
target argument to _cpu_up/down.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.862113133@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c | 31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 797723e..a00f8f6 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -547,7 +547,8 @@ static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
+			   enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int prev_state, ret = 0;
@@ -564,7 +565,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_OFFLINE;
+	st->target = target;
 	for (; st->state > st->target; st->state--) {
 		struct cpuhp_step *step = cpuhp_bp_states + st->state;
 
@@ -584,7 +585,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 	return ret;
 }
 
-int cpu_down(unsigned int cpu)
+static int do_cpu_down(unsigned int cpu, enum cpuhp_state target)
 {
 	int err;
 
@@ -595,12 +596,16 @@ int cpu_down(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_down(cpu, 0);
+	err = _cpu_down(cpu, 0, target);
 
 out:
 	cpu_maps_update_done();
 	return err;
 }
+int cpu_down(unsigned int cpu)
+{
+	return do_cpu_down(cpu, CPUHP_OFFLINE);
+}
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
@@ -669,7 +674,7 @@ static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int _cpu_up(unsigned int cpu, int tasks_frozen)
+static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
@@ -692,7 +697,7 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_ONLINE;
+	st->target = target;
 	while (st->state < st->target) {
 		struct cpuhp_step *step;
 
@@ -710,7 +715,7 @@ out:
 	return ret;
 }
 
-int cpu_up(unsigned int cpu)
+static int do_cpu_up(unsigned int cpu, enum cpuhp_state target)
 {
 	int err = 0;
 
@@ -734,12 +739,16 @@ int cpu_up(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_up(cpu, 0);
-
+	err = _cpu_up(cpu, 0, target);
 out:
 	cpu_maps_update_done();
 	return err;
 }
+
+int cpu_up(unsigned int cpu)
+{
+	return do_cpu_up(cpu, CPUHP_ONLINE);
+}
 EXPORT_SYMBOL_GPL(cpu_up);
 
 #ifdef CONFIG_PM_SLEEP_SMP
@@ -762,7 +771,7 @@ int disable_nonboot_cpus(void)
 		if (cpu == first_cpu)
 			continue;
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, true);
-		error = _cpu_down(cpu, 1);
+		error = _cpu_down(cpu, 1, CPUHP_OFFLINE);
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, false);
 		if (!error)
 			cpumask_set_cpu(cpu, frozen_cpus);
@@ -812,7 +821,7 @@ void enable_nonboot_cpus(void)
 
 	for_each_cpu(cpu, frozen_cpus) {
 		trace_suspend_resume(TPS("CPU_ON"), cpu, true);
-		error = _cpu_up(cpu, 1);
+		error = _cpu_up(cpu, 1, CPUHP_ONLINE);
 		trace_suspend_resume(TPS("CPU_ON"), cpu, false);
 		if (!error) {
 			pr_info("CPU%d is up\n", cpu);

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Add sysfs state interface
  2016-02-26 18:43 ` [patch 09/20] cpu/hotplug: Add sysfs state interface Thomas Gleixner
@ 2016-03-01 19:54   ` tip-bot for Thomas Gleixner
  2016-03-02 12:40     ` Peter Zijlstra
  0 siblings, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:54 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: tglx, torvalds, rafael.j.wysocki, bigeasy, oleg, paulmck, arjan,
	pjt, hpa, peterz, mingo, tj, linux-kernel, akpm, riel, srivatsa,
	rostedt, rusty

Commit-ID:  98f8cdce1db580b99fce823a48eea2cb2bdb261e
Gitweb:     http://git.kernel.org/tip/98f8cdce1db580b99fce823a48eea2cb2bdb261e
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:31 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:55 +0100

cpu/hotplug: Add sysfs state interface

Add a sysfs interface so we can actually see in which state the cpus are in.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182340.942257522@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 100 insertions(+)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index a00f8f6..1979b89 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -56,6 +56,7 @@ struct cpuhp_step {
 	bool		skip_onerr;
 };
 
+static DEFINE_MUTEX(cpuhp_state_mutex);
 static struct cpuhp_step cpuhp_bp_states[];
 static struct cpuhp_step cpuhp_ap_states[];
 
@@ -955,6 +956,105 @@ static struct cpuhp_step cpuhp_ap_states[] = {
 	},
 };
 
+static bool cpuhp_is_ap_state(enum cpuhp_state state)
+{
+	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+}
+
+static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
+{
+	struct cpuhp_step *sp;
+
+	sp = cpuhp_is_ap_state(state) ? cpuhp_ap_states : cpuhp_bp_states;
+	return sp + state;
+}
+
+#if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
+static ssize_t show_cpuhp_state(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->state);
+}
+static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
+
+static ssize_t show_cpuhp_target(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->target);
+}
+static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+
+static struct attribute *cpuhp_cpu_attrs[] = {
+	&dev_attr_state.attr,
+	&dev_attr_target.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_attr_group = {
+	.attrs = cpuhp_cpu_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static ssize_t show_cpuhp_states(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	ssize_t cur, res = 0;
+	int i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = 0; i <= CPUHP_ONLINE; i++) {
+		struct cpuhp_step *sp = cpuhp_get_step(i);
+
+		if (sp->name) {
+			cur = sprintf(buf, "%3d: %s\n", i, sp->name);
+			buf += cur;
+			res += cur;
+		}
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	return res;
+}
+static DEVICE_ATTR(states, 0444, show_cpuhp_states, NULL);
+
+static struct attribute *cpuhp_cpu_root_attrs[] = {
+	&dev_attr_states.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_root_attr_group = {
+	.attrs = cpuhp_cpu_root_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static int __init cpuhp_sysfs_init(void)
+{
+	int cpu, ret;
+
+	ret = sysfs_create_group(&cpu_subsys.dev_root->kobj,
+				 &cpuhp_cpu_root_attr_group);
+	if (ret)
+		return ret;
+
+	for_each_possible_cpu(cpu) {
+		struct device *dev = get_cpu_device(cpu);
+
+		if (!dev)
+			continue;
+		ret = sysfs_create_group(&dev->kobj, &cpuhp_cpu_attr_group);
+		if (ret)
+			return ret;
+	}
+	return 0;
+}
+device_initcall(cpuhp_sysfs_init);
+#endif
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Make target state writeable
  2016-02-26 18:43 ` [patch 10/20] cpu/hotplug: Make target state writeable Thomas Gleixner
  2016-02-26 23:46   ` Rafael J. Wysocki
@ 2016-03-01 19:55   ` tip-bot for Thomas Gleixner
  2016-03-02 12:41     ` Peter Zijlstra
  1 sibling, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:55 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: rostedt, tglx, rafael.j.wysocki, bigeasy, riel, mingo, arjan,
	rusty, torvalds, srivatsa, peterz, hpa, akpm, oleg, pjt, tj,
	linux-kernel, paulmck

Commit-ID:  757c989b9994f51b42d6be1bd33c7c12d16a3ac7
Gitweb:     http://git.kernel.org/tip/757c989b9994f51b42d6be1bd33c7c12d16a3ac7
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:32 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:55 +0100

cpu/hotplug: Make target state writeable

Make it possible to write a target state to the per cpu state file, so we can
switch between states.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.022814799@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c      | 73 +++++++++++++++++++++++++++++++++++++++++++++++++------
 lib/Kconfig.debug | 13 ++++++++++
 2 files changed, 78 insertions(+), 8 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 1979b89..be9335d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -48,12 +48,14 @@ static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
  * @teardown:	Teardown function of the step
  * @skip_onerr:	Do not invoke the functions on error rollback
  *		Will go away once the notifiers	are gone
+ * @cant_stop:	Bringup/teardown can't be stopped at this step
  */
 struct cpuhp_step {
 	const char	*name;
 	int		(*startup)(unsigned int cpu);
 	int		(*teardown)(unsigned int cpu);
 	bool		skip_onerr;
+	bool		cant_stop;
 };
 
 static DEFINE_MUTEX(cpuhp_state_mutex);
@@ -558,7 +560,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 	if (num_online_cpus() == 1)
 		return -EBUSY;
 
-	if (!cpu_online(cpu))
+	if (!cpu_present(cpu))
 		return -EINVAL;
 
 	cpu_hotplug_begin();
@@ -683,16 +685,25 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 
 	cpu_hotplug_begin();
 
-	if (cpu_online(cpu) || !cpu_present(cpu)) {
+	if (!cpu_present(cpu)) {
 		ret = -EINVAL;
 		goto out;
 	}
 
-	/* Let it fail before we try to bring the cpu up */
-	idle = idle_thread_get(cpu);
-	if (IS_ERR(idle)) {
-		ret = PTR_ERR(idle);
+	/*
+	 * The caller of do_cpu_up might have raced with another
+	 * caller. Ignore it for now.
+	 */
+	if (st->state >= target)
 		goto out;
+
+	if (st->state == CPUHP_OFFLINE) {
+		/* Let it fail before we try to bring the cpu up */
+		idle = idle_thread_get(cpu);
+		if (IS_ERR(idle)) {
+			ret = PTR_ERR(idle);
+			goto out;
+		}
 	}
 
 	cpuhp_tasks_frozen = tasks_frozen;
@@ -909,27 +920,32 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 		.name			= "threads:create",
 		.startup		= smpboot_create_threads,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_PREPARE] = {
 		.name			= "notify:prepare",
 		.startup		= notify_prepare,
 		.teardown		= notify_dead,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_TEARDOWN_CPU] = {
 		.name			= "cpu:teardown",
 		.startup		= NULL,
 		.teardown		= takedown_cpu,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
 		.teardown		= notify_down_prepare,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -947,6 +963,7 @@ static struct cpuhp_step cpuhp_ap_states[] = {
 		.startup		= notify_starting,
 		.teardown		= notify_dying,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -979,6 +996,46 @@ static ssize_t show_cpuhp_state(struct device *dev,
 }
 static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
 
+static ssize_t write_cpuhp_target(struct device *dev,
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+	struct cpuhp_step *sp;
+	int target, ret;
+
+	ret = kstrtoint(buf, 10, &target);
+	if (ret)
+		return ret;
+
+#ifdef CONFIG_CPU_HOTPLUG_STATE_CONTROL
+	if (target < CPUHP_OFFLINE || target > CPUHP_ONLINE)
+		return -EINVAL;
+#else
+	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
+		return -EINVAL;
+#endif
+
+	ret = lock_device_hotplug_sysfs();
+	if (ret)
+		return ret;
+
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(target);
+	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
+	mutex_unlock(&cpuhp_state_mutex);
+	if (ret)
+		return ret;
+
+	if (st->state < target)
+		ret = do_cpu_up(dev->id, target);
+	else
+		ret = do_cpu_down(dev->id, target);
+
+	unlock_device_hotplug();
+	return ret ? ret : count;
+}
+
 static ssize_t show_cpuhp_target(struct device *dev,
 				 struct device_attribute *attr, char *buf)
 {
@@ -986,7 +1043,7 @@ static ssize_t show_cpuhp_target(struct device *dev,
 
 	return sprintf(buf, "%d\n", st->target);
 }
-static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+static DEVICE_ATTR(target, 0644, show_cpuhp_target, write_cpuhp_target);
 
 static struct attribute *cpuhp_cpu_attrs[] = {
 	&dev_attr_state.attr,
@@ -1007,7 +1064,7 @@ static ssize_t show_cpuhp_states(struct device *dev,
 	int i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = 0; i <= CPUHP_ONLINE; i++) {
+	for (i = CPUHP_OFFLINE; i <= CPUHP_ONLINE; i++) {
 		struct cpuhp_step *sp = cpuhp_get_step(i);
 
 		if (sp->name) {
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 8bfd1ac..f28f7fa 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1442,6 +1442,19 @@ config DEBUG_BLOCK_EXT_DEVT
 
 	  Say N if you are unsure.
 
+config CPU_HOTPLUG_STATE_CONTROL
+	bool "Enable CPU hotplug state control"
+	depends on DEBUG_KERNEL
+	depends on HOTPLUG_CPU
+	default n
+	help
+	  Allows to write steps between "offline" and "online" to the CPUs
+	  sysfs target file so states can be stepped granular. This is a debug
+	  option for now as the hotplug machinery cannot be stopped and
+	  restarted at arbitrary points yet.
+
+	  Say N if your are unsure.
+
 config NOTIFIER_ERROR_INJECTION
 	tristate "Notifier error injection"
 	depends on DEBUG_KERNEL

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Implement setup/removal interface
  2016-02-26 18:43 ` [patch 11/20] cpu/hotplug: Implement setup/removal interface Thomas Gleixner
@ 2016-03-01 19:55   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:55 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: peterz, tj, linux-kernel, arjan, mingo, pjt, tglx, paulmck,
	rostedt, riel, hpa, rusty, akpm, bigeasy, srivatsa,
	rafael.j.wysocki, torvalds, oleg

Commit-ID:  5b7aa87e0482be768486e0c2277aa4122487eb9d
Gitweb:     http://git.kernel.org/tip/5b7aa87e0482be768486e0c2277aa4122487eb9d
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:33 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:55 +0100

cpu/hotplug: Implement setup/removal interface

Implement function which allow to setup/remove hotplug state callbacks.

The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.

The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.

This includes rollback to the previous state in case of failure.

A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them

For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.

[ Dynamic registering implemented by Sebastian Siewior ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.103464877@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |  67 ++++++++++++++
 kernel/cpu.c               | 224 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 291 insertions(+)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index d9303cc..2993526 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE_DYN,
+	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
+int __cpuhp_setup_state(enum cpuhp_state state,	const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback (will be used in debug output)
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+				    const char *name,
+				    int (*startup)(unsigned int cpu),
+				    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ *			       callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback.
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+					    const char *name,
+					    int (*startup)(unsigned int cpu),
+					    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state:	The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ *				teardown
+ * @state:	The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, false);
+}
+
 #endif
diff --git a/kernel/cpu.c b/kernel/cpu.c
index be9335d..b5eacb9 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states[] = {
 	},
 };
 
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+	if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+		return -EINVAL;
+	return 0;
+}
+
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
 	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
 	return sp + state;
 }
 
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+				  const char *name,
+				  int (*startup)(unsigned int cpu),
+				  int (*teardown)(unsigned int cpu))
+{
+	/* (Un)Install the callbacks for further cpu hotplug operations */
+	struct cpuhp_step *sp;
+
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(state);
+	sp->startup = startup;
+	sp->teardown = teardown;
+	sp->name = name;
+	mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+	return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+	int (*cb)(unsigned int cpu) = __cb;
+
+	BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+			    int (*cb)(unsigned int), bool bringup)
+{
+	int ret;
+
+	if (!cb)
+		return 0;
+
+	/*
+	 * This invokes the callback directly for now. In a later step we
+	 * convert that to use cpuhp_invoke_callback().
+	 */
+	if (cpuhp_is_ap_state(state)) {
+		/*
+		 * Note, that a function called on the AP is not
+		 * allowed to fail.
+		 */
+		if (cpu_online(cpu))
+			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+		return 0;
+	}
+
+	/*
+	 * The non AP bound callbacks can fail on bringup. On teardown
+	 * e.g. module removal we crash for now.
+	 */
+	ret = cb(cpu);
+	BUG_ON(ret && !bringup);
+	return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+				   int (*teardown)(unsigned int cpu))
+{
+	int cpu;
+
+	if (!teardown)
+		return;
+
+	/* Roll back the already executed steps on the other cpus */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpu >= failedcpu)
+			break;
+
+		/* Did we invoke the startup call on that cpu ? */
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+	enum cpuhp_state i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+		if (cpuhp_bp_states[i].name)
+			continue;
+
+		cpuhp_bp_states[i].name = "Reserved";
+		mutex_unlock(&cpuhp_state_mutex);
+		return i;
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	WARN(1, "No more dynamic states available for CPU hotplug\n");
+	return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state:	The state to setup
+ * @invoke:	If true, the startup function is invoked for cpus where
+ *		cpu state >= @state
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+			const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu))
+{
+	int cpu, ret = 0;
+	int dyn_state = 0;
+
+	if (cpuhp_cb_check(state) || !name)
+		return -EINVAL;
+
+	get_online_cpus();
+
+	/* currently assignments for the ONLINE state are possible */
+	if (state == CPUHP_ONLINE_DYN) {
+		dyn_state = 1;
+		ret = cpuhp_reserve_state(state);
+		if (ret < 0)
+			goto out;
+		state = ret;
+	}
+
+	cpuhp_store_callbacks(state, name, startup, teardown);
+
+	if (!invoke || !startup)
+		goto out;
+
+	/*
+	 * Try to call the startup callback for each present cpu
+	 * depending on the hotplug state of the cpu.
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate < state)
+			continue;
+
+		ret = cpuhp_issue_call(cpu, state, startup, true);
+		if (ret) {
+			cpuhp_rollback_install(cpu, state, teardown);
+			cpuhp_store_callbacks(state, NULL, NULL, NULL);
+			goto out;
+		}
+	}
+out:
+	put_online_cpus();
+	if (!ret && dyn_state)
+		return state;
+	return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state:	The state to remove
+ * @invoke:	If true, the teardown function is invoked for cpus where
+ *		cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+	int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+	int cpu;
+
+	BUG_ON(cpuhp_cb_check(state));
+
+	get_online_cpus();
+
+	if (!invoke || !teardown)
+		goto remove;
+
+	/*
+	 * Call the teardown callback for each present cpu depending
+	 * on the hotplug state of the cpu. This function is not
+	 * allowed to fail currently!
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+remove:
+	cpuhp_store_callbacks(state, NULL, NULL, NULL);
+	put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
 #if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
 static ssize_t show_cpuhp_state(struct device *dev,
 				struct device_attribute *attr, char *buf)

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core
  2016-02-26 18:43 ` [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core Thomas Gleixner
@ 2016-03-01 19:55   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:55 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: rostedt, torvalds, akpm, bigeasy, rafael.j.wysocki, peterz,
	rusty, pjt, hpa, tglx, arjan, srivatsa, mingo, riel, oleg,
	linux-kernel, paulmck, tj

Commit-ID:  949338e35131c551f7bf54f48a2e3a227af6721b
Gitweb:     http://git.kernel.org/tip/949338e35131c551f7bf54f48a2e3a227af6721b
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:35 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:55 +0100

cpu/hotplug: Move scheduler cpu_online notifier to hotplug core

Move the scheduler cpu online notifier part to the hotplug core. This is
anyway the highest priority callback and we need that functionality right now
for the next changes.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.200791046@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |  1 +
 kernel/cpu.c               | 18 ++++++++++++++++++
 kernel/sched/core.c        | 10 ----------
 3 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 2993526..2f2e5d9 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,6 +10,7 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
+	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
diff --git a/kernel/cpu.c b/kernel/cpu.c
index b5eacb9..65e34d3 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -666,6 +666,19 @@ void notify_cpu_starting(unsigned int cpu)
 	}
 }
 
+/*
+ * Called from the idle task. We need to set active here, so we can kick off
+ * the stopper thread.
+ */
+static int cpuhp_set_cpu_active(unsigned int cpu)
+{
+	/* The cpu is marked online, set it active now */
+	set_cpu_active(cpu, true);
+	/* Unpark the stopper thread */
+	stop_machine_unpark(cpu);
+	return 0;
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -941,6 +954,11 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
+	[CPUHP_CPU_SET_ACTIVE] = {
+		.name			= "cpu:active",
+		.startup		= cpuhp_set_cpu_active,
+		.teardown		= NULL,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9503d59..6266463 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5692,16 +5692,6 @@ static int sched_cpu_active(struct notifier_block *nfb,
 		set_cpu_rq_start_time();
 		return NOTIFY_OK;
 
-	case CPU_ONLINE:
-		/*
-		 * At this point a starting CPU has marked itself as online via
-		 * set_cpu_online(). But it might not yet have marked itself
-		 * as active, which is essential from here on.
-		 */
-		set_cpu_active(cpu, true);
-		stop_machine_unpark(cpu);
-		return NOTIFY_OK;
-
 	case CPU_DOWN_FAILED:
 		set_cpu_active(cpu, true);
 		return NOTIFY_OK;

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Unpark smpboot threads from the state machine
  2016-02-26 18:43 ` [patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine Thomas Gleixner
@ 2016-03-01 19:56   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:56 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: hpa, linux-kernel, akpm, paulmck, srivatsa, tglx, arjan, bigeasy,
	mingo, rostedt, torvalds, peterz, oleg, riel, rusty, pjt,
	rafael.j.wysocki, tj

Commit-ID:  931ef163309ee955611f287dc65248b39a65fc9d
Gitweb:     http://git.kernel.org/tip/931ef163309ee955611f287dc65248b39a65fc9d
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:36 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:56 +0100

cpu/hotplug: Unpark smpboot threads from the state machine

Handle the smpboot threads in the state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.295777684@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |  7 +------
 include/linux/cpuhotplug.h |  1 +
 init/main.c                |  1 -
 kernel/cpu.c               | 39 +++++----------------------------------
 kernel/smpboot.c           |  6 ++++--
 kernel/smpboot.h           |  4 ++--
 6 files changed, 13 insertions(+), 45 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 78989f2..83f3576 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -78,7 +78,7 @@ enum {
 	/* migration should happen before other stuff but after perf */
 	CPU_PRI_PERF		= 20,
 	CPU_PRI_MIGRATION	= 10,
-	CPU_PRI_SMPBOOT		= 9,
+
 	/* bring up workqueues before normal notifiers and down after */
 	CPU_PRI_WORKQUEUE_UP	= 5,
 	CPU_PRI_WORKQUEUE_DOWN	= -5,
@@ -172,7 +172,6 @@ static inline void __unregister_cpu_notifier(struct notifier_block *nb)
 }
 #endif
 
-void smpboot_thread_init(void);
 int cpu_up(unsigned int cpu);
 void notify_cpu_starting(unsigned int cpu);
 extern void cpu_maps_update_begin(void);
@@ -221,10 +220,6 @@ static inline void cpu_notifier_register_done(void)
 {
 }
 
-static inline void smpboot_thread_init(void)
-{
-}
-
 #endif /* CONFIG_SMP */
 extern struct bus_type cpu_subsys;
 
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 2f2e5d9..3867910 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,6 +11,7 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
+	CPUHP_SMPBOOT_THREADS,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
diff --git a/init/main.c b/init/main.c
index c2ea723..55563fd 100644
--- a/init/main.c
+++ b/init/main.c
@@ -388,7 +388,6 @@ static noinline void __init_refok rest_init(void)
 	int pid;
 
 	rcu_scheduler_starting();
-	smpboot_thread_init();
 	/*
 	 * We need to spawn init first so that it obtains pid 1, however
 	 * the init task will end up wanting to create kthreads, which, if
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 65e34d3..3ec86bc 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -481,8 +481,6 @@ static int takedown_cpu(unsigned int cpu)
 	else
 		synchronize_rcu();
 
-	smpboot_park_threads(cpu);
-
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -612,38 +610,6 @@ int cpu_down(unsigned int cpu)
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
-/*
- * Unpark per-CPU smpboot kthreads at CPU-online time.
- */
-static int smpboot_thread_call(struct notifier_block *nfb,
-			       unsigned long action, void *hcpu)
-{
-	int cpu = (long)hcpu;
-
-	switch (action & ~CPU_TASKS_FROZEN) {
-
-	case CPU_DOWN_FAILED:
-	case CPU_ONLINE:
-		smpboot_unpark_threads(cpu);
-		break;
-
-	default:
-		break;
-	}
-
-	return NOTIFY_OK;
-}
-
-static struct notifier_block smpboot_thread_notifier = {
-	.notifier_call = smpboot_thread_call,
-	.priority = CPU_PRI_SMPBOOT,
-};
-
-void smpboot_thread_init(void)
-{
-	register_cpu_notifier(&smpboot_thread_notifier);
-}
-
 /**
  * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
  * @cpu: cpu that just started
@@ -959,6 +925,11 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
+	[CPUHP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index d264f59..13bc43d 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -226,7 +226,7 @@ static void smpboot_unpark_thread(struct smp_hotplug_thread *ht, unsigned int cp
 		kthread_unpark(tsk);
 }
 
-void smpboot_unpark_threads(unsigned int cpu)
+int smpboot_unpark_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -235,6 +235,7 @@ void smpboot_unpark_threads(unsigned int cpu)
 		if (cpumask_test_cpu(cpu, cur->cpumask))
 			smpboot_unpark_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
@@ -245,7 +246,7 @@ static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
 		kthread_park(tsk);
 }
 
-void smpboot_park_threads(unsigned int cpu)
+int smpboot_park_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -253,6 +254,7 @@ void smpboot_park_threads(unsigned int cpu)
 	list_for_each_entry_reverse(cur, &hotplug_threads, list)
 		smpboot_park_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
diff --git a/kernel/smpboot.h b/kernel/smpboot.h
index 72415a0..6b5f020 100644
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -14,7 +14,7 @@ static inline void idle_threads_init(void) { }
 #endif
 
 int smpboot_create_threads(unsigned int cpu);
-void smpboot_park_threads(unsigned int cpu);
-void smpboot_unpark_threads(unsigned int cpu);
+int smpboot_park_threads(unsigned int cpu);
+int smpboot_unpark_threads(unsigned int cpu);
 
 #endif

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Split out the state walk into functions
  2016-02-26 18:43 ` [patch 14/20] cpu/hotplug: Split out the state walk into functions Thomas Gleixner
@ 2016-03-01 19:56   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:56 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: arjan, rostedt, linux-kernel, bigeasy, tj, riel, paulmck, tglx,
	rusty, akpm, rafael.j.wysocki, peterz, pjt, hpa, torvalds, oleg,
	mingo, srivatsa

Commit-ID:  2e1a3483ce74d197876e58281925dd4e378a7f28
Gitweb:     http://git.kernel.org/tip/2e1a3483ce74d197876e58281925dd4e378a7f28
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:37 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:56 +0100

cpu/hotplug: Split out the state walk into functions

We need that for running callbacks on the AP and the BP.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.374946234@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c | 111 ++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 68 insertions(+), 43 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 3ec86bc..9572ca0 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+			  struct cpuhp_step *steps)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+				struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+			struct cpuhp_step *steps)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+			      struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = steps + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
-
 void unregister_cpu_notifier(struct notifier_block *nb)
 {
 	cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state++; st->state < st->target; st->state++) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->startup);
-	}
-}
 
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 
 	prev_state = st->state;
 	st->target = target;
-	for (; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
-		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_down(cpu, st);
-			break;
-		}
-	}
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
 	cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned int cpu)
 	return 0;
 }
 
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state--; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->teardown);
-	}
-}
-
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int prev_state, ret = 0;
+	int ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	prev_state = st->state;
 	st->target = target;
-	while (st->state < st->target) {
-		struct cpuhp_step *step;
-
-		st->state++;
-		step = cpuhp_bp_states + st->state;
-		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_up(cpu, st);
-			break;
-		}
-	}
+	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
 	return ret;

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Create hotplug threads
  2016-02-26 18:43 ` [patch 15/20] cpu/hotplug: Create hotplug threads Thomas Gleixner
@ 2016-03-01 19:57   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:57 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: tglx, riel, rostedt, rafael.j.wysocki, bigeasy, hpa, mingo,
	peterz, tj, arjan, rusty, oleg, linux-kernel, pjt, srivatsa,
	torvalds, paulmck, akpm

Commit-ID:  4cb28ced23c4f222ff4e3f39898017e52161a9c9
Gitweb:     http://git.kernel.org/tip/4cb28ced23c4f222ff4e3f39898017e52161a9c9
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:38 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:56 +0100

cpu/hotplug: Create hotplug threads

In order to let the hotplugged cpu take care of the setup/teardown, we need a
seperate hotplug thread.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.454541272@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c     | 145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/smp.c     |   1 +
 kernel/smpboot.h |   2 +
 3 files changed, 147 insertions(+), 1 deletion(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index 9572ca0..9048c33 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,6 +22,7 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+#include <linux/smpboot.h>
 
 #include <trace/events/power.h>
 #define CREATE_TRACE_POINTS
@@ -33,10 +34,24 @@
  * cpuhp_cpu_state - Per cpu hotplug state storage
  * @state:	The current cpu state
  * @target:	The target state
+ * @thread:	Pointer to the hotplug thread
+ * @should_run:	Thread should execute
+ * @cb_stat:	The state for a single callback (install/uninstall)
+ * @cb:		Single callback function (install/uninstall)
+ * @result:	Result of the operation
+ * @done:	Signal completion to the issuer of the task
  */
 struct cpuhp_cpu_state {
 	enum cpuhp_state	state;
 	enum cpuhp_state	target;
+#ifdef CONFIG_SMP
+	struct task_struct	*thread;
+	bool			should_run;
+	enum cpuhp_state	cb_state;
+	int			(*cb)(unsigned int cpu);
+	int			result;
+	struct completion	done;
+#endif
 };
 
 static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
@@ -394,6 +409,134 @@ static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
 	return ret;
 }
 
+/*
+ * The cpu hotplug threads manage the bringup and teardown of the cpus
+ */
+static void cpuhp_create(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	init_completion(&st->done);
+}
+
+static int cpuhp_should_run(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	return st->should_run;
+}
+
+/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
+static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+
+	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
+}
+
+/* Execute the online startup callbacks. Used to be CPU_ONLINE */
+static int cpuhp_ap_online(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	return cpuhp_up_callbacks(cpu, st, cpuhp_ap_states, st->target);
+}
+
+/*
+ * Execute teardown/startup callbacks on the plugged cpu. Also used to invoke
+ * callbacks when a state gets [un]installed at runtime.
+ */
+static void cpuhp_thread_fun(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	int ret = 0;
+
+	/*
+	 * Paired with the mb() in cpuhp_kick_ap_work and
+	 * cpuhp_invoke_ap_callback, so the work set is consistent visible.
+	 */
+	smp_mb();
+	if (!st->should_run)
+		return;
+
+	st->should_run = false;
+
+	/* Single callback invocation for [un]install ? */
+	if (st->cb) {
+		if (st->cb_state < CPUHP_AP_ONLINE) {
+			local_irq_disable();
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+			local_irq_enable();
+		} else {
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+		}
+	} else {
+		/* Regular hotplug work */
+		if (st->state < st->target)
+			ret = cpuhp_ap_online(cpu, st);
+		else if (st->state > st->target)
+			ret = cpuhp_ap_offline(cpu, st);
+	}
+	st->result = ret;
+	complete(&st->done);
+}
+
+/* Invoke a single callback on a remote cpu */
+static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
+				    int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	if (!cpu_online(cpu))
+		return 0;
+
+	st->cb_state = state;
+	st->cb = cb;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
+/* Regular hotplug invocation of the AP hotplug thread */
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	st->result = 0;
+	st->cb = NULL;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	trace_cpuhp_exit(cpu, st->state, state, st->result);
+	return st->result;
+}
+
+static struct smp_hotplug_thread cpuhp_threads = {
+	.store			= &cpuhp_state.thread,
+	.create			= &cpuhp_create,
+	.thread_should_run	= cpuhp_should_run,
+	.thread_fn		= cpuhp_thread_fun,
+	.thread_comm		= "cpuhp/%u",
+	.selfparking		= true,
+};
+
+void __init cpuhp_threads_init(void)
+{
+	BUG_ON(smpboot_register_percpu_thread(&cpuhp_threads));
+	kthread_unpark(this_cpu_read(cpuhp_state.thread));
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -997,7 +1140,7 @@ static int cpuhp_cb_check(enum cpuhp_state state)
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
diff --git a/kernel/smp.c b/kernel/smp.c
index d903c02..822ffb1 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -569,6 +569,7 @@ void __init smp_init(void)
 	unsigned int cpu;
 
 	idle_threads_init();
+	cpuhp_threads_init();
 
 	/* FIXME: This should be done in userspace --RR */
 	for_each_present_cpu(cpu) {
diff --git a/kernel/smpboot.h b/kernel/smpboot.h
index 6b5f020..485b81c 100644
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -17,4 +17,6 @@ int smpboot_create_threads(unsigned int cpu);
 int smpboot_park_threads(unsigned int cpu);
 int smpboot_unpark_threads(unsigned int cpu);
 
+void __init cpuhp_threads_init(void);
+
 #endif

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Move online calls to hotplugged cpu
  2016-02-26 18:43 ` [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu Thomas Gleixner
@ 2016-03-01 19:57   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:57 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: oleg, hpa, mingo, paulmck, torvalds, srivatsa, riel, bigeasy, tj,
	pjt, akpm, peterz, linux-kernel, rostedt, rusty,
	rafael.j.wysocki, arjan, tglx

Commit-ID:  1cf4f629d9d246519a1e76c021806f2a51ddba4d
Gitweb:     http://git.kernel.org/tip/1cf4f629d9d246519a1e76c021806f2a51ddba4d
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:39 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:57 +0100

cpu/hotplug: Move online calls to hotplugged cpu

Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.536364371@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |  10 ++--
 kernel/cpu.c               | 144 ++++++++++++++++++++++++++++++---------------
 2 files changed, 102 insertions(+), 52 deletions(-)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 3867910..8a715bb 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_SMPBOOT_THREADS,
-	CPUHP_NOTIFY_ONLINE,
-	CPUHP_ONLINE_DYN,
-	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
+	CPUHP_KICK_AP_THREAD,
+	CPUHP_BP_ONLINE,
+	CPUHP_AP_SMPBOOT_THREADS,
+	CPUHP_AP_NOTIFY_ONLINE,
+	CPUHP_AP_ONLINE_DYN,
+	CPUHP_AP_ONLINE_DYN_END		= CPUHP_AP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 9048c33..e220e56 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int cpu)
 /* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
 static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
-	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+	enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
 
 	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
 }
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned int cpu)
 			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
 		}
 	} else {
+		/* Cannot happen .... */
+		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
 		/* Regular hotplug work */
 		if (st->state < st->target)
 			ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
 }
 
 /* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
-	enum cpuhp_state state = st->state;
-
-	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
 	st->result = 0;
 	st->cb = NULL;
 	/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned int cpu)
 	smp_mb();
 	st->should_run = true;
 	wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	__cpuhp_kick_ap_work(st);
 	wait_for_completion(&st->done);
 	trace_cpuhp_exit(cpu, st->state, state, st->result);
 	return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu)
 	else
 		synchronize_rcu();
 
+	/* Park the hotplug thread */
+	kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 
 	prev_state = st->state;
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+
+		/*
+		 * We might have stopped still in the range of the AP hotplug
+		 * thread. Nothing to do anymore.
+		 */
+		if (st->state >= CPUHP_KICK_AP_THREAD)
+			goto out;
+	}
+	/*
+	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * to do the further cleanups.
+	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
 	cpu_hotplug_done();
 	/* This post dead nonsense must die */
 	if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cpu)
  */
 static int cpuhp_set_cpu_active(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread */
+	/* Unpark the stopper thread and the hotplug thread */
 	stop_machine_unpark(cpu);
+	kthread_unpark(st->thread);
 	return 0;
 }
 
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread once more.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+	}
+
+	/*
+	 * Try to reach the target state. We max out on the BP at
+	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * responsible for bringing it up to the target state.
+	 */
+	target = min((int)target, CPUHP_KICK_AP_THREAD);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
-	[CPUHP_SMPBOOT_THREADS] = {
-		.name			= "smpboot:threads",
-		.startup		= smpboot_unpark_threads,
-		.teardown		= smpboot_park_threads,
-	},
-	[CPUHP_NOTIFY_ONLINE] = {
-		.name			= "notify:online",
-		.startup		= notify_online,
-		.teardown		= notify_down_prepare,
-		.cant_stop		= true,
+	[CPUHP_KICK_AP_THREAD] = {
+		.name			= "cpuhp:kickthread",
+		.startup		= cpuhp_kick_ap_work,
+		.teardown		= cpuhp_kick_ap_work,
 	},
 #endif
-	[CPUHP_ONLINE] = {
+	[CPUHP_BP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states[] = {
 		.skip_onerr		= true,
 		.cant_stop		= true,
 	},
+	[CPUHP_AP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
+	[CPUHP_AP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
 #endif
 	[CPUHP_ONLINE] = {
 		.name			= "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_state state)
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+		return true;
+	return state > CPUHP_BP_ONLINE;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
 	return cpuhp_get_step(state)->teardown;
 }
 
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
-	int (*cb)(unsigned int cpu) = __cb;
-
-	BUG_ON(cb(smp_processor_id()));
-}
-
 /*
  * Call the startup/teardown function for a step either on the AP or
  * on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
 
 	if (!cb)
 		return 0;
-
-	/*
-	 * This invokes the callback directly for now. In a later step we
-	 * convert that to use cpuhp_invoke_callback().
-	 */
-	if (cpuhp_is_ap_state(state)) {
-		/*
-		 * Note, that a function called on the AP is not
-		 * allowed to fail.
-		 */
-		if (cpu_online(cpu))
-			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
-		return 0;
-	}
-
 	/*
 	 * The non AP bound callbacks can fail on bringup. On teardown
 	 * e.g. module removal we crash for now.
 	 */
-	ret = cb(cpu);
+#ifdef CONFIG_SMP
+	if (cpuhp_is_ap_state(state))
+		ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+	else
+		ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+	ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
 	BUG_ON(ret && !bringup);
 	return ret;
 }
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuhp_state state)
 	enum cpuhp_state i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
-		if (cpuhp_bp_states[i].name)
+	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+		if (cpuhp_ap_states[i].name)
 			continue;
 
-		cpuhp_bp_states[i].name = "Reserved";
+		cpuhp_ap_states[i].name = "Reserved";
 		mutex_unlock(&cpuhp_state_mutex);
 		return i;
 	}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state state,
 	get_online_cpus();
 
 	/* currently assignments for the ONLINE state are possible */
-	if (state == CPUHP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN) {
 		dyn_state = 1;
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] arch/hotplug: Call into idle with a proper state
  2016-02-26 18:43 ` [patch 17/20] arch/hotplug: Call into idle with a proper state Thomas Gleixner
@ 2016-03-01 19:57   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:57 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: rostedt, rafael.j.wysocki, tj, peterz, tglx, torvalds, arjan,
	srivatsa, bigeasy, paulmck, linux-kernel, pjt, mingo, riel,
	rusty, hpa, oleg, akpm

Commit-ID:  fc6d73d67436e7784758a831227bd019547a3f73
Gitweb:     http://git.kernel.org/tip/fc6d73d67436e7784758a831227bd019547a3f73
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:40 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:57 +0100

arch/hotplug: Call into idle with a proper state

Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.614102639@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/alpha/kernel/smp.c         | 2 +-
 arch/arc/kernel/smp.c           | 2 +-
 arch/arm/kernel/smp.c           | 2 +-
 arch/arm64/kernel/smp.c         | 2 +-
 arch/blackfin/mach-common/smp.c | 2 +-
 arch/hexagon/kernel/smp.c       | 2 +-
 arch/ia64/kernel/smpboot.c      | 2 +-
 arch/m32r/kernel/smpboot.c      | 2 +-
 arch/metag/kernel/smp.c         | 2 +-
 arch/mips/kernel/smp.c          | 2 +-
 arch/mn10300/kernel/smp.c       | 2 +-
 arch/parisc/kernel/smp.c        | 2 +-
 arch/powerpc/kernel/smp.c       | 2 +-
 arch/s390/kernel/smp.c          | 2 +-
 arch/sh/kernel/smp.c            | 2 +-
 arch/sparc/kernel/smp_32.c      | 2 +-
 arch/sparc/kernel/smp_64.c      | 2 +-
 arch/tile/kernel/smpboot.c      | 2 +-
 arch/x86/kernel/smpboot.c       | 2 +-
 arch/x86/xen/smp.c              | 2 +-
 arch/xtensa/kernel/smp.c        | 2 +-
 include/linux/cpuhotplug.h      | 1 +
 22 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/arch/alpha/kernel/smp.c b/arch/alpha/kernel/smp.c
index 2f24447f..46bf263 100644
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
 	      cpuid, current, current->active_mm));
 
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Wait until hwrpb->txrdy is clear for cpu.  Return -1 on timeout.  */
diff --git a/arch/arc/kernel/smp.c b/arch/arc/kernel/smp.c
index 424e937..4cb3add 100644
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
 
 	local_irq_enable();
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /*
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index 37312f6..baee702 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(void)
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index b1adc51..4607657 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(void)
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
diff --git a/arch/blackfin/mach-common/smp.c b/arch/blackfin/mach-common/smp.c
index 0030e21..23c4ef5 100644
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
 
 	/* We are done with local CPU inits, unblock the boot CPU. */
 	set_cpu_online(cpu, true);
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_prepare_boot_cpu(void)
diff --git a/arch/hexagon/kernel/smp.c b/arch/hexagon/kernel/smp.c
index ff759f2..983bae7 100644
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 
diff --git a/arch/ia64/kernel/smpboot.c b/arch/ia64/kernel/smpboot.c
index 0e76fad..74fe317 100644
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
 	preempt_disable();
 	smp_callin();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
diff --git a/arch/m32r/kernel/smpboot.c b/arch/m32r/kernel/smpboot.c
index a468467..f98d2f6 100644
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
 	 */
 	local_flush_tlb_all();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
diff --git a/arch/metag/kernel/smp.c b/arch/metag/kernel/smp.c
index c3c6f08..bad1323 100644
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(void)
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index bd4385a..f2112a8 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
 	WARN_ON_ONCE(!irqs_disabled());
 	mp_ops->smp_finish();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void stop_this_cpu(void *dummy)
diff --git a/arch/mn10300/kernel/smp.c b/arch/mn10300/kernel/smp.c
index f984193..426173c 100644
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
 #ifdef CONFIG_GENERIC_CLOCKEVENTS
 	init_clockevents();
 #endif
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
diff --git a/arch/parisc/kernel/smp.c b/arch/parisc/kernel/smp.c
index 52e8597..c2a9cc5 100644
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
 
 	local_irq_enable();  /* Interrupts have been off until now */
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* NOTREACHED */
 	panic("smp_callin() AAAAaaaaahhhh....\n");
diff --git a/arch/powerpc/kernel/smp.c b/arch/powerpc/kernel/smp.c
index ec9ec20..cc13d4c 100644
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	BUG();
 }
diff --git a/arch/s390/kernel/smp.c b/arch/s390/kernel/smp.c
index 3c65a8e..40a6b4f 100644
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cpuvoid)
 	set_cpu_online(smp_processor_id(), true);
 	inc_irq_stat(CPU_RST);
 	local_irq_enable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Upping and downing of CPUs */
diff --git a/arch/sh/kernel/smp.c b/arch/sh/kernel/smp.c
index de6be00..13f633a 100644
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
 	set_cpu_online(cpu, true);
 	per_cpu(cpu_state, cpu) = CPU_ONLINE;
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 extern struct {
diff --git a/arch/sparc/kernel/smp_32.c b/arch/sparc/kernel/smp_32.c
index b3a5d81..fb30e7c 100644
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *arg)
 	local_irq_enable();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* We should never reach here! */
 	BUG();
diff --git a/arch/sparc/kernel/smp_64.c b/arch/sparc/kernel/smp_64.c
index 19cd08d..8a6151a 100644
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void cpu_panic(void)
diff --git a/arch/tile/kernel/smpboot.c b/arch/tile/kernel/smpboot.c
index 20d52a9..6c0abaa 100644
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
 	/* Set up tile-timer clock-event device on this cpu */
 	setup_tile_timer();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 int __cpu_up(unsigned int cpu, struct task_struct *tidle)
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 24d57f7..293b22a 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void *unused)
 	x86_cpuinit.setup_percpu_clockev();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_store_boot_cpu_info(void)
diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c
index 3f4ebf0..3c6d17f 100644
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_and_idle(int cpu)
 		xen_pvh_secondary_vcpu_init(cpu);
 #endif
 	cpu_bringup();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void xen_smp_intr_free(unsigned int cpu)
diff --git a/arch/xtensa/kernel/smp.c b/arch/xtensa/kernel/smp.c
index 4d02e38..fc4ad21 100644
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
 
 	complete(&cpu_running);
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void mx_cpu_start(void *p)
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 8a715bb..4aa263a 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
 	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_KICK_AP_THREAD,
 	CPUHP_BP_ONLINE,
+	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
 	CPUHP_AP_ONLINE_DYN,

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Let upcoming cpu bring itself fully up
  2016-02-26 18:43 ` [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner
@ 2016-03-01 19:58   ` tip-bot for Thomas Gleixner
  2016-03-02 17:28   ` [patch 18/20] " Richard Cochran
  1 sibling, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:58 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: bigeasy, rostedt, arjan, paulmck, srivatsa, rusty, mingo, pjt,
	tj, tglx, oleg, riel, linux-kernel, torvalds, rafael.j.wysocki,
	hpa, akpm, peterz

Commit-ID:  8df3e07e7f21f2ed8d001e6fabf9505946b438aa
Gitweb:     http://git.kernel.org/tip/8df3e07e7f21f2ed8d001e6fabf9505946b438aa
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:41 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:57 +0100

cpu/hotplug: Let upcoming cpu bring itself fully up

Let the upcoming cpu kick the hotplug thread and let itself complete the
bringup. That way the controll side can just wait for the completion or later
when we made the hotplug machinery async not care at all.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.697655464@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |  9 ++++---
 kernel/cpu.c               | 66 ++++++++++++++++++++++++++--------------------
 kernel/sched/idle.c        |  2 ++
 3 files changed, 45 insertions(+), 32 deletions(-)

diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 4aa263a..ad5d7fc 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,9 +10,6 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
-	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_KICK_AP_THREAD,
-	CPUHP_BP_ONLINE,
 	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
@@ -86,4 +83,10 @@ static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
 	__cpuhp_remove_state(state, false);
 }
 
+#ifdef CONFIG_SMP
+void cpuhp_online_idle(enum cpuhp_state state);
+#else
+static inline void cpuhp_online_idle(enum cpuhp_state state) { }
+#endif
+
 #endif
diff --git a/kernel/cpu.c b/kernel/cpu.c
index e220e56..f1f880f 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,6 +329,14 @@ static int notify_starting(unsigned int cpu)
 	return 0;
 }
 
+static int bringup_wait_for_ap(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -340,8 +348,9 @@ static int bringup_cpu(unsigned int cpu)
 		cpu_notify(CPU_UP_CANCELED, cpu);
 		return ret;
 	}
+	ret = bringup_wait_for_ap(cpu);
 	BUG_ON(!cpu_online(cpu));
-	return 0;
+	return ret;
 }
 
 /*
@@ -470,7 +479,7 @@ static void cpuhp_thread_fun(unsigned int cpu)
 		}
 	} else {
 		/* Cannot happen .... */
-		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+		BUG_ON(st->state < CPUHP_AP_ONLINE_IDLE);
 
 		/* Regular hotplug work */
 		if (st->state < st->target)
@@ -780,7 +789,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_TEARDOWN_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -793,11 +802,11 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
 		 * We might have stopped still in the range of the AP hotplug
 		 * thread. Nothing to do anymore.
 		 */
-		if (st->state >= CPUHP_KICK_AP_THREAD)
+		if (st->state > CPUHP_TEARDOWN_CPU)
 			goto out;
 	}
 	/*
-	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * The AP brought itself down to CPUHP_TEARDOWN_CPU. So we need
 	 * to do the further cleanups.
 	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
@@ -859,18 +868,32 @@ void notify_cpu_starting(unsigned int cpu)
 
 /*
  * Called from the idle task. We need to set active here, so we can kick off
- * the stopper thread.
+ * the stopper thread and unpark the smpboot threads. If the target state is
+ * beyond CPUHP_AP_ONLINE_IDLE we kick cpuhp thread and let it bring up the
+ * cpu further.
  */
-static int cpuhp_set_cpu_active(unsigned int cpu)
+void cpuhp_online_idle(enum cpuhp_state state)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	unsigned int cpu = smp_processor_id();
+
+	/* Happens for the boot cpu */
+	if (state != CPUHP_AP_ONLINE_IDLE)
+		return;
+
+	st->state = CPUHP_AP_ONLINE_IDLE;
 
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread and the hotplug thread */
+	/* Unpark the stopper thread and the hotplug thread of this cpu */
 	stop_machine_unpark(cpu);
 	kthread_unpark(st->thread);
-	return 0;
+
+	/* Should we go further up ? */
+	if (st->target > CPUHP_AP_ONLINE_IDLE)
+		__cpuhp_kick_ap_work(st);
+	else
+		complete(&st->done);
 }
 
 /* Requires cpu_add_remove_lock to be held */
@@ -910,7 +933,7 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread once more.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_BRINGUP_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -922,10 +945,10 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 
 	/*
 	 * Try to reach the target state. We max out on the BP at
-	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * CPUHP_BRINGUP_CPU. After that the AP hotplug thread is
 	 * responsible for bringing it up to the target state.
 	 */
-	target = min((int)target, CPUHP_KICK_AP_THREAD);
+	target = min((int)target, CPUHP_BRINGUP_CPU);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1146,22 +1169,7 @@ static struct cpuhp_step cpuhp_bp_states[] = {
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
-	[CPUHP_CPU_SET_ACTIVE] = {
-		.name			= "cpu:active",
-		.startup		= cpuhp_set_cpu_active,
-		.teardown		= NULL,
-	},
-	[CPUHP_KICK_AP_THREAD] = {
-		.name			= "cpuhp:kickthread",
-		.startup		= cpuhp_kick_ap_work,
-		.teardown		= cpuhp_kick_ap_work,
-	},
 #endif
-	[CPUHP_BP_ONLINE] = {
-		.name			= "online",
-		.startup		= NULL,
-		.teardown		= NULL,
-	},
 };
 
 /* Application processor state steps */
@@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
 	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
 		return true;
-	return state > CPUHP_BP_ONLINE;
+	return state > CPUHP_BRINGUP_CPU;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 544a713..a4b9813 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -4,6 +4,7 @@
 #include <linux/sched.h>
 #include <linux/cpu.h>
 #include <linux/cpuidle.h>
+#include <linux/cpuhotplug.h>
 #include <linux/tick.h>
 #include <linux/mm.h>
 #include <linux/stackprotector.h>
@@ -291,5 +292,6 @@ void cpu_startup_entry(enum cpuhp_state state)
 	boot_init_stack_canary();
 #endif
 	arch_cpu_idle_prepare();
+	cpuhp_online_idle(state);
 	cpu_idle_loop();
 }

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Make wait for dead cpu completion based
  2016-02-26 18:43 ` [patch 19/20] cpu/hotplug: Make wait for dead cpu completion based Thomas Gleixner
@ 2016-03-01 19:58   ` tip-bot for Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:58 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: arjan, mingo, rusty, paulmck, riel, tglx, hpa, bigeasy, torvalds,
	tj, pjt, rafael.j.wysocki, oleg, akpm, rostedt, linux-kernel,
	peterz, srivatsa

Commit-ID:  e69aab13117efc1987620090e539b4ebeb33a04c
Gitweb:     http://git.kernel.org/tip/e69aab13117efc1987620090e539b4ebeb33a04c
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:43 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:58 +0100

cpu/hotplug: Make wait for dead cpu completion based

Kill the busy spinning on the control side and just wait for the hotplugged
cpu to tell that it reached the dead state.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.776157858@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |  5 +++--
 include/linux/cpuhotplug.h |  1 +
 kernel/cpu.c               | 16 ++++++++++++----
 kernel/sched/idle.c        |  5 +----
 4 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 83f3576..91a48d1 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -276,14 +276,15 @@ void arch_cpu_idle_enter(void);
 void arch_cpu_idle_exit(void);
 void arch_cpu_idle_dead(void);
 
-DECLARE_PER_CPU(bool, cpu_dead_idle);
-
 int cpu_report_state(int cpu);
 int cpu_check_up_prepare(int cpu);
 void cpu_set_state_online(int cpu);
 #ifdef CONFIG_HOTPLUG_CPU
 bool cpu_wait_death(unsigned int cpu, int seconds);
 bool cpu_report_death(void);
+void cpuhp_report_idle_dead(void);
+#else
+static inline void cpuhp_report_idle_dead(void) { }
 #endif /* #ifdef CONFIG_HOTPLUG_CPU */
 
 #endif /* _LINUX_CPU_H_ */
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index ad5d7fc..5d68e15 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,7 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
diff --git a/kernel/cpu.c b/kernel/cpu.c
index f1f880f..0e8c07f 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -688,6 +688,7 @@ static int take_cpu_down(void *_param)
 
 static int takedown_cpu(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int err;
 
 	/*
@@ -733,10 +734,8 @@ static int takedown_cpu(unsigned int cpu)
 	 *
 	 * Wait for the stop thread to go away.
 	 */
-	while (!per_cpu(cpu_dead_idle, cpu))
-		cpu_relax();
-	smp_mb(); /* Read from cpu_dead_idle before __cpu_die(). */
-	per_cpu(cpu_dead_idle, cpu) = false;
+	wait_for_completion(&st->done);
+	BUG_ON(st->state != CPUHP_AP_IDLE_DEAD);
 
 	/* Interrupts are moved away from the dying cpu, reenable alloc/free */
 	irq_unlock_sparse();
@@ -756,6 +755,15 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+void cpuhp_report_idle_dead(void)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	BUG_ON(st->state != CPUHP_AP_OFFLINE);
+	st->state = CPUHP_AP_IDLE_DEAD;
+	complete(&st->done);
+}
+
 #else
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index a4b9813..8abbe89 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -194,8 +194,6 @@ exit_idle:
 	rcu_idle_exit();
 }
 
-DEFINE_PER_CPU(bool, cpu_dead_idle);
-
 /*
  * Generic idle loop implementation
  *
@@ -224,8 +222,7 @@ static void cpu_idle_loop(void)
 			if (cpu_is_offline(smp_processor_id())) {
 				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
 					       (void *)(long)smp_processor_id());
-				smp_mb(); /* all activity before dead. */
-				this_cpu_write(cpu_dead_idle, true);
+				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}
 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call
  2016-02-26 18:43 ` [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner
  2016-02-27  2:14   ` Paul E. McKenney
@ 2016-03-01 19:58   ` tip-bot for Thomas Gleixner
  2016-03-02 20:11     ` Paul E. McKenney
  1 sibling, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-01 19:58 UTC (permalink / raw)
  To: linux-tip-commits
  Cc: torvalds, srivatsa, tj, rusty, bigeasy, pjt, peterz,
	rafael.j.wysocki, riel, arjan, tglx, rostedt, linux-kernel,
	mingo, paulmck, hpa, akpm, oleg

Commit-ID:  27d50c7eeb0f03c3d3ca72aac4d2dd487ca1f3f0
Gitweb:     http://git.kernel.org/tip/27d50c7eeb0f03c3d3ca72aac4d2dd487ca1f3f0
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Fri, 26 Feb 2016 18:43:44 +0000
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Tue, 1 Mar 2016 20:36:58 +0100

rcu: Make CPU_DYING_IDLE an explicit call

Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
invoked at the proper place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-arch@vger.kernel.org
Cc: Rik van Riel <riel@redhat.com>
Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul Turner <pjt@google.com>
Link: http://lkml.kernel.org/r/20160226182341.870167933@linutronix.de
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h      |  4 +--
 include/linux/notifier.h |  2 ++
 include/linux/rcupdate.h |  4 +--
 kernel/cpu.c             |  1 +
 kernel/rcu/tree.c        | 70 +++++++++++++++++++++++++-----------------------
 kernel/sched/idle.c      |  2 --
 6 files changed, 42 insertions(+), 41 deletions(-)

diff --git a/include/linux/cpu.h b/include/linux/cpu.h
index 91a48d1..f9b1fab 100644
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -101,9 +101,7 @@ enum {
 					* Called on the new cpu, just before
 					* enabling interrupts. Must not sleep,
 					* must not fail */
-#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
-					* idle loop. */
-#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
+#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
 					* perhaps due to preemption. */
 
 /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
diff --git a/include/linux/notifier.h b/include/linux/notifier.h
index d14a4c3..4149868 100644
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -47,6 +47,8 @@
  * runtime initialization.
  */
 
+struct notifier_block;
+
 typedef	int (*notifier_fn_t)(struct notifier_block *nb,
 			unsigned long action, void *data);
 
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 14e6f47..fc46fe3 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -332,9 +332,7 @@ void rcu_init(void);
 void rcu_sched_qs(void);
 void rcu_bh_qs(void);
 void rcu_check_callbacks(int user);
-struct notifier_block;
-int rcu_cpu_notify(struct notifier_block *self,
-		   unsigned long action, void *hcpu);
+void rcu_report_dead(unsigned int cpu);
 
 #ifndef CONFIG_TINY_RCU
 void rcu_end_inkernel_boot(void);
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 0e8c07f..ff8059b 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
 	BUG_ON(st->state != CPUHP_AP_OFFLINE);
 	st->state = CPUHP_AP_IDLE_DEAD;
 	complete(&st->done);
+	rcu_report_dead(smp_processor_id());
 }
 
 #else
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index e41dd41..85b4134 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -2607,28 +2607,6 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf)
 }
 
 /*
- * The CPU is exiting the idle loop into the arch_cpu_idle_dead()
- * function.  We now remove it from the rcu_node tree's ->qsmaskinit
- * bit masks.
- */
-static void rcu_cleanup_dying_idle_cpu(int cpu, struct rcu_state *rsp)
-{
-	unsigned long flags;
-	unsigned long mask;
-	struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
-	struct rcu_node *rnp = rdp->mynode;  /* Outgoing CPU's rdp & rnp. */
-
-	if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
-		return;
-
-	/* Remove outgoing CPU from mask in the leaf rcu_node structure. */
-	mask = rdp->grpmask;
-	raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */
-	rnp->qsmaskinitnext &= ~mask;
-	raw_spin_unlock_irqrestore(&rnp->lock, flags);
-}
-
-/*
  * The CPU has been completely removed, and some other CPU is reporting
  * this fact from process context.  Do the remainder of the cleanup,
  * including orphaning the outgoing CPU's RCU callbacks, and also
@@ -4247,6 +4225,43 @@ static void rcu_prepare_cpu(int cpu)
 		rcu_init_percpu_data(cpu, rsp);
 }
 
+#ifdef CONFIG_HOTPLUG_CPU
+/*
+ * The CPU is exiting the idle loop into the arch_cpu_idle_dead()
+ * function.  We now remove it from the rcu_node tree's ->qsmaskinit
+ * bit masks.
+ */
+static void rcu_cleanup_dying_idle_cpu(int cpu, struct rcu_state *rsp)
+{
+	unsigned long flags;
+	unsigned long mask;
+	struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
+	struct rcu_node *rnp = rdp->mynode;  /* Outgoing CPU's rdp & rnp. */
+
+	if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
+		return;
+
+	/* Remove outgoing CPU from mask in the leaf rcu_node structure. */
+	mask = rdp->grpmask;
+	raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */
+	rnp->qsmaskinitnext &= ~mask;
+	raw_spin_unlock_irqrestore(&rnp->lock, flags);
+}
+
+void rcu_report_dead(unsigned int cpu)
+{
+	struct rcu_state *rsp;
+
+	/* QS for any half-done expedited RCU-sched GP. */
+	preempt_disable();
+	rcu_report_exp_rdp(&rcu_sched_state,
+			   this_cpu_ptr(rcu_sched_state.rda), true);
+	preempt_enable();
+	for_each_rcu_flavor(rsp)
+		rcu_cleanup_dying_idle_cpu(cpu, rsp);
+}
+#endif
+
 /*
  * Handle CPU online/offline notification events.
  */
@@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block *self,
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dying_cpu(rsp);
 		break;
-	case CPU_DYING_IDLE:
-		/* QS for any half-done expedited RCU-sched GP. */
-		preempt_disable();
-		rcu_report_exp_rdp(&rcu_sched_state,
-				   this_cpu_ptr(rcu_sched_state.rda), true);
-		preempt_enable();
-
-		for_each_rcu_flavor(rsp) {
-			rcu_cleanup_dying_idle_cpu(cpu, rsp);
-		}
-		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
index 8abbe89..bd12c6c 100644
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
 			rmb();
 
 			if (cpu_is_offline(smp_processor_id())) {
-				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
-					       (void *)(long)smp_processor_id());
 				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Convert to a state machine for the control processor
  2016-03-01 19:53   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 11:23     ` Peter Zijlstra
  0 siblings, 0 replies; 69+ messages in thread
From: Peter Zijlstra @ 2016-03-02 11:23 UTC (permalink / raw)
  To: rusty, bigeasy, rostedt, oleg, tglx, pjt, riel, torvalds,
	linux-kernel, arjan, tj, srivatsa, akpm, mingo, hpa, paulmck,
	rafael.j.wysocki
  Cc: linux-tip-commits

On Tue, Mar 01, 2016 at 11:53:25AM -0800, tip-bot for Thomas Gleixner wrote:
> @@ -558,22 +648,22 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  
>  	cpuhp_tasks_frozen = tasks_frozen;
>  
> +	prev_state = st->state;
> +	st->target = CPUHP_ONLINE;
> +	while (st->state < st->target) {
> +		struct cpuhp_step *step;
> +
> +		st->state++;
> +		step = cpuhp_bp_states + st->state;
> +		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
> +		if (ret) {
> +			st->target = prev_state;
> +			undo_cpu_up(cpu, st);
> +			break;
> +		}
> +	}

All your other state loops use for().

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Add sysfs state interface
  2016-03-01 19:54   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 12:40     ` Peter Zijlstra
  0 siblings, 0 replies; 69+ messages in thread
From: Peter Zijlstra @ 2016-03-02 12:40 UTC (permalink / raw)
  To: bigeasy, rafael.j.wysocki, tglx, torvalds, pjt, hpa, paulmck,
	oleg, arjan, linux-kernel, tj, akpm, mingo, rusty, rostedt,
	srivatsa, riel
  Cc: linux-tip-commits

On Tue, Mar 01, 2016 at 11:54:37AM -0800, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  98f8cdce1db580b99fce823a48eea2cb2bdb261e
> Gitweb:     http://git.kernel.org/tip/98f8cdce1db580b99fce823a48eea2cb2bdb261e
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:31 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:55 +0100
> 
> cpu/hotplug: Add sysfs state interface
> 
> Add a sysfs interface so we can actually see in which state the cpus are in.

I feel this Changelog can use a few extra lines to explain the ABI
ramifications and why this lives here and not in drivers/base/cpu.c
where the rest of the cpu_subsys muck lives.

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Make target state writeable
  2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 12:41     ` Peter Zijlstra
  2016-03-02 19:57       ` Thomas Gleixner
  0 siblings, 1 reply; 69+ messages in thread
From: Peter Zijlstra @ 2016-03-02 12:41 UTC (permalink / raw)
  To: rostedt, tglx, riel, mingo, rafael.j.wysocki, bigeasy, arjan,
	rusty, torvalds, hpa, srivatsa, pjt, tj, akpm, oleg,
	linux-kernel, paulmck
  Cc: linux-tip-commits

On Tue, Mar 01, 2016 at 11:55:01AM -0800, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  757c989b9994f51b42d6be1bd33c7c12d16a3ac7
> Gitweb:     http://git.kernel.org/tip/757c989b9994f51b42d6be1bd33c7c12d16a3ac7
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:32 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:55 +0100
> 
> cpu/hotplug: Make target state writeable
> 
> Make it possible to write a target state to the per cpu state file, so we can
> switch between states.

But you've marked nearly all of them cant_stop, so while this implements
the interface, its not really functional just yet, right?

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up
  2016-02-26 18:43 ` [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 17:28   ` Richard Cochran
  1 sibling, 0 replies; 69+ messages in thread
From: Richard Cochran @ 2016-03-02 17:28 UTC (permalink / raw)
  To: Thomas Gleixner
  Cc: LKML, Linus Torvalds, Andrew Morton, Ingo Molnar, Peter Zijlstra,
	Peter Anvin, Oleg Nesterov, linux-arch, Tejun Heo,
	Steven Rostedt, Rusty Russell, Paul McKenney, Rafael Wysocki,
	Arjan van de Ven, Rik van Riel, Srivatsa S. Bhat,
	Sebastian Siewior, Paul Turner

On Fri, Feb 26, 2016 at 06:43:41PM -0000, Thomas Gleixner wrote:
> @@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp
>  {
>  	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
>  		return true;

FWIW, this test is superfluous after this change, because
CPUHP_BRINGUP_CPU is less than CPUHP_AP_OFFLINE.

> -	return state > CPUHP_BP_ONLINE;
> +	return state > CPUHP_BRINGUP_CPU;
>  }

Thanks,
Richard

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Make target state writeable
  2016-03-02 12:41     ` Peter Zijlstra
@ 2016-03-02 19:57       ` Thomas Gleixner
  0 siblings, 0 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-03-02 19:57 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: rostedt, riel, mingo, rafael.j.wysocki, bigeasy, arjan, rusty,
	torvalds, hpa, srivatsa, pjt, tj, akpm, oleg, linux-kernel,
	paulmck, linux-tip-commits

On Wed, 2 Mar 2016, Peter Zijlstra wrote:
> On Tue, Mar 01, 2016 at 11:55:01AM -0800, tip-bot for Thomas Gleixner wrote:
> > Commit-ID:  757c989b9994f51b42d6be1bd33c7c12d16a3ac7
> > Gitweb:     http://git.kernel.org/tip/757c989b9994f51b42d6be1bd33c7c12d16a3ac7
> > Author:     Thomas Gleixner <tglx@linutronix.de>
> > AuthorDate: Fri, 26 Feb 2016 18:43:32 +0000
> > Committer:  Thomas Gleixner <tglx@linutronix.de>
> > CommitDate: Tue, 1 Mar 2016 20:36:55 +0100
> > 
> > cpu/hotplug: Make target state writeable
> > 
> > Make it possible to write a target state to the per cpu state file, so we can
> > switch between states.
> 
> But you've marked nearly all of them cant_stop, so while this implements
> the interface, its not really functional just yet, right?

Yes, the current state is more or less wrapping the existing notifiers and
stopping them is not a brilliant idea. I've tried and it exploded nicely in my
face. The ones we convert over to the state machine will be stoppable and
reversible. And at the end the notifiers will be gone and with them the can't
stop stuff, except maybe for a few crucial points.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call
  2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 20:11     ` Paul E. McKenney
  2016-03-03  7:31       ` Thomas Gleixner
  2016-03-03 10:03       ` [tip:smp/hotplug] cpu/hotplug: Plug death reporting race tip-bot for Thomas Gleixner
  0 siblings, 2 replies; 69+ messages in thread
From: Paul E. McKenney @ 2016-03-02 20:11 UTC (permalink / raw)
  To: rafael.j.wysocki, peterz, riel, tglx, arjan, rusty, tj, srivatsa,
	torvalds, bigeasy, pjt, akpm, hpa, oleg, mingo, linux-kernel,
	rostedt
  Cc: linux-tip-commits

On Tue, Mar 01, 2016 at 11:58:59AM -0800, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  27d50c7eeb0f03c3d3ca72aac4d2dd487ca1f3f0
> Gitweb:     http://git.kernel.org/tip/27d50c7eeb0f03c3d3ca72aac4d2dd487ca1f3f0
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:44 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:58 +0100
> 
> rcu: Make CPU_DYING_IDLE an explicit call
> 
> Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
> invoked at the proper place.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182341.870167933@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

[ . . . ]

> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 0e8c07f..ff8059b 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
>  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
>  	st->state = CPUHP_AP_IDLE_DEAD;
>  	complete(&st->done);

Not to be repetitive or anything, but if we delay here, it can break
RCU on a number of architectures.  Either the CPU can be killed holding
one of RCU's locks or RCU can wrongly see the CPU as still being alive.
Either can prevent future RCU grace periods from ever completing, thus
OOMing the system.

							Thanx, Paul

> +	rcu_report_dead(smp_processor_id());
>  }
> 
>  #else
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index e41dd41..85b4134 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -2607,28 +2607,6 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf)
>  }
> 
>  /*
> - * The CPU is exiting the idle loop into the arch_cpu_idle_dead()
> - * function.  We now remove it from the rcu_node tree's ->qsmaskinit
> - * bit masks.
> - */
> -static void rcu_cleanup_dying_idle_cpu(int cpu, struct rcu_state *rsp)
> -{
> -	unsigned long flags;
> -	unsigned long mask;
> -	struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
> -	struct rcu_node *rnp = rdp->mynode;  /* Outgoing CPU's rdp & rnp. */
> -
> -	if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
> -		return;
> -
> -	/* Remove outgoing CPU from mask in the leaf rcu_node structure. */
> -	mask = rdp->grpmask;
> -	raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */
> -	rnp->qsmaskinitnext &= ~mask;
> -	raw_spin_unlock_irqrestore(&rnp->lock, flags);
> -}
> -
> -/*
>   * The CPU has been completely removed, and some other CPU is reporting
>   * this fact from process context.  Do the remainder of the cleanup,
>   * including orphaning the outgoing CPU's RCU callbacks, and also
> @@ -4247,6 +4225,43 @@ static void rcu_prepare_cpu(int cpu)
>  		rcu_init_percpu_data(cpu, rsp);
>  }
> 
> +#ifdef CONFIG_HOTPLUG_CPU
> +/*
> + * The CPU is exiting the idle loop into the arch_cpu_idle_dead()
> + * function.  We now remove it from the rcu_node tree's ->qsmaskinit
> + * bit masks.
> + */
> +static void rcu_cleanup_dying_idle_cpu(int cpu, struct rcu_state *rsp)
> +{
> +	unsigned long flags;
> +	unsigned long mask;
> +	struct rcu_data *rdp = per_cpu_ptr(rsp->rda, cpu);
> +	struct rcu_node *rnp = rdp->mynode;  /* Outgoing CPU's rdp & rnp. */
> +
> +	if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
> +		return;
> +
> +	/* Remove outgoing CPU from mask in the leaf rcu_node structure. */
> +	mask = rdp->grpmask;
> +	raw_spin_lock_irqsave_rcu_node(rnp, flags); /* Enforce GP memory-order guarantee. */
> +	rnp->qsmaskinitnext &= ~mask;
> +	raw_spin_unlock_irqrestore(&rnp->lock, flags);
> +}
> +
> +void rcu_report_dead(unsigned int cpu)
> +{
> +	struct rcu_state *rsp;
> +
> +	/* QS for any half-done expedited RCU-sched GP. */
> +	preempt_disable();
> +	rcu_report_exp_rdp(&rcu_sched_state,
> +			   this_cpu_ptr(rcu_sched_state.rda), true);
> +	preempt_enable();
> +	for_each_rcu_flavor(rsp)
> +		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> +}
> +#endif
> +
>  /*
>   * Handle CPU online/offline notification events.
>   */
> @@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block *self,
>  		for_each_rcu_flavor(rsp)
>  			rcu_cleanup_dying_cpu(rsp);
>  		break;
> -	case CPU_DYING_IDLE:
> -		/* QS for any half-done expedited RCU-sched GP. */
> -		preempt_disable();
> -		rcu_report_exp_rdp(&rcu_sched_state,
> -				   this_cpu_ptr(rcu_sched_state.rda), true);
> -		preempt_enable();
> -
> -		for_each_rcu_flavor(rsp) {
> -			rcu_cleanup_dying_idle_cpu(cpu, rsp);
> -		}
> -		break;
>  	case CPU_DEAD:
>  	case CPU_DEAD_FROZEN:
>  	case CPU_UP_CANCELED:
> diff --git a/kernel/sched/idle.c b/kernel/sched/idle.c
> index 8abbe89..bd12c6c 100644
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
>  			rmb();
> 
>  			if (cpu_is_offline(smp_processor_id())) {
> -				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
> -					       (void *)(long)smp_processor_id());
>  				cpuhp_report_idle_dead();
>  				arch_cpu_idle_dead();
>  			}
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling
  2016-03-01 19:51   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 22:35     ` Srivatsa S. Bhat
  2016-03-02 23:43     ` Srivatsa S. Bhat
  1 sibling, 0 replies; 69+ messages in thread
From: Srivatsa S. Bhat @ 2016-03-02 22:35 UTC (permalink / raw)
  To: tj, akpm, rusty, linux-kernel, rostedt, paulmck, peterz, mingo,
	arjan, tglx, rafael.j.wysocki, pjt, riel, bigeasy, oleg,
	torvalds, srivatsa, hpa, linux-tip-commits

On 3/1/16 2:51 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  090e77c391dd983c8945b8e2e16d09f378d2e334
> Gitweb:     http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
> 
> cpu/hotplug: Restructure FROZEN state handling
> 
> There are only a few callbacks which really care about FROZEN
> vs. !FROZEN. No need to have extra states for this.
> 
> Publish the frozen state in an extra variable which is updated under
> the hotplug lock and let the users interested deal with it w/o
> imposing that extra state checks on everyone.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---

Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>

Regards,
Srivatsa S. Bhat

>  include/linux/cpu.h |  2 ++
>  kernel/cpu.c        | 69 ++++++++++++++++++++++-------------------------------
>  2 files changed, 31 insertions(+), 40 deletions(-)
> 
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index d2ca8c3..f2fb549 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -118,6 +118,7 @@ enum {
>  
>  
>  #ifdef CONFIG_SMP
> +extern bool cpuhp_tasks_frozen;
>  /* Need to know about CPUs going up/down? */
>  #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
>  #define cpu_notifier(fn, pri) {					\
> @@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
>  #define cpu_notifier_register_done	cpu_maps_update_done
>  
>  #else	/* CONFIG_SMP */
> +#define cpuhp_tasks_frozen	0
>  
>  #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
>  #define __cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 5b9d396..41a6cb8 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -29,6 +29,8 @@
>  #ifdef CONFIG_SMP
>  /* Serializes the updates to cpu_online_mask, cpu_present_mask */
>  static DEFINE_MUTEX(cpu_add_remove_lock);
> +bool cpuhp_tasks_frozen;
> +EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
>  
>  /*
>   * The following two APIs (cpu_maps_update_begin/done) must be used when
> @@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notifier_block *nb)
>  	return raw_notifier_chain_register(&cpu_chain, nb);
>  }
>  
> -static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
> +static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
>  			int *nr_calls)
>  {
> +	unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
> +	void *hcpu = (void *)(long)cpu;
> +
>  	int ret;
>  
> -	ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
> +	ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
>  					nr_calls);
>  
>  	return notifier_to_errno(ret);
>  }
>  
> -static int cpu_notify(unsigned long val, void *v)
> +static int cpu_notify(unsigned long val, unsigned int cpu)
>  {
> -	return __cpu_notify(val, v, -1, NULL);
> +	return __cpu_notify(val, cpu, -1, NULL);
>  }
>  
>  #ifdef CONFIG_HOTPLUG_CPU
>  
> -static void cpu_notify_nofail(unsigned long val, void *v)
> +static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
>  {
> -	BUG_ON(cpu_notify(val, v));
> +	BUG_ON(cpu_notify(val, cpu));
>  }
>  EXPORT_SYMBOL(register_cpu_notifier);
>  EXPORT_SYMBOL(__register_cpu_notifier);
> @@ -311,27 +316,21 @@ static inline void check_for_tasks(int dead_cpu)
>  	read_unlock(&tasklist_lock);
>  }
>  
> -struct take_cpu_down_param {
> -	unsigned long mod;
> -	void *hcpu;
> -};
> -
>  /* Take this CPU down. */
>  static int take_cpu_down(void *_param)
>  {
> -	struct take_cpu_down_param *param = _param;
> -	int err;
> +	int err, cpu = smp_processor_id();
>  
>  	/* Ensure this CPU doesn't handle any more interrupts. */
>  	err = __cpu_disable();
>  	if (err < 0)
>  		return err;
>  
> -	cpu_notify(CPU_DYING | param->mod, param->hcpu);
> +	cpu_notify(CPU_DYING, cpu);
>  	/* Give up timekeeping duties */
>  	tick_handover_do_timer();
>  	/* Park the stopper thread */
> -	stop_machine_park((long)param->hcpu);
> +	stop_machine_park(cpu);
>  	return 0;
>  }
>  
> @@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
>  static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  {
>  	int err, nr_calls = 0;
> -	void *hcpu = (void *)(long)cpu;
> -	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
> -	struct take_cpu_down_param tcd_param = {
> -		.mod = mod,
> -		.hcpu = hcpu,
> -	};
>  
>  	if (num_online_cpus() == 1)
>  		return -EBUSY;
> @@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  
>  	cpu_hotplug_begin();
>  
> -	err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
> +	cpuhp_tasks_frozen = tasks_frozen;
> +
> +	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
>  	if (err) {
>  		nr_calls--;
> -		__cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
> +		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
>  		pr_warn("%s: attempt to take down CPU %u failed\n",
>  			__func__, cpu);
>  		goto out_release;
> @@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  	/*
>  	 * So now all preempt/rcu users must observe !cpu_active().
>  	 */
> -	err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
> +	err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
>  	if (err) {
>  		/* CPU didn't die: tell everyone.  Can't complain. */
> -		cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
> +		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
>  		irq_unlock_sparse();
>  		goto out_release;
>  	}
> @@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  
>  	/* CPU is completely dead: tell everyone.  Too late to complain. */
>  	tick_cleanup_dead_cpu(cpu);
> -	cpu_notify_nofail(CPU_DEAD | mod, hcpu);
> +	cpu_notify_nofail(CPU_DEAD, cpu);
>  
>  	check_for_tasks(cpu);
>  
>  out_release:
>  	cpu_hotplug_done();
>  	if (!err)
> -		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
> +		cpu_notify_nofail(CPU_POST_DEAD, cpu);
>  	return err;
>  }
>  
> @@ -485,10 +480,8 @@ void smpboot_thread_init(void)
>  /* Requires cpu_add_remove_lock to be held */
>  static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  {
> -	int ret, nr_calls = 0;
> -	void *hcpu = (void *)(long)cpu;
> -	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
>  	struct task_struct *idle;
> +	int ret, nr_calls = 0;
>  
>  	cpu_hotplug_begin();
>  
> @@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  	if (ret)
>  		goto out;
>  
> -	ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
> +	cpuhp_tasks_frozen = tasks_frozen;
> +
> +	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
>  	if (ret) {
>  		nr_calls--;
>  		pr_warn("%s: attempt to bring up CPU %u failed\n",
> @@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  	BUG_ON(!cpu_online(cpu));
>  
>  	/* Now call notifier in preparation. */
> -	cpu_notify(CPU_ONLINE | mod, hcpu);
> +	cpu_notify(CPU_ONLINE, cpu);
>  
>  out_notify:
>  	if (ret != 0)
> -		__cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
> +		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
>  out:
>  	cpu_hotplug_done();
>  
> @@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
>   */
>  void notify_cpu_starting(unsigned int cpu)
>  {
> -	unsigned long val = CPU_STARTING;
> -
> -#ifdef CONFIG_PM_SLEEP_SMP
> -	if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
> -		val = CPU_STARTING_FROZEN;
> -#endif /* CONFIG_PM_SLEEP_SMP */
> -	cpu_notify(val, (void *)(long)cpu);
> +	cpu_notify(CPU_STARTING, cpu);
>  }
>  
>  #endif /* CONFIG_SMP */
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Restructure cpu_up code
  2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 22:36     ` Srivatsa S. Bhat
  0 siblings, 0 replies; 69+ messages in thread
From: Srivatsa S. Bhat @ 2016-03-02 22:36 UTC (permalink / raw)
  To: tj, rafael.j.wysocki, pjt, bigeasy, arjan, rostedt, riel,
	paulmck, tglx, linux-kernel, mingo, akpm, srivatsa, peterz,
	torvalds, oleg, hpa, rusty, linux-tip-commits

On 3/1/16 2:52 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  ba997462435f48ad1501320e9da8770fd40c59b1
> Gitweb:     http://git.kernel.org/tip/ba997462435f48ad1501320e9da8770fd40c59b1
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:24 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
> 
> cpu/hotplug: Restructure cpu_up code
> 
> Split out into separate functions, so we can convert it to a state machine.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.429389195@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---

Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>

Regards,
Srivatsa S. Bhat

>  kernel/cpu.c | 69 +++++++++++++++++++++++++++++++++++++++++-------------------
>  1 file changed, 47 insertions(+), 22 deletions(-)
> 
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 41a6cb8..15a4136 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val, unsigned int cpu)
>  	return __cpu_notify(val, cpu, -1, NULL);
>  }
>  
> +/* Notifier wrappers for transitioning to state machine */
> +static int notify_prepare(unsigned int cpu)
> +{
> +	int nr_calls = 0;
> +	int ret;
> +
> +	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
> +	if (ret) {
> +		nr_calls--;
> +		printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
> +				__func__, cpu);
> +		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
> +	}
> +	return ret;
> +}
> +
> +static int notify_online(unsigned int cpu)
> +{
> +	cpu_notify(CPU_ONLINE, cpu);
> +	return 0;
> +}
> +
> +static int bringup_cpu(unsigned int cpu)
> +{
> +	struct task_struct *idle = idle_thread_get(cpu);
> +	int ret;
> +
> +	/* Arch-specific enabling code. */
> +	ret = __cpu_up(cpu, idle);
> +	if (ret) {
> +		cpu_notify(CPU_UP_CANCELED, cpu);
> +		return ret;
> +	}
> +	BUG_ON(!cpu_online(cpu));
> +	return 0;
> +}
> +
>  #ifdef CONFIG_HOTPLUG_CPU
>  
>  static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> @@ -481,7 +518,7 @@ void smpboot_thread_init(void)
>  static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  {
>  	struct task_struct *idle;
> -	int ret, nr_calls = 0;
> +	int ret;
>  
>  	cpu_hotplug_begin();
>  
> @@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int tasks_frozen)
>  		goto out;
>  	}
>  
> +	cpuhp_tasks_frozen = tasks_frozen;
> +
>  	ret = smpboot_create_threads(cpu);
>  	if (ret)
>  		goto out;
>  
> -	cpuhp_tasks_frozen = tasks_frozen;
> -
> -	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
> -	if (ret) {
> -		nr_calls--;
> -		pr_warn("%s: attempt to bring up CPU %u failed\n",
> -			__func__, cpu);
> -		goto out_notify;
> -	}
> -
> -	/* Arch-specific enabling code. */
> -	ret = __cpu_up(cpu, idle);
> -
> -	if (ret != 0)
> -		goto out_notify;
> -	BUG_ON(!cpu_online(cpu));
> +	ret = notify_prepare(cpu);
> +	if (ret)
> +		goto out;
>  
> -	/* Now call notifier in preparation. */
> -	cpu_notify(CPU_ONLINE, cpu);
> +	ret = bringup_cpu(cpu);
> +	if (ret)
> +		goto out;
>  
> -out_notify:
> -	if (ret != 0)
> -		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
> +	notify_online(cpu);
>  out:
>  	cpu_hotplug_done();
>  
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Split out cpu down functions
  2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
@ 2016-03-02 22:37     ` Srivatsa S. Bhat
  0 siblings, 0 replies; 69+ messages in thread
From: Srivatsa S. Bhat @ 2016-03-02 22:37 UTC (permalink / raw)
  To: arjan, paulmck, riel, peterz, tj, rusty, akpm, pjt, torvalds,
	tglx, linux-kernel, srivatsa, rafael.j.wysocki, bigeasy, oleg,
	hpa, rostedt, mingo, linux-tip-commits

On 3/1/16 2:52 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  984581728eb4b2e10baed3d606f85a119795b207
> Gitweb:     http://git.kernel.org/tip/984581728eb4b2e10baed3d606f85a119795b207
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:25 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
> 
> cpu/hotplug: Split out cpu down functions
> 
> Split cpu_down in separate functions in preparation for state machine
> conversion.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.511796562@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---

Reviewed-by: Srivatsa S. Bhat <srivatsa@csail.mit.edu>

Regards,
Srivatsa S. Bhat

>  kernel/cpu.c | 83 ++++++++++++++++++++++++++++++++++++++----------------------
>  1 file changed, 53 insertions(+), 30 deletions(-)
> 
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 15a4136..0b5d259 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
>  }
>  
>  #ifdef CONFIG_HOTPLUG_CPU
> -
> -static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> -{
> -	BUG_ON(cpu_notify(val, cpu));
> -}
>  EXPORT_SYMBOL(register_cpu_notifier);
>  EXPORT_SYMBOL(__register_cpu_notifier);
>  
> @@ -353,6 +348,25 @@ static inline void check_for_tasks(int dead_cpu)
>  	read_unlock(&tasklist_lock);
>  }
>  
> +static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
> +{
> +	BUG_ON(cpu_notify(val, cpu));
> +}
> +
> +static int notify_down_prepare(unsigned int cpu)
> +{
> +	int err, nr_calls = 0;
> +
> +	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
> +	if (err) {
> +		nr_calls--;
> +		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
> +		pr_warn("%s: attempt to take down CPU %u failed\n",
> +				__func__, cpu);
> +	}
> +	return err;
> +}
> +
>  /* Take this CPU down. */
>  static int take_cpu_down(void *_param)
>  {
> @@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
>  	return 0;
>  }
>  
> -/* Requires cpu_add_remove_lock to be held */
> -static int _cpu_down(unsigned int cpu, int tasks_frozen)
> +static int takedown_cpu(unsigned int cpu)
>  {
> -	int err, nr_calls = 0;
> -
> -	if (num_online_cpus() == 1)
> -		return -EBUSY;
> -
> -	if (!cpu_online(cpu))
> -		return -EINVAL;
> -
> -	cpu_hotplug_begin();
> -
> -	cpuhp_tasks_frozen = tasks_frozen;
> -
> -	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
> -	if (err) {
> -		nr_calls--;
> -		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
> -		pr_warn("%s: attempt to take down CPU %u failed\n",
> -			__func__, cpu);
> -		goto out_release;
> -	}
> +	int err;
>  
>  	/*
>  	 * By now we've cleared cpu_active_mask, wait for all preempt-disabled
> @@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  		/* CPU didn't die: tell everyone.  Can't complain. */
>  		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
>  		irq_unlock_sparse();
> -		goto out_release;
> +		return err;
>  	}
>  	BUG_ON(cpu_online(cpu));
>  
> @@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, int tasks_frozen)
>  	/* This actually kills the CPU. */
>  	__cpu_die(cpu);
>  
> -	/* CPU is completely dead: tell everyone.  Too late to complain. */
>  	tick_cleanup_dead_cpu(cpu);
> -	cpu_notify_nofail(CPU_DEAD, cpu);
> +	return 0;
> +}
>  
> +static int notify_dead(unsigned int cpu)
> +{
> +	cpu_notify_nofail(CPU_DEAD, cpu);
>  	check_for_tasks(cpu);
> +	return 0;
> +}
> +
> +/* Requires cpu_add_remove_lock to be held */
> +static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
> +{
> +	int err;
> +
> +	if (num_online_cpus() == 1)
> +		return -EBUSY;
> +
> +	if (!cpu_online(cpu))
> +		return -EINVAL;
> +
> +	cpu_hotplug_begin();
> +
> +	cpuhp_tasks_frozen = tasks_frozen;
> +
> +	err = notify_down_prepare(cpu);
> +	if (err)
> +		goto out_release;
> +	err = takedown_cpu(cpu);
> +	if (err)
> +		goto out_release;
> +
> +	notify_dead(cpu);
>  
>  out_release:
>  	cpu_hotplug_done();
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Restructure FROZEN state handling
  2016-03-01 19:51   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  2016-03-02 22:35     ` Srivatsa S. Bhat
@ 2016-03-02 23:43     ` Srivatsa S. Bhat
  1 sibling, 0 replies; 69+ messages in thread
From: Srivatsa S. Bhat @ 2016-03-02 23:43 UTC (permalink / raw)
  To: tj, akpm, rusty, linux-kernel, rostedt, paulmck, peterz, mingo,
	arjan, tglx, rafael.j.wysocki, pjt, riel, bigeasy, oleg,
	torvalds, srivatsa, hpa, linux-tip-commits

On 3/1/16 2:51 PM, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  090e77c391dd983c8945b8e2e16d09f378d2e334
> Gitweb:     http://git.kernel.org/tip/090e77c391dd983c8945b8e2e16d09f378d2e334
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Fri, 26 Feb 2016 18:43:23 +0000
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Tue, 1 Mar 2016 20:36:53 +0100
> 
> cpu/hotplug: Restructure FROZEN state handling
> 
> There are only a few callbacks which really care about FROZEN
> vs. !FROZEN. No need to have extra states for this.
> 
> Publish the frozen state in an extra variable which is updated under
> the hotplug lock and let the users interested deal with it w/o
> imposing that extra state checks on everyone.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: linux-arch@vger.kernel.org
> Cc: Rik van Riel <riel@redhat.com>
> Cc: Rafael Wysocki <rafael.j.wysocki@intel.com>
> Cc: "Srivatsa S. Bhat" <srivatsa@mit.edu>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Arjan van de Ven <arjan@linux.intel.com>
> Cc: Sebastian Siewior <bigeasy@linutronix.de>
> Cc: Rusty Russell <rusty@rustcorp.com.au>
> Cc: Steven Rostedt <rostedt@goodmis.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Paul Turner <pjt@google.com>
> Link: http://lkml.kernel.org/r/20160226182340.334912357@linutronix.de
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  include/linux/cpu.h |  2 ++
>  kernel/cpu.c        | 69 ++++++++++++++++++++++-------------------------------
>  2 files changed, 31 insertions(+), 40 deletions(-)
> 
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index d2ca8c3..f2fb549 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -118,6 +118,7 @@ enum {
>  
>  
>  #ifdef CONFIG_SMP
> +extern bool cpuhp_tasks_frozen;
>  /* Need to know about CPUs going up/down? */
>  #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
>  #define cpu_notifier(fn, pri) {					\
> @@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
>  #define cpu_notifier_register_done	cpu_maps_update_done
>  
>  #else	/* CONFIG_SMP */
> +#define cpuhp_tasks_frozen	0
>  
>  #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
>  #define __cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 5b9d396..41a6cb8 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -29,6 +29,8 @@
>  #ifdef CONFIG_SMP
>  /* Serializes the updates to cpu_online_mask, cpu_present_mask */
>  static DEFINE_MUTEX(cpu_add_remove_lock);
> +bool cpuhp_tasks_frozen;
> +EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
>  

One small nitpick though: we don't need to export this symbol yet; it can
be deferred until the callbacks that need it are actually modified to use
this value (presumably in a later patchset).

Regards,
Srivatsa S. Bhat

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] rcu: Make CPU_DYING_IDLE an explicit call
  2016-03-02 20:11     ` Paul E. McKenney
@ 2016-03-03  7:31       ` Thomas Gleixner
  2016-03-03 10:03       ` [tip:smp/hotplug] cpu/hotplug: Plug death reporting race tip-bot for Thomas Gleixner
  1 sibling, 0 replies; 69+ messages in thread
From: Thomas Gleixner @ 2016-03-03  7:31 UTC (permalink / raw)
  To: Paul E. McKenney
  Cc: rafael.j.wysocki, peterz, riel, arjan, rusty, tj, srivatsa,
	torvalds, bigeasy, pjt, akpm, hpa, oleg, mingo, linux-kernel,
	rostedt, linux-tip-commits

On Wed, 2 Mar 2016, Paul E. McKenney wrote:
> > --- a/kernel/cpu.c
> > +++ b/kernel/cpu.c
> > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> >  	st->state = CPUHP_AP_IDLE_DEAD;
> >  	complete(&st->done);
> 
> Not to be repetitive or anything, but if we delay here, it can break
> RCU on a number of architectures.  Either the CPU can be killed holding
> one of RCU's locks or RCU can wrongly see the CPU as still being alive.
> Either can prevent future RCU grace periods from ever completing, thus
> OOMing the system.

Thanks for the reminder. I wanted to fix that, but then forgot again. Fix is
on the way.

Thanks,

	tglx

^ permalink raw reply	[flat|nested] 69+ messages in thread

* [tip:smp/hotplug] cpu/hotplug: Plug death reporting race
  2016-03-02 20:11     ` Paul E. McKenney
  2016-03-03  7:31       ` Thomas Gleixner
@ 2016-03-03 10:03       ` tip-bot for Thomas Gleixner
  2016-03-03 14:11         ` Paul E. McKenney
  1 sibling, 1 reply; 69+ messages in thread
From: tip-bot for Thomas Gleixner @ 2016-03-03 10:03 UTC (permalink / raw)
  To: linux-tip-commits; +Cc: hpa, linux-kernel, peterz, tglx, mingo, paulmck

Commit-ID:  71f87b2fc64c2e9b6d53cb817f28711b959d3dfe
Gitweb:     http://git.kernel.org/tip/71f87b2fc64c2e9b6d53cb817f28711b959d3dfe
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Thu, 3 Mar 2016 10:52:10 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Thu, 3 Mar 2016 10:52:10 +0100

cpu/hotplug: Plug death reporting race

Paul noticed that the conversion of the death reporting introduced a race
where the outgoing cpu might be delayed after waking the controll processor,
so it might not be able to call rcu_report_dead() before being physically
removed, leading to RCU stalls.

We cant call complete after rcu_report_dead(), so instead of going back to
busy polling, simply issue a function call to do the completion.

Fixes: 27d50c7eeb0f "rcu: Make CPU_DYING_IDLE an explicit call"
Reported-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/r/20160302201127.GA23440@linux.vnet.ibm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Peter Zijlstra <peterz@infradead.org>
---
 kernel/cpu.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/kernel/cpu.c b/kernel/cpu.c
index ff8059b..93e9d89 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -755,14 +755,26 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+static void cpuhp_complete_idle_dead(void *arg)
+{
+	struct cpuhp_cpu_state *st = arg;
+
+	complete(&st->done);
+}
+
 void cpuhp_report_idle_dead(void)
 {
 	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
 
 	BUG_ON(st->state != CPUHP_AP_OFFLINE);
-	st->state = CPUHP_AP_IDLE_DEAD;
-	complete(&st->done);
 	rcu_report_dead(smp_processor_id());
+	st->state = CPUHP_AP_IDLE_DEAD;
+	/*
+	 * We cannot call complete after rcu_report_dead() so we delegate it
+	 * to an online cpu.
+	 */
+	smp_call_function_single(cpumask_first(cpu_online_mask),
+				 cpuhp_complete_idle_dead, st, 0);
 }
 
 #else

^ permalink raw reply	[flat|nested] 69+ messages in thread

* Re: [tip:smp/hotplug] cpu/hotplug: Plug death reporting race
  2016-03-03 10:03       ` [tip:smp/hotplug] cpu/hotplug: Plug death reporting race tip-bot for Thomas Gleixner
@ 2016-03-03 14:11         ` Paul E. McKenney
  0 siblings, 0 replies; 69+ messages in thread
From: Paul E. McKenney @ 2016-03-03 14:11 UTC (permalink / raw)
  To: tip-bot for Thomas Gleixner
  Cc: linux-tip-commits, hpa, linux-kernel, peterz, tglx, mingo

On Thu, Mar 03, 2016 at 02:03:35AM -0800, tip-bot for Thomas Gleixner wrote:
> Commit-ID:  71f87b2fc64c2e9b6d53cb817f28711b959d3dfe
> Gitweb:     http://git.kernel.org/tip/71f87b2fc64c2e9b6d53cb817f28711b959d3dfe
> Author:     Thomas Gleixner <tglx@linutronix.de>
> AuthorDate: Thu, 3 Mar 2016 10:52:10 +0100
> Committer:  Thomas Gleixner <tglx@linutronix.de>
> CommitDate: Thu, 3 Mar 2016 10:52:10 +0100
> 
> cpu/hotplug: Plug death reporting race
> 
> Paul noticed that the conversion of the death reporting introduced a race
> where the outgoing cpu might be delayed after waking the controll processor,
> so it might not be able to call rcu_report_dead() before being physically
> removed, leading to RCU stalls.
> 
> We cant call complete after rcu_report_dead(), so instead of going back to
> busy polling, simply issue a function call to do the completion.
> 
> Fixes: 27d50c7eeb0f "rcu: Make CPU_DYING_IDLE an explicit call"
> Reported-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
> Link: http://lkml.kernel.org/r/20160302201127.GA23440@linux.vnet.ibm.com
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Acked-by: Peter Zijlstra <peterz@infradead.org>

There are some possible issues with printk()s from WARN_ON() calls
using RCU from within smp_call_function_single(), but in normal
execution this should be OK.  Besides, the WARN_ON() contains a
cpu_online(this_cpu), so it simply won't fire in this case.

So nice fix!!!

Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>

> ---
>  kernel/cpu.c | 16 ++++++++++++++--
>  1 file changed, 14 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index ff8059b..93e9d89 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -755,14 +755,26 @@ static int notify_dead(unsigned int cpu)
>  	return 0;
>  }
> 
> +static void cpuhp_complete_idle_dead(void *arg)
> +{
> +	struct cpuhp_cpu_state *st = arg;
> +
> +	complete(&st->done);
> +}
> +
>  void cpuhp_report_idle_dead(void)
>  {
>  	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
> 
>  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> -	st->state = CPUHP_AP_IDLE_DEAD;
> -	complete(&st->done);
>  	rcu_report_dead(smp_processor_id());
> +	st->state = CPUHP_AP_IDLE_DEAD;
> +	/*
> +	 * We cannot call complete after rcu_report_dead() so we delegate it
> +	 * to an online cpu.
> +	 */
> +	smp_call_function_single(cpumask_first(cpu_online_mask),
> +				 cpuhp_complete_idle_dead, st, 0);
>  }
> 
>  #else
> 

^ permalink raw reply	[flat|nested] 69+ messages in thread

end of thread, other threads:[~2016-03-03 14:11 UTC | newest]

Thread overview: 69+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-26 18:43 [patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework Thomas Gleixner
2016-02-26 18:43 ` [patch 01/20] idle: Move x86ism out of generic code Thomas Gleixner
2016-02-27 20:29   ` Brian Gerst
2016-02-29 19:35     ` Thomas Gleixner
2016-02-29 19:48       ` Will Deacon
2016-02-29 20:06         ` Thomas Gleixner
2016-02-26 18:43 ` [patch 02/20] cpu/hotplug: Restructure FROZEN state handling Thomas Gleixner
2016-03-01 19:51   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 22:35     ` Srivatsa S. Bhat
2016-03-02 23:43     ` Srivatsa S. Bhat
2016-02-26 18:43 ` [patch 03/20] cpu/hotplug: Restructure cpu_up code Thomas Gleixner
2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 22:36     ` Srivatsa S. Bhat
2016-02-26 18:43 ` [patch 04/20] cpu/hotplug: Split out cpu down functions Thomas Gleixner
2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 22:37     ` Srivatsa S. Bhat
2016-02-26 18:43 ` [patch 05/20] cpu/hotplug: Add tracepoints Thomas Gleixner
2016-03-01 19:52   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 06/20] cpu/hotplug: Convert to a state machine for the control processor Thomas Gleixner
2016-03-01 19:53   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 11:23     ` Peter Zijlstra
2016-02-26 18:43 ` [patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine Thomas Gleixner
2016-03-01 19:53   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down Thomas Gleixner
2016-03-01 19:54   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 09/20] cpu/hotplug: Add sysfs state interface Thomas Gleixner
2016-03-01 19:54   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 12:40     ` Peter Zijlstra
2016-02-26 18:43 ` [patch 10/20] cpu/hotplug: Make target state writeable Thomas Gleixner
2016-02-26 23:46   ` Rafael J. Wysocki
2016-02-27  7:39     ` Thomas Gleixner
2016-02-27 14:43       ` Rafael J. Wysocki
2016-02-28 14:49         ` Thomas Gleixner
2016-02-29 15:49           ` Thomas Gleixner
2016-03-01  1:53             ` Rafael J. Wysocki
2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 12:41     ` Peter Zijlstra
2016-03-02 19:57       ` Thomas Gleixner
2016-02-26 18:43 ` [patch 11/20] cpu/hotplug: Implement setup/removal interface Thomas Gleixner
2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core Thomas Gleixner
2016-03-01 19:55   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine Thomas Gleixner
2016-03-01 19:56   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 14/20] cpu/hotplug: Split out the state walk into functions Thomas Gleixner
2016-03-01 19:56   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 15/20] cpu/hotplug: Create hotplug threads Thomas Gleixner
2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu Thomas Gleixner
2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 17/20] arch/hotplug: Call into idle with a proper state Thomas Gleixner
2016-03-01 19:57   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up Thomas Gleixner
2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 17:28   ` [patch 18/20] " Richard Cochran
2016-02-26 18:43 ` [patch 19/20] cpu/hotplug: Make wait for dead cpu completion based Thomas Gleixner
2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-02-26 18:43 ` [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call Thomas Gleixner
2016-02-27  2:14   ` Paul E. McKenney
2016-02-27  2:23     ` Paul E. McKenney
2016-02-27  7:47       ` Thomas Gleixner
2016-02-27 11:05         ` Paul E. McKenney
2016-02-27 11:30           ` Thomas Gleixner
2016-02-27 16:33             ` Paul E. McKenney
2016-03-01 19:58   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-03-02 20:11     ` Paul E. McKenney
2016-03-03  7:31       ` Thomas Gleixner
2016-03-03 10:03       ` [tip:smp/hotplug] cpu/hotplug: Plug death reporting race tip-bot for Thomas Gleixner
2016-03-03 14:11         ` Paul E. McKenney

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.