[patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework

[patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework

[Thomas Gleixner]

Hi folks!

the following series contains the core infrastructure of our ongoing cpu
hotplug refactoring work. It's a follow up to the work I've done in 2013

	https://lwn.net/Articles/535764/

What's wrong with the current cpu hotplug infrastructure?

 - Asymmetry

   The hotplug notifier mechanism is asymmetric versus the bringup and
   teardown. This is mostly caused by the notifier mechanism.

 - Largely undocumented dependencies

   While some notifiers use explicitely defined notifier priorities, we have
   quite some notifiers which use numerical priorities to express
   dependencies without any documentation why.

 - Control processor driven

   Most of the bringup/teardown of a cpu is driven by a control
   processor. While it is understandable, that preperatory steps, like idle
   thread creation, memory allocation for and initialization of essential
   facilities needs to be done before a cpu can boot, there is no reason why
   everything else must run on a control processor. Todays bringup looks like
   this:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
   bring the rest up


 - All or nothing approach

   There is no way to do partial bringups. That's something which is really
   desired because we waste e.g. at boot substantial amount of time just busy
   waiting that the cpu comes to life. That's stupid as we could very well do
   preparatory steps and the initial IPI for other cpus and then go back and
   do the necessary low level synchronization with the freshly booted cpu.

 - Minimal debuggability

   Due to the notifier based design, it's impossible to switch between two
   stages of the bringup/teardown back and forth in order to test the
   correctness. So in many hotplug notifiers the cancel mechanisms are either
   not existant or completely untested.

 - Notifier [un]registering is tidious

   To [un]register notifiers we need to protect against hotplug at every
   callsite. There is no mechanism that bringup/teardown callbacks are issued
   on the online cpus, so every caller needs to do it itself. That also
   includes error rollback.


What's the new design?

   The base of the new design is a symmetric state machine, where both the
   control processor and the booting/dying cpu execute a well defined set of
   states. Each state is symmetric in the end, except for some well defined
   exceptions, and the bringup/teardown can be stopped and reversed at almost
   all states.

   So the bringup of a cpu will look like this in the future:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
				bring itself up

   The synchronization step does not require the control cpu to wait. That
   mechanism can be done asynchronously via a worker or some other mechanism.

   The teardown can be made very similar, so that the dying cpu cleans up and
   brings itself down. Cleanups which need to be done after the cpu is gone,
   can be scheduled asynchronously as well.

There is a long way to this, as we need to refactor the notion when a cpu is
available. Today we set the cpu online right after it comes out of the low
level bringup, which is not really correct.

The proper mechanism is to set it to available, i.e. cpu local threads, like
softirqd, hotplug thread etc. can be scheduled on that cpu, and once it
finished all booting steps, it's set to online, so general workloads can be
scheduled on it. The reverse happens on teardown. First thing to do is to
forbid scheduling of general workloads, then teardown all the per cpu
resources and finally shut it off completely.


The following patch series implements the basic infrastructure for this at the
core level. This includes the following:

 - Basic state machine implementation with well defined states, so ordering
   and prioritization can be expressed.

 - Interfaces to [un]register state callbacks

   This invokes the bringup/teardown callback on all online cpus with the
   proper protection in place and [un]installs the callbacks in the state
   machine array.

   For callbacks which have no particular ordering requirement we have a
   dynamic state space, so that drivers don't have to register an explicit
   hotplug state.

   If a callback fails, the code automatically does a rollback to the previous
   state.


 - Sysfs interface to drive the state machine to a particular step.

   This is only partially functional today. Full functionality and therefor
   testability will be achieved once we converted all existing hotplug
   notifiers over to the new scheme.


 - Run all CPU_ONLINE/DOWN_PREPARE notifiers on the booting/dying processor:

    Control CPU  	   	Booting CPU

    do preparatory steps
    kick cpu into life

				do low level init

    sync with booting cpu	sync with control cpu
    wait for boot     		
				bring itself up

    				Signal completion to control cpu
				
In a previous step of this work we've done a full tree mechanical conversion
of all hotplug notifiers to the new scheme. The balance is a net removal of
about 4000 lines of code.

This is not included in this post, as we decided to take a different
approach. Instead of mechanically converting everything over, we will do a
proper overhaul of the usage sites one by one so they nicely fit into the
symmetric callback scheme.

I decided to do that after I looked at the ugliness of some of the converted
sites and figured out that their hotplug mechanism is completely buggered
anyway. So there is no point to do a mechanical conversion first as we need to
go through the usage sites one by one again in order to achieve a full
symmetric and testable behaviour.

The lot is also available at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.hotplug

Thanks,

	tglx

---
 arch/ia64/kernel/smpboot.c        |    2 
 arch/metag/kernel/smp.c           |    2 
 arch/sparc/kernel/smp_32.c        |    2 
 arch/sparc/kernel/smp_64.c        |    2 
 arch/tile/kernel/smpboot.c        |    2 
 arch/x86/kernel/process.c         |   12 
 arch/x86/xen/smp.c                |    2 
 b/arch/alpha/kernel/smp.c         |    2 
 b/arch/arc/kernel/smp.c           |    2 
 b/arch/arm/kernel/smp.c           |    2 
 b/arch/arm64/kernel/smp.c         |    2 
 b/arch/blackfin/mach-common/smp.c |    2 
 b/arch/hexagon/kernel/smp.c       |    2 
 b/arch/m32r/kernel/smpboot.c      |    2 
 b/arch/mips/kernel/smp.c          |    2 
 b/arch/mn10300/kernel/smp.c       |    2 
 b/arch/parisc/kernel/smp.c        |    2 
 b/arch/powerpc/kernel/smp.c       |    2 
 b/arch/s390/kernel/smp.c          |    2 
 b/arch/sh/kernel/smp.c            |    2 
 b/arch/x86/kernel/smpboot.c       |    2 
 b/arch/xtensa/kernel/smp.c        |    2 
 b/include/linux/cpuhotplug.h      |   93 +++
 b/include/trace/events/cpuhp.h    |   66 ++
 include/linux/cpu.h               |   27 
 include/linux/notifier.h          |    2 
 include/linux/rcupdate.h          |    4 
 init/main.c                       |   16 
 kernel/cpu.c                      | 1099 +++++++++++++++++++++++++++++++++-----
 kernel/rcu/tree.c                 |   26 
 kernel/sched/core.c               |   10 
 kernel/sched/idle.c               |   24 
 kernel/smp.c                      |    1 
 kernel/smpboot.c                  |    6 
 kernel/smpboot.h                  |    6 
 lib/Kconfig.debug                 |   13 
 36 files changed, 1217 insertions(+), 230 deletions(-)







   

   

[patch 00/20] cpu/hotplug: Core infrastructure for cpu hotplug rework

[Thomas Gleixner]

Hi folks!

the following series contains the core infrastructure of our ongoing cpu
hotplug refactoring work. It's a follow up to the work I've done in 2013

	https://lwn.net/Articles/535764/

What's wrong with the current cpu hotplug infrastructure?

 - Asymmetry

   The hotplug notifier mechanism is asymmetric versus the bringup and
   teardown. This is mostly caused by the notifier mechanism.

 - Largely undocumented dependencies

   While some notifiers use explicitely defined notifier priorities, we have
   quite some notifiers which use numerical priorities to express
   dependencies without any documentation why.

 - Control processor driven

   Most of the bringup/teardown of a cpu is driven by a control
   processor. While it is understandable, that preperatory steps, like idle
   thread creation, memory allocation for and initialization of essential
   facilities needs to be done before a cpu can boot, there is no reason why
   everything else must run on a control processor. Todays bringup looks like
   this:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
   bring the rest up


 - All or nothing approach

   There is no way to do partial bringups. That's something which is really
   desired because we waste e.g. at boot substantial amount of time just busy
   waiting that the cpu comes to life. That's stupid as we could very well do
   preparatory steps and the initial IPI for other cpus and then go back and
   do the necessary low level synchronization with the freshly booted cpu.

 - Minimal debuggability

   Due to the notifier based design, it's impossible to switch between two
   stages of the bringup/teardown back and forth in order to test the
   correctness. So in many hotplug notifiers the cancel mechanisms are either
   not existant or completely untested.

 - Notifier [un]registering is tidious

   To [un]register notifiers we need to protect against hotplug at every
   callsite. There is no mechanism that bringup/teardown callbacks are issued
   on the online cpus, so every caller needs to do it itself. That also
   includes error rollback.


What's the new design?

   The base of the new design is a symmetric state machine, where both the
   control processor and the booting/dying cpu execute a well defined set of
   states. Each state is symmetric in the end, except for some well defined
   exceptions, and the bringup/teardown can be stopped and reversed at almost
   all states.

   So the bringup of a cpu will look like this in the future:

   Control CPU  	   	Booting CPU

   do preparatory steps
   kick cpu into life

				do low level init

   sync with booting cpu	sync with control cpu
   
				bring itself up

   The synchronization step does not require the control cpu to wait. That
   mechanism can be done asynchronously via a worker or some other mechanism.

   The teardown can be made very similar, so that the dying cpu cleans up and
   brings itself down. Cleanups which need to be done after the cpu is gone,
   can be scheduled asynchronously as well.

There is a long way to this, as we need to refactor the notion when a cpu is
available. Today we set the cpu online right after it comes out of the low
level bringup, which is not really correct.

The proper mechanism is to set it to available, i.e. cpu local threads, like
softirqd, hotplug thread etc. can be scheduled on that cpu, and once it
finished all booting steps, it's set to online, so general workloads can be
scheduled on it. The reverse happens on teardown. First thing to do is to
forbid scheduling of general workloads, then teardown all the per cpu
resources and finally shut it off completely.


The following patch series implements the basic infrastructure for this at the
core level. This includes the following:

 - Basic state machine implementation with well defined states, so ordering
   and prioritization can be expressed.

 - Interfaces to [un]register state callbacks

   This invokes the bringup/teardown callback on all online cpus with the
   proper protection in place and [un]installs the callbacks in the state
   machine array.

   For callbacks which have no particular ordering requirement we have a
   dynamic state space, so that drivers don't have to register an explicit
   hotplug state.

   If a callback fails, the code automatically does a rollback to the previous
   state.


 - Sysfs interface to drive the state machine to a particular step.

   This is only partially functional today. Full functionality and therefor
   testability will be achieved once we converted all existing hotplug
   notifiers over to the new scheme.


 - Run all CPU_ONLINE/DOWN_PREPARE notifiers on the booting/dying processor:

    Control CPU  	   	Booting CPU

    do preparatory steps
    kick cpu into life

				do low level init

    sync with booting cpu	sync with control cpu
    wait for boot     		
				bring itself up

    				Signal completion to control cpu
				
In a previous step of this work we've done a full tree mechanical conversion
of all hotplug notifiers to the new scheme. The balance is a net removal of
about 4000 lines of code.

This is not included in this post, as we decided to take a different
approach. Instead of mechanically converting everything over, we will do a
proper overhaul of the usage sites one by one so they nicely fit into the
symmetric callback scheme.

I decided to do that after I looked at the ugliness of some of the converted
sites and figured out that their hotplug mechanism is completely buggered
anyway. So there is no point to do a mechanical conversion first as we need to
go through the usage sites one by one again in order to achieve a full
symmetric and testable behaviour.

The lot is also available at:

  git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.hotplug

Thanks,

	tglx

---
 arch/ia64/kernel/smpboot.c        |    2 
 arch/metag/kernel/smp.c           |    2 
 arch/sparc/kernel/smp_32.c        |    2 
 arch/sparc/kernel/smp_64.c        |    2 
 arch/tile/kernel/smpboot.c        |    2 
 arch/x86/kernel/process.c         |   12 
 arch/x86/xen/smp.c                |    2 
 b/arch/alpha/kernel/smp.c         |    2 
 b/arch/arc/kernel/smp.c           |    2 
 b/arch/arm/kernel/smp.c           |    2 
 b/arch/arm64/kernel/smp.c         |    2 
 b/arch/blackfin/mach-common/smp.c |    2 
 b/arch/hexagon/kernel/smp.c       |    2 
 b/arch/m32r/kernel/smpboot.c      |    2 
 b/arch/mips/kernel/smp.c          |    2 
 b/arch/mn10300/kernel/smp.c       |    2 
 b/arch/parisc/kernel/smp.c        |    2 
 b/arch/powerpc/kernel/smp.c       |    2 
 b/arch/s390/kernel/smp.c          |    2 
 b/arch/sh/kernel/smp.c            |    2 
 b/arch/x86/kernel/smpboot.c       |    2 
 b/arch/xtensa/kernel/smp.c        |    2 
 b/include/linux/cpuhotplug.h      |   93 +++
 b/include/trace/events/cpuhp.h    |   66 ++
 include/linux/cpu.h               |   27 
 include/linux/notifier.h          |    2 
 include/linux/rcupdate.h          |    4 
 init/main.c                       |   16 
 kernel/cpu.c                      | 1099 +++++++++++++++++++++++++++++++++-----
 kernel/rcu/tree.c                 |   26 
 kernel/sched/core.c               |   10 
 kernel/sched/idle.c               |   24 
 kernel/smp.c                      |    1 
 kernel/smpboot.c                  |    6 
 kernel/smpboot.h                  |    6 
 lib/Kconfig.debug                 |   13 
 36 files changed, 1217 insertions(+), 230 deletions(-)







   

   

[patch 01/20] idle: Move x86ism out of generic code

[Thomas Gleixner]

[-- Attachment #1: idle--Move-x86ism-out-of-generic-code.patch --]
[-- Type: text/plain, Size: 1800 bytes --]

We have an arch specific callback here already.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/x86/kernel/process.c |   12 ++++++++++++
 kernel/sched/idle.c       |   15 ---------------
 2 files changed, 12 insertions(+), 15 deletions(-)

Index: b/arch/x86/kernel/process.c
===================================================================
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -271,6 +271,18 @@ void exit_idle(void)
 }
 #endif
 
+void arch_cpu_idle_prepare(void)
+{
+	/*
+	 * If we're the non-boot CPU, nothing set the stack canary up
+	 * for us. The boot CPU already has it initialized but no harm
+	 * in doing it again. This is a good place for updating it, as
+	 * we wont ever return from this function (so the invalid
+	 * canaries already on the stack wont ever trigger).
+	 */
+	boot_init_stack_canary();
+}
+
 void arch_cpu_idle_enter(void)
 {
 	local_touch_nmi();
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -275,21 +275,6 @@ static void cpu_idle_loop(void)
 
 void cpu_startup_entry(enum cpuhp_state state)
 {
-	/*
-	 * This #ifdef needs to die, but it's too late in the cycle to
-	 * make this generic (arm and sh have never invoked the canary
-	 * init for the non boot cpus!). Will be fixed in 3.11
-	 */
-#ifdef CONFIG_X86
-	/*
-	 * If we're the non-boot CPU, nothing set the stack canary up
-	 * for us. The boot CPU already has it initialized but no harm
-	 * in doing it again. This is a good place for updating it, as
-	 * we wont ever return from this function (so the invalid
-	 * canaries already on the stack wont ever trigger).
-	 */
-	boot_init_stack_canary();
-#endif
 	arch_cpu_idle_prepare();
 	cpu_idle_loop();
 }

[patch 02/20] cpu/hotplug: Restructure FROZEN state handling

[Thomas Gleixner]

[-- Attachment #1: cpu-Restructure-FROZEN-state-handling.patch --]
[-- Type: text/plain, Size: 6488 bytes --]

There are only a few callbacks which really care about FROZEN
vs. !FROZEN. No need to have extra states for this.

Publish the frozen state in an extra variable which is updated under
the hotplug lock and let the users interested deal with it w/o
imposing that extra state checks on everyone.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h |    2 +
 kernel/cpu.c        |   69 +++++++++++++++++++++-------------------------------
 2 files changed, 31 insertions(+), 40 deletions(-)

--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -118,6 +118,7 @@ enum {
 
 
 #ifdef CONFIG_SMP
+extern bool cpuhp_tasks_frozen;
 /* Need to know about CPUs going up/down? */
 #if defined(CONFIG_HOTPLUG_CPU) || !defined(MODULE)
 #define cpu_notifier(fn, pri) {					\
@@ -177,6 +178,7 @@ extern void cpu_maps_update_done(void);
 #define cpu_notifier_register_done	cpu_maps_update_done
 
 #else	/* CONFIG_SMP */
+#define cpuhp_tasks_frozen	0
 
 #define cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
 #define __cpu_notifier(fn, pri)	do { (void)(fn); } while (0)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -29,6 +29,8 @@
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
+bool cpuhp_tasks_frozen;
+EXPORT_SYMBOL_GPL(cpuhp_tasks_frozen);
 
 /*
  * The following two APIs (cpu_maps_update_begin/done) must be used when
@@ -207,27 +209,30 @@ int __register_cpu_notifier(struct notif
 	return raw_notifier_chain_register(&cpu_chain, nb);
 }
 
-static int __cpu_notify(unsigned long val, void *v, int nr_to_call,
+static int __cpu_notify(unsigned long val, unsigned int cpu, int nr_to_call,
 			int *nr_calls)
 {
+	unsigned long mod = cpuhp_tasks_frozen ? CPU_TASKS_FROZEN : 0;
+	void *hcpu = (void *)(long)cpu;
+
 	int ret;
 
-	ret = __raw_notifier_call_chain(&cpu_chain, val, v, nr_to_call,
+	ret = __raw_notifier_call_chain(&cpu_chain, val | mod, hcpu, nr_to_call,
 					nr_calls);
 
 	return notifier_to_errno(ret);
 }
 
-static int cpu_notify(unsigned long val, void *v)
+static int cpu_notify(unsigned long val, unsigned int cpu)
 {
-	return __cpu_notify(val, v, -1, NULL);
+	return __cpu_notify(val, cpu, -1, NULL);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
 
-static void cpu_notify_nofail(unsigned long val, void *v)
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
 {
-	BUG_ON(cpu_notify(val, v));
+	BUG_ON(cpu_notify(val, cpu));
 }
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -311,27 +316,21 @@ static inline void check_for_tasks(int d
 	read_unlock(&tasklist_lock);
 }
 
-struct take_cpu_down_param {
-	unsigned long mod;
-	void *hcpu;
-};
-
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
-	struct take_cpu_down_param *param = _param;
-	int err;
+	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
 	err = __cpu_disable();
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING | param->mod, param->hcpu);
+	cpu_notify(CPU_DYING, cpu);
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
-	stop_machine_park((long)param->hcpu);
+	stop_machine_park(cpu);
 	return 0;
 }
 
@@ -339,12 +338,6 @@ static int take_cpu_down(void *_param)
 static int _cpu_down(unsigned int cpu, int tasks_frozen)
 {
 	int err, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
-	struct take_cpu_down_param tcd_param = {
-		.mod = mod,
-		.hcpu = hcpu,
-	};
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -354,10 +347,12 @@ static int _cpu_down(unsigned int cpu, i
 
 	cpu_hotplug_begin();
 
-	err = __cpu_notify(CPU_DOWN_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
 	if (err) {
 		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
 		pr_warn("%s: attempt to take down CPU %u failed\n",
 			__func__, cpu);
 		goto out_release;
@@ -389,10 +384,10 @@ static int _cpu_down(unsigned int cpu, i
 	/*
 	 * So now all preempt/rcu users must observe !cpu_active().
 	 */
-	err = stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu));
+	err = stop_machine(take_cpu_down, NULL, cpumask_of(cpu));
 	if (err) {
 		/* CPU didn't die: tell everyone.  Can't complain. */
-		cpu_notify_nofail(CPU_DOWN_FAILED | mod, hcpu);
+		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
 		goto out_release;
 	}
@@ -419,14 +414,14 @@ static int _cpu_down(unsigned int cpu, i
 
 	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD | mod, hcpu);
+	cpu_notify_nofail(CPU_DEAD, cpu);
 
 	check_for_tasks(cpu);
 
 out_release:
 	cpu_hotplug_done();
 	if (!err)
-		cpu_notify_nofail(CPU_POST_DEAD | mod, hcpu);
+		cpu_notify_nofail(CPU_POST_DEAD, cpu);
 	return err;
 }
 
@@ -485,10 +480,8 @@ void smpboot_thread_init(void)
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
-	int ret, nr_calls = 0;
-	void *hcpu = (void *)(long)cpu;
-	unsigned long mod = tasks_frozen ? CPU_TASKS_FROZEN : 0;
 	struct task_struct *idle;
+	int ret, nr_calls = 0;
 
 	cpu_hotplug_begin();
 
@@ -507,7 +500,9 @@ static int _cpu_up(unsigned int cpu, int
 	if (ret)
 		goto out;
 
-	ret = __cpu_notify(CPU_UP_PREPARE | mod, hcpu, -1, &nr_calls);
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
 	if (ret) {
 		nr_calls--;
 		pr_warn("%s: attempt to bring up CPU %u failed\n",
@@ -523,11 +518,11 @@ static int _cpu_up(unsigned int cpu, int
 	BUG_ON(!cpu_online(cpu));
 
 	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE | mod, hcpu);
+	cpu_notify(CPU_ONLINE, cpu);
 
 out_notify:
 	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED | mod, hcpu, nr_calls, NULL);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
 out:
 	cpu_hotplug_done();
 
@@ -719,13 +714,7 @@ core_initcall(cpu_hotplug_pm_sync_init);
  */
 void notify_cpu_starting(unsigned int cpu)
 {
-	unsigned long val = CPU_STARTING;
-
-#ifdef CONFIG_PM_SLEEP_SMP
-	if (frozen_cpus != NULL && cpumask_test_cpu(cpu, frozen_cpus))
-		val = CPU_STARTING_FROZEN;
-#endif /* CONFIG_PM_SLEEP_SMP */
-	cpu_notify(val, (void *)(long)cpu);
+	cpu_notify(CPU_STARTING, cpu);
 }
 
 #endif /* CONFIG_SMP */

[patch 03/20] cpu/hotplug: Restructure cpu_up code

[Thomas Gleixner]

[-- Attachment #1: cpu-Restructure-cpu_down-code.patch --]
[-- Type: text/plain, Size: 2435 bytes --]

Split out into separate functions, so we can convert it to a state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   69 ++++++++++++++++++++++++++++++++++++++++-------------------
 1 file changed, 47 insertions(+), 22 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -228,6 +228,43 @@ static int cpu_notify(unsigned long val,
 	return __cpu_notify(val, cpu, -1, NULL);
 }
 
+/* Notifier wrappers for transitioning to state machine */
+static int notify_prepare(unsigned int cpu)
+{
+	int nr_calls = 0;
+	int ret;
+
+	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
+	if (ret) {
+		nr_calls--;
+		printk(KERN_WARNING "%s: attempt to bring up CPU %u failed\n",
+				__func__, cpu);
+		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	}
+	return ret;
+}
+
+static int notify_online(unsigned int cpu)
+{
+	cpu_notify(CPU_ONLINE, cpu);
+	return 0;
+}
+
+static int bringup_cpu(unsigned int cpu)
+{
+	struct task_struct *idle = idle_thread_get(cpu);
+	int ret;
+
+	/* Arch-specific enabling code. */
+	ret = __cpu_up(cpu, idle);
+	if (ret) {
+		cpu_notify(CPU_UP_CANCELED, cpu);
+		return ret;
+	}
+	BUG_ON(!cpu_online(cpu));
+	return 0;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 
 static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
@@ -481,7 +518,7 @@ void smpboot_thread_init(void)
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
 	struct task_struct *idle;
-	int ret, nr_calls = 0;
+	int ret;
 
 	cpu_hotplug_begin();
 
@@ -496,33 +533,21 @@ static int _cpu_up(unsigned int cpu, int
 		goto out;
 	}
 
+	cpuhp_tasks_frozen = tasks_frozen;
+
 	ret = smpboot_create_threads(cpu);
 	if (ret)
 		goto out;
 
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	ret = __cpu_notify(CPU_UP_PREPARE, cpu, -1, &nr_calls);
-	if (ret) {
-		nr_calls--;
-		pr_warn("%s: attempt to bring up CPU %u failed\n",
-			__func__, cpu);
-		goto out_notify;
-	}
-
-	/* Arch-specific enabling code. */
-	ret = __cpu_up(cpu, idle);
-
-	if (ret != 0)
-		goto out_notify;
-	BUG_ON(!cpu_online(cpu));
+	ret = notify_prepare(cpu);
+	if (ret)
+		goto out;
 
-	/* Now call notifier in preparation. */
-	cpu_notify(CPU_ONLINE, cpu);
+	ret = bringup_cpu(cpu);
+	if (ret)
+		goto out;
 
-out_notify:
-	if (ret != 0)
-		__cpu_notify(CPU_UP_CANCELED, cpu, nr_calls, NULL);
+	notify_online(cpu);
 out:
 	cpu_hotplug_done();
 

[patch 04/20] cpu/hotplug: Split out cpu down functions

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Split-out-cpu-down-functions.patch --]
[-- Type: text/plain, Size: 3061 bytes --]

Split cpu_down in separate functions in preparation for state machine
conversion.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   83 +++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 53 insertions(+), 30 deletions(-)

--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
-	BUG_ON(cpu_notify(val, cpu));
-}
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
 
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int d
 	read_unlock(&tasklist_lock);
 }
 
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+	BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+	int err, nr_calls = 0;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+	if (err) {
+		nr_calls--;
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+		pr_warn("%s: attempt to take down CPU %u failed\n",
+				__func__, cpu);
+	}
+	return err;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
 	return 0;
 }
 
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
 {
-	int err, nr_calls = 0;
-
-	if (num_online_cpus() == 1)
-		return -EBUSY;
-
-	if (!cpu_online(cpu))
-		return -EINVAL;
-
-	cpu_hotplug_begin();
-
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
-	if (err) {
-		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
-		pr_warn("%s: attempt to take down CPU %u failed\n",
-			__func__, cpu);
-		goto out_release;
-	}
+	int err;
 
 	/*
 	 * By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, i
 		/* CPU didn't die: tell everyone.  Can't complain. */
 		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
-		goto out_release;
+		return err;
 	}
 	BUG_ON(cpu_online(cpu));
 
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, i
 	/* This actually kills the CPU. */
 	__cpu_die(cpu);
 
-	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD, cpu);
+	return 0;
+}
 
+static int notify_dead(unsigned int cpu)
+{
+	cpu_notify_nofail(CPU_DEAD, cpu);
 	check_for_tasks(cpu);
+	return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+	int err;
+
+	if (num_online_cpus() == 1)
+		return -EBUSY;
+
+	if (!cpu_online(cpu))
+		return -EINVAL;
+
+	cpu_hotplug_begin();
+
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = notify_down_prepare(cpu);
+	if (err)
+		goto out_release;
+	err = takedown_cpu(cpu);
+	if (err)
+		goto out_release;
+
+	notify_dead(cpu);
 
 out_release:
 	cpu_hotplug_done();

[patch 04/20] cpu/hotplug: Split out cpu down functions

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Split-out-cpu-down-functions.patch --]
[-- Type: text/plain, Size: 3063 bytes --]

Split cpu_down in separate functions in preparation for state machine
conversion.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   83 +++++++++++++++++++++++++++++++++++++----------------------
 1 file changed, 53 insertions(+), 30 deletions(-)

--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -266,11 +266,6 @@ static int bringup_cpu(unsigned int cpu)
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
-
-static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
-{
-	BUG_ON(cpu_notify(val, cpu));
-}
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
 
@@ -353,6 +348,25 @@ static inline void check_for_tasks(int d
 	read_unlock(&tasklist_lock);
 }
 
+static void cpu_notify_nofail(unsigned long val, unsigned int cpu)
+{
+	BUG_ON(cpu_notify(val, cpu));
+}
+
+static int notify_down_prepare(unsigned int cpu)
+{
+	int err, nr_calls = 0;
+
+	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
+	if (err) {
+		nr_calls--;
+		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
+		pr_warn("%s: attempt to take down CPU %u failed\n",
+				__func__, cpu);
+	}
+	return err;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
@@ -371,29 +385,9 @@ static int take_cpu_down(void *_param)
 	return 0;
 }
 
-/* Requires cpu_add_remove_lock to be held */
-static int _cpu_down(unsigned int cpu, int tasks_frozen)
+static int takedown_cpu(unsigned int cpu)
 {
-	int err, nr_calls = 0;
-
-	if (num_online_cpus() == 1)
-		return -EBUSY;
-
-	if (!cpu_online(cpu))
-		return -EINVAL;
-
-	cpu_hotplug_begin();
-
-	cpuhp_tasks_frozen = tasks_frozen;
-
-	err = __cpu_notify(CPU_DOWN_PREPARE, cpu, -1, &nr_calls);
-	if (err) {
-		nr_calls--;
-		__cpu_notify(CPU_DOWN_FAILED, cpu, nr_calls, NULL);
-		pr_warn("%s: attempt to take down CPU %u failed\n",
-			__func__, cpu);
-		goto out_release;
-	}
+	int err;
 
 	/*
 	 * By now we've cleared cpu_active_mask, wait for all preempt-disabled
@@ -426,7 +420,7 @@ static int _cpu_down(unsigned int cpu, i
 		/* CPU didn't die: tell everyone.  Can't complain. */
 		cpu_notify_nofail(CPU_DOWN_FAILED, cpu);
 		irq_unlock_sparse();
-		goto out_release;
+		return err;
 	}
 	BUG_ON(cpu_online(cpu));
 
@@ -449,11 +443,40 @@ static int _cpu_down(unsigned int cpu, i
 	/* This actually kills the CPU. */
 	__cpu_die(cpu);
 
-	/* CPU is completely dead: tell everyone.  Too late to complain. */
 	tick_cleanup_dead_cpu(cpu);
-	cpu_notify_nofail(CPU_DEAD, cpu);
+	return 0;
+}
 
+static int notify_dead(unsigned int cpu)
+{
+	cpu_notify_nofail(CPU_DEAD, cpu);
 	check_for_tasks(cpu);
+	return 0;
+}
+
+/* Requires cpu_add_remove_lock to be held */
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+{
+	int err;
+
+	if (num_online_cpus() == 1)
+		return -EBUSY;
+
+	if (!cpu_online(cpu))
+		return -EINVAL;
+
+	cpu_hotplug_begin();
+
+	cpuhp_tasks_frozen = tasks_frozen;
+
+	err = notify_down_prepare(cpu);
+	if (err)
+		goto out_release;
+	err = takedown_cpu(cpu);
+	if (err)
+		goto out_release;
+
+	notify_dead(cpu);
 
 out_release:
 	cpu_hotplug_done();

[patch 05/20] cpu/hotplug: Add tracepoints

[Thomas Gleixner]

[-- Attachment #1: hotplug--Add-tracepoints.patch --]
[-- Type: text/plain, Size: 1779 bytes --]

We want to trace the hotplug machinery. Add tracepoints to track the
invocation of callbacks and their result.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/trace/events/cpuhp.h |   66 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)

Index: b/include/trace/events/cpuhp.h
===================================================================
--- /dev/null
+++ b/include/trace/events/cpuhp.h
@@ -0,0 +1,66 @@
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM cpuhp
+
+#if !defined(_TRACE_CPUHP_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_CPUHP_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(cpuhp_enter,
+
+	TP_PROTO(unsigned int cpu,
+		 int target,
+		 int idx,
+		 int (*fun)(unsigned int)),
+
+	TP_ARGS(cpu, target, idx, fun),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		target		)
+		__field( int,		idx		)
+		__field( void *,	fun		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->target	= target;
+		__entry->idx	= idx;
+		__entry->fun	= fun;
+	),
+
+	TP_printk("cpu: %04u target: %3d step: %3d (%pf)",
+		  __entry->cpu, __entry->target, __entry->idx, __entry->fun)
+);
+
+TRACE_EVENT(cpuhp_exit,
+
+	TP_PROTO(unsigned int cpu,
+		 int state,
+		 int idx,
+		 int ret),
+
+	TP_ARGS(cpu, state, idx, ret),
+
+	TP_STRUCT__entry(
+		__field( unsigned int,	cpu		)
+		__field( int,		state		)
+		__field( int,		idx		)
+		__field( int,		ret		)
+	),
+
+	TP_fast_assign(
+		__entry->cpu	= cpu;
+		__entry->state	= state;
+		__entry->idx	= idx;
+		__entry->ret	= ret;
+	),
+
+	TP_printk(" cpu: %04u  state: %3d step: %3d ret: %d",
+		  __entry->cpu, __entry->state, __entry->idx,  __entry->ret)
+);
+
+#endif
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>

[patch 06/20] cpu/hotplug: Convert to a state machine for the control processor

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Convert-to-a-state-machine-for-the-contr.patch --]
[-- Type: text/plain, Size: 9363 bytes --]

Move the split out steps into a callback array and let the cpu_up/down
code iterate through the array functions. For now most of the
callbacks are asymmetric to resemble the current hotplug maze.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    9 -
 include/linux/cpuhotplug.h |   13 ++
 init/main.c                |   15 ---
 kernel/cpu.c               |  204 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 195 insertions(+), 46 deletions(-)
 create mode 100644 include/linux/cpuhotplug.h

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -16,6 +16,7 @@
 #include <linux/node.h>
 #include <linux/compiler.h>
 #include <linux/cpumask.h>
+#include <linux/cpuhotplug.h>
 
 struct device;
 struct device_node;
@@ -27,6 +28,9 @@ struct cpu {
 	struct device dev;
 };
 
+extern void boot_cpu_init(void);
+extern void boot_cpu_state_init(void);
+
 extern int register_cpu(struct cpu *cpu, int num);
 extern struct device *get_cpu_device(unsigned cpu);
 extern bool cpu_is_hotpluggable(unsigned cpu);
@@ -267,11 +271,6 @@ static inline int disable_nonboot_cpus(v
 static inline void enable_nonboot_cpus(void) {}
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
-enum cpuhp_state {
-	CPUHP_OFFLINE,
-	CPUHP_ONLINE,
-};
-
 void cpu_startup_entry(enum cpuhp_state state);
 
 void cpu_idle_poll_ctrl(bool enable);
Index: b/include/linux/cpuhotplug.h
===================================================================
--- /dev/null
+++ b/include/linux/cpuhotplug.h
@@ -0,0 +1,13 @@
+#ifndef __CPUHOTPLUG_H
+#define __CPUHOTPLUG_H
+
+enum cpuhp_state {
+	CPUHP_OFFLINE,
+	CPUHP_CREATE_THREADS,
+	CPUHP_NOTIFY_PREPARE,
+	CPUHP_BRINGUP_CPU,
+	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE,
+};
+
+#endif
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -452,20 +452,6 @@ void __init parse_early_param(void)
 	done = 1;
 }
 
-/*
- *	Activate the first processor.
- */
-
-static void __init boot_cpu_init(void)
-{
-	int cpu = smp_processor_id();
-	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
-	set_cpu_online(cpu, true);
-	set_cpu_active(cpu, true);
-	set_cpu_present(cpu, true);
-	set_cpu_possible(cpu, true);
-}
-
 void __init __weak smp_setup_processor_id(void)
 {
 }
@@ -530,6 +516,7 @@ asmlinkage __visible void __init start_k
 	setup_command_line(command_line);
 	setup_nr_cpu_ids();
 	setup_per_cpu_areas();
+	boot_cpu_state_init();
 	smp_prepare_boot_cpu();	/* arch-specific boot-cpu hooks */
 
 	build_all_zonelists(NULL, NULL);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,10 +22,64 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+
 #include <trace/events/power.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/cpuhp.h>
 
 #include "smpboot.h"
 
+/**
+ * cpuhp_cpu_state - Per cpu hotplug state storage
+ * @state:	The current cpu state
+ * @target:	The target state
+ */
+struct cpuhp_cpu_state {
+	enum cpuhp_state	state;
+	enum cpuhp_state	target;
+};
+
+static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
+
+/**
+ * cpuhp_step - Hotplug state machine step
+ * @name:	Name of the step
+ * @startup:	Startup function of the step
+ * @teardown:	Teardown function of the step
+ * @skip_onerr:	Do not invoke the functions on error rollback
+ *		Will go away once the notifiers	are gone
+ */
+struct cpuhp_step {
+	const char	*name;
+	int		(*startup)(unsigned int cpu);
+	int		(*teardown)(unsigned int cpu);
+	bool		skip_onerr;
+};
+
+static struct cpuhp_step cpuhp_bp_states[];
+
+/**
+ * cpuhp_invoke_callback _ Invoke the callbacks for a given state
+ * @cpu:	The cpu for which the callback should be invoked
+ * @step:	The step in the state machine
+ * @cb:		The callback function to invoke
+ *
+ * Called from cpu hotplug and from the state register machinery
+ */
+static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state step,
+				 int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int ret = 0;
+
+	if (cb) {
+		trace_cpuhp_enter(cpu, st->target, step, cb);
+		ret = cb(cpu);
+		trace_cpuhp_exit(cpu, st->state, step, ret);
+	}
+	return ret;
+}
+
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
@@ -454,10 +508,29 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+#else
+#define notify_down_prepare	NULL
+#define takedown_cpu		NULL
+#define notify_dead		NULL
+#endif
+
+#ifdef CONFIG_HOTPLUG_CPU
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 {
-	int err;
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int prev_state, ret = 0;
+	bool hasdied = false;
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -469,20 +542,25 @@ static int __ref _cpu_down(unsigned int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	err = notify_down_prepare(cpu);
-	if (err)
-		goto out_release;
-	err = takedown_cpu(cpu);
-	if (err)
-		goto out_release;
-
-	notify_dead(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_OFFLINE;
+	for (; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st);
+			break;
+		}
+	}
+	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
-out_release:
 	cpu_hotplug_done();
-	if (!err)
+	/* This post dead nonsense must die */
+	if (!ret && hasdied)
 		cpu_notify_nofail(CPU_POST_DEAD, cpu);
-	return err;
+	return ret;
 }
 
 int cpu_down(unsigned int cpu)
@@ -537,11 +615,22 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int ret;
+	int prev_state, ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -550,6 +639,7 @@ static int _cpu_up(unsigned int cpu, int
 		goto out;
 	}
 
+	/* Let it fail before we try to bring the cpu up */
 	idle = idle_thread_get(cpu);
 	if (IS_ERR(idle)) {
 		ret = PTR_ERR(idle);
@@ -558,22 +648,22 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	ret = smpboot_create_threads(cpu);
-	if (ret)
-		goto out;
-
-	ret = notify_prepare(cpu);
-	if (ret)
-		goto out;
-
-	ret = bringup_cpu(cpu);
-	if (ret)
-		goto out;
-
-	notify_online(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_ONLINE;
+	while (st->state < st->target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_bp_states + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st);
+			break;
+		}
+	}
 out:
 	cpu_hotplug_done();
-
 	return ret;
 }
 
@@ -767,6 +857,44 @@ void notify_cpu_starting(unsigned int cp
 
 #endif /* CONFIG_SMP */
 
+/* Boot processor state steps */
+static struct cpuhp_step cpuhp_bp_states[] = {
+	[CPUHP_OFFLINE] = {
+		.name			= "offline",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+#ifdef CONFIG_SMP
+	[CPUHP_CREATE_THREADS]= {
+		.name			= "threads:create",
+		.startup		= smpboot_create_threads,
+		.teardown		= NULL,
+	},
+	[CPUHP_NOTIFY_PREPARE] = {
+		.name			= "notify:prepare",
+		.startup		= notify_prepare,
+		.teardown		= notify_dead,
+		.skip_onerr		= true,
+	},
+	[CPUHP_BRINGUP_CPU] = {
+		.name			= "cpu:bringup",
+		.startup		= bringup_cpu,
+		.teardown		= takedown_cpu,
+		.skip_onerr		= true,
+	},
+	[CPUHP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
+#endif
+	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.
@@ -826,3 +954,25 @@ void init_cpu_online(const struct cpumas
 {
 	cpumask_copy(&__cpu_online_mask, src);
 }
+
+/*
+ * Activate the first processor.
+ */
+void __init boot_cpu_init(void)
+{
+	int cpu = smp_processor_id();
+
+	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
+	set_cpu_online(cpu, true);
+	set_cpu_active(cpu, true);
+	set_cpu_present(cpu, true);
+	set_cpu_possible(cpu, true);
+}
+
+/*
+ * Must be called _AFTER_ setting up the per_cpu areas
+ */
+void __init boot_cpu_state_init(void)
+{
+	per_cpu_ptr(&cpuhp_state, smp_processor_id())->state = CPUHP_ONLINE;
+}

[patch 06/20] cpu/hotplug: Convert to a state machine for the control processor

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Convert-to-a-state-machine-for-the-contr.patch --]
[-- Type: text/plain, Size: 9365 bytes --]

Move the split out steps into a callback array and let the cpu_up/down
code iterate through the array functions. For now most of the
callbacks are asymmetric to resemble the current hotplug maze.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    9 -
 include/linux/cpuhotplug.h |   13 ++
 init/main.c                |   15 ---
 kernel/cpu.c               |  204 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 195 insertions(+), 46 deletions(-)
 create mode 100644 include/linux/cpuhotplug.h

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -16,6 +16,7 @@
 #include <linux/node.h>
 #include <linux/compiler.h>
 #include <linux/cpumask.h>
+#include <linux/cpuhotplug.h>
 
 struct device;
 struct device_node;
@@ -27,6 +28,9 @@ struct cpu {
 	struct device dev;
 };
 
+extern void boot_cpu_init(void);
+extern void boot_cpu_state_init(void);
+
 extern int register_cpu(struct cpu *cpu, int num);
 extern struct device *get_cpu_device(unsigned cpu);
 extern bool cpu_is_hotpluggable(unsigned cpu);
@@ -267,11 +271,6 @@ static inline int disable_nonboot_cpus(v
 static inline void enable_nonboot_cpus(void) {}
 #endif /* !CONFIG_PM_SLEEP_SMP */
 
-enum cpuhp_state {
-	CPUHP_OFFLINE,
-	CPUHP_ONLINE,
-};
-
 void cpu_startup_entry(enum cpuhp_state state);
 
 void cpu_idle_poll_ctrl(bool enable);
Index: b/include/linux/cpuhotplug.h
===================================================================
--- /dev/null
+++ b/include/linux/cpuhotplug.h
@@ -0,0 +1,13 @@
+#ifndef __CPUHOTPLUG_H
+#define __CPUHOTPLUG_H
+
+enum cpuhp_state {
+	CPUHP_OFFLINE,
+	CPUHP_CREATE_THREADS,
+	CPUHP_NOTIFY_PREPARE,
+	CPUHP_BRINGUP_CPU,
+	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE,
+};
+
+#endif
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -452,20 +452,6 @@ void __init parse_early_param(void)
 	done = 1;
 }
 
-/*
- *	Activate the first processor.
- */
-
-static void __init boot_cpu_init(void)
-{
-	int cpu = smp_processor_id();
-	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
-	set_cpu_online(cpu, true);
-	set_cpu_active(cpu, true);
-	set_cpu_present(cpu, true);
-	set_cpu_possible(cpu, true);
-}
-
 void __init __weak smp_setup_processor_id(void)
 {
 }
@@ -530,6 +516,7 @@ asmlinkage __visible void __init start_k
 	setup_command_line(command_line);
 	setup_nr_cpu_ids();
 	setup_per_cpu_areas();
+	boot_cpu_state_init();
 	smp_prepare_boot_cpu();	/* arch-specific boot-cpu hooks */
 
 	build_all_zonelists(NULL, NULL);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,10 +22,64 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+
 #include <trace/events/power.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/cpuhp.h>
 
 #include "smpboot.h"
 
+/**
+ * cpuhp_cpu_state - Per cpu hotplug state storage
+ * @state:	The current cpu state
+ * @target:	The target state
+ */
+struct cpuhp_cpu_state {
+	enum cpuhp_state	state;
+	enum cpuhp_state	target;
+};
+
+static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
+
+/**
+ * cpuhp_step - Hotplug state machine step
+ * @name:	Name of the step
+ * @startup:	Startup function of the step
+ * @teardown:	Teardown function of the step
+ * @skip_onerr:	Do not invoke the functions on error rollback
+ *		Will go away once the notifiers	are gone
+ */
+struct cpuhp_step {
+	const char	*name;
+	int		(*startup)(unsigned int cpu);
+	int		(*teardown)(unsigned int cpu);
+	bool		skip_onerr;
+};
+
+static struct cpuhp_step cpuhp_bp_states[];
+
+/**
+ * cpuhp_invoke_callback _ Invoke the callbacks for a given state
+ * @cpu:	The cpu for which the callback should be invoked
+ * @step:	The step in the state machine
+ * @cb:		The callback function to invoke
+ *
+ * Called from cpu hotplug and from the state register machinery
+ */
+static int cpuhp_invoke_callback(unsigned int cpu, enum cpuhp_state step,
+				 int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int ret = 0;
+
+	if (cb) {
+		trace_cpuhp_enter(cpu, st->target, step, cb);
+		ret = cb(cpu);
+		trace_cpuhp_exit(cpu, st->state, step, ret);
+	}
+	return ret;
+}
+
 #ifdef CONFIG_SMP
 /* Serializes the updates to cpu_online_mask, cpu_present_mask */
 static DEFINE_MUTEX(cpu_add_remove_lock);
@@ -454,10 +508,29 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+#else
+#define notify_down_prepare	NULL
+#define takedown_cpu		NULL
+#define notify_dead		NULL
+#endif
+
+#ifdef CONFIG_HOTPLUG_CPU
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
 {
-	int err;
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	int prev_state, ret = 0;
+	bool hasdied = false;
 
 	if (num_online_cpus() == 1)
 		return -EBUSY;
@@ -469,20 +542,25 @@ static int __ref _cpu_down(unsigned int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	err = notify_down_prepare(cpu);
-	if (err)
-		goto out_release;
-	err = takedown_cpu(cpu);
-	if (err)
-		goto out_release;
-
-	notify_dead(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_OFFLINE;
+	for (; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st);
+			break;
+		}
+	}
+	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
-out_release:
 	cpu_hotplug_done();
-	if (!err)
+	/* This post dead nonsense must die */
+	if (!ret && hasdied)
 		cpu_notify_nofail(CPU_POST_DEAD, cpu);
-	return err;
+	return ret;
 }
 
 int cpu_down(unsigned int cpu)
@@ -537,11 +615,22 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int ret;
+	int prev_state, ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -550,6 +639,7 @@ static int _cpu_up(unsigned int cpu, int
 		goto out;
 	}
 
+	/* Let it fail before we try to bring the cpu up */
 	idle = idle_thread_get(cpu);
 	if (IS_ERR(idle)) {
 		ret = PTR_ERR(idle);
@@ -558,22 +648,22 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	ret = smpboot_create_threads(cpu);
-	if (ret)
-		goto out;
-
-	ret = notify_prepare(cpu);
-	if (ret)
-		goto out;
-
-	ret = bringup_cpu(cpu);
-	if (ret)
-		goto out;
-
-	notify_online(cpu);
+	prev_state = st->state;
+	st->target = CPUHP_ONLINE;
+	while (st->state < st->target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_bp_states + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st);
+			break;
+		}
+	}
 out:
 	cpu_hotplug_done();
-
 	return ret;
 }
 
@@ -767,6 +857,44 @@ void notify_cpu_starting(unsigned int cp
 
 #endif /* CONFIG_SMP */
 
+/* Boot processor state steps */
+static struct cpuhp_step cpuhp_bp_states[] = {
+	[CPUHP_OFFLINE] = {
+		.name			= "offline",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+#ifdef CONFIG_SMP
+	[CPUHP_CREATE_THREADS]= {
+		.name			= "threads:create",
+		.startup		= smpboot_create_threads,
+		.teardown		= NULL,
+	},
+	[CPUHP_NOTIFY_PREPARE] = {
+		.name			= "notify:prepare",
+		.startup		= notify_prepare,
+		.teardown		= notify_dead,
+		.skip_onerr		= true,
+	},
+	[CPUHP_BRINGUP_CPU] = {
+		.name			= "cpu:bringup",
+		.startup		= bringup_cpu,
+		.teardown		= takedown_cpu,
+		.skip_onerr		= true,
+	},
+	[CPUHP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
+#endif
+	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.
@@ -826,3 +954,25 @@ void init_cpu_online(const struct cpumas
 {
 	cpumask_copy(&__cpu_online_mask, src);
 }
+
+/*
+ * Activate the first processor.
+ */
+void __init boot_cpu_init(void)
+{
+	int cpu = smp_processor_id();
+
+	/* Mark the boot cpu "present", "online" etc for SMP and UP case */
+	set_cpu_online(cpu, true);
+	set_cpu_active(cpu, true);
+	set_cpu_present(cpu, true);
+	set_cpu_possible(cpu, true);
+}
+
+/*
+ * Must be called _AFTER_ setting up the per_cpu areas
+ */
+void __init boot_cpu_state_init(void)
+{
+	per_cpu_ptr(&cpuhp_state, smp_processor_id())->state = CPUHP_ONLINE;
+}

[patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Convert-the-hotplugged-processor-work-to.patch --]
[-- Type: text/plain, Size: 5104 bytes --]

Move the functions which need to run on the hotplugged processor into
a state machine array and let the code iterate through these functions.

In a later state, this will grow synchronization points between the
control processor and the hotplugged processor, so we can move the
various architecture implementations of the synchronizations to the
core.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    4 ++
 kernel/cpu.c               |   81 ++++++++++++++++++++++++++++++++++++---------
 2 files changed, 70 insertions(+), 15 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,10 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_OFFLINE,
+	CPUHP_AP_NOTIFY_STARTING,
+	CPUHP_AP_ONLINE,
+	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE,
 };
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -57,6 +57,7 @@ struct cpuhp_step {
 };
 
 static struct cpuhp_step cpuhp_bp_states[];
+static struct cpuhp_step cpuhp_ap_states[];
 
 /**
  * cpuhp_invoke_callback _ Invoke the callbacks for a given state
@@ -304,6 +305,12 @@ static int notify_online(unsigned int cp
 	return 0;
 }
 
+static int notify_starting(unsigned int cpu)
+{
+	cpu_notify(CPU_STARTING, cpu);
+	return 0;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -421,9 +428,17 @@ static int notify_down_prepare(unsigned
 	return err;
 }
 
+static int notify_dying(unsigned int cpu)
+{
+	cpu_notify(CPU_DYING, cpu);
+	return 0;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_OFFLINE);
 	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
@@ -431,7 +446,12 @@ static int take_cpu_down(void *_param)
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING, cpu);
+	/* Invoke the former CPU_DYING callbacks */
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = cpuhp_ap_states + st->state;
+
+		cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
@@ -512,6 +532,7 @@ static int notify_dead(unsigned int cpu)
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
 #define notify_dead		NULL
+#define notify_dying		NULL
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -615,6 +636,28 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+/**
+ * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
+ * @cpu: cpu that just started
+ *
+ * This function calls the cpu_chain notifiers with CPU_STARTING.
+ * It must be called by the arch code on the new cpu, before the new cpu
+ * enables interrupts and before the "boot" cpu returns from __cpu_up().
+ */
+void notify_cpu_starting(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE);
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_ap_states + st->state;
+		cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -842,19 +885,6 @@ core_initcall(cpu_hotplug_pm_sync_init);
 
 #endif /* CONFIG_PM_SLEEP_SMP */
 
-/**
- * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
- * @cpu: cpu that just started
- *
- * This function calls the cpu_chain notifiers with CPU_STARTING.
- * It must be called by the arch code on the new cpu, before the new cpu
- * enables interrupts and before the "boot" cpu returns from __cpu_up().
- */
-void notify_cpu_starting(unsigned int cpu)
-{
-	cpu_notify(CPU_STARTING, cpu);
-}
-
 #endif /* CONFIG_SMP */
 
 /* Boot processor state steps */
@@ -879,8 +909,12 @@ static struct cpuhp_step cpuhp_bp_states
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
+		.teardown		= NULL,
+	},
+	[CPUHP_TEARDOWN_CPU] = {
+		.name			= "cpu:teardown",
+		.startup		= NULL,
 		.teardown		= takedown_cpu,
-		.skip_onerr		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
@@ -889,6 +923,23 @@ static struct cpuhp_step cpuhp_bp_states
 	},
 #endif
 	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
+/* Application processor state steps */
+static struct cpuhp_step cpuhp_ap_states[] = {
+#ifdef CONFIG_SMP
+	[CPUHP_AP_NOTIFY_STARTING] = {
+		.name			= "notify:starting",
+		.startup		= notify_starting,
+		.teardown		= notify_dying,
+		.skip_onerr		= true,
+	},
+#endif
+	[CPUHP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,

[patch 07/20] cpu/hotplug: Convert the hotplugged cpu work to a state machine

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Convert-the-hotplugged-processor-work-to.patch --]
[-- Type: text/plain, Size: 5106 bytes --]

Move the functions which need to run on the hotplugged processor into
a state machine array and let the code iterate through these functions.

In a later state, this will grow synchronization points between the
control processor and the hotplugged processor, so we can move the
various architecture implementations of the synchronizations to the
core.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    4 ++
 kernel/cpu.c               |   81 ++++++++++++++++++++++++++++++++++++---------
 2 files changed, 70 insertions(+), 15 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,10 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_OFFLINE,
+	CPUHP_AP_NOTIFY_STARTING,
+	CPUHP_AP_ONLINE,
+	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE,
 };
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -57,6 +57,7 @@ struct cpuhp_step {
 };
 
 static struct cpuhp_step cpuhp_bp_states[];
+static struct cpuhp_step cpuhp_ap_states[];
 
 /**
  * cpuhp_invoke_callback _ Invoke the callbacks for a given state
@@ -304,6 +305,12 @@ static int notify_online(unsigned int cp
 	return 0;
 }
 
+static int notify_starting(unsigned int cpu)
+{
+	cpu_notify(CPU_STARTING, cpu);
+	return 0;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -421,9 +428,17 @@ static int notify_down_prepare(unsigned
 	return err;
 }
 
+static int notify_dying(unsigned int cpu)
+{
+	cpu_notify(CPU_DYING, cpu);
+	return 0;
+}
+
 /* Take this CPU down. */
 static int take_cpu_down(void *_param)
 {
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_OFFLINE);
 	int err, cpu = smp_processor_id();
 
 	/* Ensure this CPU doesn't handle any more interrupts. */
@@ -431,7 +446,12 @@ static int take_cpu_down(void *_param)
 	if (err < 0)
 		return err;
 
-	cpu_notify(CPU_DYING, cpu);
+	/* Invoke the former CPU_DYING callbacks */
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = cpuhp_ap_states + st->state;
+
+		cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
 	/* Give up timekeeping duties */
 	tick_handover_do_timer();
 	/* Park the stopper thread */
@@ -512,6 +532,7 @@ static int notify_dead(unsigned int cpu)
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
 #define notify_dead		NULL
+#define notify_dying		NULL
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
@@ -615,6 +636,28 @@ void smpboot_thread_init(void)
 	register_cpu_notifier(&smpboot_thread_notifier);
 }
 
+/**
+ * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
+ * @cpu: cpu that just started
+ *
+ * This function calls the cpu_chain notifiers with CPU_STARTING.
+ * It must be called by the arch code on the new cpu, before the new cpu
+ * enables interrupts and before the "boot" cpu returns from __cpu_up().
+ */
+void notify_cpu_starting(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state target = min((int)st->target, CPUHP_AP_ONLINE);
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = cpuhp_ap_states + st->state;
+		cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -842,19 +885,6 @@ core_initcall(cpu_hotplug_pm_sync_init);
 
 #endif /* CONFIG_PM_SLEEP_SMP */
 
-/**
- * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
- * @cpu: cpu that just started
- *
- * This function calls the cpu_chain notifiers with CPU_STARTING.
- * It must be called by the arch code on the new cpu, before the new cpu
- * enables interrupts and before the "boot" cpu returns from __cpu_up().
- */
-void notify_cpu_starting(unsigned int cpu)
-{
-	cpu_notify(CPU_STARTING, cpu);
-}
-
 #endif /* CONFIG_SMP */
 
 /* Boot processor state steps */
@@ -879,8 +909,12 @@ static struct cpuhp_step cpuhp_bp_states
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
+		.teardown		= NULL,
+	},
+	[CPUHP_TEARDOWN_CPU] = {
+		.name			= "cpu:teardown",
+		.startup		= NULL,
 		.teardown		= takedown_cpu,
-		.skip_onerr		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
@@ -889,6 +923,23 @@ static struct cpuhp_step cpuhp_bp_states
 	},
 #endif
 	[CPUHP_ONLINE] = {
+		.name			= "online",
+		.startup		= NULL,
+		.teardown		= NULL,
+	},
+};
+
+/* Application processor state steps */
+static struct cpuhp_step cpuhp_ap_states[] = {
+#ifdef CONFIG_SMP
+	[CPUHP_AP_NOTIFY_STARTING] = {
+		.name			= "notify:starting",
+		.startup		= notify_starting,
+		.teardown		= notify_dying,
+		.skip_onerr		= true,
+	},
+#endif
+	[CPUHP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,

[patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down

[Thomas Gleixner]

[-- Attachment #1: hotplug--Hand-in-target-state-to-_cpu_up-down.patch --]
[-- Type: text/plain, Size: 3302 bytes --]

We want to be able to bringup/teardown the cpu to a particular state. Add a
target argument to _cpu_up/down.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -547,7 +547,8 @@ static void undo_cpu_down(unsigned int c
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
+			   enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int prev_state, ret = 0;
@@ -564,7 +565,7 @@ static int __ref _cpu_down(unsigned int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_OFFLINE;
+	st->target = target;
 	for (; st->state > st->target; st->state--) {
 		struct cpuhp_step *step = cpuhp_bp_states + st->state;
 
@@ -584,7 +585,7 @@ static int __ref _cpu_down(unsigned int
 	return ret;
 }
 
-int cpu_down(unsigned int cpu)
+static int do_cpu_down(unsigned int cpu, enum cpuhp_state target)
 {
 	int err;
 
@@ -595,12 +596,16 @@ int cpu_down(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_down(cpu, 0);
+	err = _cpu_down(cpu, 0, target);
 
 out:
 	cpu_maps_update_done();
 	return err;
 }
+int cpu_down(unsigned int cpu)
+{
+	return do_cpu_down(cpu, CPUHP_OFFLINE);
+}
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
@@ -669,7 +674,7 @@ static void undo_cpu_up(unsigned int cpu
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int _cpu_up(unsigned int cpu, int tasks_frozen)
+static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
@@ -692,7 +697,7 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_ONLINE;
+	st->target = target;
 	while (st->state < st->target) {
 		struct cpuhp_step *step;
 
@@ -710,7 +715,7 @@ static int _cpu_up(unsigned int cpu, int
 	return ret;
 }
 
-int cpu_up(unsigned int cpu)
+static int do_cpu_up(unsigned int cpu, enum cpuhp_state target)
 {
 	int err = 0;
 
@@ -734,12 +739,16 @@ int cpu_up(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_up(cpu, 0);
-
+	err = _cpu_up(cpu, 0, target);
 out:
 	cpu_maps_update_done();
 	return err;
 }
+
+int cpu_up(unsigned int cpu)
+{
+	return do_cpu_up(cpu, CPUHP_ONLINE);
+}
 EXPORT_SYMBOL_GPL(cpu_up);
 
 #ifdef CONFIG_PM_SLEEP_SMP
@@ -762,7 +771,7 @@ int disable_nonboot_cpus(void)
 		if (cpu == first_cpu)
 			continue;
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, true);
-		error = _cpu_down(cpu, 1);
+		error = _cpu_down(cpu, 1, CPUHP_OFFLINE);
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, false);
 		if (!error)
 			cpumask_set_cpu(cpu, frozen_cpus);
@@ -812,7 +821,7 @@ void enable_nonboot_cpus(void)
 
 	for_each_cpu(cpu, frozen_cpus) {
 		trace_suspend_resume(TPS("CPU_ON"), cpu, true);
-		error = _cpu_up(cpu, 1);
+		error = _cpu_up(cpu, 1, CPUHP_ONLINE);
 		trace_suspend_resume(TPS("CPU_ON"), cpu, false);
 		if (!error) {
 			pr_info("CPU%d is up\n", cpu);

[patch 08/20] cpu/hotplug: Hand in target state to _cpu_up/down

[Thomas Gleixner]

[-- Attachment #1: hotplug--Hand-in-target-state-to-_cpu_up-down.patch --]
[-- Type: text/plain, Size: 3304 bytes --]

We want to be able to bringup/teardown the cpu to a particular state. Add a
target argument to _cpu_up/down.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -547,7 +547,8 @@ static void undo_cpu_down(unsigned int c
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int __ref _cpu_down(unsigned int cpu, int tasks_frozen)
+static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
+			   enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int prev_state, ret = 0;
@@ -564,7 +565,7 @@ static int __ref _cpu_down(unsigned int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_OFFLINE;
+	st->target = target;
 	for (; st->state > st->target; st->state--) {
 		struct cpuhp_step *step = cpuhp_bp_states + st->state;
 
@@ -584,7 +585,7 @@ static int __ref _cpu_down(unsigned int
 	return ret;
 }
 
-int cpu_down(unsigned int cpu)
+static int do_cpu_down(unsigned int cpu, enum cpuhp_state target)
 {
 	int err;
 
@@ -595,12 +596,16 @@ int cpu_down(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_down(cpu, 0);
+	err = _cpu_down(cpu, 0, target);
 
 out:
 	cpu_maps_update_done();
 	return err;
 }
+int cpu_down(unsigned int cpu)
+{
+	return do_cpu_down(cpu, CPUHP_OFFLINE);
+}
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
@@ -669,7 +674,7 @@ static void undo_cpu_up(unsigned int cpu
 }
 
 /* Requires cpu_add_remove_lock to be held */
-static int _cpu_up(unsigned int cpu, int tasks_frozen)
+static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
@@ -692,7 +697,7 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	prev_state = st->state;
-	st->target = CPUHP_ONLINE;
+	st->target = target;
 	while (st->state < st->target) {
 		struct cpuhp_step *step;
 
@@ -710,7 +715,7 @@ static int _cpu_up(unsigned int cpu, int
 	return ret;
 }
 
-int cpu_up(unsigned int cpu)
+static int do_cpu_up(unsigned int cpu, enum cpuhp_state target)
 {
 	int err = 0;
 
@@ -734,12 +739,16 @@ int cpu_up(unsigned int cpu)
 		goto out;
 	}
 
-	err = _cpu_up(cpu, 0);
-
+	err = _cpu_up(cpu, 0, target);
 out:
 	cpu_maps_update_done();
 	return err;
 }
+
+int cpu_up(unsigned int cpu)
+{
+	return do_cpu_up(cpu, CPUHP_ONLINE);
+}
 EXPORT_SYMBOL_GPL(cpu_up);
 
 #ifdef CONFIG_PM_SLEEP_SMP
@@ -762,7 +771,7 @@ int disable_nonboot_cpus(void)
 		if (cpu == first_cpu)
 			continue;
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, true);
-		error = _cpu_down(cpu, 1);
+		error = _cpu_down(cpu, 1, CPUHP_OFFLINE);
 		trace_suspend_resume(TPS("CPU_OFF"), cpu, false);
 		if (!error)
 			cpumask_set_cpu(cpu, frozen_cpus);
@@ -812,7 +821,7 @@ void enable_nonboot_cpus(void)
 
 	for_each_cpu(cpu, frozen_cpus) {
 		trace_suspend_resume(TPS("CPU_ON"), cpu, true);
-		error = _cpu_up(cpu, 1);
+		error = _cpu_up(cpu, 1, CPUHP_ONLINE);
 		trace_suspend_resume(TPS("CPU_ON"), cpu, false);
 		if (!error) {
 			pr_info("CPU%d is up\n", cpu);

[patch 09/20] cpu/hotplug: Add sysfs state interface

[Thomas Gleixner]

[-- Attachment #1: hotplug--Add-sysfs-state-interface.patch --]
[-- Type: text/plain, Size: 3062 bytes --]

Add a sysfs interface so we can actually see in which state the cpus are in.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |   98 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 98 insertions(+)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -56,6 +56,7 @@ struct cpuhp_step {
 	bool		skip_onerr;
 };
 
+static DEFINE_MUTEX(cpuhp_state_mutex);
 static struct cpuhp_step cpuhp_bp_states[];
 static struct cpuhp_step cpuhp_ap_states[];
 
@@ -955,6 +956,103 @@ static struct cpuhp_step cpuhp_ap_states
 	},
 };
 
+static bool cpuhp_is_ap_state(enum cpuhp_state state)
+{
+	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+}
+
+static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
+{
+	struct cpuhp_step *sp;
+
+	sp = cpuhp_is_ap_state(state) ? cpuhp_ap_states : cpuhp_bp_states;
+	return sp + state;
+}
+
+#if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
+static ssize_t show_cpuhp_state(struct device *dev,
+				struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->state);
+}
+static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
+
+static ssize_t show_cpuhp_target(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+
+	return sprintf(buf, "%d\n", st->target);
+}
+static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+
+static struct attribute *cpuhp_cpu_attrs[] = {
+	&dev_attr_state.attr,
+	&dev_attr_target.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_attr_group = {
+	.attrs = cpuhp_cpu_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static ssize_t show_cpuhp_states(struct device *dev,
+				 struct device_attribute *attr, char *buf)
+{
+	ssize_t cur, res = 0;
+	int i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = 0; i <= CPUHP_ONLINE; i++) {
+		struct cpuhp_step *sp = cpuhp_get_step(i);
+
+		if (sp->name) {
+			cur = sprintf(buf, "%3d: %s\n", i, sp->name);
+			buf += cur;
+			res += cur;
+		}
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	return res;
+}
+static DEVICE_ATTR(states, 0444, show_cpuhp_states, NULL);
+
+static struct attribute *cpuhp_cpu_root_attrs[] = {
+	&dev_attr_states.attr,
+	NULL
+};
+
+static struct attribute_group cpuhp_cpu_root_attr_group = {
+	.attrs = cpuhp_cpu_root_attrs,
+	.name = "hotplug",
+	NULL
+};
+
+static int __init cpuhp_sysfs_init(void)
+{
+	int cpu, ret;
+
+	ret = sysfs_create_group(&cpu_subsys.dev_root->kobj,
+				 &cpuhp_cpu_root_attr_group);
+	if (ret)
+		return ret;
+
+	for_each_possible_cpu(cpu) {
+		struct device *dev = get_cpu_device(cpu);
+
+		ret = sysfs_create_group(&dev->kobj, &cpuhp_cpu_attr_group);
+		if (ret)
+			return ret;
+	}
+	return 0;
+}
+device_initcall(cpuhp_sysfs_init);
+#endif
+
 /*
  * cpu_bit_bitmap[] is a special, "compressed" data structure that
  * represents all NR_CPUS bits binary values of 1<<nr.

[patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

[-- Attachment #1: hotplug--Make-target-state-writeable.patch --]
[-- Type: text/plain, Size: 5263 bytes --]

Make it possible to write a target state to the per cpu state file, so we can
switch between states.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c      |   66 +++++++++++++++++++++++++++++++++++++++++++++++-------
 lib/Kconfig.debug |   13 ++++++++++
 2 files changed, 71 insertions(+), 8 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -48,12 +48,14 @@ static DEFINE_PER_CPU(struct cpuhp_cpu_s
  * @teardown:	Teardown function of the step
  * @skip_onerr:	Do not invoke the functions on error rollback
  *		Will go away once the notifiers	are gone
+ * @cant_stop:	Bringup/teardown can't be stopped at this step
  */
 struct cpuhp_step {
 	const char	*name;
 	int		(*startup)(unsigned int cpu);
 	int		(*teardown)(unsigned int cpu);
 	bool		skip_onerr;
+	bool		cant_stop;
 };
 
 static DEFINE_MUTEX(cpuhp_state_mutex);
@@ -558,7 +560,7 @@ static int __ref _cpu_down(unsigned int
 	if (num_online_cpus() == 1)
 		return -EBUSY;
 
-	if (!cpu_online(cpu))
+	if (!cpu_present(cpu))
 		return -EINVAL;
 
 	cpu_hotplug_begin();
@@ -683,16 +685,25 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpu_hotplug_begin();
 
-	if (cpu_online(cpu) || !cpu_present(cpu)) {
+	if (!cpu_present(cpu)) {
 		ret = -EINVAL;
 		goto out;
 	}
 
-	/* Let it fail before we try to bring the cpu up */
-	idle = idle_thread_get(cpu);
-	if (IS_ERR(idle)) {
-		ret = PTR_ERR(idle);
+	/*
+	 * The caller of do_cpu_up might have raced with another
+	 * caller. Ignore it for now.
+	 */
+	if (st->state >= target)
 		goto out;
+
+	if (st->state == CPUHP_OFFLINE) {
+		/* Let it fail before we try to bring the cpu up */
+		idle = idle_thread_get(cpu);
+		if (IS_ERR(idle)) {
+			ret = PTR_ERR(idle);
+			goto out;
+		}
 	}
 
 	cpuhp_tasks_frozen = tasks_frozen;
@@ -909,27 +920,32 @@ static struct cpuhp_step cpuhp_bp_states
 		.name			= "threads:create",
 		.startup		= smpboot_create_threads,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_PREPARE] = {
 		.name			= "notify:prepare",
 		.startup		= notify_prepare,
 		.teardown		= notify_dead,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_TEARDOWN_CPU] = {
 		.name			= "cpu:teardown",
 		.startup		= NULL,
 		.teardown		= takedown_cpu,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
 		.teardown		= notify_down_prepare,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -947,6 +963,7 @@ static struct cpuhp_step cpuhp_ap_states
 		.startup		= notify_starting,
 		.teardown		= notify_dying,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -979,6 +996,39 @@ static ssize_t show_cpuhp_state(struct d
 }
 static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
 
+static ssize_t write_cpuhp_target(struct device *dev,
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+	struct cpuhp_step *sp;
+	int target, ret;
+
+	ret = kstrtoint(buf, 10, &target);
+	if (ret)
+		return ret;
+
+#ifdef CONFIG_CPU_HOTPLUG_STATE_CONTROL
+	if (target < CPUHP_OFFLINE || target > CPUHP_ONLINE)
+		return -EINVAL;
+#else
+	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
+		return -EINVAL;
+#endif
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(target);
+	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
+	mutex_unlock(&cpuhp_state_mutex);
+	if (ret)
+		return ret;
+
+	if (st->state < target)
+		ret = do_cpu_up(dev->id, target);
+	else
+		ret = do_cpu_down(dev->id, target);
+	return ret ? ret : count;
+}
+
 static ssize_t show_cpuhp_target(struct device *dev,
 				 struct device_attribute *attr, char *buf)
 {
@@ -986,7 +1036,7 @@ static ssize_t show_cpuhp_target(struct
 
 	return sprintf(buf, "%d\n", st->target);
 }
-static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+static DEVICE_ATTR(target, 0644, show_cpuhp_target, write_cpuhp_target);
 
 static struct attribute *cpuhp_cpu_attrs[] = {
 	&dev_attr_state.attr,
@@ -1007,7 +1057,7 @@ static ssize_t show_cpuhp_states(struct
 	int i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = 0; i <= CPUHP_ONLINE; i++) {
+	for (i = CPUHP_OFFLINE; i <= CPUHP_ONLINE; i++) {
 		struct cpuhp_step *sp = cpuhp_get_step(i);
 
 		if (sp->name) {
Index: b/lib/Kconfig.debug
===================================================================
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1442,6 +1442,19 @@ config DEBUG_BLOCK_EXT_DEVT
 
 	  Say N if you are unsure.
 
+config CPU_HOTPLUG_STATE_CONTROL
+	bool "Enable CPU hotplug state control"
+	depends on DEBUG_KERNEL
+	depends on HOTPLUG_CPU
+	default n
+	help
+	  Allows to write steps between "offline" and "online" to the CPUs
+	  sysfs target file so states can be stepped granular. This is a debug
+	  option for now as the hotplug machinery cannot be stopped and
+	  restarted at arbitrary points yet.
+
+	  Say N if your are unsure.
+
 config NOTIFIER_ERROR_INJECTION
 	tristate "Notifier error injection"
 	depends on DEBUG_KERNEL

[patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

[-- Attachment #1: hotplug--Make-target-state-writeable.patch --]
[-- Type: text/plain, Size: 5265 bytes --]

Make it possible to write a target state to the per cpu state file, so we can
switch between states.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c      |   66 +++++++++++++++++++++++++++++++++++++++++++++++-------
 lib/Kconfig.debug |   13 ++++++++++
 2 files changed, 71 insertions(+), 8 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -48,12 +48,14 @@ static DEFINE_PER_CPU(struct cpuhp_cpu_s
  * @teardown:	Teardown function of the step
  * @skip_onerr:	Do not invoke the functions on error rollback
  *		Will go away once the notifiers	are gone
+ * @cant_stop:	Bringup/teardown can't be stopped at this step
  */
 struct cpuhp_step {
 	const char	*name;
 	int		(*startup)(unsigned int cpu);
 	int		(*teardown)(unsigned int cpu);
 	bool		skip_onerr;
+	bool		cant_stop;
 };
 
 static DEFINE_MUTEX(cpuhp_state_mutex);
@@ -558,7 +560,7 @@ static int __ref _cpu_down(unsigned int
 	if (num_online_cpus() == 1)
 		return -EBUSY;
 
-	if (!cpu_online(cpu))
+	if (!cpu_present(cpu))
 		return -EINVAL;
 
 	cpu_hotplug_begin();
@@ -683,16 +685,25 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpu_hotplug_begin();
 
-	if (cpu_online(cpu) || !cpu_present(cpu)) {
+	if (!cpu_present(cpu)) {
 		ret = -EINVAL;
 		goto out;
 	}
 
-	/* Let it fail before we try to bring the cpu up */
-	idle = idle_thread_get(cpu);
-	if (IS_ERR(idle)) {
-		ret = PTR_ERR(idle);
+	/*
+	 * The caller of do_cpu_up might have raced with another
+	 * caller. Ignore it for now.
+	 */
+	if (st->state >= target)
 		goto out;
+
+	if (st->state == CPUHP_OFFLINE) {
+		/* Let it fail before we try to bring the cpu up */
+		idle = idle_thread_get(cpu);
+		if (IS_ERR(idle)) {
+			ret = PTR_ERR(idle);
+			goto out;
+		}
 	}
 
 	cpuhp_tasks_frozen = tasks_frozen;
@@ -909,27 +920,32 @@ static struct cpuhp_step cpuhp_bp_states
 		.name			= "threads:create",
 		.startup		= smpboot_create_threads,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_PREPARE] = {
 		.name			= "notify:prepare",
 		.startup		= notify_prepare,
 		.teardown		= notify_dead,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 	[CPUHP_BRINGUP_CPU] = {
 		.name			= "cpu:bringup",
 		.startup		= bringup_cpu,
 		.teardown		= NULL,
+		.cant_stop		= true,
 	},
 	[CPUHP_TEARDOWN_CPU] = {
 		.name			= "cpu:teardown",
 		.startup		= NULL,
 		.teardown		= takedown_cpu,
+		.cant_stop		= true,
 	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
 		.teardown		= notify_down_prepare,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -947,6 +963,7 @@ static struct cpuhp_step cpuhp_ap_states
 		.startup		= notify_starting,
 		.teardown		= notify_dying,
 		.skip_onerr		= true,
+		.cant_stop		= true,
 	},
 #endif
 	[CPUHP_ONLINE] = {
@@ -979,6 +996,39 @@ static ssize_t show_cpuhp_state(struct d
 }
 static DEVICE_ATTR(state, 0444, show_cpuhp_state, NULL);
 
+static ssize_t write_cpuhp_target(struct device *dev,
+				  struct device_attribute *attr,
+				  const char *buf, size_t count)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, dev->id);
+	struct cpuhp_step *sp;
+	int target, ret;
+
+	ret = kstrtoint(buf, 10, &target);
+	if (ret)
+		return ret;
+
+#ifdef CONFIG_CPU_HOTPLUG_STATE_CONTROL
+	if (target < CPUHP_OFFLINE || target > CPUHP_ONLINE)
+		return -EINVAL;
+#else
+	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
+		return -EINVAL;
+#endif
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(target);
+	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
+	mutex_unlock(&cpuhp_state_mutex);
+	if (ret)
+		return ret;
+
+	if (st->state < target)
+		ret = do_cpu_up(dev->id, target);
+	else
+		ret = do_cpu_down(dev->id, target);
+	return ret ? ret : count;
+}
+
 static ssize_t show_cpuhp_target(struct device *dev,
 				 struct device_attribute *attr, char *buf)
 {
@@ -986,7 +1036,7 @@ static ssize_t show_cpuhp_target(struct
 
 	return sprintf(buf, "%d\n", st->target);
 }
-static DEVICE_ATTR(target, 0444, show_cpuhp_target, NULL);
+static DEVICE_ATTR(target, 0644, show_cpuhp_target, write_cpuhp_target);
 
 static struct attribute *cpuhp_cpu_attrs[] = {
 	&dev_attr_state.attr,
@@ -1007,7 +1057,7 @@ static ssize_t show_cpuhp_states(struct
 	int i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = 0; i <= CPUHP_ONLINE; i++) {
+	for (i = CPUHP_OFFLINE; i <= CPUHP_ONLINE; i++) {
 		struct cpuhp_step *sp = cpuhp_get_step(i);
 
 		if (sp->name) {
Index: b/lib/Kconfig.debug
===================================================================
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1442,6 +1442,19 @@ config DEBUG_BLOCK_EXT_DEVT
 
 	  Say N if you are unsure.
 
+config CPU_HOTPLUG_STATE_CONTROL
+	bool "Enable CPU hotplug state control"
+	depends on DEBUG_KERNEL
+	depends on HOTPLUG_CPU
+	default n
+	help
+	  Allows to write steps between "offline" and "online" to the CPUs
+	  sysfs target file so states can be stepped granular. This is a debug
+	  option for now as the hotplug machinery cannot be stopped and
+	  restarted at arbitrary points yet.
+
+	  Say N if your are unsure.
+
 config NOTIFIER_ERROR_INJECTION
 	tristate "Notifier error injection"
 	depends on DEBUG_KERNEL

[patch 11/20] cpu/hotplug: Implement setup/removal interface

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Implement-setup-removal-interface.patch --]
[-- Type: text/plain, Size: 9949 bytes --]

Implement function which allow to setup/remove hotplug state callbacks.

The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.

The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.

This includes rollback to the previous state in case of failure.

A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them

For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.

[ Dynamic registering implemented by Sebastian Siewior ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   67 +++++++++++++
 kernel/cpu.c               |  224 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 291 insertions(+)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE_DYN,
+	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
+int __cpuhp_setup_state(enum cpuhp_state state,	const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback (will be used in debug output)
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+				    const char *name,
+				    int (*startup)(unsigned int cpu),
+				    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ *			       callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback.
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+					    const char *name,
+					    int (*startup)(unsigned int cpu),
+					    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state:	The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ *				teardown
+ * @state:	The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, false);
+}
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states
 	},
 };
 
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+	if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+		return -EINVAL;
+	return 0;
+}
+
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
 	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step
 	return sp + state;
 }
 
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+				  const char *name,
+				  int (*startup)(unsigned int cpu),
+				  int (*teardown)(unsigned int cpu))
+{
+	/* (Un)Install the callbacks for further cpu hotplug operations */
+	struct cpuhp_step *sp;
+
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(state);
+	sp->startup = startup;
+	sp->teardown = teardown;
+	sp->name = name;
+	mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+	return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+	int (*cb)(unsigned int cpu) = __cb;
+
+	BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+			    int (*cb)(unsigned int), bool bringup)
+{
+	int ret;
+
+	if (!cb)
+		return 0;
+
+	/*
+	 * This invokes the callback directly for now. In a later step we
+	 * convert that to use cpuhp_invoke_callback().
+	 */
+	if (cpuhp_is_ap_state(state)) {
+		/*
+		 * Note, that a function called on the AP is not
+		 * allowed to fail.
+		 */
+		if (cpu_online(cpu))
+			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+		return 0;
+	}
+
+	/*
+	 * The non AP bound callbacks can fail on bringup. On teardown
+	 * e.g. module removal we crash for now.
+	 */
+	ret = cb(cpu);
+	BUG_ON(ret && !bringup);
+	return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+				   int (*teardown)(unsigned int cpu))
+{
+	int cpu;
+
+	if (!teardown)
+		return;
+
+	/* Roll back the already executed steps on the other cpus */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpu >= failedcpu)
+			break;
+
+		/* Did we invoke the startup call on that cpu ? */
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+	enum cpuhp_state i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+		if (cpuhp_bp_states[i].name)
+			continue;
+
+		cpuhp_bp_states[i].name = "Reserved";
+		mutex_unlock(&cpuhp_state_mutex);
+		return i;
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	WARN(1, "No more dynamic states available for CPU hotplug\n");
+	return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state:	The state to setup
+ * @invoke:	If true, the startup function is invoked for cpus where
+ *		cpu state >= @state
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+			const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu))
+{
+	int cpu, ret = 0;
+	int dyn_state = 0;
+
+	if (cpuhp_cb_check(state) || !name)
+		return -EINVAL;
+
+	get_online_cpus();
+
+	/* currently assignments for the ONLINE state are possible */
+	if (state == CPUHP_ONLINE_DYN) {
+		dyn_state = 1;
+		ret = cpuhp_reserve_state(state);
+		if (ret < 0)
+			goto out;
+		state = ret;
+	}
+
+	cpuhp_store_callbacks(state, name, startup, teardown);
+
+	if (!invoke || !startup)
+		goto out;
+
+	/*
+	 * Try to call the startup callback for each present cpu
+	 * depending on the hotplug state of the cpu.
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate < state)
+			continue;
+
+		ret = cpuhp_issue_call(cpu, state, startup, true);
+		if (ret) {
+			cpuhp_rollback_install(cpu, state, teardown);
+			cpuhp_store_callbacks(state, NULL, NULL, NULL);
+			goto out;
+		}
+	}
+out:
+	put_online_cpus();
+	if (!ret && dyn_state)
+		return state;
+	return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state:	The state to remove
+ * @invoke:	If true, the teardown function is invoked for cpus where
+ *		cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+	int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+	int cpu;
+
+	BUG_ON(cpuhp_cb_check(state));
+
+	get_online_cpus();
+
+	if (!invoke || !teardown)
+		goto remove;
+
+	/*
+	 * Call the teardown callback for each present cpu depending
+	 * on the hotplug state of the cpu. This function is not
+	 * allowed to fail currently!
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+remove:
+	cpuhp_store_callbacks(state, NULL, NULL, NULL);
+	put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
 #if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
 static ssize_t show_cpuhp_state(struct device *dev,
 				struct device_attribute *attr, char *buf)

[patch 11/20] cpu/hotplug: Implement setup/removal interface

[Thomas Gleixner]

[-- Attachment #1: cpu-hotplug-Implement-setup-removal-interface.patch --]
[-- Type: text/plain, Size: 9951 bytes --]

Implement function which allow to setup/remove hotplug state callbacks.

The default behaviour for setup is to call the startup function for this state
for (or on) all cpus which have a hotplug state >= the installed state.

The default behaviour for removal is to call the teardown function for this
state for (or on) all cpus which have a hotplug state >= the installed state.

This includes rollback to the previous state in case of failure.

A special state is CPUHP_ONLINE_DYN. Its for dynamically registering a hotplug
callback pair. This is for drivers which have no dependencies to avoid that we
need to allocate CPUHP states for each of them

For both setup and remove helper functions are provided, which prevent the
core to issue the callbacks. This simplifies the conversion of existing
hotplug notifiers.

[ Dynamic registering implemented by Sebastian Siewior ]

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   67 +++++++++++++
 kernel/cpu.c               |  224 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 291 insertions(+)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,7 +11,74 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_NOTIFY_ONLINE,
+	CPUHP_ONLINE_DYN,
+	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
+int __cpuhp_setup_state(enum cpuhp_state state,	const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu));
+
+/**
+ * cpuhp_setup_state - Setup hotplug state callbacks with calling the callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback (will be used in debug output)
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Installs the callback functions and invokes the startup callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline int cpuhp_setup_state(enum cpuhp_state state,
+				    const char *name,
+				    int (*startup)(unsigned int cpu),
+				    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, true, startup, teardown);
+}
+
+/**
+ * cpuhp_setup_state_nocalls - Setup hotplug state callbacks without calling the
+ *			       callbacks
+ * @state:	The state for which the calls are installed
+ * @name:	Name of the callback.
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Same as @cpuhp_setup_state except that no calls are executed are invoked
+ * during installation of this callback. NOP if SMP=n or HOTPLUG_CPU=n.
+ */
+static inline int cpuhp_setup_state_nocalls(enum cpuhp_state state,
+					    const char *name,
+					    int (*startup)(unsigned int cpu),
+					    int (*teardown)(unsigned int cpu))
+{
+	return __cpuhp_setup_state(state, name, false, startup, teardown);
+}
+
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke);
+
+/**
+ * cpuhp_remove_state - Remove hotplug state callbacks and invoke the teardown
+ * @state:	The state for which the calls are removed
+ *
+ * Removes the callback functions and invokes the teardown callback on
+ * the present cpus which have already reached the @state.
+ */
+static inline void cpuhp_remove_state(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, true);
+}
+
+/**
+ * cpuhp_remove_state_nocalls - Remove hotplug state callbacks without invoking
+ *				teardown
+ * @state:	The state for which the calls are removed
+ */
+static inline void cpuhp_remove_state_nocalls(enum cpuhp_state state)
+{
+	__cpuhp_remove_state(state, false);
+}
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -973,6 +973,14 @@ static struct cpuhp_step cpuhp_ap_states
 	},
 };
 
+/* Sanity check for callbacks */
+static int cpuhp_cb_check(enum cpuhp_state state)
+{
+	if (state <= CPUHP_OFFLINE || state >= CPUHP_ONLINE)
+		return -EINVAL;
+	return 0;
+}
+
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
 	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
@@ -986,6 +994,222 @@ static struct cpuhp_step *cpuhp_get_step
 	return sp + state;
 }
 
+static void cpuhp_store_callbacks(enum cpuhp_state state,
+				  const char *name,
+				  int (*startup)(unsigned int cpu),
+				  int (*teardown)(unsigned int cpu))
+{
+	/* (Un)Install the callbacks for further cpu hotplug operations */
+	struct cpuhp_step *sp;
+
+	mutex_lock(&cpuhp_state_mutex);
+	sp = cpuhp_get_step(state);
+	sp->startup = startup;
+	sp->teardown = teardown;
+	sp->name = name;
+	mutex_unlock(&cpuhp_state_mutex);
+}
+
+static void *cpuhp_get_teardown_cb(enum cpuhp_state state)
+{
+	return cpuhp_get_step(state)->teardown;
+}
+
+/* Helper function to run callback on the target cpu */
+static void cpuhp_on_cpu_cb(void *__cb)
+{
+	int (*cb)(unsigned int cpu) = __cb;
+
+	BUG_ON(cb(smp_processor_id()));
+}
+
+/*
+ * Call the startup/teardown function for a step either on the AP or
+ * on the current CPU.
+ */
+static int cpuhp_issue_call(int cpu, enum cpuhp_state state,
+			    int (*cb)(unsigned int), bool bringup)
+{
+	int ret;
+
+	if (!cb)
+		return 0;
+
+	/*
+	 * This invokes the callback directly for now. In a later step we
+	 * convert that to use cpuhp_invoke_callback().
+	 */
+	if (cpuhp_is_ap_state(state)) {
+		/*
+		 * Note, that a function called on the AP is not
+		 * allowed to fail.
+		 */
+		if (cpu_online(cpu))
+			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
+		return 0;
+	}
+
+	/*
+	 * The non AP bound callbacks can fail on bringup. On teardown
+	 * e.g. module removal we crash for now.
+	 */
+	ret = cb(cpu);
+	BUG_ON(ret && !bringup);
+	return ret;
+}
+
+/*
+ * Called from __cpuhp_setup_state on a recoverable failure.
+ *
+ * Note: The teardown callbacks for rollback are not allowed to fail!
+ */
+static void cpuhp_rollback_install(int failedcpu, enum cpuhp_state state,
+				   int (*teardown)(unsigned int cpu))
+{
+	int cpu;
+
+	if (!teardown)
+		return;
+
+	/* Roll back the already executed steps on the other cpus */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpu >= failedcpu)
+			break;
+
+		/* Did we invoke the startup call on that cpu ? */
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+}
+
+/*
+ * Returns a free for dynamic slot assignment of the Online state. The states
+ * are protected by the cpuhp_slot_states mutex and an empty slot is identified
+ * by having no name assigned.
+ */
+static int cpuhp_reserve_state(enum cpuhp_state state)
+{
+	enum cpuhp_state i;
+
+	mutex_lock(&cpuhp_state_mutex);
+	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
+		if (cpuhp_bp_states[i].name)
+			continue;
+
+		cpuhp_bp_states[i].name = "Reserved";
+		mutex_unlock(&cpuhp_state_mutex);
+		return i;
+	}
+	mutex_unlock(&cpuhp_state_mutex);
+	WARN(1, "No more dynamic states available for CPU hotplug\n");
+	return -ENOSPC;
+}
+
+/**
+ * __cpuhp_setup_state - Setup the callbacks for an hotplug machine state
+ * @state:	The state to setup
+ * @invoke:	If true, the startup function is invoked for cpus where
+ *		cpu state >= @state
+ * @startup:	startup callback function
+ * @teardown:	teardown callback function
+ *
+ * Returns 0 if successful, otherwise a proper error code
+ */
+int __cpuhp_setup_state(enum cpuhp_state state,
+			const char *name, bool invoke,
+			int (*startup)(unsigned int cpu),
+			int (*teardown)(unsigned int cpu))
+{
+	int cpu, ret = 0;
+	int dyn_state = 0;
+
+	if (cpuhp_cb_check(state) || !name)
+		return -EINVAL;
+
+	get_online_cpus();
+
+	/* currently assignments for the ONLINE state are possible */
+	if (state == CPUHP_ONLINE_DYN) {
+		dyn_state = 1;
+		ret = cpuhp_reserve_state(state);
+		if (ret < 0)
+			goto out;
+		state = ret;
+	}
+
+	cpuhp_store_callbacks(state, name, startup, teardown);
+
+	if (!invoke || !startup)
+		goto out;
+
+	/*
+	 * Try to call the startup callback for each present cpu
+	 * depending on the hotplug state of the cpu.
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate < state)
+			continue;
+
+		ret = cpuhp_issue_call(cpu, state, startup, true);
+		if (ret) {
+			cpuhp_rollback_install(cpu, state, teardown);
+			cpuhp_store_callbacks(state, NULL, NULL, NULL);
+			goto out;
+		}
+	}
+out:
+	put_online_cpus();
+	if (!ret && dyn_state)
+		return state;
+	return ret;
+}
+EXPORT_SYMBOL(__cpuhp_setup_state);
+
+/**
+ * __cpuhp_remove_state - Remove the callbacks for an hotplug machine state
+ * @state:	The state to remove
+ * @invoke:	If true, the teardown function is invoked for cpus where
+ *		cpu state >= @state
+ *
+ * The teardown callback is currently not allowed to fail. Think
+ * about module removal!
+ */
+void __cpuhp_remove_state(enum cpuhp_state state, bool invoke)
+{
+	int (*teardown)(unsigned int cpu) = cpuhp_get_teardown_cb(state);
+	int cpu;
+
+	BUG_ON(cpuhp_cb_check(state));
+
+	get_online_cpus();
+
+	if (!invoke || !teardown)
+		goto remove;
+
+	/*
+	 * Call the teardown callback for each present cpu depending
+	 * on the hotplug state of the cpu. This function is not
+	 * allowed to fail currently!
+	 */
+	for_each_present_cpu(cpu) {
+		struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+		int cpustate = st->state;
+
+		if (cpustate >= state)
+			cpuhp_issue_call(cpu, state, teardown, false);
+	}
+remove:
+	cpuhp_store_callbacks(state, NULL, NULL, NULL);
+	put_online_cpus();
+}
+EXPORT_SYMBOL(__cpuhp_remove_state);
+
 #if defined(CONFIG_SYSFS) && defined(CONFIG_HOTPLUG_CPU)
 static ssize_t show_cpuhp_state(struct device *dev,
 				struct device_attribute *attr, char *buf)

[patch 12/20] cpu/hotplug: Move scheduler cpu_online notifier to hotplug core

[Thomas Gleixner]

[-- Attachment #1: hotplug--Move-scheduler-cpu_online-notifier-to-hotplug-core.patch --]
[-- Type: text/plain, Size: 2431 bytes --]

Move the scheduler cpu online notifier part to the hotplug core. This is
anyway the highest priority callback and we need that functionality right now
for the next changes.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    1 +
 kernel/cpu.c               |   18 ++++++++++++++++++
 kernel/sched/core.c        |   10 ----------
 3 files changed, 19 insertions(+), 10 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,6 +10,7 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
+	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -666,6 +666,19 @@ void notify_cpu_starting(unsigned int cp
 	}
 }
 
+/*
+ * Called from the idle task. We need to set active here, so we can kick off
+ * the stopper thread.
+ */
+static int cpuhp_set_cpu_active(unsigned int cpu)
+{
+	/* The cpu is marked online, set it active now */
+	set_cpu_active(cpu, true);
+	/* Unpark the stopper thread */
+	stop_machine_unpark(cpu);
+	return 0;
+}
+
 static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
 	for (st->state--; st->state > st->target; st->state--) {
@@ -941,6 +954,11 @@ static struct cpuhp_step cpuhp_bp_states
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
+	[CPUHP_CPU_SET_ACTIVE] = {
+		.name			= "cpu:active",
+		.startup		= cpuhp_set_cpu_active,
+		.teardown		= NULL,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
Index: b/kernel/sched/core.c
===================================================================
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -5692,16 +5692,6 @@ static int sched_cpu_active(struct notif
 		set_cpu_rq_start_time();
 		return NOTIFY_OK;
 
-	case CPU_ONLINE:
-		/*
-		 * At this point a starting CPU has marked itself as online via
-		 * set_cpu_online(). But it might not yet have marked itself
-		 * as active, which is essential from here on.
-		 */
-		set_cpu_active(cpu, true);
-		stop_machine_unpark(cpu);
-		return NOTIFY_OK;
-
 	case CPU_DOWN_FAILED:
 		set_cpu_active(cpu, true);
 		return NOTIFY_OK;

[patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine

[Thomas Gleixner]

[-- Attachment #1: hotplug--Handle-smpboot-threads-in-state-machine.patch --]
[-- Type: text/plain, Size: 5177 bytes --]

Handle the smpboot threads in the state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    7 +------
 include/linux/cpuhotplug.h |    1 +
 init/main.c                |    1 -
 kernel/cpu.c               |   39 +++++----------------------------------
 kernel/smpboot.c           |    6 ++++--
 kernel/smpboot.h           |    4 ++--
 6 files changed, 13 insertions(+), 45 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -78,7 +78,7 @@ enum {
 	/* migration should happen before other stuff but after perf */
 	CPU_PRI_PERF		= 20,
 	CPU_PRI_MIGRATION	= 10,
-	CPU_PRI_SMPBOOT		= 9,
+
 	/* bring up workqueues before normal notifiers and down after */
 	CPU_PRI_WORKQUEUE_UP	= 5,
 	CPU_PRI_WORKQUEUE_DOWN	= -5,
@@ -172,7 +172,6 @@ static inline void __unregister_cpu_noti
 }
 #endif
 
-void smpboot_thread_init(void);
 int cpu_up(unsigned int cpu);
 void notify_cpu_starting(unsigned int cpu);
 extern void cpu_maps_update_begin(void);
@@ -221,10 +220,6 @@ static inline void cpu_notifier_register
 {
 }
 
-static inline void smpboot_thread_init(void)
-{
-}
-
 #endif /* CONFIG_SMP */
 extern struct bus_type cpu_subsys;
 
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,6 +11,7 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
+	CPUHP_SMPBOOT_THREADS,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -388,7 +388,6 @@ static noinline void __init_refok rest_i
 	int pid;
 
 	rcu_scheduler_starting();
-	smpboot_thread_init();
 	/*
 	 * We need to spawn init first so that it obtains pid 1, however
 	 * the init task will end up wanting to create kthreads, which, if
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -481,8 +481,6 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
-	smpboot_park_threads(cpu);
-
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -612,38 +610,6 @@ int cpu_down(unsigned int cpu)
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
-/*
- * Unpark per-CPU smpboot kthreads at CPU-online time.
- */
-static int smpboot_thread_call(struct notifier_block *nfb,
-			       unsigned long action, void *hcpu)
-{
-	int cpu = (long)hcpu;
-
-	switch (action & ~CPU_TASKS_FROZEN) {
-
-	case CPU_DOWN_FAILED:
-	case CPU_ONLINE:
-		smpboot_unpark_threads(cpu);
-		break;
-
-	default:
-		break;
-	}
-
-	return NOTIFY_OK;
-}
-
-static struct notifier_block smpboot_thread_notifier = {
-	.notifier_call = smpboot_thread_call,
-	.priority = CPU_PRI_SMPBOOT,
-};
-
-void smpboot_thread_init(void)
-{
-	register_cpu_notifier(&smpboot_thread_notifier);
-}
-
 /**
  * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
  * @cpu: cpu that just started
@@ -959,6 +925,11 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
+	[CPUHP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
Index: b/kernel/smpboot.c
===================================================================
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -226,7 +226,7 @@ static void smpboot_unpark_thread(struct
 		kthread_unpark(tsk);
 }
 
-void smpboot_unpark_threads(unsigned int cpu)
+int smpboot_unpark_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -235,6 +235,7 @@ void smpboot_unpark_threads(unsigned int
 		if (cpumask_test_cpu(cpu, cur->cpumask))
 			smpboot_unpark_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
@@ -245,7 +246,7 @@ static void smpboot_park_thread(struct s
 		kthread_park(tsk);
 }
 
-void smpboot_park_threads(unsigned int cpu)
+int smpboot_park_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -253,6 +254,7 @@ void smpboot_park_threads(unsigned int c
 	list_for_each_entry_reverse(cur, &hotplug_threads, list)
 		smpboot_park_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -14,7 +14,7 @@ static inline void idle_threads_init(voi
 #endif
 
 int smpboot_create_threads(unsigned int cpu);
-void smpboot_park_threads(unsigned int cpu);
-void smpboot_unpark_threads(unsigned int cpu);
+int smpboot_park_threads(unsigned int cpu);
+int smpboot_unpark_threads(unsigned int cpu);
 
 #endif

[patch 13/20] cpu/hotplug: Unpark smpboot threads from the state machine

[Thomas Gleixner]

[-- Attachment #1: hotplug--Handle-smpboot-threads-in-state-machine.patch --]
[-- Type: text/plain, Size: 5179 bytes --]

Handle the smpboot threads in the state machine.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    7 +------
 include/linux/cpuhotplug.h |    1 +
 init/main.c                |    1 -
 kernel/cpu.c               |   39 +++++----------------------------------
 kernel/smpboot.c           |    6 ++++--
 kernel/smpboot.h           |    4 ++--
 6 files changed, 13 insertions(+), 45 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -78,7 +78,7 @@ enum {
 	/* migration should happen before other stuff but after perf */
 	CPU_PRI_PERF		= 20,
 	CPU_PRI_MIGRATION	= 10,
-	CPU_PRI_SMPBOOT		= 9,
+
 	/* bring up workqueues before normal notifiers and down after */
 	CPU_PRI_WORKQUEUE_UP	= 5,
 	CPU_PRI_WORKQUEUE_DOWN	= -5,
@@ -172,7 +172,6 @@ static inline void __unregister_cpu_noti
 }
 #endif
 
-void smpboot_thread_init(void);
 int cpu_up(unsigned int cpu);
 void notify_cpu_starting(unsigned int cpu);
 extern void cpu_maps_update_begin(void);
@@ -221,10 +220,6 @@ static inline void cpu_notifier_register
 {
 }
 
-static inline void smpboot_thread_init(void)
-{
-}
-
 #endif /* CONFIG_SMP */
 extern struct bus_type cpu_subsys;
 
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,6 +11,7 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
+	CPUHP_SMPBOOT_THREADS,
 	CPUHP_NOTIFY_ONLINE,
 	CPUHP_ONLINE_DYN,
 	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
Index: b/init/main.c
===================================================================
--- a/init/main.c
+++ b/init/main.c
@@ -388,7 +388,6 @@ static noinline void __init_refok rest_i
 	int pid;
 
 	rcu_scheduler_starting();
-	smpboot_thread_init();
 	/*
 	 * We need to spawn init first so that it obtains pid 1, however
 	 * the init task will end up wanting to create kthreads, which, if
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -481,8 +481,6 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
-	smpboot_park_threads(cpu);
-
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -612,38 +610,6 @@ int cpu_down(unsigned int cpu)
 EXPORT_SYMBOL(cpu_down);
 #endif /*CONFIG_HOTPLUG_CPU*/
 
-/*
- * Unpark per-CPU smpboot kthreads at CPU-online time.
- */
-static int smpboot_thread_call(struct notifier_block *nfb,
-			       unsigned long action, void *hcpu)
-{
-	int cpu = (long)hcpu;
-
-	switch (action & ~CPU_TASKS_FROZEN) {
-
-	case CPU_DOWN_FAILED:
-	case CPU_ONLINE:
-		smpboot_unpark_threads(cpu);
-		break;
-
-	default:
-		break;
-	}
-
-	return NOTIFY_OK;
-}
-
-static struct notifier_block smpboot_thread_notifier = {
-	.notifier_call = smpboot_thread_call,
-	.priority = CPU_PRI_SMPBOOT,
-};
-
-void smpboot_thread_init(void)
-{
-	register_cpu_notifier(&smpboot_thread_notifier);
-}
-
 /**
  * notify_cpu_starting(cpu) - call the CPU_STARTING notifiers
  * @cpu: cpu that just started
@@ -959,6 +925,11 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
+	[CPUHP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
 	[CPUHP_NOTIFY_ONLINE] = {
 		.name			= "notify:online",
 		.startup		= notify_online,
Index: b/kernel/smpboot.c
===================================================================
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -226,7 +226,7 @@ static void smpboot_unpark_thread(struct
 		kthread_unpark(tsk);
 }
 
-void smpboot_unpark_threads(unsigned int cpu)
+int smpboot_unpark_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -235,6 +235,7 @@ void smpboot_unpark_threads(unsigned int
 		if (cpumask_test_cpu(cpu, cur->cpumask))
 			smpboot_unpark_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_park_thread(struct smp_hotplug_thread *ht, unsigned int cpu)
@@ -245,7 +246,7 @@ static void smpboot_park_thread(struct s
 		kthread_park(tsk);
 }
 
-void smpboot_park_threads(unsigned int cpu)
+int smpboot_park_threads(unsigned int cpu)
 {
 	struct smp_hotplug_thread *cur;
 
@@ -253,6 +254,7 @@ void smpboot_park_threads(unsigned int c
 	list_for_each_entry_reverse(cur, &hotplug_threads, list)
 		smpboot_park_thread(cur, cpu);
 	mutex_unlock(&smpboot_threads_lock);
+	return 0;
 }
 
 static void smpboot_destroy_threads(struct smp_hotplug_thread *ht)
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -14,7 +14,7 @@ static inline void idle_threads_init(voi
 #endif
 
 int smpboot_create_threads(unsigned int cpu);
-void smpboot_park_threads(unsigned int cpu);
-void smpboot_unpark_threads(unsigned int cpu);
+int smpboot_park_threads(unsigned int cpu);
+int smpboot_unpark_threads(unsigned int cpu);
 
 #endif

[patch 14/20] cpu/hotplug: Split out the state walk into functions

[Thomas Gleixner]

[-- Attachment #1: hotplug--Create-state-walk-functions.patch --]
[-- Type: text/plain, Size: 4493 bytes --]

We need that for running callbacks on the AP and the BP.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |  111 ++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 68 insertions(+), 43 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+			  struct cpuhp_step *steps)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+				struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+			struct cpuhp_step *steps)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+			      struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = steps + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
-
 void unregister_cpu_notifier(struct notifier_block *nb)
 {
 	cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state++; st->state < st->target; st->state++) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->startup);
-	}
-}
 
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
-	for (; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
-		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_down(cpu, st);
-			break;
-		}
-	}
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
 	cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned
 	return 0;
 }
 
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state--; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->teardown);
-	}
-}
-
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int prev_state, ret = 0;
+	int ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	prev_state = st->state;
 	st->target = target;
-	while (st->state < st->target) {
-		struct cpuhp_step *step;
-
-		st->state++;
-		step = cpuhp_bp_states + st->state;
-		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_up(cpu, st);
-			break;
-		}
-	}
+	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
 	return ret;

[patch 14/20] cpu/hotplug: Split out the state walk into functions

[Thomas Gleixner]

[-- Attachment #1: hotplug--Create-state-walk-functions.patch --]
[-- Type: text/plain, Size: 4495 bytes --]

We need that for running callbacks on the AP and the BP.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c |  111 ++++++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 68 insertions(+), 43 deletions(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,10 +329,74 @@ static int bringup_cpu(unsigned int cpu)
 	return 0;
 }
 
+/*
+ * Hotplug state machine related functions
+ */
+static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st,
+			  struct cpuhp_step *steps)
+{
+	for (st->state++; st->state < st->target; st->state++) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->startup);
+	}
+}
+
+static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+				struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	for (; st->state > target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_down(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
+static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st,
+			struct cpuhp_step *steps)
+{
+	for (st->state--; st->state > st->target; st->state--) {
+		struct cpuhp_step *step = steps + st->state;
+
+		if (!step->skip_onerr)
+			cpuhp_invoke_callback(cpu, st->state, step->teardown);
+	}
+}
+
+static int cpuhp_up_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st,
+			      struct cpuhp_step *steps, enum cpuhp_state target)
+{
+	enum cpuhp_state prev_state = st->state;
+	int ret = 0;
+
+	while (st->state < target) {
+		struct cpuhp_step *step;
+
+		st->state++;
+		step = steps + st->state;
+		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
+		if (ret) {
+			st->target = prev_state;
+			undo_cpu_up(cpu, st, steps);
+			break;
+		}
+	}
+	return ret;
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
-
 void unregister_cpu_notifier(struct notifier_block *nb)
 {
 	cpu_maps_update_begin();
@@ -537,15 +601,6 @@ static int notify_dead(unsigned int cpu)
 #endif
 
 #ifdef CONFIG_HOTPLUG_CPU
-static void undo_cpu_down(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state++; st->state < st->target; st->state++) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->startup);
-	}
-}
 
 /* Requires cpu_add_remove_lock to be held */
 static int __ref _cpu_down(unsigned int cpu, int tasks_frozen,
@@ -567,16 +622,8 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
-	for (; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
+	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
-		ret = cpuhp_invoke_callback(cpu, st->state, step->teardown);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_down(cpu, st);
-			break;
-		}
-	}
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
 
 	cpu_hotplug_done();
@@ -645,22 +692,12 @@ static int cpuhp_set_cpu_active(unsigned
 	return 0;
 }
 
-static void undo_cpu_up(unsigned int cpu, struct cpuhp_cpu_state *st)
-{
-	for (st->state--; st->state > st->target; st->state--) {
-		struct cpuhp_step *step = cpuhp_bp_states + st->state;
-
-		if (!step->skip_onerr)
-			cpuhp_invoke_callback(cpu, st->state, step->teardown);
-	}
-}
-
 /* Requires cpu_add_remove_lock to be held */
 static int _cpu_up(unsigned int cpu, int tasks_frozen, enum cpuhp_state target)
 {
 	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	struct task_struct *idle;
-	int prev_state, ret = 0;
+	int ret = 0;
 
 	cpu_hotplug_begin();
 
@@ -687,20 +724,8 @@ static int _cpu_up(unsigned int cpu, int
 
 	cpuhp_tasks_frozen = tasks_frozen;
 
-	prev_state = st->state;
 	st->target = target;
-	while (st->state < st->target) {
-		struct cpuhp_step *step;
-
-		st->state++;
-		step = cpuhp_bp_states + st->state;
-		ret = cpuhp_invoke_callback(cpu, st->state, step->startup);
-		if (ret) {
-			st->target = prev_state;
-			undo_cpu_up(cpu, st);
-			break;
-		}
-	}
+	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
 	return ret;

[patch 15/20] cpu/hotplug: Create hotplug threads

[Thomas Gleixner]

[-- Attachment #1: hotplug--Create-hotplug-threads.patch --]
[-- Type: text/plain, Size: 6098 bytes --]

In order to let the hotplugged cpu take care of the setup/teardown, we need a
seperate hotplug thread.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c     |  145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/smp.c     |    1 
 kernel/smpboot.h |    2 
 3 files changed, 147 insertions(+), 1 deletion(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,6 +22,7 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+#include <linux/smpboot.h>
 
 #include <trace/events/power.h>
 #define CREATE_TRACE_POINTS
@@ -33,10 +34,24 @@
  * cpuhp_cpu_state - Per cpu hotplug state storage
  * @state:	The current cpu state
  * @target:	The target state
+ * @thread:	Pointer to the hotplug thread
+ * @should_run:	Thread should execute
+ * @cb_stat:	The state for a single callback (install/uninstall)
+ * @cb:		Single callback function (install/uninstall)
+ * @result:	Result of the operation
+ * @done:	Signal completion to the issuer of the task
  */
 struct cpuhp_cpu_state {
 	enum cpuhp_state	state;
 	enum cpuhp_state	target;
+#ifdef CONFIG_SMP
+	struct task_struct	*thread;
+	bool			should_run;
+	enum cpuhp_state	cb_state;
+	int			(*cb)(unsigned int cpu);
+	int			result;
+	struct completion	done;
+#endif
 };
 
 static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
@@ -394,6 +409,134 @@ static int cpuhp_up_callbacks(unsigned i
 	return ret;
 }
 
+/*
+ * The cpu hotplug threads manage the bringup and teardown of the cpus
+ */
+static void cpuhp_create(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	init_completion(&st->done);
+}
+
+static int cpuhp_should_run(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	return st->should_run;
+}
+
+/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
+static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+
+	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
+}
+
+/* Execute the online startup callbacks. Used to be CPU_ONLINE */
+static int cpuhp_ap_online(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	return cpuhp_up_callbacks(cpu, st, cpuhp_ap_states, st->target);
+}
+
+/*
+ * Execute teardown/startup callbacks on the plugged cpu. Also used to invoke
+ * callbacks when a state gets [un]installed at runtime.
+ */
+static void cpuhp_thread_fun(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	int ret = 0;
+
+	/*
+	 * Paired with the mb() in cpuhp_kick_ap_work and
+	 * cpuhp_invoke_ap_callback, so the work set is consistent visible.
+	 */
+	smp_mb();
+	if (!st->should_run)
+		return;
+
+	st->should_run = false;
+
+	/* Single callback invocation for [un]install ? */
+	if (st->cb) {
+		if (st->cb_state < CPUHP_AP_ONLINE) {
+			local_irq_disable();
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+			local_irq_enable();
+		} else {
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+		}
+	} else {
+		/* Regular hotplug work */
+		if (st->state < st->target)
+			ret = cpuhp_ap_online(cpu, st);
+		else if (st->state > st->target)
+			ret = cpuhp_ap_offline(cpu, st);
+	}
+	st->result = ret;
+	complete(&st->done);
+}
+
+/* Invoke a single callback on a remote cpu */
+static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
+				    int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	if (!cpu_online(cpu))
+		return 0;
+
+	st->cb_state = state;
+	st->cb = cb;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
+/* Regular hotplug invocation of the AP hotplug thread */
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	st->result = 0;
+	st->cb = NULL;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	trace_cpuhp_exit(cpu, st->state, state, st->result);
+	return st->result;
+}
+
+static struct smp_hotplug_thread cpuhp_threads = {
+	.store			= &cpuhp_state.thread,
+	.create			= &cpuhp_create,
+	.thread_should_run	= cpuhp_should_run,
+	.thread_fn		= cpuhp_thread_fun,
+	.thread_comm		= "cpuhp/%u",
+	.selfparking		= true,
+};
+
+void __init cpuhp_threads_init(void)
+{
+	BUG_ON(smpboot_register_percpu_thread(&cpuhp_threads));
+	kthread_unpark(this_cpu_read(cpuhp_state.thread));
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -997,7 +1140,7 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/smp.c
===================================================================
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -569,6 +569,7 @@ void __init smp_init(void)
 	unsigned int cpu;
 
 	idle_threads_init();
+	cpuhp_threads_init();
 
 	/* FIXME: This should be done in userspace --RR */
 	for_each_present_cpu(cpu) {
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -17,4 +17,6 @@ int smpboot_create_threads(unsigned int
 int smpboot_park_threads(unsigned int cpu);
 int smpboot_unpark_threads(unsigned int cpu);
 
+void __init cpuhp_threads_init(void);
+
 #endif

[patch 15/20] cpu/hotplug: Create hotplug threads

[Thomas Gleixner]

[-- Attachment #1: hotplug--Create-hotplug-threads.patch --]
[-- Type: text/plain, Size: 6100 bytes --]

In order to let the hotplugged cpu take care of the setup/teardown, we need a
seperate hotplug thread.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 kernel/cpu.c     |  145 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 kernel/smp.c     |    1 
 kernel/smpboot.h |    2 
 3 files changed, 147 insertions(+), 1 deletion(-)

Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -22,6 +22,7 @@
 #include <linux/lockdep.h>
 #include <linux/tick.h>
 #include <linux/irq.h>
+#include <linux/smpboot.h>
 
 #include <trace/events/power.h>
 #define CREATE_TRACE_POINTS
@@ -33,10 +34,24 @@
  * cpuhp_cpu_state - Per cpu hotplug state storage
  * @state:	The current cpu state
  * @target:	The target state
+ * @thread:	Pointer to the hotplug thread
+ * @should_run:	Thread should execute
+ * @cb_stat:	The state for a single callback (install/uninstall)
+ * @cb:		Single callback function (install/uninstall)
+ * @result:	Result of the operation
+ * @done:	Signal completion to the issuer of the task
  */
 struct cpuhp_cpu_state {
 	enum cpuhp_state	state;
 	enum cpuhp_state	target;
+#ifdef CONFIG_SMP
+	struct task_struct	*thread;
+	bool			should_run;
+	enum cpuhp_state	cb_state;
+	int			(*cb)(unsigned int cpu);
+	int			result;
+	struct completion	done;
+#endif
 };
 
 static DEFINE_PER_CPU(struct cpuhp_cpu_state, cpuhp_state);
@@ -394,6 +409,134 @@ static int cpuhp_up_callbacks(unsigned i
 	return ret;
 }
 
+/*
+ * The cpu hotplug threads manage the bringup and teardown of the cpus
+ */
+static void cpuhp_create(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	init_completion(&st->done);
+}
+
+static int cpuhp_should_run(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	return st->should_run;
+}
+
+/* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
+static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+
+	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
+}
+
+/* Execute the online startup callbacks. Used to be CPU_ONLINE */
+static int cpuhp_ap_online(unsigned int cpu, struct cpuhp_cpu_state *st)
+{
+	return cpuhp_up_callbacks(cpu, st, cpuhp_ap_states, st->target);
+}
+
+/*
+ * Execute teardown/startup callbacks on the plugged cpu. Also used to invoke
+ * callbacks when a state gets [un]installed at runtime.
+ */
+static void cpuhp_thread_fun(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	int ret = 0;
+
+	/*
+	 * Paired with the mb() in cpuhp_kick_ap_work and
+	 * cpuhp_invoke_ap_callback, so the work set is consistent visible.
+	 */
+	smp_mb();
+	if (!st->should_run)
+		return;
+
+	st->should_run = false;
+
+	/* Single callback invocation for [un]install ? */
+	if (st->cb) {
+		if (st->cb_state < CPUHP_AP_ONLINE) {
+			local_irq_disable();
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+			local_irq_enable();
+		} else {
+			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
+		}
+	} else {
+		/* Regular hotplug work */
+		if (st->state < st->target)
+			ret = cpuhp_ap_online(cpu, st);
+		else if (st->state > st->target)
+			ret = cpuhp_ap_offline(cpu, st);
+	}
+	st->result = ret;
+	complete(&st->done);
+}
+
+/* Invoke a single callback on a remote cpu */
+static int cpuhp_invoke_ap_callback(int cpu, enum cpuhp_state state,
+				    int (*cb)(unsigned int))
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	if (!cpu_online(cpu))
+		return 0;
+
+	st->cb_state = state;
+	st->cb = cb;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
+/* Regular hotplug invocation of the AP hotplug thread */
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	st->result = 0;
+	st->cb = NULL;
+	/*
+	 * Make sure the above stores are visible before should_run becomes
+	 * true. Paired with the mb() above in cpuhp_thread_fun()
+	 */
+	smp_mb();
+	st->should_run = true;
+	wake_up_process(st->thread);
+	wait_for_completion(&st->done);
+	trace_cpuhp_exit(cpu, st->state, state, st->result);
+	return st->result;
+}
+
+static struct smp_hotplug_thread cpuhp_threads = {
+	.store			= &cpuhp_state.thread,
+	.create			= &cpuhp_create,
+	.thread_should_run	= cpuhp_should_run,
+	.thread_fn		= cpuhp_thread_fun,
+	.thread_comm		= "cpuhp/%u",
+	.selfparking		= true,
+};
+
+void __init cpuhp_threads_init(void)
+{
+	BUG_ON(smpboot_register_percpu_thread(&cpuhp_threads));
+	kthread_unpark(this_cpu_read(cpuhp_state.thread));
+}
+
 #ifdef CONFIG_HOTPLUG_CPU
 EXPORT_SYMBOL(register_cpu_notifier);
 EXPORT_SYMBOL(__register_cpu_notifier);
@@ -997,7 +1140,7 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state > CPUHP_AP_OFFLINE && state < CPUHP_AP_ONLINE);
+	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/smp.c
===================================================================
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -569,6 +569,7 @@ void __init smp_init(void)
 	unsigned int cpu;
 
 	idle_threads_init();
+	cpuhp_threads_init();
 
 	/* FIXME: This should be done in userspace --RR */
 	for_each_present_cpu(cpu) {
Index: b/kernel/smpboot.h
===================================================================
--- a/kernel/smpboot.h
+++ b/kernel/smpboot.h
@@ -17,4 +17,6 @@ int smpboot_create_threads(unsigned int
 int smpboot_park_threads(unsigned int cpu);
 int smpboot_unpark_threads(unsigned int cpu);
 
+void __init cpuhp_threads_init(void);
+
 #endif

[patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu

[Thomas Gleixner]

[-- Attachment #1: hotplug--Move-online-calls-to-hotplugged-cpu.patch --]
[-- Type: text/plain, Size: 8671 bytes --]

Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   10 +--
 kernel/cpu.c               |  144 ++++++++++++++++++++++++++++++---------------
 2 files changed, 102 insertions(+), 52 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_SMPBOOT_THREADS,
-	CPUHP_NOTIFY_ONLINE,
-	CPUHP_ONLINE_DYN,
-	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
+	CPUHP_KICK_AP_THREAD,
+	CPUHP_BP_ONLINE,
+	CPUHP_AP_SMPBOOT_THREADS,
+	CPUHP_AP_NOTIFY_ONLINE,
+	CPUHP_AP_ONLINE_DYN,
+	CPUHP_AP_ONLINE_DYN_END		= CPUHP_AP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int
 /* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
 static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
-	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+	enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
 
 	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
 }
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned in
 			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
 		}
 	} else {
+		/* Cannot happen .... */
+		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
 		/* Regular hotplug work */
 		if (st->state < st->target)
 			ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int
 }
 
 /* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
-	enum cpuhp_state state = st->state;
-
-	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
 	st->result = 0;
 	st->cb = NULL;
 	/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned i
 	smp_mb();
 	st->should_run = true;
 	wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	__cpuhp_kick_ap_work(st);
 	wait_for_completion(&st->done);
 	trace_cpuhp_exit(cpu, st->state, state, st->result);
 	return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
+	/* Park the hotplug thread */
+	kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+
+		/*
+		 * We might have stopped still in the range of the AP hotplug
+		 * thread. Nothing to do anymore.
+		 */
+		if (st->state >= CPUHP_KICK_AP_THREAD)
+			goto out;
+	}
+	/*
+	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * to do the further cleanups.
+	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
 	cpu_hotplug_done();
 	/* This post dead nonsense must die */
 	if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cp
  */
 static int cpuhp_set_cpu_active(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread */
+	/* Unpark the stopper thread and the hotplug thread */
 	stop_machine_unpark(cpu);
+	kthread_unpark(st->thread);
 	return 0;
 }
 
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread once more.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+	}
+
+	/*
+	 * Try to reach the target state. We max out on the BP at
+	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * responsible for bringing it up to the target state.
+	 */
+	target = min((int)target, CPUHP_KICK_AP_THREAD);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
-	[CPUHP_SMPBOOT_THREADS] = {
-		.name			= "smpboot:threads",
-		.startup		= smpboot_unpark_threads,
-		.teardown		= smpboot_park_threads,
-	},
-	[CPUHP_NOTIFY_ONLINE] = {
-		.name			= "notify:online",
-		.startup		= notify_online,
-		.teardown		= notify_down_prepare,
-		.cant_stop		= true,
+	[CPUHP_KICK_AP_THREAD] = {
+		.name			= "cpuhp:kickthread",
+		.startup		= cpuhp_kick_ap_work,
+		.teardown		= cpuhp_kick_ap_work,
 	},
 #endif
-	[CPUHP_ONLINE] = {
+	[CPUHP_BP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states
 		.skip_onerr		= true,
 		.cant_stop		= true,
 	},
+	[CPUHP_AP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
+	[CPUHP_AP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
 #endif
 	[CPUHP_ONLINE] = {
 		.name			= "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+		return true;
+	return state > CPUHP_BP_ONLINE;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum
 	return cpuhp_get_step(state)->teardown;
 }
 
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
-	int (*cb)(unsigned int cpu) = __cb;
-
-	BUG_ON(cb(smp_processor_id()));
-}
-
 /*
  * Call the startup/teardown function for a step either on the AP or
  * on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enu
 
 	if (!cb)
 		return 0;
-
-	/*
-	 * This invokes the callback directly for now. In a later step we
-	 * convert that to use cpuhp_invoke_callback().
-	 */
-	if (cpuhp_is_ap_state(state)) {
-		/*
-		 * Note, that a function called on the AP is not
-		 * allowed to fail.
-		 */
-		if (cpu_online(cpu))
-			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
-		return 0;
-	}
-
 	/*
 	 * The non AP bound callbacks can fail on bringup. On teardown
 	 * e.g. module removal we crash for now.
 	 */
-	ret = cb(cpu);
+#ifdef CONFIG_SMP
+	if (cpuhp_is_ap_state(state))
+		ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+	else
+		ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+	ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
 	BUG_ON(ret && !bringup);
 	return ret;
 }
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuh
 	enum cpuhp_state i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
-		if (cpuhp_bp_states[i].name)
+	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+		if (cpuhp_ap_states[i].name)
 			continue;
 
-		cpuhp_bp_states[i].name = "Reserved";
+		cpuhp_ap_states[i].name = "Reserved";
 		mutex_unlock(&cpuhp_state_mutex);
 		return i;
 	}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state
 	get_online_cpus();
 
 	/* currently assignments for the ONLINE state are possible */
-	if (state == CPUHP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN) {
 		dyn_state = 1;
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)

[patch 16/20] cpu/hotplug: Move online calls to hotplugged cpu

[Thomas Gleixner]

[-- Attachment #1: hotplug--Move-online-calls-to-hotplugged-cpu.patch --]
[-- Type: text/plain, Size: 8673 bytes --]

Let the hotplugged cpu invoke the setup/teardown callbacks
(CPU_ONLINE/CPU_DOWN_PREPARE) itself.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |   10 +--
 kernel/cpu.c               |  144 ++++++++++++++++++++++++++++++---------------
 2 files changed, 102 insertions(+), 52 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -11,10 +11,12 @@ enum cpuhp_state {
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
 	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_SMPBOOT_THREADS,
-	CPUHP_NOTIFY_ONLINE,
-	CPUHP_ONLINE_DYN,
-	CPUHP_ONLINE_DYN_END		= CPUHP_ONLINE_DYN + 30,
+	CPUHP_KICK_AP_THREAD,
+	CPUHP_BP_ONLINE,
+	CPUHP_AP_SMPBOOT_THREADS,
+	CPUHP_AP_NOTIFY_ONLINE,
+	CPUHP_AP_ONLINE_DYN,
+	CPUHP_AP_ONLINE_DYN_END		= CPUHP_AP_ONLINE_DYN + 30,
 	CPUHP_ONLINE,
 };
 
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -429,7 +429,7 @@ static int cpuhp_should_run(unsigned int
 /* Execute the teardown callbacks. Used to be CPU_DOWN_PREPARE */
 static int cpuhp_ap_offline(unsigned int cpu, struct cpuhp_cpu_state *st)
 {
-	enum cpuhp_state target = max((int)st->target, CPUHP_AP_ONLINE);
+	enum cpuhp_state target = max((int)st->target, CPUHP_TEARDOWN_CPU);
 
 	return cpuhp_down_callbacks(cpu, st, cpuhp_ap_states, target);
 }
@@ -469,6 +469,9 @@ static void cpuhp_thread_fun(unsigned in
 			ret = cpuhp_invoke_callback(cpu, st->cb_state, st->cb);
 		}
 	} else {
+		/* Cannot happen .... */
+		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+
 		/* Regular hotplug work */
 		if (st->state < st->target)
 			ret = cpuhp_ap_online(cpu, st);
@@ -502,12 +505,8 @@ static int cpuhp_invoke_ap_callback(int
 }
 
 /* Regular hotplug invocation of the AP hotplug thread */
-static int cpuhp_kick_ap_work(unsigned int cpu)
+static void __cpuhp_kick_ap_work(struct cpuhp_cpu_state *st)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
-	enum cpuhp_state state = st->state;
-
-	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
 	st->result = 0;
 	st->cb = NULL;
 	/*
@@ -517,6 +516,15 @@ static int cpuhp_kick_ap_work(unsigned i
 	smp_mb();
 	st->should_run = true;
 	wake_up_process(st->thread);
+}
+
+static int cpuhp_kick_ap_work(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	enum cpuhp_state state = st->state;
+
+	trace_cpuhp_enter(cpu, st->target, state, cpuhp_kick_ap_work);
+	__cpuhp_kick_ap_work(st);
 	wait_for_completion(&st->done);
 	trace_cpuhp_exit(cpu, st->state, state, st->result);
 	return st->result;
@@ -688,6 +696,9 @@ static int takedown_cpu(unsigned int cpu
 	else
 		synchronize_rcu();
 
+	/* Park the hotplug thread */
+	kthread_park(per_cpu_ptr(&cpuhp_state, cpu)->thread);
+
 	/*
 	 * Prevent irq alloc/free while the dying cpu reorganizes the
 	 * interrupt affinities.
@@ -765,10 +776,34 @@ static int __ref _cpu_down(unsigned int
 
 	prev_state = st->state;
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+
+		/*
+		 * We might have stopped still in the range of the AP hotplug
+		 * thread. Nothing to do anymore.
+		 */
+		if (st->state >= CPUHP_KICK_AP_THREAD)
+			goto out;
+	}
+	/*
+	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * to do the further cleanups.
+	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
 
 	hasdied = prev_state != st->state && st->state == CPUHP_OFFLINE;
-
+out:
 	cpu_hotplug_done();
 	/* This post dead nonsense must die */
 	if (!ret && hasdied)
@@ -828,10 +863,13 @@ void notify_cpu_starting(unsigned int cp
  */
 static int cpuhp_set_cpu_active(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread */
+	/* Unpark the stopper thread and the hotplug thread */
 	stop_machine_unpark(cpu);
+	kthread_unpark(st->thread);
 	return 0;
 }
 
@@ -868,6 +906,26 @@ static int _cpu_up(unsigned int cpu, int
 	cpuhp_tasks_frozen = tasks_frozen;
 
 	st->target = target;
+	/*
+	 * If the current CPU state is in the range of the AP hotplug thread,
+	 * then we need to kick the thread once more.
+	 */
+	if (st->state >= CPUHP_KICK_AP_THREAD) {
+		ret = cpuhp_kick_ap_work(cpu);
+		/*
+		 * The AP side has done the error rollback already. Just
+		 * return the error code..
+		 */
+		if (ret)
+			goto out;
+	}
+
+	/*
+	 * Try to reach the target state. We max out on the BP at
+	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * responsible for bringing it up to the target state.
+	 */
+	target = min((int)target, CPUHP_KICK_AP_THREAD);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1093,19 +1151,13 @@ static struct cpuhp_step cpuhp_bp_states
 		.startup		= cpuhp_set_cpu_active,
 		.teardown		= NULL,
 	},
-	[CPUHP_SMPBOOT_THREADS] = {
-		.name			= "smpboot:threads",
-		.startup		= smpboot_unpark_threads,
-		.teardown		= smpboot_park_threads,
-	},
-	[CPUHP_NOTIFY_ONLINE] = {
-		.name			= "notify:online",
-		.startup		= notify_online,
-		.teardown		= notify_down_prepare,
-		.cant_stop		= true,
+	[CPUHP_KICK_AP_THREAD] = {
+		.name			= "cpuhp:kickthread",
+		.startup		= cpuhp_kick_ap_work,
+		.teardown		= cpuhp_kick_ap_work,
 	},
 #endif
-	[CPUHP_ONLINE] = {
+	[CPUHP_BP_ONLINE] = {
 		.name			= "online",
 		.startup		= NULL,
 		.teardown		= NULL,
@@ -1122,6 +1174,16 @@ static struct cpuhp_step cpuhp_ap_states
 		.skip_onerr		= true,
 		.cant_stop		= true,
 	},
+	[CPUHP_AP_SMPBOOT_THREADS] = {
+		.name			= "smpboot:threads",
+		.startup		= smpboot_unpark_threads,
+		.teardown		= smpboot_park_threads,
+	},
+	[CPUHP_AP_NOTIFY_ONLINE] = {
+		.name			= "notify:online",
+		.startup		= notify_online,
+		.teardown		= notify_down_prepare,
+	},
 #endif
 	[CPUHP_ONLINE] = {
 		.name			= "online",
@@ -1140,7 +1202,9 @@ static int cpuhp_cb_check(enum cpuhp_sta
 
 static bool cpuhp_is_ap_state(enum cpuhp_state state)
 {
-	return (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE);
+	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
+		return true;
+	return state > CPUHP_BP_ONLINE;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
@@ -1172,14 +1236,6 @@ static void *cpuhp_get_teardown_cb(enum
 	return cpuhp_get_step(state)->teardown;
 }
 
-/* Helper function to run callback on the target cpu */
-static void cpuhp_on_cpu_cb(void *__cb)
-{
-	int (*cb)(unsigned int cpu) = __cb;
-
-	BUG_ON(cb(smp_processor_id()));
-}
-
 /*
  * Call the startup/teardown function for a step either on the AP or
  * on the current CPU.
@@ -1191,26 +1247,18 @@ static int cpuhp_issue_call(int cpu, enu
 
 	if (!cb)
 		return 0;
-
-	/*
-	 * This invokes the callback directly for now. In a later step we
-	 * convert that to use cpuhp_invoke_callback().
-	 */
-	if (cpuhp_is_ap_state(state)) {
-		/*
-		 * Note, that a function called on the AP is not
-		 * allowed to fail.
-		 */
-		if (cpu_online(cpu))
-			smp_call_function_single(cpu, cpuhp_on_cpu_cb, cb, 1);
-		return 0;
-	}
-
 	/*
 	 * The non AP bound callbacks can fail on bringup. On teardown
 	 * e.g. module removal we crash for now.
 	 */
-	ret = cb(cpu);
+#ifdef CONFIG_SMP
+	if (cpuhp_is_ap_state(state))
+		ret = cpuhp_invoke_ap_callback(cpu, state, cb);
+	else
+		ret = cpuhp_invoke_callback(cpu, state, cb);
+#else
+	ret = cpuhp_invoke_callback(cpu, state, cb);
+#endif
 	BUG_ON(ret && !bringup);
 	return ret;
 }
@@ -1252,11 +1300,11 @@ static int cpuhp_reserve_state(enum cpuh
 	enum cpuhp_state i;
 
 	mutex_lock(&cpuhp_state_mutex);
-	for (i = CPUHP_ONLINE_DYN; i <= CPUHP_ONLINE_DYN_END; i++) {
-		if (cpuhp_bp_states[i].name)
+	for (i = CPUHP_AP_ONLINE_DYN; i <= CPUHP_AP_ONLINE_DYN_END; i++) {
+		if (cpuhp_ap_states[i].name)
 			continue;
 
-		cpuhp_bp_states[i].name = "Reserved";
+		cpuhp_ap_states[i].name = "Reserved";
 		mutex_unlock(&cpuhp_state_mutex);
 		return i;
 	}
@@ -1289,7 +1337,7 @@ int __cpuhp_setup_state(enum cpuhp_state
 	get_online_cpus();
 
 	/* currently assignments for the ONLINE state are possible */
-	if (state == CPUHP_ONLINE_DYN) {
+	if (state == CPUHP_AP_ONLINE_DYN) {
 		dyn_state = 1;
 		ret = cpuhp_reserve_state(state);
 		if (ret < 0)

[patch 17/20] arch/hotplug: Call into idle with a proper state

[Thomas Gleixner]

[-- Attachment #1: arch-hotplug--Call-into-idle-with-a-proper-state.patch --]
[-- Type: text/plain, Size: 9949 bytes --]

Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/alpha/kernel/smp.c         |    2 +-
 arch/arc/kernel/smp.c           |    2 +-
 arch/arm/kernel/smp.c           |    2 +-
 arch/arm64/kernel/smp.c         |    2 +-
 arch/blackfin/mach-common/smp.c |    2 +-
 arch/hexagon/kernel/smp.c       |    2 +-
 arch/ia64/kernel/smpboot.c      |    2 +-
 arch/m32r/kernel/smpboot.c      |    2 +-
 arch/metag/kernel/smp.c         |    2 +-
 arch/mips/kernel/smp.c          |    2 +-
 arch/mn10300/kernel/smp.c       |    2 +-
 arch/parisc/kernel/smp.c        |    2 +-
 arch/powerpc/kernel/smp.c       |    2 +-
 arch/s390/kernel/smp.c          |    2 +-
 arch/sh/kernel/smp.c            |    2 +-
 arch/sparc/kernel/smp_32.c      |    2 +-
 arch/sparc/kernel/smp_64.c      |    2 +-
 arch/tile/kernel/smpboot.c      |    2 +-
 arch/x86/kernel/smpboot.c       |    2 +-
 arch/x86/xen/smp.c              |    2 +-
 arch/xtensa/kernel/smp.c        |    2 +-
 include/linux/cpuhotplug.h      |    1 +
 22 files changed, 22 insertions(+), 21 deletions(-)

Index: b/arch/alpha/kernel/smp.c
===================================================================
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
 	      cpuid, current, current->active_mm));
 
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Wait until hwrpb->txrdy is clear for cpu.  Return -1 on timeout.  */
Index: b/arch/arc/kernel/smp.c
===================================================================
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
 
 	local_irq_enable();
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /*
Index: b/arch/arm/kernel/smp.c
===================================================================
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/arm64/kernel/smp.c
===================================================================
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
Index: b/arch/blackfin/mach-common/smp.c
===================================================================
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
 
 	/* We are done with local CPU inits, unblock the boot CPU. */
 	set_cpu_online(cpu, true);
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_prepare_boot_cpu(void)
Index: b/arch/hexagon/kernel/smp.c
===================================================================
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 
Index: b/arch/ia64/kernel/smpboot.c
===================================================================
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
 	preempt_disable();
 	smp_callin();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/m32r/kernel/smpboot.c
===================================================================
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
 	 */
 	local_flush_tlb_all();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/metag/kernel/smp.c
===================================================================
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/mips/kernel/smp.c
===================================================================
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
 	WARN_ON_ONCE(!irqs_disabled());
 	mp_ops->smp_finish();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void stop_this_cpu(void *dummy)
Index: b/arch/mn10300/kernel/smp.c
===================================================================
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
 #ifdef CONFIG_GENERIC_CLOCKEVENTS
 	init_clockevents();
 #endif
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/parisc/kernel/smp.c
===================================================================
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
 
 	local_irq_enable();  /* Interrupts have been off until now */
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* NOTREACHED */
 	panic("smp_callin() AAAAaaaaahhhh....\n");
Index: b/arch/powerpc/kernel/smp.c
===================================================================
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	BUG();
 }
Index: b/arch/s390/kernel/smp.c
===================================================================
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cp
 	set_cpu_online(smp_processor_id(), true);
 	inc_irq_stat(CPU_RST);
 	local_irq_enable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Upping and downing of CPUs */
Index: b/arch/sh/kernel/smp.c
===================================================================
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
 	set_cpu_online(cpu, true);
 	per_cpu(cpu_state, cpu) = CPU_ONLINE;
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 extern struct {
Index: b/arch/sparc/kernel/smp_32.c
===================================================================
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *
 	local_irq_enable();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* We should never reach here! */
 	BUG();
Index: b/arch/sparc/kernel/smp_64.c
===================================================================
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void cpu_panic(void)
Index: b/arch/tile/kernel/smpboot.c
===================================================================
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
 	/* Set up tile-timer clock-event device on this cpu */
 	setup_tile_timer();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 int __cpu_up(unsigned int cpu, struct task_struct *tidle)
Index: b/arch/x86/kernel/smpboot.c
===================================================================
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void
 	x86_cpuinit.setup_percpu_clockev();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_store_boot_cpu_info(void)
Index: b/arch/x86/xen/smp.c
===================================================================
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_an
 		xen_pvh_secondary_vcpu_init(cpu);
 #endif
 	cpu_bringup();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void xen_smp_intr_free(unsigned int cpu)
Index: b/arch/xtensa/kernel/smp.c
===================================================================
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
 
 	complete(&cpu_running);
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void mx_cpu_start(void *p)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
 	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_KICK_AP_THREAD,
 	CPUHP_BP_ONLINE,
+	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
 	CPUHP_AP_ONLINE_DYN,

[patch 17/20] arch/hotplug: Call into idle with a proper state

[Thomas Gleixner]

[-- Attachment #1: arch-hotplug--Call-into-idle-with-a-proper-state.patch --]
[-- Type: text/plain, Size: 9951 bytes --]

Let the non boot cpus call into idle with the corresponding hotplug state, so
the hotplug core can handle the further bringup. That's a first step to
convert the boot side of the hotplugged cpus to do all the synchronization
with the other side through the state machine. For now it'll only start the
hotplug thread and kick the full bringup of the cpu.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 arch/alpha/kernel/smp.c         |    2 +-
 arch/arc/kernel/smp.c           |    2 +-
 arch/arm/kernel/smp.c           |    2 +-
 arch/arm64/kernel/smp.c         |    2 +-
 arch/blackfin/mach-common/smp.c |    2 +-
 arch/hexagon/kernel/smp.c       |    2 +-
 arch/ia64/kernel/smpboot.c      |    2 +-
 arch/m32r/kernel/smpboot.c      |    2 +-
 arch/metag/kernel/smp.c         |    2 +-
 arch/mips/kernel/smp.c          |    2 +-
 arch/mn10300/kernel/smp.c       |    2 +-
 arch/parisc/kernel/smp.c        |    2 +-
 arch/powerpc/kernel/smp.c       |    2 +-
 arch/s390/kernel/smp.c          |    2 +-
 arch/sh/kernel/smp.c            |    2 +-
 arch/sparc/kernel/smp_32.c      |    2 +-
 arch/sparc/kernel/smp_64.c      |    2 +-
 arch/tile/kernel/smpboot.c      |    2 +-
 arch/x86/kernel/smpboot.c       |    2 +-
 arch/x86/xen/smp.c              |    2 +-
 arch/xtensa/kernel/smp.c        |    2 +-
 include/linux/cpuhotplug.h      |    1 +
 22 files changed, 22 insertions(+), 21 deletions(-)

Index: b/arch/alpha/kernel/smp.c
===================================================================
--- a/arch/alpha/kernel/smp.c
+++ b/arch/alpha/kernel/smp.c
@@ -168,7 +168,7 @@ smp_callin(void)
 	      cpuid, current, current->active_mm));
 
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Wait until hwrpb->txrdy is clear for cpu.  Return -1 on timeout.  */
Index: b/arch/arc/kernel/smp.c
===================================================================
--- a/arch/arc/kernel/smp.c
+++ b/arch/arc/kernel/smp.c
@@ -142,7 +142,7 @@ void start_kernel_secondary(void)
 
 	local_irq_enable();
 	preempt_disable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /*
Index: b/arch/arm/kernel/smp.c
===================================================================
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -409,7 +409,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/arm64/kernel/smp.c
===================================================================
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -195,7 +195,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU
Index: b/arch/blackfin/mach-common/smp.c
===================================================================
--- a/arch/blackfin/mach-common/smp.c
+++ b/arch/blackfin/mach-common/smp.c
@@ -333,7 +333,7 @@ void secondary_start_kernel(void)
 
 	/* We are done with local CPU inits, unblock the boot CPU. */
 	set_cpu_online(cpu, true);
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_prepare_boot_cpu(void)
Index: b/arch/hexagon/kernel/smp.c
===================================================================
--- a/arch/hexagon/kernel/smp.c
+++ b/arch/hexagon/kernel/smp.c
@@ -180,7 +180,7 @@ void start_secondary(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 
Index: b/arch/ia64/kernel/smpboot.c
===================================================================
--- a/arch/ia64/kernel/smpboot.c
+++ b/arch/ia64/kernel/smpboot.c
@@ -454,7 +454,7 @@ start_secondary (void *unused)
 	preempt_disable();
 	smp_callin();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/m32r/kernel/smpboot.c
===================================================================
--- a/arch/m32r/kernel/smpboot.c
+++ b/arch/m32r/kernel/smpboot.c
@@ -432,7 +432,7 @@ int __init start_secondary(void *unused)
 	 */
 	local_flush_tlb_all();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/metag/kernel/smp.c
===================================================================
--- a/arch/metag/kernel/smp.c
+++ b/arch/metag/kernel/smp.c
@@ -396,7 +396,7 @@ asmlinkage void secondary_start_kernel(v
 	/*
 	 * OK, it's off to the idle thread for us
 	 */
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_cpus_done(unsigned int max_cpus)
Index: b/arch/mips/kernel/smp.c
===================================================================
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -191,7 +191,7 @@ asmlinkage void start_secondary(void)
 	WARN_ON_ONCE(!irqs_disabled());
 	mp_ops->smp_finish();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void stop_this_cpu(void *dummy)
Index: b/arch/mn10300/kernel/smp.c
===================================================================
--- a/arch/mn10300/kernel/smp.c
+++ b/arch/mn10300/kernel/smp.c
@@ -675,7 +675,7 @@ int __init start_secondary(void *unused)
 #ifdef CONFIG_GENERIC_CLOCKEVENTS
 	init_clockevents();
 #endif
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 	return 0;
 }
 
Index: b/arch/parisc/kernel/smp.c
===================================================================
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -305,7 +305,7 @@ void __init smp_callin(void)
 
 	local_irq_enable();  /* Interrupts have been off until now */
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* NOTREACHED */
 	panic("smp_callin() AAAAaaaaahhhh....\n");
Index: b/arch/powerpc/kernel/smp.c
===================================================================
--- a/arch/powerpc/kernel/smp.c
+++ b/arch/powerpc/kernel/smp.c
@@ -727,7 +727,7 @@ void start_secondary(void *unused)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	BUG();
 }
Index: b/arch/s390/kernel/smp.c
===================================================================
--- a/arch/s390/kernel/smp.c
+++ b/arch/s390/kernel/smp.c
@@ -798,7 +798,7 @@ static void smp_start_secondary(void *cp
 	set_cpu_online(smp_processor_id(), true);
 	inc_irq_stat(CPU_RST);
 	local_irq_enable();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 /* Upping and downing of CPUs */
Index: b/arch/sh/kernel/smp.c
===================================================================
--- a/arch/sh/kernel/smp.c
+++ b/arch/sh/kernel/smp.c
@@ -203,7 +203,7 @@ asmlinkage void start_secondary(void)
 	set_cpu_online(cpu, true);
 	per_cpu(cpu_state, cpu) = CPU_ONLINE;
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 extern struct {
Index: b/arch/sparc/kernel/smp_32.c
===================================================================
--- a/arch/sparc/kernel/smp_32.c
+++ b/arch/sparc/kernel/smp_32.c
@@ -364,7 +364,7 @@ static void sparc_start_secondary(void *
 	local_irq_enable();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 
 	/* We should never reach here! */
 	BUG();
Index: b/arch/sparc/kernel/smp_64.c
===================================================================
--- a/arch/sparc/kernel/smp_64.c
+++ b/arch/sparc/kernel/smp_64.c
@@ -134,7 +134,7 @@ void smp_callin(void)
 
 	local_irq_enable();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void cpu_panic(void)
Index: b/arch/tile/kernel/smpboot.c
===================================================================
--- a/arch/tile/kernel/smpboot.c
+++ b/arch/tile/kernel/smpboot.c
@@ -208,7 +208,7 @@ void online_secondary(void)
 	/* Set up tile-timer clock-event device on this cpu */
 	setup_tile_timer();
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 int __cpu_up(unsigned int cpu, struct task_struct *tidle)
Index: b/arch/x86/kernel/smpboot.c
===================================================================
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -248,7 +248,7 @@ static void notrace start_secondary(void
 	x86_cpuinit.setup_percpu_clockev();
 
 	wmb();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 void __init smp_store_boot_cpu_info(void)
Index: b/arch/x86/xen/smp.c
===================================================================
--- a/arch/x86/xen/smp.c
+++ b/arch/x86/xen/smp.c
@@ -112,7 +112,7 @@ asmlinkage __visible void cpu_bringup_an
 		xen_pvh_secondary_vcpu_init(cpu);
 #endif
 	cpu_bringup();
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void xen_smp_intr_free(unsigned int cpu)
Index: b/arch/xtensa/kernel/smp.c
===================================================================
--- a/arch/xtensa/kernel/smp.c
+++ b/arch/xtensa/kernel/smp.c
@@ -157,7 +157,7 @@ void secondary_start_kernel(void)
 
 	complete(&cpu_running);
 
-	cpu_startup_entry(CPUHP_ONLINE);
+	cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }
 
 static void mx_cpu_start(void *p)
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -13,6 +13,7 @@ enum cpuhp_state {
 	CPUHP_CPU_SET_ACTIVE,
 	CPUHP_KICK_AP_THREAD,
 	CPUHP_BP_ONLINE,
+	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
 	CPUHP_AP_ONLINE_DYN,

[patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up

[Thomas Gleixner]

[-- Attachment #1: hotplug--Let-upcoming-cpu-bring-itself-fully-up.patch --]
[-- Type: text/plain, Size: 6415 bytes --]

Let the upcoming cpu kick the hotplug thread and let itself complete the
bringup. That way the controll side can just wait for the completion or later
when we made the hotplug machinery async not care at all.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    9 ++++--
 kernel/cpu.c               |   66 +++++++++++++++++++++++++--------------------
 kernel/sched/idle.c        |    2 +
 3 files changed, 45 insertions(+), 32 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,9 +10,6 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
-	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_KICK_AP_THREAD,
-	CPUHP_BP_ONLINE,
 	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
@@ -86,4 +83,10 @@ static inline void cpuhp_remove_state_no
 	__cpuhp_remove_state(state, false);
 }
 
+#ifdef CONFIG_SMP
+void cpuhp_online_idle(enum cpuhp_state state);
+#else
+static inline void cpuhp_online_idle(enum cpuhp_state state) { }
+#endif
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,6 +329,14 @@ static int notify_starting(unsigned int
 	return 0;
 }
 
+static int bringup_wait_for_ap(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -340,8 +348,9 @@ static int bringup_cpu(unsigned int cpu)
 		cpu_notify(CPU_UP_CANCELED, cpu);
 		return ret;
 	}
+	ret = bringup_wait_for_ap(cpu);
 	BUG_ON(!cpu_online(cpu));
-	return 0;
+	return ret;
 }
 
 /*
@@ -470,7 +479,7 @@ static void cpuhp_thread_fun(unsigned in
 		}
 	} else {
 		/* Cannot happen .... */
-		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+		BUG_ON(st->state < CPUHP_AP_ONLINE_IDLE);
 
 		/* Regular hotplug work */
 		if (st->state < st->target)
@@ -780,7 +789,7 @@ static int __ref _cpu_down(unsigned int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_TEARDOWN_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -793,11 +802,11 @@ static int __ref _cpu_down(unsigned int
 		 * We might have stopped still in the range of the AP hotplug
 		 * thread. Nothing to do anymore.
 		 */
-		if (st->state >= CPUHP_KICK_AP_THREAD)
+		if (st->state > CPUHP_TEARDOWN_CPU)
 			goto out;
 	}
 	/*
-	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * The AP brought itself down to CPUHP_TEARDOWN_CPU. So we need
 	 * to do the further cleanups.
 	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
@@ -859,18 +868,32 @@ void notify_cpu_starting(unsigned int cp
 
 /*
  * Called from the idle task. We need to set active here, so we can kick off
- * the stopper thread.
+ * the stopper thread and unpark the smpboot threads. If the target state is
+ * beyond CPUHP_AP_ONLINE_IDLE we kick cpuhp thread and let it bring up the
+ * cpu further.
  */
-static int cpuhp_set_cpu_active(unsigned int cpu)
+void cpuhp_online_idle(enum cpuhp_state state)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	unsigned int cpu = smp_processor_id();
+
+	/* Happens for the boot cpu */
+	if (state != CPUHP_AP_ONLINE_IDLE)
+		return;
+
+	st->state = CPUHP_AP_ONLINE_IDLE;
 
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread and the hotplug thread */
+	/* Unpark the stopper thread and the hotplug thread of this cpu */
 	stop_machine_unpark(cpu);
 	kthread_unpark(st->thread);
-	return 0;
+
+	/* Should we go further up ? */
+	if (st->target > CPUHP_AP_ONLINE_IDLE)
+		__cpuhp_kick_ap_work(st);
+	else
+		complete(&st->done);
 }
 
 /* Requires cpu_add_remove_lock to be held */
@@ -910,7 +933,7 @@ static int _cpu_up(unsigned int cpu, int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread once more.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_BRINGUP_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -922,10 +945,10 @@ static int _cpu_up(unsigned int cpu, int
 
 	/*
 	 * Try to reach the target state. We max out on the BP at
-	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * CPUHP_BRINGUP_CPU. After that the AP hotplug thread is
 	 * responsible for bringing it up to the target state.
 	 */
-	target = min((int)target, CPUHP_KICK_AP_THREAD);
+	target = min((int)target, CPUHP_BRINGUP_CPU);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1146,22 +1169,7 @@ static struct cpuhp_step cpuhp_bp_states
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
-	[CPUHP_CPU_SET_ACTIVE] = {
-		.name			= "cpu:active",
-		.startup		= cpuhp_set_cpu_active,
-		.teardown		= NULL,
-	},
-	[CPUHP_KICK_AP_THREAD] = {
-		.name			= "cpuhp:kickthread",
-		.startup		= cpuhp_kick_ap_work,
-		.teardown		= cpuhp_kick_ap_work,
-	},
 #endif
-	[CPUHP_BP_ONLINE] = {
-		.name			= "online",
-		.startup		= NULL,
-		.teardown		= NULL,
-	},
 };
 
 /* Application processor state steps */
@@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp
 {
 	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
 		return true;
-	return state > CPUHP_BP_ONLINE;
+	return state > CPUHP_BRINGUP_CPU;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -4,6 +4,7 @@
 #include <linux/sched.h>
 #include <linux/cpu.h>
 #include <linux/cpuidle.h>
+#include <linux/cpuhotplug.h>
 #include <linux/tick.h>
 #include <linux/mm.h>
 #include <linux/stackprotector.h>
@@ -276,5 +277,6 @@ static void cpu_idle_loop(void)
 void cpu_startup_entry(enum cpuhp_state state)
 {
 	arch_cpu_idle_prepare();
+	cpuhp_online_idle(state);
 	cpu_idle_loop();
 }

[patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up

[Thomas Gleixner]

[-- Attachment #1: hotplug--Let-upcoming-cpu-bring-itself-fully-up.patch --]
[-- Type: text/plain, Size: 6417 bytes --]

Let the upcoming cpu kick the hotplug thread and let itself complete the
bringup. That way the controll side can just wait for the completion or later
when we made the hotplug machinery async not care at all.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpuhotplug.h |    9 ++++--
 kernel/cpu.c               |   66 +++++++++++++++++++++++++--------------------
 kernel/sched/idle.c        |    2 +
 3 files changed, 45 insertions(+), 32 deletions(-)

Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -10,9 +10,6 @@ enum cpuhp_state {
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
 	CPUHP_TEARDOWN_CPU,
-	CPUHP_CPU_SET_ACTIVE,
-	CPUHP_KICK_AP_THREAD,
-	CPUHP_BP_ONLINE,
 	CPUHP_AP_ONLINE_IDLE,
 	CPUHP_AP_SMPBOOT_THREADS,
 	CPUHP_AP_NOTIFY_ONLINE,
@@ -86,4 +83,10 @@ static inline void cpuhp_remove_state_no
 	__cpuhp_remove_state(state, false);
 }
 
+#ifdef CONFIG_SMP
+void cpuhp_online_idle(enum cpuhp_state state);
+#else
+static inline void cpuhp_online_idle(enum cpuhp_state state) { }
+#endif
+
 #endif
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -329,6 +329,14 @@ static int notify_starting(unsigned int
 	return 0;
 }
 
+static int bringup_wait_for_ap(unsigned int cpu)
+{
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+	wait_for_completion(&st->done);
+	return st->result;
+}
+
 static int bringup_cpu(unsigned int cpu)
 {
 	struct task_struct *idle = idle_thread_get(cpu);
@@ -340,8 +348,9 @@ static int bringup_cpu(unsigned int cpu)
 		cpu_notify(CPU_UP_CANCELED, cpu);
 		return ret;
 	}
+	ret = bringup_wait_for_ap(cpu);
 	BUG_ON(!cpu_online(cpu));
-	return 0;
+	return ret;
 }
 
 /*
@@ -470,7 +479,7 @@ static void cpuhp_thread_fun(unsigned in
 		}
 	} else {
 		/* Cannot happen .... */
-		BUG_ON(st->state < CPUHP_KICK_AP_THREAD);
+		BUG_ON(st->state < CPUHP_AP_ONLINE_IDLE);
 
 		/* Regular hotplug work */
 		if (st->state < st->target)
@@ -780,7 +789,7 @@ static int __ref _cpu_down(unsigned int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_TEARDOWN_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -793,11 +802,11 @@ static int __ref _cpu_down(unsigned int
 		 * We might have stopped still in the range of the AP hotplug
 		 * thread. Nothing to do anymore.
 		 */
-		if (st->state >= CPUHP_KICK_AP_THREAD)
+		if (st->state > CPUHP_TEARDOWN_CPU)
 			goto out;
 	}
 	/*
-	 * The AP brought itself down below CPUHP_KICK_AP_THREAD. So we need
+	 * The AP brought itself down to CPUHP_TEARDOWN_CPU. So we need
 	 * to do the further cleanups.
 	 */
 	ret = cpuhp_down_callbacks(cpu, st, cpuhp_bp_states, target);
@@ -859,18 +868,32 @@ void notify_cpu_starting(unsigned int cp
 
 /*
  * Called from the idle task. We need to set active here, so we can kick off
- * the stopper thread.
+ * the stopper thread and unpark the smpboot threads. If the target state is
+ * beyond CPUHP_AP_ONLINE_IDLE we kick cpuhp thread and let it bring up the
+ * cpu further.
  */
-static int cpuhp_set_cpu_active(unsigned int cpu)
+void cpuhp_online_idle(enum cpuhp_state state)
 {
-	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+	unsigned int cpu = smp_processor_id();
+
+	/* Happens for the boot cpu */
+	if (state != CPUHP_AP_ONLINE_IDLE)
+		return;
+
+	st->state = CPUHP_AP_ONLINE_IDLE;
 
 	/* The cpu is marked online, set it active now */
 	set_cpu_active(cpu, true);
-	/* Unpark the stopper thread and the hotplug thread */
+	/* Unpark the stopper thread and the hotplug thread of this cpu */
 	stop_machine_unpark(cpu);
 	kthread_unpark(st->thread);
-	return 0;
+
+	/* Should we go further up ? */
+	if (st->target > CPUHP_AP_ONLINE_IDLE)
+		__cpuhp_kick_ap_work(st);
+	else
+		complete(&st->done);
 }
 
 /* Requires cpu_add_remove_lock to be held */
@@ -910,7 +933,7 @@ static int _cpu_up(unsigned int cpu, int
 	 * If the current CPU state is in the range of the AP hotplug thread,
 	 * then we need to kick the thread once more.
 	 */
-	if (st->state >= CPUHP_KICK_AP_THREAD) {
+	if (st->state > CPUHP_BRINGUP_CPU) {
 		ret = cpuhp_kick_ap_work(cpu);
 		/*
 		 * The AP side has done the error rollback already. Just
@@ -922,10 +945,10 @@ static int _cpu_up(unsigned int cpu, int
 
 	/*
 	 * Try to reach the target state. We max out on the BP at
-	 * CPUHP_KICK_AP_THREAD. After that the AP hotplug thread is
+	 * CPUHP_BRINGUP_CPU. After that the AP hotplug thread is
 	 * responsible for bringing it up to the target state.
 	 */
-	target = min((int)target, CPUHP_KICK_AP_THREAD);
+	target = min((int)target, CPUHP_BRINGUP_CPU);
 	ret = cpuhp_up_callbacks(cpu, st, cpuhp_bp_states, target);
 out:
 	cpu_hotplug_done();
@@ -1146,22 +1169,7 @@ static struct cpuhp_step cpuhp_bp_states
 		.teardown		= takedown_cpu,
 		.cant_stop		= true,
 	},
-	[CPUHP_CPU_SET_ACTIVE] = {
-		.name			= "cpu:active",
-		.startup		= cpuhp_set_cpu_active,
-		.teardown		= NULL,
-	},
-	[CPUHP_KICK_AP_THREAD] = {
-		.name			= "cpuhp:kickthread",
-		.startup		= cpuhp_kick_ap_work,
-		.teardown		= cpuhp_kick_ap_work,
-	},
 #endif
-	[CPUHP_BP_ONLINE] = {
-		.name			= "online",
-		.startup		= NULL,
-		.teardown		= NULL,
-	},
 };
 
 /* Application processor state steps */
@@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp
 {
 	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
 		return true;
-	return state > CPUHP_BP_ONLINE;
+	return state > CPUHP_BRINGUP_CPU;
 }
 
 static struct cpuhp_step *cpuhp_get_step(enum cpuhp_state state)
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -4,6 +4,7 @@
 #include <linux/sched.h>
 #include <linux/cpu.h>
 #include <linux/cpuidle.h>
+#include <linux/cpuhotplug.h>
 #include <linux/tick.h>
 #include <linux/mm.h>
 #include <linux/stackprotector.h>
@@ -276,5 +277,6 @@ static void cpu_idle_loop(void)
 void cpu_startup_entry(enum cpuhp_state state)
 {
 	arch_cpu_idle_prepare();
+	cpuhp_online_idle(state);
 	cpu_idle_loop();
 }

[patch 19/20] cpu/hotplug: Make wait for dead cpu completion based

[Thomas Gleixner]

[-- Attachment #1: hotplug--Make-wait-for-dead-cpu-completion-based.patch --]
[-- Type: text/plain, Size: 3225 bytes --]

Kill the busy spinning on the control side and just wait for the hotplugged
cpu to tell that it reached the dead state.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    5 +++--
 include/linux/cpuhotplug.h |    1 +
 kernel/cpu.c               |   16 ++++++++++++----
 kernel/sched/idle.c        |    5 +----
 4 files changed, 17 insertions(+), 10 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -276,14 +276,15 @@ void arch_cpu_idle_enter(void);
 void arch_cpu_idle_exit(void);
 void arch_cpu_idle_dead(void);
 
-DECLARE_PER_CPU(bool, cpu_dead_idle);
-
 int cpu_report_state(int cpu);
 int cpu_check_up_prepare(int cpu);
 void cpu_set_state_online(int cpu);
 #ifdef CONFIG_HOTPLUG_CPU
 bool cpu_wait_death(unsigned int cpu, int seconds);
 bool cpu_report_death(void);
+void cpuhp_report_idle_dead(void);
+#else
+static inline void cpuhp_report_idle_dead(void) { }
 #endif /* #ifdef CONFIG_HOTPLUG_CPU */
 
 #endif /* _LINUX_CPU_H_ */
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,7 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -688,6 +688,7 @@ static int take_cpu_down(void *_param)
 
 static int takedown_cpu(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int err;
 
 	/*
@@ -733,10 +734,8 @@ static int takedown_cpu(unsigned int cpu
 	 *
 	 * Wait for the stop thread to go away.
 	 */
-	while (!per_cpu(cpu_dead_idle, cpu))
-		cpu_relax();
-	smp_mb(); /* Read from cpu_dead_idle before __cpu_die(). */
-	per_cpu(cpu_dead_idle, cpu) = false;
+	wait_for_completion(&st->done);
+	BUG_ON(st->state != CPUHP_AP_IDLE_DEAD);
 
 	/* Interrupts are moved away from the dying cpu, reenable alloc/free */
 	irq_unlock_sparse();
@@ -756,6 +755,15 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+void cpuhp_report_idle_dead(void)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	BUG_ON(st->state != CPUHP_AP_OFFLINE);
+	st->state = CPUHP_AP_IDLE_DEAD;
+	complete(&st->done);
+}
+
 #else
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -194,8 +194,6 @@ static void cpuidle_idle_call(void)
 	rcu_idle_exit();
 }
 
-DEFINE_PER_CPU(bool, cpu_dead_idle);
-
 /*
  * Generic idle loop implementation
  *
@@ -224,8 +222,7 @@ static void cpu_idle_loop(void)
 			if (cpu_is_offline(smp_processor_id())) {
 				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
 					       (void *)(long)smp_processor_id());
-				smp_mb(); /* all activity before dead. */
-				this_cpu_write(cpu_dead_idle, true);
+				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}
 

[patch 19/20] cpu/hotplug: Make wait for dead cpu completion based

[Thomas Gleixner]

[-- Attachment #1: hotplug--Make-wait-for-dead-cpu-completion-based.patch --]
[-- Type: text/plain, Size: 3227 bytes --]

Kill the busy spinning on the control side and just wait for the hotplugged
cpu to tell that it reached the dead state.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h        |    5 +++--
 include/linux/cpuhotplug.h |    1 +
 kernel/cpu.c               |   16 ++++++++++++----
 kernel/sched/idle.c        |    5 +----
 4 files changed, 17 insertions(+), 10 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -276,14 +276,15 @@ void arch_cpu_idle_enter(void);
 void arch_cpu_idle_exit(void);
 void arch_cpu_idle_dead(void);
 
-DECLARE_PER_CPU(bool, cpu_dead_idle);
-
 int cpu_report_state(int cpu);
 int cpu_check_up_prepare(int cpu);
 void cpu_set_state_online(int cpu);
 #ifdef CONFIG_HOTPLUG_CPU
 bool cpu_wait_death(unsigned int cpu, int seconds);
 bool cpu_report_death(void);
+void cpuhp_report_idle_dead(void);
+#else
+static inline void cpuhp_report_idle_dead(void) { }
 #endif /* #ifdef CONFIG_HOTPLUG_CPU */
 
 #endif /* _LINUX_CPU_H_ */
Index: b/include/linux/cpuhotplug.h
===================================================================
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -6,6 +6,7 @@ enum cpuhp_state {
 	CPUHP_CREATE_THREADS,
 	CPUHP_NOTIFY_PREPARE,
 	CPUHP_BRINGUP_CPU,
+	CPUHP_AP_IDLE_DEAD,
 	CPUHP_AP_OFFLINE,
 	CPUHP_AP_NOTIFY_STARTING,
 	CPUHP_AP_ONLINE,
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -688,6 +688,7 @@ static int take_cpu_down(void *_param)
 
 static int takedown_cpu(unsigned int cpu)
 {
+	struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
 	int err;
 
 	/*
@@ -733,10 +734,8 @@ static int takedown_cpu(unsigned int cpu
 	 *
 	 * Wait for the stop thread to go away.
 	 */
-	while (!per_cpu(cpu_dead_idle, cpu))
-		cpu_relax();
-	smp_mb(); /* Read from cpu_dead_idle before __cpu_die(). */
-	per_cpu(cpu_dead_idle, cpu) = false;
+	wait_for_completion(&st->done);
+	BUG_ON(st->state != CPUHP_AP_IDLE_DEAD);
 
 	/* Interrupts are moved away from the dying cpu, reenable alloc/free */
 	irq_unlock_sparse();
@@ -756,6 +755,15 @@ static int notify_dead(unsigned int cpu)
 	return 0;
 }
 
+void cpuhp_report_idle_dead(void)
+{
+	struct cpuhp_cpu_state *st = this_cpu_ptr(&cpuhp_state);
+
+	BUG_ON(st->state != CPUHP_AP_OFFLINE);
+	st->state = CPUHP_AP_IDLE_DEAD;
+	complete(&st->done);
+}
+
 #else
 #define notify_down_prepare	NULL
 #define takedown_cpu		NULL
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -194,8 +194,6 @@ static void cpuidle_idle_call(void)
 	rcu_idle_exit();
 }
 
-DEFINE_PER_CPU(bool, cpu_dead_idle);
-
 /*
  * Generic idle loop implementation
  *
@@ -224,8 +222,7 @@ static void cpu_idle_loop(void)
 			if (cpu_is_offline(smp_processor_id())) {
 				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
 					       (void *)(long)smp_processor_id());
-				smp_mb(); /* all activity before dead. */
-				this_cpu_write(cpu_dead_idle, true);
+				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}
 

[patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Thomas Gleixner]

[-- Attachment #1: rcu--Make-CPU_DYING_IDLE-an-explicit-call.patch --]
[-- Type: text/plain, Size: 3849 bytes --]

Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
invoked at the proper place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h      |    4 +---
 include/linux/notifier.h |    2 ++
 include/linux/rcupdate.h |    4 +---
 kernel/cpu.c             |    1 +
 kernel/rcu/tree.c        |   26 +++++++++++++++-----------
 kernel/sched/idle.c      |    2 --
 6 files changed, 20 insertions(+), 19 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -101,9 +101,7 @@ enum {
 					* Called on the new cpu, just before
 					* enabling interrupts. Must not sleep,
 					* must not fail */
-#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
-					* idle loop. */
-#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
+#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
 					* perhaps due to preemption. */
 
 /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
Index: b/include/linux/notifier.h
===================================================================
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -47,6 +47,8 @@
  * runtime initialization.
  */
 
+struct notifier_block;
+
 typedef	int (*notifier_fn_t)(struct notifier_block *nb,
 			unsigned long action, void *data);
 
Index: b/include/linux/rcupdate.h
===================================================================
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -332,9 +332,7 @@ void rcu_init(void);
 void rcu_sched_qs(void);
 void rcu_bh_qs(void);
 void rcu_check_callbacks(int user);
-struct notifier_block;
-int rcu_cpu_notify(struct notifier_block *self,
-		   unsigned long action, void *hcpu);
+void rcu_report_dead(unsigned int cpu);
 
 #ifndef CONFIG_TINY_RCU
 void rcu_end_inkernel_boot(void);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
 	BUG_ON(st->state != CPUHP_AP_OFFLINE);
 	st->state = CPUHP_AP_IDLE_DEAD;
 	complete(&st->done);
+	rcu_report_dead(smp_processor_id());
 }
 
 #else
Index: b/kernel/rcu/tree.c
===================================================================
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
 		rcu_init_percpu_data(cpu, rsp);
 }
 
+#ifdef CONFIG_HOTPLUG_CPU
+void rcu_report_dead(unsigned int cpu)
+{
+	struct rcu_state *rsp;
+
+	/* QS for any half-done expedited RCU-sched GP. */
+	preempt_disable();
+	rcu_report_exp_rdp(&rcu_sched_state,
+			   this_cpu_ptr(rcu_sched_state.rda), true);
+	preempt_enable();
+	for_each_rcu_flavor(rsp)
+		rcu_cleanup_dying_idle_cpu(cpu, rsp);
+}
+#endif
+
 /*
  * Handle CPU online/offline notification events.
  */
@@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dying_cpu(rsp);
 		break;
-	case CPU_DYING_IDLE:
-		/* QS for any half-done expedited RCU-sched GP. */
-		preempt_disable();
-		rcu_report_exp_rdp(&rcu_sched_state,
-				   this_cpu_ptr(rcu_sched_state.rda), true);
-		preempt_enable();
-
-		for_each_rcu_flavor(rsp) {
-			rcu_cleanup_dying_idle_cpu(cpu, rsp);
-		}
-		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
 			rmb();
 
 			if (cpu_is_offline(smp_processor_id())) {
-				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
-					       (void *)(long)smp_processor_id());
 				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}

[patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Thomas Gleixner]

[-- Attachment #1: rcu--Make-CPU_DYING_IDLE-an-explicit-call.patch --]
[-- Type: text/plain, Size: 3851 bytes --]

Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
invoked at the proper place.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
 include/linux/cpu.h      |    4 +---
 include/linux/notifier.h |    2 ++
 include/linux/rcupdate.h |    4 +---
 kernel/cpu.c             |    1 +
 kernel/rcu/tree.c        |   26 +++++++++++++++-----------
 kernel/sched/idle.c      |    2 --
 6 files changed, 20 insertions(+), 19 deletions(-)

Index: b/include/linux/cpu.h
===================================================================
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -101,9 +101,7 @@ enum {
 					* Called on the new cpu, just before
 					* enabling interrupts. Must not sleep,
 					* must not fail */
-#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
-					* idle loop. */
-#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
+#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
 					* perhaps due to preemption. */
 
 /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
Index: b/include/linux/notifier.h
===================================================================
--- a/include/linux/notifier.h
+++ b/include/linux/notifier.h
@@ -47,6 +47,8 @@
  * runtime initialization.
  */
 
+struct notifier_block;
+
 typedef	int (*notifier_fn_t)(struct notifier_block *nb,
 			unsigned long action, void *data);
 
Index: b/include/linux/rcupdate.h
===================================================================
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -332,9 +332,7 @@ void rcu_init(void);
 void rcu_sched_qs(void);
 void rcu_bh_qs(void);
 void rcu_check_callbacks(int user);
-struct notifier_block;
-int rcu_cpu_notify(struct notifier_block *self,
-		   unsigned long action, void *hcpu);
+void rcu_report_dead(unsigned int cpu);
 
 #ifndef CONFIG_TINY_RCU
 void rcu_end_inkernel_boot(void);
Index: b/kernel/cpu.c
===================================================================
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
 	BUG_ON(st->state != CPUHP_AP_OFFLINE);
 	st->state = CPUHP_AP_IDLE_DEAD;
 	complete(&st->done);
+	rcu_report_dead(smp_processor_id());
 }
 
 #else
Index: b/kernel/rcu/tree.c
===================================================================
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
 		rcu_init_percpu_data(cpu, rsp);
 }
 
+#ifdef CONFIG_HOTPLUG_CPU
+void rcu_report_dead(unsigned int cpu)
+{
+	struct rcu_state *rsp;
+
+	/* QS for any half-done expedited RCU-sched GP. */
+	preempt_disable();
+	rcu_report_exp_rdp(&rcu_sched_state,
+			   this_cpu_ptr(rcu_sched_state.rda), true);
+	preempt_enable();
+	for_each_rcu_flavor(rsp)
+		rcu_cleanup_dying_idle_cpu(cpu, rsp);
+}
+#endif
+
 /*
  * Handle CPU online/offline notification events.
  */
@@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
 		for_each_rcu_flavor(rsp)
 			rcu_cleanup_dying_cpu(rsp);
 		break;
-	case CPU_DYING_IDLE:
-		/* QS for any half-done expedited RCU-sched GP. */
-		preempt_disable();
-		rcu_report_exp_rdp(&rcu_sched_state,
-				   this_cpu_ptr(rcu_sched_state.rda), true);
-		preempt_enable();
-
-		for_each_rcu_flavor(rsp) {
-			rcu_cleanup_dying_idle_cpu(cpu, rsp);
-		}
-		break;
 	case CPU_DEAD:
 	case CPU_DEAD_FROZEN:
 	case CPU_UP_CANCELED:
Index: b/kernel/sched/idle.c
===================================================================
--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
 			rmb();
 
 			if (cpu_is_offline(smp_processor_id())) {
-				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
-					       (void *)(long)smp_processor_id());
 				cpuhp_report_idle_dead();
 				arch_cpu_idle_dead();
 			}

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Rafael J. Wysocki]

Hi Thomas,

On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> Make it possible to write a target state to the per cpu state file, so we can
> switch between states.

One thing that potentially may be problematic here is that any kind of
"offline" operations needs to be carried out under device_hotplug_lock,
because there are cases in which devices (including CPUs) are taken
offline in groups and if one offline fails, the whole operation has to
be rolled back.

So if you put a CPU into one of the intermediate states manually and
something like the above happens in parallel with it, they may not
play well together IMO.

Thanks,
Rafael

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Paul E. McKenney]

On Fri, Feb 26, 2016 at 06:43:44PM -0000, Thomas Gleixner wrote:
> Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
> invoked at the proper place.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

A question below...

> ---
>  include/linux/cpu.h      |    4 +---
>  include/linux/notifier.h |    2 ++
>  include/linux/rcupdate.h |    4 +---
>  kernel/cpu.c             |    1 +
>  kernel/rcu/tree.c        |   26 +++++++++++++++-----------
>  kernel/sched/idle.c      |    2 --
>  6 files changed, 20 insertions(+), 19 deletions(-)
> 
> Index: b/include/linux/cpu.h
> ===================================================================
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -101,9 +101,7 @@ enum {
>  					* Called on the new cpu, just before
>  					* enabling interrupts. Must not sleep,
>  					* must not fail */
> -#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
> -					* idle loop. */
> -#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
> +#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
>  					* perhaps due to preemption. */
> 
>  /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
> Index: b/include/linux/notifier.h
> ===================================================================
> --- a/include/linux/notifier.h
> +++ b/include/linux/notifier.h
> @@ -47,6 +47,8 @@
>   * runtime initialization.
>   */
> 
> +struct notifier_block;
> +
>  typedef	int (*notifier_fn_t)(struct notifier_block *nb,
>  			unsigned long action, void *data);
> 
> Index: b/include/linux/rcupdate.h
> ===================================================================
> --- a/include/linux/rcupdate.h
> +++ b/include/linux/rcupdate.h
> @@ -332,9 +332,7 @@ void rcu_init(void);
>  void rcu_sched_qs(void);
>  void rcu_bh_qs(void);
>  void rcu_check_callbacks(int user);
> -struct notifier_block;
> -int rcu_cpu_notify(struct notifier_block *self,
> -		   unsigned long action, void *hcpu);
> +void rcu_report_dead(unsigned int cpu);
> 
>  #ifndef CONFIG_TINY_RCU
>  void rcu_end_inkernel_boot(void);
> Index: b/kernel/cpu.c
> ===================================================================
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
>  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
>  	st->state = CPUHP_AP_IDLE_DEAD;
>  	complete(&st->done);

What prevents the other CPU from killing this CPU at this point, so
that this CPU does not tell RCU that it is dead?

I agree that the odds should be low, but there are all manner of things
that might delay a CPU for just a little bit too long...

Or am I missing something subtle here?

							Thanx, Paul

> +	rcu_report_dead(smp_processor_id());
>  }
> 
>  #else
> Index: b/kernel/rcu/tree.c
> ===================================================================
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
>  		rcu_init_percpu_data(cpu, rsp);
>  }
> 
> +#ifdef CONFIG_HOTPLUG_CPU
> +void rcu_report_dead(unsigned int cpu)
> +{
> +	struct rcu_state *rsp;
> +
> +	/* QS for any half-done expedited RCU-sched GP. */
> +	preempt_disable();
> +	rcu_report_exp_rdp(&rcu_sched_state,
> +			   this_cpu_ptr(rcu_sched_state.rda), true);
> +	preempt_enable();
> +	for_each_rcu_flavor(rsp)
> +		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> +}
> +#endif
> +
>  /*
>   * Handle CPU online/offline notification events.
>   */
> @@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
>  		for_each_rcu_flavor(rsp)
>  			rcu_cleanup_dying_cpu(rsp);
>  		break;
> -	case CPU_DYING_IDLE:
> -		/* QS for any half-done expedited RCU-sched GP. */
> -		preempt_disable();
> -		rcu_report_exp_rdp(&rcu_sched_state,
> -				   this_cpu_ptr(rcu_sched_state.rda), true);
> -		preempt_enable();
> -
> -		for_each_rcu_flavor(rsp) {
> -			rcu_cleanup_dying_idle_cpu(cpu, rsp);
> -		}
> -		break;
>  	case CPU_DEAD:
>  	case CPU_DEAD_FROZEN:
>  	case CPU_UP_CANCELED:
> Index: b/kernel/sched/idle.c
> ===================================================================
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
>  			rmb();
> 
>  			if (cpu_is_offline(smp_processor_id())) {
> -				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
> -					       (void *)(long)smp_processor_id());
>  				cpuhp_report_idle_dead();
>  				arch_cpu_idle_dead();
>  			}
> 
> 

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Paul E. McKenney]

On Fri, Feb 26, 2016 at 06:14:29PM -0800, Paul E. McKenney wrote:
> On Fri, Feb 26, 2016 at 06:43:44PM -0000, Thomas Gleixner wrote:
> > Make the RCU CPU_DYING_IDLE callback an explicit function call, so it gets
> > invoked at the proper place.
> > 
> > Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> 
> A question below...
> 
> > ---
> >  include/linux/cpu.h      |    4 +---
> >  include/linux/notifier.h |    2 ++
> >  include/linux/rcupdate.h |    4 +---
> >  kernel/cpu.c             |    1 +
> >  kernel/rcu/tree.c        |   26 +++++++++++++++-----------
> >  kernel/sched/idle.c      |    2 --
> >  6 files changed, 20 insertions(+), 19 deletions(-)
> > 
> > Index: b/include/linux/cpu.h
> > ===================================================================
> > --- a/include/linux/cpu.h
> > +++ b/include/linux/cpu.h
> > @@ -101,9 +101,7 @@ enum {
> >  					* Called on the new cpu, just before
> >  					* enabling interrupts. Must not sleep,
> >  					* must not fail */
> > -#define CPU_DYING_IDLE		0x000B /* CPU (unsigned)v dying, reached
> > -					* idle loop. */
> > -#define CPU_BROKEN		0x000C /* CPU (unsigned)v did not die properly,
> > +#define CPU_BROKEN		0x000B /* CPU (unsigned)v did not die properly,
> >  					* perhaps due to preemption. */
> > 
> >  /* Used for CPU hotplug events occurring while tasks are frozen due to a suspend
> > Index: b/include/linux/notifier.h
> > ===================================================================
> > --- a/include/linux/notifier.h
> > +++ b/include/linux/notifier.h
> > @@ -47,6 +47,8 @@
> >   * runtime initialization.
> >   */
> > 
> > +struct notifier_block;
> > +
> >  typedef	int (*notifier_fn_t)(struct notifier_block *nb,
> >  			unsigned long action, void *data);
> > 
> > Index: b/include/linux/rcupdate.h
> > ===================================================================
> > --- a/include/linux/rcupdate.h
> > +++ b/include/linux/rcupdate.h
> > @@ -332,9 +332,7 @@ void rcu_init(void);
> >  void rcu_sched_qs(void);
> >  void rcu_bh_qs(void);
> >  void rcu_check_callbacks(int user);
> > -struct notifier_block;
> > -int rcu_cpu_notify(struct notifier_block *self,
> > -		   unsigned long action, void *hcpu);
> > +void rcu_report_dead(unsigned int cpu);
> > 
> >  #ifndef CONFIG_TINY_RCU
> >  void rcu_end_inkernel_boot(void);
> > Index: b/kernel/cpu.c
> > ===================================================================
> > --- a/kernel/cpu.c
> > +++ b/kernel/cpu.c
> > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> >  	st->state = CPUHP_AP_IDLE_DEAD;
> >  	complete(&st->done);
> 
> What prevents the other CPU from killing this CPU at this point, so
> that this CPU does not tell RCU that it is dead?
> 
> I agree that the odds should be low, but there are all manner of things
> that might delay a CPU for just a little bit too long...
> 
> Or am I missing something subtle here?

Just in case I am not missing anything...

One approach is to go back to the spinning, but to do rcu_report_dead()
just before kicking the other CPU.  This would also fix some issues with
use of RCU of the offline path, so would definitely be better than my
earlier approach of notifying RCU from within the idle loop.

This assumes that all the offline paths have been consolidated into
this path.  (Yes, I was too lazy and cowardly to consolidate them all
last I touched this code, but perhaps that has happened elsewise?)

							Thanx, Paul

> > +	rcu_report_dead(smp_processor_id());
> >  }
> > 
> >  #else
> > Index: b/kernel/rcu/tree.c
> > ===================================================================
> > --- a/kernel/rcu/tree.c
> > +++ b/kernel/rcu/tree.c
> > @@ -4247,6 +4247,21 @@ static void rcu_prepare_cpu(int cpu)
> >  		rcu_init_percpu_data(cpu, rsp);
> >  }
> > 
> > +#ifdef CONFIG_HOTPLUG_CPU
> > +void rcu_report_dead(unsigned int cpu)
> > +{
> > +	struct rcu_state *rsp;
> > +
> > +	/* QS for any half-done expedited RCU-sched GP. */
> > +	preempt_disable();
> > +	rcu_report_exp_rdp(&rcu_sched_state,
> > +			   this_cpu_ptr(rcu_sched_state.rda), true);
> > +	preempt_enable();
> > +	for_each_rcu_flavor(rsp)
> > +		rcu_cleanup_dying_idle_cpu(cpu, rsp);
> > +}
> > +#endif
> > +
> >  /*
> >   * Handle CPU online/offline notification events.
> >   */
> > @@ -4278,17 +4293,6 @@ int rcu_cpu_notify(struct notifier_block
> >  		for_each_rcu_flavor(rsp)
> >  			rcu_cleanup_dying_cpu(rsp);
> >  		break;
> > -	case CPU_DYING_IDLE:
> > -		/* QS for any half-done expedited RCU-sched GP. */
> > -		preempt_disable();
> > -		rcu_report_exp_rdp(&rcu_sched_state,
> > -				   this_cpu_ptr(rcu_sched_state.rda), true);
> > -		preempt_enable();
> > -
> > -		for_each_rcu_flavor(rsp) {
> > -			rcu_cleanup_dying_idle_cpu(cpu, rsp);
> > -		}
> > -		break;
> >  	case CPU_DEAD:
> >  	case CPU_DEAD_FROZEN:
> >  	case CPU_UP_CANCELED:
> > Index: b/kernel/sched/idle.c
> > ===================================================================
> > --- a/kernel/sched/idle.c
> > +++ b/kernel/sched/idle.c
> > @@ -220,8 +220,6 @@ static void cpu_idle_loop(void)
> >  			rmb();
> > 
> >  			if (cpu_is_offline(smp_processor_id())) {
> > -				rcu_cpu_notify(NULL, CPU_DYING_IDLE,
> > -					       (void *)(long)smp_processor_id());
> >  				cpuhp_report_idle_dead();
> >  				arch_cpu_idle_dead();
> >  			}
> > 
> > 

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

Rafael,

On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> > Make it possible to write a target state to the per cpu state file, so we can
> > switch between states.
> 
> One thing that potentially may be problematic here is that any kind of
> "offline" operations needs to be carried out under device_hotplug_lock,
> because there are cases in which devices (including CPUs) are taken
> offline in groups and if one offline fails, the whole operation has to
> be rolled back.
>
> So if you put a CPU into one of the intermediate states manually and
> something like the above happens in parallel with it, they may not
> play well together IMO.

I don't see how that is related. device_hotplug_lock is completely independent
of cpu hotplug today, unless I'm missing some magic connection here.

Physical CPU hotplug is a different story, but that's about bringing the cpus
into the system or taking them out. Sure, if you want to take one or more cpus
physically out, you have to bring them offline first. If you plug them in then
it's not necessarily related to actually bringing them online. That's a
different set of operations.

We surely need to look into that aspect, but I don't see a reason why e.g. a
device hotplug operation should be in any way related to the intermediate
state of a particular cpu. If that's the case, then there is something really
wrong.

I'm aware that we have a gazillion of silly assumptions all over the place and
some of them are wrong today and just do not explode in our face simply
because it's extremly hard to trigger. That's one reason why we need to go
through all the cpu notifier related sites and inspect them deeply.

Thanks,

	tglx

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Thomas Gleixner]

On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > --- a/kernel/cpu.c
> > > +++ b/kernel/cpu.c
> > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > >  	complete(&st->done);
> > 
> > What prevents the other CPU from killing this CPU at this point, so
> > that this CPU does not tell RCU that it is dead?
> >
> > I agree that the odds should be low, but there are all manner of things
> > that might delay a CPU for just a little bit too long...
> > 
> > Or am I missing something subtle here?

No. The reason why I moved the rcu call past the complete is, that otherwise
complete() complains about rcu being dead already. Hmm, but you are right. In
theory the other side could allow physical removal before it actually told rcu
that it's gone.

> Just in case I am not missing anything...
> 
> One approach is to go back to the spinning, but to do rcu_report_dead()
> just before kicking the other CPU.  This would also fix some issues with
> use of RCU of the offline path, so would definitely be better than my
> earlier approach of notifying RCU from within the idle loop.
> 
> This assumes that all the offline paths have been consolidated into
> this path.  (Yes, I was too lazy and cowardly to consolidate them all
> last I touched this code, but perhaps that has happened elsewise?)

The question is whether the rcu dead notification has to happen
instantaniously and needs to be done on the dead cpu. If we can avoid both,
then there is a very simple solution.

Thanks,

	tglx

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Paul E. McKenney]

On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > --- a/kernel/cpu.c
> > > > +++ b/kernel/cpu.c
> > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > >  	complete(&st->done);
> > > 
> > > What prevents the other CPU from killing this CPU at this point, so
> > > that this CPU does not tell RCU that it is dead?
> > >
> > > I agree that the odds should be low, but there are all manner of things
> > > that might delay a CPU for just a little bit too long...
> > > 
> > > Or am I missing something subtle here?
> 
> No. The reason why I moved the rcu call past the complete is, that otherwise
> complete() complains about rcu being dead already. Hmm, but you are right. In
> theory the other side could allow physical removal before it actually told rcu
> that it's gone.

There is one case where this is OK, and that is where the outgoing CPU
puts itself to sleep (or whatever) without help from the other CPU.

> > Just in case I am not missing anything...
> > 
> > One approach is to go back to the spinning, but to do rcu_report_dead()
> > just before kicking the other CPU.  This would also fix some issues with
> > use of RCU of the offline path, so would definitely be better than my
> > earlier approach of notifying RCU from within the idle loop.
> > 
> > This assumes that all the offline paths have been consolidated into
> > this path.  (Yes, I was too lazy and cowardly to consolidate them all
> > last I touched this code, but perhaps that has happened elsewise?)
> 
> The question is whether the rcu dead notification has to happen
> instantaniously and needs to be done on the dead cpu. If we can avoid both,
> then there is a very simple solution.

Hmmm...

The rcu_cleanup_dying_idle_cpu() can be invoked from the surviving CPU,
-as- -long- -as- nothing in the intervening code path waits for a grace
period.  The wakeup path itself had better not wait for a grace period,
of course.  The concern would be that the task running on the surviving
CPU might be waiting for a grace period before sleeping -- which used to
be possible due to the CPU-hotplug notifiers that it might be executing
before getting to RCU's CPU-hotplug notifiers.

The rcu_report_exp_rdp() is considerably more scary.  At first glance,
it looks OK, but I will need to stare at it for a bit.

Of course, if the task running on the surviving CPU can be waiting for a
grace period, one of two problems can happen:

o	RCU times out the outgoing CPU before it has really left.
	This is the current state, and needs to change.  The dying idle
	stuff was half of the needed change, the other half being on
	the incoming side.

o	Deadlock -- the outgoing CPU won't respond to RCU, so the
	task running on the surviving CPU never wakes up from its
	wait on a grace period.

So if rcu_report_exp_rdp() turns out to be OK -and- if the outgoing
task never waits on a grace period during the CPU-shutdown process,
this might work.

Of course, my ability to test this sufficiently viciously is currently
blocked by the lost-wakeup problem I am currently chasing.  (Hey, at
least I finally get ftrace output!  Completely baffling output, but so
it goes...)  :-/

							Thanx, Paul

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Thomas Gleixner]

On Sat, 27 Feb 2016, Paul E. McKenney wrote:
> On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> > On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > > --- a/kernel/cpu.c
> > > > > +++ b/kernel/cpu.c
> > > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > > >  	complete(&st->done);
> > > > 
> > > > What prevents the other CPU from killing this CPU at this point, so
> > > > that this CPU does not tell RCU that it is dead?
> > > >
> > > > I agree that the odds should be low, but there are all manner of things
> > > > that might delay a CPU for just a little bit too long...
> > > > 
> > > > Or am I missing something subtle here?
> > 
> > No. The reason why I moved the rcu call past the complete is, that otherwise
> > complete() complains about rcu being dead already. Hmm, but you are right. In
> > theory the other side could allow physical removal before it actually told rcu
> > that it's gone.
> 
> There is one case where this is OK, and that is where the outgoing CPU
> puts itself to sleep (or whatever) without help from the other CPU.

That's the case. It's the last call before the outgoing CPU goes into
arch_cpu_idle_dead(). There is no involvement of the controlling CPU at this
point. It just wants to know, that the outgoing one is dead finally.

Thanks,

	tglx

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Thomas Gleixner]

On Sat, 27 Feb 2016, Paul E. McKenney wrote:
> On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> > On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > > --- a/kernel/cpu.c
> > > > > +++ b/kernel/cpu.c
> > > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > > >  	complete(&st->done);
> > > > 
> > > > What prevents the other CPU from killing this CPU at this point, so
> > > > that this CPU does not tell RCU that it is dead?
> > > >
> > > > I agree that the odds should be low, but there are all manner of things
> > > > that might delay a CPU for just a little bit too long...
> > > > 
> > > > Or am I missing something subtle here?
> > 
> > No. The reason why I moved the rcu call past the complete is, that otherwise
> > complete() complains about rcu being dead already. Hmm, but you are right. In
> > theory the other side could allow physical removal before it actually told rcu
> > that it's gone.
> 
> There is one case where this is OK, and that is where the outgoing CPU
> puts itself to sleep (or whatever) without help from the other CPU.

That's the case. It's the last call before the outgoing CPU goes into
arch_cpu_idle_dead(). There is no involvement of the controlling CPU at this
point. It just wants to know, that the outgoing one is dead finally.

Thanks,

	tglx

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Rafael J. Wysocki]

On Saturday, February 27, 2016 08:39:42 AM Thomas Gleixner wrote:
> Rafael,
> 
> On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> > On Friday, February 26, 2016 06:43:32 PM Thomas Gleixner wrote:
> > > Make it possible to write a target state to the per cpu state file, so we can
> > > switch between states.
> > 
> > One thing that potentially may be problematic here is that any kind of
> > "offline" operations needs to be carried out under device_hotplug_lock,
> > because there are cases in which devices (including CPUs) are taken
> > offline in groups and if one offline fails, the whole operation has to
> > be rolled back.
> >
> > So if you put a CPU into one of the intermediate states manually and
> > something like the above happens in parallel with it, they may not
> > play well together IMO.
> 
> I don't see how that is related. device_hotplug_lock is completely independent
> of cpu hotplug today, unless I'm missing some magic connection here.

Well, there is a magic connection which is my point.  That's mostly about
physical hot-remove.

> Physical CPU hotplug is a different story, but that's about bringing the cpus
> into the system or taking them out. Sure, if you want to take one or more cpus
> physically out, you have to bring them offline first. If you plug them in then
> it's not necessarily related to actually bringing them online. That's a
> different set of operations.

So that's mostly about the hot-remove part.  Namely, devices may need to go
away together (like in one package), so we need to offline them together first.

That's because generally offline may fail, for example for memory, and now
if a CPU is bundled to a set of memory that cannot be taken offline, it
can't be hot-removed too.  If offline fails for one component, we roll back,
but if it is successful for all of them, we can eject the whole bundle and
that's where the problem resides.

Say we've taken all of them offline and now we are ready to eject.  If an
online from sysfs (or any other place) comes in at this point, we'll be
ejecting a CPU that's potentially doing something which is not awesome.

That's why we have device_hotplug_lock and some ugly code related to it.

It extends to parents and children somewhat because of device objects
representing packages (we want those to be "offline" only if all their
children are offline) and that's why the lock is held around offline from
sysfs too.

I'm not entirely happy with this for quite obvious reasons, but it gets
the job done ATM.

> We surely need to look into that aspect, but I don't see a reason why e.g. a
> device hotplug operation should be in any way related to the intermediate
> state of a particular cpu. If that's the case, then there is something really
> wrong.

If that state is different from complete offline, we should not try to eject
(the package containing) that CPU.

> I'm aware that we have a gazillion of silly assumptions all over the place and
> some of them are wrong today and just do not explode in our face simply
> because it's extremly hard to trigger. That's one reason why we need to go
> through all the cpu notifier related sites and inspect them deeply.

Agreed, but the particular concern I'm talking about is not in that category IMO.

Thanks,
Rafael

Re: [patch 20/20] rcu: Make CPU_DYING_IDLE an explicit call

[Paul E. McKenney]

On Sat, Feb 27, 2016 at 12:30:33PM +0100, Thomas Gleixner wrote:
> On Sat, 27 Feb 2016, Paul E. McKenney wrote:
> > On Sat, Feb 27, 2016 at 08:47:41AM +0100, Thomas Gleixner wrote:
> > > On Fri, 26 Feb 2016, Paul E. McKenney wrote:
> > > > > > --- a/kernel/cpu.c
> > > > > > +++ b/kernel/cpu.c
> > > > > > @@ -762,6 +762,7 @@ void cpuhp_report_idle_dead(void)
> > > > > >  	BUG_ON(st->state != CPUHP_AP_OFFLINE);
> > > > > >  	st->state = CPUHP_AP_IDLE_DEAD;
> > > > > >  	complete(&st->done);
> > > > > 
> > > > > What prevents the other CPU from killing this CPU at this point, so
> > > > > that this CPU does not tell RCU that it is dead?
> > > > >
> > > > > I agree that the odds should be low, but there are all manner of things
> > > > > that might delay a CPU for just a little bit too long...
> > > > > 
> > > > > Or am I missing something subtle here?
> > > 
> > > No. The reason why I moved the rcu call past the complete is, that otherwise
> > > complete() complains about rcu being dead already. Hmm, but you are right. In
> > > theory the other side could allow physical removal before it actually told rcu
> > > that it's gone.
> > 
> > There is one case where this is OK, and that is where the outgoing CPU
> > puts itself to sleep (or whatever) without help from the other CPU.
> 
> That's the case. It's the last call before the outgoing CPU goes into
> arch_cpu_idle_dead(). There is no involvement of the controlling CPU at this
> point. It just wants to know, that the outgoing one is dead finally.

Ah, so you have gotten rid of all the things like arm's and xtensa's
platform_cpu_kill(), where the surviving CPU does things like stopping
the outgoing CPU's clock?  That would make things simpler!

							Thanx, Paul

Re: [patch 01/20] idle: Move x86ism out of generic code

[Brian Gerst]

On Fri, Feb 26, 2016 at 1:43 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> We have an arch specific callback here already.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
>  arch/x86/kernel/process.c |   12 ++++++++++++
>  kernel/sched/idle.c       |   15 ---------------
>  2 files changed, 12 insertions(+), 15 deletions(-)
>
> Index: b/arch/x86/kernel/process.c
> ===================================================================
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -271,6 +271,18 @@ void exit_idle(void)
>  }
>  #endif
>
> +void arch_cpu_idle_prepare(void)
> +{
> +       /*
> +        * If we're the non-boot CPU, nothing set the stack canary up
> +        * for us. The boot CPU already has it initialized but no harm
> +        * in doing it again. This is a good place for updating it, as
> +        * we wont ever return from this function (so the invalid
> +        * canaries already on the stack wont ever trigger).
> +        */
> +       boot_init_stack_canary();
> +}
> +
>  void arch_cpu_idle_enter(void)
>  {
>         local_touch_nmi();
> Index: b/kernel/sched/idle.c
> ===================================================================
> --- a/kernel/sched/idle.c
> +++ b/kernel/sched/idle.c
> @@ -275,21 +275,6 @@ static void cpu_idle_loop(void)
>
>  void cpu_startup_entry(enum cpuhp_state state)
>  {
> -       /*
> -        * This #ifdef needs to die, but it's too late in the cycle to
> -        * make this generic (arm and sh have never invoked the canary
> -        * init for the non boot cpus!). Will be fixed in 3.11
> -        */
> -#ifdef CONFIG_X86
> -       /*
> -        * If we're the non-boot CPU, nothing set the stack canary up
> -        * for us. The boot CPU already has it initialized but no harm
> -        * in doing it again. This is a good place for updating it, as
> -        * we wont ever return from this function (so the invalid
> -        * canaries already on the stack wont ever trigger).
> -        */
> -       boot_init_stack_canary();
> -#endif
>         arch_cpu_idle_prepare();
>         cpu_idle_loop();
>  }

Does this actually work with stack protector enabled?
boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
not.

--
Brian Gerst

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

Rafael,

On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> Say we've taken all of them offline and now we are ready to eject.  If an
> online from sysfs (or any other place) comes in at this point, we'll be
> ejecting a CPU that's potentially doing something which is not awesome.
> 
> That's why we have device_hotplug_lock and some ugly code related to it.
> 
> It extends to parents and children somewhat because of device objects
> representing packages (we want those to be "offline" only if all their
> children are offline) and that's why the lock is held around offline from
> sysfs too.
> 
> I'm not entirely happy with this for quite obvious reasons, but it gets
> the job done ATM.

Understood. I'll fix that thing up so that won't happen and I put it on the
list of things to look at deeper.

Thanks,

	tglx

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

Rafael,

On Sat, 27 Feb 2016, Rafael J. Wysocki wrote:
> Say we've taken all of them offline and now we are ready to eject.  If an
> online from sysfs (or any other place) comes in at this point, we'll be
> ejecting a CPU that's potentially doing something which is not awesome.
> 
> That's why we have device_hotplug_lock and some ugly code related to it.
> 
> It extends to parents and children somewhat because of device objects
> representing packages (we want those to be "offline" only if all their
> children are offline) and that's why the lock is held around offline from
> sysfs too.
> 
> I'm not entirely happy with this for quite obvious reasons, but it gets
> the job done ATM.

Understood. I'll fix that thing up so that won't happen and I put it on the
list of things to look at deeper.

Thanks,

	tglx

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Thomas Gleixner]

On Sun, 28 Feb 2016, Thomas Gleixner wrote:
> 
> Understood. I'll fix that thing up so that won't happen and I put it on the
> list of things to look at deeper.

The below delta patch should do what you need, right?

Thanks,

	tglx

8<-------------

--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1461,6 +1461,11 @@ static ssize_t write_cpuhp_target(struct
 	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
 		return -EINVAL;
 #endif
+
+	ret = lock_device_hotplug_sysfs();
+	if (ret)
+		return ret;
+
 	mutex_lock(&cpuhp_state_mutex);
 	sp = cpuhp_get_step(target);
 	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
@@ -1472,6 +1477,8 @@ static ssize_t write_cpuhp_target(struct
 		ret = do_cpu_up(dev->id, target);
 	else
 		ret = do_cpu_down(dev->id, target);
+
+	unlock_device_hotplug();
 	return ret ? ret : count;
 }
 

Re: [patch 01/20] idle: Move x86ism out of generic code

[Thomas Gleixner]

On Sat, 27 Feb 2016, Brian Gerst wrote:
> >         arch_cpu_idle_prepare();
> >         cpu_idle_loop();
> >  }
> 
> Does this actually work with stack protector enabled?
> boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> not.

Stupid me. No it does of course not. I could have sworn that I tested that,
but obvioulsy not.

I drop that patch, but actually the real question is whether we can drop that
'#ifdef x86' around that boot_init_stack_canary() invocation.

AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
boot cpu. I can't see why that would be an issue on those architectures and
why it would be a problem if the boot cpu calls it again here.

CC'ed the relevant maintainers. Is there any issue with the patch below?

Thanks,

	tglx

8<------------------

--- a/kernel/sched/idle.c
+++ b/kernel/sched/idle.c
@@ -276,12 +276,6 @@ static void cpu_idle_loop(void)
 void cpu_startup_entry(enum cpuhp_state state)
 {
 	/*
-	 * This #ifdef needs to die, but it's too late in the cycle to
-	 * make this generic (arm and sh have never invoked the canary
-	 * init for the non boot cpus!). Will be fixed in 3.11
-	 */
-#ifdef CONFIG_X86
-	/*
 	 * If we're the non-boot CPU, nothing set the stack canary up
 	 * for us. The boot CPU already has it initialized but no harm
 	 * in doing it again. This is a good place for updating it, as
@@ -289,7 +283,7 @@ void cpu_startup_entry(enum cpuhp_state
 	 * canaries already on the stack wont ever trigger).
 	 */
 	boot_init_stack_canary();
-#endif
+
 	arch_cpu_idle_prepare();
 	cpu_idle_loop();
 }

Re: [patch 01/20] idle: Move x86ism out of generic code

[Will Deacon]

On Mon, Feb 29, 2016 at 08:35:41PM +0100, Thomas Gleixner wrote:
> On Sat, 27 Feb 2016, Brian Gerst wrote:
> > >         arch_cpu_idle_prepare();
> > >         cpu_idle_loop();
> > >  }
> > 
> > Does this actually work with stack protector enabled?
> > boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> > not.
> 
> Stupid me. No it does of course not. I could have sworn that I tested that,
> but obvioulsy not.
> 
> I drop that patch, but actually the real question is whether we can drop that
> '#ifdef x86' around that boot_init_stack_canary() invocation.
> 
> AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
> boot cpu. I can't see why that would be an issue on those architectures and
> why it would be a problem if the boot cpu calls it again here.
> 
> CC'ed the relevant maintainers. Is there any issue with the patch below?

On arm[64], the canary is unfortunately global, so I don't think it would
be safe to update it live like this without effectively stopping the
machine and forcing everybody into idle.

Will

Re: [patch 01/20] idle: Move x86ism out of generic code

[Thomas Gleixner]

On Mon, 29 Feb 2016, Will Deacon wrote:

> On Mon, Feb 29, 2016 at 08:35:41PM +0100, Thomas Gleixner wrote:
> > On Sat, 27 Feb 2016, Brian Gerst wrote:
> > > >         arch_cpu_idle_prepare();
> > > >         cpu_idle_loop();
> > > >  }
> > > 
> > > Does this actually work with stack protector enabled?
> > > boot_init_stack_canary() is inlined while arch_cpu_idle_prepare() is
> > > not.
> > 
> > Stupid me. No it does of course not. I could have sworn that I tested that,
> > but obvioulsy not.
> > 
> > I drop that patch, but actually the real question is whether we can drop that
> > '#ifdef x86' around that boot_init_stack_canary() invocation.
> > 
> > AFAICT, neither arm, arm64 nor mips and sh call it on anything else than the
> > boot cpu. I can't see why that would be an issue on those architectures and
> > why it would be a problem if the boot cpu calls it again here.
> > 
> > CC'ed the relevant maintainers. Is there any issue with the patch below?
> 
> On arm[64], the canary is unfortunately global, so I don't think it would
> be safe to update it live like this without effectively stopping the
> machine and forcing everybody into idle.

Thanks for clarification.

       Thomas

Re: [patch 10/20] cpu/hotplug: Make target state writeable

[Rafael J. Wysocki]

On Monday, February 29, 2016 04:49:24 PM Thomas Gleixner wrote:
> On Sun, 28 Feb 2016, Thomas Gleixner wrote:
> > 
> > Understood. I'll fix that thing up so that won't happen and I put it on the
> > list of things to look at deeper.
> 
> The below delta patch should do what you need, right?

Yup, thanks!

Rafael


> 8<-------------
> 
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1461,6 +1461,11 @@ static ssize_t write_cpuhp_target(struct
>  	if (target != CPUHP_OFFLINE && target != CPUHP_ONLINE)
>  		return -EINVAL;
>  #endif
> +
> +	ret = lock_device_hotplug_sysfs();
> +	if (ret)
> +		return ret;
> +
>  	mutex_lock(&cpuhp_state_mutex);
>  	sp = cpuhp_get_step(target);
>  	ret = !sp->name || sp->cant_stop ? -EINVAL : 0;
> @@ -1472,6 +1477,8 @@ static ssize_t write_cpuhp_target(struct
>  		ret = do_cpu_up(dev->id, target);
>  	else
>  		ret = do_cpu_down(dev->id, target);
> +
> +	unlock_device_hotplug();
>  	return ret ? ret : count;
>  }

Re: [patch 18/20] cpu/hotplug: Let upcoming cpu bring itself fully up

[Richard Cochran]

On Fri, Feb 26, 2016 at 06:43:41PM -0000, Thomas Gleixner wrote:
> @@ -1204,7 +1212,7 @@ static bool cpuhp_is_ap_state(enum cpuhp
>  {
>  	if (state >= CPUHP_AP_OFFLINE && state <= CPUHP_AP_ONLINE)
>  		return true;

FWIW, this test is superfluous after this change, because
CPUHP_BRINGUP_CPU is less than CPUHP_AP_OFFLINE.

> -	return state > CPUHP_BP_ONLINE;
> +	return state > CPUHP_BRINGUP_CPU;
>  }

Thanks,
Richard