[PATCH] smp: smp_call_on_cpu(): use INIT_WORK_ONSTACK() for automatic work_struct

[PATCH] smp: smp_call_on_cpu(): use INIT_WORK_ONSTACK() for automatic work_struct

[Nicolai Stange]

This new warning

  INFO: trying to register non-static key.
  the code is fine but needs lockdep annotation.
  turning off the locking correctness validator.
  CPU: 0 PID: 82 Comm: kworker/0:1 Not tainted 4.8.0-rc6+ #360
  Hardware name: Dell Inc. Latitude E6540/0725FP, BIOS A10 06/26/2014
  Workqueue: events smp_call_on_cpu_callback
[...]
  Call Trace:
   [<ffffffff9542fa55>] dump_stack+0x68/0x93
   [<ffffffff950ede3a>] register_lock_class+0x54a/0x550
   [<ffffffff950f0178>] __lock_acquire+0x88/0x12a0
   [<ffffffff95106f0d>] ? debug_lockdep_rcu_enabled+0x1d/0x20
   [<ffffffff950f17b9>] lock_acquire+0xe9/0x1d0
   [<ffffffff950b2fe7>] ? process_one_work+0x197/0x6c0
   [<ffffffff950b3040>] process_one_work+0x1f0/0x6c0
   [<ffffffff950b2fe7>] ? process_one_work+0x197/0x6c0
   [<ffffffff950b355e>] worker_thread+0x4e/0x4a0
   [<ffffffff950b3510>] ? process_one_work+0x6c0/0x6c0
   [<ffffffff950b3510>] ? process_one_work+0x6c0/0x6c0
   [<ffffffff950baf0f>] kthread+0xff/0x120
   [<ffffffff950bae10>] ? kthread_park+0x60/0x60
   [<ffffffff9584d217>] ret_from_fork+0x27/0x40
  dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)

can be bisected to commit e23f22b5cb9e ("dcdbas: Make use of
smp_call_on_cpu()").

The reason is that smp_call_on_cpu(), introduced with commit df8ce9d78a4e
("smp: Add function to execute a function synchronously on a CPU"),
initializes a work_struct with automatic storage duration by means of
__WORK_INITIALIZER().

Use INIT_WORK_ONSTACK() there instead, just like the very similar
work_on_cpu() does.

Fixes: df8ce9d78a4e ("smp: Add function to execute a function
                      synchronously on a CPU")
Signed-off-by: Nicolai Stange <nicstange@gmail.com>
---
 kernel/smp.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index f4f6137..a129ac3 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -759,7 +759,6 @@ static void smp_call_on_cpu_callback(struct work_struct *work)
 int smp_call_on_cpu(unsigned int cpu, int (*func)(void *), void *par, bool phys)
 {
 	struct smp_call_on_cpu_struct sscs = {
-		.work = __WORK_INITIALIZER(sscs.work, smp_call_on_cpu_callback),
 		.done = COMPLETION_INITIALIZER_ONSTACK(sscs.done),
 		.func = func,
 		.data = par,
@@ -769,9 +768,13 @@ int smp_call_on_cpu(unsigned int cpu, int (*func)(void *), void *par, bool phys)
 	if (cpu >= nr_cpu_ids || !cpu_online(cpu))
 		return -ENXIO;
 
+	INIT_WORK_ONSTACK(&sscs.work, smp_call_on_cpu_callback);
+
 	queue_work_on(cpu, system_wq, &sscs.work);
 	wait_for_completion(&sscs.done);
 
+	destroy_work_on_stack(&sscs.work);
+
 	return sscs.ret;
 }
 EXPORT_SYMBOL_GPL(smp_call_on_cpu);
-- 
2.10.0

Re: [PATCH] smp: smp_call_on_cpu(): use INIT_WORK_ONSTACK() for automatic work_struct

[Peter Zijlstra]

On Thu, Sep 22, 2016 at 08:17:58PM +0200, Nicolai Stange wrote:
> This new warning
> 
>   INFO: trying to register non-static key.
>   the code is fine but needs lockdep annotation.
>   turning off the locking correctness validator.
>   CPU: 0 PID: 82 Comm: kworker/0:1 Not tainted 4.8.0-rc6+ #360
>   Hardware name: Dell Inc. Latitude E6540/0725FP, BIOS A10 06/26/2014
>   Workqueue: events smp_call_on_cpu_callback

https://lkml.kernel.org/r/tip-8db549491c4a3ce9e1d509b75f78516e497f48ec@git.kernel.org

Re: [PATCH] smp: smp_call_on_cpu(): use INIT_WORK_ONSTACK() for automatic work_struct

[Nicolai Stange]

Peter Zijlstra <peterz@infradead.org> writes:

> On Thu, Sep 22, 2016 at 08:17:58PM +0200, Nicolai Stange wrote:
>> This new warning
>> 
>>   INFO: trying to register non-static key.
>>   the code is fine but needs lockdep annotation.
>>   turning off the locking correctness validator.
>>   CPU: 0 PID: 82 Comm: kworker/0:1 Not tainted 4.8.0-rc6+ #360
>>   Hardware name: Dell Inc. Latitude E6540/0725FP, BIOS A10 06/26/2014
>>   Workqueue: events smp_call_on_cpu_callback
>
> https://lkml.kernel.org/r/tip-8db549491c4a3ce9e1d509b75f78516e497f48ec@git.kernel.org


Ah, thanks for letting me know.

One minor question regarding your patch though: it hasn't got a
destroy_work_on_stack(). Isn't one needed because of
  INIT_WORK_ONSTACK() -> __INIT_WORK() -> __init_work() ->
  debug_object_init_on_stack() ?
At least work_on_cpu() destroys its local work_struct that way...


Thanks,

Nicolai

Re: [PATCH] smp: smp_call_on_cpu(): use INIT_WORK_ONSTACK() for automatic work_struct

[Peter Zijlstra]

On Thu, Sep 22, 2016 at 09:15:52PM +0200, Nicolai Stange wrote:

> One minor question regarding your patch though: it hasn't got a
> destroy_work_on_stack(). Isn't one needed because of
>   INIT_WORK_ONSTACK() -> __INIT_WORK() -> __init_work() ->
>   debug_object_init_on_stack() ?
> At least work_on_cpu() destroys its local work_struct that way...

I wasn't aware of destroy_work_on_stack(), and I'm not familiar enough
with the debug objects stuff to see hwo important it is to call.

Also, there's more INIT_WORK_ONSTACK usage lacking
destroy_work_on_stack() calls.

Maybe it would be good to do a patch fixing them all up and adding a
comment with INIT_WORK_ONSTACK() that says it ought to be paired with
destroy_work_on_stack().