From: Oleg Nesterov <oleg@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: "Kohli, Gaurav" <gkohli@codeaurora.org>,
tglx@linutronix.de, mpe@ellerman.id.au, mingo@kernel.org,
bigeasy@linutronix.de, linux-kernel@vger.kernel.org,
linux-arm-msm@vger.kernel.org,
Neeraj Upadhyay <neeraju@codeaurora.org>,
Will Deacon <will.deacon@arm.com>
Subject: Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
Date: Wed, 6 Jun 2018 15:51:16 +0200 [thread overview]
Message-ID: <20180606135115.GA4609@redhat.com> (raw)
In-Reply-To: <20180605201316.GZ12198@hirez.programming.kicks-ass.net>
On 06/05, Peter Zijlstra wrote:
>
> Also, I think we still need TASK_PARKED as a special state for that.
I think it would be nice to kill the TASK_PARKED state altogether. But I don't
know how. I'll try to look at this code later, but I am not sure I will find a
way to cleanup it...
> --- a/kernel/kthread.c
> +++ b/kernel/kthread.c
> @@ -177,12 +177,24 @@ void *kthread_probe_data(struct task_struct *task)
> static void __kthread_parkme(struct kthread *self)
> {
> for (;;) {
> - set_current_state(TASK_PARKED);
> + /*
> + * TASK_PARKED is a special state; we must serialize against
> + * possible pending wakeups to avoid store-store collisions on
> + * task->state.
> + *
> + * Such a collision might possibly result in the task state
> + * changin from TASK_PARKED and us failing the
> + * wait_task_inactive() in kthread_park().
> + */
> + set_special_state(TASK_PARKED);
Agreed,
> if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
> break;
> +
> + complete_all(&self->parked);
> schedule();
> }
> __set_current_state(TASK_RUNNING);
> + reinit_completion(&self->parked);
But how can we know that all the callers of kthread_park() have already returned
from wait_for_completion() ?
Oh. The very fact that __kthread_parkme() does complete_all() proves that we need
some serious cleanups. In particular, I think that kthread_park() on a parked kthread
must not be possible.
Just look at this code. It looks as if __kthread_parkme() can race with _unpark()
and thus we need this wait-event-like loop.
But if it can race with _unpark() then kthread_park() can block forever.
For the start, can't we change kthread_park()
- set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+ if (test_and_set_bit(...))
+ return -EAGAIN;
and s/complete_all/complete/ in __kthread_parkme() ?
IIUC, this will only affect smpboot_update_cpumask_percpu_thread() which can hit
an already parked thread, but it doesn't need to wait.
And it seems that smpboot_update_cpumask_percpu_thread() in turn needs some cleanups.
Hmm. and its single user: kernel/watchdog.c.
And speaking of watchdog.c, can't we simply kill the "watchdog/%u" threads? This is
off-topic, but can't watchdog_timer_fn() use stop_one_cpu_nowait(watchdog) ?
And I really think we should unexport kthread_park/unpark(), only smpboot_thread_fn()
should use them. kthread() should not play with __kthread_parkme(). And even
KTHREAD_SHOULD_PARK must die, I mean it should live in struct smp_hotplug_thread,
not in struct kthread.
OK, this is off-topic too.
In short, I think this patch is fine but I didn't read it carefully, will try tomorrow.
And, let me repeat, can't we avoid complete_all() ?
Oleg.
next prev parent reply other threads:[~2018-06-06 13:51 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-25 8:33 [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup Gaurav Kohli
2018-04-25 20:09 ` Peter Zijlstra
2018-04-26 4:04 ` Kohli, Gaurav
2018-04-26 9:14 ` Peter Zijlstra
2018-04-26 8:41 ` Peter Zijlstra
2018-04-26 8:57 ` Peter Zijlstra
2018-04-26 15:53 ` Kohli, Gaurav
2018-04-30 11:17 ` Peter Zijlstra
2018-05-01 7:50 ` Kohli, Gaurav
2018-05-01 10:18 ` Peter Zijlstra
2018-05-01 10:40 ` Peter Zijlstra
2018-05-01 10:40 ` Kohli, Gaurav
2018-05-01 11:31 ` Peter Zijlstra
2018-05-01 11:46 ` Kohli, Gaurav
2018-05-01 13:19 ` Peter Zijlstra
2018-05-02 5:15 ` Kohli, Gaurav
2018-05-02 8:20 ` Peter Zijlstra
2018-05-02 10:13 ` Kohli, Gaurav
2018-05-07 11:09 ` Kohli, Gaurav
2018-05-07 11:23 ` Kohli, Gaurav
2018-06-05 11:13 ` Kohli, Gaurav
2018-06-05 15:08 ` Oleg Nesterov
2018-06-05 15:22 ` Peter Zijlstra
2018-06-05 15:40 ` Peter Zijlstra
2018-06-05 16:35 ` Oleg Nesterov
2018-06-05 18:21 ` Kohli, Gaurav
2018-06-05 20:13 ` Peter Zijlstra
2018-06-06 13:51 ` Oleg Nesterov [this message]
2018-06-06 15:03 ` Peter Zijlstra
2018-06-06 15:04 ` Peter Zijlstra
2018-06-06 15:22 ` Peter Zijlstra
2018-06-06 18:59 ` Peter Zijlstra
2018-06-07 8:30 ` Kohli, Gaurav
2018-05-01 10:44 ` Peter Zijlstra
2018-04-26 16:02 ` Andrea Parri
2018-04-26 16:18 ` Oleg Nesterov
2018-04-30 11:20 ` Peter Zijlstra
2018-04-30 11:56 ` Peter Zijlstra
2018-04-28 6:43 ` [lkp-robot] [kthread/smpboot] cad8e99675: inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180606135115.GA4609@redhat.com \
--to=oleg@redhat.com \
--cc=bigeasy@linutronix.de \
--cc=gkohli@codeaurora.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=neeraju@codeaurora.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).