From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934843AbbI2Mup (ORCPT ); Tue, 29 Sep 2015 08:50:45 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:55323 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755467AbbI2Mui (ORCPT ); Tue, 29 Sep 2015 08:50:38 -0400 Date: Tue, 29 Sep 2015 14:45:09 +0200 From: Peter Zijlstra To: mingo@kernel.org Cc: linux-kernel@vger.kernel.org, tglx@linutronix.de, manfred@colorfullife.com, torvalds@linux-foundation.org, oleg@redhat.com, will.deacon@arm.com Subject: [RFC][PATCH] sched: Fix TASK_DEAD race in finish_task_switch() Message-ID: <20150929124509.GG3816@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Reviving a discussion at least a year old, I just remembered we had an issue here while doing those PREEMPT_ACTIVE patches. --- So the problem this patch is trying to address is as follows: CPU0 CPU1 context_switch(A, B) ttwu(A) LOCK A->pi_lock A->on_cpu == 0 finish_task_switch(A) prev_state = A->state <-. WMB | A->on_cpu = 0; | UNLOCK rq0->lock | | context_switch(C, A) `-- A->state = TASK_DEAD prev_state == TASK_DEAD put_task_struct(A) context_switch(A, C) finish_task_switch(A) A->state == TASK_DEAD put_task_struct(A) The argument being that the WMB will allow the load of A->state on CPU0 to cross over and observe CPU1's store of A->state, which will then result in a double-drop and use-after-free. Now the comment states (and this was true once upon a long time ago) that we need to observe A->state while holding rq->lock because that will order us against the wakeup; however the wakeup will not in fact acquire (that) rq->lock; it takes A->pi_lock these days. We can obviously fix this by upgrading the WMB to an MB, but that is expensive, so we'd rather avoid that. The alternative this patch takes is: smp_store_release(A->on_cpu, 0), which avoids the MB on some archs, but not important ones like ARM. Fixes: e4a52bcb9a18 ("sched: Remove rq->lock from the first half of ttwu()") Cc: Ingo Molnar Cc: Thomas Gleixner Cc: Manfred Spraul Cc: Linus Torvalds Reported-by: Oleg Nesterov Signed-off-by: Peter Zijlstra (Intel) --- kernel/sched/core.c | 10 +++++----- kernel/sched/sched.h | 5 +++-- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index e3a81ad9721a..7346133f1f4d 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2515,11 +2515,11 @@ static struct rq *finish_task_switch(struct task_struct *prev) * If a task dies, then it sets TASK_DEAD in tsk->state and calls * schedule one last time. The schedule call will never return, and * the scheduled task must drop that reference. - * The test for TASK_DEAD must occur while the runqueue locks are - * still held, otherwise prev could be scheduled on another cpu, die - * there before we look at prev->state, and then the reference would - * be dropped twice. - * Manfred Spraul + * + * We must observe prev->state before clearing prev->on_cpu (in + * finish_lock_switch), otherwise a concurrent wakeup can get prev + * running on another CPU and we could rave with its RUNNING -> DEAD + * transition, resulting in a double drop. */ prev_state = prev->state; vtime_task_switch(prev); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index af6f252e7e34..046242feff3a 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1072,9 +1072,10 @@ static inline void finish_lock_switch(struct rq *rq, struct task_struct *prev) * After ->on_cpu is cleared, the task can be moved to a different CPU. * We must ensure this doesn't happen until the switch is completely * finished. + * + * Pairs with the control dependency and rmb in try_to_wake_up(). */ - smp_wmb(); - prev->on_cpu = 0; + smp_store_release(&prev->on_cpu, 0); #endif #ifdef CONFIG_DEBUG_SPINLOCK /* this is a valid case when another task releases the spinlock */