Re: [PATCH 4/3] rtmutex: Avoid barrier in rt_mutex_handle_deadlock

From: Heiko Carstens <heiko.carstens@de.ibm.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>,
	tglx@linutronix.de, mingo@kernel.org, bigeasy@linutronix.de,
	umgwanakikbuti@gmail.com, paulmck@linux.vnet.ibm.com,
	linux-kernel@vger.kernel.org, kmo@daterainc.com
Subject: Re: [PATCH 4/3] rtmutex: Avoid barrier in rt_mutex_handle_deadlock
Date: Tue, 22 Mar 2016 15:45:37 +0100	[thread overview]
Message-ID: <20160322144537.GE3921@osiris> (raw)
In-Reply-To: <20160322135530.GR6344@twins.programming.kicks-ass.net>

On Tue, Mar 22, 2016 at 02:55:30PM +0100, Peter Zijlstra wrote:
> On Tue, Mar 22, 2016 at 02:26:00PM +0100, Heiko Carstens wrote:
> > > Clearly something magical is going on and its not clear.
> > 
> > The mechanism of our pfault code: if Linux is running as guest, runs a user
> > space process and the user space process accesses a page that the host has
> > paged out we get a pfault interrupt.
> > 
> > This allows us, within the guest, to schedule a different process. Without
> > this mechanism the host would have to suspend the whole virtual CPU until
> > the page has been paged in.
> > 
> > So when we get such an interrupt then we set the state of the current task
> > to uninterruptible and also set the need_resched flag. Both happens within
> > interrupt context(!). If we later on want to return to user space we
> > recognize the need_resched flag and then call schedule().
> > It's not very obvious how this works...
> 
> A few lines like the above near that function would go a long while I
> think.
> 
> And, ah!, you rely on the return to user resched to not be a
> preempt_schedule, how very icky :-)
> 
> Now, what happens if that task gets a spurious wakeup? Will it take the
> fault again, raise the PF int again etc.. ?

Yes, it will fault again etc. We actually do the spurious wakeup thing on
cpu hotplug (down), since unfortunately the original protocal has a flaw:
all pending completion interrupts of the "downed" cpu got lost in the host
and we do not know which ones.

So we wake all tasks up and see what happens... see pfault_cpu_notify().

> > Of course we have a lot of additional fun with the completion interrupt (->
> > host signals that a page of a process has been paged in and the process can
> > continue to run). This interrupt can arrive on any cpu and, since we have
> > virtual cpus, actually appear before the interrupt that signals that a page
> > is missing.
> 
> Of course :-)
> 
> Something like the below perhaps?
> 
> ---
>  arch/s390/mm/fault.c | 44 ++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 36 insertions(+), 8 deletions(-)

Sure, looks nice and makes a lot of sense. And the text looks a bit familiar
to me ;)

Could you provide From: and Signed-off-by: lines?