From: "Michael S. Tsirkin" <mst@redhat.com>
To: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: linux-arch@vger.kernel.org, heiko.carstens@de.ibm.com,
linux-kernel@vger.kernel.org,
David Hildenbrand <dahi@linux.vnet.ibm.com>,
paulus@samba.org, schwidefsky@de.ibm.com,
akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org,
mingo@kernel.org
Subject: Re: [RFC 0/2] Reenable might_sleep() checks for might_fault() when atomic
Date: Wed, 26 Nov 2014 18:50:24 +0200 [thread overview]
Message-ID: <20141126165024.GA11202@redhat.com> (raw)
In-Reply-To: <5476002B.30900@de.ibm.com>
On Wed, Nov 26, 2014 at 05:30:35PM +0100, Christian Borntraeger wrote:
> Am 26.11.2014 um 17:19 schrieb Michael S. Tsirkin:
> > On Wed, Nov 26, 2014 at 05:02:23PM +0100, David Hildenbrand wrote:
> >>>> This is what happened on our side (very recent kernel):
> >>>>
> >>>> spin_lock(&lock)
> >>>> copy_to_user(...)
> >>>> spin_unlock(&lock)
> >>>
> >>> That's a deadlock even without copy_to_user - it's
> >>> enough for the thread to be preempted and another one
> >>> to try taking the lock.
> >>>
> >>>
> >>>> 1. s390 locks/unlocks a spin lock with a compare and swap, using the _cpu id_
> >>>> as "old value"
> >>>> 2. we slept during copy_to_user()
> >>>> 3. the thread got scheduled onto another cpu
> >>>> 4. spin_unlock failed as the _cpu id_ didn't match (another cpu that locked
> >>>> the spinlock tried to unlocked it).
> >>>> 5. lock remained locked -> deadlock
> >>>>
> >>>> Christian came up with the following explanation:
> >>>> Without preemption, spin_lock() will not touch the preempt counter.
> >>>> disable_pfault() will always touch it.
> >>>>
> >>>> Therefore, with preemption disabled, copy_to_user() has no idea that it is
> >>>> running in atomic context - and will therefore try to sleep.
> >>>>
> >>>> So copy_to_user() will on s390:
> >>>> 1. run "as atomic" while spin_lock() with preemption enabled.
> >>>> 2. run "as not atomic" while spin_lock() with preemption disabled.
> >>>> 3. run "as atomic" while pagefault_disabled() with preemption enabled or
> >>>> disabled.
> >>>> 4. run "as not atomic" when really not atomic.
> >>
> >> should have been more clear at that point:
> >> preemption enabled == kernel compiled with preemption support
> >> preemption disabled == kernel compiled without preemption support
> >>
> >>>>
> >>>> And exactly nr 2. is the thing that produced the deadlock in our scenario and
> >>>> the reason why I want a might_sleep() :)
> >>>
> >>> IMHO it's not copy to user that causes the problem.
> >>> It's the misuse of spinlocks with preemption on.
> >>
> >> As I said, preemption was off.
> >
> > off -> disabled at compile time?
> >
> > But the code is broken for people that do enable it.
> [...]
> > You should normally disable preemption if you take
> > spinlocks.
>
> Your are telling that any sequence of
> spin_lock
> ...
> spin_unlock
>
> is broken with CONFIG_PREEMPT?
> Michael, that is bullshit. spin_lock will take care of CONFIG_PREEMPT just fine.
>
> Only sequences like
> spin_lock
> ...
> schedule
> ...
> spin_unlock
> are broken.
>
> But as I said. That is not the problem that we are discussing here.
>
> Christian
I'm saying spin_lock without _irqsave is often a bug.
I am also saying this code in mm/fault.c:
__do_page_fault
...
/*
* If we're in an interrupt, have no user context or are running
* in an atomic region then we must not take the fault:
*/
if (unlikely(in_atomic() || !mm)) {
bad_area_nosemaphore(regs, error_code, address);
return;
}
means that a fault won't cause sleep if called in atomic context.
And a bunch of code relies on this.
This is why might_fault does:
* it would be nicer only to annotate paths which are not under
* pagefault_disable, however that requires a larger audit and
* providing helpers like get_user_atomic.
*/
if (in_atomic())
return;
__might_sleep(__FILE__, __LINE__, 0);
If you see this violated, let's figure out why.
--
MST
next prev parent reply other threads:[~2014-11-26 16:50 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-11-25 11:43 [RFC 0/2] Reenable might_sleep() checks for might_fault() when atomic David Hildenbrand
2014-11-25 11:43 ` [RFC 1/2] powerpc/fsl-pci: atomic get_user when pagefault_disabled David Hildenbrand
2015-01-30 5:15 ` [RFC,1/2] " Scott Wood
2015-01-30 7:58 ` David Hildenbrand
2014-11-25 11:43 ` [RFC 2/2] mm, sched: trigger might_sleep() in might_fault() when atomic David Hildenbrand
2014-11-26 7:02 ` [RFC 0/2] Reenable might_sleep() checks for " Michael S. Tsirkin
2014-11-26 10:05 ` David Hildenbrand
2014-11-26 15:17 ` Michael S. Tsirkin
2014-11-26 15:23 ` Michael S. Tsirkin
2014-11-26 15:32 ` David Hildenbrand
2014-11-26 15:47 ` Michael S. Tsirkin
2014-11-26 16:02 ` David Hildenbrand
2014-11-26 16:19 ` Michael S. Tsirkin
2014-11-26 16:30 ` Christian Borntraeger
2014-11-26 16:50 ` Michael S. Tsirkin [this message]
2014-11-26 16:07 ` Christian Borntraeger
2014-11-26 16:32 ` Michael S. Tsirkin
2014-11-26 16:51 ` Christian Borntraeger
2014-11-26 17:04 ` Michael S. Tsirkin
2014-11-26 17:21 ` Michael S. Tsirkin
2014-11-27 7:09 ` Heiko Carstens
2014-11-27 7:40 ` Michael S. Tsirkin
2014-11-27 8:03 ` David Hildenbrand
2014-11-27 12:04 ` Heiko Carstens
2014-11-27 12:08 ` David Hildenbrand
2014-11-27 15:07 ` Thomas Gleixner
2014-11-27 15:19 ` David Hildenbrand
2014-11-27 15:37 ` David Laight
2014-11-27 15:45 ` David Hildenbrand
2014-11-27 16:27 ` David Laight
2014-11-27 16:49 ` David Hildenbrand
2014-11-27 21:52 ` Thomas Gleixner
2014-11-28 7:34 ` David Hildenbrand
2014-11-26 15:30 ` Christian Borntraeger
2014-11-26 15:37 ` Michael S. Tsirkin
2014-11-26 16:02 ` Christian Borntraeger
2014-11-26 15:22 ` Michael S. Tsirkin
2014-11-27 17:10 ` [PATCH RFC " David Hildenbrand
2014-11-27 17:10 ` [PATCH RFC 1/2] preempt: track pagefault_disable() calls in the preempt counter David Hildenbrand
2014-11-27 17:10 ` [PATCH RFC 2/2] mm, sched: trigger might_sleep() in might_fault() when pagefaults are disabled David Hildenbrand
2014-11-27 17:24 ` Michael S. Tsirkin
2014-11-27 17:32 ` Michael S. Tsirkin
2014-11-27 18:08 ` David Hildenbrand
2014-11-27 18:27 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20141126165024.GA11202@redhat.com \
--to=mst@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=borntraeger@de.ibm.com \
--cc=dahi@linux.vnet.ibm.com \
--cc=heiko.carstens@de.ibm.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mingo@kernel.org \
--cc=paulus@samba.org \
--cc=schwidefsky@de.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).