From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v2 15/20] mm: Try spin lock in speculative path Date: Fri, 18 Aug 2017 00:05:14 +0200 [thread overview] Message-ID: <1503007519-26777-16-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1503007519-26777-1-git-send-email-ldufour@linux.vnet.ibm.com> There is a deadlock when a CPU is doing a speculative page fault and another one is calling do_unmap(). The deadlock occurred because the speculative path try to spinlock the pte while the interrupt are disabled. When the other CPU in the unmap's path has locked the pte then is waiting for all the CPU to invalidate the TLB. As the CPU doing the speculative fault have the interrupt disable it can't invalidate the TLB, and can't get the lock. Since we are in a speculative path, we can race with other mm action. So let assume that the lock may not get acquired and fail the speculative page fault. Here are the stacks captured during the deadlock: CPU 0 native_flush_tlb_others+0x7c/0x260 flush_tlb_mm_range+0x6a/0x220 tlb_flush_mmu_tlbonly+0x63/0xc0 unmap_page_range+0x897/0x9d0 ? unmap_single_vma+0x7d/0xe0 ? release_pages+0x2b3/0x360 unmap_single_vma+0x7d/0xe0 unmap_vmas+0x51/0xa0 unmap_region+0xbd/0x130 do_munmap+0x279/0x460 SyS_munmap+0x53/0x70 CPU 1 do_raw_spin_lock+0x14e/0x160 _raw_spin_lock+0x5d/0x80 ? pte_map_lock+0x169/0x1b0 pte_map_lock+0x169/0x1b0 handle_pte_fault+0xbf2/0xd80 ? trace_hardirqs_on+0xd/0x10 handle_speculative_fault+0x272/0x280 handle_speculative_fault+0x5/0x280 __do_page_fault+0x187/0x580 trace_do_page_fault+0x52/0x260 do_async_page_fault+0x19/0x70 async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- mm/memory.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0ba14a5797b2..8c701e4f59d3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2300,7 +2300,8 @@ static bool pte_spinlock(struct vm_fault *vmf) goto out; vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); - spin_lock(vmf->ptl); + if (unlikely(!spin_trylock(vmf->ptl))) + goto out; if (vma_has_changed(vmf)) { spin_unlock(vmf->ptl); @@ -2336,8 +2337,20 @@ static bool pte_map_lock(struct vm_fault *vmf) if (vma_has_changed(vmf)) goto out; - pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &ptl); + /* + * Same as pte_offset_map_lock() except that we call + * spin_trylock() in place of spin_lock() to avoid race with + * unmap path which may have the lock and wait for this CPU + * to invalidate TLB but this CPU has irq disabled. + * Since we are in a speculative patch, accept it could fail + */ + ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); + pte = pte_offset_map(vmf->pmd, vmf->address); + if (unlikely(!spin_trylock(ptl))) { + pte_unmap(pte); + goto out; + } + if (vma_has_changed(vmf)) { pte_unmap_unlock(pte, ptl); goto out; -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v2 15/20] mm: Try spin lock in speculative path Date: Fri, 18 Aug 2017 00:05:14 +0200 [thread overview] Message-ID: <1503007519-26777-16-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1503007519-26777-1-git-send-email-ldufour@linux.vnet.ibm.com> There is a deadlock when a CPU is doing a speculative page fault and another one is calling do_unmap(). The deadlock occurred because the speculative path try to spinlock the pte while the interrupt are disabled. When the other CPU in the unmap's path has locked the pte then is waiting for all the CPU to invalidate the TLB. As the CPU doing the speculative fault have the interrupt disable it can't invalidate the TLB, and can't get the lock. Since we are in a speculative path, we can race with other mm action. So let assume that the lock may not get acquired and fail the speculative page fault. Here are the stacks captured during the deadlock: CPU 0 native_flush_tlb_others+0x7c/0x260 flush_tlb_mm_range+0x6a/0x220 tlb_flush_mmu_tlbonly+0x63/0xc0 unmap_page_range+0x897/0x9d0 ? unmap_single_vma+0x7d/0xe0 ? release_pages+0x2b3/0x360 unmap_single_vma+0x7d/0xe0 unmap_vmas+0x51/0xa0 unmap_region+0xbd/0x130 do_munmap+0x279/0x460 SyS_munmap+0x53/0x70 CPU 1 do_raw_spin_lock+0x14e/0x160 _raw_spin_lock+0x5d/0x80 ? pte_map_lock+0x169/0x1b0 pte_map_lock+0x169/0x1b0 handle_pte_fault+0xbf2/0xd80 ? trace_hardirqs_on+0xd/0x10 handle_speculative_fault+0x272/0x280 handle_speculative_fault+0x5/0x280 __do_page_fault+0x187/0x580 trace_do_page_fault+0x52/0x260 do_async_page_fault+0x19/0x70 async_page_fault+0x28/0x30 Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- mm/memory.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 0ba14a5797b2..8c701e4f59d3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2300,7 +2300,8 @@ static bool pte_spinlock(struct vm_fault *vmf) goto out; vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); - spin_lock(vmf->ptl); + if (unlikely(!spin_trylock(vmf->ptl))) + goto out; if (vma_has_changed(vmf)) { spin_unlock(vmf->ptl); @@ -2336,8 +2337,20 @@ static bool pte_map_lock(struct vm_fault *vmf) if (vma_has_changed(vmf)) goto out; - pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, - vmf->address, &ptl); + /* + * Same as pte_offset_map_lock() except that we call + * spin_trylock() in place of spin_lock() to avoid race with + * unmap path which may have the lock and wait for this CPU + * to invalidate TLB but this CPU has irq disabled. + * Since we are in a speculative patch, accept it could fail + */ + ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd); + pte = pte_offset_map(vmf->pmd, vmf->address); + if (unlikely(!spin_trylock(ptl))) { + pte_unmap(pte); + goto out; + } + if (vma_has_changed(vmf)) { pte_unmap_unlock(pte, ptl); goto out; -- 2.7.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-08-17 22:06 UTC|newest] Thread overview: 122+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-08-17 22:04 [PATCH v2 00/20] Speculative page faults Laurent Dufour 2017-08-17 22:04 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 01/20] mm: Dont assume page-table invariance during faults Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 03/20] mm: Introduce pte_spinlock " Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 04/20] mm: VMA sequence count Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 05/20] mm: Protect VMA modifications using " Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 06/20] mm: RCU free VMAs Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 07/20] mm: Cache some VMA fields in the vm_fault structure Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 08/20] mm: Protect SPF handler against anon_vma changes Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 10/20] mm: Introduce __lru_cache_add_active_or_unevictable Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 11/20] mm: Introduce __maybe_mkwrite() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 12/20] mm: Introduce __vm_normal_page() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 13/20] mm: Introduce __page_add_new_anon_rmap() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 14/20] mm: Provide speculative fault infrastructure Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-20 12:11 ` Sergey Senozhatsky 2017-08-20 12:11 ` Sergey Senozhatsky 2017-08-25 8:52 ` Laurent Dufour 2017-08-25 8:52 ` Laurent Dufour 2017-08-27 0:18 ` Kirill A. Shutemov 2017-08-27 0:18 ` Kirill A. Shutemov 2017-08-28 9:37 ` Peter Zijlstra 2017-08-28 9:37 ` Peter Zijlstra 2017-08-28 21:14 ` Benjamin Herrenschmidt 2017-08-28 21:14 ` Benjamin Herrenschmidt 2017-08-28 22:35 ` Andi Kleen 2017-08-28 22:35 ` Andi Kleen 2017-08-29 8:15 ` Peter Zijlstra 2017-08-29 8:15 ` Peter Zijlstra 2017-08-29 8:33 ` Peter Zijlstra 2017-08-29 8:33 ` Peter Zijlstra 2017-08-29 11:27 ` Peter Zijlstra 2017-08-29 11:27 ` Peter Zijlstra 2017-08-29 21:19 ` Benjamin Herrenschmidt 2017-08-29 21:19 ` Benjamin Herrenschmidt 2017-08-30 6:13 ` Peter Zijlstra 2017-08-30 6:13 ` Peter Zijlstra 2017-08-29 7:59 ` Laurent Dufour 2017-08-29 7:59 ` Laurent Dufour 2017-08-29 12:04 ` Peter Zijlstra 2017-08-29 12:04 ` Peter Zijlstra 2017-08-29 13:18 ` Laurent Dufour 2017-08-29 13:18 ` Laurent Dufour 2017-08-29 13:45 ` Peter Zijlstra 2017-08-29 13:45 ` Peter Zijlstra 2017-08-30 5:03 ` Anshuman Khandual 2017-08-30 5:03 ` Anshuman Khandual 2017-08-30 5:58 ` Peter Zijlstra 2017-08-30 5:58 ` Peter Zijlstra 2017-08-30 9:32 ` Laurent Dufour 2017-08-30 9:32 ` Laurent Dufour 2017-08-31 6:55 ` Anshuman Khandual 2017-08-31 6:55 ` Anshuman Khandual 2017-08-31 7:31 ` Peter Zijlstra 2017-08-31 7:31 ` Peter Zijlstra 2017-08-30 9:53 ` Laurent Dufour 2017-08-30 9:53 ` Laurent Dufour 2017-08-30 3:48 ` Anshuman Khandual 2017-08-30 3:48 ` Anshuman Khandual 2017-08-30 5:25 ` Anshuman Khandual 2017-08-30 5:25 ` Anshuman Khandual 2017-08-30 8:56 ` Laurent Dufour 2017-08-30 8:56 ` Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour [this message] 2017-08-17 22:05 ` [PATCH v2 15/20] mm: Try spin lock in speculative path Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 16/20] mm: Adding speculative page fault failure trace events Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 17/20] perf: Add a speculative page fault sw event Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 8:55 ` Anshuman Khandual 2017-08-21 8:55 ` Anshuman Khandual 2017-08-22 1:46 ` Michael Ellerman 2017-08-22 1:46 ` Michael Ellerman 2017-08-17 22:05 ` [PATCH v2 18/20] perf tools: Add support for the SPF perf event Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 8:48 ` Anshuman Khandual 2017-08-21 8:48 ` Anshuman Khandual 2017-08-25 8:53 ` Laurent Dufour 2017-08-25 8:53 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 19/20] x86/mm: Add speculative pagefault handling Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 7:29 ` Anshuman Khandual 2017-08-21 7:29 ` Anshuman Khandual 2017-08-29 14:50 ` Laurent Dufour 2017-08-29 14:50 ` Laurent Dufour 2017-08-29 14:58 ` Laurent Dufour 2017-08-29 14:58 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 20/20] powerpc/mm: Add speculative page fault Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 6:58 ` Anshuman Khandual 2017-08-21 6:58 ` Anshuman Khandual 2017-08-29 15:13 ` Laurent Dufour 2017-08-29 15:13 ` Laurent Dufour 2017-08-21 2:26 ` [PATCH v2 00/20] Speculative page faults Sergey Senozhatsky 2017-08-21 2:26 ` Sergey Senozhatsky 2017-09-08 9:24 ` Laurent Dufour 2017-09-08 9:24 ` Laurent Dufour 2017-09-11 0:45 ` Sergey Senozhatsky 2017-09-11 0:45 ` Sergey Senozhatsky 2017-09-11 6:28 ` Laurent Dufour 2017-09-11 6:28 ` Laurent Dufour 2017-08-21 6:28 ` Anshuman Khandual 2017-08-21 6:28 ` Anshuman Khandual 2017-08-22 0:41 ` Paul E. McKenney 2017-08-22 0:41 ` Paul E. McKenney 2017-08-25 9:41 ` Laurent Dufour 2017-08-25 9:41 ` Laurent Dufour
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1503007519-26777-16-git-send-email-ldufour@linux.vnet.ibm.com \ --to=ldufour@linux.vnet.ibm.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=dave@stgolabs.net \ --cc=haren@linux.vnet.ibm.com \ --cc=hpa@zytor.com \ --cc=jack@suse.cz \ --cc=khandual@linux.vnet.ibm.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mhocko@kernel.org \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=npiggin@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=tim.c.chen@linux.intel.com \ --cc=will.deacon@arm.com \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.