From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com>, Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v3 20/20] powerpc/mm: Add speculative page fault Date: Fri, 8 Sep 2017 20:07:04 +0200 [thread overview] Message-ID: <1504894024-2750-21-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1504894024-2750-1-git-send-email-ldufour@linux.vnet.ibm.com> This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with VM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Support is only provide for BOOK3S_64 currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- arch/powerpc/include/asm/book3s/64/pgtable.h | 5 +++++ arch/powerpc/mm/fault.c | 15 +++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index b9aff515b4de..470203daf97e 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -314,6 +314,11 @@ extern unsigned long pci_io_base; /* Advertise support for _PAGE_SPECIAL */ #define __HAVE_ARCH_PTE_SPECIAL +/* Advertise that we call the Speculative Page Fault handler */ +#if defined(CONFIG_PPC_BOOK3S_64) +#define __HAVE_ARCH_CALL_SPF +#endif + #ifndef __ASSEMBLY__ /* diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 4797d08581ce..97c7de242627 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -442,6 +442,20 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, if (is_exec) flags |= FAULT_FLAG_INSTRUCTION; +#if defined(__HAVE_ARCH_CALL_SPF) + if (is_user) { + /* let's try a speculative page fault without grabbing the + * mmap_sem. + */ + fault = handle_speculative_fault(mm, address, flags); + if (!(fault & VM_FAULT_RETRY)) { + perf_sw_event(PERF_COUNT_SW_SPF, 1, + regs, address); + goto done; + } + } +#endif /* defined(__HAVE_ARCH_CALL_SPF) */ + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -526,6 +540,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, up_read(¤t->mm->mmap_sem); +done: if (unlikely(fault & VM_FAULT_ERROR)) return mm_fault_error(regs, address, fault); -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com>, Sergey Senozhatsky <sergey.senozhatsky@gmail.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v3 20/20] powerpc/mm: Add speculative page fault Date: Fri, 8 Sep 2017 20:07:04 +0200 [thread overview] Message-ID: <1504894024-2750-21-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1504894024-2750-1-git-send-email-ldufour@linux.vnet.ibm.com> This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with VM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Support is only provide for BOOK3S_64 currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- arch/powerpc/include/asm/book3s/64/pgtable.h | 5 +++++ arch/powerpc/mm/fault.c | 15 +++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index b9aff515b4de..470203daf97e 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -314,6 +314,11 @@ extern unsigned long pci_io_base; /* Advertise support for _PAGE_SPECIAL */ #define __HAVE_ARCH_PTE_SPECIAL +/* Advertise that we call the Speculative Page Fault handler */ +#if defined(CONFIG_PPC_BOOK3S_64) +#define __HAVE_ARCH_CALL_SPF +#endif + #ifndef __ASSEMBLY__ /* diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 4797d08581ce..97c7de242627 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -442,6 +442,20 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, if (is_exec) flags |= FAULT_FLAG_INSTRUCTION; +#if defined(__HAVE_ARCH_CALL_SPF) + if (is_user) { + /* let's try a speculative page fault without grabbing the + * mmap_sem. + */ + fault = handle_speculative_fault(mm, address, flags); + if (!(fault & VM_FAULT_RETRY)) { + perf_sw_event(PERF_COUNT_SW_SPF, 1, + regs, address); + goto done; + } + } +#endif /* defined(__HAVE_ARCH_CALL_SPF) */ + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -526,6 +540,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address, up_read(¤t->mm->mmap_sem); +done: if (unlikely(fault & VM_FAULT_ERROR)) return mm_fault_error(regs, address, fault); -- 2.7.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-09-08 18:08 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-09-08 18:06 [PATCH v3 00/20] Speculative page faults Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 01/20] mm: Dont assume page-table invariance during faults Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 03/20] mm: Introduce pte_spinlock " Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 04/20] mm: VMA sequence count Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-13 11:53 ` Sergey Senozhatsky 2017-09-13 11:53 ` Sergey Senozhatsky 2017-09-13 16:56 ` Laurent Dufour 2017-09-13 16:56 ` Laurent Dufour 2017-09-14 0:31 ` Sergey Senozhatsky 2017-09-14 0:31 ` Sergey Senozhatsky 2017-09-14 7:55 ` Laurent Dufour 2017-09-14 7:55 ` Laurent Dufour 2017-09-14 8:13 ` Sergey Senozhatsky 2017-09-14 8:13 ` Sergey Senozhatsky 2017-09-14 8:58 ` Laurent Dufour 2017-09-14 8:58 ` Laurent Dufour 2017-09-14 9:11 ` Sergey Senozhatsky 2017-09-14 9:11 ` Sergey Senozhatsky 2017-09-14 9:15 ` Laurent Dufour 2017-09-14 9:15 ` Laurent Dufour 2017-09-14 9:40 ` Sergey Senozhatsky 2017-09-14 9:40 ` Sergey Senozhatsky 2017-09-15 12:38 ` Laurent Dufour 2017-09-15 12:38 ` Laurent Dufour 2017-09-25 12:22 ` Peter Zijlstra 2017-09-25 12:22 ` Peter Zijlstra 2017-09-08 18:06 ` [PATCH v3 05/20] mm: Protect VMA modifications using " Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 06/20] mm: RCU free VMAs Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 07/20] mm: Cache some VMA fields in the vm_fault structure Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 08/20] mm: Protect SPF handler against anon_vma changes Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page() Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 10/20] mm: Introduce __lru_cache_add_active_or_unevictable Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 11/20] mm: Introduce __maybe_mkwrite() Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 12/20] mm: Introduce __vm_normal_page() Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 13/20] mm: Introduce __page_add_new_anon_rmap() Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 14/20] mm: Provide speculative fault infrastructure Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:06 ` [PATCH v3 15/20] mm: Try spin lock in speculative path Laurent Dufour 2017-09-08 18:06 ` Laurent Dufour 2017-09-08 18:07 ` [PATCH v3 16/20] mm: Adding speculative page fault failure trace events Laurent Dufour 2017-09-08 18:07 ` Laurent Dufour 2017-09-08 18:07 ` [PATCH v3 17/20] perf: Add a speculative page fault sw event Laurent Dufour 2017-09-08 18:07 ` Laurent Dufour 2017-09-08 18:07 ` [PATCH v3 18/20] perf tools: Add support for the SPF perf event Laurent Dufour 2017-09-08 18:07 ` Laurent Dufour 2017-09-08 18:07 ` [PATCH v3 19/20] x86/mm: Add speculative pagefault handling Laurent Dufour 2017-09-08 18:07 ` Laurent Dufour 2017-09-08 18:07 ` Laurent Dufour [this message] 2017-09-08 18:07 ` [PATCH v3 20/20] powerpc/mm: Add speculative page fault Laurent Dufour 2017-09-18 7:15 ` [PATCH v3 00/20] Speculative page faults Laurent Dufour 2017-09-18 7:15 ` Laurent Dufour
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1504894024-2750-21-git-send-email-ldufour@linux.vnet.ibm.com \ --to=ldufour@linux.vnet.ibm.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=dave@stgolabs.net \ --cc=haren@linux.vnet.ibm.com \ --cc=hpa@zytor.com \ --cc=jack@suse.cz \ --cc=khandual@linux.vnet.ibm.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mhocko@kernel.org \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=npiggin@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=sergey.senozhatsky@gmail.com \ --cc=tglx@linutronix.de \ --cc=tim.c.chen@linux.intel.com \ --cc=will.deacon@arm.com \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.