From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v2 20/20] powerpc/mm: Add speculative page fault Date: Fri, 18 Aug 2017 00:05:19 +0200 [thread overview] Message-ID: <1503007519-26777-21-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1503007519-26777-1-git-send-email-ldufour@linux.vnet.ibm.com> This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Support is only provide for BOOK3S_64 currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- arch/powerpc/include/asm/book3s/64/pgtable.h | 5 +++++ arch/powerpc/mm/fault.c | 30 +++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 818a58fc3f4f..897f8b9f67e6 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -313,6 +313,11 @@ extern unsigned long pci_io_base; /* Advertise support for _PAGE_SPECIAL */ #define __HAVE_ARCH_PTE_SPECIAL +/* Advertise that we call the Speculative Page Fault handler */ +#if defined(CONFIG_PPC_BOOK3S_64) +#define __HAVE_ARCH_CALL_SPF +#endif + #ifndef __ASSEMBLY__ /* diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 4c422632047b..7b3cc4c30eab 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -291,9 +291,36 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, if (is_write && is_user) store_update_sp = store_updates_sp(regs); - if (is_user) + if (is_user) { flags |= FAULT_FLAG_USER; +#if defined(__HAVE_ARCH_CALL_SPF) + /* let's try a speculative page fault without grabbing the + * mmap_sem. + */ + + /* + * flags is set later based on the VMA's flags, for the common + * speculative service, we need some flags to be set. + */ + if (is_write) + flags |= FAULT_FLAG_WRITE; + + fault = handle_speculative_fault(mm, address, flags); + if (!(fault & VM_FAULT_RETRY || fault & VM_FAULT_ERROR)) { + perf_sw_event(PERF_COUNT_SW_SPF_DONE, 1, + regs, address); + goto done; + } + + /* + * Resetting flags since the following code assumes + * FAULT_FLAG_WRITE is not set. + */ + flags &= ~FAULT_FLAG_WRITE; +#endif /* defined(__HAVE_ARCH_CALL_SPF) */ + } + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -479,6 +506,7 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, rc = 0; } +done: /* * Major/minor page fault accounting. */ -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Laurent Dufour <ldufour@linux.vnet.ibm.com> To: paulmck@linux.vnet.ibm.com, peterz@infradead.org, akpm@linux-foundation.org, kirill@shutemov.name, ak@linux.intel.com, mhocko@kernel.org, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, khandual@linux.vnet.ibm.com, npiggin@gmail.com, bsingharora@gmail.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v2 20/20] powerpc/mm: Add speculative page fault Date: Fri, 18 Aug 2017 00:05:19 +0200 [thread overview] Message-ID: <1503007519-26777-21-git-send-email-ldufour@linux.vnet.ibm.com> (raw) In-Reply-To: <1503007519-26777-1-git-send-email-ldufour@linux.vnet.ibm.com> This patch enable the speculative page fault on the PowerPC architecture. This will try a speculative page fault without holding the mmap_sem, if it returns with WM_FAULT_RETRY, the mmap_sem is acquired and the traditional page fault processing is done. Support is only provide for BOOK3S_64 currently because: - require CONFIG_PPC_STD_MMU because checks done in set_access_flags_filter() - require BOOK3S because we can't support for book3e_hugetlb_preload() called by update_mmu_cache() Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com> --- arch/powerpc/include/asm/book3s/64/pgtable.h | 5 +++++ arch/powerpc/mm/fault.c | 30 +++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 818a58fc3f4f..897f8b9f67e6 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -313,6 +313,11 @@ extern unsigned long pci_io_base; /* Advertise support for _PAGE_SPECIAL */ #define __HAVE_ARCH_PTE_SPECIAL +/* Advertise that we call the Speculative Page Fault handler */ +#if defined(CONFIG_PPC_BOOK3S_64) +#define __HAVE_ARCH_CALL_SPF +#endif + #ifndef __ASSEMBLY__ /* diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 4c422632047b..7b3cc4c30eab 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -291,9 +291,36 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, if (is_write && is_user) store_update_sp = store_updates_sp(regs); - if (is_user) + if (is_user) { flags |= FAULT_FLAG_USER; +#if defined(__HAVE_ARCH_CALL_SPF) + /* let's try a speculative page fault without grabbing the + * mmap_sem. + */ + + /* + * flags is set later based on the VMA's flags, for the common + * speculative service, we need some flags to be set. + */ + if (is_write) + flags |= FAULT_FLAG_WRITE; + + fault = handle_speculative_fault(mm, address, flags); + if (!(fault & VM_FAULT_RETRY || fault & VM_FAULT_ERROR)) { + perf_sw_event(PERF_COUNT_SW_SPF_DONE, 1, + regs, address); + goto done; + } + + /* + * Resetting flags since the following code assumes + * FAULT_FLAG_WRITE is not set. + */ + flags &= ~FAULT_FLAG_WRITE; +#endif /* defined(__HAVE_ARCH_CALL_SPF) */ + } + /* When running in the kernel we expect faults to occur only to * addresses in user space. All other faults represent errors in the * kernel and should generate an OOPS. Unfortunately, in the case of an @@ -479,6 +506,7 @@ int do_page_fault(struct pt_regs *regs, unsigned long address, rc = 0; } +done: /* * Major/minor page fault accounting. */ -- 2.7.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-08-17 22:06 UTC|newest] Thread overview: 122+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-08-17 22:04 [PATCH v2 00/20] Speculative page faults Laurent Dufour 2017-08-17 22:04 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 01/20] mm: Dont assume page-table invariance during faults Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 02/20] mm: Prepare for FAULT_FLAG_SPECULATIVE Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 03/20] mm: Introduce pte_spinlock " Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 04/20] mm: VMA sequence count Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 05/20] mm: Protect VMA modifications using " Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 06/20] mm: RCU free VMAs Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 07/20] mm: Cache some VMA fields in the vm_fault structure Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 08/20] mm: Protect SPF handler against anon_vma changes Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 09/20] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 10/20] mm: Introduce __lru_cache_add_active_or_unevictable Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 11/20] mm: Introduce __maybe_mkwrite() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 12/20] mm: Introduce __vm_normal_page() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 13/20] mm: Introduce __page_add_new_anon_rmap() Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 14/20] mm: Provide speculative fault infrastructure Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-20 12:11 ` Sergey Senozhatsky 2017-08-20 12:11 ` Sergey Senozhatsky 2017-08-25 8:52 ` Laurent Dufour 2017-08-25 8:52 ` Laurent Dufour 2017-08-27 0:18 ` Kirill A. Shutemov 2017-08-27 0:18 ` Kirill A. Shutemov 2017-08-28 9:37 ` Peter Zijlstra 2017-08-28 9:37 ` Peter Zijlstra 2017-08-28 21:14 ` Benjamin Herrenschmidt 2017-08-28 21:14 ` Benjamin Herrenschmidt 2017-08-28 22:35 ` Andi Kleen 2017-08-28 22:35 ` Andi Kleen 2017-08-29 8:15 ` Peter Zijlstra 2017-08-29 8:15 ` Peter Zijlstra 2017-08-29 8:33 ` Peter Zijlstra 2017-08-29 8:33 ` Peter Zijlstra 2017-08-29 11:27 ` Peter Zijlstra 2017-08-29 11:27 ` Peter Zijlstra 2017-08-29 21:19 ` Benjamin Herrenschmidt 2017-08-29 21:19 ` Benjamin Herrenschmidt 2017-08-30 6:13 ` Peter Zijlstra 2017-08-30 6:13 ` Peter Zijlstra 2017-08-29 7:59 ` Laurent Dufour 2017-08-29 7:59 ` Laurent Dufour 2017-08-29 12:04 ` Peter Zijlstra 2017-08-29 12:04 ` Peter Zijlstra 2017-08-29 13:18 ` Laurent Dufour 2017-08-29 13:18 ` Laurent Dufour 2017-08-29 13:45 ` Peter Zijlstra 2017-08-29 13:45 ` Peter Zijlstra 2017-08-30 5:03 ` Anshuman Khandual 2017-08-30 5:03 ` Anshuman Khandual 2017-08-30 5:58 ` Peter Zijlstra 2017-08-30 5:58 ` Peter Zijlstra 2017-08-30 9:32 ` Laurent Dufour 2017-08-30 9:32 ` Laurent Dufour 2017-08-31 6:55 ` Anshuman Khandual 2017-08-31 6:55 ` Anshuman Khandual 2017-08-31 7:31 ` Peter Zijlstra 2017-08-31 7:31 ` Peter Zijlstra 2017-08-30 9:53 ` Laurent Dufour 2017-08-30 9:53 ` Laurent Dufour 2017-08-30 3:48 ` Anshuman Khandual 2017-08-30 3:48 ` Anshuman Khandual 2017-08-30 5:25 ` Anshuman Khandual 2017-08-30 5:25 ` Anshuman Khandual 2017-08-30 8:56 ` Laurent Dufour 2017-08-30 8:56 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 15/20] mm: Try spin lock in speculative path Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 16/20] mm: Adding speculative page fault failure trace events Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 17/20] perf: Add a speculative page fault sw event Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 8:55 ` Anshuman Khandual 2017-08-21 8:55 ` Anshuman Khandual 2017-08-22 1:46 ` Michael Ellerman 2017-08-22 1:46 ` Michael Ellerman 2017-08-17 22:05 ` [PATCH v2 18/20] perf tools: Add support for the SPF perf event Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 8:48 ` Anshuman Khandual 2017-08-21 8:48 ` Anshuman Khandual 2017-08-25 8:53 ` Laurent Dufour 2017-08-25 8:53 ` Laurent Dufour 2017-08-17 22:05 ` [PATCH v2 19/20] x86/mm: Add speculative pagefault handling Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour 2017-08-21 7:29 ` Anshuman Khandual 2017-08-21 7:29 ` Anshuman Khandual 2017-08-29 14:50 ` Laurent Dufour 2017-08-29 14:50 ` Laurent Dufour 2017-08-29 14:58 ` Laurent Dufour 2017-08-29 14:58 ` Laurent Dufour 2017-08-17 22:05 ` Laurent Dufour [this message] 2017-08-17 22:05 ` [PATCH v2 20/20] powerpc/mm: Add speculative page fault Laurent Dufour 2017-08-21 6:58 ` Anshuman Khandual 2017-08-21 6:58 ` Anshuman Khandual 2017-08-29 15:13 ` Laurent Dufour 2017-08-29 15:13 ` Laurent Dufour 2017-08-21 2:26 ` [PATCH v2 00/20] Speculative page faults Sergey Senozhatsky 2017-08-21 2:26 ` Sergey Senozhatsky 2017-09-08 9:24 ` Laurent Dufour 2017-09-08 9:24 ` Laurent Dufour 2017-09-11 0:45 ` Sergey Senozhatsky 2017-09-11 0:45 ` Sergey Senozhatsky 2017-09-11 6:28 ` Laurent Dufour 2017-09-11 6:28 ` Laurent Dufour 2017-08-21 6:28 ` Anshuman Khandual 2017-08-21 6:28 ` Anshuman Khandual 2017-08-22 0:41 ` Paul E. McKenney 2017-08-22 0:41 ` Paul E. McKenney 2017-08-25 9:41 ` Laurent Dufour 2017-08-25 9:41 ` Laurent Dufour
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1503007519-26777-21-git-send-email-ldufour@linux.vnet.ibm.com \ --to=ldufour@linux.vnet.ibm.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=dave@stgolabs.net \ --cc=haren@linux.vnet.ibm.com \ --cc=hpa@zytor.com \ --cc=jack@suse.cz \ --cc=khandual@linux.vnet.ibm.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mhocko@kernel.org \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=npiggin@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=tglx@linutronix.de \ --cc=tim.c.chen@linux.intel.com \ --cc=will.deacon@arm.com \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.