From: Laurent Dufour <ldufour@linux.ibm.com> To: akpm@linux-foundation.org, mhocko@kernel.org, peterz@infradead.org, kirill@shutemov.name, ak@linux.intel.com, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, aneesh.kumar@linux.ibm.com, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com>, Sergey Senozhatsky <sergey.senozhatsky@gmail.com>, sergey.senozhatsky.work@gmail.com, Andrea Arcangeli <aarcange@redhat.com>, Alexei Starovoitov <alexei.starovoitov@gmail.com>, kemi.wang@intel.com, Daniel Jordan <daniel.m.jordan@oracle.com>, David Rientjes <rientjes@google.com>, Jerome Glisse <jglisse@redhat.com>, Ganesh Mahendran <opensource.ganesh@gmail.com>, Minchan Kim <minchan@kernel.org>, Punit Agrawal <punitagrawal@gmail.com>, vinayak menon <vinayakm.list@gmail.com>, Yang Shi <yang.shi@linux.alibaba.com>, zhong jiang <zhongjiang@huawei.com>, Haiyan Song <haiyanx.song@intel.com>, Balbir Singh <bsingharora@gmail.com>, sj38.park@gmail.com, Michel Lespinasse <walken@google.com>, Mike Rapoport <rppt@linux.ibm.com> Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, haren@linux.vnet.ibm.com, npiggin@gmail.com, paulmck@linux.vnet.ibm.com, Tim Chen <tim.c.chen@linux.intel.com>, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [PATCH v12 12/31] mm: protect SPF handler against anon_vma changes Date: Tue, 16 Apr 2019 15:45:03 +0200 [thread overview] Message-ID: <20190416134522.17540-13-ldufour@linux.ibm.com> (raw) In-Reply-To: <20190416134522.17540-1-ldufour@linux.ibm.com> The speculative page fault handler must be protected against anon_vma changes. This is because page_add_new_anon_rmap() is called during the speculative path. In addition, don't try speculative page fault if the VMA don't have an anon_vma structure allocated because its allocation should be protected by the mmap_sem. In __vma_adjust() when importer->anon_vma is set, there is no need to protect against speculative page faults since speculative page fault is aborted if the vma->anon_vma is not set. When calling page_add_new_anon_rmap() vma->anon_vma is necessarily valid since we checked for it when locking the pte and the anon_vma is removed once the pte is unlocked. So even if the speculative page fault handler is running concurrently with do_unmap(), as the pte is locked in unmap_region() - through unmap_vmas() - and the anon_vma unlinked later, because we check for the vma sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com> --- mm/memory.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index 423fa8ea0569..2cf7b6185daa 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -377,7 +377,9 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma, * Hide vma from rmap and truncate_pagecache before freeing * pgtables */ + vm_write_begin(vma); unlink_anon_vmas(vma); + vm_write_end(vma); unlink_file_vma(vma); if (is_vm_hugetlb_page(vma)) { @@ -391,7 +393,9 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma, && !is_vm_hugetlb_page(next)) { vma = next; next = vma->vm_next; + vm_write_begin(vma); unlink_anon_vmas(vma); + vm_write_end(vma); unlink_file_vma(vma); } free_pgd_range(tlb, addr, vma->vm_end, -- 2.21.0
WARNING: multiple messages have this Message-ID (diff)
From: Laurent Dufour <ldufour@linux.ibm.com> To: akpm@linux-foundation.org, mhocko@kernel.org, peterz@infradead.org, kirill@shutemov.name, ak@linux.intel.com, dave@stgolabs.net, jack@suse.cz, Matthew Wilcox <willy@infradead.org>, aneesh.kumar@linux.ibm.com, benh@kernel.crashing.org, mpe@ellerman.id.au, paulus@samba.org, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, hpa@zytor.com, Will Deacon <will.deacon@arm.com>, Sergey Senozhatsky <sergey.senozhatsky@gmail.com>, sergey.senozhatsky.work@gmail.com, Andrea Arcangeli <aarcange@redhat.com>, Alexei Starovoitov <alexei.starovoitov@gmail.com>, kemi.wang@intel.com, Daniel Jordan <daniel.m.jordan@oracle.com>, David Rientjes <rientjes@google.com>, Jerome Glisse <jglisse@redhat.com>, Ganesh Mahendran <opensource.ganesh@gmail.com>, Minchan Kim <minchan@kernel.org>, Punit Agrawal <punitagrawal@gmail.com>, vinayak menon <vinayakm.list@gmail.com>, Yang Shi <yang.shi@linux.alibaba.com>, zhong jiang <zhongjiang@huawei.com>, Haiyan Song <haiyanx.song@intel.com>, Balbir Singh <bsingharora@gmail.com>, sj38.park@gmail.com, Michel Lespinasse <walken@google.com>, Mike Rapoport <rppt@linux.ibm.com> Cc: linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org, npiggin@gmail.com, linux-mm@kvack.org, paulmck@linux.vnet.ibm.com, Tim Chen <tim.c.chen@linux.intel.com>, haren@linux.vnet.ibm.com Subject: [PATCH v12 12/31] mm: protect SPF handler against anon_vma changes Date: Tue, 16 Apr 2019 15:45:03 +0200 [thread overview] Message-ID: <20190416134522.17540-13-ldufour@linux.ibm.com> (raw) In-Reply-To: <20190416134522.17540-1-ldufour@linux.ibm.com> The speculative page fault handler must be protected against anon_vma changes. This is because page_add_new_anon_rmap() is called during the speculative path. In addition, don't try speculative page fault if the VMA don't have an anon_vma structure allocated because its allocation should be protected by the mmap_sem. In __vma_adjust() when importer->anon_vma is set, there is no need to protect against speculative page faults since speculative page fault is aborted if the vma->anon_vma is not set. When calling page_add_new_anon_rmap() vma->anon_vma is necessarily valid since we checked for it when locking the pte and the anon_vma is removed once the pte is unlocked. So even if the speculative page fault handler is running concurrently with do_unmap(), as the pte is locked in unmap_region() - through unmap_vmas() - and the anon_vma unlinked later, because we check for the vma sequence counter which is updated in unmap_page_range() before locking the pte, and then in free_pgtables() so when locking the pte the change will be detected. Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com> --- mm/memory.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index 423fa8ea0569..2cf7b6185daa 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -377,7 +377,9 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma, * Hide vma from rmap and truncate_pagecache before freeing * pgtables */ + vm_write_begin(vma); unlink_anon_vmas(vma); + vm_write_end(vma); unlink_file_vma(vma); if (is_vm_hugetlb_page(vma)) { @@ -391,7 +393,9 @@ void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *vma, && !is_vm_hugetlb_page(next)) { vma = next; next = vma->vm_next; + vm_write_begin(vma); unlink_anon_vmas(vma); + vm_write_end(vma); unlink_file_vma(vma); } free_pgd_range(tlb, addr, vma->vm_end, -- 2.21.0
next prev parent reply other threads:[~2019-04-16 13:47 UTC|newest] Thread overview: 197+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-04-16 13:44 [PATCH v12 00/31] Speculative page faults Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-16 13:44 ` [PATCH v12 01/31] mm: introduce CONFIG_SPECULATIVE_PAGE_FAULT Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 21:47 ` Jerome Glisse 2019-04-18 21:47 ` Jerome Glisse 2019-04-23 15:21 ` Laurent Dufour 2019-04-23 15:21 ` Laurent Dufour 2019-04-16 13:44 ` [PATCH v12 02/31] x86/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 21:48 ` Jerome Glisse 2019-04-18 21:48 ` Jerome Glisse 2019-04-16 13:44 ` [PATCH v12 03/31] powerpc/mm: set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 21:49 ` Jerome Glisse 2019-04-18 21:49 ` Jerome Glisse 2019-04-16 13:44 ` [PATCH v12 04/31] arm64/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-16 14:27 ` Mark Rutland 2019-04-16 14:27 ` Mark Rutland 2019-04-16 14:31 ` Laurent Dufour 2019-04-16 14:31 ` Laurent Dufour 2019-04-16 14:41 ` Mark Rutland 2019-04-16 14:41 ` Mark Rutland 2019-04-18 21:51 ` Jerome Glisse 2019-04-18 21:51 ` Jerome Glisse 2019-04-23 15:36 ` Laurent Dufour 2019-04-23 15:36 ` Laurent Dufour 2019-04-23 16:19 ` Mark Rutland 2019-04-23 16:19 ` Mark Rutland 2019-04-24 10:34 ` Laurent Dufour 2019-04-24 10:34 ` Laurent Dufour 2019-04-16 13:44 ` [PATCH v12 05/31] mm: prepare for FAULT_FLAG_SPECULATIVE Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 22:04 ` Jerome Glisse 2019-04-18 22:04 ` Jerome Glisse 2019-04-23 15:45 ` Laurent Dufour 2019-04-23 15:45 ` Laurent Dufour 2019-04-16 13:44 ` [PATCH v12 06/31] mm: introduce pte_spinlock " Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 22:05 ` Jerome Glisse 2019-04-18 22:05 ` Jerome Glisse 2019-04-16 13:44 ` [PATCH v12 07/31] mm: make pte_unmap_same compatible with SPF Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 22:10 ` Jerome Glisse 2019-04-18 22:10 ` Jerome Glisse 2019-04-23 15:43 ` Matthew Wilcox 2019-04-23 15:43 ` Matthew Wilcox 2019-04-23 15:47 ` Laurent Dufour 2019-04-23 15:47 ` Laurent Dufour 2019-04-16 13:44 ` [PATCH v12 08/31] mm: introduce INIT_VMA() Laurent Dufour 2019-04-16 13:44 ` Laurent Dufour 2019-04-18 22:22 ` Jerome Glisse 2019-04-18 22:22 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 09/31] mm: VMA sequence count Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-18 22:48 ` Jerome Glisse 2019-04-18 22:48 ` Jerome Glisse 2019-04-19 15:45 ` Laurent Dufour 2019-04-19 15:45 ` Laurent Dufour 2019-04-22 15:51 ` Jerome Glisse 2019-04-22 15:51 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 10/31] mm: protect VMA modifications using " Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 19:43 ` Jerome Glisse 2019-04-22 19:43 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 11/31] mm: protect mremap() against SPF hanlder Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 19:51 ` Jerome Glisse 2019-04-22 19:51 ` Jerome Glisse 2019-04-23 15:51 ` Laurent Dufour 2019-04-23 15:51 ` Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour [this message] 2019-04-16 13:45 ` [PATCH v12 12/31] mm: protect SPF handler against anon_vma changes Laurent Dufour 2019-04-22 19:53 ` Jerome Glisse 2019-04-22 19:53 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 13/31] mm: cache some VMA fields in the vm_fault structure Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:06 ` Jerome Glisse 2019-04-22 20:06 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 14/31] mm/migrate: Pass vm_fault pointer to migrate_misplaced_page() Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:09 ` Jerome Glisse 2019-04-22 20:09 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 15/31] mm: introduce __lru_cache_add_active_or_unevictable Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:11 ` Jerome Glisse 2019-04-22 20:11 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 16/31] mm: introduce __vm_normal_page() Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:15 ` Jerome Glisse 2019-04-22 20:15 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 17/31] mm: introduce __page_add_new_anon_rmap() Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:18 ` Jerome Glisse 2019-04-22 20:18 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 18/31] mm: protect against PTE changes done by dup_mmap() Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:32 ` Jerome Glisse 2019-04-22 20:32 ` Jerome Glisse 2019-04-24 10:33 ` Laurent Dufour 2019-04-24 10:33 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 19/31] mm: protect the RB tree with a sequence lock Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:33 ` Jerome Glisse 2019-04-22 20:33 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 20/31] mm: introduce vma reference counter Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:36 ` Jerome Glisse 2019-04-22 20:36 ` Jerome Glisse 2019-04-24 14:26 ` Laurent Dufour 2019-04-24 14:26 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 21/31] mm: Introduce find_vma_rcu() Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 20:57 ` Jerome Glisse 2019-04-22 20:57 ` Jerome Glisse 2019-04-24 14:39 ` Laurent Dufour 2019-04-24 14:39 ` Laurent Dufour 2019-04-23 9:27 ` Peter Zijlstra 2019-04-23 9:27 ` Peter Zijlstra 2019-04-23 18:13 ` Davidlohr Bueso 2019-04-23 18:13 ` Davidlohr Bueso 2019-04-24 7:57 ` Laurent Dufour 2019-04-24 7:57 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 22/31] mm: provide speculative fault infrastructure Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 21:26 ` Jerome Glisse 2019-04-22 21:26 ` Jerome Glisse 2019-04-24 14:56 ` Laurent Dufour 2019-04-24 14:56 ` Laurent Dufour 2019-04-24 15:13 ` Jerome Glisse 2019-04-24 15:13 ` Jerome Glisse 2019-04-16 13:45 ` [PATCH v12 23/31] mm: don't do swap readahead during speculative page fault Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 21:36 ` Jerome Glisse 2019-04-22 21:36 ` Jerome Glisse 2019-04-24 14:57 ` Laurent Dufour 2019-04-24 14:57 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 24/31] mm: adding speculative page fault failure trace events Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 25/31] perf: add a speculative page fault sw event Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 26/31] perf tools: add support for the SPF perf event Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 27/31] mm: add speculative page fault vmstats Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 28/31] x86/mm: add speculative pagefault handling Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 29/31] powerpc/mm: add speculative page fault Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 30/31] arm64/mm: " Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-16 13:45 ` [PATCH v12 31/31] mm: Add a speculative page fault switch in sysctl Laurent Dufour 2019-04-16 13:45 ` Laurent Dufour 2019-04-22 21:29 ` [PATCH v12 00/31] Speculative page faults Michel Lespinasse 2019-04-22 21:29 ` Michel Lespinasse 2019-04-22 21:29 ` Michel Lespinasse 2019-04-23 9:38 ` Peter Zijlstra 2019-04-23 9:38 ` Peter Zijlstra 2019-04-24 7:33 ` Laurent Dufour 2019-04-24 7:33 ` Laurent Dufour 2019-04-27 1:53 ` Michel Lespinasse 2019-04-27 1:53 ` Michel Lespinasse 2019-04-23 10:47 ` Michal Hocko 2019-04-23 10:47 ` Michal Hocko 2019-04-23 12:41 ` Matthew Wilcox 2019-04-23 12:41 ` Matthew Wilcox 2019-04-23 12:48 ` Peter Zijlstra 2019-04-23 12:48 ` Peter Zijlstra 2019-04-23 13:42 ` Michal Hocko 2019-04-23 13:42 ` Michal Hocko 2019-04-24 18:01 ` Laurent Dufour 2019-04-24 18:01 ` Laurent Dufour 2019-04-27 6:00 ` Michel Lespinasse 2019-04-27 6:00 ` Michel Lespinasse 2019-04-23 11:35 ` Anshuman Khandual 2019-04-23 11:35 ` Anshuman Khandual 2019-06-06 6:51 ` Haiyan Song 2019-06-06 6:51 ` Haiyan Song 2019-06-14 8:37 ` Laurent Dufour 2019-06-14 8:37 ` Laurent Dufour 2019-06-14 8:44 ` Laurent Dufour 2019-06-14 8:44 ` Laurent Dufour 2019-06-20 8:19 ` Haiyan Song 2019-06-20 8:19 ` Haiyan Song 2020-07-06 9:25 ` Chinwen Chang 2020-07-06 9:25 ` Chinwen Chang 2020-07-06 12:27 ` Laurent Dufour 2020-07-06 12:27 ` Laurent Dufour 2020-07-07 5:31 ` Chinwen Chang 2020-07-07 5:31 ` Chinwen Chang 2020-12-14 2:03 ` Joel Fernandes 2020-12-14 2:03 ` Joel Fernandes 2020-12-14 9:36 ` Laurent Dufour 2020-12-14 9:36 ` Laurent Dufour 2020-12-14 18:10 ` Joel Fernandes 2020-12-14 18:10 ` Joel Fernandes
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190416134522.17540-13-ldufour@linux.ibm.com \ --to=ldufour@linux.ibm.com \ --cc=aarcange@redhat.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=alexei.starovoitov@gmail.com \ --cc=aneesh.kumar@linux.ibm.com \ --cc=benh@kernel.crashing.org \ --cc=bsingharora@gmail.com \ --cc=daniel.m.jordan@oracle.com \ --cc=dave@stgolabs.net \ --cc=haiyanx.song@intel.com \ --cc=haren@linux.vnet.ibm.com \ --cc=hpa@zytor.com \ --cc=jack@suse.cz \ --cc=jglisse@redhat.com \ --cc=kemi.wang@intel.com \ --cc=kirill@shutemov.name \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linuxppc-dev@lists.ozlabs.org \ --cc=mhocko@kernel.org \ --cc=minchan@kernel.org \ --cc=mingo@redhat.com \ --cc=mpe@ellerman.id.au \ --cc=npiggin@gmail.com \ --cc=opensource.ganesh@gmail.com \ --cc=paulmck@linux.vnet.ibm.com \ --cc=paulus@samba.org \ --cc=peterz@infradead.org \ --cc=punitagrawal@gmail.com \ --cc=rientjes@google.com \ --cc=rppt@linux.ibm.com \ --cc=sergey.senozhatsky.work@gmail.com \ --cc=sergey.senozhatsky@gmail.com \ --cc=sj38.park@gmail.com \ --cc=tglx@linutronix.de \ --cc=tim.c.chen@linux.intel.com \ --cc=vinayakm.list@gmail.com \ --cc=walken@google.com \ --cc=will.deacon@arm.com \ --cc=willy@infradead.org \ --cc=x86@kernel.org \ --cc=yang.shi@linux.alibaba.com \ --cc=zhongjiang@huawei.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.