From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> To: Andrea Arcangeli <aarcange@redhat.com>, Andrew Morton <akpm@linux-foundation.org> Cc: Al Viro <viro@zeniv.linux.org.uk>, Hugh Dickins <hughd@google.com>, Wu Fengguang <fengguang.wu@intel.com>, Jan Kara <jack@suse.cz>, Mel Gorman <mgorman@suse.de>, linux-mm@kvack.org, Andi Kleen <ak@linux.intel.com>, Matthew Wilcox <matthew.r.wilcox@intel.com>, "Kirill A. Shutemov" <kirill@shutemov.name>, Hillf Danton <dhillf@gmail.com>, Dave Hansen <dave@sr71.net>, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: [PATCHv3, RFC 21/34] thp: wait_split_huge_page(): serialize over i_mmap_mutex too Date: Fri, 5 Apr 2013 14:59:45 +0300 [thread overview] Message-ID: <1365163198-29726-22-git-send-email-kirill.shutemov@linux.intel.com> (raw) In-Reply-To: <1365163198-29726-1-git-send-email-kirill.shutemov@linux.intel.com> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Since we're going to have huge pages backed by files, wait_split_huge_page() has to serialize not only over anon_vma_lock, but over i_mmap_mutex too. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> --- include/linux/huge_mm.h | 15 ++++++++++++--- mm/huge_memory.c | 4 ++-- mm/memory.c | 4 ++-- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a54939c..b53e295 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -113,11 +113,20 @@ extern void __split_huge_page_pmd(struct vm_area_struct *vma, __split_huge_page_pmd(__vma, __address, \ ____pmd); \ } while (0) -#define wait_split_huge_page(__anon_vma, __pmd) \ +#define wait_split_huge_page(__vma, __pmd) \ do { \ pmd_t *____pmd = (__pmd); \ - anon_vma_lock_write(__anon_vma); \ - anon_vma_unlock_write(__anon_vma); \ + struct address_space *__mapping = \ + vma->vm_file->f_mapping; \ + struct anon_vma *__anon_vma = (__vma)->anon_vma; \ + if (__mapping) \ + mutex_lock(&__mapping->i_mmap_mutex); \ + if (__anon_vma) { \ + anon_vma_lock_write(__anon_vma); \ + anon_vma_unlock_write(__anon_vma); \ + } \ + if (__mapping) \ + mutex_unlock(&__mapping->i_mmap_mutex); \ BUG_ON(pmd_trans_splitting(*____pmd) || \ pmd_trans_huge(*____pmd)); \ } while (0) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ac0dc80..7c48f58 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -907,7 +907,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, spin_unlock(&dst_mm->page_table_lock); pte_free(dst_mm, pgtable); - wait_split_huge_page(vma->anon_vma, src_pmd); /* src_vma */ + wait_split_huge_page(vma, src_pmd); /* src_vma */ goto out; } src_page = pmd_page(pmd); @@ -1480,7 +1480,7 @@ int __pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) if (likely(pmd_trans_huge(*pmd))) { if (unlikely(pmd_trans_splitting(*pmd))) { spin_unlock(&vma->vm_mm->page_table_lock); - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); return -1; } else { /* Thp mapped by 'pmd' is stable, so we can diff --git a/mm/memory.c b/mm/memory.c index 9da540f..2895f0e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -619,7 +619,7 @@ int __pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, if (new) pte_free(mm, new); if (wait_split_huge_page) - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); return 0; } @@ -1529,7 +1529,7 @@ struct page *follow_page_mask(struct vm_area_struct *vma, if (likely(pmd_trans_huge(*pmd))) { if (unlikely(pmd_trans_splitting(*pmd))) { spin_unlock(&mm->page_table_lock); - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); } else { page = follow_trans_huge_pmd(vma, address, pmd, flags); -- 1.7.10.4
WARNING: multiple messages have this Message-ID (diff)
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> To: Andrea Arcangeli <aarcange@redhat.com>, Andrew Morton <akpm@linux-foundation.org> Cc: Al Viro <viro@zeniv.linux.org.uk>, Hugh Dickins <hughd@google.com>, Wu Fengguang <fengguang.wu@intel.com>, Jan Kara <jack@suse.cz>, Mel Gorman <mgorman@suse.de>, linux-mm@kvack.org, Andi Kleen <ak@linux.intel.com>, Matthew Wilcox <matthew.r.wilcox@intel.com>, "Kirill A. Shutemov" <kirill@shutemov.name>, Hillf Danton <dhillf@gmail.com>, Dave Hansen <dave@sr71.net>, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Subject: [PATCHv3, RFC 21/34] thp: wait_split_huge_page(): serialize over i_mmap_mutex too Date: Fri, 5 Apr 2013 14:59:45 +0300 [thread overview] Message-ID: <1365163198-29726-22-git-send-email-kirill.shutemov@linux.intel.com> (raw) In-Reply-To: <1365163198-29726-1-git-send-email-kirill.shutemov@linux.intel.com> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com> Since we're going to have huge pages backed by files, wait_split_huge_page() has to serialize not only over anon_vma_lock, but over i_mmap_mutex too. Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> --- include/linux/huge_mm.h | 15 ++++++++++++--- mm/huge_memory.c | 4 ++-- mm/memory.c | 4 ++-- 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a54939c..b53e295 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -113,11 +113,20 @@ extern void __split_huge_page_pmd(struct vm_area_struct *vma, __split_huge_page_pmd(__vma, __address, \ ____pmd); \ } while (0) -#define wait_split_huge_page(__anon_vma, __pmd) \ +#define wait_split_huge_page(__vma, __pmd) \ do { \ pmd_t *____pmd = (__pmd); \ - anon_vma_lock_write(__anon_vma); \ - anon_vma_unlock_write(__anon_vma); \ + struct address_space *__mapping = \ + vma->vm_file->f_mapping; \ + struct anon_vma *__anon_vma = (__vma)->anon_vma; \ + if (__mapping) \ + mutex_lock(&__mapping->i_mmap_mutex); \ + if (__anon_vma) { \ + anon_vma_lock_write(__anon_vma); \ + anon_vma_unlock_write(__anon_vma); \ + } \ + if (__mapping) \ + mutex_unlock(&__mapping->i_mmap_mutex); \ BUG_ON(pmd_trans_splitting(*____pmd) || \ pmd_trans_huge(*____pmd)); \ } while (0) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index ac0dc80..7c48f58 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -907,7 +907,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, spin_unlock(&dst_mm->page_table_lock); pte_free(dst_mm, pgtable); - wait_split_huge_page(vma->anon_vma, src_pmd); /* src_vma */ + wait_split_huge_page(vma, src_pmd); /* src_vma */ goto out; } src_page = pmd_page(pmd); @@ -1480,7 +1480,7 @@ int __pmd_trans_huge_lock(pmd_t *pmd, struct vm_area_struct *vma) if (likely(pmd_trans_huge(*pmd))) { if (unlikely(pmd_trans_splitting(*pmd))) { spin_unlock(&vma->vm_mm->page_table_lock); - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); return -1; } else { /* Thp mapped by 'pmd' is stable, so we can diff --git a/mm/memory.c b/mm/memory.c index 9da540f..2895f0e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -619,7 +619,7 @@ int __pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, if (new) pte_free(mm, new); if (wait_split_huge_page) - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); return 0; } @@ -1529,7 +1529,7 @@ struct page *follow_page_mask(struct vm_area_struct *vma, if (likely(pmd_trans_huge(*pmd))) { if (unlikely(pmd_trans_splitting(*pmd))) { spin_unlock(&mm->page_table_lock); - wait_split_huge_page(vma->anon_vma, pmd); + wait_split_huge_page(vma, pmd); } else { page = follow_trans_huge_pmd(vma, address, pmd, flags); -- 1.7.10.4 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-04-05 11:59 UTC|newest] Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top 2013-04-05 11:59 [PATCHv3, RFC 00/34] Transparent huge page cache Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 01/34] mm: drop actor argument of do_generic_file_read() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 02/34] block: implement add_bdi_stat() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 03/34] mm: implement zero_huge_user_segment and friends Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 04/34] radix-tree: implement preload for multiple contiguous elements Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 05/34] memcg, thp: charge huge cache pages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 06/34] thp, mm: avoid PageUnevictable on active/inactive lru lists Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 07/34] thp, mm: basic defines for transparent huge page cache Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 08/34] thp, mm: introduce mapping_can_have_hugepages() predicate Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 09/34] thp: represent file thp pages in meminfo and friends Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-08 19:38 ` Dave Hansen 2013-04-08 19:38 ` Dave Hansen 2013-04-16 14:49 ` Kirill A. Shutemov 2013-04-16 14:49 ` Kirill A. Shutemov 2013-04-16 15:11 ` Dave Hansen 2013-04-16 15:11 ` Dave Hansen 2013-04-05 11:59 ` [PATCHv3, RFC 10/34] thp, mm: rewrite add_to_page_cache_locked() to support huge pages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 11/34] mm: trace filemap: dump page order Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 12/34] thp, mm: rewrite delete_from_page_cache() to support huge pages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 13/34] thp, mm: trigger bug in replace_page_cache_page() on THP Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 14/34] thp, mm: locking tail page is a bug Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 15/34] thp, mm: handle tail pages in page_cache_get_speculative() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 16/34] thp, mm: add event counters for huge page alloc on write to a file Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 17/34] thp, mm: implement grab_thp_write_begin() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 18/34] thp, mm: naive support of thp in generic read/write routines Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 19/34] thp, libfs: initial support of thp in simple_read/write_begin/write_end Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 20/34] thp: handle file pages in split_huge_page() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov [this message] 2013-04-05 11:59 ` [PATCHv3, RFC 21/34] thp: wait_split_huge_page(): serialize over i_mmap_mutex too Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 22/34] thp, mm: truncate support for transparent huge page cache Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 23/34] thp, mm: split huge page on mmap file page Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 24/34] ramfs: enable transparent huge page cache Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 25/34] x86-64, mm: proper alignment mappings with hugepages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 26/34] mm: add huge_fault() callback to vm_operations_struct Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 27/34] thp: prepare zap_huge_pmd() to uncharge file pages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 28/34] thp: move maybe_pmd_mkwrite() out of mk_huge_pmd() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 29/34] thp, mm: basic huge_fault implementation for generic_file_vm_ops Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 30/34] thp: extract fallback path from do_huge_pmd_anonymous_page() to a function Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 31/34] thp: initial implementation of do_huge_linear_fault() Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-08 18:46 ` Dave Hansen 2013-04-08 18:46 ` Dave Hansen 2013-04-08 18:52 ` Dave Hansen 2013-04-08 18:52 ` Dave Hansen 2013-04-17 14:38 ` Kirill A. Shutemov 2013-04-17 14:38 ` Kirill A. Shutemov 2013-04-17 22:07 ` Dave Hansen 2013-04-18 16:09 ` Kirill A. Shutemov 2013-04-18 16:09 ` Kirill A. Shutemov 2013-04-18 16:19 ` Kirill A. Shutemov 2013-04-18 16:19 ` Kirill A. Shutemov 2013-04-18 16:19 ` Kirill A. Shutemov 2013-04-18 16:20 ` Dave Hansen 2013-04-18 16:20 ` Dave Hansen 2013-04-18 16:38 ` Kirill A. Shutemov 2013-04-18 16:38 ` Kirill A. Shutemov 2013-04-18 16:42 ` Dave Hansen 2013-04-18 16:42 ` Dave Hansen 2013-04-05 11:59 ` [PATCHv3, RFC 32/34] thp: handle write-protect exception to file-backed huge pages Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-08 19:07 ` Dave Hansen 2013-04-08 19:07 ` Dave Hansen 2013-04-26 15:31 ` Kirill A. Shutemov 2013-04-26 15:31 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 33/34] thp: call __vma_adjust_trans_huge() for file-backed VMA Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-05 11:59 ` [PATCHv3, RFC 34/34] thp: map file-backed huge pages on fault Kirill A. Shutemov 2013-04-05 11:59 ` Kirill A. Shutemov 2013-04-07 0:40 ` [PATCHv3, RFC 00/34] Transparent huge page cache Ric Mason 2013-04-07 0:40 ` Ric Mason 2013-04-15 16:02 ` IOZone with transparent " Kirill A. Shutemov 2013-04-15 16:02 ` Kirill A. Shutemov 2013-04-15 18:17 ` [RESEND] " Kirill A. Shutemov 2013-04-15 18:17 ` Kirill A. Shutemov 2013-04-15 23:19 ` Dave Hansen 2013-04-15 23:19 ` Dave Hansen 2013-04-16 5:57 ` Kirill A. Shutemov 2013-04-16 5:57 ` Kirill A. Shutemov 2013-04-16 6:11 ` Dave Hansen 2013-04-16 6:11 ` Dave Hansen
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1365163198-29726-22-git-send-email-kirill.shutemov@linux.intel.com \ --to=kirill.shutemov@linux.intel.com \ --cc=aarcange@redhat.com \ --cc=ak@linux.intel.com \ --cc=akpm@linux-foundation.org \ --cc=dave@sr71.net \ --cc=dhillf@gmail.com \ --cc=fengguang.wu@intel.com \ --cc=hughd@google.com \ --cc=jack@suse.cz \ --cc=kirill@shutemov.name \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=matthew.r.wilcox@intel.com \ --cc=mgorman@suse.de \ --cc=viro@zeniv.linux.org.uk \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.