Does this look good to you? I want it to stay in this thread so I didn't send it by git imap-send. >From e637cfc915107f0b7df11ec58046c1f7fae87a10 Mon Sep 17 00:00:00 2001 From: Ning Qu Date: Fri, 9 Aug 2013 13:54:24 -0700 Subject: [PATCH] thp: Fix deadlock situation in vma_adjust with huge page in page cache. In vma_adjust, the current code grabs i_mmap_mutex before calling vma_adjust_trans_huge. This used to be fine until huge page in page cache comes in. The problem is the underlying function split_file_huge_page will also grab the i_mmap_mutex before splitting the huge page in page cache. Obviously this is causing deadlock situation. This fix is to move the vma_adjust_trans_huge before grab the lock for file, the same as what the function is currently doing for anonymous memory. Tested, everything works fine so far. Signed-off-by: Ning Qu --- mm/mmap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 519ce78..accf1b3 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -750,6 +750,8 @@ again: remove_next = 1 + (end > next->vm_end); } } + vma_adjust_trans_huge(vma, start, end, adjust_next); + if (file) { mapping = file->f_mapping; if (!(vma->vm_flags & VM_NONLINEAR)) { @@ -773,8 +775,6 @@ again: remove_next = 1 + (end > next->vm_end); } } - vma_adjust_trans_huge(vma, start, end, adjust_next); - anon_vma = vma->anon_vma; if (!anon_vma && adjust_next) anon_vma = next->anon_vma; -- 1.8.3 Best wishes, -- Ning Qu (曲宁) | Software Engineer | quning@google.com | +1-408-418-6066 On Fri, Aug 9, 2013 at 7:49 AM, Ning Qu wrote: > Sure! > On Aug 9, 2013 7:42 AM, "Kirill A. Shutemov" < > kirill.shutemov@linux.intel.com> wrote: > >> Ning Qu wrote: >> > I just tried, and it seems working fine now without the deadlock >> anymore. I >> > can run some big internal test with about 40GB files in sysv shm. Just >> move >> > the line before the locking happens in vma_adjust, something as below, >> the >> > line number is not accurate because my patch is based on another tree >> right >> > now. >> >> Looks okay to me. Could you prepare real patch (description, etc.). I'll >> add it to my patchset. >> >> > >> > --- a/mm/mmap.c >> > +++ b/mm/mmap.c >> > @@ -581,6 +581,8 @@ again: remove_next = 1 + (end > >> > next->vm_end); >> > } >> > } >> > >> > + vma_adjust_trans_huge(vma, start, end, adjust_next); >> > + >> > if (file) { >> > mapping = file->f_mapping; >> > if (!(vma->vm_flags & VM_NONLINEAR)) >> > @@ -597,8 +599,6 @@ again: remove_next = 1 + (end > >> > next->vm_end); >> > } >> > } >> > >> > - vma_adjust_trans_huge(vma, start, end, adjust_next); >> > - >> > anon_vma = vma->anon_vma; >> > if (!anon_vma && adjust_next) >> > anon_vma = next->anon_vma; >> > >> > >> > Best wishes, >> > -- >> > Ning Qu (曲宁) | Software Engineer | quning@google.com | +1-408-418-6066 >> >> -- >> Kirill A. Shutemov >> >