* [PATCH] mm/huge_memory: fix the memory leak due to the race
@ 2016-06-21 14:05 zhongjiang
2016-06-21 14:32 ` kbuild test robot
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: zhongjiang @ 2016-06-21 14:05 UTC (permalink / raw)
To: mhocko, akpm; +Cc: linux-mm, linux-kernel
From: zhong jiang <zhongjiang@huawei.com>
with great pressure, I run some test cases. As a result, I found
that the THP is not freed, it is detected by check_mm().
BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
Consider the following race :
CPU0 CPU1
__handle_mm_fault()
wp_huge_pmd()
do_huge_pmd_wp_page()
pmdp_huge_clear_flush_notify()
(pmd_none = true)
exit_mmap()
unmap_vmas()
zap_pmd_range()
pmd_none_or_trans_huge_or_clear_bad()
(result in memory leak)
set_pmd_at()
because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
and it make the pmd entry to be null. Therefore, The memory leak can occur.
The patch fix the scenario that the pmd entry can lead to be null.
Signed-off-by: zhong jiang <zhongjiang@huawei.com>
---
mm/huge_memory.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e10a4fe..ef04b94 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1340,11 +1340,11 @@ alloc:
pmd_t entry;
entry = mk_huge_pmd(new_page, vma->vm_page_prot);
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
- pmdp_huge_clear_flush_notify(vma, haddr, pmd);
+ pmdp_invalidate(vma, haddr, pmd);
page_add_new_anon_rmap(new_page, vma, haddr, true);
mem_cgroup_commit_charge(new_page, memcg, false, true);
lru_cache_add_active_or_unevictable(new_page, vma);
- set_pmd_at(mm, haddr, pmd, entry);
+ pmd_populate(mm, pmd, entry);
update_mmu_cache_pmd(vma, address, pmd);
if (!page) {
add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 14:05 [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang
@ 2016-06-21 14:32 ` kbuild test robot
2016-06-21 14:37 ` Kirill A. Shutemov
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: kbuild test robot @ 2016-06-21 14:32 UTC (permalink / raw)
To: zhongjiang; +Cc: kbuild-all, mhocko, akpm, linux-mm, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2494 bytes --]
Hi,
[auto build test ERROR on v4.7-rc4]
[also build test ERROR on next-20160621]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/zhongjiang/mm-huge_memory-fix-the-memory-leak-due-to-the-race/20160621-221736
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 5.3.1-8) 5.3.1 20160205
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=sparc64
All errors (new ones prefixed by >>):
In file included from arch/sparc/include/asm/pgalloc.h:4:0,
from arch/sparc/include/asm/tlb_64.h:6,
from arch/sparc/include/asm/tlb.h:4,
from mm/huge_memory.c:34:
mm/huge_memory.c: In function 'do_huge_pmd_wp_page':
>> mm/huge_memory.c:1383:25: error: incompatible type for argument 3 of 'pmd_set'
pmd_populate(mm, pmd, entry);
^
arch/sparc/include/asm/pgalloc_64.h:72:54: note: in definition of macro 'pmd_populate'
#define pmd_populate(MM, PMD, PTE) pmd_set(MM, PMD, PTE)
^
In file included from arch/sparc/include/asm/pgtable.h:4:0,
from include/linux/mm.h:68,
from mm/huge_memory.c:10:
arch/sparc/include/asm/pgtable_64.h:796:20: note: expected 'pte_t * {aka struct <anonymous> *}' but argument is of type 'pmd_t {aka struct <anonymous>}'
static inline void pmd_set(struct mm_struct *mm, pmd_t *pmdp, pte_t *ptep)
^
vim +/pmd_set +1383 mm/huge_memory.c
1377 entry = mk_huge_pmd(new_page, vma->vm_page_prot);
1378 entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
1379 pmdp_invalidate(vma, haddr, pmd);
1380 page_add_new_anon_rmap(new_page, vma, haddr, true);
1381 mem_cgroup_commit_charge(new_page, memcg, false, true);
1382 lru_cache_add_active_or_unevictable(new_page, vma);
> 1383 pmd_populate(mm, pmd, entry);
1384 update_mmu_cache_pmd(vma, address, pmd);
1385 if (!page) {
1386 add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 46455 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 14:05 [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang
2016-06-21 14:32 ` kbuild test robot
@ 2016-06-21 14:37 ` Kirill A. Shutemov
[not found] ` <57695AEB.8030509@huawei.com>
2016-06-21 14:37 ` Michal Hocko
2016-06-21 14:42 ` kbuild test robot
3 siblings, 1 reply; 9+ messages in thread
From: Kirill A. Shutemov @ 2016-06-21 14:37 UTC (permalink / raw)
To: zhongjiang; +Cc: mhocko, akpm, linux-mm, linux-kernel
On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
>
> with great pressure, I run some test cases. As a result, I found
> that the THP is not freed, it is detected by check_mm().
>
> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>
> Consider the following race :
>
> CPU0 CPU1
> __handle_mm_fault()
> wp_huge_pmd()
> do_huge_pmd_wp_page()
> pmdp_huge_clear_flush_notify()
> (pmd_none = true)
> exit_mmap()
> unmap_vmas()
> zap_pmd_range()
> pmd_none_or_trans_huge_or_clear_bad()
> (result in memory leak)
> set_pmd_at()
>
> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> and it make the pmd entry to be null. Therefore, The memory leak can occur.
>
> The patch fix the scenario that the pmd entry can lead to be null.
I don't think the scenario is possible.
exit_mmap() called when all mm users have gone, so no parallel threads
exist.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 14:05 [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang
2016-06-21 14:32 ` kbuild test robot
2016-06-21 14:37 ` Kirill A. Shutemov
@ 2016-06-21 14:37 ` Michal Hocko
2016-06-21 14:42 ` kbuild test robot
3 siblings, 0 replies; 9+ messages in thread
From: Michal Hocko @ 2016-06-21 14:37 UTC (permalink / raw)
To: zhongjiang; +Cc: akpm, linux-mm, linux-kernel, Kirill A. Shutemov
[CCing Kirill]
On Tue 21-06-16 22:05:56, zhongjiang wrote:
> From: zhong jiang <zhongjiang@huawei.com>
>
> with great pressure, I run some test cases. As a result, I found
> that the THP is not freed, it is detected by check_mm().
>
> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>
> Consider the following race :
>
> CPU0 CPU1
> __handle_mm_fault()
> wp_huge_pmd()
> do_huge_pmd_wp_page()
> pmdp_huge_clear_flush_notify()
> (pmd_none = true)
> exit_mmap()
> unmap_vmas()
> zap_pmd_range()
> pmd_none_or_trans_huge_or_clear_bad()
> (result in memory leak)
> set_pmd_at()
>
> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> and it make the pmd entry to be null. Therefore, The memory leak can occur.
I do not understand this description. CPU1 is in the exit path with last
mm user gone. So CPU0 is a different process with its own mm. How can
they influence each other. But maybe I am just missing your point.
> The patch fix the scenario that the pmd entry can lead to be null.
>
> Signed-off-by: zhong jiang <zhongjiang@huawei.com>
> ---
> mm/huge_memory.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e10a4fe..ef04b94 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -1340,11 +1340,11 @@ alloc:
> pmd_t entry;
> entry = mk_huge_pmd(new_page, vma->vm_page_prot);
> entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
> - pmdp_huge_clear_flush_notify(vma, haddr, pmd);
> + pmdp_invalidate(vma, haddr, pmd);
> page_add_new_anon_rmap(new_page, vma, haddr, true);
> mem_cgroup_commit_charge(new_page, memcg, false, true);
> lru_cache_add_active_or_unevictable(new_page, vma);
> - set_pmd_at(mm, haddr, pmd, entry);
> + pmd_populate(mm, pmd, entry);
> update_mmu_cache_pmd(vma, address, pmd);
> if (!page) {
> add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
> --
> 1.8.3.1
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 14:05 [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang
` (2 preceding siblings ...)
2016-06-21 14:37 ` Michal Hocko
@ 2016-06-21 14:42 ` kbuild test robot
3 siblings, 0 replies; 9+ messages in thread
From: kbuild test robot @ 2016-06-21 14:42 UTC (permalink / raw)
To: zhongjiang; +Cc: kbuild-all, mhocko, akpm, linux-mm, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]
Hi,
[auto build test ERROR on v4.7-rc4]
[also build test ERROR on next-20160621]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/zhongjiang/mm-huge_memory-fix-the-memory-leak-due-to-the-race/20160621-221736
config: s390-allyesconfig (attached as .config)
compiler: s390x-linux-gnu-gcc (Debian 5.3.1-8) 5.3.1 20160205
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=s390
All errors (new ones prefixed by >>):
mm/huge_memory.c: In function 'do_huge_pmd_wp_page':
>> mm/huge_memory.c:1383:25: error: incompatible type for argument 3 of 'pmd_populate'
pmd_populate(mm, pmd, entry);
^
In file included from arch/s390/include/asm/tlbflush.h:7:0,
from include/linux/hugetlb.h:21,
from mm/huge_memory.c:13:
arch/s390/include/asm/pgalloc.h:120:20: note: expected 'pgtable_t {aka struct <anonymous> *}' but argument is of type 'pmd_t {aka struct <anonymous>}'
static inline void pmd_populate(struct mm_struct *mm,
^
vim +/pmd_populate +1383 mm/huge_memory.c
1377 entry = mk_huge_pmd(new_page, vma->vm_page_prot);
1378 entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
1379 pmdp_invalidate(vma, haddr, pmd);
1380 page_add_new_anon_rmap(new_page, vma, haddr, true);
1381 mem_cgroup_commit_charge(new_page, memcg, false, true);
1382 lru_cache_add_active_or_unevictable(new_page, vma);
> 1383 pmd_populate(mm, pmd, entry);
1384 update_mmu_cache_pmd(vma, address, pmd);
1385 if (!page) {
1386 add_mm_counter(mm, MM_ANONPAGES, HPAGE_PMD_NR);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 41724 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
[not found] ` <57695AEB.8030509@huawei.com>
@ 2016-06-21 15:29 ` Kirill A. Shutemov
2016-06-22 2:55 ` zhong jiang
2016-06-22 9:52 ` zhong jiang
0 siblings, 2 replies; 9+ messages in thread
From: Kirill A. Shutemov @ 2016-06-21 15:29 UTC (permalink / raw)
To: zhong jiang; +Cc: mhocko, akpm, linux-mm, linux-kernel
On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
> > On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
> >> From: zhong jiang <zhongjiang@huawei.com>
> >>
> >> with great pressure, I run some test cases. As a result, I found
> >> that the THP is not freed, it is detected by check_mm().
> >>
> >> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
> >>
> >> Consider the following race :
> >>
> >> CPU0 CPU1
> >> __handle_mm_fault()
> >> wp_huge_pmd()
> >> do_huge_pmd_wp_page()
> >> pmdp_huge_clear_flush_notify()
> >> (pmd_none = true)
> >> exit_mmap()
> >> unmap_vmas()
> >> zap_pmd_range()
> >> pmd_none_or_trans_huge_or_clear_bad()
> >> (result in memory leak)
> >> set_pmd_at()
> >>
> >> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
> >> and it make the pmd entry to be null. Therefore, The memory leak can occur.
> >>
> >> The patch fix the scenario that the pmd entry can lead to be null.
> > I don't think the scenario is possible.
> >
> > exit_mmap() called when all mm users have gone, so no parallel threads
> > exist.
> >
> Forget this patch. It 's my fault , it indeed don not exist.
> But I hit the following problem. we can see the memory leak when the process exit.
>
>
> Any suggestion will be apprecaited.
Could you try this:
http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 15:29 ` Kirill A. Shutemov
@ 2016-06-22 2:55 ` zhong jiang
2016-06-22 9:52 ` zhong jiang
1 sibling, 0 replies; 9+ messages in thread
From: zhong jiang @ 2016-06-22 2:55 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: mhocko, akpm, linux-mm, linux-kernel
On 2016/6/21 23:29, Kirill A. Shutemov wrote:
> On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
>> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
>>> On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> with great pressure, I run some test cases. As a result, I found
>>>> that the THP is not freed, it is detected by check_mm().
>>>>
>>>> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>>>>
>>>> Consider the following race :
>>>>
>>>> CPU0 CPU1
>>>> __handle_mm_fault()
>>>> wp_huge_pmd()
>>>> do_huge_pmd_wp_page()
>>>> pmdp_huge_clear_flush_notify()
>>>> (pmd_none = true)
>>>> exit_mmap()
>>>> unmap_vmas()
>>>> zap_pmd_range()
>>>> pmd_none_or_trans_huge_or_clear_bad()
>>>> (result in memory leak)
>>>> set_pmd_at()
>>>>
>>>> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
>>>> and it make the pmd entry to be null. Therefore, The memory leak can occur.
>>>>
>>>> The patch fix the scenario that the pmd entry can lead to be null.
>>> I don't think the scenario is possible.
>>>
>>> exit_mmap() called when all mm users have gone, so no parallel threads
>>> exist.
>>>
>> Forget this patch. It 's my fault , it indeed don not exist.
>> But I hit the following problem. we can see the memory leak when the process exit.
>>
>>
>> Any suggestion will be apprecaited.
> Could you try this:
>
> http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name
>
I fails to open it. can you display or add attachmemts ? :-) thx
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] mm/huge_memory: fix the memory leak due to the race
2016-06-21 15:29 ` Kirill A. Shutemov
2016-06-22 2:55 ` zhong jiang
@ 2016-06-22 9:52 ` zhong jiang
1 sibling, 0 replies; 9+ messages in thread
From: zhong jiang @ 2016-06-22 9:52 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: mhocko, akpm, linux-mm, linux-kernel
On 2016/6/21 23:29, Kirill A. Shutemov wrote:
> On Tue, Jun 21, 2016 at 11:19:07PM +0800, zhong jiang wrote:
>> On 2016/6/21 22:37, Kirill A. Shutemov wrote:
>>> On Tue, Jun 21, 2016 at 10:05:56PM +0800, zhongjiang wrote:
>>>> From: zhong jiang <zhongjiang@huawei.com>
>>>>
>>>> with great pressure, I run some test cases. As a result, I found
>>>> that the THP is not freed, it is detected by check_mm().
>>>>
>>>> BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
>>>>
>>>> Consider the following race :
>>>>
>>>> CPU0 CPU1
>>>> __handle_mm_fault()
>>>> wp_huge_pmd()
>>>> do_huge_pmd_wp_page()
>>>> pmdp_huge_clear_flush_notify()
>>>> (pmd_none = true)
>>>> exit_mmap()
>>>> unmap_vmas()
>>>> zap_pmd_range()
>>>> pmd_none_or_trans_huge_or_clear_bad()
>>>> (result in memory leak)
>>>> set_pmd_at()
>>>>
>>>> because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
>>>> and it make the pmd entry to be null. Therefore, The memory leak can occur.
>>>>
>>>> The patch fix the scenario that the pmd entry can lead to be null.
>>> I don't think the scenario is possible.
>>>
>>> exit_mmap() called when all mm users have gone, so no parallel threads
>>> exist.
>>>
>> Forget this patch. It 's my fault , it indeed don not exist.
>> But I hit the following problem. we can see the memory leak when the process exit.
>>
>>
>> Any suggestion will be apprecaited.
> Could you try this:
>
> http://lkml.kernel.org/r/20160621150433.GA7536@node.shutemov.name
The patch I have seen , but I don not think this patch can fix so problem . if that race occur, pmd entry points to
the huge page will be changed , and freeze_page spilt pmd will fail. subsequent vm_bug_on() will fired.
freeze_page()
try_to_unmap()
split_huge_pmd_address() (return fail) result in page_mapcount is not zero
vm_bug_on()
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] mm/huge_memory: fix the memory leak due to the race
@ 2016-06-21 14:57 zhongjiang
0 siblings, 0 replies; 9+ messages in thread
From: zhongjiang @ 2016-06-21 14:57 UTC (permalink / raw)
To: mhocko, akpm; +Cc: linux-mm, linux-kernel
From: zhong jiang <zhongjiang@huawei.com>
with great pressure, I run some test cases. As a result, I found
that the THP is not freed, it is detected by check_mm().
BUG: Bad rss-counter state mm:ffff8827edb70000 idx:1 val:512
Consider the following race :
CPU0 CPU1
__handle_mm_fault()
wp_huge_pmd()
do_huge_pmd_wp_page()
pmdp_huge_clear_flush_notify()
(pmd_none = true)
exit_mmap()
unmap_vmas()
zap_pmd_range()
pmd_none_or_trans_huge_or_clear_bad()
(result in memory leak)
set_pmd_at()
because of CPU0 have allocated huge page before pmdp_huge_clear_notify,
and it make the pmd entry to be null. Therefore, The memory leak can occur.
The patch fix the scenario that the pmd entry can lead to be null.
Signed-off-by: zhong jiang <zhongjiang@huawei.com>
---
mm/huge_memory.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e10a4fe..95c7dfe 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1340,7 +1340,7 @@ alloc:
pmd_t entry;
entry = mk_huge_pmd(new_page, vma->vm_page_prot);
entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
- pmdp_huge_clear_flush_notify(vma, haddr, pmd);
+ pmdp_invalidate(vma, haddr, pmd);
page_add_new_anon_rmap(new_page, vma, haddr, true);
mem_cgroup_commit_charge(new_page, memcg, false, true);
lru_cache_add_active_or_unevictable(new_page, vma);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-06-22 10:01 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-21 14:05 [PATCH] mm/huge_memory: fix the memory leak due to the race zhongjiang
2016-06-21 14:32 ` kbuild test robot
2016-06-21 14:37 ` Kirill A. Shutemov
[not found] ` <57695AEB.8030509@huawei.com>
2016-06-21 15:29 ` Kirill A. Shutemov
2016-06-22 2:55 ` zhong jiang
2016-06-22 9:52 ` zhong jiang
2016-06-21 14:37 ` Michal Hocko
2016-06-21 14:42 ` kbuild test robot
2016-06-21 14:57 zhongjiang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).