* vmscan: Reclaim unevictable pages
[not found] <SG2PR02MB3098A05E09B0D3F3CB1C3B9BE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
@ 2019-03-14 7:53 ` Pankaj Suryawanshi
2019-03-14 8:23 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-14 7:53 UTC (permalink / raw)
To: linux-kernel; +Cc: minchan
Hello ,
shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
Regards,
Pankaj
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: vmscan: Reclaim unevictable pages
2019-03-14 7:53 ` vmscan: Reclaim unevictable pages Pankaj Suryawanshi
@ 2019-03-14 8:23 ` Pankaj Suryawanshi
2019-03-14 8:41 ` Michal Hocko
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-14 8:23 UTC (permalink / raw)
To: linux-kernel; +Cc: minchan
Below is the Error Msg :
[ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
[ 24.726949] page->mem_cgroup:bd008c00
[ 24.730693] ------------[ cut here ]------------
[ 24.735304] kernel BUG at mm/vmscan.c:1350!
[ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Solved the issue by below patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..2e51edc 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
page = lru_to_page(page_list);
list_del(&page->lru);
if (!trylock_page(page)) {
goto keep;
}
VM_BUG_ON_PAGE(PageActive(page), page);
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,6 +1333,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
From: Pankaj Suryawanshi
Sent: 14 March 2019 13:23:53
To: linux-kernel@vger.kernel.org
Cc: minchan@kernel.org
Subject: vmscan: Reclaim unevictable pages
Hello ,
shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
Regards,
Pankaj
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: vmscan: Reclaim unevictable pages
2019-03-14 8:23 ` Pankaj Suryawanshi
@ 2019-03-14 8:41 ` Michal Hocko
2019-03-14 8:52 ` [External] " Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Michal Hocko @ 2019-03-14 8:41 UTC (permalink / raw)
To: Pankaj Suryawanshi; +Cc: linux-kernel, minchan
On Thu 14-03-19 08:23:10, Pankaj Suryawanshi wrote:
>
> Below is the Error Msg :
>
> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
> [ 24.726949] page->mem_cgroup:bd008c00
> [ 24.730693] ------------[ cut here ]------------
> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Just curious. Which kernel version is that? Are there any addional
patches applied on top of vanilla?
> Solved the issue by below patch.
>
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..2e51edc 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> page = lru_to_page(page_list);
> list_del(&page->lru);
>
> if (!trylock_page(page)) {
> goto keep;
> }
>
> VM_BUG_ON_PAGE(PageActive(page), page);
>
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,6 +1333,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
>
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
>
>
>
>
> From: Pankaj Suryawanshi
> Sent: 14 March 2019 13:23:53
> To: linux-kernel@vger.kernel.org
> Cc: minchan@kernel.org
> Subject: vmscan: Reclaim unevictable pages
>
>
>
> Hello ,
>
> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>
> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>
> Regards,
> Pankaj
>
> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company
accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-14 8:41 ` Michal Hocko
@ 2019-03-14 8:52 ` Pankaj Suryawanshi
2019-03-14 9:25 ` Kirill Tkhai
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-14 8:52 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-kernel, minchan
I am using kernel version 4.14.65 (on Android pie [ARM]).
No additional patches applied on top of vanilla.(Core MM).
If I change in the vmscan.c as below patch, it will work.
Regards,
Pankaj
From: Michal Hocko <mhocko@kernel.org>
Sent: 14 March 2019 14:11:20
To: Pankaj Suryawanshi
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: [External] Re: vmscan: Reclaim unevictable pages
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
On Thu 14-03-19 08:23:10, Pankaj Suryawanshi wrote:
>
> Below is the Error Msg :
>
> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
> [ 24.726949] page->mem_cgroup:bd008c00
> [ 24.730693] ------------[ cut here ]------------
> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Just curious. Which kernel version is that? Are there any addional
patches applied on top of vanilla?
> Solved the issue by below patch.
>
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..2e51edc 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> page = lru_to_page(page_list);
> list_del(&page->lru);
>
> if (!trylock_page(page)) {
> goto keep;
> }
>
> VM_BUG_ON_PAGE(PageActive(page), page);
>
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,6 +1333,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
>
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
>
>
>
>
> From: Pankaj Suryawanshi
> Sent: 14 March 2019 13:23:53
> To: linux-kernel@vger.kernel.org
> Cc: minchan@kernel.org
> Subject: vmscan: Reclaim unevictable pages
>
>
>
> Hello ,
>
> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>
> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>
> Regards,
> Pankaj
>
> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-14 8:52 ` [External] " Pankaj Suryawanshi
@ 2019-03-14 9:25 ` Kirill Tkhai
2019-03-14 11:09 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Kirill Tkhai @ 2019-03-14 9:25 UTC (permalink / raw)
To: Pankaj Suryawanshi, Michal Hocko; +Cc: linux-kernel, minchan
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-14 9:25 ` Kirill Tkhai
@ 2019-03-14 11:09 ` Pankaj Suryawanshi
2019-03-14 13:44 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-14 11:09 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko; +Cc: linux-kernel, minchan
Hello ,
The curly braces are just for debugging purpose so please ignore it.
Below is the updated patch :
if (!trylock_page(page))
goto keep;
VM_BUG_ON_PAGE(PageActive(page), page);
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,6 +1333,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-14 11:09 ` Pankaj Suryawanshi
@ 2019-03-14 13:44 ` Pankaj Suryawanshi
2019-03-15 6:05 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-14 13:44 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko; +Cc: linux-kernel, minchan
Hello ,
Please ignore the curly braces, they are just for debugging.
Below is the updated patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
Regards,
Pankaj
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-14 13:44 ` Pankaj Suryawanshi
@ 2019-03-15 6:05 ` Pankaj Suryawanshi
2019-03-15 10:11 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-15 6:05 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko; +Cc: linux-kernel, minchan, linux-mm
[ cc linux-mm ]
From: Pankaj Suryawanshi
Sent: 14 March 2019 19:14:40
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
Hello ,
Please ignore the curly braces, they are just for debugging.
Below is the updated patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
Regards,
Pankaj
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-15 6:05 ` Pankaj Suryawanshi
@ 2019-03-15 10:11 ` Pankaj Suryawanshi
2019-03-18 7:45 ` Pankaj Suryawanshi
2019-03-18 8:42 ` [External] " Vlastimil Babka
0 siblings, 2 replies; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-15 10:11 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj, vbabka
[ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
From: Pankaj Suryawanshi
Sent: 15 March 2019 11:35:05
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
[ cc linux-mm ]
From: Pankaj Suryawanshi
Sent: 14 March 2019 19:14:40
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
Hello ,
Please ignore the curly braces, they are just for debugging.
Below is the updated patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
Regards,
Pankaj
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-15 10:11 ` Pankaj Suryawanshi
@ 2019-03-18 7:45 ` Pankaj Suryawanshi
2019-03-18 8:56 ` Pankaj Suryawanshi
2019-03-18 8:42 ` [External] " Vlastimil Babka
1 sibling, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-18 7:45 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, vbabka
It fixes the below issue.
1. Large size buffer allocation using cma_alloc successful with unevictable pages.
cma_alloc of current kernel will fail due to unevictable pages.
Solved the below issue of cma_alloc
-----------------------------------------------------------------------------------------------------------------------------------------
[ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
[ 24.726949] page->mem_cgroup:bd008c00
[ 24.730693] ------------[ cut here ]------------
[ 24.735304] kernel BUG at mm/vmscan.c:1350!
[ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
-----------------------------------------------------------------------------------------------------------------------------------------
From: Pankaj Suryawanshi
Sent: 15 March 2019 15:41:57
To: Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com; vbabka@suse.cz
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
[ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
From: Pankaj Suryawanshi
Sent: 15 March 2019 11:35:05
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
[ cc linux-mm ]
From: Pankaj Suryawanshi
Sent: 14 March 2019 19:14:40
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
Hello ,
Please ignore the curly braces, they are just for debugging.
Below is the updated patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
Regards,
Pankaj
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-15 10:11 ` Pankaj Suryawanshi
2019-03-18 7:45 ` Pankaj Suryawanshi
@ 2019-03-18 8:42 ` Vlastimil Babka
2019-03-18 9:09 ` Pankaj Suryawanshi
1 sibling, 1 reply; 25+ messages in thread
From: Vlastimil Babka @ 2019-03-18 8:42 UTC (permalink / raw)
To: Pankaj Suryawanshi, Kirill Tkhai, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj
On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>
> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
Can you send a proper patch with changelog explaining the change? I
don't know the context of this thread.
> From: Pankaj Suryawanshi
> Sent: 15 March 2019 11:35:05
> To: Kirill Tkhai; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
>
> [ cc linux-mm ]
>
>
> From: Pankaj Suryawanshi
> Sent: 14 March 2019 19:14:40
> To: Kirill Tkhai; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
>
> Hello ,
>
> Please ignore the curly braces, they are just for debugging.
>
> Below is the updated patch.
>
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..12ac353 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> -
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>
>
>
> Regards,
> Pankaj
>
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 14 March 2019 14:55:34
> To: Pankaj Suryawanshi; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>
>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>
>> No additional patches applied on top of vanilla.(Core MM).
>>
>> If I change in the vmscan.c as below patch, it will work.
>
> Sorry, but 4.14.65 does not have braces around trylock_page(),
> like in your patch below.
>
> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>
> [...]
>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..2e51edc 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> page = lru_to_page(page_list);
>>> list_del(&page->lru);
>>>
>>> if (!trylock_page(page)) {
>>> goto keep;
>>> }
>
> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: vmscan: Reclaim unevictable pages
2019-03-18 7:45 ` Pankaj Suryawanshi
@ 2019-03-18 8:56 ` Pankaj Suryawanshi
0 siblings, 0 replies; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-18 8:56 UTC (permalink / raw)
To: Kirill Tkhai, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, vbabka
Hello
shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
[ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
[ 24.726949] page->mem_cgroup:bd008c00
[ 24.730693] ------------[ cut here ]------------
[ 24.735304] kernel BUG at mm/vmscan.c:1350!
[ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Below is the patch which solved this issue :
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
It fixes the below issue.
1. Large size buffer allocation using cma_alloc successful with unevictable pages.
cma_alloc of current kernel will fail due to unevictable page
Please let me know if anything i am missing.
Regards,
Pankaj
From: Pankaj Suryawanshi
Sent: 18 March 2019 13:15:22
To: Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; vbabka@suse.cz
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
It fixes the below issue.
1. Large size buffer allocation using cma_alloc successful with unevictable pages.
cma_alloc of current kernel will fail due to unevictable pages.
Solved the below issue of cma_alloc
-----------------------------------------------------------------------------------------------------------------------------------------
[ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
[ 24.726949] page->mem_cgroup:bd008c00
[ 24.730693] ------------[ cut here ]------------
[ 24.735304] kernel BUG at mm/vmscan.c:1350!
[ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
-----------------------------------------------------------------------------------------------------------------------------------------
From: Pankaj Suryawanshi
Sent: 15 March 2019 15:41:57
To: Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com; vbabka@suse.cz
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
[ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
From: Pankaj Suryawanshi
Sent: 15 March 2019 11:35:05
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
[ cc linux-mm ]
From: Pankaj Suryawanshi
Sent: 14 March 2019 19:14:40
To: Kirill Tkhai; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
Hello ,
Please ignore the curly braces, they are just for debugging.
Below is the updated patch.
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
Regards,
Pankaj
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 14 March 2019 14:55:34
To: Pankaj Suryawanshi; Michal Hocko
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>
> I am using kernel version 4.14.65 (on Android pie [ARM]).
>
> No additional patches applied on top of vanilla.(Core MM).
>
> If I change in the vmscan.c as below patch, it will work.
Sorry, but 4.14.65 does not have braces around trylock_page(),
like in your patch below.
See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
[...]
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..2e51edc 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> page = lru_to_page(page_list);
>> list_del(&page->lru);
>>
>> if (!trylock_page(page)) {
>> goto keep;
>> }
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 8:42 ` [External] " Vlastimil Babka
@ 2019-03-18 9:09 ` Pankaj Suryawanshi
2019-03-18 9:33 ` Kirill Tkhai
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-18 9:09 UTC (permalink / raw)
To: Vlastimil Babka, Kirill Tkhai, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj
Hello
shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
[ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
[ 24.726949] page->mem_cgroup:bd008c00
[ 24.730693] ------------[ cut here ]------------
[ 24.735304] kernel BUG at mm/vmscan.c:1350!
[ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
Below is the patch which solved this issue :
diff --git a/mm/vmscan.c b/mm/vmscan.c
index be56e2e..12ac353 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
sc->nr_scanned++;
if (unlikely(!page_evictable(page)))
- goto activate_locked;
+ goto cull_mlocked;
if (!sc->may_unmap && page_mapped(page))
goto keep_locked;
@@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
} else
list_add(&page->lru, &free_pages);
continue;
-
+cull_mlocked:
+ if (PageSwapCache(page))
+ try_to_free_swap(page);
+ unlock_page(page);
+ list_add(&page->lru, &ret_pages);
+ continue;
activate_locked:
/* Not a candidate for swapping, so reclaim swap space. */
if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
It fixes the below issue.
1. Large size buffer allocation using cma_alloc successful with unevictable pages.
cma_alloc of current kernel will fail due to unevictable page
Please let me know if anything i am missing.
Regards,
Pankaj
From: Vlastimil Babka <vbabka@suse.cz>
Sent: 18 March 2019 14:12:50
To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>
> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
Can you send a proper patch with changelog explaining the change? I
don't know the context of this thread.
> From: Pankaj Suryawanshi
> Sent: 15 March 2019 11:35:05
> To: Kirill Tkhai; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
>
> [ cc linux-mm ]
>
>
> From: Pankaj Suryawanshi
> Sent: 14 March 2019 19:14:40
> To: Kirill Tkhai; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
>
> Hello ,
>
> Please ignore the curly braces, they are just for debugging.
>
> Below is the updated patch.
>
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..12ac353 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> -
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>
>
>
> Regards,
> Pankaj
>
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 14 March 2019 14:55:34
> To: Pankaj Suryawanshi; Michal Hocko
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>
>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>
>> No additional patches applied on top of vanilla.(Core MM).
>>
>> If I change in the vmscan.c as below patch, it will work.
>
> Sorry, but 4.14.65 does not have braces around trylock_page(),
> like in your patch below.
>
> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>
> [...]
>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..2e51edc 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> page = lru_to_page(page_list);
>>> list_del(&page->lru);
>>>
>>> if (!trylock_page(page)) {
>>> goto keep;
>>> }
>
> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 9:09 ` Pankaj Suryawanshi
@ 2019-03-18 9:33 ` Kirill Tkhai
2019-03-18 9:43 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Kirill Tkhai @ 2019-03-18 9:33 UTC (permalink / raw)
To: Pankaj Suryawanshi, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj
Hi, Pankaj,
On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>
> Hello
>
> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
PageUnevictable() pages are never being added to lists, which shrink_page_list()
uses for iteration. Also, a page can't be marked as PageUnevictable(), when
it's attached to a shrinkable list.
So, the problem should be somewhere outside shrink_page_list().
I won't suggest you something about CMA, since I haven't dived in that code.
> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>
> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>
>
> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>
> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
> [ 24.726949] page->mem_cgroup:bd008c00
> [ 24.730693] ------------[ cut here ]------------
> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>
>
> Below is the patch which solved this issue :
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..12ac353 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> -
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>
>
>
>
> It fixes the below issue.
>
> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>
> cma_alloc of current kernel will fail due to unevictable page
>
> Please let me know if anything i am missing.
>
> Regards,
> Pankaj
>
> From: Vlastimil Babka <vbabka@suse.cz>
> Sent: 18 March 2019 14:12:50
> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>
>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>
> Can you send a proper patch with changelog explaining the change? I
> don't know the context of this thread.
>
>> From: Pankaj Suryawanshi
>> Sent: 15 March 2019 11:35:05
>> To: Kirill Tkhai; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>>
>> [ cc linux-mm ]
>>
>>
>> From: Pankaj Suryawanshi
>> Sent: 14 March 2019 19:14:40
>> To: Kirill Tkhai; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>>
>> Hello ,
>>
>> Please ignore the curly braces, they are just for debugging.
>>
>> Below is the updated patch.
>>
>>
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..12ac353 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> sc->nr_scanned++;
>>
>> if (unlikely(!page_evictable(page)))
>> - goto activate_locked;
>> + goto cull_mlocked;
>>
>> if (!sc->may_unmap && page_mapped(page))
>> goto keep_locked;
>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> } else
>> list_add(&page->lru, &free_pages);
>> continue;
>> -
>> +cull_mlocked:
>> + if (PageSwapCache(page))
>> + try_to_free_swap(page);
>> + unlock_page(page);
>> + list_add(&page->lru, &ret_pages);
>> + continue;
>> activate_locked:
>> /* Not a candidate for swapping, so reclaim swap space. */
>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>
>>
>>
>> Regards,
>> Pankaj
>>
>>
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Sent: 14 March 2019 14:55:34
>> To: Pankaj Suryawanshi; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>
>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>
>>> No additional patches applied on top of vanilla.(Core MM).
>>>
>>> If I change in the vmscan.c as below patch, it will work.
>>
>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>> like in your patch below.
>>
>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>
>> [...]
>>
>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>> index be56e2e..2e51edc 100644
>>>> --- a/mm/vmscan.c
>>>> +++ b/mm/vmscan.c
>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> page = lru_to_page(page_list);
>>>> list_del(&page->lru);
>>>>
>>>> if (!trylock_page(page)) {
>>>> goto keep;
>>>> }
>>
>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>
>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 9:33 ` Kirill Tkhai
@ 2019-03-18 9:43 ` Pankaj Suryawanshi
2019-03-18 9:47 ` Kirill Tkhai
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-18 9:43 UTC (permalink / raw)
To: Kirill Tkhai, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj
Hi Kirill Tkhai,
Please see mm/vmscan.c in which it first added to list and than throw the error :
--------------------------------------------------------------------------------------------------
keep:
list_add(&page->lru, &ret_pages);
VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
---------------------------------------------------------------------------------------------------
Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 18 March 2019 15:03:15
To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
Hi, Pankaj,
On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>
> Hello
>
> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
PageUnevictable() pages are never being added to lists, which shrink_page_list()
uses for iteration. Also, a page can't be marked as PageUnevictable(), when
it's attached to a shrinkable list.
So, the problem should be somewhere outside shrink_page_list().
I won't suggest you something about CMA, since I haven't dived in that code.
> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>
> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>
>
> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>
> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
> [ 24.726949] page->mem_cgroup:bd008c00
> [ 24.730693] ------------[ cut here ]------------
> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>
>
> Below is the patch which solved this issue :
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index be56e2e..12ac353 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> sc->nr_scanned++;
>
> if (unlikely(!page_evictable(page)))
> - goto activate_locked;
> + goto cull_mlocked;
>
> if (!sc->may_unmap && page_mapped(page))
> goto keep_locked;
> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> } else
> list_add(&page->lru, &free_pages);
> continue;
> -
> +cull_mlocked:
> + if (PageSwapCache(page))
> + try_to_free_swap(page);
> + unlock_page(page);
> + list_add(&page->lru, &ret_pages);
> + continue;
> activate_locked:
> /* Not a candidate for swapping, so reclaim swap space. */
> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>
>
>
>
> It fixes the below issue.
>
> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>
> cma_alloc of current kernel will fail due to unevictable page
>
> Please let me know if anything i am missing.
>
> Regards,
> Pankaj
>
> From: Vlastimil Babka <vbabka@suse.cz>
> Sent: 18 March 2019 14:12:50
> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>
>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>
> Can you send a proper patch with changelog explaining the change? I
> don't know the context of this thread.
>
>> From: Pankaj Suryawanshi
>> Sent: 15 March 2019 11:35:05
>> To: Kirill Tkhai; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>>
>> [ cc linux-mm ]
>>
>>
>> From: Pankaj Suryawanshi
>> Sent: 14 March 2019 19:14:40
>> To: Kirill Tkhai; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>>
>> Hello ,
>>
>> Please ignore the curly braces, they are just for debugging.
>>
>> Below is the updated patch.
>>
>>
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..12ac353 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> sc->nr_scanned++;
>>
>> if (unlikely(!page_evictable(page)))
>> - goto activate_locked;
>> + goto cull_mlocked;
>>
>> if (!sc->may_unmap && page_mapped(page))
>> goto keep_locked;
>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> } else
>> list_add(&page->lru, &free_pages);
>> continue;
>> -
>> +cull_mlocked:
>> + if (PageSwapCache(page))
>> + try_to_free_swap(page);
>> + unlock_page(page);
>> + list_add(&page->lru, &ret_pages);
>> + continue;
>> activate_locked:
>> /* Not a candidate for swapping, so reclaim swap space. */
>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>
>>
>>
>> Regards,
>> Pankaj
>>
>>
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Sent: 14 March 2019 14:55:34
>> To: Pankaj Suryawanshi; Michal Hocko
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>
>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>
>>> No additional patches applied on top of vanilla.(Core MM).
>>>
>>> If I change in the vmscan.c as below patch, it will work.
>>
>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>> like in your patch below.
>>
>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>
>> [...]
>>
>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>> index be56e2e..2e51edc 100644
>>>> --- a/mm/vmscan.c
>>>> +++ b/mm/vmscan.c
>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> page = lru_to_page(page_list);
>>>> list_del(&page->lru);
>>>>
>>>> if (!trylock_page(page)) {
>>>> goto keep;
>>>> }
>>
>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>
>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 9:43 ` Pankaj Suryawanshi
@ 2019-03-18 9:47 ` Kirill Tkhai
2019-03-18 9:59 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Kirill Tkhai @ 2019-03-18 9:47 UTC (permalink / raw)
To: Pankaj Suryawanshi, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual, hillf.zj
On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
> Hi Kirill Tkhai,
>
Please, do not top posting: https://kernelnewbies.org/mailinglistguidelines
> Please see mm/vmscan.c in which it first added to list and than throw the error :
> --------------------------------------------------------------------------------------------------
> keep:
> list_add(&page->lru, &ret_pages);
> VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
> ---------------------------------------------------------------------------------------------------
>
> Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
I say about about the list, which is passed to shrink_page_list() as first argument.
shrink_inactive_list()
{
isolate_lru_pages(&page_list); // <-- you can't obtain unevictable pages here.
shrink_page_list(&page_list);
}
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:03:15
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> Hi, Pankaj,
>
> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>
>> Hello
>>
>> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>
> the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
> PageUnevictable() pages are never being added to lists, which shrink_page_list()
> uses for iteration. Also, a page can't be marked as PageUnevictable(), when
> it's attached to a shrinkable list.
>
> So, the problem should be somewhere outside shrink_page_list().
>
> I won't suggest you something about CMA, since I haven't dived in that code.
>
>> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>>
>> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
>> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>>
>>
>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>>
>> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
>> [ 24.726949] page->mem_cgroup:bd008c00
>> [ 24.730693] ------------[ cut here ]------------
>> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
>> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>
>>
>> Below is the patch which solved this issue :
>>
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..12ac353 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> sc->nr_scanned++;
>>
>> if (unlikely(!page_evictable(page)))
>> - goto activate_locked;
>> + goto cull_mlocked;
>>
>> if (!sc->may_unmap && page_mapped(page))
>> goto keep_locked;
>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> } else
>> list_add(&page->lru, &free_pages);
>> continue;
>> -
>> +cull_mlocked:
>> + if (PageSwapCache(page))
>> + try_to_free_swap(page);
>> + unlock_page(page);
>> + list_add(&page->lru, &ret_pages);
>> + continue;
>> activate_locked:
>> /* Not a candidate for swapping, so reclaim swap space. */
>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>
>>
>>
>>
>> It fixes the below issue.
>>
>> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>>
>> cma_alloc of current kernel will fail due to unevictable page
>>
>> Please let me know if anything i am missing.
>>
>> Regards,
>> Pankaj
>>
>> From: Vlastimil Babka <vbabka@suse.cz>
>> Sent: 18 March 2019 14:12:50
>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>
>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>
>> Can you send a proper patch with changelog explaining the change? I
>> don't know the context of this thread.
>>
>>> From: Pankaj Suryawanshi
>>> Sent: 15 March 2019 11:35:05
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> [ cc linux-mm ]
>>>
>>>
>>> From: Pankaj Suryawanshi
>>> Sent: 14 March 2019 19:14:40
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> Hello ,
>>>
>>> Please ignore the curly braces, they are just for debugging.
>>>
>>> Below is the updated patch.
>>>
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> sc->nr_scanned++;
>>>
>>> if (unlikely(!page_evictable(page)))
>>> - goto activate_locked;
>>> + goto cull_mlocked;
>>>
>>> if (!sc->may_unmap && page_mapped(page))
>>> goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> } else
>>> list_add(&page->lru, &free_pages);
>>> continue;
>>> -
>>> +cull_mlocked:
>>> + if (PageSwapCache(page))
>>> + try_to_free_swap(page);
>>> + unlock_page(page);
>>> + list_add(&page->lru, &ret_pages);
>>> + continue;
>>> activate_locked:
>>> /* Not a candidate for swapping, so reclaim swap space. */
>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>> Regards,
>>> Pankaj
>>>
>>>
>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>> Sent: 14 March 2019 14:55:34
>>> To: Pankaj Suryawanshi; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>
>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>
>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>
>>>> If I change in the vmscan.c as below patch, it will work.
>>>
>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>> like in your patch below.
>>>
>>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>>
>>> [...]
>>>
>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>> index be56e2e..2e51edc 100644
>>>>> --- a/mm/vmscan.c
>>>>> +++ b/mm/vmscan.c
>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>>> page = lru_to_page(page_list);
>>>>> list_del(&page->lru);
>>>>>
>>>>> if (!trylock_page(page)) {
>>>>> goto keep;
>>>>> }
>>>
>>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>>
>>
>>
>>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 9:47 ` Kirill Tkhai
@ 2019-03-18 9:59 ` Pankaj Suryawanshi
2019-03-18 10:38 ` Kirill Tkhai
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-18 9:59 UTC (permalink / raw)
To: Kirill Tkhai, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 18 March 2019 15:17:56
To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
> Hi Kirill Tkhai,
>
Please, do not top posting: https://kernelnewbies.org/mailinglistguidelines
Okay.
mailinglistguidelines - Linux Kernel Newbies
kernelnewbies.org
Set of FAQs for kernelnewbies mailing list. If you are new to this list please read this page before you go on your quest for squeezing all the knowledge from fellow members.
> Please see mm/vmscan.c in which it first added to list and than throw the error :
> --------------------------------------------------------------------------------------------------
> keep:
> list_add(&page->lru, &ret_pages);
> VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
> ---------------------------------------------------------------------------------------------------
>
> Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
I say about about the list, which is passed to shrink_page_list() as first argument.
Did you mean candidate list which is passed to shrink_page_list().
shrink_inactive_list()
{
isolate_lru_pages(&page_list); // <-- you can't obtain unevictable pages here.
shrink_page_list(&page_list);
}
below is the overview of flow of calls for your information.
cma_alloc() ->
alloc_contig_range() ->
start_isolate_page_range() ->
__alloc_contig_migrate_range() ->
isolate_migratepages_range() ->
reclaim_clean_pages_from_list() ->
shrink_page_list()
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:03:15
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
>
> Hi, Pankaj,
>
> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>
>> Hello
>>
>> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>
> the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
> PageUnevictable() pages are never being added to lists, which shrink_page_list()
> uses for iteration. Also, a page can't be marked as PageUnevictable(), when
> it's attached to a shrinkable list.
>
> So, the problem should be somewhere outside shrink_page_list().
>
> I won't suggest you something about CMA, since I haven't dived in that code.
>
>> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>>
>> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
>> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>>
>>
>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>>
>> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
>> [ 24.726949] page->mem_cgroup:bd008c00
>> [ 24.730693] ------------[ cut here ]------------
>> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
>> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>
>>
>> Below is the patch which solved this issue :
>>
>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>> index be56e2e..12ac353 100644
>> --- a/mm/vmscan.c
>> +++ b/mm/vmscan.c
>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> sc->nr_scanned++;
>>
>> if (unlikely(!page_evictable(page)))
>> - goto activate_locked;
>> + goto cull_mlocked;
>>
>> if (!sc->may_unmap && page_mapped(page))
>> goto keep_locked;
>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>> } else
>> list_add(&page->lru, &free_pages);
>> continue;
>> -
>> +cull_mlocked:
>> + if (PageSwapCache(page))
>> + try_to_free_swap(page);
>> + unlock_page(page);
>> + list_add(&page->lru, &ret_pages);
>> + continue;
>> activate_locked:
>> /* Not a candidate for swapping, so reclaim swap space. */
>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>
>>
>>
>>
>> It fixes the below issue.
>>
>> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>>
>> cma_alloc of current kernel will fail due to unevictable page
>>
>> Please let me know if anything i am missing.
>>
>> Regards,
>> Pankaj
>>
>> From: Vlastimil Babka <vbabka@suse.cz>
>> Sent: 18 March 2019 14:12:50
>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>
>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>
>> Can you send a proper patch with changelog explaining the change? I
>> don't know the context of this thread.
>>
>>> From: Pankaj Suryawanshi
>>> Sent: 15 March 2019 11:35:05
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> [ cc linux-mm ]
>>>
>>>
>>> From: Pankaj Suryawanshi
>>> Sent: 14 March 2019 19:14:40
>>> To: Kirill Tkhai; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>>
>>> Hello ,
>>>
>>> Please ignore the curly braces, they are just for debugging.
>>>
>>> Below is the updated patch.
>>>
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> sc->nr_scanned++;
>>>
>>> if (unlikely(!page_evictable(page)))
>>> - goto activate_locked;
>>> + goto cull_mlocked;
>>>
>>> if (!sc->may_unmap && page_mapped(page))
>>> goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> } else
>>> list_add(&page->lru, &free_pages);
>>> continue;
>>> -
>>> +cull_mlocked:
>>> + if (PageSwapCache(page))
>>> + try_to_free_swap(page);
>>> + unlock_page(page);
>>> + list_add(&page->lru, &ret_pages);
>>> + continue;
>>> activate_locked:
>>> /* Not a candidate for swapping, so reclaim swap space. */
>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>> Regards,
>>> Pankaj
>>>
>>>
>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>> Sent: 14 March 2019 14:55:34
>>> To: Pankaj Suryawanshi; Michal Hocko
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>
>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>
>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>
>>>> If I change in the vmscan.c as below patch, it will work.
>>>
>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>> like in your patch below.
>>>
>>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>>
>>> [...]
>>>
>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>> index be56e2e..2e51edc 100644
>>>>> --- a/mm/vmscan.c
>>>>> +++ b/mm/vmscan.c
>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>>> page = lru_to_page(page_list);
>>>>> list_del(&page->lru);
>>>>>
>>>>> if (!trylock_page(page)) {
>>>>> goto keep;
>>>>> }
>>>
>>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>>
>>
>>
>>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 9:59 ` Pankaj Suryawanshi
@ 2019-03-18 10:38 ` Kirill Tkhai
2019-03-20 6:48 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Kirill Tkhai @ 2019-03-18 10:38 UTC (permalink / raw)
To: Pankaj Suryawanshi, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual
On 18.03.2019 12:59, Pankaj Suryawanshi wrote:
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:17:56
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
Also, please, avoid irritating quoting like below ^^^. They just distract attention.
> On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
>> Hi Kirill Tkhai,
>>
>
> Please, do not top posting: https://kernelnewbies.org/mailinglistguidelines
>
> Okay.
>
> mailinglistguidelines - Linux Kernel Newbies
> kernelnewbies.org
> Set of FAQs for kernelnewbies mailing list. If you are new to this list please read this page before you go on your quest for squeezing all the knowledge from fellow members.
And this spew ^^^.
>> Please see mm/vmscan.c in which it first added to list and than throw the error :
>> --------------------------------------------------------------------------------------------------
>> keep:
>> list_add(&page->lru, &ret_pages);
>> VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
>> ---------------------------------------------------------------------------------------------------
>>
>> Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
>
> I say about about the list, which is passed to shrink_page_list() as first argument.
> Did you mean candidate list which is passed to shrink_page_list().
>
> shrink_inactive_list()
> {
> isolate_lru_pages(&page_list); // <-- you can't obtain unevictable pages here.
> shrink_page_list(&page_list);
> }
>
> below is the overview of flow of calls for your information.
>
> cma_alloc() ->
> alloc_contig_range() ->
> start_isolate_page_range() ->
> __alloc_contig_migrate_range() ->
> isolate_migratepages_range() ->
> reclaim_clean_pages_from_list() ->
> shrink_page_list()
Hm, isolate_migratepages_range() can take unevictable pages,
but then with your patch we just skip them in shrink_page_list().
Without your patch we bump into bug on.
These both look incorrect for me. Let's wait someone who familiar
with this logic.
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Sent: 18 March 2019 15:03:15
>> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> Hi, Pankaj,
>>
>> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>>
>>> Hello
>>>
>>> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>>
>> the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
>> PageUnevictable() pages are never being added to lists, which shrink_page_list()
>> uses for iteration. Also, a page can't be marked as PageUnevictable(), when
>> it's attached to a shrinkable list.
>>
>> So, the problem should be somewhere outside shrink_page_list().
>>
>> I won't suggest you something about CMA, since I haven't dived in that code.
>>
>>> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>>>
>>> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
>>> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>>>
>>>
>>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>>>
>>> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
>>> [ 24.726949] page->mem_cgroup:bd008c00
>>> [ 24.730693] ------------[ cut here ]------------
>>> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
>>> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>>
>>>
>>> Below is the patch which solved this issue :
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> sc->nr_scanned++;
>>>
>>> if (unlikely(!page_evictable(page)))
>>> - goto activate_locked;
>>> + goto cull_mlocked;
>>>
>>> if (!sc->may_unmap && page_mapped(page))
>>> goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> } else
>>> list_add(&page->lru, &free_pages);
>>> continue;
>>> -
>>> +cull_mlocked:
>>> + if (PageSwapCache(page))
>>> + try_to_free_swap(page);
>>> + unlock_page(page);
>>> + list_add(&page->lru, &ret_pages);
>>> + continue;
>>> activate_locked:
>>> /* Not a candidate for swapping, so reclaim swap space. */
>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>>
>>> It fixes the below issue.
>>>
>>> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>>>
>>> cma_alloc of current kernel will fail due to unevictable page
>>>
>>> Please let me know if anything i am missing.
>>>
>>> Regards,
>>> Pankaj
>>>
>>> From: Vlastimil Babka <vbabka@suse.cz>
>>> Sent: 18 March 2019 14:12:50
>>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>>
>>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>>
>>> Can you send a proper patch with changelog explaining the change? I
>>> don't know the context of this thread.
>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 15 March 2019 11:35:05
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> [ cc linux-mm ]
>>>>
>>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 14 March 2019 19:14:40
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> Hello ,
>>>>
>>>> Please ignore the curly braces, they are just for debugging.
>>>>
>>>> Below is the updated patch.
>>>>
>>>>
>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>> index be56e2e..12ac353 100644
>>>> --- a/mm/vmscan.c
>>>> +++ b/mm/vmscan.c
>>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> sc->nr_scanned++;
>>>>
>>>> if (unlikely(!page_evictable(page)))
>>>> - goto activate_locked;
>>>> + goto cull_mlocked;
>>>>
>>>> if (!sc->may_unmap && page_mapped(page))
>>>> goto keep_locked;
>>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> } else
>>>> list_add(&page->lru, &free_pages);
>>>> continue;
>>>> -
>>>> +cull_mlocked:
>>>> + if (PageSwapCache(page))
>>>> + try_to_free_swap(page);
>>>> + unlock_page(page);
>>>> + list_add(&page->lru, &ret_pages);
>>>> + continue;
>>>> activate_locked:
>>>> /* Not a candidate for swapping, so reclaim swap space. */
>>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Pankaj
>>>>
>>>>
>>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>>> Sent: 14 March 2019 14:55:34
>>>> To: Pankaj Suryawanshi; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>>
>>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>>
>>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>>
>>>>> If I change in the vmscan.c as below patch, it will work.
>>>>
>>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>>> like in your patch below.
>>>>
>>>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>>>
>>>> [...]
>>>>
>>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>>> index be56e2e..2e51edc 100644
>>>>>> --- a/mm/vmscan.c
>>>>>> +++ b/mm/vmscan.c
>>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>>>> page = lru_to_page(page_list);
>>>>>> list_del(&page->lru);
>>>>>>
>>>>>> if (!trylock_page(page)) {
>>>>>> goto keep;
>>>>>> }
>>>>
>>>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>>>
>>>
>>>
>>>
>>
>>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-18 10:38 ` Kirill Tkhai
@ 2019-03-20 6:48 ` Pankaj Suryawanshi
2019-03-26 7:53 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-20 6:48 UTC (permalink / raw)
To: Kirill Tkhai, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual
________________________________________
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 18 March 2019 16:08
To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
On 18.03.2019 12:59, Pankaj Suryawanshi wrote:
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:17:56
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
Also, please, avoid irritating quoting like below ^^^. They just distract attention.
> On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
>> Hi Kirill Tkhai,
>>
>
> Please, do not top posting: https://kernelnewbies.org/mailinglistguidelines
>
> Okay.
>
> mailinglistguidelines - Linux Kernel Newbies
> kernelnewbies.org
> Set of FAQs for kernelnewbies mailing list. If you are new to this list please read this page before you go on your quest for squeezing all the knowledge from fellow members.
And this spew ^^^.
>> Please see mm/vmscan.c in which it first added to list and than throw the error :
>> --------------------------------------------------------------------------------------------------
>> keep:
>> list_add(&page->lru, &ret_pages);
>> VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
>> ---------------------------------------------------------------------------------------------------
>>
>> Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
>
> I say about about the list, which is passed to shrink_page_list() as first argument.
> Did you mean candidate list which is passed to shrink_page_list().
>
> shrink_inactive_list()
> {
> isolate_lru_pages(&page_list); // <-- you can't obtain unevictable pages here.
> shrink_page_list(&page_list);
> }
>
> below is the overview of flow of calls for your information.
>
> cma_alloc() ->
> alloc_contig_range() ->
> start_isolate_page_range() ->
> __alloc_contig_migrate_range() ->
> isolate_migratepages_range() ->
> reclaim_clean_pages_from_list() ->
> shrink_page_list()
Hm, isolate_migratepages_range() can take unevictable pages,
but then with your patch we just skip them in shrink_page_list().
Without your patch we bump into bug on.
I don't see any other issue/effect if i apply this patch.
These both look incorrect for me. Let's wait someone who familiar
with this logic.
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Sent: 18 March 2019 15:03:15
>> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> Hi, Pankaj,
>>
>> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>>
>>> Hello
>>>
>>> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>>
>> the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
>> PageUnevictable() pages are never being added to lists, which shrink_page_list()
>> uses for iteration. Also, a page can't be marked as PageUnevictable(), when
>> it's attached to a shrinkable list.
>>
>> So, the problem should be somewhere outside shrink_page_list().
>>
>> I won't suggest you something about CMA, since I haven't dived in that code.
>>
>>> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>>>
>>> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
>>> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>>>
>>>
>>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>>>
>>> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
>>> [ 24.726949] page->mem_cgroup:bd008c00
>>> [ 24.730693] ------------[ cut here ]------------
>>> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
>>> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>>
>>>
>>> Below is the patch which solved this issue :
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> sc->nr_scanned++;
>>>
>>> if (unlikely(!page_evictable(page)))
>>> - goto activate_locked;
>>> + goto cull_mlocked;
>>>
>>> if (!sc->may_unmap && page_mapped(page))
>>> goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> } else
>>> list_add(&page->lru, &free_pages);
>>> continue;
>>> -
>>> +cull_mlocked:
>>> + if (PageSwapCache(page))
>>> + try_to_free_swap(page);
>>> + unlock_page(page);
>>> + list_add(&page->lru, &ret_pages);
>>> + continue;
>>> activate_locked:
>>> /* Not a candidate for swapping, so reclaim swap space. */
>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>>
>>> It fixes the below issue.
>>>
>>> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>>>
>>> cma_alloc of current kernel will fail due to unevictable page
>>>
>>> Please let me know if anything i am missing.
>>>
>>> Regards,
>>> Pankaj
>>>
>>> From: Vlastimil Babka <vbabka@suse.cz>
>>> Sent: 18 March 2019 14:12:50
>>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>>
>>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>>
>>> Can you send a proper patch with changelog explaining the change? I
>>> don't know the context of this thread.
>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 15 March 2019 11:35:05
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> [ cc linux-mm ]
>>>>
>>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 14 March 2019 19:14:40
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> Hello ,
>>>>
>>>> Please ignore the curly braces, they are just for debugging.
>>>>
>>>> Below is the updated patch.
>>>>
>>>>
>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>> index be56e2e..12ac353 100644
>>>> --- a/mm/vmscan.c
>>>> +++ b/mm/vmscan.c
>>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> sc->nr_scanned++;
>>>>
>>>> if (unlikely(!page_evictable(page)))
>>>> - goto activate_locked;
>>>> + goto cull_mlocked;
>>>>
>>>> if (!sc->may_unmap && page_mapped(page))
>>>> goto keep_locked;
>>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> } else
>>>> list_add(&page->lru, &free_pages);
>>>> continue;
>>>> -
>>>> +cull_mlocked:
>>>> + if (PageSwapCache(page))
>>>> + try_to_free_swap(page);
>>>> + unlock_page(page);
>>>> + list_add(&page->lru, &ret_pages);
>>>> + continue;
>>>> activate_locked:
>>>> /* Not a candidate for swapping, so reclaim swap space. */
>>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Pankaj
>>>>
>>>>
>>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>>> Sent: 14 March 2019 14:55:34
>>>> To: Pankaj Suryawanshi; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>>
>>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>>
>>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>>
>>>>> If I change in the vmscan.c as below patch, it will work.
>>>>
>>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>>> like in your patch below.
>>>>
>>>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>>>
>>>> [...]
>>>>
>>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>>> index be56e2e..2e51edc 100644
>>>>>> --- a/mm/vmscan.c
>>>>>> +++ b/mm/vmscan.c
>>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>>>> page = lru_to_page(page_list);
>>>>>> list_del(&page->lru);
>>>>>>
>>>>>> if (!trylock_page(page)) {
>>>>>> goto keep;
>>>>>> }
>>>>
>>>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>>>
>>>
>>>
>>>
>>
>>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-20 6:48 ` Pankaj Suryawanshi
@ 2019-03-26 7:53 ` Pankaj Suryawanshi
2019-03-26 9:01 ` Michal Hocko
0 siblings, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-26 7:53 UTC (permalink / raw)
To: Kirill Tkhai, Vlastimil Babka, Michal Hocko, aneesh.kumar
Cc: linux-kernel, minchan, linux-mm, khandual
________________________________________
From: Pankaj Suryawanshi
Sent: 20 March 2019 12:18
To: Kirill Tkhai; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
________________________________________
From: Kirill Tkhai <ktkhai@virtuozzo.com>
Sent: 18 March 2019 16:08
To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
On 18.03.2019 12:59, Pankaj Suryawanshi wrote:
>
> From: Kirill Tkhai <ktkhai@virtuozzo.com>
> Sent: 18 March 2019 15:17:56
> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
Also, please, avoid irritating quoting like below ^^^. They just distract attention.
> On 18.03.2019 12:43, Pankaj Suryawanshi wrote:
>> Hi Kirill Tkhai,
>>
>
> Please, do not top posting: https://kernelnewbies.org/mailinglistguidelines
>
> Okay.
>
> mailinglistguidelines - Linux Kernel Newbies
> kernelnewbies.org
> Set of FAQs for kernelnewbies mailing list. If you are new to this list please read this page before you go on your quest for squeezing all the knowledge from fellow members.
And this spew ^^^.
>> Please see mm/vmscan.c in which it first added to list and than throw the error :
>> --------------------------------------------------------------------------------------------------
>> keep:
>> list_add(&page->lru, &ret_pages);
>> VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
>> ---------------------------------------------------------------------------------------------------
>>
>> Before throwing error, pages are added to list, this is under iteration of shrink_page_list().
>
> I say about about the list, which is passed to shrink_page_list() as first argument.
> Did you mean candidate list which is passed to shrink_page_list().
>
> shrink_inactive_list()
> {
> isolate_lru_pages(&page_list); // <-- you can't obtain unevictable pages here.
> shrink_page_list(&page_list);
> }
>
> below is the overview of flow of calls for your information.
>
> cma_alloc() ->
> alloc_contig_range() ->
> start_isolate_page_range() ->
> __alloc_contig_migrate_range() ->
> isolate_migratepages_range() ->
> reclaim_clean_pages_from_list() ->
> shrink_page_list()
Hm, isolate_migratepages_range() can take unevictable pages,
but then with your patch we just skip them in shrink_page_list().
Without your patch we bump into bug on.
I don't see any other issue/effect if i apply this patch.
These both look incorrect for me. Let's wait someone who familiar
with this logic.
Is there anyone who is familiar with this? Please Comment.
>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>> Sent: 18 March 2019 15:03:15
>> To: Pankaj Suryawanshi; Vlastimil Babka; Michal Hocko; aneesh.kumar@linux.ibm.com
>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>
>>
>> Hi, Pankaj,
>>
>> On 18.03.2019 12:09, Pankaj Suryawanshi wrote:
>>>
>>> Hello
>>>
>>> shrink_page_list() returns , number of pages reclaimed, when pages is unevictable it returns VM_BUG_ON_PAGE(PageLRU(page) || PageUnevicatble(page),page);
>>
>> the general idea is shrink_page_list() can't iterate PageUnevictable() pages.
>> PageUnevictable() pages are never being added to lists, which shrink_page_list()
>> uses for iteration. Also, a page can't be marked as PageUnevictable(), when
>> it's attached to a shrinkable list.
>>
>> So, the problem should be somewhere outside shrink_page_list().
>>
>> I won't suggest you something about CMA, since I haven't dived in that code.
>>
>>> We can add the unevictable pages in reclaim list in shrink_page_list(), return total number of reclaim pages including unevictable pages, let the caller handle unevictable pages.
>>>
>>> I think the problem is shrink_page_list is awkard. If page is unevictable it goto activate_locked->keep_locked->keep lables, keep lable list_add the unevictable pages and throw the VM_BUG instead of passing it to caller while it relies on caller for non-reclaimed-non-unevictable page's putback.
>>> I think we can make it consistent so that shrink_page_list could return non-reclaimed pages via page_list and caller can handle it. As an advance, it could try to migrate mlocked pages without retrial.
>>>
>>>
>>> Below is the issue of CMA_ALLOC of large size buffer : (Kernel version - 4.14.65 (On Android pie [ARM])).
>>>
>>> [ 24.718792] page dumped because: VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page))
>>> [ 24.726949] page->mem_cgroup:bd008c00
>>> [ 24.730693] ------------[ cut here ]------------
>>> [ 24.735304] kernel BUG at mm/vmscan.c:1350!
>>> [ 24.739478] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>>>
>>>
>>> Below is the patch which solved this issue :
>>>
>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>> index be56e2e..12ac353 100644
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> sc->nr_scanned++;
>>>
>>> if (unlikely(!page_evictable(page)))
>>> - goto activate_locked;
>>> + goto cull_mlocked;
>>>
>>> if (!sc->may_unmap && page_mapped(page))
>>> goto keep_locked;
>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>> } else
>>> list_add(&page->lru, &free_pages);
>>> continue;
>>> -
>>> +cull_mlocked:
>>> + if (PageSwapCache(page))
>>> + try_to_free_swap(page);
>>> + unlock_page(page);
>>> + list_add(&page->lru, &ret_pages);
>>> + continue;
>>> activate_locked:
>>> /* Not a candidate for swapping, so reclaim swap space. */
>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>
>>>
>>>
>>>
>>> It fixes the below issue.
>>>
>>> 1. Large size buffer allocation using cma_alloc successful with unevictable pages.
>>>
>>> cma_alloc of current kernel will fail due to unevictable page
>>>
>>> Please let me know if anything i am missing.
>>>
>>> Regards,
>>> Pankaj
>>>
>>> From: Vlastimil Babka <vbabka@suse.cz>
>>> Sent: 18 March 2019 14:12:50
>>> To: Pankaj Suryawanshi; Kirill Tkhai; Michal Hocko; aneesh.kumar@linux.ibm.com
>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com; hillf.zj@alibaba-inc.com
>>> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>
>>>
>>> On 3/15/19 11:11 AM, Pankaj Suryawanshi wrote:
>>>>
>>>> [ cc Aneesh kumar, Anshuman, Hillf, Vlastimil]
>>>
>>> Can you send a proper patch with changelog explaining the change? I
>>> don't know the context of this thread.
>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 15 March 2019 11:35:05
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> [ cc linux-mm ]
>>>>
>>>>
>>>> From: Pankaj Suryawanshi
>>>> Sent: 14 March 2019 19:14:40
>>>> To: Kirill Tkhai; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>>
>>>> Hello ,
>>>>
>>>> Please ignore the curly braces, they are just for debugging.
>>>>
>>>> Below is the updated patch.
>>>>
>>>>
>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>> index be56e2e..12ac353 100644
>>>> --- a/mm/vmscan.c
>>>> +++ b/mm/vmscan.c
>>>> @@ -998,7 +998,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> sc->nr_scanned++;
>>>>
>>>> if (unlikely(!page_evictable(page)))
>>>> - goto activate_locked;
>>>> + goto cull_mlocked;
>>>>
>>>> if (!sc->may_unmap && page_mapped(page))
>>>> goto keep_locked;
>>>> @@ -1331,7 +1331,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>> } else
>>>> list_add(&page->lru, &free_pages);
>>>> continue;
>>>> -
>>>> +cull_mlocked:
>>>> + if (PageSwapCache(page))
>>>> + try_to_free_swap(page);
>>>> + unlock_page(page);
>>>> + list_add(&page->lru, &ret_pages);
>>>> + continue;
>>>> activate_locked:
>>>> /* Not a candidate for swapping, so reclaim swap space. */
>>>> if (PageSwapCache(page) && (mem_cgroup_swap_full(page) ||
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Pankaj
>>>>
>>>>
>>>> From: Kirill Tkhai <ktkhai@virtuozzo.com>
>>>> Sent: 14 March 2019 14:55:34
>>>> To: Pankaj Suryawanshi; Michal Hocko
>>>> Cc: linux-kernel@vger.kernel.org; minchan@kernel.org
>>>> Subject: Re: Re: [External] Re: vmscan: Reclaim unevictable pages
>>>>
>>>>
>>>> On 14.03.2019 11:52, Pankaj Suryawanshi wrote:
>>>>>
>>>>> I am using kernel version 4.14.65 (on Android pie [ARM]).
>>>>>
>>>>> No additional patches applied on top of vanilla.(Core MM).
>>>>>
>>>>> If I change in the vmscan.c as below patch, it will work.
>>>>
>>>> Sorry, but 4.14.65 does not have braces around trylock_page(),
>>>> like in your patch below.
>>>>
>>>> See https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/mm/vmscan.c?h=v4.14.65
>>>>
>>>> [...]
>>>>
>>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>>> index be56e2e..2e51edc 100644
>>>>>> --- a/mm/vmscan.c
>>>>>> +++ b/mm/vmscan.c
>>>>>> @@ -990,15 +990,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
>>>>>> page = lru_to_page(page_list);
>>>>>> list_del(&page->lru);
>>>>>>
>>>>>> if (!trylock_page(page)) {
>>>>>> goto keep;
>>>>>> }
>>>>
>>>> ************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
>>>>
>>>
>>>
>>>
>>
>>
>
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-26 7:53 ` Pankaj Suryawanshi
@ 2019-03-26 9:01 ` Michal Hocko
2019-03-26 9:12 ` Pankaj Suryawanshi
2019-03-26 9:16 ` Pankaj Suryawanshi
0 siblings, 2 replies; 25+ messages in thread
From: Michal Hocko @ 2019-03-26 9:01 UTC (permalink / raw)
To: Pankaj Suryawanshi
Cc: Kirill Tkhai, Vlastimil Babka, aneesh.kumar, linux-kernel,
minchan, linux-mm, khandual
[You were asked to use a reasonable quoting several times. This is
really annoying because it turns the email thread into a complete mess]
On Tue 26-03-19 07:53:14, Pankaj Suryawanshi wrote:
> Is there anyone who is familiar with this? Please Comment.
Not really. You are observing an unexpected behavior of the page reclaim
which hasn't changed for quite some time. So I find more probable that
your non-vanilla kernel is doing something unexpected. It would help if
you could track down how does the unevictable page get down to the
reclaim path. I assume this is a CMA page or something like that but
those shouldn't get to the reclaim path AFIR.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-26 9:01 ` Michal Hocko
@ 2019-03-26 9:12 ` Pankaj Suryawanshi
2019-03-26 9:16 ` Pankaj Suryawanshi
1 sibling, 0 replies; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-26 9:12 UTC (permalink / raw)
To: Michal Hocko
Cc: Kirill Tkhai, Vlastimil Babka, aneesh.kumar, linux-kernel,
minchan, linux-mm, khandual
________________________________________
From: Michal Hocko <mhocko@kernel.org>
Sent: 26 March 2019 14:31
To: Pankaj Suryawanshi
Cc: Kirill Tkhai; Vlastimil Babka; aneesh.kumar@linux.ibm.com; linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
[You were asked to use a reasonable quoting several times. This is
really annoying because it turns the email thread into a complete mess]
I already fix the email client, dont know the reason for quoting.
On Tue 26-03-19 07:53:14, Pankaj Suryawanshi wrote:
> Is there anyone who is familiar with this? Please Comment.
Not really. You are observing an unexpected behavior of the page reclaim
which hasn't changed for quite some time. So I find more probable that
your non-vanilla kernel is doing something unexpected. It would help if
you could track down how does the unevictable page get down to the
reclaim path. I assume this is a CMA page or something like that but
those shouldn't get to the reclaim path AFIR.
As i said earlier, the kernel i am using is vanilla kernel.
--
Michal Hocko
SUSE Labs
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-26 9:01 ` Michal Hocko
2019-03-26 9:12 ` Pankaj Suryawanshi
@ 2019-03-26 9:16 ` Pankaj Suryawanshi
2019-03-26 9:36 ` Michal Hocko
1 sibling, 1 reply; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-26 9:16 UTC (permalink / raw)
To: Michal Hocko
Cc: Kirill Tkhai, Vlastimil Babka, aneesh.kumar, linux-kernel,
minchan, linux-mm, khandual
________________________________________
From: Michal Hocko <mhocko@kernel.org>
Sent: 26 March 2019 14:31
To: Pankaj Suryawanshi
Cc: Kirill Tkhai; Vlastimil Babka; aneesh.kumar@linux.ibm.com; linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
[You were asked to use a reasonable quoting several times. This is
really annoying because it turns the email thread into a complete mess]
[Already fix the email client, but dont know the reason for quoting Maybe account issue.]
On Tue 26-03-19 07:53:14, Pankaj Suryawanshi wrote:
> Is there anyone who is familiar with this? Please Comment.
Not really. You are observing an unexpected behavior of the page reclaim
which hasn't changed for quite some time. So I find more probable that
your non-vanilla kernel is doing something unexpected. It would help if
you could track down how does the unevictable page get down to the
reclaim path. I assume this is a CMA page or something like that but
those shouldn't get to the reclaim path AFIR.
As i said earlier, i am using vanilla kernel 4.14.65.
--
Michal Hocko
SUSE Labs
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-26 9:16 ` Pankaj Suryawanshi
@ 2019-03-26 9:36 ` Michal Hocko
2019-03-29 5:29 ` Pankaj Suryawanshi
0 siblings, 1 reply; 25+ messages in thread
From: Michal Hocko @ 2019-03-26 9:36 UTC (permalink / raw)
To: Pankaj Suryawanshi
Cc: Kirill Tkhai, Vlastimil Babka, aneesh.kumar, linux-kernel,
minchan, linux-mm, khandual
On Tue 26-03-19 09:16:11, Pankaj Suryawanshi wrote:
>
> ________________________________________
> From: Michal Hocko <mhocko@kernel.org>
> Sent: 26 March 2019 14:31
> To: Pankaj Suryawanshi
> Cc: Kirill Tkhai; Vlastimil Babka; aneesh.kumar@linux.ibm.com; linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
> [You were asked to use a reasonable quoting several times. This is
> really annoying because it turns the email thread into a complete mess]
>
> [Already fix the email client, but dont know the reason for quoting Maybe account issue.]
You clearly haven't
> As i said earlier, i am using vanilla kernel 4.14.65.
This got lost in the quoting mess. Can you reproduce with 5.0?
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] Re: vmscan: Reclaim unevictable pages
2019-03-26 9:36 ` Michal Hocko
@ 2019-03-29 5:29 ` Pankaj Suryawanshi
0 siblings, 0 replies; 25+ messages in thread
From: Pankaj Suryawanshi @ 2019-03-29 5:29 UTC (permalink / raw)
To: Michal Hocko
Cc: Kirill Tkhai, Vlastimil Babka, aneesh.kumar, linux-kernel,
minchan, linux-mm, khandual
________________________________________
From: Michal Hocko <mhocko@kernel.org>
Sent: 26 March 2019 15:06
To: Pankaj Suryawanshi
Cc: Kirill Tkhai; Vlastimil Babka; aneesh.kumar@linux.ibm.com; linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
On Tue 26-03-19 09:16:11, Pankaj Suryawanshi wrote:
>
> ________________________________________
> From: Michal Hocko <mhocko@kernel.org>
> Sent: 26 March 2019 14:31
> To: Pankaj Suryawanshi
> Cc: Kirill Tkhai; Vlastimil Babka; aneesh.kumar@linux.ibm.com; linux-kernel@vger.kernel.org; minchan@kernel.org; linux-mm@kvack.org; khandual@linux.vnet.ibm.com
> Subject: Re: [External] Re: vmscan: Reclaim unevictable pages
>
> [You were asked to use a reasonable quoting several times. This is
> really annoying because it turns the email thread into a complete mess]
>
> [Already fix the email client, but dont know the reason for quoting Maybe account issue.]
You clearly haven't
> As i said earlier, i am using vanilla kernel 4.14.65.
This got lost in the quoting mess. Can you reproduce with 5.0?
Actually i am using android pie-9.0, can i replace kernel 4.14.65 to 5.0 ?
--
Michal Hocko
SUSE Labs
************************************************************************************************************************************************************* eInfochips Business Disclaimer: This e-mail message and all attachments transmitted with it are intended solely for the use of the addressee and may contain legally privileged and confidential information. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, copying, or other use of this message or its attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately by replying to this message and please delete it from your computer. Any views expressed in this message are those of the individual sender unless otherwise stated. Company has taken enough precautions to prevent the spread of viruses. However the company accepts no liability for any damage caused by any virus transmitted by this email. *************************************************************************************************************************************************************
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2019-03-29 5:30 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <SG2PR02MB3098A05E09B0D3F3CB1C3B9BE84B0@SG2PR02MB3098.apcprd02.prod.outlook.com>
2019-03-14 7:53 ` vmscan: Reclaim unevictable pages Pankaj Suryawanshi
2019-03-14 8:23 ` Pankaj Suryawanshi
2019-03-14 8:41 ` Michal Hocko
2019-03-14 8:52 ` [External] " Pankaj Suryawanshi
2019-03-14 9:25 ` Kirill Tkhai
2019-03-14 11:09 ` Pankaj Suryawanshi
2019-03-14 13:44 ` Pankaj Suryawanshi
2019-03-15 6:05 ` Pankaj Suryawanshi
2019-03-15 10:11 ` Pankaj Suryawanshi
2019-03-18 7:45 ` Pankaj Suryawanshi
2019-03-18 8:56 ` Pankaj Suryawanshi
2019-03-18 8:42 ` [External] " Vlastimil Babka
2019-03-18 9:09 ` Pankaj Suryawanshi
2019-03-18 9:33 ` Kirill Tkhai
2019-03-18 9:43 ` Pankaj Suryawanshi
2019-03-18 9:47 ` Kirill Tkhai
2019-03-18 9:59 ` Pankaj Suryawanshi
2019-03-18 10:38 ` Kirill Tkhai
2019-03-20 6:48 ` Pankaj Suryawanshi
2019-03-26 7:53 ` Pankaj Suryawanshi
2019-03-26 9:01 ` Michal Hocko
2019-03-26 9:12 ` Pankaj Suryawanshi
2019-03-26 9:16 ` Pankaj Suryawanshi
2019-03-26 9:36 ` Michal Hocko
2019-03-29 5:29 ` Pankaj Suryawanshi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).