Anshuman Khandual wrote:
> On 04/21/2017 02:17 AM, Zi Yan wrote:
>> From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>>
>> This patch enables thp migration for soft offline.
>>
>> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>>
>> ChangeLog: v1 -> v5:
>> - fix page isolation counting error
>>
>> Signed-off-by: Zi Yan <zi.yan@cs.rutgers.edu>
>> ---
>>  mm/memory-failure.c | 35 ++++++++++++++---------------------
>>  1 file changed, 14 insertions(+), 21 deletions(-)
>>
>> diff --git a/mm/memory-failure.c b/mm/memory-failure.c
>> index 9b77476ef31f..23ff02eb3ed4 100644
>> --- a/mm/memory-failure.c
>> +++ b/mm/memory-failure.c
>> @@ -1481,7 +1481,17 @@ static struct page *new_page(struct page *p, unsigned long private, int **x)
>>  	if (PageHuge(p))
>>  		return alloc_huge_page_node(page_hstate(compound_head(p)),
>>  						   nid);
>> -	else
>> +	else if (thp_migration_supported() && PageTransHuge(p)) {
>> +		struct page *thp;
>> +
>> +		thp = alloc_pages_node(nid,
>> +			(GFP_TRANSHUGE | __GFP_THISNODE) & ~__GFP_RECLAIM,
> 
> Why not __GFP_RECLAIM ? Its soft offline path we wait a bit before
> declaring that THP page cannot be allocated and hence should invoke
> reclaim methods as well.

I am not sure how much effort the kernel wants to put here to soft
offline a THP. Naoya knows more here.


> 
>> +			HPAGE_PMD_ORDER);
>> +		if (!thp)
>> +			return NULL;
>> +		prep_transhuge_page(thp);
>> +		return thp;
>> +	} else
>>  		return __alloc_pages_node(nid, GFP_HIGHUSER_MOVABLE, 0);
>>  }
>>  
>> @@ -1665,8 +1675,8 @@ static int __soft_offline_page(struct page *page, int flags)
>>  		 * cannot have PAGE_MAPPING_MOVABLE.
>>  		 */
>>  		if (!__PageMovable(page))
>> -			inc_node_page_state(page, NR_ISOLATED_ANON +
>> -						page_is_file_cache(page));
>> +			mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON +
>> +						page_is_file_cache(page), hpage_nr_pages(page));
>>  		list_add(&page->lru, &pagelist);
>>  		ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL,
>>  					MIGRATE_SYNC, MR_MEMORY_FAILURE);
>> @@ -1689,28 +1699,11 @@ static int __soft_offline_page(struct page *page, int flags)
>>  static int soft_offline_in_use_page(struct page *page, int flags)
>>  {
>>  	int ret;
>> -	struct page *hpage = compound_head(page);
>> -
>> -	if (!PageHuge(page) && PageTransHuge(hpage)) {
>> -		lock_page(hpage);
>> -		if (!PageAnon(hpage) || unlikely(split_huge_page(hpage))) {
>> -			unlock_page(hpage);
>> -			if (!PageAnon(hpage))
>> -				pr_info("soft offline: %#lx: non anonymous thp\n", page_to_pfn(page));
>> -			else
>> -				pr_info("soft offline: %#lx: thp split failed\n", page_to_pfn(page));
>> -			put_hwpoison_page(hpage);
>> -			return -EBUSY;
>> -		}
>> -		unlock_page(hpage);
>> -		get_hwpoison_page(page);
>> -		put_hwpoison_page(hpage);
>> -	}
>>  
>>  	if (PageHuge(page))
>>  		ret = soft_offline_huge_page(page, flags);
>>  	else
>> -		ret = __soft_offline_page(page, flags);
>> +		ret = __soft_offline_page(compound_head(page), flags);
> 
> Hmm, what if the THP allocation fails in the new_page() path and
> we fallback for general page allocation. In that case we will
> always be still calling with the head page ? Because we dont
> split the huge page any more.

This could be a problem if the user wants to offline a TailPage but due
to THP allocation failure, the HeadPage is offlined.

It may be better to only soft offline THPs if page ==
compound_head(page). If page != compound_head(page), we still split THPs
like before.

Because in migrate_pages(), we cannot guarantee any TailPages in that
THP are migrated (1. THP allocation failure causes THP splitting, then
only HeadPage is going to be migrated; 2. even if we change existing
migrate_pages() implementation to add all TailPages to migration list
instead of LRU list, we still cannot guarantee the TailPage we want to
migrate is migrated.).

Naoya, what do you think?

-- 
Best Regards,
Yan Zi