All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mel Gorman <mgorman@suse.de>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Lin Feng <linfeng@cn.fujitsu.com>,
	bcrl@kvack.org, viro@zeniv.linux.org.uk, khlebnikov@openvz.org,
	walken@google.com, kamezawa.hiroyu@jp.fujitsu.com,
	minchan@kernel.org, riel@redhat.com, rientjes@google.com,
	isimatu.yasuaki@jp.fujitsu.com, wency@cn.fujitsu.com,
	laijs@cn.fujitsu.com, jiang.liu@huawei.com, mhocko@suse.cz,
	linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()
Date: Tue, 5 Feb 2013 13:32:44 +0000	[thread overview]
Message-ID: <20130205133244.GH21389@suse.de> (raw)
In-Reply-To: <20130205115722.GF21389@suse.de>

On Tue, Feb 05, 2013 at 11:57:22AM +0000, Mel Gorman wrote:
> 
> > > +				migrate_pre_flag = 1;
> > > +			}
> > > +
> > > +			if (!isolate_lru_page(pages[i])) {
> > > +				inc_zone_page_state(pages[i], NR_ISOLATED_ANON +
> > > +						 page_is_file_cache(pages[i]));
> > > +				list_add_tail(&pages[i]->lru, &pagelist);
> > > +			} else {
> > > +				isolate_err = 1;
> > > +				goto put_page;
> > > +			}
> 
> isolate_lru_page() takes the LRU lock every time.

Credit to Michal Hocko for bringing this up but with the number of
other issues I missed that this is also broken with respect to huge page
handling. hugetlbfs pages will not be on the LRU so the isolation will mess
up and the migration has to be handled differently.  Ordinarily hugetlbfs
pages cannot be allocated from ZONE_MOVABLE but it is possible to configure
it to be allowed via /proc/sys/vm/hugepages_treat_as_movable. If this
encounters a hugetlbfs page, it'll just blow up.

The other is that this almost certainly broken for transhuge page
handling. gup returns the head and tail pages and ordinarily this is ok
because the caller only cares about the physical address. Migration will
also split a hugepage if it receives it but you are potentially adding
tail pages to a list here and then migrating them. The split of the first
page will get very confused. I'm not exactly sure what the result will be
but it won't be pretty.

Was THP enabled when this was tested? Was CONFIG_DEBUG_LIST enabled
during testing?

-- 
Mel Gorman
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Lin Feng <linfeng@cn.fujitsu.com>,
	bcrl@kvack.org, viro@zeniv.linux.org.uk, khlebnikov@openvz.org,
	walken@google.com, kamezawa.hiroyu@jp.fujitsu.com,
	minchan@kernel.org, riel@redhat.com, rientjes@google.com,
	isimatu.yasuaki@jp.fujitsu.com, wency@cn.fujitsu.com,
	laijs@cn.fujitsu.com, jiang.liu@huawei.com, mhocko@suse.cz,
	linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()
Date: Tue, 5 Feb 2013 13:32:44 +0000	[thread overview]
Message-ID: <20130205133244.GH21389@suse.de> (raw)
In-Reply-To: <20130205115722.GF21389@suse.de>

On Tue, Feb 05, 2013 at 11:57:22AM +0000, Mel Gorman wrote:
> 
> > > +				migrate_pre_flag = 1;
> > > +			}
> > > +
> > > +			if (!isolate_lru_page(pages[i])) {
> > > +				inc_zone_page_state(pages[i], NR_ISOLATED_ANON +
> > > +						 page_is_file_cache(pages[i]));
> > > +				list_add_tail(&pages[i]->lru, &pagelist);
> > > +			} else {
> > > +				isolate_err = 1;
> > > +				goto put_page;
> > > +			}
> 
> isolate_lru_page() takes the LRU lock every time.

Credit to Michal Hocko for bringing this up but with the number of
other issues I missed that this is also broken with respect to huge page
handling. hugetlbfs pages will not be on the LRU so the isolation will mess
up and the migration has to be handled differently.  Ordinarily hugetlbfs
pages cannot be allocated from ZONE_MOVABLE but it is possible to configure
it to be allowed via /proc/sys/vm/hugepages_treat_as_movable. If this
encounters a hugetlbfs page, it'll just blow up.

The other is that this almost certainly broken for transhuge page
handling. gup returns the head and tail pages and ordinarily this is ok
because the caller only cares about the physical address. Migration will
also split a hugepage if it receives it but you are potentially adding
tail pages to a list here and then migrating them. The split of the first
page will get very confused. I'm not exactly sure what the result will be
but it won't be pretty.

Was THP enabled when this was tested? Was CONFIG_DEBUG_LIST enabled
during testing?

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to majordomo@kvack.org.  For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>

WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Lin Feng <linfeng@cn.fujitsu.com>,
	bcrl@kvack.org, viro@zeniv.linux.org.uk, khlebnikov@openvz.org,
	walken@google.com, kamezawa.hiroyu@jp.fujitsu.com,
	minchan@kernel.org, riel@redhat.com, rientjes@google.com,
	isimatu.yasuaki@jp.fujitsu.com, wency@cn.fujitsu.com,
	laijs@cn.fujitsu.com, jiang.liu@huawei.com, mhocko@suse.cz,
	linux-mm@kvack.org, linux-aio@kvack.org,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable()
Date: Tue, 5 Feb 2013 13:32:44 +0000	[thread overview]
Message-ID: <20130205133244.GH21389@suse.de> (raw)
In-Reply-To: <20130205115722.GF21389@suse.de>

On Tue, Feb 05, 2013 at 11:57:22AM +0000, Mel Gorman wrote:
> 
> > > +				migrate_pre_flag = 1;
> > > +			}
> > > +
> > > +			if (!isolate_lru_page(pages[i])) {
> > > +				inc_zone_page_state(pages[i], NR_ISOLATED_ANON +
> > > +						 page_is_file_cache(pages[i]));
> > > +				list_add_tail(&pages[i]->lru, &pagelist);
> > > +			} else {
> > > +				isolate_err = 1;
> > > +				goto put_page;
> > > +			}
> 
> isolate_lru_page() takes the LRU lock every time.

Credit to Michal Hocko for bringing this up but with the number of
other issues I missed that this is also broken with respect to huge page
handling. hugetlbfs pages will not be on the LRU so the isolation will mess
up and the migration has to be handled differently.  Ordinarily hugetlbfs
pages cannot be allocated from ZONE_MOVABLE but it is possible to configure
it to be allowed via /proc/sys/vm/hugepages_treat_as_movable. If this
encounters a hugetlbfs page, it'll just blow up.

The other is that this almost certainly broken for transhuge page
handling. gup returns the head and tail pages and ordinarily this is ok
because the caller only cares about the physical address. Migration will
also split a hugepage if it receives it but you are potentially adding
tail pages to a list here and then migrating them. The split of the first
page will get very confused. I'm not exactly sure what the result will be
but it won't be pretty.

Was THP enabled when this was tested? Was CONFIG_DEBUG_LIST enabled
during testing?

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2013-02-05 13:32 UTC|newest]

Thread overview: 88+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-02-04 10:04 [PATCH 0/2] mm: hotplug: implement non-movable version of get_user_pages() to kill long-time pin pages Lin Feng
2013-02-04 10:04 ` Lin Feng
2013-02-04 10:04 ` [PATCH 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable() Lin Feng
2013-02-04 10:04   ` Lin Feng
2013-02-04 10:04   ` Lin Feng
2013-02-05  0:06   ` Andrew Morton
2013-02-05  0:06     ` Andrew Morton
2013-02-05  0:06     ` Andrew Morton
2013-02-05  0:18     ` Andrew Morton
2013-02-05  0:18       ` Andrew Morton
2013-02-05  3:09     ` Lin Feng
2013-02-05  3:09       ` Lin Feng
2013-02-05  3:09       ` Lin Feng
2013-02-05 21:13       ` Andrew Morton
2013-02-05 21:13         ` Andrew Morton
2013-02-05 21:13         ` Andrew Morton
2013-02-05 11:57     ` Mel Gorman
2013-02-05 11:57       ` Mel Gorman
2013-02-05 11:57       ` Mel Gorman
2013-02-05 13:32       ` Mel Gorman [this message]
2013-02-05 13:32         ` Mel Gorman
2013-02-05 13:32         ` Mel Gorman
2013-02-19 13:37         ` Lin Feng
2013-02-19 13:37           ` Lin Feng
2013-02-20  2:34           ` Lin Feng
2013-02-20  2:34             ` Lin Feng
2013-02-20  2:34             ` Lin Feng
2013-02-20  2:44             ` Wanpeng Li
2013-02-20  2:44             ` Wanpeng Li
2013-02-20  2:44             ` Wanpeng Li
2013-02-20  2:59               ` Lin Feng
2013-02-20  2:59                 ` Lin Feng
2013-02-20  9:58         ` Simon Jeons
2013-02-20  9:58           ` Simon Jeons
2013-02-20 10:23           ` Lin Feng
2013-02-20 10:23             ` Lin Feng
2013-02-20 10:23             ` Lin Feng
2013-02-20 11:31             ` Simon Jeons
2013-02-20 11:31               ` Simon Jeons
2013-02-20 11:54               ` Lin Feng
2013-02-20 11:54                 ` Lin Feng
2013-02-20 11:54                 ` Lin Feng
2013-02-06  2:26       ` Michel Lespinasse
2013-02-06  2:26         ` Michel Lespinasse
2013-02-06  2:26         ` Michel Lespinasse
2013-02-06 10:41         ` Mel Gorman
2013-02-06 10:41           ` Mel Gorman
2013-02-18 10:34       ` Lin Feng
2013-02-18 10:34         ` Lin Feng
2013-02-18 10:34         ` Lin Feng
2013-02-18 15:17         ` Mel Gorman
2013-02-18 15:17           ` Mel Gorman
2013-02-18 15:17           ` Mel Gorman
2013-02-19  9:55           ` Lin Feng
2013-02-19  9:55             ` Lin Feng
2013-02-19 10:34             ` Mel Gorman
2013-02-19 10:34               ` Mel Gorman
2013-02-19 10:34               ` Mel Gorman
2013-02-04 10:04 ` [PATCH 2/2] fs/aio.c: use get_user_pages_non_movable() to pin ring pages when support memory hotremove Lin Feng
2013-02-04 10:04   ` Lin Feng
2013-02-04 10:04   ` Lin Feng
2013-02-04 15:18   ` Jeff Moyer
2013-02-04 15:18     ` Jeff Moyer
2013-02-04 15:18     ` Jeff Moyer
2013-02-04 23:02     ` Zach Brown
2013-02-04 23:02       ` Zach Brown
2013-02-04 23:02       ` Zach Brown
2013-02-05  5:35       ` Lin Feng
2013-02-05  5:35         ` Lin Feng
2013-02-05  5:35         ` Lin Feng
2013-02-05  5:06     ` Lin Feng
2013-02-05  5:06       ` Lin Feng
2013-02-05  0:58 ` [PATCH 0/2] mm: hotplug: implement non-movable version of get_user_pages() to kill long-time pin pages Minchan Kim
2013-02-05  0:58   ` Minchan Kim
2013-02-05  0:58   ` Minchan Kim
2013-02-05  4:42   ` Lin Feng
2013-02-05  4:42     ` Lin Feng
2013-02-05  5:25     ` Minchan Kim
2013-02-05  5:25       ` Minchan Kim
2013-02-05  5:25       ` Minchan Kim
2013-02-05  6:18       ` Lin Feng
2013-02-05  6:18         ` Lin Feng
2013-02-05  7:45         ` Minchan Kim
2013-02-05  7:45           ` Minchan Kim
2013-02-05  7:45           ` Minchan Kim
2013-02-05  8:27           ` Lin Feng
2013-02-05  8:27             ` Lin Feng
2013-02-05  8:27             ` Lin Feng

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20130205133244.GH21389@suse.de \
    --to=mgorman@suse.de \
    --cc=akpm@linux-foundation.org \
    --cc=bcrl@kvack.org \
    --cc=isimatu.yasuaki@jp.fujitsu.com \
    --cc=jiang.liu@huawei.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=khlebnikov@openvz.org \
    --cc=laijs@cn.fujitsu.com \
    --cc=linfeng@cn.fujitsu.com \
    --cc=linux-aio@kvack.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.cz \
    --cc=minchan@kernel.org \
    --cc=riel@redhat.com \
    --cc=rientjes@google.com \
    --cc=viro@zeniv.linux.org.uk \
    --cc=walken@google.com \
    --cc=wency@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.