From: Mel Gorman <mgorman@suse.de> To: Andrew Morton <akpm@linux-foundation.org> Cc: Andrea Arcangeli <aarcange@redhat.com>, Minchan Kim <minchan.kim@gmail.com>, Dave Jones <davej@redhat.com>, Jan Kara <jack@suse.cz>, Andy Isaacson <adi@hexapodia.org>, Johannes Weiner <jweiner@redhat.com>, Rik van Riel <riel@redhat.com>, Nai Xia <nai.xia@gmail.com>, Linux-MM <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org> Subject: Re: [PATCH 05/11] mm: compaction: Determine if dirty pages can be migrated without blocking within ->migratepage Date: Mon, 19 Dec 2011 11:05:51 +0000 [thread overview] Message-ID: <20111219110551.GJ3487@suse.de> (raw) In-Reply-To: <20111216152054.f7445e98.akpm@linux-foundation.org> On Fri, Dec 16, 2011 at 03:20:54PM -0800, Andrew Morton wrote: > On Wed, 14 Dec 2011 15:41:27 +0000 > Mel Gorman <mgorman@suse.de> wrote: > > > Asynchronous compaction is used when allocating transparent hugepages > > to avoid blocking for long periods of time. Due to reports of > > stalling, there was a debate on disabling synchronous compaction > > but this severely impacted allocation success rates. Part of the > > reason was that many dirty pages are skipped in asynchronous compaction > > by the following check; > > > > if (PageDirty(page) && !sync && > > mapping->a_ops->migratepage != migrate_page) > > rc = -EBUSY; > > > > This skips over all mapping aops using buffer_migrate_page() > > even though it is possible to migrate some of these pages without > > blocking. This patch updates the ->migratepage callback with a "sync" > > parameter. It is the responsibility of the callback to fail gracefully > > if migration would block. > > > > ... > > > > @@ -259,6 +309,19 @@ static int migrate_page_move_mapping(struct address_space *mapping, > > } > > > > /* > > + * In the async migration case of moving a page with buffers, lock the > > + * buffers using trylock before the mapping is moved. If the mapping > > + * was moved, we later failed to lock the buffers and could not move > > + * the mapping back due to an elevated page count, we would have to > > + * block waiting on other references to be dropped. > > + */ > > + if (!sync && head && !buffer_migrate_lock_buffers(head, sync)) { > > Once it has been established that "sync" is true, I find it clearer to > pass in plain old "true" to buffer_migrate_lock_buffers(). Minor point. > Later in the series, sync changes to "mode" to distinguish between async, sync-light and sync compaction. At that point, this becomes if (mode == MIGRATE_ASYNC && head && !buffer_migrate_lock_buffers(head, mode)) { Passing true in here would be fine, but it would just end up being changed back later in the series so it can be left alone. > I hadn't paid a lot of attention to buffer_migrate_page() before. > Scary function. I'm rather worried about its interactions with ext3 > journal commit which locks buffers then plays with them while leaving > the page unlocked. How vigorously has this been whitebox-tested? > Blackbox testing only AFAIK. This has been tested recently with ext3 and nothing unusual was reported. The list of events for migration looks like isolate page from LRU migrate_pages unmap_and_move lock_page(src_page) if page under writeback, either bail or wait on writeback try_to_unmap move_to_new_page lock_page(dst_page) buffer_migrate_page migrate_page_move_mapping spin_lock_irq(&mapping->tree_lock) lookup in radix tree check reference counts to make sure no one else has references lock buffers if async mode replace page in radix tree with new page spin_unlock_irq lock buffers if !async mode copy buffers unlock buffers unlock_page(dst_page) The critical part is that the copying of buffer data is happening with both page and buffer locks held and no other references to the page exists - it has already been unmapped for example. Journal commit minimally acquires the buffer lock. If migration is in the process of copying the buffers, the buffer lock will prevent journal commit starting at the same time buffers are being copied. block_write_full_page and friends should be taking the buffer lock so they should also be ok. For other accessors, the mapping tree_lock should prevent other users looking up the page in the radix tree in the first place while the radix tree replacement is taking place. Racing against try_to_free_buffer should also be a problem. According to buffer.c, exclusion from try_to_free_buffer "may be obtained by either locking the page or holding the mappings private_lock". Migration is holding the page lock. Taking private_lock would give additional protection but I haven't heard or seen a case where it is necessary. -- Mel Gorman SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de> To: Andrew Morton <akpm@linux-foundation.org> Cc: Andrea Arcangeli <aarcange@redhat.com>, Minchan Kim <minchan.kim@gmail.com>, Dave Jones <davej@redhat.com>, Jan Kara <jack@suse.cz>, Andy Isaacson <adi@hexapodia.org>, Johannes Weiner <jweiner@redhat.com>, Rik van Riel <riel@redhat.com>, Nai Xia <nai.xia@gmail.com>, Linux-MM <linux-mm@kvack.org>, LKML <linux-kernel@vger.kernel.org> Subject: Re: [PATCH 05/11] mm: compaction: Determine if dirty pages can be migrated without blocking within ->migratepage Date: Mon, 19 Dec 2011 11:05:51 +0000 [thread overview] Message-ID: <20111219110551.GJ3487@suse.de> (raw) In-Reply-To: <20111216152054.f7445e98.akpm@linux-foundation.org> On Fri, Dec 16, 2011 at 03:20:54PM -0800, Andrew Morton wrote: > On Wed, 14 Dec 2011 15:41:27 +0000 > Mel Gorman <mgorman@suse.de> wrote: > > > Asynchronous compaction is used when allocating transparent hugepages > > to avoid blocking for long periods of time. Due to reports of > > stalling, there was a debate on disabling synchronous compaction > > but this severely impacted allocation success rates. Part of the > > reason was that many dirty pages are skipped in asynchronous compaction > > by the following check; > > > > if (PageDirty(page) && !sync && > > mapping->a_ops->migratepage != migrate_page) > > rc = -EBUSY; > > > > This skips over all mapping aops using buffer_migrate_page() > > even though it is possible to migrate some of these pages without > > blocking. This patch updates the ->migratepage callback with a "sync" > > parameter. It is the responsibility of the callback to fail gracefully > > if migration would block. > > > > ... > > > > @@ -259,6 +309,19 @@ static int migrate_page_move_mapping(struct address_space *mapping, > > } > > > > /* > > + * In the async migration case of moving a page with buffers, lock the > > + * buffers using trylock before the mapping is moved. If the mapping > > + * was moved, we later failed to lock the buffers and could not move > > + * the mapping back due to an elevated page count, we would have to > > + * block waiting on other references to be dropped. > > + */ > > + if (!sync && head && !buffer_migrate_lock_buffers(head, sync)) { > > Once it has been established that "sync" is true, I find it clearer to > pass in plain old "true" to buffer_migrate_lock_buffers(). Minor point. > Later in the series, sync changes to "mode" to distinguish between async, sync-light and sync compaction. At that point, this becomes if (mode == MIGRATE_ASYNC && head && !buffer_migrate_lock_buffers(head, mode)) { Passing true in here would be fine, but it would just end up being changed back later in the series so it can be left alone. > I hadn't paid a lot of attention to buffer_migrate_page() before. > Scary function. I'm rather worried about its interactions with ext3 > journal commit which locks buffers then plays with them while leaving > the page unlocked. How vigorously has this been whitebox-tested? > Blackbox testing only AFAIK. This has been tested recently with ext3 and nothing unusual was reported. The list of events for migration looks like isolate page from LRU migrate_pages unmap_and_move lock_page(src_page) if page under writeback, either bail or wait on writeback try_to_unmap move_to_new_page lock_page(dst_page) buffer_migrate_page migrate_page_move_mapping spin_lock_irq(&mapping->tree_lock) lookup in radix tree check reference counts to make sure no one else has references lock buffers if async mode replace page in radix tree with new page spin_unlock_irq lock buffers if !async mode copy buffers unlock buffers unlock_page(dst_page) The critical part is that the copying of buffer data is happening with both page and buffer locks held and no other references to the page exists - it has already been unmapped for example. Journal commit minimally acquires the buffer lock. If migration is in the process of copying the buffers, the buffer lock will prevent journal commit starting at the same time buffers are being copied. block_write_full_page and friends should be taking the buffer lock so they should also be ok. For other accessors, the mapping tree_lock should prevent other users looking up the page in the radix tree in the first place while the radix tree replacement is taking place. Racing against try_to_free_buffer should also be a problem. According to buffer.c, exclusion from try_to_free_buffer "may be obtained by either locking the page or holding the mappings private_lock". Migration is holding the page lock. Taking private_lock would give additional protection but I haven't heard or seen a case where it is necessary. -- Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-12-19 11:05 UTC|newest] Thread overview: 100+ messages / expand[flat|nested] mbox.gz Atom feed top 2011-12-14 15:41 [PATCH 0/11] Reduce compaction-related stalls and improve asynchronous migration of dirty pages v6 Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-14 15:41 ` [PATCH 01/11] mm: compaction: Allow compaction to isolate dirty pages Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-14 15:41 ` [PATCH 02/11] mm: compaction: Use synchronous compaction for /proc/sys/vm/compact_memory Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-14 15:41 ` [PATCH 03/11] mm: vmscan: Check if we isolated a compound page during lumpy scan Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-15 23:21 ` Rik van Riel 2011-12-15 23:21 ` Rik van Riel 2011-12-14 15:41 ` [PATCH 04/11] mm: vmscan: Do not OOM if aborting reclaim to start compaction Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-15 23:36 ` Rik van Riel 2011-12-15 23:36 ` Rik van Riel 2011-12-14 15:41 ` [PATCH 05/11] mm: compaction: Determine if dirty pages can be migrated without blocking within ->migratepage Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 3:32 ` Rik van Riel 2011-12-16 3:32 ` Rik van Riel 2011-12-16 23:20 ` Andrew Morton 2011-12-16 23:20 ` Andrew Morton 2011-12-17 3:03 ` Nai Xia 2011-12-17 3:03 ` Nai Xia 2011-12-17 3:26 ` Andrew Morton 2011-12-17 3:26 ` Andrew Morton 2011-12-19 11:05 ` Mel Gorman [this message] 2011-12-19 11:05 ` Mel Gorman 2011-12-19 13:12 ` nai.xia 2011-12-19 13:12 ` nai.xia 2011-12-14 15:41 ` [PATCH 06/11] mm: compaction: make isolate_lru_page() filter-aware again Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 3:34 ` Rik van Riel 2011-12-16 3:34 ` Rik van Riel 2011-12-18 1:53 ` Minchan Kim 2011-12-18 1:53 ` Minchan Kim 2011-12-14 15:41 ` [PATCH 07/11] mm: page allocator: Do not call direct reclaim for THP allocations while compaction is deferred Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 4:10 ` Rik van Riel 2011-12-16 4:10 ` Rik van Riel 2011-12-14 15:41 ` [PATCH 08/11] mm: compaction: Introduce sync-light migration for use by compaction Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 4:31 ` Rik van Riel 2011-12-16 4:31 ` Rik van Riel 2011-12-18 2:05 ` Minchan Kim 2011-12-18 2:05 ` Minchan Kim 2011-12-19 11:45 ` Mel Gorman 2011-12-19 11:45 ` Mel Gorman 2011-12-20 7:18 ` Minchan Kim 2011-12-20 7:18 ` Minchan Kim 2012-01-13 21:25 ` Andrew Morton 2012-01-13 21:25 ` Andrew Morton 2012-01-16 11:33 ` Mel Gorman 2012-01-16 11:33 ` Mel Gorman 2011-12-14 15:41 ` [PATCH 09/11] mm: vmscan: When reclaiming for compaction, ensure there are sufficient free pages available Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 4:35 ` Rik van Riel 2011-12-16 4:35 ` Rik van Riel 2011-12-14 15:41 ` [PATCH 10/11] mm: vmscan: Check if reclaim should really abort even if compaction_ready() is true for one zone Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 4:38 ` Rik van Riel 2011-12-16 4:38 ` Rik van Riel 2011-12-16 11:29 ` Mel Gorman 2011-12-16 11:29 ` Mel Gorman 2011-12-14 15:41 ` [PATCH 11/11] mm: Isolate pages for immediate reclaim on their own LRU Mel Gorman 2011-12-14 15:41 ` Mel Gorman 2011-12-16 4:47 ` Rik van Riel 2011-12-16 4:47 ` Rik van Riel 2011-12-16 12:26 ` Mel Gorman 2011-12-16 12:26 ` Mel Gorman 2011-12-16 15:17 ` Johannes Weiner 2011-12-16 15:17 ` Johannes Weiner 2011-12-16 16:07 ` Mel Gorman 2011-12-16 16:07 ` Mel Gorman 2011-12-19 16:14 ` Johannes Weiner 2011-12-19 16:14 ` Johannes Weiner 2011-12-17 16:08 ` Minchan Kim 2011-12-17 16:08 ` Minchan Kim 2011-12-19 13:26 ` Mel Gorman 2011-12-19 13:26 ` Mel Gorman 2011-12-20 7:10 ` Minchan Kim 2011-12-20 7:10 ` Minchan Kim 2011-12-20 9:55 ` Mel Gorman 2011-12-20 9:55 ` Mel Gorman 2011-12-23 19:08 ` Hugh Dickins 2011-12-23 19:08 ` Hugh Dickins 2011-12-29 16:59 ` Mel Gorman 2011-12-29 16:59 ` Mel Gorman 2011-12-29 19:31 ` Rik van Riel 2011-12-29 19:31 ` Rik van Riel 2011-12-30 11:27 ` Mel Gorman 2011-12-30 11:27 ` Mel Gorman 2011-12-16 22:56 ` [PATCH 0/11] Reduce compaction-related stalls and improve asynchronous migration of dirty pages v6 Andrew Morton 2011-12-16 22:56 ` Andrew Morton 2011-12-19 14:40 ` Mel Gorman 2011-12-19 14:40 ` Mel Gorman 2011-12-16 23:37 ` Andrew Morton 2011-12-16 23:37 ` Andrew Morton 2011-12-19 14:20 ` Mel Gorman 2011-12-19 14:20 ` Mel Gorman -- strict thread matches above, loose matches on Subject: below -- 2011-12-01 17:36 [PATCH 0/11] Reduce compaction-related stalls and improve asynchronous migration of dirty pages v5 Mel Gorman 2011-12-01 17:36 ` [PATCH 05/11] mm: compaction: Determine if dirty pages can be migrated without blocking within ->migratepage Mel Gorman 2011-12-01 17:36 ` Mel Gorman
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20111219110551.GJ3487@suse.de \ --to=mgorman@suse.de \ --cc=aarcange@redhat.com \ --cc=adi@hexapodia.org \ --cc=akpm@linux-foundation.org \ --cc=davej@redhat.com \ --cc=jack@suse.cz \ --cc=jweiner@redhat.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=minchan.kim@gmail.com \ --cc=nai.xia@gmail.com \ --cc=riel@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.