From: Vlastimil Babka <vbabka@suse.cz> To: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org>, Andrew Morton <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Mel Gorman <mgorman@techsingularity.net>, David Rientjes <rientjes@google.com>, kernel-team@fb.com, kernel-team@lge.com Subject: Re: [PATCH v3 7/8] mm, compaction: restrict async compaction to pageblocks of same migratetype Date: Thu, 4 May 2017 08:12:56 +0200 [thread overview] Message-ID: <45323114-97d7-f34a-8336-51efff26bc8b@suse.cz> (raw) In-Reply-To: <20170407003851.GA17231@js1304-P5Q-DELUXE> On 04/07/2017 02:38 AM, Joonsoo Kim wrote: > On Wed, Mar 29, 2017 at 06:06:41PM +0200, Vlastimil Babka wrote: >> On 03/16/2017 03:14 AM, Joonsoo Kim wrote: >>> On Tue, Mar 07, 2017 at 02:15:44PM +0100, Vlastimil Babka wrote: >>>> The migrate scanner in async compaction is currently limited to MIGRATE_MOVABLE >>>> pageblocks. This is a heuristic intended to reduce latency, based on the >>>> assumption that non-MOVABLE pageblocks are unlikely to contain movable pages. >>>> >>>> However, with the exception of THP's, most high-order allocations are not >>>> movable. Should the async compaction succeed, this increases the chance that >>>> the non-MOVABLE allocations will fallback to a MOVABLE pageblock, making the >>>> long-term fragmentation worse. >>> >>> I agree with this idea but have some concerns on this change. >>> >>> *ASYNC* compaction is designed for reducing latency and this change >>> doesn't fit it. If everything works fine, there is a few movable pages >>> in non-MOVABLE pageblocks as you noted above. Moreover, there is quite >>> less the number of non-MOVABLE pageblock than MOVABLE one so finding >>> non-MOVABLE pageblock takes long time. These two factors will increase >>> the latency of *ASYNC* compaction. >> >> Right. I lately started to doubt the whole idea of async compaction (for >> non-movable allocations). Seems it's one of the compaction heuristics tuned >> towards the THP usecase. But for non-movable allocations, we just can't have >> both the low latency and long-term fragmentation avoidance. I see now even my >> own skip_on_failure mode in isolate_migratepages_block() as a mistake for >> non-movable allocations. > > Why do you think that skip_on_failure mode is a mistake? I think that > it would lead to reduce the latency and it fits the goal of async > compaction. Yes, but the downside is that compaction will create just the single high-order page that is requested, while previously it would also migrate away some more lower-order pages. When compacting for MIGRATE_UNMOVABLE allocation, we then can't steal extra pages, so next allocation might pollute a different pageblock. It's not a good tradeoff. >> >> Ideally I'd like to make async compaction redundant by kcompactd, and direct >> compaction would mean a serious situation which should warrant sync compaction. >> Meanwhile I see several options to modify this patch >> - async compaction for non-movable allocations will stop doing the >> skip_on_failure mode, and won't restrict the pageblock at all. patch 8/8 will >> make sure that also this kind of compaction finishes the whole pageblock >> - non-movable allocations will skip async compaction completely and go for sync >> compaction immediately > > IMO, concept of async compaction is also important for non-movable allocation. > Non-movable allocation is essential for some workload and they hope > the low latency. The low latency should not be at the expense of making long-term fragmentation worse. > Thanks. >
WARNING: multiple messages have this Message-ID (diff)
From: Vlastimil Babka <vbabka@suse.cz> To: Joonsoo Kim <iamjoonsoo.kim@lge.com> Cc: Johannes Weiner <hannes@cmpxchg.org>, Andrew Morton <akpm@linux-foundation.org>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Mel Gorman <mgorman@techsingularity.net>, David Rientjes <rientjes@google.com>, kernel-team@fb.com, kernel-team@lge.com Subject: Re: [PATCH v3 7/8] mm, compaction: restrict async compaction to pageblocks of same migratetype Date: Thu, 4 May 2017 08:12:56 +0200 [thread overview] Message-ID: <45323114-97d7-f34a-8336-51efff26bc8b@suse.cz> (raw) In-Reply-To: <20170407003851.GA17231@js1304-P5Q-DELUXE> On 04/07/2017 02:38 AM, Joonsoo Kim wrote: > On Wed, Mar 29, 2017 at 06:06:41PM +0200, Vlastimil Babka wrote: >> On 03/16/2017 03:14 AM, Joonsoo Kim wrote: >>> On Tue, Mar 07, 2017 at 02:15:44PM +0100, Vlastimil Babka wrote: >>>> The migrate scanner in async compaction is currently limited to MIGRATE_MOVABLE >>>> pageblocks. This is a heuristic intended to reduce latency, based on the >>>> assumption that non-MOVABLE pageblocks are unlikely to contain movable pages. >>>> >>>> However, with the exception of THP's, most high-order allocations are not >>>> movable. Should the async compaction succeed, this increases the chance that >>>> the non-MOVABLE allocations will fallback to a MOVABLE pageblock, making the >>>> long-term fragmentation worse. >>> >>> I agree with this idea but have some concerns on this change. >>> >>> *ASYNC* compaction is designed for reducing latency and this change >>> doesn't fit it. If everything works fine, there is a few movable pages >>> in non-MOVABLE pageblocks as you noted above. Moreover, there is quite >>> less the number of non-MOVABLE pageblock than MOVABLE one so finding >>> non-MOVABLE pageblock takes long time. These two factors will increase >>> the latency of *ASYNC* compaction. >> >> Right. I lately started to doubt the whole idea of async compaction (for >> non-movable allocations). Seems it's one of the compaction heuristics tuned >> towards the THP usecase. But for non-movable allocations, we just can't have >> both the low latency and long-term fragmentation avoidance. I see now even my >> own skip_on_failure mode in isolate_migratepages_block() as a mistake for >> non-movable allocations. > > Why do you think that skip_on_failure mode is a mistake? I think that > it would lead to reduce the latency and it fits the goal of async > compaction. Yes, but the downside is that compaction will create just the single high-order page that is requested, while previously it would also migrate away some more lower-order pages. When compacting for MIGRATE_UNMOVABLE allocation, we then can't steal extra pages, so next allocation might pollute a different pageblock. It's not a good tradeoff. >> >> Ideally I'd like to make async compaction redundant by kcompactd, and direct >> compaction would mean a serious situation which should warrant sync compaction. >> Meanwhile I see several options to modify this patch >> - async compaction for non-movable allocations will stop doing the >> skip_on_failure mode, and won't restrict the pageblock at all. patch 8/8 will >> make sure that also this kind of compaction finishes the whole pageblock >> - non-movable allocations will skip async compaction completely and go for sync >> compaction immediately > > IMO, concept of async compaction is also important for non-movable allocation. > Non-movable allocation is essential for some workload and they hope > the low latency. The low latency should not be at the expense of making long-term fragmentation worse. > Thanks. > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-05-04 6:13 UTC|newest] Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-03-07 13:15 [PATCH v3 0/8] try to reduce fragmenting fallbacks Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 1/8] mm, compaction: reorder fields in struct compact_control Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 2/8] mm, compaction: remove redundant watermark check in compact_finished() Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-16 1:30 ` Joonsoo Kim 2017-03-16 1:30 ` Joonsoo Kim 2017-03-29 15:30 ` Vlastimil Babka 2017-03-29 15:30 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 3/8] mm, page_alloc: split smallest stolen page in fallback Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 4/8] mm, page_alloc: count movable pages when stealing from pageblock Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-16 1:53 ` Joonsoo Kim 2017-03-16 1:53 ` Joonsoo Kim 2017-03-29 15:49 ` Vlastimil Babka 2017-03-29 15:49 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 5/8] mm, compaction: change migrate_async_suitable() to suitable_migration_source() Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 6/8] mm, compaction: add migratetype to compact_control Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 7/8] mm, compaction: restrict async compaction to pageblocks of same migratetype Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-16 2:14 ` Joonsoo Kim 2017-03-16 2:14 ` Joonsoo Kim 2017-03-29 16:06 ` Vlastimil Babka 2017-03-29 16:06 ` Vlastimil Babka 2017-04-07 0:38 ` Joonsoo Kim 2017-04-07 0:38 ` Joonsoo Kim 2017-05-04 6:12 ` Vlastimil Babka [this message] 2017-05-04 6:12 ` Vlastimil Babka 2017-03-07 13:15 ` [PATCH v3 8/8] mm, compaction: finish whole pageblock to reduce fragmentation Vlastimil Babka 2017-03-07 13:15 ` Vlastimil Babka 2017-03-16 2:18 ` Joonsoo Kim 2017-03-16 2:18 ` Joonsoo Kim 2017-03-29 16:13 ` Vlastimil Babka 2017-03-29 16:13 ` Vlastimil Babka 2017-03-08 16:46 ` [PATCH v3 0/8] try to reduce fragmenting fallbacks Johannes Weiner 2017-03-08 16:46 ` Johannes Weiner 2017-03-08 19:17 ` Vlastimil Babka 2017-03-08 19:17 ` Vlastimil Babka 2017-03-16 18:34 ` Johannes Weiner 2017-03-16 18:34 ` Johannes Weiner 2017-03-17 18:29 ` Vlastimil Babka 2017-03-17 18:29 ` Vlastimil Babka 2017-03-19 21:23 ` Johannes Weiner 2017-03-19 21:23 ` Johannes Weiner
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=45323114-97d7-f34a-8336-51efff26bc8b@suse.cz \ --to=vbabka@suse.cz \ --cc=akpm@linux-foundation.org \ --cc=hannes@cmpxchg.org \ --cc=iamjoonsoo.kim@lge.com \ --cc=kernel-team@fb.com \ --cc=kernel-team@lge.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mgorman@techsingularity.net \ --cc=rientjes@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.