From: Brice Goglin <Brice.Goglin@inria.fr> To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Cc: Christoph Lameter <cl@linux-foundation.org>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org> Subject: Re: [PATCH] migration: only migrate_prep() once per move_pages() Date: Wed, 15 Apr 2009 11:33:16 +0200 [thread overview] Message-ID: <49E5A9DC.2050309@inria.fr> (raw) In-Reply-To: <20090415164955.41746866.kamezawa.hiroyu@jp.fujitsu.com> KAMEZAWA Hiroyuki wrote: > On Wed, 15 Apr 2009 09:32:10 +0200 > Brice Goglin <Brice.Goglin@ens-lyon.org> wrote: > > >> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz). >> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages() >> throughput by breaking it into chunks, but it also made migrate_prep() >> be called once per chunk (every 128pages or so) instead of once per >> move_pages(). >> >> This patch reverts to calling migrate_prep() only once per chunk >> as we did before 2.6.29. >> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d >> mm: move migrate_prep out from under mmap_sem >> >> This improves migration throughput on the above machine from 600MB/s >> to 750MB/s. >> >> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr> >> >> > Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > I think this patch is good. page migration is best-effort syscall ;) > My next feeling now is about improving migrate_prep() itself. It makes the move_pages() startup overhead very high. But lru_add_drain_all() touches some code that I am far from understanding :/ Can we imagine using IPI instead of a deferred work_struct for this kind of things? Or maybe, for each processor, check whether drain_cpu_pagevecs() would have something to do before actually scheduling the local work_struct? It's racy, but migrate_prep() doesn't guarantee anyway that pages won't be moved out of the LRU before the actual migration, so... Also I don't see why the cost of lru_add_drain_all() seems to increase linearly with the number of cores in the machine. There may be some lock contention, but it should scale better when there's pretty-much nothing in the CPU lists... > BTW, current users of sys_move_pages() does retry when it gets -EBUSY ? > I'd say they ignore it since it doesn't happen often :) Brice
WARNING: multiple messages have this Message-ID (diff)
From: Brice Goglin <Brice.Goglin@inria.fr> To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> Cc: Christoph Lameter <cl@linux-foundation.org>, linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org> Subject: Re: [PATCH] migration: only migrate_prep() once per move_pages() Date: Wed, 15 Apr 2009 11:33:16 +0200 [thread overview] Message-ID: <49E5A9DC.2050309@inria.fr> (raw) In-Reply-To: <20090415164955.41746866.kamezawa.hiroyu@jp.fujitsu.com> KAMEZAWA Hiroyuki wrote: > On Wed, 15 Apr 2009 09:32:10 +0200 > Brice Goglin <Brice.Goglin@ens-lyon.org> wrote: > > >> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz). >> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages() >> throughput by breaking it into chunks, but it also made migrate_prep() >> be called once per chunk (every 128pages or so) instead of once per >> move_pages(). >> >> This patch reverts to calling migrate_prep() only once per chunk >> as we did before 2.6.29. >> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d >> mm: move migrate_prep out from under mmap_sem >> >> This improves migration throughput on the above machine from 600MB/s >> to 750MB/s. >> >> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr> >> >> > Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > I think this patch is good. page migration is best-effort syscall ;) > My next feeling now is about improving migrate_prep() itself. It makes the move_pages() startup overhead very high. But lru_add_drain_all() touches some code that I am far from understanding :/ Can we imagine using IPI instead of a deferred work_struct for this kind of things? Or maybe, for each processor, check whether drain_cpu_pagevecs() would have something to do before actually scheduling the local work_struct? It's racy, but migrate_prep() doesn't guarantee anyway that pages won't be moved out of the LRU before the actual migration, so... Also I don't see why the cost of lru_add_drain_all() seems to increase linearly with the number of cores in the machine. There may be some lock contention, but it should scale better when there's pretty-much nothing in the CPU lists... > BTW, current users of sys_move_pages() does retry when it gets -EBUSY ? > I'd say they ignore it since it doesn't happen often :) Brice -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2009-04-15 9:33 UTC|newest] Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top 2009-04-15 7:32 [PATCH] migration: only migrate_prep() once per move_pages() Brice Goglin 2009-04-15 7:32 ` Brice Goglin 2009-04-15 7:49 ` KAMEZAWA Hiroyuki 2009-04-15 7:49 ` KAMEZAWA Hiroyuki 2009-04-15 9:33 ` Brice Goglin [this message] 2009-04-15 9:33 ` Brice Goglin 2009-04-15 12:41 ` Christoph Lameter 2009-04-15 12:41 ` Christoph Lameter 2009-05-13 5:52 ` Brice Goglin 2009-05-13 5:52 ` Brice Goglin 2009-05-14 20:11 ` Christoph Lameter 2009-05-14 20:11 ` Christoph Lameter
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=49E5A9DC.2050309@inria.fr \ --to=brice.goglin@inria.fr \ --cc=cl@linux-foundation.org \ --cc=kamezawa.hiroyu@jp.fujitsu.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.