From: tip-bot for Mel Gorman <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: torvalds@linux-foundation.org, srikar@linux.vnet.ibm.com,
mgorman@techsingularity.net, riel@surriel.com,
linux-mm@kvack.org, linux-kernel@vger.kernel.org, hpa@zytor.com,
a.p.zijlstra@chello.nl, mingo@kernel.org, tglx@linutronix.de,
jhladky@redhat.com
Subject: [tip:sched/urgent] sched/numa: Migrate pages to local nodes quicker early in the lifetime of a task
Date: Tue, 2 Oct 2018 03:17:34 -0700 [thread overview]
Message-ID: <tip-37355bdc5a129899f6b245900a8eb944a092f7fd@git.kernel.org> (raw)
In-Reply-To: <20181001100525.29789-3-mgorman@techsingularity.net>
Commit-ID: 37355bdc5a129899f6b245900a8eb944a092f7fd
Gitweb: https://git.kernel.org/tip/37355bdc5a129899f6b245900a8eb944a092f7fd
Author: Mel Gorman <mgorman@techsingularity.net>
AuthorDate: Mon, 1 Oct 2018 11:05:25 +0100
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 2 Oct 2018 11:31:33 +0200
sched/numa: Migrate pages to local nodes quicker early in the lifetime of a task
Automatic NUMA Balancing uses a multi-stage pass to decide whether a page
should migrate to a local node. This filter avoids excessive ping-ponging
if a page is shared or used by threads that migrate cross-node frequently.
Threads inherit both page tables and the preferred node ID from the
parent. This means that threads can trigger hinting faults earlier than
a new task which delays scanning for a number of seconds. As it can be
load balanced very early in its lifetime there can be an unnecessary delay
before it starts migrating thread-local data. This patch migrates private
pages faster early in the lifetime of a thread using the sequence counter
as an identifier of new tasks.
With this patch applied, STREAM performance is the same as 4.17 even though
processes are not spread cross-node prematurely. Other workloads showed
a mix of minor gains and losses. This is somewhat expected most workloads
are not very sensitive to the starting conditions of a process.
4.19.0-rc5 4.19.0-rc5 4.17.0
numab-v1r1 fastmigrate-v1r1 vanilla
MB/sec copy 43298.52 ( 0.00%) 47335.46 ( 9.32%) 47219.24 ( 9.06%)
MB/sec scale 30115.06 ( 0.00%) 32568.12 ( 8.15%) 32527.56 ( 8.01%)
MB/sec add 32825.12 ( 0.00%) 36078.94 ( 9.91%) 35928.02 ( 9.45%)
MB/sec triad 32549.52 ( 0.00%) 35935.94 ( 10.40%) 35969.88 ( 10.51%)
Signed-off-by: Mel Gorman <mgorman@techsingularity.net>
Reviewed-by: Rik van Riel <riel@surriel.com>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Jirka Hladky <jhladky@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Linux-MM <linux-mm@kvack.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20181001100525.29789-3-mgorman@techsingularity.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
kernel/sched/fair.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 25c7c7e09cbd..7fc4a371bdd2 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1392,6 +1392,17 @@ bool should_numa_migrate_memory(struct task_struct *p, struct page * page,
int last_cpupid, this_cpupid;
this_cpupid = cpu_pid_to_cpupid(dst_cpu, current->pid);
+ last_cpupid = page_cpupid_xchg_last(page, this_cpupid);
+
+ /*
+ * Allow first faults or private faults to migrate immediately early in
+ * the lifetime of a task. The magic number 4 is based on waiting for
+ * two full passes of the "multi-stage node selection" test that is
+ * executed below.
+ */
+ if ((p->numa_preferred_nid == -1 || p->numa_scan_seq <= 4) &&
+ (cpupid_pid_unset(last_cpupid) || cpupid_match_pid(p, last_cpupid)))
+ return true;
/*
* Multi-stage node selection is used in conjunction with a periodic
@@ -1410,7 +1421,6 @@ bool should_numa_migrate_memory(struct task_struct *p, struct page * page,
* This quadric squishes small probabilities, making it less likely we
* act on an unlikely task<->page relation.
*/
- last_cpupid = page_cpupid_xchg_last(page, this_cpupid);
if (!cpupid_pid_unset(last_cpupid) &&
cpupid_to_nid(last_cpupid) != dst_nid)
return false;
next prev parent reply other threads:[~2018-10-02 10:17 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-10-01 10:05 [PATCH 0/2] Faster migration for automatic NUMA balancing Mel Gorman
2018-10-01 10:05 ` [PATCH 1/2] mm, numa: Remove rate-limiting of automatic numa balancing migration Mel Gorman
2018-10-01 15:39 ` Rik van Riel
2018-10-02 10:17 ` [tip:sched/urgent] mm, sched/numa: Remove rate-limiting of automatic NUMA " tip-bot for Mel Gorman
2018-10-02 11:54 ` [PATCH 1/2] mm, numa: Remove rate-limiting of automatic numa " Srikar Dronamraju
2018-10-01 10:05 ` [PATCH 2/2] mm, numa: Migrate pages to local nodes quicker early in the lifetime of a task Mel Gorman
2018-10-01 15:41 ` Rik van Riel
2018-10-02 10:17 ` tip-bot for Mel Gorman [this message]
2018-10-02 12:41 ` Srikar Dronamraju
2018-10-02 13:54 ` Mel Gorman
2018-10-02 17:30 ` Srikar Dronamraju
2018-10-02 18:22 ` Mel Gorman
2018-10-03 13:15 ` Srikar Dronamraju
2018-10-03 13:07 ` Srikar Dronamraju
2018-10-03 13:21 ` Mel Gorman
2018-10-03 14:08 ` Srikar Dronamraju
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=tip-37355bdc5a129899f6b245900a8eb944a092f7fd@git.kernel.org \
--to=tipbot@zytor.com \
--cc=a.p.zijlstra@chello.nl \
--cc=hpa@zytor.com \
--cc=jhladky@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@kernel.org \
--cc=riel@surriel.com \
--cc=srikar@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).