From: Kemi Wang <kemi.wang@intel.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Andrew Morton <akpm@linux-foundation.org>,
Michal Hocko <mhocko@suse.com>, Vlastimil Babka <vbabka@suse.cz>,
Mel Gorman <mgorman@techsingularity.net>,
Johannes Weiner <hannes@cmpxchg.org>,
Christopher Lameter <cl@linux.com>,
YASUAKI ISHIMATSU <yasu.isimatu@gmail.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Nikolay Borisov <nborisov@suse.com>,
Pavel Tatashin <pasha.tatashin@oracle.com>,
David Rientjes <rientjes@google.com>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Dave <dave.hansen@linux.intel.com>,
Andi Kleen <andi.kleen@intel.com>,
Tim Chen <tim.c.chen@intel.com>,
Jesper Dangaard Brouer <brouer@redhat.com>,
Ying Huang <ying.huang@intel.com>, Aaron Lu <aaron.lu@intel.com>,
Aubrey Li <aubrey.li@intel.com>, Kemi Wang <kemi.wang@intel.com>,
Linux MM <linux-mm@kvack.org>,
Linux Kernel <linux-kernel@vger.kernel.org>
Subject: [PATCH v2 3/5] mm: enlarge NUMA counters threshold size
Date: Tue, 19 Dec 2017 14:39:24 +0800 [thread overview]
Message-ID: <1513665566-4465-4-git-send-email-kemi.wang@intel.com> (raw)
In-Reply-To: <1513665566-4465-1-git-send-email-kemi.wang@intel.com>
We have seen significant overhead in cache bouncing caused by NUMA counters
update in multi-threaded page allocation. See 'commit 1d90ca897cb0 ("mm:
update NUMA counter threshold size")' for more details.
This patch updates NUMA counters to a fixed size of (MAX_S16 - 2) and deals
with global counter update using different threshold size for node page
stats.
Signed-off-by: Kemi Wang <kemi.wang@intel.com>
---
mm/vmstat.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 9c681cc..64e08ae 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -30,6 +30,8 @@
#include "internal.h"
+#define VM_NUMA_STAT_THRESHOLD (S16_MAX - 2)
+
#ifdef CONFIG_NUMA
int sysctl_vm_numa_stat = ENABLE_NUMA_STAT;
@@ -394,7 +396,11 @@ void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item item)
s16 v, t;
v = __this_cpu_inc_return(*p);
- t = __this_cpu_read(pcp->stat_threshold);
+ if (item >= NR_VM_NUMA_STAT_ITEMS)
+ t = __this_cpu_read(pcp->stat_threshold);
+ else
+ t = VM_NUMA_STAT_THRESHOLD;
+
if (unlikely(v > t)) {
s16 overstep = t >> 1;
@@ -549,7 +555,10 @@ static inline void mod_node_state(struct pglist_data *pgdat,
* Most of the time the thresholds are the same anyways
* for all cpus in a node.
*/
- t = this_cpu_read(pcp->stat_threshold);
+ if (item >= NR_VM_NUMA_STAT_ITEMS)
+ t = this_cpu_read(pcp->stat_threshold);
+ else
+ t = VM_NUMA_STAT_THRESHOLD;
o = this_cpu_read(*p);
n = delta + o;
--
2.7.4
next prev parent reply other threads:[~2017-12-19 6:41 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-19 6:39 [PATCH v2 0/5] mm: NUMA stats code cleanup and enhancement Kemi Wang
2017-12-19 6:39 ` [PATCH v2 1/5] mm: migrate NUMA stats from per-zone to per-node Kemi Wang
2017-12-19 12:28 ` Michal Hocko
2017-12-20 5:32 ` kemi
2017-12-19 6:39 ` [PATCH v2 2/5] mm: Extends local cpu counter vm_diff_nodestat from s8 to s16 Kemi Wang
2017-12-19 12:38 ` Michal Hocko
2017-12-20 3:05 ` kemi
2017-12-19 16:05 ` Christopher Lameter
2017-12-19 16:20 ` Michal Hocko
2017-12-19 17:21 ` Christopher Lameter
2017-12-20 6:45 ` kemi
2017-12-19 6:39 ` Kemi Wang [this message]
2017-12-19 12:40 ` [PATCH v2 3/5] mm: enlarge NUMA counters threshold size Michal Hocko
2017-12-20 5:52 ` kemi
2017-12-20 10:12 ` Michal Hocko
2017-12-20 10:21 ` kemi
2017-12-21 8:06 ` kemi
2017-12-21 8:17 ` Michal Hocko
2017-12-21 8:23 ` kemi
2017-12-21 8:59 ` Michal Hocko
2017-12-21 10:31 ` kemi
2017-12-22 12:31 ` Michal Hocko
2017-12-21 17:10 ` Christopher Lameter
2017-12-22 2:06 ` kemi
2017-12-26 19:05 ` Christopher Lameter
2017-12-19 6:39 ` [PATCH v2 4/5] mm: use node_page_state_snapshot to avoid deviation Kemi Wang
2017-12-19 12:43 ` Michal Hocko
2017-12-20 6:07 ` kemi
2017-12-20 10:06 ` Michal Hocko
2017-12-20 10:24 ` kemi
2017-12-20 15:58 ` Christopher Lameter
2017-12-21 1:39 ` kemi
2017-12-19 6:39 ` [PATCH v2 5/5] mm: Rename zone_statistics() to numa_statistics() Kemi Wang
2017-12-19 12:44 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1513665566-4465-4-git-send-email-kemi.wang@intel.com \
--to=kemi.wang@intel.com \
--cc=aaron.lu@intel.com \
--cc=akpm@linux-foundation.org \
--cc=andi.kleen@intel.com \
--cc=aryabinin@virtuozzo.com \
--cc=aubrey.li@intel.com \
--cc=bigeasy@linutronix.de \
--cc=brouer@redhat.com \
--cc=cl@linux.com \
--cc=dave.hansen@linux.intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@techsingularity.net \
--cc=mhocko@suse.com \
--cc=nborisov@suse.com \
--cc=pasha.tatashin@oracle.com \
--cc=rientjes@google.com \
--cc=tim.c.chen@intel.com \
--cc=vbabka@suse.cz \
--cc=yasu.isimatu@gmail.com \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).