From: Vlastimil Babka <vbabka@suse.cz>
To: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org,
Andrew Morton <akpm@linux-foundation.org>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Minchan Kim <minchan@kernel.org>,
Sasha Levin <sasha.levin@oracle.com>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Mel Gorman <mgorman@suse.de>, Michal Hocko <mhocko@suse.cz>,
Vlastimil Babka <vbabka@suse.cz>
Subject: [PATCH v2 3/9] mm, page_owner: convert page_owner_inited to static key
Date: Tue, 24 Nov 2015 13:36:15 +0100 [thread overview]
Message-ID: <1448368581-6923-4-git-send-email-vbabka@suse.cz> (raw)
In-Reply-To: <1448368581-6923-1-git-send-email-vbabka@suse.cz>
CONFIG_PAGE_OWNER attempts to impose negligible runtime overhead when enabled
during compilation, but not actually enabled during runtime by boot param
page_owner=on. This overhead can be further reduced using the static key
mechanism, which this patch does.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
Documentation/vm/page_owner.txt | 9 +++++----
include/linux/page_owner.h | 22 ++++++++++------------
mm/page_owner.c | 9 +++++----
mm/vmstat.c | 2 +-
4 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/Documentation/vm/page_owner.txt b/Documentation/vm/page_owner.txt
index 8f3ce9b..ffff143 100644
--- a/Documentation/vm/page_owner.txt
+++ b/Documentation/vm/page_owner.txt
@@ -28,10 +28,11 @@ with page owner and page owner is disabled in runtime due to no enabling
boot option, runtime overhead is marginal. If disabled in runtime, it
doesn't require memory to store owner information, so there is no runtime
memory overhead. And, page owner inserts just two unlikely branches into
-the page allocator hotpath and if it returns false then allocation is
-done like as the kernel without page owner. These two unlikely branches
-would not affect to allocation performance. Following is the kernel's
-code size change due to this facility.
+the page allocator hotpath and if not enabled, then allocation is done
+like as the kernel without page owner. These two unlikely branches should
+not affect to allocation performance, especially if the static keys jump
+label patching functionality is available. Following is the kernel's code
+size change due to this facility.
- Without page owner
text data bss dec hex filename
diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
index cacaabe..8e2eb15 100644
--- a/include/linux/page_owner.h
+++ b/include/linux/page_owner.h
@@ -1,8 +1,10 @@
#ifndef __LINUX_PAGE_OWNER_H
#define __LINUX_PAGE_OWNER_H
+#include <linux/jump_label.h>
+
#ifdef CONFIG_PAGE_OWNER
-extern bool page_owner_inited;
+extern struct static_key_false page_owner_inited;
extern struct page_ext_operations page_owner_ops;
extern void __reset_page_owner(struct page *page, unsigned int order);
@@ -12,27 +14,23 @@ extern gfp_t __get_page_owner_gfp(struct page *page);
static inline void reset_page_owner(struct page *page, unsigned int order)
{
- if (likely(!page_owner_inited))
- return;
-
- __reset_page_owner(page, order);
+ if (static_branch_unlikely(&page_owner_inited))
+ __reset_page_owner(page, order);
}
static inline void set_page_owner(struct page *page,
unsigned int order, gfp_t gfp_mask)
{
- if (likely(!page_owner_inited))
- return;
-
- __set_page_owner(page, order, gfp_mask);
+ if (static_branch_unlikely(&page_owner_inited))
+ __set_page_owner(page, order, gfp_mask);
}
static inline gfp_t get_page_owner_gfp(struct page *page)
{
- if (likely(!page_owner_inited))
+ if (static_branch_unlikely(&page_owner_inited))
+ return __get_page_owner_gfp(page);
+ else
return 0;
-
- return __get_page_owner_gfp(page);
}
#else
static inline void reset_page_owner(struct page *page, unsigned int order)
diff --git a/mm/page_owner.c b/mm/page_owner.c
index f35826e..10a6a46 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -5,10 +5,11 @@
#include <linux/bootmem.h>
#include <linux/stacktrace.h>
#include <linux/page_owner.h>
+#include <linux/jump_label.h>
#include "internal.h"
static bool page_owner_disabled = true;
-bool page_owner_inited __read_mostly;
+DEFINE_STATIC_KEY_FALSE(page_owner_inited);
static void init_early_allocated_pages(void);
@@ -37,7 +38,7 @@ static void init_page_owner(void)
if (page_owner_disabled)
return;
- page_owner_inited = true;
+ static_branch_enable(&page_owner_inited);
init_early_allocated_pages();
}
@@ -157,7 +158,7 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)
struct page *page;
struct page_ext *page_ext;
- if (!page_owner_inited)
+ if (!static_branch_unlikely(&page_owner_inited))
return -EINVAL;
page = NULL;
@@ -305,7 +306,7 @@ static int __init pageowner_init(void)
{
struct dentry *dentry;
- if (!page_owner_inited) {
+ if (!static_branch_unlikely(&page_owner_inited)) {
pr_info("page_owner is disabled\n");
return 0;
}
diff --git a/mm/vmstat.c b/mm/vmstat.c
index 53b722b..3f7ec14 100644
--- a/mm/vmstat.c
+++ b/mm/vmstat.c
@@ -1117,7 +1117,7 @@ static void pagetypeinfo_showmixedcount(struct seq_file *m, pg_data_t *pgdat)
#ifdef CONFIG_PAGE_OWNER
int mtype;
- if (!page_owner_inited)
+ if (!static_branch_unlikely(&page_owner_inited))
return;
drain_all_pages(NULL);
--
2.6.3
next prev parent reply other threads:[~2015-11-24 12:36 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-24 12:36 [PATCH v2 0/9] page_owner improvements for debugging Vlastimil Babka
2015-11-24 12:36 ` [PATCH v2 1/9] mm, debug: fix wrongly filtered flags in dump_vma() Vlastimil Babka
2015-11-27 9:52 ` Vlastimil Babka
2015-11-24 12:36 ` [PATCH v2 2/9] mm, page_owner: print symbolic migratetype of both page and pageblock Vlastimil Babka
2015-11-25 8:11 ` Joonsoo Kim
2015-11-24 12:36 ` Vlastimil Babka [this message]
2015-11-25 14:52 ` [PATCH v2 3/9] mm, page_owner: convert page_owner_inited to static key Michal Hocko
2015-11-25 15:08 ` Vlastimil Babka
2015-11-25 15:25 ` Peter Zijlstra
2015-11-25 15:46 ` Michal Hocko
2015-11-24 12:36 ` [PATCH v2 4/9] mm, page_owner: copy page owner info during migration Vlastimil Babka
2015-11-24 12:36 ` [PATCH v2 5/9] mm, page_owner: track and print last migrate reason Vlastimil Babka
2015-11-25 8:13 ` Joonsoo Kim
2015-11-26 10:39 ` Vlastimil Babka
2015-11-24 12:36 ` [PATCH v2 6/9] mm, debug: introduce dump_gfpflag_names() for symbolic printing of gfp_flags Vlastimil Babka
2015-11-25 8:16 ` Joonsoo Kim
2015-11-25 10:28 ` Vlastimil Babka
2015-11-27 3:40 ` yalin wang
2015-11-24 12:36 ` [PATCH v2 7/9] mm, page_owner: dump page owner info from dump_page() Vlastimil Babka
2015-11-25 14:58 ` Michal Hocko
2015-11-26 10:43 ` Vlastimil Babka
2015-11-24 12:36 ` [PATCH v2 8/9] mm, page_alloc: print symbolic gfp_flags on allocation failure Vlastimil Babka
2015-11-25 14:33 ` Michal Hocko
2015-11-24 12:36 ` [PATCH v2 9/9] mm, oom: print symbolic gfp_flags in oom warning Vlastimil Babka
2015-11-25 14:31 ` Michal Hocko
2016-01-07 21:29 ` David Rientjes
2016-01-08 11:34 ` Vlastimil Babka
2015-11-25 14:30 ` [PATCH v2 0/9] page_owner improvements for debugging Michal Hocko
2015-11-30 16:10 ` [PATCH 1/2] mm, printk: introduce new format string for flags Vlastimil Babka
2015-11-30 16:10 ` [PATCH 2/2] mm, page_owner: provide symbolic page flags and gfp_flags Vlastimil Babka
2015-12-02 11:01 ` [PATCH 1/2] mm, printk: introduce new format string for flags Rasmus Villemoes
2015-12-02 20:34 ` Vlastimil Babka
2015-12-03 12:37 ` Rasmus Villemoes
2015-12-03 13:46 ` Vlastimil Babka
2015-12-04 15:16 ` [PATCH v2 1/3] " Vlastimil Babka
2015-12-04 15:16 ` [PATCH v2 2/3] mm, page_owner: provide symbolic page flags and gfp_flags Vlastimil Babka
2015-12-04 15:16 ` [PATCH v2 3/3] mm, debug: move bad flags printing to bad_page() Vlastimil Babka
2015-12-05 20:00 ` [PATCH v2 1/3] mm, printk: introduce new format string for flags Rasmus Villemoes
2015-12-09 11:29 ` Arnd Bergmann
2015-12-09 20:48 ` Vlastimil Babka
2015-12-10 12:26 ` James Hogan
2015-12-10 2:59 ` Joonsoo Kim
2015-12-10 4:04 ` Steven Rostedt
2015-12-10 4:12 ` Joonsoo Kim
2015-12-10 8:41 ` Rasmus Villemoes
2015-12-10 10:03 ` Vlastimil Babka
2015-12-14 3:03 ` Joonsoo Kim
2015-12-10 3:51 ` Steven Rostedt
2015-12-10 9:51 ` Vlastimil Babka
2015-12-02 17:40 ` [PATCH 1/2] " yalin wang
2015-12-02 21:04 ` Vlastimil Babka
2015-12-03 0:11 ` yalin wang
2015-12-03 8:03 ` Rasmus Villemoes
2015-12-03 18:38 ` yalin wang
2015-12-04 1:04 ` yalin wang
2015-12-04 14:15 ` Vlastimil Babka
2015-12-10 4:03 ` Steven Rostedt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1448368581-6923-4-git-send-email-vbabka@suse.cz \
--to=vbabka@suse.cz \
--cc=akpm@linux-foundation.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.cz \
--cc=minchan@kernel.org \
--cc=sasha.levin@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).