From: Juerg Haefliger <juerg.haefliger@hpe.com>
To: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
kernel-hardening@lists.openwall.com,
linux-x86_64@vger.kernel.org
Cc: vpk@cs.columbia.edu, juerg.haefliger@hpe.com
Subject: [RFC PATCH v3 2/2] xpfo: Only put previous userspace pages into the hot cache
Date: Fri, 4 Nov 2016 15:45:34 +0100 [thread overview]
Message-ID: <20161104144534.14790-3-juerg.haefliger@hpe.com> (raw)
In-Reply-To: <20161104144534.14790-1-juerg.haefliger@hpe.com>
Allocating a page to userspace that was previously allocated to the
kernel requires an expensive TLB shootdown. To minimize this, we only
put non-kernel pages into the hot cache to favor their allocation.
Signed-off-by: Juerg Haefliger <juerg.haefliger@hpe.com>
---
include/linux/xpfo.h | 2 ++
mm/page_alloc.c | 8 +++++++-
mm/xpfo.c | 8 ++++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/include/linux/xpfo.h b/include/linux/xpfo.h
index 77187578ca33..077d1cfadfa2 100644
--- a/include/linux/xpfo.h
+++ b/include/linux/xpfo.h
@@ -24,6 +24,7 @@ extern void xpfo_alloc_page(struct page *page, int order, gfp_t gfp);
extern void xpfo_free_page(struct page *page, int order);
extern bool xpfo_page_is_unmapped(struct page *page);
+extern bool xpfo_page_is_kernel(struct page *page);
#else /* !CONFIG_XPFO */
@@ -33,6 +34,7 @@ static inline void xpfo_alloc_page(struct page *page, int order, gfp_t gfp) { }
static inline void xpfo_free_page(struct page *page, int order) { }
static inline bool xpfo_page_is_unmapped(struct page *page) { return false; }
+static inline bool xpfo_page_is_kernel(struct page *page) { return false; }
#endif /* CONFIG_XPFO */
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 100e80e008e2..09ef4f7cfd14 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2440,7 +2440,13 @@ void free_hot_cold_page(struct page *page, bool cold)
}
pcp = &this_cpu_ptr(zone->pageset)->pcp;
- if (!cold)
+ /*
+ * XPFO: Allocating a page to userspace that was previously allocated
+ * to the kernel requires an expensive TLB shootdown. To minimize this,
+ * we only put non-kernel pages into the hot cache to favor their
+ * allocation.
+ */
+ if (!cold && !xpfo_page_is_kernel(page))
list_add(&page->lru, &pcp->lists[migratetype]);
else
list_add_tail(&page->lru, &pcp->lists[migratetype]);
diff --git a/mm/xpfo.c b/mm/xpfo.c
index 8e3a6a694b6a..0e447e38008a 100644
--- a/mm/xpfo.c
+++ b/mm/xpfo.c
@@ -204,3 +204,11 @@ inline bool xpfo_page_is_unmapped(struct page *page)
return test_bit(PAGE_EXT_XPFO_UNMAPPED, &lookup_page_ext(page)->flags);
}
EXPORT_SYMBOL(xpfo_page_is_unmapped);
+
+inline bool xpfo_page_is_kernel(struct page *page)
+{
+ if (!static_branch_unlikely(&xpfo_inited))
+ return false;
+
+ return test_bit(PAGE_EXT_XPFO_KERNEL, &lookup_page_ext(page)->flags);
+}
--
2.10.1
prev parent reply other threads:[~2016-11-04 14:46 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-26 14:21 [RFC PATCH] Add support for eXclusive Page Frame Ownership (XPFO) Juerg Haefliger
2016-03-01 1:31 ` Laura Abbott
2016-03-21 8:37 ` Juerg Haefliger
2016-03-28 19:29 ` Laura Abbott
2016-03-01 2:10 ` Balbir Singh
2016-03-21 8:44 ` Juerg Haefliger
2016-04-01 0:21 ` Balbir Singh
2016-09-02 11:39 ` [RFC PATCH v2 0/3] " Juerg Haefliger
2016-09-02 11:39 ` [RFC PATCH v2 1/3] " Juerg Haefliger
2016-09-02 11:39 ` [RFC PATCH v2 2/3] xpfo: Only put previous userspace pages into the hot cache Juerg Haefliger
2016-09-02 20:39 ` Dave Hansen
2016-09-05 11:54 ` Juerg Haefliger
2016-09-02 11:39 ` [RFC PATCH v2 3/3] block: Always use a bounce buffer when XPFO is enabled Juerg Haefliger
2016-09-14 7:18 ` [RFC PATCH v2 0/3] Add support for eXclusive Page Frame Ownership (XPFO) Juerg Haefliger
2016-09-14 7:18 ` [RFC PATCH v2 1/3] " Juerg Haefliger
2016-09-14 7:19 ` [RFC PATCH v2 2/3] xpfo: Only put previous userspace pages into the hot cache Juerg Haefliger
2016-09-14 14:33 ` [kernel-hardening] " Dave Hansen
2016-09-14 14:40 ` Juerg Haefliger
2016-09-14 14:48 ` Dave Hansen
2016-09-21 5:32 ` Juerg Haefliger
2016-09-14 7:19 ` [RFC PATCH v2 3/3] block: Always use a bounce buffer when XPFO is enabled Juerg Haefliger
2016-09-14 7:33 ` Christoph Hellwig
2016-09-14 7:23 ` [RFC PATCH v2 0/3] Add support for eXclusive Page Frame Ownership (XPFO) Juerg Haefliger
2016-09-14 9:36 ` [kernel-hardening] " Mark Rutland
2016-09-14 9:49 ` Mark Rutland
2016-11-04 14:45 ` [RFC PATCH v3 0/2] " Juerg Haefliger
2016-11-04 14:45 ` [RFC PATCH v3 1/2] " Juerg Haefliger
2016-11-04 14:50 ` Christoph Hellwig
2016-11-10 5:53 ` [kernel-hardening] " ZhaoJunmin Zhao(Junmin)
2016-11-10 19:11 ` Kees Cook
2016-11-15 11:15 ` Juerg Haefliger
2016-11-10 19:24 ` Kees Cook
2016-11-15 11:18 ` Juerg Haefliger
2016-11-24 10:56 ` AKASHI Takahiro
2016-11-28 11:15 ` Juerg Haefliger
2016-12-09 9:02 ` AKASHI Takahiro
2016-11-04 14:45 ` Juerg Haefliger [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20161104144534.14790-3-juerg.haefliger@hpe.com \
--to=juerg.haefliger@hpe.com \
--cc=kernel-hardening@lists.openwall.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-x86_64@vger.kernel.org \
--cc=vpk@cs.columbia.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).