From: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
To: Dave Hansen <dave.hansen@intel.com>,
linux-kernel@vger.kernel.org,
Christoph Hellwig <hch@infradead.org>,
linux-mm@kvack.org, Andy Lutomirski <luto@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>
Subject: [PATCH 4/4] kernel/fork: add option to use virtually mapped stacks as fallback
Date: Tue, 23 Jan 2018 13:55:32 +0300 [thread overview]
Message-ID: <151670493255.658225.2881484505285363395.stgit@buzz> (raw)
In-Reply-To: <151670492223.658225.4605377710524021456.stgit@buzz>
Virtually mapped stack have two bonuses: it eats order-0 pages and
adds guard page at the end. But it slightly slower if system have
plenty free high-order pages.
This patch adds option to use virtually bapped stack as fallback for
atomic allocation of traditional high-order page.
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
arch/Kconfig | 14 ++++++++++++++
kernel/fork.c | 11 +++++++++++
2 files changed, 25 insertions(+)
diff --git a/arch/Kconfig b/arch/Kconfig
index 400b9e1b2f27..c181ab263e7f 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -904,6 +904,20 @@ config VMAP_STACK
the stack to map directly to the KASAN shadow map using a formula
that is incorrect if the stack is in vmalloc space.
+config VMAP_STACK_AS_FALLBACK
+ default n
+ bool "Use a virtually-mapped stack as fallback for directly-mapped"
+ depends on VMAP_STACK
+ help
+ With this option kernel first tries to allocate directly-mapped stack
+ without calling direct memory reclaim and fallback to vmap stack.
+
+ Allocation of directly mapped stack faster than vmap if system a lot
+ of free memory and much slower if all memory is used or fragmented.
+
+ This option neutralize stack overflow protection but allows to
+ achieve best performance for syscalls fork() and clone().
+
config ARCH_OPTIONAL_KERNEL_RWX
def_bool n
diff --git a/kernel/fork.c b/kernel/fork.c
index 457c9151f3c8..cc61a083954d 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -207,6 +207,17 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node)
struct vm_struct *stack;
int i;
+#ifdef CONFIG_VMAP_STACK_AS_FALLBACK
+ struct page *page;
+
+ page = alloc_pages_node(node, THREADINFO_GFP & ~__GFP_DIRECT_RECLAIM,
+ THREAD_SIZE_ORDER);
+ if (page) {
+ tsk->stack_vm_area = NULL;
+ return page_address(page);
+ }
+#endif
+
for (i = 0; i < NR_CACHED_STACKS; i++) {
stack = this_cpu_xchg(cached_stacks[i], NULL);
if (!stack)
next prev parent reply other threads:[~2018-01-23 10:55 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-01-23 10:55 [PATCH 1/4] vmalloc: add vm_flags argument to internal __vmalloc_node() Konstantin Khlebnikov
2018-01-23 10:55 ` [PATCH 2/4] vmalloc: add __vmalloc_area() Konstantin Khlebnikov
2018-01-23 10:55 ` [PATCH 3/4] kernel/fork: switch vmapped stack callation to __vmalloc_area() Konstantin Khlebnikov
2018-01-23 13:57 ` Konstantin Khlebnikov
2018-02-21 0:16 ` Andrew Morton
2018-02-21 7:23 ` Konstantin Khlebnikov
2018-02-21 16:35 ` Andy Lutomirski
2018-01-23 10:55 ` Konstantin Khlebnikov [this message]
2018-02-21 15:42 ` Use higher-order pages in vmalloc Matthew Wilcox
2018-02-21 16:11 ` Andy Lutomirski
2018-02-21 16:50 ` Matthew Wilcox
2018-02-21 16:16 ` Dave Hansen
2018-02-21 17:01 ` Matthew Wilcox
2018-02-22 6:59 ` Michal Hocko
2018-02-22 12:22 ` Matthew Wilcox
2018-02-22 13:36 ` Michal Hocko
2018-02-22 19:01 ` Andy Lutomirski
2018-02-22 19:19 ` Dave Hansen
2018-02-22 19:27 ` Andy Lutomirski
2018-02-22 19:36 ` Dave Hansen
2018-02-23 12:13 ` Michal Hocko
2018-03-01 18:16 ` Eric Dumazet
2018-02-21 12:24 ` [PATCH 1/4] vmalloc: add vm_flags argument to internal __vmalloc_node() Matthew Wilcox
2018-02-21 12:39 ` Andrey Ryabinin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=151670493255.658225.2881484505285363395.stgit@buzz \
--to=khlebnikov@yandex-team.ru \
--cc=akpm@linux-foundation.org \
--cc=dave.hansen@intel.com \
--cc=hch@infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).