Re: [RFC 00/14] Dynamic Kernel Stacks

From: Matthew Wilcox <willy@infradead.org>
To: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>,
	Kent Overstreet <kent.overstreet@linux.dev>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de,
	brauner@kernel.org, bristot@redhat.com, bsegall@google.com,
	dave.hansen@linux.intel.com, dianders@chromium.org,
	dietmar.eggemann@arm.com, eric.devolder@oracle.com,
	hca@linux.ibm.com, hch@infradead.org,
	jacob.jun.pan@linux.intel.com, jgg@ziepe.ca, jpoimboe@kernel.org,
	jroedel@suse.de, juri.lelli@redhat.com, kinseyho@google.com,
	kirill.shutemov@linux.intel.com, lstoakes@gmail.com,
	luto@kernel.org, mgorman@suse.de, mic@digikod.net,
	michael.christie@oracle.com, mingo@redhat.com, mjguzik@gmail.com,
	mst@redhat.com, npiggin@gmail.com, peterz@infradead.org,
	pmladek@suse.com, rick.p.edgecombe@intel.com,
	rostedt@goodmis.org, surenb@google.com, tglx@linutronix.de,
	urezki@gmail.com, vincent.guittot@linaro.org,
	vschneid@redhat.com
Subject: Re: [RFC 00/14] Dynamic Kernel Stacks
Date: Sun, 17 Mar 2024 00:41:33 +0000	[thread overview]
Message-ID: <ZfY8PSnsLtkHBBZF@casper.infradead.org> (raw)
In-Reply-To: <CA+CK2bDothmwdJ86K1LiKWDKdWdYDjg5WCwdbapL9c3Y_Sf+kg@mail.gmail.com>

On Sat, Mar 16, 2024 at 03:17:57PM -0400, Pasha Tatashin wrote:
> Expanding on Mathew's idea of an interface for dynamic kernel stack
> sizes, here's what I'm thinking:
> 
> - Kernel Threads: Create all kernel threads with a fully populated
> THREAD_SIZE stack.  (i.e. 16K)
> - User Threads: Create all user threads with THREAD_SIZE kernel stack
> but only the top page mapped. (i.e. 4K)
> - In enter_from_user_mode(): Expand the thread stack to 16K by mapping
> three additional pages from the per-CPU stack cache. This function is
> called early in kernel entry points.
> - exit_to_user_mode(): Unmap the extra three pages and return them to
> the per-CPU cache. This function is called late in the kernel exit
> path.
> 
> Both of the above hooks are called with IRQ disabled on all kernel
> entries whether through interrupts and syscalls, and they are called
> early/late enough that 4K is enough to handle the rest of entry/exit.

At what point do we replenish the per-CPU stash of pages?  If we're
12kB deep in the stack and call mutex_lock(), we can be scheduled out,
and then the new thread can make a syscall.  Do we just assume that
get_free_page() can sleep at kernel entry (seems reasonable)?  I don't
think this is an infeasible problem, I'd just like it to be described.