From: Pasha Tatashin <pasha.tatashin@soleen.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
akpm@linux-foundation.org, x86@kernel.org, bp@alien8.de,
brauner@kernel.org, bristot@redhat.com, bsegall@google.com,
dave.hansen@linux.intel.com, dianders@chromium.org,
dietmar.eggemann@arm.com, eric.devolder@oracle.com,
hca@linux.ibm.com, hch@infradead.org, hpa@zytor.com,
jacob.jun.pan@linux.intel.com, jgg@ziepe.ca,
jpoimboe@kernel.org, jroedel@suse.de, juri.lelli@redhat.com,
kent.overstreet@linux.dev, kinseyho@google.com,
kirill.shutemov@linux.intel.com, lstoakes@gmail.com,
luto@kernel.org, mgorman@suse.de, mic@digikod.net,
michael.christie@oracle.com, mingo@redhat.com,
mjguzik@gmail.com, mst@redhat.com, npiggin@gmail.com,
peterz@infradead.org, pmladek@suse.com,
rick.p.edgecombe@intel.com, rostedt@goodmis.org,
surenb@google.com, urezki@gmail.com, vincent.guittot@linaro.org,
vschneid@redhat.com
Subject: Re: [RFC 11/14] x86: add support for Dynamic Kernel Stacks
Date: Wed, 13 Mar 2024 11:28:26 -0400 [thread overview]
Message-ID: <CA+CK2bAzJuCe06g_TEOh3B-hK+dXfUaGaOSTgzyxkN4zqpSU_A@mail.gmail.com> (raw)
In-Reply-To: <CA+CK2bBgeK=rW3=RJSx8zxST8xnJ-VXSSYmKPVK9gHX3pxEWuQ@mail.gmail.com>
On Wed, Mar 13, 2024 at 9:43 AM Pasha Tatashin
<pasha.tatashin@soleen.com> wrote:
>
> On Wed, Mar 13, 2024 at 6:23 AM Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> > On Mon, Mar 11 2024 at 16:46, Pasha Tatashin wrote:
> > > @@ -413,6 +413,9 @@ DEFINE_IDTENTRY_DF(exc_double_fault)
> > > }
> > > #endif
> > >
> > > + if (dynamic_stack_fault(current, address))
> > > + return;
> > > +
> > > irqentry_nmi_enter(regs);
> > > instrumentation_begin();
> > > notify_die(DIE_TRAP, str, regs, error_code, X86_TRAP_DF, SIGSEGV);
> > > diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > > index d6375b3c633b..651c558b10eb 100644
> > > --- a/arch/x86/mm/fault.c
> > > +++ b/arch/x86/mm/fault.c
> > > @@ -1198,6 +1198,9 @@ do_kern_addr_fault(struct pt_regs *regs, unsigned long hw_error_code,
> > > if (is_f00f_bug(regs, hw_error_code, address))
> > > return;
> > >
> > > + if (dynamic_stack_fault(current, address))
> > > + return;
> >
> > T1 schedules out with stack used close to the fault boundary.
> >
> > switch_to(T2)
> >
> > Now T1 schedules back in
> >
> > switch_to(T1)
> > __switch_to_asm()
> > ...
> > switch_stacks() <- SP on T1 stack
> > ! ...
> > ! jmp __switch_to()
> > ! __switch_to()
> > ! ...
> > ! raw_cpu_write(pcpu_hot.current_task, next_p);
> >
> > After switching SP to T1's stack and up to the point where
> > pcpu_hot.current_task (aka current) is updated to T1 a stack fault will
> > invoke dynamic_stack_fault(T2, address) which will return false here:
> >
> > /* check if address is inside the kernel stack area */
> > stack = (unsigned long)tsk->stack;
> > if (address < stack || address >= stack + THREAD_SIZE)
> > return false;
> >
> > because T2's stack does obviously not cover the faulting address on T1's
> > stack. As a consequence double fault will panic the machine.
>
> Hi Thomas,
>
> Thank you, you are absolutely right, we can't trust "current" in the
> fault handler.
>
> We can change dynamic_stack_fault() to only accept fault_address as an
> argument, and let it determine the right task_struct pointer
> internally.
>
> Let's modify dynamic_stack_fault() to accept only the fault_address.
> It can then determine the correct task_struct pointer internally.
>
> Here's a potential solution that is fast, avoids locking, and ensures atomicity:
>
> 1. Kernel Stack VA Space
> Dedicate a virtual address range ([KSTACK_START_VA - KSTACK_END_VA])
> exclusively for kernel stacks. This simplifies validation of faulting
> addresses to be part of a stack.
>
> 2. Finding the faulty task
> - Use ALIGN(fault_address, THREAD_SIZE) to calculate the end of the
> topmost stack page (since stack addresses are aligned to THREAD_SIZE).
> - Store the task_struct pointer as the last word on this topmost page,
> that is always present as it is a pre-allcated stack page.
>
> 3. Stack Padding
> Increase padding to 8 bytes on x86_64 (TOP_OF_KERNEL_STACK_PADDING 8)
> to accommodate the task_struct pointer.
Alternatively, do not even look-up the task_struct in
dynamic_stack_fault(), but only install the mapping to the faulting
address, store va in the per-cpu array, and handle the rest in
dynamic_stack() during context switching. At that time spin locks can
be taken, and we can do a find_vm_area(addr) call.
This way, we would not need to modify TOP_OF_KERNEL_STACK_PADDING to
keep task_struct in there.
Pasha
next prev parent reply other threads:[~2024-03-13 15:29 UTC|newest]
Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-03-11 16:46 [RFC 00/14] Dynamic Kernel Stacks Pasha Tatashin
2024-03-11 16:46 ` [RFC 01/14] task_stack.h: remove obsolete __HAVE_ARCH_KSTACK_END check Pasha Tatashin
2024-03-17 14:36 ` Christophe JAILLET
2024-03-17 15:13 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 02/14] fork: Clean-up ifdef logic around stack allocation Pasha Tatashin
2024-03-11 16:46 ` [RFC 03/14] fork: Clean-up naming of vm_strack/vm_struct variables in vmap stacks code Pasha Tatashin
2024-03-17 14:42 ` Christophe JAILLET
2024-03-19 16:32 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 04/14] fork: Remove assumption that vm_area->nr_pages equals to THREAD_SIZE Pasha Tatashin
2024-03-17 14:45 ` Christophe JAILLET
2024-03-17 15:14 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 05/14] fork: check charging success before zeroing stack Pasha Tatashin
2024-03-12 15:57 ` Kirill A. Shutemov
2024-03-12 16:52 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 06/14] fork: zero vmap stack using clear_page() instead of memset() Pasha Tatashin
2024-03-12 7:15 ` Nikolay Borisov
2024-03-12 16:53 ` Pasha Tatashin
2024-03-14 7:55 ` Christophe Leroy
2024-03-14 13:52 ` Pasha Tatashin
2024-03-17 14:48 ` Christophe JAILLET
2024-03-17 15:15 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 07/14] fork: use the first page in stack to store vm_stack in cached_stacks Pasha Tatashin
2024-03-11 16:46 ` [RFC 08/14] fork: separate vmap stack alloction and free calls Pasha Tatashin
2024-03-14 15:18 ` Jeff Xie
2024-03-14 17:14 ` Pasha Tatashin
2024-03-17 14:51 ` Christophe JAILLET
2024-03-17 15:15 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 09/14] mm/vmalloc: Add a get_vm_area_node() and vmap_pages_range_noflush() public functions Pasha Tatashin
2024-03-11 16:46 ` [RFC 10/14] fork: Dynamic Kernel Stacks Pasha Tatashin
2024-03-11 19:32 ` Randy Dunlap
2024-03-11 19:55 ` Pasha Tatashin
2024-03-11 16:46 ` [RFC 11/14] x86: add support for " Pasha Tatashin
2024-03-11 22:17 ` Andy Lutomirski
2024-03-11 23:10 ` Pasha Tatashin
2024-03-11 23:33 ` Thomas Gleixner
2024-03-11 23:34 ` Andy Lutomirski
2024-03-12 0:08 ` Pasha Tatashin
2024-03-12 0:23 ` Pasha Tatashin
2024-03-11 23:34 ` Dave Hansen
2024-03-11 23:41 ` Andy Lutomirski
2024-03-11 23:56 ` Nadav Amit
2024-03-12 0:02 ` Andy Lutomirski
2024-03-12 7:20 ` Nadav Amit
2024-03-12 0:53 ` Dave Hansen
2024-03-12 1:25 ` H. Peter Anvin
2024-03-12 2:16 ` Andy Lutomirski
2024-03-12 2:20 ` H. Peter Anvin
2024-03-12 21:58 ` Andi Kleen
2024-03-13 10:23 ` Thomas Gleixner
2024-03-13 13:43 ` Pasha Tatashin
2024-03-13 15:28 ` Pasha Tatashin [this message]
2024-03-13 16:12 ` Thomas Gleixner
2024-03-14 14:03 ` Pasha Tatashin
2024-03-14 18:26 ` Thomas Gleixner
2024-03-11 16:46 ` [RFC 12/14] task_stack.h: Clean-up stack_not_used() implementation Pasha Tatashin
2024-03-11 16:46 ` [RFC 13/14] task_stack.h: Add stack_not_used() support for dynamic stack Pasha Tatashin
2024-03-11 16:46 ` [RFC 14/14] fork: Dynamic Kernel Stack accounting Pasha Tatashin
2024-03-11 17:09 ` [RFC 00/14] Dynamic Kernel Stacks Mateusz Guzik
2024-03-11 18:58 ` Pasha Tatashin
2024-03-11 19:21 ` Mateusz Guzik
2024-03-11 19:55 ` Pasha Tatashin
2024-03-12 17:18 ` H. Peter Anvin
2024-03-12 19:45 ` Pasha Tatashin
2024-03-12 21:36 ` H. Peter Anvin
2024-03-14 19:05 ` Kent Overstreet
2024-03-14 19:23 ` Pasha Tatashin
2024-03-14 19:28 ` Kent Overstreet
2024-03-14 19:34 ` Pasha Tatashin
2024-03-14 19:49 ` Kent Overstreet
2024-03-12 22:18 ` David Laight
2024-03-14 19:43 ` Matthew Wilcox
2024-03-14 19:53 ` Kent Overstreet
2024-03-14 19:57 ` Matthew Wilcox
2024-03-14 19:58 ` Kent Overstreet
2024-03-15 3:13 ` Pasha Tatashin
2024-03-15 3:39 ` H. Peter Anvin
2024-03-16 19:17 ` Pasha Tatashin
2024-03-17 0:41 ` Matthew Wilcox
2024-03-17 1:32 ` Kent Overstreet
2024-03-17 14:19 ` Pasha Tatashin
2024-03-17 14:43 ` Brian Gerst
2024-03-17 16:15 ` Pasha Tatashin
2024-03-17 21:30 ` Brian Gerst
2024-03-18 14:59 ` Pasha Tatashin
2024-03-18 21:02 ` Brian Gerst
2024-03-19 14:56 ` Pasha Tatashin
2024-03-17 18:57 ` David Laight
2024-03-18 15:09 ` Pasha Tatashin
2024-03-18 15:13 ` Pasha Tatashin
2024-03-18 15:19 ` Matthew Wilcox
2024-03-18 15:30 ` Pasha Tatashin
2024-03-18 15:53 ` David Laight
2024-03-18 16:57 ` Pasha Tatashin
2024-03-18 15:38 ` David Laight
2024-03-18 17:00 ` Pasha Tatashin
2024-03-18 17:37 ` Pasha Tatashin
2024-03-15 4:17 ` H. Peter Anvin
2024-03-17 0:47 ` H. Peter Anvin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+CK2bAzJuCe06g_TEOh3B-hK+dXfUaGaOSTgzyxkN4zqpSU_A@mail.gmail.com \
--to=pasha.tatashin@soleen.com \
--cc=akpm@linux-foundation.org \
--cc=bp@alien8.de \
--cc=brauner@kernel.org \
--cc=bristot@redhat.com \
--cc=bsegall@google.com \
--cc=dave.hansen@linux.intel.com \
--cc=dianders@chromium.org \
--cc=dietmar.eggemann@arm.com \
--cc=eric.devolder@oracle.com \
--cc=hca@linux.ibm.com \
--cc=hch@infradead.org \
--cc=hpa@zytor.com \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jgg@ziepe.ca \
--cc=jpoimboe@kernel.org \
--cc=jroedel@suse.de \
--cc=juri.lelli@redhat.com \
--cc=kent.overstreet@linux.dev \
--cc=kinseyho@google.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lstoakes@gmail.com \
--cc=luto@kernel.org \
--cc=mgorman@suse.de \
--cc=mic@digikod.net \
--cc=michael.christie@oracle.com \
--cc=mingo@redhat.com \
--cc=mjguzik@gmail.com \
--cc=mst@redhat.com \
--cc=npiggin@gmail.com \
--cc=peterz@infradead.org \
--cc=pmladek@suse.com \
--cc=rick.p.edgecombe@intel.com \
--cc=rostedt@goodmis.org \
--cc=surenb@google.com \
--cc=tglx@linutronix.de \
--cc=urezki@gmail.com \
--cc=vincent.guittot@linaro.org \
--cc=vschneid@redhat.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).