All of lore.kernel.org
 help / color / mirror / Atom feed
From: Joerg Roedel <joro@8bytes.org>
To: Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@kernel.org>, "H . Peter Anvin" <hpa@zytor.com>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andy Lutomirski <luto@kernel.org>,
	Dave Hansen <dave.hansen@intel.com>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	Juergen Gross <jgross@suse.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Borislav Petkov <bp@alien8.de>, Jiri Kosina <jkosina@suse.cz>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Brian Gerst <brgerst@gmail.com>,
	David Laight <David.Laight@aculab.com>,
	Denys Vlasenko <dvlasenk@redhat.com>,
	Eduardo Valentin <eduval@amazon.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	Will Deacon <will.deacon@arm.com>,
	aliguori@amazon.com, daniel.gruss@iaik.tugraz.at,
	hughd@google.com, keescook@google.com,
	Andrea Arcangeli <aarcange@redhat.com>,
	Waiman Long <llong@redhat.com>, Pavel Machek <pavel@ucw.cz>,
	"David H . Gutteridge" <dhgutteridge@sympatico.ca>,
	jroedel@suse.de, Arnaldo Carvalho de Melo <acme@kernel.org>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
	joro@8bytes.org
Subject: [PATCH 3/3] x86/entry/32: Copy only ptregs on paranoid entry/exit path
Date: Fri, 20 Jul 2018 18:22:24 +0200	[thread overview]
Message-ID: <1532103744-31902-4-git-send-email-joro@8bytes.org> (raw)
In-Reply-To: <1532103744-31902-1-git-send-email-joro@8bytes.org>

From: Joerg Roedel <jroedel@suse.de>

The code that switches from entry- to task-stack when we
enter from kernel-mode copies the full entry-stack contents
to the task-stack.

That is because we don't trust that the entry-stack
contents. But actually we can trust its contents if we are
not scheduled between entry and exit.

So do less copying and move only the ptregs over to the
task-stack in this code-path.

Suggested-by: Andy Lutomirski <luto@amacapital.net>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 arch/x86/entry/entry_32.S | 70 +++++++++++++++++++++++++----------------------
 1 file changed, 38 insertions(+), 32 deletions(-)

diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
index 2767c62..90166b2 100644
--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -469,33 +469,48 @@
 	 * segment registers on the way back to user-space or when the
 	 * sysenter handler runs with eflags.tf set.
 	 *
-	 * When we switch to the task-stack here, we can't trust the
-	 * contents of the entry-stack anymore, as the exception handler
-	 * might be scheduled out or moved to another CPU. Therefore we
-	 * copy the complete entry-stack to the task-stack and set a
-	 * marker in the iret-frame (bit 31 of the CS dword) to detect
-	 * what we've done on the iret path.
+	 * When we switch to the task-stack here, we extend the
+	 * stack-frame we copy to include the entry-stack %esp and a
+	 * pseudo %ss value so that we have a full ptregs struct on the
+	 * stack. We set a marker in the frame (bit 31 of the CS dword).
 	 *
-	 * On the iret path we copy everything back and switch to the
-	 * entry-stack, so that the interrupted kernel code-path
-	 * continues on the same stack it was interrupted with.
+	 * On the iret path we read %esp from the PT_OLDESP slot on the
+	 * stack and copy ptregs (except oldesp and oldss) to it, when
+	 * we find the marker set. Then we switch to the %esp we read,
+	 * so that the interrupted kernel code-path continues on the
+	 * same stack it was interrupted with.
 	 *
 	 * Be aware that an NMI can happen anytime in this code.
 	 *
+	 * Register values here are:
+	 *
 	 * %esi: Entry-Stack pointer (same as %esp)
 	 * %edi: Top of the task stack
 	 * %eax: CR3 on kernel entry
 	 */
 
-	/* Calculate number of bytes on the entry stack in %ecx */
-	movl	%esi, %ecx
+	/* Allocate full pt_regs on task-stack */
+	subl	$PTREGS_SIZE, %edi
+
+	/* Switch to task-stack */
+	movl	%edi, %esp
 
-	/* %ecx to the top of entry-stack */
-	andl	$(MASK_entry_stack), %ecx
-	addl	$(SIZEOF_entry_stack), %ecx
+	/* Populate pt_regs on task-stack */
+	movl	$__KERNEL_DS, PT_OLDSS(%esp)	/* Check: Is this needed? */
 
-	/* Number of bytes on the entry stack to %ecx */
-	sub	%esi, %ecx
+	/*
+	 * Save entry-stack pointer on task-stack so that we can switch back to
+	 * it on the the iret path.
+	 */
+	movl	%esi, PT_OLDESP(%esp)
+
+	/* sizeof(pt_regs) minus space for %esp and %ss to %ecx */
+	movl	$(PTREGS_SIZE - 8), %ecx
+
+	/* Copy rest */
+	shrl	$2, %ecx
+	cld
+	rep movsl
 
 	/* Mark stackframe as coming from entry stack */
 	orl	$CS_FROM_ENTRY_STACK, PT_CS(%esp)
@@ -505,16 +520,9 @@
 	 * so that we can switch back to it before iret.
 	 */
 	testl	$PTI_SWITCH_MASK, %eax
-	jz	.Lcopy_pt_regs_\@
+	jz	.Lend_\@
 	orl	$CS_FROM_USER_CR3, PT_CS(%esp)
 
-	/*
-	 * %esi and %edi are unchanged, %ecx contains the number of
-	 * bytes to copy. The code at .Lcopy_pt_regs_\@ will allocate
-	 * the stack-frame on task-stack and copy everything over
-	 */
-	jmp .Lcopy_pt_regs_\@
-
 .Lend_\@:
 .endm
 
@@ -594,16 +602,14 @@
 	/* Clear marker from stack-frame */
 	andl	$(~CS_FROM_ENTRY_STACK), PT_CS(%esp)
 
-	/* Copy the remaining task-stack contents to entry-stack */
+	/*
+	 * Copy the remaining 'struct ptregs' to entry-stack. Leave out
+	 * OLDESP and OLDSS as we didn't copy that over on entry.
+	 */
 	movl	%esp, %esi
-	movl	PER_CPU_VAR(cpu_tss_rw + TSS_sp0), %edi
+	movl	PT_OLDESP(%esp), %edi
 
-	/* Bytes on the task-stack to ecx */
-	movl	PER_CPU_VAR(cpu_tss_rw + TSS_sp1), %ecx
-	subl	%esi, %ecx
-
-	/* Allocate stack-frame on entry-stack */
-	subl	%ecx, %edi
+	movl	$(PTREGS_SIZE - 8), %ecx
 
 	/*
 	 * Save future stack-pointer, we must not switch until the
-- 
2.7.4


  parent reply	other threads:[~2018-07-20 16:22 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-20 16:22 [PATCH 0/3] PTI for x86-32 Fixes and Updates Joerg Roedel
2018-07-20 16:22 ` [PATCH 1/3] perf/core: Make sure the ring-buffer is mapped in all page-tables Joerg Roedel
2018-07-20 17:06   ` Andy Lutomirski
2018-07-20 17:48     ` Joerg Roedel
2018-07-20 19:32       ` Andy Lutomirski
2018-07-20 21:37         ` Joerg Roedel
2018-07-20 22:20           ` Andy Lutomirski
2018-07-21 21:06             ` Linus Torvalds
2018-07-20 19:27     ` Thomas Gleixner
2018-07-20 19:33       ` Andy Lutomirski
2018-07-20 19:43         ` Thomas Gleixner
2018-07-20 19:53           ` Thomas Gleixner
2018-07-20 19:37   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-20 20:36   ` tip-bot for Joerg Roedel
2018-07-20 16:22 ` [PATCH 2/3] x86/entry/32: Check for VM86 mode in slow-path check Joerg Roedel
2018-07-20 19:37   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-20 20:37   ` tip-bot for Joerg Roedel
2018-07-21 16:06   ` [PATCH 2/3] " Pavel Machek
2018-07-20 16:22 ` Joerg Roedel [this message]
2018-07-20 17:09   ` [PATCH 3/3] x86/entry/32: Copy only ptregs on paranoid entry/exit path Andy Lutomirski
2018-07-20 21:42     ` Joerg Roedel
2018-07-23  3:49 ` [PATCH 0/3] PTI for x86-32 Fixes and Updates David H. Gutteridge
2018-07-23  7:29   ` Joerg Roedel
2018-07-26  3:47     ` David H. Gutteridge
2018-07-23 14:09 ` Pavel Machek
2018-07-23 19:00   ` Linus Torvalds
2018-07-23 21:38     ` Pavel Machek
2018-07-23 21:50       ` Andy Lutomirski
2018-07-23 21:55         ` Pavel Machek
2018-07-24 21:18         ` Pavel Machek
2018-07-23 21:59       ` Josh Poimboeuf
2018-07-23 22:07         ` Dave Hansen
2018-07-24 13:39     ` Pavel Machek
2018-07-24 14:39       ` Andy Lutomirski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1532103744-31902-4-git-send-email-joro@8bytes.org \
    --to=joro@8bytes.org \
    --cc=David.Laight@aculab.com \
    --cc=aarcange@redhat.com \
    --cc=acme@kernel.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=aliguori@amazon.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=daniel.gruss@iaik.tugraz.at \
    --cc=dave.hansen@intel.com \
    --cc=dhgutteridge@sympatico.ca \
    --cc=dvlasenk@redhat.com \
    --cc=eduval@amazon.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hpa@zytor.com \
    --cc=hughd@google.com \
    --cc=jgross@suse.com \
    --cc=jkosina@suse.cz \
    --cc=jolsa@redhat.com \
    --cc=jpoimboe@redhat.com \
    --cc=jroedel@suse.de \
    --cc=keescook@google.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=llong@redhat.com \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=namhyung@kernel.org \
    --cc=pavel@ucw.cz \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=will.deacon@arm.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.