linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: tip-bot for Joerg Roedel <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: torvalds@linux-foundation.org, llong@redhat.com,
	dave.hansen@intel.com, jpoimboe@redhat.com, luto@kernel.org,
	mingo@kernel.org, jgross@suse.com, tglx@linutronix.de,
	jkosina@suse.cz, linux-kernel@vger.kernel.org,
	will.deacon@arm.com, David.Laight@aculab.com,
	boris.ostrovsky@oracle.com, bp@alien8.de, hpa@zytor.com,
	eduval@amazon.com, dhgutteridge@sympatico.ca,
	dvlasenk@redhat.com, brgerst@gmail.com, jroedel@suse.de,
	gregkh@linuxfoundation.org, pavel@ucw.cz, aarcange@redhat.com,
	peterz@infradead.org
Subject: [tip:x86/pti] x86/entry/32: Add PTI CR3 switches to NMI handler code
Date: Thu, 19 Jul 2018 16:25:10 -0700	[thread overview]
Message-ID: <tip-b65bef400689ceee7108c2d47fb97ae91f4d1440@git.kernel.org> (raw)
In-Reply-To: <1531906876-13451-14-git-send-email-joro@8bytes.org>

Commit-ID:  b65bef400689ceee7108c2d47fb97ae91f4d1440
Gitweb:     https://git.kernel.org/tip/b65bef400689ceee7108c2d47fb97ae91f4d1440
Author:     Joerg Roedel <jroedel@suse.de>
AuthorDate: Wed, 18 Jul 2018 11:40:50 +0200
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Fri, 20 Jul 2018 01:11:39 +0200

x86/entry/32: Add PTI CR3 switches to NMI handler code

The NMI handler is special, as it needs to leave with the same CR3 as it
was entered with. This is required because the NMI can happen within kernel
context but with user CR3 already loaded, i.e. after switching to user CR3
but before returning to user space.

Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Pavel Machek <pavel@ucw.cz>
Cc: "H . Peter Anvin" <hpa@zytor.com>
Cc: linux-mm@kvack.org
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: David Laight <David.Laight@aculab.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Eduardo Valentin <eduval@amazon.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Will Deacon <will.deacon@arm.com>
Cc: aliguori@amazon.com
Cc: daniel.gruss@iaik.tugraz.at
Cc: hughd@google.com
Cc: keescook@google.com
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Waiman Long <llong@redhat.com>
Cc: "David H . Gutteridge" <dhgutteridge@sympatico.ca>
Cc: joro@8bytes.org
Link: https://lkml.kernel.org/r/1531906876-13451-14-git-send-email-joro@8bytes.org

---
 arch/x86/entry/entry_32.S | 39 +++++++++++++++++++++++++++++++++------
 1 file changed, 33 insertions(+), 6 deletions(-)

diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
index 60b28dfa00dc..b1541c74c71a 100644
--- a/arch/x86/entry/entry_32.S
+++ b/arch/x86/entry/entry_32.S
@@ -210,8 +210,19 @@
 
 .endm
 
-.macro SAVE_ALL_NMI
+.macro SAVE_ALL_NMI cr3_reg:req
 	SAVE_ALL
+
+	/*
+	 * Now switch the CR3 when PTI is enabled.
+	 *
+	 * We can enter with either user or kernel cr3, the code will
+	 * store the old cr3 in \cr3_reg and switches to the kernel cr3
+	 * if necessary.
+	 */
+	SWITCH_TO_KERNEL_CR3 scratch_reg=\cr3_reg
+
+.Lend_\@:
 .endm
 /*
  * This is a sneaky trick to help the unwinder find pt_regs on the stack.  The
@@ -259,7 +270,23 @@
 	POP_GS_EX
 .endm
 
-.macro RESTORE_ALL_NMI pop=0
+.macro RESTORE_ALL_NMI cr3_reg:req pop=0
+	/*
+	 * Now switch the CR3 when PTI is enabled.
+	 *
+	 * We enter with kernel cr3 and switch the cr3 to the value
+	 * stored on \cr3_reg, which is either a user or a kernel cr3.
+	 */
+	ALTERNATIVE "jmp .Lswitched_\@", "", X86_FEATURE_PTI
+
+	testl	$PTI_SWITCH_MASK, \cr3_reg
+	jz	.Lswitched_\@
+
+	/* User cr3 in \cr3_reg - write it to hardware cr3 */
+	movl	\cr3_reg, %cr3
+
+.Lswitched_\@:
+
 	RESTORE_REGS pop=\pop
 .endm
 
@@ -1331,7 +1358,7 @@ ENTRY(nmi)
 #endif
 
 	pushl	%eax				# pt_regs->orig_ax
-	SAVE_ALL_NMI
+	SAVE_ALL_NMI cr3_reg=%edi
 	ENCODE_FRAME_POINTER
 	xorl	%edx, %edx			# zero error code
 	movl	%esp, %eax			# pt_regs pointer
@@ -1359,7 +1386,7 @@ ENTRY(nmi)
 
 .Lnmi_return:
 	CHECK_AND_APPLY_ESPFIX
-	RESTORE_ALL_NMI pop=4
+	RESTORE_ALL_NMI cr3_reg=%edi pop=4
 	jmp	.Lirq_return
 
 #ifdef CONFIG_X86_ESPFIX32
@@ -1375,12 +1402,12 @@ ENTRY(nmi)
 	pushl	16(%esp)
 	.endr
 	pushl	%eax
-	SAVE_ALL_NMI
+	SAVE_ALL_NMI cr3_reg=%edi
 	ENCODE_FRAME_POINTER
 	FIXUP_ESPFIX_STACK			# %eax == %esp
 	xorl	%edx, %edx			# zero error code
 	call	do_nmi
-	RESTORE_ALL_NMI
+	RESTORE_ALL_NMI cr3_reg=%edi
 	lss	12+4(%esp), %esp		# back to espfix stack
 	jmp	.Lirq_return
 #endif

  reply	other threads:[~2018-07-19 23:26 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-18  9:40 [PATCH 00/39 v8] PTI support for x86-32 Joerg Roedel
2018-07-18  9:40 ` [PATCH 01/39] x86/asm-offsets: Move TSS_sp0 and TSS_sp1 to asm-offsets.c Joerg Roedel
2018-07-19 23:18   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 02/39] x86/entry/32: Rename TSS_sysenter_sp0 to TSS_entry2task_stack Joerg Roedel
2018-07-19 23:19   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 03/39] x86/entry/32: Load task stack from x86_tss.sp1 in SYSENTER handler Joerg Roedel
2018-07-19 23:19   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 04/39] x86/entry/32: Put ESPFIX code into a macro Joerg Roedel
2018-07-19 23:20   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 05/39] x86/entry/32: Unshare NMI return path Joerg Roedel
2018-07-19 23:21   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 06/39] x86/entry/32: Split off return-to-kernel path Joerg Roedel
2018-07-19 23:21   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 07/39] x86/entry/32: Enter the kernel via trampoline stack Joerg Roedel
2018-07-18 18:09   ` Brian Gerst
2018-07-19 20:52     ` Thomas Gleixner
2018-07-19 23:22   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 08/39] x86/entry/32: Leave " Joerg Roedel
2018-07-19 23:22   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 09/39] x86/entry/32: Introduce SAVE_ALL_NMI and RESTORE_ALL_NMI Joerg Roedel
2018-07-19 23:23   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 10/39] x86/entry/32: Handle Entry from Kernel-Mode on Entry-Stack Joerg Roedel
2018-07-19 23:23   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-10-12 18:29   ` [PATCH 10/39] " Jan Kiszka
2018-10-13  9:54     ` [PATCH] x86/entry/32: Fix setup of CS high bits Jan Kiszka
2018-10-13 15:12       ` Andy Lutomirski
2018-10-15 13:08         ` Jan Kiszka
2018-10-15 13:14           ` David Laight
2018-10-15 13:18             ` Jan Kiszka
2018-10-15 13:29               ` David Laight
2018-10-15  9:10       ` Joerg Roedel
2018-10-15 14:09       ` [PATCH v2] " Jan Kiszka
2018-10-15 15:09         ` [tip:x86/urgent] x86/entry/32: Clear the " tip-bot for Jan Kiszka
2018-10-18  6:21         ` tip-bot for Jan Kiszka
2018-07-18  9:40 ` [PATCH 11/39] x86/entry/32: Simplify debug entry point Joerg Roedel
2018-07-19 23:24   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 12/39] x86/entry/32: Add PTI cr3 switch to non-NMI entry/exit points Joerg Roedel
2018-07-19 23:24   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 13/39] x86/entry/32: Add PTI cr3 switches to NMI handler code Joerg Roedel
2018-07-19 23:25   ` tip-bot for Joerg Roedel [this message]
2018-07-18  9:40 ` [PATCH 14/39] x86/entry: Rename update_sp0 to update_task_stack Joerg Roedel
2018-07-19 23:25   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 15/39] x86/pgtable: Rename pti_set_user_pgd to pti_set_user_pgtbl Joerg Roedel
2018-07-19 23:26   ` [tip:x86/pti] x86/pgtable: Rename pti_set_user_pgd() to pti_set_user_pgtbl() tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 16/39] x86/pgtable/pae: Unshare kernel PMDs when PTI is enabled Joerg Roedel
2018-07-19 23:26   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 17/39] x86/pgtable/32: Allocate 8k page-tables " Joerg Roedel
2018-07-19 23:27   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 18/39] x86/pgtable: Move pgdp kernel/user conversion functions to pgtable.h Joerg Roedel
2018-07-19 23:27   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 19/39] x86/pgtable: Move pti_set_user_pgtbl() " Joerg Roedel
2018-07-19 23:28   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 20/39] x86/pgtable: Move two more functions from pgtable_64.h " Joerg Roedel
2018-07-19 23:28   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 21/39] x86/mm/pae: Populate valid user PGD entries Joerg Roedel
2018-07-19 23:29   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:40 ` [PATCH 22/39] x86/mm/pae: Populate the user page-table with user pgd's Joerg Roedel
2018-07-19 23:30   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 23/39] x86/mm/legacy: " Joerg Roedel
2018-07-19 23:30   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 24/39] x86/mm/pti: Add an overflow check to pti_clone_pmds() Joerg Roedel
2018-07-19 23:31   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 25/39] x86/mm/pti: Define X86_CR3_PTI_PCID_USER_BIT on x86_32 Joerg Roedel
2018-07-19 23:31   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 26/39] x86/mm/pti: Clone CPU_ENTRY_AREA on PMD level " Joerg Roedel
2018-07-19 23:32   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 27/39] x86/mm/pti: Make pti_clone_kernel_text() compile on 32 bit Joerg Roedel
2018-07-19 23:32   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 28/39] x86/mm/pti: Keep permissions when cloning kernel text in pti_clone_kernel_text() Joerg Roedel
2018-07-19 23:33   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 29/39] x86/mm/pti: Introduce pti_finalize() Joerg Roedel
2018-07-19 23:33   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 30/39] x86/mm/pti: Clone entry-text again in pti_finalize() Joerg Roedel
2018-07-19 23:34   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 31/39] x86/mm/dump_pagetables: Define INIT_PGD Joerg Roedel
2018-07-19 23:34   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 32/39] x86/pgtable/pae: Use separate kernel PMDs for user page-table Joerg Roedel
2018-07-19 23:35   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-10-05 14:06   ` [PATCH 32/39] " Arnd Bergmann
2018-07-18  9:41 ` [PATCH 33/39] x86/ldt: Reserve address-space range on 32 bit for the LDT Joerg Roedel
2018-07-19 23:35   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 34/39] x86/ldt: Define LDT_END_ADDR Joerg Roedel
2018-07-19 23:36   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 35/39] x86/ldt: Split out sanity check in map_ldt_struct() Joerg Roedel
2018-07-19 23:36   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 36/39] x86/ldt: Enable LDT user-mapping for PAE Joerg Roedel
2018-07-19 23:37   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 37/39] x86/pti: Allow CONFIG_PAGE_TABLE_ISOLATION for x86_32 Joerg Roedel
2018-07-19 23:37   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 38/39] x86/mm/pti: Add Warning when booting on a PCID capable CPU Joerg Roedel
2018-07-19 23:38   ` [tip:x86/pti] " tip-bot for Joerg Roedel
2018-07-18  9:41 ` [PATCH 39/39] x86/entry/32: Add debug code to check entry/exit cr3 Joerg Roedel
2018-07-19 23:38   ` [tip:x86/pti] x86/entry/32: Add debug code to check entry/exit CR3 tip-bot for Joerg Roedel
2018-07-18 11:59 ` [PATCH 00/39 v8] PTI support for x86-32 Pavel Machek
2018-07-19 23:21 ` Thomas Gleixner
2018-07-20  7:59   ` Joerg Roedel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tip-b65bef400689ceee7108c2d47fb97ae91f4d1440@git.kernel.org \
    --to=tipbot@zytor.com \
    --cc=David.Laight@aculab.com \
    --cc=aarcange@redhat.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=dave.hansen@intel.com \
    --cc=dhgutteridge@sympatico.ca \
    --cc=dvlasenk@redhat.com \
    --cc=eduval@amazon.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hpa@zytor.com \
    --cc=jgross@suse.com \
    --cc=jkosina@suse.cz \
    --cc=jpoimboe@redhat.com \
    --cc=jroedel@suse.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=llong@redhat.com \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=pavel@ucw.cz \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=will.deacon@arm.com \
    --subject='Re: [tip:x86/pti] x86/entry/32: Add PTI CR3 switches to NMI handler code' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).