All of lore.kernel.org
 help / color / mirror / Atom feed
From: Willy Tarreau <w@1wt.eu>
To: linux-kernel@vger.kernel.org, x86@kernel.org
Cc: Willy Tarreau <w@1wt.eu>, Andy Lutomirski <luto@kernel.org>,
	Borislav Petkov <bp@alien8.de>, Brian Gerst <brgerst@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Ingo Molnar <mingo@kernel.org>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Kees Cook <keescook@chromium.org>
Subject: [RFC PATCH v3 7/8] x86/entry/pti: avoid setting CR3 when it's already correct
Date: Wed, 10 Jan 2018 20:28:19 +0100	[thread overview]
Message-ID: <1515612500-14505-8-git-send-email-w@1wt.eu> (raw)
In-Reply-To: <1515612500-14505-1-git-send-email-w@1wt.eu>

When entering the kernel with CR3 pointing to the kernel's PGD, there's
no need to set it again. This will avoid a TLB flush on syscalls for tasks
running with the kernel's PGD (see next patch).

Signed-off-by: Willy Tarreau <w@1wt.eu>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Kees Cook <keescook@chromium.org>

v2:
  - updated comments according to Ingo's suggestions
  - split the code to keep only the CR3 changes here
---
 arch/x86/entry/calling.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h
index 45a63e0..19c6790 100644
--- a/arch/x86/entry/calling.h
+++ b/arch/x86/entry/calling.h
@@ -214,6 +214,11 @@
 .macro SWITCH_TO_KERNEL_CR3 scratch_reg:req
 	ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI
 	mov	%cr3, \scratch_reg
+
+	/* If we're already on the kernel PGD, we don't switch */
+	testq $(PTI_SWITCH_PGTABLES_MASK), \scratch_reg
+	jz .Lend_\@
+
 	ADJUST_KERNEL_CR3 \scratch_reg
 	mov	\scratch_reg, %cr3
 .Lend_\@:
@@ -262,6 +267,14 @@
 	ALTERNATIVE "jmp .Ldone_\@", "", X86_FEATURE_PTI
 	movq	%cr3, \scratch_reg
 	movq	\scratch_reg, \save_reg
+
+	/*
+	 * If we're already on the kernel PGD, we don't switch,
+	 * we just save the current CR3.
+	 */
+	testq $(PTI_SWITCH_PGTABLES_MASK), \scratch_reg
+	jz .Ldone_\@
+
 	/*
 	 * Is the "switch mask" all zero?  That means that both of
 	 * these are zero:
@@ -284,6 +297,13 @@
 .macro RESTORE_CR3 scratch_reg:req save_reg:req
 	ALTERNATIVE "jmp .Lend_\@", "", X86_FEATURE_PTI
 
+	/*
+	 * If we saved a kernel context on entry, we didn't switch the CR3,
+	 * so we don't need to restore it on the way out either:
+	 */
+	testq $(PTI_SWITCH_PGTABLES_MASK), \save_reg
+	jz .Lend_\@
+
 	ALTERNATIVE "jmp .Lwrcr3_\@", "", X86_FEATURE_PCID
 
 	/*
-- 
1.7.12.1

  parent reply	other threads:[~2018-01-10 19:29 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-10 19:28 [RFC PATCH v3 0/8] Per process PTI activation Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 1/8] x86/thread_info: add TIF_DISABLE_PTI_{NOW,NEXT} to disable PTI per task Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 2/8] x86/pti: add new config option PER_PROCESS_PTI Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 3/8] x86/pti: create the pti_adjust sysctl Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 4/8] x86/arch_prctl: add ARCH_DISABLE_PTI_{NOW,NEXT} to enable/disable PTI Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 5/8] exec: take care of disabling PTI upon execve() Willy Tarreau
2018-01-10 19:28 ` [RFC PATCH v3 6/8] x86/pti: don't mark the user PGD with _PAGE_NX Willy Tarreau
2018-01-10 19:54   ` Linus Torvalds
2018-01-10 19:59     ` Andy Lutomirski
2018-01-10 20:28       ` Woodhouse, David
2018-01-10 20:28         ` Woodhouse, David
2018-01-11  6:23         ` Willy Tarreau
2018-02-23 17:58         ` Konrad Rzeszutek Wilk
2018-02-23 19:30           ` Dave Hansen
2018-01-10 20:20   ` Dave Hansen
2018-01-11  6:27     ` Willy Tarreau
2018-01-10 19:28 ` Willy Tarreau [this message]
2018-01-10 19:28 ` [RFC PATCH v3 8/8] x86/entry/pti: don't switch PGD when TIF_DISABLE_PTI_NOW is set Willy Tarreau
2018-01-10 19:35 ` [RFC PATCH v3 0/8] Per process PTI activation Linus Torvalds

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1515612500-14505-8-git-send-email-w@1wt.eu \
    --to=w@1wt.eu \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=hpa@zytor.com \
    --cc=jpoimboe@redhat.com \
    --cc=keescook@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.