All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andy Lutomirski <luto@kernel.org>
To: x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Borislav Petkov <bp@alien8.de>, Brian Gerst <brgerst@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Oleg Nesterov <oleg@redhat.com>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	Andrey Ryabinin <aryabinin@virtuozzo.com>,
	Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v3 3/3] x86/mm: If INVPCID is available, use it to flush global mappings
Date: Fri, 29 Jan 2016 11:42:59 -0800	[thread overview]
Message-ID: <ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org> (raw)
In-Reply-To: <cover.1454096309.git.luto@kernel.org>
In-Reply-To: <cover.1454096309.git.luto@kernel.org>

On my Skylake laptop, INVPCID function 2 (flush absolutely
everything) takes about 376ns, whereas saving flags, twiddling
CR4.PGE to flush global mappings, and restoring flags takes about
539ns.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/include/asm/tlbflush.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 8b576832777e..985f1162c505 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -152,6 +152,15 @@ static inline void __native_flush_tlb_global(void)
 {
 	unsigned long flags;
 
+	if (static_cpu_has_safe(X86_FEATURE_INVPCID)) {
+		/*
+		 * Using INVPCID is considerably faster than a pair of writes
+		 * to CR4 sandwiched inside an IRQ flag save/restore.
+		 */
+		invpcid_flush_all();
+		return;
+	}
+
 	/*
 	 * Read-modify-write to CR4 - protect it from preemption and
 	 * from interrupts. (Use the raw variant because this code can
-- 
2.5.0

WARNING: multiple messages have this Message-ID (diff)
From: Andy Lutomirski <luto@kernel.org>
To: x86@kernel.org, linux-kernel@vger.kernel.org
Cc: Borislav Petkov <bp@alien8.de>, Brian Gerst <brgerst@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Oleg Nesterov <oleg@redhat.com>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	Andrey Ryabinin <aryabinin@virtuozzo.com>,
	Andy Lutomirski <luto@kernel.org>
Subject: [PATCH v3 3/3] x86/mm: If INVPCID is available, use it to flush global mappings
Date: Fri, 29 Jan 2016 11:42:59 -0800	[thread overview]
Message-ID: <ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org> (raw)
In-Reply-To: <cover.1454096309.git.luto@kernel.org>
In-Reply-To: <cover.1454096309.git.luto@kernel.org>

On my Skylake laptop, INVPCID function 2 (flush absolutely
everything) takes about 376ns, whereas saving flags, twiddling
CR4.PGE to flush global mappings, and restoring flags takes about
539ns.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
 arch/x86/include/asm/tlbflush.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 8b576832777e..985f1162c505 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -152,6 +152,15 @@ static inline void __native_flush_tlb_global(void)
 {
 	unsigned long flags;
 
+	if (static_cpu_has_safe(X86_FEATURE_INVPCID)) {
+		/*
+		 * Using INVPCID is considerably faster than a pair of writes
+		 * to CR4 sandwiched inside an IRQ flag save/restore.
+		 */
+		invpcid_flush_all();
+		return;
+	}
+
 	/*
 	 * Read-modify-write to CR4 - protect it from preemption and
 	 * from interrupts. (Use the raw variant because this code can
-- 
2.5.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2016-01-29 19:43 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-01-29 19:42 [PATCH v3 0/3] x86/mm: INVPCID support Andy Lutomirski
2016-01-29 19:42 ` Andy Lutomirski
2016-01-29 19:42 ` [PATCH v3 1/3] x86/mm: Add INVPCID helpers Andy Lutomirski
2016-01-29 19:42   ` Andy Lutomirski
2016-02-09 16:07   ` [tip:x86/mm] " tip-bot for Andy Lutomirski
2016-02-10 12:08     ` Borislav Petkov
2016-02-10 13:48       ` Michael Matz
2016-02-10 14:51         ` [PATCH -v1.1] x86/mm: Fix INVPCID asm constraint Borislav Petkov
2016-02-11  2:48           ` Andy Lutomirski
2016-01-29 19:42 ` [PATCH v3 2/3] x86/mm: Add a noinvpcid option to turn off INVPCID Andy Lutomirski
2016-01-29 19:42   ` Andy Lutomirski
2016-02-09 16:07   ` [tip:x86/mm] x86/mm: Add a 'noinvpcid' boot " tip-bot for Andy Lutomirski
2016-01-29 19:42 ` Andy Lutomirski [this message]
2016-01-29 19:42   ` [PATCH v3 3/3] x86/mm: If INVPCID is available, use it to flush global mappings Andy Lutomirski
2016-02-09 16:08   ` [tip:x86/mm] " tip-bot for Andy Lutomirski
2016-02-01 10:51 ` [PATCH v3 0/3] x86/mm: INVPCID support Borislav Petkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ed0ef62581c0ea9c99b9bf6df726015e96d44743.1454096309.git.luto@kernel.org \
    --to=luto@kernel.org \
    --cc=aryabinin@virtuozzo.com \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=dave.hansen@linux.intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=oleg@redhat.com \
    --cc=torvalds@linux-foundation.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.