[PATCH 3/3] x86: add local_tlb_flush_kernel_range()

From: Seth Jennings <sjenning@linux.vnet.ibm.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Seth Jennings <sjenning@linux.vnet.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Dan Magenheimer <dan.magenheimer@oracle.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Nitin Gupta <ngupta@vflare.org>, Minchan Kim <minchan@kernel.org>,
	Robert Jennings <rcj@linux.vnet.ibm.com>,
	linux-mm@kvack.org, devel@driverdev.osuosl.org,
	linux-kernel@vger.kernel.org
Subject: [PATCH 3/3] x86: add local_tlb_flush_kernel_range()
Date: Mon, 25 Jun 2012 11:14:38 -0500	[thread overview]
Message-ID: <1340640878-27536-4-git-send-email-sjenning@linux.vnet.ibm.com> (raw)
In-Reply-To: <1340640878-27536-1-git-send-email-sjenning@linux.vnet.ibm.com>

This patch adds support for a local_tlb_flush_kernel_range()
function for the x86 arch.  This function allows for CPU-local
TLB flushing, potentially using invlpg for single entry flushing,
using an arch independent function name.

Signed-off-by: Seth Jennings <sjenning@linux.vnet.ibm.com>
---
 arch/x86/include/asm/tlbflush.h |   21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 36a1a2a..92a280b 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -168,4 +168,25 @@ static inline void flush_tlb_kernel_range(unsigned long start,
 	flush_tlb_all();
 }
 
+#define __HAVE_LOCAL_FLUSH_TLB_KERNEL_RANGE
+/*
+ * INVLPG_BREAK_EVEN_PAGES is the number of pages after which single tlb
+ * flushing becomes more costly than just doing a complete tlb flush.
+ * While this break even point varies among x86 hardware, tests have shown
+ * that 8 is a good generic value.
+*/
+#define INVLPG_BREAK_EVEN_PAGES 8
+static inline void local_flush_tlb_kernel_range(unsigned long start,
+		unsigned long end)
+{
+	if (cpu_has_invlpg &&
+		(end - start)/PAGE_SIZE <= INVLPG_BREAK_EVEN_PAGES) {
+		while (start < end) {
+			__flush_tlb_single(start);
+			start += PAGE_SIZE;
+		}
+	} else
+		local_flush_tlb();
+}
+
 #endif /* _ASM_X86_TLBFLUSH_H */
-- 
1.7.9.5

