From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757793AbbKFXNF (ORCPT ); Fri, 6 Nov 2015 18:13:05 -0500 Received: from mail.kernel.org ([198.145.29.136]:53653 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757751AbbKFXMz (ORCPT ); Fri, 6 Nov 2015 18:12:55 -0500 From: Andy Lutomirski To: x86@kernel.org, linux-kernel@vger.kernel.org Cc: Brian Gerst , Linus Torvalds , Borislav Petkov , =?UTF-8?q?Fr=C3=A9d=C3=A9ric=20Weisbecker?= , Peter Zijlstra , Andy Lutomirski Subject: [PATCH 4/4] x86/entry/64: Bypass enter_from_user_mode on non-context-tracking boots Date: Fri, 6 Nov 2015 15:12:46 -0800 Message-Id: <32bb09004dc9b005d5c14f021e42eb1f83ce3db3.1446849780.git.luto@kernel.org> X-Mailer: git-send-email 2.4.3 In-Reply-To: References: In-Reply-To: References: Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On CONFIG_CONTEXT_TRACKING kernels that have context tracking disabled at runtime (which includes most distro kernels), we still have the overhead of a call to enter_from_user_mode in interrupt and exception entries. If jump labels are available, this uses the jump label infrastructure to skip the call. Signed-off-by: Andy Lutomirski --- arch/x86/entry/calling.h | 15 +++++++++++++++ arch/x86/entry/entry_64.S | 8 ++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/arch/x86/entry/calling.h b/arch/x86/entry/calling.h index 3c71dd947c7b..271e30c585bc 100644 --- a/arch/x86/entry/calling.h +++ b/arch/x86/entry/calling.h @@ -1,3 +1,5 @@ +#include + /* x86 function call convention, 64-bit: @@ -232,3 +234,16 @@ For 32-bit we have the following conventions - kernel is built with #endif /* CONFIG_X86_64 */ +/* + * This does 'call enter_from_user_mode' unless we can avoid it based on + * kernel config or using the static jump infrastructure. + */ +.macro CALL_ENTER_FROM_USER_MODE +#ifdef CONFIG_CONTEXT_TRACKING +#ifdef HAVE_JUMP_LABEL + STATIC_JUMP_IF_FALSE .Lafter_call_\@, context_tracking_enabled, def=0 +#endif + call enter_from_user_mode +.Lafter_call_\@: +#endif +.endm diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index f585df24ab3d..9b49b56efa29 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -517,9 +517,7 @@ END(irq_entries_start) */ TRACE_IRQS_OFF -#ifdef CONFIG_CONTEXT_TRACKING - call enter_from_user_mode -#endif + CALL_ENTER_FROM_USER_MODE 1: /* @@ -1058,9 +1056,7 @@ ENTRY(error_entry) .Lerror_entry_from_usermode_after_swapgs: TRACE_IRQS_OFF -#ifdef CONFIG_CONTEXT_TRACKING - call enter_from_user_mode -#endif + CALL_ENTER_FROM_USER_MODE ret .Lerror_entry_done: -- 2.4.3