From: tip-bot for Oleg Nesterov <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, hpa@zytor.com, mingo@kernel.org,
torvalds@linux-foundation.org, peterz@infradead.org,
luto@amacapital.net, dvlasenk@redhat.com, tglx@linutronix.de,
oleg@redhat.com
Subject: [tip:x86/asm] x86: Speed up ___preempt_schedule*() by using THUNK helpers
Date: Wed, 24 Sep 2014 08:02:06 -0700 [thread overview]
Message-ID: <tip-0ad6e3c5199be12c9745da8f8b9e3c9f8066c235@git.kernel.org> (raw)
In-Reply-To: <20140921184153.GA23727@redhat.com>
Commit-ID: 0ad6e3c5199be12c9745da8f8b9e3c9f8066c235
Gitweb: http://git.kernel.org/tip/0ad6e3c5199be12c9745da8f8b9e3c9f8066c235
Author: Oleg Nesterov <oleg@redhat.com>
AuthorDate: Sun, 21 Sep 2014 20:41:53 +0200
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Wed, 24 Sep 2014 15:15:38 +0200
x86: Speed up ___preempt_schedule*() by using THUNK helpers
___preempt_schedule() does SAVE_ALL/RESTORE_ALL but this is
suboptimal, we do not need to save/restore the callee-saved
register. And we already have arch/x86/lib/thunk_*.S which
implements the similar asm wrappers, so it makes sense to
redefine ___preempt_schedule() as "THUNK ..." and remove
preempt.S altogether.
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Andy Lutomirski <luto@amacapital.net>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: http://lkml.kernel.org/r/20140921184153.GA23727@redhat.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
arch/x86/kernel/Makefile | 2 --
arch/x86/kernel/preempt.S | 25 -------------------------
arch/x86/lib/thunk_32.S | 20 ++++++++++++++++----
arch/x86/lib/thunk_64.S | 7 +++++++
4 files changed, 23 insertions(+), 31 deletions(-)
diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
index ada2e2d..8f1e774 100644
--- a/arch/x86/kernel/Makefile
+++ b/arch/x86/kernel/Makefile
@@ -39,8 +39,6 @@ obj-y += tsc.o tsc_msr.o io_delay.o rtc.o
obj-y += pci-iommu_table.o
obj-y += resource.o
-obj-$(CONFIG_PREEMPT) += preempt.o
-
obj-y += process.o
obj-y += i387.o xsave.o
obj-y += ptrace.o
diff --git a/arch/x86/kernel/preempt.S b/arch/x86/kernel/preempt.S
deleted file mode 100644
index ca7f0d5..0000000
--- a/arch/x86/kernel/preempt.S
+++ /dev/null
@@ -1,25 +0,0 @@
-
-#include <linux/linkage.h>
-#include <asm/dwarf2.h>
-#include <asm/asm.h>
-#include <asm/calling.h>
-
-ENTRY(___preempt_schedule)
- CFI_STARTPROC
- SAVE_ALL
- call preempt_schedule
- RESTORE_ALL
- ret
- CFI_ENDPROC
-
-#ifdef CONFIG_CONTEXT_TRACKING
-
-ENTRY(___preempt_schedule_context)
- CFI_STARTPROC
- SAVE_ALL
- call preempt_schedule_context
- RESTORE_ALL
- ret
- CFI_ENDPROC
-
-#endif
diff --git a/arch/x86/lib/thunk_32.S b/arch/x86/lib/thunk_32.S
index 28f85c91..e9acf5f 100644
--- a/arch/x86/lib/thunk_32.S
+++ b/arch/x86/lib/thunk_32.S
@@ -7,16 +7,19 @@
#include <linux/linkage.h>
#include <asm/asm.h>
-#ifdef CONFIG_TRACE_IRQFLAGS
/* put return address in eax (arg1) */
- .macro thunk_ra name,func
+ .macro THUNK name, func, put_ret_addr_in_eax=0
.globl \name
\name:
pushl %eax
pushl %ecx
pushl %edx
+
+ .if \put_ret_addr_in_eax
/* Place EIP in the arg1 */
movl 3*4(%esp), %eax
+ .endif
+
call \func
popl %edx
popl %ecx
@@ -25,6 +28,15 @@
_ASM_NOKPROBE(\name)
.endm
- thunk_ra trace_hardirqs_on_thunk,trace_hardirqs_on_caller
- thunk_ra trace_hardirqs_off_thunk,trace_hardirqs_off_caller
+#ifdef CONFIG_TRACE_IRQFLAGS
+ THUNK trace_hardirqs_on_thunk,trace_hardirqs_on_caller,1
+ THUNK trace_hardirqs_off_thunk,trace_hardirqs_off_caller,1
+#endif
+
+#ifdef CONFIG_PREEMPT
+ THUNK ___preempt_schedule, preempt_schedule
+#ifdef CONFIG_CONTEXT_TRACKING
+ THUNK ___preempt_schedule_context, preempt_schedule_context
#endif
+#endif
+
diff --git a/arch/x86/lib/thunk_64.S b/arch/x86/lib/thunk_64.S
index 92d9fea..b30b5eb 100644
--- a/arch/x86/lib/thunk_64.S
+++ b/arch/x86/lib/thunk_64.S
@@ -38,6 +38,13 @@
THUNK lockdep_sys_exit_thunk,lockdep_sys_exit
#endif
+#ifdef CONFIG_PREEMPT
+ THUNK ___preempt_schedule, preempt_schedule
+#ifdef CONFIG_CONTEXT_TRACKING
+ THUNK ___preempt_schedule_context, preempt_schedule_context
+#endif
+#endif
+
/* SAVE_ARGS below is used only for the .cfi directives it contains. */
CFI_STARTPROC
SAVE_ARGS
next prev parent reply other threads:[~2014-09-24 15:04 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-09-21 18:41 [PATCH v3 0/2] x86: reimplement ___preempt_schedule*() using THUNK helpers Oleg Nesterov
2014-09-21 18:41 ` [PATCH v3 1/2] " Oleg Nesterov
2014-09-24 15:02 ` tip-bot for Oleg Nesterov [this message]
2014-10-03 4:50 ` [tip:x86/asm] x86: Speed up ___preempt_schedule*() by " Sasha Levin
2014-10-03 13:39 ` Chuck Ebbert
2014-10-03 21:41 ` Oleg Nesterov
2014-10-03 21:56 ` Andy Lutomirski
2014-10-03 23:48 ` Linus Torvalds
2014-10-03 23:51 ` Oleg Nesterov
2014-10-03 22:48 ` Chuck Ebbert
2014-10-03 22:53 ` Andy Lutomirski
2014-10-03 23:13 ` H. Peter Anvin
2014-10-03 23:37 ` Oleg Nesterov
2014-10-03 21:16 ` Oleg Nesterov
2014-10-03 23:26 ` Oleg Nesterov
2014-10-04 0:01 ` Linus Torvalds
2014-10-04 0:11 ` Linus Torvalds
2014-10-04 0:33 ` Oleg Nesterov
2014-10-05 20:23 ` [PATCH 0/1] stop the unbound recursion in preempt_schedule_context() Oleg Nesterov
2014-10-05 20:23 ` [PATCH 1/1] " Oleg Nesterov
2014-10-28 11:03 ` [tip:sched/core] sched: " tip-bot for Oleg Nesterov
2014-10-05 23:53 ` [PATCH 0/1] " Oleg Nesterov
2014-10-04 0:19 ` [tip:x86/asm] x86: Speed up ___preempt_schedule*() by using THUNK helpers Oleg Nesterov
2014-09-21 18:42 ` [PATCH v3 2/2] x86, lib/Makefile: remove the unnecessary "+= thunk_64.o" Oleg Nesterov
2014-09-24 15:02 ` [tip:x86/asm] x86/lib/Makefile: Remove " tip-bot for Oleg Nesterov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=tip-0ad6e3c5199be12c9745da8f8b9e3c9f8066c235@git.kernel.org \
--to=tipbot@zytor.com \
--cc=dvlasenk@redhat.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).