From: Lai Jiangshan <laijs@linux.alibaba.com>
To: linux-kernel@vger.kernel.org
Cc: Lai Jiangshan <laijs@linux.alibaba.com>,
Andy Lutomirski <luto@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH 2/5] x86/entry: avoid calling into sync_regs() when entering from userspace
Date: Wed, 27 May 2020 07:31:04 +0000 [thread overview]
Message-ID: <20200527073107.2127-3-laijs@linux.alibaba.com> (raw)
In-Reply-To: <20200527073107.2127-1-laijs@linux.alibaba.com>
7f2590a110b8("x86/entry/64: Use a per-CPU trampoline stack for IDT entries")
made a change that when any exception happens on userspace, the
entry code will save the pt_regs on the sp0 stack, and then copy it
to the thread stack via sync_regs() and switch to thread stack
afterward.
This is hot path, such overhead should be avoided. This patch
borrows the way how interrupt_entry handles it.
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
---
arch/x86/entry/entry_64.S | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 5e983506f82e..e8817ae31390 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -1321,19 +1321,13 @@ SYM_CODE_END(paranoid_exit)
SYM_CODE_START_LOCAL(error_entry)
UNWIND_HINT_FUNC
cld
- PUSH_AND_CLEAR_REGS save_ret=1
- ENCODE_FRAME_POINTER 8
- testb $3, CS+8(%rsp)
+ testb $3, CS-ORIG_RAX+8(%rsp)
jz .Lerror_kernelspace
- /*
- * We entered from user mode or we're pretending to have entered
- * from user mode due to an IRET fault.
- */
- SWAPGS
- FENCE_SWAPGS_USER_ENTRY
- /* We have user CR3. Change to kernel CR3. */
- SWITCH_TO_KERNEL_CR3 scratch_reg=%rax
+ idtentry_swapgs_and_switch_to_kernel_stack
+ PUSH_AND_CLEAR_REGS save_ret=1
+ ENCODE_FRAME_POINTER 8
+ ret
.Lerror_entry_from_usermode_after_swapgs:
/* Put us onto the real thread stack. */
@@ -1357,6 +1351,8 @@ SYM_CODE_START_LOCAL(error_entry)
* for these here too.
*/
.Lerror_kernelspace:
+ PUSH_AND_CLEAR_REGS save_ret=1
+ ENCODE_FRAME_POINTER 8
leaq native_irq_return_iret(%rip), %rcx
cmpq %rcx, RIP+8(%rsp)
je .Lerror_bad_iret
--
2.20.1
next prev parent reply other threads:[~2020-05-27 7:31 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-27 7:31 [PATCH 0/5] x86/entry: simply stack switching when exception on userspace Lai Jiangshan
2020-05-27 7:31 ` [PATCH 1/5] x86/entry: introduce macro idtentry_swapgs_and_switch_to_kernel_stack Lai Jiangshan
2020-05-28 19:12 ` Thomas Gleixner
2020-05-29 8:26 ` [PATCH V2 0/4] x86/entry: simply stack switching when exception on userspace Lai Jiangshan
2020-05-29 8:26 ` [PATCH V2 1/4] x86/entry: avoid calling into sync_regs() when entering from userspace Lai Jiangshan
2020-05-29 8:26 ` [PATCH V2 2/4] x86/entry: directly switch to kernel stack when .Lerror_bad_iret Lai Jiangshan
2020-05-29 8:26 ` [PATCH V2 3/4] x86/entry: remove unused sync_regs() Lai Jiangshan
2020-05-29 8:26 ` [PATCH V2 4/4] x86/entry: don't copy to tmp in fixup_bad_iret Lai Jiangshan
2020-05-29 18:32 ` [PATCH V2 0/4] x86/entry: simply stack switching when exception on userspace Andy Lutomirski
2020-06-16 1:56 ` Lai Jiangshan
2020-06-18 13:52 ` Lai Jiangshan
2020-06-18 14:10 ` Thomas Gleixner
2020-06-27 21:03 ` Andy Lutomirski
[not found] ` <20200817062355.2884-1-jiangshanlai@gmail.com>
2020-08-17 5:31 ` [PATCH V3 0/3] " Lai Jiangshan
2020-09-10 10:12 ` Lai Jiangshan
2020-08-17 6:23 ` [PATCH V3 1/3] x86/entry: avoid calling into sync_regs() when entering from userspace Lai Jiangshan
2020-09-11 21:24 ` Jann Horn
2020-09-15 7:55 ` Lai Jiangshan
2020-08-17 6:23 ` [PATCH V3 2/3] x86/entry: directly switch to kernel stack when .Lerror_bad_iret Lai Jiangshan
2020-08-17 6:23 ` [PATCH V3 3/3] x86/entry: remove unused sync_regs() Lai Jiangshan
2020-05-27 7:31 ` Lai Jiangshan [this message]
2020-05-27 7:31 ` [PATCH 3/5] x86/entry: directly switch to kernel stack when .Lerror_bad_iret Lai Jiangshan
2020-05-27 7:31 ` [PATCH 4/5] x86/entry: remove unused sync_regs() Lai Jiangshan
2020-05-27 7:31 ` [PATCH 5/5] x86/entry: don't copy to tmp in fixup_bad_iret Lai Jiangshan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200527073107.2127-3-laijs@linux.alibaba.com \
--to=laijs@linux.alibaba.com \
--cc=bp@alien8.de \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).