From: Lai Jiangshan <jiangshanlai@gmail.com>
To: linux-kernel@vger.kernel.org
Cc: x86@kernel.org, Borislav Petkov <bp@alien8.de>,
Peter Zijlstra <peterz@infradead.org>,
Lai Jiangshan <jiangshan.ljs@antgroup.com>,
Andy Lutomirski <luto@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
"H. Peter Anvin" <hpa@zytor.com>
Subject: [PATCH V3 3/7] x86/entry: Switch the stack after error_entry() returns
Date: Tue, 15 Mar 2022 15:39:45 +0800 [thread overview]
Message-ID: <20220315073949.7541-4-jiangshanlai@gmail.com> (raw)
In-Reply-To: <20220315073949.7541-1-jiangshanlai@gmail.com>
From: Lai Jiangshan <jiangshan.ljs@antgroup.com>
error_entry() calls sync_regs() to settle/copy the pt_regs and switches
the stack directly after sync_regs(). But error_entry() itself is also
a function call, the switching has to handle the return address of it
together, which causes the work complicated and tangly.
Switching to the stack after error_entry() makes the code simpler and
intuitive.
The behavior/logic is unchanged:
1) (opt) feed fixup_bad_iret() with the pt_regs pushed by ASM code
2) (opt) fixup_bad_iret() moves the partial pt_regs up
3) feed sync_regs() with the pt_regs pushed by ASM code or returned
by fixup_bad_iret()
4) sync_regs() copies the whole pt_regs to kernel stack if needed
5) after error_entry() and switching %rsp, it is in kernel stack with
the pt_regs
Changes only in calling:
Old code switches to copied pt_regs immediately twice in
error_entry() while new code switches to the copied pt_regs only
once after error_entry() returns.
It is correct since sync_regs() doesn't need to be called close
to the pt_regs it handles.
Old code stashes the return-address of error_entry() in a scratch
register and new code doesn't stash it.
It relies on the fact that fixup_bad_iret() and sync_regs() don't
corrupt the return-address of error_entry() on the stack. But even
the old code also relies on the fact that fixup_bad_iret() and
sync_regs() don't corrupt the return-address of themselves.
They are the same reliances and are assured.
After this change, error_entry() will not do fancy things with the stack
except when in the prolog which will be fixed in the next patch ("move
PUSH_AND_CLEAR_REGS out of error_entry"). This patch and the next patch
can't be swapped because the next patch relies on this patch's stopping
fiddling with the return-address of error_entry(), otherwise the objtool
would complain.
Signed-off-by: Lai Jiangshan <jiangshan.ljs@antgroup.com>
---
arch/x86/entry/entry_64.S | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 37505331b7f1..7768cdd0c7ed 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -326,6 +326,8 @@ SYM_CODE_END(ret_from_fork)
.macro idtentry_body cfunc has_error_code:req
call error_entry
+ movq %rax, %rsp /* switch stack settled by sync_regs() */
+ ENCODE_FRAME_POINTER
UNWIND_HINT_REGS
movq %rsp, %rdi /* pt_regs pointer into 1st argument*/
@@ -1014,14 +1016,10 @@ SYM_CODE_START_LOCAL(error_entry)
/* We have user CR3. Change to kernel CR3. */
SWITCH_TO_KERNEL_CR3 scratch_reg=%rax
+ leaq 8(%rsp), %rdi /* arg0 = pt_regs pointer */
.Lerror_entry_from_usermode_after_swapgs:
/* Put us onto the real thread stack. */
- popq %r12 /* save return addr in %12 */
- movq %rsp, %rdi /* arg0 = pt_regs pointer */
call sync_regs
- movq %rax, %rsp /* switch stack */
- ENCODE_FRAME_POINTER
- pushq %r12
RET
/*
@@ -1053,6 +1051,7 @@ SYM_CODE_START_LOCAL(error_entry)
*/
.Lerror_entry_done_lfence:
FENCE_SWAPGS_KERNEL_ENTRY
+ leaq 8(%rsp), %rax /* return pt_regs pointer */
RET
.Lbstep_iret:
@@ -1073,12 +1072,9 @@ SYM_CODE_START_LOCAL(error_entry)
* Pretend that the exception came from user mode: set up pt_regs
* as if we faulted immediately after IRET.
*/
- popq %r12 /* save return addr in %12 */
- movq %rsp, %rdi /* arg0 = pt_regs pointer */
+ leaq 8(%rsp), %rdi /* arg0 = pt_regs pointer */
call fixup_bad_iret
- mov %rax, %rsp
- ENCODE_FRAME_POINTER
- pushq %r12
+ mov %rax, %rdi
jmp .Lerror_entry_from_usermode_after_swapgs
SYM_CODE_END(error_entry)
--
2.19.1.6.gb485710b
next prev parent reply other threads:[~2022-03-15 7:39 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-15 7:39 [PATCH V3 0/7] x86/entry: Clean up entry code Lai Jiangshan
2022-03-15 7:39 ` [PATCH V3 1/7] x86/entry: Use idtentry macro for entry_INT80_compat Lai Jiangshan
2022-03-16 13:48 ` Peter Zijlstra
2022-03-15 7:39 ` [PATCH V3 2/7] x86/traps: Move pt_regs only in fixup_bad_iret() Lai Jiangshan
2022-03-15 7:39 ` Lai Jiangshan [this message]
2022-03-16 2:37 ` [PATCH V3 3/7] x86/entry: Switch the stack after error_entry() returns Josh Poimboeuf
2022-03-16 3:02 ` Lai Jiangshan
2022-03-16 15:05 ` Peter Zijlstra
2022-03-16 16:43 ` Lai Jiangshan
2022-03-18 11:12 ` Peter Zijlstra
2022-03-15 7:39 ` [PATCH V3 4/7] x86/entry: move PUSH_AND_CLEAR_REGS out of error_entry Lai Jiangshan
2022-03-16 15:07 ` Peter Zijlstra
2022-03-16 16:18 ` Lai Jiangshan
2022-03-15 7:39 ` [PATCH V3 5/7] x86/entry: Move cld to the start of idtentry Lai Jiangshan
2022-03-15 7:39 ` [PATCH V3 6/7] x86/entry: Don't call error_entry for XENPV Lai Jiangshan
2022-03-16 2:59 ` Josh Poimboeuf
2022-03-16 15:09 ` Peter Zijlstra
2022-03-16 16:48 ` Lai Jiangshan
2022-03-15 7:39 ` [PATCH V3 7/7] x86/entry: Convert SWAPGS to swapgs and remove the definition of SWAPGS Lai Jiangshan
2022-03-16 15:12 ` [PATCH V3 0/7] x86/entry: Clean up entry code Peter Zijlstra
2022-03-16 15:13 ` Peter Zijlstra
2022-03-17 12:58 ` Lai Jiangshan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220315073949.7541-4-jiangshanlai@gmail.com \
--to=jiangshanlai@gmail.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jiangshan.ljs@antgroup.com \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.