LKML Archive on lore.kernel.org
 help / color / Atom feed
From: tip-bot for Andy Lutomirski <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: hpa@zytor.com, oleg@redhat.com, luto@kernel.org,
	fweisbec@gmail.com, dvlasenk@redhat.com,
	linux-kernel@vger.kernel.org, riel@redhat.com,
	tglx@linutronix.de, peterz@infradead.org, keescook@chromium.org,
	brgerst@gmail.com, mingo@kernel.org, vda.linux@googlemail.com,
	torvalds@linux-foundation.org, luto@amacapital.net, bp@alien8.de
Subject: [tip:x86/asm] x86/entry: Add new, comprehensible entry and exit handlers written in C
Date: Tue, 7 Jul 2015 03:51:48 -0700
Message-ID: <tip-c5c46f59e4e7c1ab244b8d38f2b61d317df90bba@git.kernel.org> (raw)
In-Reply-To: <c57c8b87661a4152801d7d3786eac2d1a2f209dd.1435952415.git.luto@kernel.org>

Commit-ID:  c5c46f59e4e7c1ab244b8d38f2b61d317df90bba
Gitweb:     http://git.kernel.org/tip/c5c46f59e4e7c1ab244b8d38f2b61d317df90bba
Author:     Andy Lutomirski <luto@kernel.org>
AuthorDate: Fri, 3 Jul 2015 12:44:26 -0700
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Tue, 7 Jul 2015 10:59:06 +0200

x86/entry: Add new, comprehensible entry and exit handlers written in C

The current x86 entry and exit code, written in a mixture of assembly and
C code, is incomprehensible due to being open-coded in a lot of places
without coherent documentation.

It appears to work primary by luck and duct tape: i.e. obvious runtime
failures were fixed on-demand, without re-thinking the design.

Due to those reasons our confidence level in that code is low, and it is
very difficult to incrementally improve.

Add new code written in C, in preparation for simply deleting the old
entry code.

prepare_exit_to_usermode() is a new function that will handle all
slow path exits to user mode.  It is called with IRQs disabled
and it leaves us in a state in which it is safe to immediately
return to user mode.  IRQs must not be re-enabled at any point
after prepare_exit_to_usermode() returns and user mode is actually
entered. (We can, of course, fail to enter user mode and treat
that failure as a fresh entry to kernel mode.)

All callers of do_notify_resume() will be migrated to call
prepare_exit_to_usermode() instead; prepare_exit_to_usermode() needs
to do everything that do_notify_resume() does today, but it also
takes care of scheduling and context tracking.  Unlike
do_notify_resume(), it does not need to be called in a loop.

syscall_return_slowpath() is exactly what it sounds like: it will
be called on any syscall exit slow path. It will replace
syscall_trace_leave() and it calls prepare_exit_to_usermode() on the
way out.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Denys Vlasenko <vda.linux@googlemail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: paulmck@linux.vnet.ibm.com
Link: http://lkml.kernel.org/r/c57c8b87661a4152801d7d3786eac2d1a2f209dd.1435952415.git.luto@kernel.org
[ Improved the changelog a bit. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/entry/common.c | 112 +++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 111 insertions(+), 1 deletion(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 9a327ee..febc530 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -207,6 +207,7 @@ long syscall_trace_enter(struct pt_regs *regs)
 		return syscall_trace_enter_phase2(regs, arch, phase1_result);
 }
 
+/* Deprecated. */
 void syscall_trace_leave(struct pt_regs *regs)
 {
 	bool step;
@@ -237,8 +238,117 @@ void syscall_trace_leave(struct pt_regs *regs)
 	user_enter();
 }
 
+static struct thread_info *pt_regs_to_thread_info(struct pt_regs *regs)
+{
+	unsigned long top_of_stack =
+		(unsigned long)(regs + 1) + TOP_OF_KERNEL_STACK_PADDING;
+	return (struct thread_info *)(top_of_stack - THREAD_SIZE);
+}
+
+/* Called with IRQs disabled. */
+__visible void prepare_exit_to_usermode(struct pt_regs *regs)
+{
+	if (WARN_ON(!irqs_disabled()))
+		local_irq_disable();
+
+	/*
+	 * In order to return to user mode, we need to have IRQs off with
+	 * none of _TIF_SIGPENDING, _TIF_NOTIFY_RESUME, _TIF_USER_RETURN_NOTIFY,
+	 * _TIF_UPROBE, or _TIF_NEED_RESCHED set.  Several of these flags
+	 * can be set at any time on preemptable kernels if we have IRQs on,
+	 * so we need to loop.  Disabling preemption wouldn't help: doing the
+	 * work to clear some of the flags can sleep.
+	 */
+	while (true) {
+		u32 cached_flags =
+			READ_ONCE(pt_regs_to_thread_info(regs)->flags);
+
+		if (!(cached_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_RESUME |
+				      _TIF_UPROBE | _TIF_NEED_RESCHED)))
+			break;
+
+		/* We have work to do. */
+		local_irq_enable();
+
+		if (cached_flags & _TIF_NEED_RESCHED)
+			schedule();
+
+		if (cached_flags & _TIF_UPROBE)
+			uprobe_notify_resume(regs);
+
+		/* deal with pending signal delivery */
+		if (cached_flags & _TIF_SIGPENDING)
+			do_signal(regs);
+
+		if (cached_flags & _TIF_NOTIFY_RESUME) {
+			clear_thread_flag(TIF_NOTIFY_RESUME);
+			tracehook_notify_resume(regs);
+		}
+
+		if (cached_flags & _TIF_USER_RETURN_NOTIFY)
+			fire_user_return_notifiers();
+
+		/* Disable IRQs and retry */
+		local_irq_disable();
+	}
+
+	user_enter();
+}
+
+/*
+ * Called with IRQs on and fully valid regs.  Returns with IRQs off in a
+ * state such that we can immediately switch to user mode.
+ */
+__visible void syscall_return_slowpath(struct pt_regs *regs)
+{
+	struct thread_info *ti = pt_regs_to_thread_info(regs);
+	u32 cached_flags = READ_ONCE(ti->flags);
+	bool step;
+
+	CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
+
+	if (WARN(irqs_disabled(), "syscall %ld left IRQs disabled",
+		 regs->orig_ax))
+		local_irq_enable();
+
+	/*
+	 * First do one-time work.  If these work items are enabled, we
+	 * want to run them exactly once per syscall exit with IRQs on.
+	 */
+	if (cached_flags & (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT |
+			    _TIF_SINGLESTEP | _TIF_SYSCALL_TRACEPOINT)) {
+		audit_syscall_exit(regs);
+
+		if (cached_flags & _TIF_SYSCALL_TRACEPOINT)
+			trace_sys_exit(regs, regs->ax);
+
+		/*
+		 * If TIF_SYSCALL_EMU is set, we only get here because of
+		 * TIF_SINGLESTEP (i.e. this is PTRACE_SYSEMU_SINGLESTEP).
+		 * We already reported this syscall instruction in
+		 * syscall_trace_enter().
+		 */
+		step = unlikely(
+			(cached_flags & (_TIF_SINGLESTEP | _TIF_SYSCALL_EMU))
+			== _TIF_SINGLESTEP);
+		if (step || cached_flags & _TIF_SYSCALL_TRACE)
+			tracehook_report_syscall_exit(regs, step);
+	}
+
+#ifdef CONFIG_COMPAT
+	/*
+	 * Compat syscalls set TS_COMPAT.  Make sure we clear it before
+	 * returning to user mode.
+	 */
+	ti->status &= ~TS_COMPAT;
+#endif
+
+	local_irq_disable();
+	prepare_exit_to_usermode(regs);
+}
+
 /*
- * notification of userspace execution resumption
+ * Deprecated notification of userspace execution resumption
  * - triggered by the TIF_WORK_MASK flags
  */
 __visible void

  reply index

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-07-03 19:44 [PATCH v5 00/17] x86: Rewrite exit-to-userspace code Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 01/17] selftests/x86: Add a test for 32-bit fast syscall arg faults Andy Lutomirski
2015-07-07 10:49   ` [tip:x86/asm] x86/entry, " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 02/17] x86/entry/64/compat: Fix bad fast syscall arg failure path Andy Lutomirski
2015-07-07 10:49   ` [tip:x86/asm] " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 03/17] uml: Fix do_signal() prototype Andy Lutomirski
2015-07-07 10:49   ` [tip:x86/asm] um: " tip-bot for Ingo Molnar
2015-07-03 19:44 ` [PATCH v5 04/17] context_tracking: Add ct_state and CT_WARN_ON Andy Lutomirski
2015-07-07 10:50   ` [tip:x86/asm] context_tracking: Add ct_state() and CT_WARN_ON() tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 05/17] notifiers: Assert that RCU is watching in notify_die Andy Lutomirski
2015-07-07 10:50   ` [tip:x86/asm] notifiers, RCU: Assert that RCU is watching in notify_die() tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 06/17] x86: Move C entry and exit code to arch/x86/entry/common.c Andy Lutomirski
2015-07-07 10:50   ` [tip:x86/asm] x86/entry: Move C entry and exit code to arch/x86/ entry/common.c tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 07/17] x86/traps: Assert that we're in CONTEXT_KERNEL in exception entries Andy Lutomirski
2015-07-07 10:51   ` [tip:x86/asm] x86/traps, context_tracking: Assert that we' re " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 08/17] x86/entry: Add enter_from_user_mode and use it in syscalls Andy Lutomirski
2015-07-07 10:51   ` [tip:x86/asm] x86/entry: Add enter_from_user_mode() " tip-bot for Andy Lutomirski
2015-07-14 23:00     ` Frederic Weisbecker
2015-07-14 23:04       ` Andy Lutomirski
2015-07-14 23:28         ` Frederic Weisbecker
2015-12-21 20:50   ` [PATCH v5 08/17] x86/entry: Add enter_from_user_mode " Sasha Levin
2015-12-21 22:44     ` Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 09/17] x86/entry: Add new, comprehensible entry and exit hooks Andy Lutomirski
2015-07-07 10:51   ` tip-bot for Andy Lutomirski [this message]
2015-07-14 23:07     ` [tip:x86/asm] x86/entry: Add new, comprehensible entry and exit handlers written in C Frederic Weisbecker
2015-07-15 19:56       ` Linus Torvalds
2015-07-15 20:46         ` Andy Lutomirski
2015-07-15 21:25           ` [PATCH] x86/entry: Fix _TIF_USER_RETURN_NOTIFY check in prepare_exit_to_usermode Andy Lutomirski
2015-07-18  3:25             ` [tip:x86/asm] " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 10/17] x86/entry/64: Really create an error-entry-from-usermode code path Andy Lutomirski
2015-07-07 10:52   ` [tip:x86/asm] " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 11/17] x86/entry/64: Migrate 64-bit and compat syscalls to new exit hooks Andy Lutomirski
2015-07-07 10:52   ` [tip:x86/asm] x86/entry/64: Migrate 64-bit and compat syscalls to the new exit handlers and remove old assembly code tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 12/17] x86/asm/entry/64: Save all regs on interrupt entry Andy Lutomirski
2015-07-07 10:52   ` [tip:x86/asm] " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 13/17] x86/asm/entry/64: Simplify irq stack pt_regs handling Andy Lutomirski
2015-07-07 10:53   ` [tip:x86/asm] x86/asm/entry/64: Simplify IRQ " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 14/17] x86/asm/entry/64: Migrate error and interrupt exit work to C Andy Lutomirski
2015-07-07 10:53   ` [tip:x86/asm] x86/asm/entry/64: Migrate error and IRQ exit work to C and remove old assembly code tip-bot for Andy Lutomirski
2015-08-11 22:18     ` Frederic Weisbecker
2015-08-11 22:25       ` Andy Lutomirski
2015-08-11 22:49         ` Frederic Weisbecker
2015-08-11 22:59           ` Andy Lutomirski
2015-08-12  1:02             ` Paul E. McKenney
2015-08-12 13:13             ` Frederic Weisbecker
2015-08-11 22:38     ` Frederic Weisbecker
2015-08-11 22:51       ` Andy Lutomirski
2015-08-11 23:22         ` Frederic Weisbecker
2015-08-11 23:33           ` Andy Lutomirski
2015-08-12 13:32             ` Frederic Weisbecker
2015-08-12 14:59               ` Andy Lutomirski
2015-08-18 22:34                 ` Frederic Weisbecker
2015-08-18 22:40                   ` Andy Lutomirski
2015-08-19 17:18                     ` Frederic Weisbecker
2015-08-19 18:02                       ` Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 15/17] x86/entry: Remove exception_enter from most trap handlers Andy Lutomirski
2015-07-07 10:53   ` [tip:x86/asm] x86/entry: Remove exception_enter() " tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 16/17] x86/entry: Remove SCHEDULE_USER and asm/context-tracking.h Andy Lutomirski
2015-07-07 10:54   ` [tip:x86/asm] x86/entry: Remove SCHEDULE_USER and asm/ context-tracking.h tip-bot for Andy Lutomirski
2015-07-03 19:44 ` [PATCH v5 17/17] x86/irq: Document how IRQ context tracking works and add an assertion Andy Lutomirski
2015-07-07 10:54   ` [tip:x86/asm] x86/irq, context_tracking: Document how IRQ context tracking works and add an RCU assertion tip-bot for Andy Lutomirski
2015-07-14 23:26     ` Frederic Weisbecker
2015-07-14 23:33       ` Andy Lutomirski
2015-07-18 13:23         ` Frederic Weisbecker
2015-07-18 14:10           ` Paul E. McKenney
2015-07-07 11:12 ` [PATCH v5 00/17] x86: Rewrite exit-to-userspace code Ingo Molnar
2015-07-07 16:03   ` Andy Lutomirski
2015-07-07 17:55     ` [PATCH] x86/entry/64: Fix warning on compat syscalls with CONFIG_AUDITSYSCALL=n Andy Lutomirski
2015-07-08  9:57       ` [tip:x86/asm] x86/entry/64: Fix IRQ state confusion and related warning on compat syscalls with CONFIG_AUDITSYSCALL =n tip-bot for Andy Lutomirski
2015-07-08 19:12       ` tip-bot for Andy Lutomirski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tip-c5c46f59e4e7c1ab244b8d38f2b61d317df90bba@git.kernel.org \
    --to=tipbot@zytor.com \
    --cc=bp@alien8.de \
    --cc=brgerst@gmail.com \
    --cc=dvlasenk@redhat.com \
    --cc=fweisbec@gmail.com \
    --cc=hpa@zytor.com \
    --cc=keescook@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=luto@amacapital.net \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=oleg@redhat.com \
    --cc=peterz@infradead.org \
    --cc=riel@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=vda.linux@googlemail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git