Re: [tip:x86/asm] x86/asm/entry/64: Migrate error and IRQ exit work to C and remove old assembly code

[Andy Lutomirski <luto@amacapital.net>]

On Wed, Aug 19, 2015 at 10:18 AM, Frederic Weisbecker
<fweisbec@gmail.com> wrote:
> On Tue, Aug 18, 2015 at 03:40:20PM -0700, Andy Lutomirski wrote:
>>
>> I sure hope not, unless it nests inside an NMI-like thing.  It's
>> conceivable that this might happen due to perf NMIs causing a failed
>> MSR read or similar.  We might need to relax the assertions to check
>> that we're either in kernel or NMI context.  If so, that's
>> straightforward.  Meanwhile no one has reported this happening.
>
> But we can still have #DB on entry code right? We blocked breakpoints on entry
> code (I still don't get why and it looks to me like an overkill) but we still
> have watchpoints.

The actual reason is buried in the many threads about NMIs.
Basically, we want to start using RET to return from exceptions to
contexts with IF=0, but we can't do that if we need RF to work
correctly, and we need RF to work correctly if we allow breakpoints in
entry asm (otherwise we risk random infinite loops).  So we're
disallowing breakpoints in entry asm.

> But an exception slow path based on static key would the most lightweight
> thing for context tracking off-case (which is 99.9999% of usecases) and we
> would keep it robust (ie: no need to enumerate all the fragile non-possibility
> for an exception in entry code to get it safe).
>

IRQs work more or less like this in -tip (restructured, but this gets the gist):

if (user_mode(regs)) {
  swapgs;
  enter_from_user_mode;
  do_IRQ;
  prepare_exit_to_usermode;
  swapgs;
  iret;
} else {
  do_IRQ;
  check for preemption;
  iret;
}

In 4.2 and before, the enter_from_user_mode wasn't there, and instead
of calling prepare_exit_to_usermode in a known context
(CONTEXT_KERNEL), we went through the maze of retint_user in an
unknown context.  That meant that we needed things like SCHEDULE_USER
(which had a bug at some point), do_notify_resume (probably had tons
of bugs), etc, and somehow we still needed to end up in CONTEXT_USER
at the end.

I think the new state of affairs is much nicer.  It means that we
finally actually know what state we're in throughout the entry asm.
The only real downsides that I can see are:

1. There's an unnecessary pair of branches due to rcu_irq_enter and
rcu_irq_exit when an IRQ hits user mode.

2. If user_exit is indeed much more expensive than rcu_irq_enter, then
we pay that cost.

If you have suggestions for how to make this faster without making it
uglier, please let me know. :)

--Andy