From: Andy Lutomirski <luto@amacapital.net>
To: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andy Lutomirski <luto@kernel.org>, X86 ML <x86@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
live-patching@vger.kernel.org,
Linus Torvalds <torvalds@linux-foundation.org>,
Jiri Slaby <jslaby@suse.cz>, Ingo Molnar <mingo@kernel.org>,
"H. Peter Anvin" <hpa@zytor.com>,
Peter Zijlstra <peterz@infradead.org>,
Mike Galbraith <efault@gmx.de>
Subject: Re: [PATCH v2 6/8] x86/entry: add unwind hint annotations
Date: Thu, 29 Jun 2017 15:59:04 -0700 [thread overview]
Message-ID: <F1E11F3A-039D-48B2-A57D-7881E93028DD@amacapital.net> (raw)
In-Reply-To: <20170629214134.c36krjhvzegwkfjk@treble>
--Andy
> On Jun 29, 2017, at 2:41 PM, Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>
>> On Thu, Jun 29, 2017 at 02:09:54PM -0700, Andy Lutomirski wrote:
>>> On Thu, Jun 29, 2017 at 12:05 PM, Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>>>> On Thu, Jun 29, 2017 at 11:50:18AM -0700, Andy Lutomirski wrote:
>>>>> On Thu, Jun 29, 2017 at 10:53 AM, Josh Poimboeuf <jpoimboe@redhat.com> wrote:
>>>>> There's a bug here that will need a small change to the entry code.
>>>>>
>>>>> Mike Galbraith reported:
>>>>>
>>>>> WARNING: can't dereference registers at ffffc900089d7e08 for ip ffffffff81740bbb
>>>>>
>>>>> After some looking I found that it's caused by the following code
>>>>> snippet in the 'interrupt' macro in entry_64.S:
>>>>>
>>>>> /*
>>>>> * Save previous stack pointer, optionally switch to interrupt stack.
>>>>> * irq_count is used to check if a CPU is already on an interrupt stack
>>>>> * or not. While this is essentially redundant with preempt_count it is
>>>>> * a little cheaper to use a separate counter in the PDA (short of
>>>>> * moving irq_enter into assembly, which would be too much work)
>>>>> */
>>>>> movq %rsp, %rdi
>>>>> incl PER_CPU_VAR(irq_count)
>>>>> cmovzq PER_CPU_VAR(irq_stack_ptr), %rsp
>>>>> UNWIND_HINT_REGS base=rdi
>>>>> pushq %rdi
>>>>> UNWIND_HINT_REGS indirect=1
>>>>>
>>>>> The problem is that it's changing the stack pointer *before* writing the
>>>>> previous stack pointer (push %rdi). So when unwinding from an NMI which
>>>>> hit between the rsp write and the rdi push, the unwinder tries to access
>>>>> the regs on the previous stack (by reading rdi), but the previous stack
>>>>> pointer isn't there yet, so the access is considered out of bounds.
>>>>
>>>> Ugh, that code. Does this problem go away with this patch applied:
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git/commit/?h=x86/entry_ist&id=2231ec7e0bcc1a2bc94a17081511ab54cc6badd1
>>>>
>>>> If so, want to update the patch for new kernels (shouldn't conflict
>>>> with anything except your unwind hints)?
>>>
>>> I don't think that patch will fix it, because it still updates rsp
>>> *before* writing the old rsp on the new stack. So there's still a
>>> window where the "previous stack" pointer is missing.
>>
>> But it's in a register. Is undwarf not able to grok that?
>
> Sorry, I didn't explain it very well. Undwarf can find the regs pointer
> in rdi, it just doesn't trust its value.
>
> See the stack_info.next_sp field, which is set in in_irq_stack():
>
> /*
> * The next stack pointer is the first thing pushed by the entry code
> * after switching to the irq stack.
> */
> info->next_sp = (unsigned long *)*(end - 1);
>
> It's a safety mechanism. The unwinder needs the last word of the irq
> stack page to point to the previous stack. That way it can double check
> that the stack pointer it calculates is within the bounds of either the
> current stack or the previous stack.
>
> In the above code, the previous stack pointer (or next stack pointer,
> depending on your perspective) hasn't been set up before it switches
> stacks. So the unwinder reads an uninitialized value into
> info->next_sp, and compares that with the regs pointer, and then stops
> the unwind because it thinks it went off into the weeds.
>
That should be manageable, though, I think. With my patch applied (and maybe even without it), the only exception to that rule is if regs->sp points just above the top of the IRQ stack and the next instruction is push reg. In that case, the reg is exactly as trustworthy as the normal rule.* Can you teach the unwinding code that this is okay?
* If an NMI hits right there, then it relies on unwinding out of the NMI correctly. But the usual checks that the target stack is a valid stack should prevent us from going off into the weeds regardless.
> --
> Josh
next prev parent reply other threads:[~2017-06-29 22:59 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-06-28 15:11 [PATCH v2 0/8] x86: undwarf unwinder Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 1/8] objtool: move checking code to check.c Josh Poimboeuf
2017-06-30 13:12 ` [tip:core/objtool] objtool: Move " tip-bot for Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 2/8] objtool, x86: add several functions and files to the objtool whitelist Josh Poimboeuf
2017-06-30 13:12 ` [tip:core/objtool] objtool, x86: Add " tip-bot for Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 3/8] objtool: stack validation 2.0 Josh Poimboeuf
2017-06-30 8:32 ` Ingo Molnar
2017-06-30 13:23 ` Josh Poimboeuf
2017-06-30 13:26 ` Josh Poimboeuf
2017-06-30 14:09 ` [PATCH] objtool: silence warnings for functions which use iret Josh Poimboeuf
2017-06-30 17:49 ` [tip:core/objtool] objtool: Silence warnings for functions which use IRET tip-bot for Josh Poimboeuf
2017-06-30 13:13 ` [tip:core/objtool] objtool: Implement stack validation 2.0 tip-bot for Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 4/8] objtool: add undwarf debuginfo generation Josh Poimboeuf
2017-06-29 7:14 ` Ingo Molnar
2017-06-29 13:40 ` Josh Poimboeuf
2017-06-29 7:25 ` Ingo Molnar
2017-06-29 14:04 ` Josh Poimboeuf
2017-06-29 14:46 ` Ingo Molnar
2017-06-29 15:06 ` Josh Poimboeuf
2017-07-06 20:36 ` Josh Poimboeuf
2017-07-07 9:44 ` Ingo Molnar
2017-07-11 2:58 ` Josh Poimboeuf
2017-07-11 8:40 ` Ingo Molnar
2017-06-28 15:11 ` [PATCH v2 5/8] objtool, x86: add facility for asm code to provide unwind hints Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 6/8] x86/entry: add unwind hint annotations Josh Poimboeuf
2017-06-29 17:53 ` Josh Poimboeuf
2017-06-29 18:50 ` Andy Lutomirski
2017-06-29 19:05 ` Josh Poimboeuf
2017-06-29 21:09 ` Andy Lutomirski
2017-06-29 21:41 ` Josh Poimboeuf
2017-06-29 22:59 ` Andy Lutomirski [this message]
2017-06-30 2:12 ` Josh Poimboeuf
2017-06-30 5:05 ` Andy Lutomirski
2017-06-30 5:41 ` Andy Lutomirski
2017-06-30 13:11 ` Josh Poimboeuf
2017-06-30 15:44 ` Andy Lutomirski
2017-06-30 15:55 ` Josh Poimboeuf
2017-06-30 15:56 ` Andy Lutomirski
2017-06-30 16:16 ` Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 7/8] x86/asm: add unwind hint annotations to sync_core() Josh Poimboeuf
2017-06-28 15:11 ` [PATCH v2 8/8] x86/unwind: add undwarf unwinder Josh Poimboeuf
2017-06-29 7:55 ` [PATCH v2 0/8] x86: " Ingo Molnar
2017-06-29 14:12 ` Josh Poimboeuf
2017-06-29 19:13 ` Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=F1E11F3A-039D-48B2-A57D-7881E93028DD@amacapital.net \
--to=luto@amacapital.net \
--cc=efault@gmx.de \
--cc=hpa@zytor.com \
--cc=jpoimboe@redhat.com \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=live-patching@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).