linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Josh Poimboeuf <jpoimboe@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Vince Weaver <vincent.weaver@maine.edu>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	"davej@codemonkey.org.uk" <davej@codemonkey.org.uk>,
	"dvyukov@google.com" <dvyukov@google.com>,
	Stephane Eranian <eranian@gmail.com>
Subject: Re: perf: fuzzer KASAN unwind_get_return_address
Date: Wed, 16 Nov 2016 22:48:28 -0600	[thread overview]
Message-ID: <20161117044828.vedc3whqkuki624r@treble> (raw)
In-Reply-To: <20161116145849.GR3157@twins.programming.kicks-ass.net>

On Wed, Nov 16, 2016 at 03:58:49PM +0100, Peter Zijlstra wrote:
> 3BUG: KASAN: stack-out-of-bounds in unwind_get_return_address+0x1fb/0x220 at addr ffff88042f88bba0

So I dug through the disassembly (thanks for the vmlinux), and I'm
pretty sure the stack-out-of-bounds address is on the NMI stack, in the
kasan redzone in the stack frame of intel_pmu_handle_irq().

What's weird though is that perf_callchain_kernel() passes the pt_regs
from the IRQ, not from the NMI.  The unwinder should have started from
the IRQ stack.  But somehow it ended up unwinding to the middle of the
NMI stack.

So it seems like stack corruption in the IRQ or task stack, with a frame
pointer that points back to the middle of the NMI stack for some reason.
But then again, the kasan error report dumped the stack fine.  So that
would seem to rule out stack corruption...  So I have no idea what's
going on.

I got perf_fuzzer running and tried to recreate, but no luck.

Peter or Vince, can you try to recreate with this patch?  It dumps the
raw stack contents during a stack dump.  Hopefully that would give a
clue about what's going wrong.

diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index 499aa6f..67ff3ac 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -48,6 +48,30 @@ static void printk_stack_address(unsigned long address, int reliable,
 	printk("%s %s%pB\n", log_lvl, reliable ? "" : "? ", (void *)address);
 }
 
+static void raw_stack_dump(struct stack_info *info)
+{
+	unsigned long *s, word[4];
+	int skip = 0;
+
+	for (s = info->begin; s < info->end; s += 4) {
+		word[0] = READ_ONCE_NOCHECK(s[0]);
+		word[1] = READ_ONCE_NOCHECK(s[1]);
+		word[2] = READ_ONCE_NOCHECK(s[2]);
+		word[3] = READ_ONCE_NOCHECK(s[3]);
+
+		if (!word[0] && !word[1] && !word[2] && !word[3]) {
+			if (!skip)
+				printk("%p: %016x ...\n", s, 0);
+			skip = 1;
+			continue;
+		}
+
+		skip = 0;
+		printk("%p: %016lx %016lx %016lx %016lx\n",
+		       s, word[0], word[1], word[2], word[3]);
+	}
+}
+
 void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,
 			unsigned long *stack, char *log_lvl)
 {
@@ -156,6 +180,8 @@ void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs,
 
 		if (str_end)
 			printk("%s <%s>\n", log_lvl, str_end);
+
+		raw_stack_dump(&stack_info);
 	}
 }
 

  reply	other threads:[~2016-11-17  4:48 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-11-15 17:43 perf: fuzzer KASAN unwind_get_return_address Vince Weaver
2016-11-15 18:57 ` Peter Zijlstra
2016-11-15 19:04   ` Dmitry Vyukov
2016-11-15 20:56     ` Josh Poimboeuf
2016-11-15 19:05   ` Vince Weaver
2016-11-15 20:57     ` Josh Poimboeuf
2016-11-16 13:03       ` Peter Zijlstra
2016-11-16 13:18         ` Dmitry Vyukov
2016-11-16 14:37         ` Josh Poimboeuf
2016-11-16 14:49           ` Peter Zijlstra
2016-11-16 14:58             ` Josh Poimboeuf
2016-11-16 14:58             ` Peter Zijlstra
2016-11-17  4:48               ` Josh Poimboeuf [this message]
2016-11-17  9:04                 ` Peter Zijlstra
2016-11-17  9:13                   ` Peter Zijlstra
2016-11-17  9:30                     ` Peter Zijlstra
2016-11-17  9:48                       ` Dmitry Vyukov
2016-11-17 14:01                         ` Josh Poimboeuf
2016-11-17 14:25                           ` Vince Weaver
2016-11-17 14:36                             ` Josh Poimboeuf
2016-11-17 14:58                               ` Dmitry Vyukov
2016-11-17 17:15                                 ` Vince Weaver
2016-11-17 15:18                   ` Josh Poimboeuf
2016-11-17 16:07                     ` Peter Zijlstra
2016-11-17 17:17                       ` Peter Zijlstra
2016-11-22 12:30                         ` [tip:perf/urgent] perf/x86/intel: Cure bogus unwind from PEBS entries tip-bot for Peter Zijlstra
2016-11-16 15:06             ` perf: fuzzer KASAN unwind_get_return_address Vince Weaver
2016-11-17 15:57         ` [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder Josh Poimboeuf
2016-11-17 15:57           ` [PATCH 2/2] dumpstack: prevent KASAN false positive warnings Josh Poimboeuf
2016-11-18  9:04             ` [tip:x86/urgent] x86/dumpstack: Prevent " tip-bot for Josh Poimboeuf
2016-11-17 20:26           ` [PATCH 1/2] unwind: prevent KASAN false positive warnings in guess unwinder Josh Poimboeuf
2016-11-18  8:38             ` Ingo Molnar
2016-11-18  9:04           ` [tip:x86/urgent] x86/unwind: Prevent " tip-bot for Josh Poimboeuf

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20161117044828.vedc3whqkuki624r@treble \
    --to=jpoimboe@redhat.com \
    --cc=acme@kernel.org \
    --cc=davej@codemonkey.org.uk \
    --cc=dvyukov@google.com \
    --cc=eranian@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=vincent.weaver@maine.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).