From: Vince Weaver <vincent.weaver@maine.edu>
To: linux-kernel@vger.kernel.org
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Peter Zijlstra <peterz@infradead.org>,
Stephane Eranian <eranian@google.com>,
Ingo Molnar <mingo@kernel.org>
Subject: perf: perf_fuzzer lockup in perf_cgroup_attach
Date: Wed, 14 Sep 2016 22:43:29 -0400 (EDT) [thread overview]
Message-ID: <alpine.DEB.2.20.1609142224480.12612@macbook-air> (raw)
so the skylake that was fuzzing finally is mostly locked up.
Really hard to tell what's going, especially as KASLR made looking up the
addresses a big pain.
The best I can tell things are getting wedged somehow in
perf_cgroup_switch() while interrupts are disabled. Interrupts are never
getting re-enabled, causing the RCU and NMI watchdogs to trigger (and more
alarming things like the SATA bus resetting).
[26292.413603] Task dump for CPU 4:
[26292.413604] perf_fuzzer R running task 0 8870 1096 0x10000008
[26292.413605] ffff9045f29e1100 00000000e7143ab0 ffff9045fdd03db8 ffffffff938accef
[26292.413606] 0000000000000004 0000000000000087 ffff9045fdd03dd0 ffffffff938af5e9
[26292.413607] 0000000000000004 ffff9045fdd03e00 ffffffff93984928 ffff9045fdd19440
[26292.413608] Call Trace:
[26292.413609] <IRQ> [<ffffffff938accef>] sched_show_task+0xaf/0x110
[26292.413611] [<ffffffff938af5e9>] dump_cpu_task+0x39/0x40
[26292.413613] [<ffffffff93984928>] rcu_dump_cpu_stacks+0x80/0xbb
[26292.413614] [<ffffffff938e78ce>] rcu_check_callbacks+0x71e/0x880
[26292.413615] [<ffffffff9393364c>] ? acct_account_cputime+0x1c/0x20
[26292.413616] [<ffffffff938b0049>] ? account_system_time+0x79/0x120
[26292.413617] [<ffffffff938fe8b0>] ? tick_sched_do_timer+0x30/0x30
[26292.413619] [<ffffffff938ee88f>] update_process_times+0x2f/0x60
[26292.413619] [<ffffffff938fe285>] tick_sched_handle.isra.13+0x25/0x60
[26292.413620] [<ffffffff938fe8ed>] tick_sched_timer+0x3d/0x70
[26292.413621] [<ffffffff938ef633>] __hrtimer_run_queues+0xf3/0x280
[26292.413623] [<ffffffff938efb08>] hrtimer_interrupt+0xa8/0x1a0
[26292.413624] [<ffffffff93852d88>] local_apic_timer_interrupt+0x38/0x60
[26292.413625] [<ffffffff93e1dee4>] smp_trace_apic_timer_interrupt+0x44/0xde
[26292.413627] [<ffffffff9396ff60>] ? perf_cgroup_attach+0x70/0x70
[26292.413628] [<ffffffff93e1d232>] trace_apic_timer_interrupt+0x82/0x90
[26292.413628] <EOI> [<ffffffff9396ff60>] ? perf_cgroup_attach+0x70/0x70
[26292.413631] [<ffffffff939039d6>] ? smp_call_function_single+0xd6/0x130
[26292.413632] [<ffffffff9396ff60>] ? perf_cgroup_attach+0x70/0x70
[26292.413633] [<ffffffff9396ec53>] cpu_function_call+0x43/0x60
[26292.413634] [<ffffffff93976840>] ? __perf_event_enable+0x260/0x260
[26292.413635] [<ffffffff939737e1>] perf_install_in_context+0x141/0x150
[26292.413636] [<ffffffff9397956e>] SYSC_perf_event_open+0x70e/0xfe0
[26292.413637] [<ffffffff938a99e4>] ? check_preempt_curr+0x54/0x90
[26292.413639] [<ffffffff9397c099>] SyS_perf_event_open+0x9/0x10
[26292.413640] [<ffffffff93803bb4>] do_syscall_64+0x64/0x160
[26292.413641] [<ffffffff93e1b625>] entry_SYSCALL64_slow_path+0x25/0x25
[26316.489382] NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [perf_fuzzer:8870]
[26316.497309] CPU: 4 PID: 8870 Comm: perf_fuzzer Tainted: G W 4.8.0-rc6+ #5
[26316.497310] Hardware name: LENOVO 10FY0017US/SKYBAY, BIOS FWKT53A 06/06/2016
[26316.497310] task: ffff9045f29e1100 task.stack: ffff9045f0a48000
[26316.497311] RIP: 0010:[<ffffffff939039d6>] [<ffffffff939039d6>] smp_call_function_single+0xd6/0x130
[26316.497312] RSP: 0018:ffff9045f0a4bd30 EFLAGS: 00000202
[26316.497313] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 0000000000000000
[26316.497313] RDX: 0000000000000001 RSI: 00000000000000fb RDI: 0000000000000286
[26316.497314] RBP: ffff9045f0a4bd78 R08: ffff9045fdc9bed0 R09: ffff9045f2ac8000
[26316.497314] R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff9396ff60
[26316.497315] R13: ffff9045fdc9bed0 R14: 000000000000001e R15: ffff9045fdc9bee0
[26316.497315] FS: 00007f4157a08700(0000) GS:ffff9045fdd00000(0000) knlGS:0000000000000000
[26316.497316] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[26316.497316] CR2: 000000000693a048 CR3: 0000000231049000 CR4: 00000000003407e0
[26316.497317] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[26316.497317] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[26316.497318] Stack:
[26316.497318] 0000000000000000 ffff9045ee35da80 0000000000000000 ffffffff9396ff60
[26316.497319] ffff9045f0a4bd88 0000000000000003 00000000e7143ab0 ffff9045ee47613a
[26316.497321] ffff9045ee476000 ffff9045f0a4bdb0 ffffffff9396ec53 0000000000000000
[26316.497322] Call Trace:
[26316.497323] [<ffffffff9396ff60>] ? perf_cgroup_attach+0x70/0x70
[26316.497324] [<ffffffff9396ec53>] cpu_function_call+0x43/0x60
[26316.497325] [<ffffffff93976840>] ? __perf_event_enable+0x260/0x260
[26316.497326] [<ffffffff939737e1>] perf_install_in_context+0x141/0x150
[26316.497327] [<ffffffff9397956e>] SYSC_perf_event_open+0x70e/0xfe0
[26316.497328] [<ffffffff938a99e4>] ? check_preempt_curr+0x54/0x90
[26316.497330] [<ffffffff9397c099>] SyS_perf_event_open+0x9/0x10
[26316.497331] [<ffffffff93803bb4>] do_syscall_64+0x64/0x160
[26316.497332] [<ffffffff93e1b625>] entry_SYSCALL64_slow_path+0x25/0x25
[26316.497332] Code: 25 28 00 00 00 75 70 48 83 c4 38 5b 41 5c 5d c3 48 8d 75 c8 48 89 d1 89 df 4c 89 e2 e8 14 fe ff ff 8b 55 e0 83 e2 01 74 0a f3 90 <8b> 55 e0 83 e2 01 75 f6 eb c3 8b 05 ba e5 dd 00 85 c0 75 85 80
next reply other threads:[~2016-09-15 2:43 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-15 2:43 Vince Weaver [this message]
2016-09-15 5:35 ` perf: perf_fuzzer lockup in perf_cgroup_attach Stephane Eranian
2016-09-15 7:24 ` Peter Zijlstra
2016-09-15 12:10 ` Josh Poimboeuf
2016-09-16 14:48 ` [PATCH] scripts: add script for translating stack dump function offsets Josh Poimboeuf
2016-09-16 18:12 ` Linus Torvalds
2016-09-16 19:17 ` Josh Poimboeuf
2016-09-16 19:26 ` Linus Torvalds
2016-09-16 21:26 ` [PATCH v2] " Josh Poimboeuf
2016-09-17 0:09 ` Linus Torvalds
2016-09-17 0:42 ` Josh Poimboeuf
2016-09-17 1:59 ` Linus Torvalds
2016-09-17 2:31 ` Linus Torvalds
2016-09-17 18:45 ` Josh Poimboeuf
2016-09-19 15:52 ` [PATCH v3] scripts: add script for translating stack dump function Josh Poimboeuf
2016-09-19 18:56 ` Linus Torvalds
2016-09-19 19:15 ` Linus Torvalds
2016-09-19 19:28 ` Josh Poimboeuf
2016-09-19 20:00 ` Rabin Vincent
2016-09-19 20:24 ` Linus Torvalds
2016-09-19 20:56 ` Josh Poimboeuf
2016-09-19 21:02 ` Linus Torvalds
2016-09-19 21:07 ` Josh Poimboeuf
2016-09-17 1:25 ` [PATCH v2] scripts: add script for translating stack dump function offsets Peter Zijlstra
2016-09-17 8:15 ` Rabin Vincent
2016-09-17 18:48 ` Josh Poimboeuf
2016-09-17 9:11 ` [PATCH] " Vegard Nossum
2016-09-17 18:54 ` Josh Poimboeuf
2016-09-15 12:47 ` perf: perf_fuzzer lockup in perf_cgroup_attach Vince Weaver
2016-09-15 7:15 ` Peter Zijlstra
2016-09-15 12:41 ` Vince Weaver
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.20.1609142224480.12612@macbook-air \
--to=vincent.weaver@maine.edu \
--cc=alexander.shishkin@linux.intel.com \
--cc=eranian@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.