From: Sasha Levin <levinsasha928@gmail.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
"linux-kernel@vger.kernel.org List"
<linux-kernel@vger.kernel.org>,
alexander.levin@verizon.com
Subject: Re: [PATCH 3/4] sched: WARN when migrating to an offline CPU
Date: Thu, 28 Sep 2017 02:14:15 -0700 [thread overview]
Message-ID: <CA+1xoqe0AW9acyvdtE+=CN8FisQrXH2wnwvakSXapg3sKE4khw@mail.gmail.com> (raw)
In-Reply-To: <20170907150614.094206976@infradead.org>
On Thu, Sep 7, 2017 at 8:03 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> Migrating tasks to offline CPUs is a pretty big fail, warn about it.
Hey Peter,
This seems to get hit on the latest -next:
[2035565360.446794] Unregister pv shared memory for cpu 2
[2035565360.467930] numa_remove_cpu cpu 2 node 2: mask now 6
[2035565360.471431] ------------[ cut here ]------------
[2035565360.472548] WARNING: CPU: 2 PID: 24 at
kernel/sched/core.c:1178 set_task_cpu (kernel/sched/core.c:1157)
[2035565360.473840] Modules linked in:
[2035565360.474632] CPU: 2 PID: 24 Comm: migration/2 Not tainted
4.14.0-rc2-next-20170927+ #252
[2035565360.476128] task: ffff8801ab7dc040 task.stack: ffff8801ab7e0000
[2035565360.477108] RIP: 0010:set_task_cpu (??:?)
[2035565360.477850] RSP: 0018:ffff8801ab807b18 EFLAGS: 00010096
[2035565360.478734] RAX: 0000000000000002 RBX: ffff8801a8d94040 RCX:
1ffff1003577cb3c
[2035565360.479872] RDX: 0000000000000004 RSI: 0000000000000002 RDI:
ffffffff9177d940
[2035565360.480998] RBP: ffff8801ab807b58 R08: 000000000000b6b2 R09:
ffff8801abbe5a2c
[2035565360.482356] R10: 0000000000000001 R11: 0000000000000001 R12:
ffff8801a8d94048
[2035565360.483490] R13: ffff8801ab807c88 R14: 0000000000000002 R15:
ffff8801a8d940a4
[2035565360.484617] FS: 0000000000000000(0000)
GS:ffff8801ab800000(0000) knlGS:0000000000000000
[2035565360.485881] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[2035565360.490482] CR2: 00000000012dfae4 CR3: 00000001a30c5002 CR4:
00000000000606a0
[2035565360.491836] Call Trace:
[2035565360.492301] <IRQ>
[2035565360.492710] detach_task.isra.80 (kernel/sched/fair.c:6816)
[2035565360.493430] load_balance (./include/linux/list.h:78
kernel/sched/fair.c:6902 kernel/sched/fair.c:8201)
[2035565360.494094] ? find_busiest_group (kernel/sched/fair.c:8133)
[2035565360.494861] ? rebalance_domains (kernel/sched/fair.c:8785)
[2035565360.495571] rebalance_domains (kernel/sched/fair.c:8829)
[2035565360.496292] ? find_next_bit (lib/find_bit.c:65)
[2035565360.496961] ? pick_next_task_fair (kernel/sched/fair.c:8779)
[2035565360.497753] ? check_preemption_disabled (lib/smp_processor_id.c:52)
[2035565360.498574] run_rebalance_domains (kernel/sched/fair.c:9052)
[2035565360.499318] ? check_preemption_disabled (lib/smp_processor_id.c:52)
[2035565360.500388] __do_softirq (kernel/softirq.c:284
./include/linux/jump_label.h:141 ./include/trace/events/irq.h:141
kernel/softirq.c:285)
[2035565360.501039] ? hrtimer_interrupt (kernel/time/hrtimer.c:1370)
[2035565360.502014] irq_exit (kernel/softirq.c:364 kernel/softirq.c:405)
[2035565360.502598] smp_apic_timer_interrupt
(./arch/x86/include/asm/irq_regs.h:26
arch/x86/kernel/apic/apic.c:1043)
[2035565360.503605] apic_timer_interrupt (arch/x86/entry/entry_64.S:770)
[2035565360.504313] </IRQ>
[2035565360.504724] RIP: 0010:multi_cpu_stop (??:?)
[2035565360.505503] RSP: 0018:ffff8801ab7e7d88 EFLAGS: 00000292
ORIG_RAX: ffffffffffffff11
[2035565360.506721] RAX: 0000000000000003 RBX: 0000000000000004 RCX:
000000000000000a
[2035565360.507862] RDX: 0000000000000004 RSI: 0000000000000040 RDI:
0000000000000292
[2035565360.508997] RBP: ffff8801ab7e7de0 R08: ffff8801ab7dc040 R09:
0000000000000000
[2035565360.510566] R10: 0000000000000001 R11: 1ffff1003577cb55 R12:
ffff8801aae2f890
[2035565360.511701] R13: ffff8801aae2f8b4 R14: dffffc0000000000 R15:
0000000000000003
[2035565360.512843] ? cpu_stop_queue_work (kernel/stop_machine.c:176)
[2035565360.513591] cpu_stopper_thread (kernel/stop_machine.c:480)
[2035565360.514316] ? cpu_stop_create (kernel/stop_machine.c:458)
[2035565360.515218] smpboot_thread_fn (kernel/smpboot.c:164)
[2035565360.515918] ? sort_range (kernel/smpboot.c:107)
[2035565360.516528] ? schedule (./arch/x86/include/asm/bitops.h:324
(discriminator 1) ./include/linux/thread_info.h:79 (discriminator 1)
./include/linux/sched.h:1605 (discriminator 1)
kernel/sched/core.c:3435 (discriminator 1))
[2035565360.517119] ? __kthread_parkme (kernel/kthread.c:188)
[2035565360.517835] kthread (kernel/kthread.c:242)
[2035565360.518413] ? sort_range (kernel/smpboot.c:107)
[2035565360.519008] ? kthread_create_on_node (kernel/kthread.c:198)
[2035565360.519792] ret_from_fork (arch/x86/entry/entry_64.S:437)
[2035565360.520407] Code: 09 84 d2 74 05 e8 94 38 4b 00 f7 83 84 00 00
00 fd ff ff ff 0f 84 a0 f9 ff ff 0f ff e9 99 f9 ff ff e8 68 18 ff ff
e9 c7 fd ff ff <0f> ff e9 ea f9 ff ff e8 57 18 ff ff e9 f5 f9 ff ff e8
0d 84 0f
All code
========
0: 09 84 d2 74 05 e8 94 or %eax,-0x6b17fa8c(%rdx,%rdx,8)
7: 38 4b 00 cmp %cl,0x0(%rbx)
a: f7 83 84 00 00 00 fd testl $0xfffffffd,0x84(%rbx)
11: ff ff ff
14: 0f 84 a0 f9 ff ff je 0xfffffffffffff9ba
1a: 0f ff (bad)
1c: e9 99 f9 ff ff jmpq 0xfffffffffffff9ba
21: e8 68 18 ff ff callq 0xffffffffffff188e
26: e9 c7 fd ff ff jmpq 0xfffffffffffffdf2
2b:* 0f ff (bad) <-- trapping instruction
2d: e9 ea f9 ff ff jmpq 0xfffffffffffffa1c
32: e8 57 18 ff ff callq 0xffffffffffff188e
37: e9 f5 f9 ff ff jmpq 0xfffffffffffffa31
3c: e8 0d 84 0f 00 callq 0xf844e
Code starting with the faulting instruction
===========================================
0: 0f ff (bad)
2: e9 ea f9 ff ff jmpq 0xfffffffffffff9f1
7: e8 57 18 ff ff callq 0xffffffffffff1863
c: e9 f5 f9 ff ff jmpq 0xfffffffffffffa06
11: e8 0d 84 0f 00 callq 0xf8423
[2035565360.523585] ---[ end trace 812b7ff74114db6c ]---
[2035565360.524744] smpboot: CPU 2 is now offline
Thanks,
Sasha
next prev parent reply other threads:[~2017-09-28 9:14 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-07 15:03 [PATCH 0/4] sched: Fix some load-balancer vs hotplug holes Peter Zijlstra
2017-09-07 15:03 ` [PATCH 1/4] sched/fair: Avoid newidle balance for !active CPUs Peter Zijlstra
2017-09-12 18:04 ` [tip:sched/urgent] " tip-bot for Peter Zijlstra
2017-09-07 15:03 ` [PATCH 2/4] sched/fair: Plug hole between hotplug and active load_balance Peter Zijlstra
2017-09-12 18:05 ` [tip:sched/urgent] sched/fair: Plug hole between hotplug and active_load_balance() tip-bot for Peter Zijlstra
2017-09-07 15:03 ` [PATCH 3/4] sched: WARN when migrating to an offline CPU Peter Zijlstra
2017-09-12 18:05 ` [tip:sched/urgent] sched/core: WARN() " tip-bot for Peter Zijlstra
2017-09-28 9:14 ` Sasha Levin [this message]
2017-09-28 10:35 ` [PATCH 3/4] sched: WARN " Peter Zijlstra
2017-09-28 11:03 ` Levin, Alexander (Sasha Levin)
2017-09-28 11:42 ` Peter Zijlstra
2017-09-29 11:11 ` Peter Zijlstra
2017-10-07 2:07 ` Levin, Alexander (Sasha Levin)
2017-10-07 9:15 ` Peter Zijlstra
[not found] ` <20171007174327.ky6g5viokxg5ysdm@sasha-lappy>
2017-10-09 8:04 ` Peter Zijlstra
2017-10-10 1:18 ` Levin, Alexander (Sasha Levin)
2017-09-07 15:03 ` [PATCH 4/4] sched/debug: Add debugfs knob for "sched_debug" Peter Zijlstra
2017-09-12 18:05 ` [tip:sched/urgent] " tip-bot for Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CA+1xoqe0AW9acyvdtE+=CN8FisQrXH2wnwvakSXapg3sKE4khw@mail.gmail.com' \
--to=levinsasha928@gmail.com \
--cc=alexander.levin@verizon.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).