All of lore.kernel.org
 help / color / mirror / Atom feed
From: Qian Cai <cai@lca.pw>
To: axboe@kernel.dk
Cc: akpm@linux-foundation.org, hch@lst.de, peterz@infradead.org,
	oleg@redhat.com, gkohli@codeaurora.org, mingo@redhat.com,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	Qian Cai <cai@lca.pw>
Subject: [PATCH] block: fix a crash in do_task_dead()
Date: Wed, 29 May 2019 16:25:26 -0400	[thread overview]
Message-ID: <1559161526-618-1-git-send-email-cai@lca.pw> (raw)

The commit 0619317ff8ba ("block: add polled wakeup task helper")
replaced wake_up_process() with blk_wake_io_task() in
end_swap_bio_read() which triggers a crash when running heavy swapping
workloads.

[T114538] kernel BUG at kernel/sched/core.c:3462!
[T114538] Process oom01 (pid: 114538, stack limit = 0x000000004f40e0c1)
[T114538] Call trace:
[T114538]  do_task_dead+0xf0/0xf8
[T114538]  do_exit+0xd5c/0x10fc
[T114538]  do_group_exit+0xf4/0x110
[T114538]  get_signal+0x280/0xdd8
[T114538]  do_notify_resume+0x720/0x968
[T114538]  work_pending+0x8/0x10

This is because shortly after set_special_state(TASK_DEAD),
end_swap_bio_read() is called from an interrupt handler that revive the
task state to TASK_RUNNING causes __schedule() to return and trip the
BUG() later.

[  C206] Call trace:
[  C206]  dump_backtrace+0x0/0x268
[  C206]  show_stack+0x20/0x2c
[  C206]  dump_stack+0xb4/0x108
[  C206]  blk_wake_io_task+0x7c/0x80
[  C206]  end_swap_bio_read+0x22c/0x31c
[  C206]  bio_endio+0x3d8/0x414
[  C206]  dec_pending+0x280/0x378 [dm_mod]
[  C206]  clone_endio+0x128/0x2ac [dm_mod]
[  C206]  bio_endio+0x3d8/0x414
[  C206]  blk_update_request+0x3ac/0x924
[  C206]  scsi_end_request+0x54/0x350
[  C206]  scsi_io_completion+0xf0/0x6f4
[  C206]  scsi_finish_command+0x214/0x228
[  C206]  scsi_softirq_done+0x170/0x1a4
[  C206]  blk_done_softirq+0x100/0x194
[  C206]  __do_softirq+0x350/0x790
[  C206]  irq_exit+0x200/0x26c
[  C206]  handle_IPI+0x2e8/0x514
[  C206]  gic_handle_irq+0x224/0x228
[  C206]  el1_irq+0xb8/0x140
[  C206]  _raw_spin_unlock_irqrestore+0x3c/0x74
[  C206]  do_task_dead+0x88/0xf8
[  C206]  do_exit+0xd5c/0x10fc
[  C206]  do_group_exit+0xf4/0x110
[  C206]  get_signal+0x280/0xdd8
[  C206]  do_notify_resume+0x720/0x968
[  C206]  work_pending+0x8/0x10

Before the offensive commit, wake_up_process() will prevent this from
happening by taking the pi_lock and bail out immediately if TASK_DEAD is
set.

if (!(p->state & TASK_NORMAL))
	goto out;

Fix it by calling wake_up_process() if it is in a non-task context.

Fixes: 0619317ff8ba ("block: add polled wakeup task helper")
Signed-off-by: Qian Cai <cai@lca.pw>
---
 include/linux/blkdev.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 592669bcc536..290eb7528f54 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1803,7 +1803,7 @@ static inline void blk_wake_io_task(struct task_struct *waiter)
 	 * that case, we don't need to signal a wakeup, it's enough to just
 	 * mark us as RUNNING.
 	 */
-	if (waiter == current)
+	if (waiter == current && in_task())
 		__set_current_state(TASK_RUNNING);
 	else
 		wake_up_process(waiter);
-- 
1.8.3.1


             reply	other threads:[~2019-05-29 20:26 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-29 20:25 Qian Cai [this message]
2019-05-29 20:31 ` [PATCH] block: fix a crash in do_task_dead() Jens Axboe
2019-05-30  8:03 ` Peter Zijlstra
2019-05-31 21:12   ` Jens Axboe
2019-06-03 12:37     ` Peter Zijlstra
2019-06-03 12:44       ` Peter Zijlstra
2019-06-03 16:09         ` Oleg Nesterov
2019-06-03 16:19           ` Peter Zijlstra
2019-06-03 16:23       ` Jens Axboe
2019-06-05 15:04       ` Jens Axboe
2019-06-07 13:35         ` Peter Zijlstra
2019-06-07 14:23           ` Peter Zijlstra
2019-06-08  8:39             ` Jens Axboe
2019-06-10 13:13             ` Gaurav Kohli
2019-06-10 14:46               ` Oleg Nesterov
2019-06-11  4:39                 ` Gaurav Kohli
2019-06-30 23:06         ` Hugh Dickins
2019-06-30 23:06           ` Hugh Dickins
2019-07-01 14:22           ` Jens Axboe
2019-07-02 22:06             ` Andrew Morton
2019-07-03 17:35               ` Oleg Nesterov
2019-07-03 17:44                 ` Hugh Dickins
2019-07-03 17:44                   ` Hugh Dickins
2019-07-04 16:00                   ` Oleg Nesterov
2019-07-03 17:52                 ` Jens Axboe
2019-05-30 11:15 ` Oleg Nesterov
2019-05-31 21:10   ` Jens Axboe
2019-07-04 16:03 ` [PATCH] swap_readpage: avoid blk_wake_io_task() if !synchronous Oleg Nesterov
2019-07-04 19:32   ` Andrew Morton
2019-07-04 21:15     ` Hugh Dickins
2019-07-04 21:15       ` Hugh Dickins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1559161526-618-1-git-send-email-cai@lca.pw \
    --to=cai@lca.pw \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=gkohli@codeaurora.org \
    --cc=hch@lst.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mingo@redhat.com \
    --cc=oleg@redhat.com \
    --cc=peterz@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.