From: David Hill <dhill@redhat.com>
To: kvm@vger.kernel.org
Subject: Shutting down a VM with Kernel 4.14 will sometime hang and a reboot is the only way to recover. [1]
Date: Mon, 13 Nov 2017 10:54:09 -0500 [thread overview]
Message-ID: <92c4f997-80db-fabf-98c8-fcb92da064a7@redhat.com> (raw)
In-Reply-To: <efd45fba-5724-0036-8473-0274b5816ae9@redhat.com>
Hi guys,
Starting with kernel 4.14-rc1, my CI failed to completely shutdown
one of the
VMs and it stuck in "in shutdown" while sending this kernel message:
[ 7496.552971] INFO: task qemu-system-x86:5978 blocked for more than 120
seconds.
[ 7496.552987] Tainted: G I
4.14.0-0.rc1.git3.1.fc28.x86_64 #1
[ 7496.552996] "echo 0 /proc/sys/kernel/hung_task_timeout_secs" disables
this message.
[ 7496.553006] qemu-system-x86 D12240 5978 1 0x00000004
[ 7496.553024] Call Trace:
[ 7496.553044] __schedule+0x2dc/0xbb0
[ 7496.553055] ? trace_hardirqs_on+0xd/0x10
[ 7496.553074] schedule+0x3d/0x90
[ 7496.553087] vhost_net_ubuf_put_and_wait+0x73/0xa0 [vhost_net]
[ 7496.553100] ? finish_wait+0x90/0x90
[ 7496.553115] vhost_net_ioctl+0x542/0x910 [vhost_net]
[ 7496.553144] do_vfs_ioctl+0xa6/0x6c0
[ 7496.553166] SyS_ioctl+0x79/0x90
[ 7496.553182] entry_SYSCALL_64_fastpath+0x1f/0xbe
[ 7496.553190] RIP: 0033:0x7fa1ea0e1817
[ 7496.553196] RSP: 002b:00007ffe3d854bc8 EFLAGS: 00000246 ORIG_RAX:
0000000000000010
[ 7496.553209] RAX: ffffffffffffffda RBX: 000000000000001d RCX:
00007fa1ea0e1817
[ 7496.553215] RDX: 00007ffe3d854bd0 RSI: 000000004008af30 RDI:
0000000000000021
[ 7496.553222] RBP: 000055e33352b610 R08: 000055e33024a6f0 R09:
000055e330245d92
[ 7496.553228] R10: 000055e33344e7f0 R11: 0000000000000246 R12:
000055e33351a000
[ 7496.553235] R13: 0000000000000001 R14: 0000000400000000 R15:
0000000000000000
[ 7496.553284]
Showing all locks held in the system:
[ 7496.553313] 1 lock held by khungtaskd/161:
[ 7496.553319] #0: (tasklist_lock){.+.+}, at: [<ffffffff8111740d>]
debug_show_all_locks+0x3d/0x1a0
[ 7496.553373] 1 lock held by in:imklog/1194:
[ 7496.553379] #0: (&f->f_pos_lock){+.+.}, at: [<ffffffff8130ecfc>]
__fdget_pos+0x4c/0x60
[ 7496.553541] 1 lock held by qemu-system-x86/5978:
[ 7496.553547] #0: (&dev->mutex#3){+.+.}, at: [<ffffffffc077e498>]
vhost_net_ioctl+0x358/0x910 [vhost_net]
I'm currently bisecting to figure out which commit breaks this but for
some reasons, when hitting this commit:
# bad: [46d4b68f891bee5d83a32508bfbd9778be6b1b63] Merge tag
'wireless-drivers-next-for-davem-2017-08-07' of
git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next
git bisect bad 46d4b68f891bee5d83a32508bfbd9778be6b1b63
the host will not allow SSHd to establish a new session and when
starting a KVM guest, the host will hard lock. I'm still bisecting but
I marked that commit as bad even though perhaps it would be good.
Hopefully, this commit was a bad one and my bisection will pinpoint which
commit broke the kernel. If you have an idea of which commit might break
the system, please let me know which one I should test first.
Thank you very much,
David Hill
[1] https://bugzilla.kernel.org/show_bug.cgi?id=197861
next parent reply other threads:[~2017-11-13 15:54 UTC|newest]
Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <efd45fba-5724-0036-8473-0274b5816ae9@redhat.com>
2017-11-13 15:54 ` David Hill [this message]
[not found] ` <CALapVYHmf7gG25nA-5LkoaTDR8gB0xQ1Ro_FyyCQNbzrfSp+aQ@mail.gmail.com>
2017-11-15 21:08 ` Shutting down a VM with Kernel 4.14 will sometime hang and a reboot is the only way to recover. [1] David Hill
2017-11-22 18:22 ` Shutting down a VM with Kernel 4.14 will sometime hang and a reboot is the only way to recover David Hill
2017-11-23 23:48 ` Paolo Bonzini
2017-11-24 3:11 ` Jason Wang
2017-11-24 16:19 ` David Hill
2017-11-24 16:22 ` David Hill
2017-11-27 3:44 ` Jason Wang
2017-11-27 19:38 ` David Hill
2017-11-28 18:00 ` David Hill
2017-11-29 1:52 ` Jason Wang
2017-11-29 2:52 ` Dave Hill
2017-11-29 5:15 ` Jason Wang
2017-11-29 19:13 ` David Hill
2017-11-30 2:42 ` Jason Wang
2017-11-30 20:52 ` David Hill
2017-11-30 20:59 ` David Hill
2017-12-01 16:38 ` David Hill
2017-12-04 4:08 ` Jason Wang
2017-12-04 19:51 ` David Hill
2017-12-07 4:34 ` David Hill
2017-12-07 4:42 ` David Hill
2017-12-07 5:13 ` Jason Wang
2017-12-08 18:03 ` David Hill
2017-12-12 3:53 ` David Hill
2017-12-19 3:36 ` Jason Wang
2017-12-19 16:19 ` Willem de Bruijn
2017-12-07 5:12 ` Jason Wang
2017-12-02 12:16 ` Harald Moeller
2017-12-02 16:37 ` Harald Moeller
2017-12-07 2:44 ` David Hill
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=92c4f997-80db-fabf-98c8-fcb92da064a7@redhat.com \
--to=dhill@redhat.com \
--cc=kvm@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.