From: Ilya Dryomov <ilya.dryomov@inktank.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>,
oleg@redhat.com, Linus Torvalds <torvalds@linux-foundation.org>,
tglx@linutronix.de, Mike Galbraith <umgwanakikbuti@gmail.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
netdev@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [RFC][PATCH 0/7] nested sleeps, fixes and debug infra
Date: Tue, 5 Aug 2014 12:33:16 +0400 [thread overview]
Message-ID: <CALFYKtBo2p5uNtkJZOy_rN7JbdFs1RbB1OfcF7TR+qDaMU0Kvg@mail.gmail.com> (raw)
In-Reply-To: <20140804103025.478913141@infradead.org>
On Mon, Aug 4, 2014 at 2:30 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> Hi,
>
> Ilya recently tripped over a nested sleep which made Ingo suggest we should
> have debug checks for that. So I did some, see patch 7. Of course that
> triggered a whole bunch of fail the instant I tried to boot my machine.
>
> With this series I can boot my test box and build a kernel on it, I'm fairly
> sure that's far too limited a test to have found all, but its a start.
FWIW, I'm getting a lot of these during light rbd testing. CC'ed
netdev and linux-mm.
WARNING: CPU: 2 PID: 1978 at kernel/sched/core.c:7094 __might_sleep+0x5b/0x1e0()
do not call blocking ops when !TASK_RUNNING; state=1 set at
[<ffffffff81070640>] prepare_to_wait+0x50/0xa0
Modules linked in:
CPU: 2 PID: 1978 Comm: ceph-osd Not tainted 3.16.0-vm+ #109
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
0000000000001bb6 ffff8800126739e8 ffffffff8156ec1d 0000000000000000
ffff880012673a38 ffff880012673a28 ffffffff81032c27 ffff880012673a58
0000000000000200 ffff8800150fa060 00000000000007ad ffffffff817ed352
Call Trace:
[<ffffffff8156ec1d>] dump_stack+0x4f/0x7c
[<ffffffff81032c27>] warn_slowpath_common+0x87/0xb0
[<ffffffff81032cf1>] warn_slowpath_fmt+0x41/0x50
[<ffffffff814f23cf>] ? tcp_v4_do_rcv+0x10f/0x4a0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff8105b53b>] __might_sleep+0x5b/0x1e0
[<ffffffff8148d73d>] release_sock+0x13d/0x200
[<ffffffff81498223>] sk_stream_wait_memory+0x133/0x2d0
[<ffffffff810701d0>] ? woken_wake_function+0x10/0x10
[<ffffffff814dfdbf>] tcp_sendmsg+0xb6f/0xd70
[<ffffffff815096cf>] inet_sendmsg+0xdf/0x100
[<ffffffff815095f0>] ? inet_recvmsg+0x100/0x100
[<ffffffff814896d7>] sock_sendmsg+0x67/0x90
[<ffffffff810fd961>] ? might_fault+0x51/0xb0
[<ffffffff81489a22>] ___sys_sendmsg+0x2d2/0x2e0
[<ffffffff81095e58>] ? futex_wake+0x128/0x140
[<ffffffff81095d31>] ? futex_wake+0x1/0x140
[<ffffffff81141dd0>] ? do_dup2+0xd0/0xd0
[<ffffffff8105fa31>] ? get_parent_ip+0x11/0x50
[<ffffffff813cea27>] ? debug_smp_processor_id+0x17/0x20
[<ffffffff813c33c5>] ? delay_tsc+0x85/0xb0
[<ffffffff81141ead>] ? __fget+0xdd/0xf0
[<ffffffff81141dd0>] ? do_dup2+0xd0/0xd0
[<ffffffff81141f05>] ? __fget_light+0x45/0x60
[<ffffffff81141f2e>] ? __fdget+0xe/0x10
[<ffffffff8148a4e4>] __sys_sendmsg+0x44/0x70
[<ffffffff8148a519>] SyS_sendmsg+0x9/0x10
[<ffffffff81575b92>] system_call_fastpath+0x16/0x1b
WARNING: CPU: 0 PID: 380 at kernel/sched/core.c:7094 __might_sleep+0x5b/0x1e0()
do not call blocking ops when !TASK_RUNNING; state=1 set at
[<ffffffff81070640>] prepare_to_wait+0x50/0xa0
Modules linked in:
CPU: 0 PID: 380 Comm: kswapd0 Tainted: G W 3.16.0-vm+ #109
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
0000000000001bb6 ffff88007b64bc68 ffffffff8156ec1d 0000000000000000
ffff88007b64bcb8 ffff88007b64bca8 ffffffff81032c27 0000000000000000
0000000000000000 ffff88007c062060 0000000000000065 ffffffff8179ca1f
Call Trace:
[<ffffffff8156ec1d>] dump_stack+0x4f/0x7c
[<ffffffff81032c27>] warn_slowpath_common+0x87/0xb0
[<ffffffff81032cf1>] warn_slowpath_fmt+0x41/0x50
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff8105b53b>] __might_sleep+0x5b/0x1e0
[<ffffffff810f7fd3>] __reset_isolation_suitable+0x83/0x140
[<ffffffff810f83f3>] reset_isolation_suitable+0x33/0x50
[<ffffffff810eb717>] kswapd+0x2e7/0x4d0
[<ffffffff810701d0>] ? woken_wake_function+0x10/0x10
[<ffffffff810eb430>] ? balance_pgdat+0x5b0/0x5b0
[<ffffffff810539ab>] kthread+0xfb/0x110
[<ffffffff810538b0>] ? flush_kthread_worker+0x130/0x130
[<ffffffff81575aec>] ret_from_fork+0x7c/0xb0
[<ffffffff810538b0>] ? flush_kthread_worker+0x130/0x130
Thanks,
Ilya
WARNING: multiple messages have this Message-ID (diff)
From: Ilya Dryomov <ilya.dryomov@inktank.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@kernel.org>,
oleg@redhat.com, Linus Torvalds <torvalds@linux-foundation.org>,
tglx@linutronix.de, Mike Galbraith <umgwanakikbuti@gmail.com>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
netdev@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [RFC][PATCH 0/7] nested sleeps, fixes and debug infra
Date: Tue, 5 Aug 2014 12:33:16 +0400 [thread overview]
Message-ID: <CALFYKtBo2p5uNtkJZOy_rN7JbdFs1RbB1OfcF7TR+qDaMU0Kvg@mail.gmail.com> (raw)
In-Reply-To: <20140804103025.478913141@infradead.org>
On Mon, Aug 4, 2014 at 2:30 PM, Peter Zijlstra <peterz@infradead.org> wrote:
> Hi,
>
> Ilya recently tripped over a nested sleep which made Ingo suggest we should
> have debug checks for that. So I did some, see patch 7. Of course that
> triggered a whole bunch of fail the instant I tried to boot my machine.
>
> With this series I can boot my test box and build a kernel on it, I'm fairly
> sure that's far too limited a test to have found all, but its a start.
FWIW, I'm getting a lot of these during light rbd testing. CC'ed
netdev and linux-mm.
WARNING: CPU: 2 PID: 1978 at kernel/sched/core.c:7094 __might_sleep+0x5b/0x1e0()
do not call blocking ops when !TASK_RUNNING; state=1 set at
[<ffffffff81070640>] prepare_to_wait+0x50/0xa0
Modules linked in:
CPU: 2 PID: 1978 Comm: ceph-osd Not tainted 3.16.0-vm+ #109
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
0000000000001bb6 ffff8800126739e8 ffffffff8156ec1d 0000000000000000
ffff880012673a38 ffff880012673a28 ffffffff81032c27 ffff880012673a58
0000000000000200 ffff8800150fa060 00000000000007ad ffffffff817ed352
Call Trace:
[<ffffffff8156ec1d>] dump_stack+0x4f/0x7c
[<ffffffff81032c27>] warn_slowpath_common+0x87/0xb0
[<ffffffff81032cf1>] warn_slowpath_fmt+0x41/0x50
[<ffffffff814f23cf>] ? tcp_v4_do_rcv+0x10f/0x4a0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff8105b53b>] __might_sleep+0x5b/0x1e0
[<ffffffff8148d73d>] release_sock+0x13d/0x200
[<ffffffff81498223>] sk_stream_wait_memory+0x133/0x2d0
[<ffffffff810701d0>] ? woken_wake_function+0x10/0x10
[<ffffffff814dfdbf>] tcp_sendmsg+0xb6f/0xd70
[<ffffffff815096cf>] inet_sendmsg+0xdf/0x100
[<ffffffff815095f0>] ? inet_recvmsg+0x100/0x100
[<ffffffff814896d7>] sock_sendmsg+0x67/0x90
[<ffffffff810fd961>] ? might_fault+0x51/0xb0
[<ffffffff81489a22>] ___sys_sendmsg+0x2d2/0x2e0
[<ffffffff81095e58>] ? futex_wake+0x128/0x140
[<ffffffff81095d31>] ? futex_wake+0x1/0x140
[<ffffffff81141dd0>] ? do_dup2+0xd0/0xd0
[<ffffffff8105fa31>] ? get_parent_ip+0x11/0x50
[<ffffffff813cea27>] ? debug_smp_processor_id+0x17/0x20
[<ffffffff813c33c5>] ? delay_tsc+0x85/0xb0
[<ffffffff81141ead>] ? __fget+0xdd/0xf0
[<ffffffff81141dd0>] ? do_dup2+0xd0/0xd0
[<ffffffff81141f05>] ? __fget_light+0x45/0x60
[<ffffffff81141f2e>] ? __fdget+0xe/0x10
[<ffffffff8148a4e4>] __sys_sendmsg+0x44/0x70
[<ffffffff8148a519>] SyS_sendmsg+0x9/0x10
[<ffffffff81575b92>] system_call_fastpath+0x16/0x1b
WARNING: CPU: 0 PID: 380 at kernel/sched/core.c:7094 __might_sleep+0x5b/0x1e0()
do not call blocking ops when !TASK_RUNNING; state=1 set at
[<ffffffff81070640>] prepare_to_wait+0x50/0xa0
Modules linked in:
CPU: 0 PID: 380 Comm: kswapd0 Tainted: G W 3.16.0-vm+ #109
Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
0000000000001bb6 ffff88007b64bc68 ffffffff8156ec1d 0000000000000000
ffff88007b64bcb8 ffff88007b64bca8 ffffffff81032c27 0000000000000000
0000000000000000 ffff88007c062060 0000000000000065 ffffffff8179ca1f
Call Trace:
[<ffffffff8156ec1d>] dump_stack+0x4f/0x7c
[<ffffffff81032c27>] warn_slowpath_common+0x87/0xb0
[<ffffffff81032cf1>] warn_slowpath_fmt+0x41/0x50
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff81070640>] ? prepare_to_wait+0x50/0xa0
[<ffffffff8105b53b>] __might_sleep+0x5b/0x1e0
[<ffffffff810f7fd3>] __reset_isolation_suitable+0x83/0x140
[<ffffffff810f83f3>] reset_isolation_suitable+0x33/0x50
[<ffffffff810eb717>] kswapd+0x2e7/0x4d0
[<ffffffff810701d0>] ? woken_wake_function+0x10/0x10
[<ffffffff810eb430>] ? balance_pgdat+0x5b0/0x5b0
[<ffffffff810539ab>] kthread+0xfb/0x110
[<ffffffff810538b0>] ? flush_kthread_worker+0x130/0x130
[<ffffffff81575aec>] ret_from_fork+0x7c/0xb0
[<ffffffff810538b0>] ? flush_kthread_worker+0x130/0x130
Thanks,
Ilya
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2014-08-05 8:33 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-08-04 10:30 [RFC][PATCH 0/7] nested sleeps, fixes and debug infra Peter Zijlstra
2014-08-04 10:30 ` [RFC][PATCH 1/7] wait: Provide infrastructure to deal with nested blocking Peter Zijlstra
2014-08-04 13:44 ` Peter Zijlstra
2014-08-04 18:35 ` Oleg Nesterov
2014-08-04 10:30 ` [RFC][PATCH 2/7] wait: Provide Peter Zijlstra
2014-08-04 10:30 ` [RFC][PATCH 3/7] exit: Desl with nested sleeps Peter Zijlstra
2014-08-04 18:53 ` Oleg Nesterov
2014-08-04 10:30 ` [RFC][PATCH 4/7] inotify: Deal " Peter Zijlstra
2014-08-04 19:23 ` Oleg Nesterov
2014-08-04 21:02 ` Peter Zijlstra
2014-08-05 2:22 ` Lai Jiangshan
2014-08-05 7:28 ` Peter Zijlstra
2014-08-04 10:30 ` [RFC][PATCH 5/7] tty: " Peter Zijlstra
2014-08-05 23:29 ` Greg Kroah-Hartman
2014-08-04 10:30 ` [RFC][PATCH 6/7] smp: Correctly deal " Peter Zijlstra
2014-08-04 10:30 ` [RFC][PATCH 7/7] sched: Debug " Peter Zijlstra
2014-08-05 8:33 ` Ilya Dryomov [this message]
2014-08-05 8:33 ` [RFC][PATCH 0/7] nested sleeps, fixes and debug infra Ilya Dryomov
2014-08-05 13:06 ` Peter Zijlstra
2014-08-06 7:51 ` Ilya Dryomov
2014-08-06 7:51 ` Ilya Dryomov
2014-08-06 8:31 ` Peter Zijlstra
2014-08-06 21:16 ` David Miller
2014-08-06 21:16 ` David Miller
2014-08-07 8:10 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CALFYKtBo2p5uNtkJZOy_rN7JbdFs1RbB1OfcF7TR+qDaMU0Kvg@mail.gmail.com \
--to=ilya.dryomov@inktank.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=umgwanakikbuti@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.