* WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
@ 2016-12-09 15:12 Michal Hocko
2016-12-12 22:03 ` Tejun Heo
0 siblings, 1 reply; 7+ messages in thread
From: Michal Hocko @ 2016-12-09 15:12 UTC (permalink / raw)
To: Tejun Heo; +Cc: LKML
Hi Tejun,
I am seeing the following warning during boot in my kvm testing
environment:
[ 3.549649] WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
[ 3.551279] do not call blocking ops when !TASK_RUNNING; state=2 set at [<ffffffff81098621>] prepare_to_wait+0x65/0xac
[ 3.553354] Modules linked in:
[ 3.554032] CPU: 3 PID: 1568 Comm: modprobe Not tainted 4.9.0-rc8-nofstest5-next-20161209-00013-g3134766e941a #1018
[ 3.556099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
[ 3.557330] Call Trace:
[ 3.557330] dump_stack+0x68/0x92
[ 3.557330] __warn+0xc2/0xdd
[ 3.557330] warn_slowpath_fmt+0x4b/0x53
[ 3.557330] ? prepare_to_wait+0x65/0xac
[ 3.557330] ? prepare_to_wait+0x65/0xac
[ 3.557330] __might_sleep+0x69/0x7e
[ 3.557330] do_ide_request+0x47/0x540
[ 3.557330] ? __lock_is_held+0x47/0x7a
[ 3.557330] ? hrtimer_try_to_cancel+0x15f/0x1c6
[ 3.557330] ? _raw_spin_unlock_irqrestore+0x4c/0x61
[ 3.557330] __blk_run_queue+0x31/0x3a
[ 3.557330] cfq_insert_request+0x331/0x429
[ 3.557330] __elv_add_request+0x2bb/0x346
[ 3.557330] blk_flush_plug_list+0x1da/0x21f
[ 3.557330] ? bit_wait+0x50/0x50
[ 3.557330] io_schedule_timeout+0x47/0x102
[ 3.557330] bit_wait_io+0x1b/0x5a
[ 3.557330] __wait_on_bit+0x4d/0x7b
[ 3.557330] out_of_line_wait_on_bit+0x72/0x7d
[ 3.557330] ? bit_wait+0x50/0x50
[ 3.557330] ? autoremove_wake_function+0x4e/0x4e
[ 3.557330] __wait_on_buffer+0x40/0x43
[ 3.557330] bh_submit_read+0x60/0x72
[ 3.557330] ext4_get_branch+0x91/0x115
[ 3.557330] ext4_ind_map_blocks+0x181/0xa16
[ 3.557330] ? __lock_is_held+0x47/0x7a
[ 3.557330] ? lock_acquire+0x133/0x1c7
[ 3.557330] ext4_map_blocks+0x120/0x4ca
[ 3.557330] ext4_mpage_readpages+0x2a7/0x70a
[ 3.557330] ext4_readpages+0x33/0x35
[ 3.557330] __do_page_cache_readahead+0x239/0x32a
[ 3.557330] filemap_fault+0x180/0x602
[ 3.557330] ? filemap_fault+0x180/0x602
[ 3.557330] ext4_filemap_fault+0x37/0x4b
[ 3.557330] __do_fault+0x1f/0x94
[ 3.557330] handle_mm_fault+0x946/0xfca
[ 3.557330] __do_page_fault+0x361/0x518
[ 3.557330] do_page_fault+0x42/0x5f
[ 3.557330] page_fault+0x22/0x30
At first I thought this would be related to you recent work
http://lkml.kernel.org/r/1477673892-28940-1-git-send-email-tj@kernel.org
but that doesn't seem to be in the linux-next I am using as a base.
io_schedule_timeout is on the stack though so you might have an idea
what is going on here. That might_sleep check has been added a long ago
02e7cf8f8488 ("ide-cd,atapi: use bio for internal commands") in 2.6.31
so I am wondering what has changed since than.
Thanks!
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-09 15:12 WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e Michal Hocko
@ 2016-12-12 22:03 ` Tejun Heo
2016-12-12 22:30 ` Cong Wang
2016-12-12 22:45 ` Shaohua Li
0 siblings, 2 replies; 7+ messages in thread
From: Tejun Heo @ 2016-12-12 22:03 UTC (permalink / raw)
To: Michal Hocko; +Cc: LKML, Shaohua Li
Hello, Michal.
On Fri, Dec 09, 2016 at 04:12:29PM +0100, Michal Hocko wrote:
> Hi Tejun,
> I am seeing the following warning during boot in my kvm testing
> environment:
> [ 3.549649] WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
> [ 3.551279] do not call blocking ops when !TASK_RUNNING; state=2 set at [<ffffffff81098621>] prepare_to_wait+0x65/0xac
> [ 3.553354] Modules linked in:
> [ 3.554032] CPU: 3 PID: 1568 Comm: modprobe Not tainted 4.9.0-rc8-nofstest5-next-20161209-00013-g3134766e941a #1018
> [ 3.556099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
> [ 3.557330] Call Trace:
> [ 3.557330] dump_stack+0x68/0x92
> [ 3.557330] __warn+0xc2/0xdd
> [ 3.557330] warn_slowpath_fmt+0x4b/0x53
> [ 3.557330] ? prepare_to_wait+0x65/0xac
> [ 3.557330] ? prepare_to_wait+0x65/0xac
> [ 3.557330] __might_sleep+0x69/0x7e
> [ 3.557330] do_ide_request+0x47/0x540
> [ 3.557330] ? __lock_is_held+0x47/0x7a
> [ 3.557330] ? hrtimer_try_to_cancel+0x15f/0x1c6
> [ 3.557330] ? _raw_spin_unlock_irqrestore+0x4c/0x61
> [ 3.557330] __blk_run_queue+0x31/0x3a
> [ 3.557330] cfq_insert_request+0x331/0x429
> [ 3.557330] __elv_add_request+0x2bb/0x346
> [ 3.557330] blk_flush_plug_list+0x1da/0x21f
> [ 3.557330] ? bit_wait+0x50/0x50
> [ 3.557330] io_schedule_timeout+0x47/0x102
> [ 3.557330] bit_wait_io+0x1b/0x5a
> [ 3.557330] __wait_on_bit+0x4d/0x7b
> [ 3.557330] out_of_line_wait_on_bit+0x72/0x7d
> [ 3.557330] ? bit_wait+0x50/0x50
> [ 3.557330] ? autoremove_wake_function+0x4e/0x4e
> [ 3.557330] __wait_on_buffer+0x40/0x43
> [ 3.557330] bh_submit_read+0x60/0x72
> [ 3.557330] ext4_get_branch+0x91/0x115
> [ 3.557330] ext4_ind_map_blocks+0x181/0xa16
> [ 3.557330] ? __lock_is_held+0x47/0x7a
> [ 3.557330] ? lock_acquire+0x133/0x1c7
> [ 3.557330] ext4_map_blocks+0x120/0x4ca
> [ 3.557330] ext4_mpage_readpages+0x2a7/0x70a
> [ 3.557330] ext4_readpages+0x33/0x35
> [ 3.557330] __do_page_cache_readahead+0x239/0x32a
> [ 3.557330] filemap_fault+0x180/0x602
> [ 3.557330] ? filemap_fault+0x180/0x602
> [ 3.557330] ext4_filemap_fault+0x37/0x4b
> [ 3.557330] __do_fault+0x1f/0x94
> [ 3.557330] handle_mm_fault+0x946/0xfca
> [ 3.557330] __do_page_fault+0x361/0x518
> [ 3.557330] do_page_fault+0x42/0x5f
> [ 3.557330] page_fault+0x22/0x30
>
> At first I thought this would be related to you recent work
> http://lkml.kernel.org/r/1477673892-28940-1-git-send-email-tj@kernel.org
> but that doesn't seem to be in the linux-next I am using as a base.
> io_schedule_timeout is on the stack though so you might have an idea
> what is going on here. That might_sleep check has been added a long ago
> 02e7cf8f8488 ("ide-cd,atapi: use bio for internal commands") in 2.6.31
> so I am wondering what has changed since than.
10d784eae2b4 ("sched: always use blk_schedule_flush_plug in
io_schedule_out") seems related. might_sleep is complaining that it's
being called while !TASK_RUNNING. Shaohua, any ideas?
Thanks.
--
tejun
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-12 22:03 ` Tejun Heo
@ 2016-12-12 22:30 ` Cong Wang
2016-12-12 22:45 ` Shaohua Li
1 sibling, 0 replies; 7+ messages in thread
From: Cong Wang @ 2016-12-12 22:30 UTC (permalink / raw)
To: Tejun Heo; +Cc: Michal Hocko, LKML, Shaohua Li
On Mon, Dec 12, 2016 at 2:03 PM, Tejun Heo <tj@kernel.org> wrote:
> Hello, Michal.
>
> On Fri, Dec 09, 2016 at 04:12:29PM +0100, Michal Hocko wrote:
>> At first I thought this would be related to you recent work
>> http://lkml.kernel.org/r/1477673892-28940-1-git-send-email-tj@kernel.org
>> but that doesn't seem to be in the linux-next I am using as a base.
>> io_schedule_timeout is on the stack though so you might have an idea
>> what is going on here. That might_sleep check has been added a long ago
>> 02e7cf8f8488 ("ide-cd,atapi: use bio for internal commands") in 2.6.31
>> so I am wondering what has changed since than.
>
> 10d784eae2b4 ("sched: always use blk_schedule_flush_plug in
> io_schedule_out") seems related. might_sleep is complaining that it's
> being called while !TASK_RUNNING. Shaohua, any ideas?
Yeah, it is pretty much similar to the one I fixed for networking:
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d9dc8b0f8b4ec8cdc48ad5a20a3105387138be82
But in your case, it is complicated because of the wait bits API.
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-12 22:03 ` Tejun Heo
2016-12-12 22:30 ` Cong Wang
@ 2016-12-12 22:45 ` Shaohua Li
2016-12-21 10:47 ` Michal Hocko
1 sibling, 1 reply; 7+ messages in thread
From: Shaohua Li @ 2016-12-12 22:45 UTC (permalink / raw)
To: Tejun Heo; +Cc: Michal Hocko, LKML
On Mon, Dec 12, 2016 at 05:03:49PM -0500, Tejun Heo wrote:
> Hello, Michal.
>
> On Fri, Dec 09, 2016 at 04:12:29PM +0100, Michal Hocko wrote:
> > Hi Tejun,
> > I am seeing the following warning during boot in my kvm testing
> > environment:
> > [ 3.549649] WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
> > [ 3.551279] do not call blocking ops when !TASK_RUNNING; state=2 set at [<ffffffff81098621>] prepare_to_wait+0x65/0xac
> > [ 3.553354] Modules linked in:
> > [ 3.554032] CPU: 3 PID: 1568 Comm: modprobe Not tainted 4.9.0-rc8-nofstest5-next-20161209-00013-g3134766e941a #1018
> > [ 3.556099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
> > [ 3.557330] Call Trace:
> > [ 3.557330] dump_stack+0x68/0x92
> > [ 3.557330] __warn+0xc2/0xdd
> > [ 3.557330] warn_slowpath_fmt+0x4b/0x53
> > [ 3.557330] ? prepare_to_wait+0x65/0xac
> > [ 3.557330] ? prepare_to_wait+0x65/0xac
> > [ 3.557330] __might_sleep+0x69/0x7e
> > [ 3.557330] do_ide_request+0x47/0x540
> > [ 3.557330] ? __lock_is_held+0x47/0x7a
> > [ 3.557330] ? hrtimer_try_to_cancel+0x15f/0x1c6
> > [ 3.557330] ? _raw_spin_unlock_irqrestore+0x4c/0x61
> > [ 3.557330] __blk_run_queue+0x31/0x3a
> > [ 3.557330] cfq_insert_request+0x331/0x429
> > [ 3.557330] __elv_add_request+0x2bb/0x346
> > [ 3.557330] blk_flush_plug_list+0x1da/0x21f
> > [ 3.557330] ? bit_wait+0x50/0x50
> > [ 3.557330] io_schedule_timeout+0x47/0x102
> > [ 3.557330] bit_wait_io+0x1b/0x5a
> > [ 3.557330] __wait_on_bit+0x4d/0x7b
> > [ 3.557330] out_of_line_wait_on_bit+0x72/0x7d
> > [ 3.557330] ? bit_wait+0x50/0x50
> > [ 3.557330] ? autoremove_wake_function+0x4e/0x4e
> > [ 3.557330] __wait_on_buffer+0x40/0x43
> > [ 3.557330] bh_submit_read+0x60/0x72
> > [ 3.557330] ext4_get_branch+0x91/0x115
> > [ 3.557330] ext4_ind_map_blocks+0x181/0xa16
> > [ 3.557330] ? __lock_is_held+0x47/0x7a
> > [ 3.557330] ? lock_acquire+0x133/0x1c7
> > [ 3.557330] ext4_map_blocks+0x120/0x4ca
> > [ 3.557330] ext4_mpage_readpages+0x2a7/0x70a
> > [ 3.557330] ext4_readpages+0x33/0x35
> > [ 3.557330] __do_page_cache_readahead+0x239/0x32a
> > [ 3.557330] filemap_fault+0x180/0x602
> > [ 3.557330] ? filemap_fault+0x180/0x602
> > [ 3.557330] ext4_filemap_fault+0x37/0x4b
> > [ 3.557330] __do_fault+0x1f/0x94
> > [ 3.557330] handle_mm_fault+0x946/0xfca
> > [ 3.557330] __do_page_fault+0x361/0x518
> > [ 3.557330] do_page_fault+0x42/0x5f
> > [ 3.557330] page_fault+0x22/0x30
> >
> > At first I thought this would be related to you recent work
> > http://lkml.kernel.org/r/1477673892-28940-1-git-send-email-tj@kernel.org
> > but that doesn't seem to be in the linux-next I am using as a base.
> > io_schedule_timeout is on the stack though so you might have an idea
> > what is going on here. That might_sleep check has been added a long ago
> > 02e7cf8f8488 ("ide-cd,atapi: use bio for internal commands") in 2.6.31
> > so I am wondering what has changed since than.
>
> 10d784eae2b4 ("sched: always use blk_schedule_flush_plug in
> io_schedule_out") seems related. might_sleep is complaining that it's
> being called while !TASK_RUNNING. Shaohua, any ideas?
I can't see how the patch is related. Both blk_flush_plug and
blk_schedule_flush_plug can sleep. The patch should only change behavior for plug which accepts from_schedule, which IIRC only raid code uses it.
Thanks,
Shaohua
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-12 22:45 ` Shaohua Li
@ 2016-12-21 10:47 ` Michal Hocko
2016-12-21 19:24 ` Cong Wang
0 siblings, 1 reply; 7+ messages in thread
From: Michal Hocko @ 2016-12-21 10:47 UTC (permalink / raw)
To: Shaohua Li; +Cc: Tejun Heo, LKML
On Mon 12-12-16 14:45:31, Shaohua Li wrote:
> On Mon, Dec 12, 2016 at 05:03:49PM -0500, Tejun Heo wrote:
> > Hello, Michal.
> >
> > On Fri, Dec 09, 2016 at 04:12:29PM +0100, Michal Hocko wrote:
> > > Hi Tejun,
> > > I am seeing the following warning during boot in my kvm testing
> > > environment:
> > > [ 3.549649] WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
> > > [ 3.551279] do not call blocking ops when !TASK_RUNNING; state=2 set at [<ffffffff81098621>] prepare_to_wait+0x65/0xac
> > > [ 3.553354] Modules linked in:
> > > [ 3.554032] CPU: 3 PID: 1568 Comm: modprobe Not tainted 4.9.0-rc8-nofstest5-next-20161209-00013-g3134766e941a #1018
> > > [ 3.556099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.9.3-20161025_171302-gandalf 04/01/2014
> > > [ 3.557330] Call Trace:
> > > [ 3.557330] dump_stack+0x68/0x92
> > > [ 3.557330] __warn+0xc2/0xdd
> > > [ 3.557330] warn_slowpath_fmt+0x4b/0x53
> > > [ 3.557330] ? prepare_to_wait+0x65/0xac
> > > [ 3.557330] ? prepare_to_wait+0x65/0xac
> > > [ 3.557330] __might_sleep+0x69/0x7e
> > > [ 3.557330] do_ide_request+0x47/0x540
> > > [ 3.557330] ? __lock_is_held+0x47/0x7a
> > > [ 3.557330] ? hrtimer_try_to_cancel+0x15f/0x1c6
> > > [ 3.557330] ? _raw_spin_unlock_irqrestore+0x4c/0x61
> > > [ 3.557330] __blk_run_queue+0x31/0x3a
> > > [ 3.557330] cfq_insert_request+0x331/0x429
> > > [ 3.557330] __elv_add_request+0x2bb/0x346
> > > [ 3.557330] blk_flush_plug_list+0x1da/0x21f
> > > [ 3.557330] ? bit_wait+0x50/0x50
> > > [ 3.557330] io_schedule_timeout+0x47/0x102
> > > [ 3.557330] bit_wait_io+0x1b/0x5a
> > > [ 3.557330] __wait_on_bit+0x4d/0x7b
> > > [ 3.557330] out_of_line_wait_on_bit+0x72/0x7d
> > > [ 3.557330] ? bit_wait+0x50/0x50
> > > [ 3.557330] ? autoremove_wake_function+0x4e/0x4e
> > > [ 3.557330] __wait_on_buffer+0x40/0x43
> > > [ 3.557330] bh_submit_read+0x60/0x72
> > > [ 3.557330] ext4_get_branch+0x91/0x115
> > > [ 3.557330] ext4_ind_map_blocks+0x181/0xa16
> > > [ 3.557330] ? __lock_is_held+0x47/0x7a
> > > [ 3.557330] ? lock_acquire+0x133/0x1c7
> > > [ 3.557330] ext4_map_blocks+0x120/0x4ca
> > > [ 3.557330] ext4_mpage_readpages+0x2a7/0x70a
> > > [ 3.557330] ext4_readpages+0x33/0x35
> > > [ 3.557330] __do_page_cache_readahead+0x239/0x32a
> > > [ 3.557330] filemap_fault+0x180/0x602
> > > [ 3.557330] ? filemap_fault+0x180/0x602
> > > [ 3.557330] ext4_filemap_fault+0x37/0x4b
> > > [ 3.557330] __do_fault+0x1f/0x94
> > > [ 3.557330] handle_mm_fault+0x946/0xfca
> > > [ 3.557330] __do_page_fault+0x361/0x518
> > > [ 3.557330] do_page_fault+0x42/0x5f
> > > [ 3.557330] page_fault+0x22/0x30
> > >
> > > At first I thought this would be related to you recent work
> > > http://lkml.kernel.org/r/1477673892-28940-1-git-send-email-tj@kernel.org
> > > but that doesn't seem to be in the linux-next I am using as a base.
> > > io_schedule_timeout is on the stack though so you might have an idea
> > > what is going on here. That might_sleep check has been added a long ago
> > > 02e7cf8f8488 ("ide-cd,atapi: use bio for internal commands") in 2.6.31
> > > so I am wondering what has changed since than.
> >
> > 10d784eae2b4 ("sched: always use blk_schedule_flush_plug in
> > io_schedule_out") seems related. might_sleep is complaining that it's
> > being called while !TASK_RUNNING. Shaohua, any ideas?
>
> I can't see how the patch is related. Both blk_flush_plug and
> blk_schedule_flush_plug can sleep. The patch should only change
> behavior for plug which accepts from_schedule, which IIRC only raid
> code uses it.
Is there anything I could test?
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-21 10:47 ` Michal Hocko
@ 2016-12-21 19:24 ` Cong Wang
2016-12-21 20:57 ` Michal Hocko
0 siblings, 1 reply; 7+ messages in thread
From: Cong Wang @ 2016-12-21 19:24 UTC (permalink / raw)
To: Michal Hocko; +Cc: Shaohua Li, Tejun Heo, LKML
On Wed, Dec 21, 2016 at 2:47 AM, Michal Hocko <mhocko@kernel.org> wrote:
>
> Is there anything I could test?
I am working on a patchset, will send them for you to test. Meanwhile,
if you could provide me a reproducer, that would help a lot.
Thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e
2016-12-21 19:24 ` Cong Wang
@ 2016-12-21 20:57 ` Michal Hocko
0 siblings, 0 replies; 7+ messages in thread
From: Michal Hocko @ 2016-12-21 20:57 UTC (permalink / raw)
To: Cong Wang; +Cc: Shaohua Li, Tejun Heo, LKML
On Wed 21-12-16 11:24:47, Cong Wang wrote:
> On Wed, Dec 21, 2016 at 2:47 AM, Michal Hocko <mhocko@kernel.org> wrote:
> >
> > Is there anything I could test?
>
> I am working on a patchset, will send them for you to test. Meanwhile,
> if you could provide me a reproducer, that would help a lot.
I just boot my kvm image. Nothing really special here.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-12-21 20:57 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-12-09 15:12 WARNING: CPU: 3 PID: 1568 at kernel/sched/core.c:7738 __might_sleep+0x69/0x7e Michal Hocko
2016-12-12 22:03 ` Tejun Heo
2016-12-12 22:30 ` Cong Wang
2016-12-12 22:45 ` Shaohua Li
2016-12-21 10:47 ` Michal Hocko
2016-12-21 19:24 ` Cong Wang
2016-12-21 20:57 ` Michal Hocko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).