All of lore.kernel.org
 help / color / mirror / Atom feed
From: TomK <tk@mdevsys.com>
To: linux-scsi@vger.kernel.org
Subject: Re: Kernel 4.8.4: INFO: task kworker/u16:8:289 blocked for more than 120 seconds.
Date: Mon, 24 Oct 2016 00:45:22 -0400	[thread overview]
Message-ID: <0851dba9-ad6c-c419-dc92-93c5d7d2f54a@mdevsys.com> (raw)
In-Reply-To: <64ef386e-8de2-0b1c-9322-f02530a3e783@mdevsys.com>

On 10/24/2016 12:32 AM, TomK wrote:
> On 10/23/2016 10:03 PM, TomK wrote:
>> Hey,
>>
>> Has anyone seen this and could have a workaround?  Seems like it is more
>> Kernel related with various apps not just target apparently not but
>> wondering if there is an interim solution
>> (https://access.redhat.com/solutions/408833)
>>
>> Getting this message after few minutes of usage from the QLA2xxx driver.
>>  This is after some activity on an ESXi server (15 VM's) that I'm
>> connecting to this HBA.  I've tried the following tuning parameters but
>> there was no change in behaviour:
>>
>> vm.dirty_background_ratio = 5
>> vm.dirty_ratio = 10
>>
>> Details:
>>
>>
>> Oct 23 21:28:25 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:28:29 mbpc-pc kernel: ABORT_TASK: Found referenced qla2xxx
>> task_tag: 1128612
>> Oct 23 21:28:42 mbpc-pc kernel: ABORT_TASK: Sending
>> TMR_FUNCTION_COMPLETE for ref_tag: 1128612
>> Oct 23 21:28:42 mbpc-pc kernel: ABORT_TASK: Found referenced qla2xxx
>> task_tag: 1129116
>> Jan  6 23:52:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:30:18 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Jan  6 23:54:01 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:32:16 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:32:24 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:32:24 mbpc-pc kernel:      Not tainted 4.8.4 #2
>> Oct 23 21:32:24 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:32:24 mbpc-pc kernel: kworker/u16:8   D ffff88011113ba18     0
>>   289      2 0x00000000
>> Oct 23 21:32:24 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:32:24 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:32:24 mbpc-pc kernel: Call Trace:
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8115cc5c>] ? printk+0x46/0x48
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f70c4>]
>> transport_wait_for_tasks+0x44/0x60 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f46e2>]
>> core_tmr_abort_task+0xf2/0x160 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f6aa4>]
>> target_tmr_work+0x154/0x160 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:32:24 mbpc-pc kernel: INFO: task kworker/1:48:6089 blocked for
>> more than 120 seconds.
>> Oct 23 21:32:24 mbpc-pc kernel:      Not tainted 4.8.4 #2
>> Oct 23 21:32:24 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:32:24 mbpc-pc kernel: kworker/1:48    D ffff88004017f968     0
>>  6089      2 0x00000080
>> Oct 23 21:32:24 mbpc-pc kernel: Workqueue: events qlt_free_session_done
>> [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88004017f968 ffff88004017f8f8
>> ffff88011a83a300 0000000000000004
>> Oct 23 21:32:24 mbpc-pc kernel: ffff88004017a600 ffff88004017f938
>> ffffffff810a0bb6 ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: ffff880110fd0840 ffff880000000000
>> ffffffff81090728 ffff880100000000
>> Oct 23 21:32:24 mbpc-pc kernel: Call Trace:
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a0bb6>] ?
>> enqueue_task_fair+0x66/0x410
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81090728>] ?
>> check_preempt_curr+0x78/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109075d>] ?
>> ttwu_do_wakeup+0x1d/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81090de0>] ?
>> ttwu_queue+0x180/0x190
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa08f76ad>]
>> target_wait_for_sess_cmds+0x4d/0x1b0 [target_core_mod]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa00e7188>] ?
>> qla2x00_post_work+0x58/0x70 [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa0286f69>]
>> tcm_qla2xxx_free_session+0x49/0x90 [tcm_qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffffa01447e9>]
>> qlt_free_session_done+0xf9/0x3d0 [qla2xxx]
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff815092fc>] ?
>> dbs_work_handler+0x5c/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8107f960>] ?
>> pwq_dec_nr_in_flight+0x50/0xa0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810d060c>] ?
>> del_timer_sync+0x4c/0x60
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108131e>] ?
>> maybe_create_worker+0x8e/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:32:24 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Jan  6 23:56:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:34:22 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:34:22 mbpc-pc kernel: hpet1: lost 3 rtc interrupts
>> Oct 23 21:34:27 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:34:27 mbpc-pc kernel:      Not tainted 4.8.4 #2
>> Oct 23 21:34:27 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:34:27 mbpc-pc kernel: kworker/u16:8   D ffff88011113ba18     0
>>   289      2 0x00000000
>> Oct 23 21:34:27 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:34:27 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:34:27 mbpc-pc kernel: Call Trace:
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8115cc5c>] ? printk+0x46/0x48
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f70c4>]
>> transport_wait_for_tasks+0x44/0x60 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f46e2>]
>> core_tmr_abort_task+0xf2/0x160 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f6aa4>]
>> target_tmr_work+0x154/0x160 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:34:27 mbpc-pc kernel: INFO: task kworker/1:48:6089 blocked for
>> more than 120 seconds.
>> Oct 23 21:34:27 mbpc-pc kernel:      Not tainted 4.8.4 #2
>> Oct 23 21:34:27 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:34:27 mbpc-pc kernel: kworker/1:48    D ffff88004017f968     0
>>  6089      2 0x00000080
>> Oct 23 21:34:27 mbpc-pc kernel: Workqueue: events qlt_free_session_done
>> [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88004017f968 ffff88004017f8f8
>> ffff88011a83a300 0000000000000004
>> Oct 23 21:34:27 mbpc-pc kernel: ffff88004017a600 ffff88004017f938
>> ffffffff810a0bb6 ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: ffff880110fd0840 ffff880000000000
>> ffffffff81090728 ffff880100000000
>> Oct 23 21:34:27 mbpc-pc kernel: Call Trace:
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a0bb6>] ?
>> enqueue_task_fair+0x66/0x410
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81090728>] ?
>> check_preempt_curr+0x78/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109075d>] ?
>> ttwu_do_wakeup+0x1d/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81090de0>] ?
>> ttwu_queue+0x180/0x190
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa08f76ad>]
>> target_wait_for_sess_cmds+0x4d/0x1b0 [target_core_mod]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa00e7188>] ?
>> qla2x00_post_work+0x58/0x70 [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa0286f69>]
>> tcm_qla2xxx_free_session+0x49/0x90 [tcm_qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffffa01447e9>]
>> qlt_free_session_done+0xf9/0x3d0 [qla2xxx]
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff815092fc>] ?
>> dbs_work_handler+0x5c/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8107f960>] ?
>> pwq_dec_nr_in_flight+0x50/0xa0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81080639>]
>> process_one_work+0x189/0x4e0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810d060c>] ?
>> del_timer_sync+0x4c/0x60
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108131e>] ?
>> maybe_create_worker+0x8e/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8108150d>]
>> worker_thread+0x16d/0x520
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810923f2>] ?
>> default_wake_function+0x12/0x20
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810a6f06>] ?
>> __wake_up_common+0x56/0x90
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff810813a0>] ?
>> maybe_create_worker+0x110/0x110
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8109130e>] ?
>> schedule_tail+0x1e/0xc0
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff8162f60f>]
>> ret_from_fork+0x1f/0x40
>> Oct 23 21:34:27 mbpc-pc kernel: [<ffffffff81085f20>] ?
>> kthread_freezable_should_stop+0x70/0x70
>> Oct 23 21:36:04 mbpc-pc kernel: hpet1: lost 9600 rtc interrupts
>> Oct 23 21:36:04 mbpc-pc kernel: hpet1: lost 3 rtc interrupts
>> Jan  6 23:58:00 192.168.0.2 syslog: dhcpfwd : dhcp forwarder daemon
>> successfully started
>> Oct 23 21:36:30 mbpc-pc kernel: INFO: task kworker/u16:8:289 blocked for
>> more than 120 seconds.
>> Oct 23 21:36:30 mbpc-pc kernel:      Not tainted 4.8.4 #2
>> Oct 23 21:36:30 mbpc-pc kernel: "echo 0 >
>> /proc/sys/kernel/hung_task_timeout_secs" disables this message.
>> Oct 23 21:36:30 mbpc-pc kernel: kworker/u16:8   D ffff88011113ba18     0
>>   289      2 0x00000000
>> Oct 23 21:36:30 mbpc-pc kernel: Workqueue: tmr-fileio target_tmr_work
>> [target_core_mod]
>> Oct 23 21:36:30 mbpc-pc kernel: ffff88011113ba18 0000000000000400
>> ffff880049e926c0 ffff88011113b998
>> Oct 23 21:36:30 mbpc-pc kernel: ffff880111134600 ffffffff81f99ca0
>> ffffffff81f998ef ffff880100000000
>> Oct 23 21:36:30 mbpc-pc kernel: ffffffff812f27d9 0000000000000000
>> ffffe8ffffc9a000 ffff880000000000
>> Oct 23 21:36:30 mbpc-pc kernel: Call Trace:
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff812f27d9>] ? number+0x2e9/0x310
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162c040>] schedule+0x40/0xb0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff81080169>] ?
>> start_flush_work+0x49/0x180
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162e7ec>]
>> schedule_timeout+0x9c/0xe0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810802ba>] ?
>> flush_work+0x1a/0x40
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810bd15c>] ?
>> console_unlock+0x35c/0x380
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff8162cfa0>]
>> wait_for_completion+0xc0/0xf0
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810923e0>] ?
>> try_to_wake_up+0x260/0x260
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffffa08f6f84>]
>> __transport_wait_for_tasks+0xb4/0x1b0 [target_core_mod]
>> Oct 23 21:36:30 mbpc-pc kernel: [<ffffffff810bdd1f>] ?
>> vprintk_default+0x1f/0x30
>>
>>
>
>
> Including the full log:
>
> http://microdevsys.com/linux-lio/messages-mailing-list
>


When tryint to shut down target using /etc/init.d/target stop, the 
following is printed repeatedly:

Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e837:20: 
ABTS_RECV_24XX: instance 0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f811:20: 
qla_target(0): task abort (s_id=1:5:0, tag=1177068, param=0)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f812:20: 
qla_target(0): task abort for non-existant session
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f80e:20: 
Scheduling work (type 1, prm ffff880093365680) to find session for param 
ffff88010f8c7680 (size 64, tgt ffff880111f06600)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-f800:20: Sess 
work (tgt ffff880111f06600)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e806:20: Sending 
task mgmt ABTS response (ha=ffff88010fae0000, atio=ffff880093365694, 
status=4
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e838:20: 
ABTS_RESP_24XX: compl_status 31
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e807:20: Sending 
retry TERM EXCH CTIO7 (ha=ffff88010fae0000)
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e806:20: Sending 
task mgmt ABTS response (ha=ffff88010fae0000, atio=ffff88010f8c76c0, 
status=0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e838:20: 
ABTS_RESP_24XX: compl_status 0
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e872:20: 
qlt_24xx_atio_pkt_all_vps: qla_target(0): type 6 ox_id 029c
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-3861:20: New 
command while device ffff880111f06600 is shutting down
Oct 24 00:39:48 mbpc-pc kernel: qla2xxx [0000:04:00.0]-e859:20: 
qla_target: Unable to send command to target for req, ignoring.



+ when I disable the ports on the brocade switch that we're using then 
try to stop target, the following is printed:



Oct 24 00:41:31 mbpc-pc kernel: qla2xxx [0000:04:00.1]-680a:21: Loop 
down - seconds remaining 231.
Oct 24 00:41:32 mbpc-pc kernel: qla2xxx [0000:04:00.0]-680a:20: Loop 
down - seconds remaining 153.
Oct 24 00:41:32 mbpc-pc kernel: ------------[ cut here ]------------
Oct 24 00:41:32 mbpc-pc kernel: WARNING: CPU: 2 PID: 8615 at 
lib/list_debug.c:33 __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: list_add corruption. prev->next should 
be next (ffff88009e83b330), but was ffff88011fc972a0. 
(prev=ffff880118ada4c0).
Oct 24 00:41:32 mbpc-pc kernel: Modules linked in: tcm_qla2xxx tcm_fc 
tcm_loop target_core_file target_core_iblock target_core_pscsi 
target_core_mod configfs ip6table_filter ip6_tables ebtable_nat ebtables 
nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_CHECKSUM 
iptable_mangle bridge nfsd lockd grace nfs_acl auth_rpcgss autofs4 it87 
hwmon_vid bnx2fc cnic uio fcoe libfcoe libfc 8021q garp stp llc ppdev 
parport_pc parport sunrpc cpufreq_ondemand bonding ipv6 crc_ccitt 
ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_tables fuse 
vfat fat xfs vhost_net macvtap macvlan vhost tun uinput raid456 
async_raid6_recov async_pq async_xor xor async_memcpy async_tx raid6_pq 
libcrc32c joydev sg serio_raw e1000 kvm_amd kvm irqbypass r8169 mii 
pcspkr k10temp snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel 
snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm 
snd_timer snd soundcore i2c_piix4 i2c_core wmi acpi_cpufreq shpchp ext4 
mbcache jbd2 qla2xxx scsi_transport_fc floppy firewire_ohci f
Oct 24 00:41:32 mbpc-pc kernel: irewire_core crc_itu_t sd_mod pata_acpi 
ata_generic pata_jmicron ahci libahci usb_storage dm_mirror 
dm_region_hash dm_log dm_mod
Oct 24 00:41:32 mbpc-pc kernel: CPU: 2 PID: 8615 Comm: kworker/2:3 Not 
tainted 4.8.4 #2
Oct 24 00:41:32 mbpc-pc kernel: Hardware name: Gigabyte Technology Co., 
Ltd. GA-890XA-UD3/GA-890XA-UD3, BIOS FC 08/02/2010
Oct 24 00:41:32 mbpc-pc kernel: 0000000000000000 ffff880092b83b48 
ffffffff812e88e9 ffffffff8130753e
Oct 24 00:41:32 mbpc-pc kernel: ffff880092b83ba8 ffff880092b83ba8 
0000000000000000 ffff880092b83b98
Oct 24 00:41:32 mbpc-pc kernel: ffffffff81066a7d ffff88000058f952 
0000002100000046 ffffffff8101eae8
Oct 24 00:41:32 mbpc-pc kernel: Call Trace:
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff812e88e9>] dump_stack+0x51/0x78
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130753e>] ? __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066a7d>] __warn+0xfd/0x120
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8101eae8>] ? 
__switch_to+0x398/0x7e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066b59>] 
warn_slowpath_fmt+0x49/0x50
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130753e>] __list_add+0xbe/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8107d0b2>] 
move_linked_works+0x62/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108070c>] 
process_one_work+0x25c/0x4e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108150d>] 
worker_thread+0x16d/0x520
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162bb3d>] ? 
__schedule+0x2fd/0x6a0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810923f2>] ? 
default_wake_function+0x12/0x20
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810a6f06>] ? 
__wake_up_common+0x56/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ? 
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ? 
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8109130e>] ? 
schedule_tail+0x1e/0xc0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162f60f>] ret_from_fork+0x1f/0x40
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085f20>] ? 
kthread_freezable_should_stop+0x70/0x70
Oct 24 00:41:32 mbpc-pc kernel: ---[ end trace 713a9071c9f5f464 ]---
Oct 24 00:41:32 mbpc-pc kernel: ------------[ cut here ]------------
Oct 24 00:41:32 mbpc-pc kernel: WARNING: CPU: 2 PID: 8615 at 
lib/list_debug.c:36 __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: list_add double add: 
new=ffff880118ada4c0, prev=ffff880118ada4c0, next=ffff88009e83b330.
Oct 24 00:41:32 mbpc-pc kernel: Modules linked in: tcm_qla2xxx tcm_fc 
tcm_loop target_core_file target_core_iblock target_core_pscsi 
target_core_mod configfs ip6table_filter ip6_tables ebtable_nat ebtables 
nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_CHECKSUM 
iptable_mangle bridge nfsd lockd grace nfs_acl auth_rpcgss autofs4 it87 
hwmon_vid bnx2fc cnic uio fcoe libfcoe libfc 8021q garp stp llc ppdev 
parport_pc parport sunrpc cpufreq_ondemand bonding ipv6 crc_ccitt 
ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter ip_tables fuse 
vfat fat xfs vhost_net macvtap macvlan vhost tun uinput raid456 
async_raid6_recov async_pq async_xor xor async_memcpy async_tx raid6_pq 
libcrc32c joydev sg serio_raw e1000 kvm_amd kvm irqbypass r8169 mii 
pcspkr k10temp snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel 
snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm 
snd_timer snd soundcore i2c_piix4 i2c_core wmi acpi_cpufreq shpchp ext4 
mbcache jbd2 qla2xxx scsi_transport_fc floppy firewire_ohci f
Oct 24 00:41:32 mbpc-pc kernel: irewire_core crc_itu_t sd_mod pata_acpi 
ata_generic pata_jmicron ahci libahci usb_storage dm_mirror 
dm_region_hash dm_log dm_mod
Oct 24 00:41:32 mbpc-pc kernel: CPU: 2 PID: 8615 Comm: kworker/2:3 
Tainted: G        W       4.8.4 #2
Oct 24 00:41:32 mbpc-pc kernel: Hardware name: Gigabyte Technology Co., 
Ltd. GA-890XA-UD3/GA-890XA-UD3, BIOS FC 08/02/2010
Oct 24 00:41:32 mbpc-pc kernel: 0000000000000000 ffff880092b83b48 
ffffffff812e88e9 ffffffff8130751c
Oct 24 00:41:32 mbpc-pc kernel: ffff880092b83ba8 ffff880092b83ba8 
0000000000000000 ffff880092b83b98
Oct 24 00:41:32 mbpc-pc kernel: ffffffff81066a7d ffff88000058f952 
0000002400000046 ffffffff8101eae8
Oct 24 00:41:32 mbpc-pc kernel: Call Trace:
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff812e88e9>] dump_stack+0x51/0x78
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130751c>] ? __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066a7d>] __warn+0xfd/0x120
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8101eae8>] ? 
__switch_to+0x398/0x7e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81066b59>] 
warn_slowpath_fmt+0x49/0x50
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8130751c>] __list_add+0x9c/0xd0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8107d0b2>] 
move_linked_works+0x62/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108070c>] 
process_one_work+0x25c/0x4e0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8108150d>] 
worker_thread+0x16d/0x520
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162bb3d>] ? 
__schedule+0x2fd/0x6a0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810923f2>] ? 
default_wake_function+0x12/0x20
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810a6f06>] ? 
__wake_up_common+0x56/0x90
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ? 
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162c040>] ? schedule+0x40/0xb0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff810813a0>] ? 
maybe_create_worker+0x110/0x110
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085fec>] kthread+0xcc/0xf0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8109130e>] ? 
schedule_tail+0x1e/0xc0
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff8162f60f>] ret_from_fork+0x1f/0x40
Oct 24 00:41:32 mbpc-pc kernel: [<ffffffff81085f20>] ? 
kthread_freezable_should_stop+0x70/0x70
Oct 24 00:41:32 mbpc-pc kernel: ---[ end trace 713a9071c9f5f465 ]---
Oct 24 00:41:32 mbpc-pc kernel: qla2xxx [0000:04:00.1]-680a:21: Loop 
down - seconds remaining 230.
Oct 24 00:41:33 mbpc-pc kernel: qla2xxx [0000:04:00.0]-680a:20: Loop 
down - seconds remaining 152.


-- 
Cheers,
Tom K.
-------------------------------------------------------------------------------------

Living on earth is expensive, but it includes a free trip around the sun.


  reply	other threads:[~2016-10-24  4:45 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-10-24  2:03 Kernel 4.8.4: INFO: task kworker/u16:8:289 blocked for more than 120 seconds TomK
2016-10-24  4:32 ` TomK
2016-10-24  4:45   ` TomK [this message]
2016-10-24  6:36     ` Nicholas A. Bellinger
2016-10-25  5:28       ` TomK
2016-10-26  2:05         ` TomK
2016-10-26  7:20           ` Nicholas A. Bellinger
2016-10-26 12:08             ` TomK
2016-10-28  6:01               ` TomK
2016-10-29  7:50                 ` Nicholas A. Bellinger
2016-10-29 18:10                   ` TomK
2016-10-29 21:44                     ` Nicholas A. Bellinger
2016-10-30 18:50                       ` TomK
2016-11-01  2:44                         ` TomK

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0851dba9-ad6c-c419-dc92-93c5d7d2f54a@mdevsys.com \
    --to=tk@mdevsys.com \
    --cc=linux-scsi@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.