Re: [bug report][bisected] WARNING: CPU: 109 PID: 739473 at block/blk-stat.c:218 blk_free_queue_stats+0x3c/0x80

[Yi Zhang <yi.zhang@redhat.com>]

Hi Christoph

It still can be reproduced with the patch, here is the log:
[   60.427575] run blktests block/001 at 2021-10-26 05:46:07
[   60.449111] scsi_debug:sdebug_driver_probe: scsi_debug: trim
poll_queues to 0. poll_q/nr_hw = (0/1)
[   60.458422] scsi_debug:sdebug_driver_probe: scsi_debug: trim
poll_queues to 0. poll_q/nr_hw = (0/1)
[   60.467694] sd 0:0:0:0: Power-on or device reset occurred
[   60.467719] scsi_debug:sdebug_driver_probe: scsi_debug: trim
poll_queues to 0. poll_q/nr_hw = (0/1)
[   60.467958] scsi_debug:sdebug_driver_probe: scsi_debug: trim
poll_queues to 0. poll_q/nr_hw = (0/1)
[   60.491409] sd 1:0:0:0: Power-on or device reset occurred
[   60.491596] sd 2:0:0:0: Power-on or device reset occurred
[   60.491893] sd 3:0:0:0: Power-on or device reset occurred
[   61.493843] sd 3:0:0:0: Power-on or device reset occurred
[   61.499283] sd 3:0:0:0: [sdd] Asking for cache data failed
[   61.504765] sd 3:0:0:0: [sdd] Assuming drive cache: write through
[   61.513449] sd 1:0:0:0: Power-on or device reset occurred
[   61.513608] sd 0:0:0:0: Power-on or device reset occurred
[   61.524250] sd 1:0:0:0: [sda] Asking for cache data failed
[   61.529722] sd 1:0:0:0: [sda] Assuming drive cache: write through
[   61.535811] sd 0:0:0:0: [sdb] Asking for cache data failed
[   61.541287] sd 0:0:0:0: [sdb] Assuming drive cache: write through
[   61.547537] sd 2:0:0:0: Power-on or device reset occurred
[   61.773630] ------------[ cut here ]------------
[   61.778238] WARNING: CPU: 77 PID: 1673 at block/blk-stat.c:218
blk_free_queue_stats+0x3c/0x80
[   61.786755] Modules linked in: scsi_debug rfkill sunrpc vfat fat
dm_service_time dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua
sbsa_gwdt acpi_ipmi ipmi_ssif igb ipmi_devintf ipmi_msghandler arm_cmn
cppc_cpufreq arm_dsu_pmu fuse zram ip_tables xfs ast i2c_algo_bit
drm_vram_helper drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops cec crct10dif_ce drm_ttm_helper ttm ghash_ce nvme drm
nvme_core xgene_hwmon aes_neon_bs
[   61.824812] CPU: 77 PID: 1673 Comm: check Not tainted 5.15.0-rc6.rc6.hch+ #4
[   61.831848] Hardware name: GIGABYTE R272-P30-JG/MP32-AR0-JG, BIOS
F07 03/22/2021
[   61.839231] pstate: 80400009 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   61.846181] pc : blk_free_queue_stats+0x3c/0x80
[   61.850699] lr : blk_release_queue+0x48/0x140
[   61.855045] sp : ffff800015cdbaa0
[   61.858346] x29: ffff800015cdbaa0 x28: ffff07ff84bfc200 x27: 0000000000000000
[   61.865470] x26: ffff07ff8f9d4428 x25: dead000000000100 x24: dead000000000122
[   61.872593] x23: ffff07ff8f9ec150 x22: 0000000000000000 x21: ffff07ff8c5f74e0
[   61.879716] x20: ffffc5e925264cd8 x19: ffff07ff8c5f7460 x18: ffffffffffffffff
[   61.886839] x17: 303a327465677261 x16: 742f3274736f682f x15: 0000000000000000
[   61.893962] x14: 0000000000000001 x13: 0000000000000040 x12: 0000000000000040
[   61.901085] x11: ffff07ff80407d78 x10: ffff07ff80407d7a x9 : ffffc5e92399b55c
[   61.908208] x8 : ffff07ff80403dc0 x7 : 0000000000000000 x6 : ffff07ff80403dd8
[   61.915331] x5 : 0000000000000000 x4 : 0000000000000003 x3 : 0000000000000000
[   61.922454] x2 : 0000000000002710 x1 : ffff07ffcfd60100 x0 : ffff07ff83732b00
[   61.929577] Call trace:
[   61.932011]  blk_free_queue_stats+0x3c/0x80
[   61.936182]  blk_release_queue+0x48/0x140
[   61.940179]  kobject_cleanup+0x4c/0x180
[   61.944004]  kobject_put+0x50/0xd0
[   61.947393]  blk_put_queue+0x20/0x30
[   61.950957]  scsi_device_dev_release_usercontext+0x160/0x244
[   61.956605]  execute_in_process_context+0x50/0xa0
[   61.961298]  scsi_device_dev_release+0x28/0x3c
[   61.965732]  device_release+0x40/0xa0
[   61.969384]  kobject_cleanup+0x4c/0x180
[   61.973207]  kobject_put+0x50/0xd0
[   61.976596]  put_device+0x20/0x30
[   61.979900]  scsi_device_put+0x38/0x50
[   61.983638]  sdev_store_delete+0x90/0xf0
[   61.987549]  dev_attr_store+0x24/0x40
[   61.991199]  sysfs_kf_write+0x50/0x60
[   61.994850]  kernfs_fop_write_iter+0x134/0x1c4
[   61.999282]  new_sync_write+0xdc/0x15c
[   62.003020]  vfs_write+0x230/0x2d0
[   62.006410]  ksys_write+0x64/0xec
[   62.009713]  __arm64_sys_write+0x28/0x34
[   62.013623]  invoke_syscall+0x50/0x120
[   62.017362]  el0_svc_common.constprop.0+0x4c/0x100
[   62.022142]  do_el0_svc+0x34/0xa0
[   62.025446]  el0_svc+0x30/0xd0
[   62.028490]  el0t_64_sync_handler+0xa4/0x130
[   62.032748]  el0t_64_sync+0x1a4/0x1a8
[   62.036399] ---[ end trace ecf3f33db601e65a ]---
[   62.041567] sd 2:0:0:0: Power-on or device reset occurred
[   62.045090] ------------[ cut here ]------------
[   62.045549] ------------[ cut here ]------------
[   62.045552] WARNING: CPU: 60 PID: 1672 at block/blk-stat.c:218
blk_free_queue_stats+0x3c/0x80



On Tue, Oct 26, 2021 at 4:36 PM Christoph Hellwig <hch@lst.de> wrote:
>
> Hi Yi,
>
> can you try the patch below?  This changes the teardown code to not
> re-enable writeback tracking when we're shutting the queue down, which
> is what I suspect is on the ->callbacks list.
>
> diff --git a/block/elevator.c b/block/elevator.c
> index ff45d8388f487..bb5c6ee4546cd 100644
> --- a/block/elevator.c
> +++ b/block/elevator.c
> @@ -523,8 +523,6 @@ void elv_unregister_queue(struct request_queue *q)
>                 kobject_del(&e->kobj);
>
>                 e->registered = 0;
> -               /* Re-enable throttling in case elevator disabled it */
> -               wbt_enable_default(q);
>         }
>  }
>
> @@ -591,8 +589,11 @@ int elevator_switch_mq(struct request_queue *q,
>         lockdep_assert_held(&q->sysfs_lock);
>
>         if (q->elevator) {
> -               if (q->elevator->registered)
> +               if (q->elevator->registered) {
>                         elv_unregister_queue(q);
> +                       /* Re-enable throttling in case elevator disabled it */
> +                       wbt_enable_default(q);
> +               }
>
>                 ioc_clear_queue(q);
>                 elevator_exit(q, q->elevator);
>


-- 
Best Regards,
  Yi Zhang