All of lore.kernel.org
 help / color / mirror / Atom feed
* v4.16-rc2: I/O hang with dm-rq + Kyber
@ 2018-02-22 21:10 Bart Van Assche
  2018-02-22 22:42 ` Omar Sandoval
  2018-02-23 16:26 ` Ming Lei
  0 siblings, 2 replies; 5+ messages in thread
From: Bart Van Assche @ 2018-02-22 21:10 UTC (permalink / raw)
  To: osandov; +Cc: linux-block

[-- Attachment #1: Type: text/plain, Size: 474 bytes --]

Hello Omar,

I/O hangs if I run the following command on top of kernel v4.16-rc2 + the
ib_srpt patch that adds RDMA/CM support:

srp-test/run_tests -c -d -r 10 -t 02-mq -e kyber

This does not happen with the deadline scheduler nor without a scheduler.
This test passed a few months ago. I have attached the output of the following
command to this e-mail:

(cd /sys/kernel/debug/block && grep -r .) | grep -v /poll_stat

Can you have a look?

Thanks,

Bart.

[-- Attachment #2: pending --]
[-- Type: text/plain, Size: 60124 bytes --]

dm-1/sched/async_depth:48
dm-1/sched/other_tokens:depth=64
dm-1/sched/other_tokens:busy=0
dm-1/sched/other_tokens:bits_per_word=64
dm-1/sched/other_tokens:map_nr=1
dm-1/sched/other_tokens:alloc_hint={33, 1415, 1816, 307}
dm-1/sched/other_tokens:wake_batch=8
dm-1/sched/other_tokens:wake_index=0
dm-1/sched/other_tokens:ws={
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/other_tokens:}
dm-1/sched/other_tokens:round_robin=0
dm-1/sched/sync_write_tokens:depth=128
dm-1/sched/sync_write_tokens:busy=128
dm-1/sched/sync_write_tokens:bits_per_word=64
dm-1/sched/sync_write_tokens:map_nr=2
dm-1/sched/sync_write_tokens:alloc_hint={85, 0, 0, 90}
dm-1/sched/sync_write_tokens:wake_batch=8
dm-1/sched/sync_write_tokens:wake_index=2
dm-1/sched/sync_write_tokens:ws={
dm-1/sched/sync_write_tokens:	{.wait_cnt=1, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=active},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/sync_write_tokens:}
dm-1/sched/sync_write_tokens:round_robin=0
dm-1/sched/read_tokens:depth=256
dm-1/sched/read_tokens:busy=256
dm-1/sched/read_tokens:bits_per_word=64
dm-1/sched/read_tokens:map_nr=4
dm-1/sched/read_tokens:alloc_hint={89, 0, 90, 64}
dm-1/sched/read_tokens:wake_batch=8
dm-1/sched/read_tokens:wake_index=0
dm-1/sched/read_tokens:ws={
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=active},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:	{.wait_cnt=8, .wait=inactive},
dm-1/sched/read_tokens:}
dm-1/sched/read_tokens:round_robin=0
dm-1/hctx0/sched/batching:0
dm-1/hctx0/sched/cur_domain:READ
dm-1/hctx0/sched/other_waiting:0
dm-1/hctx0/sched/sync_write_waiting:1
dm-1/hctx0/sched/sync_write_rqs:000000008b7da875 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x260/0x0, .tag=-1, .internal_tag=48}
dm-1/hctx0/sched/sync_write_rqs:000000004b242673 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x1b4/0x0, .tag=-1, .internal_tag=46}
dm-1/hctx0/sched/sync_write_rqs:0000000083108835 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=183}
dm-1/hctx0/sched/sync_write_rqs:00000000b01459f7 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x2a4/0x0, .tag=-1, .internal_tag=54}
dm-1/hctx0/sched/sync_write_rqs:00000000f56c232f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x23c/0x0, .tag=-1, .internal_tag=62}
dm-1/hctx0/sched/sync_write_rqs:000000008ccce380 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x290/0x0, .tag=-1, .internal_tag=28}
dm-1/hctx0/sched/sync_write_rqs:0000000089435afc {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x15c/0x0, .tag=-1, .internal_tag=122}
dm-1/hctx0/sched/sync_write_rqs:00000000e683a4e2 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x268/0x0, .tag=-1, .internal_tag=127}
dm-1/hctx0/sched/sync_write_rqs:00000000c4ada57b {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x2e4/0x0, .tag=-1, .internal_tag=64}
dm-1/hctx0/sched/sync_write_rqs:00000000ab365972 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x230/0x0, .tag=-1, .internal_tag=101}
dm-1/hctx0/sched/sync_write_rqs:00000000c9e14c7b {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x22c/0x0, .tag=-1, .internal_tag=82}
dm-1/hctx0/sched/sync_write_rqs:0000000072115405 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x284/0x0, .tag=-1, .internal_tag=91}
dm-1/hctx0/sched/sync_write_rqs:000000003051d25c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x268/0x0, .tag=-1, .internal_tag=239}
dm-1/hctx0/sched/sync_write_rqs:000000006e79a598 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x214/0x0, .tag=-1, .internal_tag=220}
dm-1/hctx0/sched/sync_write_rqs:0000000065192a1c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x19c/0x0, .tag=-1, .internal_tag=206}
dm-1/hctx0/sched/sync_write_rqs:000000008f158a81 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=208}
dm-1/hctx0/sched/sync_write_rqs:00000000725d0b8f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x178/0x0, .tag=-1, .internal_tag=209}
dm-1/hctx0/sched/sync_write_rqs:00000000f4b47838 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x28c/0x0, .tag=-1, .internal_tag=212}
dm-1/hctx0/sched/sync_write_rqs:00000000e66c923a {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x1ac/0x0, .tag=-1, .internal_tag=223}
dm-1/hctx0/sched/sync_write_rqs:00000000852db7b7 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x1ec/0x0, .tag=-1, .internal_tag=240}
dm-1/hctx0/sched/sync_write_rqs:000000007bb5be5e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x240/0x0, .tag=-1, .internal_tag=248}
dm-1/hctx0/sched/sync_write_rqs:00000000775baee1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=251}
dm-1/hctx0/sched/sync_write_rqs:000000005e64185f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x1cc/0x0, .tag=-1, .internal_tag=192}
dm-1/hctx0/sched/sync_write_rqs:000000009d31f7ab {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x2b4/0x0, .tag=-1, .internal_tag=198}
dm-1/hctx0/sched/sync_write_rqs:000000004ee7e3a1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x258/0x0, .tag=-1, .internal_tag=205}
dm-1/hctx0/sched/sync_write_rqs:0000000027577848 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=24}
dm-1/hctx0/sched/sync_write_rqs:000000004591a711 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=31}
dm-1/hctx0/sched/sync_write_rqs:000000000706aac4 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x1f8/0x0, .tag=-1, .internal_tag=34}
dm-1/hctx0/sched/sync_write_rqs:000000005b228eef {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x384/0x0, .tag=-1, .internal_tag=49}
dm-1/hctx0/sched/sync_write_rqs:000000006dff26ba {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x298/0x0, .tag=-1, .internal_tag=61}
dm-1/hctx0/sched/sync_write_rqs:000000004b4c45a7 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=ELVPRIV|IO_STAT, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=174}
dm-1/hctx0/sched/sync_write_rqs:00000000d3a14798 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x264/0x0, .tag=-1, .internal_tag=16}
dm-1/hctx0/sched/sync_write_rqs:000000003d197adf {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x22c/0x0, .tag=-1, .internal_tag=47}
dm-1/hctx0/sched/sync_write_rqs:000000009b396ab5 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1dc/0x0, .tag=-1, .internal_tag=103}
dm-1/hctx0/sched/sync_write_rqs:000000004695516a {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x25c/0x0, .tag=-1, .internal_tag=115}
dm-1/hctx0/sched/sync_write_rqs:00000000a7e7a830 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c4/0x0, .tag=-1, .internal_tag=155}
dm-1/hctx0/sched/sync_write_rqs:0000000090082987 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x16c/0x0, .tag=-1, .internal_tag=171}
dm-1/hctx0/sched/sync_write_rqs:000000009079cd33 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=1}
dm-1/hctx0/sched/sync_write_rqs:00000000c36aa1fe {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1cc/0x0, .tag=-1, .internal_tag=73}
dm-1/hctx0/sched/sync_write_rqs:0000000004224383 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2f4/0x0, .tag=-1, .internal_tag=95}
dm-1/hctx0/sched/sync_write_rqs:000000007a083bfe {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x3bc/0x0, .tag=-1, .internal_tag=136}
dm-1/hctx0/sched/sync_write_rqs:00000000525fd360 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x180/0x0, .tag=-1, .internal_tag=237}
dm-1/hctx0/sched/sync_write_rqs:00000000abb21577 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x310/0x0, .tag=-1, .internal_tag=67}
dm-1/hctx0/sched/sync_write_rqs:000000005cf06964 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x294/0x0, .tag=-1, .internal_tag=85}
dm-1/hctx0/sched/sync_write_rqs:00000000b3397688 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c8/0x0, .tag=-1, .internal_tag=66}
dm-1/hctx0/sched/sync_write_rqs:0000000052254ee2 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x158/0x0, .tag=-1, .internal_tag=235}
dm-1/hctx0/sched/sync_write_rqs:0000000009bfcc60 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1ac/0x0, .tag=-1, .internal_tag=245}
dm-1/hctx0/sched/sync_write_rqs:000000008bfb7c45 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x258/0x0, .tag=-1, .internal_tag=252}
dm-1/hctx0/sched/sync_write_rqs:00000000f134b187 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x310/0x0, .tag=-1, .internal_tag=37}
dm-1/hctx0/sched/sync_write_rqs:000000002e39c8ba {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=74}
dm-1/hctx0/sched/sync_write_rqs:00000000b9b71d1c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x208/0x0, .tag=-1, .internal_tag=110}
dm-1/hctx0/sched/sync_write_rqs:00000000b43cbe0c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=210}
dm-1/hctx0/sched/sync_write_rqs:00000000b325502f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1bc/0x0, .tag=-1, .internal_tag=215}
dm-1/hctx0/sched/sync_write_rqs:00000000254cd553 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=222}
dm-1/hctx0/sched/sync_write_rqs:00000000c09fab9f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1b4/0x0, .tag=-1, .internal_tag=226}
dm-1/hctx0/sched/sync_write_rqs:0000000003270702 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=236}
dm-1/hctx0/sched/sync_write_rqs:000000006787fd88 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x208/0x0, .tag=-1, .internal_tag=6}
dm-1/hctx0/sched/sync_write_rqs:00000000c4041a1c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c8/0x0, .tag=-1, .internal_tag=18}
dm-1/hctx0/sched/sync_write_rqs:0000000032783010 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x24c/0x0, .tag=-1, .internal_tag=214}
dm-1/hctx0/sched/sync_write_rqs:000000003809f132 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2f4/0x0, .tag=-1, .internal_tag=197}
dm-1/hctx0/sched/sync_write_rqs:000000003811a321 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f4/0x0, .tag=-1, .internal_tag=195}
dm-1/hctx0/sched/sync_write_rqs:00000000d053cf06 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x148/0x0, .tag=-1, .internal_tag=230}
dm-1/hctx0/sched/sync_write_rqs:00000000a4772cc4 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x140/0x0, .tag=-1, .internal_tag=244}
dm-1/hctx0/sched/sync_write_rqs:0000000020dca4d5 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=86}
dm-1/hctx0/sched/sync_write_rqs:000000005554a0ce {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x29c/0x0, .tag=-1, .internal_tag=160}
dm-1/hctx0/sched/sync_write_rqs:0000000063e32ceb {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e0/0x0, .tag=-1, .internal_tag=89}
dm-1/hctx0/sched/sync_write_rqs:0000000043e2b562 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x230/0x0, .tag=-1, .internal_tag=94}
dm-1/hctx0/sched/sync_write_rqs:00000000b759c011 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2e4/0x0, .tag=-1, .internal_tag=93}
dm-1/hctx0/sched/sync_write_rqs:0000000010282088 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x29c/0x0, .tag=-1, .internal_tag=99}
dm-1/hctx0/sched/sync_write_rqs:00000000310cd57b {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=102}
dm-1/hctx0/sched/sync_write_rqs:000000009f3cfe7b {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x198/0x0, .tag=-1, .internal_tag=108}
dm-1/hctx0/sched/sync_write_rqs:0000000044c72ae1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x244/0x0, .tag=-1, .internal_tag=113}
dm-1/hctx0/sched/sync_write_rqs:00000000719f2305 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x164/0x0, .tag=-1, .internal_tag=123}
dm-1/hctx0/sched/sync_write_rqs:0000000010a00a3a {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=143}
dm-1/hctx0/sched/sync_write_rqs:00000000fb0161bd {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x218/0x0, .tag=-1, .internal_tag=217}
dm-1/hctx0/sched/sync_write_rqs:00000000d8e78cb2 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x178/0x0, .tag=-1, .internal_tag=232}
dm-1/hctx0/sched/sync_write_rqs:00000000b504707f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f0/0x0, .tag=-1, .internal_tag=196}
dm-1/hctx0/sched/sync_write_rqs:0000000058eb1652 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1bc/0x0, .tag=-1, .internal_tag=202}
dm-1/hctx0/sched/sync_write_rqs:0000000082db1cc1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x354/0x0, .tag=-1, .internal_tag=21}
dm-1/hctx0/sched/sync_write_rqs:00000000c9123239 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b4/0x0, .tag=-1, .internal_tag=40}
dm-1/hctx0/sched/sync_write_rqs:00000000756407af {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c8/0x0, .tag=-1, .internal_tag=78}
dm-1/hctx0/sched/sync_write_rqs:00000000903ee3f3 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x280/0x0, .tag=-1, .internal_tag=92}
dm-1/hctx0/sched/sync_write_rqs:00000000da81e778 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e4/0x0, .tag=-1, .internal_tag=106}
dm-1/hctx0/sched/sync_write_rqs:000000000d4bca70 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c0/0x0, .tag=-1, .internal_tag=111}
dm-1/hctx0/sched/sync_write_rqs:000000007725300f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2a0/0x0, .tag=-1, .internal_tag=119}
dm-1/hctx0/sched/sync_write_rqs:000000007c8b44d6 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b0/0x0, .tag=-1, .internal_tag=154}
dm-1/hctx0/sched/sync_write_rqs:00000000a50797ee {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=194}
dm-1/hctx0/sched/sync_write_rqs:000000003ddea3af {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=27}
dm-1/hctx0/sched/sync_write_rqs:000000007ac78217 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e0/0x0, .tag=-1, .internal_tag=165}
dm-1/hctx0/sched/sync_write_rqs:00000000744ab35e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x214/0x0, .tag=-1, .internal_tag=189}
dm-1/hctx0/sched/sync_write_rqs:00000000a819edf5 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d0/0x0, .tag=-1, .internal_tag=3}
dm-1/hctx0/sched/sync_write_rqs:00000000594fd463 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x220/0x0, .tag=-1, .internal_tag=4}
dm-1/hctx0/sched/sync_write_rqs:000000004002fbdc {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x184/0x0, .tag=-1, .internal_tag=17}
dm-1/hctx0/sched/sync_write_rqs:000000006e5310d8 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c4/0x0, .tag=-1, .internal_tag=80}
dm-1/hctx0/sched/sync_write_rqs:000000001982a658 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=84}
dm-1/hctx0/sched/sync_write_rqs:000000005346d29c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x290/0x0, .tag=-1, .internal_tag=109}
dm-1/hctx0/sched/sync_write_rqs:00000000dba6ccb6 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x358/0x0, .tag=-1, .internal_tag=152}
dm-1/hctx0/sched/sync_write_rqs:0000000042264b16 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1cc/0x0, .tag=-1, .internal_tag=219}
dm-1/hctx0/sched/sync_write_rqs:000000001bbfb611 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2fc/0x0, .tag=-1, .internal_tag=199}
dm-1/hctx0/sched/sync_write_rqs:000000006dcd0fec {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=204}
dm-1/hctx0/sched/sync_write_rqs:000000005e03f6f5 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2e4/0x0, .tag=-1, .internal_tag=243}
dm-1/hctx0/sched/sync_write_rqs:000000005f4598ae {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x238/0x0, .tag=-1, .internal_tag=90}
dm-1/hctx0/sched/sync_write_rqs:0000000041ae7477 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x18c/0x0, .tag=-1, .internal_tag=117}
dm-1/hctx0/sched/sync_write_rqs:000000002b6d7df9 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f4/0x0, .tag=-1, .internal_tag=130}
dm-1/hctx0/sched/sync_write_rqs:0000000074f4cc3c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d8/0x0, .tag=-1, .internal_tag=168}
dm-1/hctx0/sched/sync_write_rqs:0000000027a81808 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d8/0x0, .tag=-1, .internal_tag=29}
dm-1/hctx0/sched/sync_write_rqs:00000000e9917e3f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b0/0x0, .tag=-1, .internal_tag=77}
dm-1/hctx0/sched/sync_write_rqs:000000004b7372f0 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=156}
dm-1/hctx0/sched/sync_write_rqs:00000000bce8fc12 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x24c/0x0, .tag=-1, .internal_tag=100}
dm-1/hctx0/sched/sync_write_rqs:00000000a7d69b70 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2e0/0x0, .tag=-1, .internal_tag=177}
dm-1/hctx0/sched/sync_write_rqs:00000000b2fd8a11 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x228/0x0, .tag=-1, .internal_tag=186}
dm-1/hctx0/sched/sync_write_rqs:00000000d31a86ae {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x300/0x0, .tag=-1, .internal_tag=216}
dm-1/hctx0/sched/sync_write_rqs:00000000f4df3632 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x254/0x0, .tag=-1, .internal_tag=221}
dm-1/hctx0/sched/sync_write_rqs:00000000e92fdd6d {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x13c/0x0, .tag=-1, .internal_tag=241}
dm-1/hctx0/sched/sync_write_rqs:000000002e0257fe {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d8/0x0, .tag=-1, .internal_tag=233}
dm-1/hctx0/sched/sync_write_rqs:00000000ce405353 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x230/0x0, .tag=-1, .internal_tag=238}
dm-1/hctx0/sched/sync_write_rqs:00000000a4a7abeb {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c8/0x0, .tag=-1, .internal_tag=253}
dm-1/hctx0/sched/sync_write_rqs:0000000053b12eef {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=255}
dm-1/hctx0/sched/sync_write_rqs:00000000bd34ad61 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x3e8/0x0, .tag=-1, .internal_tag=0}
dm-1/hctx0/sched/sync_write_rqs:00000000014a3018 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x210/0x0, .tag=-1, .internal_tag=227}
dm-1/hctx0/sched/sync_write_rqs:00000000e18318f3 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d4/0x0, .tag=-1, .internal_tag=166}
dm-1/hctx0/sched/sync_write_rqs:00000000529eecb4 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=179}
dm-1/hctx0/sched/sync_write_rqs:00000000907ec494 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x284/0x0, .tag=-1, .internal_tag=181}
dm-1/hctx0/sched/sync_write_rqs:0000000020aad20f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x280/0x0, .tag=-1, .internal_tag=131}
dm-1/hctx0/sched/sync_write_rqs:000000009fb190b7 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x20c/0x0, .tag=-1, .internal_tag=141}
dm-1/hctx0/sched/sync_write_rqs:000000003fadc759 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x264/0x0, .tag=-1, .internal_tag=147}
dm-1/hctx0/sched/sync_write_rqs:00000000941291c7 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x270/0x0, .tag=-1, .internal_tag=211}
dm-1/hctx0/sched/sync_write_rqs:0000000047b4b25f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1a8/0x0, .tag=-1, .internal_tag=213}
dm-1/hctx0/sched/sync_write_rqs:00000000d6cdd6a8 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2a0/0x0, .tag=-1, .internal_tag=225}
dm-1/hctx0/sched/sync_write_rqs:000000009d2ae080 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x210/0x0, .tag=-1, .internal_tag=250}
dm-1/hctx0/sched/sync_write_rqs:0000000017d480d1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x258/0x0, .tag=-1, .internal_tag=254}
dm-1/hctx0/sched/sync_write_rqs:00000000fe8b01f1 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=11}
dm-1/hctx0/sched/sync_write_rqs:00000000f56c3a5a {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x258/0x0, .tag=-1, .internal_tag=25}
dm-1/hctx0/sched/sync_write_rqs:000000004659015c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2cc/0x0, .tag=-1, .internal_tag=32}
dm-1/hctx0/sched/sync_write_rqs:000000008d4f1770 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1b4/0x0, .tag=-1, .internal_tag=33}
dm-1/hctx0/sched/sync_write_rqs:000000001ab15a9e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x290/0x0, .tag=-1, .internal_tag=35}
dm-1/hctx0/sched/sync_write_rqs:00000000f8711dc5 {.op=WRITE, .cmd_flags=SYNC|META|PRIO, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x198/0x0, .tag=-1, .internal_tag=229}
dm-1/hctx0/sched/sync_write_rqs:00000000319ee6c9 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=138}
dm-1/hctx0/sched/sync_write_rqs:000000009bce151f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1cc/0x0, .tag=-1, .internal_tag=140}
dm-1/hctx0/sched/sync_write_rqs:00000000459eb332 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1ec/0x0, .tag=-1, .internal_tag=142}
dm-1/hctx0/sched/sync_write_rqs:0000000019ccf47e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c4/0x0, .tag=-1, .internal_tag=145}
dm-1/hctx0/sched/sync_write_rqs:000000003470e9e4 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x24c/0x0, .tag=-1, .internal_tag=158}
dm-1/hctx0/sched/sync_write_rqs:00000000a98dd771 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1fc/0x0, .tag=-1, .internal_tag=159}
dm-1/hctx0/sched/sync_write_rqs:00000000ef5235c5 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f0/0x0, .tag=-1, .internal_tag=178}
dm-1/hctx0/sched/sync_write_rqs:0000000097e7dc5f {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b4/0x0, .tag=-1, .internal_tag=188}
dm-1/hctx0/sched/sync_write_rqs:0000000017cee035 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x240/0x0, .tag=-1, .internal_tag=132}
dm-1/hctx0/sched/sync_write_rqs:00000000cc107071 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=133}
dm-1/hctx0/sched/sync_write_rqs:0000000060ef72aa {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=135}
dm-1/hctx0/sched/sync_write_rqs:000000006bcc76ce {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x29c/0x0, .tag=-1, .internal_tag=88}
dm-1/hctx0/sched/sync_write_rqs:000000005b450d6e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x184/0x0, .tag=-1, .internal_tag=20}
dm-1/hctx0/sched/sync_write_rqs:00000000574afa37 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2a4/0x0, .tag=-1, .internal_tag=44}
dm-1/hctx0/sched/sync_write_rqs:00000000ed150f7c {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x174/0x0, .tag=-1, .internal_tag=45}
dm-1/hctx0/sched/sync_write_rqs:000000000b750371 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1b8/0x0, .tag=-1, .internal_tag=51}
dm-1/hctx0/sched/sync_write_rqs:0000000051f9b77e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x244/0x0, .tag=-1, .internal_tag=53}
dm-1/hctx0/sched/sync_write_rqs:00000000d44d71dd {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x280/0x0, .tag=-1, .internal_tag=60}
dm-1/hctx0/sched/sync_write_rqs:000000003e72ed6d {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x19c/0x0, .tag=-1, .internal_tag=98}
dm-1/hctx0/sched/sync_write_rqs:0000000049e4a52e {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=129}
dm-1/hctx0/sched/sync_write_rqs:00000000f1a55708 {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x22c/0x0, .tag=-1, .internal_tag=144}
dm-1/hctx0/sched/sync_write_rqs:00000000bcd7c38d {.op=WRITE, .cmd_flags=SYNC|IDLE, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x264/0x0, .tag=-1, .internal_tag=172}
dm-1/hctx0/sched/read_waiting:1
dm-1/hctx0/sched/read_rqs:00000000e422b915 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1ac/0x0, .tag=-1, .internal_tag=190}
dm-1/hctx0/sched/read_rqs:0000000027f3e5c5 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x288/0x0, .tag=-1, .internal_tag=42}
dm-1/hctx0/sched/read_rqs:000000002b1e24f2 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c8/0x0, .tag=-1, .internal_tag=153}
dm-1/hctx0/sched/read_rqs:00000000344632cd {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x230/0x0, .tag=-1, .internal_tag=161}
dm-1/hctx0/sched/read_rqs:00000000d3770f22 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1a0/0x0, .tag=-1, .internal_tag=162}
dm-1/hctx0/sched/read_rqs:00000000c3356fb1 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c4/0x0, .tag=-1, .internal_tag=169}
dm-1/hctx0/sched/read_rqs:00000000fc3a5b5f {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c8/0x0, .tag=-1, .internal_tag=175}
dm-1/hctx0/sched/read_rqs:00000000478197e4 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c8/0x0, .tag=-1, .internal_tag=176}
dm-1/hctx0/sched/read_rqs:00000000b27da33c {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2a4/0x0, .tag=-1, .internal_tag=19}
dm-1/hctx0/sched/read_rqs:0000000031ccce39 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1fc/0x0, .tag=-1, .internal_tag=30}
dm-1/hctx0/sched/read_rqs:000000003f9c7bda {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x294/0x0, .tag=-1, .internal_tag=43}
dm-1/hctx0/sched/read_rqs:00000000e2022a5b {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d0/0x0, .tag=-1, .internal_tag=50}
dm-1/hctx0/sched/read_rqs:000000002f61c2ee {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x364/0x0, .tag=-1, .internal_tag=55}
dm-1/hctx0/sched/read_rqs:000000000cfb89c7 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2ac/0x0, .tag=-1, .internal_tag=63}
dm-1/hctx0/sched/read_rqs:00000000a895fbea {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x254/0x0, .tag=-1, .internal_tag=65}
dm-1/hctx0/sched/read_rqs:0000000094404f73 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x324/0x0, .tag=-1, .internal_tag=71}
dm-1/hctx0/sched/read_rqs:00000000b32cd601 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2ac/0x0, .tag=-1, .internal_tag=76}
dm-1/hctx0/sched/read_rqs:000000008e7bb220 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c4/0x0, .tag=-1, .internal_tag=81}
dm-1/hctx0/sched/read_rqs:00000000333ad0b4 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e4/0x0, .tag=-1, .internal_tag=124}
dm-1/hctx0/sched/read_rqs:00000000889ffde5 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x26c/0x0, .tag=-1, .internal_tag=139}
dm-1/hctx0/sched/read_rqs:00000000f784275d {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c4/0x0, .tag=-1, .internal_tag=149}
dm-1/hctx0/sched/read_rqs:000000004e95ae4f {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x188/0x0, .tag=-1, .internal_tag=163}
dm-1/hctx0/sched/read_rqs:0000000019b51a4f {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f8/0x0, .tag=-1, .internal_tag=184}
dm-1/hctx0/sched/read_rqs:00000000d55e26c9 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x224/0x0, .tag=-1, .internal_tag=187}
dm-1/hctx0/sched/read_rqs:00000000045d1b41 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x208/0x0, .tag=-1, .internal_tag=70}
dm-1/hctx0/sched/read_rqs:00000000f3b346b7 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2f0/0x0, .tag=-1, .internal_tag=87}
dm-1/hctx0/sched/read_rqs:00000000b1b97de5 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x26c/0x0, .tag=-1, .internal_tag=96}
dm-1/hctx0/sched/read_rqs:00000000f259090a {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x22c/0x0, .tag=-1, .internal_tag=105}
dm-1/hctx0/sched/read_rqs:000000007612b798 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x200/0x0, .tag=-1, .internal_tag=107}
dm-1/hctx0/sched/read_rqs:0000000085617e1f {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2ec/0x0, .tag=-1, .internal_tag=116}
dm-1/hctx0/sched/read_rqs:00000000dd830169 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c0/0x0, .tag=-1, .internal_tag=126}
dm-1/hctx0/sched/read_rqs:00000000b4282665 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x218/0x0, .tag=-1, .internal_tag=137}
dm-1/hctx0/sched/read_rqs:00000000c978709d {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x244/0x0, .tag=-1, .internal_tag=148}
dm-1/hctx0/sched/read_rqs:000000002465347a {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x284/0x0, .tag=-1, .internal_tag=164}
dm-1/hctx0/sched/read_rqs:0000000097a5172b {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x234/0x0, .tag=-1, .internal_tag=173}
dm-1/hctx0/sched/read_rqs:00000000562cad09 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x290/0x0, .tag=-1, .internal_tag=185}
dm-1/hctx0/sched/read_rqs:0000000034b30eb1 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x29c/0x0, .tag=-1, .internal_tag=151}
dm-1/hctx0/sched/read_rqs:00000000faee6fba {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x248/0x0, .tag=-1, .internal_tag=97}
dm-1/hctx0/sched/read_rqs:00000000ffae64ea {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1dc/0x0, .tag=-1, .internal_tag=104}
dm-1/hctx0/sched/read_rqs:00000000eebd316c {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x288/0x0, .tag=-1, .internal_tag=114}
dm-1/hctx0/sched/read_rqs:00000000bb84f791 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x278/0x0, .tag=-1, .internal_tag=207}
dm-1/hctx0/sched/read_rqs:000000004b52ce6a {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x204/0x0, .tag=-1, .internal_tag=22}
dm-1/hctx0/sched/read_rqs:00000000795532f2 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x350/0x0, .tag=-1, .internal_tag=52}
dm-1/hctx0/sched/read_rqs:00000000dab1f2da {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x228/0x0, .tag=-1, .internal_tag=201}
dm-1/hctx0/sched/read_rqs:00000000b1364ace {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x24c/0x0, .tag=-1, .internal_tag=200}
dm-1/hctx0/sched/read_rqs:000000004504b556 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1c4/0x0, .tag=-1, .internal_tag=2}
dm-1/hctx0/sched/read_rqs:0000000061a30738 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=224}
dm-1/hctx0/sched/read_rqs:000000009e24c42d {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x368/0x0, .tag=-1, .internal_tag=125}
dm-1/hctx0/sched/read_rqs:00000000e3ab75e2 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x180/0x0, .tag=-1, .internal_tag=120}
dm-1/hctx0/sched/read_rqs:00000000856af234 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1a8/0x0, .tag=-1, .internal_tag=157}
dm-1/hctx0/sched/read_rqs:00000000e2646deb {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x278/0x0, .tag=-1, .internal_tag=167}
dm-1/hctx0/sched/read_rqs:000000005f397521 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x20c/0x0, .tag=-1, .internal_tag=170}
dm-1/hctx0/sched/read_rqs:00000000c50d9ea4 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x244/0x0, .tag=-1, .internal_tag=182}
dm-1/hctx0/sched/read_rqs:00000000bc2a2d59 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e4/0x0, .tag=-1, .internal_tag=193}
dm-1/hctx0/sched/read_rqs:00000000d9217a8f {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x220/0x0, .tag=-1, .internal_tag=203}
dm-1/hctx0/sched/read_rqs:00000000592613be {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2ac/0x0, .tag=-1, .internal_tag=218}
dm-1/hctx0/sched/read_rqs:00000000d4bbab6c {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x224/0x0, .tag=-1, .internal_tag=228}
dm-1/hctx0/sched/read_rqs:000000008d647926 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1d4/0x0, .tag=-1, .internal_tag=234}
dm-1/hctx0/sched/read_rqs:000000001218fe59 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x324/0x0, .tag=-1, .internal_tag=247}
dm-1/hctx0/sched/read_rqs:00000000cd3ea438 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=7}
dm-1/hctx0/sched/read_rqs:00000000d6ba8d11 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x264/0x0, .tag=-1, .internal_tag=8}
dm-1/hctx0/sched/read_rqs:0000000036f4c531 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b4/0x0, .tag=-1, .internal_tag=12}
dm-1/hctx0/sched/read_rqs:00000000c9ff22c6 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x334/0x0, .tag=-1, .internal_tag=23}
dm-1/hctx0/sched/read_rqs:00000000f58089b9 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x184/0x0, .tag=-1, .internal_tag=38}
dm-1/hctx0/sched/read_rqs:00000000b9e811fa {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x26c/0x0, .tag=-1, .internal_tag=39}
dm-1/hctx0/sched/read_rqs:00000000a1dc159a {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x350/0x0, .tag=-1, .internal_tag=56}
dm-1/hctx0/sched/read_rqs:00000000af762d0e {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x3a8/0x0, .tag=-1, .internal_tag=57}
dm-1/hctx0/sched/read_rqs:00000000285df5d5 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x370/0x0, .tag=-1, .internal_tag=58}
dm-1/hctx0/sched/read_rqs:00000000bff485c7 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x238/0x0, .tag=-1, .internal_tag=59}
dm-1/hctx0/sched/read_rqs:00000000b5f0013a {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=121}
dm-1/hctx0/sched/read_rqs:00000000a2e293e1 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=118}
dm-1/hctx0/sched/read_rqs:00000000ed6fd859 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x224/0x0, .tag=-1, .internal_tag=9}
dm-1/hctx0/sched/read_rqs:0000000098178778 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2ec/0x0, .tag=-1, .internal_tag=231}
dm-1/hctx0/sched/read_rqs:00000000c072b159 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2b4/0x0, .tag=-1, .internal_tag=15}
dm-1/hctx0/sched/read_rqs:000000008025d113 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=36}
dm-1/hctx0/sched/read_rqs:000000005f7ef2df {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x250/0x0, .tag=-1, .internal_tag=41}
dm-1/hctx0/sched/read_rqs:00000000edb6ea7d {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e8/0x0, .tag=-1, .internal_tag=72}
dm-1/hctx0/sched/read_rqs:00000000d1150e24 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x24c/0x0, .tag=-1, .internal_tag=112}
dm-1/hctx0/sched/read_rqs:000000003e18f788 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x330/0x0, .tag=-1, .internal_tag=128}
dm-1/hctx0/sched/read_rqs:00000000d276f825 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x340/0x0, .tag=-1, .internal_tag=191}
dm-1/hctx0/sched/read_rqs:00000000e75088bb {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1e0/0x0, .tag=-1, .internal_tag=246}
dm-1/hctx0/sched/read_rqs:0000000046032b79 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1ec/0x0, .tag=-1, .internal_tag=249}
dm-1/hctx0/sched/read_rqs:00000000cf31c849 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2bc/0x0, .tag=-1, .internal_tag=242}
dm-1/hctx0/sched/read_rqs:00000000259471ca {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x230/0x0, .tag=-1, .internal_tag=5}
dm-1/hctx0/sched/read_rqs:000000006375ca8b {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1dc/0x0, .tag=-1, .internal_tag=10}
dm-1/hctx0/sched/read_rqs:000000008291c8bf {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x200/0x0, .tag=-1, .internal_tag=13}
dm-1/hctx0/sched/read_rqs:00000000e11a9149 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x288/0x0, .tag=-1, .internal_tag=14}
dm-1/hctx0/sched/read_rqs:00000000d2781eb6 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x23c/0x0, .tag=-1, .internal_tag=26}
dm-1/hctx0/sched/read_rqs:000000009f53b432 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2c0/0x0, .tag=-1, .internal_tag=68}
dm-1/hctx0/sched/read_rqs:00000000dd790c1b {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x34c/0x0, .tag=-1, .internal_tag=69}
dm-1/hctx0/sched/read_rqs:00000000a0e82022 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x1f4/0x0, .tag=-1, .internal_tag=75}
dm-1/hctx0/sched/read_rqs:0000000018d1d3ae {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x21c/0x0, .tag=-1, .internal_tag=79}
dm-1/hctx0/sched/read_rqs:000000002fce17b9 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x200/0x0, .tag=-1, .internal_tag=83}
dm-1/hctx0/sched/read_rqs:00000000f19b5734 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x28c/0x0, .tag=-1, .internal_tag=134}
dm-1/hctx0/sched/read_rqs:00000000d0589cab {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x2dc/0x0, .tag=-1, .internal_tag=146}
dm-1/hctx0/sched/read_rqs:0000000028a100b4 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x274/0x0, .tag=-1, .internal_tag=150}
dm-1/hctx0/sched/read_rqs:000000009b1b0c71 {.op=READ, .cmd_flags=, .rq_flags=SORTED|ELVPRIV|IO_STAT|STATS, .state=idle, .gstate=0x344/0x0, .tag=-1, .internal_tag=180}
dm-1/hctx0/cpu3/completed:4015 0
dm-1/hctx0/cpu3/merged:452
dm-1/hctx0/cpu3/dispatched:4103 0
dm-1/hctx0/cpu2/completed:3869 0
dm-1/hctx0/cpu2/merged:528
dm-1/hctx0/cpu2/dispatched:3960 0
dm-1/hctx0/cpu1/completed:4029 0
dm-1/hctx0/cpu1/merged:483
dm-1/hctx0/cpu1/dispatched:4086 0
dm-1/hctx0/cpu0/completed:3845 0
dm-1/hctx0/cpu0/merged:569
dm-1/hctx0/cpu0/dispatched:3865 0
dm-1/hctx0/active:0
dm-1/hctx0/run:21117
dm-1/hctx0/queued:16004
dm-1/hctx0/dispatched:       0	20287
dm-1/hctx0/dispatched:       1	11933
dm-1/hctx0/dispatched:       2	405
dm-1/hctx0/dispatched:       4	251
dm-1/hctx0/dispatched:       8	140
dm-1/hctx0/dispatched:      16	29
dm-1/hctx0/dispatched:      32+	0
dm-1/hctx0/io_poll:considered=0
dm-1/hctx0/io_poll:invoked=0
dm-1/hctx0/io_poll:success=0
dm-1/hctx0/sched_tags_bitmap:00000000: ffff ffff ffff ffff ffff ffff ffff ffff
dm-1/hctx0/sched_tags_bitmap:00000010: ffff ffff ffff ffff ffff ffff ffff ffff
dm-1/hctx0/sched_tags:nr_tags=256
dm-1/hctx0/sched_tags:nr_reserved_tags=0
dm-1/hctx0/sched_tags:active_queues=0
dm-1/hctx0/sched_tags:bitmap_tags:
dm-1/hctx0/sched_tags:depth=256
dm-1/hctx0/sched_tags:busy=256
dm-1/hctx0/sched_tags:bits_per_word=64
dm-1/hctx0/sched_tags:map_nr=4
dm-1/hctx0/sched_tags:alloc_hint={240, 0, 0, 0}
dm-1/hctx0/sched_tags:wake_batch=8
dm-1/hctx0/sched_tags:wake_index=6
dm-1/hctx0/sched_tags:ws={
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:	{.wait_cnt=8, .wait=active},
dm-1/hctx0/sched_tags:}
dm-1/hctx0/sched_tags:round_robin=0
dm-1/hctx0/tags_bitmap:00000000: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000010: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000020: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000030: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000040: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000050: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000060: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000070: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000080: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:00000090: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000a0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000b0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000c0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000d0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000e0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags_bitmap:000000f0: 0000 0000 0000 0000 0000 0000 0000 0000
dm-1/hctx0/tags:nr_tags=2048
dm-1/hctx0/tags:nr_reserved_tags=0
dm-1/hctx0/tags:active_queues=0
dm-1/hctx0/tags:bitmap_tags:
dm-1/hctx0/tags:depth=2048
dm-1/hctx0/tags:busy=0
dm-1/hctx0/tags:bits_per_word=64
dm-1/hctx0/tags:map_nr=32
dm-1/hctx0/tags:alloc_hint={1527, 1587, 1586, 1558}
dm-1/hctx0/tags:wake_batch=8
dm-1/hctx0/tags:wake_index=0
dm-1/hctx0/tags:ws={
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
dm-1/hctx0/tags:}
dm-1/hctx0/tags:round_robin=0
dm-1/hctx0/ctx_map:00000000: 00
dm-1/hctx0/flags:alloc_policy=FIFO SHOULD_MERGE|SG_MERGE
dm-1/write_hints:hint0: 0
dm-1/write_hints:hint1: 0
dm-1/write_hints:hint2: 0
dm-1/write_hints:hint3: 0
dm-1/write_hints:hint4: 0
dm-1/state:SAME_COMP|IO_STAT|DISCARD|INIT_DONE|NO_SG_MERGE|POLL|STATS|REGISTERED
sdb/hctx0/cpu3/completed:56 4
sdb/hctx0/cpu3/merged:0
sdb/hctx0/cpu3/dispatched:56 4
sdb/hctx0/cpu2/completed:82 11
sdb/hctx0/cpu2/merged:0
sdb/hctx0/cpu2/dispatched:82 11
sdb/hctx0/cpu1/completed:69 65
sdb/hctx0/cpu1/merged:0
sdb/hctx0/cpu1/dispatched:69 65
sdb/hctx0/cpu0/completed:31 4
sdb/hctx0/cpu0/merged:0
sdb/hctx0/cpu0/dispatched:31 4
sdb/hctx0/active:0
sdb/hctx0/run:316
sdb/hctx0/queued:322
sdb/hctx0/dispatched:       0	0
sdb/hctx0/dispatched:       1	322
sdb/hctx0/dispatched:       2	0
sdb/hctx0/dispatched:       4	0
sdb/hctx0/dispatched:       8	0
sdb/hctx0/dispatched:      16	0
sdb/hctx0/dispatched:      32+	0
sdb/hctx0/io_poll:considered=0
sdb/hctx0/io_poll:invoked=0
sdb/hctx0/io_poll:success=0
sdb/hctx0/sched_tags_bitmap:00000000: 0000 0000 0000 0000 0000 0000 0000 0000
sdb/hctx0/sched_tags_bitmap:00000010: 0000 0000 0000 0000 0000 0000 0000 0000
sdb/hctx0/sched_tags:nr_tags=256
sdb/hctx0/sched_tags:nr_reserved_tags=0
sdb/hctx0/sched_tags:active_queues=0
sdb/hctx0/sched_tags:bitmap_tags:
sdb/hctx0/sched_tags:depth=256
sdb/hctx0/sched_tags:busy=0
sdb/hctx0/sched_tags:bits_per_word=64
sdb/hctx0/sched_tags:map_nr=4
sdb/hctx0/sched_tags:alloc_hint={66, 53, 244, 86}
sdb/hctx0/sched_tags:wake_batch=8
sdb/hctx0/sched_tags:wake_index=0
sdb/hctx0/sched_tags:ws={
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/sched_tags:}
sdb/hctx0/sched_tags:round_robin=0
sdb/hctx0/tags_bitmap:00000000: 0000 0000 0000 0000 0000 0000 0000 0000
sdb/hctx0/tags_bitmap:00000010: 0000 0000 0000 0000
sdb/hctx0/tags:nr_tags=192
sdb/hctx0/tags:nr_reserved_tags=0
sdb/hctx0/tags:active_queues=0
sdb/hctx0/tags:bitmap_tags:
sdb/hctx0/tags:depth=192
sdb/hctx0/tags:busy=0
sdb/hctx0/tags:bits_per_word=32
sdb/hctx0/tags:map_nr=6
sdb/hctx0/tags:alloc_hint={105, 104, 8, 185}
sdb/hctx0/tags:wake_batch=8
sdb/hctx0/tags:wake_index=0
sdb/hctx0/tags:ws={
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sdb/hctx0/tags:}
sdb/hctx0/tags:round_robin=0
sdb/hctx0/ctx_map:00000000: 00
sdb/hctx0/flags:alloc_policy=FIFO SHOULD_MERGE|SG_MERGE
sdb/sched/starved:0
sdb/sched/batching:1
sdb/write_hints:hint0: 0
sdb/write_hints:hint1: 0
sdb/write_hints:hint2: 0
sdb/write_hints:hint3: 0
sdb/write_hints:hint4: 0
sdb/state:BIDI|SAME_COMP|NONROT|IO_STAT|INIT_DONE|POLL|WC|FUA|REGISTERED|SCSI_PASSTHROUGH
sda/hctx0/cpu3/completed:0 1
sda/hctx0/cpu3/merged:0
sda/hctx0/cpu3/dispatched:0 1
sda/hctx0/cpu2/completed:81 11
sda/hctx0/cpu2/merged:0
sda/hctx0/cpu2/dispatched:81 11
sda/hctx0/cpu1/completed:2 53
sda/hctx0/cpu1/merged:0
sda/hctx0/cpu1/dispatched:2 53
sda/hctx0/cpu0/completed:7 8
sda/hctx0/cpu0/merged:0
sda/hctx0/cpu0/dispatched:7 8
sda/hctx0/active:0
sda/hctx0/run:135
sda/hctx0/queued:163
sda/hctx0/dispatched:       0	0
sda/hctx0/dispatched:       1	163
sda/hctx0/dispatched:       2	0
sda/hctx0/dispatched:       4	0
sda/hctx0/dispatched:       8	0
sda/hctx0/dispatched:      16	0
sda/hctx0/dispatched:      32+	0
sda/hctx0/io_poll:considered=0
sda/hctx0/io_poll:invoked=0
sda/hctx0/io_poll:success=0
sda/hctx0/tags_bitmap:00000000: 0000 0000 0000 0000 0000 0000 0000 0000
sda/hctx0/tags:nr_tags=128
sda/hctx0/tags:nr_reserved_tags=0
sda/hctx0/tags:active_queues=0
sda/hctx0/tags:bitmap_tags:
sda/hctx0/tags:depth=128
sda/hctx0/tags:busy=0
sda/hctx0/tags:bits_per_word=32
sda/hctx0/tags:map_nr=4
sda/hctx0/tags:alloc_hint={109, 99, 82, 15}
sda/hctx0/tags:wake_batch=8
sda/hctx0/tags:wake_index=0
sda/hctx0/tags:ws={
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
sda/hctx0/tags:}
sda/hctx0/tags:round_robin=0
sda/hctx0/ctx_map:00000000: 00
sda/hctx0/flags:alloc_policy=FIFO SHOULD_MERGE|SG_MERGE
sda/write_hints:hint0: 0
sda/write_hints:hint1: 0
sda/write_hints:hint2: 0
sda/write_hints:hint3: 0
sda/write_hints:hint4: 0
sda/state:SAME_COMP|IO_STAT|DISCARD|INIT_DONE|POLL|WC|REGISTERED|SCSI_PASSTHROUGH
vda/hctx0/cpu3/completed:1417 37
vda/hctx0/cpu3/merged:0
vda/hctx0/cpu3/dispatched:1417 37
vda/hctx0/cpu2/completed:2287 35
vda/hctx0/cpu2/merged:2
vda/hctx0/cpu2/dispatched:2287 35
vda/hctx0/cpu1/completed:1646 0
vda/hctx0/cpu1/merged:0
vda/hctx0/cpu1/dispatched:1646 0
vda/hctx0/cpu0/completed:1235 4
vda/hctx0/cpu0/merged:15
vda/hctx0/cpu0/dispatched:1235 4
vda/hctx0/active:0
vda/hctx0/run:6520
vda/hctx0/queued:6661
vda/hctx0/dispatched:       0	0
vda/hctx0/dispatched:       1	6482
vda/hctx0/dispatched:       2	16
vda/hctx0/dispatched:       4	8
vda/hctx0/dispatched:       8	9
vda/hctx0/dispatched:      16	2
vda/hctx0/dispatched:      32+	0
vda/hctx0/io_poll:considered=0
vda/hctx0/io_poll:invoked=0
vda/hctx0/io_poll:success=0
vda/hctx0/tags_bitmap:00000000: 0000 0000 0000 0000 0000 0000 0000 0000
vda/hctx0/tags:nr_tags=128
vda/hctx0/tags:nr_reserved_tags=0
vda/hctx0/tags:active_queues=0
vda/hctx0/tags:bitmap_tags:
vda/hctx0/tags:depth=128
vda/hctx0/tags:busy=0
vda/hctx0/tags:bits_per_word=32
vda/hctx0/tags:map_nr=4
vda/hctx0/tags:alloc_hint={111, 62, 55, 120}
vda/hctx0/tags:wake_batch=8
vda/hctx0/tags:wake_index=0
vda/hctx0/tags:ws={
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:	{.wait_cnt=8, .wait=inactive},
vda/hctx0/tags:}
vda/hctx0/tags:round_robin=0
vda/hctx0/ctx_map:00000000: 00
vda/hctx0/flags:alloc_policy=FIFO SHOULD_MERGE
vda/write_hints:hint0: 0
vda/write_hints:hint1: 0
vda/write_hints:hint2: 0
vda/write_hints:hint3: 0
vda/write_hints:hint4: 0
vda/state:SAME_COMP|IO_STAT|INIT_DONE|NO_SG_MERGE|POLL|WC|REGISTERED
sr0/hctx0/cpu3/completed:16 94
sr0/hctx0/cpu3/merged:0
sr0/hctx0/cpu3/dispatched:16 94
sr0/hctx0/cpu2/completed:11 2
sr0/hctx0/cpu2/merged:0
sr0/hctx0/cpu2/dispatched:11 2
sr0/hctx0/cpu1/completed:19 14
sr0/hctx0/cpu1/merged:0
sr0/hctx0/cpu1/dispatched:19 14
sr0/hctx0/cpu0/completed:0 22
sr0/hctx0/cpu0/merged:0
sr0/hctx0/cpu0/dispatched:0 22
sr0/hctx0/active:0
sr0/hctx0/run:178
sr0/hctx0/queued:178
sr0/hctx0/dispatched:       0	0
sr0/hctx0/dispatched:       1	178
sr0/hctx0/dispatched:       2	0
sr0/hctx0/dispatched:       4	0
sr0/hctx0/dispatched:       8	0
sr0/hctx0/dispatched:      16	0
sr0/hctx0/dispatched:      32+	0
sr0/hctx0/io_poll:considered=0
sr0/hctx0/io_poll:invoked=0
sr0/hctx0/io_poll:success=0
sr0/hctx0/tags_bitmap:00000000: 00
sr0/hctx0/tags:nr_tags=1
sr0/hctx0/tags:nr_reserved_tags=0
sr0/hctx0/tags:active_queues=0
sr0/hctx0/tags:bitmap_tags:
sr0/hctx0/tags:depth=1
sr0/hctx0/tags:busy=0
sr0/hctx0/tags:bits_per_word=64
sr0/hctx0/tags:map_nr=1
sr0/hctx0/tags:alloc_hint={0, 0, 0, 0}
sr0/hctx0/tags:wake_batch=1
sr0/hctx0/tags:wake_index=7
sr0/hctx0/tags:ws={
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:	{.wait_cnt=1, .wait=inactive},
sr0/hctx0/tags:}
sr0/hctx0/tags:round_robin=1
sr0/hctx0/ctx_map:00000000: 00
sr0/hctx0/flags:alloc_policy=RR SHOULD_MERGE|SG_MERGE
sr0/write_hints:hint0: 0
sr0/write_hints:hint1: 0
sr0/write_hints:hint2: 0
sr0/write_hints:hint3: 0
sr0/write_hints:hint4: 0
sr0/state:SAME_COMP|IO_STAT|INIT_DONE|POLL|FLUSH_NQ|REGISTERED|SCSI_PASSTHROUGH

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: v4.16-rc2: I/O hang with dm-rq + Kyber
  2018-02-22 21:10 v4.16-rc2: I/O hang with dm-rq + Kyber Bart Van Assche
@ 2018-02-22 22:42 ` Omar Sandoval
  2018-02-22 23:22   ` Bart Van Assche
  2018-02-23 16:26 ` Ming Lei
  1 sibling, 1 reply; 5+ messages in thread
From: Omar Sandoval @ 2018-02-22 22:42 UTC (permalink / raw)
  To: Bart Van Assche; +Cc: linux-block

On Thu, Feb 22, 2018 at 09:10:23PM +0000, Bart Van Assche wrote:
> Hello Omar,
> 
> I/O hangs if I run the following command on top of kernel v4.16-rc2 + the
> ib_srpt patch that adds RDMA/CM support:
> 
> srp-test/run_tests -c -d -r 10 -t 02-mq -e kyber
> 
> This does not happen with the deadline scheduler nor without a scheduler.
> This test passed a few months ago. I have attached the output of the following
> command to this e-mail:
> 
> (cd /sys/kernel/debug/block && grep -r .) | grep -v /poll_stat
> 
> Can you have a look?
> 
> Thanks,
> 
> Bart.

Hey, Bart, thanks for the report. Can you clarify what the device
topology is w.r.t. the dm devices?

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: v4.16-rc2: I/O hang with dm-rq + Kyber
  2018-02-22 22:42 ` Omar Sandoval
@ 2018-02-22 23:22   ` Bart Van Assche
  0 siblings, 0 replies; 5+ messages in thread
From: Bart Van Assche @ 2018-02-22 23:22 UTC (permalink / raw)
  To: osandov; +Cc: linux-block

T24gVGh1LCAyMDE4LTAyLTIyIGF0IDE0OjQyIC0wODAwLCBPbWFyIFNhbmRvdmFsIHdyb3RlOg0K
PiBPbiBUaHUsIEZlYiAyMiwgMjAxOCBhdCAwOToxMDoyM1BNICswMDAwLCBCYXJ0IFZhbiBBc3Nj
aGUgd3JvdGU6DQo+ID4gSS9PIGhhbmdzIGlmIEkgcnVuIHRoZSBmb2xsb3dpbmcgY29tbWFuZCBv
biB0b3Agb2Yga2VybmVsIHY0LjE2LXJjMiArIHRoZQ0KPiA+IGliX3NycHQgcGF0Y2ggdGhhdCBh
ZGRzIFJETUEvQ00gc3VwcG9ydDoNCj4gPiANCj4gPiBzcnAtdGVzdC9ydW5fdGVzdHMgLWMgLWQg
LXIgMTAgLXQgMDItbXEgLWUga3liZXINCj4gPiANCj4gPiBUaGlzIGRvZXMgbm90IGhhcHBlbiB3
aXRoIHRoZSBkZWFkbGluZSBzY2hlZHVsZXIgbm9yIHdpdGhvdXQgYSBzY2hlZHVsZXIuDQo+ID4g
VGhpcyB0ZXN0IHBhc3NlZCBhIGZldyBtb250aHMgYWdvLiBJIGhhdmUgYXR0YWNoZWQgdGhlIG91
dHB1dCBvZiB0aGUgZm9sbG93aW5nDQo+ID4gY29tbWFuZCB0byB0aGlzIGUtbWFpbDoNCj4gPiAN
Cj4gPiAoY2QgL3N5cy9rZXJuZWwvZGVidWcvYmxvY2sgJiYgZ3JlcCAtciAuKSB8IGdyZXAgLXYg
L3BvbGxfc3RhdA0KPiA+IA0KPiA+IENhbiB5b3UgaGF2ZSBhIGxvb2s/DQo+IA0KPiBIZXksIEJh
cnQsIHRoYW5rcyBmb3IgdGhlIHJlcG9ydC4gQ2FuIHlvdSBjbGFyaWZ5IHdoYXQgdGhlIGRldmlj
ZQ0KPiB0b3BvbG9neSBpcyB3LnIudC4gdGhlIGRtIGRldmljZXM/DQoNCkhlbGxvIE9tYXIsDQoN
ClRoZSB0b3BvbG9neSB3YXMgbm90aGluZyBmYW5jeTogZG0tcnEgaW4gYmxrLW1xIG1vZGUgd2Fz
IGNvbmZpZ3VyZWQgdG8gdXNlDQpvbmUgc2NzaS1tcSBTUlAtb3Zlci1Sb0NFIHBhdGguIFRoZSBt
dWx0aXBhdGggb3V0cHV0IGlzIGFzIGZvbGxvd3M6DQoNCiMgbXVsdGlwYXRoIC1sbA0KbXBhdGhi
ICgzNjAwMTQwNTcyNjE2ZDY0Njk3MzZiMzEwMDAwMDAwMDApIGRtLTAgTElPLU9SRyxJQkxPQ0sN
CnNpemU9MzJNIGZlYXR1cmVzPSczIHF1ZXVlX2lmX25vX3BhdGggcXVldWVfbW9kZSBtcScgaHdo
YW5kbGVyPScwJyB3cD1ydw0KYC0rLSBwb2xpY3k9J3NlcnZpY2UtdGltZSAwJyBwcmlvPTUwIHN0
YXR1cz1hY3RpdmUNCiAgYC0gNDowOjA6MCBzZGMgODozMiBhY3RpdmUgcmVhZHkgcnVubmluZw0K
DQojIGZvciBkIGluIC9zeXMvYmxvY2sve2RtLTAsc2RjfTsgZG8gZWNobyAkKGJhc2VuYW1lICRk
KTogJCg8JGQvcXVldWUvc2NoZWR1bGVyKTsgZG9uZQ0KZG0tMDogYmZxIFtreWJlcl0gbXEtZGVh
ZGxpbmUgbm9uZQ0Kc2RjOiBiZnEgW2t5YmVyXSBtcS1kZWFkbGluZSBub25lDQoNCkJhcnQu

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: v4.16-rc2: I/O hang with dm-rq + Kyber
  2018-02-22 21:10 v4.16-rc2: I/O hang with dm-rq + Kyber Bart Van Assche
  2018-02-22 22:42 ` Omar Sandoval
@ 2018-02-23 16:26 ` Ming Lei
  2018-02-23 17:13   ` Bart Van Assche
  1 sibling, 1 reply; 5+ messages in thread
From: Ming Lei @ 2018-02-23 16:26 UTC (permalink / raw)
  To: Bart Van Assche; +Cc: osandov, linux-block

On Thu, Feb 22, 2018 at 09:10:23PM +0000, Bart Van Assche wrote:
> Hello Omar,
> 
> I/O hangs if I run the following command on top of kernel v4.16-rc2 + the
> ib_srpt patch that adds RDMA/CM support:
> 
> srp-test/run_tests -c -d -r 10 -t 02-mq -e kyber
> 
> This does not happen with the deadline scheduler nor without a scheduler.
> This test passed a few months ago. I have attached the output of the following
> command to this e-mail:
> 
> (cd /sys/kernel/debug/block && grep -r .) | grep -v /poll_stat
> 
> Can you have a look?

The following 2 patch fixes one IO hang on kyber in my test on USB, could
you test it and see if your case can be fixed?

	https://marc.info/?l=linux-block&m=151940022831994&w=2

BTW, from your attached log, looks it belongs to domain token leak
issue too, so it should have been addressed by above patches.

Thanks,
Ming

^ permalink raw reply	[flat|nested] 5+ messages in thread
* Re: v4.16-rc2: I/O hang with dm-rq + Kyber
  2018-02-23 16:26 ` Ming Lei
@ 2018-02-23 17:13   ` Bart Van Assche
  0 siblings, 0 replies; 5+ messages in thread
From: Bart Van Assche @ 2018-02-23 17:13 UTC (permalink / raw)
  To: ming.lei; +Cc: osandov, linux-block

T24gU2F0LCAyMDE4LTAyLTI0IGF0IDAwOjI2ICswODAwLCBNaW5nIExlaSB3cm90ZToNCj4gVGhl
IGZvbGxvd2luZyAyIHBhdGNoIGZpeGVzIG9uZSBJTyBoYW5nIG9uIGt5YmVyIGluIG15IHRlc3Qg
b24gVVNCLCBjb3VsZA0KPiB5b3UgdGVzdCBpdCBhbmQgc2VlIGlmIHlvdXIgY2FzZSBjYW4gYmUg
Zml4ZWQ/DQo+IA0KPiAJaHR0cHM6Ly9tYXJjLmluZm8vP2w9bGludXgtYmxvY2smbT0xNTE5NDAw
MjI4MzE5OTQmdz0yDQoNClRoZXNlIHR3byBwYXRjaGVzIGFyZSBzdWZmaWNpZW50IHRvIG1ha2Ug
bXkgdGVzdCBwYXNzLg0KDQpUaGFua3MhDQoNCkJhcnQuDQoNCg0K

^ permalink raw reply	[flat|nested] 5+ messages in thread
end of thread, other threads:[~2018-02-23 17:13 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-22 21:10 v4.16-rc2: I/O hang with dm-rq + Kyber Bart Van Assche
2018-02-22 22:42 ` Omar Sandoval
2018-02-22 23:22   ` Bart Van Assche
2018-02-23 16:26 ` Ming Lei
2018-02-23 17:13   ` Bart Van Assche

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.