randrw_16th_8k: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=psync, iodepth=1 ... fio-3.1 Starting 16 threads randrw_16th_8k: (groupid=0, jobs=16): err= 0: pid=5294: Mon Mar 25 03:15:49 2019 read: IOPS=1409, BW=11.0MiB/s (11.5MB/s)(662MiB/60100msec) clat (usec): min=3, max=1488.6k, avg=11313.66, stdev=22312.95 lat (usec): min=3, max=1488.6k, avg=11313.90, stdev=22312.95 clat percentiles (usec): | 1.00th=[ 5], 5.00th=[ 6], 10.00th=[ 8], 20.00th=[ 3392], | 30.00th=[ 5145], 40.00th=[ 6718], 50.00th=[ 8094], 60.00th=[ 9503], | 70.00th=[ 11338], 80.00th=[ 14484], 90.00th=[ 21103], 95.00th=[ 29754], | 99.00th=[ 68682], 99.50th=[109577], 99.90th=[278922], 99.95th=[379585], | 99.99th=[893387] bw ( KiB/s): min= 32, max= 1202, per=6.30%, avg=710.12, stdev=188.47, samples=1909 iops : min= 4, max= 150, avg=88.72, stdev=23.54, samples=1909 write: IOPS=1418, BW=11.1MiB/s (11.6MB/s)(666MiB/60100msec) clat (usec): min=4, max=7202, avg=19.68, stdev=40.91 lat (usec): min=5, max=7203, avg=20.09, stdev=40.97 clat percentiles (usec): | 1.00th=[ 7], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 12], | 30.00th=[ 13], 40.00th=[ 15], 50.00th=[ 18], 60.00th=[ 20], | 70.00th=[ 22], 80.00th=[ 26], 90.00th=[ 32], 95.00th=[ 37], | 99.00th=[ 56], 99.50th=[ 64], 99.90th=[ 91], 99.95th=[ 117], | 99.99th=[ 379] bw ( KiB/s): min= 16, max= 1523, per=6.30%, avg=715.24, stdev=247.64, samples=1908 iops : min= 2, max= 190, avg=89.36, stdev=30.93, samples=1908 lat (usec) : 4=0.02%, 10=8.31%, 20=29.16%, 50=17.64%, 100=0.69% lat (usec) : 250=0.04%, 500=0.66%, 750=0.08%, 1000=0.02% lat (msec) : 2=0.55%, 4=4.52%, 10=20.33%, 20=12.50%, 50=4.56% lat (msec) : 100=0.63%, 250=0.21%, 500=0.05%, 750=0.01%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.09%, sys=0.48%, ctx=75050, majf=0, minf=0 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwt: total=84702,85250,0, short=0,0,0, dropped=0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=11.0MiB/s (11.5MB/s), 11.0MiB/s-11.0MiB/s (11.5MB/s-11.5MB/s), io=662MiB (694MB), run=60100-60100msec WRITE: bw=11.1MiB/s (11.6MB/s), 11.1MiB/s-11.1MiB/s (11.6MB/s-11.6MB/s), io=666MiB (698MB), run=60100-60100msec Disk stats (read/write): sdb: ios=74911/60641, merge=0/13, ticks=950347/34711, in_queue=987372, util=100.00%