[PATCH V2 0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup

[PATCH V2 0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup

[Chaitanya Kulkarni]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="a", Size: 38129 bytes --]

Hi,

In order to test the non-trivial I/O path in the block layer for
REQ_OP_WRITE_ZEROES allow write-zeroes on null_blk so we can write
testcases with and without non-memory backed mode followed by
few cleanup patches.

Below is the test report with ext2/ext4 mkfs and a blktest
waiting to get upstream for this patch-series.

-ck

v1->v2:-

* Merge patch 1 and 2 (Damien).
* Change SECT_INX_IN_PAGE->SECT_OFFSET_IN_PAGE. (Damien)
* Add reviewed-by tags. (Damien)
* Add discard/write-zeroes module param to set the respective
  max sector length.

Chaitanya Kulkarni (7):
  null_blk: allow REQ_OP_WRITE_ZEROES
  null_blk: code cleaup
  null_blk: initialize cmd->bio in __alloc_cmd()
  null_blk: don't use magic numbers in the code
  null_blk: remove extra space in switch condition
  null_blk: add param to set max disacrd sectors
  null_blk: add param to set max write-zeroes sects

 drivers/block/null_blk/main.c     | 149 ++++++++++++++++++++++--------
 drivers/block/null_blk/null_blk.h |   3 +
 2 files changed, 116 insertions(+), 36 deletions(-)

linux-block (for-next) # gitlog -8
ab66a9484551 (HEAD -> for-next) null_blk: add param to set max write-zeroes sects
4ad2d08f4ea4 null_blk: add param to set max disacrd sectors
008e19cd64e6 null_blk: remove extra space in switch condition
b92a9bf280bb null_blk: don't use magic numbers in the code
cc56f23fb6bc null_blk: initialize cmd->bio in __alloc_cmd()
36037b231dee null_blk: code cleaup
d23c0fdf1ee2 null_blk: allow REQ_OP_WRITE_ZEROES
86e551dc35c9 Merge branch 'for-next' of git://git.kernel.dk/linux-block into for-next
linux-block (for-next) # ./compile_nullb.sh 
+ umount /mnt/nullb0
umount: /mnt/nullb0: not mounted.
+ rmdir 'config/nullb/nullb*'
rmdir: failed to remove 'config/nullb/nullb*': No such file or directory
+ dmesg -c
+ modprobe -r null_blk
+ lsmod
+ grep null_blk
++ nproc
+ make -j 48 M=drivers/block modules
  CC [M]  drivers/block/null_blk/main.o
  CC [M]  drivers/block/null_blk/trace.o
  CC [M]  drivers/block/null_blk/zoned.o
  LD [M]  drivers/block/null_blk/null_blk.o
  MODPOST drivers/block/Module.symvers
  LD [M]  drivers/block/null_blk/null_blk.ko
+ HOST=drivers/block/null_blk/
++ uname -r
+ HOST_DEST=/lib/modules/6.0.0-rc7blk+/kernel/drivers/block/null_blk/
+ cp drivers/block/null_blk//null_blk.ko /lib/modules/6.0.0-rc7blk+/kernel/drivers/block/null_blk//
+ ls -lrth /lib/modules/6.0.0-rc7blk+/kernel/drivers/block/null_blk//null_blk.ko
-rw-r--r--. 1 root root 1.2M Oct  5 19:55 /lib/modules/6.0.0-rc7blk+/kernel/drivers/block/null_blk//null_blk.ko
+ sleep 1
+ dmesg -c

blktests (master) # ./check block
block/001 (stress device hotplugging)                        [passed]
    runtime  53.159s  ...  49.759s
block/002 (remove a device while running blktrace)           [passed]
    runtime  0.798s  ...  1.267s
block/006 (run null-blk in blocking mode)                    [passed]
    read iops  8867450  ...  8810329
    runtime    9.669s   ...  9.850s
block/007 => nullb1 (test classic and hybrid IO polling)     [passed]
    auto hybrid poll read iops       619725      ...  615361
    auto hybrid poll read lat mean   1.366826    ...  1.371562
    auto hybrid poll system cpu      76.066351%  ...  74.352941%
    classic poll read iops           618264      ...  615361
    classic poll read lat mean       1.366413    ...  1.371557
    classic poll system cpu          73.995272%  ...  72.470588%
    fixed hybrid poll read iops      625642      ...  622669
    fixed hybrid poll read lat mean  1.352212    ...  1.357977
    fixed hybrid poll system cpu     75.598086%  ...  73.809524%
    no poll read iops                630153      ...  619725
    no poll read lat mean            1.345745    ...  1.362713
    no poll system cpu               73.975904%  ...  72.037915%
    runtime                          2.499s      ...  2.515s
block/009 (check page-cache coherency after BLKDISCARD)      [passed]
    runtime  0.590s  ...  0.560s
block/010 (run I/O on null_blk with shared and non-shared tags) [passed]
    Individual tags read iops  7911915  ...  7883103
    runtime                    21.139s  ...  20.894s
    Shared tags read iops      7837073  ...  7849908
block/014 (run null-blk with blk-mq and timeout injection configured) [passed]
    runtime  86.076s  ...  60.765s
block/015 (run null-blk on different schedulers with requeue injection configured) [passed]
    runtime  44.143s  ...  43.989s
block/016 (send a signal to a process waiting on a frozen queue) [passed]
    runtime  7.085s  ...  7.091s
block/017 (do I/O and check the inflight counter)            [passed]
    runtime  1.703s  ...  1.692s
block/018 (do I/O and check iostats times)                   [passed]
    runtime  5.111s  ...  5.107s
block/020 (run null-blk on different schedulers with only one hardware tag) [passed]
    runtime  41.501s  ...  41.558s
block/021 (read/write nr_requests on null-blk with different schedulers) [passed]
    runtime  5.854s  ...  5.854s
block/022 (Test hang caused by freeze/unfreeze sequence)     [passed]
    runtime  30.137s  ...  30.144s
block/023 (do I/O on all null_blk queue modes)               [passed]
    runtime  0.274s  ...  0.267s
block/024 (do I/O faster than a jiffy and check iostats times) [passed]
    runtime  2.657s  ...  2.660s
block/025 (do a huge discard with 4k sector size)            [passed]
    runtime  3.968s  ...  3.948s
block/027 (stress device hotplugging with running fio jobs and different schedulers) [passed]
    runtime  18.602s  ...  18.900s
block/028 (do I/O on scsi_debug with DIF/DIX enabled)        [passed]
    runtime  13.869s  ...  13.742s
block/029 (trigger blk_mq_update_nr_hw_queues())             [passed]
    runtime  30.283s  ...  30.297s
block/030 (trigger the blk_mq_realloc_hw_ctxs() error path)  [passed]
    runtime  2.142s  ...  2.141s
block/031 (do IO on null-blk with a host tag set)            [passed]
    runtime  30.324s  ...  30.327s


linux-block (for-next) # ./zeroout.sh 1
#################### BLKISZ 512 #####################
config/nullb/nullb0
├── badblocks
├── blocking
├── blocksize
├── cache_size
├── completion_nsec
├── discard
├── home_node
├── hw_queue_depth
├── index
├── irqmode
├── max_discard_sectors
├── max_sectors
├── max_write_zeroes_sectors
├── mbps
├── memory_backed
├── no_sched
├── poll_queues
├── power
├── queue_mode
├── shared_tag_bitmap
├── size
├── submit_queues
├── use_per_node_hctx
├── virt_boundary
├── write_zeroes
├── zone_capacity
├── zoned
├── zone_max_active
├── zone_max_open
├── zone_nr_conv
└── zone_size

0 directories, 31 files
MKFS:-
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 1b2daa8b-59f3-40d7-9e06-fa9f5347d08f
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel)
[18136.386533] null_blk: disk nullb0 created
[18137.411780] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem
[18137.412380] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][75.0%][r=1091MiB/s][r=279k IOPS][eta 00m:05s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=8936: Wed Oct  5 19:55:38 2022
  read: IOPS=278k, BW=1088MiB/s (1141MB/s)(2529MiB/2325msec)
    slat (nsec): min=2164, max=57528, avg=2545.52, stdev=505.53
    clat (nsec): min=1102, max=145345, avg=54147.07, stdev=2689.73
     lat (usec): min=3, max=148, avg=56.74, stdev= 2.76
    clat percentiles (usec):
     |  1.00th=[   52],  5.00th=[   52], 10.00th=[   52], 20.00th=[   53],
     | 30.00th=[   53], 40.00th=[   53], 50.00th=[   54], 60.00th=[   54],
     | 70.00th=[   55], 80.00th=[   55], 90.00th=[   59], 95.00th=[   61],
     | 99.00th=[   63], 99.50th=[   65], 99.90th=[   74], 99.95th=[   83],
     | 99.99th=[  108]
  write: IOPS=78.3k, BW=306MiB/s (321MB/s)(4000MiB/13070msec); 0 zone resets
    slat (usec): min=3, max=486, avg=12.06, stdev=10.45
    clat (nsec): min=781, max=937009, avg=191772.55, stdev=46098.76
     lat (usec): min=7, max=1274, avg=203.90, stdev=47.92
    clat percentiles (usec):
     |  1.00th=[  128],  5.00th=[  143], 10.00th=[  151], 20.00th=[  163],
     | 30.00th=[  172], 40.00th=[  180], 50.00th=[  186], 60.00th=[  194],
     | 70.00th=[  200], 80.00th=[  208], 90.00th=[  219], 95.00th=[  306],
     | 99.00th=[  363], 99.50th=[  383], 99.90th=[  537], 99.95th=[  594],
     | 99.99th=[  701]
   bw (  KiB/s): min=45520, max=368544, per=96.81%, avg=303407.41, stdev=59205.58, samples=27
   iops        : min=11380, max=92136, avg=75851.85, stdev=14801.40, samples=27
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=38.73%
  lat (usec)   : 250=57.66%, 500=3.50%, 750=0.11%, 1000=0.01%
  cpu          : usr=22.24%, sys=75.62%, ctx=3208, majf=0, minf=15193
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647410,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1088MiB/s (1141MB/s), 1088MiB/s-1088MiB/s (1141MB/s-1141MB/s), io=2529MiB (2652MB), run=2325-2325msec
  WRITE: bw=306MiB/s (321MB/s), 306MiB/s-306MiB/s (321MB/s-321MB/s), io=4000MiB (4194MB), run=13070-13070msec

Disk stats (read/write):
  nullb0: ios=645166/1024000, merge=0/0, ticks=714/1821, in_queue=2534, util=99.36%
mke2fs 1.45.6 (20-Mar-2020)
/dev/nullb0 contains a ext2 file system
	last mounted on /mnt/nullb0 on Wed Oct  5 19:55:22 2022
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 524cb8db-9768-493b-b6ad-abb1db199bcd
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel)
[18153.154196] EXT4-fs (nullb0): unmounting filesystem.
[18153.182343] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=0): [f(1)][100.0%][r=1043MiB/s][r=267k IOPS][eta 00m:00s]                 
write-and-verify: (groupid=0, jobs=1): err= 0: pid=8951: Wed Oct  5 19:55:51 2022
  read: IOPS=273k, BW=1066MiB/s (1118MB/s)(2528MiB/2370msec)
    slat (nsec): min=2174, max=97034, avg=2606.58, stdev=551.65
    clat (nsec): min=1112, max=151898, avg=55202.08, stdev=3056.22
     lat (usec): min=3, max=155, avg=57.85, stdev= 3.15
    clat percentiles (nsec):
     |  1.00th=[52480],  5.00th=[52992], 10.00th=[52992], 20.00th=[53504],
     | 30.00th=[54016], 40.00th=[54016], 50.00th=[54528], 60.00th=[54528],
     | 70.00th=[55040], 80.00th=[55552], 90.00th=[59136], 95.00th=[61696],
     | 99.00th=[66048], 99.50th=[70144], 99.90th=[79360], 99.95th=[82432],
     | 99.99th=[93696]
  write: IOPS=97.6k, BW=381MiB/s (400MB/s)(4000MiB/10489msec); 0 zone resets
    slat (usec): min=3, max=3626, avg= 9.62, stdev= 9.41
    clat (nsec): min=982, max=3790.8k, avg=153946.33, stdev=38887.01
     lat (usec): min=8, max=3800, avg=163.62, stdev=40.33
    clat percentiles (usec):
     |  1.00th=[  119],  5.00th=[  128], 10.00th=[  133], 20.00th=[  139],
     | 30.00th=[  145], 40.00th=[  149], 50.00th=[  151], 60.00th=[  155],
     | 70.00th=[  159], 80.00th=[  165], 90.00th=[  172], 95.00th=[  180],
     | 99.00th=[  269], 99.50th=[  318], 99.90th=[  351], 99.95th=[  494],
     | 99.99th=[  947]
   bw (  KiB/s): min=357048, max=428968, per=99.90%, avg=390095.24, stdev=23668.21, samples=21
   iops        : min=89262, max=107242, avg=97523.81, stdev=5917.05, samples=21
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.01%, 100=38.72%
  lat (usec)   : 250=60.59%, 500=0.66%, 750=0.02%, 1000=0.01%
  lat (msec)   : 4=0.01%
  cpu          : usr=26.27%, sys=73.56%, ctx=29, majf=0, minf=15182
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647042,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1066MiB/s (1118MB/s), 1066MiB/s-1066MiB/s (1118MB/s-1118MB/s), io=2528MiB (2650MB), run=2370-2370msec
  WRITE: bw=381MiB/s (400MB/s), 381MiB/s-381MiB/s (400MB/s-400MB/s), io=4000MiB (4194MB), run=10489-10489msec

Disk stats (read/write):
  nullb0: ios=587867/1024234, merge=0/5709, ticks=651/1908, in_queue=2559, util=99.27%




#################### BLKISZ 1024 #####################
config/nullb/nullb0
├── badblocks
├── blocking
├── blocksize
├── cache_size
├── completion_nsec
├── discard
├── home_node
├── hw_queue_depth
├── index
├── irqmode
├── max_discard_sectors
├── max_sectors
├── max_write_zeroes_sectors
├── mbps
├── memory_backed
├── no_sched
├── poll_queues
├── power
├── queue_mode
├── shared_tag_bitmap
├── size
├── submit_queues
├── use_per_node_hctx
├── virt_boundary
├── write_zeroes
├── zone_capacity
├── zoned
├── zone_max_active
├── zone_max_open
├── zone_nr_conv
└── zone_size

0 directories, 31 files
MKFS:-
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 66fa20c2-4bd7-4dd5-9002-8f832b198068
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel)
[18166.870587] null_blk: disk nullb0 created
[18167.892430] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem
[18167.893033] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][71.4%][r=1127MiB/s][r=289k IOPS][eta 00m:06s]                  
write-and-verify: (groupid=0, jobs=1): err= 0: pid=8976: Wed Oct  5 19:56:09 2022
  read: IOPS=288k, BW=1123MiB/s (1178MB/s)(2528MiB/2251msec)
    slat (nsec): min=2014, max=92396, avg=2422.41, stdev=512.19
    clat (nsec): min=1303, max=167628, avg=52433.36, stdev=2464.96
     lat (usec): min=3, max=170, avg=54.90, stdev= 2.53
    clat percentiles (usec):
     |  1.00th=[   50],  5.00th=[   51], 10.00th=[   51], 20.00th=[   51],
     | 30.00th=[   52], 40.00th=[   52], 50.00th=[   52], 60.00th=[   52],
     | 70.00th=[   53], 80.00th=[   53], 90.00th=[   57], 95.00th=[   59],
     | 99.00th=[   61], 99.50th=[   62], 99.90th=[   67], 99.95th=[   69],
     | 99.99th=[  106]
  write: IOPS=75.7k, BW=296MiB/s (310MB/s)(4000MiB/13530msec); 0 zone resets
    slat (usec): min=3, max=466, avg=12.47, stdev=10.01
    clat (nsec): min=1052, max=903526, avg=198530.45, stdev=47804.47
     lat (usec): min=27, max=1064, avg=211.08, stdev=49.81
    clat percentiles (usec):
     |  1.00th=[  133],  5.00th=[  147], 10.00th=[  157], 20.00th=[  167],
     | 30.00th=[  178], 40.00th=[  186], 50.00th=[  194], 60.00th=[  200],
     | 70.00th=[  206], 80.00th=[  215], 90.00th=[  229], 95.00th=[  314],
     | 99.00th=[  375], 99.50th=[  404], 99.90th=[  562], 99.95th=[  611],
     | 99.99th=[  701]
   bw (  KiB/s): min=16080, max=355544, per=96.64%, avg=292571.43, stdev=60750.67, samples=28
   iops        : min= 4020, max=88886, avg=73142.86, stdev=15187.66, samples=28
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.50%, 100=38.23%
  lat (usec)   : 250=57.21%, 500=3.95%, 750=0.11%, 1000=0.01%
  cpu          : usr=23.21%, sys=74.54%, ctx=3346, majf=0, minf=15186
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647231,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1123MiB/s (1178MB/s), 1123MiB/s-1123MiB/s (1178MB/s-1178MB/s), io=2528MiB (2651MB), run=2251-2251msec
  WRITE: bw=296MiB/s (310MB/s), 296MiB/s-296MiB/s (310MB/s-310MB/s), io=4000MiB (4194MB), run=13530-13530msec

Disk stats (read/write):
  nullb0: ios=605621/1024000, merge=0/0, ticks=602/1664, in_queue=2266, util=99.36%
mke2fs 1.45.6 (20-Mar-2020)
/dev/nullb0 contains a ext2 file system
	last mounted on /mnt/nullb0 on Wed Oct  5 19:55:53 2022
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: eeac79fa-2a6f-497c-a04a-e22e48867b4a
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel)
[18184.019532] EXT4-fs (nullb0): unmounting filesystem.
[18184.043893] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][75.0%][r=1123MiB/s][r=287k IOPS][eta 00m:04s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=8990: Wed Oct  5 19:56:22 2022
  read: IOPS=287k, BW=1119MiB/s (1174MB/s)(2530MiB/2260msec)
    slat (nsec): min=2024, max=94349, avg=2444.00, stdev=532.18
    clat (nsec): min=1383, max=148101, avg=52612.45, stdev=2521.26
     lat (usec): min=3, max=150, avg=55.10, stdev= 2.58
    clat percentiles (nsec):
     |  1.00th=[50432],  5.00th=[50432], 10.00th=[50944], 20.00th=[51456],
     | 30.00th=[51456], 40.00th=[51456], 50.00th=[51968], 60.00th=[51968],
     | 70.00th=[52480], 80.00th=[52992], 90.00th=[56064], 95.00th=[58624],
     | 99.00th=[60672], 99.50th=[61696], 99.90th=[66048], 99.95th=[76288],
     | 99.99th=[98816]
  write: IOPS=101k, BW=396MiB/s (415MB/s)(4000MiB/10107msec); 0 zone resets
    slat (usec): min=3, max=3934, avg= 9.24, stdev= 8.67
    clat (nsec): min=792, max=7280.8k, avg=148351.59, stdev=40152.32
     lat (usec): min=6, max=7292, avg=157.65, stdev=41.65
    clat percentiles (usec):
     |  1.00th=[  114],  5.00th=[  123], 10.00th=[  128], 20.00th=[  135],
     | 30.00th=[  139], 40.00th=[  143], 50.00th=[  147], 60.00th=[  151],
     | 70.00th=[  155], 80.00th=[  161], 90.00th=[  167], 95.00th=[  176],
     | 99.00th=[  202], 99.50th=[  223], 99.90th=[  277], 99.95th=[  433],
     | 99.99th=[  486]
   bw (  KiB/s): min=93312, max=450696, per=96.26%, avg=390095.24, stdev=73136.38, samples=21
   iops        : min=23328, max=112674, avg=97523.81, stdev=18284.09, samples=21
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=0.16%, 100=38.59%
  lat (usec)   : 250=61.09%, 500=0.16%, 750=0.01%
  lat (msec)   : 4=0.01%, 10=0.01%
  cpu          : usr=27.71%, sys=72.12%, ctx=29, majf=0, minf=15196
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647570,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1119MiB/s (1174MB/s), 1119MiB/s-1119MiB/s (1174MB/s-1174MB/s), io=2530MiB (2652MB), run=2260-2260msec
  WRITE: bw=396MiB/s (415MB/s), 396MiB/s-396MiB/s (415MB/s-415MB/s), io=4000MiB (4194MB), run=10107-10107msec

Disk stats (read/write):
  nullb0: ios=583689/1024237, merge=0/5798, ticks=571/1672, in_queue=2244, util=99.23%




#################### BLKISZ 2048 #####################
config/nullb/nullb0
├── badblocks
├── blocking
├── blocksize
├── cache_size
├── completion_nsec
├── discard
├── home_node
├── hw_queue_depth
├── index
├── irqmode
├── max_discard_sectors
├── max_sectors
├── max_write_zeroes_sectors
├── mbps
├── memory_backed
├── no_sched
├── poll_queues
├── power
├── queue_mode
├── shared_tag_bitmap
├── size
├── submit_queues
├── use_per_node_hctx
├── virt_boundary
├── write_zeroes
├── zone_capacity
├── zoned
├── zone_max_active
├── zone_max_open
├── zone_nr_conv
└── zone_size

0 directories, 31 files
MKFS:-
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 95af17f3-1251-4eb3-bb08-ef59d64de711
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel)
[18197.234676] null_blk: disk nullb0 created
[18198.256145] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem
[18198.256835] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][75.0%][r=1176MiB/s][r=301k IOPS][eta 00m:05s]                  
write-and-verify: (groupid=0, jobs=1): err= 0: pid=9014: Wed Oct  5 19:56:39 2022
  read: IOPS=300k, BW=1172MiB/s (1229MB/s)(2526MiB/2156msec)
    slat (nsec): min=1883, max=95211, avg=2289.83, stdev=538.98
    clat (nsec): min=1102, max=144034, avg=50260.51, stdev=3111.72
     lat (usec): min=3, max=146, avg=52.59, stdev= 3.20
    clat percentiles (usec):
     |  1.00th=[   48],  5.00th=[   49], 10.00th=[   49], 20.00th=[   49],
     | 30.00th=[   49], 40.00th=[   49], 50.00th=[   50], 60.00th=[   50],
     | 70.00th=[   50], 80.00th=[   51], 90.00th=[   55], 95.00th=[   57],
     | 99.00th=[   62], 99.50th=[   66], 99.90th=[   79], 99.95th=[   84],
     | 99.99th=[  104]
  write: IOPS=77.2k, BW=301MiB/s (316MB/s)(4000MiB/13272msec); 0 zone resets
    slat (usec): min=3, max=392, avg=12.21, stdev= 9.32
    clat (nsec): min=792, max=996792, avg=194754.81, stdev=45154.41
     lat (usec): min=6, max=1032, avg=207.04, stdev=47.09
    clat percentiles (usec):
     |  1.00th=[  127],  5.00th=[  143], 10.00th=[  151], 20.00th=[  163],
     | 30.00th=[  174], 40.00th=[  182], 50.00th=[  190], 60.00th=[  198],
     | 70.00th=[  206], 80.00th=[  215], 90.00th=[  227], 95.00th=[  289],
     | 99.00th=[  363], 99.50th=[  375], 99.90th=[  529], 99.95th=[  619],
     | 99.99th=[  725]
   bw (  KiB/s): min=192784, max=369664, per=98.30%, avg=303385.96, stdev=40667.67, samples=27
   iops        : min=48196, max=92416, avg=75846.48, stdev=10166.92, samples=27
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=28.00%, 100=10.71%
  lat (usec)   : 250=57.62%, 500=3.59%, 750=0.08%, 1000=0.01%
  cpu          : usr=23.52%, sys=74.41%, ctx=3289, majf=0, minf=15175
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=646749,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1172MiB/s (1229MB/s), 1172MiB/s-1172MiB/s (1229MB/s-1229MB/s), io=2526MiB (2649MB), run=2156-2156msec
  WRITE: bw=301MiB/s (316MB/s), 301MiB/s-301MiB/s (316MB/s-316MB/s), io=4000MiB (4194MB), run=13272-13272msec

Disk stats (read/write):
  nullb0: ios=634659/1024000, merge=0/0, ticks=558/1600, in_queue=2157, util=99.38%
mke2fs 1.45.6 (20-Mar-2020)
/dev/nullb0 contains a ext2 file system
	last mounted on /mnt/nullb0 on Wed Oct  5 19:56:23 2022
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: b3f69fcb-abd2-41eb-aaf5-621503259a75
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel)
[18214.019199] EXT4-fs (nullb0): unmounting filesystem.
[18214.041644] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][80.0%][r=1168MiB/s][r=299k IOPS][eta 00m:03s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=9027: Wed Oct  5 19:56:52 2022
  read: IOPS=297k, BW=1161MiB/s (1217MB/s)(2528MiB/2178msec)
    slat (nsec): min=1903, max=61776, avg=2321.68, stdev=544.98
    clat (nsec): min=1102, max=111392, avg=50739.27, stdev=3167.99
     lat (usec): min=3, max=113, avg=53.10, stdev= 3.27
    clat percentiles (usec):
     |  1.00th=[   48],  5.00th=[   49], 10.00th=[   49], 20.00th=[   49],
     | 30.00th=[   50], 40.00th=[   50], 50.00th=[   50], 60.00th=[   50],
     | 70.00th=[   51], 80.00th=[   51], 90.00th=[   55], 95.00th=[   58],
     | 99.00th=[   65], 99.50th=[   71], 99.90th=[   77], 99.95th=[   80],
     | 99.99th=[  104]
  write: IOPS=101k, BW=396MiB/s (415MB/s)(4000MiB/10105msec); 0 zone resets
    slat (usec): min=3, max=3540, avg= 9.23, stdev= 8.10
    clat (nsec): min=831, max=3715.1k, avg=148313.45, stdev=34493.81
     lat (usec): min=17, max=3722, avg=157.60, stdev=35.79
    clat percentiles (usec):
     |  1.00th=[  113],  5.00th=[  122], 10.00th=[  127], 20.00th=[  135],
     | 30.00th=[  139], 40.00th=[  143], 50.00th=[  147], 60.00th=[  151],
     | 70.00th=[  155], 80.00th=[  161], 90.00th=[  169], 95.00th=[  178],
     | 99.00th=[  215], 99.50th=[  243], 99.90th=[  281], 99.95th=[  334],
     | 99.99th=[  404]
   bw (  KiB/s): min=89824, max=454648, per=96.24%, avg=390095.24, stdev=74144.52, samples=21
   iops        : min=22456, max=113662, avg=97523.81, stdev=18536.13, samples=21
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=21.93%, 100=16.81%
  lat (usec)   : 250=61.03%, 500=0.22%
  lat (msec)   : 4=0.01%
  cpu          : usr=28.16%, sys=71.64%, ctx=191, majf=0, minf=15190
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647285,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1161MiB/s (1217MB/s), 1161MiB/s-1161MiB/s (1217MB/s-1217MB/s), io=2528MiB (2651MB), run=2178-2178msec
  WRITE: bw=396MiB/s (415MB/s), 396MiB/s-396MiB/s (415MB/s-415MB/s), io=4000MiB (4194MB), run=10105-10105msec

Disk stats (read/write):
  nullb0: ios=606388/1024235, merge=0/5718, ticks=528/1602, in_queue=2130, util=99.24%




#################### BLKISZ 4096 #####################
config/nullb/nullb0
├── badblocks
├── blocking
├── blocksize
├── cache_size
├── completion_nsec
├── discard
├── home_node
├── hw_queue_depth
├── index
├── irqmode
├── max_discard_sectors
├── max_sectors
├── max_write_zeroes_sectors
├── mbps
├── memory_backed
├── no_sched
├── poll_queues
├── power
├── queue_mode
├── shared_tag_bitmap
├── size
├── submit_queues
├── use_per_node_hctx
├── virt_boundary
├── write_zeroes
├── zone_capacity
├── zoned
├── zone_max_active
├── zone_max_open
├── zone_nr_conv
└── zone_size

0 directories, 31 files
MKFS:-
mke2fs 1.45.6 (20-Mar-2020)
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 90636218-f980-4d5f-a306-d0b38fd30e92
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext2 (rw,relatime,seclabel)
[18227.145657] null_blk: disk nullb0 created
[18228.166279] EXT4-fs (nullb0): mounting ext2 file system using the ext4 subsystem
[18228.166990] EXT4-fs (nullb0): mounted filesystem without journal. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][78.9%][r=1174MiB/s][r=301k IOPS][eta 00m:04s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=9053: Wed Oct  5 19:57:08 2022
  read: IOPS=298k, BW=1165MiB/s (1221MB/s)(2530MiB/2172msec)
    slat (nsec): min=1923, max=112183, avg=2305.07, stdev=500.91
    clat (nsec): min=1342, max=256057, avg=50578.91, stdev=2797.70
     lat (usec): min=3, max=261, avg=52.93, stdev= 2.88
    clat percentiles (usec):
     |  1.00th=[   48],  5.00th=[   49], 10.00th=[   49], 20.00th=[   49],
     | 30.00th=[   50], 40.00th=[   50], 50.00th=[   50], 60.00th=[   50],
     | 70.00th=[   51], 80.00th=[   51], 90.00th=[   55], 95.00th=[   57],
     | 99.00th=[   60], 99.50th=[   61], 99.90th=[   68], 99.95th=[   81],
     | 99.99th=[  109]
  write: IOPS=79.2k, BW=309MiB/s (325MB/s)(4000MiB/12925msec); 0 zone resets
    slat (usec): min=3, max=527, avg=11.92, stdev= 9.64
    clat (nsec): min=942, max=974140, avg=189663.16, stdev=44574.80
     lat (usec): min=8, max=1353, avg=201.65, stdev=46.43
    clat percentiles (usec):
     |  1.00th=[  127],  5.00th=[  143], 10.00th=[  151], 20.00th=[  161],
     | 30.00th=[  169], 40.00th=[  178], 50.00th=[  186], 60.00th=[  192],
     | 70.00th=[  198], 80.00th=[  206], 90.00th=[  219], 95.00th=[  273],
     | 99.00th=[  363], 99.50th=[  400], 99.90th=[  545], 99.95th=[  611],
     | 99.99th=[  807]
   bw (  KiB/s): min=251544, max=368624, per=99.42%, avg=315076.92, stdev=30985.16, samples=26
   iops        : min=62886, max=92156, avg=78769.23, stdev=7746.29, samples=26
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=21.02%, 100=17.72%
  lat (usec)   : 250=58.00%, 500=3.16%, 750=0.09%, 1000=0.01%
  cpu          : usr=23.65%, sys=74.12%, ctx=3203, majf=0, minf=15194
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647556,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1165MiB/s (1221MB/s), 1165MiB/s-1165MiB/s (1221MB/s-1221MB/s), io=2530MiB (2652MB), run=2172-2172msec
  WRITE: bw=309MiB/s (325MB/s), 309MiB/s-309MiB/s (325MB/s-325MB/s), io=4000MiB (4194MB), run=12925-12925msec

Disk stats (read/write):
  nullb0: ios=583450/1024000, merge=0/0, ticks=517/1562, in_queue=2079, util=99.33%
mke2fs 1.45.6 (20-Mar-2020)
/dev/nullb0 contains a ext2 file system
	last mounted on /mnt/nullb0 on Wed Oct  5 19:56:53 2022
Creating filesystem with 1310720 4k blocks and 327680 inodes
Filesystem UUID: 03970406-594a-4d78-b966-d6dbdc697b16
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

/dev/nullb0 on /mnt/nullb0 type ext4 (rw,relatime,seclabel)
[18243.612194] EXT4-fs (nullb0): unmounting filesystem.
[18243.634860] EXT4-fs (nullb0): mounted filesystem with ordered data mode. Quota mode: none.
write-and-verify: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16
fio-3.27
Starting 1 process
write-and-verify: Laying out IO file (1 file / 4000MiB)
Jobs: 1 (f=1): [V(1)][80.0%][r=1161MiB/s][r=297k IOPS][eta 00m:03s]                   
write-and-verify: (groupid=0, jobs=1): err= 0: pid=9066: Wed Oct  5 19:57:21 2022
  read: IOPS=295k, BW=1151MiB/s (1207MB/s)(2528MiB/2196msec)
    slat (nsec): min=1944, max=76366, avg=2349.04, stdev=512.68
    clat (nsec): min=1092, max=132292, avg=51163.41, stdev=2525.10
     lat (usec): min=3, max=134, avg=53.56, stdev= 2.60
    clat percentiles (usec):
     |  1.00th=[   49],  5.00th=[   49], 10.00th=[   49], 20.00th=[   50],
     | 30.00th=[   50], 40.00th=[   51], 50.00th=[   51], 60.00th=[   51],
     | 70.00th=[   52], 80.00th=[   52], 90.00th=[   55], 95.00th=[   58],
     | 99.00th=[   60], 99.50th=[   61], 99.90th=[   65], 99.95th=[   68],
     | 99.99th=[  109]
  write: IOPS=101k, BW=396MiB/s (416MB/s)(4000MiB/10093msec); 0 zone resets
    slat (usec): min=3, max=3218, avg= 9.23, stdev= 7.92
    clat (nsec): min=772, max=3407.3k, avg=148142.35, stdev=32842.67
     lat (usec): min=6, max=3419, avg=157.43, stdev=34.05
    clat percentiles (usec):
     |  1.00th=[  114],  5.00th=[  123], 10.00th=[  128], 20.00th=[  135],
     | 30.00th=[  139], 40.00th=[  143], 50.00th=[  147], 60.00th=[  151],
     | 70.00th=[  155], 80.00th=[  161], 90.00th=[  167], 95.00th=[  176],
     | 99.00th=[  198], 99.50th=[  210], 99.90th=[  273], 99.95th=[  338],
     | 99.99th=[  457]
   bw (  KiB/s): min=78848, max=452720, per=96.12%, avg=390095.24, stdev=76125.82, samples=21
   iops        : min=19712, max=113180, avg=97523.81, stdev=19031.45, samples=21
  lat (nsec)   : 1000=0.01%
  lat (usec)   : 2=0.01%, 10=0.01%, 20=0.01%, 50=12.29%, 100=26.44%
  lat (usec)   : 250=61.14%, 500=0.13%
  lat (msec)   : 4=0.01%
  cpu          : usr=27.40%, sys=72.43%, ctx=28, majf=0, minf=15187
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=100.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=647188,1024000,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=16

Run status group 0 (all jobs):
   READ: bw=1151MiB/s (1207MB/s), 1151MiB/s-1151MiB/s (1207MB/s-1207MB/s), io=2528MiB (2651MB), run=2196-2196msec
  WRITE: bw=396MiB/s (416MB/s), 396MiB/s-396MiB/s (416MB/s-416MB/s), io=4000MiB (4194MB), run=10093-10093msec

Disk stats (read/write):
  nullb0: ios=604283/1024239, merge=0/5858, ticks=528/1597, in_queue=2125, util=99.24%



* Without max discard and write-zeroes set :-

linux-block (for-next) # for i in `ls /sys/block/nullb0/queue/*discard*`;
do echo -n $i:   ;cat $i; echo "";  done 
/sys/block/nullb0/queue/discard_granularity:4096

/sys/block/nullb0/queue/discard_max_bytes:4294966784

/sys/block/nullb0/queue/discard_max_hw_bytes:4294966784

/sys/block/nullb0/queue/discard_zeroes_data:0

/sys/block/nullb0/queue/max_discard_segments:1

linux-block (for-next) # for i in `ls /sys/block/nullb0/queue/*zeroes*`;
do echo -n $i:   ;cat $i; echo "";  done 
/sys/block/nullb0/queue/discard_zeroes_data:0

/sys/block/nullb0/queue/write_zeroes_max_bytes:4294966784
* With max discard and write-zeroes set to 1  :-

linux-block (for-next) # for i in `ls /sys/block/nullb0/queue/*discard*`;
do echo -n $i:   ;cat $i; echo "";  done 
/sys/block/nullb0/queue/discard_granularity:4096

/sys/block/nullb0/queue/discard_max_bytes:512

/sys/block/nullb0/queue/discard_max_hw_bytes:512

/sys/block/nullb0/queue/discard_zeroes_data:0

/sys/block/nullb0/queue/max_discard_segments:1

linux-block (for-next) # for i in `ls /sys/block/nullb0/queue/*zeroes*`; do echo -n $i:   ;cat $i; echo "";  done /sys/block/nullb0/queue/discard_zeroes_data:0

/sys/block/nullb0/queue/write_zeroes_max_bytes:512

linux-block (for-next) # 





-- 
2.29.0

[PATCH V2 1/7] null_blk: allow REQ_OP_WRITE_ZEROES

[Chaitanya Kulkarni]

Add a helper function to enable the REQ_OP_WRITE_ZEROES operations
when null_blk.

Since write-zeroes is a non-trivial I/O operation we need this to
add a blktest so we can test the non-trivial I/O path from the
application to the block layer.

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/block/null_blk/main.c     | 59 ++++++++++++++++++++++++++++++-
 drivers/block/null_blk/null_blk.h |  1 +
 2 files changed, 59 insertions(+), 1 deletion(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 1f154f92f4c2..2d592b4eb815 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -209,6 +209,10 @@ static bool g_discard;
 module_param_named(discard, g_discard, bool, 0444);
 MODULE_PARM_DESC(discard, "Support discard operations (requires memory-backed null_blk device). Default: false");
 
+static bool g_write_zeroes;
+module_param_named(write_zeroes, g_write_zeroes, bool, 0444);
+MODULE_PARM_DESC(write_zeroes, "Support write-zeores operations. Default: false");
+
 static unsigned long g_cache_size;
 module_param_named(cache_size, g_cache_size, ulong, 0444);
 MODULE_PARM_DESC(mbps, "Cache size in MiB for memory-backed device. Default: 0 (none)");
@@ -416,6 +420,7 @@ NULLB_DEVICE_ATTR(blocking, bool, NULL);
 NULLB_DEVICE_ATTR(use_per_node_hctx, bool, NULL);
 NULLB_DEVICE_ATTR(memory_backed, bool, NULL);
 NULLB_DEVICE_ATTR(discard, bool, NULL);
+NULLB_DEVICE_ATTR(write_zeroes, bool, NULL);
 NULLB_DEVICE_ATTR(mbps, uint, NULL);
 NULLB_DEVICE_ATTR(cache_size, ulong, NULL);
 NULLB_DEVICE_ATTR(zoned, bool, NULL);
@@ -540,6 +545,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
 	&nullb_device_attr_power,
 	&nullb_device_attr_memory_backed,
 	&nullb_device_attr_discard,
+	&nullb_device_attr_write_zeroes,
 	&nullb_device_attr_mbps,
 	&nullb_device_attr_cache_size,
 	&nullb_device_attr_badblocks,
@@ -614,7 +620,7 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page)
 			"poll_queues,power,queue_mode,shared_tag_bitmap,size,"
 			"submit_queues,use_per_node_hctx,virt_boundary,zoned,"
 			"zone_capacity,zone_max_active,zone_max_open,"
-			"zone_nr_conv,zone_size\n");
+			"zone_nr_conv,zone_size,write_zeroes\n");
 }
 
 CONFIGFS_ATTR_RO(memb_group_, features);
@@ -678,6 +684,7 @@ static struct nullb_device *null_alloc_dev(void)
 	dev->blocking = g_blocking;
 	dev->memory_backed = g_memory_backed;
 	dev->discard = g_discard;
+	dev->write_zeroes = g_write_zeroes;
 	dev->cache_size = g_cache_size;
 	dev->mbps = g_mbps;
 	dev->use_per_node_hctx = g_use_per_node_hctx;
@@ -870,6 +877,24 @@ static void null_free_sector(struct nullb *nullb, sector_t sector,
 	}
 }
 
+static void null_zero_sector(struct nullb_device *d, sector_t sect,
+			     sector_t nr_sects, bool cache)
+{
+	struct radix_tree_root *root = cache ? &d->cache : &d->data;
+	struct nullb_page *t_page;
+	unsigned int offset;
+	void *dest;
+
+	t_page = radix_tree_lookup(root, sect >> PAGE_SECTORS_SHIFT);
+	if (!t_page)
+		return;
+
+	offset = (sect & SECTOR_MASK) << SECTOR_SHIFT;
+	dest = kmap_atomic(t_page->page);
+	memset(dest + offset, 0, SECTOR_SIZE * nr_sects);
+	kunmap_atomic(dest);
+}
+
 static struct nullb_page *null_radix_tree_insert(struct nullb *nullb, u64 idx,
 	struct nullb_page *t_page, bool is_cache)
 {
@@ -1186,6 +1211,27 @@ blk_status_t null_handle_discard(struct nullb_device *dev,
 	return BLK_STS_OK;
 }
 
+static blk_status_t null_handle_write_zeroes(struct nullb_device *dev,
+					     sector_t sector, sector_t nr_sectors)
+{
+	unsigned int bytes_left = nr_sectors << 9;
+	struct nullb *nullb = dev->nullb;
+	size_t curr_bytes;
+
+	spin_lock_irq(&nullb->lock);
+	while (bytes_left > 0) {
+		curr_bytes = min_t(size_t, bytes_left, nullb->dev->blocksize);
+		nr_sectors = curr_bytes >> SECTOR_SHIFT;
+		null_zero_sector(nullb->dev, sector, nr_sectors, false);
+		if (null_cache_active(nullb))
+			null_zero_sector(nullb->dev, sector, nr_sectors, true);
+		sector += nr_sectors;
+		bytes_left -= curr_bytes;
+	}
+	spin_unlock_irq(&nullb->lock);
+	return BLK_STS_OK;
+}
+
 static int null_handle_flush(struct nullb *nullb)
 {
 	int err;
@@ -1352,6 +1398,9 @@ static inline blk_status_t null_handle_memory_backed(struct nullb_cmd *cmd,
 	if (op == REQ_OP_DISCARD)
 		return null_handle_discard(dev, sector, nr_sectors);
 
+	if (op == REQ_OP_WRITE_ZEROES)
+		return null_handle_write_zeroes(dev, sector, nr_sectors);
+
 	if (dev->queue_mode == NULL_Q_BIO)
 		err = null_handle_bio(cmd);
 	else
@@ -1800,6 +1849,13 @@ static void null_config_discard(struct nullb *nullb)
 	blk_queue_max_discard_sectors(nullb->q, UINT_MAX >> 9);
 }
 
+static void null_config_write_zeroes(struct nullb *nullb)
+{
+	if (!nullb->dev->write_zeroes)
+		return;
+	blk_queue_max_write_zeroes_sectors(nullb->q, UINT_MAX >> 9);
+}
+
 static const struct block_device_operations null_bio_ops = {
 	.owner		= THIS_MODULE,
 	.submit_bio	= null_submit_bio,
@@ -2111,6 +2167,7 @@ static int null_add_dev(struct nullb_device *dev)
 		blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1);
 
 	null_config_discard(nullb);
+	null_config_write_zeroes(nullb);
 
 	if (config_item_name(&dev->item)) {
 		/* Use configfs dir name as the device name */
diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h
index 94ff68052b1e..2c0c9c29158f 100644
--- a/drivers/block/null_blk/null_blk.h
+++ b/drivers/block/null_blk/null_blk.h
@@ -111,6 +111,7 @@ struct nullb_device {
 	bool power; /* power on/off the device */
 	bool memory_backed; /* if data is stored in memory */
 	bool discard; /* if support discard */
+	bool write_zeroes; /* if support write_zeroes */
 	bool zoned; /* if device is zoned */
 	bool virt_boundary; /* virtual boundary on/off for the device */
 	bool no_sched; /* no IO scheduler for the device */
-- 
2.29.0

[PATCH V2 2/7] null_blk: code cleaup

[Chaitanya Kulkarni]

Introduce and use two new macros for calculating the page index from
given sector and index (offset) of the sector in the page.
The newly added macros makes code easy to read with meaningful name and
explanation comments attached to it.

While at it adjust the code in the null_free_sector() to return early
to get rid of the extra identation.

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/block/null_blk/main.c | 37 ++++++++++++++++++++---------------
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 2d592b4eb815..fa47fab279c3 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -14,6 +14,11 @@
 #undef pr_fmt
 #define pr_fmt(fmt)	"null_blk: " fmt
 
+/* Gives page index for which this sector belongs to. */
+#define PAGE_IDX_FROM_SECT(sect)	(sect >> PAGE_SECTORS_SHIFT)
+/* Gives index (offset) of the sector within page. */
+#define SECT_OFFSET_IN_PAGE(sect)		((sect & SECTOR_MASK) << SECTOR_SHIFT)
+
 #define FREE_BATCH		16
 
 #define TICKS_PER_SEC		50ULL
@@ -860,20 +865,20 @@ static void null_free_sector(struct nullb *nullb, sector_t sector,
 	struct radix_tree_root *root;
 
 	root = is_cache ? &nullb->dev->cache : &nullb->dev->data;
-	idx = sector >> PAGE_SECTORS_SHIFT;
+	idx = PAGE_IDX_FROM_SECT(sector);
 	sector_bit = (sector & SECTOR_MASK);
 
 	t_page = radix_tree_lookup(root, idx);
-	if (t_page) {
-		__clear_bit(sector_bit, t_page->bitmap);
-
-		if (null_page_empty(t_page)) {
-			ret = radix_tree_delete_item(root, idx, t_page);
-			WARN_ON(ret != t_page);
-			null_free_page(ret);
-			if (is_cache)
-				nullb->dev->curr_cache -= PAGE_SIZE;
-		}
+	if (!t_page)
+		return;
+	__clear_bit(sector_bit, t_page->bitmap);
+
+	if (null_page_empty(t_page)) {
+		ret = radix_tree_delete_item(root, idx, t_page);
+		WARN_ON(ret != t_page);
+		null_free_page(ret);
+		if (is_cache)
+			nullb->dev->curr_cache -= PAGE_SIZE;
 	}
 }
 
@@ -885,11 +890,11 @@ static void null_zero_sector(struct nullb_device *d, sector_t sect,
 	unsigned int offset;
 	void *dest;
 
-	t_page = radix_tree_lookup(root, sect >> PAGE_SECTORS_SHIFT);
+	t_page = radix_tree_lookup(root, PAGE_IDX_FROM_SECT(sect));
 	if (!t_page)
 		return;
 
-	offset = (sect & SECTOR_MASK) << SECTOR_SHIFT;
+	offset = SECT_OFFSET_IN_PAGE(sect);
 	dest = kmap_atomic(t_page->page);
 	memset(dest + offset, 0, SECTOR_SIZE * nr_sects);
 	kunmap_atomic(dest);
@@ -949,7 +954,7 @@ static struct nullb_page *__null_lookup_page(struct nullb *nullb,
 	struct nullb_page *t_page;
 	struct radix_tree_root *root;
 
-	idx = sector >> PAGE_SECTORS_SHIFT;
+	idx = PAGE_IDX_FROM_SECT(sector);
 	sector_bit = (sector & SECTOR_MASK);
 
 	root = is_cache ? &nullb->dev->cache : &nullb->dev->data;
@@ -1125,7 +1130,7 @@ static int copy_to_nullb(struct nullb *nullb, struct page *source,
 		if (null_cache_active(nullb) && !is_fua)
 			null_make_cache_space(nullb, PAGE_SIZE);
 
-		offset = (sector & SECTOR_MASK) << SECTOR_SHIFT;
+		offset = SECT_OFFSET_IN_PAGE(sector);
 		t_page = null_insert_page(nullb, sector,
 			!null_cache_active(nullb) || is_fua);
 		if (!t_page)
@@ -1159,7 +1164,7 @@ static int copy_from_nullb(struct nullb *nullb, struct page *dest,
 	while (count < n) {
 		temp = min_t(size_t, nullb->dev->blocksize, n - count);
 
-		offset = (sector & SECTOR_MASK) << SECTOR_SHIFT;
+		offset = SECT_OFFSET_IN_PAGE(sector);
 		t_page = null_lookup_page(nullb, sector, false,
 			!null_cache_active(nullb));
 
-- 
2.29.0

[PATCH V2 3/7] null_blk: initialize cmd->bio in __alloc_cmd()

[Chaitanya Kulkarni]

The function __alloc_cmd() is responsible to allocate tag and
initializae the different members of the null_cmd structure e.g.
cmd->tag, cmd->error, and cmd->nq, Move only member bio that is initialized
from alloc_cmd() into __alloc_cmd().

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
---
 drivers/block/null_blk/main.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index fa47fab279c3..84f5ca0cc79c 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -743,7 +743,7 @@ static void free_cmd(struct nullb_cmd *cmd)
 
 static enum hrtimer_restart null_cmd_timer_expired(struct hrtimer *timer);
 
-static struct nullb_cmd *__alloc_cmd(struct nullb_queue *nq)
+static struct nullb_cmd *__alloc_cmd(struct nullb_queue *nq, struct bio *bio)
 {
 	struct nullb_cmd *cmd;
 	unsigned int tag;
@@ -754,6 +754,7 @@ static struct nullb_cmd *__alloc_cmd(struct nullb_queue *nq)
 		cmd->tag = tag;
 		cmd->error = BLK_STS_OK;
 		cmd->nq = nq;
+		cmd->bio = bio;
 		if (nq->dev->irqmode == NULL_IRQ_TIMER) {
 			hrtimer_init(&cmd->timer, CLOCK_MONOTONIC,
 				     HRTIMER_MODE_REL);
@@ -775,11 +776,9 @@ static struct nullb_cmd *alloc_cmd(struct nullb_queue *nq, struct bio *bio)
 		 * This avoids multiple return statements, multiple calls to
 		 * __alloc_cmd() and a fast path call to prepare_to_wait().
 		 */
-		cmd = __alloc_cmd(nq);
-		if (cmd) {
-			cmd->bio = bio;
+		cmd = __alloc_cmd(nq, bio);
+		if (cmd)
 			return cmd;
-		}
 		prepare_to_wait(&nq->wait, &wait, TASK_UNINTERRUPTIBLE);
 		io_schedule();
 		finish_wait(&nq->wait, &wait);
-- 
2.29.0

[PATCH V2 4/7] null_blk: don't use magic numbers in the code

[Chaitanya Kulkarni]

Insteasd of using the hardcoded value use meaningful macro for tag
available value of -1U in get_tag() and __alloc_cmd().

While at it return early on error to get rid of the extra indentation
in __alloc_cmd().

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
---
 drivers/block/null_blk/main.c | 30 +++++++++++++++---------------
 1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 84f5ca0cc79c..db849e6bc475 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -24,6 +24,8 @@
 #define TICKS_PER_SEC		50ULL
 #define TIMER_INTERVAL		(NSEC_PER_SEC / TICKS_PER_SEC)
 
+#define NULL_REQ_TAG_NOT_AVAILABLE (-1U)
+
 #ifdef CONFIG_BLK_DEV_NULL_BLK_FAULT_INJECTION
 static DECLARE_FAULT_ATTR(null_timeout_attr);
 static DECLARE_FAULT_ATTR(null_requeue_attr);
@@ -730,7 +732,7 @@ static unsigned int get_tag(struct nullb_queue *nq)
 	do {
 		tag = find_first_zero_bit(nq->tag_map, nq->queue_depth);
 		if (tag >= nq->queue_depth)
-			return -1U;
+			return NULL_REQ_TAG_NOT_AVAILABLE;
 	} while (test_and_set_bit_lock(tag, nq->tag_map));
 
 	return tag;
@@ -749,21 +751,19 @@ static struct nullb_cmd *__alloc_cmd(struct nullb_queue *nq, struct bio *bio)
 	unsigned int tag;
 
 	tag = get_tag(nq);
-	if (tag != -1U) {
-		cmd = &nq->cmds[tag];
-		cmd->tag = tag;
-		cmd->error = BLK_STS_OK;
-		cmd->nq = nq;
-		cmd->bio = bio;
-		if (nq->dev->irqmode == NULL_IRQ_TIMER) {
-			hrtimer_init(&cmd->timer, CLOCK_MONOTONIC,
-				     HRTIMER_MODE_REL);
-			cmd->timer.function = null_cmd_timer_expired;
-		}
-		return cmd;
+	if (tag == NULL_REQ_TAG_NOT_AVAILABLE)
+		return NULL;
+	cmd = &nq->cmds[tag];
+	cmd->tag = tag;
+	cmd->error = BLK_STS_OK;
+	cmd->nq = nq;
+	cmd->bio = bio;
+	if (nq->dev->irqmode == NULL_IRQ_TIMER) {
+		hrtimer_init(&cmd->timer, CLOCK_MONOTONIC,
+				HRTIMER_MODE_REL);
+		cmd->timer.function = null_cmd_timer_expired;
 	}
-
-	return NULL;
+	return cmd;
 }
 
 static struct nullb_cmd *alloc_cmd(struct nullb_queue *nq, struct bio *bio)
-- 
2.29.0

[PATCH V2 5/7] null_blk: remove extra space in switch condition

[Chaitanya Kulkarni]

The extra space in after switch condition does not follow kernel coding
standards, remove extra space in switch condition end_cmd().

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: Damien Le Moal <damien.lemoal@opensource.wdc.com>
---
 drivers/block/null_blk/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index db849e6bc475..96b8aca5abda 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -789,7 +789,7 @@ static void end_cmd(struct nullb_cmd *cmd)
 {
 	int queue_mode = cmd->nq->dev->queue_mode;
 
-	switch (queue_mode)  {
+	switch (queue_mode) {
 	case NULL_Q_MQ:
 		blk_mq_end_request(cmd->rq, cmd->error);
 		return;
-- 
2.29.0

[PATCH V2 6/7] null_blk: add param to set max disacrd sectors

[Chaitanya Kulkarni]

Instead of hardcoding value for the maximum discard sector to
UINT_MAX >> 9, allow user to set the value with newly added module
parameter max_discard_sectors.

To retain the backward compatibility make set default value to
UINT_MAX >> 9.

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/block/null_blk/main.c     | 10 +++++++++-
 drivers/block/null_blk/null_blk.h |  1 +
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 96b8aca5abda..2f787807cf63 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -216,6 +216,10 @@ static bool g_discard;
 module_param_named(discard, g_discard, bool, 0444);
 MODULE_PARM_DESC(discard, "Support discard operations (requires memory-backed null_blk device). Default: false");
 
+static unsigned int g_max_discard_sectors = UINT_MAX >> 9;
+module_param_named(max_discard_sectors, g_max_discard_sectors, uint, 0444);
+MODULE_PARM_DESC(max_discard_sectors, "Maximum size of a REQ_OP_DISCARD command (in 512B sectors).");
+
 static bool g_write_zeroes;
 module_param_named(write_zeroes, g_write_zeroes, bool, 0444);
 MODULE_PARM_DESC(write_zeroes, "Support write-zeores operations. Default: false");
@@ -420,6 +424,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL);
 NULLB_DEVICE_ATTR(queue_mode, uint, NULL);
 NULLB_DEVICE_ATTR(blocksize, uint, NULL);
 NULLB_DEVICE_ATTR(max_sectors, uint, NULL);
+NULLB_DEVICE_ATTR(max_discard_sectors, uint, NULL);
 NULLB_DEVICE_ATTR(irqmode, uint, NULL);
 NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL);
 NULLB_DEVICE_ATTR(index, uint, NULL);
@@ -544,6 +549,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
 	&nullb_device_attr_queue_mode,
 	&nullb_device_attr_blocksize,
 	&nullb_device_attr_max_sectors,
+	&nullb_device_attr_max_discard_sectors,
 	&nullb_device_attr_irqmode,
 	&nullb_device_attr_hw_queue_depth,
 	&nullb_device_attr_index,
@@ -686,6 +692,7 @@ static struct nullb_device *null_alloc_dev(void)
 	dev->queue_mode = g_queue_mode;
 	dev->blocksize = g_bs;
 	dev->max_sectors = g_max_sectors;
+	dev->max_discard_sectors = g_max_discard_sectors;
 	dev->irqmode = g_irqmode;
 	dev->hw_queue_depth = g_hw_queue_depth;
 	dev->blocking = g_blocking;
@@ -1850,7 +1857,8 @@ static void null_config_discard(struct nullb *nullb)
 	}
 
 	nullb->q->limits.discard_granularity = nullb->dev->blocksize;
-	blk_queue_max_discard_sectors(nullb->q, UINT_MAX >> 9);
+	blk_queue_max_discard_sectors(nullb->q,
+				      nullb->dev->max_discard_sectors);
 }
 
 static void null_config_write_zeroes(struct nullb *nullb)
diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h
index 2c0c9c29158f..09940211326d 100644
--- a/drivers/block/null_blk/null_blk.h
+++ b/drivers/block/null_blk/null_blk.h
@@ -102,6 +102,7 @@ struct nullb_device {
 	unsigned int queue_mode; /* block interface */
 	unsigned int blocksize; /* block size */
 	unsigned int max_sectors; /* Max sectors per command */
+	unsigned int max_discard_sectors; /* Max discard sectors per command */
 	unsigned int irqmode; /* IRQ completion handler */
 	unsigned int hw_queue_depth; /* queue depth */
 	unsigned int index; /* index of the disk, only valid with a disk */
-- 
2.29.0

[PATCH V2 7/7] null_blk: add param to set max write-zeroes sects

[Chaitanya Kulkarni]

Instead of hardcoding value for the maximum write-zeroes sector to
UINT_MAX >> 9, allow user to set the value with newly added module
parameter max_write_zeroes_sectors.

Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
---
 drivers/block/null_blk/main.c     | 10 +++++++++-
 drivers/block/null_blk/null_blk.h |  1 +
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index 2f787807cf63..8b7f42024f14 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -224,6 +224,10 @@ static bool g_write_zeroes;
 module_param_named(write_zeroes, g_write_zeroes, bool, 0444);
 MODULE_PARM_DESC(write_zeroes, "Support write-zeores operations. Default: false");
 
+static unsigned int g_max_write_zeroes_sectors = UINT_MAX >> 9;
+module_param_named(max_write_zeroes_sectors, g_max_write_zeroes_sectors, uint, 0444);
+MODULE_PARM_DESC(max_write_zeroes_sectors, "Maximum size of a REQ_OP_WRITE_ZEROES command (in 512B sectors).");
+
 static unsigned long g_cache_size;
 module_param_named(cache_size, g_cache_size, ulong, 0444);
 MODULE_PARM_DESC(mbps, "Cache size in MiB for memory-backed device. Default: 0 (none)");
@@ -425,6 +429,7 @@ NULLB_DEVICE_ATTR(queue_mode, uint, NULL);
 NULLB_DEVICE_ATTR(blocksize, uint, NULL);
 NULLB_DEVICE_ATTR(max_sectors, uint, NULL);
 NULLB_DEVICE_ATTR(max_discard_sectors, uint, NULL);
+NULLB_DEVICE_ATTR(max_write_zeroes_sectors, uint, NULL);
 NULLB_DEVICE_ATTR(irqmode, uint, NULL);
 NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL);
 NULLB_DEVICE_ATTR(index, uint, NULL);
@@ -550,6 +555,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
 	&nullb_device_attr_blocksize,
 	&nullb_device_attr_max_sectors,
 	&nullb_device_attr_max_discard_sectors,
+	&nullb_device_attr_max_write_zeroes_sectors,
 	&nullb_device_attr_irqmode,
 	&nullb_device_attr_hw_queue_depth,
 	&nullb_device_attr_index,
@@ -693,6 +699,7 @@ static struct nullb_device *null_alloc_dev(void)
 	dev->blocksize = g_bs;
 	dev->max_sectors = g_max_sectors;
 	dev->max_discard_sectors = g_max_discard_sectors;
+	dev->max_write_zeroes_sectors = g_max_write_zeroes_sectors;
 	dev->irqmode = g_irqmode;
 	dev->hw_queue_depth = g_hw_queue_depth;
 	dev->blocking = g_blocking;
@@ -1865,7 +1872,8 @@ static void null_config_write_zeroes(struct nullb *nullb)
 {
 	if (!nullb->dev->write_zeroes)
 		return;
-	blk_queue_max_write_zeroes_sectors(nullb->q, UINT_MAX >> 9);
+	blk_queue_max_write_zeroes_sectors(nullb->q,
+			nullb->dev->max_write_zeroes_sectors);
 }
 
 static const struct block_device_operations null_bio_ops = {
diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h
index 09940211326d..e692c2a7369e 100644
--- a/drivers/block/null_blk/null_blk.h
+++ b/drivers/block/null_blk/null_blk.h
@@ -103,6 +103,7 @@ struct nullb_device {
 	unsigned int blocksize; /* block size */
 	unsigned int max_sectors; /* Max sectors per command */
 	unsigned int max_discard_sectors; /* Max discard sectors per command */
+	unsigned int max_write_zeroes_sectors; /* Max write-zeroes sectors per command */
 	unsigned int irqmode; /* IRQ completion handler */
 	unsigned int hw_queue_depth; /* queue depth */
 	unsigned int index; /* index of the disk, only valid with a disk */
-- 
2.29.0

Re: [PATCH V2 0/7] null_blk: allow REQ_OP_WRITE_ZEROES and cleanup

[Chaitanya Kulkarni]

On 10/5/22 20:18, Chaitanya Kulkarni wrote:
> Hi,
> 
> In order to test the non-trivial I/O path in the block layer for
> REQ_OP_WRITE_ZEROES allow write-zeroes on null_blk so we can write
> testcases with and without non-memory backed mode followed by
> few cleanup patches.
> 
> Below is the test report with ext2/ext4 mkfs and a blktest
> waiting to get upstream for this patch-series.
> 
> -ck
> 

I applied these patches on the latest linux-block/for-next [1],
they apply seamlessly, is there anything missing in order to get
it merged ?

-ck

[1]
commit 6c944cdf52bb05a6572800a14edcea5a928a98a4 (origin/for-next)
Merge: 939f803c54b0 5626196a5ae0
Author: Jens Axboe <axboe@kernel.dk>
Date:   Mon Nov 14 12:58:09 2022 -0700

     Merge branch 'for-6.2/block' into for-next

     * for-6.2/block:
       md/raid1: stop mdx_raid1 thread when raid1 array run failed
       md/raid5: use bdev_write_cache instead of open coding it
       md: fix a crash in mempool_free
       md/raid0, raid10: Don't set discard sectors for request queue
       md/bitmap: Fix bitmap chunk size overflow issues
       md: introduce md_ro_state
       md: factor out __md_set_array_info()
       lib/raid6: drop RAID6_USE_EMPTY_ZERO_PAGE

Re: [PATCH V2 3/7] null_blk: initialize cmd->bio in __alloc_cmd()

[Bart Van Assche]

On 10/5/22 20:18, Chaitanya Kulkarni wrote:
> The function __alloc_cmd() is responsible to allocate tag and
> initializae the different members of the null_cmd structure e.g.

initializae -> initialize?

> cmd->tag, cmd->error, and cmd->nq, Move only member bio that is initialized
> from alloc_cmd() into __alloc_cmd().

Thanks,

Bart.

Re: [PATCH V2 4/7] null_blk: don't use magic numbers in the code

[Bart Van Assche]

On 10/5/22 20:18, Chaitanya Kulkarni wrote:
> Insteasd of using the hardcoded value use meaningful macro for tag

Insteasd -> Instead?

> available value of -1U in get_tag() and __alloc_cmd().
> 
> While at it return early on error to get rid of the extra indentation
> in __alloc_cmd().

Thanks,

Bart.

Re: [PATCH V2 4/7] null_blk: don't use magic numbers in the code

[Chaitanya Kulkarni]

On 11/14/22 16:40, Bart Van Assche wrote:
> On 10/5/22 20:18, Chaitanya Kulkarni wrote:
>> Insteasd of using the hardcoded value use meaningful macro for tag
> 
> Insteasd -> Instead?
> 
>> available value of -1U in get_tag() and __alloc_cmd().
>>
>> While at it return early on error to get rid of the extra indentation
>> in __alloc_cmd().
> 
> Thanks,
> 
> Bart.
> 

thanks for your comment Bart, I'll send out V3 with fixes..

-ck