[PATCH 3/6] sg: improve sg_write_mode option names

[PATCH 3/6] sg: improve sg_write_mode option names

[Vincent Fu]

There is a name collision for the sg_write_mode options for the WRITE AND
VERIFY and VERIFY commands. Deprecate the 'verify' option and use
'write_and_verify' instead. Do the same thing for 'same' and 'write_same' to
have a consistent naming scheme. The original option names are still supported
for backward compatibility but list them as deprecated.

Here are the new sg_write_mode options:

Option			SCSI command
write			WRITE (default)
write_and_verify	WRITE AND VERIFY
verify (deprecated)	WRITE AND VERIFY
write_same		WRITE SAME
same (deprecated)	WRITE SAME
write_same_ndob         WRITE SAME with NDOB flag set
verify_bytchk_00	VERIFY with BYTCHK set to 00
verify_bytchk_01	VERIFY with BYTCHK set to 01
verify_bytchk_11	VERIFY with BYTCHK set to 11

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO        |  8 ++++++--
 engines/sg.c | 14 ++++++++++++--
 fio.1        | 10 ++++++++--
 3 files changed, 26 insertions(+), 6 deletions(-)

diff --git a/HOWTO b/HOWTO
index 3c507d50..0f204afa 100644
--- a/HOWTO
+++ b/HOWTO
@@ -2486,11 +2486,13 @@ with the caveat that when used on the command line, they must come after the
 
 	**write**
 		This is the default where write opcodes are issued as usual.
-	**verify**
+	**write_and_verify**
 		Issue WRITE AND VERIFY commands. The BYTCHK bit is set to 0. This
 		directs the device to carry out a medium verification with no data
 		comparison. The writefua option is ignored with this selection.
-	**same**
+	**verify**
+		This option is deprecated. Use write_and_verify instead.
+	**write_same**
 		Issue WRITE SAME commands. This transfers a single block to the device
 		and writes this same block of data to a contiguous sequence of LBAs
 		beginning at the specified offset. fio's block size parameter specifies
@@ -2501,6 +2503,8 @@ with the caveat that when used on the command line, they must come after the
 		for each command but only the first 512 bytes will be used and
 		transferred to the device. The writefua option is ignored with this
 		selection.
+	**same**
+		This option is deprecated. Use write_same instead.
 	**write_same_ndob**
 		Issue WRITE SAME(16) commands as above but with the No Data Output
 		Buffer (NDOB) bit set. No data will be transferred to the device with
diff --git a/engines/sg.c b/engines/sg.c
index 3c4e986d..d15b438f 100644
--- a/engines/sg.c
+++ b/engines/sg.c
@@ -124,14 +124,24 @@ static struct fio_option options[] = {
 			    .oval = FIO_SG_WRITE,
 			    .help = "Issue standard SCSI WRITE commands",
 			  },
-			  { .ival = "verify",
+			  { .ival = "write_and_verify",
 			    .oval = FIO_SG_WRITE_VERIFY,
 			    .help = "Issue SCSI WRITE AND VERIFY commands",
 			  },
-			  { .ival = "same",
+			  { .ival = "verify",
+			    .oval = FIO_SG_WRITE_VERIFY,
+			    .help = "Issue SCSI WRITE AND VERIFY commands. This "
+				    "option is deprecated. Use write_and_verify instead.",
+			  },
+			  { .ival = "write_same",
 			    .oval = FIO_SG_WRITE_SAME,
 			    .help = "Issue SCSI WRITE SAME commands",
 			  },
+			  { .ival = "same",
+			    .oval = FIO_SG_WRITE_SAME,
+			    .help = "Issue SCSI WRITE SAME commands. This "
+				    "option is deprecated. Use write_same instead.",
+			  },
 			  { .ival = "write_same_ndob",
 			    .oval = FIO_SG_WRITE_SAME_NDOB,
 			    .help = "Issue SCSI WRITE SAME(16) commands with NDOB flag set",
diff --git a/fio.1 b/fio.1
index b1af6e8f..71f98135 100644
--- a/fio.1
+++ b/fio.1
@@ -2289,12 +2289,15 @@ values:
 .B write (default)
 Write opcodes are issued as usual
 .TP
-.B verify
+.B write_and_verify
 Issue WRITE AND VERIFY commands. The BYTCHK bit is set to 00b. This directs the
 device to carry out a medium verification with no data comparison for the data
 that was written. The writefua option is ignored with this selection.
 .TP
-.B same
+.B verify
+This option is deprecated. Use write_and_verify instead.
+.TP
+.B write_same
 Issue WRITE SAME commands. This transfers a single block to the device
 and writes this same block of data to a contiguous sequence of LBAs
 beginning at the specified offset. fio's block size parameter
@@ -2306,6 +2309,9 @@ generate 8k of data for each command butonly the first 512 bytes will
 be used and transferred to the device. The writefua option is ignored
 with this selection.
 .TP
+.B same
+This option is deprecated. Use write_same instead.
+.TP
 .B write_same_ndob
 Issue WRITE SAME(16) commands as above but with the No Data Output
 Buffer (NDOB) bit set. No data will be transferred to the device with
-- 
2.25.1

[PATCH 0/6] additional sg ionegine commands

[Vincent Fu]

Jens, please consider this patch series that adds to the sg ioengine
support for the SCSI commands VERIFY, WRITE SAME(16), and WRITE
STREAM(16).

Vincent


Vincent Fu (6):
  sg: add support for VERIFY command using write modes
  sg: add support for WRITE SAME(16) commands with NDOB flag set
  sg: improve sg_write_mode option names
  sg: add support for WRITE STREAM(16) commands
  sg: allow fio to open and close streams for WRITE STREAM(16) commands
  docs: documentation for sg WRITE STREAM(16)

 HOWTO                           |  36 ++++++-
 engines/sg.c                    | 181 ++++++++++++++++++++++++++++++--
 examples/sg_verify-fail.fio     |  48 +++++++++
 examples/sg_verify.fio          |  57 ++++++++++
 examples/sg_write_same_ndob.fio |  44 ++++++++
 fio.1                           |  47 ++++++++-
 6 files changed, 397 insertions(+), 16 deletions(-)
 create mode 100644 examples/sg_verify-fail.fio
 create mode 100644 examples/sg_verify.fio
 create mode 100644 examples/sg_write_same_ndob.fio

-- 
2.25.1

[PATCH 1/6] sg: add support for VERIFY command using write modes

[Vincent Fu]

fio does not have an explicit verify data direction and creating a new data
direction just for SCSI VERIFY commands probably is not worthwhile. The format
of SCSI VERIFY commands matches that of write operations since VERIFY commands
can include data transfer to the device. So it seems reasonable to have VERIFY
commands be accounted for as write operations by fio.

Use the sg_write_mode option to support SCSI VERIFY commands with different
BYTCHK values.

BYTCHK	Description
00	No data is transferred to the device; device data is checked
01	Device data is compared with data transferred to device
11	Same as 01 except that only one sector of data is transferred to the
device and each sector specified in the verification extent is compared against
this transferred data.

Also update documentation and add a couple example jobs files.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO                       | 14 +++++++++
 engines/sg.c                | 41 ++++++++++++++++++++++++--
 examples/sg_verify-fail.fio | 48 +++++++++++++++++++++++++++++++
 examples/sg_verify.fio      | 57 +++++++++++++++++++++++++++++++++++++
 fio.1                       | 24 +++++++++++++---
 5 files changed, 178 insertions(+), 6 deletions(-)
 create mode 100644 examples/sg_verify-fail.fio
 create mode 100644 examples/sg_verify.fio

diff --git a/HOWTO b/HOWTO
index 196bca6c..c641a2a5 100644
--- a/HOWTO
+++ b/HOWTO
@@ -2501,6 +2501,20 @@ with the caveat that when used on the command line, they must come after the
 		for each command but only the first 512 bytes will be used and
 		transferred to the device. The writefua option is ignored with this
 		selection.
+	**verify_bytchk_00**
+		Issue VERIFY commands with BYTCHK set to 00. This directs the
+		device to carry out a medium verification with no data comparison.
+	**verify_bytchk_01**
+		Issue VERIFY commands with BYTCHK set to 01. This directs the device to
+		compare the data on the device with the data transferred to the device.
+	**verify_bytchk_11**
+		Issue VERIFY commands with BYTCHK set to 11. This transfers a
+		single block to the device and compares the contents of this block with the
+		data on the device beginning at the specified offset. fio's block size
+		parameter specifies the total amount of data compared with this command.
+		However, only one block (sector) worth of data is transferred to the device.
+		This is similar to the WRITE SAME command except that data is compared instead
+		of written.
 
 .. option:: hipri : [sg]
 
diff --git a/engines/sg.c b/engines/sg.c
index 1c019384..0c525fae 100644
--- a/engines/sg.c
+++ b/engines/sg.c
@@ -66,8 +66,11 @@
 
 enum {
 	FIO_SG_WRITE		= 1,
-	FIO_SG_WRITE_VERIFY	= 2,
-	FIO_SG_WRITE_SAME	= 3
+	FIO_SG_WRITE_VERIFY,
+	FIO_SG_WRITE_SAME,
+	FIO_SG_VERIFY_BYTCHK_00,
+	FIO_SG_VERIFY_BYTCHK_01,
+	FIO_SG_VERIFY_BYTCHK_11,
 };
 
 struct sg_options {
@@ -128,6 +131,18 @@ static struct fio_option options[] = {
 			    .oval = FIO_SG_WRITE_SAME,
 			    .help = "Issue SCSI WRITE SAME commands",
 			  },
+			  { .ival = "verify_bytchk_00",
+			    .oval = FIO_SG_VERIFY_BYTCHK_00,
+			    .help = "Issue SCSI VERIFY commands with BYTCHK set to 00",
+			  },
+			  { .ival = "verify_bytchk_01",
+			    .oval = FIO_SG_VERIFY_BYTCHK_01,
+			    .help = "Issue SCSI VERIFY commands with BYTCHK set to 01",
+			  },
+			  { .ival = "verify_bytchk_11",
+			    .oval = FIO_SG_VERIFY_BYTCHK_11,
+			    .help = "Issue SCSI VERIFY commands with BYTCHK set to 11",
+			  },
 		},
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_SG,
@@ -576,6 +591,28 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			else
 				hdr->cmdp[0] = 0x93; // write same(16)
 			break;
+		case FIO_SG_VERIFY_BYTCHK_00:
+			if (lba < MAX_10B_LBA)
+				hdr->cmdp[0] = 0x2f; // VERIFY(10)
+			else
+				hdr->cmdp[0] = 0x8f; // VERIFY(16)
+			hdr->dxfer_len = 0;
+			break;
+		case FIO_SG_VERIFY_BYTCHK_01:
+			if (lba < MAX_10B_LBA)
+				hdr->cmdp[0] = 0x2f; // VERIFY(10)
+			else
+				hdr->cmdp[0] = 0x8f; // VERIFY(16)
+			hdr->cmdp[1] |= 0x02;		// BYTCHK = 01b
+			break;
+		case FIO_SG_VERIFY_BYTCHK_11:
+			if (lba < MAX_10B_LBA)
+				hdr->cmdp[0] = 0x2f; // VERIFY(10)
+			else
+				hdr->cmdp[0] = 0x8f; // VERIFY(16)
+			hdr->cmdp[1] |= 0x06;		// BYTCHK = 11b
+			hdr->dxfer_len = sd->bs;
+			break;
 		};
 
 		fio_sgio_rw_lba(hdr, lba, nr_blocks);
diff --git a/examples/sg_verify-fail.fio b/examples/sg_verify-fail.fio
new file mode 100644
index 00000000..64feece3
--- /dev/null
+++ b/examples/sg_verify-fail.fio
@@ -0,0 +1,48 @@
+#
+# **********************************
+# * !!THIS IS A DESTRUCTIVE TEST!! *
+# * IF NOT CHANGED THIS TEST WILL  *
+# * DESTROY DATA ON /dev/sdb       *
+# **********************************
+#
+# Test SCSI VERIFY commands issued via the sg ioengine
+# The jobs with fail in the name should produce errors
+#
+# job			description
+# precon		precondition the device by writing with a known
+#			pattern
+# verify01		verify each block one at a time by comparing to known
+#			pattern
+# verify01-fail		verifying one too many blocks should produce a failure
+# verify11-one_ios	verify all 20 blocks by sending only 512 bytes
+# verify11-fail		verifying beyond the preconditioned region should
+#			produce a failure
+
+[global]
+filename=/dev/sdb
+buffer_pattern=0x01
+ioengine=sg
+rw=write
+bs=512
+number_ios=20
+stonewall
+
+[precon]
+
+[verify01]
+sg_write_mode=verify_bytchk_01
+number_ios=20
+
+[verify01-fail]
+sg_write_mode=verify_bytchk_01
+number_ios=21
+
+[verify11-one_ios]
+sg_write_mode=verify_bytchk_11
+number_ios=1
+bs=10240
+
+[verify11-fail]
+sg_write_mode=verify_bytchk_11
+number_ios=1
+bs=10752
diff --git a/examples/sg_verify.fio b/examples/sg_verify.fio
new file mode 100644
index 00000000..6db0dd0a
--- /dev/null
+++ b/examples/sg_verify.fio
@@ -0,0 +1,57 @@
+#
+# **********************************
+# * !!THIS IS A DESTRUCTIVE TEST!! *
+# * IF NOT CHANGED THIS TEST WILL  *
+# * DESTROY DATA ON /dev/sdb       *
+# **********************************
+#
+# Test SCSI VERIFY commands issued via the sg ioengine
+# All of the jobs below should complete without error
+#
+# job			description
+# precon		precondition the device by writing with a known
+#			pattern
+# verify00		verify written data on medium only
+# verify01		verify each block one at a time by comparing to known
+#			pattern
+# verify01-two_ios	verify same data but with only two VERIFY operations
+# verify11		verify each block one at a time
+# verify11-five_ios	verify data with five IOs, four blocks at a time,
+#			sending 512 bytes for each IO
+# verify11-one_ios	verify all 20 blocks by sending only 512 bytes
+#
+
+[global]
+filename=/dev/sdb
+buffer_pattern=0x01
+ioengine=sg
+rw=write
+bs=512
+number_ios=20
+stonewall
+
+[precon]
+
+[verify00]
+sg_write_mode=verify_bytchk_00
+
+[verify01]
+sg_write_mode=verify_bytchk_01
+
+[verify01-two_ios]
+sg_write_mode=verify_bytchk_01
+bs=5120
+number_ios=2
+
+[verify11]
+sg_write_mode=verify_bytchk_11
+
+[verify11-five_ios]
+sg_write_mode=verify_bytchk_11
+bs=2048
+number_ios=5
+
+[verify11-one_ios]
+sg_write_mode=verify_bytchk_11
+bs=10240
+number_ios=1
diff --git a/fio.1 b/fio.1
index e3c3feae..2d69615b 100644
--- a/fio.1
+++ b/fio.1
@@ -2281,7 +2281,7 @@ With writefua option set to 1, write operations include the force
 unit access (fua) flag. Default: 0.
 .TP
 .BI (sg)sg_write_mode \fR=\fPstr
-Specify the type of write commands to issue. This option can take three
+Specify the type of write commands to issue. This option can take multiple
 values:
 .RS
 .RS
@@ -2290,9 +2290,9 @@ values:
 Write opcodes are issued as usual
 .TP
 .B verify
-Issue WRITE AND VERIFY commands. The BYTCHK bit is set to 0. This
-directs the device to carry out a medium verification with no data
-comparison. The writefua option is ignored with this selection.
+Issue WRITE AND VERIFY commands. The BYTCHK bit is set to 00b. This directs the
+device to carry out a medium verification with no data comparison for the data
+that was written. The writefua option is ignored with this selection.
 .TP
 .B same
 Issue WRITE SAME commands. This transfers a single block to the device
@@ -2305,6 +2305,22 @@ blocksize=8k will write 16 sectors with each command. fio will still
 generate 8k of data for each command butonly the first 512 bytes will
 be used and transferred to the device. The writefua option is ignored
 with this selection.
+.TP
+.B verify_bytchk_00
+Issue VERIFY commands with BYTCHK set to 00. This directs the device to carry
+out a medium verification with no data comparison.
+.TP
+.B verify_bytchk_01
+Issue VERIFY commands with BYTCHK set to 01. This directs the device to
+compare the data on the device with the data transferred to the device.
+.TP
+.B verify_bytchk_11
+Issue VERIFY commands with BYTCHK set to 11. This transfers a single block to
+the device and compares the contents of this block with the data on the device
+beginning at the specified offset. fio's block size parameter specifies the
+total amount of data compared with this command. However, only one block
+(sector) worth of data is transferred to the device. This is similar to the
+WRITE SAME command except that data is compared instead of written.
 .RE
 .RE
 .TP
-- 
2.25.1

[PATCH 5/6] sg: allow fio to open and close streams for WRITE STREAM(16) commands

[Vincent Fu]

If --stream_id=0 then fio will open a stream for WRITE STREAM(16) commands and
close the stream when the device file is closed.

Example:
./fio --name=test --filename=/dev/sdb --ioengine=sg --number_ios=1 --debug=file,io --sg_write_mode=write_stream --rw=randwrite
fio: set debug option file
fio: set debug option io
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sg, iodepth=1
fio-3.27
Starting 1 process
file     1072297 setup files
file     1072297 get file size for 0x7f0306fa5110/0//dev/sdb
file     1072307 trying file /dev/sdb 290
file     1072307 fd open /dev/sdb
file     1072307 file not found in hash /dev/sdb
file     1072307 sgio_stream_control: opened stream 1
file     1072307 get file /dev/sdb, ref=0
io       1072307 drop page cache /dev/sdb
file     1072307 goodf=1, badf=2, ff=2b1
file     1072307 get_next_file_rr: 0x7f0306fa5110
file     1072307 get_next_file: 0x7f0306fa5110 [/dev/sdb]
file     1072307 get file /dev/sdb, ref=1
io       1072307 fill: io_u 0xb55700: off=0x35ef554000,len=0x1000,ddir=1,file=/dev/sdb
io       1072307 prep: io_u 0xb55700: off=0x35ef554000,len=0x1000,ddir=1,file=/dev/sdb
io       1072307 prep: io_u 0xb55700: ret=0
io       1072307 queue: io_u 0xb55700: off=0x35ef554000,len=0x1000,ddir=1,file=/dev/sdb
io       1072307 complete: io_u 0xb55700: off=0x35ef554000,len=0x1000,ddir=1,file=/dev/sdb
file     1072307 put file /dev/sdb, ref=2
file     1072307 close files
file     1072307 put file /dev/sdb, ref=1
file     1072307 sgio_stream_control: closed stream 1
file     1072307 fd close /dev/sdb
io       1072307 close ioengine sg
io       1072307 free ioengine sg

test: (groupid=0, jobs=1): err= 0: pid=1072307: Mon Aug 16 14:25:45 2021
  write: IOPS=200, BW=800KiB/s (819kB/s)(4096B/5msec); 0 zone resets
    clat (nsec): min=93339, max=93339, avg=93339.00, stdev= 0.00
     lat (nsec): min=96201, max=96201, avg=96201.00, stdev= 0.00
    clat percentiles (nsec):
     |  1.00th=[93696],  5.00th=[93696], 10.00th=[93696], 20.00th=[93696],
     | 30.00th=[93696], 40.00th=[93696], 50.00th=[93696], 60.00th=[93696],
     | 70.00th=[93696], 80.00th=[93696], 90.00th=[93696], 95.00th=[93696],
     | 99.00th=[93696], 99.50th=[93696], 99.90th=[93696], 99.95th=[93696],
     | 99.99th=[93696]
  lat (usec)   : 100=100.00%
  cpu          : usr=100.00%, sys=0.00%, ctx=2, majf=0, minf=20
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,1,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=800KiB/s (819kB/s), 800KiB/s-800KiB/s (819kB/s-819kB/s), io=4096B (4096B), run=5-5msec

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 engines/sg.c | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 82 insertions(+), 2 deletions(-)

diff --git a/engines/sg.c b/engines/sg.c
index b51edb07..72ee07ba 100644
--- a/engines/sg.c
+++ b/engines/sg.c
@@ -217,6 +217,11 @@ struct sgio_data {
 #endif
 };
 
+static inline uint16_t sgio_get_be16(uint8_t *buf)
+{
+	return be16_to_cpu(*((uint16_t *) buf));
+}
+
 static inline uint32_t sgio_get_be32(uint8_t *buf)
 {
 	return be32_to_cpu(*((uint32_t *) buf));
@@ -632,7 +637,7 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			if (o->writefua)
 				hdr->cmdp[1] |= 0x08;
 			sgio_set_be64(lba, &hdr->cmdp[2]);
-			sgio_set_be16(o->stream_id, &hdr->cmdp[10]);
+			sgio_set_be16((uint16_t) io_u->file->engine_pos, &hdr->cmdp[10]);
 			sgio_set_be16((uint16_t) nr_blocks, &hdr->cmdp[12]);
 			break;
 		case FIO_SG_VERIFY_BYTCHK_00:
@@ -1053,9 +1058,60 @@ static int fio_sgio_type_check(struct thread_data *td, struct fio_file *f)
 	return 0;
 }
 
+static int fio_sgio_stream_control(struct fio_file *f, bool open_stream, uint16_t *stream_id)
+{
+	struct sg_io_hdr hdr;
+	unsigned char cmd[16];
+	unsigned char sb[64];
+	unsigned char buf[8];
+	int ret;
+
+	memset(&hdr, 0, sizeof(hdr));
+	memset(cmd, 0, sizeof(cmd));
+	memset(sb, 0, sizeof(sb));
+	memset(buf, 0, sizeof(buf));
+
+	hdr.interface_id = 'S';
+	hdr.cmdp = cmd;
+	hdr.cmd_len = 16;
+	hdr.sbp = sb;
+	hdr.mx_sb_len = sizeof(sb);
+	hdr.timeout = SCSI_TIMEOUT_MS;
+	hdr.cmdp[0] = 0x9e;
+	hdr.dxfer_direction = SG_DXFER_FROM_DEV;
+	hdr.dxferp = buf;
+	hdr.dxfer_len = sizeof(buf);
+	sgio_set_be32(sizeof(buf), &hdr.cmdp[10]);
+
+	if (open_stream)
+		hdr.cmdp[1] = 0x34;
+	else {
+		hdr.cmdp[1] = 0x54;
+		sgio_set_be16(*stream_id, &hdr.cmdp[4]);
+	}
+
+	ret = ioctl(f->fd, SG_IO, &hdr);
+
+	if (ret < 0)
+		return ret;
+
+	if (hdr.info & SG_INFO_CHECK)
+		return 1;
+
+	if (open_stream) {
+		*stream_id = sgio_get_be16(&buf[4]);
+		dprint(FD_FILE, "sgio_stream_control: opened stream %u\n", (unsigned int) *stream_id);
+		assert(*stream_id != 0);
+	} else
+		dprint(FD_FILE, "sgio_stream_control: closed stream %u\n", (unsigned int) *stream_id);
+
+	return 0;
+}
+
 static int fio_sgio_open(struct thread_data *td, struct fio_file *f)
 {
 	struct sgio_data *sd = td->io_ops_data;
+	struct sg_options *o = td->eo;
 	int ret;
 
 	ret = generic_open_file(td, f);
@@ -1067,9 +1123,33 @@ static int fio_sgio_open(struct thread_data *td, struct fio_file *f)
 		return ret;
 	}
 
+	if (o->write_mode == FIO_SG_WRITE_STREAM) {
+		if (o->stream_id)
+			f->engine_pos = o->stream_id;
+		else {
+			ret = fio_sgio_stream_control(f, true, (uint16_t *) &f->engine_pos);
+			if (ret)
+				return ret;
+		}
+	}
+
 	return 0;
 }
 
+int fio_sgio_close(struct thread_data *td, struct fio_file *f)
+{
+	struct sg_options *o = td->eo;
+	int ret;
+
+	if (!o->stream_id && o->write_mode == FIO_SG_WRITE_STREAM) {
+		ret = fio_sgio_stream_control(f, false, (uint16_t *) &f->engine_pos);
+		if (ret)
+			return ret;
+	}
+
+	return generic_close_file(td, f);
+}
+
 /*
  * Build an error string with details about the driver, host or scsi
  * error contained in the sg header Caller will use as necessary.
@@ -1344,7 +1424,7 @@ static struct ioengine_ops ioengine = {
 	.event		= fio_sgio_event,
 	.cleanup	= fio_sgio_cleanup,
 	.open_file	= fio_sgio_open,
-	.close_file	= generic_close_file,
+	.close_file	= fio_sgio_close,
 	.get_file_size	= fio_sgio_get_file_size,
 	.flags		= FIO_SYNCIO | FIO_RAWIO,
 	.options	= options,
-- 
2.25.1

[PATCH 2/6] sg: add support for WRITE SAME(16) commands with NDOB flag set

[Vincent Fu]

Add the sg_write_mode option write_same_ndob to issue WRITE SAME(16) commands
with the no data output buffer flag set. This flag is not supported for WRITE
SAME(10). So all commands with this option will be WRITE SAME(16).

Also include an example job file.

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO                           |  5 ++++
 engines/sg.c                    | 19 +++++++++++---
 examples/sg_write_same_ndob.fio | 44 +++++++++++++++++++++++++++++++++
 fio.1                           |  6 +++++
 4 files changed, 70 insertions(+), 4 deletions(-)
 create mode 100644 examples/sg_write_same_ndob.fio

diff --git a/HOWTO b/HOWTO
index c641a2a5..3c507d50 100644
--- a/HOWTO
+++ b/HOWTO
@@ -2501,6 +2501,11 @@ with the caveat that when used on the command line, they must come after the
 		for each command but only the first 512 bytes will be used and
 		transferred to the device. The writefua option is ignored with this
 		selection.
+	**write_same_ndob**
+		Issue WRITE SAME(16) commands as above but with the No Data Output
+		Buffer (NDOB) bit set. No data will be transferred to the device with
+		this bit set. Data written will be a pre-determined pattern such as
+		all zeroes.
 	**verify_bytchk_00**
 		Issue VERIFY commands with BYTCHK set to 00. This directs the
 		device to carry out a medium verification with no data comparison.
diff --git a/engines/sg.c b/engines/sg.c
index 0c525fae..3c4e986d 100644
--- a/engines/sg.c
+++ b/engines/sg.c
@@ -68,6 +68,7 @@ enum {
 	FIO_SG_WRITE		= 1,
 	FIO_SG_WRITE_VERIFY,
 	FIO_SG_WRITE_SAME,
+	FIO_SG_WRITE_SAME_NDOB,
 	FIO_SG_VERIFY_BYTCHK_00,
 	FIO_SG_VERIFY_BYTCHK_01,
 	FIO_SG_VERIFY_BYTCHK_11,
@@ -131,6 +132,10 @@ static struct fio_option options[] = {
 			    .oval = FIO_SG_WRITE_SAME,
 			    .help = "Issue SCSI WRITE SAME commands",
 			  },
+			  { .ival = "write_same_ndob",
+			    .oval = FIO_SG_WRITE_SAME_NDOB,
+			    .help = "Issue SCSI WRITE SAME(16) commands with NDOB flag set",
+			  },
 			  { .ival = "verify_bytchk_00",
 			    .oval = FIO_SG_VERIFY_BYTCHK_00,
 			    .help = "Issue SCSI VERIFY commands with BYTCHK set to 00",
@@ -517,9 +522,9 @@ static enum fio_q_status fio_sgio_doio(struct thread_data *td,
 }
 
 static void fio_sgio_rw_lba(struct sg_io_hdr *hdr, unsigned long long lba,
-			    unsigned long long nr_blocks)
+			    unsigned long long nr_blocks, bool override16)
 {
-	if (lba < MAX_10B_LBA) {
+	if (lba < MAX_10B_LBA && !override16) {
 		sgio_set_be32((uint32_t) lba, &hdr->cmdp[2]);
 		sgio_set_be16((uint16_t) nr_blocks, &hdr->cmdp[7]);
 	} else {
@@ -560,7 +565,7 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 		if (o->readfua)
 			hdr->cmdp[1] |= 0x08;
 
-		fio_sgio_rw_lba(hdr, lba, nr_blocks);
+		fio_sgio_rw_lba(hdr, lba, nr_blocks, false);
 
 	} else if (io_u->ddir == DDIR_WRITE) {
 		sgio_hdr_init(sd, hdr, io_u, 1);
@@ -591,6 +596,11 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			else
 				hdr->cmdp[0] = 0x93; // write same(16)
 			break;
+		case FIO_SG_WRITE_SAME_NDOB:
+			hdr->cmdp[0] = 0x93; // write same(16)
+			hdr->cmdp[1] |= 0x1; // no data output buffer
+			hdr->dxfer_len = 0;
+			break;
 		case FIO_SG_VERIFY_BYTCHK_00:
 			if (lba < MAX_10B_LBA)
 				hdr->cmdp[0] = 0x2f; // VERIFY(10)
@@ -615,7 +625,8 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			break;
 		};
 
-		fio_sgio_rw_lba(hdr, lba, nr_blocks);
+		fio_sgio_rw_lba(hdr, lba, nr_blocks,
+			o->write_mode == FIO_SG_WRITE_SAME_NDOB);
 
 	} else if (io_u->ddir == DDIR_TRIM) {
 		struct sgio_trim *st;
diff --git a/examples/sg_write_same_ndob.fio b/examples/sg_write_same_ndob.fio
new file mode 100644
index 00000000..fb047319
--- /dev/null
+++ b/examples/sg_write_same_ndob.fio
@@ -0,0 +1,44 @@
+#
+# **********************************
+# * !!THIS IS A DESTRUCTIVE TEST!! *
+# * IF NOT CHANGED THIS TEST WILL  *
+# * DESTROY DATA ON /dev/sdb       *
+# **********************************
+#
+# Test WRITE SAME commands with the NDOB flag set
+# issued via the sg ioengine
+# All of the jobs below should complete without error
+# except the last one
+#
+# job			description
+# precon		Precondition the device by writing 20 blocks with a
+# 			known pattern
+# write_same_ndob	Write 19 sectors of all zeroes with the NDOB flag set
+# verify-pass		Verify 19 blocks of all zeroes
+# verify-fail		Verify 20 blocks of all zeroes. This should fail.
+#
+
+[global]
+filename=/dev/sdb
+buffer_pattern=0x01
+ioengine=sg
+rw=write
+bs=512
+stonewall
+
+[precon]
+number_ios=20
+
+[write_same_ndob]
+sg_write_mode=write_same_ndob
+number_ios=19
+
+[verify-pass]
+sg_write_mode=verify_bytchk_01
+buffer_pattern=0x00
+number_ios=19
+
+[verify-fail]
+sg_write_mode=verify_bytchk_01
+buffer_pattern=0x00
+number_ios=20
diff --git a/fio.1 b/fio.1
index 2d69615b..b1af6e8f 100644
--- a/fio.1
+++ b/fio.1
@@ -2306,6 +2306,12 @@ generate 8k of data for each command butonly the first 512 bytes will
 be used and transferred to the device. The writefua option is ignored
 with this selection.
 .TP
+.B write_same_ndob
+Issue WRITE SAME(16) commands as above but with the No Data Output
+Buffer (NDOB) bit set. No data will be transferred to the device with
+this bit set. Data written will be a pre-determined pattern such as
+all zeroes.
+.TP
 .B verify_bytchk_00
 Issue VERIFY commands with BYTCHK set to 00. This directs the device to carry
 out a medium verification with no data comparison.
-- 
2.25.1

[PATCH 6/6] docs: documentation for sg WRITE STREAM(16)

[Vincent Fu]

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 HOWTO | 9 +++++++++
 fio.1 | 9 +++++++++
 2 files changed, 18 insertions(+)

diff --git a/HOWTO b/HOWTO
index 0f204afa..ccf2442d 100644
--- a/HOWTO
+++ b/HOWTO
@@ -2510,6 +2510,9 @@ with the caveat that when used on the command line, they must come after the
 		Buffer (NDOB) bit set. No data will be transferred to the device with
 		this bit set. Data written will be a pre-determined pattern such as
 		all zeroes.
+	**write_stream**
+		Issue WRITE STREAM(16) commands. Use the **stream_id** option to specify
+		the stream identifier.
 	**verify_bytchk_00**
 		Issue VERIFY commands with BYTCHK set to 00. This directs the
 		device to carry out a medium verification with no data comparison.
@@ -2525,6 +2528,12 @@ with the caveat that when used on the command line, they must come after the
 		This is similar to the WRITE SAME command except that data is compared instead
 		of written.
 
+.. option:: stream_id=int : [sg]
+
+	Set the stream identifier for WRITE STREAM commands. If this is set to 0 (which is not
+	a valid stream identifier) fio will open a stream and then close it when done. Default
+	is 0.
+
 .. option:: hipri : [sg]
 
 	If this option is set, fio will attempt to use polled IO completions.
diff --git a/fio.1 b/fio.1
index 71f98135..823c925d 100644
--- a/fio.1
+++ b/fio.1
@@ -2318,6 +2318,10 @@ Buffer (NDOB) bit set. No data will be transferred to the device with
 this bit set. Data written will be a pre-determined pattern such as
 all zeroes.
 .TP
+.B write_stream
+Issue WRITE STREAM(16) commands. Use the stream_id option to specify
+the stream identifier.
+.TP
 .B verify_bytchk_00
 Issue VERIFY commands with BYTCHK set to 00. This directs the device to carry
 out a medium verification with no data comparison.
@@ -2336,6 +2340,11 @@ WRITE SAME command except that data is compared instead of written.
 .RE
 .RE
 .TP
+.BI (sg)stream_id \fR=\fPint
+Set the stream identifier for WRITE STREAM commands. If this is set to 0 (which is not
+a valid stream identifier) fio will open a stream and then close it when done. Default
+is 0.
+.TP
 .BI (nbd)uri \fR=\fPstr
 Specify the NBD URI of the server to test.
 The string is a standard NBD URI (see
-- 
2.25.1

[PATCH 4/6] sg: add support for WRITE STREAM(16) commands

[Vincent Fu]

Add the "write_stream" option to sg_write_mode to send WRITE STREAM(16)
commands. Use the new stream_id option to set the stream identifier.

Example:

sg_stream_ctl -o /dev/sdb
Assigned stream id: 1
./fio --name=test --filename=/dev/sdb --ioengine=sg --sg_write_mode=write_stream --stream_id=1 --rw=randwrite --time_based --runtime=10s
...
sg_stream_ctl -c --id=1 /dev/sdb

Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
---
 engines/sg.c | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/engines/sg.c b/engines/sg.c
index d15b438f..b51edb07 100644
--- a/engines/sg.c
+++ b/engines/sg.c
@@ -69,6 +69,7 @@ enum {
 	FIO_SG_WRITE_VERIFY,
 	FIO_SG_WRITE_SAME,
 	FIO_SG_WRITE_SAME_NDOB,
+	FIO_SG_WRITE_STREAM,
 	FIO_SG_VERIFY_BYTCHK_00,
 	FIO_SG_VERIFY_BYTCHK_01,
 	FIO_SG_VERIFY_BYTCHK_11,
@@ -80,6 +81,7 @@ struct sg_options {
 	unsigned int readfua;
 	unsigned int writefua;
 	unsigned int write_mode;
+	uint16_t stream_id;
 };
 
 static struct fio_option options[] = {
@@ -158,10 +160,24 @@ static struct fio_option options[] = {
 			    .oval = FIO_SG_VERIFY_BYTCHK_11,
 			    .help = "Issue SCSI VERIFY commands with BYTCHK set to 11",
 			  },
+			  { .ival = "write_stream",
+			    .oval = FIO_SG_WRITE_STREAM,
+			    .help = "Issue SCSI WRITE STREAM(16) commands",
+			  },
 		},
 		.category = FIO_OPT_C_ENGINE,
 		.group	= FIO_OPT_G_SG,
 	},
+	{
+		.name	= "stream_id",
+		.lname	= "stream id for WRITE STREAM(16) commands",
+		.type	= FIO_OPT_INT,
+		.off1	= offsetof(struct sg_options, stream_id),
+		.help	= "Stream ID for WRITE STREAM(16) commands",
+		.def	= "0",
+		.category = FIO_OPT_C_ENGINE,
+		.group	= FIO_OPT_G_SG,
+	},
 	{
 		.name	= NULL,
 	},
@@ -611,6 +627,14 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			hdr->cmdp[1] |= 0x1; // no data output buffer
 			hdr->dxfer_len = 0;
 			break;
+		case FIO_SG_WRITE_STREAM:
+			hdr->cmdp[0] = 0x9a; // write stream (16)
+			if (o->writefua)
+				hdr->cmdp[1] |= 0x08;
+			sgio_set_be64(lba, &hdr->cmdp[2]);
+			sgio_set_be16(o->stream_id, &hdr->cmdp[10]);
+			sgio_set_be16((uint16_t) nr_blocks, &hdr->cmdp[12]);
+			break;
 		case FIO_SG_VERIFY_BYTCHK_00:
 			if (lba < MAX_10B_LBA)
 				hdr->cmdp[0] = 0x2f; // VERIFY(10)
@@ -635,8 +659,9 @@ static int fio_sgio_prep(struct thread_data *td, struct io_u *io_u)
 			break;
 		};
 
-		fio_sgio_rw_lba(hdr, lba, nr_blocks,
-			o->write_mode == FIO_SG_WRITE_SAME_NDOB);
+		if (o->write_mode != FIO_SG_WRITE_STREAM)
+			fio_sgio_rw_lba(hdr, lba, nr_blocks,
+				o->write_mode == FIO_SG_WRITE_SAME_NDOB);
 
 	} else if (io_u->ddir == DDIR_TRIM) {
 		struct sgio_trim *st;
-- 
2.25.1

RE: [PATCH 0/6] additional sg ionegine commands

[Vincent Fu]

Jens, please let me know if you have any feedback about this patch series.

Vincent

> -----Original Message-----
> From: Vincent Fu
> Sent: Monday, November 15, 2021 3:07 PM
> To: axboe@kernel.dk; fio@vger.kernel.org
> Cc: Vincent Fu <vincent.fu@samsung.com>
> Subject: [PATCH 0/6] additional sg ionegine commands
> 
> Jens, please consider this patch series that adds to the sg ioengine support
> for the SCSI commands VERIFY, WRITE SAME(16), and WRITE STREAM(16).
> 
> Vincent
> 
> 
> Vincent Fu (6):
>   sg: add support for VERIFY command using write modes
>   sg: add support for WRITE SAME(16) commands with NDOB flag set
>   sg: improve sg_write_mode option names
>   sg: add support for WRITE STREAM(16) commands
>   sg: allow fio to open and close streams for WRITE STREAM(16) commands
>   docs: documentation for sg WRITE STREAM(16)
> 
>  HOWTO                           |  36 ++++++-
>  engines/sg.c                    | 181 ++++++++++++++++++++++++++++++--
>  examples/sg_verify-fail.fio     |  48 +++++++++
>  examples/sg_verify.fio          |  57 ++++++++++
>  examples/sg_write_same_ndob.fio |  44 ++++++++
>  fio.1                           |  47 ++++++++-
>  6 files changed, 397 insertions(+), 16 deletions(-)  create mode 100644
> examples/sg_verify-fail.fio  create mode 100644 examples/sg_verify.fio
> create mode 100644 examples/sg_write_same_ndob.fio
> 
> --
> 2.25.1

RE: [PATCH 0/6] additional sg ionegine commands

[Vincent Fu]

Jens, please let me know your feedback on this series.

Vincent

> -----Original Message-----
> From: Vincent Fu
> Sent: Monday, November 29, 2021 10:20 AM
> To: axboe@kernel.dk; fio@vger.kernel.org
> Subject: RE: [PATCH 0/6] additional sg ionegine commands
> 
> Jens, please let me know if you have any feedback about this patch series.
> 
> Vincent
> 
> > -----Original Message-----
> > From: Vincent Fu
> > Sent: Monday, November 15, 2021 3:07 PM
> > To: axboe@kernel.dk; fio@vger.kernel.org
> > Cc: Vincent Fu <vincent.fu@samsung.com>
> > Subject: [PATCH 0/6] additional sg ionegine commands
> >
> > Jens, please consider this patch series that adds to the sg ioengine
> > support for the SCSI commands VERIFY, WRITE SAME(16), and WRITE
> STREAM(16).
> >
> > Vincent
> >
> >
> > Vincent Fu (6):
> >   sg: add support for VERIFY command using write modes
> >   sg: add support for WRITE SAME(16) commands with NDOB flag set
> >   sg: improve sg_write_mode option names
> >   sg: add support for WRITE STREAM(16) commands
> >   sg: allow fio to open and close streams for WRITE STREAM(16) commands
> >   docs: documentation for sg WRITE STREAM(16)
> >
> >  HOWTO                           |  36 ++++++-
> >  engines/sg.c                    | 181 ++++++++++++++++++++++++++++++--
> >  examples/sg_verify-fail.fio     |  48 +++++++++
> >  examples/sg_verify.fio          |  57 ++++++++++
> >  examples/sg_write_same_ndob.fio |  44 ++++++++
> >  fio.1                           |  47 ++++++++-
> >  6 files changed, 397 insertions(+), 16 deletions(-)  create mode
> > 100644 examples/sg_verify-fail.fio  create mode 100644
> > examples/sg_verify.fio create mode 100644
> > examples/sg_write_same_ndob.fio
> >
> > --
> > 2.25.1

Re: [PATCH 0/6] additional sg ionegine commands

[Jens Axboe]

On Mon, 15 Nov 2021 20:07:17 +0000, Vincent Fu wrote:
> Jens, please consider this patch series that adds to the sg ioengine
> support for the SCSI commands VERIFY, WRITE SAME(16), and WRITE
> STREAM(16).
> 
> Vincent
> 
> 
> [...]

Applied, thanks!

[1/6] sg: add support for VERIFY command using write modes
      commit: e8ab121c88d61624c0925b54013cd57c2dc171f2
[2/6] sg: add support for WRITE SAME(16) commands with NDOB flag set
      commit: 91e13ff509253667c11fce864693849d2cb77b67
[3/6] sg: improve sg_write_mode option names
      commit: eadf32608213dc2a44303fce0b417fbb95d1a0a6
[4/6] sg: add support for WRITE STREAM(16) commands
      commit: 9917adb5d3b9c480ac710af4bc2c9c0421c6a5df
[5/6] sg: allow fio to open and close streams for WRITE STREAM(16) commands
      commit: 5f647ee93b0b8ee07d6cbcef571986f8a18b1234
[6/6] docs: documentation for sg WRITE STREAM(16)
      commit: 71efbed61dfb157dfa7fe550f500b53f9731e1cb

Best regards,
-- 
Jens Axboe