Linux-Block Archive on lore.kernel.org
 help / Atom feed
* [PATCH blktests 00/14] Implement zoned block device support
@ 2019-01-09  1:35 Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 01/14] config: Introduce ZONED variable Damien Le Moal
                   ` (13 more replies)
  0 siblings, 14 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

The current blktests infrastucture and test cases do not support zoned block
devices and no specific test cases exist to test these block devices special
features (zone report and reset, sequential write constraint). This patch series
implement this missing support.

The series addresses two aspects: the first 6 patches introduce changes to the
common scripts and configuration are introduced to allow existing test cases to
run against a null_blk device with zone mode enabled (new ZONED config variable)
or for these test cases to be skipped if a test declare itself as not zoned
compliant. Helper functions are introduced to facilitate checking a device
zone model.

The second part, composed of the last 8 patches, introduce the new zbd test
group to cover zoned block device specific test cases. All these test cases are
implemented using the test_device() function so that target devices can be
specified in the TEST_DEVS config variable, to cover a variety of zoned block
devices: physical real drives, partitions and dm-linear setups on top of zoned
block devices, etc. Furthermore, using the infrastructure changes of the first
part, the TEST_DEVS definition can be left empty, resulting in the zbd test
cases to be run against an automatically created null_blk device with zoned
mode enabled.

5 test cases are added to the new zbd test group to check the kernel ioctl and
sysfs interface, zone report operation, zone reset and write command handling.
These tests are simple but only a start. We will in the future send more test
cases to cover at least the regressions and bugs found and fixed in the zoned
block device code since its introduction with kernel 4.10.

Another still to be added part is support for host-managed ZBC emulation in
scsi-debug to further improve test coverage without requiring a physical SMR
disk. This work is ongoing and will be added to blktests once the relevant
scsi-debug changes are accepted in the kernel.

Masato Suzuki (6):
  tests: Introduce zbd test group
  zbd/001: sysfs and ioctl consistency test
  zbd/002: report zone test
  zbd/003: Test sequential zones reset
  zbd/004: Check write split accross sequential zones
  zbd/005: Test write ordering

Shin'ichiro Kawasaki (8):
  config: Introduce ZONED variable
  common: Introduce _test_dev_is_zoned() helper function
  common: Move set_scheduler() function from multipath-over-rdma to rc
  block/004: Adjust fio conditions for zoned block device
  block/013: Skip for zoned block devices
  block/018,024: Skip when ZONED is set
  check: Introduce group_exit() function
  src: Introduce zbdioctl program

 Documentation/running-tests.md |  11 ++
 check                          |   6 +
 common/multipath-over-rdma     |  24 ----
 common/null_blk                |  23 ++-
 common/rc                      |  33 +++++
 src/.gitignore                 |   1 +
 src/Makefile                   |   3 +-
 src/zbdioctl.c                 |  83 +++++++++++
 tests/block/004                |  12 +-
 tests/block/013                |   8 ++
 tests/block/018                |   2 +-
 tests/block/024                |   2 +-
 tests/zbd/001                  |  66 +++++++++
 tests/zbd/001.out              |   2 +
 tests/zbd/002                  |  99 +++++++++++++
 tests/zbd/002.out              |   2 +
 tests/zbd/003                  |  69 +++++++++
 tests/zbd/003.out              |   2 +
 tests/zbd/004                  |  81 +++++++++++
 tests/zbd/004.out              |   2 +
 tests/zbd/005                  |  57 ++++++++
 tests/zbd/005.out              |   2 +
 tests/zbd/rc                   | 250 +++++++++++++++++++++++++++++++++
 23 files changed, 811 insertions(+), 29 deletions(-)
 create mode 100644 src/zbdioctl.c
 create mode 100755 tests/zbd/001
 create mode 100644 tests/zbd/001.out
 create mode 100755 tests/zbd/002
 create mode 100644 tests/zbd/002.out
 create mode 100755 tests/zbd/003
 create mode 100644 tests/zbd/003.out
 create mode 100755 tests/zbd/004
 create mode 100644 tests/zbd/004.out
 create mode 100755 tests/zbd/005
 create mode 100644 tests/zbd/005.out
 create mode 100644 tests/zbd/rc

-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 01/14] config: Introduce ZONED variable
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function Damien Le Moal
                   ` (12 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

To allow running tests against null_blk devices with the zoned mode
enabled, introduce the config ZONED variable. If set (ZONED=1), tests
that create a null_blk device as a test target device will be executed
against a zoned null device.

_init_null_blk is modified to prepare null_blk as a zoned blocked device
if ZONED is set. To avoid "modprobe -r null_blk" failures, rmdir calls
on all sysfs nullbX directories is added.

Introduce a helper function _null_blk_not_zoned() to check if
_init_null_blk() creates zoned device or not. This function can be used
within test script implementation of the requires() function to ensure
that the test is run only for non-zoned null_blk device.

The use of the ZONED variable requires that the kernel be compiled with
CONFIG_BLK_DEV_ZONED enabled.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 Documentation/running-tests.md | 11 +++++++++++
 check                          |  1 +
 common/null_blk                | 23 ++++++++++++++++++++++-
 3 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/Documentation/running-tests.md b/Documentation/running-tests.md
index 8f32af3..0416d6d 100644
--- a/Documentation/running-tests.md
+++ b/Documentation/running-tests.md
@@ -82,6 +82,17 @@ passing the `-d` command line option or setting the `DEVICE_ONLY` variable.
 DEVICE_ONLY=1
 ```
 
+### Zoned Block Device
+
+Some test items prepare virtual devices such as null_blk. To configure the
+virtual device as a zoned block device, set `ZONED` variable. The use of the
+ZONED variable requires that the kernel be compiled with CONFIG_BLK_DEV_ZONED
+enabled.
+
+```sh
+ZONED=1
+```
+
 ### Custom Setup
 
 The `config` file is really just a bash file that is sourced at the beginning
diff --git a/check b/check
index 6c6d9f5..141ebb6 100755
--- a/check
+++ b/check
@@ -591,6 +591,7 @@ fi
 # Default configuration.
 : "${DEVICE_ONLY:=0}"
 : "${QUICK_RUN:=0}"
+: "${ZONED:=0}"
 : "${OUTPUT:=results}"
 if [[ -v EXCLUDE ]] && ! declare -p EXCLUDE | grep -q '^declare -a'; then
 	# If EXCLUDE was not defined as an array, convert it to one.
diff --git a/common/null_blk b/common/null_blk
index 937ece0..a78c95d 100644
--- a/common/null_blk
+++ b/common/null_blk
@@ -8,8 +8,29 @@ _have_null_blk() {
 	_have_modules null_blk
 }
 
+_null_blk_not_zoned() {
+	if [[ "${ZONED}" != "0" ]]; then
+		# shellcheck disable=SC2034
+		SKIP_REASON="null_blk zoned mode not supported"
+		return 1
+	fi
+	return 0
+}
+
 _init_null_blk() {
-	if ! modprobe -r null_blk || ! modprobe null_blk "$@"; then
+	for d in /sys/kernel/config/nullb/*;
+	do [[ -d "$d" ]] && rmdir "$d";	done
+
+	local _zoned=""
+	if [[ ${ZONED} -ne 0 ]] ; then
+		if ! _have_kernel_option BLK_DEV_ZONED ; then
+			echo -n "ZONED specified for kernel with "
+			echo "CONFIG_BLK_DEV_ZONED disabled"
+			return 1
+		fi
+		_zoned="zoned=1"
+	fi
+	if ! modprobe -r null_blk || ! modprobe null_blk "$@" "${_zoned}" ; then
 		return 1
 	fi
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 01/14] config: Introduce ZONED variable Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  8:46   ` Johannes Thumshirn
  2019-01-09  1:35 ` [PATCH blktests 03/14] common: Move set_scheduler() function from multipath-over-rdma to rc Damien Le Moal
                   ` (11 subsequent siblings)
  13 siblings, 1 reply; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Introduce the helper function _test_dev_is_zoned() to check if a test
device is a zoned block device. Using this function, test scripts can
adjust test conditions based on the device zone model.
This function can also be used within test scripts implementation of the
device_requires() function to ensure that the test is run only for zoned
block device or only for non-zoned block device.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 common/rc | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/common/rc b/common/rc
index 153a323..999590d 100644
--- a/common/rc
+++ b/common/rc
@@ -214,3 +214,12 @@ _test_dev_in_hotplug_slot() {
 _filter_xfs_io_error() {
 	sed -e 's/^\(.*\)64\(: .*$\)/\1\2/'
 }
+
+_test_dev_is_zoned() {
+	local zoned_file="${TEST_DEV_SYSFS}/queue/zoned"
+	if grep -q -e "none" "${zoned_file}" ; then
+		SKIP_REASON="${TEST_DEV} is not a zoned block device"
+		return 1
+	fi
+	return 0
+}
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 03/14] common: Move set_scheduler() function from multipath-over-rdma to rc
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 01/14] config: Introduce ZONED variable Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 04/14] block/004: Adjust fio conditions for zoned block device Damien Le Moal
                   ` (10 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

set_scheduler() function defined in common/multipath-over-rdma is useful
to set up specific IO scheduler not only for multipath tests but also for
zoned block device tests. Move the function to common/rc to reuse it for
test groups other than multipath.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 common/multipath-over-rdma | 24 ------------------------
 common/rc                  | 24 ++++++++++++++++++++++++
 2 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/common/multipath-over-rdma b/common/multipath-over-rdma
index 308c9b5..6bea8e4 100644
--- a/common/multipath-over-rdma
+++ b/common/multipath-over-rdma
@@ -275,30 +275,6 @@ mount_configfs() {
 	fi
 }
 
-# Set scheduler of block device $1 to $2.
-set_scheduler() {
-	local b=$1 p s=$2
-
-	p=/sys/class/block/$b/queue/scheduler
-	if [ -e "/sys/block/$b/mq" ]; then
-		case "$s" in
-			noop)        s=none;;
-			deadline)    s=mq-deadline;;
-			bfq)         s=bfq;;
-		esac
-	else
-		case "$s" in
-			none)        s=noop;;
-			mq-deadline) s=deadline;;
-			bfq-mq)      s=bfq;;
-		esac
-	fi
-	if ! echo "$s" > "$p"; then
-		echo "Changing scheduler of $b from $(<"$p") into $s failed"
-		return 1
-	fi
-}
-
 # Get a /dev/... path that points at dm device number $1. Set its I/O scheduler
 # to $2 and its timeout to $3. The shell script that includes this file must
 # define a function get_bdev_path() that translates device number $1 into a
diff --git a/common/rc b/common/rc
index 999590d..85f6468 100644
--- a/common/rc
+++ b/common/rc
@@ -223,3 +223,27 @@ _test_dev_is_zoned() {
 	fi
 	return 0
 }
+
+# Set scheduler of block device $1 to $2.
+set_scheduler() {
+	local b=$1 p s=$2
+
+	p=/sys/class/block/$b/queue/scheduler
+	if [ -e "/sys/block/$b/mq" ]; then
+		case "$s" in
+			noop)        s=none;;
+			deadline)    s=mq-deadline;;
+			bfq)         s=bfq;;
+		esac
+	else
+		case "$s" in
+			none)        s=noop;;
+			mq-deadline) s=deadline;;
+			bfq-mq)      s=bfq;;
+		esac
+	fi
+	if ! echo "$s" > "$p"; then
+		echo "Changing scheduler of $b from $(<"$p") into $s failed"
+		return 1
+	fi
+}
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 04/14] block/004: Adjust fio conditions for zoned block device
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (2 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 03/14] common: Move set_scheduler() function from multipath-over-rdma to rc Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 05/14] block/013: Skip for zoned block devices Damien Le Moal
                   ` (9 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

For a random write pattern to a zoned block device, fio requires --direct=1
and --zonemode=zbd options as well as deadline I/O scheduler to be
specified. Specify these options and set the I/O scheduler if the target
device is a zoned block device.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/block/004 | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/tests/block/004 b/tests/block/004
index 4c14c4b..d9b8209 100755
--- a/tests/block/004
+++ b/tests/block/004
@@ -16,10 +16,20 @@ requires() {
 test_device() {
 	echo "Running ${TEST_NAME}"
 
+	local directio=""
+	local zbdmode=""
+
+	if _test_dev_is_zoned; then
+		directio="--direct=1"
+		zbdmode="--zonemode=zbd"
+		set_scheduler "$(basename "$(readlink -f "${TEST_DEV}")")" \
+			      deadline
+	fi
+
 	FIO_PERF_FIELDS=("write iops")
 	_fio_perf --bs=4k --rw=randwrite --norandommap --fsync=1 \
 		--number_ios=256 --numjobs=64 --name=flushes \
-		--filename="$TEST_DEV"
+		${directio} ${zbdmode} --filename="$TEST_DEV"
 
 	echo "Test complete"
 }
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 05/14] block/013: Skip for zoned block devices
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (3 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 04/14] block/004: Adjust fio conditions for zoned block device Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 06/14] block/018,024: Skip when ZONED is set Damien Le Moal
                   ` (8 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

block/013 runs mkfs.ext3 but ext3 file system does not support zoned
block devices. Add device_requires() implementation to automatically
skip the test if the target device is a zoned block device.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/block/013 | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tests/block/013 b/tests/block/013
index 35fc181..fe68cc1 100755
--- a/tests/block/013
+++ b/tests/block/013
@@ -19,6 +19,14 @@ requires() {
 	_have_program mkfs.ext3
 }
 
+device_requires() {
+	if _test_dev_is_zoned; then
+		SKIP_REASON="${TEST_DEV} is a zoned block device (not supported by ext3)"
+		return 1
+	fi
+	return 0
+}
+
 test_device() {
 	echo "Running ${TEST_NAME}"
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 06/14] block/018,024: Skip when ZONED is set
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (4 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 05/14] block/013: Skip for zoned block devices Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 07/14] check: Introduce group_exit() function Damien Le Moal
                   ` (7 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

These tests do not execute a purely sequential write pattern which will
cause I/O failures when ZONED is set and null_blk is created with zoned
mode. Skip the tests in such case.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 tests/block/018 | 2 +-
 tests/block/024 | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/block/018 b/tests/block/018
index 7312723..3c1eca4 100755
--- a/tests/block/018
+++ b/tests/block/018
@@ -11,7 +11,7 @@ DESCRIPTION="do I/O and check iostats times"
 QUICK=1
 
 requires() {
-	_have_null_blk
+	_have_null_blk && _null_blk_not_zoned
 }
 
 init_times() {
diff --git a/tests/block/024 b/tests/block/024
index cf14707..c8dd1f7 100755
--- a/tests/block/024
+++ b/tests/block/024
@@ -13,7 +13,7 @@ DESCRIPTION="do I/O faster than a jiffy and check iostats times"
 QUICK=1
 
 requires() {
-	_have_null_blk
+	_have_null_blk && _null_blk_not_zoned
 }
 
 init_times() {
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 07/14] check: Introduce group_exit() function
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (5 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 06/14] block/018,024: Skip when ZONED is set Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 08/14] src: Introduce zbdioctl program Damien Le Moal
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

This optional function can be defined by a test group rc script. When
defined, it is executed after all tests of the group complete, allowing
cleaning up resources used by all tests of the group.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 check | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/check b/check
index 141ebb6..360f61b 100755
--- a/check
+++ b/check
@@ -467,6 +467,11 @@ _run_group() {
 			ret=1
 		fi
 	done
+
+	if declare -fF group_exit >/dev/null ; then
+		group_exit
+	fi
+
 	return $ret
 }
 
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 08/14] src: Introduce zbdioctl program
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (6 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 07/14] check: Introduce group_exit() function Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 09/14] tests: Introduce zbd test group Damien Le Moal
                   ` (5 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

zbdioctl implements calls to zoned block devices ioctls that are not
supported currently by sys-utils blkzone utility, namely BLKGETZONESZ
and BLKGETNRZONES.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
 src/.gitignore |  1 +
 src/Makefile   |  3 +-
 src/zbdioctl.c | 83 ++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 86 insertions(+), 1 deletion(-)
 create mode 100644 src/zbdioctl.c

diff --git a/src/.gitignore b/src/.gitignore
index 8c95785..2108f56 100644
--- a/src/.gitignore
+++ b/src/.gitignore
@@ -6,3 +6,4 @@
 /nbdsetsize
 /sg/dxfer-from-dev
 /sg/syzkaller1
+/zbdioctl
diff --git a/src/Makefile b/src/Makefile
index c4094b4..5a0556f 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -5,7 +5,8 @@ C_TARGETS := \
 	sg/dxfer-from-dev \
 	sg/syzkaller1 \
 	nbdsetsize \
-	loop_change_fd
+	loop_change_fd \
+	zbdioctl
 
 CXX_TARGETS := \
 	discontiguous-io
diff --git a/src/zbdioctl.c b/src/zbdioctl.c
new file mode 100644
index 0000000..1ea72e8
--- /dev/null
+++ b/src/zbdioctl.c
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-3.0+
+// Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/ioctl.h>
+#include <linux/blkzoned.h>
+#include <linux/types.h>
+
+#if !defined(BLKGETZONESZ) || !defined(BLKGETNRZONES)
+
+int main(int argc, char **argv)
+{
+	return EXIT_FAILURE;
+}
+
+#else
+
+struct request {
+	const char *name;
+	unsigned long code;
+} requests[] = {
+	{ "-s", BLKGETZONESZ},
+	{ "-n", BLKGETNRZONES},
+	{ NULL, 0},
+};
+
+void usage(const char *progname)
+{
+	int i = 0;
+
+	fprintf(stderr, "usage: %s <request> <device file>\n", progname);
+	fprintf(stderr, "<request> can be:\n");
+	while (requests[i].name) {
+		fprintf(stderr, "\t%s\n", requests[i].name);
+		i++;
+	}
+	exit(EXIT_FAILURE);
+}
+
+int main(int argc, char **argv)
+{
+	int i = 0, fd, ret;
+	unsigned int val;
+	unsigned long code = 0;
+
+	if (argc != 3)
+		usage(argv[0]);
+
+	while (requests[i].name) {
+		if (strcmp(argv[1], requests[i].name) == 0) {
+			code = requests[i].code;
+			break;
+		}
+		i++;
+	}
+	if (code == 0)
+		usage(argv[0]);
+
+	fd = open(argv[2], O_RDWR);
+	if (fd < 0) {
+		perror("open");
+		return EXIT_FAILURE;
+	}
+
+	ret = ioctl(fd, code, &val);
+	if (ret < 0) {
+		perror("ioctl");
+		close(fd);
+		return EXIT_FAILURE;
+	}
+
+	printf("%u\n", val);
+
+	close(fd);
+
+	return EXIT_SUCCESS;
+}
+
+#endif
+
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 09/14] tests: Introduce zbd test group
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (7 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 08/14] src: Introduce zbdioctl program Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 10/14] zbd/001: sysfs and ioctl consistency test Damien Le Moal
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

The zoned block device (zbd) test group is used to gather all tests
specific to zoned block devices (null_blk device with zoned mode enabled,
SMR disks, dm-linear on top of zoned devices, etc). Execution of this group
requires that the kernel be compiled with the block layer
CONFIG_BLK_DEV_ZONED option enabled and also requires the null_blk driver
to have zoned mode support (added in kernel 4.19).

This group rc script allows all tests to operate even if TEST_DEVS is not
set in the config. If TEST_DEVS is not set, the rc script creates a
null_blk device with zoned mode enabled and the created device assigned to
TEST_DEVS. If TEST_DEVS is set, all tests are executed against the
specified devices, which all must be zoned. With this, all tests scripts
can be written by only defining the test_device() function while allowing
operation on both nullblk and user specified devices.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/rc | 250 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 250 insertions(+)
 create mode 100644 tests/zbd/rc

diff --git a/tests/zbd/rc b/tests/zbd/rc
new file mode 100644
index 0000000..bf68264
--- /dev/null
+++ b/tests/zbd/rc
@@ -0,0 +1,250 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Tests for Zone Block Device.
+
+. common/rc
+. common/null_blk
+
+#
+# Test requirement check functions
+#
+
+group_requires() {
+	_have_root || return $?
+	_have_program blkzone || return $?
+	_have_program dd || return $?
+	_have_kernel_option BLK_DEV_ZONED || return $?
+	_have_modules null_blk && _have_module_param null_blk zoned
+
+	# If TEST_DEVS is set in config file, use it as is.
+	# Otherwise, create a zoned null_blk device and set TEST_DEVS.
+	if [[ -n "${TEST_DEVS}" ]] ; then
+		return 0
+	fi
+
+	local test_dev
+	local sysfs_dir
+	if ! _init_null_blk zone_size=4 gb=1 zoned=1 ; then
+		return 1
+	fi
+	test_dev=/dev/nullb0
+	if ! sysfs_dir="$(_find_sysfs_dir "$test_dev")"; then
+		_error "could not find sysfs directory for ${test_dev}"
+		return 1
+	fi
+	export _NULL_BLK_ZONED_CREATED=1
+	TEST_DEVS+=( "${test_dev}" )
+
+	# shellcheck disable=SC2034
+	TEST_DEV_SYSFS_DIRS["$test_dev"]="$sysfs_dir"
+
+	return 0
+}
+
+group_device_requires() {
+	_test_dev_is_zoned
+}
+
+group_exit() {
+	if [[ -n "${_NULL_BLK_ZONED_CREATED}" ]] ; then
+		_exit_null_blk
+		unset _NULL_BLK_ZONED_CREATED
+	fi
+}
+
+#
+# Zone types and conditions
+#
+export ZONE_TYPE_CONVENTIONAL=1
+export ZONE_TYPE_SEQ_WRITE_REQUIRED=2
+export ZONE_TYPE_SEQ_WRITE_PREFERRED=3
+
+export ZONE_COND_EMPTY=1
+export ZONE_COND_IMPLICIT_OPEN=2
+export ZONE_COND_FULL=14
+
+export ZONE_TYPE_ARRAY=(
+	[1]="CONVENTIONAL"
+	[2]="SEQ_WRITE_REQUIRED"
+	[3]="SEQ_WRITE_PREFERRED"
+)
+
+export ZONE_COND_ARRAY=(
+	[0]="NOT_WP"
+	[1]="EMPTY"
+	[2]="IMPLICIT_OPEN"
+	[3]="EXPLICIT_OPEN"
+	[4]="CLOSE"
+	[13]="READ_ONLY"
+	[14]="FULL"
+	[15]="OFFLINE"
+)
+
+# sysfs variable array indices
+export SV_CAPACITY=0
+export SV_CHUNK_SECTORS=1
+export SV_PHYS_BLK_SIZE=2
+export SV_PHYS_BLK_SECTORS=3
+export SV_NR_ZONES=4
+
+#
+# Helper functions
+#
+
+# Obtain zone related sysfs variables and keep in a global array until put
+# function call.
+_get_sysfs_variable() {
+	unset SYSFS_VARS
+	local _dir=${TEST_DEV_SYSFS}
+	SYSFS_VARS[$SV_CAPACITY]=$(<"${_dir}"/size)
+	SYSFS_VARS[$SV_CHUNK_SECTORS]=$(<"${_dir}"/queue/chunk_sectors)
+	SYSFS_VARS[$SV_PHYS_BLK_SIZE]=$(<"${_dir}"/queue/physical_block_size)
+	SYSFS_VARS[$SV_PHYS_BLK_SECTORS]=$((SYSFS_VARS[SV_PHYS_BLK_SIZE] / 512))
+
+	# If the nr_zones sysfs attribute exists, get its value. Otherwise,
+	# calculate its value based on the total capacity and zone size, taking
+	# into account that the last zone can be smaller than other zones.
+	if [[ -e ${TEST_DEV_SYSFS}/queue/nr_zones ]]; then
+		SYSFS_VARS[$SV_NR_ZONES]=$(<"${_dir}"/queue/nr_zones)
+	else
+		SYSFS_VARS[$SV_NR_ZONES]=$(( (SYSFS_VARS[SV_CAPACITY] - 1) \
+				/ SYSFS_VARS[SV_CHUNK_SECTORS] + 1 ))
+	fi
+}
+
+_put_sysfs_variable() {
+	unset SYSFS_VARS
+}
+
+# Issue zone report command and keep reported information in global arrays
+# until put function call.
+_get_blkzone_report() {
+	local target_dev=${1}
+
+	# Initialize arrays to store parsed blkzone reports.
+	# Number of reported zones is set in REPORTED_COUNT.
+	# The arrays have REPORTED_COUNT+1 elements with additional one at tail
+	# to simplify loop operation.
+	ZONE_STARTS=()
+	ZONE_LENGTHS=()
+	ZONE_WPTRS=()
+	ZONE_CONDS=()
+	ZONE_TYPES=()
+	NR_CONV_ZONES=0
+	REPORTED_COUNT=0
+
+	TMP_REPORT_FILE=${TMPDIR}/blkzone_report
+	if ! blkzone report "${target_dev}" > "${TMP_REPORT_FILE}"; then
+		echo "blkzone command failed"
+		return $?
+	fi
+
+	local _IFS=$IFS
+	local -i loop=0
+	IFS=$' ,:'
+	while read -r -a _tokens
+	do
+		ZONE_STARTS+=($((_tokens[1])))
+		ZONE_LENGTHS+=($((_tokens[3])))
+		ZONE_WPTRS+=($((_tokens[5])))
+		ZONE_CONDS+=($((${_tokens[11]%\(*})))
+		ZONE_TYPES+=($((${_tokens[13]%\(*})))
+		if [[ ${ZONE_TYPES[-1]} -eq ${ZONE_TYPE_CONVENTIONAL} ]]; then
+			(( NR_CONV_ZONES++ ))
+		fi
+		(( loop++ ))
+	done < "${TMP_REPORT_FILE}"
+	IFS="$_IFS"
+	REPORTED_COUNT=${loop}
+
+	if [[ ${REPORTED_COUNT} -eq 0 ]] ; then
+		echo "blkzone report returned no zone"
+		return 1
+	fi
+
+	# Set value to allow additioanl element access at array end
+	local -i max_idx=$((REPORTED_COUNT - 1))
+	ZONE_STARTS+=( $((ZONE_STARTS[max_idx] + ZONE_LENGTHS[max_idx])) )
+	ZONE_LENGTHS+=( "${ZONE_LENGTHS[max_idx]}" )
+	ZONE_WPTRS+=( "${ZONE_WPTRS[max_idx]}" )
+	ZONE_CONDS+=( "${ZONE_CONDS[max_idx]}" )
+	ZONE_TYPES+=( "${ZONE_TYPES[max_idx]}" )
+
+	rm -f "${TMP_REPORT_FILE}"
+}
+
+_put_blkzone_report() {
+	unset ZONE_STARTS
+	unset ZONE_LENGTHS
+	unset ZONE_WPTRS
+	unset ZONE_CONDS
+	unset ZONE_TYPES
+	unset REPORTED_COUNT
+	unset NR_CONV_ZONES
+}
+
+# Issue reset zone command with zone count option.
+# Call _get_blkzone_report() beforehand.
+_reset_zones() {
+	local target_dev=${1}
+	local -i idx=${2}
+	local -i count=${3}
+
+	if ! blkzone reset -o "${ZONE_STARTS[idx]}" -c "${count}" \
+	     "${target_dev}" >> "$FULL" 2>&1 ; then
+		echo "blkzone reset command failed"
+		return 1
+	fi
+}
+
+# Issue dd command with five arguments and record command line in FULL file.
+# args: target device, r/w, start sector, sector len, block size in bytes
+_dd() {
+	local target_dev=${1}
+	local rw=${2}
+	local -i start_sector=${3}
+	local -i start_byte=$(( start_sector * 512 ))
+	local -i sector_count=${4}
+	local -i bs=${5}
+	local -i block_count=$(( sector_count * 512 / bs ))
+
+	local _cmd="dd bs=${bs} count=${block_count}"
+
+	if [[ ${rw} = "read" ]]; then
+		_cmd="${_cmd} if=${target_dev} of=/dev/null"
+		_cmd="${_cmd} iflag=skip_bytes skip=${start_byte}"
+	elif [[ ${rw} = "write" ]]; then
+		_cmd="${_cmd} if=/dev/zero of=${target_dev}"
+		_cmd="${_cmd} oflag=seek_bytes,direct seek=${start_byte}"
+	fi
+
+	echo "${_cmd}" >> "$FULL" 2>&1
+
+	if ! eval "${_cmd}" >> "$FULL" 2>&1 ; then
+		echo "dd command failed"
+		return 1
+	fi
+
+	sync
+}
+
+# Search zones and find two contiguous sequential required zones.
+# Return index of the first zone of the found two zones.
+# Call _get_blkzone_report() beforehand.
+_find_two_contiguous_seq_zones() {
+	local -i type_seq=${ZONE_TYPE_SEQ_WRITE_REQUIRED}
+	for ((idx = NR_CONV_ZONES; idx < REPORTED_COUNT; idx++)); do
+		if [[ ${ZONE_TYPES[idx]} -eq ${type_seq} &&
+		      ${ZONE_TYPES[idx+1]} -eq ${type_seq} ]];
+		then
+			echo "${idx}"
+			return 0
+		fi
+	done
+
+	echo "Contiguous sequential write required zones not found"
+	return 1
+}
+
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 10/14] zbd/001: sysfs and ioctl consistency test
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (8 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 09/14] tests: Introduce zbd test group Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 11/14] zbd/002: report zone test Damien Le Moal
                   ` (3 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

Obtain a zoned block device attributes from sysfs and using ioctl calls
and confirm the consistency of the values.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/001     | 66 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/zbd/001.out |  2 ++
 2 files changed, 68 insertions(+)
 create mode 100755 tests/zbd/001
 create mode 100644 tests/zbd/001.out

diff --git a/tests/zbd/001 b/tests/zbd/001
new file mode 100755
index 0000000..7f16c9a
--- /dev/null
+++ b/tests/zbd/001
@@ -0,0 +1,66 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Check zoned block device sysfs and ioctl parameter availability and
+# confirm consistency among them.
+
+. tests/zbd/rc
+
+DESCRIPTION="sysfs and ioctl"
+QUICK=1
+
+requires() {
+	_have_src_program zbdioctl
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	# Get and keep sysfs variables
+	_get_sysfs_variable "${TEST_DEV}" || return $?
+	local -i capacity=${SYSFS_VARS[$SV_CAPACITY]}
+	local -i chunk_sectors=${SYSFS_VARS[$SV_CHUNK_SECTORS]}
+
+	# Calculate expected number of zones based on the total capacity and
+	# zone size, taking into account that the last zone can be smaller
+	# than other zones.
+	local -i nr_zones=$(( (capacity - 1) / chunk_sectors + 1 ))
+
+	# Compare sysfs values and ioctl values
+	if [[ -e "${TEST_DEV_SYSFS}"/queue/nr_zones ]]; then
+		local -i sysfs_nr_zones=${SYSFS_VARS[$SV_NR_ZONES]}
+		local -i ioctl_nr_zones
+		local -i ioctl_zonesize
+
+		ioctl_zonesize=$(src/zbdioctl -s "${TEST_DEV}")
+		if [[ ${chunk_sectors} -ne ${ioctl_zonesize} ]]; then
+			echo -n "ioctl zone size:${ioctl_zonesize} != "
+			echo "sysfs chunk_sectors:${chunk_sectors}"
+			return 1
+		fi
+
+		ioctl_nr_zones=$(src/zbdioctl -n "${TEST_DEV}")
+		if [[ ${nr_zones} -ne ${ioctl_nr_zones} ]]; then
+			echo -n "ioctl nr_zones:${ioctl_nr_zones} != "
+			echo "nr_zones:${nr_zones}"
+			return 1
+		fi
+
+		if [[ ${nr_zones} -ne ${sysfs_nr_zones} ]]; then
+			echo -n "sysfs nr_zones:${sysfs_nr_zones} != "
+			echo "nr_zones:${nr_zones}"
+			return 1
+		fi
+	fi
+
+	_put_sysfs_variable
+	{
+		echo "Capacity: ${capacity} sectors"
+		echo "Zone: ${chunk_sectors} sectors"
+		echo "Number of zones: ${nr_zones} zones"
+	} >> "$FULL" 2>&1
+
+	echo "Test complete"
+}
+
diff --git a/tests/zbd/001.out b/tests/zbd/001.out
new file mode 100644
index 0000000..7d72a8f
--- /dev/null
+++ b/tests/zbd/001.out
@@ -0,0 +1,2 @@
+Running zbd/001
+Test complete
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 11/14] zbd/002: report zone test
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (9 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 10/14] zbd/001: sysfs and ioctl consistency test Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 12/14] zbd/003: Test sequential zones reset Damien Le Moal
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

Get a report for all zones and confirm that the reported values are
valid and consistent with regard to the device capacity and zone size.

Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/002     | 99 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/zbd/002.out |  2 +
 2 files changed, 101 insertions(+)
 create mode 100755 tests/zbd/002
 create mode 100644 tests/zbd/002.out

diff --git a/tests/zbd/002 b/tests/zbd/002
new file mode 100755
index 0000000..aec4813
--- /dev/null
+++ b/tests/zbd/002
@@ -0,0 +1,99 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Issue report zone command and confirm consistency of reported values.
+
+. tests/zbd/rc
+
+DESCRIPTION="report zone"
+
+_check_blkzone_report() {
+	# Utilize local variables as much as possible to speed up loop execution
+	local -i max_idx=$((REPORTED_COUNT - 1))
+	local -i chunk_sectors=${SYSFS_VARS[$SV_CHUNK_SECTORS]}
+	local -i cur_start=${ZONE_STARTS[0]}
+	local -i next_start=0
+	local -i len=0
+	local -i wptr=0
+	local -i cond=0
+	local -i zone_type=0
+
+	# Confirm number of reported zones
+	if [[ ${REPORTED_COUNT} -ne ${SYSFS_VARS[$SV_NR_ZONES]} ]]; then
+		echo "The number of zones reported differ from sysfs nr_zones"
+		echo -n "Reported zones count: ${REPORTED_COUNT}  "
+		echo "sysfs nr_zones: ${SYSFS_VARS[$SV_NR_ZONES]}"
+		return 1
+	fi
+
+	# Check consistency between last zone size and capacity
+	local -i last_zone_end=$((ZONE_STARTS[max_idx] + ZONE_LENGTHS[max_idx]))
+	if [[ ${last_zone_end} -gt ${SYSFS_VARS[$SV_CAPACITY]} ]]; then
+		echo "Last zone start sector + length exceeds capacity"
+		echo -n "Capacity: ${SYSFS_VARS[$SV_CAPACITY]}, "
+		echo "last zone start sector + length: ${last_zone_end}"
+		return 1
+	fi
+
+	# Check each zone parameter validity and that all zones are contiguous
+	for ((idx = 0; idx <= max_idx; idx++)); do
+
+		next_start=${ZONE_STARTS[$((idx+1))]}
+		len=${ZONE_LENGTHS[$idx]}
+		wptr=${ZONE_WPTRS[$idx]}
+		cond=${ZONE_CONDS[$idx]}
+		zone_type=${ZONE_TYPES[$idx]}
+
+		# Check two zones are contiguous
+		if [[ $((cur_start+len)) -ne ${next_start} ]]; then
+			echo -n "Zones are not contiguous at zone ${idx}. "
+			echo "cur:${cur_start}+${len}, next:${next_start}"
+			return 1
+		fi
+
+		# Check zone size
+		if [[ ${len} -ne ${chunk_sectors} &&
+		      ${idx} -ne ${max_idx} ]]; then
+			echo -n "Zone size is not same as chunk_sectors "
+			echo -n "at zone ${idx}. "
+			echo "size: ${len}, chunk_sectors: ${chunk_sectors}"
+			return 1
+		fi
+
+		# Check write pointer
+		if [[ ${wptr} -lt 0 || ${wptr} -gt ${len} ]]; then
+			echo -n "Write pointer is invalid at zone ${idx}. "
+			echo "wp:${wptr}"
+			return 1
+		fi
+
+		# Check zone condition
+		if [[ ! ${ZONE_COND_ARRAY[cond]} ]]; then
+			echo -n "Zone condition is incorrect at zone ${idx}. "
+			echo "condition: ${cond}"
+			return 1
+		fi
+
+		# Check zone type
+		if [[ ! ${ZONE_TYPE_ARRAY[zone_type]} ]]; then
+			echo -n "Zone type is incorrect at zone ${idx}. "
+			echo "type: ${zone_type}"
+			return 1
+		fi
+		cur_start=${next_start}
+	done
+	return 0
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	_get_sysfs_variable "${TEST_DEV}" || return $?
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	_check_blkzone_report || return $?
+	_put_blkzone_report
+	_put_sysfs_variable
+
+	echo "Test complete"
+}
diff --git a/tests/zbd/002.out b/tests/zbd/002.out
new file mode 100644
index 0000000..7317541
--- /dev/null
+++ b/tests/zbd/002.out
@@ -0,0 +1,2 @@
+Running zbd/002
+Test complete
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 12/14] zbd/003: Test sequential zones reset
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (10 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 11/14] zbd/002: report zone test Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 13/14] zbd/004: Check write split accross sequential zones Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 14/14] zbd/005: Test write ordering Damien Le Moal
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

Test zone reset operation to make sure that the BLKRESETZONE ioctl call
works as expected but also that the zone sector remapping that may be
done for logical devices (partitions or dm-linear devices) is correct.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/003     | 69 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/zbd/003.out |  2 ++
 2 files changed, 71 insertions(+)
 create mode 100755 tests/zbd/003
 create mode 100644 tests/zbd/003.out

diff --git a/tests/zbd/003 b/tests/zbd/003
new file mode 100755
index 0000000..a44960d
--- /dev/null
+++ b/tests/zbd/003
@@ -0,0 +1,69 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Confirm that reset zone command for 2 contiguous sequential write
+# requried zones works as expected.
+
+. tests/zbd/rc
+
+DESCRIPTION="reset sequential required zones"
+QUICK=1
+
+requires() {
+	_have_program blkzone
+}
+
+test_device() {
+	local -i zone_idx
+	local -a target_zones
+	local -i phys_blk_size
+
+	echo "Running ${TEST_NAME}"
+
+	# Get physical block size for dd
+	_get_sysfs_variable "${TEST_DEV}" || return $?
+	phys_blk_size=${SYSFS_VARS[$SV_PHYS_BLK_SIZE]}
+	_put_sysfs_variable
+
+	# Select 2 target zones so that reset zone range also get tested
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	zone_idx=$(_find_two_contiguous_seq_zones) || return $?
+	target_zones=( "${zone_idx}" "$((zone_idx + 1))" )
+
+	# Reset the 2 target zones and write 4KB in each zone to increment
+	# their write pointer positions
+	_reset_zones "${TEST_DEV}" "${zone_idx}" "2"
+	for i in "${target_zones[@]}"; do
+		_dd "${TEST_DEV}" "write" "${ZONE_STARTS[$i]}" \
+		    "8" "${phys_blk_size}"
+	done
+	_put_blkzone_report
+
+	# Check that the write pointers have moved by 8x512 B sectors
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	for i in "${target_zones[@]}"; do
+		if [[ ${ZONE_WPTRS[$i]} -ne 8 ]]; then
+			echo -n "Unexpected write poiter position in zone ${i} "
+			echo "wp: ${ZONE_WPTRS[i]}"
+			return 1
+		fi
+	done
+
+	# Reset the 2 target zones
+	_reset_zones "${TEST_DEV}" "${zone_idx}" "2"
+	_put_blkzone_report
+
+	# Check that the write pointers have moved back to position 0
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	for i in "${target_zones[@]}"; do
+		if [[ ${ZONE_WPTRS[i]} -ne 0 ]]; then
+			echo -n "Unexpected non-zero write poiter in zone ${i} "
+			echo "wp: ${ZONE_WPTRS[i]}"
+			return 1
+		fi
+	done
+	_put_blkzone_report
+
+	echo "Test complete"
+}
diff --git a/tests/zbd/003.out b/tests/zbd/003.out
new file mode 100644
index 0000000..7326ae4
--- /dev/null
+++ b/tests/zbd/003.out
@@ -0,0 +1,2 @@
+Running zbd/003
+Test complete
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 13/14] zbd/004: Check write split accross sequential zones
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (11 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 12/14] zbd/003: Test sequential zones reset Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  2019-01-09  1:35 ` [PATCH blktests 14/14] zbd/005: Test write ordering Damien Le Moal
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

Check that write operations spanning a zone boundary are correctly
processed as 2 different write operations each fully within a single
zone.

Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/004     | 81 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/zbd/004.out |  2 ++
 2 files changed, 83 insertions(+)
 create mode 100755 tests/zbd/004
 create mode 100644 tests/zbd/004.out

diff --git a/tests/zbd/004 b/tests/zbd/004
new file mode 100755
index 0000000..7d708f4
--- /dev/null
+++ b/tests/zbd/004
@@ -0,0 +1,81 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Check kernel splits write operations across a zone border. Select two
+# contiguous sequential write required zones and confirm write oprations
+# across the two zones succeed.
+
+. tests/zbd/rc
+
+DESCRIPTION="write split across sequential zones"
+QUICK=1
+
+_check_zone_cond() {
+	local -i idx=${1}
+	local -i cond=${2}
+
+	if [[ ${ZONE_CONDS[idx]} -ne ${cond} ]]; then
+		echo -n "Zone ${idx} condition is not ${ZONE_COND_ARRAY[cond]} "
+		echo "cond: ${ZONE_COND_ARRAY[ZONE_CONDS[idx]]}"
+		return 1
+	fi
+}
+
+test_device() {
+	local -i idx
+	local -i phys_blk_size
+	local -i phys_blk_sectors
+
+	echo "Running ${TEST_NAME}"
+
+	# Get physical block size and sectors for dd.
+	_get_sysfs_variable "${TEST_DEV}" || return $?
+	phys_blk_size=${SYSFS_VARS[SV_PHYS_BLK_SIZE]}
+	phys_blk_sectors=${SYSFS_VARS[SV_PHYS_BLK_SECTORS]}
+	_put_sysfs_variable
+
+	# Find target sequential required zones and reset write pointers
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	idx=$(_find_two_contiguous_seq_zones) || return $?
+	_reset_zones "${TEST_DEV}" "${idx}" "2"
+
+	# Confirm the zones are initialized
+	_put_blkzone_report
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	_check_zone_cond "${idx}" "${ZONE_COND_EMPTY}" || return $?
+	_check_zone_cond "$((idx+1))" "${ZONE_COND_EMPTY}" || return $?
+
+	# Fill first target zone, remaining a physical block to write
+	if ! _dd "${TEST_DEV}" "write" "${ZONE_STARTS[idx]}" \
+	     $((ZONE_LENGTHS[idx] - phys_blk_sectors)) ${phys_blk_size} ; then
+		echo "Fill zone failed"
+		return 1
+	fi
+
+	# Write across the zone border as a single block write
+	local -i start_sector=$((ZONE_STARTS[idx+1] - phys_blk_sectors))
+	if ! _dd "${TEST_DEV}" "write" ${start_sector} \
+	     $((phys_blk_sectors * 2)) $((phys_blk_size * 2)) ; then
+		echo "Write across zone border failed"
+		return 1
+	fi
+
+	# Confirm the zone conditions are as expected
+	_put_blkzone_report
+	_get_blkzone_report "${TEST_DEV}" || return $?
+	_check_zone_cond "${idx}" "${ZONE_COND_FULL}" || return $?
+	_check_zone_cond "$((idx+1))" "${ZONE_COND_IMPLICIT_OPEN}" || return $?
+	if [[ ${ZONE_WPTRS[idx+1]} -ne ${phys_blk_sectors} ]]; then
+		echo -n "Unexpected write pointer for zone $((idx+1)) "
+		echo "wp: ${ZONE_WPTRS[idx+1]}"
+		return 1
+	fi
+
+	# Clean up
+	_reset_zones "${TEST_DEV}" "${idx}" "2"
+	_put_blkzone_report
+
+	echo "Test complete"
+}
+
diff --git a/tests/zbd/004.out b/tests/zbd/004.out
new file mode 100644
index 0000000..dd4ea94
--- /dev/null
+++ b/tests/zbd/004.out
@@ -0,0 +1,2 @@
+Running zbd/004
+Test complete
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH blktests 14/14] zbd/005: Test write ordering
  2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
                   ` (12 preceding siblings ...)
  2019-01-09  1:35 ` [PATCH blktests 13/14] zbd/004: Check write split accross sequential zones Damien Le Moal
@ 2019-01-09  1:35 ` Damien Le Moal
  13 siblings, 0 replies; 16+ messages in thread
From: Damien Le Moal @ 2019-01-09  1:35 UTC (permalink / raw)
  To: linux-block, Omar Sandoval, Masato Suzuki, Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

From: Masato Suzuki <masato.suzuki@wdc.com>

Run a high queue depth direct sequential write fio job to check that
write requests are not being reordered when the deadline scheduler is
used. This test allowed to catch a bug fixed with commit
80e02039721 "block: mq-deadline: Fix write completion handling".

Signed-off-by: Masato Suzuki <masato.suzuki@wdc.com>
---
 tests/zbd/005     | 57 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/zbd/005.out |  2 ++
 2 files changed, 59 insertions(+)
 create mode 100755 tests/zbd/005
 create mode 100644 tests/zbd/005.out

diff --git a/tests/zbd/005 b/tests/zbd/005
new file mode 100755
index 0000000..7b38462
--- /dev/null
+++ b/tests/zbd/005
@@ -0,0 +1,57 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2018 Western Digital Corporation or its affiliates.
+#
+# Test zoned block device write ordering. Run simultaneous write operations
+# to a single zone and confirm no write ordering failure.
+# It checks fix by commit 80e02039721 ("block: mq-deadline: Fix write
+# completion handling").
+
+. tests/zbd/rc
+
+DESCRIPTION="write command ordering"
+TIMED=1
+
+requires() {
+	_have_fio
+}
+
+_find_first_sequential_zone() {
+	for ((idx =  NR_CONV_ZONES; idx < REPORTED_COUNT; idx++)); do
+		if [[ ${ZONE_TYPES[idx]} -eq ${ZONE_TYPE_SEQ_WRITE_REQUIRED} ]];
+		then
+			echo "${idx}"
+			return 0
+		fi
+	done
+	echo "Sequential write required zone not found"
+
+	return 1
+}
+
+test_device() {
+	local -i zone_idx
+	local -i offset
+
+	echo "Running ${TEST_NAME}"
+
+	_get_sysfs_variable "${TEST_DEV}" || return $?
+	_get_blkzone_report "${TEST_DEV}" || return $?
+
+	zone_idx=$(_find_first_sequential_zone) || return $?
+	offset=$((ZONE_STARTS[zone_idx] * 512))
+
+	blkzone reset -o "${ZONE_STARTS[zone_idx]}" "${TEST_DEV}"
+
+	: "${TIMEOUT:=30}"
+	set_scheduler "$(basename "$(readlink -f "${TEST_DEV}")")" deadline
+	FIO_PERF_FIELDS=("write io" "write iops")
+	_fio_perf --filename="${TEST_DEV}" --name zbdwo --rw=write --direct=1 \
+		  --ioengine=libaio --iodepth=128 --bs=256k \
+		  --offset="${offset}"
+
+	_put_blkzone_report
+	_put_sysfs_variable
+
+	echo "Test complete"
+}
diff --git a/tests/zbd/005.out b/tests/zbd/005.out
new file mode 100644
index 0000000..3ff78c6
--- /dev/null
+++ b/tests/zbd/005.out
@@ -0,0 +1,2 @@
+Running zbd/005
+Test complete
-- 
2.20.1


^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function
  2019-01-09  1:35 ` [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function Damien Le Moal
@ 2019-01-09  8:46   ` Johannes Thumshirn
  0 siblings, 0 replies; 16+ messages in thread
From: Johannes Thumshirn @ 2019-01-09  8:46 UTC (permalink / raw)
  To: Damien Le Moal, linux-block, Omar Sandoval, Masato Suzuki,
	Shinichiro Kawasaki
  Cc: Omar Sandoval, Jens Axboe, Matias Bjorling, Hannes Reinecke,
	Mike Snitzer, Martin K . Petersen

On 09/01/2019 02:35, Damien Le Moal wrote:
> From: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
> +_test_dev_is_zoned() {
> +	local zoned_file="${TEST_DEV_SYSFS}/queue/zoned"
> +	if grep -q -e "none" "${zoned_file}" ; then

Nit: I think we can leave the zoned_file variable out
	if grep -qe "none" "${TEST_DEV_SYSFS}/queue/zoned"; then
	[...]
works as well and is a bit more descriptive


-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, back to index

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-01-09  1:35 [PATCH blktests 00/14] Implement zoned block device support Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 01/14] config: Introduce ZONED variable Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 02/14] common: Introduce _test_dev_is_zoned() helper function Damien Le Moal
2019-01-09  8:46   ` Johannes Thumshirn
2019-01-09  1:35 ` [PATCH blktests 03/14] common: Move set_scheduler() function from multipath-over-rdma to rc Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 04/14] block/004: Adjust fio conditions for zoned block device Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 05/14] block/013: Skip for zoned block devices Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 06/14] block/018,024: Skip when ZONED is set Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 07/14] check: Introduce group_exit() function Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 08/14] src: Introduce zbdioctl program Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 09/14] tests: Introduce zbd test group Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 10/14] zbd/001: sysfs and ioctl consistency test Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 11/14] zbd/002: report zone test Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 12/14] zbd/003: Test sequential zones reset Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 13/14] zbd/004: Check write split accross sequential zones Damien Le Moal
2019-01-09  1:35 ` [PATCH blktests 14/14] zbd/005: Test write ordering Damien Le Moal

Linux-Block Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \
		linux-block@vger.kernel.org linux-block@archiver.kernel.org
	public-inbox-index linux-block


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block


AGPL code for this site: git clone https://public-inbox.org/ public-inbox