[PATCH 0/5] fstests: add checks for testing zoned btrfs

[PATCH 0/5] fstests: add checks for testing zoned btrfs

[Naohiro Aota]

Several tests are failing on zoned btrfs, but actually they are
invalid. There are two reasons of the failures. One is creating too
small filesystem. Since zoned btrfs needs at lease 5 zones (= 1.25 GB
if zone size = 256MB) to create a filesystem, tests creating e.g., 1
GB filesystem will fail.

The other reason is lacking of zone support of some dm targets and
loop device. So, they need to skip the test if the testing device is
zoned.

Patches 1 to 3 handle the too small file system failure.

And, patches 4 and 5 add checks for tests requiring non-zoned devices.

Naohiro Aota (5):
  common/rc: introduce minimal fs size check
  btrfs/057: use _scratch_mkfs_sized to set filesystem size
  btrfs: add minimal file system size check
  common: add zoned block device checks
  shared/032: add check for zoned block device

 README            |  4 ++++
 common/dmerror    |  3 +++
 common/dmhugedisk |  3 +++
 common/rc         | 15 +++++++++++++++
 tests/btrfs/057   |  2 +-
 tests/btrfs/141   |  1 +
 tests/btrfs/142   |  1 +
 tests/btrfs/143   |  1 +
 tests/btrfs/150   |  1 +
 tests/btrfs/151   |  1 +
 tests/btrfs/156   |  1 +
 tests/btrfs/157   |  1 +
 tests/btrfs/158   |  1 +
 tests/btrfs/175   |  1 +
 tests/shared/032  |  2 ++
 15 files changed, 37 insertions(+), 1 deletion(-)

-- 
2.31.1

[PATCH 1/5] common/rc: introduce minimal fs size check

[Naohiro Aota]

_scratch_mkfs_sized() create a file system with specified size
limit. It can, however, too small for certain kind of devices. For
example, zoned btrfs requires at least 5 zones to make a file system.

This commit introduces MIN_FSSIZE, which specify the minimum size of the
possible file system. We can set this variable e.g. $ZONE_SIZE *
$MIN_ZONE_COUNT.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
---
 README    |  4 ++++
 common/rc | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/README b/README
index 048491a68838..a7888281aa09 100644
--- a/README
+++ b/README
@@ -117,6 +117,10 @@ Preparing system for tests:
                name of a file to compress; and it must accept '-d -f -k' and
                the name of a file to decompress.  In other words, it must
                emulate gzip.
+	     - Set MIN_FSSIZE to specify the minimal size (bytes) of a
+               filesystem we can create. Setting this parameter will
+               skip the tests creating a filesystem less than
+               MIN_FSSIZE.
 
         - or add a case to the switch in common/config assigning
           these variables based on the hostname of your test
diff --git a/common/rc b/common/rc
index b18cf61e8a96..9a8458d4a3b6 100644
--- a/common/rc
+++ b/common/rc
@@ -956,6 +956,16 @@ _available_memory_bytes()
 	fi
 }
 
+_check_minimal_fs_size()
+{
+	local fssize=$1
+
+	if [ -n "$MIN_FSSIZE" ]; then
+		[ $MIN_FSSIZE -gt "$fssize" ] &&
+			_notrun "specified filesystem size is too small"
+	fi
+}
+
 # Create fs of certain size on scratch device
 # _scratch_mkfs_sized <size in bytes> [optional blocksize]
 _scratch_mkfs_sized()
@@ -989,6 +999,8 @@ _scratch_mkfs_sized()
 
 	local blocks=`expr $fssize / $blocksize`
 
+	_check_minimal_fs_size $fssize
+
 	if [ -b "$SCRATCH_DEV" ]; then
 		local devsize=`blockdev --getsize64 $SCRATCH_DEV`
 		[ "$fssize" -gt "$devsize" ] && _notrun "Scratch device too small"
-- 
2.31.1

[PATCH 2/5] btrfs/057: use _scratch_mkfs_sized to set filesystem size

[Naohiro Aota]

Btrfs/057 is using _scratch_mkfs directly to set filesystem size. This
can be _scratch_mkfs_sized instead, to go through several
checks (e.g., minimal filesystem size check).

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
---
 tests/btrfs/057 | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/btrfs/057 b/tests/btrfs/057
index bf9ab14a89b2..e2c651911ded 100755
--- a/tests/btrfs/057
+++ b/tests/btrfs/057
@@ -31,7 +31,7 @@ _require_scratch
 
 rm -f $seqres.full
 
-run_check _scratch_mkfs "-b 1g"
+_scratch_mkfs_sized $((1024 * 1024 * 1024))
 
 _scratch_mount
 
-- 
2.31.1

[PATCH 3/5] btrfs: add minimal file system size check

[Naohiro Aota]

Some btrfs tests call _scratch_pool_mkfs or _scratch_mkfs by themselves to
specify file system size limit. It slips through the check in
_scratch_mkfs_sized(). Let's add size check call for each of them.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
---
 tests/btrfs/141 | 1 +
 tests/btrfs/142 | 1 +
 tests/btrfs/143 | 1 +
 tests/btrfs/150 | 1 +
 tests/btrfs/151 | 1 +
 tests/btrfs/156 | 1 +
 tests/btrfs/157 | 1 +
 tests/btrfs/158 | 1 +
 tests/btrfs/175 | 1 +
 9 files changed, 9 insertions(+)

diff --git a/tests/btrfs/141 b/tests/btrfs/141
index bc4ba52d011d..409cb8ee4099 100755
--- a/tests/btrfs/141
+++ b/tests/btrfs/141
@@ -71,6 +71,7 @@ _scratch_dev_pool_get 2
 # step 1, create a raid1 btrfs which contains one 128k file.
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 mkfs_opts="-d raid1 -b 1G"
 _scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
 
diff --git a/tests/btrfs/142 b/tests/btrfs/142
index c8660fd60e5e..e7170cb4e20b 100755
--- a/tests/btrfs/142
+++ b/tests/btrfs/142
@@ -49,6 +49,7 @@ _scratch_dev_pool_get 2
 # step 1, create a raid1 btrfs which contains one 128k file.
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 mkfs_opts="-d raid1 -b 1G"
 _scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
 
diff --git a/tests/btrfs/143 b/tests/btrfs/143
index 88fdbe60c123..399146063c40 100755
--- a/tests/btrfs/143
+++ b/tests/btrfs/143
@@ -56,6 +56,7 @@ _scratch_dev_pool_get 2
 # step 1, create a raid1 btrfs which contains one 128k file.
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 mkfs_opts="-d raid1 -b 1G"
 _scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
 
diff --git a/tests/btrfs/150 b/tests/btrfs/150
index 0840e0a0cc7d..12826dc0469e 100755
--- a/tests/btrfs/150
+++ b/tests/btrfs/150
@@ -59,6 +59,7 @@ disable_io_failure()
 	echo 0 > $SYSFS_BDEV/make-it-fail
 }
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 _scratch_pool_mkfs "-d raid1 -b 1G" >> $seqres.full 2>&1
 
 # It doesn't matter which compression algorithm we use.
diff --git a/tests/btrfs/151 b/tests/btrfs/151
index cd18f895fd69..d363874f491a 100755
--- a/tests/btrfs/151
+++ b/tests/btrfs/151
@@ -39,6 +39,7 @@ _supported_fs btrfs
 _require_scratch
 _require_scratch_dev_pool 3
 _require_btrfs_dev_del_by_devid
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 
 # We need exactly 3 disks to form a fixed stripe layout for this test.
 _scratch_dev_pool_get 3
diff --git a/tests/btrfs/156 b/tests/btrfs/156
index 89c80e7161e2..9126dbab1dde 100755
--- a/tests/btrfs/156
+++ b/tests/btrfs/156
@@ -53,6 +53,7 @@ nr_files=$(( $fs_size / $file_size / 2))
 # Force to use single data and meta profile.
 # Since the test relies on fstrim output, which will differ for different
 # profiles
+_check_minimal_fs_size $fs_size
 _scratch_mkfs -b $fs_size -m single -d single > /dev/null
 _scratch_mount
 
diff --git a/tests/btrfs/157 b/tests/btrfs/157
index e7118dbcad0f..e90349b8ce16 100755
--- a/tests/btrfs/157
+++ b/tests/btrfs/157
@@ -76,6 +76,7 @@ _scratch_dev_pool_get 4
 # step 1: create a raid6 btrfs and create a 128K file
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 mkfs_opts="-d raid6 -b 1G"
 _scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
 
diff --git a/tests/btrfs/158 b/tests/btrfs/158
index 803d6b9ea37f..e5bf2f102e15 100755
--- a/tests/btrfs/158
+++ b/tests/btrfs/158
@@ -68,6 +68,7 @@ _scratch_dev_pool_get 4
 # step 1: create a raid6 btrfs and create a 128K file
 echo "step 1......mkfs.btrfs" >>$seqres.full
 
+_check_minimal_fs_size $(( 1024 * 1024 * 1024 ))
 mkfs_opts="-d raid6 -b 1G"
 _scratch_pool_mkfs $mkfs_opts >>$seqres.full 2>&1
 
diff --git a/tests/btrfs/175 b/tests/btrfs/175
index 94f56284a717..75704c43e27d 100755
--- a/tests/btrfs/175
+++ b/tests/btrfs/175
@@ -29,6 +29,7 @@ rm -f $seqres.full
 _supported_fs btrfs
 _require_scratch_dev_pool 2
 _require_scratch_swapfile
+_check_minimal_fs_size $((1024 * 1024 * 1024))
 
 cycle_swapfile() {
 	local sz=${1:-$(($(get_page_size) * 10))}
-- 
2.31.1

[PATCH 4/5] common: add zoned block device checks

[Naohiro Aota]

dm-error and dm-snapshot does not have DM_TARGET_ZONED_HM nor
DM_TARGET_MIXED_ZONED_MODEL feature and does not implement
.report_zones(). So, it cannot pass the zone information from the down
layer (zoned device) to the upper layer.

Loop device also cannot pass the zone information.

This patch requires non-zoned block device for the tests using these
ones.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
---
 common/dmerror    | 3 +++
 common/dmhugedisk | 3 +++
 common/rc         | 3 +++
 3 files changed, 9 insertions(+)

diff --git a/common/dmerror b/common/dmerror
index 7d12e0a1843c..84d2c338df11 100644
--- a/common/dmerror
+++ b/common/dmerror
@@ -15,6 +15,9 @@ _dmerror_setup()
 	DMLINEAR_TABLE="0 $blk_dev_size linear $dm_backing_dev 0"
 
 	DMERROR_TABLE="0 $blk_dev_size error $dm_backing_dev 0"
+
+	# dm-error cannot handle zone information
+	_require_non_zoned_device "${dm_backing_dev}"
 }
 
 _dmerror_init()
diff --git a/common/dmhugedisk b/common/dmhugedisk
index 502f0243772d..715f95efde29 100644
--- a/common/dmhugedisk
+++ b/common/dmhugedisk
@@ -16,6 +16,9 @@ _dmhugedisk_init()
 	local dm_backing_dev=$SCRATCH_DEV
 	local chunk_size="$2"
 
+	# We cannot ensure sequential writes on the backing device
+	_require_non_zoned_device $dm_backing_dev
+
 	if [ -z "$chunk_size" ]; then
 		chunk_size=512
 	fi
diff --git a/common/rc b/common/rc
index 9a8458d4a3b6..55cd8e8df8f6 100644
--- a/common/rc
+++ b/common/rc
@@ -1812,6 +1812,9 @@ _require_loop()
     else
 	_notrun "This test requires loopback device support"
     fi
+
+    # loop device does not handle zone information
+    _require_non_zoned_device ${TEST_DEV}
 }
 
 # this test requires kernel support for a secondary filesystem
-- 
2.31.1

[PATCH 5/5] shared/032: add check for zoned block device

[Naohiro Aota]

Mkfs on zoned block device won't work on most filesystem. Let's
disable the test.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
---
 tests/shared/032 | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/shared/032 b/tests/shared/032
index 360087ee1fb9..31ac52303dde 100755
--- a/tests/shared/032
+++ b/tests/shared/032
@@ -25,6 +25,8 @@ _supported_fs xfs btrfs
 
 _require_scratch_nocheck
 _require_no_large_scratch_dev
+# not all the FS support zoned block device
+_require_non_zoned_device "${SCRATCH_DEV}"
 
 # mkfs.btrfs did not have overwrite detection at first
 if [ "$FSTYP" == "btrfs" ]; then
-- 
2.31.1

Re: [PATCH 0/5] fstests: add checks for testing zoned btrfs

[Johannes Thumshirn]

On 21/05/2021 06:58, Naohiro Aota wrote:
> Several tests are failing on zoned btrfs, but actually they are
> invalid. There are two reasons of the failures. One is creating too
> small filesystem. Since zoned btrfs needs at lease 5 zones (= 1.25 GB
> if zone size = 256MB) to create a filesystem, tests creating e.g., 1
> GB filesystem will fail.
> 
> The other reason is lacking of zone support of some dm targets and
> loop device. So, they need to skip the test if the testing device is
> zoned.
> 
> Patches 1 to 3 handle the too small file system failure.
> 
> And, patches 4 and 5 add checks for tests requiring non-zoned devices.
> 
> Naohiro Aota (5):
>   common/rc: introduce minimal fs size check
>   btrfs/057: use _scratch_mkfs_sized to set filesystem size
>   btrfs: add minimal file system size check
>   common: add zoned block device checks
>   shared/032: add check for zoned block device
> 
>  README            |  4 ++++
>  common/dmerror    |  3 +++
>  common/dmhugedisk |  3 +++
>  common/rc         | 15 +++++++++++++++
>  tests/btrfs/057   |  2 +-
>  tests/btrfs/141   |  1 +
>  tests/btrfs/142   |  1 +
>  tests/btrfs/143   |  1 +
>  tests/btrfs/150   |  1 +
>  tests/btrfs/151   |  1 +
>  tests/btrfs/156   |  1 +
>  tests/btrfs/157   |  1 +
>  tests/btrfs/158   |  1 +
>  tests/btrfs/175   |  1 +
>  tests/shared/032  |  2 ++
>  15 files changed, 37 insertions(+), 1 deletion(-)
> 

For the whole series:
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Tested-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>

Re: [PATCH 0/5] fstests: add checks for testing zoned btrfs

[Johannes Thumshirn]

On 21/05/2021 12:17, Johannes Thumshirn wrote:
> On 21/05/2021 06:58, Naohiro Aota wrote:
>> Several tests are failing on zoned btrfs, but actually they are
>> invalid. There are two reasons of the failures. One is creating too
>> small filesystem. Since zoned btrfs needs at lease 5 zones (= 1.25 GB
>> if zone size = 256MB) to create a filesystem, tests creating e.g., 1
>> GB filesystem will fail.
>>
>> The other reason is lacking of zone support of some dm targets and
>> loop device. So, they need to skip the test if the testing device is
>> zoned.
>>
>> Patches 1 to 3 handle the too small file system failure.
>>
>> And, patches 4 and 5 add checks for tests requiring non-zoned devices.
>>
>> Naohiro Aota (5):
>>   common/rc: introduce minimal fs size check
>>   btrfs/057: use _scratch_mkfs_sized to set filesystem size
>>   btrfs: add minimal file system size check
>>   common: add zoned block device checks
>>   shared/032: add check for zoned block device
>>
>>  README            |  4 ++++
>>  common/dmerror    |  3 +++
>>  common/dmhugedisk |  3 +++
>>  common/rc         | 15 +++++++++++++++
>>  tests/btrfs/057   |  2 +-
>>  tests/btrfs/141   |  1 +
>>  tests/btrfs/142   |  1 +
>>  tests/btrfs/143   |  1 +
>>  tests/btrfs/150   |  1 +
>>  tests/btrfs/151   |  1 +
>>  tests/btrfs/156   |  1 +
>>  tests/btrfs/157   |  1 +
>>  tests/btrfs/158   |  1 +
>>  tests/btrfs/175   |  1 +
>>  tests/shared/032  |  2 ++
>>  15 files changed, 37 insertions(+), 1 deletion(-)
>>
> 
> For the whole series:
> Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> Tested-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
> 

Spoke too early, I've only tested with a zoned device. I get the 
following splat with non zoned btrfs:

btrfs/146       - output mismatch (see /home/johannes/src/xfstests-dev/results//btrfs/146.out.bad)                                                                                                                                                                                                                                                                          
    --- tests/btrfs/146.out     2020-01-07 15:49:53.000000000 +0000                                                                                                                                                                                                                                                                                                         
    +++ /home/johannes/src/xfstests-dev/results//btrfs/146.out.bad      2021-05-21 09:56:43.963914581 +0000                                                                                                                                                                                                                                                                 
    @@ -1,3 +1,5 @@                                                                                                                                                                                                                                                                                                                                                         
     QA output created by 146                                                                                                                                                                                                                                                                                                                                               
     Format and mount                                                                                                                                                                                                                                                                                                                                                       
    +./common/dmerror: line 20: _require_non_zoned_device: command not found                                                                                                                                                                                                                                                                                                
    +./common/dmerror: line 20: _require_non_zoned_device: command not found                                                                                                                                                                                                                                                                                                
     Test passed!                                                                                                                                                                                                                                                                                                                                                           
    ...                                                                                                                                                                                                                                                                                                                                                                     
    (Run 'diff -u /home/johannes/src/xfstests-dev/tests/btrfs/146.out /home/johannes/src/xfstests-dev/results//btrfs/146.out.bad'  to see the entire diff)