[PATCH 0/2] Fix input value to _scratch_mkfs_sized

[PATCH 0/2] Fix input value to _scratch_mkfs_sized

[An Long]

Function _scratch_mkfs_sized cannot recognize the size descriptor.

For example, we set MKFS_OPTIONS="-b 4k" and then run generic/416 on
ext4, will fail with "mkfs.ext4: invalid block size - 4".

The [PATCH 2/2] based on [PATCH 1/2].

Full story is here:
https://lore.kernel.org/fstests/Yre%2F9qV62dtVyCvn@mit.edu/T/#t
https://lore.kernel.org/fstests/20220621044038.ppvdexgob3kzs46m@zlang-mailbox/T/#t

An Long (2):
  common/config: add _check_mkfs_block_options
  common/rc: fix input value to _scratch_mkfs_sized

 README        |  4 ++++
 common/config | 30 ++++++++++++++++++++++++++++++
 common/rc     | 16 +---------------
 3 files changed, 35 insertions(+), 15 deletions(-)


base-commit: 568ac9fffeb6afec03e5d6c9936617232fd7fc6d
-- 
2.35.3

[PATCH 1/2] common/config: add _check_mkfs_block_options

[An Long]

Add a helper to check MKFS_OPTIONS to use pure integer in bytes, and
create variable FS_BLOCK_SIZE=blocksize.
For example, set MKFS_OPTIONS="-b 4096" but not "-b 4k", then
FS_BLOCK_SIZE=4096.

Signed-off-by: An Long <lan@suse.com>
---
 README        |  4 ++++
 common/config | 30 ++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/README b/README
index 80d148be..e80b8457 100644
--- a/README
+++ b/README
@@ -241,6 +241,10 @@ Misc:
    this option is supported for all filesystems currently only -overlay is
    expected to run without issues. For other filesystems additional patches
    and fixes to the test suite might be needed.
+ - If MKFS_OPTIONS contains block size, the value must be an integer number of
+   bytes without units. And the variable FS_BLOCK_SIZE will be created.
+   For example, set MKFS_OPTIONS="-b 4096" but not "-b 4k", then
+   FS_BLOCK_SIZE=4096.
 
 ______________________
 USING THE FSQA SUITE
diff --git a/common/config b/common/config
index de3aba15..d34b1bf3 100644
--- a/common/config
+++ b/common/config
@@ -623,6 +623,34 @@ _check_device()
 	esac
 }
 
+# Check block size in $MKFS_OPTIONS is a valid integer
+# And then set the value in $FS_BLOCK_SIZE
+_check_mkfs_block_options()
+{
+        # Avoid that FS_BLOCK_SIZE is different from MKFS_OPTIONS
+        # Otherwise it will cause confusion
+        unset FS_BLOCK_SIZE
+
+        case $FSTYP in
+        xfs)
+                FS_BLOCK_SIZE=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+[a-zA-Z]?).*/\1/p'`
+                ;;
+        btrfs)
+                FS_BLOCK_SIZE=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+[a-zA-Z]?).*/\1/p'`
+                ;;
+        ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
+                FS_BLOCK_SIZE=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+[a-zA-Z]?).*/\1/p'`
+                ;;
+        jfs)
+                FS_BLOCK_SIZE=4096
+                ;;
+        esac
+
+        if [ -n "$FS_BLOCK_SIZE" ] && ! [[ $FS_BLOCK_SIZE =~ ^[0-9]+$ ]] ; then
+		_fatal "\$MKFS_OPTIONS: block size \"$FS_BLOCK_SIZE\" not an integer."
+        fi
+}
+
 # check and return a canonical mount point path
 _canonicalize_mountpoint()
 {
@@ -896,5 +924,7 @@ else
 	fi
 fi
 
+_check_mkfs_block_options
+
 # make sure this script returns success
 /bin/true
-- 
2.35.3

[PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized

[An Long]

_scratch_mkfs_sized only receive integer number of bytes as a valid
input. But if the MKFS_OPTIONS variable exists, it will use the value of
block size in MKFS_OPTIONS to override input. In case of
MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096. This
will give errors to ext2/3/4 etc, and brings potential bugs to xfs or
btrfs.

This patch depends on patch ("common/config: add _check_mkfs_block_options").

Signed-off-by: An Long <lan@suse.com>
---
 common/rc | 16 +---------------
 1 file changed, 1 insertion(+), 15 deletions(-)

diff --git a/common/rc b/common/rc
index 3c072c16..8b8c3b51 100644
--- a/common/rc
+++ b/common/rc
@@ -1036,21 +1036,7 @@ _scratch_mkfs_sized()
 	local blocksize=$2
 	local def_blksz
 
-	case $FSTYP in
-	xfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+).*/\1/p'`
-		;;
-	btrfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+).*/\1/p'`
-		;;
-	ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+).*/\1/p'`
-		;;
-	jfs)
-		def_blksz=4096
-		;;
-	esac
-
+	[ -n "$FS_BLOCK_SIZE" ] && def_blksz=$FS_BLOCK_SIZE
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
 	[ -z "$blocksize" ] && blocksize=4096
 
-- 
2.35.3

Re: [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized

[Long An]

On Wed, 2022-06-15 at 12:15 +0800, Zorro Lang wrote:
> On Tue, Jun 14, 2022 at 01:08:43PM +0800, An Long wrote:
> > _scratch_mkfs_sized only receive integer number of bytes as a valid
> > input. But if the MKFS_OPTIONS variable exists, it will use the
> > value of
> > block size in MKFS_OPTIONS to override input. In case of
> > MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096.
> > This
> 
> Hi An,
> 
> Good catch. Can you provide an example that a case fails on this
> issue, and
> then it tests passed after having this patch? I'm wondering why no
> one notice
> that before.
> 

For example, I set MKFS_OPTIONS="-b 4k" and then run generic/416 on
ext4 filesystem.

# ./check tests/generic/416
FSTYP         -- ext4
PLATFORM      -- Linux/x86_64 bogon 5.3.18-57-default #1 SMP Wed Apr 28
10:54:41 UTC 2021 (ba3c2e9)
MKFS_OPTIONS  -- -b 4k /dev/loop1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt/scratch

generic/416 90s ... [failed, exit status 1]- output mismatch (see
/home/lan/work/xfstests/results//generic/416.out.bad)
    --- tests/generic/416.out	2021-06-25 23:18:09.595254118 +0800
    +++ /home/lan/work/xfstests/results//generic/416.out.bad	2022-
06-15 16:33:19.584206316 +0800
    @@ -1,3 +1,4 @@
     QA output created by 416
    -wrote 16777216/16777216 bytes at offset 0
    -XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
    +mount: /mnt/scratch: wrong fs type, bad option, bad superblock on
/dev/loop1, missing codepage or helper program, or other error.
    +mount -o acl,user_xattr /dev/loop1 /mnt/scratch failed
    +(see /home/lan/work/xfstests/results//generic/416.full for
details)
    ...
    (Run 'diff -u /home/lan/work/xfstests/tests/generic/416.out
/home/lan/work/xfstests/results//generic/416.out.bad'  to see the
entire diff)
Ran: generic/416
Failures: generic/416
Failed 1 of 1 tests

But the more common result as below:
# ./check tests/generic/416
FSTYP         -- ext4
PLATFORM      -- Linux/x86_64 bogon 5.3.18-57-default #1 SMP Wed Apr 28
10:54:41 UTC 2021 (ba3c2e9)
MKFS_OPTIONS  -- -b 4k /dev/loop1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt/scratch

umount: /mnt/scratch: target is busy.
generic/416 77s ...  90s
Ran: generic/416
Passed all 1 tests

Test passed due to _require_scratch_nocheck() ignore the umount error.
However, "mkfs.ext4: invalid block size - 4" was found in the 416.full

Apart from this, the error only occurs when MKFS_OPTIONS is set, and
doesn't affect xfs or btrfs. Since ext4 only supports 4k blocksize on
most platforms, we usually don't set blocksize in MKFS_OPTIONS.

I think that's why this problem hasn't been found before. In fact, I
found this issue when I tested on ppc64 with 64k kernel and 4k
subblocksize fs.

> > will give errors to ext2/3/4 etc, and brings potential bugs to xfs
> > or
> > btrfs.
> > 
> > In addition, since we can receive various strings, so remove
> > integer
> > number check.
> > 
> > Signed-off-by: An Long <lan@suse.com>
> > ---
> >  common/rc | 18 ++++++------------
> >  1 file changed, 6 insertions(+), 12 deletions(-)
> > 
> > diff --git a/common/rc b/common/rc
> > index 22050bc2..026007d3 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -1077,7 +1077,7 @@ _parse_size_from_string()
> >  }
> >  
> >  # Create fs of certain size on scratch device
> > -# _scratch_mkfs_sized <size in bytes> [optional blocksize]
> > +# _scratch_mkfs_sized <size> [optional blocksize]
> >  _scratch_mkfs_sized()
> >  {
> >  	local fssize=$1
> > @@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
> >  
> >  	case $FSTYP in
> >  	xfs)
> > -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size=
> > ?+([0-9]+).*/\1/p'`
> > +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size=
> > ?+([0-9]+[a-zA-Z]?).*/\1/p'`
> >  		;;
> >  	btrfs)
> > -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-
> > 9]+).*/\1/p'`
> > +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-
> > 9]+[a-zA-Z]?).*/\1/p'`
> >  		;;
> >  	ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
> > -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-
> > 9]+).*/\1/p'`
> > +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-
> > 9]+[a-zA-Z]?).*/\1/p'`
> >  		;;
> >  	jfs)
> >  		def_blksz=4096
> > @@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
> >  
> >  	[ -n "$def_blksz" ] && blocksize=$def_blksz
> >  	[ -z "$blocksize" ] && blocksize=4096
> > -
> > -	local re='^[0-9]+$'
> > -	if ! [[ $fssize =~ $re ]] ; then
> > -		_notrun "error: _scratch_mkfs_sized: fs size
> > \"$fssize\" not an integer."
> > -	fi
> > -	if ! [[ $blocksize =~ $re ]] ; then
> > -		_notrun "error: _scratch_mkfs_sized: block size
> > \"$blocksize\" not an integer."
> > -	fi
> > +	blocksize=$(_parse_size_from_string $blocksize)
> > +	fssize=$(_parse_size_from_string $fssize)
> 
> For now, the _parse_size_from_string is only used for this patch, I
> think these
> two patches can be merged into one patch, as it trys to fix one
> single problem.
> 

There are a total of two logical changes. In particular it also include
a Function update. So it's better to keep two separate patch. But I'll
add patch depends on info to commit message.

Thank you for your time!

> Thanks,
> Zorro
> 
> >  
> >  	local blocks=`expr $fssize / $blocksize`
> >  
> > -- 
> > 2.35.3
> > 
-- 
An Long <lan@suse.com>
SUSE QE LSG, QE 2, Beijing

Re: [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized

[Zorro Lang]

On Tue, Jun 14, 2022 at 01:08:43PM +0800, An Long wrote:
> _scratch_mkfs_sized only receive integer number of bytes as a valid
> input. But if the MKFS_OPTIONS variable exists, it will use the value of
> block size in MKFS_OPTIONS to override input. In case of
> MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096. This

Hi An,

Good catch. Can you provide an example that a case fails on this issue, and
then it tests passed after having this patch? I'm wondering why no one notice
that before.

> will give errors to ext2/3/4 etc, and brings potential bugs to xfs or
> btrfs.
> 
> In addition, since we can receive various strings, so remove integer
> number check.
> 
> Signed-off-by: An Long <lan@suse.com>
> ---
>  common/rc | 18 ++++++------------
>  1 file changed, 6 insertions(+), 12 deletions(-)
> 
> diff --git a/common/rc b/common/rc
> index 22050bc2..026007d3 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1077,7 +1077,7 @@ _parse_size_from_string()
>  }
>  
>  # Create fs of certain size on scratch device
> -# _scratch_mkfs_sized <size in bytes> [optional blocksize]
> +# _scratch_mkfs_sized <size> [optional blocksize]
>  _scratch_mkfs_sized()
>  {
>  	local fssize=$1
> @@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
>  
>  	case $FSTYP in
>  	xfs)
> -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+).*/\1/p'`
> +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+[a-zA-Z]?).*/\1/p'`
>  		;;
>  	btrfs)
> -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+).*/\1/p'`
> +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+[a-zA-Z]?).*/\1/p'`
>  		;;
>  	ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
> -		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+).*/\1/p'`
> +		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+[a-zA-Z]?).*/\1/p'`
>  		;;
>  	jfs)
>  		def_blksz=4096
> @@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
>  
>  	[ -n "$def_blksz" ] && blocksize=$def_blksz
>  	[ -z "$blocksize" ] && blocksize=4096
> -
> -	local re='^[0-9]+$'
> -	if ! [[ $fssize =~ $re ]] ; then
> -		_notrun "error: _scratch_mkfs_sized: fs size \"$fssize\" not an integer."
> -	fi
> -	if ! [[ $blocksize =~ $re ]] ; then
> -		_notrun "error: _scratch_mkfs_sized: block size \"$blocksize\" not an integer."
> -	fi
> +	blocksize=$(_parse_size_from_string $blocksize)
> +	fssize=$(_parse_size_from_string $fssize)

For now, the _parse_size_from_string is only used for this patch, I think these
two patches can be merged into one patch, as it trys to fix one single problem.

Thanks,
Zorro

>  
>  	local blocks=`expr $fssize / $blocksize`
>  
> -- 
> 2.35.3
> 

[PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized

[An Long]

_scratch_mkfs_sized only receive integer number of bytes as a valid
input. But if the MKFS_OPTIONS variable exists, it will use the value of
block size in MKFS_OPTIONS to override input. In case of
MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096. This
will give errors to ext2/3/4 etc, and brings potential bugs to xfs or
btrfs.

In addition, since we can receive various strings, so remove integer
number check.

Signed-off-by: An Long <lan@suse.com>
---
 common/rc | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/common/rc b/common/rc
index 22050bc2..026007d3 100644
--- a/common/rc
+++ b/common/rc
@@ -1077,7 +1077,7 @@ _parse_size_from_string()
 }
 
 # Create fs of certain size on scratch device
-# _scratch_mkfs_sized <size in bytes> [optional blocksize]
+# _scratch_mkfs_sized <size> [optional blocksize]
 _scratch_mkfs_sized()
 {
 	local fssize=$1
@@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
 
 	case $FSTYP in
 	xfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+).*/\1/p'`
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+[a-zA-Z]?).*/\1/p'`
 		;;
 	btrfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+).*/\1/p'`
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+[a-zA-Z]?).*/\1/p'`
 		;;
 	ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
-		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+).*/\1/p'`
+		def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+[a-zA-Z]?).*/\1/p'`
 		;;
 	jfs)
 		def_blksz=4096
@@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
 
 	[ -n "$def_blksz" ] && blocksize=$def_blksz
 	[ -z "$blocksize" ] && blocksize=4096
-
-	local re='^[0-9]+$'
-	if ! [[ $fssize =~ $re ]] ; then
-		_notrun "error: _scratch_mkfs_sized: fs size \"$fssize\" not an integer."
-	fi
-	if ! [[ $blocksize =~ $re ]] ; then
-		_notrun "error: _scratch_mkfs_sized: block size \"$blocksize\" not an integer."
-	fi
+	blocksize=$(_parse_size_from_string $blocksize)
+	fssize=$(_parse_size_from_string $fssize)
 
 	local blocks=`expr $fssize / $blocksize`
 
-- 
2.35.3