[PATCHSET 0/3] fstests: updates for xfsprogs 5.15

[PATCHSET 0/3] fstests: updates for xfsprogs 5.15

[Darrick J. Wong]

Hi all,

This series contains test updates and fixes for things that were fixed
in 5.15.  Specifically, we now let the xfs_scrub binary tell us if it
supports Unicode checking (instead of grepping the binary!); test for a
regression in mkfs config file support prior to this release; and have
some adjustments for setting the minimum log size to 64MB>

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=xfsprogs-merge-5.15
---
 common/rc         |   12 +++++++++
 tests/xfs/216.out |   14 +++++------
 tests/xfs/831     |   68 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/831.out |    2 ++
 4 files changed, 89 insertions(+), 7 deletions(-)
 create mode 100755 tests/xfs/831
 create mode 100644 tests/xfs/831.out

[PATCH 1/3] common/rc: let xfs_scrub tell us about its unicode checker

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that xfs_scrub can report whether or not it was built with the
Unicode name checker, rewrite _check_xfs_scrub_does_unicode to take
advantage of that.  This supersedes the old method of trying to observe
dynamic library linkages and grepping the binary, neither of which
worked very well.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc |   12 ++++++++++++
 1 file changed, 12 insertions(+)


diff --git a/common/rc b/common/rc
index 17629801..ec146c4e 100644
--- a/common/rc
+++ b/common/rc
@@ -4800,6 +4800,18 @@ _check_xfs_scrub_does_unicode() {
 
 	_supports_xfs_scrub "${mount}" "${dev}" || return 1
 
+	# Newer versions of xfs_scrub advertise whether or not it supports
+	# Unicode name checks.
+	local xfs_scrub_ver="$("${XFS_SCRUB_PROG}" -VV)"
+
+	if echo "${xfs_scrub_ver}" | grep -q -- '-Unicode'; then
+		return 1
+	fi
+
+	if echo "${xfs_scrub_ver}" | grep -q -- '+Unicode'; then
+		return 0
+	fi
+
 	# If the xfs_scrub binary contains the string "Unicode name.*%s", then
 	# we know that it has the ability to complain about improper Unicode
 	# names.

[PATCH 2/3] xfs: test mkfs.xfs config file stack corruption issues

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Add a new regression test for a stack corruption problem uncovered in
the mkfs config file parsing code.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/831     |   68 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/xfs/831.out |    2 ++
 2 files changed, 70 insertions(+)
 create mode 100755 tests/xfs/831
 create mode 100644 tests/xfs/831.out


diff --git a/tests/xfs/831 b/tests/xfs/831
new file mode 100755
index 00000000..a73f14ff
--- /dev/null
+++ b/tests/xfs/831
@@ -0,0 +1,68 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022 Oracle.  All Rights Reserved.
+#
+# FS QA Test 831
+#
+# Regression test for xfsprogs commit:
+#
+# 99c78777 ("mkfs: prevent corruption of passed-in suboption string values")
+#
+. ./common/preamble
+_begin_fstest auto quick mkfs
+
+_cleanup()
+{
+	rm -f $TEST_DIR/fubar.img
+	cd /
+	rm -r -f $tmp.*
+}
+
+# Import common functions.
+# . ./common/filter
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs xfs
+_require_test
+_require_xfs_mkfs_cfgfile
+
+# Set up a configuration file with an exact block size and log stripe unit
+# so that mkfs won't complain about having to correct the log stripe unit
+# size that is implied by the provided data device stripe unit.
+cfgfile=$tmp.cfg
+cat << EOF >> $tmp.cfg
+[block]
+size=4096
+
+[data]
+su=2097152
+sw=1
+EOF
+
+# Some mkfs options store the user's value string for processing after certain
+# geometry parameters (e.g. the fs block size) have been settled.  This is how
+# the su= option can accept arguments such as "8b" to mean eight filesystem
+# blocks.
+#
+# Unfortunately, on Ubuntu 20.04, the libini parser uses an onstack char[]
+# array to store value that it parse, and it passes the address of this array
+# to the parse_cfgopt.  The getstr function returns its argument, which is
+# stored in the cli_params structure by the D_SU parsing code.  By the time we
+# get around to interpreting this string, of course, the stack array has long
+# since lost scope and is now full of garbage.  If we're lucky, the value will
+# cause a number interpretation failure.  If not, the fs is configured with
+# garbage geometry.
+#
+# Either way, set up a config file to exploit this vulnerability so that we
+# can prove that current mkfs works correctly.
+$XFS_IO_PROG -f -c "truncate 1g" $TEST_DIR/fubar.img
+options=(-c options=$cfgfile -l sunit=8 -f -N $TEST_DIR/fubar.img)
+$MKFS_XFS_PROG "${options[@]}" >> $seqres.full ||
+	echo "mkfs failed"
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/xfs/831.out b/tests/xfs/831.out
new file mode 100644
index 00000000..abe137e3
--- /dev/null
+++ b/tests/xfs/831.out
@@ -0,0 +1,2 @@
+QA output created by 831
+Silence is golden

[PATCH 3/3] xfs/216: handle larger log sizes

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

mkfs will soon refuse to format a log smaller than 64MB, so update this
test to reflect the new log sizing calculations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/216.out |   14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)


diff --git a/tests/xfs/216.out b/tests/xfs/216.out
index cbd7b652..3c12085f 100644
--- a/tests/xfs/216.out
+++ b/tests/xfs/216.out
@@ -1,10 +1,10 @@
 QA output created by 216
-fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
-fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
-fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
-fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
-fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
-fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
-fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
+fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
 fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
 fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2

Re: [PATCH 3/3] xfs/216: handle larger log sizes

[Zorro Lang]

On Mon, Apr 11, 2022 at 03:55:13PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> mkfs will soon refuse to format a log smaller than 64MB, so update this
> test to reflect the new log sizing calculations.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/216.out |   14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 
> 
> diff --git a/tests/xfs/216.out b/tests/xfs/216.out
> index cbd7b652..3c12085f 100644
> --- a/tests/xfs/216.out
> +++ b/tests/xfs/216.out
> @@ -1,10 +1,10 @@
>  QA output created by 216
> -fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
> -fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
> -fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
> -fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
> -fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
> -fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
> -fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
> +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2

So this will break downstream kernel testing too, except it follows this new
xfs behavior change. Is it possible to get the minimal log size, then help to
avoid the failure (if it won't mess up the code:)?

>  fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
>  fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> 

Re: [PATCH 2/3] xfs: test mkfs.xfs config file stack corruption issues

[Zorro Lang]

On Mon, Apr 11, 2022 at 03:55:08PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Add a new regression test for a stack corruption problem uncovered in
> the mkfs config file parsing code.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---

Good to me,
Reviewed-by: Zorro Lang <zlang@redhat.com>

>  tests/xfs/831     |   68 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tests/xfs/831.out |    2 ++
>  2 files changed, 70 insertions(+)
>  create mode 100755 tests/xfs/831
>  create mode 100644 tests/xfs/831.out
> 
> 
> diff --git a/tests/xfs/831 b/tests/xfs/831
> new file mode 100755
> index 00000000..a73f14ff
> --- /dev/null
> +++ b/tests/xfs/831
> @@ -0,0 +1,68 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022 Oracle.  All Rights Reserved.
> +#
> +# FS QA Test 831
> +#
> +# Regression test for xfsprogs commit:
> +#
> +# 99c78777 ("mkfs: prevent corruption of passed-in suboption string values")
> +#
> +. ./common/preamble
> +_begin_fstest auto quick mkfs
> +
> +_cleanup()
> +{
> +	rm -f $TEST_DIR/fubar.img
> +	cd /
> +	rm -r -f $tmp.*
> +}
> +
> +# Import common functions.
> +# . ./common/filter
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs xfs
> +_require_test
> +_require_xfs_mkfs_cfgfile
> +
> +# Set up a configuration file with an exact block size and log stripe unit
> +# so that mkfs won't complain about having to correct the log stripe unit
> +# size that is implied by the provided data device stripe unit.
> +cfgfile=$tmp.cfg
> +cat << EOF >> $tmp.cfg
> +[block]
> +size=4096
> +
> +[data]
> +su=2097152
> +sw=1
> +EOF
> +
> +# Some mkfs options store the user's value string for processing after certain
> +# geometry parameters (e.g. the fs block size) have been settled.  This is how
> +# the su= option can accept arguments such as "8b" to mean eight filesystem
> +# blocks.
> +#
> +# Unfortunately, on Ubuntu 20.04, the libini parser uses an onstack char[]
> +# array to store value that it parse, and it passes the address of this array
> +# to the parse_cfgopt.  The getstr function returns its argument, which is
> +# stored in the cli_params structure by the D_SU parsing code.  By the time we
> +# get around to interpreting this string, of course, the stack array has long
> +# since lost scope and is now full of garbage.  If we're lucky, the value will
> +# cause a number interpretation failure.  If not, the fs is configured with
> +# garbage geometry.
> +#
> +# Either way, set up a config file to exploit this vulnerability so that we
> +# can prove that current mkfs works correctly.
> +$XFS_IO_PROG -f -c "truncate 1g" $TEST_DIR/fubar.img
> +options=(-c options=$cfgfile -l sunit=8 -f -N $TEST_DIR/fubar.img)
> +$MKFS_XFS_PROG "${options[@]}" >> $seqres.full ||
> +	echo "mkfs failed"
> +
> +# success, all done
> +echo Silence is golden
> +status=0
> +exit
> diff --git a/tests/xfs/831.out b/tests/xfs/831.out
> new file mode 100644
> index 00000000..abe137e3
> --- /dev/null
> +++ b/tests/xfs/831.out
> @@ -0,0 +1,2 @@
> +QA output created by 831
> +Silence is golden
> 

Re: [PATCH 1/3] common/rc: let xfs_scrub tell us about its unicode checker

[Zorro Lang]

On Mon, Apr 11, 2022 at 03:55:02PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Now that xfs_scrub can report whether or not it was built with the
> Unicode name checker, rewrite _check_xfs_scrub_does_unicode to take
> advantage of that.  This supersedes the old method of trying to observe
> dynamic library linkages and grepping the binary, neither of which
> worked very well.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---

Good to me,
Reviewed-by: Zorro Lang <zlang@redhat.com>

>  common/rc |   12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> 
> diff --git a/common/rc b/common/rc
> index 17629801..ec146c4e 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -4800,6 +4800,18 @@ _check_xfs_scrub_does_unicode() {
>  
>  	_supports_xfs_scrub "${mount}" "${dev}" || return 1
>  
> +	# Newer versions of xfs_scrub advertise whether or not it supports
> +	# Unicode name checks.
> +	local xfs_scrub_ver="$("${XFS_SCRUB_PROG}" -VV)"
> +
> +	if echo "${xfs_scrub_ver}" | grep -q -- '-Unicode'; then
> +		return 1
> +	fi
> +
> +	if echo "${xfs_scrub_ver}" | grep -q -- '+Unicode'; then
> +		return 0
> +	fi
> +
>  	# If the xfs_scrub binary contains the string "Unicode name.*%s", then
>  	# we know that it has the ability to complain about improper Unicode
>  	# names.
> 

Re: [PATCH 3/3] xfs/216: handle larger log sizes

[Darrick J. Wong]

On Thu, Apr 14, 2022 at 01:44:00AM +0800, Zorro Lang wrote:
> On Mon, Apr 11, 2022 at 03:55:13PM -0700, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > 
> > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > test to reflect the new log sizing calculations.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/216.out |   14 +++++++-------
> >  1 file changed, 7 insertions(+), 7 deletions(-)
> > 
> > 
> > diff --git a/tests/xfs/216.out b/tests/xfs/216.out
> > index cbd7b652..3c12085f 100644
> > --- a/tests/xfs/216.out
> > +++ b/tests/xfs/216.out
> > @@ -1,10 +1,10 @@
> >  QA output created by 216
> > -fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
> > -fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
> > -fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
> > -fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
> > -fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
> > -fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
> > -fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
> > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> 
> So this will break downstream kernel testing too, except it follows this new
> xfs behavior change. Is it possible to get the minimal log size, then help to
> avoid the failure (if it won't mess up the code:)?

Hmm.  I suppose we could do a .out.XXX switcheroo type thing, though I
don't know of a good way to detect which mkfs behavior you've got.

--D

> 
> >  fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> >  fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > 
> 

Re: [PATCH 3/3] xfs/216: handle larger log sizes

[Zorro Lang]

On Wed, Apr 13, 2022 at 06:51:49PM -0700, Darrick J. Wong wrote:
> On Thu, Apr 14, 2022 at 01:44:00AM +0800, Zorro Lang wrote:
> > On Mon, Apr 11, 2022 at 03:55:13PM -0700, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <djwong@kernel.org>
> > > 
> > > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > > test to reflect the new log sizing calculations.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  tests/xfs/216.out |   14 +++++++-------
> > >  1 file changed, 7 insertions(+), 7 deletions(-)
> > > 
> > > 
> > > diff --git a/tests/xfs/216.out b/tests/xfs/216.out
> > > index cbd7b652..3c12085f 100644
> > > --- a/tests/xfs/216.out
> > > +++ b/tests/xfs/216.out
> > > @@ -1,10 +1,10 @@
> > >  QA output created by 216
> > > -fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
> > > -fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
> > > -fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
> > > -fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
> > > -fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
> > > -fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
> > > -fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
> > > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> > 
> > So this will break downstream kernel testing too, except it follows this new
> > xfs behavior change. Is it possible to get the minimal log size, then help to
> > avoid the failure (if it won't mess up the code:)?
> 
> Hmm.  I suppose we could do a .out.XXX switcheroo type thing, though I
> don't know of a good way to detect which mkfs behavior you've got.

Don't need to take much time to handle it :) How about use a specified filter function,
filter all log blocks number <= 16384, if the number of blocks=$number <= 16384, transform
it to blocks=* or what anything else do you like ?

I think we don't really care how much the log size less than 64M, right? Just hope it
works (can be mounted and read/write)?

Thanks,
Zorro

> 
> --D
> 
> > 
> > >  fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> > >  fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > > 
> > 
> 

Re: [PATCH 3/3] xfs/216: handle larger log sizes

[Darrick J. Wong]

On Fri, Apr 15, 2022 at 03:25:31AM +0800, Zorro Lang wrote:
> On Wed, Apr 13, 2022 at 06:51:49PM -0700, Darrick J. Wong wrote:
> > On Thu, Apr 14, 2022 at 01:44:00AM +0800, Zorro Lang wrote:
> > > On Mon, Apr 11, 2022 at 03:55:13PM -0700, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > 
> > > > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > > > test to reflect the new log sizing calculations.
> > > > 
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  tests/xfs/216.out |   14 +++++++-------
> > > >  1 file changed, 7 insertions(+), 7 deletions(-)
> > > > 
> > > > 
> > > > diff --git a/tests/xfs/216.out b/tests/xfs/216.out
> > > > index cbd7b652..3c12085f 100644
> > > > --- a/tests/xfs/216.out
> > > > +++ b/tests/xfs/216.out
> > > > @@ -1,10 +1,10 @@
> > > >  QA output created by 216
> > > > -fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > -fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > -fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > -fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > -fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > -fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
> > > > -fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
> > > > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> > > 
> > > So this will break downstream kernel testing too, except it follows this new
> > > xfs behavior change. Is it possible to get the minimal log size, then help to
> > > avoid the failure (if it won't mess up the code:)?
> > 
> > Hmm.  I suppose we could do a .out.XXX switcheroo type thing, though I
> > don't know of a good way to detect which mkfs behavior you've got.
> 
> Don't need to take much time to handle it :) How about use a specified filter function,
> filter all log blocks number <= 16384, if the number of blocks=$number <= 16384, transform
> it to blocks=* or what anything else do you like ?
> 
> I think we don't really care how much the log size less than 64M, right? Just hope it
> works (can be mounted and read/write)?

<shrug> Well I already reworked this patch to create 216.out.64mblog and
216.out.classic, and symlink them to 216.out as appropriate given the
log size for a 512m log format, since it probably *is* a good idea to
make sure older mkfs doesn't stray.

--D

> Thanks,
> Zorro
> 
> > 
> > --D
> > 
> > > 
> > > >  fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> > > >  fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > > > 
> > > 
> > 
> 

Re: [PATCH 3/3] xfs/216: handle larger log sizes

[Zorro Lang]

On Thu, Apr 14, 2022 at 12:36:35PM -0700, Darrick J. Wong wrote:
> On Fri, Apr 15, 2022 at 03:25:31AM +0800, Zorro Lang wrote:
> > On Wed, Apr 13, 2022 at 06:51:49PM -0700, Darrick J. Wong wrote:
> > > On Thu, Apr 14, 2022 at 01:44:00AM +0800, Zorro Lang wrote:
> > > > On Mon, Apr 11, 2022 at 03:55:13PM -0700, Darrick J. Wong wrote:
> > > > > From: Darrick J. Wong <djwong@kernel.org>
> > > > > 
> > > > > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > > > > test to reflect the new log sizing calculations.
> > > > > 
> > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > > ---
> > > > >  tests/xfs/216.out |   14 +++++++-------
> > > > >  1 file changed, 7 insertions(+), 7 deletions(-)
> > > > > 
> > > > > 
> > > > > diff --git a/tests/xfs/216.out b/tests/xfs/216.out
> > > > > index cbd7b652..3c12085f 100644
> > > > > --- a/tests/xfs/216.out
> > > > > +++ b/tests/xfs/216.out
> > > > > @@ -1,10 +1,10 @@
> > > > >  QA output created by 216
> > > > > -fssize=1g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > > -fssize=2g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > > -fssize=4g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > > -fssize=8g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > > -fssize=16g log      =internal log           bsize=4096   blocks=2560, version=2
> > > > > -fssize=32g log      =internal log           bsize=4096   blocks=4096, version=2
> > > > > -fssize=64g log      =internal log           bsize=4096   blocks=8192, version=2
> > > > > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > 
> > > > So this will break downstream kernel testing too, except it follows this new
> > > > xfs behavior change. Is it possible to get the minimal log size, then help to
> > > > avoid the failure (if it won't mess up the code:)?
> > > 
> > > Hmm.  I suppose we could do a .out.XXX switcheroo type thing, though I
> > > don't know of a good way to detect which mkfs behavior you've got.
> > 
> > Don't need to take much time to handle it :) How about use a specified filter function,
> > filter all log blocks number <= 16384, if the number of blocks=$number <= 16384, transform
> > it to blocks=* or what anything else do you like ?
> > 
> > I think we don't really care how much the log size less than 64M, right? Just hope it
> > works (can be mounted and read/write)?
> 
> <shrug> Well I already reworked this patch to create 216.out.64mblog and
> 216.out.classic, and symlink them to 216.out as appropriate given the
> log size for a 512m log format, since it probably *is* a good idea to
> make sure older mkfs doesn't stray.

Thanks for doing this! I saw Dave has described the history reason[1] of this test, so
I think your new change on this patch is good to me. I've acked two other patches, I'll
give RVB to this one ASAP after you send the new version.

Thanks,
Zorro


[1]
05:58 < dchinner> sandeen: I'm betting that I wrote that test to ensure the changes I made in mkfs commit a6634fba3dec ("mkfs: allow to make larger logs") didn't change the log size for 
                  small filesystems
05:58 < dchinner> that was the commit that changed the max log size from 128MB to 2GB

> 
> --D
> 
> > Thanks,
> > Zorro
> > 
> > > 
> > > --D
> > > 
> > > > 
> > > > >  fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> > > > >  fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > > > > 
> > > > 
> > > 
> > 
> 

[PATCH v1.1 3/3] xfs/216: handle larger log sizes

[Darrick J. Wong]

mkfs will soon refuse to format a log smaller than 64MB, so update this
test to reflect the new log sizing calculations.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/216             |   19 +++++++++++++++++++
 tests/xfs/216.out.64mblog |   10 ++++++++++
 tests/xfs/216.out.classic |    0 
 3 files changed, 29 insertions(+)
 create mode 100644 tests/xfs/216.out.64mblog
 rename tests/xfs/{216.out => 216.out.classic} (100%)

diff --git a/tests/xfs/216 b/tests/xfs/216
index c3697db7..ebae8979 100755
--- a/tests/xfs/216
+++ b/tests/xfs/216
@@ -29,6 +29,23 @@ $MKFS_XFS_PROG 2>&1 | grep -q rmapbt && \
 $MKFS_XFS_PROG 2>&1 | grep -q reflink && \
 	loop_mkfs_opts="$loop_mkfs_opts -m reflink=0"
 
+# Decide which golden output file we're using.  Starting with mkfs.xfs 5.15,
+# the default minimum log size was raised to 64MB for all cases, so we detect
+# that by test-formatting with a 512M filesystem.  This is a little handwavy,
+# but it's the best we can do.
+choose_golden_output() {
+	local seqfull=$1
+	local file=$2
+
+	if $MKFS_XFS_PROG -f -b size=4096 -l version=2 \
+			-d name=$file,size=512m $loop_mkfs_opts | \
+			grep -q 'log.*blocks=16384'; then
+		ln -f -s $seqfull.out.64mblog $seqfull.out
+	else
+		ln -f -s $seqfull.out.classic $seqfull.out
+	fi
+}
+
 _do_mkfs()
 {
 	for i in $*; do
@@ -43,6 +60,8 @@ _do_mkfs()
 # make large holey file
 $XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV
 
+choose_golden_output $0 $LOOP_DEV
+
 #make loopback mount dir
 mkdir $LOOP_MNT
 
diff --git a/tests/xfs/216.out.64mblog b/tests/xfs/216.out.64mblog
new file mode 100644
index 00000000..3c12085f
--- /dev/null
+++ b/tests/xfs/216.out.64mblog
@@ -0,0 +1,10 @@
+QA output created by 216
+fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
+fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
diff --git a/tests/xfs/216.out b/tests/xfs/216.out.classic
similarity index 100%
rename from tests/xfs/216.out
rename to tests/xfs/216.out.classic

Re: [PATCH v1.1 3/3] xfs/216: handle larger log sizes

[Zorro Lang]

On Fri, Apr 15, 2022 at 08:04:58AM -0700, Darrick J. Wong wrote:
> mkfs will soon refuse to format a log smaller than 64MB, so update this
> test to reflect the new log sizing calculations.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/216             |   19 +++++++++++++++++++
>  tests/xfs/216.out.64mblog |   10 ++++++++++
>  tests/xfs/216.out.classic |    0 
>  3 files changed, 29 insertions(+)
>  create mode 100644 tests/xfs/216.out.64mblog
>  rename tests/xfs/{216.out => 216.out.classic} (100%)
> 
> diff --git a/tests/xfs/216 b/tests/xfs/216
> index c3697db7..ebae8979 100755
> --- a/tests/xfs/216
> +++ b/tests/xfs/216
> @@ -29,6 +29,23 @@ $MKFS_XFS_PROG 2>&1 | grep -q rmapbt && \
>  $MKFS_XFS_PROG 2>&1 | grep -q reflink && \
>  	loop_mkfs_opts="$loop_mkfs_opts -m reflink=0"
>  
> +# Decide which golden output file we're using.  Starting with mkfs.xfs 5.15,
> +# the default minimum log size was raised to 64MB for all cases, so we detect
> +# that by test-formatting with a 512M filesystem.  This is a little handwavy,
> +# but it's the best we can do.
> +choose_golden_output() {
> +	local seqfull=$1
> +	local file=$2
> +
> +	if $MKFS_XFS_PROG -f -b size=4096 -l version=2 \
> +			-d name=$file,size=512m $loop_mkfs_opts | \
> +			grep -q 'log.*blocks=16384'; then
> +		ln -f -s $seqfull.out.64mblog $seqfull.out
> +	else
> +		ln -f -s $seqfull.out.classic $seqfull.out
> +	fi

Actually there's a old common function in common/rc named _link_out_file(),
xfstests generally use it to deal with multiple .out files. It would be
better to keep in step with common helpers, but your "ln" command
isn't wrong :)

Reviewed-by: Zorro Lang <zlang@redhat.com>

> +}
> +
>  _do_mkfs()
>  {
>  	for i in $*; do
> @@ -43,6 +60,8 @@ _do_mkfs()
>  # make large holey file
>  $XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV
>  
> +choose_golden_output $0 $LOOP_DEV
> +
>  #make loopback mount dir
>  mkdir $LOOP_MNT
>  
> diff --git a/tests/xfs/216.out.64mblog b/tests/xfs/216.out.64mblog
> new file mode 100644
> index 00000000..3c12085f
> --- /dev/null
> +++ b/tests/xfs/216.out.64mblog
> @@ -0,0 +1,10 @@
> +QA output created by 216
> +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> +fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> diff --git a/tests/xfs/216.out b/tests/xfs/216.out.classic
> similarity index 100%
> rename from tests/xfs/216.out
> rename to tests/xfs/216.out.classic
> 

Re: [PATCH v1.1 3/3] xfs/216: handle larger log sizes

[Eryu Guan]

On Sat, Apr 16, 2022 at 09:35:18PM +0800, Zorro Lang wrote:
> On Fri, Apr 15, 2022 at 08:04:58AM -0700, Darrick J. Wong wrote:
> > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > test to reflect the new log sizing calculations.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  tests/xfs/216             |   19 +++++++++++++++++++
> >  tests/xfs/216.out.64mblog |   10 ++++++++++
> >  tests/xfs/216.out.classic |    0 
> >  3 files changed, 29 insertions(+)
> >  create mode 100644 tests/xfs/216.out.64mblog
> >  rename tests/xfs/{216.out => 216.out.classic} (100%)
> > 
> > diff --git a/tests/xfs/216 b/tests/xfs/216
> > index c3697db7..ebae8979 100755
> > --- a/tests/xfs/216
> > +++ b/tests/xfs/216
> > @@ -29,6 +29,23 @@ $MKFS_XFS_PROG 2>&1 | grep -q rmapbt && \
> >  $MKFS_XFS_PROG 2>&1 | grep -q reflink && \
> >  	loop_mkfs_opts="$loop_mkfs_opts -m reflink=0"
> >  
> > +# Decide which golden output file we're using.  Starting with mkfs.xfs 5.15,
> > +# the default minimum log size was raised to 64MB for all cases, so we detect
> > +# that by test-formatting with a 512M filesystem.  This is a little handwavy,
> > +# but it's the best we can do.
> > +choose_golden_output() {
> > +	local seqfull=$1
> > +	local file=$2
> > +
> > +	if $MKFS_XFS_PROG -f -b size=4096 -l version=2 \
> > +			-d name=$file,size=512m $loop_mkfs_opts | \
> > +			grep -q 'log.*blocks=16384'; then
> > +		ln -f -s $seqfull.out.64mblog $seqfull.out
> > +	else
> > +		ln -f -s $seqfull.out.classic $seqfull.out
> > +	fi
> 
> Actually there's a old common function in common/rc named _link_out_file(),
> xfstests generally use it to deal with multiple .out files. It would be
> better to keep in step with common helpers, but your "ln" command
> isn't wrong :)

I added tests/xfs/216.cfg file and updated test to use
_link_out_file_named().

> 
> Reviewed-by: Zorro Lang <zlang@redhat.com>

Thanks!
Eryu

> 
> > +}
> > +
> >  _do_mkfs()
> >  {
> >  	for i in $*; do
> > @@ -43,6 +60,8 @@ _do_mkfs()
> >  # make large holey file
> >  $XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV
> >  
> > +choose_golden_output $0 $LOOP_DEV
> > +
> >  #make loopback mount dir
> >  mkdir $LOOP_MNT
> >  
> > diff --git a/tests/xfs/216.out.64mblog b/tests/xfs/216.out.64mblog
> > new file mode 100644
> > index 00000000..3c12085f
> > --- /dev/null
> > +++ b/tests/xfs/216.out.64mblog
> > @@ -0,0 +1,10 @@
> > +QA output created by 216
> > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> > +fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > diff --git a/tests/xfs/216.out b/tests/xfs/216.out.classic
> > similarity index 100%
> > rename from tests/xfs/216.out
> > rename to tests/xfs/216.out.classic
> > 

Re: [PATCH v1.1 3/3] xfs/216: handle larger log sizes

[Zorro Lang]

On Sun, Apr 17, 2022 at 10:42:15PM +0800, Eryu Guan wrote:
> On Sat, Apr 16, 2022 at 09:35:18PM +0800, Zorro Lang wrote:
> > On Fri, Apr 15, 2022 at 08:04:58AM -0700, Darrick J. Wong wrote:
> > > mkfs will soon refuse to format a log smaller than 64MB, so update this
> > > test to reflect the new log sizing calculations.
> > > 
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  tests/xfs/216             |   19 +++++++++++++++++++
> > >  tests/xfs/216.out.64mblog |   10 ++++++++++
> > >  tests/xfs/216.out.classic |    0 
> > >  3 files changed, 29 insertions(+)
> > >  create mode 100644 tests/xfs/216.out.64mblog
> > >  rename tests/xfs/{216.out => 216.out.classic} (100%)
> > > 
> > > diff --git a/tests/xfs/216 b/tests/xfs/216
> > > index c3697db7..ebae8979 100755
> > > --- a/tests/xfs/216
> > > +++ b/tests/xfs/216
> > > @@ -29,6 +29,23 @@ $MKFS_XFS_PROG 2>&1 | grep -q rmapbt && \
> > >  $MKFS_XFS_PROG 2>&1 | grep -q reflink && \
> > >  	loop_mkfs_opts="$loop_mkfs_opts -m reflink=0"
> > >  
> > > +# Decide which golden output file we're using.  Starting with mkfs.xfs 5.15,
> > > +# the default minimum log size was raised to 64MB for all cases, so we detect
> > > +# that by test-formatting with a 512M filesystem.  This is a little handwavy,
> > > +# but it's the best we can do.
> > > +choose_golden_output() {
> > > +	local seqfull=$1
> > > +	local file=$2
> > > +
> > > +	if $MKFS_XFS_PROG -f -b size=4096 -l version=2 \
> > > +			-d name=$file,size=512m $loop_mkfs_opts | \
> > > +			grep -q 'log.*blocks=16384'; then
> > > +		ln -f -s $seqfull.out.64mblog $seqfull.out
> > > +	else
> > > +		ln -f -s $seqfull.out.classic $seqfull.out
> > > +	fi
> > 
> > Actually there's a old common function in common/rc named _link_out_file(),
> > xfstests generally use it to deal with multiple .out files. It would be
> > better to keep in step with common helpers, but your "ln" command
> > isn't wrong :)
> 
> I added tests/xfs/216.cfg file and updated test to use
> _link_out_file_named().

Thanks! Sorry have to trouble you, my request to kernel.org is still blocking.

`_link_out_file_named $seqfull.out 64mblog` is equal to
`_link_out_file 64mblog`

So the later one might be enough. Anyway, both are good to me!

Thanks,
Zorro

> 
> > 
> > Reviewed-by: Zorro Lang <zlang@redhat.com>
> 
> Thanks!
> Eryu
> 
> > 
> > > +}
> > > +
> > >  _do_mkfs()
> > >  {
> > >  	for i in $*; do
> > > @@ -43,6 +60,8 @@ _do_mkfs()
> > >  # make large holey file
> > >  $XFS_IO_PROG -f -c "truncate 256g" $LOOP_DEV
> > >  
> > > +choose_golden_output $0 $LOOP_DEV
> > > +
> > >  #make loopback mount dir
> > >  mkdir $LOOP_MNT
> > >  
> > > diff --git a/tests/xfs/216.out.64mblog b/tests/xfs/216.out.64mblog
> > > new file mode 100644
> > > index 00000000..3c12085f
> > > --- /dev/null
> > > +++ b/tests/xfs/216.out.64mblog
> > > @@ -0,0 +1,10 @@
> > > +QA output created by 216
> > > +fssize=1g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=2g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=4g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=8g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=16g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=32g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=64g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=128g log      =internal log           bsize=4096   blocks=16384, version=2
> > > +fssize=256g log      =internal log           bsize=4096   blocks=32768, version=2
> > > diff --git a/tests/xfs/216.out b/tests/xfs/216.out.classic
> > > similarity index 100%
> > > rename from tests/xfs/216.out
> > > rename to tests/xfs/216.out.classic
> > > 
>