[PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

With the change to CRCs by default, some tests are updated to call mkfs
with "-m crc=0" option directly, and this breaks testings on older
distros where mkfs.xfs doesn't have crc support.

Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
tweaks in _scratch_mkfs_xfs_opts() based on it.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---

This is based on Dave's commit "filter: inode size output of mkfs.xfs can change"

Tested on upstream kernel with upstream xfsprogs(v5 as default) and on RHEL7(v4
as default) and on RHEL6(no v5 support), all seem fine, no failures caused by
crc options.

 check         | 10 +++++++++-
 common/rc     |  5 +++++
 tests/xfs/073 |  4 +++-
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/check b/check
index 4fa96ed..568c05b 100755
--- a/check
+++ b/check
@@ -505,9 +505,17 @@ for section in $HOST_OPTIONS_SECTIONS; do
 
 	if [ ! -z "$SCRATCH_DEV" ]; then
 	  umount $SCRATCH_DEV 2>/dev/null
+
+	  # check if mkfs.xfs supports v5 xfs
+	  if [ "$FSTYP" == "xfs" ]; then
+		export XFS_MKFS_HAS_NO_META_SUPPORT=""
+		if ! _scratch_mkfs -m crc=0 >/dev/null 2>&1; then
+			export XFS_MKFS_HAS_NO_META_SUPPORT=true
+		fi
+	  fi
+
 	  # call the overridden mkfs - make sure the FS is built
 	  # the same as we'll create it later.
-
 	  if ! _scratch_mkfs $flag >$tmp.err 2>&1
 	  then
 	      echo "our local _scratch_mkfs routine ..."
diff --git a/common/rc b/common/rc
index 242dedb..6af7f14 100644
--- a/common/rc
+++ b/common/rc
@@ -308,6 +308,11 @@ _scratch_mkfs_xfs_opts()
 {
 	mkfs_opts=$*
 
+	# remove crc related mkfs options if mkfs.xfs doesn't support v5 support
+	if [ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
+		mkfs_opts=`echo $mkfs_opts | sed "s/-m\s\+crc=.//"`
+	fi
+
 	_scratch_options mkfs
 
 	$MKFS_XFS_PROG $SCRATCH_OPTIONS $mkfs_opts $SCRATCH_DEV
diff --git a/tests/xfs/073 b/tests/xfs/073
index 45a3fdf..acac8b8 100755
--- a/tests/xfs/073
+++ b/tests/xfs/073
@@ -156,7 +156,9 @@ _verify_copy $imgs.image $SCRATCH_DEV $SCRATCH_MNT
 
 echo 
 echo === copying scratch device to single target, large ro device
-${MKFS_XFS_PROG} -m crc=0 -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
+mkfs_crc_opts="-m crc=0"
+[ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ] && mkfs_crc_opts=""
+${MKFS_XFS_PROG} $mkfs_crc_opts -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
 rmdir $imgs.source_dir 2>/dev/null
 mkdir $imgs.source_dir
 
-- 
1.8.3.1

Re: [PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

On Sun, May 10, 2015 at 07:50:43PM +0800, Eryu Guan wrote:
> With the change to CRCs by default, some tests are updated to call mkfs
> with "-m crc=0" option directly, and this breaks testings on older
> distros where mkfs.xfs doesn't have crc support.
> 
> Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
> tweaks in _scratch_mkfs_xfs_opts() based on it.
> 
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
> 
> This is based on Dave's commit "filter: inode size output of mkfs.xfs can change"
> 
> Tested on upstream kernel with upstream xfsprogs(v5 as default) and on RHEL7(v4

Correction: tested with upstream xfsprogs with CRC default patch
applied, it was not there when I did the test.

Eryu

Re: [PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Dave Chinner]

On Sun, May 10, 2015 at 07:50:43PM +0800, Eryu Guan wrote:
> With the change to CRCs by default, some tests are updated to call mkfs
> with "-m crc=0" option directly, and this breaks testings on older
> distros where mkfs.xfs doesn't have crc support.
> 
> Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
> tweaks in _scratch_mkfs_xfs_opts() based on it.
> 
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
> 
> This is based on Dave's commit "filter: inode size output of mkfs.xfs can change"
> 
> Tested on upstream kernel with upstream xfsprogs(v5 as default) and on RHEL7(v4
> as default) and on RHEL6(no v5 support), all seem fine, no failures caused by
> crc options.
> 
>  check         | 10 +++++++++-
>  common/rc     |  5 +++++
>  tests/xfs/073 |  4 +++-
>  3 files changed, 17 insertions(+), 2 deletions(-)
> 
> diff --git a/check b/check
> index 4fa96ed..568c05b 100755
> --- a/check
> +++ b/check
> @@ -505,9 +505,17 @@ for section in $HOST_OPTIONS_SECTIONS; do
>  
>  	if [ ! -z "$SCRATCH_DEV" ]; then
>  	  umount $SCRATCH_DEV 2>/dev/null
> +
> +	  # check if mkfs.xfs supports v5 xfs
> +	  if [ "$FSTYP" == "xfs" ]; then
> +		export XFS_MKFS_HAS_NO_META_SUPPORT=""
> +		if ! _scratch_mkfs -m crc=0 >/dev/null 2>&1; then
> +			export XFS_MKFS_HAS_NO_META_SUPPORT=true
> +		fi
> +	  fi
> +

Shouldn't this be somewhere in common/config, not in the check
script itself?

Also, _scratch_mkfs_xfs_supported already exists for checking
various options exist in mkfs.xfs....

> diff --git a/tests/xfs/073 b/tests/xfs/073
> index 45a3fdf..acac8b8 100755
> --- a/tests/xfs/073
> +++ b/tests/xfs/073
> @@ -156,7 +156,9 @@ _verify_copy $imgs.image $SCRATCH_DEV $SCRATCH_MNT
>  
>  echo 
>  echo === copying scratch device to single target, large ro device
> -${MKFS_XFS_PROG} -m crc=0 -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
> +mkfs_crc_opts="-m crc=0"
> +[ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ] && mkfs_crc_opts=""
> +${MKFS_XFS_PROG} $mkfs_crc_opts -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null

This is kinda messy. Just use if/then/else, it's much easy to read.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

On Thu, May 14, 2015 at 12:14:06PM +1000, Dave Chinner wrote:
> On Sun, May 10, 2015 at 07:50:43PM +0800, Eryu Guan wrote:
> > With the change to CRCs by default, some tests are updated to call mkfs
> > with "-m crc=0" option directly, and this breaks testings on older
> > distros where mkfs.xfs doesn't have crc support.
> > 
> > Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
> > tweaks in _scratch_mkfs_xfs_opts() based on it.
> > 
> > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > ---
> > 
> > This is based on Dave's commit "filter: inode size output of mkfs.xfs can change"
> > 
> > Tested on upstream kernel with upstream xfsprogs(v5 as default) and on RHEL7(v4
> > as default) and on RHEL6(no v5 support), all seem fine, no failures caused by
> > crc options.
> > 
> >  check         | 10 +++++++++-
> >  common/rc     |  5 +++++
> >  tests/xfs/073 |  4 +++-
> >  3 files changed, 17 insertions(+), 2 deletions(-)
> > 
> > diff --git a/check b/check
> > index 4fa96ed..568c05b 100755
> > --- a/check
> > +++ b/check
> > @@ -505,9 +505,17 @@ for section in $HOST_OPTIONS_SECTIONS; do
> >  
> >  	if [ ! -z "$SCRATCH_DEV" ]; then
> >  	  umount $SCRATCH_DEV 2>/dev/null
> > +
> > +	  # check if mkfs.xfs supports v5 xfs
> > +	  if [ "$FSTYP" == "xfs" ]; then
> > +		export XFS_MKFS_HAS_NO_META_SUPPORT=""
> > +		if ! _scratch_mkfs -m crc=0 >/dev/null 2>&1; then
> > +			export XFS_MKFS_HAS_NO_META_SUPPORT=true
> > +		fi
> > +	  fi
> > +
> 
> Shouldn't this be somewhere in common/config, not in the check
> script itself?

When sourcing common/config, SCRATCH_DEV has not been validated, and
things get messier when config sections are used.

In check here, we have done all the validations and preparations,
everything is ready to go.

> 
> Also, _scratch_mkfs_xfs_supported already exists for checking
> various options exist in mkfs.xfs....

I missed it, it's better than _scratch_mkfs, thanks! But it's not
available in common/config if we want to do the check there.

> 
> > diff --git a/tests/xfs/073 b/tests/xfs/073
> > index 45a3fdf..acac8b8 100755
> > --- a/tests/xfs/073
> > +++ b/tests/xfs/073
> > @@ -156,7 +156,9 @@ _verify_copy $imgs.image $SCRATCH_DEV $SCRATCH_MNT
> >  
> >  echo 
> >  echo === copying scratch device to single target, large ro device
> > -${MKFS_XFS_PROG} -m crc=0 -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
> > +mkfs_crc_opts="-m crc=0"
> > +[ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ] && mkfs_crc_opts=""
> > +${MKFS_XFS_PROG} $mkfs_crc_opts -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
> 
> This is kinda messy. Just use if/then/else, it's much easy to read.

OK.

Thanks for the review!

Eryu

[PATCH v2] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

With the change to CRCs by default, some tests are updated to call mkfs
with "-m crc=0" option directly, and this breaks testings on older
distros where mkfs.xfs doesn't have crc support.

Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
tweaks in _scratch_mkfs_xfs_opts() based on it.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---

This depends on Dave's commit "filter: inode size output of mkfs.xfs can change"

v2:
- use _scratch_mkfs_xfs_supported not _scratch_mkfs
- use if/else not [ ] && ... in xfs/073

 check         | 10 +++++++++-
 common/rc     |  5 +++++
 tests/xfs/073 |  7 ++++++-
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/check b/check
index 4fa96ed..9eecb44 100755
--- a/check
+++ b/check
@@ -505,9 +505,17 @@ for section in $HOST_OPTIONS_SECTIONS; do
 
 	if [ ! -z "$SCRATCH_DEV" ]; then
 	  umount $SCRATCH_DEV 2>/dev/null
+
+	  # check if mkfs.xfs supports v5 xfs
+	  if [ "$FSTYP" == "xfs" ]; then
+		export XFS_MKFS_HAS_NO_META_SUPPORT=""
+		if ! _scratch_mkfs_xfs_supported -m crc=1 >/dev/null 2>&1; then
+			export XFS_MKFS_HAS_NO_META_SUPPORT=true
+		fi
+	  fi
+
 	  # call the overridden mkfs - make sure the FS is built
 	  # the same as we'll create it later.
-
 	  if ! _scratch_mkfs $flag >$tmp.err 2>&1
 	  then
 	      echo "our local _scratch_mkfs routine ..."
diff --git a/common/rc b/common/rc
index 242dedb..6af7f14 100644
--- a/common/rc
+++ b/common/rc
@@ -308,6 +308,11 @@ _scratch_mkfs_xfs_opts()
 {
 	mkfs_opts=$*
 
+	# remove crc related mkfs options if mkfs.xfs doesn't support v5 support
+	if [ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
+		mkfs_opts=`echo $mkfs_opts | sed "s/-m\s\+crc=.//"`
+	fi
+
 	_scratch_options mkfs
 
 	$MKFS_XFS_PROG $SCRATCH_OPTIONS $mkfs_opts $SCRATCH_DEV
diff --git a/tests/xfs/073 b/tests/xfs/073
index 45a3fdf..32faffd 100755
--- a/tests/xfs/073
+++ b/tests/xfs/073
@@ -156,7 +156,12 @@ _verify_copy $imgs.image $SCRATCH_DEV $SCRATCH_MNT
 
 echo 
 echo === copying scratch device to single target, large ro device
-${MKFS_XFS_PROG} -m crc=0 -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
+mkfs_crc_opts="-m crc=0"
+if [ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
+	mkfs_crc_opts=""
+fi
+${MKFS_XFS_PROG} $mkfs_crc_opts -dfile,name=$imgs.source,size=100g \
+	| _filter_mkfs 2>/dev/null
 rmdir $imgs.source_dir 2>/dev/null
 mkdir $imgs.source_dir
 
-- 
1.8.3.1

Re: [PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Dave Chinner]

On Thu, May 14, 2015 at 12:40:31PM +0800, Eryu Guan wrote:
> On Thu, May 14, 2015 at 12:14:06PM +1000, Dave Chinner wrote:
> > On Sun, May 10, 2015 at 07:50:43PM +0800, Eryu Guan wrote:
> > > With the change to CRCs by default, some tests are updated to call mkfs
> > > with "-m crc=0" option directly, and this breaks testings on older
> > > distros where mkfs.xfs doesn't have crc support.
> > > 
> > > Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
> > > tweaks in _scratch_mkfs_xfs_opts() based on it.
> > > 
> > > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > > ---
> > > 
> > > This is based on Dave's commit "filter: inode size output of mkfs.xfs can change"
> > > 
> > > Tested on upstream kernel with upstream xfsprogs(v5 as default) and on RHEL7(v4
> > > as default) and on RHEL6(no v5 support), all seem fine, no failures caused by
> > > crc options.
> > > 
> > >  check         | 10 +++++++++-
> > >  common/rc     |  5 +++++
> > >  tests/xfs/073 |  4 +++-
> > >  3 files changed, 17 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/check b/check
> > > index 4fa96ed..568c05b 100755
> > > --- a/check
> > > +++ b/check
> > > @@ -505,9 +505,17 @@ for section in $HOST_OPTIONS_SECTIONS; do
> > >  
> > >  	if [ ! -z "$SCRATCH_DEV" ]; then
> > >  	  umount $SCRATCH_DEV 2>/dev/null
> > > +
> > > +	  # check if mkfs.xfs supports v5 xfs
> > > +	  if [ "$FSTYP" == "xfs" ]; then
> > > +		export XFS_MKFS_HAS_NO_META_SUPPORT=""
> > > +		if ! _scratch_mkfs -m crc=0 >/dev/null 2>&1; then
> > > +			export XFS_MKFS_HAS_NO_META_SUPPORT=true
> > > +		fi
> > > +	  fi
> > > +
> > 
> > Shouldn't this be somewhere in common/config, not in the check
> > script itself?
> 
> When sourcing common/config, SCRATCH_DEV has not been validated, and
> things get messier when config sections are used.

Don't need a scratch dev:

touch /tmp/foo
mkfs.xfs -N -d file,name=/tmp/foo,size=32m -m crc=0
rm /tmp/foo

mkfs will succeed if "-m crc=X" is recognised, fail if it isn't.
the "-N" means it just does arg parsing and checks geometry, etc but
does not do try to create the filesystem....

> I missed it, it's better than _scratch_mkfs, thanks! But it's not
> available in common/config if we want to do the check there.

*nod*. In which case you'd just use MKFS_XFS_PROG directly, like
above.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

On Thu, May 14, 2015 at 08:24:11PM +1000, Dave Chinner wrote:
> On Thu, May 14, 2015 at 12:40:31PM +0800, Eryu Guan wrote:
> > On Thu, May 14, 2015 at 12:14:06PM +1000, Dave Chinner wrote:
> > > On Sun, May 10, 2015 at 07:50:43PM +0800, Eryu Guan wrote:
> > > > With the change to CRCs by default, some tests are updated to call mkfs
> > > > with "-m crc=0" option directly, and this breaks testings on older
> > > > distros where mkfs.xfs doesn't have crc support.
> > > > 
> > > > Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
> > > > tweaks in _scratch_mkfs_xfs_opts() based on it.
> > > > 
> > > > Signed-off-by: Eryu Guan <eguan@redhat.com>
[snip]
> > > 
> > > Shouldn't this be somewhere in common/config, not in the check
> > > script itself?
> > 
> > When sourcing common/config, SCRATCH_DEV has not been validated, and
> > things get messier when config sections are used.
> 
> Don't need a scratch dev:
> 
> touch /tmp/foo
> mkfs.xfs -N -d file,name=/tmp/foo,size=32m -m crc=0
> rm /tmp/foo

Ah right, thanks, I'll work on v3.

Thanks,
Eryu
> 
> mkfs will succeed if "-m crc=X" is recognised, fail if it isn't.
> the "-N" means it just does arg parsing and checks geometry, etc but
> does not do try to create the filesystem....
> 
> > I missed it, it's better than _scratch_mkfs, thanks! But it's not
> > available in common/config if we want to do the check there.
> 
> *nod*. In which case you'd just use MKFS_XFS_PROG directly, like
> above.
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com

[PATCH v3] fstests: be compatible with older mkfs.xfs which has no v5 support

[Eryu Guan]

With the change to CRCs by default, some tests are updated to call mkfs
with "-m crc=0" option directly, and this breaks testings on older
distros where mkfs.xfs doesn't have crc support.

Introduce a new variable to tell if mkfs.xfs supports v5 xfs and do
tweaks in _scratch_mkfs_xfs_opts() based on it.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---

This depends on Dave's commit "filter: inode size output of mkfs.xfs can change"

v3:
- move mkfs.xfs check to common/config

v2:
- use _scratch_mkfs_xfs_supported not _scratch_mkfs
- use if/else not [ ] && ... in xfs/073

 common/config | 11 +++++++++++
 common/rc     |  5 +++++
 tests/xfs/073 |  7 ++++++-
 3 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/common/config b/common/config
index 3732287..8e21c28 100644
--- a/common/config
+++ b/common/config
@@ -239,6 +239,17 @@ if [ -x /usr/sbin/selinuxenabled ] && /usr/sbin/selinuxenabled; then
 	export SELINUX_MOUNT_OPTIONS
 fi
 
+# check if mkfs.xfs supports v5 xfs
+XFS_MKFS_HAS_NO_META_SUPPORT=""
+touch /tmp/crc_check.img
+$MKFS_XFS_PROG -N -d file,name=/tmp/crc_check.img,size=32m -m crc=0 \
+	>/dev/null 2>&1;
+if [ $? -ne 0 ]; then
+	XFS_MKFS_HAS_NO_META_SUPPORT=true
+fi
+rm -f /tmp/crc_check.img
+export XFS_MKFS_HAS_NO_META_SUPPORT
+
 _mount_opts()
 {
 	case $FSTYP in
diff --git a/common/rc b/common/rc
index 242dedb..d55dda8 100644
--- a/common/rc
+++ b/common/rc
@@ -308,6 +308,11 @@ _scratch_mkfs_xfs_opts()
 {
 	mkfs_opts=$*
 
+	# remove crc related mkfs options if mkfs.xfs doesn't support v5 xfs
+	if [ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
+		mkfs_opts=`echo $mkfs_opts | sed "s/-m\s\+crc=.//"`
+	fi
+
 	_scratch_options mkfs
 
 	$MKFS_XFS_PROG $SCRATCH_OPTIONS $mkfs_opts $SCRATCH_DEV
diff --git a/tests/xfs/073 b/tests/xfs/073
index 45a3fdf..32faffd 100755
--- a/tests/xfs/073
+++ b/tests/xfs/073
@@ -156,7 +156,12 @@ _verify_copy $imgs.image $SCRATCH_DEV $SCRATCH_MNT
 
 echo 
 echo === copying scratch device to single target, large ro device
-${MKFS_XFS_PROG} -m crc=0 -dfile,name=$imgs.source,size=100g | _filter_mkfs 2>/dev/null
+mkfs_crc_opts="-m crc=0"
+if [ -n "$XFS_MKFS_HAS_NO_META_SUPPORT" ]; then
+	mkfs_crc_opts=""
+fi
+${MKFS_XFS_PROG} $mkfs_crc_opts -dfile,name=$imgs.source,size=100g \
+	| _filter_mkfs 2>/dev/null
 rmdir $imgs.source_dir 2>/dev/null
 mkdir $imgs.source_dir
 
-- 
1.8.3.1