[PATCH v2 0/3] fix failures caused by mount error msg change in util-linux v2.30

[PATCH v2 0/3] fix failures caused by mount error msg change in util-linux v2.30

[Eryu Guan]

util-linux v2.30 changed mount error message again and caused several
tests to fail (reported by Misono Tomohiro, thanks!), this is mainly due
to two util-linux commits in v2.30-rc1

6dede2f2f7c5 libmount: support MS_RDONLY on write-protected devices
ea848180dd34 libmount: add mnt_context_get_excode()

This patchset introduced _filter_ending_dot, _filter_error_mount and
_filter_busy_mount and improved _filter_ro_mount to deal with all these
output differences between old and new util-linux versions.

I've tested and confirmed generic/050, xfs/005 and overlay/03[567] all
passed with both old and new util-linux. xfs/333 is always _notrun
because there's no realtime-rmapbt support yet.

Perhaps there're better and cleaner ways to fix the problems, please
help review and advise, thanks!

Eryu

v2:
- remove not-needed _filter_scratch from tests

Eryu Guan (3):
  fstests: filter mount error message for EUCLEAN and ESTALE
  overlay/036: filter busy mount message
  fstests: filter readonly mount error messages

 common/filter         | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++-
 tests/generic/050     |  8 +++---
 tests/generic/050.out |  8 +++---
 tests/overlay/035     |  9 +++++--
 tests/overlay/036     |  4 +--
 tests/overlay/036.out |  4 +--
 tests/overlay/037     |  4 +--
 tests/overlay/037.out |  4 +--
 tests/xfs/005         |  7 +-----
 tests/xfs/333         |  2 +-
 tests/xfs/333.out     |  2 +-
 11 files changed, 95 insertions(+), 27 deletions(-)

-- 
2.13.6

[PATCH v2 1/3] fstests: filter mount error message for EUCLEAN and ESTALE

[Eryu Guan]

util-linux commit ea848180dd34 ("libmount: add
mnt_context_get_excode()") since v2.30 changed the error message on
EUCLEAN and ESTALE again (and maybe other errno too):

 - mount: <device> on <mountpoint> failed: Structure needs cleaning
 + mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.

and it causes xfs/005, overlay/037 to fail (and probably xfs/333 too,
but it's always _notrun for now).

So let's filter out the changing parts and keep the error message
simple.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
 common/filter         | 23 +++++++++++++++++++++++
 tests/overlay/037     |  4 ++--
 tests/overlay/037.out |  4 ++--
 tests/xfs/005         |  7 +------
 tests/xfs/333         |  2 +-
 tests/xfs/333.out     |  2 +-
 6 files changed, 30 insertions(+), 12 deletions(-)

diff --git a/common/filter b/common/filter
index 0cb458b0051f..d1bb94818e3d 100644
--- a/common/filter
+++ b/common/filter
@@ -390,6 +390,13 @@ _filter_fstrim()
 	egrep -o "[0-9]+ bytes" | $AWK_PROG '{print $1}'
 }
 
+# Remove the ending dot appended to mount error message, util-linux 2.30
+# starts to do so.
+_filter_ending_dot()
+{
+	sed -e "s/\.$//"
+}
+
 # Older mount output referred to "block device" when mounting RO devices
 # It's gone in newer versions
 _filter_ro_mount() {
@@ -397,6 +404,22 @@ _filter_ro_mount() {
 	    -e "s/mount: cannot mount block device/mount: cannot mount/g"
 }
 
+# Filter a failed mount output, util-linux changed the message several times.
+#
+# prior to v2.21:
+# mount: Structure needs cleaning
+# v2.21 to v2.29:
+# mount: mount <device> on <mountpoint> failed: Structure needs cleaning
+# v2.30 and later:
+# mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.
+#
+# This is also true for ESTALE error. So let's remove all the changing parts
+# and keep the 'prior to v2.21' format.
+_filter_error_mount()
+{
+	sed -e "s/mount:\(.*failed:\)/mount:/" | _filter_ending_dot
+}
+
 _filter_od()
 {
 	BLOCK_SIZE=$(_get_block_size $SCRATCH_MNT)
diff --git a/tests/overlay/037 b/tests/overlay/037
index 728732903358..6710ddaf4802 100755
--- a/tests/overlay/037
+++ b/tests/overlay/037
@@ -76,12 +76,12 @@ $UMOUNT_PROG $SCRATCH_MNT
 
 # Try to mount an overlay with the same upperdir and different lowerdir - expect ESTALE
 _overlay_scratch_mount_dirs $lowerdir2 $upperdir $workdir -oindex=on \
-	2>&1 | _filter_scratch
+	2>&1 | _filter_error_mount
 $UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
 
 # Try to mount an overlay with the same workdir and different upperdir - expect ESTALE
 _overlay_scratch_mount_dirs $lowerdir $upperdir2 $workdir -oindex=on \
-	2>&1 | _filter_scratch
+	2>&1 | _filter_error_mount
 $UMOUNT_PROG $SCRATCH_MNT 2>/dev/null
 
 # Mount overlay with original lowerdir, upperdir, workdir and index=on - expect success
diff --git a/tests/overlay/037.out b/tests/overlay/037.out
index 786ef85c29e3..d6ad7cfad687 100644
--- a/tests/overlay/037.out
+++ b/tests/overlay/037.out
@@ -1,3 +1,3 @@
 QA output created by 037
-mount: mount SCRATCH_DEV on SCRATCH_MNT failed: Stale file handle
-mount: mount SCRATCH_DEV on SCRATCH_MNT failed: Stale file handle
+mount: Stale file handle
+mount: Stale file handle
diff --git a/tests/xfs/005 b/tests/xfs/005
index fade4bbf282b..ebf4b15ec9b5 100755
--- a/tests/xfs/005
+++ b/tests/xfs/005
@@ -40,11 +40,6 @@ _cleanup()
     rm -f $tmp.*
 }
 
-filter_mount()
-{
-	sed -e "s/mount .* failed: //"
-}
-
 # get standard environment, filters and checks
 . ./common/rc
 . ./common/filter
@@ -64,7 +59,7 @@ _scratch_mkfs_xfs -m crc=1 >> $seqres.full 2>&1 || _fail "mkfs failed"
 $XFS_IO_PROG -c "pwrite 224 4" -c fsync $SCRATCH_DEV | _filter_xfs_io
 
 # should FAIL, the crc is bad; golden output contains mount failure
-_scratch_mount 2>&1 | filter_mount
+_scratch_mount 2>&1 | _filter_error_mount
 
 # success, all done
 status=0
diff --git a/tests/xfs/333 b/tests/xfs/333
index bf0c811d2435..2f394feda6c9 100755
--- a/tests/xfs/333
+++ b/tests/xfs/333
@@ -64,7 +64,7 @@ _scratch_unmount
 
 echo "Corrupt fs"
 _scratch_xfs_db -x -c 'sb 0' -c "write rrmapino $ino" >> $seqres.full
-_scratch_mount 2>&1 | _filter_scratch
+_scratch_mount 2>&1 | _filter_error_mount
 
 echo "Test done, mount should have failed"
 
diff --git a/tests/xfs/333.out b/tests/xfs/333.out
index f7518f46d900..b3c698750f8f 100644
--- a/tests/xfs/333.out
+++ b/tests/xfs/333.out
@@ -2,5 +2,5 @@ QA output created by 333
 Format and mount
 Create some files
 Corrupt fs
-mount: mount SCRATCH_DEV on SCRATCH_MNT failed: Structure needs cleaning
+mount: Structure needs cleaning
 Test done, mount should have failed
-- 
2.13.6

[PATCH v2 2/3] overlay/036: filter busy mount message

[Eryu Guan]

util-linux v2.30 changed error message of a busy mount and caused
overlay/036 to fail. e.g.

 - mount: <device> is already mounted or <mountpoint> busy
 + mount: <mountpoint>: <device> already mounted or mount point busy.

Filter the mount output by a newly introduced _filter_busy_mount
into a unified format.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
 common/filter         | 12 ++++++++++++
 tests/overlay/036     |  4 ++--
 tests/overlay/036.out |  4 ++--
 3 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/common/filter b/common/filter
index d1bb94818e3d..b1cd558ab0e1 100644
--- a/common/filter
+++ b/common/filter
@@ -420,6 +420,18 @@ _filter_error_mount()
 	sed -e "s/mount:\(.*failed:\)/mount:/" | _filter_ending_dot
 }
 
+# Similar to _filter_error_mount, filter a busy mount output.
+# Turn both old (prior to util-linux v2.30) and new (v2.30 and later) format to
+# a simple one. e.g.
+# old: mount: <device> is already mounted or <mountpoint> busy
+# new: mount: <mountpoint>: <device> already mounted or mount point busy.
+# filtered: mount: <device> already mounted or mount point busy
+_filter_busy_mount()
+{
+	sed -e "s/.*: \([^ ]*\).* already mounted.*busy/mount: \1 already mounted or mount point busy/" | \
+		_filter_ending_dot
+}
+
 _filter_od()
 {
 	BLOCK_SIZE=$(_get_block_size $SCRATCH_MNT)
diff --git a/tests/overlay/036 b/tests/overlay/036
index 544d4e4eaa87..e0c13ae88c92 100755
--- a/tests/overlay/036
+++ b/tests/overlay/036
@@ -103,12 +103,12 @@ _overlay_mount_dirs $lowerdir2 $upperdir2 $workdir \
 # Try to mount another overlay with the same upperdir
 # with index=on - expect EBUSY
 _overlay_mount_dirs $lowerdir2 $upperdir $workdir2 \
-		    overlay2 $SCRATCH_MNT -oindex=on 2>&1 | _filter_scratch
+	    overlay2 $SCRATCH_MNT -oindex=on 2>&1 | _filter_busy_mount
 
 # Try to mount another overlay with the same workdir
 # with index=on - expect EBUSY
 _overlay_mount_dirs $lowerdir2 $upperdir2 $workdir \
-		    overlay3 $SCRATCH_MNT -oindex=on 2>&1 | _filter_scratch
+	    overlay3 $SCRATCH_MNT -oindex=on 2>&1 | _filter_busy_mount
 
 
 # success, all done
diff --git a/tests/overlay/036.out b/tests/overlay/036.out
index 51746114db1c..fbe224e1bac7 100644
--- a/tests/overlay/036.out
+++ b/tests/overlay/036.out
@@ -1,3 +1,3 @@
 QA output created by 036
-mount: overlay2 is already mounted or SCRATCH_MNT busy
-mount: overlay3 is already mounted or SCRATCH_MNT busy
+mount: overlay2 already mounted or mount point busy
+mount: overlay3 already mounted or mount point busy
-- 
2.13.6

[PATCH v2 3/3] fstests: filter readonly mount error messages

[Eryu Guan]

util-linux commit 6dede2f2f7c5 ("libmount: support MS_RDONLY on
write-protected devices") changed the error message on read-only
block device, and in the failure case printed one line message
instead of two (for details please see comments in common/filter),
and this change broke generic/050 and overlay/035.

Fix it by adding more filter rules to _filter_ro_mount and updating
associated .out files to unify the output from both old and new
util-linux versions.

Signed-off-by: Eryu Guan <eguan@redhat.com>
---
 common/filter         | 35 ++++++++++++++++++++++++++++++++++-
 tests/generic/050     |  8 ++++----
 tests/generic/050.out |  8 ++++----
 tests/overlay/035     |  9 +++++++--
 4 files changed, 49 insertions(+), 11 deletions(-)

diff --git a/common/filter b/common/filter
index b1cd558ab0e1..357c4c9357c3 100644
--- a/common/filter
+++ b/common/filter
@@ -399,9 +399,42 @@ _filter_ending_dot()
 
 # Older mount output referred to "block device" when mounting RO devices
 # It's gone in newer versions
+#
+# And util-linux v2.30 changed the output again, e.g.
+# for a successful ro mount:
+# prior to v2.30:  mount: <device> is write-protected, mounting read-only
+# v2.30 and later: mount: <mountpoint>: WARNING: device write-protected, mounted read-only.
+#
+# a failed ro mount:
+# prior to v2.30:
+# mount: <device> is write-protected, mounting read-only
+# mount: cannot mount <device> read-only
+# v2.30 and later:
+# mount: <mountpoint>: cannot mount <device> read-only.
+#
+# a failed rw remount:
+# prior to v2.30:  mount: cannot remount <device> read-write, is write-protected
+# v2.30 and later: mount: <mountpoint>: cannot remount <device> read-write, is write-protected.
+#
+# Now use _filter_ro_mount to unify all these differences across old & new
+# util-linux versions.
 _filter_ro_mount() {
 	sed -e "s/mount: block device/mount:/g" \
-	    -e "s/mount: cannot mount block device/mount: cannot mount/g"
+	    -e "s/mount: cannot mount block device/mount: cannot mount/g" | \
+	_filter_ending_dot | \
+	perl -ne '
+	if (/write-protected, mount.*read-only/) {
+		print "mount: device write-protected, mounting read-only\n";
+	} elsif (/mount: .*: cannot mount.*read-only/) {
+		print "mount: device write-protected, mounting read-only\n";
+		print "mount: cannot mount read-only\n";
+	} elsif (/mount: .*: cannot remount (.*) read-write.*/) {
+		print "mount: cannot remount $1 read-write, is write-protected\n";
+	} elsif (/(^mount: cannot mount) .* (read-only$)/) {
+		print "$1 $2\n";
+	} else {
+		print "$_";
+	}'
 }
 
 # Filter a failed mount output, util-linux changed the message several times.
diff --git a/tests/generic/050 b/tests/generic/050
index 5fa28a7648e5..efa45f04825b 100755
--- a/tests/generic/050
+++ b/tests/generic/050
@@ -60,7 +60,7 @@ blockdev --setro $SCRATCH_DEV
 # Mount it, and make sure we can't write to it, and we can unmount it again
 #
 echo "mounting read-only block device:"
-_scratch_mount 2>&1 | _filter_scratch | _filter_ro_mount
+_scratch_mount 2>&1 | _filter_ro_mount
 
 echo "touching file on read-only filesystem (should fail)"
 touch $SCRATCH_MNT/foo 2>&1 | _filter_scratch
@@ -95,10 +95,10 @@ blockdev --setro $SCRATCH_DEV
 # -o norecovery is used.
 #
 echo "mounting filesystem that needs recovery on a read-only device:"
-_scratch_mount 2>&1 | _filter_scratch | _filter_ro_mount
+_scratch_mount 2>&1 | _filter_ro_mount
 
 echo "unmounting read-only filesystem"
-_scratch_unmount 2>&1 | _filter_scratch
+_scratch_unmount 2>&1 | _filter_scratch | _filter_ending_dot
 
 #
 # This is the way out if the underlying device really is read-only.
@@ -106,7 +106,7 @@ _scratch_unmount 2>&1 | _filter_scratch
 # data recovery hack.
 #
 echo "mounting filesystem with -o norecovery on a read-only device:"
-_scratch_mount -o norecovery 2>&1 | _filter_scratch | _filter_ro_mount
+_scratch_mount -o norecovery 2>&1 | _filter_ro_mount
 
 echo "unmounting read-only filesystem"
 _scratch_unmount 2>&1 | _filter_scratch
diff --git a/tests/generic/050.out b/tests/generic/050.out
index fb90f6ea5819..2187d16fa328 100644
--- a/tests/generic/050.out
+++ b/tests/generic/050.out
@@ -1,7 +1,7 @@
 QA output created by 050
 setting device read-only
 mounting read-only block device:
-mount: SCRATCH_DEV is write-protected, mounting read-only
+mount: device write-protected, mounting read-only
 touching file on read-only filesystem (should fail)
 touch: cannot touch 'SCRATCH_MNT/foo': Read-only file system
 unmounting read-only filesystem
@@ -12,12 +12,12 @@ going down:
 unmounting shutdown filesystem:
 setting device read-only
 mounting filesystem that needs recovery on a read-only device:
-mount: SCRATCH_DEV is write-protected, mounting read-only
-mount: cannot mount SCRATCH_DEV read-only
+mount: device write-protected, mounting read-only
+mount: cannot mount read-only
 unmounting read-only filesystem
 umount: SCRATCH_DEV: not mounted
 mounting filesystem with -o norecovery on a read-only device:
-mount: SCRATCH_DEV is write-protected, mounting read-only
+mount: device write-protected, mounting read-only
 unmounting read-only filesystem
 setting device read-write
 mounting filesystem that needs recovery with -o ro:
diff --git a/tests/overlay/035 b/tests/overlay/035
index 64fcd708105e..d285d44ae9df 100755
--- a/tests/overlay/035
+++ b/tests/overlay/035
@@ -45,6 +45,11 @@ _cleanup()
 . ./common/rc
 . ./common/filter
 
+filter_mount()
+{
+	_filter_scratch | _filter_ro_mount
+}
+
 # remove previous $seqres.full before test
 rm -f $seqres.full
 
@@ -69,7 +74,7 @@ mkdir -p $lowerdir1 $lowerdir2 $upperdir $workdir
 $MOUNT_PROG -t overlay -o"lowerdir=$lowerdir2:$lowerdir1" \
 			$OVL_BASE_SCRATCH_MNT $SCRATCH_MNT
 touch $SCRATCH_MNT/foo 2>&1 | _filter_scratch
-_scratch_remount rw 2>&1 | _filter_scratch
+_scratch_remount rw 2>&1 | filter_mount
 $UMOUNT_PROG $SCRATCH_MNT
 
 # Make workdir immutable to prevent workdir re-create on mount
@@ -79,7 +84,7 @@ $CHATTR_PROG +i $workdir
 # Verify that overlay is mounted read-only and that it cannot be remounted rw.
 _overlay_scratch_mount_dirs $lowerdir2 $upperdir $workdir
 touch $SCRATCH_MNT/bar 2>&1 | _filter_scratch
-_scratch_remount rw 2>&1 | _filter_scratch
+_scratch_remount rw 2>&1 | filter_mount
 
 # success, all done
 status=0
-- 
2.13.6

Re: [PATCH v2 0/3] fix failures caused by mount error msg change in util-linux v2.30

[Eryu Guan]

On Tue, Nov 14, 2017 at 08:33:07PM +0800, Eryu Guan wrote:
> util-linux v2.30 changed mount error message again and caused several
> tests to fail (reported by Misono Tomohiro, thanks!), this is mainly due
> to two util-linux commits in v2.30-rc1
> 
> 6dede2f2f7c5 libmount: support MS_RDONLY on write-protected devices
> ea848180dd34 libmount: add mnt_context_get_excode()
> 
> This patchset introduced _filter_ending_dot, _filter_error_mount and
> _filter_busy_mount and improved _filter_ro_mount to deal with all these
> output differences between old and new util-linux versions.
> 
> I've tested and confirmed generic/050, xfs/005 and overlay/03[567] all
> passed with both old and new util-linux. xfs/333 is always _notrun
> because there's no realtime-rmapbt support yet.
> 
> Perhaps there're better and cleaner ways to fix the problems, please
> help review and advise, thanks!

Could someone help review this patchset please? Newer distributions like
Fedora 27 have already shipped util-linux v2.30. I'd like to see them
fixed before they're causing problems for more and more people.

Thanks,
Eryu

> 
> Eryu
> 
> v2:
> - remove not-needed _filter_scratch from tests
> 
> Eryu Guan (3):
>   fstests: filter mount error message for EUCLEAN and ESTALE
>   overlay/036: filter busy mount message
>   fstests: filter readonly mount error messages
> 
>  common/filter         | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++-
>  tests/generic/050     |  8 +++---
>  tests/generic/050.out |  8 +++---
>  tests/overlay/035     |  9 +++++--
>  tests/overlay/036     |  4 +--
>  tests/overlay/036.out |  4 +--
>  tests/overlay/037     |  4 +--
>  tests/overlay/037.out |  4 +--
>  tests/xfs/005         |  7 +-----
>  tests/xfs/333         |  2 +-
>  tests/xfs/333.out     |  2 +-
>  11 files changed, 95 insertions(+), 27 deletions(-)
> 
> -- 
> 2.13.6
> 

Re: [PATCH v2 3/3] fstests: filter readonly mount error messages

[Amir Goldstein]

On Tue, Nov 14, 2017 at 2:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> util-linux commit 6dede2f2f7c5 ("libmount: support MS_RDONLY on
> write-protected devices") changed the error message on read-only
> block device, and in the failure case printed one line message
> instead of two (for details please see comments in common/filter),
> and this change broke generic/050 and overlay/035.
>
> Fix it by adding more filter rules to _filter_ro_mount and updating
> associated .out files to unify the output from both old and new
> util-linux versions.
>
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
>  common/filter         | 35 ++++++++++++++++++++++++++++++++++-
>  tests/generic/050     |  8 ++++----
>  tests/generic/050.out |  8 ++++----
>  tests/overlay/035     |  9 +++++++--
>  4 files changed, 49 insertions(+), 11 deletions(-)
>
> diff --git a/common/filter b/common/filter
> index b1cd558ab0e1..357c4c9357c3 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -399,9 +399,42 @@ _filter_ending_dot()
>
>  # Older mount output referred to "block device" when mounting RO devices
>  # It's gone in newer versions
> +#
> +# And util-linux v2.30 changed the output again, e.g.
> +# for a successful ro mount:
> +# prior to v2.30:  mount: <device> is write-protected, mounting read-only
> +# v2.30 and later: mount: <mountpoint>: WARNING: device write-protected, mounted read-only.
> +#
> +# a failed ro mount:
> +# prior to v2.30:
> +# mount: <device> is write-protected, mounting read-only
> +# mount: cannot mount <device> read-only
> +# v2.30 and later:
> +# mount: <mountpoint>: cannot mount <device> read-only.
> +#
> +# a failed rw remount:
> +# prior to v2.30:  mount: cannot remount <device> read-write, is write-protected
> +# v2.30 and later: mount: <mountpoint>: cannot remount <device> read-write, is write-protected.
> +#
> +# Now use _filter_ro_mount to unify all these differences across old & new
> +# util-linux versions.

Please document the unified filter result as you did with _filter_busy_mount

>  _filter_ro_mount() {
>         sed -e "s/mount: block device/mount:/g" \
> -           -e "s/mount: cannot mount block device/mount: cannot mount/g"
> +           -e "s/mount: cannot mount block device/mount: cannot mount/g" | \

Let's convert all new error formats to this legacy format, so we don't need
this extra sed to convert the legacy format to an xfstest invented format.

Besides, this filter is making an effort to preserve the <device> variable
and for what? for having to add _filter_scratch after it and covert <device>
to SCRATCH_DEV?
Better to let this filter already replace <device> with "block device" and do
away with the need to _filter_scratch.

> +       _filter_ending_dot | \
> +       perl -ne '
> +       if (/write-protected, mount.*read-only/) {
> +               print "mount: device write-protected, mounting read-only\n";
> +       } elsif (/mount: .*: cannot mount.*read-only/) {
> +               print "mount: device write-protected, mounting read-only\n";
> +               print "mount: cannot mount read-only\n";
> +       } elsif (/mount: .*: cannot remount (.*) read-write.*/) {
> +               print "mount: cannot remount $1 read-write, is write-protected\n";
> +       } elsif (/(^mount: cannot mount) .* (read-only$)/) {
> +               print "$1 $2\n";
> +       } else {
> +               print "$_";
> +       }'
>  }
>
[...]
> --- a/tests/overlay/035
> +++ b/tests/overlay/035
> @@ -45,6 +45,11 @@ _cleanup()
>  . ./common/rc
>  . ./common/filter
>
> +filter_mount()
> +{
> +       _filter_scratch | _filter_ro_mount
> +}

_filter_scratch not needed. helper not needed.

Amir.

Re: [PATCH v2 3/3] fstests: filter readonly mount error messages

[Eryu Guan]

On Thu, Nov 23, 2017 at 10:32:36AM +0200, Amir Goldstein wrote:
> On Tue, Nov 14, 2017 at 2:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> > util-linux commit 6dede2f2f7c5 ("libmount: support MS_RDONLY on
> > write-protected devices") changed the error message on read-only
> > block device, and in the failure case printed one line message
> > instead of two (for details please see comments in common/filter),
> > and this change broke generic/050 and overlay/035.
> >
> > Fix it by adding more filter rules to _filter_ro_mount and updating
> > associated .out files to unify the output from both old and new
> > util-linux versions.
> >
> > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > ---
> >  common/filter         | 35 ++++++++++++++++++++++++++++++++++-
> >  tests/generic/050     |  8 ++++----
> >  tests/generic/050.out |  8 ++++----
> >  tests/overlay/035     |  9 +++++++--
> >  4 files changed, 49 insertions(+), 11 deletions(-)
> >
> > diff --git a/common/filter b/common/filter
> > index b1cd558ab0e1..357c4c9357c3 100644
> > --- a/common/filter
> > +++ b/common/filter
> > @@ -399,9 +399,42 @@ _filter_ending_dot()
> >
> >  # Older mount output referred to "block device" when mounting RO devices
> >  # It's gone in newer versions
> > +#
> > +# And util-linux v2.30 changed the output again, e.g.
> > +# for a successful ro mount:
> > +# prior to v2.30:  mount: <device> is write-protected, mounting read-only
> > +# v2.30 and later: mount: <mountpoint>: WARNING: device write-protected, mounted read-only.
> > +#
> > +# a failed ro mount:
> > +# prior to v2.30:
> > +# mount: <device> is write-protected, mounting read-only
> > +# mount: cannot mount <device> read-only
> > +# v2.30 and later:
> > +# mount: <mountpoint>: cannot mount <device> read-only.
> > +#
> > +# a failed rw remount:
> > +# prior to v2.30:  mount: cannot remount <device> read-write, is write-protected
> > +# v2.30 and later: mount: <mountpoint>: cannot remount <device> read-write, is write-protected.
> > +#
> > +# Now use _filter_ro_mount to unify all these differences across old & new
> > +# util-linux versions.
> 
> Please document the unified filter result as you did with _filter_busy_mount

Sounds good, will do.

> 
> >  _filter_ro_mount() {
> >         sed -e "s/mount: block device/mount:/g" \
> > -           -e "s/mount: cannot mount block device/mount: cannot mount/g"
> > +           -e "s/mount: cannot mount block device/mount: cannot mount/g" | \
> 
> Let's convert all new error formats to this legacy format, so we don't need
> this extra sed to convert the legacy format to an xfstest invented format.
> 
> Besides, this filter is making an effort to preserve the <device> variable
> and for what? for having to add _filter_scratch after it and covert <device>
> to SCRATCH_DEV?

No special reason, I just wanted to preserve the format as much as
possible, but seems that's not necessary.

> Better to let this filter already replace <device> with "block device" and do
> away with the need to _filter_scratch.

OK, I'll rework this filter.

> 
> > +       _filter_ending_dot | \
> > +       perl -ne '
> > +       if (/write-protected, mount.*read-only/) {
> > +               print "mount: device write-protected, mounting read-only\n";
> > +       } elsif (/mount: .*: cannot mount.*read-only/) {
> > +               print "mount: device write-protected, mounting read-only\n";
> > +               print "mount: cannot mount read-only\n";
> > +       } elsif (/mount: .*: cannot remount (.*) read-write.*/) {
> > +               print "mount: cannot remount $1 read-write, is write-protected\n";
> > +       } elsif (/(^mount: cannot mount) .* (read-only$)/) {
> > +               print "$1 $2\n";
> > +       } else {
> > +               print "$_";
> > +       }'
> >  }
> >
> [...]
> > --- a/tests/overlay/035
> > +++ b/tests/overlay/035
> > @@ -45,6 +45,11 @@ _cleanup()
> >  . ./common/rc
> >  . ./common/filter
> >
> > +filter_mount()
> > +{
> > +       _filter_scratch | _filter_ro_mount
> > +}
> 
> _filter_scratch not needed. helper not needed.

I remembered that I checked this filter and found _filter_scratch was
still needed. But if we remove all <device>/<mountpoint> variables by
the new filters, _filter_scratch isn't needed for sure.

Thanks for the review!

Eryu

Re: [PATCH v2 2/3] overlay/036: filter busy mount message

[Amir Goldstein]

On Tue, Nov 14, 2017 at 2:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> util-linux v2.30 changed error message of a busy mount and caused
> overlay/036 to fail. e.g.
>
>  - mount: <device> is already mounted or <mountpoint> busy
>  + mount: <mountpoint>: <device> already mounted or mount point busy.
>
> Filter the mount output by a newly introduced _filter_busy_mount
> into a unified format.
>
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
>  common/filter         | 12 ++++++++++++
>  tests/overlay/036     |  4 ++--
>  tests/overlay/036.out |  4 ++--
>  3 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/common/filter b/common/filter
> index d1bb94818e3d..b1cd558ab0e1 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -420,6 +420,18 @@ _filter_error_mount()
>         sed -e "s/mount:\(.*failed:\)/mount:/" | _filter_ending_dot
>  }
>
> +# Similar to _filter_error_mount, filter a busy mount output.
> +# Turn both old (prior to util-linux v2.30) and new (v2.30 and later) format to
> +# a simple one. e.g.
> +# old: mount: <device> is already mounted or <mountpoint> busy
> +# new: mount: <mountpoint>: <device> already mounted or mount point busy.
> +# filtered: mount: <device> already mounted or mount point busy
> +_filter_busy_mount()
> +{
> +       sed -e "s/.*: \([^ ]*\).* already mounted.*busy/mount: \1 already mounted or mount point busy/" | \
> +               _filter_ending_dot

Same comment as for romount error. Please canonicalize output to:
"mount: block device already mounted or mount point busy"
so we can do away with _filter_scratch.

_filter_scratch is not needed for overlay/036, but that is not the common case.

And a nit: I would keep the word "or" in the left side.
"already mounted or .*busy" is a nicer expression to read IMO.

Amir.

Re: [PATCH v2 1/3] fstests: filter mount error message for EUCLEAN and ESTALE

[Amir Goldstein]

On Tue, Nov 14, 2017 at 2:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> util-linux commit ea848180dd34 ("libmount: add
> mnt_context_get_excode()") since v2.30 changed the error message on
> EUCLEAN and ESTALE again (and maybe other errno too):
>
>  - mount: <device> on <mountpoint> failed: Structure needs cleaning
>  + mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.
>
> and it causes xfs/005, overlay/037 to fail (and probably xfs/333 too,
> but it's always _notrun for now).
>
> So let's filter out the changing parts and keep the error message
> simple.
>
> Signed-off-by: Eryu Guan <eguan@redhat.com>
> ---
>  common/filter         | 23 +++++++++++++++++++++++
>  tests/overlay/037     |  4 ++--
>  tests/overlay/037.out |  4 ++--
>  tests/xfs/005         |  7 +------
>  tests/xfs/333         |  2 +-
>  tests/xfs/333.out     |  2 +-
>  6 files changed, 30 insertions(+), 12 deletions(-)
>
> diff --git a/common/filter b/common/filter
> index 0cb458b0051f..d1bb94818e3d 100644
> --- a/common/filter
> +++ b/common/filter
> @@ -390,6 +390,13 @@ _filter_fstrim()
>         egrep -o "[0-9]+ bytes" | $AWK_PROG '{print $1}'
>  }
>
> +# Remove the ending dot appended to mount error message, util-linux 2.30
> +# starts to do so.
> +_filter_ending_dot()
> +{
> +       sed -e "s/\.$//"
> +}
> +
>  # Older mount output referred to "block device" when mounting RO devices
>  # It's gone in newer versions
>  _filter_ro_mount() {
> @@ -397,6 +404,22 @@ _filter_ro_mount() {
>             -e "s/mount: cannot mount block device/mount: cannot mount/g"
>  }
>
> +# Filter a failed mount output, util-linux changed the message several times.
> +#
> +# prior to v2.21:
> +# mount: Structure needs cleaning

So the mentioned tests are currently also broken with util-linux < v2.21?
Maybe mention this is commit message.

> +# v2.21 to v2.29:
> +# mount: mount <device> on <mountpoint> failed: Structure needs cleaning
> +# v2.30 and later:
> +# mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.
> +#

Please document filtered format.

Thanks,
Amir.

Re: [PATCH v2 1/3] fstests: filter mount error message for EUCLEAN and ESTALE

[Eryu Guan]

On Thu, Nov 23, 2017 at 11:04:57AM +0200, Amir Goldstein wrote:
> On Tue, Nov 14, 2017 at 2:33 PM, Eryu Guan <eguan@redhat.com> wrote:
> > util-linux commit ea848180dd34 ("libmount: add
> > mnt_context_get_excode()") since v2.30 changed the error message on
> > EUCLEAN and ESTALE again (and maybe other errno too):
> >
> >  - mount: <device> on <mountpoint> failed: Structure needs cleaning
> >  + mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.
> >
> > and it causes xfs/005, overlay/037 to fail (and probably xfs/333 too,
> > but it's always _notrun for now).
> >
> > So let's filter out the changing parts and keep the error message
> > simple.
> >
> > Signed-off-by: Eryu Guan <eguan@redhat.com>
> > ---
> >  common/filter         | 23 +++++++++++++++++++++++
> >  tests/overlay/037     |  4 ++--
> >  tests/overlay/037.out |  4 ++--
> >  tests/xfs/005         |  7 +------
> >  tests/xfs/333         |  2 +-
> >  tests/xfs/333.out     |  2 +-
> >  6 files changed, 30 insertions(+), 12 deletions(-)
> >
> > diff --git a/common/filter b/common/filter
> > index 0cb458b0051f..d1bb94818e3d 100644
> > --- a/common/filter
> > +++ b/common/filter
> > @@ -390,6 +390,13 @@ _filter_fstrim()
> >         egrep -o "[0-9]+ bytes" | $AWK_PROG '{print $1}'
> >  }
> >
> > +# Remove the ending dot appended to mount error message, util-linux 2.30
> > +# starts to do so.
> > +_filter_ending_dot()
> > +{
> > +       sed -e "s/\.$//"
> > +}
> > +
> >  # Older mount output referred to "block device" when mounting RO devices
> >  # It's gone in newer versions
> >  _filter_ro_mount() {
> > @@ -397,6 +404,22 @@ _filter_ro_mount() {
> >             -e "s/mount: cannot mount block device/mount: cannot mount/g"
> >  }
> >
> > +# Filter a failed mount output, util-linux changed the message several times.
> > +#
> > +# prior to v2.21:
> > +# mount: Structure needs cleaning
> 
> So the mentioned tests are currently also broken with util-linux < v2.21?
> Maybe mention this is commit message.

Yeah, I think so, just v2.21 is pretty old, the affected tests are
usually _notrun on such old distributions that ship util-linux < v2.21,
e.g. RHEL6, so no one noticed it. I'll mention it in commit log.

> 
> > +# v2.21 to v2.29:
> > +# mount: mount <device> on <mountpoint> failed: Structure needs cleaning
> > +# v2.30 and later:
> > +# mount: <mountpoint>: mount(2) system call failed: Structure needs cleaning.
> > +#
> 
> Please document filtered format.

OK.

Thanks,
Eryu