[PATCH 0/3] fstests: various fixes

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

Fix broken cleanup in some tests.

Fix the mixed buffered/directio filtering function to pick up the new
locations of the iomap directio code, fix generic/081's broken lvm
cleanup, and reduce the time generic/561 waits for duperemove to finish.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes

[PATCH 1/3] generic/081: fix lvm config not being cleaned up properly

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Fix a race between _cleanup and dmeventd that causes the lvm
configuration not to be cleaned up and subsequent tests to fail.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/081 |   29 +++++++++++++++++++++++------
 1 file changed, 23 insertions(+), 6 deletions(-)


diff --git a/tests/generic/081 b/tests/generic/081
index 10f4a186..e8f4f5b5 100755
--- a/tests/generic/081
+++ b/tests/generic/081
@@ -19,12 +19,29 @@ _cleanup()
 {
 	cd /
 	rm -f $tmp.*
-	# lvm may have umounted it on I/O error, but in case it does not
-	# wait a bit for lvm to settle, sigh..
-	sleep 2
-	$UMOUNT_PROG $mnt >/dev/null 2>&1
-	$LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1
-	$LVM_PROG pvremove -f $SCRATCH_DEV >>$seqres.full 2>&1
+
+	# Tear down the lvm vg and snapshot.
+	#
+	# NOTE: We do the unmount and {vg,pv}remove in a loop here because
+	# dmeventd could be configured to unmount the filesystem automatically
+	# after the IO errors.  That is racy with the umount we're trying to do
+	# here because there's a window in which the directory tree has been
+	# removed from the mount namespaces (so the umount call here sees no
+	# mount and exits) but the filesystem hasn't yet released the block
+	# device, which causes the vgremove here to fail.
+	#
+	# We "solve" the race by repeating the umount/lvm teardown until the
+	# block device goes away, because we cannot exit this test without
+	# removing the lvm devices from the scratch device -- this will cause
+	# other tests to fail.
+	while test -e /dev/mapper/$vgname-$snapname || \
+	      test -e /dev/mapper/$vgname-$lvname; do
+		$UMOUNT_PROG $mnt >> $seqres.full 2>&1
+		$LVM_PROG vgremove -f $vgname >>$seqres.full 2>&1
+		$LVM_PROG pvremove -f $SCRATCH_DEV >>$seqres.full 2>&1
+		test $? -eq 0 && break
+		sleep 2
+	done
 }
 
 # get standard environment, filters and checks

[PATCH 2/3] generic/561: kill duperemove directly

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

While the kill statement added in the previous patch usually suffices to
shut down the bash loop that runs the duperemove processes, for whatever
reason this sometimes fails to kill duperemove.  Kill the duperemove
processes directly after removing the run file, which should cause the
bash loop to exit immediately.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 tests/generic/561 |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


diff --git a/tests/generic/561 b/tests/generic/561
index 2f3eff3c..26ecff5d 100755
--- a/tests/generic/561
+++ b/tests/generic/561
@@ -47,7 +47,7 @@ function end_test()
 	# stop duperemove running
 	if [ -e $dupe_run ]; then
 		rm -f $dupe_run
-		kill -INT $dedup_pids
+		$KILLALL_PROG -q $DUPEREMOVE_PROG > /dev/null 2>&1
 		wait $dedup_pids
 	fi
 
@@ -74,7 +74,7 @@ for ((i = 0; i < $((2 * LOAD_FACTOR)); i++)); do
 	while [ -e $dupe_run ]; do
 		$DUPEREMOVE_PROG -dr --dedupe-options=same $testdir \
 			>>$seqres.full 2>&1
-	done &
+	done 2>&1 | sed -e '/Terminated/d' &
 	dedup_pids="$! $dedup_pids"
 done
 

[PATCH 3/3] common: filter aiodio dmesg after fs/iomap.c to fs/iomap/ move

[Darrick J. Wong]

From: Darrick J. Wong <darrick.wong@oracle.com>

Since the iomap code are moving to fs/iomap/ we have to add new entries
to the aiodio dmesg filter to reflect this.  It's still possible for
filesystems using iomap for directio to cough up WARNings when a direct
write collides with a buffered write, since in some cases we catch that
early enough to have the direct write return EIO.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 common/filter |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)


diff --git a/common/filter b/common/filter
index ed082d24..26fc2132 100644
--- a/common/filter
+++ b/common/filter
@@ -555,6 +555,7 @@ _filter_aiodio_dmesg()
 	local warn7="WARNING:.*fs/iomap\.c:.*iomap_dio_actor.*"
 	local warn8="WARNING:.*fs/iomap\.c:.*iomap_dio_complete.*"
 	local warn9="WARNING:.*fs/direct-io\.c:.*dio_complete.*"
+	local warn10="WARNING:.*fs/iomap/direct-io\.c:.*iomap_dio_actor.*"
 	sed -e "s#$warn1#Intentional warnings in xfs_file_dio_aio_write#" \
 	    -e "s#$warn2#Intentional warnings in xfs_file_dio_aio_read#" \
 	    -e "s#$warn3#Intentional warnings in xfs_file_read_iter#" \
@@ -563,7 +564,8 @@ _filter_aiodio_dmesg()
 	    -e "s#$warn6#Intentional warnings in __xfs_get_blocks#" \
 	    -e "s#$warn7#Intentional warnings in iomap_dio_actor#" \
 	    -e "s#$warn8#Intentional warnings in iomap_dio_complete#" \
-	    -e "s#$warn9#Intentional warnings in dio_complete#"
+	    -e "s#$warn9#Intentional warnings in dio_complete#" \
+	    -e "s#$warn10#Intentional warnings in iomap_dio_actor#"
 }
 
 # We generate assert related WARNINGs on purpose and make sure test doesn't fail

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

Fix the mixed buffered/directio filtering function to pick up the new
locations of the iomap directio code, fix the bogus description in
xfs/504, and reduce the time generic/561 waits for duperemove to finish.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

Fix some problems with the xfs min log size calculation code.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

The first patch reduces the likelihood that the test control process
will get OOM killed by increasing the chance that the test processes
themselves will be targeted.

The second patch teaches xfs/122 about some new structures.

The third patch converts some typedef usage.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

The first two patches fix t_open_tmpfiles to shut down the scratch
filesystem properly by reverting a broken fix and teaching xfstests to
pass the relevant handles around.  The final patch cleans up some open
coded src/godown calls against the scratch fs to use the wrapper.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes

Re: [PATCH 0/3] fstests: various fixes

[xuyang]

on 2019/05/10 16:47, Eryu Guan wrote:

> On Thu, May 09, 2019 at 10:21:12AM +0800, xuyang wrote:
>> on 2019/05/08 0:56, Darrick J. Wong wrote:
>>> Hi all,
>>>
>>> Here are three patches fixing various regressions in xfstests when
>>> mkfs.xfs defaults to enabling reflink and/or rmap by default.  Most of
>>> the changes deal with the change in minimum log size requirements.  They
>>> weren't caught until now because there are a number of tests that call
>>> mkfs on a loop device or a file without using MKFS_OPTIONS.
>>>
>>> If you're going to start using this mess, you probably ought to just
>>> pull from my git trees, which are linked below.
>>>
>>> This is an extraordinary way to destroy everything.  Enjoy!
>>> Comments and questions are, as always, welcome.
>>>
>>> --D
>>>
>>> fstests git tree:
>>> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
>>>
>>>
>>>
>> Hi
>>
>> Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>
> Thanks for the testing! Just want to make sure that you tested all the
> three patches so that I can add your Tested-by tag too all of them?
>
> Thanks,
> Eryu
Hi Eryu

Yes.  I tested all the three patches.

>
>

Re: [PATCH 0/3] fstests: various fixes

[Eryu Guan]

On Thu, May 09, 2019 at 10:21:12AM +0800, xuyang wrote:
> on 2019/05/08 0:56, Darrick J. Wong wrote:
> > Hi all,
> > 
> > Here are three patches fixing various regressions in xfstests when
> > mkfs.xfs defaults to enabling reflink and/or rmap by default.  Most of
> > the changes deal with the change in minimum log size requirements.  They
> > weren't caught until now because there are a number of tests that call
> > mkfs on a loop device or a file without using MKFS_OPTIONS.
> > 
> > If you're going to start using this mess, you probably ought to just
> > pull from my git trees, which are linked below.
> > 
> > This is an extraordinary way to destroy everything.  Enjoy!
> > Comments and questions are, as always, welcome.
> > 
> > --D
> > 
> > fstests git tree:
> > https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
> > 
> > 
> > 
> Hi
> 
> Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>

Thanks for the testing! Just want to make sure that you tested all the
three patches so that I can add your Tested-by tag too all of them?

Thanks,
Eryu

Re: [PATCH 0/3] fstests: various fixes

[xuyang]

on 2019/05/08 0:56, Darrick J. Wong wrote:
> Hi all,
>
> Here are three patches fixing various regressions in xfstests when
> mkfs.xfs defaults to enabling reflink and/or rmap by default.  Most of
> the changes deal with the change in minimum log size requirements.  They
> weren't caught until now because there are a number of tests that call
> mkfs on a loop device or a file without using MKFS_OPTIONS.
>
> If you're going to start using this mess, you probably ought to just
> pull from my git trees, which are linked below.
>
> This is an extraordinary way to destroy everything.  Enjoy!
> Comments and questions are, as always, welcome.
>
> --D
>
> fstests git tree:
> https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes
>
>
>
Hi

Tested-by: Yang Xu<xuyang2018.jy@cn.fujitsu.com>

[PATCH 0/3] fstests: various fixes

[Darrick J. Wong]

Hi all,

Here are three patches fixing various regressions in xfstests when
mkfs.xfs defaults to enabling reflink and/or rmap by default.  Most of
the changes deal with the change in minimum log size requirements.  They
weren't caught until now because there are a number of tests that call
mkfs on a loop device or a file without using MKFS_OPTIONS.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=random-fixes