[PATCH 0/2] skip tests that don't work with an external log

[PATCH 0/2] skip tests that don't work with an external log

[Theodore Ts'o]

I recently tried running xfstests with an external log device, and
found a number of failures caused by the fact that we use dmflakey and
dmerror to take down a disk to simulate a power failure or a complete
hard disk failure.  But with an external log device, the tests don't
correctly simulate these scenarios, so skip them.

With these patches, testing with ext4 using an external log is passing
almost all of the auto group tests, with the exception of ext4/028
(which appears to be a genuine ext4 fsmap bug not handling external
log devices correctly).

Theodore Ts'o (2):
  dmflakey: don't run dmflakey tests with an external log device
  generic/475: skip test when using an external log device

 common/dmflakey   | 4 ++++
 tests/generic/475 | 4 ++++
 2 files changed, 8 insertions(+)

-- 
2.31.0

[PATCH 1/2] dmflakey: don't run dmflakey tests with an external log device

[Theodore Ts'o]

dmflakey works by dropping all writes before unmounting to similate a
crash/power loss.  This doesn't work if there is an external log
device, since we only drop writes to the primary block device, and not
the external log device.  Fixing this for real would require somehow
arranging to atomically loading a new dmflakey table for two block
devices at the same time, so for now, just skip tests using dmflakey
if the external log device is enabled.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 common/dmflakey | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/common/dmflakey b/common/dmflakey
index b4e11ae9..3bbf6e2b 100644
--- a/common/dmflakey
+++ b/common/dmflakey
@@ -11,6 +11,10 @@ FLAKEY_ERROR_WRITES=2
 _init_flakey()
 {
 	local BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
+
+	if test "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ; then
+		_notrun "dmflakey tests don't work with an external log device"
+	fi
 	FLAKEY_DEV=/dev/mapper/flakey-test
 	FLAKEY_TABLE="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 180 0"
 	FLAKEY_TABLE_DROP="0 $BLK_DEV_SIZE flakey $SCRATCH_DEV 0 0 180 1 drop_writes"
-- 
2.31.0

[PATCH 2/2] generic/475: skip test when using an external log device

[Theodore Ts'o]

This test tests log recovery by kicking off fstress on the scratch fs,
then switch out the underlying the device with dm-error to see what
happens when the disk goes down.  But when there is an external log
device writes to the log device still succeed, while writes to the
rest of the file system fail, and this leads to file system
corruption.  So skip this test when there is an external log.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
---
 tests/generic/475 | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/generic/475 b/tests/generic/475
index 62894491..4f230430 100755
--- a/tests/generic/475
+++ b/tests/generic/475
@@ -34,6 +34,10 @@ _require_scratch
 _require_dm_target error
 _require_command "$KILLALL_PROG" "killall"
 
+if test "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ; then
+    _notrun "Cannot have external log device"
+fi
+
 echo "Silence is golden."
 
 _scratch_mkfs >> $seqres.full 2>&1
-- 
2.31.0

Re: [PATCH 0/2] skip tests that don't work with an external log

[Darrick J. Wong]

On Mon, Jul 05, 2021 at 12:03:05AM -0400, Theodore Ts'o wrote:
> I recently tried running xfstests with an external log device, and
> found a number of failures caused by the fact that we use dmflakey and
> dmerror to take down a disk to simulate a power failure or a complete
> hard disk failure.  But with an external log device, the tests don't
> correctly simulate these scenarios, so skip them.

Counter-proposal: If I sent out my series to fix all that[1], would you
review them?

--D

[1] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=dmerror-on-rt-devices

> 
> With these patches, testing with ext4 using an external log is passing
> almost all of the auto group tests, with the exception of ext4/028
> (which appears to be a genuine ext4 fsmap bug not handling external
> log devices correctly).
> 
> Theodore Ts'o (2):
>   dmflakey: don't run dmflakey tests with an external log device
>   generic/475: skip test when using an external log device
> 
>  common/dmflakey   | 4 ++++
>  tests/generic/475 | 4 ++++
>  2 files changed, 8 insertions(+)
> 
> -- 
> 2.31.0
> 

Re: [PATCH 0/2] skip tests that don't work with an external log

[Eryu Guan]

On Tue, Jul 06, 2021 at 10:48:37AM -0700, Darrick J. Wong wrote:
> On Mon, Jul 05, 2021 at 12:03:05AM -0400, Theodore Ts'o wrote:
> > I recently tried running xfstests with an external log device, and
> > found a number of failures caused by the fact that we use dmflakey and
> > dmerror to take down a disk to simulate a power failure or a complete
> > hard disk failure.  But with an external log device, the tests don't
> > correctly simulate these scenarios, so skip them.
> 
> Counter-proposal: If I sent out my series to fix all that[1], would you
> review them?

Yeah, I was going to ask if xfs supports dmflakey tests with external
log dev. So Ted's proposal will break your work.

And if we want to disable dmflakey/dmerror tests with external log only
for ext4, I think we could do it in _require_dm_target() not in
_init_flakey(), something like:

_require_dm_target()
{
	local target=$1
	......

	case "$target" in
	flakey|error)
		if [ "$FSTYP" == "ext4" -o "$FSTYP" == "ext3" ] && _has_logdev; then
			_notrun "dmflakey/dmerror tests don't work with an external log device for $FSTYP"
		fi
		;;
	*)
		;;
	esac
	......
}

Then generic/475 doesn't need any update.

Thanks,
Eryu

> 
> --D
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfstests-dev.git/log/?h=dmerror-on-rt-devices
> 
> > 
> > With these patches, testing with ext4 using an external log is passing
> > almost all of the auto group tests, with the exception of ext4/028
> > (which appears to be a genuine ext4 fsmap bug not handling external
> > log devices correctly).
> > 
> > Theodore Ts'o (2):
> >   dmflakey: don't run dmflakey tests with an external log device
> >   generic/475: skip test when using an external log device
> > 
> >  common/dmflakey   | 4 ++++
> >  tests/generic/475 | 4 ++++
> >  2 files changed, 8 insertions(+)
> > 
> > -- 
> > 2.31.0
> >