Re: [PATCH 5/9] tests/xfs/group: add group for tests which require a logdev

From: "Luis R. Rodriguez" <mcgrof@kernel.org>
To: Dave Chinner <david@fromorbit.com>
Cc: "Luis R. Rodriguez" <mcgrof@kernel.org>,
	fstests@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH 5/9] tests/xfs/group: add group for tests which require a logdev
Date: Thu, 14 Dec 2017 18:48:53 +0100	[thread overview]
Message-ID: <20171214174853.GK16026@wotan.suse.de> (raw)
In-Reply-To: <20171213233914.GD5858@dastard>

On Thu, Dec 14, 2017 at 10:39:14AM +1100, Dave Chinner wrote:
> You can't just run an internal log test then add couple of extra
> external log tests and say "external logs work fine".
> 
> > Automatic detection if the requirements are met is fine, but this doesn't 
> > let me easily use say:
> > 
> > 	./check -s logdev_xfs -g logdev
> 
> You can do that if we ignore the fact that a large number of tests
> need to be run on both internal and external log devices to cover
> the differences in behaviour between them.
> 
> > > And, FWIW, we already have a "log" group to indicate tests that
> > > exercise the log, and that mostly includes all the tests that use
> > > external logs. It would be better to tag all the tests that exercise
> > > the log with "log" rather than create some new group that doesn't
> > > really provide any added benefit....
> > 
> > So for my case would one better goal be to just run check without the external
> > one and one with the external log?
> 
> See above. Your test coverage assumptions are wrong, so what you are
> trying to do really doesn't tell you whether external logs work
> correctly or not. It's worse that not testing external logs at all,
> because it gives the false impression that they have been
> exhaustively tested and work just fine when that really isn't the
> case.

Makes sense, thanks.

  Luis