[RFC] [PATCH 0/18] xfstests: move tests out of top level

* [RFC] [PATCH 0/18] xfstests: move tests out of top level
@ 2012-07-26  9:27 Dave Chinner
  2012-07-26  9:27 ` [PATCH 01/18] xfstests: remove remake script Dave Chinner
                   ` (19 more replies)
  0 siblings, 20 replies; 49+ messages in thread
From: Dave Chinner @ 2012-07-26  9:27 UTC (permalink / raw)
  To: xfs

Alt-Subject: Games with Sed, Grep and Awk.

This series is based on top of the large filesystem test series.

This moves all the tests into a ./tests subdirectory, and sorts them into
classes of related tests. Those are:

	tests/generic:	valid for all filesystems
	tests/shared:	valid for a limited number of filesystems
	tests/xfs:	xfs specific tests
	tests/btrfs	btrfs specific tests
	tests/ext4	ext4 specific tests
	tests/udf	udf specific tests

Each directory has it's own group file to determine what groups the
tests are associated with. Tests are run in exactly the same was as
before, but when trying to run individual tests you need to specify
the class as well. e.g. the old way:

# ./check 001

The new way:

# ./check generic/001

The output also indicates what class the test came from:

$ sudo ./check -g auto
FSTYP         -- xfs (debug)
PLATFORM      -- Linux/x86_64 test-1 3.5.0-rc5-dgc+
MKFS_OPTIONS  -- -f -bsize=4096 /dev/vdb
MOUNT_OPTIONS -- /dev/vdb /mnt/scratch

generic/001      3s
generic/002      0s
generic/005      1s
generic/006      10s
generic/007      2s
generic/010      [not run] dbtest was not built for this platform
generic/011      15s
generic/013      43s
generic/014      2s
generic/015      0s
generic/020      3s
generic/053      0s
....

[I'd post more example output, but a lightning strike took out power
a little while ago and so all the output in my scrollback buffers
went bye-bye...]

The test classes that are run are generic, shared and $FSTYP, hence
avoiding most "notrun, wrong filesystem" cases.

Further, the test result/status files (e.g. 001.full) are also moved
out of the top level directory into a new results directory. This
defaults to ./results ($RESULT_BASE) and duplicates the heirarchy of
the tests/ directory. It is created on demand. Each test is passed
$RESULT_DIR which points to the directory it should dump it's output
files in. i.e. "echo foo > $RESULT_DIR/$seq.full".

There's a bunch of cleanup at the start of the series, removing
stuff that I don't think has been used for years. e.g. i didn't even
know the remake script existed, but it's usefulness is minimal are
we rarely, if ever, regenerate every single .out file in the test
suite. Hence stuff is removed to make it easy to convert the
important stuff to use the new structure....

Feel free to debate things like the renaming of variables - I just
used sed scripts to do most of the conversion and most of them are
in the commit messages so I can easily re-run them to do global
search/replace if you've got better ideas for naming stuff ($seqres
is pretty 'orrible)....

Overall, this series shows the direction I want to take xfstests in.
The next steps are:

	- remove remaining limitations on test naming (i.e.  must be
	  numbered) so that we can have descriptive names
	- move all of the output into the results directory and
	  enable it to be hosted externally so it can be archived
	  and data mined easily
	- move all the common* files to a subdirectory
	- allow running of test classes, not just groups
	- re-introduce the expunged file functionailty (which I
	  didn't know existed) because I can see how useful that is
	  for running regular QA with a current xfstests on an older
	  distro (e.g. RHEL5) to avoid running tests that are known
	  to fail or test features that aren't in old kernels...

I sent the patches in git format for all the renames - you don't
need to see a patchset that is this size:

1280 files changed, 102397 insertions(+), 104307 deletions(-)

When turning on rename detection makes it this size:

686 files changed, 1722 insertions(+), 3632 deletions(-)

Which is much more manageable to review....

I'm certain there are problems still in there - I haven't done a lot
of weird command line testing and really only just enough testing to
make sure a typical auto group test run mostly passes.....

Comments, additional ideas, new functionality, modifications, etc 
are all welcome.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 49+ messages in thread