[PATCHSET 0/5] fstests: document all test groups

[PATCHSET 0/5] fstests: document all test groups

[Darrick J. Wong]

Hi all,

I've noticed that fstests doesn't document what the test groups mean.  I
attempted to document what each group seems to symbolize, to reduce the
amount of confusion for new contributors.  While doing so, I noticed
that there were a handful of tests that aren't in the right group.  The
first three patches dix that problem.  The next test adds a build-time
check to ensure that all groups are listed in the documentation file so
that we don't lose track of things.  The last patch updates ./new to use
the documentation file instead of having to build group.list files.

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=document-test-groups
---
 doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/buildgrouplist |    1 
 new                    |   24 ++++----
 tests/ceph/001         |    2 -
 tests/ceph/002         |    2 -
 tests/ceph/003         |    2 -
 tests/xfs/491          |    2 -
 tests/xfs/492          |    2 -
 tests/xfs/493          |    2 -
 tests/xfs/519          |    2 -
 tests/xfs/520          |    2 -
 tests/xfs/535          |    2 -
 tests/xfs/536          |    2 -
 tools/check-groups     |   33 ++++++++++++
 14 files changed, 191 insertions(+), 23 deletions(-)
 create mode 100644 doc/group-names.txt
 create mode 100755 tools/check-groups

[PATCH 1/5] ceph: re-tag copy_file_range as being in the copy_range group

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

copy_range is the group name for copy_file_range tests, so reclassify
these tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/ceph/001 |    2 +-
 tests/ceph/002 |    2 +-
 tests/ceph/003 |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/tests/ceph/001 b/tests/ceph/001
index aca77168..c00de308 100755
--- a/tests/ceph/001
+++ b/tests/ceph/001
@@ -11,7 +11,7 @@
 # file and 3) the middle of the dst file.
 #
 . ./common/preamble
-_begin_fstest auto quick copy
+_begin_fstest auto quick copy_range
 
 # get standard environment
 . common/filter
diff --git a/tests/ceph/002 b/tests/ceph/002
index 428f23a9..9bc728fd 100755
--- a/tests/ceph/002
+++ b/tests/ceph/002
@@ -20,7 +20,7 @@
 #   linux kernel: 78beb0ff2fec ("ceph: use copy-from2 op in copy_file_range")
 #
 . ./common/preamble
-_begin_fstest auto quick copy
+_begin_fstest auto quick copy_range
 
 # get standard environment
 . common/filter
diff --git a/tests/ceph/003 b/tests/ceph/003
index 9f8c6068..faedb48c 100755
--- a/tests/ceph/003
+++ b/tests/ceph/003
@@ -7,7 +7,7 @@
 # Test copy_file_range with infile = outfile
 #
 . ./common/preamble
-_begin_fstest auto quick copy
+_begin_fstest auto quick copy_range
 
 # get standard environment
 . common/filter

[PATCH 2/5] xfs: move reflink tests into the clone group

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

"clone" is the group for tests that exercise FICLONERANGE, so move these
tests.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/519 |    2 +-
 tests/xfs/520 |    2 +-
 tests/xfs/535 |    2 +-
 tests/xfs/536 |    2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)


diff --git a/tests/xfs/519 b/tests/xfs/519
index 675ec07e..49c62b56 100755
--- a/tests/xfs/519
+++ b/tests/xfs/519
@@ -9,7 +9,7 @@
 # flushing the log and then remounting to check file contents.
 
 . ./common/preamble
-_begin_fstest auto quick reflink
+_begin_fstest auto quick clone
 
 # Import common functions.
 . ./common/filter
diff --git a/tests/xfs/520 b/tests/xfs/520
index 8410f2ba..2fceb07c 100755
--- a/tests/xfs/520
+++ b/tests/xfs/520
@@ -12,7 +12,7 @@
 # is included in the current kernel.
 #
 . ./common/preamble
-_begin_fstest auto quick reflink
+_begin_fstest auto quick clone
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/xfs/535 b/tests/xfs/535
index 4c883675..1a5da61b 100755
--- a/tests/xfs/535
+++ b/tests/xfs/535
@@ -7,7 +7,7 @@
 # Verify that XFS does not cause inode fork's extent count to overflow when
 # writing to a shared extent.
 . ./common/preamble
-_begin_fstest auto quick reflink
+_begin_fstest auto quick clone
 
 # Import common functions.
 . ./common/filter
diff --git a/tests/xfs/536 b/tests/xfs/536
index e5f904f5..64fa4fbf 100755
--- a/tests/xfs/536
+++ b/tests/xfs/536
@@ -7,7 +7,7 @@
 # Verify that XFS does not cause inode fork's extent count to overflow when
 # remapping extents from one file's inode fork to another.
 . ./common/preamble
-_begin_fstest auto quick reflink
+_begin_fstest auto quick clone
 
 # Import common functions.
 . ./common/filter

[PATCH 3/5] xfs: fix incorrect fuzz test group name

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

The group name for fuzz tests is 'fuzzers'.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/491 |    2 +-
 tests/xfs/492 |    2 +-
 tests/xfs/493 |    2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)


diff --git a/tests/xfs/491 b/tests/xfs/491
index 5c7c5d1f..7402b09a 100755
--- a/tests/xfs/491
+++ b/tests/xfs/491
@@ -7,7 +7,7 @@
 # Test detection & fixing of bad summary block counts at mount time.
 #
 . ./common/preamble
-_begin_fstest auto quick fuzz
+_begin_fstest auto quick fuzzers
 
 # Import common functions.
 . ./common/filter
diff --git a/tests/xfs/492 b/tests/xfs/492
index 8258e5d8..514ac1e4 100755
--- a/tests/xfs/492
+++ b/tests/xfs/492
@@ -7,7 +7,7 @@
 # Test detection & fixing of bad summary inode counts at mount time.
 #
 . ./common/preamble
-_begin_fstest auto quick fuzz
+_begin_fstest auto quick fuzzers
 
 # Import common functions.
 . ./common/filter
diff --git a/tests/xfs/493 b/tests/xfs/493
index 58fd9c99..58091ad7 100755
--- a/tests/xfs/493
+++ b/tests/xfs/493
@@ -8,7 +8,7 @@
 # Corrupt the AGFs to test mount failure when mount-fixing fails.
 #
 . ./common/preamble
-_begin_fstest auto quick fuzz
+_begin_fstest auto quick fuzzers
 
 # Import common functions.
 . ./common/filter

[PATCH 4/5] tools: make sure that test groups are described in the documentation

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Create a file to document the purpose of each test group that is
currently defined in fstests, and add a build script to check that every
group mentioned in the tests is also mentioned in the documentation.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/buildgrouplist |    1 
 tools/check-groups     |   33 ++++++++++++
 3 files changed, 170 insertions(+)
 create mode 100644 doc/group-names.txt
 create mode 100755 tools/check-groups


diff --git a/doc/group-names.txt b/doc/group-names.txt
new file mode 100644
index 00000000..ae517328
--- /dev/null
+++ b/doc/group-names.txt
@@ -0,0 +1,136 @@
+======================= =======================================================
+Group Name:		Description:
+======================= =======================================================
+all			All known tests, automatically generated by ./check at
+			runtime
+auto			Tests that should be run automatically.  These should
+			not require more than ~5 minutes to run.
+quick			Tests that should run in under 30 seconds.
+deprecated		Old tests that should not be run.
+
+acl			Access Control Lists
+admin			xfs_admin functionality
+aio			general libaio async io tests
+atime			file access time
+attr			extended attributes
+attr2			xfs v2 extended aributes
+balance			btrfs tree rebalance
+bigtime			timestamps beyond the year 2038
+blockdev		block device functionality
+broken			broken tests
+cap			Linux capabilities
+casefold		directory name casefolding
+ci			ASCII case-insensitive directory name lookups
+clone			FICLONE/FICLONERANGE ioctls
+clone_stress		stress testing FICLONE/FICLONERANGE
+collapse		fallocate collapse_range
+compress		file compression
+convert			btrfs ext[34] conversion tool
+copy			xfs_copy functionality
+copy_range		copy_file_range syscall
+copyup			overlayfs copyup support
+dangerous		dangerous test that can crash the system
+dangerous_bothrepair	fuzzers to evaluate xfs_scrub + xfs_repair repair
+dangerous_fuzzers	fuzzers that can crash your computer
+dangerous_norepair	fuzzers to evaluate kernel metadata verifiers
+dangerous_online_repair	fuzzers to evaluate xfs_scrub online repair
+dangerous_repair	fuzzers to evaluate xfs_repair offline repair
+dangerous_scrub		fuzzers to evaluate xfs_scrub checking
+data			data loss checkers
+dax			direct access mode for persistent memory files
+db			xfs_db functional tests
+dedupe			FIEDEDUPERANGE ioctl
+defrag			filesystem defragmenters
+dir			directory test functions
+dump			dump and restore utilities
+eio			IO error reporting
+encrypt			encrypted file contents
+enospc			ENOSPC error reporting
+exportfs		file handles
+filestreams		XFS filestreams allocator
+freeze			filesystem freeze tests
+fsck			general fsck tests
+fsmap			FS_IOC_GETFSMAP ioctl
+fsr			XFS free space reorganizer
+fuzzers			filesystem fuzz tests
+growfs			increasing the size of a filesystem
+hardlink		hardlinks
+health			XFS health reporting
+idmapped		idmapped mount functionality
+inobtcount		XFS inode btree count tests
+insert			fallocate insert_range
+ioctl			general ioctl tests
+io_uring		general io_uring async io tests
+label			filesystem labelling
+limit			resource limits
+locks			file locking
+log			metadata logging
+logprint		xfs_logprint functional tests
+long_rw			long-soak read write IO path exercisers
+metacopy		overlayfs metadata-only copy-up
+metadata		filesystem metadata update exercisers
+metadump		xfs_metadump/xfs_mdrestore functionality
+mkfs			filesystem formatting tools
+mount			mount option and functionality checks
+nested			nested overlayfs instances
+nfs4_acl		NFSv4 access control lists
+nonsamefs		overlayfs layers on different filesystems
+online_repair		online repair functionality tests
+other			dumping ground, do not add more tests to this group
+overlay			using overlayfs on top of FSTYP
+pattern			specific IO pattern tests
+perms			access control and permission checking
+pipe			pipe functionality
+pnfs			PNFS
+posix			POSIX behavior conformance
+prealloc		fallocate
+preallocrw		fallocate, then read and write
+punch			fallocate punch_hole
+qgroup			btrfs qgroup feature
+quota			filesystem usage quotas
+raid			btrfs RAID
+realtime		XFS realtime volumes
+recoveryloop		crash recovery loops
+redirect		overlayfs redirect_dir feature
+remote			dump and restore with a remote tape
+remount			remounting filesystems
+rename			rename system call
+repair			xfs_repair functional tests
+replace			btrfs device replace
+replay			dm-logwrites replays
+resize			resize2fs functionality tests
+richacl			rich ACL feature
+rmap			XFS reverse mapping exercisers
+rotate			overlayfs feature of some sort
+rw			read/write IO tests
+samefs			overlayfs when all layers are on the same fs
+scrub			filesystem metadata scrubbers
+seed			btrfs seeded filesystems
+seek			llseek functionality
+send			btrfs send/receive
+shrinkfs		decreasing the size of a filesystem
+shutdown		FS_IOC_SHUTDOWN ioctl
+snapshot		btrfs snapshots
+soak			long soak tests of any kind
+spaceman		xfs_spaceman functional tests
+splice			splice system call
+stress			fsstress filesystem exerciser
+subvol			btrfs subvolumes
+subvolume		btrfs subvolumes (again?)
+swap			swap files
+symlink			symbolic links
+tape			dump and restore with a tape
+thin			thin provisioning
+trim			FITRIM ioctl
+udf			UDF functionality tests
+union			tests from the unionmount test suite
+unlink			O_TMPFILE unlinked files
+unshare			fallocate unshare_range
+v2log			XFS v2 log format tests
+verity			fsverity
+volume			btrfs volume management
+whiteout		overlayfs whiteout functionality
+xino			overlayfs xino feature
+zero			fallocate zero_range
+zone			zoned (SMR) device support
+======================= =======================================================
diff --git a/include/buildgrouplist b/include/buildgrouplist
index d898efa3..489de965 100644
--- a/include/buildgrouplist
+++ b/include/buildgrouplist
@@ -6,3 +6,4 @@
 group.list:
 	@echo " [GROUP] $$PWD/$@"
 	$(Q)$(TOPDIR)/tools/mkgroupfile $@
+	$(Q)$(TOPDIR)/tools/check-groups $(TOPDIR)/doc/group-names.txt $@
diff --git a/tools/check-groups b/tools/check-groups
new file mode 100755
index 00000000..bac45bbd
--- /dev/null
+++ b/tools/check-groups
@@ -0,0 +1,33 @@
+#!/bin/bash
+
+# Make sure that all groups listed in a group.list file are mentioned in the
+# group description file.
+
+if [ -z "$1" ] || [ "$1" = "--help" ]; then
+	echo "Usage: $0 path_to_group_names [group.list files...]"
+	exit 1
+fi
+
+groups_doc_file="$1"
+shift
+
+get_group_list() {
+	for file in "$@"; do
+		while read testname groups; do
+			test -z "${testname}" && continue
+			test "${testname:0:1}" = "#" && continue
+
+			echo "${groups}" | tr ' ' '\n'
+		done < "${file}"
+	done | sort | uniq
+}
+
+ret=0
+while read group; do
+	if ! grep -q "^${group}[[:space:]]" "${groups_doc_file}"; then
+		echo "${group}: group not mentioned in documentation." 1>&2
+		ret=1
+	fi
+done < <(get_group_list "$@")
+
+exit "${ret}"

[PATCH 5/5] new: only allow documented test group names

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

Now that we require all group names to be listed in doc/group-names.txt,
we can use that (instead of running mkgroupfile) to check if the group
name(s) supplied by the user actually exist.  This has the secondary
effect of being a second nudge towards keeping the description of groups
up to date.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 new |   24 +++++++++++-------------
 1 file changed, 11 insertions(+), 13 deletions(-)


diff --git a/new b/new
index 2097a883..44777bd6 100755
--- a/new
+++ b/new
@@ -83,6 +83,14 @@ then
     exit 1
 fi
 
+# Extract group names from the documentation.
+group_names() {
+	grep '^[[:lower:][:digit:]_]' doc/group-names.txt | awk '
+{if ($1 != "" && $1 != "Group" && $2 != "Name:" && $1 != "all")
+	printf("%s\n", $1);
+}'
+}
+
 if [ $# -eq 0 ]
 then
 
@@ -93,16 +101,7 @@ then
 	[ -z "$ans" ] && ans=other
 	if [ "X$ans" = "X?" ]
 	then
-	    for d in $SRC_GROUPS; do
-		(cd "tests/$d/" ; ../../tools/mkgroupfile "$tmpfile")
-		l=$(sed -n < "$tmpfile" \
-		    -e 's/#.*//' \
-		    -e 's/$/ /' \
-		    -e 's;\(^[0-9][0-9][0-9]\)\(.*$\);\2;p')
-		grpl="$grpl $l"
-	    done
-	    lst=`for word in $grpl; do echo $word; done | sort| uniq `
-	    echo $lst
+	    echo $(group_names)
 	else
 	    # only allow lower cases, spaces, digits and underscore in group
 	    inval=`echo $ans | tr -d '[:lower:][:space:][:digit:]_'`
@@ -120,11 +119,10 @@ then
 else
     # expert mode, groups are on the command line
     #
-    (cd "$tdir" ; ../../tools/mkgroupfile "$tmpfile")
     for g in $*
     do
-	if ! grep -q "[[:space:]]$g" "$tmpfile"; then
-	    echo "Warning: group \"$g\" not defined in $tdir tests"
+	if ! grep -q "^$g" doc/group-names.txt; then
+	    echo "Warning: group \"$g\" not defined in documentation"
 	fi
     done
     ans="$*"

Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

[Amir Goldstein]

On Wed, Sep 1, 2021 at 3:37 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> From: Darrick J. Wong <djwong@kernel.org>
>
> Create a file to document the purpose of each test group that is
> currently defined in fstests, and add a build script to check that every
> group mentioned in the tests is also mentioned in the documentation.
>

This is awesome and long due.
Thanks for doing that!

Minor nits about overlayfs groups below...

> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
>  include/buildgrouplist |    1
>  tools/check-groups     |   33 ++++++++++++
>  3 files changed, 170 insertions(+)
>  create mode 100644 doc/group-names.txt
>  create mode 100755 tools/check-groups
>
>
> diff --git a/doc/group-names.txt b/doc/group-names.txt
> new file mode 100644
> index 00000000..ae517328
> --- /dev/null
> +++ b/doc/group-names.txt
> @@ -0,0 +1,136 @@
> +======================= =======================================================
> +Group Name:            Description:
> +======================= =======================================================
> +all                    All known tests, automatically generated by ./check at
> +                       runtime
> +auto                   Tests that should be run automatically.  These should
> +                       not require more than ~5 minutes to run.
> +quick                  Tests that should run in under 30 seconds.
> +deprecated             Old tests that should not be run.
> +
> +acl                    Access Control Lists
> +admin                  xfs_admin functionality
> +aio                    general libaio async io tests
> +atime                  file access time
> +attr                   extended attributes
> +attr2                  xfs v2 extended aributes
> +balance                        btrfs tree rebalance
> +bigtime                        timestamps beyond the year 2038
> +blockdev               block device functionality
> +broken                 broken tests
> +cap                    Linux capabilities
> +casefold               directory name casefolding
> +ci                     ASCII case-insensitive directory name lookups
> +clone                  FICLONE/FICLONERANGE ioctls
> +clone_stress           stress testing FICLONE/FICLONERANGE
> +collapse               fallocate collapse_range
> +compress               file compression
> +convert                        btrfs ext[34] conversion tool
> +copy                   xfs_copy functionality
> +copy_range             copy_file_range syscall
> +copyup                 overlayfs copyup support

The tests in this group exercise copy up.
There is no such thing as overlayfs without "copyup support",
so guess my point is - remove the word "support"

> +dangerous              dangerous test that can crash the system
> +dangerous_bothrepair   fuzzers to evaluate xfs_scrub + xfs_repair repair
> +dangerous_fuzzers      fuzzers that can crash your computer
> +dangerous_norepair     fuzzers to evaluate kernel metadata verifiers
> +dangerous_online_repair        fuzzers to evaluate xfs_scrub online repair
> +dangerous_repair       fuzzers to evaluate xfs_repair offline repair
> +dangerous_scrub                fuzzers to evaluate xfs_scrub checking
> +data                   data loss checkers
> +dax                    direct access mode for persistent memory files
> +db                     xfs_db functional tests
> +dedupe                 FIEDEDUPERANGE ioctl
> +defrag                 filesystem defragmenters
> +dir                    directory test functions
> +dump                   dump and restore utilities
> +eio                    IO error reporting
> +encrypt                        encrypted file contents
> +enospc                 ENOSPC error reporting
> +exportfs               file handles
> +filestreams            XFS filestreams allocator
> +freeze                 filesystem freeze tests
> +fsck                   general fsck tests
> +fsmap                  FS_IOC_GETFSMAP ioctl
> +fsr                    XFS free space reorganizer
> +fuzzers                        filesystem fuzz tests
> +growfs                 increasing the size of a filesystem
> +hardlink               hardlinks
> +health                 XFS health reporting
> +idmapped               idmapped mount functionality
> +inobtcount             XFS inode btree count tests
> +insert                 fallocate insert_range
> +ioctl                  general ioctl tests
> +io_uring               general io_uring async io tests
> +label                  filesystem labelling
> +limit                  resource limits
> +locks                  file locking
> +log                    metadata logging
> +logprint               xfs_logprint functional tests
> +long_rw                        long-soak read write IO path exercisers
> +metacopy               overlayfs metadata-only copy-up
> +metadata               filesystem metadata update exercisers
> +metadump               xfs_metadump/xfs_mdrestore functionality
> +mkfs                   filesystem formatting tools
> +mount                  mount option and functionality checks
> +nested                 nested overlayfs instances
> +nfs4_acl               NFSv4 access control lists
> +nonsamefs              overlayfs layers on different filesystems
> +online_repair          online repair functionality tests
> +other                  dumping ground, do not add more tests to this group
> +overlay                        using overlayfs on top of FSTYP

This description is a bit confusing, because the recommended
way to run overlayfs tests as described in README.overlay is
to set FSTYP=xfs and run ./check -overlay

I'm struggling for a better description but perhaps:
"using overlayfs regardless of ./check -overlay flag"?


> +pattern                        specific IO pattern tests
> +perms                  access control and permission checking
> +pipe                   pipe functionality
> +pnfs                   PNFS
> +posix                  POSIX behavior conformance
> +prealloc               fallocate
> +preallocrw             fallocate, then read and write
> +punch                  fallocate punch_hole
> +qgroup                 btrfs qgroup feature
> +quota                  filesystem usage quotas
> +raid                   btrfs RAID
> +realtime               XFS realtime volumes
> +recoveryloop           crash recovery loops
> +redirect               overlayfs redirect_dir feature
> +remote                 dump and restore with a remote tape
> +remount                        remounting filesystems
> +rename                 rename system call
> +repair                 xfs_repair functional tests
> +replace                        btrfs device replace
> +replay                 dm-logwrites replays
> +resize                 resize2fs functionality tests
> +richacl                        rich ACL feature
> +rmap                   XFS reverse mapping exercisers
> +rotate                 overlayfs feature of some sort"

I guess that works :-D
but to be accurate, this is actually a unionmount testsuite feature -
at selected test points in the workload, a new upper layer is stacked
on to overlayfs, so maybe:
"upper layer rotate tests from the unionmount test suite"?

> +rw                     read/write IO tests
> +samefs                 overlayfs when all layers are on the same fs
> +scrub                  filesystem metadata scrubbers
> +seed                   btrfs seeded filesystems
> +seek                   llseek functionality
> +send                   btrfs send/receive
> +shrinkfs               decreasing the size of a filesystem
> +shutdown               FS_IOC_SHUTDOWN ioctl
> +snapshot               btrfs snapshots
> +soak                   long soak tests of any kind
> +spaceman               xfs_spaceman functional tests
> +splice                 splice system call
> +stress                 fsstress filesystem exerciser
> +subvol                 btrfs subvolumes
> +subvolume              btrfs subvolumes (again?)

A cleanup patch to fix this typo in btrfs/233?

Thanks,
Amir.

Re: [PATCH 1/5] ceph: re-tag copy_file_range as being in the copy_range group

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/5] xfs: move reflink tests into the clone group

[Christoph Hellwig]

On Tue, Aug 31, 2021 at 05:12:40PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> "clone" is the group for tests that exercise FICLONERANGE, so move these
> tests.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 3/5] xfs: fix incorrect fuzz test group name

[Christoph Hellwig]

On Tue, Aug 31, 2021 at 05:12:46PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> The group name for fuzz tests is 'fuzzers'.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

[Christoph Hellwig]

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 5/5] new: only allow documented test group names

[Christoph Hellwig]

On Tue, Aug 31, 2021 at 05:12:57PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Now that we require all group names to be listed in doc/group-names.txt,
> we can use that (instead of running mkgroupfile) to check if the group
> name(s) supplied by the user actually exist.  This has the secondary
> effect of being a second nudge towards keeping the description of groups
> up to date.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

[Darrick J. Wong]

On Wed, Sep 01, 2021 at 07:46:01AM +0300, Amir Goldstein wrote:
> On Wed, Sep 1, 2021 at 3:37 AM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > Create a file to document the purpose of each test group that is
> > currently defined in fstests, and add a build script to check that every
> > group mentioned in the tests is also mentioned in the documentation.
> >
> 
> This is awesome and long due.
> Thanks for doing that!
> 
> Minor nits about overlayfs groups below...

Heh, yeah, thanks for making corrections. :)

> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
> >  include/buildgrouplist |    1
> >  tools/check-groups     |   33 ++++++++++++
> >  3 files changed, 170 insertions(+)
> >  create mode 100644 doc/group-names.txt
> >  create mode 100755 tools/check-groups
> >
> >
> > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > new file mode 100644
> > index 00000000..ae517328
> > --- /dev/null
> > +++ b/doc/group-names.txt
> > @@ -0,0 +1,136 @@
> > +======================= =======================================================
> > +Group Name:            Description:
> > +======================= =======================================================
> > +all                    All known tests, automatically generated by ./check at
> > +                       runtime
> > +auto                   Tests that should be run automatically.  These should
> > +                       not require more than ~5 minutes to run.
> > +quick                  Tests that should run in under 30 seconds.
> > +deprecated             Old tests that should not be run.
> > +
> > +acl                    Access Control Lists
> > +admin                  xfs_admin functionality
> > +aio                    general libaio async io tests
> > +atime                  file access time
> > +attr                   extended attributes
> > +attr2                  xfs v2 extended aributes
> > +balance                        btrfs tree rebalance
> > +bigtime                        timestamps beyond the year 2038
> > +blockdev               block device functionality
> > +broken                 broken tests
> > +cap                    Linux capabilities
> > +casefold               directory name casefolding
> > +ci                     ASCII case-insensitive directory name lookups
> > +clone                  FICLONE/FICLONERANGE ioctls
> > +clone_stress           stress testing FICLONE/FICLONERANGE
> > +collapse               fallocate collapse_range
> > +compress               file compression
> > +convert                        btrfs ext[34] conversion tool
> > +copy                   xfs_copy functionality
> > +copy_range             copy_file_range syscall
> > +copyup                 overlayfs copyup support
> 
> The tests in this group exercise copy up.
> There is no such thing as overlayfs without "copyup support",
> so guess my point is - remove the word "support"

OK.

> > +dangerous              dangerous test that can crash the system
> > +dangerous_bothrepair   fuzzers to evaluate xfs_scrub + xfs_repair repair
> > +dangerous_fuzzers      fuzzers that can crash your computer
> > +dangerous_norepair     fuzzers to evaluate kernel metadata verifiers
> > +dangerous_online_repair        fuzzers to evaluate xfs_scrub online repair
> > +dangerous_repair       fuzzers to evaluate xfs_repair offline repair
> > +dangerous_scrub                fuzzers to evaluate xfs_scrub checking
> > +data                   data loss checkers
> > +dax                    direct access mode for persistent memory files
> > +db                     xfs_db functional tests
> > +dedupe                 FIEDEDUPERANGE ioctl
> > +defrag                 filesystem defragmenters
> > +dir                    directory test functions
> > +dump                   dump and restore utilities
> > +eio                    IO error reporting
> > +encrypt                        encrypted file contents
> > +enospc                 ENOSPC error reporting
> > +exportfs               file handles
> > +filestreams            XFS filestreams allocator
> > +freeze                 filesystem freeze tests
> > +fsck                   general fsck tests
> > +fsmap                  FS_IOC_GETFSMAP ioctl
> > +fsr                    XFS free space reorganizer
> > +fuzzers                        filesystem fuzz tests
> > +growfs                 increasing the size of a filesystem
> > +hardlink               hardlinks
> > +health                 XFS health reporting
> > +idmapped               idmapped mount functionality
> > +inobtcount             XFS inode btree count tests
> > +insert                 fallocate insert_range
> > +ioctl                  general ioctl tests
> > +io_uring               general io_uring async io tests
> > +label                  filesystem labelling
> > +limit                  resource limits
> > +locks                  file locking
> > +log                    metadata logging
> > +logprint               xfs_logprint functional tests
> > +long_rw                        long-soak read write IO path exercisers
> > +metacopy               overlayfs metadata-only copy-up
> > +metadata               filesystem metadata update exercisers
> > +metadump               xfs_metadump/xfs_mdrestore functionality
> > +mkfs                   filesystem formatting tools
> > +mount                  mount option and functionality checks
> > +nested                 nested overlayfs instances
> > +nfs4_acl               NFSv4 access control lists
> > +nonsamefs              overlayfs layers on different filesystems
> > +online_repair          online repair functionality tests
> > +other                  dumping ground, do not add more tests to this group
> > +overlay                        using overlayfs on top of FSTYP
> 
> This description is a bit confusing, because the recommended
> way to run overlayfs tests as described in README.overlay is
> to set FSTYP=xfs and run ./check -overlay
> 
> I'm struggling for a better description but perhaps:
> "using overlayfs regardless of ./check -overlay flag"?

Hmm.  Since I'm the author of the only test that uses this tag, I guess
I'm the authority (ha!) on what the name actually means.

That test (generic/631) is a regression test for a XFS whiteout handling
bug that can only be reproduced by layering overlayfs atop xfs.
Overlayfs is incidental to reproducing the XFS bug, but AFAIK overlayfs
is the only in-kernel user of whiteout, which is why it's critical here.

It's not right to make it "_supported_fs overlay" because we're not
testing overlayfs functionality; we're merely using it as a stick to
poke another filesystem.

How about: "regression tests that require the use of overlayfs in a
targetted configuration" ?

> > +pattern                        specific IO pattern tests
> > +perms                  access control and permission checking
> > +pipe                   pipe functionality
> > +pnfs                   PNFS
> > +posix                  POSIX behavior conformance
> > +prealloc               fallocate
> > +preallocrw             fallocate, then read and write
> > +punch                  fallocate punch_hole
> > +qgroup                 btrfs qgroup feature
> > +quota                  filesystem usage quotas
> > +raid                   btrfs RAID
> > +realtime               XFS realtime volumes
> > +recoveryloop           crash recovery loops
> > +redirect               overlayfs redirect_dir feature
> > +remote                 dump and restore with a remote tape
> > +remount                        remounting filesystems
> > +rename                 rename system call
> > +repair                 xfs_repair functional tests
> > +replace                        btrfs device replace
> > +replay                 dm-logwrites replays
> > +resize                 resize2fs functionality tests
> > +richacl                        rich ACL feature
> > +rmap                   XFS reverse mapping exercisers
> > +rotate                 overlayfs feature of some sort"
> 
> I guess that works :-D
> but to be accurate, this is actually a unionmount testsuite feature -
> at selected test points in the workload, a new upper layer is stacked
> on to overlayfs, so maybe:
> "upper layer rotate tests from the unionmount test suite"?

I've changed it to 'overlayfs upper layer rotate tests from the
unionmount test suite'.

> > +rw                     read/write IO tests
> > +samefs                 overlayfs when all layers are on the same fs
> > +scrub                  filesystem metadata scrubbers
> > +seed                   btrfs seeded filesystems
> > +seek                   llseek functionality
> > +send                   btrfs send/receive
> > +shrinkfs               decreasing the size of a filesystem
> > +shutdown               FS_IOC_SHUTDOWN ioctl
> > +snapshot               btrfs snapshots
> > +soak                   long soak tests of any kind
> > +spaceman               xfs_spaceman functional tests
> > +splice                 splice system call
> > +stress                 fsstress filesystem exerciser
> > +subvol                 btrfs subvolumes
> > +subvolume              btrfs subvolumes (again?)
> 
> A cleanup patch to fix this typo in btrfs/233?

Will do.

--D

> 
> Thanks,
> Amir.

Re: [PATCH 5/5] new: only allow documented test group names

[Dave Chinner]

On Tue, Aug 31, 2021 at 05:12:57PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> Now that we require all group names to be listed in doc/group-names.txt,
> we can use that (instead of running mkgroupfile) to check if the group
> name(s) supplied by the user actually exist.  This has the secondary
> effect of being a second nudge towards keeping the description of groups
> up to date.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  new |   24 +++++++++++-------------
>  1 file changed, 11 insertions(+), 13 deletions(-)
> 
> 
> diff --git a/new b/new
> index 2097a883..44777bd6 100755
> --- a/new
> +++ b/new
> @@ -83,6 +83,14 @@ then
>      exit 1
>  fi
>  
> +# Extract group names from the documentation.
> +group_names() {
> +	grep '^[[:lower:][:digit:]_]' doc/group-names.txt | awk '
> +{if ($1 != "" && $1 != "Group" && $2 != "Name:" && $1 != "all")
> +	printf("%s\n", $1);
> +}'
> +}

Took me a while to realise this was running an awk script for output
slection but using grep for regex based line selection. Awk can do
both of these things just fine, and the result is much more
readable:

group_names() {
	awk '/^[[:lower:][:digit:]_]/ {
		if ($1 != "Group" && $2 != "Name:" && $1 != "all")
			printf("%s\n", $1);
	}' doc/group-names.txt
}


$ awk '/^[[:lower:][:digit:]_]/ { if ($1 != "Group" && $2 != "Name:" && $1 != "all") printf("%s\n", $1); }' t.t
auto
quick
deprecated
acl
admin
aio
atime
....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

[Amir Goldstein]

On Wed, Sep 1, 2021 at 7:43 PM Darrick J. Wong <djwong@kernel.org> wrote:
>
> On Wed, Sep 01, 2021 at 07:46:01AM +0300, Amir Goldstein wrote:
> > On Wed, Sep 1, 2021 at 3:37 AM Darrick J. Wong <djwong@kernel.org> wrote:
> > >
> > > From: Darrick J. Wong <djwong@kernel.org>
> > >
> > > Create a file to document the purpose of each test group that is
> > > currently defined in fstests, and add a build script to check that every
> > > group mentioned in the tests is also mentioned in the documentation.
> > >
> >
> > This is awesome and long due.
> > Thanks for doing that!
> >
> > Minor nits about overlayfs groups below...
>
> Heh, yeah, thanks for making corrections. :)
>
> > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > ---
> > >  doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
> > >  include/buildgrouplist |    1
> > >  tools/check-groups     |   33 ++++++++++++
> > >  3 files changed, 170 insertions(+)
> > >  create mode 100644 doc/group-names.txt
> > >  create mode 100755 tools/check-groups
> > >
> > >
> > > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > > new file mode 100644
> > > index 00000000..ae517328
> > > --- /dev/null
> > > +++ b/doc/group-names.txt
> > > @@ -0,0 +1,136 @@
> > > +======================= =======================================================
> > > +Group Name:            Description:
> > > +======================= =======================================================
> > > +all                    All known tests, automatically generated by ./check at
> > > +                       runtime
> > > +auto                   Tests that should be run automatically.  These should
> > > +                       not require more than ~5 minutes to run.
> > > +quick                  Tests that should run in under 30 seconds.
> > > +deprecated             Old tests that should not be run.
> > > +
> > > +acl                    Access Control Lists
> > > +admin                  xfs_admin functionality
> > > +aio                    general libaio async io tests
> > > +atime                  file access time
> > > +attr                   extended attributes
> > > +attr2                  xfs v2 extended aributes
> > > +balance                        btrfs tree rebalance
> > > +bigtime                        timestamps beyond the year 2038
> > > +blockdev               block device functionality
> > > +broken                 broken tests
> > > +cap                    Linux capabilities
> > > +casefold               directory name casefolding
> > > +ci                     ASCII case-insensitive directory name lookups
> > > +clone                  FICLONE/FICLONERANGE ioctls
> > > +clone_stress           stress testing FICLONE/FICLONERANGE
> > > +collapse               fallocate collapse_range
> > > +compress               file compression
> > > +convert                        btrfs ext[34] conversion tool
> > > +copy                   xfs_copy functionality
> > > +copy_range             copy_file_range syscall
> > > +copyup                 overlayfs copyup support
> >
> > The tests in this group exercise copy up.
> > There is no such thing as overlayfs without "copyup support",
> > so guess my point is - remove the word "support"
>
> OK.
>
> > > +dangerous              dangerous test that can crash the system
> > > +dangerous_bothrepair   fuzzers to evaluate xfs_scrub + xfs_repair repair
> > > +dangerous_fuzzers      fuzzers that can crash your computer
> > > +dangerous_norepair     fuzzers to evaluate kernel metadata verifiers
> > > +dangerous_online_repair        fuzzers to evaluate xfs_scrub online repair
> > > +dangerous_repair       fuzzers to evaluate xfs_repair offline repair
> > > +dangerous_scrub                fuzzers to evaluate xfs_scrub checking
> > > +data                   data loss checkers
> > > +dax                    direct access mode for persistent memory files
> > > +db                     xfs_db functional tests
> > > +dedupe                 FIEDEDUPERANGE ioctl
> > > +defrag                 filesystem defragmenters
> > > +dir                    directory test functions
> > > +dump                   dump and restore utilities
> > > +eio                    IO error reporting
> > > +encrypt                        encrypted file contents
> > > +enospc                 ENOSPC error reporting
> > > +exportfs               file handles
> > > +filestreams            XFS filestreams allocator
> > > +freeze                 filesystem freeze tests
> > > +fsck                   general fsck tests
> > > +fsmap                  FS_IOC_GETFSMAP ioctl
> > > +fsr                    XFS free space reorganizer
> > > +fuzzers                        filesystem fuzz tests
> > > +growfs                 increasing the size of a filesystem
> > > +hardlink               hardlinks
> > > +health                 XFS health reporting
> > > +idmapped               idmapped mount functionality
> > > +inobtcount             XFS inode btree count tests
> > > +insert                 fallocate insert_range
> > > +ioctl                  general ioctl tests
> > > +io_uring               general io_uring async io tests
> > > +label                  filesystem labelling
> > > +limit                  resource limits
> > > +locks                  file locking
> > > +log                    metadata logging
> > > +logprint               xfs_logprint functional tests
> > > +long_rw                        long-soak read write IO path exercisers
> > > +metacopy               overlayfs metadata-only copy-up
> > > +metadata               filesystem metadata update exercisers
> > > +metadump               xfs_metadump/xfs_mdrestore functionality
> > > +mkfs                   filesystem formatting tools
> > > +mount                  mount option and functionality checks
> > > +nested                 nested overlayfs instances
> > > +nfs4_acl               NFSv4 access control lists
> > > +nonsamefs              overlayfs layers on different filesystems
> > > +online_repair          online repair functionality tests
> > > +other                  dumping ground, do not add more tests to this group
> > > +overlay                        using overlayfs on top of FSTYP
> >
> > This description is a bit confusing, because the recommended
> > way to run overlayfs tests as described in README.overlay is
> > to set FSTYP=xfs and run ./check -overlay
> >
> > I'm struggling for a better description but perhaps:
> > "using overlayfs regardless of ./check -overlay flag"?
>
> Hmm.  Since I'm the author of the only test that uses this tag, I guess
> I'm the authority (ha!) on what the name actually means.
>
> That test (generic/631) is a regression test for a XFS whiteout handling
> bug that can only be reproduced by layering overlayfs atop xfs.
> Overlayfs is incidental to reproducing the XFS bug, but AFAIK overlayfs
> is the only in-kernel user of whiteout, which is why it's critical here.
>
> It's not right to make it "_supported_fs overlay" because we're not
> testing overlayfs functionality; we're merely using it as a stick to
> poke another filesystem.

Yes. I know.
Note that while this is the only case of _require_extra_fs overaly
there is another case of _require_extra_fs ext2 (xfs/049)

>
> How about: "regression tests that require the use of overlayfs in a
> targetted configuration" ?
>

TBH, I do not think it is wise to tag the test by the test method
rather than the tested functionality.

What is more likely?
that a tester wants to run all tests that use overlay over FSTYP?
Or that a tester wants to run all tests to verify whiteout related
behavior after changing whiteout related code?

I suggest that you re-tag this test as 'whiteout', which is documented
already.

If you want to be more specific, you can create a group
rename_whiteout, because RENAME_WHITEOUT is the vfs
interface that this test is actually exercising.

Thanks,
Amir.

Re: [PATCH 4/5] tools: make sure that test groups are described in the documentation

[Darrick J. Wong]

On Thu, Sep 02, 2021 at 07:49:51AM +0300, Amir Goldstein wrote:
> On Wed, Sep 1, 2021 at 7:43 PM Darrick J. Wong <djwong@kernel.org> wrote:
> >
> > On Wed, Sep 01, 2021 at 07:46:01AM +0300, Amir Goldstein wrote:
> > > On Wed, Sep 1, 2021 at 3:37 AM Darrick J. Wong <djwong@kernel.org> wrote:
> > > >
> > > > From: Darrick J. Wong <djwong@kernel.org>
> > > >
> > > > Create a file to document the purpose of each test group that is
> > > > currently defined in fstests, and add a build script to check that every
> > > > group mentioned in the tests is also mentioned in the documentation.
> > > >
> > >
> > > This is awesome and long due.
> > > Thanks for doing that!
> > >
> > > Minor nits about overlayfs groups below...
> >
> > Heh, yeah, thanks for making corrections. :)
> >
> > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > > > ---
> > > >  doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
> > > >  include/buildgrouplist |    1
> > > >  tools/check-groups     |   33 ++++++++++++
> > > >  3 files changed, 170 insertions(+)
> > > >  create mode 100644 doc/group-names.txt
> > > >  create mode 100755 tools/check-groups
> > > >
> > > >
> > > > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > > > new file mode 100644
> > > > index 00000000..ae517328
> > > > --- /dev/null
> > > > +++ b/doc/group-names.txt
> > > > @@ -0,0 +1,136 @@
> > > > +======================= =======================================================
> > > > +Group Name:            Description:
> > > > +======================= =======================================================
> > > > +all                    All known tests, automatically generated by ./check at
> > > > +                       runtime
> > > > +auto                   Tests that should be run automatically.  These should
> > > > +                       not require more than ~5 minutes to run.
> > > > +quick                  Tests that should run in under 30 seconds.
> > > > +deprecated             Old tests that should not be run.
> > > > +
> > > > +acl                    Access Control Lists
> > > > +admin                  xfs_admin functionality
> > > > +aio                    general libaio async io tests
> > > > +atime                  file access time
> > > > +attr                   extended attributes
> > > > +attr2                  xfs v2 extended aributes
> > > > +balance                        btrfs tree rebalance
> > > > +bigtime                        timestamps beyond the year 2038
> > > > +blockdev               block device functionality
> > > > +broken                 broken tests
> > > > +cap                    Linux capabilities
> > > > +casefold               directory name casefolding
> > > > +ci                     ASCII case-insensitive directory name lookups
> > > > +clone                  FICLONE/FICLONERANGE ioctls
> > > > +clone_stress           stress testing FICLONE/FICLONERANGE
> > > > +collapse               fallocate collapse_range
> > > > +compress               file compression
> > > > +convert                        btrfs ext[34] conversion tool
> > > > +copy                   xfs_copy functionality
> > > > +copy_range             copy_file_range syscall
> > > > +copyup                 overlayfs copyup support
> > >
> > > The tests in this group exercise copy up.
> > > There is no such thing as overlayfs without "copyup support",
> > > so guess my point is - remove the word "support"
> >
> > OK.
> >
> > > > +dangerous              dangerous test that can crash the system
> > > > +dangerous_bothrepair   fuzzers to evaluate xfs_scrub + xfs_repair repair
> > > > +dangerous_fuzzers      fuzzers that can crash your computer
> > > > +dangerous_norepair     fuzzers to evaluate kernel metadata verifiers
> > > > +dangerous_online_repair        fuzzers to evaluate xfs_scrub online repair
> > > > +dangerous_repair       fuzzers to evaluate xfs_repair offline repair
> > > > +dangerous_scrub                fuzzers to evaluate xfs_scrub checking
> > > > +data                   data loss checkers
> > > > +dax                    direct access mode for persistent memory files
> > > > +db                     xfs_db functional tests
> > > > +dedupe                 FIEDEDUPERANGE ioctl
> > > > +defrag                 filesystem defragmenters
> > > > +dir                    directory test functions
> > > > +dump                   dump and restore utilities
> > > > +eio                    IO error reporting
> > > > +encrypt                        encrypted file contents
> > > > +enospc                 ENOSPC error reporting
> > > > +exportfs               file handles
> > > > +filestreams            XFS filestreams allocator
> > > > +freeze                 filesystem freeze tests
> > > > +fsck                   general fsck tests
> > > > +fsmap                  FS_IOC_GETFSMAP ioctl
> > > > +fsr                    XFS free space reorganizer
> > > > +fuzzers                        filesystem fuzz tests
> > > > +growfs                 increasing the size of a filesystem
> > > > +hardlink               hardlinks
> > > > +health                 XFS health reporting
> > > > +idmapped               idmapped mount functionality
> > > > +inobtcount             XFS inode btree count tests
> > > > +insert                 fallocate insert_range
> > > > +ioctl                  general ioctl tests
> > > > +io_uring               general io_uring async io tests
> > > > +label                  filesystem labelling
> > > > +limit                  resource limits
> > > > +locks                  file locking
> > > > +log                    metadata logging
> > > > +logprint               xfs_logprint functional tests
> > > > +long_rw                        long-soak read write IO path exercisers
> > > > +metacopy               overlayfs metadata-only copy-up
> > > > +metadata               filesystem metadata update exercisers
> > > > +metadump               xfs_metadump/xfs_mdrestore functionality
> > > > +mkfs                   filesystem formatting tools
> > > > +mount                  mount option and functionality checks
> > > > +nested                 nested overlayfs instances
> > > > +nfs4_acl               NFSv4 access control lists
> > > > +nonsamefs              overlayfs layers on different filesystems
> > > > +online_repair          online repair functionality tests
> > > > +other                  dumping ground, do not add more tests to this group
> > > > +overlay                        using overlayfs on top of FSTYP
> > >
> > > This description is a bit confusing, because the recommended
> > > way to run overlayfs tests as described in README.overlay is
> > > to set FSTYP=xfs and run ./check -overlay
> > >
> > > I'm struggling for a better description but perhaps:
> > > "using overlayfs regardless of ./check -overlay flag"?
> >
> > Hmm.  Since I'm the author of the only test that uses this tag, I guess
> > I'm the authority (ha!) on what the name actually means.
> >
> > That test (generic/631) is a regression test for a XFS whiteout handling
> > bug that can only be reproduced by layering overlayfs atop xfs.
> > Overlayfs is incidental to reproducing the XFS bug, but AFAIK overlayfs
> > is the only in-kernel user of whiteout, which is why it's critical here.
> >
> > It's not right to make it "_supported_fs overlay" because we're not
> > testing overlayfs functionality; we're merely using it as a stick to
> > poke another filesystem.
> 
> Yes. I know.
> Note that while this is the only case of _require_extra_fs overaly
> there is another case of _require_extra_fs ext2 (xfs/049)
> 
> >
> > How about: "regression tests that require the use of overlayfs in a
> > targetted configuration" ?
> >
> 
> TBH, I do not think it is wise to tag the test by the test method
> rather than the tested functionality.

<nod> It seemed kinda awkward to me.

> What is more likely?
> that a tester wants to run all tests that use overlay over FSTYP?
> Or that a tester wants to run all tests to verify whiteout related
> behavior after changing whiteout related code?
> 
> I suggest that you re-tag this test as 'whiteout', which is documented
> already.

Ooh!  That's a much better suggestion.  I'll do that instead! :)

> If you want to be more specific, you can create a group
> rename_whiteout, because RENAME_WHITEOUT is the vfs
> interface that this test is actually exercising.

Eh, if we want to split the groups someday we can always revisit it.

Thanks for the suggestion; I'll have a v2 series out shortly.

--D

> 
> Thanks,
> Amir.