All of
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <>
Subject: [PATCH 4/5] tools: make sure that test groups are described in the documentation
Date: Tue, 31 Aug 2021 17:12:51 -0700	[thread overview]
Message-ID: <163045517173.771564.1524162806861567173.stgit@magnolia> (raw)
In-Reply-To: <163045514980.771564.6282165259140399788.stgit@magnolia>

From: Darrick J. Wong <>

Create a file to document the purpose of each test group that is
currently defined in fstests, and add a build script to check that every
group mentioned in the tests is also mentioned in the documentation.

Signed-off-by: Darrick J. Wong <>
 doc/group-names.txt    |  136 ++++++++++++++++++++++++++++++++++++++++++++++++
 include/buildgrouplist |    1 
 tools/check-groups     |   33 ++++++++++++
 3 files changed, 170 insertions(+)
 create mode 100644 doc/group-names.txt
 create mode 100755 tools/check-groups

diff --git a/doc/group-names.txt b/doc/group-names.txt
new file mode 100644
index 00000000..ae517328
--- /dev/null
+++ b/doc/group-names.txt
@@ -0,0 +1,136 @@
+======================= =======================================================
+Group Name:		Description:
+======================= =======================================================
+all			All known tests, automatically generated by ./check at
+			runtime
+auto			Tests that should be run automatically.  These should
+			not require more than ~5 minutes to run.
+quick			Tests that should run in under 30 seconds.
+deprecated		Old tests that should not be run.
+acl			Access Control Lists
+admin			xfs_admin functionality
+aio			general libaio async io tests
+atime			file access time
+attr			extended attributes
+attr2			xfs v2 extended aributes
+balance			btrfs tree rebalance
+bigtime			timestamps beyond the year 2038
+blockdev		block device functionality
+broken			broken tests
+cap			Linux capabilities
+casefold		directory name casefolding
+ci			ASCII case-insensitive directory name lookups
+clone_stress		stress testing FICLONE/FICLONERANGE
+collapse		fallocate collapse_range
+compress		file compression
+convert			btrfs ext[34] conversion tool
+copy			xfs_copy functionality
+copy_range		copy_file_range syscall
+copyup			overlayfs copyup support
+dangerous		dangerous test that can crash the system
+dangerous_bothrepair	fuzzers to evaluate xfs_scrub + xfs_repair repair
+dangerous_fuzzers	fuzzers that can crash your computer
+dangerous_norepair	fuzzers to evaluate kernel metadata verifiers
+dangerous_online_repair	fuzzers to evaluate xfs_scrub online repair
+dangerous_repair	fuzzers to evaluate xfs_repair offline repair
+dangerous_scrub		fuzzers to evaluate xfs_scrub checking
+data			data loss checkers
+dax			direct access mode for persistent memory files
+db			xfs_db functional tests
+dedupe			FIEDEDUPERANGE ioctl
+defrag			filesystem defragmenters
+dir			directory test functions
+dump			dump and restore utilities
+eio			IO error reporting
+encrypt			encrypted file contents
+enospc			ENOSPC error reporting
+exportfs		file handles
+filestreams		XFS filestreams allocator
+freeze			filesystem freeze tests
+fsck			general fsck tests
+fsmap			FS_IOC_GETFSMAP ioctl
+fsr			XFS free space reorganizer
+fuzzers			filesystem fuzz tests
+growfs			increasing the size of a filesystem
+hardlink		hardlinks
+health			XFS health reporting
+idmapped		idmapped mount functionality
+inobtcount		XFS inode btree count tests
+insert			fallocate insert_range
+ioctl			general ioctl tests
+io_uring		general io_uring async io tests
+label			filesystem labelling
+limit			resource limits
+locks			file locking
+log			metadata logging
+logprint		xfs_logprint functional tests
+long_rw			long-soak read write IO path exercisers
+metacopy		overlayfs metadata-only copy-up
+metadata		filesystem metadata update exercisers
+metadump		xfs_metadump/xfs_mdrestore functionality
+mkfs			filesystem formatting tools
+mount			mount option and functionality checks
+nested			nested overlayfs instances
+nfs4_acl		NFSv4 access control lists
+nonsamefs		overlayfs layers on different filesystems
+online_repair		online repair functionality tests
+other			dumping ground, do not add more tests to this group
+overlay			using overlayfs on top of FSTYP
+pattern			specific IO pattern tests
+perms			access control and permission checking
+pipe			pipe functionality
+pnfs			PNFS
+posix			POSIX behavior conformance
+prealloc		fallocate
+preallocrw		fallocate, then read and write
+punch			fallocate punch_hole
+qgroup			btrfs qgroup feature
+quota			filesystem usage quotas
+raid			btrfs RAID
+realtime		XFS realtime volumes
+recoveryloop		crash recovery loops
+redirect		overlayfs redirect_dir feature
+remote			dump and restore with a remote tape
+remount			remounting filesystems
+rename			rename system call
+repair			xfs_repair functional tests
+replace			btrfs device replace
+replay			dm-logwrites replays
+resize			resize2fs functionality tests
+richacl			rich ACL feature
+rmap			XFS reverse mapping exercisers
+rotate			overlayfs feature of some sort
+rw			read/write IO tests
+samefs			overlayfs when all layers are on the same fs
+scrub			filesystem metadata scrubbers
+seed			btrfs seeded filesystems
+seek			llseek functionality
+send			btrfs send/receive
+shrinkfs		decreasing the size of a filesystem
+shutdown		FS_IOC_SHUTDOWN ioctl
+snapshot		btrfs snapshots
+soak			long soak tests of any kind
+spaceman		xfs_spaceman functional tests
+splice			splice system call
+stress			fsstress filesystem exerciser
+subvol			btrfs subvolumes
+subvolume		btrfs subvolumes (again?)
+swap			swap files
+symlink			symbolic links
+tape			dump and restore with a tape
+thin			thin provisioning
+trim			FITRIM ioctl
+udf			UDF functionality tests
+union			tests from the unionmount test suite
+unlink			O_TMPFILE unlinked files
+unshare			fallocate unshare_range
+v2log			XFS v2 log format tests
+verity			fsverity
+volume			btrfs volume management
+whiteout		overlayfs whiteout functionality
+xino			overlayfs xino feature
+zero			fallocate zero_range
+zone			zoned (SMR) device support
+======================= =======================================================
diff --git a/include/buildgrouplist b/include/buildgrouplist
index d898efa3..489de965 100644
--- a/include/buildgrouplist
+++ b/include/buildgrouplist
@@ -6,3 +6,4 @@
 	@echo " [GROUP] $$PWD/$@"
 	$(Q)$(TOPDIR)/tools/mkgroupfile $@
+	$(Q)$(TOPDIR)/tools/check-groups $(TOPDIR)/doc/group-names.txt $@
diff --git a/tools/check-groups b/tools/check-groups
new file mode 100755
index 00000000..bac45bbd
--- /dev/null
+++ b/tools/check-groups
@@ -0,0 +1,33 @@
+# Make sure that all groups listed in a group.list file are mentioned in the
+# group description file.
+if [ -z "$1" ] || [ "$1" = "--help" ]; then
+	echo "Usage: $0 path_to_group_names [group.list files...]"
+	exit 1
+get_group_list() {
+	for file in "$@"; do
+		while read testname groups; do
+			test -z "${testname}" && continue
+			test "${testname:0:1}" = "#" && continue
+			echo "${groups}" | tr ' ' '\n'
+		done < "${file}"
+	done | sort | uniq
+while read group; do
+	if ! grep -q "^${group}[[:space:]]" "${groups_doc_file}"; then
+		echo "${group}: group not mentioned in documentation." 1>&2
+		ret=1
+	fi
+done < <(get_group_list "$@")
+exit "${ret}"

  parent reply	other threads:[~2021-09-01  0:12 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-01  0:12 [PATCHSET 0/5] fstests: document all test groups Darrick J. Wong
2021-09-01  0:12 ` [PATCH 1/5] ceph: re-tag copy_file_range as being in the copy_range group Darrick J. Wong
2021-09-01  8:44   ` Christoph Hellwig
2021-09-01  0:12 ` [PATCH 2/5] xfs: move reflink tests into the clone group Darrick J. Wong
2021-09-01  8:46   ` Christoph Hellwig
2021-09-01  0:12 ` [PATCH 3/5] xfs: fix incorrect fuzz test group name Darrick J. Wong
2021-09-01  8:48   ` Christoph Hellwig
2021-09-01  0:12 ` Darrick J. Wong [this message]
2021-09-01  4:46   ` [PATCH 4/5] tools: make sure that test groups are described in the documentation Amir Goldstein
2021-09-01 16:43     ` Darrick J. Wong
2021-09-02  4:49       ` Amir Goldstein
2021-09-02 15:03         ` Darrick J. Wong
2021-09-01  8:50   ` Christoph Hellwig
2021-09-01  0:12 ` [PATCH 5/5] new: only allow documented test group names Darrick J. Wong
2021-09-01  8:53   ` Christoph Hellwig
2021-09-01 21:29   ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=163045517173.771564.1524162806861567173.stgit@magnolia \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.