Hi all, I've noticed that fstests doesn't document what the test groups mean. I attempted to document what each group seems to symbolize, to reduce the amount of confusion for new contributors. While doing so, I noticed that there were a handful of tests that aren't in the right group. The first three patches dix that problem. The next test adds a build-time check to ensure that all groups are listed in the documentation file so that we don't lose track of things. The last patch updates ./new to use the documentation file instead of having to build group.list files. v2: fix the 'subvolume' group tests, tweak some of the wordings of the group description file, add missing license declarations, remove the one-off overlay group If you're going to start using this mess, you probably ought to just pull from my git trees, which are linked below. This is an extraordinary way to destroy everything. Enjoy! Comments and questions are, as always, welcome. --D fstests git tree: https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=document-test-groups --- doc/group-names.txt | 135 ++++++++++++++++++++++++++++++++++++++++++++++++ include/buildgrouplist | 1 new | 24 ++++----- tests/btrfs/233 | 2 - tests/btrfs/245 | 2 - tests/ceph/001 | 2 - tests/ceph/002 | 2 - tests/ceph/003 | 2 - tests/generic/631 | 2 - tests/xfs/491 | 2 - tests/xfs/492 | 2 - tests/xfs/493 | 2 - tests/xfs/519 | 2 - tests/xfs/520 | 2 - tests/xfs/535 | 2 - tests/xfs/536 | 2 - tools/check-groups | 35 ++++++++++++ tools/mkgroupfile | 4 + tools/mvtest | 5 +- tools/nextid | 4 + 20 files changed, 205 insertions(+), 29 deletions(-) create mode 100644 doc/group-names.txt create mode 100755 tools/check-groups
From: Darrick J. Wong <djwong@kernel.org> copy_range is the group name for copy_file_range tests, so reclassify these tests. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> --- tests/ceph/001 | 2 +- tests/ceph/002 | 2 +- tests/ceph/003 | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/ceph/001 b/tests/ceph/001 index aca77168..c00de308 100755 --- a/tests/ceph/001 +++ b/tests/ceph/001 @@ -11,7 +11,7 @@ # file and 3) the middle of the dst file. # . ./common/preamble -_begin_fstest auto quick copy +_begin_fstest auto quick copy_range # get standard environment . common/filter diff --git a/tests/ceph/002 b/tests/ceph/002 index 428f23a9..9bc728fd 100755 --- a/tests/ceph/002 +++ b/tests/ceph/002 @@ -20,7 +20,7 @@ # linux kernel: 78beb0ff2fec ("ceph: use copy-from2 op in copy_file_range") # . ./common/preamble -_begin_fstest auto quick copy +_begin_fstest auto quick copy_range # get standard environment . common/filter diff --git a/tests/ceph/003 b/tests/ceph/003 index 9f8c6068..faedb48c 100755 --- a/tests/ceph/003 +++ b/tests/ceph/003 @@ -7,7 +7,7 @@ # Test copy_file_range with infile = outfile # . ./common/preamble -_begin_fstest auto quick copy +_begin_fstest auto quick copy_range # get standard environment . common/filter
From: Darrick J. Wong <djwong@kernel.org> "clone" is the group for tests that exercise FICLONERANGE, so move these tests. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> --- tests/xfs/519 | 2 +- tests/xfs/520 | 2 +- tests/xfs/535 | 2 +- tests/xfs/536 | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/xfs/519 b/tests/xfs/519 index 675ec07e..49c62b56 100755 --- a/tests/xfs/519 +++ b/tests/xfs/519 @@ -9,7 +9,7 @@ # flushing the log and then remounting to check file contents. . ./common/preamble -_begin_fstest auto quick reflink +_begin_fstest auto quick clone # Import common functions. . ./common/filter diff --git a/tests/xfs/520 b/tests/xfs/520 index 8410f2ba..2fceb07c 100755 --- a/tests/xfs/520 +++ b/tests/xfs/520 @@ -12,7 +12,7 @@ # is included in the current kernel. # . ./common/preamble -_begin_fstest auto quick reflink +_begin_fstest auto quick clone # Override the default cleanup function. _cleanup() diff --git a/tests/xfs/535 b/tests/xfs/535 index 4c883675..1a5da61b 100755 --- a/tests/xfs/535 +++ b/tests/xfs/535 @@ -7,7 +7,7 @@ # Verify that XFS does not cause inode fork's extent count to overflow when # writing to a shared extent. . ./common/preamble -_begin_fstest auto quick reflink +_begin_fstest auto quick clone # Import common functions. . ./common/filter diff --git a/tests/xfs/536 b/tests/xfs/536 index e5f904f5..64fa4fbf 100755 --- a/tests/xfs/536 +++ b/tests/xfs/536 @@ -7,7 +7,7 @@ # Verify that XFS does not cause inode fork's extent count to overflow when # remapping extents from one file's inode fork to another. . ./common/preamble -_begin_fstest auto quick reflink +_begin_fstest auto quick clone # Import common functions. . ./common/filter
From: Darrick J. Wong <djwong@kernel.org> The group name for fuzz tests is 'fuzzers'. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> --- tests/xfs/491 | 2 +- tests/xfs/492 | 2 +- tests/xfs/493 | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/xfs/491 b/tests/xfs/491 index 5c7c5d1f..7402b09a 100755 --- a/tests/xfs/491 +++ b/tests/xfs/491 @@ -7,7 +7,7 @@ # Test detection & fixing of bad summary block counts at mount time. # . ./common/preamble -_begin_fstest auto quick fuzz +_begin_fstest auto quick fuzzers # Import common functions. . ./common/filter diff --git a/tests/xfs/492 b/tests/xfs/492 index 8258e5d8..514ac1e4 100755 --- a/tests/xfs/492 +++ b/tests/xfs/492 @@ -7,7 +7,7 @@ # Test detection & fixing of bad summary inode counts at mount time. # . ./common/preamble -_begin_fstest auto quick fuzz +_begin_fstest auto quick fuzzers # Import common functions. . ./common/filter diff --git a/tests/xfs/493 b/tests/xfs/493 index 58fd9c99..58091ad7 100755 --- a/tests/xfs/493 +++ b/tests/xfs/493 @@ -8,7 +8,7 @@ # Corrupt the AGFs to test mount failure when mount-fixing fails. # . ./common/preamble -_begin_fstest auto quick fuzz +_begin_fstest auto quick fuzzers # Import common functions. . ./common/filter
From: Darrick J. Wong <djwong@kernel.org> The group for testing subvolume functionality is 'subvol', not 'subvolume'. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- tests/btrfs/233 | 2 +- tests/btrfs/245 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/btrfs/233 b/tests/btrfs/233 index f3e3762c..6a414443 100755 --- a/tests/btrfs/233 +++ b/tests/btrfs/233 @@ -9,7 +9,7 @@ # performed. # . ./common/preamble -_begin_fstest auto quick subvolume +_begin_fstest auto quick subvol # Override the default cleanup function. _cleanup() diff --git a/tests/btrfs/245 b/tests/btrfs/245 index 2b9c63c7..f3380ac2 100755 --- a/tests/btrfs/245 +++ b/tests/btrfs/245 @@ -8,7 +8,7 @@ # as subvolume and snapshot creation and deletion. # . ./common/preamble -_begin_fstest auto quick idmapped subvolume +_begin_fstest auto quick idmapped subvol # get standard environment, filters and checks . ./common/rc
From: Darrick J. Wong <djwong@kernel.org> This test isn't really an overlay test; it's a regression test for a bug that someone found in xfs handling of whiteout files. Since the 'overlay' group has one member, let's move it to 'whiteout'. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- tests/generic/631 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/generic/631 b/tests/generic/631 index a1acdedb..aae181dd 100755 --- a/tests/generic/631 +++ b/tests/generic/631 @@ -22,7 +22,7 @@ # in xfs_rename"). . ./common/preamble -_begin_fstest auto rw overlay rename +_begin_fstest auto rw whiteout rename # Override the default cleanup function. _cleanup()
From: Darrick J. Wong <djwong@kernel.org> Create a file to document the purpose of each test group that is currently defined in fstests, and add a build script to check that every group mentioned in the tests is also mentioned in the documentation. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- doc/group-names.txt | 135 ++++++++++++++++++++++++++++++++++++++++++++++++ include/buildgrouplist | 1 tools/check-groups | 35 ++++++++++++ 3 files changed, 171 insertions(+) create mode 100644 doc/group-names.txt create mode 100755 tools/check-groups diff --git a/doc/group-names.txt b/doc/group-names.txt new file mode 100644 index 00000000..e8e3477e --- /dev/null +++ b/doc/group-names.txt @@ -0,0 +1,135 @@ +======================= ======================================================= +Group Name: Description: +======================= ======================================================= +all All known tests, automatically generated by ./check at + runtime +auto Tests that should be run automatically. These should + not require more than ~5 minutes to run. +quick Tests that should run in under 30 seconds. +deprecated Old tests that should not be run. + +acl Access Control Lists +admin xfs_admin functionality +aio general libaio async io tests +atime file access time +attr extended attributes +attr2 xfs v2 extended aributes +balance btrfs tree rebalance +bigtime timestamps beyond the year 2038 +blockdev block device functionality +broken broken tests +cap Linux capabilities +casefold directory name casefolding +ci ASCII case-insensitive directory name lookups +clone FICLONE/FICLONERANGE ioctls +clone_stress stress testing FICLONE/FICLONERANGE +collapse fallocate FALLOC_FL_COLLAPSE_RANGE +compress file compression +convert btrfs ext[34] conversion tool +copy xfs_copy functionality +copy_range copy_file_range syscall +copyup overlayfs copyup +dangerous dangerous test that can crash the system +dangerous_bothrepair fuzzers to evaluate xfs_scrub + xfs_repair repair +dangerous_fuzzers fuzzers that can crash your computer +dangerous_norepair fuzzers to evaluate kernel metadata verifiers +dangerous_online_repair fuzzers to evaluate xfs_scrub online repair +dangerous_repair fuzzers to evaluate xfs_repair offline repair +dangerous_scrub fuzzers to evaluate xfs_scrub checking +data data loss checkers +dax direct access mode for persistent memory files +db xfs_db functional tests +dedupe FIEDEDUPERANGE ioctl +defrag filesystem defragmenters +dir directory test functions +dump dump and restore utilities +eio IO error reporting +encrypt encrypted file contents +enospc ENOSPC error reporting +exportfs file handles +filestreams XFS filestreams allocator +freeze filesystem freeze tests +fsck general fsck tests +fsmap FS_IOC_GETFSMAP ioctl +fsr XFS free space reorganizer +fuzzers filesystem fuzz tests +growfs increasing the size of a filesystem +hardlink hardlinks +health XFS health reporting +idmapped idmapped mount functionality +inobtcount XFS inode btree count tests +insert fallocate FALLOC_FL_INSERT_RANGE +ioctl general ioctl tests +io_uring general io_uring async io tests +label filesystem labelling +limit resource limits +locks file locking +log metadata logging +logprint xfs_logprint functional tests +long_rw long-soak read write IO path exercisers +metacopy overlayfs metadata-only copy-up +metadata filesystem metadata update exercisers +metadump xfs_metadump/xfs_mdrestore functionality +mkfs filesystem formatting tools +mount mount option and functionality checks +nested nested overlayfs instances +nfs4_acl NFSv4 access control lists +nonsamefs overlayfs layers on different filesystems +online_repair online repair functionality tests +other dumping ground, do not add more tests to this group +pattern specific IO pattern tests +perms access control and permission checking +pipe pipe functionality +pnfs PNFS +posix POSIX behavior conformance +prealloc fallocate for preallocating unwritten space +preallocrw fallocate, then read and write +punch fallocate FALLOC_FL_PUNCH_HOLE +qgroup btrfs qgroup feature +quota filesystem usage quotas +raid btrfs RAID +realtime XFS realtime volumes +recoveryloop crash recovery loops +redirect overlayfs redirect_dir feature +remote dump and restore with a remote tape +remount remounting filesystems +rename rename system call +repair xfs_repair functional tests +replace btrfs device replace +replay dm-logwrites replays +resize resize2fs functionality tests +richacl rich ACL feature +rmap XFS reverse mapping exercisers +rotate overlayfs upper layer rotate tests from the unionmount + test suite +rw read/write IO tests +samefs overlayfs when all layers are on the same fs +scrub filesystem metadata scrubbers +seed btrfs seeded filesystems +seek llseek functionality +send btrfs send/receive +shrinkfs decreasing the size of a filesystem +shutdown FS_IOC_SHUTDOWN ioctl +snapshot btrfs snapshots +soak long running soak tests of any kind +spaceman xfs_spaceman functional tests +splice splice system call +stress fsstress filesystem exerciser +subvol btrfs subvolumes +swap swap files +symlink symbolic links +tape dump and restore with a tape +thin thin provisioning +trim FITRIM ioctl +udf UDF functionality tests +union tests from the unionmount test suite +unlink O_TMPFILE unlinked files +unshare fallocate FALLOC_FL_UNSHARE_RANGE +v2log XFS v2 log format tests +verity fsverity +volume btrfs volume management +whiteout overlayfs whiteout functionality +xino overlayfs xino feature +zero fallocate FALLOC_FL_ZERO_RANGE +zone zoned (SMR) device support +======================= ======================================================= diff --git a/include/buildgrouplist b/include/buildgrouplist index d898efa3..489de965 100644 --- a/include/buildgrouplist +++ b/include/buildgrouplist @@ -6,3 +6,4 @@ group.list: @echo " [GROUP] $$PWD/$@" $(Q)$(TOPDIR)/tools/mkgroupfile $@ + $(Q)$(TOPDIR)/tools/check-groups $(TOPDIR)/doc/group-names.txt $@ diff --git a/tools/check-groups b/tools/check-groups new file mode 100755 index 00000000..0d193615 --- /dev/null +++ b/tools/check-groups @@ -0,0 +1,35 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Oracle. All Rights Reserved. +# +# Make sure that all groups listed in a group.list file are mentioned in the +# group description file. + +if [ -z "$1" ] || [ "$1" = "--help" ]; then + echo "Usage: $0 path_to_group_names [group.list files...]" + exit 1 +fi + +groups_doc_file="$1" +shift + +get_group_list() { + for file in "$@"; do + while read testname groups; do + test -z "${testname}" && continue + test "${testname:0:1}" = "#" && continue + + echo "${groups}" | tr ' ' '\n' + done < "${file}" + done | sort | uniq +} + +ret=0 +while read group; do + if ! grep -q "^${group}[[:space:]]" "${groups_doc_file}"; then + echo "${group}: group not mentioned in documentation." 1>&2 + ret=1 + fi +done < <(get_group_list "$@") + +exit "${ret}"
From: Darrick J. Wong <djwong@kernel.org> I forgot to add spdx license tags and copyright statements to some of the tools that I've contributed to fstests. Fix this to be explicit. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- tools/mkgroupfile | 4 +++- tools/mvtest | 5 ++++- tools/nextid | 4 +++- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/tools/mkgroupfile b/tools/mkgroupfile index 0681e5d2..f96c329b 100755 --- a/tools/mkgroupfile +++ b/tools/mkgroupfile @@ -1,5 +1,7 @@ #!/bin/bash - +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Oracle. All Rights Reserved. +# # Generate a group file from the _begin_fstest call in each test. if [ "$1" = "--help" ]; then diff --git a/tools/mvtest b/tools/mvtest index 5088b45f..99b15414 100755 --- a/tools/mvtest +++ b/tools/mvtest @@ -1,6 +1,9 @@ #!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2015 Oracle. All Rights Reserved. +# +# Move a test and update the golden output file. -# Renumber a test dir="$(dirname "$0")" if [ -z "$1" ] || [ "$1" = "--help" ]; then diff --git a/tools/nextid b/tools/nextid index 9507de29..9e31718c 100755 --- a/tools/nextid +++ b/tools/nextid @@ -1,5 +1,7 @@ #!/bin/bash - +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2015 Oracle. All Rights Reserved. +# # Compute the next available test id in a given test directory. if [ $# != 1 ] || [ "$1" = "--help" ] || [ ! -d "tests/$1/" ]; then
From: Darrick J. Wong <djwong@kernel.org> Now that we require all group names to be listed in doc/group-names.txt, we can use that (instead of running mkgroupfile) to check if the group name(s) supplied by the user actually exist. This has the secondary effect of being a second nudge towards keeping the description of groups up to date. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> --- new | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/new b/new index 2097a883..6b7dc5d4 100755 --- a/new +++ b/new @@ -83,6 +83,14 @@ then exit 1 fi +# Extract group names from the documentation. +group_names() { + awk '/^[[:lower:][:digit:]_]/ { + if ($1 != "" && $1 != "Group" && $2 != "Name:" && $1 != "all") + printf("%s\n", $1); + }' doc/group-names.txt +} + if [ $# -eq 0 ] then @@ -93,16 +101,7 @@ then [ -z "$ans" ] && ans=other if [ "X$ans" = "X?" ] then - for d in $SRC_GROUPS; do - (cd "tests/$d/" ; ../../tools/mkgroupfile "$tmpfile") - l=$(sed -n < "$tmpfile" \ - -e 's/#.*//' \ - -e 's/$/ /' \ - -e 's;\(^[0-9][0-9][0-9]\)\(.*$\);\2;p') - grpl="$grpl $l" - done - lst=`for word in $grpl; do echo $word; done | sort| uniq ` - echo $lst + echo $(group_names) else # only allow lower cases, spaces, digits and underscore in group inval=`echo $ans | tr -d '[:lower:][:space:][:digit:]_'` @@ -120,11 +119,10 @@ then else # expert mode, groups are on the command line # - (cd "$tdir" ; ../../tools/mkgroupfile "$tmpfile") for g in $* do - if ! grep -q "[[:space:]]$g" "$tmpfile"; then - echo "Warning: group \"$g\" not defined in $tdir tests" + if ! grep -q "^$g" doc/group-names.txt; then + echo "Warning: group \"$g\" not defined in documentation" fi done ans="$*"
> diff --git a/include/buildgrouplist b/include/buildgrouplist > index d898efa3..489de965 100644 > --- a/include/buildgrouplist > +++ b/include/buildgrouplist > @@ -6,3 +6,4 @@ > group.list: > @echo " [GROUP] $$PWD/$@" > $(Q)$(TOPDIR)/tools/mkgroupfile $@ > + $(Q)$(TOPDIR)/tools/check-groups $(TOPDIR)/doc/group-names.txt $@ I would like to argue against checking groups post mkgroupfile and for checking groups during mkgroupfile > diff --git a/tools/check-groups b/tools/check-groups > new file mode 100755 > index 00000000..0d193615 > --- /dev/null > +++ b/tools/check-groups > @@ -0,0 +1,35 @@ > +#!/bin/bash > +# SPDX-License-Identifier: GPL-2.0 > +# Copyright (c) 2021 Oracle. All Rights Reserved. > +# > +# Make sure that all groups listed in a group.list file are mentioned in the > +# group description file. > + > +if [ -z "$1" ] || [ "$1" = "--help" ]; then > + echo "Usage: $0 path_to_group_names [group.list files...]" > + exit 1 > +fi > + > +groups_doc_file="$1" > +shift > + > +get_group_list() { > + for file in "$@"; do > + while read testname groups; do > + test -z "${testname}" && continue > + test "${testname:0:1}" = "#" && continue > + > + echo "${groups}" | tr ' ' '\n' > + done < "${file}" > + done | sort | uniq > +} > + > +ret=0 > +while read group; do > + if ! grep -q "^${group}[[:space:]]" "${groups_doc_file}"; then > + echo "${group}: group not mentioned in documentation." 1>&2 This message would have been more informative with the offending test file. Now after you crunched all the test files into group.list files and all the group.list files into a unique group set, this is too late. But this same check during generate_groupfile() would have been trivial and would allow reporting the offending test. While we are on the subject of generate_groupfile(), can you please explain the rationale behind the method of extracting the test file groups by executing the test with GENERATE_GROUPS=yes? As opposed to just getting the list of groups on the stop from the file using grep? Thanks, Amir.
On Fri, Sep 03, 2021 at 06:38:38AM +0300, Amir Goldstein wrote: > > diff --git a/include/buildgrouplist b/include/buildgrouplist > > index d898efa3..489de965 100644 > > --- a/include/buildgrouplist > > +++ b/include/buildgrouplist > > @@ -6,3 +6,4 @@ > > group.list: > > @echo " [GROUP] $$PWD/$@" > > $(Q)$(TOPDIR)/tools/mkgroupfile $@ > > + $(Q)$(TOPDIR)/tools/check-groups $(TOPDIR)/doc/group-names.txt $@ > > I would like to argue against checking groups post mkgroupfile > and for checking groups during mkgroupfile Done. > > diff --git a/tools/check-groups b/tools/check-groups > > new file mode 100755 > > index 00000000..0d193615 > > --- /dev/null > > +++ b/tools/check-groups > > @@ -0,0 +1,35 @@ > > +#!/bin/bash > > +# SPDX-License-Identifier: GPL-2.0 > > +# Copyright (c) 2021 Oracle. All Rights Reserved. > > +# > > +# Make sure that all groups listed in a group.list file are mentioned in the > > +# group description file. > > + > > +if [ -z "$1" ] || [ "$1" = "--help" ]; then > > + echo "Usage: $0 path_to_group_names [group.list files...]" > > + exit 1 > > +fi > > + > > +groups_doc_file="$1" > > +shift > > + > > +get_group_list() { > > + for file in "$@"; do > > + while read testname groups; do > > + test -z "${testname}" && continue > > + test "${testname:0:1}" = "#" && continue > > + > > + echo "${groups}" | tr ' ' '\n' > > + done < "${file}" > > + done | sort | uniq > > +} > > + > > +ret=0 > > +while read group; do > > + if ! grep -q "^${group}[[:space:]]" "${groups_doc_file}"; then > > + echo "${group}: group not mentioned in documentation." 1>&2 > > This message would have been more informative with the offending > test file. Hm. This becomes much easier if I make the _begin_fstest helper do the checking of the group names. > Now after you crunched all the test files into group.list files and > all the group.list files into a unique group set, this is too late. > But this same check during generate_groupfile() would have > been trivial and would allow reporting the offending test. > > While we are on the subject of generate_groupfile(), can you please > explain the rationale behind the method of extracting the test file > groups by executing the test with GENERATE_GROUPS=yes? > As opposed to just getting the list of groups on the stop from the file > using grep? Well... now that you point that out, it's so that we can put in custom logic like checking group names. ;) --D > > Thanks, > Amir.
From: Darrick J. Wong <djwong@kernel.org> Create a file to document the purpose of each test group that is currently defined in fstests, and change mkgroupfile to check that every group mentioned in the tests is also mentioned in the documentation. Signed-off-by: Darrick J. Wong <djwong@kernel.org> --- common/preamble | 21 ++++++++ doc/group-names.txt | 135 +++++++++++++++++++++++++++++++++++++++++++++++++++ tools/mkgroupfile | 15 +++++- 3 files changed, 168 insertions(+), 3 deletions(-) create mode 100644 doc/group-names.txt diff --git a/common/preamble b/common/preamble index 66b0ed05..64d79385 100644 --- a/common/preamble +++ b/common/preamble @@ -23,6 +23,26 @@ _register_cleanup() trap "${cleanup}exit \$status" EXIT HUP INT QUIT TERM $* } +# Make sure each group is in the documentation file. +_check_groups() { + test -n "$GROUPNAME_DOC_FILE" || return 0 + + local testname="$(echo "$0" | sed -e 's/^.*tests\///g')" + declare -a missing=() + + for group in "$@"; do + if ! grep -q "^${group}[[:space:]]" "$GROUPNAME_DOC_FILE"; then + missing+=("\"${group}\"") + fi + done + test "${#missing}" -eq 0 && return 0 + + local suffix= + test "${#missing}" -gt 1 && suffix="s" + echo "$testname: group$suffix ${missing[@]} not mentioned in documentation." 1>&2 + return 1 +} + # Prepare to run a fstest by initializing the required global variables to # their defaults, sourcing common functions, registering a cleanup function, # and removing the $seqres.full file. @@ -42,6 +62,7 @@ _begin_fstest() # If we're only running the test to generate a group.list file, # spit out the group data and exit. if [ -n "$GENERATE_GROUPS" ]; then + _check_groups "$@" || exit 1 echo "$seq $@" exit 0 fi diff --git a/doc/group-names.txt b/doc/group-names.txt new file mode 100644 index 00000000..e8e3477e --- /dev/null +++ b/doc/group-names.txt @@ -0,0 +1,135 @@ +======================= ======================================================= +Group Name: Description: +======================= ======================================================= +all All known tests, automatically generated by ./check at + runtime +auto Tests that should be run automatically. These should + not require more than ~5 minutes to run. +quick Tests that should run in under 30 seconds. +deprecated Old tests that should not be run. + +acl Access Control Lists +admin xfs_admin functionality +aio general libaio async io tests +atime file access time +attr extended attributes +attr2 xfs v2 extended aributes +balance btrfs tree rebalance +bigtime timestamps beyond the year 2038 +blockdev block device functionality +broken broken tests +cap Linux capabilities +casefold directory name casefolding +ci ASCII case-insensitive directory name lookups +clone FICLONE/FICLONERANGE ioctls +clone_stress stress testing FICLONE/FICLONERANGE +collapse fallocate FALLOC_FL_COLLAPSE_RANGE +compress file compression +convert btrfs ext[34] conversion tool +copy xfs_copy functionality +copy_range copy_file_range syscall +copyup overlayfs copyup +dangerous dangerous test that can crash the system +dangerous_bothrepair fuzzers to evaluate xfs_scrub + xfs_repair repair +dangerous_fuzzers fuzzers that can crash your computer +dangerous_norepair fuzzers to evaluate kernel metadata verifiers +dangerous_online_repair fuzzers to evaluate xfs_scrub online repair +dangerous_repair fuzzers to evaluate xfs_repair offline repair +dangerous_scrub fuzzers to evaluate xfs_scrub checking +data data loss checkers +dax direct access mode for persistent memory files +db xfs_db functional tests +dedupe FIEDEDUPERANGE ioctl +defrag filesystem defragmenters +dir directory test functions +dump dump and restore utilities +eio IO error reporting +encrypt encrypted file contents +enospc ENOSPC error reporting +exportfs file handles +filestreams XFS filestreams allocator +freeze filesystem freeze tests +fsck general fsck tests +fsmap FS_IOC_GETFSMAP ioctl +fsr XFS free space reorganizer +fuzzers filesystem fuzz tests +growfs increasing the size of a filesystem +hardlink hardlinks +health XFS health reporting +idmapped idmapped mount functionality +inobtcount XFS inode btree count tests +insert fallocate FALLOC_FL_INSERT_RANGE +ioctl general ioctl tests +io_uring general io_uring async io tests +label filesystem labelling +limit resource limits +locks file locking +log metadata logging +logprint xfs_logprint functional tests +long_rw long-soak read write IO path exercisers +metacopy overlayfs metadata-only copy-up +metadata filesystem metadata update exercisers +metadump xfs_metadump/xfs_mdrestore functionality +mkfs filesystem formatting tools +mount mount option and functionality checks +nested nested overlayfs instances +nfs4_acl NFSv4 access control lists +nonsamefs overlayfs layers on different filesystems +online_repair online repair functionality tests +other dumping ground, do not add more tests to this group +pattern specific IO pattern tests +perms access control and permission checking +pipe pipe functionality +pnfs PNFS +posix POSIX behavior conformance +prealloc fallocate for preallocating unwritten space +preallocrw fallocate, then read and write +punch fallocate FALLOC_FL_PUNCH_HOLE +qgroup btrfs qgroup feature +quota filesystem usage quotas +raid btrfs RAID +realtime XFS realtime volumes +recoveryloop crash recovery loops +redirect overlayfs redirect_dir feature +remote dump and restore with a remote tape +remount remounting filesystems +rename rename system call +repair xfs_repair functional tests +replace btrfs device replace +replay dm-logwrites replays +resize resize2fs functionality tests +richacl rich ACL feature +rmap XFS reverse mapping exercisers +rotate overlayfs upper layer rotate tests from the unionmount + test suite +rw read/write IO tests +samefs overlayfs when all layers are on the same fs +scrub filesystem metadata scrubbers +seed btrfs seeded filesystems +seek llseek functionality +send btrfs send/receive +shrinkfs decreasing the size of a filesystem +shutdown FS_IOC_SHUTDOWN ioctl +snapshot btrfs snapshots +soak long running soak tests of any kind +spaceman xfs_spaceman functional tests +splice splice system call +stress fsstress filesystem exerciser +subvol btrfs subvolumes +swap swap files +symlink symbolic links +tape dump and restore with a tape +thin thin provisioning +trim FITRIM ioctl +udf UDF functionality tests +union tests from the unionmount test suite +unlink O_TMPFILE unlinked files +unshare fallocate FALLOC_FL_UNSHARE_RANGE +v2log XFS v2 log format tests +verity fsverity +volume btrfs volume management +whiteout overlayfs whiteout functionality +xino overlayfs xino feature +zero fallocate FALLOC_FL_ZERO_RANGE +zone zoned (SMR) device support +======================= ======================================================= diff --git a/tools/mkgroupfile b/tools/mkgroupfile index 0681e5d2..767bac90 100755 --- a/tools/mkgroupfile +++ b/tools/mkgroupfile @@ -9,6 +9,8 @@ fi test_dir="$PWD" groupfile="$1" +GROUPNAME_DOC_FILE="$(readlink -m ../../doc/group-names.txt)" +export GROUPNAME_DOC_FILE if [ ! -x ../../check ]; then echo "$0: Run this from tests/XXX/." @@ -24,19 +26,26 @@ ENDL cd ../../ export GENERATE_GROUPS=yes grep -R -l "^_begin_fstest" "$test_dir/" 2>/dev/null | while read testfile; do - test -x "$testfile" && "$testfile" + test -x "$testfile" && "$testfile" || return 1 done | sort -g + local ret="${PIPESTATUS[1]}" cd "$test_dir" + return $ret } if [ -z "$groupfile" ] || [ "$groupfile" = "-" ]; then # Dump the group file to stdout and exit generate_groupfile - exit 0 fi # Otherwise, write the group file to disk somewhere. ngroupfile="${groupfile}.new" rm -f "$ngroupfile" generate_groupfile >> "$ngroupfile" -mv "$ngroupfile" "$groupfile" +ret=$? +if [ $ret -eq 0 ]; then + mv "$ngroupfile" "$groupfile" +else + rm -f "$ngroupfile" +fi +exit $ret
On Fri, Sep 3, 2021 at 5:14 AM Darrick J. Wong <djwong@kernel.org> wrote:
>
> From: Darrick J. Wong <djwong@kernel.org>
>
> Now that we require all group names to be listed in doc/group-names.txt,
> we can use that (instead of running mkgroupfile) to check if the group
> name(s) supplied by the user actually exist. This has the secondary
> effect of being a second nudge towards keeping the description of groups
> up to date.
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> ---
> new | 24 +++++++++++-------------
> 1 file changed, 11 insertions(+), 13 deletions(-)
>
>
> diff --git a/new b/new
> index 2097a883..6b7dc5d4 100755
> --- a/new
> +++ b/new
> @@ -83,6 +83,14 @@ then
> exit 1
> fi
>
> +# Extract group names from the documentation.
> +group_names() {
> + awk '/^[[:lower:][:digit:]_]/ {
> + if ($1 != "" && $1 != "Group" && $2 != "Name:" && $1 != "all")
> + printf("%s\n", $1);
> + }' doc/group-names.txt
> +}
> +
> if [ $# -eq 0 ]
> then
>
> @@ -93,16 +101,7 @@ then
> [ -z "$ans" ] && ans=other
> if [ "X$ans" = "X?" ]
> then
> - for d in $SRC_GROUPS; do
> - (cd "tests/$d/" ; ../../tools/mkgroupfile "$tmpfile")
> - l=$(sed -n < "$tmpfile" \
> - -e 's/#.*//' \
> - -e 's/$/ /' \
> - -e 's;\(^[0-9][0-9][0-9]\)\(.*$\);\2;p')
> - grpl="$grpl $l"
> - done
> - lst=`for word in $grpl; do echo $word; done | sort| uniq `
> - echo $lst
> + echo $(group_names)
> else
> # only allow lower cases, spaces, digits and underscore in group
> inval=`echo $ans | tr -d '[:lower:][:space:][:digit:]_'`
> @@ -120,11 +119,10 @@ then
> else
> # expert mode, groups are on the command line
> #
> - (cd "$tdir" ; ../../tools/mkgroupfile "$tmpfile")
> for g in $*
> do
> - if ! grep -q "[[:space:]]$g" "$tmpfile"; then
> - echo "Warning: group \"$g\" not defined in $tdir tests"
> + if ! grep -q "^$g" doc/group-names.txt; then
> + echo "Warning: group \"$g\" not defined in documentation"
> fi
Do you want to warn (or fail) on new uses of the group "other"?
Thanks,
Amir.
On Sat, Sep 4, 2021 at 6:06 AM Darrick J. Wong <djwong@kernel.org> wrote: > > From: Darrick J. Wong <djwong@kernel.org> > > Create a file to document the purpose of each test group that is > currently defined in fstests, and change mkgroupfile to check that every > group mentioned in the tests is also mentioned in the documentation. > > Signed-off-by: Darrick J. Wong <djwong@kernel.org> > --- > common/preamble | 21 ++++++++ > doc/group-names.txt | 135 +++++++++++++++++++++++++++++++++++++++++++++++++++ > tools/mkgroupfile | 15 +++++- > 3 files changed, 168 insertions(+), 3 deletions(-) > create mode 100644 doc/group-names.txt > > diff --git a/common/preamble b/common/preamble > index 66b0ed05..64d79385 100644 > --- a/common/preamble > +++ b/common/preamble > @@ -23,6 +23,26 @@ _register_cleanup() > trap "${cleanup}exit \$status" EXIT HUP INT QUIT TERM $* > } > > +# Make sure each group is in the documentation file. > +_check_groups() { > + test -n "$GROUPNAME_DOC_FILE" || return 0 > + > + local testname="$(echo "$0" | sed -e 's/^.*tests\///g')" > + declare -a missing=() > + > + for group in "$@"; do > + if ! grep -q "^${group}[[:space:]]" "$GROUPNAME_DOC_FILE"; then > + missing+=("\"${group}\"") > + fi > + done > + test "${#missing}" -eq 0 && return 0 > + > + local suffix= > + test "${#missing}" -gt 1 && suffix="s" > + echo "$testname: group$suffix ${missing[@]} not mentioned in documentation." 1>&2 > + return 1 > +} > + > # Prepare to run a fstest by initializing the required global variables to > # their defaults, sourcing common functions, registering a cleanup function, > # and removing the $seqres.full file. > @@ -42,6 +62,7 @@ _begin_fstest() > # If we're only running the test to generate a group.list file, > # spit out the group data and exit. > if [ -n "$GENERATE_GROUPS" ]; then > + _check_groups "$@" || exit 1 > echo "$seq $@" > exit 0 > fi > diff --git a/doc/group-names.txt b/doc/group-names.txt > new file mode 100644 > index 00000000..e8e3477e > --- /dev/null > +++ b/doc/group-names.txt > @@ -0,0 +1,135 @@ > +======================= ======================================================= > +Group Name: Description: > +======================= ======================================================= > +all All known tests, automatically generated by ./check at > + runtime > +auto Tests that should be run automatically. These should > + not require more than ~5 minutes to run. > +quick Tests that should run in under 30 seconds. > +deprecated Old tests that should not be run. > + > +acl Access Control Lists > +admin xfs_admin functionality > +aio general libaio async io tests > +atime file access time > +attr extended attributes > +attr2 xfs v2 extended aributes > +balance btrfs tree rebalance > +bigtime timestamps beyond the year 2038 > +blockdev block device functionality > +broken broken tests > +cap Linux capabilities > +casefold directory name casefolding > +ci ASCII case-insensitive directory name lookups > +clone FICLONE/FICLONERANGE ioctls > +clone_stress stress testing FICLONE/FICLONERANGE > +collapse fallocate FALLOC_FL_COLLAPSE_RANGE > +compress file compression > +convert btrfs ext[34] conversion tool > +copy xfs_copy functionality > +copy_range copy_file_range syscall > +copyup overlayfs copyup > +dangerous dangerous test that can crash the system > +dangerous_bothrepair fuzzers to evaluate xfs_scrub + xfs_repair repair > +dangerous_fuzzers fuzzers that can crash your computer > +dangerous_norepair fuzzers to evaluate kernel metadata verifiers > +dangerous_online_repair fuzzers to evaluate xfs_scrub online repair > +dangerous_repair fuzzers to evaluate xfs_repair offline repair > +dangerous_scrub fuzzers to evaluate xfs_scrub checking > +data data loss checkers > +dax direct access mode for persistent memory files > +db xfs_db functional tests > +dedupe FIEDEDUPERANGE ioctl > +defrag filesystem defragmenters > +dir directory test functions > +dump dump and restore utilities > +eio IO error reporting > +encrypt encrypted file contents > +enospc ENOSPC error reporting > +exportfs file handles > +filestreams XFS filestreams allocator > +freeze filesystem freeze tests > +fsck general fsck tests > +fsmap FS_IOC_GETFSMAP ioctl > +fsr XFS free space reorganizer > +fuzzers filesystem fuzz tests > +growfs increasing the size of a filesystem > +hardlink hardlinks > +health XFS health reporting > +idmapped idmapped mount functionality > +inobtcount XFS inode btree count tests > +insert fallocate FALLOC_FL_INSERT_RANGE > +ioctl general ioctl tests > +io_uring general io_uring async io tests > +label filesystem labelling > +limit resource limits > +locks file locking > +log metadata logging > +logprint xfs_logprint functional tests > +long_rw long-soak read write IO path exercisers > +metacopy overlayfs metadata-only copy-up > +metadata filesystem metadata update exercisers > +metadump xfs_metadump/xfs_mdrestore functionality > +mkfs filesystem formatting tools > +mount mount option and functionality checks > +nested nested overlayfs instances > +nfs4_acl NFSv4 access control lists > +nonsamefs overlayfs layers on different filesystems > +online_repair online repair functionality tests > +other dumping ground, do not add more tests to this group > +pattern specific IO pattern tests > +perms access control and permission checking > +pipe pipe functionality > +pnfs PNFS > +posix POSIX behavior conformance > +prealloc fallocate for preallocating unwritten space > +preallocrw fallocate, then read and write > +punch fallocate FALLOC_FL_PUNCH_HOLE > +qgroup btrfs qgroup feature > +quota filesystem usage quotas > +raid btrfs RAID > +realtime XFS realtime volumes > +recoveryloop crash recovery loops > +redirect overlayfs redirect_dir feature > +remote dump and restore with a remote tape > +remount remounting filesystems > +rename rename system call > +repair xfs_repair functional tests > +replace btrfs device replace > +replay dm-logwrites replays > +resize resize2fs functionality tests > +richacl rich ACL feature > +rmap XFS reverse mapping exercisers > +rotate overlayfs upper layer rotate tests from the unionmount > + test suite > +rw read/write IO tests > +samefs overlayfs when all layers are on the same fs > +scrub filesystem metadata scrubbers > +seed btrfs seeded filesystems > +seek llseek functionality > +send btrfs send/receive > +shrinkfs decreasing the size of a filesystem > +shutdown FS_IOC_SHUTDOWN ioctl > +snapshot btrfs snapshots > +soak long running soak tests of any kind > +spaceman xfs_spaceman functional tests > +splice splice system call > +stress fsstress filesystem exerciser > +subvol btrfs subvolumes > +swap swap files > +symlink symbolic links > +tape dump and restore with a tape > +thin thin provisioning > +trim FITRIM ioctl > +udf UDF functionality tests > +union tests from the unionmount test suite > +unlink O_TMPFILE unlinked files > +unshare fallocate FALLOC_FL_UNSHARE_RANGE > +v2log XFS v2 log format tests > +verity fsverity > +volume btrfs volume management > +whiteout overlayfs whiteout functionality > +xino overlayfs xino feature > +zero fallocate FALLOC_FL_ZERO_RANGE > +zone zoned (SMR) device support > +======================= ======================================================= > diff --git a/tools/mkgroupfile b/tools/mkgroupfile > index 0681e5d2..767bac90 100755 > --- a/tools/mkgroupfile > +++ b/tools/mkgroupfile > @@ -9,6 +9,8 @@ fi > > test_dir="$PWD" > groupfile="$1" > +GROUPNAME_DOC_FILE="$(readlink -m ../../doc/group-names.txt)" > +export GROUPNAME_DOC_FILE > > if [ ! -x ../../check ]; then > echo "$0: Run this from tests/XXX/." > @@ -24,19 +26,26 @@ ENDL > cd ../../ > export GENERATE_GROUPS=yes > grep -R -l "^_begin_fstest" "$test_dir/" 2>/dev/null | while read testfile; do > - test -x "$testfile" && "$testfile" > + test -x "$testfile" && "$testfile" || return 1 > done | sort -g > + local ret="${PIPESTATUS[1]}" > cd "$test_dir" > + return $ret > } > > if [ -z "$groupfile" ] || [ "$groupfile" = "-" ]; then > # Dump the group file to stdout and exit > generate_groupfile > - exit 0 Now the comment above is incorrect, but I think you did want to exit $ret? > fi > > # Otherwise, write the group file to disk somewhere. > ngroupfile="${groupfile}.new" > rm -f "$ngroupfile" > generate_groupfile >> "$ngroupfile" > -mv "$ngroupfile" "$groupfile" > +ret=$? > +if [ $ret -eq 0 ]; then > + mv "$ngroupfile" "$groupfile" > +else > + rm -f "$ngroupfile" > +fi > +exit $ret I think that a cleanup() trap would have made this script a lot cleaner, not having to deal with ${PIPESTATUS[1]} and whatnot, but if there is a reason not to use cleanup() trap, I'm fine with this as well. Thanks, Amir.
On Sat, Sep 04, 2021 at 11:52:32AM +0300, Amir Goldstein wrote: > On Sat, Sep 4, 2021 at 6:06 AM Darrick J. Wong <djwong@kernel.org> wrote: > > > > From: Darrick J. Wong <djwong@kernel.org> > > > > Create a file to document the purpose of each test group that is > > currently defined in fstests, and change mkgroupfile to check that every > > group mentioned in the tests is also mentioned in the documentation. > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org> > > --- > > common/preamble | 21 ++++++++ > > doc/group-names.txt | 135 +++++++++++++++++++++++++++++++++++++++++++++++++++ > > tools/mkgroupfile | 15 +++++- > > 3 files changed, 168 insertions(+), 3 deletions(-) > > create mode 100644 doc/group-names.txt > > > > diff --git a/common/preamble b/common/preamble > > index 66b0ed05..64d79385 100644 > > --- a/common/preamble > > +++ b/common/preamble > > @@ -23,6 +23,26 @@ _register_cleanup() > > trap "${cleanup}exit \$status" EXIT HUP INT QUIT TERM $* > > } > > > > +# Make sure each group is in the documentation file. > > +_check_groups() { > > + test -n "$GROUPNAME_DOC_FILE" || return 0 > > + > > + local testname="$(echo "$0" | sed -e 's/^.*tests\///g')" > > + declare -a missing=() > > + > > + for group in "$@"; do > > + if ! grep -q "^${group}[[:space:]]" "$GROUPNAME_DOC_FILE"; then > > + missing+=("\"${group}\"") > > + fi > > + done > > + test "${#missing}" -eq 0 && return 0 > > + > > + local suffix= > > + test "${#missing}" -gt 1 && suffix="s" > > + echo "$testname: group$suffix ${missing[@]} not mentioned in documentation." 1>&2 > > + return 1 > > +} > > + > > # Prepare to run a fstest by initializing the required global variables to > > # their defaults, sourcing common functions, registering a cleanup function, > > # and removing the $seqres.full file. > > @@ -42,6 +62,7 @@ _begin_fstest() > > # If we're only running the test to generate a group.list file, > > # spit out the group data and exit. > > if [ -n "$GENERATE_GROUPS" ]; then > > + _check_groups "$@" || exit 1 > > echo "$seq $@" > > exit 0 > > fi > > diff --git a/doc/group-names.txt b/doc/group-names.txt > > new file mode 100644 > > index 00000000..e8e3477e > > --- /dev/null > > +++ b/doc/group-names.txt > > @@ -0,0 +1,135 @@ > > +======================= ======================================================= > > +Group Name: Description: > > +======================= ======================================================= > > +all All known tests, automatically generated by ./check at > > + runtime > > +auto Tests that should be run automatically. These should > > + not require more than ~5 minutes to run. > > +quick Tests that should run in under 30 seconds. > > +deprecated Old tests that should not be run. > > + > > +acl Access Control Lists > > +admin xfs_admin functionality > > +aio general libaio async io tests > > +atime file access time > > +attr extended attributes > > +attr2 xfs v2 extended aributes > > +balance btrfs tree rebalance > > +bigtime timestamps beyond the year 2038 > > +blockdev block device functionality > > +broken broken tests > > +cap Linux capabilities > > +casefold directory name casefolding > > +ci ASCII case-insensitive directory name lookups > > +clone FICLONE/FICLONERANGE ioctls > > +clone_stress stress testing FICLONE/FICLONERANGE > > +collapse fallocate FALLOC_FL_COLLAPSE_RANGE > > +compress file compression > > +convert btrfs ext[34] conversion tool > > +copy xfs_copy functionality > > +copy_range copy_file_range syscall > > +copyup overlayfs copyup > > +dangerous dangerous test that can crash the system > > +dangerous_bothrepair fuzzers to evaluate xfs_scrub + xfs_repair repair > > +dangerous_fuzzers fuzzers that can crash your computer > > +dangerous_norepair fuzzers to evaluate kernel metadata verifiers > > +dangerous_online_repair fuzzers to evaluate xfs_scrub online repair > > +dangerous_repair fuzzers to evaluate xfs_repair offline repair > > +dangerous_scrub fuzzers to evaluate xfs_scrub checking > > +data data loss checkers > > +dax direct access mode for persistent memory files > > +db xfs_db functional tests > > +dedupe FIEDEDUPERANGE ioctl > > +defrag filesystem defragmenters > > +dir directory test functions > > +dump dump and restore utilities > > +eio IO error reporting > > +encrypt encrypted file contents > > +enospc ENOSPC error reporting > > +exportfs file handles > > +filestreams XFS filestreams allocator > > +freeze filesystem freeze tests > > +fsck general fsck tests > > +fsmap FS_IOC_GETFSMAP ioctl > > +fsr XFS free space reorganizer > > +fuzzers filesystem fuzz tests > > +growfs increasing the size of a filesystem > > +hardlink hardlinks > > +health XFS health reporting > > +idmapped idmapped mount functionality > > +inobtcount XFS inode btree count tests > > +insert fallocate FALLOC_FL_INSERT_RANGE > > +ioctl general ioctl tests > > +io_uring general io_uring async io tests > > +label filesystem labelling > > +limit resource limits > > +locks file locking > > +log metadata logging > > +logprint xfs_logprint functional tests > > +long_rw long-soak read write IO path exercisers > > +metacopy overlayfs metadata-only copy-up > > +metadata filesystem metadata update exercisers > > +metadump xfs_metadump/xfs_mdrestore functionality > > +mkfs filesystem formatting tools > > +mount mount option and functionality checks > > +nested nested overlayfs instances > > +nfs4_acl NFSv4 access control lists > > +nonsamefs overlayfs layers on different filesystems > > +online_repair online repair functionality tests > > +other dumping ground, do not add more tests to this group > > +pattern specific IO pattern tests > > +perms access control and permission checking > > +pipe pipe functionality > > +pnfs PNFS > > +posix POSIX behavior conformance > > +prealloc fallocate for preallocating unwritten space > > +preallocrw fallocate, then read and write > > +punch fallocate FALLOC_FL_PUNCH_HOLE > > +qgroup btrfs qgroup feature > > +quota filesystem usage quotas > > +raid btrfs RAID > > +realtime XFS realtime volumes > > +recoveryloop crash recovery loops > > +redirect overlayfs redirect_dir feature > > +remote dump and restore with a remote tape > > +remount remounting filesystems > > +rename rename system call > > +repair xfs_repair functional tests > > +replace btrfs device replace > > +replay dm-logwrites replays > > +resize resize2fs functionality tests > > +richacl rich ACL feature > > +rmap XFS reverse mapping exercisers > > +rotate overlayfs upper layer rotate tests from the unionmount > > + test suite > > +rw read/write IO tests > > +samefs overlayfs when all layers are on the same fs > > +scrub filesystem metadata scrubbers > > +seed btrfs seeded filesystems > > +seek llseek functionality > > +send btrfs send/receive > > +shrinkfs decreasing the size of a filesystem > > +shutdown FS_IOC_SHUTDOWN ioctl > > +snapshot btrfs snapshots > > +soak long running soak tests of any kind > > +spaceman xfs_spaceman functional tests > > +splice splice system call > > +stress fsstress filesystem exerciser > > +subvol btrfs subvolumes > > +swap swap files > > +symlink symbolic links > > +tape dump and restore with a tape > > +thin thin provisioning > > +trim FITRIM ioctl > > +udf UDF functionality tests > > +union tests from the unionmount test suite > > +unlink O_TMPFILE unlinked files > > +unshare fallocate FALLOC_FL_UNSHARE_RANGE > > +v2log XFS v2 log format tests > > +verity fsverity > > +volume btrfs volume management > > +whiteout overlayfs whiteout functionality > > +xino overlayfs xino feature > > +zero fallocate FALLOC_FL_ZERO_RANGE > > +zone zoned (SMR) device support > > +======================= ======================================================= > > diff --git a/tools/mkgroupfile b/tools/mkgroupfile > > index 0681e5d2..767bac90 100755 > > --- a/tools/mkgroupfile > > +++ b/tools/mkgroupfile > > @@ -9,6 +9,8 @@ fi > > > > test_dir="$PWD" > > groupfile="$1" > > +GROUPNAME_DOC_FILE="$(readlink -m ../../doc/group-names.txt)" > > +export GROUPNAME_DOC_FILE > > > > if [ ! -x ../../check ]; then > > echo "$0: Run this from tests/XXX/." > > @@ -24,19 +26,26 @@ ENDL > > cd ../../ > > export GENERATE_GROUPS=yes > > grep -R -l "^_begin_fstest" "$test_dir/" 2>/dev/null | while read testfile; do > > - test -x "$testfile" && "$testfile" > > + test -x "$testfile" && "$testfile" || return 1 > > done | sort -g > > + local ret="${PIPESTATUS[1]}" > > cd "$test_dir" > > + return $ret > > } > > > > if [ -z "$groupfile" ] || [ "$groupfile" = "-" ]; then > > # Dump the group file to stdout and exit > > generate_groupfile > > - exit 0 > > Now the comment above is incorrect, but I think you > did want to exit $ret? "exit" by itself will cause the script to exit with the last accumulated return value (which is the 'return $ret' in the last line of generat_groupfile), but yes, thanks for pointing out that I'd deleted the whole line by mistake. Fixed. > > > fi > > > > # Otherwise, write the group file to disk somewhere. > > ngroupfile="${groupfile}.new" > > rm -f "$ngroupfile" > > generate_groupfile >> "$ngroupfile" > > -mv "$ngroupfile" "$groupfile" > > +ret=$? > > +if [ $ret -eq 0 ]; then > > + mv "$ngroupfile" "$groupfile" > > +else > > + rm -f "$ngroupfile" > > +fi > > +exit $ret > > I think that a cleanup() trap would have made this script a lot cleaner, > not having to deal with ${PIPESTATUS[1]} and whatnot, but if there is > a reason not to use cleanup() trap, I'm fine with this as well. No real reason. I'll reimplement the prologue as a cleanup trap so that we don't leave junk everywhere if someone ^Cs during build. --D > > Thanks, > Amir.
On Sat, Sep 04, 2021 at 11:43:58AM +0300, Amir Goldstein wrote: > On Fri, Sep 3, 2021 at 5:14 AM Darrick J. Wong <djwong@kernel.org> wrote: > > > > From: Darrick J. Wong <djwong@kernel.org> > > > > Now that we require all group names to be listed in doc/group-names.txt, > > we can use that (instead of running mkgroupfile) to check if the group > > name(s) supplied by the user actually exist. This has the secondary > > effect of being a second nudge towards keeping the description of groups > > up to date. > > > > Signed-off-by: Darrick J. Wong <djwong@kernel.org> > > Reviewed-by: Christoph Hellwig <hch@lst.de> > > --- > > new | 24 +++++++++++------------- > > 1 file changed, 11 insertions(+), 13 deletions(-) > > > > > > diff --git a/new b/new > > index 2097a883..6b7dc5d4 100755 > > --- a/new > > +++ b/new > > @@ -83,6 +83,14 @@ then > > exit 1 > > fi > > > > +# Extract group names from the documentation. > > +group_names() { > > + awk '/^[[:lower:][:digit:]_]/ { > > + if ($1 != "" && $1 != "Group" && $2 != "Name:" && $1 != "all") > > + printf("%s\n", $1); > > + }' doc/group-names.txt > > +} > > + > > if [ $# -eq 0 ] > > then > > > > @@ -93,16 +101,7 @@ then > > [ -z "$ans" ] && ans=other > > if [ "X$ans" = "X?" ] > > then > > - for d in $SRC_GROUPS; do > > - (cd "tests/$d/" ; ../../tools/mkgroupfile "$tmpfile") > > - l=$(sed -n < "$tmpfile" \ > > - -e 's/#.*//' \ > > - -e 's/$/ /' \ > > - -e 's;\(^[0-9][0-9][0-9]\)\(.*$\);\2;p') > > - grpl="$grpl $l" > > - done > > - lst=`for word in $grpl; do echo $word; done | sort| uniq ` > > - echo $lst > > + echo $(group_names) > > else > > # only allow lower cases, spaces, digits and underscore in group > > inval=`echo $ans | tr -d '[:lower:][:space:][:digit:]_'` > > @@ -120,11 +119,10 @@ then > > else > > # expert mode, groups are on the command line > > # > > - (cd "$tdir" ; ../../tools/mkgroupfile "$tmpfile") > > for g in $* > > do > > - if ! grep -q "[[:space:]]$g" "$tmpfile"; then > > - echo "Warning: group \"$g\" not defined in $tdir tests" > > + if ! grep -q "^$g" doc/group-names.txt; then > > + echo "Warning: group \"$g\" not defined in documentation" > > fi > > Do you want to warn (or fail) on new uses of the group "other"? Yeah, I'll add a new patch on the end to do that. --D > Thanks, > Amir.
From: Darrick J. Wong <djwong@kernel.org> Create a file to document the purpose of each test group that is currently defined in fstests, and change mkgroupfile to check that every group mentioned in the tests is also mentioned in the documentation. Signed-off-by: Darrick J. Wong <djwong@kernel.org> Reviewed-by: Amir Goldstein <amir73il@gmail.com> --- common/preamble | 21 ++++++++ doc/group-names.txt | 135 +++++++++++++++++++++++++++++++++++++++++++++++++++ tools/mkgroupfile | 33 +++++++++--- 3 files changed, 181 insertions(+), 8 deletions(-) create mode 100644 doc/group-names.txt diff --git a/common/preamble b/common/preamble index 66b0ed05..64d79385 100644 --- a/common/preamble +++ b/common/preamble @@ -23,6 +23,26 @@ _register_cleanup() trap "${cleanup}exit \$status" EXIT HUP INT QUIT TERM $* } +# Make sure each group is in the documentation file. +_check_groups() { + test -n "$GROUPNAME_DOC_FILE" || return 0 + + local testname="$(echo "$0" | sed -e 's/^.*tests\///g')" + declare -a missing=() + + for group in "$@"; do + if ! grep -q "^${group}[[:space:]]" "$GROUPNAME_DOC_FILE"; then + missing+=("\"${group}\"") + fi + done + test "${#missing}" -eq 0 && return 0 + + local suffix= + test "${#missing}" -gt 1 && suffix="s" + echo "$testname: group$suffix ${missing[@]} not mentioned in documentation." 1>&2 + return 1 +} + # Prepare to run a fstest by initializing the required global variables to # their defaults, sourcing common functions, registering a cleanup function, # and removing the $seqres.full file. @@ -42,6 +62,7 @@ _begin_fstest() # If we're only running the test to generate a group.list file, # spit out the group data and exit. if [ -n "$GENERATE_GROUPS" ]; then + _check_groups "$@" || exit 1 echo "$seq $@" exit 0 fi diff --git a/doc/group-names.txt b/doc/group-names.txt new file mode 100644 index 00000000..e8e3477e --- /dev/null +++ b/doc/group-names.txt @@ -0,0 +1,135 @@ +======================= ======================================================= +Group Name: Description: +======================= ======================================================= +all All known tests, automatically generated by ./check at + runtime +auto Tests that should be run automatically. These should + not require more than ~5 minutes to run. +quick Tests that should run in under 30 seconds. +deprecated Old tests that should not be run. + +acl Access Control Lists +admin xfs_admin functionality +aio general libaio async io tests +atime file access time +attr extended attributes +attr2 xfs v2 extended aributes +balance btrfs tree rebalance +bigtime timestamps beyond the year 2038 +blockdev block device functionality +broken broken tests +cap Linux capabilities +casefold directory name casefolding +ci ASCII case-insensitive directory name lookups +clone FICLONE/FICLONERANGE ioctls +clone_stress stress testing FICLONE/FICLONERANGE +collapse fallocate FALLOC_FL_COLLAPSE_RANGE +compress file compression +convert btrfs ext[34] conversion tool +copy xfs_copy functionality +copy_range copy_file_range syscall +copyup overlayfs copyup +dangerous dangerous test that can crash the system +dangerous_bothrepair fuzzers to evaluate xfs_scrub + xfs_repair repair +dangerous_fuzzers fuzzers that can crash your computer +dangerous_norepair fuzzers to evaluate kernel metadata verifiers +dangerous_online_repair fuzzers to evaluate xfs_scrub online repair +dangerous_repair fuzzers to evaluate xfs_repair offline repair +dangerous_scrub fuzzers to evaluate xfs_scrub checking +data data loss checkers +dax direct access mode for persistent memory files +db xfs_db functional tests +dedupe FIEDEDUPERANGE ioctl +defrag filesystem defragmenters +dir directory test functions +dump dump and restore utilities +eio IO error reporting +encrypt encrypted file contents +enospc ENOSPC error reporting +exportfs file handles +filestreams XFS filestreams allocator +freeze filesystem freeze tests +fsck general fsck tests +fsmap FS_IOC_GETFSMAP ioctl +fsr XFS free space reorganizer +fuzzers filesystem fuzz tests +growfs increasing the size of a filesystem +hardlink hardlinks +health XFS health reporting +idmapped idmapped mount functionality +inobtcount XFS inode btree count tests +insert fallocate FALLOC_FL_INSERT_RANGE +ioctl general ioctl tests +io_uring general io_uring async io tests +label filesystem labelling +limit resource limits +locks file locking +log metadata logging +logprint xfs_logprint functional tests +long_rw long-soak read write IO path exercisers +metacopy overlayfs metadata-only copy-up +metadata filesystem metadata update exercisers +metadump xfs_metadump/xfs_mdrestore functionality +mkfs filesystem formatting tools +mount mount option and functionality checks +nested nested overlayfs instances +nfs4_acl NFSv4 access control lists +nonsamefs overlayfs layers on different filesystems +online_repair online repair functionality tests +other dumping ground, do not add more tests to this group +pattern specific IO pattern tests +perms access control and permission checking +pipe pipe functionality +pnfs PNFS +posix POSIX behavior conformance +prealloc fallocate for preallocating unwritten space +preallocrw fallocate, then read and write +punch fallocate FALLOC_FL_PUNCH_HOLE +qgroup btrfs qgroup feature +quota filesystem usage quotas +raid btrfs RAID +realtime XFS realtime volumes +recoveryloop crash recovery loops +redirect overlayfs redirect_dir feature +remote dump and restore with a remote tape +remount remounting filesystems +rename rename system call +repair xfs_repair functional tests +replace btrfs device replace +replay dm-logwrites replays +resize resize2fs functionality tests +richacl rich ACL feature +rmap XFS reverse mapping exercisers +rotate overlayfs upper layer rotate tests from the unionmount + test suite +rw read/write IO tests +samefs overlayfs when all layers are on the same fs +scrub filesystem metadata scrubbers +seed btrfs seeded filesystems +seek llseek functionality +send btrfs send/receive +shrinkfs decreasing the size of a filesystem +shutdown FS_IOC_SHUTDOWN ioctl +snapshot btrfs snapshots +soak long running soak tests of any kind +spaceman xfs_spaceman functional tests +splice splice system call +stress fsstress filesystem exerciser +subvol btrfs subvolumes +swap swap files +symlink symbolic links +tape dump and restore with a tape +thin thin provisioning +trim FITRIM ioctl +udf UDF functionality tests +union tests from the unionmount test suite +unlink O_TMPFILE unlinked files +unshare fallocate FALLOC_FL_UNSHARE_RANGE +v2log XFS v2 log format tests +verity fsverity +volume btrfs volume management +whiteout overlayfs whiteout functionality +xino overlayfs xino feature +zero fallocate FALLOC_FL_ZERO_RANGE +zone zoned (SMR) device support +======================= ======================================================= diff --git a/tools/mkgroupfile b/tools/mkgroupfile index 0681e5d2..e4244507 100755 --- a/tools/mkgroupfile +++ b/tools/mkgroupfile @@ -9,12 +9,28 @@ fi test_dir="$PWD" groupfile="$1" +GROUPNAME_DOC_FILE="$(readlink -m ../../doc/group-names.txt)" +export GROUPNAME_DOC_FILE if [ ! -x ../../check ]; then echo "$0: Run this from tests/XXX/." exit 1 fi +cleanup() { + test -z "$groupfile" && return + test -z "$ngroupfile" && return + + if [ $ret -eq 0 ]; then + mv "$ngroupfile" "$groupfile" + else + rm -f "$ngroupfile" + fi +} + +ret=1 # trigger cleanup of temporary files unless we succeed +trap 'cleanup; exit $ret' EXIT INT TERM QUIT + generate_groupfile() { cat << ENDL # QA groups control file, automatically generated. @@ -24,19 +40,20 @@ ENDL cd ../../ export GENERATE_GROUPS=yes grep -R -l "^_begin_fstest" "$test_dir/" 2>/dev/null | while read testfile; do - test -x "$testfile" && "$testfile" + test -x "$testfile" && "$testfile" || return 1 done | sort -g + ret="${PIPESTATUS[1]}" cd "$test_dir" } if [ -z "$groupfile" ] || [ "$groupfile" = "-" ]; then # Dump the group file to stdout and exit + unset groupfile generate_groupfile - exit 0 +else + # Otherwise, write the group file to disk somewhere. + ngroupfile="${groupfile}.new" + rm -f "$ngroupfile" + generate_groupfile >> "$ngroupfile" + # let cleanup rename or delete ngroupfile fi - -# Otherwise, write the group file to disk somewhere. -ngroupfile="${groupfile}.new" -rm -f "$ngroupfile" -generate_groupfile >> "$ngroupfile" -mv "$ngroupfile" "$groupfile"