[PATCH v3] selftest: add tests for debugging testing setup

* [PATCH v3] selftest: add tests for debugging testing setup
@ 2023-02-21 22:36 Leah Rumancik
  2023-02-22 14:30 ` Zorro Lang
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Leah Rumancik @ 2023-02-21 22:36 UTC (permalink / raw)
  To: fstests; +Cc: zlang, djwong, amir73il, Leah Rumancik

Many people have developed infrastructure around xfstests. In order to
test a setup, it would be helpful to have dummy tests that have
consistent test outcomes. Add a new test folder with the following
tests:

selftest/001 pass
selftest/002 fail from output mismatch
selftest/003 fail via _fail
selftest/004 skip
selftest/005 crash
selftest/006 hang

Also, create two new groups: 'selftest' which includes tests 001-004 and
'dangerous_selftest' which includes tests 005-006. The selftests will
not run unless explicitly specified.

Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
---
v2 -> v3:
- remove selftest folder from SRC_GROUPS
- add README.selftest
- remove trailing blank lines from tests

v1 -> v2:
- change name of folder from 'debug' to 'selftest'
- update tags to 'selftest' and 'dangerous_selftest'
- add selftest folder to SRC_GROUPS
- updated copyright tag
- added test which _fails

v1: https://lore.kernel.org/fstests/20230209013143.2586104-1-leah.rumancik@gmail.com/
v2: https://lore.kernel.org/fstests/20230214215431.1425107-1-leah.rumancik@gmail.com/


 README.selftest         | 22 ++++++++++++++++++++++
 doc/group-names.txt     |  2 ++
 tests/selftest/001      | 13 +++++++++++++
 tests/selftest/001.out  |  2 ++
 tests/selftest/002      | 13 +++++++++++++
 tests/selftest/002.out  |  2 ++
 tests/selftest/003      | 15 +++++++++++++++
 tests/selftest/003.out  |  2 ++
 tests/selftest/004      | 15 +++++++++++++++
 tests/selftest/004.out  |  2 ++
 tests/selftest/005      | 16 ++++++++++++++++
 tests/selftest/005.out  |  2 ++
 tests/selftest/006      | 19 +++++++++++++++++++
 tests/selftest/006.out  |  2 ++
 tests/selftest/Makefile | 24 ++++++++++++++++++++++++
 15 files changed, 151 insertions(+)
 create mode 100644 README.selftest
 create mode 100755 tests/selftest/001
 create mode 100644 tests/selftest/001.out
 create mode 100755 tests/selftest/002
 create mode 100644 tests/selftest/002.out
 create mode 100755 tests/selftest/003
 create mode 100644 tests/selftest/003.out
 create mode 100755 tests/selftest/004
 create mode 100644 tests/selftest/004.out
 create mode 100755 tests/selftest/005
 create mode 100644 tests/selftest/005.out
 create mode 100755 tests/selftest/006
 create mode 100644 tests/selftest/006.out
 create mode 100644 tests/selftest/Makefile

diff --git a/README.selftest b/README.selftest
new file mode 100644
index 00000000..8f1d45af
--- /dev/null
+++ b/README.selftest
@@ -0,0 +1,22 @@
+Tests with consistent results are provided in the selftest folder.
+Since many people develop testing infrastructure around xfstests,
+these tests are helpful to confirm the testing setup is working as
+expected.
+
+The provided tests include:
+selftest/001 - pass
+selftest/002 - fail from output mismatch
+selftest/003 - fail via _fail
+selftest/004 - skip
+selftest/005 - crash
+selftest/006 - hang
+
+Two groups are used for these tests: selftest and dangerous_selftest.
+selftest/00[1-4] are in the selftest group and selftest/00[5-6] are
+in the dangerous_selftest group.
+
+The selftests will only be run if explicitly speficied. To run the
+selftests, you can specify individual tests, use "selftests/*", or
+use the groups "-g selftest/selftest -g selftest/dangerous_selftest".
+Note, you cannot use the group names without including the folder name
+(ie. "-g selftest").
diff --git a/doc/group-names.txt b/doc/group-names.txt
index 771ce937..ace59e05 100644
--- a/doc/group-names.txt
+++ b/doc/group-names.txt
@@ -38,6 +38,7 @@ dangerous_fsstress_repair	race fsstress and xfs_scrub online repair
 dangerous_fsstress_scrub	race fsstress and xfs_scrub checking
 dangerous_repair	fuzzers to evaluate xfs_repair offline repair
 dangerous_scrub		fuzzers to evaluate xfs_scrub checking
+dangerous_selftest	selftests that crash/hang
 data			data loss checkers
 dax			direct access mode for persistent memory files
 db			xfs_db functional tests
@@ -111,6 +112,7 @@ samefs			overlayfs when all layers are on the same fs
 scrub			filesystem metadata scrubbers
 seed			btrfs seeded filesystems
 seek			llseek functionality
+selftest		tests with fixed results, used to validate testing setup
 send			btrfs send/receive
 shrinkfs		decreasing the size of a filesystem
 shutdown		FS_IOC_SHUTDOWN ioctl
diff --git a/tests/selftest/001 b/tests/selftest/001
new file mode 100755
index 00000000..f3480330
--- /dev/null
+++ b/tests/selftest/001
@@ -0,0 +1,13 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 001
+#
+# A test that should always pass
+#
+. ./common/preamble
+_begin_fstest selftest
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/selftest/001.out b/tests/selftest/001.out
new file mode 100644
index 00000000..88678b8e
--- /dev/null
+++ b/tests/selftest/001.out
@@ -0,0 +1,2 @@
+QA output created by 001
+Silence is golden
diff --git a/tests/selftest/002 b/tests/selftest/002
new file mode 100755
index 00000000..189ae2cb
--- /dev/null
+++ b/tests/selftest/002
@@ -0,0 +1,13 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 002
+#
+# A test that should always fail due to output mismatch
+#
+. ./common/preamble
+_begin_fstest selftest
+
+echo "I am intentionally broken"
+status=0
+exit
diff --git a/tests/selftest/002.out b/tests/selftest/002.out
new file mode 100644
index 00000000..61705c7c
--- /dev/null
+++ b/tests/selftest/002.out
@@ -0,0 +1,2 @@
+QA output created by 002
+Silence is golden
diff --git a/tests/selftest/003 b/tests/selftest/003
new file mode 100755
index 00000000..c7a2848c
--- /dev/null
+++ b/tests/selftest/003
@@ -0,0 +1,15 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 003
+#
+# A test that _fail's
+#
+
+. ./common/preamble
+_begin_fstest selftest
+
+_fail "I have _fail'ed"
+
+status=0
+exit
diff --git a/tests/selftest/003.out b/tests/selftest/003.out
new file mode 100644
index 00000000..6895fc80
--- /dev/null
+++ b/tests/selftest/003.out
@@ -0,0 +1,2 @@
+QA output created by 003
+Silence is golden
diff --git a/tests/selftest/004 b/tests/selftest/004
new file mode 100755
index 00000000..89a3884d
--- /dev/null
+++ b/tests/selftest/004
@@ -0,0 +1,15 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 004
+#
+# A test that should always be skipped
+#
+. ./common/preamble
+_begin_fstest selftest
+
+_notrun "Always skip me"
+
+echo "I should be skipped"
+status=0
+exit
diff --git a/tests/selftest/004.out b/tests/selftest/004.out
new file mode 100644
index 00000000..af8614ae
--- /dev/null
+++ b/tests/selftest/004.out
@@ -0,0 +1,2 @@
+QA output created by 004
+Silence is golden
diff --git a/tests/selftest/005 b/tests/selftest/005
new file mode 100755
index 00000000..738cf44d
--- /dev/null
+++ b/tests/selftest/005
@@ -0,0 +1,16 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 005
+#
+# A test that crashes
+#
+. ./common/preamble
+_begin_fstest dangerous_selftest
+
+echo 1 > /proc/sys/kernel/sysrq
+echo c > /proc/sysrq-trigger
+
+echo "I should have crashed by now"
+status=0
+exit
diff --git a/tests/selftest/005.out b/tests/selftest/005.out
new file mode 100644
index 00000000..a5027f12
--- /dev/null
+++ b/tests/selftest/005.out
@@ -0,0 +1,2 @@
+QA output created by 005
+Silence is golden
diff --git a/tests/selftest/006 b/tests/selftest/006
new file mode 100755
index 00000000..150de6e8
--- /dev/null
+++ b/tests/selftest/006
@@ -0,0 +1,19 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 006
+#
+# A test that hangs
+#
+
+. ./common/preamble
+_begin_fstest dangerous_selftest
+
+while :
+do
+    sleep 1d
+done
+
+echo "I should still be sleeping"
+status=0
+exit
diff --git a/tests/selftest/006.out b/tests/selftest/006.out
new file mode 100644
index 00000000..675c1b7c
--- /dev/null
+++ b/tests/selftest/006.out
@@ -0,0 +1,2 @@
+QA output created by 006
+Silence is golden
diff --git a/tests/selftest/Makefile b/tests/selftest/Makefile
new file mode 100644
index 00000000..3ddfca37
--- /dev/null
+++ b/tests/selftest/Makefile
@@ -0,0 +1,24 @@
+#
+# Copyright (c) 2023 Google, Inc. All Rights Reserved.
+#
+
+TOPDIR = ../..
+include $(TOPDIR)/include/builddefs
+include $(TOPDIR)/include/buildgrouplist
+
+SELFTEST_DIR = selftest
+TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(SELFTEST_DIR)
+DIRT = group.list
+
+default: $(DIRT)
+
+include $(BUILDRULES)
+
+install:
+	$(INSTALL) -m 755 -d $(TARGET_DIR)
+	$(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
+	$(INSTALL) -m 644 group.list $(TARGET_DIR)
+	$(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
+
+# Nothing.
+install-dev install-lib:
-- 
2.39.2.637.g21b0678d19-goog


^ permalink raw reply related	[flat|nested] 5+ messages in thread