All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH fstests 0/3] few enhancements
@ 2024-02-16 18:18 Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 1/3] tests: augment soak test group Luis Chamberlain
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-16 18:18 UTC (permalink / raw)
  To: fstests, anand.jain, aalbersh, djwong
  Cc: linux-fsdevel, kdevops, patches, Luis Chamberlain

This adds a couple of enhancements picked up over experiences with
kdevops. The first one is to augment the set of tests which are part of
the soak group. That required careful review of all of our tests, so
might as well update the tests with this information.

To allow us to verify this, we add an option to let us list the tests
which are part of a group with --list-group-tests. This came out of
recent discussions where Darrick proposed perhaps it might be possible
to query this [0], this is an attempt to help with that. The goal here
is to ensure that any test which does use SOAK_DURATION will be
agumented properly in the future as part of the soak group.

The last patch is a simple fstest watchdog enhacement which lets
applications monitoring only a guest's kernel buffer to know when
the show has started and ended. It is completely optional to use, but
kdevops has been using this in its own wrapper oscheck.sh for years now.

[0] https://lkml.kernel.org/r/20240125222956.GD6188@frogsfrogsfrogs

Luis Chamberlain (3):
  tests: augment soak test group
  check: add support for --list-group-tests
  check: add --print-start-done to enhance watchdogs

 check             | 32 +++++++++++++++++++++++++++++++-
 tests/generic/019 |  2 +-
 tests/generic/388 |  2 +-
 tests/generic/475 |  2 +-
 tests/generic/642 |  2 +-
 tests/generic/648 |  2 +-
 tests/xfs/285     |  2 +-
 tests/xfs/517     |  2 +-
 tests/xfs/560     |  2 +-
 tests/xfs/561     |  2 +-
 tests/xfs/562     |  2 +-
 tests/xfs/565     |  2 +-
 tests/xfs/570     |  2 +-
 tests/xfs/571     |  2 +-
 tests/xfs/572     |  2 +-
 tests/xfs/573     |  2 +-
 tests/xfs/574     |  2 +-
 tests/xfs/575     |  2 +-
 tests/xfs/576     |  2 +-
 tests/xfs/577     |  2 +-
 tests/xfs/578     |  2 +-
 tests/xfs/579     |  2 +-
 tests/xfs/580     |  2 +-
 tests/xfs/581     |  2 +-
 tests/xfs/582     |  2 +-
 tests/xfs/583     |  2 +-
 tests/xfs/584     |  2 +-
 tests/xfs/585     |  2 +-
 tests/xfs/586     |  2 +-
 tests/xfs/587     |  2 +-
 tests/xfs/588     |  2 +-
 tests/xfs/589     |  2 +-
 tests/xfs/590     |  2 +-
 tests/xfs/591     |  2 +-
 tests/xfs/592     |  2 +-
 tests/xfs/593     |  2 +-
 tests/xfs/594     |  2 +-
 tests/xfs/595     |  2 +-
 tests/xfs/727     |  2 +-
 tests/xfs/729     |  2 +-
 tests/xfs/800     |  2 +-
 41 files changed, 71 insertions(+), 41 deletions(-)

-- 
2.42.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

* [PATCH 1/3] tests: augment soak test group
  2024-02-16 18:18 [PATCH fstests 0/3] few enhancements Luis Chamberlain
@ 2024-02-16 18:18 ` Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 2/3] check: add support for --list-group-tests Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 3/3] check: add --print-start-done to enhance watchdogs Luis Chamberlain
  2 siblings, 0 replies; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-16 18:18 UTC (permalink / raw)
  To: fstests, anand.jain, aalbersh, djwong
  Cc: linux-fsdevel, kdevops, patches, Luis Chamberlain

Many tests are using SOAK_DURATION but they have not been added to the
soak group. We want to have a deterministic way to query which tests are
part of the soak group, so to enable test frameworks which use fstests
to get an idea when a test may have lapsed the expected amount of time
for the test to complete. Of course such a time is subjetive to a test
environment and system, however max variables are possible and are used
for an initial test run, and later an enhanced test environement can
leverage and also use prior known test times with check.time. That is
exactly what kdevops uses to determine a timeout.

In kdevops we have to maintain a list of static array of tests which
uses soak, with this, we shold be able to grow that set dynamically.

Tests either use SOAK_DURATION directly or they use the helper loop such as
_soak_loop_running(). XFS also uses SOAK_DURATION with helpers such as
_scratch_xfs_stress_scrub().

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 tests/generic/019 | 2 +-
 tests/generic/388 | 2 +-
 tests/generic/475 | 2 +-
 tests/generic/642 | 2 +-
 tests/generic/648 | 2 +-
 tests/xfs/285     | 2 +-
 tests/xfs/517     | 2 +-
 tests/xfs/560     | 2 +-
 tests/xfs/561     | 2 +-
 tests/xfs/562     | 2 +-
 tests/xfs/565     | 2 +-
 tests/xfs/570     | 2 +-
 tests/xfs/571     | 2 +-
 tests/xfs/572     | 2 +-
 tests/xfs/573     | 2 +-
 tests/xfs/574     | 2 +-
 tests/xfs/575     | 2 +-
 tests/xfs/576     | 2 +-
 tests/xfs/577     | 2 +-
 tests/xfs/578     | 2 +-
 tests/xfs/579     | 2 +-
 tests/xfs/580     | 2 +-
 tests/xfs/581     | 2 +-
 tests/xfs/582     | 2 +-
 tests/xfs/583     | 2 +-
 tests/xfs/584     | 2 +-
 tests/xfs/585     | 2 +-
 tests/xfs/586     | 2 +-
 tests/xfs/587     | 2 +-
 tests/xfs/588     | 2 +-
 tests/xfs/589     | 2 +-
 tests/xfs/590     | 2 +-
 tests/xfs/591     | 2 +-
 tests/xfs/592     | 2 +-
 tests/xfs/593     | 2 +-
 tests/xfs/594     | 2 +-
 tests/xfs/595     | 2 +-
 tests/xfs/727     | 2 +-
 tests/xfs/729     | 2 +-
 tests/xfs/800     | 2 +-
 40 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/tests/generic/019 b/tests/generic/019
index b81c1d17ba65..a77ce1e3dad6 100755
--- a/tests/generic/019
+++ b/tests/generic/019
@@ -8,7 +8,7 @@
 # check filesystem consistency at the end.
 #
 . ./common/preamble
-_begin_fstest aio dangerous enospc rw stress recoveryloop
+_begin_fstest aio dangerous enospc rw stress recoveryloop soak
 
 fio_config=$tmp.fio
 
diff --git a/tests/generic/388 b/tests/generic/388
index 4a5be6698cbd..523f4b310b8a 100755
--- a/tests/generic/388
+++ b/tests/generic/388
@@ -15,7 +15,7 @@
 # spurious corruption reports and/or mount failures.
 #
 . ./common/preamble
-_begin_fstest shutdown auto log metadata recoveryloop
+_begin_fstest shutdown auto log metadata recoveryloop soak
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/generic/475 b/tests/generic/475
index ce7fe013b1fc..cfbbcedf80e2 100755
--- a/tests/generic/475
+++ b/tests/generic/475
@@ -12,7 +12,7 @@
 # testing efforts.
 #
 . ./common/preamble
-_begin_fstest shutdown auto log metadata eio recoveryloop smoketest
+_begin_fstest shutdown auto log metadata eio recoveryloop smoketest soak
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/generic/642 b/tests/generic/642
index 4d0c41fd5d51..9c367c653807 100755
--- a/tests/generic/642
+++ b/tests/generic/642
@@ -8,7 +8,7 @@
 # bugs in the xattr code.
 #
 . ./common/preamble
-_begin_fstest auto soak attr long_rw stress smoketest
+_begin_fstest auto soak attr long_rw stress smoketest soak
 
 _cleanup()
 {
diff --git a/tests/generic/648 b/tests/generic/648
index 3b3544ff49c3..e3f4ce7af801 100755
--- a/tests/generic/648
+++ b/tests/generic/648
@@ -12,7 +12,7 @@
 # in writeback on the host that cause VM guests to fail to recover.
 #
 . ./common/preamble
-_begin_fstest shutdown auto log metadata eio recoveryloop
+_begin_fstest shutdown auto log metadata eio recoveryloop soak
 
 _cleanup()
 {
diff --git a/tests/xfs/285 b/tests/xfs/285
index 0056baeb1c73..e0510d7f6696 100755
--- a/tests/xfs/285
+++ b/tests/xfs/285
@@ -8,7 +8,7 @@
 # or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	cd /
diff --git a/tests/xfs/517 b/tests/xfs/517
index 68438e544ea0..815c1fb40cc1 100755
--- a/tests/xfs/517
+++ b/tests/xfs/517
@@ -7,7 +7,7 @@
 # Race freeze and fsmap for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest auto quick fsmap freeze
+_begin_fstest auto quick fsmap freeze soak
 
 _register_cleanup "_cleanup" BUS
 
diff --git a/tests/xfs/560 b/tests/xfs/560
index 28b45d5f5e72..a931da7bc239 100755
--- a/tests/xfs/560
+++ b/tests/xfs/560
@@ -7,7 +7,7 @@
 # Race GETFSMAP and ro remount for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest auto quick fsmap remount
+_begin_fstest auto quick fsmap remount soak
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/xfs/561 b/tests/xfs/561
index c1d68c6fe62c..10277e8a6d75 100755
--- a/tests/xfs/561
+++ b/tests/xfs/561
@@ -8,7 +8,7 @@
 # crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/xfs/562 b/tests/xfs/562
index a5c6e88875fc..a7304cd3b551 100755
--- a/tests/xfs/562
+++ b/tests/xfs/562
@@ -8,7 +8,7 @@
 # or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 # Override the default cleanup function.
 _cleanup()
diff --git a/tests/xfs/565 b/tests/xfs/565
index 826bc5354a77..8000984bdee6 100755
--- a/tests/xfs/565
+++ b/tests/xfs/565
@@ -8,7 +8,7 @@
 # or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	cd /
diff --git a/tests/xfs/570 b/tests/xfs/570
index 9f3ba873ae3d..e8c3a315d325 100755
--- a/tests/xfs/570
+++ b/tests/xfs/570
@@ -7,7 +7,7 @@
 # Race fsstress and superblock scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/571 b/tests/xfs/571
index 9d22de8f45c5..4e5ad4b0460e 100755
--- a/tests/xfs/571
+++ b/tests/xfs/571
@@ -7,7 +7,7 @@
 # Race fsstress and AGF scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/572 b/tests/xfs/572
index b0e352af4e40..dfbed43ffa83 100755
--- a/tests/xfs/572
+++ b/tests/xfs/572
@@ -7,7 +7,7 @@
 # Race fsstress and AGFL scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/573 b/tests/xfs/573
index a2b6bef3cf3b..5be8fea7676e 100755
--- a/tests/xfs/573
+++ b/tests/xfs/573
@@ -7,7 +7,7 @@
 # Race fsstress and AGI scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/574 b/tests/xfs/574
index 5a4bad00162d..847a99bc01b7 100755
--- a/tests/xfs/574
+++ b/tests/xfs/574
@@ -8,7 +8,7 @@
 # crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/575 b/tests/xfs/575
index 3d29620f2c4b..66731af213eb 100755
--- a/tests/xfs/575
+++ b/tests/xfs/575
@@ -8,7 +8,7 @@
 # crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/576 b/tests/xfs/576
index e11476d452fd..d1b99b968068 100755
--- a/tests/xfs/576
+++ b/tests/xfs/576
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/577 b/tests/xfs/577
index d1abe6fafb15..dad9b3f400cc 100755
--- a/tests/xfs/577
+++ b/tests/xfs/577
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/578 b/tests/xfs/578
index 8160b7ef515e..28db2c53ba83 100755
--- a/tests/xfs/578
+++ b/tests/xfs/578
@@ -8,7 +8,7 @@
 # or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/579 b/tests/xfs/579
index a00ae02aa74e..bd187852419d 100755
--- a/tests/xfs/579
+++ b/tests/xfs/579
@@ -8,7 +8,7 @@
 # or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/580 b/tests/xfs/580
index f49cba6427c4..1094f04e730c 100755
--- a/tests/xfs/580
+++ b/tests/xfs/580
@@ -8,7 +8,7 @@
 # if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/581 b/tests/xfs/581
index 1d08bc7df3e6..e733bf3962ce 100755
--- a/tests/xfs/581
+++ b/tests/xfs/581
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/582 b/tests/xfs/582
index 7a8c330befd1..97c2bfde1453 100755
--- a/tests/xfs/582
+++ b/tests/xfs/582
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/583 b/tests/xfs/583
index a6121a83bb65..9eb4cefe05f0 100755
--- a/tests/xfs/583
+++ b/tests/xfs/583
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/584 b/tests/xfs/584
index c80ba57550cb..81ab3e82120b 100755
--- a/tests/xfs/584
+++ b/tests/xfs/584
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/585 b/tests/xfs/585
index ea47dada7bc3..74493ba1f3d7 100755
--- a/tests/xfs/585
+++ b/tests/xfs/585
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/586 b/tests/xfs/586
index e802ee718887..8d1e960fe0d4 100755
--- a/tests/xfs/586
+++ b/tests/xfs/586
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/587 b/tests/xfs/587
index 71e1ce69ae0b..dd9442c203ae 100755
--- a/tests/xfs/587
+++ b/tests/xfs/587
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/588 b/tests/xfs/588
index f56c50ace5f2..824f47fc8d05 100755
--- a/tests/xfs/588
+++ b/tests/xfs/588
@@ -7,7 +7,7 @@
 # Race fsstress and data fork scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/589 b/tests/xfs/589
index d9cd81e02be8..2ca3dd3d0d41 100755
--- a/tests/xfs/589
+++ b/tests/xfs/589
@@ -7,7 +7,7 @@
 # Race fsstress and attr fork scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/590 b/tests/xfs/590
index 4e39109abd9a..587c0be19cca 100755
--- a/tests/xfs/590
+++ b/tests/xfs/590
@@ -7,7 +7,7 @@
 # Race fsstress and cow fork scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/591 b/tests/xfs/591
index 00d5114e06ef..79492e8aeefb 100755
--- a/tests/xfs/591
+++ b/tests/xfs/591
@@ -7,7 +7,7 @@
 # Race fsstress and directory scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/592 b/tests/xfs/592
index 02ac456b5e2b..aacd95cbfad4 100755
--- a/tests/xfs/592
+++ b/tests/xfs/592
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/593 b/tests/xfs/593
index cf2ac506ca72..7a8b4a6010fc 100755
--- a/tests/xfs/593
+++ b/tests/xfs/593
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/594 b/tests/xfs/594
index 323b191b59ae..2f6287396be1 100755
--- a/tests/xfs/594
+++ b/tests/xfs/594
@@ -8,7 +8,7 @@
 # We can't open symlink files directly for scrubbing, so we use xfs_scrub(8).
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/595 b/tests/xfs/595
index fc2a89ed8625..4e431258ce58 100755
--- a/tests/xfs/595
+++ b/tests/xfs/595
@@ -9,7 +9,7 @@
 # xfs_scrub(8).
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/727 b/tests/xfs/727
index 6c5ac7db5e47..81be43cc521d 100755
--- a/tests/xfs/727
+++ b/tests/xfs/727
@@ -8,7 +8,7 @@
 # livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/729 b/tests/xfs/729
index 235cb175d259..70ed67eb24f3 100755
--- a/tests/xfs/729
+++ b/tests/xfs/729
@@ -7,7 +7,7 @@
 # Race fsstress and nlinks scrub for a while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	_scratch_xfs_stress_scrub_cleanup &> /dev/null
diff --git a/tests/xfs/800 b/tests/xfs/800
index a23e47338e59..6086a4ee2fa2 100755
--- a/tests/xfs/800
+++ b/tests/xfs/800
@@ -8,7 +8,7 @@
 # while to see if we crash or livelock.
 #
 . ./common/preamble
-_begin_fstest scrub dangerous_fsstress_scrub
+_begin_fstest scrub dangerous_fsstress_scrub soak
 
 _cleanup() {
 	cd /
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 2/3] check: add support for --list-group-tests
  2024-02-16 18:18 [PATCH fstests 0/3] few enhancements Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 1/3] tests: augment soak test group Luis Chamberlain
@ 2024-02-16 18:18 ` Luis Chamberlain
  2024-02-19  3:38   ` Dave Chinner
  2024-02-16 18:18 ` [PATCH 3/3] check: add --print-start-done to enhance watchdogs Luis Chamberlain
  2 siblings, 1 reply; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-16 18:18 UTC (permalink / raw)
  To: fstests, anand.jain, aalbersh, djwong
  Cc: linux-fsdevel, kdevops, patches, Luis Chamberlain

Since the prior commit adds the ability to list groups but is used
only when we use --start-after, let's add an option which leverages this
to also allow us to easily query which tests are part of the groups
specified.

This can be used for dynamic test configuration suites such as kdevops
which may want to take advantage of this information to deterministically
determine if a test falls part of a specific group.

Demo:

root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak

generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 check | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/check b/check
index f081bf8ce685..523cf024c139 100755
--- a/check
+++ b/check
@@ -19,6 +19,7 @@ have_test_arg=false
 randomize=false
 exact_order=false
 start_after_test=""
+list_group_tests=false
 export here=`pwd`
 xfile=""
 subdir_xfile=""
@@ -81,6 +82,7 @@ check options
     -b			brief test summary
     -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
     --large-fs		optimise scratch device for large filesystems
+    --list-group-tests	only list tests part of the groups you specified, do not run the tests
     --start-after	only start testing after the test specified
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
@@ -276,8 +278,16 @@ _prepare_test_list()
 			done
 			group_all="$group_all $list"
 		done
+
+		group_all=$(echo $group_all | sed -e 's|tests/||g')
+
+		# Keep it simple, allow for easy machine scraping
+		if $list_group_tests ; then
+			echo $group_all
+			exit 0
+		fi
+
 		if [[ "$start_after_test" != "" && $start_after_found -ne 1 ]]; then
-			group_all=$(echo $group_all | sed -e 's|tests/||g')
 			echo "Start after test $start_after_test not found in any group specified."
 			echo "Be sure you specify a test present in one of your test run groups if using --start-after."
 			echo
@@ -366,6 +376,9 @@ while [ $# -gt 0 ]; do
 		start_after_test="$2"
 		shift
 		;;
+	--list-group-tests)
+		list_group_tests=true
+		;;
 	-s)	RUN_SECTION="$RUN_SECTION $2"; shift ;;
 	-S)	EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;;
 	-l)	diff="diff" ;;
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [PATCH 3/3] check: add --print-start-done to enhance watchdogs
  2024-02-16 18:18 [PATCH fstests 0/3] few enhancements Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 1/3] tests: augment soak test group Luis Chamberlain
  2024-02-16 18:18 ` [PATCH 2/3] check: add support for --list-group-tests Luis Chamberlain
@ 2024-02-16 18:18 ` Luis Chamberlain
  2024-02-19  3:42   ` Dave Chinner
  2 siblings, 1 reply; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-16 18:18 UTC (permalink / raw)
  To: fstests, anand.jain, aalbersh, djwong
  Cc: linux-fsdevel, kdevops, patches, Luis Chamberlain

fstests specific watchdogs want to know when the full test suite will
start and end. Right now the kernel ring buffer can get augmented but we
can't know for sure if it was due to a test or some odd hardware issue
after fstests ran. This is specially true for systems left running tests in
loops in automation where we are not running things ourselves but rather just
get access to kernel logs, or for filesystem runner watdogs such as the one
in kdevops [0]. It is also often not easy to determine for sure based on
just logs when fstests check really has completed unless we have a
matching log of who spawned that test runner. Although we could keep track of
this ourselves by an annotation locally on the test runner, it is useful to
have independent tools which are not attached to the process which spawned
check to just peak into a system and verify the system's progress with
fstests by just using the kernel log. Keeping this in the test target kernel
ring buffer enables these use cases.

This is useful for example for filesyste checker specific watchdogs like the
one in kdevops so that the watchdog knows when to start hunting for crashes
based just on the kernel ring buffer, and so it also knows when the show is
over.

[0] https://github.com/linux-kdevops/kdevops/blob/master/scripts/workflows/fstests/fstests_watchdog.py

Demo:

root@demo-xfs-reflink /var/lib/xfstests # dmesg -c > /dev/null
root@demo-xfs-reflink /var/lib/xfstests # ./check -s xfs_reflink --print-start-done generic/002
SECTION       -- xfs_reflink
RECREATING    -- xfs on /dev/loop16
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/x86_64 demo-xfs-reflink 6.5.0-5-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.13-1 (2023-11-29)
MKFS_OPTIONS  -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, /dev/loop5
MOUNT_OPTIONS -- /dev/loop5 /media/scratch

generic/002        1s
Ran: generic/002
Passed all 1 tests

SECTION       -- xfs_reflink
=========================
Ran: generic/002
Passed all 1 tests

root@demo-xfs-reflink /var/lib/xfstests # dmesg -c
[61392.525562] XFS (loop16): Mounting V5 Filesystem 20c3bdf7-69d8-42b2-8a7c-fccff0949dcc
[61392.536059] XFS (loop16): Ending clean mount
[61392.684417] run fstests fstestsstart/000 at 2024-02-16 17:51:28
[61392.726709] XFS (loop16): Unmounting Filesystem 20c3bdf7-69d8-42b2-8a7c-fccff0949dcc
[61392.779791] XFS (loop16): Mounting V5 Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61392.791217] XFS (loop16): Ending clean mount
[61393.328386] XFS (loop5): Mounting V5 Filesystem 6e3f6c64-5b48-41b6-8810-d2a41fbcd125
[61393.340019] XFS (loop5): Ending clean mount
[61393.347636] XFS (loop5): Unmounting Filesystem 6e3f6c64-5b48-41b6-8810-d2a41fbcd125
[61393.403519] XFS (loop16): Unmounting Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61393.456945] XFS (loop16): Mounting V5 Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61393.466506] XFS (loop16): Ending clean mount
[61393.504926] run fstests generic/002 at 2024-02-16 17:51:29
[61394.579638] XFS (loop16): Unmounting Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61394.637296] XFS (loop16): Mounting V5 Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61394.646365] XFS (loop16): Ending clean mount
[61394.762667] XFS (loop16): Unmounting Filesystem ce4188e8-8da8-474a-acb4-b1d8c3e7edb7
[61394.790581] run fstests fstestsdone/000 at 2024-02-16 17:51:30

Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
---
 check | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/check b/check
index 523cf024c139..b3fdda57f665 100755
--- a/check
+++ b/check
@@ -19,6 +19,7 @@ have_test_arg=false
 randomize=false
 exact_order=false
 start_after_test=""
+print_start_done=false
 list_group_tests=false
 export here=`pwd`
 xfile=""
@@ -84,6 +85,7 @@ check options
     --large-fs		optimise scratch device for large filesystems
     --list-group-tests	only list tests part of the groups you specified, do not run the tests
     --start-after	only start testing after the test specified
+    --print-start-done  append to /dev/kmsg when available test start and end time
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
     -L <n>		loop tests <n> times following a failure, measuring aggregate pass/fail metrics
@@ -379,6 +381,11 @@ while [ $# -gt 0 ]; do
 	--list-group-tests)
 		list_group_tests=true
 		;;
+	--print-start-done)
+		if [ -w /dev/kmsg ]; then
+			print_start_done=true
+		fi
+		;;
 	-s)	RUN_SECTION="$RUN_SECTION $2"; shift ;;
 	-S)	EXCLUDE_SECTION="$EXCLUDE_SECTION $2"; shift ;;
 	-l)	diff="diff" ;;
@@ -1161,6 +1168,11 @@ function run_section()
 	_scratch_unmount 2> /dev/null
 }
 
+if $print_start_done; then
+	start_time=`date +"%F %T"`
+	echo "run fstests fstestsstart/000 at $start_time" > /dev/kmsg
+fi
+
 for ((iters = 0; iters < $iterations; iters++)) do
 	for section in $HOST_OPTIONS_SECTIONS; do
 		run_section $section
@@ -1172,6 +1184,11 @@ for ((iters = 0; iters < $iterations; iters++)) do
 	done
 done
 
+if $print_start_done; then
+	end_time=`date +"%F %T"`
+	echo "run fstests fstestsdone/000 at $end_time" > /dev/kmsg
+fi
+
 interrupt=false
 status=`expr $sum_bad != 0`
 exit
-- 
2.42.0


^ permalink raw reply related	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/3] check: add support for --list-group-tests
  2024-02-16 18:18 ` [PATCH 2/3] check: add support for --list-group-tests Luis Chamberlain
@ 2024-02-19  3:38   ` Dave Chinner
  2024-02-21 16:45     ` Luis Chamberlain
  0 siblings, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2024-02-19  3:38 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: fstests, anand.jain, aalbersh, djwong, linux-fsdevel, kdevops, patches

On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> Since the prior commit adds the ability to list groups but is used
> only when we use --start-after, let's add an option which leverages this
> to also allow us to easily query which tests are part of the groups
> specified.
> 
> This can be used for dynamic test configuration suites such as kdevops
> which may want to take advantage of this information to deterministically
> determine if a test falls part of a specific group.
> Demo:
> 
> root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> 
> generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800

So how is this different to ./check -n -g soak?

'-n' is supposed to show you want tests are going to be run
without actually running them, so why can't you use that?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] check: add --print-start-done to enhance watchdogs
  2024-02-16 18:18 ` [PATCH 3/3] check: add --print-start-done to enhance watchdogs Luis Chamberlain
@ 2024-02-19  3:42   ` Dave Chinner
  2024-02-21 16:46     ` Luis Chamberlain
  0 siblings, 1 reply; 9+ messages in thread
From: Dave Chinner @ 2024-02-19  3:42 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: fstests, anand.jain, aalbersh, djwong, linux-fsdevel, kdevops, patches

On Fri, Feb 16, 2024 at 10:18:59AM -0800, Luis Chamberlain wrote:
> fstests specific watchdogs want to know when the full test suite will
> start and end. Right now the kernel ring buffer can get augmented but we
> can't know for sure if it was due to a test or some odd hardware issue
> after fstests ran. This is specially true for systems left running tests in
> loops in automation where we are not running things ourselves but rather just
> get access to kernel logs, or for filesystem runner watdogs such as the one
> in kdevops [0]. It is also often not easy to determine for sure based on
> just logs when fstests check really has completed unless we have a
> matching log of who spawned that test runner. Although we could keep track of
> this ourselves by an annotation locally on the test runner, it is useful to
> have independent tools which are not attached to the process which spawned
> check to just peak into a system and verify the system's progress with
> fstests by just using the kernel log. Keeping this in the test target kernel
> ring buffer enables these use cases.
> 
> This is useful for example for filesyste checker specific watchdogs like the
> one in kdevops so that the watchdog knows when to start hunting for crashes
> based just on the kernel ring buffer, and so it also knows when the show is
> over.

Why can't the runner that requires timing information in the
kernel log just emit a message to the kernel log before it
runs check and again immediately after completion of the check
script?

-Dave.
-- 
Dave Chinner
david@fromorbit.com

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/3] check: add support for --list-group-tests
  2024-02-19  3:38   ` Dave Chinner
@ 2024-02-21 16:45     ` Luis Chamberlain
  2024-02-25 16:08       ` Zorro Lang
  0 siblings, 1 reply; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-21 16:45 UTC (permalink / raw)
  To: Dave Chinner
  Cc: fstests, anand.jain, aalbersh, djwong, linux-fsdevel, kdevops, patches

On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote:
> On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> > Since the prior commit adds the ability to list groups but is used
> > only when we use --start-after, let's add an option which leverages this
> > to also allow us to easily query which tests are part of the groups
> > specified.
> > 
> > This can be used for dynamic test configuration suites such as kdevops
> > which may want to take advantage of this information to deterministically
> > determine if a test falls part of a specific group.
> > Demo:
> > 
> > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> > 
> > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800
> 
> So how is this different to ./check -n -g soak?
> 
> '-n' is supposed to show you want tests are going to be run
> without actually running them, so why can't you use that?

'-n' will replicate as if you are running all tests but just skip while
--list-group-tests will just look for the tests for the group and bail right
away, and it is machine readable.

  Luis

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 3/3] check: add --print-start-done to enhance watchdogs
  2024-02-19  3:42   ` Dave Chinner
@ 2024-02-21 16:46     ` Luis Chamberlain
  0 siblings, 0 replies; 9+ messages in thread
From: Luis Chamberlain @ 2024-02-21 16:46 UTC (permalink / raw)
  To: Dave Chinner
  Cc: fstests, anand.jain, aalbersh, djwong, linux-fsdevel, kdevops, patches

On Mon, Feb 19, 2024 at 02:42:32PM +1100, Dave Chinner wrote:
> On Fri, Feb 16, 2024 at 10:18:59AM -0800, Luis Chamberlain wrote:
> > fstests specific watchdogs want to know when the full test suite will
> > start and end. Right now the kernel ring buffer can get augmented but we
> > can't know for sure if it was due to a test or some odd hardware issue
> > after fstests ran. This is specially true for systems left running tests in
> > loops in automation where we are not running things ourselves but rather just
> > get access to kernel logs, or for filesystem runner watdogs such as the one
> > in kdevops [0]. It is also often not easy to determine for sure based on
> > just logs when fstests check really has completed unless we have a
> > matching log of who spawned that test runner. Although we could keep track of
> > this ourselves by an annotation locally on the test runner, it is useful to
> > have independent tools which are not attached to the process which spawned
> > check to just peak into a system and verify the system's progress with
> > fstests by just using the kernel log. Keeping this in the test target kernel
> > ring buffer enables these use cases.
> > 
> > This is useful for example for filesyste checker specific watchdogs like the
> > one in kdevops so that the watchdog knows when to start hunting for crashes
> > based just on the kernel ring buffer, and so it also knows when the show is
> > over.
> 
> Why can't the runner that requires timing information in the
> kernel log just emit a message to the kernel log before it
> runs check and again immediately after completion of the check
> script?

That's exactly what is done today, it just seemed to me that since this
has been useful to a test runner now for years, it might make sense to
generalize this.
 
  Luis

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: [PATCH 2/3] check: add support for --list-group-tests
  2024-02-21 16:45     ` Luis Chamberlain
@ 2024-02-25 16:08       ` Zorro Lang
  0 siblings, 0 replies; 9+ messages in thread
From: Zorro Lang @ 2024-02-25 16:08 UTC (permalink / raw)
  To: Luis Chamberlain
  Cc: Dave Chinner, fstests, anand.jain, aalbersh, djwong,
	linux-fsdevel, kdevops, patches

On Wed, Feb 21, 2024 at 08:45:37AM -0800, Luis Chamberlain wrote:
> On Mon, Feb 19, 2024 at 02:38:12PM +1100, Dave Chinner wrote:
> > On Fri, Feb 16, 2024 at 10:18:58AM -0800, Luis Chamberlain wrote:
> > > Since the prior commit adds the ability to list groups but is used
> > > only when we use --start-after, let's add an option which leverages this
> > > to also allow us to easily query which tests are part of the groups
> > > specified.
> > > 
> > > This can be used for dynamic test configuration suites such as kdevops
> > > which may want to take advantage of this information to deterministically
> > > determine if a test falls part of a specific group.
> > > Demo:
> > > 
> > > root@demo-xfs-reflink /var/lib/xfstests # ./check --list-group-tests -g soak
> > > 
> > > generic/019 generic/388 generic/475 generic/476 generic/521 generic/522 generic/616 generic/617 generic/642 generic/648 generic/650 xfs/285 xfs/517 xfs/560 xfs/561 xfs/562 xfs/565 xfs/570 xfs/571 xfs/572 xfs/573 xfs/574 xfs/575 xfs/576 xfs/577 xfs/578 xfs/579 xfs/580 xfs/581 xfs/582 xfs/583 xfs/584 xfs/585 xfs/586 xfs/587 xfs/588 xfs/589 xfs/590 xfs/591 xfs/592 xfs/593 xfs/594 xfs/595 xfs/727 xfs/729 xfs/800
> > 
> > So how is this different to ./check -n -g soak?
> > 
> > '-n' is supposed to show you want tests are going to be run
> > without actually running them, so why can't you use that?
> 
> '-n' will replicate as if you are running all tests but just skip while
> --list-group-tests will just look for the tests for the group and bail right
> away, and it is machine readable.

What do you mean "replicate as if you are running all tests but just skip"?
Sorry I don't understand this explanation 100%, can you show us some examples
to explain what kind of job you hope "--list-group-tests" to do, but the "-n"
is helpless?

Thanks,
Zorro

> 
>   Luis
> 


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2024-02-25 16:09 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-16 18:18 [PATCH fstests 0/3] few enhancements Luis Chamberlain
2024-02-16 18:18 ` [PATCH 1/3] tests: augment soak test group Luis Chamberlain
2024-02-16 18:18 ` [PATCH 2/3] check: add support for --list-group-tests Luis Chamberlain
2024-02-19  3:38   ` Dave Chinner
2024-02-21 16:45     ` Luis Chamberlain
2024-02-25 16:08       ` Zorro Lang
2024-02-16 18:18 ` [PATCH 3/3] check: add --print-start-done to enhance watchdogs Luis Chamberlain
2024-02-19  3:42   ` Dave Chinner
2024-02-21 16:46     ` Luis Chamberlain

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.