* [PATCH] debug: add tests for debugging testing setup
@ 2023-02-09 1:31 Leah Rumancik
2023-02-09 2:47 ` Darrick J. Wong
0 siblings, 1 reply; 6+ messages in thread
From: Leah Rumancik @ 2023-02-09 1:31 UTC (permalink / raw)
To: fstests; +Cc: zlang, Leah Rumancik
Many people have developed infrastructure around xfstests. In order to
test a setup, it would be helpful to have dummy tests that have
consistent test outcomes. Add a new test folder with the following
debugging tests:
debug/001 pass
debug/002 fail
debug/003 skip
debug/004 crash
debug/005 hang
Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
---
doc/group-names.txt | 1 +
tests/debug/001 | 14 ++++++++++++++
tests/debug/001.out | 2 ++
tests/debug/002 | 14 ++++++++++++++
tests/debug/002.out | 2 ++
tests/debug/003 | 16 ++++++++++++++++
tests/debug/003.out | 2 ++
tests/debug/004 | 17 +++++++++++++++++
tests/debug/004.out | 2 ++
tests/debug/005 | 20 ++++++++++++++++++++
tests/debug/005.out | 2 ++
tests/debug/Makefile | 24 ++++++++++++++++++++++++
12 files changed, 116 insertions(+)
create mode 100755 tests/debug/001
create mode 100644 tests/debug/001.out
create mode 100755 tests/debug/002
create mode 100644 tests/debug/002.out
create mode 100755 tests/debug/003
create mode 100644 tests/debug/003.out
create mode 100755 tests/debug/004
create mode 100644 tests/debug/004.out
create mode 100755 tests/debug/005
create mode 100644 tests/debug/005.out
create mode 100644 tests/debug/Makefile
diff --git a/doc/group-names.txt b/doc/group-names.txt
index 6cc9af78..212da7e7 100644
--- a/doc/group-names.txt
+++ b/doc/group-names.txt
@@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
data data loss checkers
dax direct access mode for persistent memory files
db xfs_db functional tests
+debug tests with fixed results, used to validate testing setup
dedupe FIEDEDUPERANGE ioctl
defrag filesystem defragmenters
dir directory test functions
diff --git a/tests/debug/001 b/tests/debug/001
new file mode 100755
index 00000000..9567449c
--- /dev/null
+++ b/tests/debug/001
@@ -0,0 +1,14 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 001
+#
+# A test that should always pass
+#
+. ./common/preamble
+_begin_fstest debug
+
+echo "Silence is golden"
+status=0
+exit
+
diff --git a/tests/debug/001.out b/tests/debug/001.out
new file mode 100644
index 00000000..88678b8e
--- /dev/null
+++ b/tests/debug/001.out
@@ -0,0 +1,2 @@
+QA output created by 001
+Silence is golden
diff --git a/tests/debug/002 b/tests/debug/002
new file mode 100755
index 00000000..12b45a94
--- /dev/null
+++ b/tests/debug/002
@@ -0,0 +1,14 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 002
+#
+# A test that should always fail
+#
+. ./common/preamble
+_begin_fstest debug
+
+echo "I am intentionally broken"
+status=0
+exit
+
diff --git a/tests/debug/002.out b/tests/debug/002.out
new file mode 100644
index 00000000..61705c7c
--- /dev/null
+++ b/tests/debug/002.out
@@ -0,0 +1,2 @@
+QA output created by 002
+Silence is golden
diff --git a/tests/debug/003 b/tests/debug/003
new file mode 100755
index 00000000..813e6ff6
--- /dev/null
+++ b/tests/debug/003
@@ -0,0 +1,16 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 003
+#
+# A test that should always be skipped
+#
+. ./common/preamble
+_begin_fstest debug
+
+_notrun "Always skip me"
+
+echo "I should be skipped"
+status=0
+exit
+
diff --git a/tests/debug/003.out b/tests/debug/003.out
new file mode 100644
index 00000000..6895fc80
--- /dev/null
+++ b/tests/debug/003.out
@@ -0,0 +1,2 @@
+QA output created by 003
+Silence is golden
diff --git a/tests/debug/004 b/tests/debug/004
new file mode 100755
index 00000000..2e5f1e43
--- /dev/null
+++ b/tests/debug/004
@@ -0,0 +1,17 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 004
+#
+# A test that crashes
+#
+. ./common/preamble
+_begin_fstest debug dangerous
+
+echo 1 > /proc/sys/kernel/sysrq
+echo c > /proc/sysrq-trigger
+
+echo "I should have crashed by now"
+status=0
+exit
+
diff --git a/tests/debug/004.out b/tests/debug/004.out
new file mode 100644
index 00000000..af8614ae
--- /dev/null
+++ b/tests/debug/004.out
@@ -0,0 +1,2 @@
+QA output created by 004
+Silence is golden
diff --git a/tests/debug/005 b/tests/debug/005
new file mode 100755
index 00000000..1a213926
--- /dev/null
+++ b/tests/debug/005
@@ -0,0 +1,20 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# FS QA Test No. 005
+#
+# A test that hangs
+#
+
+. ./common/preamble
+_begin_fstest debug dangerous
+
+while :
+do
+ sleep 1d
+done
+
+echo "I should still be sleeping"
+status=0
+exit
+
diff --git a/tests/debug/005.out b/tests/debug/005.out
new file mode 100644
index 00000000..a5027f12
--- /dev/null
+++ b/tests/debug/005.out
@@ -0,0 +1,2 @@
+QA output created by 005
+Silence is golden
diff --git a/tests/debug/Makefile b/tests/debug/Makefile
new file mode 100644
index 00000000..84556199
--- /dev/null
+++ b/tests/debug/Makefile
@@ -0,0 +1,24 @@
+#
+# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
+#
+
+TOPDIR = ../..
+include $(TOPDIR)/include/builddefs
+include $(TOPDIR)/include/buildgrouplist
+
+DEBUG_DIR = debug
+TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
+DIRT = group.list
+
+default: $(DIRT)
+
+include $(BUILDRULES)
+
+install:
+ $(INSTALL) -m 755 -d $(TARGET_DIR)
+ $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
+ $(INSTALL) -m 644 group.list $(TARGET_DIR)
+ $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
+
+# Nothing.
+install-dev install-lib:
--
2.39.1.519.gcb327c4b5f-goog
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] debug: add tests for debugging testing setup
2023-02-09 1:31 [PATCH] debug: add tests for debugging testing setup Leah Rumancik
@ 2023-02-09 2:47 ` Darrick J. Wong
2023-02-09 5:53 ` Zorro Lang
2023-02-09 21:35 ` Leah Rumancik
0 siblings, 2 replies; 6+ messages in thread
From: Darrick J. Wong @ 2023-02-09 2:47 UTC (permalink / raw)
To: Leah Rumancik; +Cc: fstests, zlang
On Wed, Feb 08, 2023 at 05:31:43PM -0800, Leah Rumancik wrote:
> Many people have developed infrastructure around xfstests. In order to
> test a setup, it would be helpful to have dummy tests that have
> consistent test outcomes. Add a new test folder with the following
> debugging tests:
Ooh, we've been needing selftests for a while, thanks!
> debug/001 pass
> debug/002 fail
> debug/003 skip
> debug/004 crash
> debug/005 hang
>
> Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> ---
> doc/group-names.txt | 1 +
> tests/debug/001 | 14 ++++++++++++++
> tests/debug/001.out | 2 ++
> tests/debug/002 | 14 ++++++++++++++
> tests/debug/002.out | 2 ++
> tests/debug/003 | 16 ++++++++++++++++
> tests/debug/003.out | 2 ++
> tests/debug/004 | 17 +++++++++++++++++
> tests/debug/004.out | 2 ++
> tests/debug/005 | 20 ++++++++++++++++++++
> tests/debug/005.out | 2 ++
> tests/debug/Makefile | 24 ++++++++++++++++++++++++
> 12 files changed, 116 insertions(+)
> create mode 100755 tests/debug/001
> create mode 100644 tests/debug/001.out
> create mode 100755 tests/debug/002
> create mode 100644 tests/debug/002.out
> create mode 100755 tests/debug/003
> create mode 100644 tests/debug/003.out
> create mode 100755 tests/debug/004
> create mode 100644 tests/debug/004.out
> create mode 100755 tests/debug/005
> create mode 100644 tests/debug/005.out
> create mode 100644 tests/debug/Makefile
>
> diff --git a/doc/group-names.txt b/doc/group-names.txt
> index 6cc9af78..212da7e7 100644
> --- a/doc/group-names.txt
> +++ b/doc/group-names.txt
> @@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
> data data loss checkers
> dax direct access mode for persistent memory files
> db xfs_db functional tests
> +debug tests with fixed results, used to validate testing setup
Bikeshed: Can we make the tag 'selftests' since debugging could refer to
just about anything in the kernel/userspace/otherstuff stacks?
Also a dangerous_selftests for tests that will crash or hang the test
appliances. I'd really like to run this every time I start up fstests,
albeit without the dangerous ones. :)
Or <cough> fsteststests.
(No, really not that.)
> dedupe FIEDEDUPERANGE ioctl
> defrag filesystem defragmenters
> dir directory test functions
> diff --git a/tests/debug/001 b/tests/debug/001
> new file mode 100755
> index 00000000..9567449c
> --- /dev/null
> +++ b/tests/debug/001
> @@ -0,0 +1,14 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 001
> +#
> +# A test that should always pass
> +#
> +. ./common/preamble
> +_begin_fstest debug
> +
> +echo "Silence is golden"
> +status=0
> +exit
> +
> diff --git a/tests/debug/001.out b/tests/debug/001.out
> new file mode 100644
> index 00000000..88678b8e
> --- /dev/null
> +++ b/tests/debug/001.out
> @@ -0,0 +1,2 @@
> +QA output created by 001
> +Silence is golden
> diff --git a/tests/debug/002 b/tests/debug/002
> new file mode 100755
> index 00000000..12b45a94
> --- /dev/null
> +++ b/tests/debug/002
> @@ -0,0 +1,14 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 002
> +#
> +# A test that should always fail
> +#
> +. ./common/preamble
> +_begin_fstest debug
> +
> +echo "I am intentionally broken"
Would you mind slipping in another test to simulate tests that cannot
satisfy preconditions and hence _fail?
> +status=0
> +exit
> +
> diff --git a/tests/debug/002.out b/tests/debug/002.out
> new file mode 100644
> index 00000000..61705c7c
> --- /dev/null
> +++ b/tests/debug/002.out
> @@ -0,0 +1,2 @@
> +QA output created by 002
> +Silence is golden
> diff --git a/tests/debug/003 b/tests/debug/003
> new file mode 100755
> index 00000000..813e6ff6
> --- /dev/null
> +++ b/tests/debug/003
> @@ -0,0 +1,16 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 003
> +#
> +# A test that should always be skipped
> +#
> +. ./common/preamble
> +_begin_fstest debug
> +
> +_notrun "Always skip me"
> +
> +echo "I should be skipped"
> +status=0
> +exit
> +
> diff --git a/tests/debug/003.out b/tests/debug/003.out
> new file mode 100644
> index 00000000..6895fc80
> --- /dev/null
> +++ b/tests/debug/003.out
> @@ -0,0 +1,2 @@
> +QA output created by 003
> +Silence is golden
> diff --git a/tests/debug/004 b/tests/debug/004
> new file mode 100755
> index 00000000..2e5f1e43
> --- /dev/null
> +++ b/tests/debug/004
> @@ -0,0 +1,17 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 004
> +#
> +# A test that crashes
> +#
> +. ./common/preamble
> +_begin_fstest debug dangerous
_begin_fstest dangerous_selftest?
> +
> +echo 1 > /proc/sys/kernel/sysrq
> +echo c > /proc/sysrq-trigger
> +
> +echo "I should have crashed by now"
> +status=0
> +exit
> +
> diff --git a/tests/debug/004.out b/tests/debug/004.out
> new file mode 100644
> index 00000000..af8614ae
> --- /dev/null
> +++ b/tests/debug/004.out
> @@ -0,0 +1,2 @@
> +QA output created by 004
> +Silence is golden
> diff --git a/tests/debug/005 b/tests/debug/005
> new file mode 100755
> index 00000000..1a213926
> --- /dev/null
> +++ b/tests/debug/005
> @@ -0,0 +1,20 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# FS QA Test No. 005
> +#
> +# A test that hangs
> +#
> +
> +. ./common/preamble
> +_begin_fstest debug dangerous
Here too.
> +
> +while :
> +do
> + sleep 1d
> +done
> +
> +echo "I should still be sleeping"
> +status=0
> +exit
> +
> diff --git a/tests/debug/005.out b/tests/debug/005.out
> new file mode 100644
> index 00000000..a5027f12
> --- /dev/null
> +++ b/tests/debug/005.out
> @@ -0,0 +1,2 @@
> +QA output created by 005
> +Silence is golden
> diff --git a/tests/debug/Makefile b/tests/debug/Makefile
> new file mode 100644
> index 00000000..84556199
> --- /dev/null
> +++ b/tests/debug/Makefile
> @@ -0,0 +1,24 @@
> +#
> +# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
> +#
> +
> +TOPDIR = ../..
> +include $(TOPDIR)/include/builddefs
> +include $(TOPDIR)/include/buildgrouplist
> +
> +DEBUG_DIR = debug
> +TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
> +DIRT = group.list
> +
> +default: $(DIRT)
> +
> +include $(BUILDRULES)
> +
> +install:
> + $(INSTALL) -m 755 -d $(TARGET_DIR)
> + $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
> + $(INSTALL) -m 644 group.list $(TARGET_DIR)
> + $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
> +
> +# Nothing.
> +install-dev install-lib:
> --
> 2.39.1.519.gcb327c4b5f-goog
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] debug: add tests for debugging testing setup
2023-02-09 2:47 ` Darrick J. Wong
@ 2023-02-09 5:53 ` Zorro Lang
2023-02-09 21:47 ` Leah Rumancik
2023-02-09 21:35 ` Leah Rumancik
1 sibling, 1 reply; 6+ messages in thread
From: Zorro Lang @ 2023-02-09 5:53 UTC (permalink / raw)
To: Leah Rumancik; +Cc: fstests, Darrick J. Wong
On Wed, Feb 08, 2023 at 06:47:30PM -0800, Darrick J. Wong wrote:
> On Wed, Feb 08, 2023 at 05:31:43PM -0800, Leah Rumancik wrote:
> > Many people have developed infrastructure around xfstests. In order to
> > test a setup, it would be helpful to have dummy tests that have
> > consistent test outcomes. Add a new test folder with the following
> > debugging tests:
>
> Ooh, we've been needing selftests for a while, thanks!
>
> > debug/001 pass
> > debug/002 fail
> > debug/003 skip
> > debug/004 crash
> > debug/005 hang
> >
> > Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> > ---
> > doc/group-names.txt | 1 +
> > tests/debug/001 | 14 ++++++++++++++
> > tests/debug/001.out | 2 ++
> > tests/debug/002 | 14 ++++++++++++++
> > tests/debug/002.out | 2 ++
> > tests/debug/003 | 16 ++++++++++++++++
> > tests/debug/003.out | 2 ++
> > tests/debug/004 | 17 +++++++++++++++++
> > tests/debug/004.out | 2 ++
> > tests/debug/005 | 20 ++++++++++++++++++++
> > tests/debug/005.out | 2 ++
> > tests/debug/Makefile | 24 ++++++++++++++++++++++++
> > 12 files changed, 116 insertions(+)
> > create mode 100755 tests/debug/001
> > create mode 100644 tests/debug/001.out
> > create mode 100755 tests/debug/002
> > create mode 100644 tests/debug/002.out
> > create mode 100755 tests/debug/003
> > create mode 100644 tests/debug/003.out
> > create mode 100755 tests/debug/004
> > create mode 100644 tests/debug/004.out
> > create mode 100755 tests/debug/005
> > create mode 100644 tests/debug/005.out
> > create mode 100644 tests/debug/Makefile
> >
> > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > index 6cc9af78..212da7e7 100644
> > --- a/doc/group-names.txt
> > +++ b/doc/group-names.txt
> > @@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
> > data data loss checkers
> > dax direct access mode for persistent memory files
> > db xfs_db functional tests
> > +debug tests with fixed results, used to validate testing setup
>
> Bikeshed: Can we make the tag 'selftests' since debugging could refer to
> just about anything in the kernel/userspace/otherstuff stacks?
>
> Also a dangerous_selftests for tests that will crash or hang the test
> appliances. I'd really like to run this every time I start up fstests,
> albeit without the dangerous ones. :)
I agree with this. When I saw *debug*, I thought it's about some kernel/fs
debuging test, or it need kernel debug option. So *selftest* might be a better
name, or if someone has better idea :)
With group name changing, I think better to change tests/debug/ directory to
tests/selftests/.
>
> Or <cough> fsteststests.
>
> (No, really not that.)
>
> > dedupe FIEDEDUPERANGE ioctl
> > defrag filesystem defragmenters
> > dir directory test functions
> > diff --git a/tests/debug/001 b/tests/debug/001
> > new file mode 100755
> > index 00000000..9567449c
> > --- /dev/null
> > +++ b/tests/debug/001
> > @@ -0,0 +1,14 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 001
> > +#
> > +# A test that should always pass
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +echo "Silence is golden"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/001.out b/tests/debug/001.out
> > new file mode 100644
> > index 00000000..88678b8e
> > --- /dev/null
> > +++ b/tests/debug/001.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 001
> > +Silence is golden
> > diff --git a/tests/debug/002 b/tests/debug/002
> > new file mode 100755
> > index 00000000..12b45a94
> > --- /dev/null
> > +++ b/tests/debug/002
> > @@ -0,0 +1,14 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 002
> > +#
> > +# A test that should always fail
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +echo "I am intentionally broken"
>
> Would you mind slipping in another test to simulate tests that cannot
> satisfy preconditions and hence _fail?
>
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/002.out b/tests/debug/002.out
> > new file mode 100644
> > index 00000000..61705c7c
> > --- /dev/null
> > +++ b/tests/debug/002.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 002
> > +Silence is golden
> > diff --git a/tests/debug/003 b/tests/debug/003
> > new file mode 100755
> > index 00000000..813e6ff6
> > --- /dev/null
> > +++ b/tests/debug/003
> > @@ -0,0 +1,16 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 003
> > +#
> > +# A test that should always be skipped
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +_notrun "Always skip me"
> > +
> > +echo "I should be skipped"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/003.out b/tests/debug/003.out
> > new file mode 100644
> > index 00000000..6895fc80
> > --- /dev/null
> > +++ b/tests/debug/003.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 003
> > +Silence is golden
> > diff --git a/tests/debug/004 b/tests/debug/004
> > new file mode 100755
> > index 00000000..2e5f1e43
> > --- /dev/null
> > +++ b/tests/debug/004
> > @@ -0,0 +1,17 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 004
> > +#
> > +# A test that crashes
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug dangerous
>
> _begin_fstest dangerous_selftest?
Agree
>
> > +
> > +echo 1 > /proc/sys/kernel/sysrq
> > +echo c > /proc/sysrq-trigger
> > +
> > +echo "I should have crashed by now"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/004.out b/tests/debug/004.out
> > new file mode 100644
> > index 00000000..af8614ae
> > --- /dev/null
> > +++ b/tests/debug/004.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 004
> > +Silence is golden
> > diff --git a/tests/debug/005 b/tests/debug/005
> > new file mode 100755
> > index 00000000..1a213926
> > --- /dev/null
> > +++ b/tests/debug/005
> > @@ -0,0 +1,20 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 005
> > +#
> > +# A test that hangs
> > +#
> > +
> > +. ./common/preamble
> > +_begin_fstest debug dangerous
>
> Here too.
>
> > +
> > +while :
> > +do
> > + sleep 1d
> > +done
> > +
> > +echo "I should still be sleeping"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/005.out b/tests/debug/005.out
> > new file mode 100644
> > index 00000000..a5027f12
> > --- /dev/null
> > +++ b/tests/debug/005.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 005
> > +Silence is golden
> > diff --git a/tests/debug/Makefile b/tests/debug/Makefile
> > new file mode 100644
> > index 00000000..84556199
> > --- /dev/null
> > +++ b/tests/debug/Makefile
> > @@ -0,0 +1,24 @@
> > +#
> > +# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
Do you need to change this ^^ ?
Thanks,
Zorro
> > +#
> > +
> > +TOPDIR = ../..
> > +include $(TOPDIR)/include/builddefs
> > +include $(TOPDIR)/include/buildgrouplist
> > +
> > +DEBUG_DIR = debug
> > +TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
> > +DIRT = group.list
> > +
> > +default: $(DIRT)
> > +
> > +include $(BUILDRULES)
> > +
> > +install:
> > + $(INSTALL) -m 755 -d $(TARGET_DIR)
> > + $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
> > + $(INSTALL) -m 644 group.list $(TARGET_DIR)
> > + $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
> > +
> > +# Nothing.
> > +install-dev install-lib:
> > --
> > 2.39.1.519.gcb327c4b5f-goog
> >
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] debug: add tests for debugging testing setup
2023-02-09 2:47 ` Darrick J. Wong
2023-02-09 5:53 ` Zorro Lang
@ 2023-02-09 21:35 ` Leah Rumancik
2023-02-10 1:02 ` Darrick J. Wong
1 sibling, 1 reply; 6+ messages in thread
From: Leah Rumancik @ 2023-02-09 21:35 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: fstests, zlang
On Wed, Feb 08, 2023 at 06:47:30PM -0800, Darrick J. Wong wrote:
> On Wed, Feb 08, 2023 at 05:31:43PM -0800, Leah Rumancik wrote:
> > Many people have developed infrastructure around xfstests. In order to
> > test a setup, it would be helpful to have dummy tests that have
> > consistent test outcomes. Add a new test folder with the following
> > debugging tests:
>
> Ooh, we've been needing selftests for a while, thanks!
>
> > debug/001 pass
> > debug/002 fail
> > debug/003 skip
> > debug/004 crash
> > debug/005 hang
> >
> > Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> >
> > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > index 6cc9af78..212da7e7 100644
> > --- a/doc/group-names.txt
> > +++ b/doc/group-names.txt
> > @@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
> > data data loss checkers
> > dax direct access mode for persistent memory files
> > db xfs_db functional tests
> > +debug tests with fixed results, used to validate testing setup
>
> Bikeshed: Can we make the tag 'selftests' since debugging could refer to
> just about anything in the kernel/userspace/otherstuff stacks?
Ah sure, agreed selftests would be better, thanks.
>
> Also a dangerous_selftests for tests that will crash or hang the test
> appliances. I'd really like to run this every time I start up fstests,
> albeit without the dangerous ones. :)
How exactly would you plan to run it? To run just the non-dangerous tests,
you could do -g selftests/selftests -x selftests/dangerous_selftests. You
can't simply use -g selftests -x selftests_dangerous as right now
selftests is not being added to SRC_GROUPS. It would be nice to make it
behave more like a normal group though... If it doesn't matter that
these would run with -g all, I could just add it to SRC_GROUPS. If it'd be
best to skip these during running -g all, I could also update get_all_tests
to skip over the selftests folder. Do you think these should run with
-g all?
>
> Or <cough> fsteststests.
Ha :)
>
> (No, really not that.)
>
> > dedupe FIEDEDUPERANGE ioctl
> > defrag filesystem defragmenters
> > dir directory test functions
> > diff --git a/tests/debug/001 b/tests/debug/001
> > new file mode 100755
> > index 00000000..9567449c
> > --- /dev/null
> > +++ b/tests/debug/001
> > @@ -0,0 +1,14 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 001
> > +#
> > +# A test that should always pass
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +echo "Silence is golden"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/001.out b/tests/debug/001.out
> > new file mode 100644
> > index 00000000..88678b8e
> > --- /dev/null
> > +++ b/tests/debug/001.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 001
> > +Silence is golden
> > diff --git a/tests/debug/002 b/tests/debug/002
> > new file mode 100755
> > index 00000000..12b45a94
> > --- /dev/null
> > +++ b/tests/debug/002
> > @@ -0,0 +1,14 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 002
> > +#
> > +# A test that should always fail
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +echo "I am intentionally broken"
>
> Would you mind slipping in another test to simulate tests that cannot
> satisfy preconditions and hence _fail?
What exactly do you mean? Like a test that just calls _fail to
simulate something like _scratch_mkfs || _fail "mkfs failed"?
>
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/002.out b/tests/debug/002.out
> > new file mode 100644
> > index 00000000..61705c7c
> > --- /dev/null
> > +++ b/tests/debug/002.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 002
> > +Silence is golden
> > diff --git a/tests/debug/003 b/tests/debug/003
> > new file mode 100755
> > index 00000000..813e6ff6
> > --- /dev/null
> > +++ b/tests/debug/003
> > @@ -0,0 +1,16 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 003
> > +#
> > +# A test that should always be skipped
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug
> > +
> > +_notrun "Always skip me"
> > +
> > +echo "I should be skipped"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/003.out b/tests/debug/003.out
> > new file mode 100644
> > index 00000000..6895fc80
> > --- /dev/null
> > +++ b/tests/debug/003.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 003
> > +Silence is golden
> > diff --git a/tests/debug/004 b/tests/debug/004
> > new file mode 100755
> > index 00000000..2e5f1e43
> > --- /dev/null
> > +++ b/tests/debug/004
> > @@ -0,0 +1,17 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 004
> > +#
> > +# A test that crashes
> > +#
> > +. ./common/preamble
> > +_begin_fstest debug dangerous
>
> _begin_fstest dangerous_selftest?
>
Sure, will update.
> > +
> > +echo 1 > /proc/sys/kernel/sysrq
> > +echo c > /proc/sysrq-trigger
> > +
> > +echo "I should have crashed by now"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/004.out b/tests/debug/004.out
> > new file mode 100644
> > index 00000000..af8614ae
> > --- /dev/null
> > +++ b/tests/debug/004.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 004
> > +Silence is golden
> > diff --git a/tests/debug/005 b/tests/debug/005
> > new file mode 100755
> > index 00000000..1a213926
> > --- /dev/null
> > +++ b/tests/debug/005
> > @@ -0,0 +1,20 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#
> > +# FS QA Test No. 005
> > +#
> > +# A test that hangs
> > +#
> > +
> > +. ./common/preamble
> > +_begin_fstest debug dangerous
>
> Here too.
Ack
>
> > +
> > +while :
> > +do
> > + sleep 1d
> > +done
> > +
> > +echo "I should still be sleeping"
> > +status=0
> > +exit
> > +
> > diff --git a/tests/debug/005.out b/tests/debug/005.out
> > new file mode 100644
> > index 00000000..a5027f12
> > --- /dev/null
> > +++ b/tests/debug/005.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 005
> > +Silence is golden
> > diff --git a/tests/debug/Makefile b/tests/debug/Makefile
> > new file mode 100644
> > index 00000000..84556199
> > --- /dev/null
> > +++ b/tests/debug/Makefile
> > @@ -0,0 +1,24 @@
> > +#
> > +# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
> > +#
> > +
> > +TOPDIR = ../..
> > +include $(TOPDIR)/include/builddefs
> > +include $(TOPDIR)/include/buildgrouplist
> > +
> > +DEBUG_DIR = debug
> > +TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
> > +DIRT = group.list
> > +
> > +default: $(DIRT)
> > +
> > +include $(BUILDRULES)
> > +
> > +install:
> > + $(INSTALL) -m 755 -d $(TARGET_DIR)
> > + $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
> > + $(INSTALL) -m 644 group.list $(TARGET_DIR)
> > + $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
> > +
> > +# Nothing.
> > +install-dev install-lib:
> > --
> > 2.39.1.519.gcb327c4b5f-goog
> >
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] debug: add tests for debugging testing setup
2023-02-09 5:53 ` Zorro Lang
@ 2023-02-09 21:47 ` Leah Rumancik
0 siblings, 0 replies; 6+ messages in thread
From: Leah Rumancik @ 2023-02-09 21:47 UTC (permalink / raw)
To: Zorro Lang; +Cc: fstests, Darrick J. Wong
Oops, should have put all my replies this message. Please see my
reply to Darrick's email as well.
On Thu, Feb 09, 2023 at 01:53:02PM +0800, Zorro Lang wrote:
> On Wed, Feb 08, 2023 at 06:47:30PM -0800, Darrick J. Wong wrote:
> > On Wed, Feb 08, 2023 at 05:31:43PM -0800, Leah Rumancik wrote:
> > > Many people have developed infrastructure around xfstests. In order to
> > > test a setup, it would be helpful to have dummy tests that have
> > > consistent test outcomes. Add a new test folder with the following
> > > debugging tests:
> >
> > Ooh, we've been needing selftests for a while, thanks!
> >
> > > debug/001 pass
> > > debug/002 fail
> > > debug/003 skip
> > > debug/004 crash
> > > debug/005 hang
> > >
> > > Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> > > ---
> > >
> > > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > > index 6cc9af78..212da7e7 100644
> > > --- a/doc/group-names.txt
> > > +++ b/doc/group-names.txt
> > > @@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
> > > data data loss checkers
> > > dax direct access mode for persistent memory files
> > > db xfs_db functional tests
> > > +debug tests with fixed results, used to validate testing setup
> >
> > Bikeshed: Can we make the tag 'selftests' since debugging could refer to
> > just about anything in the kernel/userspace/otherstuff stacks?
> >
> > Also a dangerous_selftests for tests that will crash or hang the test
> > appliances. I'd really like to run this every time I start up fstests,
> > albeit without the dangerous ones. :)
>
> I agree with this. When I saw *debug*, I thought it's about some kernel/fs
> debuging test, or it need kernel debug option. So *selftest* might be a better
> name, or if someone has better idea :)
>
> With group name changing, I think better to change tests/debug/ directory to
> tests/selftests/.
Will do!
> > > diff --git a/tests/debug/Makefile b/tests/debug/Makefile
> > > new file mode 100644
> > > index 00000000..84556199
> > > --- /dev/null
> > > +++ b/tests/debug/Makefile
> > > @@ -0,0 +1,24 @@
> > > +#
> > > +# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
>
> Do you need to change this ^^ ?
Ah yes, thanks!
- Leah
>
> Thanks,
> Zorro
>
> > > +#
> > > +
> > > +TOPDIR = ../..
> > > +include $(TOPDIR)/include/builddefs
> > > +include $(TOPDIR)/include/buildgrouplist
> > > +
> > > +DEBUG_DIR = debug
> > > +TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
> > > +DIRT = group.list
> > > +
> > > +default: $(DIRT)
> > > +
> > > +include $(BUILDRULES)
> > > +
> > > +install:
> > > + $(INSTALL) -m 755 -d $(TARGET_DIR)
> > > + $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
> > > + $(INSTALL) -m 644 group.list $(TARGET_DIR)
> > > + $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
> > > +
> > > +# Nothing.
> > > +install-dev install-lib:
> > > --
> > > 2.39.1.519.gcb327c4b5f-goog
> > >
> >
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] debug: add tests for debugging testing setup
2023-02-09 21:35 ` Leah Rumancik
@ 2023-02-10 1:02 ` Darrick J. Wong
0 siblings, 0 replies; 6+ messages in thread
From: Darrick J. Wong @ 2023-02-10 1:02 UTC (permalink / raw)
To: Leah Rumancik; +Cc: fstests, zlang
On Thu, Feb 09, 2023 at 01:35:08PM -0800, Leah Rumancik wrote:
> On Wed, Feb 08, 2023 at 06:47:30PM -0800, Darrick J. Wong wrote:
> > On Wed, Feb 08, 2023 at 05:31:43PM -0800, Leah Rumancik wrote:
> > > Many people have developed infrastructure around xfstests. In order to
> > > test a setup, it would be helpful to have dummy tests that have
> > > consistent test outcomes. Add a new test folder with the following
> > > debugging tests:
> >
> > Ooh, we've been needing selftests for a while, thanks!
> >
> > > debug/001 pass
> > > debug/002 fail
> > > debug/003 skip
> > > debug/004 crash
> > > debug/005 hang
> > >
> > > Signed-off-by: Leah Rumancik <leah.rumancik@gmail.com>
> > >
> > > diff --git a/doc/group-names.txt b/doc/group-names.txt
> > > index 6cc9af78..212da7e7 100644
> > > --- a/doc/group-names.txt
> > > +++ b/doc/group-names.txt
> > > @@ -39,6 +39,7 @@ dangerous_scrub fuzzers to evaluate xfs_scrub checking
> > > data data loss checkers
> > > dax direct access mode for persistent memory files
> > > db xfs_db functional tests
> > > +debug tests with fixed results, used to validate testing setup
> >
> > Bikeshed: Can we make the tag 'selftests' since debugging could refer to
> > just about anything in the kernel/userspace/otherstuff stacks?
>
> Ah sure, agreed selftests would be better, thanks.
>
> >
> > Also a dangerous_selftests for tests that will crash or hang the test
> > appliances. I'd really like to run this every time I start up fstests,
> > albeit without the dangerous ones. :)
>
> How exactly would you plan to run it? To run just the non-dangerous tests,
> you could do -g selftests/selftests -x selftests/dangerous_selftests. You
> can't simply use -g selftests -x selftests_dangerous as right now
> selftests is not being added to SRC_GROUPS.
That's exactly what I was thinking.
> It would be nice to make it
> behave more like a normal group though... If it doesn't matter that
> these would run with -g all, I could just add it to SRC_GROUPS. If it'd be
> best to skip these during running -g all, I could also update get_all_tests
> to skip over the selftests folder. Do you think these should run with
> -g all?
Yes.
>
> >
> > Or <cough> fsteststests.
>
> Ha :)
>
> >
> > (No, really not that.)
> >
> > > dedupe FIEDEDUPERANGE ioctl
> > > defrag filesystem defragmenters
> > > dir directory test functions
> > > diff --git a/tests/debug/001 b/tests/debug/001
> > > new file mode 100755
> > > index 00000000..9567449c
> > > --- /dev/null
> > > +++ b/tests/debug/001
> > > @@ -0,0 +1,14 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +#
> > > +# FS QA Test No. 001
> > > +#
> > > +# A test that should always pass
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest debug
> > > +
> > > +echo "Silence is golden"
> > > +status=0
> > > +exit
> > > +
> > > diff --git a/tests/debug/001.out b/tests/debug/001.out
> > > new file mode 100644
> > > index 00000000..88678b8e
> > > --- /dev/null
> > > +++ b/tests/debug/001.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 001
> > > +Silence is golden
> > > diff --git a/tests/debug/002 b/tests/debug/002
> > > new file mode 100755
> > > index 00000000..12b45a94
> > > --- /dev/null
> > > +++ b/tests/debug/002
> > > @@ -0,0 +1,14 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +#
> > > +# FS QA Test No. 002
> > > +#
> > > +# A test that should always fail
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest debug
> > > +
> > > +echo "I am intentionally broken"
> >
> > Would you mind slipping in another test to simulate tests that cannot
> > satisfy preconditions and hence _fail?
>
> What exactly do you mean? Like a test that just calls _fail to
> simulate something like _scratch_mkfs || _fail "mkfs failed"?
Yes.
--D
> >
> > > +status=0
> > > +exit
> > > +
> > > diff --git a/tests/debug/002.out b/tests/debug/002.out
> > > new file mode 100644
> > > index 00000000..61705c7c
> > > --- /dev/null
> > > +++ b/tests/debug/002.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 002
> > > +Silence is golden
> > > diff --git a/tests/debug/003 b/tests/debug/003
> > > new file mode 100755
> > > index 00000000..813e6ff6
> > > --- /dev/null
> > > +++ b/tests/debug/003
> > > @@ -0,0 +1,16 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +#
> > > +# FS QA Test No. 003
> > > +#
> > > +# A test that should always be skipped
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest debug
> > > +
> > > +_notrun "Always skip me"
> > > +
> > > +echo "I should be skipped"
> > > +status=0
> > > +exit
> > > +
> > > diff --git a/tests/debug/003.out b/tests/debug/003.out
> > > new file mode 100644
> > > index 00000000..6895fc80
> > > --- /dev/null
> > > +++ b/tests/debug/003.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 003
> > > +Silence is golden
> > > diff --git a/tests/debug/004 b/tests/debug/004
> > > new file mode 100755
> > > index 00000000..2e5f1e43
> > > --- /dev/null
> > > +++ b/tests/debug/004
> > > @@ -0,0 +1,17 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +#
> > > +# FS QA Test No. 004
> > > +#
> > > +# A test that crashes
> > > +#
> > > +. ./common/preamble
> > > +_begin_fstest debug dangerous
> >
> > _begin_fstest dangerous_selftest?
> >
> Sure, will update.
>
> > > +
> > > +echo 1 > /proc/sys/kernel/sysrq
> > > +echo c > /proc/sysrq-trigger
> > > +
> > > +echo "I should have crashed by now"
> > > +status=0
> > > +exit
> > > +
> > > diff --git a/tests/debug/004.out b/tests/debug/004.out
> > > new file mode 100644
> > > index 00000000..af8614ae
> > > --- /dev/null
> > > +++ b/tests/debug/004.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 004
> > > +Silence is golden
> > > diff --git a/tests/debug/005 b/tests/debug/005
> > > new file mode 100755
> > > index 00000000..1a213926
> > > --- /dev/null
> > > +++ b/tests/debug/005
> > > @@ -0,0 +1,20 @@
> > > +#! /bin/bash
> > > +# SPDX-License-Identifier: GPL-2.0
> > > +#
> > > +# FS QA Test No. 005
> > > +#
> > > +# A test that hangs
> > > +#
> > > +
> > > +. ./common/preamble
> > > +_begin_fstest debug dangerous
> >
> > Here too.
>
> Ack
> >
> > > +
> > > +while :
> > > +do
> > > + sleep 1d
> > > +done
> > > +
> > > +echo "I should still be sleeping"
> > > +status=0
> > > +exit
> > > +
> > > diff --git a/tests/debug/005.out b/tests/debug/005.out
> > > new file mode 100644
> > > index 00000000..a5027f12
> > > --- /dev/null
> > > +++ b/tests/debug/005.out
> > > @@ -0,0 +1,2 @@
> > > +QA output created by 005
> > > +Silence is golden
> > > diff --git a/tests/debug/Makefile b/tests/debug/Makefile
> > > new file mode 100644
> > > index 00000000..84556199
> > > --- /dev/null
> > > +++ b/tests/debug/Makefile
> > > @@ -0,0 +1,24 @@
> > > +#
> > > +# Copyright (c) 2003-2005 Silicon Graphics, Inc. All Rights Reserved.
> > > +#
> > > +
> > > +TOPDIR = ../..
> > > +include $(TOPDIR)/include/builddefs
> > > +include $(TOPDIR)/include/buildgrouplist
> > > +
> > > +DEBUG_DIR = debug
> > > +TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(DEBUG_DIR)
> > > +DIRT = group.list
> > > +
> > > +default: $(DIRT)
> > > +
> > > +include $(BUILDRULES)
> > > +
> > > +install:
> > > + $(INSTALL) -m 755 -d $(TARGET_DIR)
> > > + $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR)
> > > + $(INSTALL) -m 644 group.list $(TARGET_DIR)
> > > + $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR)
> > > +
> > > +# Nothing.
> > > +install-dev install-lib:
> > > --
> > > 2.39.1.519.gcb327c4b5f-goog
> > >
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-02-10 1:03 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-02-09 1:31 [PATCH] debug: add tests for debugging testing setup Leah Rumancik
2023-02-09 2:47 ` Darrick J. Wong
2023-02-09 5:53 ` Zorro Lang
2023-02-09 21:47 ` Leah Rumancik
2023-02-09 21:35 ` Leah Rumancik
2023-02-10 1:02 ` Darrick J. Wong
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.