* [PATCHSET v3 0/1] fstests: exercise code refactored in 5.14
@ 2021-09-15 23:42 Darrick J. Wong
2021-09-15 23:42 ` [PATCH 1/1] generic: fsstress with cpu offlining Darrick J. Wong
0 siblings, 1 reply; 4+ messages in thread
From: Darrick J. Wong @ 2021-09-15 23:42 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
Hi all,
Add new tests to exercise code that got refactored in 5.14. The
nested shutdown test simulates the process of recovering after a VM host
filesystem goes down and the guests have to recover.
v2: fix some bugs pointed out by the maintainer, add cpu offlining stress test
v3: run against the test fs, and limit the io thread count
If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.
This is an extraordinary way to destroy everything. Enjoy!
Comments and questions are, as always, welcome.
--D
fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=new-tests-for-5.14
---
tests/generic/726 | 74 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/726.out | 2 +
2 files changed, 76 insertions(+)
create mode 100755 tests/generic/726
create mode 100644 tests/generic/726.out
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH 1/1] generic: fsstress with cpu offlining
2021-09-15 23:42 [PATCHSET v3 0/1] fstests: exercise code refactored in 5.14 Darrick J. Wong
@ 2021-09-15 23:42 ` Darrick J. Wong
2021-09-17 18:34 ` riteshh
0 siblings, 1 reply; 4+ messages in thread
From: Darrick J. Wong @ 2021-09-15 23:42 UTC (permalink / raw)
To: djwong, guaneryu; +Cc: linux-xfs, fstests, guan
From: Darrick J. Wong <djwong@kernel.org>
Exercise filesystem operations when we're taking CPUs online and offline
throughout the test.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
tests/generic/726 | 74 +++++++++++++++++++++++++++++++++++++++++++++++++
tests/generic/726.out | 2 +
2 files changed, 76 insertions(+)
create mode 100755 tests/generic/726
create mode 100644 tests/generic/726.out
diff --git a/tests/generic/726 b/tests/generic/726
new file mode 100755
index 00000000..1a3f2fad
--- /dev/null
+++ b/tests/generic/726
@@ -0,0 +1,74 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2021 Oracle, Inc. All Rights Reserved.
+#
+# FS QA Test No. 726
+#
+# Run an all-writes fsstress run with multiple threads while exercising CPU
+# hotplugging to shake out bugs in the write path.
+#
+. ./common/preamble
+_begin_fstest auto rw stress
+
+# Override the default cleanup function.
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+ $KILLALL_PROG -9 fsstress > /dev/null 2>&1
+ wait # for exercise_cpu_hotplug subprocess
+ for i in "$sysfs_cpu_dir/"cpu*/online; do
+ echo 1 > "$i" 2>/dev/null
+ done
+ test -n "$stress_dir" && rm -r -f "$stress_dir"
+}
+
+exercise_cpu_hotplug()
+{
+ while [ -e $sentinel_file ]; do
+ local idx=$(( RANDOM % nr_hotplug_cpus ))
+ local cpu="${hotplug_cpus[idx]}"
+ local action=$(( RANDOM % 2 ))
+
+ echo "$action" > "$sysfs_cpu_dir/cpu$cpu/online" 2>/dev/null
+ sleep 0.5
+ done
+}
+
+_supported_fs generic
+_require_test
+_require_command "$KILLALL_PROG" "killall"
+
+sysfs_cpu_dir="/sys/devices/system/cpu"
+
+# Figure out which CPU(s) support hotplug.
+nrcpus=$(getconf _NPROCESSORS_CONF)
+hotplug_cpus=()
+for ((i = 0; i < nrcpus; i++ )); do
+ test -e "$sysfs_cpu_dir/cpu$i/online" && hotplug_cpus+=("$i")
+done
+nr_hotplug_cpus="${#hotplug_cpus[@]}"
+test "$nr_hotplug_cpus" -gt 0 || _notrun "CPU hotplugging not supported"
+
+stress_dir="$TEST_DIR/$seq"
+rm -r -f "$stress_dir"
+mkdir -p "$stress_dir"
+
+echo "Silence is golden."
+
+sentinel_file=$tmp.hotplug
+touch $sentinel_file
+exercise_cpu_hotplug &
+
+# Cap the number of fsstress threads at one per hotpluggable CPU if we exceed
+# 1024 IO threads, per maintainer request.
+nr_cpus=$((LOAD_FACTOR * nr_hotplug_cpus))
+test "$nr_cpus" -gt 1024 && nr_cpus="$nr_hotplug_cpus"
+
+nr_ops=$((25000 * TIME_FACTOR))
+$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $stress_dir -n $nr_ops -p $nr_cpus >> $seqres.full
+rm -f $sentinel_file
+
+# success, all done
+status=0
+exit
diff --git a/tests/generic/726.out b/tests/generic/726.out
new file mode 100644
index 00000000..6839f8ce
--- /dev/null
+++ b/tests/generic/726.out
@@ -0,0 +1,2 @@
+QA output created by 726
+Silence is golden.
^ permalink raw reply related [flat|nested] 4+ messages in thread* Re: [PATCH 1/1] generic: fsstress with cpu offlining
2021-09-15 23:42 ` [PATCH 1/1] generic: fsstress with cpu offlining Darrick J. Wong
@ 2021-09-17 18:34 ` riteshh
2021-09-17 23:48 ` Darrick J. Wong
0 siblings, 1 reply; 4+ messages in thread
From: riteshh @ 2021-09-17 18:34 UTC (permalink / raw)
To: Darrick J. Wong; +Cc: guaneryu, linux-xfs, fstests, guan
On 21/09/15 04:42PM, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
>
> Exercise filesystem operations when we're taking CPUs online and offline
> throughout the test.
Nice test coverage. Btw, I may have missed older versions, but could you point
to the link which points to the bugs which this test uncovered?
I guess it will be good to add tha in the comment section of test description
too.
This also made me think whether doing memory online/offline while running
fsstress, makes any sense?
>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
> tests/generic/726 | 74 +++++++++++++++++++++++++++++++++++++++++++++++++
> tests/generic/726.out | 2 +
> 2 files changed, 76 insertions(+)
> create mode 100755 tests/generic/726
> create mode 100644 tests/generic/726.out
>
>
> diff --git a/tests/generic/726 b/tests/generic/726
> new file mode 100755
> index 00000000..1a3f2fad
> --- /dev/null
> +++ b/tests/generic/726
> @@ -0,0 +1,74 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2021 Oracle, Inc. All Rights Reserved.
> +#
> +# FS QA Test No. 726
> +#
> +# Run an all-writes fsstress run with multiple threads while exercising CPU
> +# hotplugging to shake out bugs in the write path.
> +#
> +. ./common/preamble
> +_begin_fstest auto rw stress
Does it qualify for auto? This definitely is taking a longer time compared to
other auto tests on my qemu setup.
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> + $KILLALL_PROG -9 fsstress > /dev/null 2>&1
> + wait # for exercise_cpu_hotplug subprocess
> + for i in "$sysfs_cpu_dir/"cpu*/online; do
> + echo 1 > "$i" 2>/dev/null
> + done
> + test -n "$stress_dir" && rm -r -f "$stress_dir"
> +}
> +
> +exercise_cpu_hotplug()
> +{
> + while [ -e $sentinel_file ]; do
> + local idx=$(( RANDOM % nr_hotplug_cpus ))
> + local cpu="${hotplug_cpus[idx]}"
> + local action=$(( RANDOM % 2 ))
> +
> + echo "$action" > "$sysfs_cpu_dir/cpu$cpu/online" 2>/dev/null
> + sleep 0.5
> + done
> +}
> +
> +_supported_fs generic
> +_require_test
> +_require_command "$KILLALL_PROG" "killall"
> +
> +sysfs_cpu_dir="/sys/devices/system/cpu"
> +
> +# Figure out which CPU(s) support hotplug.
> +nrcpus=$(getconf _NPROCESSORS_CONF)
> +hotplug_cpus=()
> +for ((i = 0; i < nrcpus; i++ )); do
> + test -e "$sysfs_cpu_dir/cpu$i/online" && hotplug_cpus+=("$i")
> +done
> +nr_hotplug_cpus="${#hotplug_cpus[@]}"
> +test "$nr_hotplug_cpus" -gt 0 || _notrun "CPU hotplugging not supported"
> +
> +stress_dir="$TEST_DIR/$seq"
> +rm -r -f "$stress_dir"
> +mkdir -p "$stress_dir"
> +
> +echo "Silence is golden."
> +
> +sentinel_file=$tmp.hotplug
> +touch $sentinel_file
> +exercise_cpu_hotplug &
> +
> +# Cap the number of fsstress threads at one per hotpluggable CPU if we exceed
> +# 1024 IO threads, per maintainer request.
> +nr_cpus=$((LOAD_FACTOR * nr_hotplug_cpus))
> +test "$nr_cpus" -gt 1024 && nr_cpus="$nr_hotplug_cpus"
> +
> +nr_ops=$((25000 * TIME_FACTOR))
> +$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $stress_dir -n $nr_ops -p $nr_cpus >> $seqres.full
> +rm -f $sentinel_file
> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/generic/726.out b/tests/generic/726.out
> new file mode 100644
> index 00000000..6839f8ce
> --- /dev/null
> +++ b/tests/generic/726.out
> @@ -0,0 +1,2 @@
> +QA output created by 726
> +Silence is golden.
>
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: [PATCH 1/1] generic: fsstress with cpu offlining
2021-09-17 18:34 ` riteshh
@ 2021-09-17 23:48 ` Darrick J. Wong
0 siblings, 0 replies; 4+ messages in thread
From: Darrick J. Wong @ 2021-09-17 23:48 UTC (permalink / raw)
To: riteshh; +Cc: guaneryu, linux-xfs, fstests, guan
On Sat, Sep 18, 2021 at 12:04:49AM +0530, riteshh wrote:
> On 21/09/15 04:42PM, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > Exercise filesystem operations when we're taking CPUs online and offline
> > throughout the test.
>
> Nice test coverage. Btw, I may have missed older versions, but could you point
> to the link which points to the bugs which this test uncovered?
> I guess it will be good to add tha in the comment section of test description
> too.
I didn't uncover any bugs with 5.14, fortunately. Dave Chinner was
messing around with per-CPU lists in XFS, so I figured I had better
write something to exercise the cpu-dead handlers to try to make sure
there weren't any obvious bugs in the code, and this is the result.
> This also made me think whether doing memory online/offline while
> running
> fsstress, makes any sense?
Heh, perhaps. I hadn't gotten /that/ far... :)
--D
>
> >
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> > tests/generic/726 | 74 +++++++++++++++++++++++++++++++++++++++++++++++++
> > tests/generic/726.out | 2 +
> > 2 files changed, 76 insertions(+)
> > create mode 100755 tests/generic/726
> > create mode 100644 tests/generic/726.out
> >
> >
> > diff --git a/tests/generic/726 b/tests/generic/726
> > new file mode 100755
> > index 00000000..1a3f2fad
> > --- /dev/null
> > +++ b/tests/generic/726
> > @@ -0,0 +1,74 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2021 Oracle, Inc. All Rights Reserved.
> > +#
> > +# FS QA Test No. 726
> > +#
> > +# Run an all-writes fsstress run with multiple threads while exercising CPU
> > +# hotplugging to shake out bugs in the write path.
> > +#
> > +. ./common/preamble
> > +_begin_fstest auto rw stress
>
> Does it qualify for auto? This definitely is taking a longer time compared to
> other auto tests on my qemu setup.
>
> > +
> > +# Override the default cleanup function.
> > +_cleanup()
> > +{
> > + cd /
> > + rm -f $tmp.*
> > + $KILLALL_PROG -9 fsstress > /dev/null 2>&1
> > + wait # for exercise_cpu_hotplug subprocess
> > + for i in "$sysfs_cpu_dir/"cpu*/online; do
> > + echo 1 > "$i" 2>/dev/null
> > + done
> > + test -n "$stress_dir" && rm -r -f "$stress_dir"
> > +}
> > +
> > +exercise_cpu_hotplug()
> > +{
> > + while [ -e $sentinel_file ]; do
> > + local idx=$(( RANDOM % nr_hotplug_cpus ))
> > + local cpu="${hotplug_cpus[idx]}"
> > + local action=$(( RANDOM % 2 ))
> > +
> > + echo "$action" > "$sysfs_cpu_dir/cpu$cpu/online" 2>/dev/null
> > + sleep 0.5
> > + done
> > +}
> > +
> > +_supported_fs generic
> > +_require_test
> > +_require_command "$KILLALL_PROG" "killall"
> > +
> > +sysfs_cpu_dir="/sys/devices/system/cpu"
> > +
> > +# Figure out which CPU(s) support hotplug.
> > +nrcpus=$(getconf _NPROCESSORS_CONF)
> > +hotplug_cpus=()
> > +for ((i = 0; i < nrcpus; i++ )); do
> > + test -e "$sysfs_cpu_dir/cpu$i/online" && hotplug_cpus+=("$i")
> > +done
> > +nr_hotplug_cpus="${#hotplug_cpus[@]}"
> > +test "$nr_hotplug_cpus" -gt 0 || _notrun "CPU hotplugging not supported"
> > +
> > +stress_dir="$TEST_DIR/$seq"
> > +rm -r -f "$stress_dir"
> > +mkdir -p "$stress_dir"
> > +
> > +echo "Silence is golden."
> > +
> > +sentinel_file=$tmp.hotplug
> > +touch $sentinel_file
> > +exercise_cpu_hotplug &
> > +
> > +# Cap the number of fsstress threads at one per hotpluggable CPU if we exceed
> > +# 1024 IO threads, per maintainer request.
> > +nr_cpus=$((LOAD_FACTOR * nr_hotplug_cpus))
> > +test "$nr_cpus" -gt 1024 && nr_cpus="$nr_hotplug_cpus"
> > +
> > +nr_ops=$((25000 * TIME_FACTOR))
> > +$FSSTRESS_PROG $FSSTRESS_AVOID -w -d $stress_dir -n $nr_ops -p $nr_cpus >> $seqres.full
> > +rm -f $sentinel_file
> > +
> > +# success, all done
> > +status=0
> > +exit
> > diff --git a/tests/generic/726.out b/tests/generic/726.out
> > new file mode 100644
> > index 00000000..6839f8ce
> > --- /dev/null
> > +++ b/tests/generic/726.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 726
> > +Silence is golden.
> >
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-09-17 23:48 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-15 23:42 [PATCHSET v3 0/1] fstests: exercise code refactored in 5.14 Darrick J. Wong
2021-09-15 23:42 ` [PATCH 1/1] generic: fsstress with cpu offlining Darrick J. Wong
2021-09-17 18:34 ` riteshh
2021-09-17 23:48 ` Darrick J. Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).