* [PATCH tip/core/rcu 01/28] torturescript: Don't rerun failed rcutorture builds
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 02/28] torture: Allow 1G of memory for torture.sh kvfree testing paulmck
` (30 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
If the build fails when running multiple instances of a given rcutorture
scenario, for example, using the kvm.sh --configs "8*RUDE01" argument,
the build will be rerun an additional seven times. This is in some sense
correct, but it can waste significant time. This commit therefore checks
for a prior failed build and simply copies over that build's output.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index 536d103..9d8a82c 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -73,7 +73,7 @@ config_override_param "--kconfig argument" KcList "$TORTURE_KCONFIG_ARG"
cp $T/KcList $resdir/ConfigFragment
base_resdir=`echo $resdir | sed -e 's/\.[0-9]\+$//'`
-if test "$base_resdir" != "$resdir" -a -f $base_resdir/bzImage -a -f $base_resdir/vmlinux
+if test "$base_resdir" != "$resdir" && test -f $base_resdir/bzImage && test -f $base_resdir/vmlinux
then
# Rerunning previous test, so use that test's kernel.
QEMU="`identify_qemu $base_resdir/vmlinux`"
@@ -83,6 +83,17 @@ then
ln -s $base_resdir/.config $resdir # for kvm-recheck.sh
# Arch-independent indicator
touch $resdir/builtkernel
+elif test "$base_resdir" != "$resdir"
+then
+ # Rerunning previous test for which build failed
+ ln -s $base_resdir/Make*.out $resdir # for kvm-recheck.sh
+ ln -s $base_resdir/.config $resdir # for kvm-recheck.sh
+ echo Initial build failed, not running KVM, see $resdir.
+ if test -f $builddir.wait
+ then
+ mv $builddir.wait $builddir.ready
+ fi
+ exit 1
elif kvm-build.sh $T/KcList $resdir
then
# Had to build a kernel for this test.
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 02/28] torture: Allow 1G of memory for torture.sh kvfree testing
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
2021-03-04 0:37 ` [PATCH tip/core/rcu 01/28] torturescript: Don't rerun failed rcutorture builds paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 03/28] torture: Provide bare-metal modprobe-based advice paulmck
` (29 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Yes, I do recall a time when 512MB of memory was a lot of mass storage,
much less main memory, but the rcuscale kvfree_rcu() testing invoked by
torture.sh can sometimes exceed it on large systems, resulting in OOM.
This commit therefore causes torture.sh to pase the "--memory 1G"
argument to kvm.sh to reserve a full gigabyte for this purpose.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/torture.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/torture.sh b/tools/testing/selftests/rcutorture/bin/torture.sh
index ad7525b..56e2e1a 100755
--- a/tools/testing/selftests/rcutorture/bin/torture.sh
+++ b/tools/testing/selftests/rcutorture/bin/torture.sh
@@ -374,7 +374,7 @@ done
if test "$do_kvfree" = "yes"
then
torture_bootargs="rcuscale.kfree_rcu_test=1 rcuscale.kfree_nthreads=16 rcuscale.holdoff=20 rcuscale.kfree_loops=10000 torture.disable_onoff_at_boot"
- torture_set "rcuscale-kvfree" tools/testing/selftests/rcutorture/bin/kvm.sh --torture rcuscale --allcpus --duration 10 --kconfig "CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --trust-make
+ torture_set "rcuscale-kvfree" tools/testing/selftests/rcutorture/bin/kvm.sh --torture rcuscale --allcpus --duration 10 --kconfig "CONFIG_NR_CPUS=$HALF_ALLOTED_CPUS" --memory 1G --trust-make
fi
echo " --- " $scriptname $args
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 03/28] torture: Provide bare-metal modprobe-based advice
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
2021-03-04 0:37 ` [PATCH tip/core/rcu 01/28] torturescript: Don't rerun failed rcutorture builds paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 02/28] torture: Allow 1G of memory for torture.sh kvfree testing paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 04/28] torture: Improve readability of the testid.txt file paulmck
` (28 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
In some environments, the torture-testing use of virtualization is
inconvenient. In such cases, the modprobe and rmmod commands may be used
to do torture testing, but significant setup is required to build, boot,
and modprobe a kernel so as to match a given torture-test scenario.
This commit therefore creates a "bare-metal" file in each results
directory containing steps to run the corresponding scenario using the
modprobe command on bare metal. For example, the contents of this file
after using kvm.sh to build an rcutorture TREE01 kernel, perhaps with
the --buildonly argument, is as follows:
To run this scenario on bare metal:
1. Set your bare-metal build tree to the state shown in this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/testid.txt
2. Update your bare-metal build tree's .config based on this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/ConfigFragment
3. Make the bare-metal kernel's build system aware of your .config updates:
$ yes "" | make oldconfig
4. Build your bare-metal kernel.
5. Boot your bare-metal kernel with the following parameters:
maxcpus=8 nr_cpus=43 rcutree.gp_preinit_delay=3 rcutree.gp_init_delay=3 rcutree.gp_cleanup_delay=3 rcu_nocbs=0-1,3-7
6. Start the test with the following command:
$ modprobe rcutorture nocbs_nthreads=8 nocbs_toggle=1000 fwd_progress=0 onoff_interval=1000 onoff_holdoff=30 n_barrier_cbs=4 stat_interval=15 shutdown_secs=120 test_no_idle_hz=1 verbose=1
7. After some time, end the test with the following command:
$ rmmod rcutorture
8. Copy your bare-metal kernel's .config file, overwriting this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/.config
9. Copy the console output from just before the modprobe to just after
the rmmod into this file:
/home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19/TREE01/console.log
10. Check for runtime errors using the following command:
$ tools/testing/selftests/rcutorture/bin/kvm-recheck.sh /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2021.02.04-17.10.19
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../selftests/rcutorture/bin/kvm-test-1-run.sh | 44 +++++++++++++++++++---
tools/testing/selftests/rcutorture/bin/kvm.sh | 4 ++
2 files changed, 42 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index 9d8a82c..03c0410 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -7,15 +7,15 @@
# Execute this in the source tree. Do not run it as a background task
# because qemu does not seem to like that much.
#
-# Usage: kvm-test-1-run.sh config builddir resdir seconds qemu-args boot_args
+# Usage: kvm-test-1-run.sh config builddir resdir seconds qemu-args boot_args_in
#
# qemu-args defaults to "-enable-kvm -nographic", along with arguments
# specifying the number of CPUs and other options
# generated from the underlying CPU architecture.
-# boot_args defaults to value returned by the per_version_boot_params
+# boot_args_in defaults to value returned by the per_version_boot_params
# shell function.
#
-# Anything you specify for either qemu-args or boot_args is appended to
+# Anything you specify for either qemu-args or boot_args_in is appended to
# the default values. The "-smp" value is deduced from the contents of
# the config fragment.
#
@@ -134,7 +134,7 @@ do
done
seconds=$4
qemu_args=$5
-boot_args=$6
+boot_args_in=$6
if test -z "$TORTURE_BUILDONLY"
then
@@ -144,7 +144,7 @@ fi
# Generate -smp qemu argument.
qemu_args="-enable-kvm -nographic $qemu_args"
cpu_count=`configNR_CPUS.sh $resdir/ConfigFragment`
-cpu_count=`configfrag_boot_cpus "$boot_args" "$config_template" "$cpu_count"`
+cpu_count=`configfrag_boot_cpus "$boot_args_in" "$config_template" "$cpu_count"`
if test "$cpu_count" -gt "$TORTURE_ALLOTED_CPUS"
then
echo CPU count limited from $cpu_count to $TORTURE_ALLOTED_CPUS | tee -a $resdir/Warnings
@@ -160,13 +160,45 @@ qemu_args="$qemu_args `identify_qemu_args "$QEMU" "$resdir/console.log"`"
qemu_append="`identify_qemu_append "$QEMU"`"
# Pull in Kconfig-fragment boot parameters
-boot_args="`configfrag_boot_params "$boot_args" "$config_template"`"
+boot_args="`configfrag_boot_params "$boot_args_in" "$config_template"`"
# Generate kernel-version-specific boot parameters
boot_args="`per_version_boot_params "$boot_args" $resdir/.config $seconds`"
if test -n "$TORTURE_BOOT_GDB_ARG"
then
boot_args="$boot_args $TORTURE_BOOT_GDB_ARG"
fi
+
+# Give bare-metal advice
+modprobe_args="`echo $boot_args | tr -s ' ' '\012' | grep "^$TORTURE_MOD\." | sed -e "s/$TORTURE_MOD\.//g"`"
+kboot_args="`echo $boot_args | tr -s ' ' '\012' | grep -v "^$TORTURE_MOD\."`"
+testid_txt="`dirname $resdir`/testid.txt"
+touch $resdir/bare-metal
+echo To run this scenario on bare metal: >> $resdir/bare-metal
+echo >> $resdir/bare-metal
+echo " 1." Set your bare-metal build tree to the state shown in this file: >> $resdir/bare-metal
+echo " " $testid_txt >> $resdir/bare-metal
+echo " 2." Update your bare-metal build tree"'"s .config based on this file: >> $resdir/bare-metal
+echo " " $resdir/ConfigFragment >> $resdir/bare-metal
+echo " 3." Make the bare-metal kernel"'"s build system aware of your .config updates: >> $resdir/bare-metal
+echo " " $ 'yes "" | make oldconfig' >> $resdir/bare-metal
+echo " 4." Build your bare-metal kernel. >> $resdir/bare-metal
+echo " 5." Boot your bare-metal kernel with the following parameters: >> $resdir/bare-metal
+echo " " $kboot_args >> $resdir/bare-metal
+echo " 6." Start the test with the following command: >> $resdir/bare-metal
+echo " " $ modprobe $TORTURE_MOD $modprobe_args >> $resdir/bare-metal
+echo " 7." After some time, end the test with the following command: >> $resdir/bare-metal
+echo " " $ rmmod $TORTURE_MOD >> $resdir/bare-metal
+echo " 8." Copy your bare-metal kernel"'"s .config file, overwriting this file: >> $resdir/bare-metal
+echo " " $resdir/.config >> $resdir/bare-metal
+echo " 9." Copy the console output from just before the modprobe to just after >> $resdir/bare-metal
+echo " " the rmmod into this file: >> $resdir/bare-metal
+echo " " $resdir/console.log >> $resdir/bare-metal
+echo "10." Check for runtime errors using the following command: >> $resdir/bare-metal
+echo " " $ tools/testing/selftests/rcutorture/bin/kvm-recheck.sh `dirname $resdir` >> $resdir/bare-metal
+echo >> $resdir/bare-metal
+echo Some of the above steps may be skipped if you build your bare-metal >> $resdir/bare-metal
+echo kernel here: `head -n 1 $testid_txt | sed -e 's/^Build directory: //'` >> $resdir/bare-metal
+
echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_append $boot_args\" $TORTURE_QEMU_GDB_ARG > $resdir/qemu-cmd
echo "# TORTURE_SHUTDOWN_GRACE=$TORTURE_SHUTDOWN_GRACE" >> $resdir/qemu-cmd
echo "# seconds=$seconds" >> $resdir/qemu-cmd
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 8d3c99b..35a2132 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -29,6 +29,7 @@ PATH=${KVM}/bin:$PATH; export PATH
TORTURE_ALLOTED_CPUS="`identify_qemu_vcpus`"
TORTURE_DEFCONFIG=defconfig
TORTURE_BOOT_IMAGE=""
+TORTURE_BUILDONLY=
TORTURE_INITRD="$KVM/initrd"; export TORTURE_INITRD
TORTURE_KCONFIG_ARG=""
TORTURE_KCONFIG_GDB_ARG=""
@@ -40,6 +41,7 @@ TORTURE_KMAKE_ARG=""
TORTURE_QEMU_MEM=512
TORTURE_SHUTDOWN_GRACE=180
TORTURE_SUITE=rcu
+TORTURE_MOD=rcutorture
TORTURE_TRUST_MAKE=""
resdir=""
configs=""
@@ -215,6 +217,7 @@ do
--torture)
checkarg --torture "(suite name)" "$#" "$2" '^\(lock\|rcu\|rcuscale\|refscale\|scf\)$' '^--'
TORTURE_SUITE=$2
+ TORTURE_MOD="`echo $TORTURE_SUITE | sed -e 's/^\(lock\|rcu\|scf\)$/\1torture/'`"
shift
if test "$TORTURE_SUITE" = rcuscale || test "$TORTURE_SUITE" = refscale
then
@@ -381,6 +384,7 @@ TORTURE_QEMU_GDB_ARG="$TORTURE_QEMU_GDB_ARG"; export TORTURE_QEMU_GDB_ARG
TORTURE_KCONFIG_KASAN_ARG="$TORTURE_KCONFIG_KASAN_ARG"; export TORTURE_KCONFIG_KASAN_ARG
TORTURE_KCONFIG_KCSAN_ARG="$TORTURE_KCONFIG_KCSAN_ARG"; export TORTURE_KCONFIG_KCSAN_ARG
TORTURE_KMAKE_ARG="$TORTURE_KMAKE_ARG"; export TORTURE_KMAKE_ARG
+TORTURE_MOD="$TORTURE_MOD"; export TORTURE_MOD
TORTURE_QEMU_CMD="$TORTURE_QEMU_CMD"; export TORTURE_QEMU_CMD
TORTURE_QEMU_INTERACTIVE="$TORTURE_QEMU_INTERACTIVE"; export TORTURE_QEMU_INTERACTIVE
TORTURE_QEMU_MAC="$TORTURE_QEMU_MAC"; export TORTURE_QEMU_MAC
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 04/28] torture: Improve readability of the testid.txt file
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (2 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 03/28] torture: Provide bare-metal modprobe-based advice paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 05/28] rcuscale: Disable verbose torture-test output paulmck
` (27 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
The testid.txt file was intended for occasional in extremis use, but
now that the new "bare-metal" file references it, it might see more use.
This commit therefore labels sections of output and adds spacing to make
it easier to see what needs to be done to make a bare-metal build tree
match an rcutorture build tree.
Of course, you can avoid this whole issue by building your bare-metal
kernel in the same directory in which you ran rcutorture, but that might
not always be an option.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm.sh | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 35a2132..1de198d 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -404,11 +404,16 @@ echo $scriptname $args
touch $resdir/$ds/log
echo $scriptname $args >> $resdir/$ds/log
echo ${TORTURE_SUITE} > $resdir/$ds/TORTURE_SUITE
-pwd > $resdir/$ds/testid.txt
+echo Build directory: `pwd` > $resdir/$ds/testid.txt
if test -d .git
then
+ echo Current commit: `git rev-parse HEAD` >> $resdir/$ds/testid.txt
+ echo >> $resdir/$ds/testid.txt
+ echo ' ---' Output of "'"git status"'": >> $resdir/$ds/testid.txt
git status >> $resdir/$ds/testid.txt
- git rev-parse HEAD >> $resdir/$ds/testid.txt
+ echo >> $resdir/$ds/testid.txt
+ echo >> $resdir/$ds/testid.txt
+ echo ' ---' Output of "'"git diff HEAD"'": >> $resdir/$ds/testid.txt
git diff HEAD >> $resdir/$ds/testid.txt
fi
___EOF___
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 05/28] rcuscale: Disable verbose torture-test output
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (3 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 04/28] torture: Improve readability of the testid.txt file paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 06/28] refscale: " paulmck
` (26 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Given large numbers of threads, the quantity of torture-test output is
sufficient to sometimes result in RCU CPU stall warnings. The probability
of these stall warnings was greatly reduced by batching the output,
but the warnings were not eliminated. However, the actual test only
depends on console output that is printed even when rcuscale.verbose=0.
This commit therefore causes this test to run with rcuscale.verbose=0.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/configs/rcuscale/ver_functions.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/configs/rcuscale/ver_functions.sh b/tools/testing/selftests/rcutorture/configs/rcuscale/ver_functions.sh
index 0333e9b..ffbe151 100644
--- a/tools/testing/selftests/rcutorture/configs/rcuscale/ver_functions.sh
+++ b/tools/testing/selftests/rcutorture/configs/rcuscale/ver_functions.sh
@@ -12,5 +12,5 @@
# Adds per-version torture-module parameters to kernels supporting them.
per_version_boot_params () {
echo $1 rcuscale.shutdown=1 \
- rcuscale.verbose=1
+ rcuscale.verbose=0
}
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13
@ 2021-03-04 0:37 Paul E. McKenney
2021-03-04 0:37 ` [PATCH tip/core/rcu 01/28] torturescript: Don't rerun failed rcutorture builds paulmck
` (31 more replies)
0 siblings, 32 replies; 33+ messages in thread
From: Paul E. McKenney @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel
Hello!
This series contains updates to the torture-test scripting, perhaps most
notably the addition of kvm-again.sh, which allows rerunning an earlier
test without having to rebuild all the kernels.
1. Don't rerun failed rcutorture builds.
2. Allow 1G of memory for torture.sh kvfree testing.
3. Provide bare-metal modprobe-based advice.
4. Improve readability of the testid.txt file.
5. Disable verbose torture-test output.
6. Disable verbose torture-test output.
7. Move build/run synchronization files into scenario directories.
8. Use file-based protocol to mark batch's runs complete.
9. Use "jittering" file to control jitter.sh execution.
10. Eliminate jitter_pids file.
11. Reverse jittering and duration parameters for jitter.sh.
12. Abstract jitter.sh start/stop into scripts.
13. Record TORTURE_KCONFIG_GDB_ARG in qemu-cmd.
14. Extract kvm-test-1-run-qemu.sh from kvm-test-1-run.sh.
15. Record jitter start/stop commands.
16. Record kvm-test-1-run.sh and kvm-test-1-run-qemu.sh PIDs.
17. Remove no-mpstat error message.
18. Rename SRCU-t and SRCU-u to avoid lowercase characters.
19. Make upper-case-only no-dot no-slash scenario names official.
20. De-capitalize TORTURE_SUITE.
21. Create a "batches" file for build reuse.
22. Add kvm-again.sh to rerun a previous torture-test.
23. Add --duration argument to kvm-again.sh.
24. Make kvm-transform.sh update jitter commands.
25. Make TORTURE_TRUST_MAKE available in kvm-again.sh environment.
26. Print proper vmlinux path for kvm-again.sh runs.
27. Consolidate qemu-cmd duration editing into kvm-transform.sh.
28. Fix kvm.sh --datestamp regex check.
Thanx, Paul
------------------------------------------------------------------------
bin/cpus2use.sh | 1
bin/jitter.sh | 16 +-
bin/jitterstart.sh | 37 ++++++
bin/jitterstop.sh | 23 +++
bin/kvm-again.sh | 213 +++++++++++++++++++++++++++++++++-
bin/kvm-recheck.sh | 2
bin/kvm-test-1-run-batch.sh | 67 ++++++++++
bin/kvm-test-1-run-qemu.sh | 178 ++++++++++++++++++++++++++++-
bin/kvm-test-1-run.sh | 232 +++++++++++---------------------------
bin/kvm-transform.sh | 60 ++++++++-
bin/kvm.sh | 114 +++++++++++-------
bin/torture.sh | 2
configs/rcu/CFLIST | 4
configs/rcu/SRCU-U | 1
configs/rcuscale/ver_functions.sh | 2
configs/refscale/ver_functions.sh | 2
16 files changed, 715 insertions(+), 239 deletions(-)
^ permalink raw reply [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 06/28] refscale: Disable verbose torture-test output
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (4 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 05/28] rcuscale: Disable verbose torture-test output paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 07/28] torture: Move build/run synchronization files into scenario directories paulmck
` (25 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Given large numbers of threads, the quantity of torture-test output is
sufficient to sometimes result in RCU CPU stall warnings. The probability
of these stall warnings was greatly reduced by batching the output,
but the warnings were not eliminated. However, the actual test only
depends on console output that is printed even when refscale.verbose=0.
This commit therefore causes this test to run with refscale.verbose=0.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/configs/refscale/ver_functions.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/configs/refscale/ver_functions.sh b/tools/testing/selftests/rcutorture/configs/refscale/ver_functions.sh
index 321e826..f81fa2c 100644
--- a/tools/testing/selftests/rcutorture/configs/refscale/ver_functions.sh
+++ b/tools/testing/selftests/rcutorture/configs/refscale/ver_functions.sh
@@ -12,5 +12,5 @@
# Adds per-version torture-module parameters to kernels supporting them.
per_version_boot_params () {
echo $1 refscale.shutdown=1 \
- refscale.verbose=1
+ refscale.verbose=0
}
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 07/28] torture: Move build/run synchronization files into scenario directories
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (5 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 06/28] refscale: " paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 08/28] torture: Use file-based protocol to mark batch's runs complete paulmck
` (24 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Currently the bN.ready and bN.wait files are placed in the
rcutorture directory, which really is not at all a good place
for run-specific files. This commit therefore renames these
files to build.ready and build.wait and then moves them into the
scenario directories within the "res" directory, for example, into
tools/testing/selftests/rcutorture/res/2021.02.10-15.08.23/TINY01.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../selftests/rcutorture/bin/kvm-test-1-run.sh | 25 +++++++++++-----------
tools/testing/selftests/rcutorture/bin/kvm.sh | 10 ++++-----
2 files changed, 16 insertions(+), 19 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index 03c0410..91578d3 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -7,7 +7,7 @@
# Execute this in the source tree. Do not run it as a background task
# because qemu does not seem to like that much.
#
-# Usage: kvm-test-1-run.sh config builddir resdir seconds qemu-args boot_args_in
+# Usage: kvm-test-1-run.sh config resdir seconds qemu-args boot_args_in
#
# qemu-args defaults to "-enable-kvm -nographic", along with arguments
# specifying the number of CPUs and other options
@@ -35,8 +35,7 @@ mkdir $T
config_template=${1}
config_dir=`echo $config_template | sed -e 's,/[^/]*$,,'`
title=`echo $config_template | sed -e 's/^.*\///'`
-builddir=${2}
-resdir=${3}
+resdir=${2}
if test -z "$resdir" -o ! -d "$resdir" -o ! -w "$resdir"
then
echo "kvm-test-1-run.sh :$resdir: Not a writable directory, cannot store results into it"
@@ -89,9 +88,9 @@ then
ln -s $base_resdir/Make*.out $resdir # for kvm-recheck.sh
ln -s $base_resdir/.config $resdir # for kvm-recheck.sh
echo Initial build failed, not running KVM, see $resdir.
- if test -f $builddir.wait
+ if test -f $resdir/build.wait
then
- mv $builddir.wait $builddir.ready
+ mv $resdir/build.wait $resdir/build.ready
fi
exit 1
elif kvm-build.sh $T/KcList $resdir
@@ -118,23 +117,23 @@ else
# Build failed.
cp .config $resdir || :
echo Build failed, not running KVM, see $resdir.
- if test -f $builddir.wait
+ if test -f $resdir/build.wait
then
- mv $builddir.wait $builddir.ready
+ mv $resdir/build.wait $resdir/build.ready
fi
exit 1
fi
-if test -f $builddir.wait
+if test -f $resdir/build.wait
then
- mv $builddir.wait $builddir.ready
+ mv $resdir/build.wait $resdir/build.ready
fi
-while test -f $builddir.ready
+while test -f $resdir/build.ready
do
sleep 1
done
-seconds=$4
-qemu_args=$5
-boot_args_in=$6
+seconds=$3
+qemu_args=$4
+boot_args_in=$5
if test -z "$TORTURE_BUILDONLY"
then
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 1de198d..7944510 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -444,7 +444,6 @@ function dump(first, pastlast, batchnum)
print "needqemurun="
jn=1
for (j = first; j < pastlast; j++) {
- builddir=KVM "/b" j - first + 1
cpusr[jn] = cpus[j];
if (cfrep[cf[j]] == "") {
cfr[jn] = cf[j];
@@ -453,15 +452,15 @@ function dump(first, pastlast, batchnum)
cfrep[cf[j]]++;
cfr[jn] = cf[j] "." cfrep[cf[j]];
}
+ builddir=rd cfr[jn] "/build";
if (cpusr[jn] > ncpus && ncpus != 0)
ovf = "-ovf";
else
ovf = "";
print "echo ", cfr[jn], cpusr[jn] ovf ": Starting build. `date` | tee -a " rd "log";
- print "rm -f " builddir ".*";
- print "touch " builddir ".wait";
print "mkdir " rd cfr[jn] " || :";
- print "kvm-test-1-run.sh " CONFIGDIR cf[j], builddir, rd cfr[jn], dur " \"" TORTURE_QEMU_ARG "\" \"" TORTURE_BOOTARGS "\" > " rd cfr[jn] "/kvm-test-1-run.sh.out 2>&1 &"
+ print "touch " builddir ".wait";
+ print "kvm-test-1-run.sh " CONFIGDIR cf[j], rd cfr[jn], dur " \"" TORTURE_QEMU_ARG "\" \"" TORTURE_BOOTARGS "\" > " rd cfr[jn] "/kvm-test-1-run.sh.out 2>&1 &"
print "echo ", cfr[jn], cpusr[jn] ovf ": Waiting for build to complete. `date` | tee -a " rd "log";
print "while test -f " builddir ".wait"
print "do"
@@ -471,7 +470,7 @@ function dump(first, pastlast, batchnum)
jn++;
}
for (j = 1; j < jn; j++) {
- builddir=KVM "/b" j
+ builddir=rd cfr[j] "/build";
print "rm -f " builddir ".ready"
print "if test -f \"" rd cfr[j] "/builtkernel\""
print "then"
@@ -509,7 +508,6 @@ function dump(first, pastlast, batchnum)
print "\techo ---- No kernel runs. `date` | tee -a " rd "log";
print "fi"
for (j = 1; j < jn; j++) {
- builddir=KVM "/b" j
print "echo ----", cfr[j], cpusr[j] ovf ": Build/run results: | tee -a " rd "log";
print "cat " rd cfr[j] "/kvm-test-1-run.sh.out | tee -a " rd "log";
}
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 08/28] torture: Use file-based protocol to mark batch's runs complete
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (6 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 07/28] torture: Move build/run synchronization files into scenario directories paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 09/28] torture: Use "jittering" file to control jitter.sh execution paulmck
` (23 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Currently, the script generated by kvm.sh does a "wait" to wait on both
the current batch's guest OSes and any jitter.sh scripts. This works,
but makes it hard to abstract the jittering so that common code can be
used for both local and distributed runs. This commit therefore uses
"build.run" files in scenario directories, and these files are removed
after the corresponding scenario's guest OS has completed.
Note that --build-only runs do not create build.run files because they
also do not create guest OSes and do not run any jitter.sh scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 3 +++
tools/testing/selftests/rcutorture/bin/kvm.sh | 13 +++++++++++--
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index 91578d3..fed6f10 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -345,4 +345,7 @@ then
echo Unknown PID, cannot kill qemu command
fi
+# Tell the script that this run is done.
+rm -f $resdir/build.run
+
parse-console.sh $resdir/console.log $title
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 7944510..1f5f872 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -469,9 +469,15 @@ function dump(first, pastlast, batchnum)
print "echo ", cfr[jn], cpusr[jn] ovf ": Build complete. `date` | tee -a " rd "log";
jn++;
}
+ print "runfiles="
for (j = 1; j < jn; j++) {
builddir=rd cfr[j] "/build";
- print "rm -f " builddir ".ready"
+ if (TORTURE_BUILDONLY)
+ print "rm -f " builddir ".ready"
+ else
+ print "mv " builddir ".ready " builddir ".run"
+ print "runfiles=\"$runfiles " builddir ".run\""
+ fi
print "if test -f \"" rd cfr[j] "/builtkernel\""
print "then"
print "\techo ----", cfr[j], cpusr[j] ovf ": Kernel present. `date` | tee -a " rd "log";
@@ -501,7 +507,10 @@ function dump(first, pastlast, batchnum)
print "\tjitter.sh " j " " dur " " ja[2] " " ja[3] "&"
print "\techo $! >> " rd "jitter_pids"
}
- print "\twait"
+ print "\twhile ls $runfiles > /dev/null 2>&1"
+ print "\tdo"
+ print "\t\t:"
+ print "\tdone"
print "\techo ---- All kernel runs complete. `date` | tee -a " rd "log";
print "else"
print "\twait"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 09/28] torture: Use "jittering" file to control jitter.sh execution
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (7 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 08/28] torture: Use file-based protocol to mark batch's runs complete paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 10/28] torture: Eliminate jitter_pids file paulmck
` (22 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Currently, jitter.sh execution is controlled by a time limit and by the
"kill" command. The former allowed jitter.sh to run uselessly past
the end of a set of runs that panicked during boot, and the latter is
vulnerable to PID reuse. This commit therefore introduces a "jittering"
file in the date-stamp directory within "res" that must be present for
the jitter.sh scripts to continue executing. The time limit is still
in place in order to avoid disturbing runs featuring large trace dumps,
but the removal of the "jittering" file handles the panic-during-boot
scenario without relying on PIDs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/jitter.sh | 10 ++++++----
tools/testing/selftests/rcutorture/bin/kvm.sh | 5 ++++-
2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/jitter.sh b/tools/testing/selftests/rcutorture/bin/jitter.sh
index 188b864..ed0ea86 100755
--- a/tools/testing/selftests/rcutorture/bin/jitter.sh
+++ b/tools/testing/selftests/rcutorture/bin/jitter.sh
@@ -5,10 +5,11 @@
# of this script is to inflict random OS jitter on a concurrently running
# test.
#
-# Usage: jitter.sh me duration [ sleepmax [ spinmax ] ]
+# Usage: jitter.sh me duration jittering-path [ sleepmax [ spinmax ] ]
#
# me: Random-number-generator seed salt.
# duration: Time to run in seconds.
+# jittering-path: Path to file whose removal will stop this script.
# sleepmax: Maximum microseconds to sleep, defaults to one second.
# spinmax: Maximum microseconds to spin, defaults to one millisecond.
#
@@ -18,8 +19,9 @@
me=$(($1 * 1000))
duration=$2
-sleepmax=${3-1000000}
-spinmax=${4-1000}
+jittering=$3
+sleepmax=${4-1000000}
+spinmax=${5-1000}
n=1
@@ -47,7 +49,7 @@ do
fi
# Check for stop request.
- if test -f "$TORTURE_STOPFILE"
+ if ! test -f "$jittering"
then
exit 1;
fi
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 1f5f872..48da4cd 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -503,14 +503,17 @@ function dump(first, pastlast, batchnum)
print "then"
print "\techo ---- Starting kernels. `date` | tee -a " rd "log";
print "\techo > " rd "jitter_pids"
+ print "\ttouch " rd "jittering"
for (j = 0; j < njitter; j++) {
- print "\tjitter.sh " j " " dur " " ja[2] " " ja[3] "&"
+ print "\tjitter.sh " j " " dur " " rd "jittering " ja[2] " " ja[3] "&"
print "\techo $! >> " rd "jitter_pids"
}
print "\twhile ls $runfiles > /dev/null 2>&1"
print "\tdo"
print "\t\t:"
print "\tdone"
+ print "\trm -f " rd "jittering"
+ print "\twait"
print "\techo ---- All kernel runs complete. `date` | tee -a " rd "log";
print "else"
print "\twait"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 10/28] torture: Eliminate jitter_pids file
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (8 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 09/28] torture: Use "jittering" file to control jitter.sh execution paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 11/28] torture: Reverse jittering and duration parameters for jitter.sh paulmck
` (21 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Now that there is a reliable way to convince the jitter.sh scripts to
stop, the jitter_pids file is not needed, nor is the code that kills all
the PIDs contained in this file. This commit therefore eliminates this
file and the code using it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 14 --------------
tools/testing/selftests/rcutorture/bin/kvm.sh | 5 +----
2 files changed, 1 insertion(+), 18 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index fed6f10..eb5346b 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -270,20 +270,6 @@ do
echo "ps -fp $killpid" >> $resdir/Warnings 2>&1
ps -fp $killpid >> $resdir/Warnings 2>&1
fi
- # Reduce probability of PID reuse by allowing a one-minute buffer
- if test $((kruntime + 60)) -lt $seconds && test -s "$resdir/../jitter_pids"
- then
- awk < "$resdir/../jitter_pids" '
- NF > 0 {
- pidlist = pidlist " " $1;
- n++;
- }
- END {
- if (n > 0) {
- print "kill " pidlist;
- }
- }' | sh
- fi
else
echo ' ---' `date`: "Kernel done"
fi
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 48da4cd..de93802 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -502,12 +502,9 @@ function dump(first, pastlast, batchnum)
print "if test -n \"$needqemurun\""
print "then"
print "\techo ---- Starting kernels. `date` | tee -a " rd "log";
- print "\techo > " rd "jitter_pids"
print "\ttouch " rd "jittering"
- for (j = 0; j < njitter; j++) {
+ for (j = 0; j < njitter; j++)
print "\tjitter.sh " j " " dur " " rd "jittering " ja[2] " " ja[3] "&"
- print "\techo $! >> " rd "jitter_pids"
- }
print "\twhile ls $runfiles > /dev/null 2>&1"
print "\tdo"
print "\t\t:"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 11/28] torture: Reverse jittering and duration parameters for jitter.sh
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (9 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 10/28] torture: Eliminate jitter_pids file paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 12/28] torture: Abstract jitter.sh start/stop into scripts paulmck
` (20 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Remote rcutorture testing requires that jitter.sh continue to be
invoked from the generated script for local runs, but that it instead
be invoked on the remote system for distributed runs. This argues
for common jitterstart and jitterstop scripts. But it would be good
for jitterstart and jitterstop to control the name and location of the
"jittering" file, while continuing to have the duration controlled by
the caller of these new scripts.
This commit therefore reverses the order of the jittering and duration
parameters for jitter.sh, so that the jittering parameter precedes the
duration parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/jitter.sh | 6 +++---
tools/testing/selftests/rcutorture/bin/kvm.sh | 2 +-
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/jitter.sh b/tools/testing/selftests/rcutorture/bin/jitter.sh
index ed0ea86..ff1d3e4 100755
--- a/tools/testing/selftests/rcutorture/bin/jitter.sh
+++ b/tools/testing/selftests/rcutorture/bin/jitter.sh
@@ -5,7 +5,7 @@
# of this script is to inflict random OS jitter on a concurrently running
# test.
#
-# Usage: jitter.sh me duration jittering-path [ sleepmax [ spinmax ] ]
+# Usage: jitter.sh me jittering-path duration [ sleepmax [ spinmax ] ]
#
# me: Random-number-generator seed salt.
# duration: Time to run in seconds.
@@ -18,8 +18,8 @@
# Authors: Paul E. McKenney <paulmck@linux.ibm.com>
me=$(($1 * 1000))
-duration=$2
-jittering=$3
+jittering=$2
+duration=$3
sleepmax=${4-1000000}
spinmax=${5-1000}
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index de93802..a2ee3f2 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -504,7 +504,7 @@ function dump(first, pastlast, batchnum)
print "\techo ---- Starting kernels. `date` | tee -a " rd "log";
print "\ttouch " rd "jittering"
for (j = 0; j < njitter; j++)
- print "\tjitter.sh " j " " dur " " rd "jittering " ja[2] " " ja[3] "&"
+ print "\tjitter.sh " j " " rd "jittering " dur " " ja[2] " " ja[3] "&"
print "\twhile ls $runfiles > /dev/null 2>&1"
print "\tdo"
print "\t\t:"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 12/28] torture: Abstract jitter.sh start/stop into scripts
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (10 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 11/28] torture: Reverse jittering and duration parameters for jitter.sh paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 13/28] torture: Record TORTURE_KCONFIG_GDB_ARG in qemu-cmd paulmck
` (19 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit creates jitterstart.sh and jitterstop.sh scripts that handle
the starting and stopping of the jitter.sh scripts. These must be sourced
using the bash "." command to allow the generated script to wait on the
backgrounded jitter.sh scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../selftests/rcutorture/bin/jitterstart.sh | 37 ++++++++++++++++++++++
.../testing/selftests/rcutorture/bin/jitterstop.sh | 23 ++++++++++++++
tools/testing/selftests/rcutorture/bin/kvm.sh | 7 ++--
3 files changed, 62 insertions(+), 5 deletions(-)
create mode 100644 tools/testing/selftests/rcutorture/bin/jitterstart.sh
create mode 100644 tools/testing/selftests/rcutorture/bin/jitterstop.sh
diff --git a/tools/testing/selftests/rcutorture/bin/jitterstart.sh b/tools/testing/selftests/rcutorture/bin/jitterstart.sh
new file mode 100644
index 0000000..9fe3e83
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/jitterstart.sh
@@ -0,0 +1,37 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Start up the specified number of jitter.sh scripts in the background.
+#
+# Usage: . jitterstart.sh n jittering-dir duration [ sleepmax [ spinmax ] ]
+#
+# n: Number of jitter.sh scripts to start up.
+# jittering-dir: Directory in which to put "jittering" file.
+# duration: Time to run in seconds.
+# sleepmax: Maximum microseconds to sleep, defaults to one second.
+# spinmax: Maximum microseconds to spin, defaults to one millisecond.
+#
+# Copyright (C) 2021 Facebook, Inc.
+#
+# Authors: Paul E. McKenney <paulmck@kernel.org>
+
+jitter_n=$1
+if test -z "$jitter_n"
+then
+ echo jitterstart.sh: Missing count of jitter.sh scripts to start.
+ exit 33
+fi
+jittering_dir=$2
+if test -z "$jittering_dir"
+then
+ echo jitterstart.sh: Missing directory in which to place jittering file.
+ exit 34
+fi
+shift
+shift
+
+touch ${jittering_dir}jittering
+for ((jitter_i = 1; jitter_i <= $jitter_n; jitter_i++))
+do
+ jitter.sh $jitter_i "${jittering_dir}jittering" "$@" &
+done
diff --git a/tools/testing/selftests/rcutorture/bin/jitterstop.sh b/tools/testing/selftests/rcutorture/bin/jitterstop.sh
new file mode 100644
index 0000000..b70d1f4
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/jitterstop.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Remove the "jittering" file, signaling the jitter.sh scripts to stop,
+# then wait for them to terminate.
+#
+# Usage: . jitterstop.sh jittering-dir
+#
+# jittering-dir: Directory containing "jittering" file.
+#
+# Copyright (C) 2021 Facebook, Inc.
+#
+# Authors: Paul E. McKenney <paulmck@kernel.org>
+
+jittering_dir=$1
+if test -z "$jittering_dir"
+then
+ echo jitterstop.sh: Missing directory in which to place jittering file.
+ exit 34
+fi
+
+rm -f ${jittering_dir}jittering
+wait
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index a2ee3f2..d6973e4 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -502,15 +502,12 @@ function dump(first, pastlast, batchnum)
print "if test -n \"$needqemurun\""
print "then"
print "\techo ---- Starting kernels. `date` | tee -a " rd "log";
- print "\ttouch " rd "jittering"
- for (j = 0; j < njitter; j++)
- print "\tjitter.sh " j " " rd "jittering " dur " " ja[2] " " ja[3] "&"
+ print "\t. jitterstart.sh " njitter " " rd " " dur " " ja[2] " " ja[3]
print "\twhile ls $runfiles > /dev/null 2>&1"
print "\tdo"
print "\t\t:"
print "\tdone"
- print "\trm -f " rd "jittering"
- print "\twait"
+ print "\t. jitterstop.sh " rd
print "\techo ---- All kernel runs complete. `date` | tee -a " rd "log";
print "else"
print "\twait"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 13/28] torture: Record TORTURE_KCONFIG_GDB_ARG in qemu-cmd
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (11 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 12/28] torture: Abstract jitter.sh start/stop into scripts paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 14/28] torture: Extract kvm-test-1-run-qemu.sh from kvm-test-1-run.sh paulmck
` (18 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
When re-running old rcutorture builds, if the original run involved
gdb, the re-run also needs to do so. This commit therefore records the
TORTURE_KCONFIG_GDB_ARG environment variable into the qemu-cmd file so
that the re-run can access it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 1 +
1 file changed, 1 insertion(+)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index eb5346b..5d9ac90 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -201,6 +201,7 @@ echo kernel here: `head -n 1 $testid_txt | sed -e 's/^Build directory: //'` >>
echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_append $boot_args\" $TORTURE_QEMU_GDB_ARG > $resdir/qemu-cmd
echo "# TORTURE_SHUTDOWN_GRACE=$TORTURE_SHUTDOWN_GRACE" >> $resdir/qemu-cmd
echo "# seconds=$seconds" >> $resdir/qemu-cmd
+echo "# TORTURE_KCONFIG_GDB_ARG=\"$TORTURE_KCONFIG_GDB_ARG\"" >> $resdir/qemu-cmd
if test -n "$TORTURE_BUILDONLY"
then
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 14/28] torture: Extract kvm-test-1-run-qemu.sh from kvm-test-1-run.sh
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (12 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 13/28] torture: Record TORTURE_KCONFIG_GDB_ARG in qemu-cmd paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 15/28] torture: Record jitter start/stop commands paulmck
` (17 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Currently, kvm-test-1-run.sh both builds and runs an rcutorture kernel,
which is inconvenient when it is necessary to re-run an old run or to
carry out a run on a remote system. This commit therefore extracts the
portion of kvm-test-1-run.sh that invoke qemu to actually run rcutorture
and places it in kvm-test-1-run-qemu.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../rcutorture/bin/kvm-test-1-run-qemu.sh | 170 +++++++++++++++++++++
.../selftests/rcutorture/bin/kvm-test-1-run.sh | 127 +--------------
2 files changed, 171 insertions(+), 126 deletions(-)
create mode 100755 tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
new file mode 100755
index 0000000..6b0d71b
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
@@ -0,0 +1,170 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Carry out a kvm-based run for the specified qemu-cmd file, which might
+# have been generated by --build-only kvm.sh run.
+#
+# Usage: kvm-test-1-run-qemu.sh qemu-cmd-dir
+#
+# qemu-cmd-dir provides the directory containing qemu-cmd file.
+# This is assumed to be of the form prefix/ds/scenario, where
+# "ds" is the top-level date-stamped directory and "scenario"
+# is the scenario name. Any required adjustments to this file
+# must have been made by the caller. The shell-command comments
+# at the end of the qemu-cmd file are not optional.
+#
+# Copyright (C) 2021 Facebook, Inc.
+#
+# Authors: Paul E. McKenney <paulmck@kernel.org>
+
+T=${TMPDIR-/tmp}/kvm-test-1-run-qemu.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+resdir="$1"
+if ! test -d "$resdir"
+then
+ echo $0: Nonexistent directory: $resdir
+ exit 1
+fi
+if ! test -f "$resdir/qemu-cmd"
+then
+ echo $0: Nonexistent qemu-cmd file: $resdir/qemu-cmd
+ exit 1
+fi
+
+# Obtain settings from the qemu-cmd file.
+grep '^#' $resdir/qemu-cmd | sed -e 's/^# //' > $T/qemu-cmd-settings
+. $T/qemu-cmd-settings
+
+# Decorate qemu-cmd with redirection, backgrounding, and PID capture
+sed -e 's/$/ 2>\&1 \&/' < $resdir/qemu-cmd > $T/qemu-cmd
+echo 'echo $! > $resdir/qemu_pid' >> $T/qemu-cmd
+
+# In case qemu refuses to run...
+echo "NOTE: $QEMU either did not run or was interactive" > $resdir/console.log
+
+# Attempt to run qemu
+kstarttime=`gawk 'BEGIN { print systime() }' < /dev/null`
+( . $T/qemu-cmd; wait `cat $resdir/qemu_pid`; echo $? > $resdir/qemu-retval ) &
+commandcompleted=0
+if test -z "$TORTURE_KCONFIG_GDB_ARG"
+then
+ sleep 10 # Give qemu's pid a chance to reach the file
+ if test -s "$resdir/qemu_pid"
+ then
+ qemu_pid=`cat "$resdir/qemu_pid"`
+ echo Monitoring qemu job at pid $qemu_pid
+ else
+ qemu_pid=""
+ echo Monitoring qemu job at yet-as-unknown pid
+ fi
+fi
+if test -n "$TORTURE_KCONFIG_GDB_ARG"
+then
+ base_resdir=`echo $resdir | sed -e 's/\.[0-9]\+$//'`
+ if ! test -f $base_resdir/vmlinux
+ then
+ base_resdir=/path/to
+ fi
+ echo Waiting for you to attach a debug session, for example: > /dev/tty
+ echo " gdb $base_resdir/vmlinux" > /dev/tty
+ echo 'After symbols load and the "(gdb)" prompt appears:' > /dev/tty
+ echo " target remote :1234" > /dev/tty
+ echo " continue" > /dev/tty
+ kstarttime=`gawk 'BEGIN { print systime() }' < /dev/null`
+fi
+while :
+do
+ if test -z "$qemu_pid" -a -s "$resdir/qemu_pid"
+ then
+ qemu_pid=`cat "$resdir/qemu_pid"`
+ fi
+ kruntime=`gawk 'BEGIN { print systime() - '"$kstarttime"' }' < /dev/null`
+ if test -z "$qemu_pid" || kill -0 "$qemu_pid" > /dev/null 2>&1
+ then
+ if test -n "$TORTURE_KCONFIG_GDB_ARG"
+ then
+ :
+ elif test $kruntime -ge $seconds || test -f "$resdir/../STOP.1"
+ then
+ break;
+ fi
+ sleep 1
+ else
+ commandcompleted=1
+ if test $kruntime -lt $seconds
+ then
+ echo Completed in $kruntime vs. $seconds >> $resdir/Warnings 2>&1
+ grep "^(qemu) qemu:" $resdir/kvm-test-1-run.sh.out >> $resdir/Warnings 2>&1
+ killpid="`sed -n "s/^(qemu) qemu: terminating on signal [0-9]* from pid \([0-9]*\).*$/\1/p" $resdir/Warnings`"
+ if test -n "$killpid"
+ then
+ echo "ps -fp $killpid" >> $resdir/Warnings 2>&1
+ ps -fp $killpid >> $resdir/Warnings 2>&1
+ fi
+ else
+ echo ' ---' `date`: "Kernel done"
+ fi
+ break
+ fi
+done
+if test -z "$qemu_pid" -a -s "$resdir/qemu_pid"
+then
+ qemu_pid=`cat "$resdir/qemu_pid"`
+fi
+if test $commandcompleted -eq 0 -a -n "$qemu_pid"
+then
+ if ! test -f "$resdir/../STOP.1"
+ then
+ echo Grace period for qemu job at pid $qemu_pid
+ fi
+ oldline="`tail $resdir/console.log`"
+ while :
+ do
+ if test -f "$resdir/../STOP.1"
+ then
+ echo "PID $qemu_pid killed due to run STOP.1 request" >> $resdir/Warnings 2>&1
+ kill -KILL $qemu_pid
+ break
+ fi
+ kruntime=`gawk 'BEGIN { print systime() - '"$kstarttime"' }' < /dev/null`
+ if kill -0 $qemu_pid > /dev/null 2>&1
+ then
+ :
+ else
+ break
+ fi
+ must_continue=no
+ newline="`tail $resdir/console.log`"
+ if test "$newline" != "$oldline" && echo $newline | grep -q ' [0-9]\+us : '
+ then
+ must_continue=yes
+ fi
+ last_ts="`tail $resdir/console.log | grep '^\[ *[0-9]\+\.[0-9]\+]' | tail -1 | sed -e 's/^\[ *//' -e 's/\..*$//'`"
+ if test -z "$last_ts"
+ then
+ last_ts=0
+ fi
+ if test "$newline" != "$oldline" -a "$last_ts" -lt $((seconds + $TORTURE_SHUTDOWN_GRACE))
+ then
+ must_continue=yes
+ fi
+ if test $must_continue = no -a $kruntime -ge $((seconds + $TORTURE_SHUTDOWN_GRACE))
+ then
+ echo "!!! PID $qemu_pid hung at $kruntime vs. $seconds seconds" >> $resdir/Warnings 2>&1
+ kill -KILL $qemu_pid
+ break
+ fi
+ oldline=$newline
+ sleep 10
+ done
+elif test -z "$qemu_pid"
+then
+ echo Unknown PID, cannot kill qemu command
+fi
+
+# Tell the script that this run is done.
+rm -f $resdir/build.run
+
+parse-console.sh $resdir/console.log $title
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index 5d9ac90..f3d2ded 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -210,129 +210,4 @@ then
exit 0
fi
-# Decorate qemu-cmd with redirection, backgrounding, and PID capture
-sed -e 's/$/ 2>\&1 \&/' < $resdir/qemu-cmd > $T/qemu-cmd
-echo 'echo $! > $resdir/qemu_pid' >> $T/qemu-cmd
-
-# In case qemu refuses to run...
-echo "NOTE: $QEMU either did not run or was interactive" > $resdir/console.log
-
-# Attempt to run qemu
-kstarttime=`gawk 'BEGIN { print systime() }' < /dev/null`
-( . $T/qemu-cmd; wait `cat $resdir/qemu_pid`; echo $? > $resdir/qemu-retval ) &
-commandcompleted=0
-if test -z "$TORTURE_KCONFIG_GDB_ARG"
-then
- sleep 10 # Give qemu's pid a chance to reach the file
- if test -s "$resdir/qemu_pid"
- then
- qemu_pid=`cat "$resdir/qemu_pid"`
- echo Monitoring qemu job at pid $qemu_pid
- else
- qemu_pid=""
- echo Monitoring qemu job at yet-as-unknown pid
- fi
-fi
-if test -n "$TORTURE_KCONFIG_GDB_ARG"
-then
- echo Waiting for you to attach a debug session, for example: > /dev/tty
- echo " gdb $base_resdir/vmlinux" > /dev/tty
- echo 'After symbols load and the "(gdb)" prompt appears:' > /dev/tty
- echo " target remote :1234" > /dev/tty
- echo " continue" > /dev/tty
- kstarttime=`gawk 'BEGIN { print systime() }' < /dev/null`
-fi
-while :
-do
- if test -z "$qemu_pid" -a -s "$resdir/qemu_pid"
- then
- qemu_pid=`cat "$resdir/qemu_pid"`
- fi
- kruntime=`gawk 'BEGIN { print systime() - '"$kstarttime"' }' < /dev/null`
- if test -z "$qemu_pid" || kill -0 "$qemu_pid" > /dev/null 2>&1
- then
- if test -n "$TORTURE_KCONFIG_GDB_ARG"
- then
- :
- elif test $kruntime -ge $seconds || test -f "$resdir/../STOP.1"
- then
- break;
- fi
- sleep 1
- else
- commandcompleted=1
- if test $kruntime -lt $seconds
- then
- echo Completed in $kruntime vs. $seconds >> $resdir/Warnings 2>&1
- grep "^(qemu) qemu:" $resdir/kvm-test-1-run.sh.out >> $resdir/Warnings 2>&1
- killpid="`sed -n "s/^(qemu) qemu: terminating on signal [0-9]* from pid \([0-9]*\).*$/\1/p" $resdir/Warnings`"
- if test -n "$killpid"
- then
- echo "ps -fp $killpid" >> $resdir/Warnings 2>&1
- ps -fp $killpid >> $resdir/Warnings 2>&1
- fi
- else
- echo ' ---' `date`: "Kernel done"
- fi
- break
- fi
-done
-if test -z "$qemu_pid" -a -s "$resdir/qemu_pid"
-then
- qemu_pid=`cat "$resdir/qemu_pid"`
-fi
-if test $commandcompleted -eq 0 -a -n "$qemu_pid"
-then
- if ! test -f "$resdir/../STOP.1"
- then
- echo Grace period for qemu job at pid $qemu_pid
- fi
- oldline="`tail $resdir/console.log`"
- while :
- do
- if test -f "$resdir/../STOP.1"
- then
- echo "PID $qemu_pid killed due to run STOP.1 request" >> $resdir/Warnings 2>&1
- kill -KILL $qemu_pid
- break
- fi
- kruntime=`gawk 'BEGIN { print systime() - '"$kstarttime"' }' < /dev/null`
- if kill -0 $qemu_pid > /dev/null 2>&1
- then
- :
- else
- break
- fi
- must_continue=no
- newline="`tail $resdir/console.log`"
- if test "$newline" != "$oldline" && echo $newline | grep -q ' [0-9]\+us : '
- then
- must_continue=yes
- fi
- last_ts="`tail $resdir/console.log | grep '^\[ *[0-9]\+\.[0-9]\+]' | tail -1 | sed -e 's/^\[ *//' -e 's/\..*$//'`"
- if test -z "$last_ts"
- then
- last_ts=0
- fi
- if test "$newline" != "$oldline" -a "$last_ts" -lt $((seconds + $TORTURE_SHUTDOWN_GRACE))
- then
- must_continue=yes
- fi
- if test $must_continue = no -a $kruntime -ge $((seconds + $TORTURE_SHUTDOWN_GRACE))
- then
- echo "!!! PID $qemu_pid hung at $kruntime vs. $seconds seconds" >> $resdir/Warnings 2>&1
- kill -KILL $qemu_pid
- break
- fi
- oldline=$newline
- sleep 10
- done
-elif test -z "$qemu_pid"
-then
- echo Unknown PID, cannot kill qemu command
-fi
-
-# Tell the script that this run is done.
-rm -f $resdir/build.run
-
-parse-console.sh $resdir/console.log $title
+kvm-test-1-run-qemu.sh $resdir
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 15/28] torture: Record jitter start/stop commands
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (13 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 14/28] torture: Extract kvm-test-1-run-qemu.sh from kvm-test-1-run.sh paulmck
@ 2021-03-04 0:37 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 16/28] torture: Record kvm-test-1-run.sh and kvm-test-1-run-qemu.sh PIDs paulmck
` (16 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:37 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Distributed runs of rcutorture will need to start and stop jittering on
the remote hosts, which means that the commands must be communicated to
those hosts. The commit therefore causes kvm.sh to place these commands
in new TORTURE_JITTER_START and TORTURE_JITTER_STOP environment variables
to communicate them to the scripts that will set this up. In addition,
this commit causes kvm-test-1-run.sh to append these commands to each
generated qemu-cmd file, which allows any remotely executing script to
extract the needed commands from this file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../selftests/rcutorture/bin/kvm-test-1-run.sh | 2 ++
tools/testing/selftests/rcutorture/bin/kvm.sh | 24 +++++++++++++---------
2 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index f3d2ded..a69f8ae 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -202,6 +202,8 @@ echo $QEMU $qemu_args -m $TORTURE_QEMU_MEM -kernel $KERNEL -append \"$qemu_appen
echo "# TORTURE_SHUTDOWN_GRACE=$TORTURE_SHUTDOWN_GRACE" >> $resdir/qemu-cmd
echo "# seconds=$seconds" >> $resdir/qemu-cmd
echo "# TORTURE_KCONFIG_GDB_ARG=\"$TORTURE_KCONFIG_GDB_ARG\"" >> $resdir/qemu-cmd
+echo "# TORTURE_JITTER_START=\"$TORTURE_JITTER_START\"" >> $resdir/qemu-cmd
+echo "# TORTURE_JITTER_STOP=\"$TORTURE_JITTER_STOP\"" >> $resdir/qemu-cmd
if test -n "$TORTURE_BUILDONLY"
then
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index d6973e4..efcbd12 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -35,6 +35,8 @@ TORTURE_KCONFIG_ARG=""
TORTURE_KCONFIG_GDB_ARG=""
TORTURE_BOOT_GDB_ARG=""
TORTURE_QEMU_GDB_ARG=""
+TORTURE_JITTER_START=""
+TORTURE_JITTER_STOP=""
TORTURE_KCONFIG_KASAN_ARG=""
TORTURE_KCONFIG_KCSAN_ARG=""
TORTURE_KMAKE_ARG=""
@@ -443,6 +445,16 @@ function dump(first, pastlast, batchnum)
print "echo ----Start batch " batchnum ": `date` | tee -a " rd "log";
print "needqemurun="
jn=1
+ njitter = 0;
+ split(jitter, ja);
+ if (ja[1] == -1 && ncpus == 0)
+ njitter = 1;
+ else if (ja[1] == -1)
+ njitter = ncpus;
+ else
+ njitter = ja[1];
+ print "TORTURE_JITTER_START=\". jitterstart.sh " njitter " " rd " " dur " " ja[2] " " ja[3] "\"; export TORTURE_JITTER_START";
+ print "TORTURE_JITTER_STOP=\". jitterstop.sh " rd " \"; export TORTURE_JITTER_STOP"
for (j = first; j < pastlast; j++) {
cpusr[jn] = cpus[j];
if (cfrep[cf[j]] == "") {
@@ -484,14 +496,6 @@ function dump(first, pastlast, batchnum)
print "\tneedqemurun=1"
print "fi"
}
- njitter = 0;
- split(jitter, ja);
- if (ja[1] == -1 && ncpus == 0)
- njitter = 1;
- else if (ja[1] == -1)
- njitter = ncpus;
- else
- njitter = ja[1];
if (TORTURE_BUILDONLY && njitter != 0) {
njitter = 0;
print "echo Build-only run, so suppressing jitter | tee -a " rd "log"
@@ -502,12 +506,12 @@ function dump(first, pastlast, batchnum)
print "if test -n \"$needqemurun\""
print "then"
print "\techo ---- Starting kernels. `date` | tee -a " rd "log";
- print "\t. jitterstart.sh " njitter " " rd " " dur " " ja[2] " " ja[3]
+ print "\t$TORTURE_JITTER_START";
print "\twhile ls $runfiles > /dev/null 2>&1"
print "\tdo"
print "\t\t:"
print "\tdone"
- print "\t. jitterstop.sh " rd
+ print "\t$TORTURE_JITTER_STOP";
print "\techo ---- All kernel runs complete. `date` | tee -a " rd "log";
print "else"
print "\twait"
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 16/28] torture: Record kvm-test-1-run.sh and kvm-test-1-run-qemu.sh PIDs
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (14 preceding siblings ...)
2021-03-04 0:37 ` [PATCH tip/core/rcu 15/28] torture: Record jitter start/stop commands paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 17/28] torture: Remove no-mpstat error message paulmck
` (15 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit records the process IDs of the kvm-test-1-run.sh and
kvm-test-1-run-qemu.sh scripts to ease monitoring of remotely running
instances of these scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh | 2 ++
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
index 6b0d71b..576a9b7 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
@@ -33,6 +33,8 @@ then
exit 1
fi
+echo ' ---' `date`: Starting kernel, PID $$
+
# Obtain settings from the qemu-cmd file.
grep '^#' $resdir/qemu-cmd | sed -e 's/^# //' > $T/qemu-cmd-settings
. $T/qemu-cmd-settings
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index a69f8ae..a386ca8d 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -41,7 +41,7 @@ then
echo "kvm-test-1-run.sh :$resdir: Not a writable directory, cannot store results into it"
exit 1
fi
-echo ' ---' `date`: Starting build
+echo ' ---' `date`: Starting build, PID $$
echo ' ---' Kconfig fragment at: $config_template >> $resdir/log
touch $resdir/ConfigFragment.input
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 17/28] torture: Remove no-mpstat error message
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (15 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 16/28] torture: Record kvm-test-1-run.sh and kvm-test-1-run-qemu.sh PIDs paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 18/28] torture: Rename SRCU-t and SRCU-u to avoid lowercase characters paulmck
` (14 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
The cpus2use.sh script complains if the mpstat command is not available,
and instead uses all available CPUs. Unfortunately, this complaint
goes to stdout, where it confuses invokers who expect a single number.
This commit removes this error message in order to avoid this confusion.
The tendency of late has been to give rcutorture a full system, so this
should not cause issues.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/cpus2use.sh | 1 -
1 file changed, 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/cpus2use.sh b/tools/testing/selftests/rcutorture/bin/cpus2use.sh
index 1dbfb62..6bb9930 100755
--- a/tools/testing/selftests/rcutorture/bin/cpus2use.sh
+++ b/tools/testing/selftests/rcutorture/bin/cpus2use.sh
@@ -21,7 +21,6 @@ then
awk -v ncpus=$ncpus '{ print ncpus * ($7 + $NF) / 100 }'`
else
# No mpstat command, so use all available CPUs.
- echo The mpstat command is not available, so greedily using all CPUs.
idlecpus=$ncpus
fi
awk -v ncpus=$ncpus -v idlecpus=$idlecpus < /dev/null '
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 18/28] torture: Rename SRCU-t and SRCU-u to avoid lowercase characters
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (16 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 17/28] torture: Remove no-mpstat error message paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 19/28] torture: Make upper-case-only no-dot no-slash scenario names official paulmck
` (13 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
The convention that scenario names are all uppercase has two exceptions,
SRCU-t and SRCU-u. This commit therefore renames them to SRCU-T and
SRCU-U, respectively, to bring them in line with this convention. This in
turn permits tighter argument checking in the torture-test scripting.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/configs/rcu/CFLIST | 4 ++--
tools/testing/selftests/rcutorture/configs/rcu/{SRCU-t => SRCU-T} | 0
.../selftests/rcutorture/configs/rcu/{SRCU-t.boot => SRCU-T.boot} | 0
tools/testing/selftests/rcutorture/configs/rcu/{SRCU-u => SRCU-U} | 0
.../selftests/rcutorture/configs/rcu/{SRCU-u.boot => SRCU-U.boot} | 0
5 files changed, 2 insertions(+), 2 deletions(-)
rename tools/testing/selftests/rcutorture/configs/rcu/{SRCU-t => SRCU-T} (100%)
rename tools/testing/selftests/rcutorture/configs/rcu/{SRCU-t.boot => SRCU-T.boot} (100%)
rename tools/testing/selftests/rcutorture/configs/rcu/{SRCU-u => SRCU-U} (100%)
rename tools/testing/selftests/rcutorture/configs/rcu/{SRCU-u.boot => SRCU-U.boot} (100%)
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/CFLIST b/tools/testing/selftests/rcutorture/configs/rcu/CFLIST
index f2b20db..98b6175 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/CFLIST
+++ b/tools/testing/selftests/rcutorture/configs/rcu/CFLIST
@@ -7,8 +7,8 @@ TREE07
TREE09
SRCU-N
SRCU-P
-SRCU-t
-SRCU-u
+SRCU-T
+SRCU-U
TINY01
TINY02
TASKS01
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-T
similarity index 100%
rename from tools/testing/selftests/rcutorture/configs/rcu/SRCU-t
rename to tools/testing/selftests/rcutorture/configs/rcu/SRCU-T
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-t.boot b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-T.boot
similarity index 100%
rename from tools/testing/selftests/rcutorture/configs/rcu/SRCU-t.boot
rename to tools/testing/selftests/rcutorture/configs/rcu/SRCU-T.boot
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-U
similarity index 100%
rename from tools/testing/selftests/rcutorture/configs/rcu/SRCU-u
rename to tools/testing/selftests/rcutorture/configs/rcu/SRCU-U
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/SRCU-u.boot b/tools/testing/selftests/rcutorture/configs/rcu/SRCU-U.boot
similarity index 100%
rename from tools/testing/selftests/rcutorture/configs/rcu/SRCU-u.boot
rename to tools/testing/selftests/rcutorture/configs/rcu/SRCU-U.boot
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 19/28] torture: Make upper-case-only no-dot no-slash scenario names official
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (17 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 18/28] torture: Rename SRCU-t and SRCU-u to avoid lowercase characters paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 20/28] torture: De-capitalize TORTURE_SUITE paulmck
` (12 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit enforces the defacto restriction on scenario names, which is
that they contain neither "/", ".", nor lowercase alphabetic characters.
This restriction avoids collisions between scenario names and the torture
scripting's files and directories.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index efcbd12..03364f4 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -104,7 +104,7 @@ do
TORTURE_BUILDONLY=1
;;
--configs|--config)
- checkarg --configs "(list of config files)" "$#" "$2" '^[^/]\+$' '^--'
+ checkarg --configs "(list of config files)" "$#" "$2" '^[^/.a-z]\+$' '^--'
configs="$configs $2"
shift
;;
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 20/28] torture: De-capitalize TORTURE_SUITE
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (18 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 19/28] torture: Make upper-case-only no-dot no-slash scenario names official paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 21/28] torture: Create a "batches" file for build reuse paulmck
` (11 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Although it might be unlikely that someone would name a scenario
"TORTURE_SUITE", they are within their rights to do so. This script
therefore renames the "TORTURE_SUITE" file in the top-level date-stamped
directory within "res" to "torture_suite" to avoid this name collision.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-recheck.sh | 2 +-
tools/testing/selftests/rcutorture/bin/kvm.sh | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh
index 47cf4db..e01b31b 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh
@@ -30,7 +30,7 @@ do
resdir=`echo $i | sed -e 's,/$,,' -e 's,/[^/]*$,,'`
head -1 $resdir/log
fi
- TORTURE_SUITE="`cat $i/../TORTURE_SUITE`"
+ TORTURE_SUITE="`cat $i/../torture_suite`"
configfile=`echo $i | sed -e 's,^.*/,,'`
rm -f $i/console.log.*.diags
kvm-recheck-${TORTURE_SUITE}.sh $i
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 03364f4..a1cd05c 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -405,7 +405,7 @@ echo Results directory: $resdir/$ds
echo $scriptname $args
touch $resdir/$ds/log
echo $scriptname $args >> $resdir/$ds/log
-echo ${TORTURE_SUITE} > $resdir/$ds/TORTURE_SUITE
+echo ${TORTURE_SUITE} > $resdir/$ds/torture_suite
echo Build directory: `pwd` > $resdir/$ds/testid.txt
if test -d .git
then
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 21/28] torture: Create a "batches" file for build reuse
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (19 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 20/28] torture: De-capitalize TORTURE_SUITE paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 22/28] torture: Add kvm-again.sh to rerun a previous torture-test paulmck
` (10 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit creates a "batches" file in the res/$ds directory, where $ds
is the datestamp. This file contains the batches and the number of CPUs,
for example:
1 TREE03 16
1 SRCU-P 8
2 TREE07 16
2 TREE01 8
3 TREE02 8
3 TREE04 8
3 TREE05 8
4 SRCU-N 4
4 TRACE01 4
4 TRACE02 4
4 RUDE01 2
4 RUDE01.2 2
4 TASKS01 2
4 TASKS03 2
4 SRCU-t 1
4 SRCU-u 1
4 TASKS02 1
4 TINY01 1
5 TINY02 1
5 TREE09 1
The first column is the batch number, the second the scenario number
(possibly suffixed by a repetition number, as in "RUDE01.2"), and the
third is the number of CPUs required by that scenario. The last line
shows the number of CPUs expected by this batch file, which allows
the run to be re-batched if a different number of CPUs is available.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm.sh | 29 ++++++++++++++++-----------
1 file changed, 17 insertions(+), 12 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index a1cd05c..0add163 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -565,6 +565,18 @@ echo 'ret=$?' >> $T/script
echo "cat $T/kvm-recheck.sh.out | tee -a $resdir/$ds/log" >> $T/script
echo 'exit $ret' >> $T/script
+# Extract the tests and their batches from the script.
+egrep 'Start batch|Starting build\.' $T/script | grep -v ">>" |
+ sed -e 's/:.*$//' -e 's/^echo //' -e 's/-ovf//' |
+ awk '
+ /^----Start/ {
+ batchno = $3;
+ next;
+ }
+ {
+ print batchno, $1, $2
+ }' > $T/batches
+
if test "$dryrun" = script
then
cat $T/script
@@ -583,21 +595,14 @@ then
exit 0
elif test "$dryrun" = batches
then
- # Extract the tests and their batches from the script.
- egrep 'Start batch|Starting build\.' $T/script | grep -v ">>" |
- sed -e 's/:.*$//' -e 's/^echo //' -e 's/-ovf//' |
- awk '
- /^----Start/ {
- batchno = $3;
- next;
- }
- {
- print batchno, $1, $2
- }'
+ cat $T/batches
+ exit 0
else
- # Not a dryrun, so run the script.
+ # Not a dryrun. Record the batches and the number of CPUs, then run the script.
bash $T/script
ret=$?
+ cp $T/batches $resdir/$ds/batches
+ echo '#' cpus=$cpus >> $resdir/$ds/batches
echo " --- Done at `date` (`get_starttime_duration $starttime`) exitcode $ret" | tee -a $resdir/$ds/log
exit $ret
fi
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 22/28] torture: Add kvm-again.sh to rerun a previous torture-test
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (20 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 21/28] torture: Create a "batches" file for build reuse paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 23/28] torture: Add --duration argument to kvm-again.sh paulmck
` (9 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit adds a kvm-again.sh script that, given the results directory
of a torture-test run, re-runs that test. This means that the kernels
need not be rebuilt, but it also is a step towards running torture tests
on remote systems.
This commit also adds a kvm-test-1-run-batch.sh script that runs one
batch out of the torture test. The idea is to copy a results directory
tree to remote systems, then use kvm-test-1-run-batch.sh to run batches
on these systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../testing/selftests/rcutorture/bin/kvm-again.sh | 171 +++++++++++++++++++++
.../rcutorture/bin/kvm-test-1-run-batch.sh | 67 ++++++++
2 files changed, 238 insertions(+)
create mode 100755 tools/testing/selftests/rcutorture/bin/kvm-again.sh
create mode 100755 tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
new file mode 100755
index 0000000..92cf396
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -0,0 +1,171 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Rerun a series of tests under KVM.
+#
+# Usage: kvm.sh /path/to/old/run [ options ]
+#
+# Copyright (C) 2021 Facebook, Inc.
+#
+# Authors: Paul E. McKenney <paulmck@kernel.org>
+
+scriptname=$0
+args="$*"
+
+T=${TMPDIR-/tmp}/kvm-again.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+if ! test -d tools/testing/selftests/rcutorture/bin
+then
+ echo $scriptname must be run from top-level directory of kernel source tree.
+ exit 1
+fi
+
+oldrun=$1
+shift
+if ! test -d "$oldrun"
+then
+ echo "Usage: $scriptname /path/to/old/run [ options ]"
+ exit 1
+fi
+if ! cp "$oldrun/batches" $T/batches.oldrun
+then
+ # Later on, can reconstitute this from console.log files.
+ echo Prior run batches file does not exist: $oldrun/batches
+ exit 1
+fi
+
+if test -f "$oldrun/torture_suite"
+then
+ torture_suite="`cat $oldrun/torture_suite`"
+elif test -f "$oldrun/TORTURE_SUITE"
+then
+ torture_suite="`cat $oldrun/TORTURE_SUITE`"
+else
+ echo "Prior run torture_suite file does not exist: $oldrun/{torture_suite,TORTURE_SUITE}"
+ exit 1
+fi
+
+KVM="`pwd`/tools/testing/selftests/rcutorture"; export KVM
+PATH=${KVM}/bin:$PATH; export PATH
+. functions.sh
+
+dryrun=
+default_link="cp -R"
+rundir="`pwd`/tools/testing/selftests/rcutorture/res/`date +%Y.%m.%d-%H.%M.%S-again`"
+
+startdate="`date`"
+starttime="`get_starttime`"
+
+usage () {
+ echo "Usage: $scriptname $oldrun [ arguments ]:"
+ echo " --dryrun"
+ echo " --link hard|soft|copy"
+ echo " --remote"
+ echo " --rundir /new/res/path"
+ exit 1
+}
+
+while test $# -gt 0
+do
+ case "$1" in
+ --dryrun)
+ dryrun=1
+ ;;
+ --link)
+ checkarg --link "hard|soft|copy" "$#" "$2" 'hard\|soft\|copy' '^--'
+ case "$2" in
+ copy)
+ arg_link="cp -R"
+ ;;
+ hard)
+ arg_link="cp -Rl"
+ ;;
+ soft)
+ arg_link="cp -Rs"
+ ;;
+ esac
+ shift
+ ;;
+ --remote)
+ arg_remote=1
+ default_link="cp -as"
+ ;;
+ --rundir)
+ checkarg --rundir "(absolute pathname)" "$#" "$2" '^/' '^error'
+ rundir=$2
+ if test -e "$rundir"
+ then
+ echo "--rundir $2: Already exists."
+ usage
+ fi
+ shift
+ ;;
+ *)
+ echo Unknown argument $1
+ usage
+ ;;
+ esac
+ shift
+done
+if test -z "$arg_link"
+then
+ arg_link="$default_link"
+fi
+
+echo ---- Re-run results directory: $rundir
+
+# Copy old run directory tree over and adjust.
+mkdir -p "`dirname "$rundir"`"
+if ! $arg_link "$oldrun" "$rundir"
+then
+ echo "Cannot copy from $oldrun to $rundir."
+ usage
+fi
+rm -f "$rundir"/*/{console.log,console.log.diags,qemu_pid,qemu-retval,Warnings,kvm-test-1-run.sh.out,kvm-test-1-run-qemu.sh.out,vmlinux} "$rundir"/log
+echo $oldrun > "$rundir/re-run"
+if ! test -d "$rundir/../../bin"
+then
+ $arg_link "$oldrun/../../bin" "$rundir/../.."
+fi
+for i in $rundir/*/qemu-cmd
+do
+ cp "$i" $T
+ qemu_cmd_dir="`dirname "$i"`"
+ kernel_dir="`echo $qemu_cmd_dir | sed -e 's/\.[0-9]\+$//'`"
+ kvm-transform.sh $kernel_dir/bzImage $qemu_cmd_dir/console.log < $T/qemu-cmd > $i
+ echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
+done
+grep -v '^#' $T/batches.oldrun | awk '
+BEGIN {
+ oldbatch = 1;
+}
+
+{
+ if (oldbatch != $1) {
+ print "kvm-test-1-run-batch.sh" curbatch;
+ curbatch = "";
+ oldbatch = $1;
+ }
+ curbatch = curbatch " " $2;
+}
+
+END {
+ print "kvm-test-1-run-batch.sh" curbatch
+}' > $T/runbatches.sh
+
+if test -n "$dryrun"
+then
+ echo ---- Dryrun complete, directory: $rundir | tee -a "$rundir/log"
+else
+ ( cd "$rundir"; sh $T/runbatches.sh )
+ kcsan-collapse.sh "$rundir" | tee -a "$rundir/log"
+ echo | tee -a "$rundir/log"
+ echo ---- Results directory: $rundir | tee -a "$rundir/log"
+ kvm-recheck.sh "$rundir" > $T/kvm-recheck.sh.out 2>&1
+ ret=$?
+ cat $T/kvm-recheck.sh.out | tee -a "$rundir/log"
+ echo " --- Done at `date` (`get_starttime_duration $starttime`) exitcode $ret" | tee -a "$rundir/log"
+ exit $ret
+fi
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh
new file mode 100755
index 0000000..7ea0809e
--- /dev/null
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-batch.sh
@@ -0,0 +1,67 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0+
+#
+# Carry out a kvm-based run for the specified batch of scenarios, which
+# might have been built by --build-only kvm.sh run.
+#
+# Usage: kvm-test-1-run-batch.sh SCENARIO [ SCENARIO ... ]
+#
+# Each SCENARIO is the name of a directory in the current directory
+# containing a ready-to-run qemu-cmd file.
+#
+# Copyright (C) 2021 Facebook, Inc.
+#
+# Authors: Paul E. McKenney <paulmck@kernel.org>
+
+T=${TMPDIR-/tmp}/kvm-test-1-run-batch.sh.$$
+trap 'rm -rf $T' 0
+mkdir $T
+
+echo ---- Running batch $*
+# Check arguments
+runfiles=
+for i in "$@"
+do
+ if ! echo $i | grep -q '^[^/.a-z]\+\(\.[0-9]\+\)\?$'
+ then
+ echo Bad scenario name: \"$i\" 1>&2
+ exit 1
+ fi
+ if ! test -d "$i"
+ then
+ echo Scenario name not a directory: \"$i\" 1>&2
+ exit 2
+ fi
+ if ! test -f "$i/qemu-cmd"
+ then
+ echo Scenario lacks a command file: \"$i/qemu-cmd\" 1>&2
+ exit 3
+ fi
+ rm -f $i/build.*
+ touch $i/build.run
+ runfiles="$runfiles $i/build.run"
+done
+
+# Extract settings from the qemu-cmd file.
+grep '^#' $1/qemu-cmd | sed -e 's/^# //' > $T/qemu-cmd-settings
+. $T/qemu-cmd-settings
+
+# Start up jitter, start each scenario, wait, end jitter.
+echo ---- System running test: `uname -a`
+echo ---- Starting kernels. `date` | tee -a log
+$TORTURE_JITTER_START
+for i in "$@"
+do
+ echo ---- System running test: `uname -a` > $i/kvm-test-1-run-qemu.sh.out
+ echo > $i/kvm-test-1-run-qemu.sh.out
+ kvm-test-1-run-qemu.sh $i >> $i/kvm-test-1-run-qemu.sh.out 2>&1 &
+done
+for i in $runfiles
+do
+ while ls $i > /dev/null 2>&1
+ do
+ :
+ done
+done
+echo ---- All kernel runs complete. `date` | tee -a log
+$TORTURE_JITTER_STOP
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 23/28] torture: Add --duration argument to kvm-again.sh
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (21 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 22/28] torture: Add kvm-again.sh to rerun a previous torture-test paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 24/28] torture: Make kvm-transform.sh update jitter commands paulmck
` (8 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
This commit adds a --duration argument to kvm-again.sh to allow the user
to override the --duration specified for the original kvm.sh run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../testing/selftests/rcutorture/bin/kvm-again.sh | 25 ++++++++++++++++++-
.../selftests/rcutorture/bin/kvm-transform.sh | 29 +++++++++++++++++-----
2 files changed, 47 insertions(+), 7 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 92cf396..4de576495 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -52,6 +52,7 @@ PATH=${KVM}/bin:$PATH; export PATH
. functions.sh
dryrun=
+dur=
default_link="cp -R"
rundir="`pwd`/tools/testing/selftests/rcutorture/res/`date +%Y.%m.%d-%H.%M.%S-again`"
@@ -61,6 +62,7 @@ starttime="`get_starttime`"
usage () {
echo "Usage: $scriptname $oldrun [ arguments ]:"
echo " --dryrun"
+ echo " --duration minutes | <seconds>s | <hours>h | <days>d"
echo " --link hard|soft|copy"
echo " --remote"
echo " --rundir /new/res/path"
@@ -73,6 +75,23 @@ do
--dryrun)
dryrun=1
;;
+ --duration)
+ checkarg --duration "(minutes)" $# "$2" '^[0-9][0-9]*\(s\|m\|h\|d\|\)$' '^error'
+ mult=60
+ if echo "$2" | grep -q 's$'
+ then
+ mult=1
+ elif echo "$2" | grep -q 'h$'
+ then
+ mult=3600
+ elif echo "$2" | grep -q 'd$'
+ then
+ mult=86400
+ fi
+ ts=`echo $2 | sed -e 's/[smhd]$//'`
+ dur=$(($ts*mult))
+ shift
+ ;;
--link)
checkarg --link "hard|soft|copy" "$#" "$2" 'hard\|soft\|copy' '^--'
case "$2" in
@@ -134,7 +153,11 @@ do
cp "$i" $T
qemu_cmd_dir="`dirname "$i"`"
kernel_dir="`echo $qemu_cmd_dir | sed -e 's/\.[0-9]\+$//'`"
- kvm-transform.sh $kernel_dir/bzImage $qemu_cmd_dir/console.log < $T/qemu-cmd > $i
+ kvm-transform.sh $kernel_dir/bzImage $qemu_cmd_dir/console.log $dur < $T/qemu-cmd > $i
+ if test -n "$dur"
+ then
+ echo "# seconds=$dur" >> $i
+ fi
echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
done
grep -v '^#' $T/batches.oldrun | awk '
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
index c45a953..162dddb 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
@@ -3,7 +3,7 @@
#
# Transform a qemu-cmd file to allow reuse.
#
-# Usage: kvm-transform.sh bzImage console.log < qemu-cmd-in > qemu-cmd-out
+# Usage: kvm-transform.sh bzImage console.log [ seconds ] < qemu-cmd-in > qemu-cmd-out
#
# bzImage: Kernel and initrd from the same prior kvm.sh run.
# console.log: File into which to place console output.
@@ -29,20 +29,37 @@ then
echo "Need console log file name."
exit 1
fi
+seconds=$3
+if test -n "$seconds" && echo $seconds | grep -q '[^0-9]'
+then
+ echo "Invalid duration, should be numeric in seconds: '$seconds'"
+ exit 1
+fi
+
+awk -v image="$image" -v consolelog="$consolelog" -v seconds="$seconds" '
+/^#/ {
+ print $0;
+ next;
+}
-awk -v image="$image" -v consolelog="$consolelog" '
{
line = "";
for (i = 1; i <= NF; i++) {
- if (line == "")
+ if ("" seconds != "" && $i ~ /\.shutdown_secs=[0-9]*$/) {
+ sub(/[0-9]*$/, seconds, $i);
+ if (line == "")
+ line = $i;
+ else
+ line = line " " $i;
+ } else if (line == "") {
line = $i;
- else
+ } else {
line = line " " $i;
+ }
if ($i == "-serial") {
i++;
line = line " file:" consolelog;
- }
- if ($i == "-kernel") {
+ } else if ($i == "-kernel") {
i++;
line = line " " image;
}
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 24/28] torture: Make kvm-transform.sh update jitter commands
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (22 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 23/28] torture: Add --duration argument to kvm-again.sh paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 25/28] torture: Make TORTURE_TRUST_MAKE available in kvm-again.sh environment paulmck
` (7 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
When rerunning an old run using kvm-again.sh, the jitter commands
will re-use the original "res" directory. This works, but is clearly
an accident waiting to happen. And this accident will happen with
remote runs, where the original directory lives on some other system.
This commit therefore updates the qemu-cmd commands to use the new res
directory created for this specific run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
.../testing/selftests/rcutorture/bin/kvm-again.sh | 3 ++-
.../selftests/rcutorture/bin/kvm-transform.sh | 23 +++++++++++++++++++---
2 files changed, 22 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 4de576495..21d1e99 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -153,7 +153,8 @@ do
cp "$i" $T
qemu_cmd_dir="`dirname "$i"`"
kernel_dir="`echo $qemu_cmd_dir | sed -e 's/\.[0-9]\+$//'`"
- kvm-transform.sh $kernel_dir/bzImage $qemu_cmd_dir/console.log $dur < $T/qemu-cmd > $i
+ jitter_dir="`dirname "$kernel_dir"`"
+ kvm-transform.sh "$kernel_dir/bzImage" "$qemu_cmd_dir/console.log" "$jitter_dir" $dur < $T/qemu-cmd > $i
if test -n "$dur"
then
echo "# seconds=$dur" >> $i
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
index 162dddb..e9dcbce 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
@@ -3,7 +3,7 @@
#
# Transform a qemu-cmd file to allow reuse.
#
-# Usage: kvm-transform.sh bzImage console.log [ seconds ] < qemu-cmd-in > qemu-cmd-out
+# Usage: kvm-transform.sh bzImage console.log jitter_dir [ seconds ] < qemu-cmd-in > qemu-cmd-out
#
# bzImage: Kernel and initrd from the same prior kvm.sh run.
# console.log: File into which to place console output.
@@ -29,14 +29,31 @@ then
echo "Need console log file name."
exit 1
fi
-seconds=$3
+jitter_dir="$3"
+if test -z "$jitter_dir" || ! test -d "$jitter_dir"
+then
+ echo "Need valid jitter directory: '$jitter_dir'"
+ exit 1
+fi
+seconds="$4"
if test -n "$seconds" && echo $seconds | grep -q '[^0-9]'
then
echo "Invalid duration, should be numeric in seconds: '$seconds'"
exit 1
fi
-awk -v image="$image" -v consolelog="$consolelog" -v seconds="$seconds" '
+awk -v image="$image" -v consolelog="$consolelog" -v jitter_dir="$jitter_dir" \
+ -v seconds="$seconds" '
+/^# TORTURE_JITTER_START=/ {
+ print "# TORTURE_JITTER_START=\". jitterstart.sh " $4 " " jitter_dir " " $6 " " $7;
+ next;
+}
+
+/^# TORTURE_JITTER_STOP=/ {
+ print "# TORTURE_JITTER_STOP=\". jitterstop.sh " " " jitter_dir " " $5;
+ next;
+}
+
/^#/ {
print $0;
next;
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 25/28] torture: Make TORTURE_TRUST_MAKE available in kvm-again.sh environment
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (23 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 24/28] torture: Make kvm-transform.sh update jitter commands paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 26/28] torture: Print proper vmlinux path for kvm-again.sh runs paulmck
` (6 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Because the TORTURE_TRUST_MAKE environment variable is not recorded,
kvm-again.sh runs can result in the parse-build.sh script emitting
false-positive "BUG: TREE03 no build" messages. These messages are
intended to complain about any lack of compiler invocations when the
--trust-make flag is not given to kvm.sh. However, when this flag is
given to kvm.sh (and thus when TORTURE_TRUST_MAKE=y), lack of compiler
invocations is expected behavior when rebuilding from identical source
code.
This commit therefore makes kvm-test-1-run.sh record the value of the
TORTURE_TRUST_MAKE environment variable as an additional comment in the
qemu-cmd file, and also makes kvm-again.sh reconstitute that variable
from that comment.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-again.sh | 5 +++++
tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh | 1 +
2 files changed, 6 insertions(+)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 21d1e99..0407df8 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -161,6 +161,11 @@ do
fi
echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
done
+
+# Extract settings from the last qemu-cmd file transformed above.
+grep '^#' $i | sed -e 's/^# //' > $T/qemu-cmd-settings
+. $T/qemu-cmd-settings
+
grep -v '^#' $T/batches.oldrun | awk '
BEGIN {
oldbatch = 1;
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
index a386ca8d..420ed5c 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run.sh
@@ -204,6 +204,7 @@ echo "# seconds=$seconds" >> $resdir/qemu-cmd
echo "# TORTURE_KCONFIG_GDB_ARG=\"$TORTURE_KCONFIG_GDB_ARG\"" >> $resdir/qemu-cmd
echo "# TORTURE_JITTER_START=\"$TORTURE_JITTER_START\"" >> $resdir/qemu-cmd
echo "# TORTURE_JITTER_STOP=\"$TORTURE_JITTER_STOP\"" >> $resdir/qemu-cmd
+echo "# TORTURE_TRUST_MAKE=\"$TORTURE_TRUST_MAKE\"; export TORTURE_TRUST_MAKE" >> $resdir/qemu-cmd
if test -n "$TORTURE_BUILDONLY"
then
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 26/28] torture: Print proper vmlinux path for kvm-again.sh runs
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (24 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 25/28] torture: Make TORTURE_TRUST_MAKE available in kvm-again.sh environment paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 27/28] torture: Consolidate qemu-cmd duration editing into kvm-transform.sh paulmck
` (5 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
The kvm-again.sh script does not copy over the vmlinux files due to
their large size. This means that a gdb run must use the vmlinux file
from the original "res" directory. This commit therefore finds that
directory and prints it out so that the user can copy and pasted the
gdb command just as for the initial run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-again.sh | 5 ++++-
tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh | 6 +++++-
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 0407df8..75e6afe 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -159,7 +159,10 @@ do
then
echo "# seconds=$dur" >> $i
fi
- echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
+ if test -n "$arg_remote"
+ then
+ echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
+ fi
done
# Extract settings from the last qemu-cmd file transformed above.
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
index 576a9b7..5b1aa2a 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-test-1-run-qemu.sh
@@ -67,7 +67,11 @@ then
base_resdir=`echo $resdir | sed -e 's/\.[0-9]\+$//'`
if ! test -f $base_resdir/vmlinux
then
- base_resdir=/path/to
+ base_resdir="`cat re-run`/$resdir"
+ if ! test -f $base_resdir/vmlinux
+ then
+ base_resdir=/path/to
+ fi
fi
echo Waiting for you to attach a debug session, for example: > /dev/tty
echo " gdb $base_resdir/vmlinux" > /dev/tty
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 27/28] torture: Consolidate qemu-cmd duration editing into kvm-transform.sh
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (25 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 26/28] torture: Print proper vmlinux path for kvm-again.sh runs paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 28/28] torture: Fix kvm.sh --datestamp regex check paulmck
` (4 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Currently, kvm-again.sh updates the duration in the "seconds=" comment
in the qemu-cmd file, but kvm-transform.sh updates the duration in the
actual qemu command arguments. This is an accident waiting to happen.
This commit therefore consolidates these updates into kvm-transform.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm-again.sh | 4 ----
tools/testing/selftests/rcutorture/bin/kvm-transform.sh | 8 ++++++++
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-again.sh b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
index 75e6afe..33ba6b0 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-again.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-again.sh
@@ -155,10 +155,6 @@ do
kernel_dir="`echo $qemu_cmd_dir | sed -e 's/\.[0-9]\+$//'`"
jitter_dir="`dirname "$kernel_dir"`"
kvm-transform.sh "$kernel_dir/bzImage" "$qemu_cmd_dir/console.log" "$jitter_dir" $dur < $T/qemu-cmd > $i
- if test -n "$dur"
- then
- echo "# seconds=$dur" >> $i
- fi
if test -n "$arg_remote"
then
echo "# TORTURE_KCONFIG_GDB_ARG=''" >> $i
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
index e9dcbce..d40b4e6 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm-transform.sh
@@ -44,6 +44,14 @@ fi
awk -v image="$image" -v consolelog="$consolelog" -v jitter_dir="$jitter_dir" \
-v seconds="$seconds" '
+/^# seconds=/ {
+ if (seconds == "")
+ print $0;
+ else
+ print "# seconds=" seconds;
+ next;
+}
+
/^# TORTURE_JITTER_START=/ {
print "# TORTURE_JITTER_START=\". jitterstart.sh " $4 " " jitter_dir " " $6 " " $7;
next;
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH tip/core/rcu 28/28] torture: Fix kvm.sh --datestamp regex check
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (26 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 27/28] torture: Consolidate qemu-cmd duration editing into kvm-transform.sh paulmck
@ 2021-03-04 0:38 ` paulmck
2021-03-04 0:40 ` [PATCH kcsan 1/4] kcsan, debugfs: Move debugfs file creation out of early init paulmck
` (3 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:38 UTC (permalink / raw)
To: rcu
Cc: linux-kernel, kernel-team, mingo, jiangshanlai, akpm,
mathieu.desnoyers, josh, tglx, peterz, rostedt, dhowells,
edumazet, fweisbec, oleg, joel, Paul E. McKenney
From: "Paul E. McKenney" <paulmck@kernel.org>
Some versions of grep are happy to interpret a nonsensically placed "-"
within a "[]" pattern as a dash, while others give an error message.
This commit therefore places the "-" at the end of the expression where
it was supposed to be in the first place.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
tools/testing/selftests/rcutorture/bin/kvm.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh
index 0add163..6bf00a0 100755
--- a/tools/testing/selftests/rcutorture/bin/kvm.sh
+++ b/tools/testing/selftests/rcutorture/bin/kvm.sh
@@ -120,7 +120,7 @@ do
shift
;;
--datestamp)
- checkarg --datestamp "(relative pathname)" "$#" "$2" '^[a-zA-Z0-9._-/]*$' '^--'
+ checkarg --datestamp "(relative pathname)" "$#" "$2" '^[a-zA-Z0-9._/-]*$' '^--'
ds=$2
shift
;;
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH kcsan 1/4] kcsan, debugfs: Move debugfs file creation out of early init
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (27 preceding siblings ...)
2021-03-04 0:38 ` [PATCH tip/core/rcu 28/28] torture: Fix kvm.sh --datestamp regex check paulmck
@ 2021-03-04 0:40 ` paulmck
2021-03-04 0:40 ` [PATCH kcsan 2/4] kcsan: Make test follow KUnit style recommendations paulmck
` (2 subsequent siblings)
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:40 UTC (permalink / raw)
To: linux-kernel, kasan-dev, kernel-team, mingo
Cc: elver, andreyknvl, glider, dvyukov, cai, boqun.feng,
Rafael J. Wysocki, stable, Paul E . McKenney
From: Marco Elver <elver@google.com>
Commit 56348560d495 ("debugfs: do not attempt to create a new file
before the filesystem is initalized") forbids creating new debugfs files
until debugfs is fully initialized. This means that KCSAN's debugfs
file creation, which happened at the end of __init(), no longer works.
And was apparently never supposed to work!
However, there is no reason to create KCSAN's debugfs file so early.
This commit therefore moves its creation to a late_initcall() callback.
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: stable <stable@vger.kernel.org>
Fixes: 56348560d495 ("debugfs: do not attempt to create a new file before the filesystem is initalized")
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/kcsan/core.c | 2 --
kernel/kcsan/debugfs.c | 4 +++-
kernel/kcsan/kcsan.h | 5 -----
3 files changed, 3 insertions(+), 8 deletions(-)
diff --git a/kernel/kcsan/core.c b/kernel/kcsan/core.c
index 3bf98db..23e7acb 100644
--- a/kernel/kcsan/core.c
+++ b/kernel/kcsan/core.c
@@ -639,8 +639,6 @@ void __init kcsan_init(void)
BUG_ON(!in_task());
- kcsan_debugfs_init();
-
for_each_possible_cpu(cpu)
per_cpu(kcsan_rand_state, cpu) = (u32)get_cycles();
diff --git a/kernel/kcsan/debugfs.c b/kernel/kcsan/debugfs.c
index 3c8093a..209ad8d 100644
--- a/kernel/kcsan/debugfs.c
+++ b/kernel/kcsan/debugfs.c
@@ -261,7 +261,9 @@ static const struct file_operations debugfs_ops =
.release = single_release
};
-void __init kcsan_debugfs_init(void)
+static void __init kcsan_debugfs_init(void)
{
debugfs_create_file("kcsan", 0644, NULL, NULL, &debugfs_ops);
}
+
+late_initcall(kcsan_debugfs_init);
diff --git a/kernel/kcsan/kcsan.h b/kernel/kcsan/kcsan.h
index 8d4bf34..87ccdb3 100644
--- a/kernel/kcsan/kcsan.h
+++ b/kernel/kcsan/kcsan.h
@@ -31,11 +31,6 @@ void kcsan_save_irqtrace(struct task_struct *task);
void kcsan_restore_irqtrace(struct task_struct *task);
/*
- * Initialize debugfs file.
- */
-void kcsan_debugfs_init(void);
-
-/*
* Statistics counters displayed via debugfs; should only be modified in
* slow-paths.
*/
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH kcsan 2/4] kcsan: Make test follow KUnit style recommendations
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (28 preceding siblings ...)
2021-03-04 0:40 ` [PATCH kcsan 1/4] kcsan, debugfs: Move debugfs file creation out of early init paulmck
@ 2021-03-04 0:40 ` paulmck
2021-03-04 0:40 ` [PATCH kcsan 3/4] kcsan: Switch to KUNIT_CASE_PARAM for parameterized tests paulmck
2021-03-04 0:40 ` [PATCH kcsan 4/4] kcsan: Add missing license and copyright headers paulmck
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:40 UTC (permalink / raw)
To: linux-kernel, kasan-dev, kernel-team, mingo
Cc: elver, andreyknvl, glider, dvyukov, cai, boqun.feng, Paul E . McKenney
From: Marco Elver <elver@google.com>
Per recently added KUnit style recommendations at
Documentation/dev-tools/kunit/style.rst, make the following changes to
the KCSAN test:
1. Rename 'kcsan-test.c' to 'kcsan_test.c'.
2. Rename suite name 'kcsan-test' to 'kcsan'.
3. Rename CONFIG_KCSAN_TEST to CONFIG_KCSAN_KUNIT_TEST and
default to KUNIT_ALL_TESTS.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/kcsan/Makefile | 4 ++--
kernel/kcsan/{kcsan-test.c => kcsan_test.c} | 2 +-
lib/Kconfig.kcsan | 5 +++--
3 files changed, 6 insertions(+), 5 deletions(-)
rename kernel/kcsan/{kcsan-test.c => kcsan_test.c} (99%)
diff --git a/kernel/kcsan/Makefile b/kernel/kcsan/Makefile
index 65ca553..c2bb07f 100644
--- a/kernel/kcsan/Makefile
+++ b/kernel/kcsan/Makefile
@@ -13,5 +13,5 @@ CFLAGS_core.o := $(call cc-option,-fno-conserve-stack) \
obj-y := core.o debugfs.o report.o
obj-$(CONFIG_KCSAN_SELFTEST) += selftest.o
-CFLAGS_kcsan-test.o := $(CFLAGS_KCSAN) -g -fno-omit-frame-pointer
-obj-$(CONFIG_KCSAN_TEST) += kcsan-test.o
+CFLAGS_kcsan_test.o := $(CFLAGS_KCSAN) -g -fno-omit-frame-pointer
+obj-$(CONFIG_KCSAN_KUNIT_TEST) += kcsan_test.o
diff --git a/kernel/kcsan/kcsan-test.c b/kernel/kcsan/kcsan_test.c
similarity index 99%
rename from kernel/kcsan/kcsan-test.c
rename to kernel/kcsan/kcsan_test.c
index ebe7fd2..f16f632 100644
--- a/kernel/kcsan/kcsan-test.c
+++ b/kernel/kcsan/kcsan_test.c
@@ -1156,7 +1156,7 @@ static void test_exit(struct kunit *test)
}
static struct kunit_suite kcsan_test_suite = {
- .name = "kcsan-test",
+ .name = "kcsan",
.test_cases = kcsan_test_cases,
.init = test_init,
.exit = test_exit,
diff --git a/lib/Kconfig.kcsan b/lib/Kconfig.kcsan
index f271ff5..0440f37 100644
--- a/lib/Kconfig.kcsan
+++ b/lib/Kconfig.kcsan
@@ -69,8 +69,9 @@ config KCSAN_SELFTEST
panic. Recommended to be enabled, ensuring critical functionality
works as intended.
-config KCSAN_TEST
- tristate "KCSAN test for integrated runtime behaviour"
+config KCSAN_KUNIT_TEST
+ tristate "KCSAN test for integrated runtime behaviour" if !KUNIT_ALL_TESTS
+ default KUNIT_ALL_TESTS
depends on TRACEPOINTS && KUNIT
select TORTURE_TEST
help
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH kcsan 3/4] kcsan: Switch to KUNIT_CASE_PARAM for parameterized tests
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (29 preceding siblings ...)
2021-03-04 0:40 ` [PATCH kcsan 2/4] kcsan: Make test follow KUnit style recommendations paulmck
@ 2021-03-04 0:40 ` paulmck
2021-03-04 0:40 ` [PATCH kcsan 4/4] kcsan: Add missing license and copyright headers paulmck
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:40 UTC (permalink / raw)
To: linux-kernel, kasan-dev, kernel-team, mingo
Cc: elver, andreyknvl, glider, dvyukov, cai, boqun.feng, Paul E . McKenney
From: Marco Elver <elver@google.com>
Since KUnit now support parameterized tests via KUNIT_CASE_PARAM, update
KCSAN's test to switch to it for parameterized tests. This simplifies
parameterized tests and gets rid of the "parameters in case name"
workaround (hack).
At the same time, we can increase the maximum number of threads used,
because on systems with too few CPUs, KUnit allows us to now stop at the
maximum useful threads and not unnecessarily execute redundant test
cases with (the same) limited threads as had been the case before.
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
kernel/kcsan/kcsan_test.c | 116 +++++++++++++++++++++-------------------------
1 file changed, 54 insertions(+), 62 deletions(-)
diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c
index f16f632..b71751f 100644
--- a/kernel/kcsan/kcsan_test.c
+++ b/kernel/kcsan/kcsan_test.c
@@ -13,6 +13,8 @@
* Author: Marco Elver <elver@google.com>
*/
+#define pr_fmt(fmt) "kcsan_test: " fmt
+
#include <kunit/test.h>
#include <linux/jiffies.h>
#include <linux/kcsan-checks.h>
@@ -951,22 +953,53 @@ static void test_atomic_builtins(struct kunit *test)
}
/*
- * Each test case is run with different numbers of threads. Until KUnit supports
- * passing arguments for each test case, we encode #threads in the test case
- * name (read by get_num_threads()). [The '-' was chosen as a stylistic
- * preference to separate test name and #threads.]
+ * Generate thread counts for all test cases. Values generated are in interval
+ * [2, 5] followed by exponentially increasing thread counts from 8 to 32.
*
* The thread counts are chosen to cover potentially interesting boundaries and
- * corner cases (range 2-5), and then stress the system with larger counts.
+ * corner cases (2 to 5), and then stress the system with larger counts.
*/
-#define KCSAN_KUNIT_CASE(test_name) \
- { .run_case = test_name, .name = #test_name "-02" }, \
- { .run_case = test_name, .name = #test_name "-03" }, \
- { .run_case = test_name, .name = #test_name "-04" }, \
- { .run_case = test_name, .name = #test_name "-05" }, \
- { .run_case = test_name, .name = #test_name "-08" }, \
- { .run_case = test_name, .name = #test_name "-16" }
+static const void *nthreads_gen_params(const void *prev, char *desc)
+{
+ long nthreads = (long)prev;
+
+ if (nthreads < 0 || nthreads >= 32)
+ nthreads = 0; /* stop */
+ else if (!nthreads)
+ nthreads = 2; /* initial value */
+ else if (nthreads < 5)
+ nthreads++;
+ else if (nthreads == 5)
+ nthreads = 8;
+ else
+ nthreads *= 2;
+ if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
+ /*
+ * Without any preemption, keep 2 CPUs free for other tasks, one
+ * of which is the main test case function checking for
+ * completion or failure.
+ */
+ const long min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
+ const long min_required_cpus = 2 + min_unused_cpus;
+
+ if (num_online_cpus() < min_required_cpus) {
+ pr_err_once("Too few online CPUs (%u < %d) for test\n",
+ num_online_cpus(), min_required_cpus);
+ nthreads = 0;
+ } else if (nthreads >= num_online_cpus() - min_unused_cpus) {
+ /* Use negative value to indicate last param. */
+ nthreads = -(num_online_cpus() - min_unused_cpus);
+ pr_warn_once("Limiting number of threads to %ld (only %d online CPUs)\n",
+ -nthreads, num_online_cpus());
+ }
+ }
+
+ snprintf(desc, KUNIT_PARAM_DESC_SIZE, "threads=%ld", abs(nthreads));
+ return (void *)nthreads;
+}
+
+#define KCSAN_KUNIT_CASE(test_name) KUNIT_CASE_PARAM(test_name, nthreads_gen_params)
static struct kunit_case kcsan_test_cases[] = {
KCSAN_KUNIT_CASE(test_basic),
KCSAN_KUNIT_CASE(test_concurrent_races),
@@ -996,24 +1029,6 @@ static struct kunit_case kcsan_test_cases[] = {
/* ===== End test cases ===== */
-/* Get number of threads encoded in test name. */
-static bool __no_kcsan
-get_num_threads(const char *test, int *nthreads)
-{
- int len = strlen(test);
-
- if (WARN_ON(len < 3))
- return false;
-
- *nthreads = test[len - 1] - '0';
- *nthreads += (test[len - 2] - '0') * 10;
-
- if (WARN_ON(*nthreads < 0))
- return false;
-
- return true;
-}
-
/* Concurrent accesses from interrupts. */
__no_kcsan
static void access_thread_timer(struct timer_list *timer)
@@ -1076,9 +1091,6 @@ static int test_init(struct kunit *test)
if (!torture_init_begin((char *)test->name, 1))
return -EBUSY;
- if (!get_num_threads(test->name, &nthreads))
- goto err;
-
if (WARN_ON(threads))
goto err;
@@ -1087,38 +1099,18 @@ static int test_init(struct kunit *test)
goto err;
}
- if (!IS_ENABLED(CONFIG_PREEMPT) || !IS_ENABLED(CONFIG_KCSAN_INTERRUPT_WATCHER)) {
- /*
- * Without any preemption, keep 2 CPUs free for other tasks, one
- * of which is the main test case function checking for
- * completion or failure.
- */
- const int min_unused_cpus = IS_ENABLED(CONFIG_PREEMPT_NONE) ? 2 : 0;
- const int min_required_cpus = 2 + min_unused_cpus;
+ nthreads = abs((long)test->param_value);
+ if (WARN_ON(!nthreads))
+ goto err;
- if (num_online_cpus() < min_required_cpus) {
- pr_err("%s: too few online CPUs (%u < %d) for test",
- test->name, num_online_cpus(), min_required_cpus);
- goto err;
- } else if (nthreads > num_online_cpus() - min_unused_cpus) {
- nthreads = num_online_cpus() - min_unused_cpus;
- pr_warn("%s: limiting number of threads to %d\n",
- test->name, nthreads);
- }
- }
+ threads = kcalloc(nthreads + 1, sizeof(struct task_struct *), GFP_KERNEL);
+ if (WARN_ON(!threads))
+ goto err;
- if (nthreads) {
- threads = kcalloc(nthreads + 1, sizeof(struct task_struct *),
- GFP_KERNEL);
- if (WARN_ON(!threads))
+ threads[nthreads] = NULL;
+ for (i = 0; i < nthreads; ++i) {
+ if (torture_create_kthread(access_thread, NULL, threads[i]))
goto err;
-
- threads[nthreads] = NULL;
- for (i = 0; i < nthreads; ++i) {
- if (torture_create_kthread(access_thread, NULL,
- threads[i]))
- goto err;
- }
}
torture_init_end();
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
* [PATCH kcsan 4/4] kcsan: Add missing license and copyright headers
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
` (30 preceding siblings ...)
2021-03-04 0:40 ` [PATCH kcsan 3/4] kcsan: Switch to KUNIT_CASE_PARAM for parameterized tests paulmck
@ 2021-03-04 0:40 ` paulmck
31 siblings, 0 replies; 33+ messages in thread
From: paulmck @ 2021-03-04 0:40 UTC (permalink / raw)
To: linux-kernel, kasan-dev, kernel-team, mingo
Cc: elver, andreyknvl, glider, dvyukov, cai, boqun.feng, Paul E . McKenney
From: Marco Elver <elver@google.com>
Adds missing license and/or copyright headers for KCSAN source files.
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
Documentation/dev-tools/kcsan.rst | 3 +++
include/linux/kcsan-checks.h | 6 ++++++
include/linux/kcsan.h | 7 +++++++
kernel/kcsan/atomic.h | 5 +++++
kernel/kcsan/core.c | 5 +++++
kernel/kcsan/debugfs.c | 5 +++++
kernel/kcsan/encoding.h | 5 +++++
kernel/kcsan/kcsan.h | 3 ++-
kernel/kcsan/report.c | 5 +++++
kernel/kcsan/selftest.c | 5 +++++
10 files changed, 48 insertions(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/kcsan.rst b/Documentation/dev-tools/kcsan.rst
index be7a0b0..d85ce23 100644
--- a/Documentation/dev-tools/kcsan.rst
+++ b/Documentation/dev-tools/kcsan.rst
@@ -1,3 +1,6 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. Copyright (C) 2019, Google LLC.
+
The Kernel Concurrency Sanitizer (KCSAN)
========================================
diff --git a/include/linux/kcsan-checks.h b/include/linux/kcsan-checks.h
index cf14840..9fd0ad8 100644
--- a/include/linux/kcsan-checks.h
+++ b/include/linux/kcsan-checks.h
@@ -1,4 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KCSAN access checks and modifiers. These can be used to explicitly check
+ * uninstrumented accesses, or change KCSAN checking behaviour of accesses.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#ifndef _LINUX_KCSAN_CHECKS_H
#define _LINUX_KCSAN_CHECKS_H
diff --git a/include/linux/kcsan.h b/include/linux/kcsan.h
index 53340d8..fc266ec 100644
--- a/include/linux/kcsan.h
+++ b/include/linux/kcsan.h
@@ -1,4 +1,11 @@
/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * The Kernel Concurrency Sanitizer (KCSAN) infrastructure. Public interface and
+ * data structures to set up runtime. See kcsan-checks.h for explicit checks and
+ * modifiers. For more info please see Documentation/dev-tools/kcsan.rst.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#ifndef _LINUX_KCSAN_H
#define _LINUX_KCSAN_H
diff --git a/kernel/kcsan/atomic.h b/kernel/kcsan/atomic.h
index 75fe701..530ae1b 100644
--- a/kernel/kcsan/atomic.h
+++ b/kernel/kcsan/atomic.h
@@ -1,4 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Rules for implicitly atomic memory accesses.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#ifndef _KERNEL_KCSAN_ATOMIC_H
#define _KERNEL_KCSAN_ATOMIC_H
diff --git a/kernel/kcsan/core.c b/kernel/kcsan/core.c
index 23e7acb..45c821d 100644
--- a/kernel/kcsan/core.c
+++ b/kernel/kcsan/core.c
@@ -1,4 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
+/*
+ * KCSAN core runtime.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#define pr_fmt(fmt) "kcsan: " fmt
diff --git a/kernel/kcsan/debugfs.c b/kernel/kcsan/debugfs.c
index 209ad8d..c1dd02f 100644
--- a/kernel/kcsan/debugfs.c
+++ b/kernel/kcsan/debugfs.c
@@ -1,4 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
+/*
+ * KCSAN debugfs interface.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#define pr_fmt(fmt) "kcsan: " fmt
diff --git a/kernel/kcsan/encoding.h b/kernel/kcsan/encoding.h
index 7ee4055..170a2bb 100644
--- a/kernel/kcsan/encoding.h
+++ b/kernel/kcsan/encoding.h
@@ -1,4 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * KCSAN watchpoint encoding.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#ifndef _KERNEL_KCSAN_ENCODING_H
#define _KERNEL_KCSAN_ENCODING_H
diff --git a/kernel/kcsan/kcsan.h b/kernel/kcsan/kcsan.h
index 87ccdb3..9881099 100644
--- a/kernel/kcsan/kcsan.h
+++ b/kernel/kcsan/kcsan.h
@@ -1,8 +1,9 @@
/* SPDX-License-Identifier: GPL-2.0 */
-
/*
* The Kernel Concurrency Sanitizer (KCSAN) infrastructure. For more info please
* see Documentation/dev-tools/kcsan.rst.
+ *
+ * Copyright (C) 2019, Google LLC.
*/
#ifndef _KERNEL_KCSAN_KCSAN_H
diff --git a/kernel/kcsan/report.c b/kernel/kcsan/report.c
index d3bf87e..13dce3c 100644
--- a/kernel/kcsan/report.c
+++ b/kernel/kcsan/report.c
@@ -1,4 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
+/*
+ * KCSAN reporting.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#include <linux/debug_locks.h>
#include <linux/delay.h>
diff --git a/kernel/kcsan/selftest.c b/kernel/kcsan/selftest.c
index 9014a3a..7f29cb0 100644
--- a/kernel/kcsan/selftest.c
+++ b/kernel/kcsan/selftest.c
@@ -1,4 +1,9 @@
// SPDX-License-Identifier: GPL-2.0
+/*
+ * KCSAN short boot-time selftests.
+ *
+ * Copyright (C) 2019, Google LLC.
+ */
#define pr_fmt(fmt) "kcsan: " fmt
--
2.9.5
^ permalink raw reply related [flat|nested] 33+ messages in thread
end of thread, other threads:[~2021-03-04 1:09 UTC | newest]
Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-04 0:37 [PATCH tip/core/rcu 0/28] Torture-test scripting updates for v5.13 Paul E. McKenney
2021-03-04 0:37 ` [PATCH tip/core/rcu 01/28] torturescript: Don't rerun failed rcutorture builds paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 02/28] torture: Allow 1G of memory for torture.sh kvfree testing paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 03/28] torture: Provide bare-metal modprobe-based advice paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 04/28] torture: Improve readability of the testid.txt file paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 05/28] rcuscale: Disable verbose torture-test output paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 06/28] refscale: " paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 07/28] torture: Move build/run synchronization files into scenario directories paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 08/28] torture: Use file-based protocol to mark batch's runs complete paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 09/28] torture: Use "jittering" file to control jitter.sh execution paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 10/28] torture: Eliminate jitter_pids file paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 11/28] torture: Reverse jittering and duration parameters for jitter.sh paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 12/28] torture: Abstract jitter.sh start/stop into scripts paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 13/28] torture: Record TORTURE_KCONFIG_GDB_ARG in qemu-cmd paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 14/28] torture: Extract kvm-test-1-run-qemu.sh from kvm-test-1-run.sh paulmck
2021-03-04 0:37 ` [PATCH tip/core/rcu 15/28] torture: Record jitter start/stop commands paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 16/28] torture: Record kvm-test-1-run.sh and kvm-test-1-run-qemu.sh PIDs paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 17/28] torture: Remove no-mpstat error message paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 18/28] torture: Rename SRCU-t and SRCU-u to avoid lowercase characters paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 19/28] torture: Make upper-case-only no-dot no-slash scenario names official paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 20/28] torture: De-capitalize TORTURE_SUITE paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 21/28] torture: Create a "batches" file for build reuse paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 22/28] torture: Add kvm-again.sh to rerun a previous torture-test paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 23/28] torture: Add --duration argument to kvm-again.sh paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 24/28] torture: Make kvm-transform.sh update jitter commands paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 25/28] torture: Make TORTURE_TRUST_MAKE available in kvm-again.sh environment paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 26/28] torture: Print proper vmlinux path for kvm-again.sh runs paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 27/28] torture: Consolidate qemu-cmd duration editing into kvm-transform.sh paulmck
2021-03-04 0:38 ` [PATCH tip/core/rcu 28/28] torture: Fix kvm.sh --datestamp regex check paulmck
2021-03-04 0:40 ` [PATCH kcsan 1/4] kcsan, debugfs: Move debugfs file creation out of early init paulmck
2021-03-04 0:40 ` [PATCH kcsan 2/4] kcsan: Make test follow KUnit style recommendations paulmck
2021-03-04 0:40 ` [PATCH kcsan 3/4] kcsan: Switch to KUNIT_CASE_PARAM for parameterized tests paulmck
2021-03-04 0:40 ` [PATCH kcsan 4/4] kcsan: Add missing license and copyright headers paulmck
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).