From: Patrick Steinhardt <ps@pks.im>
To: git@vger.kernel.org
Subject: [PATCH 1/8] perf: fix when running with TEST_OUTPUT_DIRECTORY
Date: Wed, 19 May 2021 21:13:22 +0200 [thread overview]
Message-ID: <5488f0f4888fbc46378e2777d7daba1ec627b6c6.1621451532.git.ps@pks.im> (raw)
In-Reply-To: <cover.1621451532.git.ps@pks.im>
[-- Attachment #1: Type: text/plain, Size: 5317 bytes --]
When the TEST_OUTPUT_DIRECTORY is defined, then all test data will be
written in that directory instead of the default directory located in
"t/". While this works as expected for our normal tests, performance
tests fail to locate and aggregate performance data because they don't
know to handle TEST_OUTPUT_DIRECTORY correctly and always look at the
default location.
Fix the issue by adding a `--results-dir` parameter to "aggregate.perl"
which identifies the directory where results are and by making the "run"
script awake of the TEST_OUTPUT_DIRECTORY variable.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
t/perf/aggregate.perl | 8 ++++++--
t/perf/perf-lib.sh | 4 ++--
t/perf/run | 25 ++++++++++++++++---------
3 files changed, 24 insertions(+), 13 deletions(-)
diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index 14e4cda287..5d4964c5c6 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -58,6 +58,7 @@ sub usage {
Options:
--codespeed * Format output for Codespeed
--reponame <str> * Send given reponame to codespeed
+ --results-dir <str> * Directory where test results are located
--sort-by <str> * Sort output (only "regression" criteria is supported)
--subsection <str> * Use results from given subsection
@@ -90,12 +91,13 @@ sub sane_backticks {
}
my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests,
- $codespeed, $sortby, $subsection, $reponame);
+ $codespeed, $sortby, $subsection, $reponame, $resultsdir);
Getopt::Long::Configure qw/ require_order /;
my $rc = GetOptions("codespeed" => \$codespeed,
"reponame=s" => \$reponame,
+ "results-dir=s" => \$resultsdir,
"sort-by=s" => \$sortby,
"subsection=s" => \$subsection);
usage() unless $rc;
@@ -137,7 +139,9 @@ sub sane_backticks {
@tests = glob "p????-*.sh";
}
-my $resultsdir = "test-results";
+if (not $resultsdir) {
+ $resultsdir = "test-results";
+}
if (! $subsection and
exists $ENV{GIT_PERF_SUBSECTION} and
diff --git a/t/perf/perf-lib.sh b/t/perf/perf-lib.sh
index 601d9f67dd..8ca6dd0297 100644
--- a/t/perf/perf-lib.sh
+++ b/t/perf/perf-lib.sh
@@ -45,7 +45,7 @@ export TEST_DIRECTORY TRASH_DIRECTORY GIT_BUILD_DIR GIT_TEST_CMP
MODERN_GIT=$GIT_BUILD_DIR/bin-wrappers/git
export MODERN_GIT
-perf_results_dir=$TEST_OUTPUT_DIRECTORY/test-results
+perf_results_dir=$TEST_RESULTS_DIR
test -n "$GIT_PERF_SUBSECTION" && perf_results_dir="$perf_results_dir/$GIT_PERF_SUBSECTION"
mkdir -p "$perf_results_dir"
rm -f "$perf_results_dir"/$(basename "$0" .sh).subtests
@@ -253,7 +253,7 @@ test_size () {
# and does it after running everything)
test_at_end_hook_ () {
if test -z "$GIT_PERF_AGGREGATING_LATER"; then
- ( cd "$TEST_DIRECTORY"/perf && ./aggregate.perl $(basename "$0") )
+ ( cd "$TEST_DIRECTORY"/perf && ./aggregate.perl --results-dir="$TEST_RESULTS_DIR" $(basename "$0") )
fi
}
diff --git a/t/perf/run b/t/perf/run
index c7b86104e1..03128d440a 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -188,10 +188,10 @@ run_subsection () {
if test -z "$GIT_PERF_SEND_TO_CODESPEED"
then
- ./aggregate.perl $codespeed_opt "$@"
+ ./aggregate.perl --results-dir="$TEST_RESULTS_DIR" $codespeed_opt "$@"
else
- json_res_file="test-results/$GIT_PERF_SUBSECTION/aggregate.json"
- ./aggregate.perl --codespeed "$@" | tee "$json_res_file"
+ json_res_file=""$TEST_RESULTS_DIR"/$GIT_PERF_SUBSECTION/aggregate.json"
+ ./aggregate.perl --results-dir="$TEST_RESULTS_DIR" --codespeed "$@" | tee "$json_res_file"
send_data_url="$GIT_PERF_SEND_TO_CODESPEED/result/add/json/"
curl -v --request POST --data-urlencode "json=$(cat "$json_res_file")" "$send_data_url"
fi
@@ -203,10 +203,17 @@ get_var_from_env_or_config "GIT_PERF_SEND_TO_CODESPEED" "perf" "sendToCodespeed"
cd "$(dirname $0)"
. ../../GIT-BUILD-OPTIONS
-mkdir -p test-results
-get_subsections "perf" >test-results/run_subsections.names
+if test -n "$TEST_OUTPUT_DIRECTORY"
+then
+ TEST_RESULTS_DIR="$TEST_OUTPUT_DIRECTORY/test-results"
+else
+ TEST_RESULTS_DIR=test-results
+fi
-if test $(wc -l <test-results/run_subsections.names) -eq 0
+mkdir -p "$TEST_RESULTS_DIR"
+get_subsections "perf" >"$TEST_RESULTS_DIR"/run_subsections.names
+
+if test $(wc -l <"$TEST_RESULTS_DIR"/run_subsections.names) -eq 0
then
if test -n "$GIT_PERF_SUBSECTION"
then
@@ -222,10 +229,10 @@ then
)
elif test -n "$GIT_PERF_SUBSECTION"
then
- egrep "^$GIT_PERF_SUBSECTION\$" test-results/run_subsections.names >/dev/null ||
+ egrep "^$GIT_PERF_SUBSECTION\$" "$TEST_RESULTS_DIR"/run_subsections.names >/dev/null ||
die "subsection '$GIT_PERF_SUBSECTION' not found in '$GIT_PERF_CONFIG_FILE'"
- egrep "^$GIT_PERF_SUBSECTION\$" test-results/run_subsections.names | while read -r subsec
+ egrep "^$GIT_PERF_SUBSECTION\$" "$TEST_RESULTS_DIR"/run_subsections.names | while read -r subsec
do
(
GIT_PERF_SUBSECTION="$subsec"
@@ -243,5 +250,5 @@ else
echo "======== Run for subsection '$GIT_PERF_SUBSECTION' ========"
run_subsection "$@"
)
- done <test-results/run_subsections.names
+ done <"$TEST_RESULTS_DIR"/run_subsections.names
fi
--
2.31.1
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-05-19 19:13 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-19 19:13 [PATCH 0/8] Speed up connectivity checks via quarantine dir Patrick Steinhardt
2021-05-19 19:13 ` Patrick Steinhardt [this message]
2021-05-20 2:03 ` [PATCH 1/8] perf: fix when running with TEST_OUTPUT_DIRECTORY Chris Torek
2021-05-19 19:13 ` [PATCH 2/8] p5400: add perf tests for git-receive-pack(1) Patrick Steinhardt
2021-05-20 2:09 ` Chris Torek
2021-05-20 17:04 ` Jeff King
2021-05-21 15:03 ` SZEDER Gábor
2021-05-19 19:13 ` [PATCH 3/8] tmp-objdir: expose function to retrieve path Patrick Steinhardt
2021-05-20 0:16 ` Elijah Newren
2021-05-19 19:13 ` [PATCH 4/8] packfile: have `for_each_file_in_pack_dir()` return error codes Patrick Steinhardt
2021-05-19 19:13 ` [PATCH 5/8] object-file: allow reading loose objects without reading their contents Patrick Steinhardt
2021-05-19 19:13 ` [PATCH 6/8] connected: implement connectivity check via temporary object dirs Patrick Steinhardt
2021-05-19 19:13 ` [PATCH 7/8] receive-pack: skip connectivity checks on delete-only commands Patrick Steinhardt
2021-05-21 18:53 ` Felipe Contreras
2021-05-27 14:38 ` Jeff King
2021-05-19 19:13 ` [PATCH 8/8] receive-pack: check connectivity via quarantined objects Patrick Steinhardt
2021-05-20 2:19 ` [PATCH 0/8] Speed up connectivity checks via quarantine dir Chris Torek
2021-05-20 16:50 ` Jeff King
2021-05-20 21:45 ` Junio C Hamano
2021-05-21 9:30 ` Jeff King
2021-05-21 9:42 ` Patrick Steinhardt
2021-05-21 11:20 ` Ævar Arnfjörð Bjarmason
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5488f0f4888fbc46378e2777d7daba1ec627b6c6.1621451532.git.ps@pks.im \
--to=ps@pks.im \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.