All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: zlang@redhat.com, djwong@kernel.org
Cc: Leah Rumancik <leah.rumancik@gmail.com>,
	Leah Rumancik <leah.rumancik@gmail.com>,
	linux-xfs@vger.kernel.org, fstests@vger.kernel.org, guan@eryu.me,
	leah.rumancik@gmail.com, quwenruo.btrfs@gmx.com, tytso@mit.edu
Subject: [PATCH 01/15] check: generate section reports between tests
Date: Tue, 14 Mar 2023 17:52:36 -0700	[thread overview]
Message-ID: <167884155648.2482843.4182317488319546153.stgit@magnolia> (raw)
In-Reply-To: <167884155064.2482843.4310780034948240980.stgit@magnolia>

From: Darrick J. Wong <djwong@kernel.org>

Generate the section report between tests so that the summary report
always reflects the outcome of the most recent test.  Two usecases are
envisioned here -- if a cluster-based test runner anticipates that the
testrun could crash the VM, they can set REPORT_DIR to (say) an NFS
mount to preserve the intermediate results.  If the VM does indeed
crash, the scheduler can examine the state of the crashed VM and move
the tests to another VM.  The second usecase is a reporting agent that
runs in the VM to upload live results to a test dashboard.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
Reviewed-by: Leah Rumancik <leah.rumancik@gmail.com>
Tested-by: Leah Rumancik <leah.rumancik@gmail.com>
---
 check |    9 +++++++++
 1 file changed, 9 insertions(+)


diff --git a/check b/check
index 0bf5b22e06..14b398fd73 100755
--- a/check
+++ b/check
@@ -844,6 +844,15 @@ function run_section()
 		fi
 		seqres="$REPORT_DIR/$seqnum"
 
+		# Generate the entire section report with whatever test results
+		# we have so far.  Leave the $sect_time parameter empty so that
+		# it's a little more obvious that this test run is incomplete.
+		if $do_report; then
+			_make_section_report "$section" "${#try[*]}" \
+					     "${#bad[*]}" "${#notrun[*]}" \
+					     "" &> /dev/null
+		fi
+
 		mkdir -p $RESULT_DIR
 		rm -f ${RESULT_DIR}/require_scratch*
 		rm -f ${RESULT_DIR}/require_test*


  reply	other threads:[~2023-03-15  0:52 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-15  0:52 [PATCHSET v3 00/15] fstests: improve junit xml reporting Darrick J. Wong
2023-03-15  0:52 ` Darrick J. Wong [this message]
2023-03-15  0:52 ` [PATCH 02/15] report: derive an xml schema for the xunit report Darrick J. Wong
2023-03-15  0:52 ` [PATCH 03/15] report: capture the time zone in the test report timestamp Darrick J. Wong
2023-03-15  0:52 ` [PATCH 04/15] report: clarify the meaning of the timestamp attribute Darrick J. Wong
2023-03-15  0:52 ` [PATCH 05/15] report: record fstests start and report generation timestamps Darrick J. Wong
2023-03-15  0:53 ` [PATCH 06/15] report: encode cdata sections correctly Darrick J. Wong
2023-03-15  0:53 ` [PATCH 07/15] report: encode the kernel log as a separate xml element Darrick J. Wong
2023-03-15  0:53 ` [PATCH 08/15] report: sort properties by name Darrick J. Wong
2023-03-15  0:53 ` [PATCH 09/15] report: pass property value to _xunit_add_property Darrick J. Wong
2023-03-15  0:53 ` [PATCH 10/15] report: encode xml entities in property values Darrick J. Wong
2023-03-15  0:53 ` [PATCH 11/15] report: collect basic information about a test run Darrick J. Wong
2023-03-15  0:53 ` [PATCH 12/15] report: record optional environment variables Darrick J. Wong
2023-03-15  0:53 ` [PATCH 13/15] report: record xfs-specific information about a test run Darrick J. Wong
2023-03-15  0:53 ` [PATCH 14/15] report: record ext*-specific " Darrick J. Wong
2023-03-15  0:53 ` [PATCH 15/15] report: allow test runners to inject arbitrary values Darrick J. Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=167884155648.2482843.4182317488319546153.stgit@magnolia \
    --to=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=guan@eryu.me \
    --cc=leah.rumancik@gmail.com \
    --cc=linux-xfs@vger.kernel.org \
    --cc=quwenruo.btrfs@gmx.com \
    --cc=tytso@mit.edu \
    --cc=zlang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.