git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: git@vger.kernel.org
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"John Cai" <johncai86@gmail.com>,
	"Elijah Newren" <newren@gmail.com>,
	"Derrick Stolee" <stolee@gmail.com>,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: [PATCH 0/7] test-lib-functions: a better "test_expect_failure"
Date: Fri, 18 Mar 2022 01:33:55 +0100	[thread overview]
Message-ID: <cover-0.7-00000000000-20220318T002951Z-avarab@gmail.com> (raw)

As explained in detail in 1/7 there's several major shortcomings to
"test_expect_failure", this series adds a "test_expect_todo".

It allows us to assert exactly how something that fails .. fails (no
more conflating segfaults for "exit 1"), but also emits all the right
output & metadata about the test being "TODO" that
"test_expect_failure" does.

The rest of the series then expands on that to convert several more
tests, along with adding helpers like:

    todo_test_cmp want expect actual

Where the "want" is what we want once we'd mark the test as
"test_expect_success", but the "expect" is what we get right now with
the failing TODO test.

This series has a (rather trivial) textual conflict with
js/ci-github-workflow-markup in "seen".

Ævar Arnfjörð Bjarmason (7):
  test-lib: add a "test_expect_todo", similar to "test_expect_failure"
  test-lib-functions: add and use a "test_todo" helper
  tests: allow test_* in "test_must_fail_acceptable" for "test_todo"
  test-lib-functions: add and use a "todo_test_cmp" helper
  test-lib-functions: add and use a "todo_test_path" helper
  test-lib-functions: make test_todo support a --reset
  sparse tests: convert a TODO test to use "test_expect_todo"

 t/README                           |  20 +++-
 t/t0000-basic.sh                   |  48 +++++++-
 t/t1021-rerere-in-workdir.sh       |   9 +-
 t/t1060-object-corruption.sh       |   4 +-
 t/t1091-sparse-checkout-builtin.sh |  17 ++-
 t/t1309-early-config.sh            |  14 ++-
 t/t1410-reflog.sh                  |   4 +-
 t/t3600-rm.sh                      |  23 +++-
 t/t6403-merge-file.sh              |  32 +++++-
 t/t7814-grep-recurse-submodules.sh |  60 ++++++----
 t/t7815-grep-binary.sh             |   4 +-
 t/test-lib-functions.sh            | 172 ++++++++++++++++++++++++++++-
 t/test-lib.sh                      |  32 +++++-
 13 files changed, 379 insertions(+), 60 deletions(-)

-- 
2.35.1.1436.g756b814e59f


             reply	other threads:[~2022-03-18  0:34 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-03-18  0:33 Ævar Arnfjörð Bjarmason [this message]
2022-03-18  0:33 ` [PATCH 1/7] test-lib: add a "test_expect_todo", similar to "test_expect_failure" Ævar Arnfjörð Bjarmason
2022-03-18 18:59   ` Junio C Hamano
2022-03-18 20:50     ` Junio C Hamano
2022-03-18 23:07       ` Ævar Arnfjörð Bjarmason
2022-03-19  7:12         ` Junio C Hamano
2022-03-19 11:11           ` Ævar Arnfjörð Bjarmason
2022-03-20 15:13             ` Phillip Wood
2022-03-20 18:07               ` Junio C Hamano
2022-03-22 14:43                 ` Derrick Stolee
2022-03-23 22:13                   ` Junio C Hamano
2022-03-24 11:24                     ` Phillip Wood
2022-03-18  0:33 ` [PATCH 2/7] test-lib-functions: add and use a "test_todo" helper Ævar Arnfjörð Bjarmason
2022-03-18  0:33 ` [PATCH 3/7] tests: allow test_* in "test_must_fail_acceptable" for "test_todo" Ævar Arnfjörð Bjarmason
2022-03-18  0:33 ` [PATCH 4/7] test-lib-functions: add and use a "todo_test_cmp" helper Ævar Arnfjörð Bjarmason
2022-03-18  0:34 ` [PATCH 5/7] test-lib-functions: add and use a "todo_test_path" helper Ævar Arnfjörð Bjarmason
2022-03-18  0:34 ` [PATCH 6/7] test-lib-functions: make test_todo support a --reset Ævar Arnfjörð Bjarmason
2022-03-18  0:34 ` [PATCH 7/7] sparse tests: convert a TODO test to use "test_expect_todo" Ævar Arnfjörð Bjarmason

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover-0.7-00000000000-20220318T002951Z-avarab@gmail.com \
    --to=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=johncai86@gmail.com \
    --cc=newren@gmail.com \
    --cc=stolee@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).