Git Mailing List Archive on lore.kernel.org
 help / color / Atom feed
From: Johannes Sixt <j6t@kdbg.org>
To: Denton Liu <liu.denton@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>
Subject: Re: [PATCH 03/15] t0000: replace test_must_fail with ! for run_sub_test_lib_test()
Date: Tue, 17 Dec 2019 21:23:57 +0100
Message-ID: <de3d194a-bb05-8049-673b-267f53a271d6@kdbg.org> (raw)
In-Reply-To: <fcfccebd7aaf120674691ba92a657802c2482d7e.1576583819.git.liu.denton@gmail.com>

Am 17.12.19 um 13:01 schrieb Denton Liu:
> The test_must_fail function should only be used for git commands since
> we should assume that external commands work sanely. We use
> test_must_fail to test run_sub_test_lib_test() but that function does
> not invoke any git commands internally. Replace these instances of
> `test_must_fail` with `!`.
> 
> Signed-off-by: Denton Liu <liu.denton@gmail.com>
> ---
>  t/t0000-basic.sh | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
> index 8a81a249d0..d60ad4b78b 100755
> --- a/t/t0000-basic.sh
> +++ b/t/t0000-basic.sh
> @@ -155,7 +155,7 @@ test_expect_success 'pretend we have a fully passing test suite' "
>  "
>  
>  test_expect_success 'pretend we have a partially passing test suite' "
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		partial-pass '2/3 tests passing' <<-\\EOF &&

It is a very uncommon situation (read: I doubt that it ever occurs) in
our test suite that we expect a shell function to fail, but that we do
*not* care at all which of its sub-commands actually failed. We actually
do care which sub-command failed. Therefore, we have, e.g., the idiom
"test_i18n_grep ! ...". And in fact, in the case of
run_sub_test_lib_test we have the form run_sub_test_lib_test_err to
check for error exit in the subordinate test. All of the cases you
change here should use it.

>  	test_expect_success 'passing test #1' 'true'
>  	test_expect_success 'failing test #2' 'false'
> @@ -219,7 +219,7 @@ test_expect_success 'pretend we have fixed one of two known breakages (run in su
>  "
>  
>  test_expect_success 'pretend we have a pass, fail, and known breakage' "
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		mixed-results1 'mixed results #1' <<-\\EOF &&
>  	test_expect_success 'passing test' 'true'
>  	test_expect_success 'failing test' 'false'
> @@ -238,7 +238,7 @@ test_expect_success 'pretend we have a pass, fail, and known breakage' "
>  "
>  
>  test_expect_success 'pretend we have a mix of all possible results' "
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		mixed-results2 'mixed results #2' <<-\\EOF &&
>  	test_expect_success 'passing test' 'true'
>  	test_expect_success 'passing test' 'true'
> @@ -274,7 +274,7 @@ test_expect_success 'pretend we have a mix of all possible results' "
>  "
>  
>  test_expect_success C_LOCALE_OUTPUT 'test --verbose' '
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		t1234-verbose "test verbose" --verbose <<-\EOF &&
>  	test_expect_success "passing test" true
>  	test_expect_success "test with output" "echo foo"
> @@ -301,7 +301,7 @@ test_expect_success C_LOCALE_OUTPUT 'test --verbose' '
>  '
>  
>  test_expect_success 'test --verbose-only' '
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		t2345-verbose-only-2 "test verbose-only=2" \
>  		--verbose-only=2 <<-\EOF &&
>  	test_expect_success "passing test" true
> @@ -834,7 +834,7 @@ then
>  fi
>  
>  test_expect_success 'tests clean up even on failures' "
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		failing-cleanup 'Failing tests with cleanup commands' <<-\\EOF &&
>  	test_expect_success 'tests clean up even after a failure' '
>  		touch clean-after-failure &&
> @@ -863,7 +863,7 @@ test_expect_success 'tests clean up even on failures' "
>  "
>  
>  test_expect_success 'test_atexit is run' "
> -	test_must_fail run_sub_test_lib_test \
> +	! run_sub_test_lib_test \
>  		atexit-cleanup 'Run atexit commands' -i <<-\\EOF &&
>  	test_expect_success 'tests clean up even after a failure' '
>  		> ../../clean-atexit &&
> 

-- Hannes

  reply index

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-17 12:01 [PATCH 00/15] t: replace incorrect test_must_fail usage (part 1) Denton Liu
2019-12-17 12:01 ` [PATCH 01/15] test-lib-functions: introduce test_non_git_might_fail() Denton Liu
2019-12-17 12:01 ` [PATCH 02/15] t/lib-git-p4: use test_path_is_missing() Denton Liu
2019-12-17 12:01 ` [PATCH 03/15] t0000: replace test_must_fail with ! for run_sub_test_lib_test() Denton Liu
2019-12-17 20:23   ` Johannes Sixt [this message]
2019-12-17 12:01 ` [PATCH 04/15] t0003: use named parameters in attr_check() Denton Liu
2019-12-17 18:35   ` Junio C Hamano
2019-12-17 18:43   ` Eric Sunshine
2019-12-17 12:01 ` [PATCH 05/15] t0003: use test_must_be_empty() Denton Liu
2019-12-17 12:01 ` [PATCH 06/15] t0003: don't use `test_must_fail attr_check` Denton Liu
2019-12-17 18:44   ` Junio C Hamano
2019-12-17 12:01 ` [PATCH 07/15] t0020: drop redirections to /dev/null Denton Liu
2019-12-17 19:32   ` Eric Sunshine
2019-12-17 12:01 ` [PATCH 08/15] t0020: s/test_must_fail has_cr/! has_cr/ Denton Liu
2019-12-17 19:36   ` Eric Sunshine
2019-12-17 12:01 ` [PATCH 09/15] t0020: use ! check_packed_refs_marked Denton Liu
2019-12-17 12:01 ` [PATCH 10/15] t1306: convert `test_might_fail rm` to `rm -f` Denton Liu
2019-12-17 18:55   ` Junio C Hamano
2019-12-17 12:01 ` [PATCH 11/15] t1307: reorder `nongit test_must_fail` Denton Liu
2019-12-17 12:01 ` [PATCH 12/15] t1409: let sed open its own files Denton Liu
2019-12-17 18:57   ` Junio C Hamano
2019-12-17 12:01 ` [PATCH 13/15] t1409: use test_path_is_missing() Denton Liu
2019-12-17 19:01   ` Junio C Hamano
2019-12-17 12:01 ` [PATCH 14/15] t1501: remove use of `test_might_fail cp` Denton Liu
2019-12-17 19:46   ` Eric Sunshine
2019-12-17 12:01 ` [PATCH 15/15] t1507: teach full_name() to accept `!` arg Denton Liu
2019-12-17 19:54   ` Eric Sunshine
2019-12-17 18:28 ` [PATCH 00/15] t: replace incorrect test_must_fail usage (part 1) Junio C Hamano
2019-12-19 22:22 ` [PATCH v2 00/16] " Denton Liu
2019-12-19 22:22   ` [PATCH v2 01/16] test-lib-functions: introduce test_non_git_might_fail() Denton Liu
2019-12-19 22:22   ` [PATCH v2 02/16] t/lib-git-p4: use test_path_is_missing() Denton Liu
2019-12-19 22:22   ` [PATCH v2 03/16] t0000: replace test_must_fail with run_sub_test_lib_test_err() Denton Liu
2019-12-19 22:22   ` [PATCH v2 04/16] t0003: use named parameters in attr_check() Denton Liu
2019-12-19 22:22   ` [PATCH v2 05/16] t0003: use test_must_be_empty() Denton Liu
2019-12-19 22:22   ` [PATCH v2 06/16] t0003: don't use `test_must_fail attr_check` Denton Liu
2019-12-19 22:22   ` [PATCH v2 07/16] t0020: don't use `test_must_fail has_cr` Denton Liu
2019-12-19 22:22   ` [PATCH v2 08/16] t0020: use ! check_packed_refs_marked Denton Liu
2019-12-19 22:22   ` [PATCH v2 09/16] t1306: convert `test_might_fail rm` to `rm -f` Denton Liu
2019-12-19 22:22   ` [PATCH v2 10/16] t1307: reorder `nongit test_must_fail` Denton Liu
2019-12-19 22:22   ` [PATCH v2 11/16] t1409: let sed open its own input file Denton Liu
2019-12-19 22:22   ` [PATCH v2 12/16] t1409: use test_path_is_missing() Denton Liu
2019-12-19 22:22   ` [PATCH v2 13/16] t1501: remove use of `test_might_fail cp` Denton Liu
2019-12-19 22:52     ` Eric Sunshine
2019-12-19 23:19       ` Denton Liu
2019-12-20 17:32         ` Eric Sunshine
2019-12-19 22:22   ` [PATCH v2 14/16] t1507: stop losing return codes of git commands Denton Liu
2019-12-19 22:22   ` [PATCH v2 15/16] t1507: run commands within test_expect_success Denton Liu
2019-12-19 22:22   ` [PATCH v2 16/16] t1507: inline full_name() Denton Liu
2019-12-20 18:15   ` [PATCH v3 00/15] t: replace incorrect test_must_fail usage (part 1) Denton Liu
2019-12-20 18:15     ` [PATCH v3 01/15] t/lib-git-p4: use test_path_is_missing() Denton Liu
2019-12-20 18:15     ` [PATCH v3 02/15] t0000: replace test_must_fail with run_sub_test_lib_test_err() Denton Liu
2019-12-20 18:15     ` [PATCH v3 03/15] t0003: use named parameters in attr_check() Denton Liu
2019-12-20 18:15     ` [PATCH v3 04/15] t0003: use test_must_be_empty() Denton Liu
2019-12-20 18:15     ` [PATCH v3 05/15] t0003: don't use `test_must_fail attr_check` Denton Liu
2019-12-20 18:15     ` [PATCH v3 06/15] t0020: don't use `test_must_fail has_cr` Denton Liu
2019-12-20 18:15     ` [PATCH v3 07/15] t0020: use ! check_packed_refs_marked Denton Liu
2019-12-20 18:15     ` [PATCH v3 08/15] t1306: convert `test_might_fail rm` to `rm -f` Denton Liu
2019-12-20 18:15     ` [PATCH v3 09/15] t1307: reorder `nongit test_must_fail` Denton Liu
2019-12-20 18:15     ` [PATCH v3 10/15] t1409: let sed open its own input file Denton Liu
2019-12-20 18:15     ` [PATCH v3 11/15] t1409: use test_path_is_missing() Denton Liu
2019-12-20 18:15     ` [PATCH v3 12/15] t1501: remove use of `test_might_fail cp` Denton Liu
2019-12-20 18:16     ` [PATCH v3 13/15] t1507: stop losing return codes of git commands Denton Liu
2019-12-20 18:16     ` [PATCH v3 14/15] t1507: run commands within test_expect_success Denton Liu
2019-12-20 18:16     ` [PATCH v3 15/15] t1507: inline full_name() Denton Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=de3d194a-bb05-8049-673b-267f53a271d6@kdbg.org \
    --to=j6t@kdbg.org \
    --cc=git@vger.kernel.org \
    --cc=liu.denton@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Mailing List Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/git/0 git/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 git git/ https://lore.kernel.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.git


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git