[PATCH 0/2] completion: silence pseudo-ref existence check

[PATCH 0/2] completion: silence pseudo-ref existence check

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 536 bytes --]

Hi,

I recently noticed that the Bash completion script started to output
object IDs in repositories which use the "reftable" backend when
completing certain commands. This patch series fixes this issue.

Patrick

Patrick Steinhardt (2):
  t9902: verify that completion does not print anything
  completion: silence pseudoref existence check

 contrib/completion/git-completion.bash |  2 +-
 t/t9902-completion.sh                  | 14 ++++++++++++--
 2 files changed, 13 insertions(+), 3 deletions(-)

-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 1/2] t9902: verify that completion does not print anything

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 908 bytes --]

The Bash completion script must not print anything to either stdout or
stderr. Instead, it is only expected to populate certain variables.
Tighten our `test_completion ()` test helper to verify this requirement.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t9902-completion.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index aa9a614de3..78cb93bea7 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -87,9 +87,11 @@ test_completion ()
 	else
 		sed -e 's/Z$//' |sort >expected
 	fi &&
-	run_completion "$1" &&
+	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&
 	sort out >out_sorted &&
-	test_cmp expected out_sorted
+	test_cmp expected out_sorted &&
+	test_must_be_empty "$TRASH_DIRECTORY"/output &&
+	rm "$TRASH_DIRECTORY"/output
 }
 
 # Test __gitcomp.
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 2/2] completion: silence pseudoref existence check

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2092 bytes --]

In 44dbb3bf29 (completion: support pseudoref existence checks for
reftables, 2023-12-19), we have extended the Bash completion script to
support future ref backends better by using git-rev-parse(1) to check
for pseudo-ref existence. This conversion has introduced a bug, because
even though we pass `--quiet` to git-rev-parse(1) it would still output
the resolved object ID of the ref in question if it exists.

Fix this by redirecting its stdout to `/dev/null` and add a test that
catches this behaviour. Note that the test passes even without the fix
for the "files" backend because we parse pseudo refs via the filesystem
directly in that case. But the test will fail with the "reftable"
backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 contrib/completion/git-completion.bash | 2 +-
 t/t9902-completion.sh                  | 8 ++++++++
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 8c40ade494..8872192e2b 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -146,7 +146,7 @@ __git_pseudoref_exists ()
 	if __git_eread "$__git_repo_path/HEAD" head; then
 		b="${head#ref: }"
 		if [ "$b" == "refs/heads/.invalid" ]; then
-			__git -C "$__git_repo_path" rev-parse --verify --quiet "$ref" 2>/dev/null
+			__git -C "$__git_repo_path" rev-parse --verify --quiet "$ref" >/dev/null 2>&1
 			return $?
 		fi
 	fi
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 78cb93bea7..b14ae4de14 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1927,6 +1927,14 @@ test_expect_success 'git checkout - --orphan with branch already provided comple
 	EOF
 '
 
+test_expect_success 'git reset completes modified files' '
+	test_commit A a.file &&
+	echo B >a.file &&
+	test_completion "git restore a." <<-\EOF
+	a.file
+	EOF
+'
+
 test_expect_success 'teardown after ref completion' '
 	git branch -d matching-branch &&
 	git tag -d matching-tag &&
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> The Bash completion script must not print anything to either stdout or
> stderr. Instead, it is only expected to populate certain variables.
> Tighten our `test_completion ()` test helper to verify this requirement.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/t9902-completion.sh | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index aa9a614de3..78cb93bea7 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -87,9 +87,11 @@ test_completion ()
>  	else
>  		sed -e 's/Z$//' |sort >expected
>  	fi &&
> -	run_completion "$1" &&
> +	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&

OK, there is a call to test_completion after cd and this is a
reasonable way to make sure we write to a known location.

I would have done the "run_completion should be totally silent"
check immediately after this line, as they logically are more
related to each other than to the two lines that implement "the
'out' file when sorted must match the expectation we just prepared",
but that is not a huge deal.

The filename "output" is something that may tempt folks who add more
tests to this file to use for their own purpose; because we use it
only three times locally here, it might be safer to give it a bit
more specific name, e.g., "run-completion-output" or something.



>  	sort out >out_sorted &&
> -	test_cmp expected out_sorted
> +	test_cmp expected out_sorted &&
> +	test_must_be_empty "$TRASH_DIRECTORY"/output &&
> +	rm "$TRASH_DIRECTORY"/output
>  }
>  
>  # Test __gitcomp.

Re: [PATCH 2/2] completion: silence pseudoref existence check

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> In 44dbb3bf29 (completion: support pseudoref existence checks for
> reftables, 2023-12-19), we have extended the Bash completion script to
> support future ref backends better by using git-rev-parse(1) to check
> for pseudo-ref existence. This conversion has introduced a bug, because
> even though we pass `--quiet` to git-rev-parse(1) it would still output
> the resolved object ID of the ref in question if it exists.
>
> Fix this by redirecting its stdout to `/dev/null` and add a test that
> catches this behaviour. Note that the test passes even without the fix
> for the "files" backend because we parse pseudo refs via the filesystem
> directly in that case. But the test will fail with the "reftable"
> backend.

Thanks.  Will queue.

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Johannes Schindelin]

Hi Patrick,

On Thu, 11 Jan 2024, Patrick Steinhardt wrote:

> The Bash completion script must not print anything to either stdout or
> stderr. Instead, it is only expected to populate certain variables.
> Tighten our `test_completion ()` test helper to verify this requirement.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/t9902-completion.sh | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index aa9a614de3..78cb93bea7 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -87,9 +87,11 @@ test_completion ()
>  	else
>  		sed -e 's/Z$//' |sort >expected
>  	fi &&
> -	run_completion "$1" &&
> +	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&
>  	sort out >out_sorted &&
> -	test_cmp expected out_sorted
> +	test_cmp expected out_sorted &&
> +	test_must_be_empty "$TRASH_DIRECTORY"/output &&

It seems that this fails CI on macOS, most likely because we're running
with `set -x` and that output somehow ends up in `output`, see e.g. here:
https://github.com/git/git/actions/runs/7496790359/job/20409405194#step:4:1880

  [...]
  ++ test_completion 'git switch '
  ++ test 1 -gt 1
  ++ sed -e 's/Z$//'
  ++ sort
  ++ run_completion 'git switch '
  ++ sort out
  ++ test_cmp expected out_sorted
  ++ test 2 -ne 2
  ++ eval 'diff -u' '"$@"'
  +++ diff -u expected out_sorted
  ++ test_must_be_empty '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
  ++ test 1 -ne 1
  ++ test_path_is_file '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
  ++ test 1 -ne 1
  ++ test -f '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
  ++ test -s '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
  ++ echo ''\''/Users/runner/work/git/git/t/trash directory.t9902-completion/output'\'' is not empty, it contains:'
  '/Users/runner/work/git/git/t/trash directory.t9902-completion/output' is not empty, it contains:
  ++ cat '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
  ++ local -a COMPREPLY _words
  ++ local _cword
  [...]

Maybe this is running in Dash and therefore `BASH_XTRACEFD=4` in
`test-lib.sh` has not the intended effect?

Ciao,
Johannes

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3285 bytes --]

On Fri, Jan 12, 2024 at 11:08:21AM +0100, Johannes Schindelin wrote:
> Hi Patrick,
> 
> On Thu, 11 Jan 2024, Patrick Steinhardt wrote:
> 
> > The Bash completion script must not print anything to either stdout or
> > stderr. Instead, it is only expected to populate certain variables.
> > Tighten our `test_completion ()` test helper to verify this requirement.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  t/t9902-completion.sh | 6 ++++--
> >  1 file changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> > index aa9a614de3..78cb93bea7 100755
> > --- a/t/t9902-completion.sh
> > +++ b/t/t9902-completion.sh
> > @@ -87,9 +87,11 @@ test_completion ()
> >  	else
> >  		sed -e 's/Z$//' |sort >expected
> >  	fi &&
> > -	run_completion "$1" &&
> > +	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&
> >  	sort out >out_sorted &&
> > -	test_cmp expected out_sorted
> > +	test_cmp expected out_sorted &&
> > +	test_must_be_empty "$TRASH_DIRECTORY"/output &&
> 
> It seems that this fails CI on macOS, most likely because we're running
> with `set -x` and that output somehow ends up in `output`, see e.g. here:
> https://github.com/git/git/actions/runs/7496790359/job/20409405194#step:4:1880
> 
>   [...]
>   ++ test_completion 'git switch '
>   ++ test 1 -gt 1
>   ++ sed -e 's/Z$//'
>   ++ sort
>   ++ run_completion 'git switch '
>   ++ sort out
>   ++ test_cmp expected out_sorted
>   ++ test 2 -ne 2
>   ++ eval 'diff -u' '"$@"'
>   +++ diff -u expected out_sorted
>   ++ test_must_be_empty '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
>   ++ test 1 -ne 1
>   ++ test_path_is_file '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
>   ++ test 1 -ne 1
>   ++ test -f '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
>   ++ test -s '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
>   ++ echo ''\''/Users/runner/work/git/git/t/trash directory.t9902-completion/output'\'' is not empty, it contains:'
>   '/Users/runner/work/git/git/t/trash directory.t9902-completion/output' is not empty, it contains:
>   ++ cat '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
>   ++ local -a COMPREPLY _words
>   ++ local _cword
>   [...]
> 
> Maybe this is running in Dash and therefore `BASH_XTRACEFD=4` in
> `test-lib.sh` has not the intended effect?

Meh, thanks for the heads up. Another test gap in GitLab CI which I'm
going to address soon via a new macOS job.

In any case, Dash indeed does not honor the above envvar. I also could
not find any alternate solutions to redirect the tracing output. So for
all I can see there are a few ways to handle this:

  - `set -x` and then restore the previous value after having called
    `run_completion`.

  - Filter the output so that any line starting with "${PS4}" gets
    removed. 

  - Don't test for this bug.

Not sure which way to go, but the first alternative feels a bit more
sensible to me. It does remove the ability to see what's going on in the
completion script though in case one wants to debug it.

I'll reroll this patch series early next week.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Johannes Schindelin]

Hi Patrick,

On Fri, 12 Jan 2024, Patrick Steinhardt wrote:

> On Fri, Jan 12, 2024 at 11:08:21AM +0100, Johannes Schindelin wrote:
>
> > On Thu, 11 Jan 2024, Patrick Steinhardt wrote:
> >
> > > The Bash completion script must not print anything to either stdout or
> > > stderr. Instead, it is only expected to populate certain variables.
> > > Tighten our `test_completion ()` test helper to verify this requirement.
> > >
> > > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > > ---
> > >  t/t9902-completion.sh | 6 ++++--
> > >  1 file changed, 4 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> > > index aa9a614de3..78cb93bea7 100755
> > > --- a/t/t9902-completion.sh
> > > +++ b/t/t9902-completion.sh
> > > @@ -87,9 +87,11 @@ test_completion ()
> > >  	else
> > >  		sed -e 's/Z$//' |sort >expected
> > >  	fi &&
> > > -	run_completion "$1" &&
> > > +	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&
> > >  	sort out >out_sorted &&
> > > -	test_cmp expected out_sorted
> > > +	test_cmp expected out_sorted &&
> > > +	test_must_be_empty "$TRASH_DIRECTORY"/output &&
> >
> > It seems that this fails CI on macOS, most likely because we're running
> > with `set -x` and that output somehow ends up in `output`, see e.g. here:
> > https://github.com/git/git/actions/runs/7496790359/job/20409405194#step:4:1880
> >
> >   [...]
> >   ++ test_completion 'git switch '
> >   ++ test 1 -gt 1
> >   ++ sed -e 's/Z$//'
> >   ++ sort
> >   ++ run_completion 'git switch '
> >   ++ sort out
> >   ++ test_cmp expected out_sorted
> >   ++ test 2 -ne 2
> >   ++ eval 'diff -u' '"$@"'
> >   +++ diff -u expected out_sorted
> >   ++ test_must_be_empty '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
> >   ++ test 1 -ne 1
> >   ++ test_path_is_file '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
> >   ++ test 1 -ne 1
> >   ++ test -f '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
> >   ++ test -s '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
> >   ++ echo ''\''/Users/runner/work/git/git/t/trash directory.t9902-completion/output'\'' is not empty, it contains:'
> >   '/Users/runner/work/git/git/t/trash directory.t9902-completion/output' is not empty, it contains:
> >   ++ cat '/Users/runner/work/git/git/t/trash directory.t9902-completion/output'
> >   ++ local -a COMPREPLY _words
> >   ++ local _cword
> >   [...]
> >
> > Maybe this is running in Dash and therefore `BASH_XTRACEFD=4` in
> > `test-lib.sh` has not the intended effect?
>
> Meh, thanks for the heads up. Another test gap in GitLab CI which I'm
> going to address soon via a new macOS job.
>
> In any case, Dash indeed does not honor the above envvar.

Hmm. I had a closer look and now I am thoroughly confused. In
`t/lib-bash.exe`, we make sure that this test is run in Bash. And indeed,
when I call

  BASH=1 POSIXLY_CORRECT=1 TEST_SHELL_PATH=/bin/dash dash t9902-completion.sh -iVx

I am greeted by this error message:

  git-completion.bash: Syntax error: "(" unexpected (expecting "fi")

So something else must be going on here.

> I also could not find any alternate solutions to redirect the tracing
> output. So for all I can see there are a few ways to handle this:
>
>   - `set -x` and then restore the previous value after having called
>     `run_completion`.
>
>   - Filter the output so that any line starting with "${PS4}" gets
>     removed.
>
>   - Don't test for this bug.
>
> Not sure which way to go, but the first alternative feels a bit more
> sensible to me. It does remove the ability to see what's going on in the
> completion script though in case one wants to debug it.

Personally, I would go for option 2, filtering out the xtrace output. This
here seems to work:

-- snip --
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index b14ae4de14e..23cd1cd9508 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -87,11 +87,14 @@ test_completion ()
 	else
 		sed -e 's/Z$//' |sort >expected
 	fi &&
+	PS4=+ &&
 	run_completion "$1" >"$TRASH_DIRECTORY"/output 2>&1 &&
+	sed "/^+/{:1;s/'[^']*'//g;/'/{N;b1};d}" \
+		<"$TRASH_DIRECTORY"/output >"$TRASH_DIRECTORY"/output.x &&
+	test_must_be_empty "$TRASH_DIRECTORY"/output.x &&
+	rm "$TRASH_DIRECTORY"/output "$TRASH_DIRECTORY"/output.x &&
 	sort out >out_sorted &&
-	test_cmp expected out_sorted &&
-	test_must_be_empty "$TRASH_DIRECTORY"/output &&
-	rm "$TRASH_DIRECTORY"/output
+	test_cmp expected out_sorted
 }

 # Test __gitcomp.
-- snap --

It is a bit ugly, in particular the `sed` expression (which is a bit
complex because the `output` file can contain multi-line strings enclosed
in single-quotes), and we could probably lose the `"$TRASH_DIRECTORY"/`
part to improve the reading experience somewhat.

But my main concern is: Why does this happen in the first place? If we are
running with Bash, why does `BASH_XTRACEFD` to work as intended here and
makes it necessary to filter out the traced commands?

Ciao,
Johannes

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Jeff King]

On Fri, Jan 12, 2024 at 02:12:43PM +0100, Johannes Schindelin wrote:

> But my main concern is: Why does this happen in the first place? If we are
> running with Bash, why does `BASH_XTRACEFD` to work as intended here and
> makes it necessary to filter out the traced commands?

BASH_XTRACEFD was introduced in bash 4.1. macOS ships with the ancient
bash 3.2.57, which is the last GPLv2 version.

One simple solution is to mark the script with test_untraceable. See
5fc98e79fc (t: add means to disable '-x' tracing for individual test
scripts, 2018-02-24) and 5827506928 (t1510-repo-setup: mark as
untraceable with '-x', 2018-02-24).

That will disable tracing entirely in the script for older versions of
bash, which could make debugging harder. But it will still work as
expected for people on reasonable versions of bash, and doesn't
introduce any complicated code.

-Peff

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Fri, Jan 12, 2024 at 02:12:43PM +0100, Johannes Schindelin wrote:
>
>> But my main concern is: Why does this happen in the first place? If we are
>> running with Bash, why does `BASH_XTRACEFD` to work as intended here and
>> makes it necessary to filter out the traced commands?
>
> BASH_XTRACEFD was introduced in bash 4.1. macOS ships with the ancient
> bash 3.2.57, which is the last GPLv2 version.
>
> One simple solution is to mark the script with test_untraceable. See
> 5fc98e79fc (t: add means to disable '-x' tracing for individual test
> scripts, 2018-02-24) and 5827506928 (t1510-repo-setup: mark as
> untraceable with '-x', 2018-02-24).
>
> That will disable tracing entirely in the script for older versions of
> bash, which could make debugging harder. But it will still work as
> expected for people on reasonable versions of bash, and doesn't
> introduce any complicated code.

Thank you, all three of you, for digging through to the bottom
quickly.

I too suspected a version of bash that is ancient and found out
about the "untraceable" bit just before I started reading this
thread ;-)

Re: [PATCH 2/2] completion: silence pseudoref existence check

[SZEDER Gábor]

On Thu, Jan 11, 2024 at 11:41:59AM +0100, Patrick Steinhardt wrote:
> In 44dbb3bf29 (completion: support pseudoref existence checks for
> reftables, 2023-12-19), we have extended the Bash completion script to
> support future ref backends better by using git-rev-parse(1) to check
> for pseudo-ref existence. This conversion has introduced a bug, because
> even though we pass `--quiet` to git-rev-parse(1) it would still output
> the resolved object ID of the ref in question if it exists.
> 
> Fix this by redirecting its stdout to `/dev/null` and add a test that
> catches this behaviour. Note that the test passes even without the fix
> for the "files" backend because we parse pseudo refs via the filesystem
> directly in that case. But the test will fail with the "reftable"
> backend.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  contrib/completion/git-completion.bash | 2 +-
>  t/t9902-completion.sh                  | 8 ++++++++
>  2 files changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 8c40ade494..8872192e2b 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -146,7 +146,7 @@ __git_pseudoref_exists ()
>  	if __git_eread "$__git_repo_path/HEAD" head; then
>  		b="${head#ref: }"
>  		if [ "$b" == "refs/heads/.invalid" ]; then

Nit: I guess these last two lines above came from the git prompt
script, where we do need the name of the ref, but here we don't need
that, so the condition could have simply been

  if [ "$head" = "ref: refs/heads/.invalid" ]

With a single = instead of ==, because there is no pattern matching
here.

> -			__git -C "$__git_repo_path" rev-parse --verify --quiet "$ref" 2>/dev/null
> +			__git -C "$__git_repo_path" rev-parse --verify --quiet "$ref" >/dev/null 2>&1

You don't need the '-C $__git_repo_path' option and you don't have to
redirect stderr either, because the purpose of the __git wrapper
function is to take care of both.

>  			return $?
>  		fi
>  	fi
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 78cb93bea7..b14ae4de14 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1927,6 +1927,14 @@ test_expect_success 'git checkout - --orphan with branch already provided comple
>  	EOF
>  '
>  
> +test_expect_success 'git reset completes modified files' '

The description of the test case mentions 'git reset' ...

> +	test_commit A a.file &&
> +	echo B >a.file &&
> +	test_completion "git restore a." <<-\EOF

... but it invokes 'git restore'.

Anyway, I think it would be better to add a dedicated test or two to
exercise the __git_pseudoref_exists helper function instead (or
perhaps in addition).

> +	a.file
> +	EOF
> +'
> +
>  test_expect_success 'teardown after ref completion' '
>  	git branch -d matching-branch &&
>  	git tag -d matching-tag &&
> -- 
> 2.43.GIT
> 

Re: [PATCH 1/2] t9902: verify that completion does not print anything

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On Fri, Jan 12, 2024 at 10:16:55AM -0500, Jeff King wrote:
> On Fri, Jan 12, 2024 at 02:12:43PM +0100, Johannes Schindelin wrote:
> 
> > But my main concern is: Why does this happen in the first place? If we are
> > running with Bash, why does `BASH_XTRACEFD` to work as intended here and
> > makes it necessary to filter out the traced commands?
> 
> BASH_XTRACEFD was introduced in bash 4.1. macOS ships with the ancient
> bash 3.2.57, which is the last GPLv2 version.
> 
> One simple solution is to mark the script with test_untraceable. See
> 5fc98e79fc (t: add means to disable '-x' tracing for individual test
> scripts, 2018-02-24) and 5827506928 (t1510-repo-setup: mark as
> untraceable with '-x', 2018-02-24).
> 
> That will disable tracing entirely in the script for older versions of
> bash, which could make debugging harder. But it will still work as
> expected for people on reasonable versions of bash, and doesn't
> introduce any complicated code.
> 
> -Peff

Ah, this is indeed the best solution. Thanks for the hints and
investigations everyone, will fix in the next iteration.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 0/5] completion: silence pseudo-ref existence check

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

Hi,

this is the second version of my patch series that refactors the Bash
completion scripts to work better with reftables. Changes compared to
v1:

  - I've incorporated SZEDER's feedback that he has sent in reply to the
    original patch series that is already in `master`. Most importantly,
    we now discover the repo path in `__git_pseudoref_exists ()` itself.

  - I've included some refactorings to stop leaking local variables in
    `__git_pseudoref_exists ()`.

  - I've added tests for `__git_pseudoref_exists ()` directly, which was
    also suggested by SZEDER.

  - I've marked t9902 with `test_untraceable` now to unbreak macOS CI.

  - I've refactored the completion test to use `git show-ref --exists`
    to fix an issue with unborn pseudorefs that I didn't notice
    previously.

I didn't include a range-diff because quite a lof of things have moved
around.

Patrick

Patrick Steinhardt (5):
  completion: discover repo path in `__git_pseudoref_exists ()`
  t9902: verify that completion does not print anything
  completion: improve existence check for pseudo-refs
  completion: silence pseudoref existence check
  completion: treat dangling symrefs as existing pseudorefs

 contrib/completion/git-completion.bash | 11 +++---
 t/t9902-completion.sh                  | 47 ++++++++++++++++++++++++--
 2 files changed, 50 insertions(+), 8 deletions(-)
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 1/5] completion: discover repo path in `__git_pseudoref_exists ()`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1695 bytes --]

The helper function `__git_pseudoref_exists ()` expects that the repo
path has already been discovered by its callers, which makes for a
rather fragile calling convention. Refactor the function to discover the
repo path itself to make it more self-contained, which also removes the
need to discover the path in some of its callers.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 contrib/completion/git-completion.bash | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 8c40ade494..06a9107449 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -138,6 +138,8 @@ __git_pseudoref_exists ()
 {
 	local ref=$1
 
+	__git_find_repo_path
+
 	# If the reftable is in use, we have to shell out to 'git rev-parse'
 	# to determine whether the ref exists instead of looking directly in
 	# the filesystem to determine whether the ref exists. Otherwise, use
@@ -1656,7 +1658,6 @@ __git_cherry_pick_inprogress_options=$__git_sequencer_inprogress_options
 
 _git_cherry_pick ()
 {
-	__git_find_repo_path
 	if __git_pseudoref_exists CHERRY_PICK_HEAD; then
 		__gitcomp "$__git_cherry_pick_inprogress_options"
 		return
@@ -2966,7 +2967,6 @@ _git_reset ()
 
 _git_restore ()
 {
-	__git_find_repo_path
 	case "$prev" in
 	-s)
 		__git_complete_refs
@@ -2995,7 +2995,6 @@ __git_revert_inprogress_options=$__git_sequencer_inprogress_options
 
 _git_revert ()
 {
-	__git_find_repo_path
 	if __git_pseudoref_exists REVERT_HEAD; then
 		__gitcomp "$__git_revert_inprogress_options"
 		return
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 2/5] t9902: verify that completion does not print anything

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

The Bash completion script must not print anything to either stdout or
stderr. Instead, it is only expected to populate certain variables.
Tighten our `test_completion ()` test helper to verify this requirement.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t9902-completion.sh | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index aa9a614de3..95ec762a74 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -5,6 +5,12 @@
 
 test_description='test bash completion'
 
+# The Bash completion scripts must not print anything to either stdout or
+# stderr, which we try to verify. When tracing is enabled without support for
+# BASH_XTRACEFD this assertion will fail, so we have to mark the test as
+# untraceable with such ancient Bash versions.
+test_untraceable=UnfortunatelyYes
+
 . ./lib-bash.sh
 
 complete ()
@@ -87,9 +93,11 @@ test_completion ()
 	else
 		sed -e 's/Z$//' |sort >expected
 	fi &&
-	run_completion "$1" &&
+	run_completion "$1" >"$TRASH_DIRECTORY"/bash-completion-output 2>&1 &&
 	sort out >out_sorted &&
-	test_cmp expected out_sorted
+	test_cmp expected out_sorted &&
+	test_must_be_empty "$TRASH_DIRECTORY"/bash-completion-output &&
+	rm "$TRASH_DIRECTORY"/bash-completion-output
 }
 
 # Test __gitcomp.
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 3/5] completion: improve existence check for pseudo-refs

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1606 bytes --]

Improve the existence check along the following lines:

  - Stop stripping the "ref :" prefix and compare to the expected value
    directly. This allows us to drop a now-unused variable that was
    previously leaking into the user's shell.

  - Mark the "head" variable as local so that we don't leak its value
    into the user's shell.

  - Stop manually handling the `-C $__git_repo_path` option, which the
    `__git ()` wrapper aleady does for us.

  - In simlar spirit, stop redirecting stderr, which is also handled by
    the wrapper already.

Suggested-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 contrib/completion/git-completion.bash | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 06a9107449..d703e3e64f 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -137,6 +137,7 @@ __git_eread ()
 __git_pseudoref_exists ()
 {
 	local ref=$1
+	local head
 
 	__git_find_repo_path
 
@@ -146,9 +147,8 @@ __git_pseudoref_exists ()
 	# Bash builtins since executing Git commands are expensive on some
 	# platforms.
 	if __git_eread "$__git_repo_path/HEAD" head; then
-		b="${head#ref: }"
-		if [ "$b" == "refs/heads/.invalid" ]; then
-			__git -C "$__git_repo_path" rev-parse --verify --quiet "$ref" 2>/dev/null
+		if [ "$head" == "ref: refs/heads/.invalid" ]; then
+			__git rev-parse --verify --quiet "$ref"
 			return $?
 		fi
 	fi
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 4/5] completion: silence pseudoref existence check

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2991 bytes --]

In 44dbb3bf29 (completion: support pseudoref existence checks for
reftables, 2023-12-19), we have extended the Bash completion script to
support future ref backends better by using git-rev-parse(1) to check
for pseudo-ref existence. This conversion has introduced a bug, because
even though we pass `--quiet` to git-rev-parse(1) it would still output
the resolved object ID of the ref in question if it exists.

Fix this by redirecting its stdout to `/dev/null` and add a test that
catches this behaviour. Note that the test passes even without the fix
for the "files" backend because we parse pseudo refs via the filesystem
directly in that case. But the test will fail with the "reftable"
backend.

Helped-by: Jeff King <peff@peff.net>
Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 contrib/completion/git-completion.bash |  2 +-
 t/t9902-completion.sh                  | 31 ++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 1 deletion(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d703e3e64f..54ce58f73d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -148,7 +148,7 @@ __git_pseudoref_exists ()
 	# platforms.
 	if __git_eread "$__git_repo_path/HEAD" head; then
 		if [ "$head" == "ref: refs/heads/.invalid" ]; then
-			__git rev-parse --verify --quiet "$ref"
+			__git rev-parse --verify --quiet "$ref" >/dev/null
 			return $?
 		fi
 	fi
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 95ec762a74..56dc7343a2 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1933,6 +1933,14 @@ test_expect_success 'git checkout - --orphan with branch already provided comple
 	EOF
 '
 
+test_expect_success 'git restore completes modified files' '
+	test_commit A a.file &&
+	echo B >a.file &&
+	test_completion "git restore a." <<-\EOF
+	a.file
+	EOF
+'
+
 test_expect_success 'teardown after ref completion' '
 	git branch -d matching-branch &&
 	git tag -d matching-tag &&
@@ -2728,4 +2736,27 @@ test_expect_success '__git_complete' '
 	test_must_fail __git_complete ga missing
 '
 
+test_expect_success '__git_pseudoref_exists' '
+	test_when_finished "rm -rf repo" &&
+	git init repo &&
+	(
+		cd repo &&
+		sane_unset __git_repo_path &&
+
+		# HEAD points to an existing branch, so it should exist.
+		test_commit A &&
+		__git_pseudoref_exists HEAD >output 2>&1 &&
+		test_must_be_empty output &&
+
+		# CHERRY_PICK_HEAD does not exist, so the existence check should fail.
+		! __git_pseudoref_exists CHERRY_PICK_HEAD >output 2>&1 &&
+		test_must_be_empty output &&
+
+		# CHERRY_PICK_HEAD points to a commit, so it should exist.
+		git update-ref CHERRY_PICK_HEAD A &&
+		__git_pseudoref_exists CHERRY_PICK_HEAD >output 2>&1 &&
+		test_must_be_empty output
+	)
+'
+
 test_done
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 5/5] completion: treat dangling symrefs as existing pseudorefs

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1927 bytes --]

The `__git_pseudoref_exists ()` helper function back to git-rev-parse(1)
in case the reftable backend is in use. This is not in the same spirit
as the simple existence check that the "files" backend does though,
because there we only check for the pseudo-ref to exist with `test -f`.
With git-rev-parse(1) we not only check for existence, but also verify
that the pseudo-ref resolves to an object, which may not be the case
when the pseudo-ref points to an unborn branch.

Fix this issue by using `git show-ref --exists` instead. Note that we do
not have to silence stdout anymore as git-show-ref(1) will not print
anything.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 contrib/completion/git-completion.bash | 2 +-
 t/t9902-completion.sh                  | 4 ++++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 54ce58f73d..6662db221d 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -148,7 +148,7 @@ __git_pseudoref_exists ()
 	# platforms.
 	if __git_eread "$__git_repo_path/HEAD" head; then
 		if [ "$head" == "ref: refs/heads/.invalid" ]; then
-			__git rev-parse --verify --quiet "$ref" >/dev/null
+			__git show-ref --exists "$ref"
 			return $?
 		fi
 	fi
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 56dc7343a2..35eb534fdd 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -2743,6 +2743,10 @@ test_expect_success '__git_pseudoref_exists' '
 		cd repo &&
 		sane_unset __git_repo_path &&
 
+		# HEAD should exist, even if it points to an unborn branch.
+		__git_pseudoref_exists HEAD >output 2>&1 &&
+		test_must_be_empty output &&
+
 		# HEAD points to an existing branch, so it should exist.
 		test_commit A &&
 		__git_pseudoref_exists HEAD >output 2>&1 &&
-- 
2.43.GIT


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]