[PATCH 00/11] t: reduce direct disk access to data structures

[PATCH 00/11] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2841 bytes --]

Hi,

this patch series refactors a bunch of our tests to perform less direct
disk access to on-disk data structures. Instead, the tests are converted
to use Git tools or our test-tool to access data to the best extent
possible. This serves two benefits:

    - We increase test coverage of our own code base.

    - We become less dependent on the actual on-disk format.

The main motivation for this patch series was the second bullet point as
it is preparatory work to get the reftable backend upstreamed. My intent
is to get rid of many or even most of the current blockers in the Git
project before trying to send the reftable implementation upstream.
While this will be a lot of up-front work that is going to span over a
long time period, I think this approach will make everyones live easier
by doing comparatively small and incremental improvements to the Git
project. Ultimately, the final patch series should in the best case only
contain the new backend as well as testing infrastructure, but not much
else.

Patrick

Patrick Steinhardt (11):
  t: add helpers to test for reference existence
  t: allow skipping expected object ID in `ref-store update-ref`
  t: convert tests to use helpers for reference existence
  t: convert tests to not write references via the filesystem
  t: convert tests to not access symrefs via the filesystem
  t: convert tests to not access reflog via the filesystem
  t1450: convert tests to remove worktrees via git-worktree(1)
  t4207: delete replace references via git-update-ref(1)
  t7300: assert exact states of repo
  t7900: assert the absence of refs via git-for-each-ref(1)
  t: mark several tests that assume the files backend with REFFILES

 t/README                           |  9 ++++
 t/helper/test-ref-store.c          | 38 +++++++++++++--
 t/t1400-update-ref.sh              | 49 ++++++++++----------
 t/t1430-bad-ref-name.sh            | 39 ++++++++++------
 t/t1450-fsck.sh                    | 46 ++++++++++---------
 t/t2011-checkout-invalid-head.sh   | 16 +++----
 t/t3200-branch.sh                  | 74 ++++++++++++++++--------------
 t/t3400-rebase.sh                  |  2 +-
 t/t3404-rebase-interactive.sh      |  2 +-
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t4207-log-decoration-colors.sh   |  4 +-
 t/t5521-pull-options.sh            |  4 +-
 t/t5526-fetch-submodules.sh        |  2 +-
 t/t5605-clone-local.sh             |  6 +--
 t/t5702-protocol-v2.sh             | 24 +++++++---
 t/t7300-clean.sh                   | 23 ++++++----
 t/t7900-maintenance.sh             |  3 +-
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 t/test-lib-functions.sh            | 66 ++++++++++++++++++++++++++
 20 files changed, 277 insertions(+), 136 deletions(-)

-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 01/11] t: add helpers to test for reference existence

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 6798 bytes --]

There are two major ways to check for the existence of a reference in
our tests:

    - `git rev-parse --verify` can be used to check for existence of a
      reference. This only works in the case where the reference is well
      formed though and resolves to an actual object ID. This does not
      work with malformed reference names or invalid contents.

    - `test_path_is_file` can be used to check for existence of a loose
      reference if it is known to not resolve to an actual object ID. It
      by necessity reaches into implementation details of the reference
      backend though.

Similarly, there are two equivalent ways to check for the absence of a
reference:

    - `test_must_fail git rev-parse` can be used to check for the
      absence of a reference. It could fail due to a number of reasons
      though, and all of these reasons will be thrown into the same bag
      as an absent reference.

    - `test_path_is_missing` can be used to check explicitly for the
      absence of a loose reference, but again reaches into internal
      implementation details of the reference backend.

So both our tooling to check for the presence and for the absence of
references in tests is lacking as either failure cases are thrown into
the same bag or we need to reach into internal implementation details of
the respective reference backend.

Introduce a new subcommand for our ref-store test helper that explicitly
checks only for the presence or absence of a reference. This addresses
these limitations:

    - We can check for the presence of references with malformed names.

    - We can check for the presence of references that don't resolve.

    - We can explicitly handle the case where a reference is missing by
      special-casing ENOENT errors.

    - We don't need to reach into implementation details of the backend,
      which would allow us to use this helper for the future reftable
      backend.

Next to this subcommand we also provide two wrappers `test_ref_exists`
and `test_ref_missing` that make the helper easier to use.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/README                  |  9 ++++++
 t/helper/test-ref-store.c | 27 +++++++++++++++-
 t/test-lib-functions.sh   | 66 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 101 insertions(+), 1 deletion(-)

diff --git a/t/README b/t/README
index 61080859899..779f7e7dd86 100644
--- a/t/README
+++ b/t/README
@@ -928,6 +928,15 @@ see test-lib-functions.sh for the full list and their options.
    committer times to defined state.  Subsequent calls will
    advance the times by a fixed amount.
 
+ - test_ref_exists <ref>, test_ref_missing <ref>
+
+   Check whether a reference exists or is missing. In contrast to
+   git-rev-parse(1), these helpers also work with invalid reference
+   names and references whose contents are unresolvable. The latter
+   function also distinguishes generic errors from the case where a
+   reference explicitly doesn't exist and is thus safer to use than
+   `test_must_fail git rev-parse`.
+
  - test_commit <message> [<filename> [<contents>]]
 
    Creates a commit with the given message, committing the given
diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 48552e6a9e0..7400f560ab6 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -1,6 +1,6 @@
 #include "test-tool.h"
 #include "hex.h"
-#include "refs.h"
+#include "refs/refs-internal.h"
 #include "setup.h"
 #include "worktree.h"
 #include "object-store-ll.h"
@@ -221,6 +221,30 @@ static int cmd_verify_ref(struct ref_store *refs, const char **argv)
 	return ret;
 }
 
+static int cmd_ref_exists(struct ref_store *refs, const char **argv)
+{
+	const char *refname = notnull(*argv++, "refname");
+	struct strbuf unused_referent = STRBUF_INIT;
+	struct object_id unused_oid;
+	unsigned int unused_type;
+	int failure_errno;
+
+	if (refs_read_raw_ref(refs, refname, &unused_oid, &unused_referent,
+			      &unused_type, &failure_errno)) {
+		/*
+		 * We handle ENOENT separately here such that it is possible to
+		 * distinguish actually-missing references from any kind of
+		 * generic error.
+		 */
+		if (failure_errno == ENOENT)
+			return 17;
+		return -1;
+	}
+
+	strbuf_release(&unused_referent);
+	return 0;
+}
+
 static int cmd_for_each_reflog(struct ref_store *refs,
 			       const char **argv UNUSED)
 {
@@ -325,6 +349,7 @@ static struct command commands[] = {
 	{ "for-each-ref--exclude", cmd_for_each_ref__exclude },
 	{ "resolve-ref", cmd_resolve_ref },
 	{ "verify-ref", cmd_verify_ref },
+	{ "ref-exists", cmd_ref_exists },
 	{ "for-each-reflog", cmd_for_each_reflog },
 	{ "for-each-reflog-ent", cmd_for_each_reflog_ent },
 	{ "for-each-reflog-ent-reverse", cmd_for_each_reflog_ent_reverse },
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 2f8868caa17..212fddffa96 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -251,6 +251,72 @@ debug () {
 	done
 }
 
+# Usage: test_ref_exists [options] <ref>
+#   -C <dir>:
+#	Run all git commands in directory <dir>
+#   --refdb <refdb>:
+#	The reference database to run in. One of:
+#		- "main", the main reference database (default).
+#		- "submodule:<submodule>", the reference database of a
+#		  submodule.
+#		- "worktree:<worktree>", the reference database for a
+#		  worktree's per-worktree references.
+#
+# This helper function checks whether a reference exists. Symrefs will not be
+# resolved. Can be used to check references with bad names.
+test_ref_exists () {
+	local indir=
+	local refdb=main
+
+	while test $# != 0
+	do
+		case "$1" in
+		-C)
+			indir="$2"
+			shift
+			;;
+		--refdb)
+			refdb="$2"
+			shift
+			;;
+		*)
+			break
+			;;
+		esac
+		shift
+	done &&
+
+	indir=${indir:+"$indir"/} &&
+
+	if test "$#" != 1
+	then
+		BUG "expected exactly one reference"
+	fi &&
+
+	test-tool ${indir:+ -C "$indir"} ref-store "${refdb}" ref-exists "$1"
+}
+
+# Behaves the same as test_ref_exists, except that it checks for the absence of
+# a reference. This is preferable to `! test_ref_exists` as this function is
+# able to distinguish actually-missing references from other, generic errors.
+test_ref_missing () {
+	test_ref_exists "$@"
+	case "$?" in
+	17)
+		# This is the good case.
+		return 0
+		;;
+	0)
+		echo >&4 "test_ref_missing: reference exists"
+		return 1
+		;;
+	*)
+		echo >&4 "test_ref_missing: generic error"
+		return 1
+		;;
+	esac
+}
+
 # Usage: test_commit [options] <message> [<file> [<contents> [<tag>]]]
 #   -C <dir>:
 #	Run all git commands in directory <dir>
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 02/11] t: allow skipping expected object ID in `ref-store update-ref`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1771 bytes --]

We require the caller to pass both the old and new expected object ID to
our `test-tool ref-store update-ref` helper. When trying to update a
symbolic reference though it's impossible to specify the expected object
ID, which means that the test would instead have to force-update the
reference. This is currently impossible though.

Update the helper to optionally skip verification of the old object ID
in case the test passes in an empty old object ID as input.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/helper/test-ref-store.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 7400f560ab6..7dc83137584 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -322,16 +322,19 @@ static int cmd_update_ref(struct ref_store *refs, const char **argv)
 	const char *new_sha1_buf = notnull(*argv++, "new-sha1");
 	const char *old_sha1_buf = notnull(*argv++, "old-sha1");
 	unsigned int flags = arg_flags(*argv++, "flags", transaction_flags);
-	struct object_id old_oid;
+	struct object_id old_oid, *old_oid_ptr = NULL;
 	struct object_id new_oid;
 
-	if (get_oid_hex(old_sha1_buf, &old_oid))
-		die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+	if (*old_sha1_buf) {
+		if (get_oid_hex(old_sha1_buf, &old_oid))
+			die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+		old_oid_ptr = &old_oid;
+	}
 	if (get_oid_hex(new_sha1_buf, &new_oid))
 		die("cannot parse %s as %s", new_sha1_buf, the_hash_algo->name);
 
 	return refs_update_ref(refs, msg, refname,
-			       &new_oid, &old_oid,
+			       &new_oid, old_oid_ptr,
 			       flags, UPDATE_REFS_DIE_ON_ERR);
 }
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 03/11] t: convert tests to use helpers for reference existence

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 12736 bytes --]

Convert tests that use `test_path_is_file` and `test_path_is_missing` to
instead use our new helpers `test_ref_exists` and `test_ref_missing`.
These hook into the reference database directly and thus work indepently
of the actual reference backend that is being used.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1430-bad-ref-name.sh | 27 ++++++++++++++++++---------
 t/t3200-branch.sh       | 33 ++++++++++++++++++---------------
 t/t5521-pull-options.sh |  4 ++--
 t/t5605-clone-local.sh  |  2 +-
 4 files changed, 39 insertions(+), 27 deletions(-)

diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index ff1c967d550..7b7d6953c62 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -205,8 +205,9 @@ test_expect_success 'update-ref --no-deref -d can delete symref to broken name'
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
 	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
+	test_ref_exists refs/heads/badname &&
 	git update-ref --no-deref -d refs/heads/badname >output 2>error &&
-	test_path_is_missing .git/refs/heads/badname &&
+	test_ref_missing refs/heads/badname &&
 	test_must_be_empty output &&
 	test_must_be_empty error
 '
@@ -216,8 +217,9 @@ test_expect_success 'branch -d can delete symref to broken name' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
 	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
+	test_ref_exists refs/heads/badname &&
 	git branch -d badname >output 2>error &&
-	test_path_is_missing .git/refs/heads/badname &&
+	test_ref_missing refs/heads/badname &&
 	test_i18ngrep "Deleted branch badname (was refs/heads/broken\.\.\.ref)" output &&
 	test_must_be_empty error
 '
@@ -225,8 +227,9 @@ test_expect_success 'branch -d can delete symref to broken name' '
 test_expect_success 'update-ref --no-deref -d can delete dangling symref to broken name' '
 	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
+	test_ref_exists refs/heads/badname &&
 	git update-ref --no-deref -d refs/heads/badname >output 2>error &&
-	test_path_is_missing .git/refs/heads/badname &&
+	test_ref_missing refs/heads/badname &&
 	test_must_be_empty output &&
 	test_must_be_empty error
 '
@@ -234,8 +237,9 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref to brok
 test_expect_success 'branch -d can delete dangling symref to broken name' '
 	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
+	test_ref_exists refs/heads/badname &&
 	git branch -d badname >output 2>error &&
-	test_path_is_missing .git/refs/heads/badname &&
+	test_ref_missing refs/heads/badname &&
 	test_i18ngrep "Deleted branch badname (was refs/heads/broken\.\.\.ref)" output &&
 	test_must_be_empty error
 '
@@ -245,8 +249,9 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
 	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
+	test_ref_exists refs/heads/broken...ref &&
 	git update-ref -d refs/heads/badname >output 2>error &&
-	test_path_is_missing .git/refs/heads/broken...ref &&
+	test_ref_missing refs/heads/broken...ref &&
 	test_must_be_empty output &&
 	test_must_be_empty error
 '
@@ -254,8 +259,9 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
 test_expect_success 'update-ref --no-deref -d can delete symref with broken name' '
 	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
+	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
-	test_path_is_missing .git/refs/heads/broken...symref &&
+	test_ref_missing refs/heads/broken...symref &&
 	test_must_be_empty output &&
 	test_must_be_empty error
 '
@@ -263,8 +269,9 @@ test_expect_success 'update-ref --no-deref -d can delete symref with broken name
 test_expect_success 'branch -d can delete symref with broken name' '
 	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
+	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
-	test_path_is_missing .git/refs/heads/broken...symref &&
+	test_ref_missing refs/heads/broken...symref &&
 	test_i18ngrep "Deleted branch broken...symref (was refs/heads/main)" output &&
 	test_must_be_empty error
 '
@@ -272,8 +279,9 @@ test_expect_success 'branch -d can delete symref with broken name' '
 test_expect_success 'update-ref --no-deref -d can delete dangling symref with broken name' '
 	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
+	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
-	test_path_is_missing .git/refs/heads/broken...symref &&
+	test_ref_missing refs/heads/broken...symref &&
 	test_must_be_empty output &&
 	test_must_be_empty error
 '
@@ -281,8 +289,9 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
 test_expect_success 'branch -d can delete dangling symref with broken name' '
 	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
+	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
-	test_path_is_missing .git/refs/heads/broken...symref &&
+	test_ref_missing refs/heads/broken...symref &&
 	test_i18ngrep "Deleted branch broken...symref (was refs/heads/idonotexist)" output &&
 	test_must_be_empty error
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 080e4f24a6e..bde4f1485b7 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -25,7 +25,7 @@ test_expect_success 'prepare a trivial repository' '
 
 test_expect_success 'git branch --help should not have created a bogus branch' '
 	test_might_fail git branch --man --help </dev/null >/dev/null 2>&1 &&
-	test_path_is_missing .git/refs/heads/--help
+	test_ref_missing refs/heads/--help
 '
 
 test_expect_success 'branch -h in broken repository' '
@@ -40,7 +40,8 @@ test_expect_success 'branch -h in broken repository' '
 '
 
 test_expect_success 'git branch abc should create a branch' '
-	git branch abc && test_path_is_file .git/refs/heads/abc
+	git branch abc &&
+	test_ref_exists refs/heads/abc
 '
 
 test_expect_success 'git branch abc should fail when abc exists' '
@@ -61,11 +62,13 @@ test_expect_success 'git branch --force abc should succeed when abc exists' '
 '
 
 test_expect_success 'git branch a/b/c should create a branch' '
-	git branch a/b/c && test_path_is_file .git/refs/heads/a/b/c
+	git branch a/b/c &&
+	test_ref_exists refs/heads/a/b/c
 '
 
 test_expect_success 'git branch mb main... should create a branch' '
-	git branch mb main... && test_path_is_file .git/refs/heads/mb
+	git branch mb main... &&
+	test_ref_exists refs/heads/mb
 '
 
 test_expect_success 'git branch HEAD should fail' '
@@ -78,14 +81,14 @@ EOF
 test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
-	test_path_is_file .git/refs/heads/d/e/f &&
+	test_ref_exists refs/heads/d/e/f &&
 	test_path_is_file .git/logs/refs/heads/d/e/f &&
 	test_cmp expect .git/logs/refs/heads/d/e/f
 '
 
 test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
 	git branch -d d/e/f &&
-	test_path_is_missing .git/refs/heads/d/e/f &&
+	test_ref_missing refs/heads/d/e/f &&
 	test_must_fail git reflog exists refs/heads/d/e/f
 '
 
@@ -213,7 +216,7 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 		test_commit initial &&
 		git checkout --orphan lonely &&
 		grep lonely .git/HEAD &&
-		test_path_is_missing .git/refs/head/lonely &&
+		test_ref_missing refs/head/lonely &&
 		git branch -M main mistress &&
 		grep lonely .git/HEAD
 	)
@@ -799,8 +802,8 @@ test_expect_success 'deleting a symref' '
 	git symbolic-ref refs/heads/symref refs/heads/target &&
 	echo "Deleted branch symref (was refs/heads/target)." >expect &&
 	git branch -d symref >actual &&
-	test_path_is_file .git/refs/heads/target &&
-	test_path_is_missing .git/refs/heads/symref &&
+	test_ref_exists refs/heads/target &&
+	test_ref_missing refs/heads/symref &&
 	test_cmp expect actual
 '
 
@@ -809,16 +812,16 @@ test_expect_success 'deleting a dangling symref' '
 	test_path_is_file .git/refs/heads/dangling-symref &&
 	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
 	git branch -d dangling-symref >actual &&
-	test_path_is_missing .git/refs/heads/dangling-symref &&
+	test_ref_missing refs/heads/dangling-symref &&
 	test_cmp expect actual
 '
 
 test_expect_success 'deleting a self-referential symref' '
 	git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
-	test_path_is_file .git/refs/heads/self-reference &&
+	test_ref_exists refs/heads/self-reference &&
 	echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
 	git branch -d self-reference >actual &&
-	test_path_is_missing .git/refs/heads/self-reference &&
+	test_ref_missing refs/heads/self-reference &&
 	test_cmp expect actual
 '
 
@@ -826,8 +829,8 @@ test_expect_success 'renaming a symref is not allowed' '
 	git symbolic-ref refs/heads/topic refs/heads/main &&
 	test_must_fail git branch -m topic new-topic &&
 	git symbolic-ref refs/heads/topic &&
-	test_path_is_file .git/refs/heads/main &&
-	test_path_is_missing .git/refs/heads/new-topic
+	test_ref_exists refs/heads/main &&
+	test_ref_missing refs/heads/new-topic
 '
 
 test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
@@ -1142,7 +1145,7 @@ EOF
 test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git checkout -b g/h/i -l main &&
-	test_path_is_file .git/refs/heads/g/h/i &&
+	test_ref_exists refs/heads/g/h/i &&
 	test_path_is_file .git/logs/refs/heads/g/h/i &&
 	test_cmp expect .git/logs/refs/heads/g/h/i
 '
diff --git a/t/t5521-pull-options.sh b/t/t5521-pull-options.sh
index 079b2f2536e..3681859f983 100755
--- a/t/t5521-pull-options.sh
+++ b/t/t5521-pull-options.sh
@@ -143,7 +143,7 @@ test_expect_success 'git pull --dry-run' '
 		cd clonedry &&
 		git pull --dry-run ../parent &&
 		test_path_is_missing .git/FETCH_HEAD &&
-		test_path_is_missing .git/refs/heads/main &&
+		test_ref_missing refs/heads/main &&
 		test_path_is_missing .git/index &&
 		test_path_is_missing file
 	)
@@ -157,7 +157,7 @@ test_expect_success 'git pull --all --dry-run' '
 		git remote add origin ../parent &&
 		git pull --all --dry-run &&
 		test_path_is_missing .git/FETCH_HEAD &&
-		test_path_is_missing .git/refs/remotes/origin/main &&
+		test_ref_missing refs/remotes/origin/main &&
 		test_path_is_missing .git/index &&
 		test_path_is_missing file
 	)
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index 1d7b1abda1a..946c5751885 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -69,7 +69,7 @@ test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
 	git clone a d &&
 	(cd d &&
 	git fetch &&
-	test ! -e .git/refs/remotes/origin/HEAD)
+	test_ref_missing refs/remotes/origin/HEAD)
 '
 
 test_expect_success 'bundle clone without .bundle suffix' '
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 04/11] t: convert tests to not write references via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 9212 bytes --]

Some of our tests manually create, update or delete references by
writing the respective new values into the filesystem directly. While
this works with the current files reference backend, this will break
once we have a second reference backend implementation in our codebase.

Refactor these tests to instead use git-update-ref(1) or our `ref-store`
test tool. The latter is required in some cases where safety checks of
git-update-ref(1) would otherwise reject a reference update.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh         | 16 +++++++---------
 t/t1450-fsck.sh               | 28 +++++++++++++++-------------
 t/t3404-rebase-interactive.sh |  2 +-
 t/t5526-fetch-submodules.sh   |  2 +-
 4 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 4d66cd7f4a1..cd24018ce99 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -9,8 +9,6 @@ test_description='Test git update-ref and basic ref logging'
 Z=$ZERO_OID
 
 m=refs/heads/main
-n_dir=refs/heads/gu
-n=$n_dir/fixes
 outside=refs/foo
 bare=bare-repo
 
@@ -62,10 +60,10 @@ test_expect_success "delete $m without oldvalue verification" '
 	test_must_fail git show-ref --verify -q $m
 '
 
-test_expect_success "fail to create $n" '
-	test_when_finished "rm -f .git/$n_dir" &&
-	touch .git/$n_dir &&
-	test_must_fail git update-ref $n $A
+test_expect_success "fail to create $n due to file/directory conflict" '
+	test_when_finished "git update-ref -d refs/heads/gu" &&
+	git update-ref refs/heads/gu $A &&
+	test_must_fail git update-ref refs/heads/gu/fixes $A
 '
 
 test_expect_success "create $m (by HEAD)" '
@@ -222,7 +220,7 @@ test_expect_success 'delete symref without dereference when the referred ref is
 
 test_expect_success 'update-ref -d is not confused by self-reference' '
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
 	test_path_is_file .git/refs/heads/self
@@ -230,7 +228,7 @@ test_expect_success 'update-ref -d is not confused by self-reference' '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
@@ -434,7 +432,7 @@ test_expect_success 'Query "main@{2005-05-28}" (past end of history)' '
 	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
 '
 
-rm -f .git/$m .git/logs/$m expect
+git update-ref -d $m
 
 test_expect_success 'creating initial files' '
 	test_when_finished rm -f M &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 10a539158c4..5cce24f1006 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -115,15 +115,16 @@ test_expect_success 'zlib corrupt loose object output ' '
 '
 
 test_expect_success 'branch pointing to non-commit' '
-	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
+	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
 	test_when_finished "git update-ref -d refs/heads/invalid" &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "not a commit" out
 '
 
 test_expect_success 'HEAD link pointing at a funny object' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	saved_head=$(git rev-parse --verify HEAD) &&
+	test_when_finished "git update-ref HEAD ${saved_head}" &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -131,8 +132,8 @@ test_expect_success 'HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'HEAD link pointing at a funny place' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	saved_head=$(git rev-parse --verify HEAD) &&
+	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&
 	echo "ref: refs/funny/place" >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -140,10 +141,10 @@ test_expect_success 'HEAD link pointing at a funny place' '
 '
 
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
+	saved_head=$(git rev-parse --verify HEAD) &&
+	test_when_finished "git update-ref HEAD $saved_head" &&
 	test_when_finished "rm -rf .git/worktrees wt" &&
 	git worktree add wt &&
-	mv .git/HEAD .git/SAVED_HEAD &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail git -C wt fsck 2>out &&
@@ -161,7 +162,8 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 test_expect_success 'other worktree HEAD link pointing at missing object' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD &&
+	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
+	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD: invalid sha1 pointer" out
 '
@@ -391,7 +393,7 @@ test_expect_success 'tag pointing to nonexistent' '
 
 	tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/invalid &&
+	git update-ref refs/tags/invalid $tag &&
 	test_when_finished "git update-ref -d refs/tags/invalid" &&
 	test_must_fail git fsck --tags >out &&
 	test_i18ngrep "broken link" out
@@ -411,7 +413,7 @@ test_expect_success 'tag pointing to something else than its type' '
 
 	tag=$(git hash-object -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags
 '
@@ -428,7 +430,7 @@ test_expect_success 'tag with incorrect tag name & missing tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	git fsck --tags 2>out &&
 
@@ -452,7 +454,7 @@ test_expect_success 'tag with bad tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag .*: invalid author/committer" out
@@ -471,7 +473,7 @@ test_expect_success 'tag with NUL in header' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <tag-NUL-header) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag $tag.*unterminated header: NUL at offset" out
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 8ea2bf13026..d2a7a91f170 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -2160,7 +2160,7 @@ test_expect_success '--update-refs: check failed ref update' '
 	# recorded in the update-refs file. We will force-update the
 	# "second" ref, but "git branch -f" will not work because of
 	# the lock in the update-refs file.
-	git rev-parse third >.git/refs/heads/second &&
+	git update-ref refs/heads/second third &&
 
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 26e933f93ae..7ab220fa313 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -771,7 +771,7 @@ test_expect_success 'fetching submodule into a broken repository' '
 	git -C dst fetch --recurse-submodules &&
 
 	# Break the receiving submodule
-	rm -f dst/sub/.git/HEAD &&
+	test-tool -C dst/sub ref-store main delete-refs REF_NO_DEREF msg HEAD &&
 
 	# NOTE: without the fix the following tests will recurse forever!
 	# They should terminate with an error.
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 05/11] t: convert tests to not access symrefs via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 13718 bytes --]

Some of our tests access symbolic references via the filesystem
directly. While this works with the current files reference backend, it
this will break once we have a second reference backend in our codebase.

Refactor these tests to instead use git-symbolic-ref(1) or our
`ref-store` test tool. The latter is required in some cases where safety
checks of git-symbolic-ref(1) would otherwise reject writing a symbolic
reference.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh              |  8 ++++----
 t/t1430-bad-ref-name.sh            | 12 ++++++------
 t/t1450-fsck.sh                    |  4 ++--
 t/t3200-branch.sh                  |  9 ++++++---
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t5605-clone-local.sh             |  2 +-
 t/t5702-protocol-v2.sh             | 24 ++++++++++++++++++------
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 9 files changed, 40 insertions(+), 25 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index cd24018ce99..5f505e2f353 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -221,15 +221,15 @@ test_expect_success 'delete symref without dereference when the referred ref is
 test_expect_success 'update-ref -d is not confused by self-reference' '
 	git symbolic-ref refs/heads/self refs/heads/self &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
-	test_path_is_file .git/refs/heads/self
+	git symbolic-ref --no-recurse refs/heads/self
 '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	git symbolic-ref refs/heads/self refs/heads/self &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
@@ -239,7 +239,7 @@ test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
 	test_when_finished "git update-ref -d refs/heads/ref-to-bad" &&
-	test_path_is_file .git/refs/heads/ref-to-bad &&
+	git symbolic-ref --no-recurse refs/heads/ref-to-bad &&
 	git update-ref --no-deref -d refs/heads/ref-to-bad &&
 	test_must_fail git show-ref --verify -q refs/heads/ref-to-bad
 '
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index 7b7d6953c62..5debb91f7b7 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -164,9 +164,9 @@ test_expect_success 'rev-parse skips symref pointing to broken name' '
 test_expect_success 'for-each-ref emits warnings for broken names' '
 	test-tool ref-store main update-ref msg "refs/heads/broken...ref" $main_sha1 $ZERO_OID REF_SKIP_REFNAME_VERIFICATION &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
-	printf "ref: refs/heads/broken...ref\n" >.git/refs/heads/badname &&
+	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git for-each-ref >output 2>error &&
 	! grep -e "broken\.\.\.ref" output &&
@@ -257,7 +257,7 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
@@ -267,7 +267,7 @@ test_expect_success 'update-ref --no-deref -d can delete symref with broken name
 '
 
 test_expect_success 'branch -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
@@ -277,7 +277,7 @@ test_expect_success 'branch -d can delete symref with broken name' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
@@ -287,7 +287,7 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
 '
 
 test_expect_success 'branch -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 5cce24f1006..804a5594ddd 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -134,7 +134,7 @@ test_expect_success 'HEAD link pointing at a funny object' '
 test_expect_success 'HEAD link pointing at a funny place' '
 	saved_head=$(git rev-parse --verify HEAD) &&
 	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&
-	echo "ref: refs/funny/place" >.git/HEAD &&
+	test-tool ref-store main create-symref HEAD refs/funny/place &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
 	test_i18ngrep "HEAD points to something strange" out
@@ -171,7 +171,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "ref: refs/funny/place" >.git/worktrees/other/HEAD &&
+	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD points to something strange" out
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index bde4f1485b7..874520e3f10 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -215,10 +215,13 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 		cd orphan &&
 		test_commit initial &&
 		git checkout --orphan lonely &&
-		grep lonely .git/HEAD &&
+		git symbolic-ref HEAD >expect &&
+		echo refs/heads/lonely >actual &&
+		test_cmp expect actual &&
 		test_ref_missing refs/head/lonely &&
 		git branch -M main mistress &&
-		grep lonely .git/HEAD
+		git symbolic-ref HEAD >expect &&
+		test_cmp expect actual
 	)
 '
 
@@ -809,7 +812,7 @@ test_expect_success 'deleting a symref' '
 
 test_expect_success 'deleting a dangling symref' '
 	git symbolic-ref refs/heads/dangling-symref nowhere &&
-	test_path_is_file .git/refs/heads/dangling-symref &&
+	git symbolic-ref --no-recurse refs/heads/dangling-symref &&
 	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
 	git branch -d dangling-symref >actual &&
 	test_ref_missing refs/heads/dangling-symref &&
diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
index 5de1d190759..5abbea36b39 100755
--- a/t/t4013-diff-various.sh
+++ b/t/t4013-diff-various.sh
@@ -514,7 +514,7 @@ test_expect_success 'log -S requires an argument' '
 '
 
 test_expect_success 'diff --cached on unborn branch' '
-	echo ref: refs/heads/unborn >.git/HEAD &&
+	git symbolic-ref HEAD refs/heads/unborn &&
 	git diff --cached >result &&
 	process_diffs result >actual &&
 	process_diffs "$TEST_DIRECTORY/t4013/diff.diff_--cached" >expected &&
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index af4a123cd22..57b298a4e22 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2265,7 +2265,7 @@ test_expect_success REFFILES 'log diagnoses bogus HEAD hash' '
 
 test_expect_success REFFILES 'log diagnoses bogus HEAD symref' '
 	git init empty &&
-	echo "ref: refs/heads/invalid.lock" > empty/.git/HEAD &&
+	test-tool -C empty ref-store main create-symref HEAD refs/heads/invalid.lock &&
 	test_must_fail git -C empty log 2>stderr &&
 	test_i18ngrep broken stderr &&
 	test_must_fail git -C empty log --default totally-bogus 2>stderr &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index 946c5751885..bedd29d0550 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -65,7 +65,7 @@ test_expect_success 'Even without -l, local will make a hardlink' '
 '
 
 test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
-	echo "ref: refs/heads/nonexistent" > a.git/HEAD &&
+	git -C a.git symbolic-ref HEAD refs/heads/nonexistent &&
 	git clone a d &&
 	(cd d &&
 	git fetch &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 6af5c2062fd..dcc4cd95fe7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -221,7 +221,9 @@ test_expect_success 'clone of empty repo propagates name of default branch' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/mydefaultbranch >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success '...but not if explicitly forbidden by config' '
@@ -234,7 +236,9 @@ test_expect_success '...but not if explicitly forbidden by config' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	! grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/main >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'bare clone propagates empty default branch' '
@@ -247,7 +251,9 @@ test_expect_success 'bare clone propagates empty default branch' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare \
 		"file://$(pwd)/file_empty_parent" file_empty_child.git &&
-	grep "refs/heads/mydefaultbranch" file_empty_child.git/HEAD
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_empty_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
@@ -265,7 +271,9 @@ test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	grep "warning: remote HEAD refers to nonexistent ref" stderr
 '
 
@@ -295,7 +303,9 @@ test_expect_success 'bare clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child.git 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	! grep "warning:" stderr
 '
 
@@ -315,7 +325,9 @@ test_expect_success 'defaulted HEAD uses remote branch if available' '
 	git -c init.defaultBranch=branchwithstuff -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/branchwithstuff" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/branchwithstuff" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	test_path_is_file file_unborn_child/stuff.t &&
 	! grep "warning:" stderr
 '
diff --git a/t/t9133-git-svn-nested-git-repo.sh b/t/t9133-git-svn-nested-git-repo.sh
index d8d536269cf..8ca24670acb 100755
--- a/t/t9133-git-svn-nested-git-repo.sh
+++ b/t/t9133-git-svn-nested-git-repo.sh
@@ -11,7 +11,7 @@ test_expect_success 'setup repo with a git repo inside it' '
 	(
 		cd s &&
 		git init &&
-		test -f .git/HEAD &&
+		git symbolic-ref HEAD &&
 		> .git/a &&
 		echo a > a &&
 		svn_cmd add .git a &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 06/11] t: convert tests to not access reflog via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 7125 bytes --]

Some of our tests reach directly into the filesystem in order to both
read or modify the reflog, which will break once we have a second
reference backend in our codebase that stores reflogs differently.

Refactor these tests to either use git-reflog(1) or the ref-store test
helper. Note that the refactoring to use git-reflog(1) also requires us
to adapt our expectations in some cases where we previously verified the
exact on-disk log entries. This seems like an acceptable tradeoff though
to ensure that different backends have the same user-visible behaviour
as any user would typically use git-reflog(1) anyway to access the logs.
Any backend-specific verification of the written on-disk format should
be implemented in a separate, backend-specific test.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh | 17 +++++++++++------
 t/t3200-branch.sh     | 24 ++++++++++++------------
 2 files changed, 23 insertions(+), 18 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 5f505e2f353..b1d2c014132 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -90,7 +90,8 @@ test_expect_success "deleting current branch adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-$m -d $m &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-$m$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-$m$" actual
 '
 
 test_expect_success "deleting by HEAD adds message to HEAD's log" '
@@ -99,7 +100,8 @@ test_expect_success "deleting by HEAD adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-by-head -d HEAD &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-by-head$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-by-head$" actual
 '
 
 test_expect_success 'update-ref does not create reflogs by default' '
@@ -130,7 +132,7 @@ test_expect_success 'creates no reflog in bare repository' '
 
 test_expect_success 'core.logAllRefUpdates=true creates reflog in bare repository' '
 	test_when_finished "git -C $bare config --unset core.logAllRefUpdates && \
-		rm $bare/logs/$m" &&
+		test-tool ref-store main delete-reflog $m" &&
 	git -C $bare config core.logAllRefUpdates true &&
 	git -C $bare update-ref $m $bareB &&
 	git -C $bare rev-parse $bareB >expect &&
@@ -263,7 +265,10 @@ test_expect_success "(not) changed .git/$m" '
 	! test $B = $(git show-ref -s --verify $m)
 '
 
-rm -f .git/logs/refs/heads/main
+test_expect_success "clean up reflog" '
+	test-tool ref-store main delete-reflog $m
+'
+
 test_expect_success "create $m (logged by touch)" '
 	test_config core.logAllRefUpdates false &&
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
@@ -316,7 +321,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150260 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000
 EOF
 test_expect_success "verifying $m's log (logged by touch)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
@@ -346,7 +351,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 +0000
 EOF
 test_expect_success "verifying $m's log (logged by config)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 874520e3f10..933aa9eebbd 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -76,14 +76,14 @@ test_expect_success 'git branch HEAD should fail' '
 '
 
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/d/e/f@{0}: branch: Created from main
 EOF
 test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
 	test_ref_exists refs/heads/d/e/f &&
-	test_path_is_file .git/logs/refs/heads/d/e/f &&
-	test_cmp expect .git/logs/refs/heads/d/e/f
+	git reflog show --no-abbrev-commit refs/heads/d/e/f >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
@@ -203,10 +203,9 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	test $(git rev-parse --abbrev-ref HEAD) = bam
 '
 
-test_expect_success 'git branch -M baz bam should add entries to .git/logs/HEAD' '
-	msg="Branch: renamed refs/heads/baz to refs/heads/bam" &&
-	grep " $ZERO_OID.*$msg$" .git/logs/HEAD &&
-	grep "^$ZERO_OID.*$msg$" .git/logs/HEAD
+test_expect_success 'git branch -M baz bam should add entries to HEAD reflog' '
+	git reflog show HEAD >actual &&
+	grep "HEAD@{0}: Branch: renamed refs/heads/baz to refs/heads/bam" actual
 '
 
 test_expect_success 'git branch -M should leave orphaned HEAD alone' '
@@ -228,7 +227,7 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 test_expect_success 'resulting reflog can be shown by log -g' '
 	oid=$(git rev-parse HEAD) &&
 	cat >expect <<-EOF &&
-	HEAD@{0} $oid $msg
+	HEAD@{0} $oid Branch: renamed refs/heads/baz to refs/heads/bam
 	HEAD@{2} $oid checkout: moving from foo to baz
 	EOF
 	git log -g --format="%gd %H %gs" -2 HEAD >actual &&
@@ -702,7 +701,8 @@ test_expect_success 'git branch -C c1 c2 should succeed when c1 is checked out'
 
 test_expect_success 'git branch -C c1 c2 should never touch HEAD' '
 	msg="Branch: copied refs/heads/c1 to refs/heads/c2" &&
-	! grep "$msg$" .git/logs/HEAD
+	git reflog HEAD >actual &&
+	! grep "$msg$" actual
 '
 
 test_expect_success 'git branch -C main should work when main is checked out' '
@@ -1143,14 +1143,14 @@ test_expect_success '--set-upstream-to notices an error to set branch as own ups
 
 # Keep this test last, as it changes the current branch
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/g/h/i@{0}: branch: Created from main
 EOF
 test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git checkout -b g/h/i -l main &&
 	test_ref_exists refs/heads/g/h/i &&
-	test_path_is_file .git/logs/refs/heads/g/h/i &&
-	test_cmp expect .git/logs/refs/heads/g/h/i
+	git reflog show --no-abbrev-commit refs/heads/g/h/i >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'checkout -b makes reflog by default' '
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 07/11] t1450: convert tests to remove worktrees via git-worktree(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2562 bytes --]

Some of our tests in t1450 create worktrees and then corrupt them.
As it is impossible to delete such worktrees via a normal call to `git
worktree remove`, we instead opt to remove them manually by calling
rm(1) instead.

This is ultimately unnecessary though as we can use the `-f` switch to
remove the worktree. Let's convert the tests to do so such that we don't
have to reach into internal implementation details of worktrees.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1450-fsck.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 804a5594ddd..d356605d132 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -143,7 +143,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	saved_head=$(git rev-parse --verify HEAD) &&
 	test_when_finished "git update-ref HEAD $saved_head" &&
-	test_when_finished "rm -rf .git/worktrees wt" &&
+	test_when_finished "git worktree remove -f wt" &&
 	git worktree add wt &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
@@ -152,7 +152,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
 	test_must_fail git fsck 2>out &&
@@ -160,7 +160,7 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at missing object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
 	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
@@ -169,7 +169,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 08/11] t4207: delete replace references via git-update-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1752 bytes --]

In t4207 we set up a set of replace objects via git-replace(1). Because
these references should not be impacting subsequent tests we also set up
some cleanup logic that deletes the replacement references via a call to
`rm -rf`. This reaches into the internal implementation details of the
reference backend and will thus break when we grow an alternative refdb
implementation.

Refactor the tests to delete the replacement refs via Git commands so
that we become independent of the actual refdb that's in use. As we
don't have a nice way to delete all replacements or all references in a
certain namespace, we opt for a combination of git-for-each-ref(1) and
git-update-ref(1)'s `--stdin` mode.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t4207-log-decoration-colors.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
index 21986a866df..d138e513a04 100755
--- a/t/t4207-log-decoration-colors.sh
+++ b/t/t4207-log-decoration-colors.sh
@@ -71,7 +71,7 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
 '
 
 test_expect_success 'test coloring with replace-objects' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
 	test_commit C &&
 	test_commit D &&
 
@@ -99,7 +99,7 @@ EOF
 '
 
 test_expect_success 'test coloring with grafted commit' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
 
 	git replace --graft HEAD HEAD~2 &&
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 09/11] t7300: assert exact states of repo

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]

Some of the tests in t7300 verify that git-clean(1) doesn't touch
repositories that are embedded into the main repository. This is done by
asserting a small set of substructures that are assumed to always exist,
like the "refs/", "objects/" or "HEAD". This has the downside that we
need to assume a specific repository structure that may be subject to
change when new backends for the refdb land. At the same time, we don't
thoroughly assert that git-clean(1) really didn't end up cleaning any
files in the repository either.

Convert the tests to instead assert that all files continue to exist
after git-clean(1) by comparing a file listing via find(1) before and
after executing clean. This makes our actual assertions stricter while
having to care less about the repository's actual on-disk format.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7300-clean.sh | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/t/t7300-clean.sh b/t/t7300-clean.sh
index 0ef7b784573..d7d9202f37f 100755
--- a/t/t7300-clean.sh
+++ b/t/t7300-clean.sh
@@ -517,8 +517,12 @@ test_expect_success 'nested (empty) git should be kept' '
 	git init empty_repo &&
 	mkdir to_clean &&
 	>to_clean/should_clean.this &&
+	# Note that we put the expect file in the .git directory so that it
+	# does not get cleaned.
+	find empty_repo | sort >.git/expect &&
 	git clean -f -d &&
-	test_path_is_file empty_repo/.git/HEAD &&
+	find empty_repo | sort >actual &&
+	test_cmp .git/expect actual &&
 	test_path_is_missing to_clean
 '
 
@@ -559,10 +563,10 @@ test_expect_success 'giving path in nested git work tree will NOT remove it' '
 		mkdir -p bar/baz &&
 		test_commit msg bar/baz/hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/bar/baz &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/bar/ &&
-	test_path_is_file repo/bar/baz/hello.world
+	find repo | sort >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'giving path to nested .git will not remove it' '
@@ -573,10 +577,10 @@ test_expect_success 'giving path to nested .git will not remove it' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/.git/refs &&
-	test_path_is_dir repo/.git/objects &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
@@ -588,9 +592,10 @@ test_expect_success 'giving path to nested .git/ will NOT remove contents' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git/ &&
-	test_path_is_dir repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 10/11] t7900: assert the absence of refs via git-for-each-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

We're asserting that a prefetch of remotes via git-maintenance(1)
doesn't write any references in refs/remotes by validating that the
directory ".git/refs/remotes" is missing. This is quite roundabout: we
don't care about the directory existing, we care about the references
not existing, and the way these are stored is on the behest of the
reference database.

Convert the test to instead check via git-for-each-ref(1) whether any
remote reference exist.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7900-maintenance.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index e56f5980dc4..cefecee732f 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -157,7 +157,8 @@ test_expect_success 'prefetch multiple remotes' '
 	fetchargs="--prefetch --prune --no-tags --no-write-fetch-head --recurse-submodules=no --quiet" &&
 	test_subcommand git fetch remote1 $fetchargs <run-prefetch.txt &&
 	test_subcommand git fetch remote2 $fetchargs <run-prefetch.txt &&
-	test_path_is_missing .git/refs/remotes &&
+	git for-each-ref refs/remotes >actual &&
+	test_must_be_empty actual &&
 	git log prefetch/remotes/remote1/one &&
 	git log prefetch/remotes/remote2/two &&
 	git fetch --all &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH 11/11] t: mark several tests that assume the files backend with REFFILES

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 8915 bytes --]

Add the REFFILES prerequisite to several tests that assume we're using
the files backend. There are various reasons why we cannot easily
convert those tests to be backend-independent, where the most common
one is that we have no way to write corrupt references into the refdb
via our tooling. We may at a later point in time grow the tooling to
make this possible, but for now we just mark these tests as requiring
the files backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh            |  8 ++++----
 t/t1450-fsck.sh                  |  6 +++---
 t/t2011-checkout-invalid-head.sh | 16 ++++++++--------
 t/t3200-branch.sh                |  8 ++++----
 t/t3400-rebase.sh                |  2 +-
 t/t5605-clone-local.sh           |  2 +-
 6 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index b1d2c014132..feeba6ed7cd 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -236,7 +236,7 @@ test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
 
-test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+test_expect_success REFFILES 'update-ref --no-deref -d can delete reference to bad ref' '
 	>.git/refs/heads/bad &&
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
@@ -288,7 +288,7 @@ test_expect_success "set $m (logged by touch)" '
 	test $A = $(git show-ref -s --verify $m)
 '
 
-test_expect_success 'empty directory removal' '
+test_expect_success REFFILES 'empty directory removal' '
 	git branch d1/d2/r1 HEAD &&
 	git branch d1/r2 HEAD &&
 	test_path_is_file .git/refs/heads/d1/d2/r1 &&
@@ -300,7 +300,7 @@ test_expect_success 'empty directory removal' '
 	test_path_is_file .git/logs/refs/heads/d1/r2
 '
 
-test_expect_success 'symref empty directory removal' '
+test_expect_success REFFILES 'symref empty directory removal' '
 	git branch e1/e2/r1 HEAD &&
 	git branch e1/r2 HEAD &&
 	git checkout e1/e2/r1 &&
@@ -1638,7 +1638,7 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
-test_expect_success 'directory not created deleting packed ref' '
+test_expect_success REFFILES 'directory not created deleting packed ref' '
 	git branch d1/d2/r1 HEAD &&
 	git pack-refs --all &&
 	test_path_is_missing .git/refs/heads/d1/d2 &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index d356605d132..ca5a2719a19 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -122,7 +122,7 @@ test_expect_success 'branch pointing to non-commit' '
 	test_i18ngrep "not a commit" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object' '
 	saved_head=$(git rev-parse --verify HEAD) &&
 	test_when_finished "git update-ref HEAD ${saved_head}" &&
 	echo $ZERO_OID >.git/HEAD &&
@@ -140,7 +140,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 	test_i18ngrep "HEAD points to something strange" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object (from different wt)' '
 	saved_head=$(git rev-parse --verify HEAD) &&
 	test_when_finished "git update-ref HEAD $saved_head" &&
 	test_when_finished "git worktree remove -f wt" &&
@@ -151,7 +151,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	test_i18ngrep "main-worktree/HEAD: detached HEAD points" out
 '
 
-test_expect_success 'other worktree HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'other worktree HEAD link pointing at a funny object' '
 	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
diff --git a/t/t2011-checkout-invalid-head.sh b/t/t2011-checkout-invalid-head.sh
index d9997e7b6b4..3c8135831b8 100755
--- a/t/t2011-checkout-invalid-head.sh
+++ b/t/t2011-checkout-invalid-head.sh
@@ -18,18 +18,18 @@ test_expect_success 'checkout should not start branch from a tree' '
 	test_must_fail git checkout -b newbranch main^{tree}
 '
 
-test_expect_success 'checkout main from invalid HEAD' '
+test_expect_success REFFILES 'checkout main from invalid HEAD' '
 	echo $ZERO_OID >.git/HEAD &&
 	git checkout main --
 '
 
-test_expect_success 'checkout notices failure to lock HEAD' '
+test_expect_success REFFILES 'checkout notices failure to lock HEAD' '
 	test_when_finished "rm -f .git/HEAD.lock" &&
 	>.git/HEAD.lock &&
 	test_must_fail git checkout -b other
 '
 
-test_expect_success 'create ref directory/file conflict scenario' '
+test_expect_success REFFILES 'create ref directory/file conflict scenario' '
 	git update-ref refs/heads/outer/inner main &&
 
 	# do not rely on symbolic-ref to get a known state,
@@ -39,26 +39,26 @@ test_expect_success 'create ref directory/file conflict scenario' '
 	}
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
 
-test_expect_success 'pack refs' '
+test_expect_success REFFILES 'pack refs' '
 	git pack-refs --all --prune
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 933aa9eebbd..606c50fe66c 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -28,7 +28,7 @@ test_expect_success 'git branch --help should not have created a bogus branch' '
 	test_ref_missing refs/heads/--help
 '
 
-test_expect_success 'branch -h in broken repository' '
+test_expect_success REFFILES 'branch -h in broken repository' '
 	mkdir broken &&
 	(
 		cd broken &&
@@ -245,7 +245,7 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	git worktree prune
 '
 
-test_expect_success 'git branch -M fails if updating any linked working tree fails' '
+test_expect_success REFFILES 'git branch -M fails if updating any linked working tree fails' '
 	git worktree add -b baz bazdir1 &&
 	git worktree add -f bazdir2 baz &&
 	touch .git/worktrees/bazdir1/HEAD.lock &&
@@ -836,14 +836,14 @@ test_expect_success 'renaming a symref is not allowed' '
 	test_ref_missing refs/heads/new-topic
 '
 
-test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m u v should fail when the reflog for u is a symlink' '
 	git branch --create-reflog u &&
 	mv .git/logs/refs/heads/u real-u &&
 	ln -s real-u .git/logs/refs/heads/u &&
 	test_must_fail git branch -m u v
 '
 
-test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m with symlinked .git/refs' '
 	test_when_finished "rm -rf subdir" &&
 	git init --bare subdir &&
 
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index d3df19a51f8..435943a0891 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -424,7 +424,7 @@ test_expect_success 'refuse to switch to branch checked out elsewhere' '
 	test_i18ngrep "already used by worktree at" err
 '
 
-test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
+test_expect_success REFFILES,MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
 	git checkout main &&
 	mv .git/logs actual_logs &&
 	cmd //c "mklink /D .git\logs ..\actual_logs" &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index bedd29d0550..a3055869bc7 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -157,7 +157,7 @@ test_expect_success 'cloning locally respects "-u" for fetching refs' '
 	test_must_fail git clone --bare -u false a should_not_work.git
 '
 
-test_expect_success 'local clone from repo with corrupt refs fails gracefully' '
+test_expect_success REFFILES 'local clone from repo with corrupt refs fails gracefully' '
 	git init corrupt &&
 	test_commit -C corrupt one &&
 	echo a >corrupt/.git/refs/heads/topic &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3303 bytes --]

On Wed, Oct 18, 2023 at 07:35:03AM +0200, Patrick Steinhardt wrote:
> Hi,
> 
> this patch series refactors a bunch of our tests to perform less direct
> disk access to on-disk data structures. Instead, the tests are converted
> to use Git tools or our test-tool to access data to the best extent
> possible. This serves two benefits:
> 
>     - We increase test coverage of our own code base.
> 
>     - We become less dependent on the actual on-disk format.
> 
> The main motivation for this patch series was the second bullet point as
> it is preparatory work to get the reftable backend upstreamed. My intent
> is to get rid of many or even most of the current blockers in the Git
> project before trying to send the reftable implementation upstream.
> While this will be a lot of up-front work that is going to span over a
> long time period, I think this approach will make everyones live easier
> by doing comparatively small and incremental improvements to the Git
> project. Ultimately, the final patch series should in the best case only
> contain the new backend as well as testing infrastructure, but not much
> else.
> 
> Patrick

As usual, I forgot to mention that this applies on top of current
master, which is at a9ecda2788 (The eighteenth batch, 2023-10-13) at the
time of writing. I look forward to the day where I remember to include
this information in the cover letter...

Patrick

> Patrick Steinhardt (11):
>   t: add helpers to test for reference existence
>   t: allow skipping expected object ID in `ref-store update-ref`
>   t: convert tests to use helpers for reference existence
>   t: convert tests to not write references via the filesystem
>   t: convert tests to not access symrefs via the filesystem
>   t: convert tests to not access reflog via the filesystem
>   t1450: convert tests to remove worktrees via git-worktree(1)
>   t4207: delete replace references via git-update-ref(1)
>   t7300: assert exact states of repo
>   t7900: assert the absence of refs via git-for-each-ref(1)
>   t: mark several tests that assume the files backend with REFFILES
> 
>  t/README                           |  9 ++++
>  t/helper/test-ref-store.c          | 38 +++++++++++++--
>  t/t1400-update-ref.sh              | 49 ++++++++++----------
>  t/t1430-bad-ref-name.sh            | 39 ++++++++++------
>  t/t1450-fsck.sh                    | 46 ++++++++++---------
>  t/t2011-checkout-invalid-head.sh   | 16 +++----
>  t/t3200-branch.sh                  | 74 ++++++++++++++++--------------
>  t/t3400-rebase.sh                  |  2 +-
>  t/t3404-rebase-interactive.sh      |  2 +-
>  t/t4013-diff-various.sh            |  2 +-
>  t/t4202-log.sh                     |  2 +-
>  t/t4207-log-decoration-colors.sh   |  4 +-
>  t/t5521-pull-options.sh            |  4 +-
>  t/t5526-fetch-submodules.sh        |  2 +-
>  t/t5605-clone-local.sh             |  6 +--
>  t/t5702-protocol-v2.sh             | 24 +++++++---
>  t/t7300-clean.sh                   | 23 ++++++----
>  t/t7900-maintenance.sh             |  3 +-
>  t/t9133-git-svn-nested-git-repo.sh |  2 +-
>  t/test-lib-functions.sh            | 66 ++++++++++++++++++++++++++
>  20 files changed, 277 insertions(+), 136 deletions(-)
> 
> -- 
> 2.42.0
> 



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 08/11] t4207: delete replace references via git-update-ref(1)

[Han-Wen Nienhuys]

On Wed, Oct 18, 2023 at 7:35 AM Patrick Steinhardt <ps@pks.im> wrote:
>
> In t4207 we set up a set of replace objects via git-replace(1). Because
> these references should not be impacting subsequent tests we also set up
> some cleanup logic that deletes the replacement references via a call to
> `rm -rf`. This reaches into the internal implementation details of the
> reference backend and will thus break when we grow an alternative refdb
> implementation.
>
> Refactor the tests to delete the replacement refs via Git commands so
> that we become independent of the actual refdb that's in use. As we
> don't have a nice way to delete all replacements or all references in a
> certain namespace, we opt for a combination of git-for-each-ref(1) and
> git-update-ref(1)'s `--stdin` mode.

There is a test helper that can directly access the ref database,
t/helper/test-ref-store.c.

If you use that manipulate refs for testing purposes, you make the
test independent of behavior
git-for-each-ref/git-update-ref, which is what you want for testing
replace-objects?

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Liana Sebastian

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> this patch series refactors a bunch of our tests to perform less direct
> disk access to on-disk data structures. Instead, the tests are converted
> to use Git tools or our test-tool to access data to the best extent
> possible. This serves two benefits:

Laudable goal.

>     - We increase test coverage of our own code base.

Meaning the new code added to test-tool for this series will also
get tested and bugs spotted?

>     - We become less dependent on the actual on-disk format.

Yes, this is very desirable.  Without looking at the implementation,
I see some issues aiming for this goal may involve. [a] using the
production code for validation would mean our expectation to be
compared to the reality to be validated can be affected by the same
bug, making two wrongs to appear right; [b] using a separate
implementation used only for validation would at least mean we will
have to make the same mistake in unique part of both implementations
that is less likely to miss bugs compared to [a], but bugs in shared
part of the production code and validation code will be hidden the
same way as [a].

But you have thought about this series a lot deeper and longer than
I have, so let me read on and find out what your solution is.


> The main motivation for this patch series was the second bullet point as
> it is preparatory work to get the reftable backend upstreamed.

Yay.

> My intent
> is to get rid of many or even most of the current blockers in the Git
> project before trying to send the reftable implementation upstream.
> While this will be a lot of up-front work that is going to span over a
> long time period, I think this approach will make everyones live easier
> by doing comparatively small and incremental improvements to the Git
> project.

Thank you very much for being considerate.  I assume this approach
is what those in the Contributors' Summit agreed to be the best way
forward to do anything sizeable.

> Ultimately, the final patch series should in the best case only
> contain the new backend as well as testing infrastructure, but not much
> else.

;-)

Re: [PATCH 01/11] t: add helpers to test for reference existence

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> There are two major ways to check for the existence of a reference in
> our tests:
>
>     - `git rev-parse --verify` can be used to check for existence of a
>       reference. This only works in the case where the reference is well
>       formed though and resolves to an actual object ID. This does not
>       work with malformed reference names or invalid contents.
>
>     - `test_path_is_file` can be used to check for existence of a loose
>       reference if it is known to not resolve to an actual object ID. It
>       by necessity reaches into implementation details of the reference
>       backend though.

True.  It would be ideal if we can limit the use of latter when we
_care_ how the ref is stored (e.g., "we expect it to be stored as a
loose ref, not packed").  "The ref R at must be pointing at the
commit X" is better asserted by using the former (or "git show-ref")
as we not just only want to see the .git/refs/R file holding the
object name X, but also want to see the production git tools observe
the same---if what rev-parse or show-ref observes is different from
the expected state and they say ref R does not point at commit X, we
should complain (rev-parse or show-ref may be broken in the version
of Git being tested, but we can assume that their breakage will be
caught elsewhere in the test suite as well, so as long as we trust
them, using them as the validator is better than going into the
implementation detail and assuming things like "new refs always
appear as a loose ref" that we might want to change in the future).

> Similarly, there are two equivalent ways to check for the absence of a
> reference:
>
>     - `test_must_fail git rev-parse` can be used to check for the
>       absence of a reference. It could fail due to a number of reasons
>       though, and all of these reasons will be thrown into the same bag
>       as an absent reference.
>
>     - `test_path_is_missing` can be used to check explicitly for the
>       absence of a loose reference, but again reaches into internal
>       implementation details of the reference backend.
>
> So both our tooling to check for the presence and for the absence of
> references in tests is lacking as either failure cases are thrown into
> the same bag or we need to reach into internal implementation details of
> the respective reference backend.

> Introduce a new subcommand for our ref-store test helper that explicitly
> checks only for the presence or absence of a reference. This addresses
> these limitations:
>
>     - We can check for the presence of references with malformed names.

But for the purpose of tests, we can control the input.  When we
perform an operation that we expect a ref R to be created, we would
know R is well formed and we can validate using a tool that we know
would be broken when fed a malformed name.  So I do not see this as
a huge "limitation".

>     - We can check for the presence of references that don't resolve.

Do you mean a dangling symbolic ref?  We are using a wrong tool if
you are using rev-parse for that, aren't we?  Isn't symbolic-ref
there for us for this exact use case?  That is

>     - We can explicitly handle the case where a reference is missing by
>       special-casing ENOENT errors.

You probably know the error conditions refs_read_raw_ref() can be
broken better than I do, but this feels a bit too intimate with how
the method for the files backend happens to be implemented, which at
the same time, can risk that [a] other backends can implement their
"ref does not resolve to an object name---is it because it is
missing?" report incorrectly and [b] we would eventually want to
know error conditions other than "the ref requested is missing" and
at that point we would need more "special casing", which does not
smell easy to scale.

>     - We don't need to reach into implementation details of the backend,
>       which would allow us to use this helper for the future reftable
>       backend.

This is exactly what we want to aim for.

> Next to this subcommand we also provide two wrappers `test_ref_exists`
> and `test_ref_missing` that make the helper easier to use.

Hmmmm.  This may introduce "who watches the watchers" problem, no?
I briefly wondered if a better approach is to teach the production
code, e.g., rev-parse, to optionally give more detailed diag.  It
essentially may be the same (making the code in test-ref-store.c
added by this patch available from rev-parse, we would easily get
there), so I do not think the distinction matters.

> diff --git a/t/README b/t/README
> index 61080859899..779f7e7dd86 100644
> --- a/t/README
> +++ b/t/README
> @@ -928,6 +928,15 @@ see test-lib-functions.sh for the full list and their options.
>     committer times to defined state.  Subsequent calls will
>     advance the times by a fixed amount.
>  
> + - test_ref_exists <ref>, test_ref_missing <ref>
> +
> +   Check whether a reference exists or is missing. In contrast to
> +   git-rev-parse(1), these helpers also work with invalid reference
> +   names and references whose contents are unresolvable. The latter
> +   function also distinguishes generic errors from the case where a
> +   reference explicitly doesn't exist and is thus safer to use than
> +   `test_must_fail git rev-parse`.
> +
>   - test_commit <message> [<filename> [<contents>]]
>  
>     Creates a commit with the given message, committing the given
> diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> index 48552e6a9e0..7400f560ab6 100644
> --- a/t/helper/test-ref-store.c
> +++ b/t/helper/test-ref-store.c
> @@ -1,6 +1,6 @@
>  #include "test-tool.h"
>  #include "hex.h"
> -#include "refs.h"
> +#include "refs/refs-internal.h"
>  #include "setup.h"
>  #include "worktree.h"
>  #include "object-store-ll.h"
> @@ -221,6 +221,30 @@ static int cmd_verify_ref(struct ref_store *refs, const char **argv)
>  	return ret;
>  }
>  
> +static int cmd_ref_exists(struct ref_store *refs, const char **argv)
> +{
> +	const char *refname = notnull(*argv++, "refname");
> +	struct strbuf unused_referent = STRBUF_INIT;
> +	struct object_id unused_oid;
> +	unsigned int unused_type;
> +	int failure_errno;
> +
> +	if (refs_read_raw_ref(refs, refname, &unused_oid, &unused_referent,
> +			      &unused_type, &failure_errno)) {
> +		/*
> +		 * We handle ENOENT separately here such that it is possible to
> +		 * distinguish actually-missing references from any kind of
> +		 * generic error.
> +		 */
> +		if (failure_errno == ENOENT)
> +			return 17;

Can we tell between the cases where the ref itself is missing, and
the requested ref is symbolic and points at a missing ref?  This
particular case might be OK, but there may other cases where this
"special case" may not be narrow enough.

As long we are going to spend cycles to refine the classification of
error conditions, which is a very good thing to aim for the reason
described in the proposed log message, namely "rev-parse can fail
for reasons other than the ref being absent", I have to wonder again
that the fruit of such an effort should become available in the
production code, instead of being kept only in test-tool.

Re: [PATCH 02/11] t: allow skipping expected object ID in `ref-store update-ref`

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> We require the caller to pass both the old and new expected object ID to
> our `test-tool ref-store update-ref` helper. When trying to update a
> symbolic reference though it's impossible to specify the expected object
> ID, which means that the test would instead have to force-update the
> reference. This is currently impossible though.
>
> Update the helper to optionally skip verification of the old object ID
> in case the test passes in an empty old object ID as input.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/helper/test-ref-store.c | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)

Good.

Even better would be to make the old one optional, though.

> diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> index 7400f560ab6..7dc83137584 100644
> --- a/t/helper/test-ref-store.c
> +++ b/t/helper/test-ref-store.c
> @@ -322,16 +322,19 @@ static int cmd_update_ref(struct ref_store *refs, const char **argv)
>  	const char *new_sha1_buf = notnull(*argv++, "new-sha1");
>  	const char *old_sha1_buf = notnull(*argv++, "old-sha1");
>  	unsigned int flags = arg_flags(*argv++, "flags", transaction_flags);
> -	struct object_id old_oid;
> +	struct object_id old_oid, *old_oid_ptr = NULL;
>  	struct object_id new_oid;
>  
> -	if (get_oid_hex(old_sha1_buf, &old_oid))
> -		die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
> +	if (*old_sha1_buf) {
> +		if (get_oid_hex(old_sha1_buf, &old_oid))
> +			die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
> +		old_oid_ptr = &old_oid;
> +	}
>  	if (get_oid_hex(new_sha1_buf, &new_oid))
>  		die("cannot parse %s as %s", new_sha1_buf, the_hash_algo->name);
>  
>  	return refs_update_ref(refs, msg, refname,
> -			       &new_oid, &old_oid,
> +			       &new_oid, old_oid_ptr,
>  			       flags, UPDATE_REFS_DIE_ON_ERR);
>  }

Re: [PATCH 03/11] t: convert tests to use helpers for reference existence

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> Convert tests that use `test_path_is_file` and `test_path_is_missing` to
> instead use our new helpers `test_ref_exists` and `test_ref_missing`.
> These hook into the reference database directly and thus work indepently
> of the actual reference backend that is being used.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/t1430-bad-ref-name.sh | 27 ++++++++++++++++++---------
>  t/t3200-branch.sh       | 33 ++++++++++++++++++---------------
>  t/t5521-pull-options.sh |  4 ++--
>  t/t5605-clone-local.sh  |  2 +-
>  4 files changed, 39 insertions(+), 27 deletions(-)

I scanned through the changes, and all of them looked sensible.  I
also very much liked the added "now we have done something to create
'badname' branch, let's make sure it exists, before trying to see if
the machinery to delete it works correctly" tests.

Looking good.

Thanks.

> diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
> index ff1c967d550..7b7d6953c62 100755
> --- a/t/t1430-bad-ref-name.sh
> +++ b/t/t1430-bad-ref-name.sh
> @@ -205,8 +205,9 @@ test_expect_success 'update-ref --no-deref -d can delete symref to broken name'
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
>  	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
> +	test_ref_exists refs/heads/badname &&
>  	git update-ref --no-deref -d refs/heads/badname >output 2>error &&
> -	test_path_is_missing .git/refs/heads/badname &&
> +	test_ref_missing refs/heads/badname &&
>  	test_must_be_empty output &&
>  	test_must_be_empty error
>  '
> @@ -216,8 +217,9 @@ test_expect_success 'branch -d can delete symref to broken name' '
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
>  	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
> +	test_ref_exists refs/heads/badname &&
>  	git branch -d badname >output 2>error &&
> -	test_path_is_missing .git/refs/heads/badname &&
> +	test_ref_missing refs/heads/badname &&
>  	test_i18ngrep "Deleted branch badname (was refs/heads/broken\.\.\.ref)" output &&
>  	test_must_be_empty error
>  '
> @@ -225,8 +227,9 @@ test_expect_success 'branch -d can delete symref to broken name' '
>  test_expect_success 'update-ref --no-deref -d can delete dangling symref to broken name' '
>  	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
> +	test_ref_exists refs/heads/badname &&
>  	git update-ref --no-deref -d refs/heads/badname >output 2>error &&
> -	test_path_is_missing .git/refs/heads/badname &&
> +	test_ref_missing refs/heads/badname &&
>  	test_must_be_empty output &&
>  	test_must_be_empty error
>  '
> @@ -234,8 +237,9 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref to brok
>  test_expect_success 'branch -d can delete dangling symref to broken name' '
>  	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
> +	test_ref_exists refs/heads/badname &&
>  	git branch -d badname >output 2>error &&
> -	test_path_is_missing .git/refs/heads/badname &&
> +	test_ref_missing refs/heads/badname &&
>  	test_i18ngrep "Deleted branch badname (was refs/heads/broken\.\.\.ref)" output &&
>  	test_must_be_empty error
>  '
> @@ -245,8 +249,9 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
>  	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref msg &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
> +	test_ref_exists refs/heads/broken...ref &&
>  	git update-ref -d refs/heads/badname >output 2>error &&
> -	test_path_is_missing .git/refs/heads/broken...ref &&
> +	test_ref_missing refs/heads/broken...ref &&
>  	test_must_be_empty output &&
>  	test_must_be_empty error
>  '
> @@ -254,8 +259,9 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
>  test_expect_success 'update-ref --no-deref -d can delete symref with broken name' '
>  	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
> +	test_ref_exists refs/heads/broken...symref &&
>  	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
> -	test_path_is_missing .git/refs/heads/broken...symref &&
> +	test_ref_missing refs/heads/broken...symref &&
>  	test_must_be_empty output &&
>  	test_must_be_empty error
>  '
> @@ -263,8 +269,9 @@ test_expect_success 'update-ref --no-deref -d can delete symref with broken name
>  test_expect_success 'branch -d can delete symref with broken name' '
>  	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
> +	test_ref_exists refs/heads/broken...symref &&
>  	git branch -d broken...symref >output 2>error &&
> -	test_path_is_missing .git/refs/heads/broken...symref &&
> +	test_ref_missing refs/heads/broken...symref &&
>  	test_i18ngrep "Deleted branch broken...symref (was refs/heads/main)" output &&
>  	test_must_be_empty error
>  '
> @@ -272,8 +279,9 @@ test_expect_success 'branch -d can delete symref with broken name' '
>  test_expect_success 'update-ref --no-deref -d can delete dangling symref with broken name' '
>  	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
> +	test_ref_exists refs/heads/broken...symref &&
>  	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
> -	test_path_is_missing .git/refs/heads/broken...symref &&
> +	test_ref_missing refs/heads/broken...symref &&
>  	test_must_be_empty output &&
>  	test_must_be_empty error
>  '
> @@ -281,8 +289,9 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
>  test_expect_success 'branch -d can delete dangling symref with broken name' '
>  	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
> +	test_ref_exists refs/heads/broken...symref &&
>  	git branch -d broken...symref >output 2>error &&
> -	test_path_is_missing .git/refs/heads/broken...symref &&
> +	test_ref_missing refs/heads/broken...symref &&
>  	test_i18ngrep "Deleted branch broken...symref (was refs/heads/idonotexist)" output &&
>  	test_must_be_empty error
>  '
> diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
> index 080e4f24a6e..bde4f1485b7 100755
> --- a/t/t3200-branch.sh
> +++ b/t/t3200-branch.sh
> @@ -25,7 +25,7 @@ test_expect_success 'prepare a trivial repository' '
>  
>  test_expect_success 'git branch --help should not have created a bogus branch' '
>  	test_might_fail git branch --man --help </dev/null >/dev/null 2>&1 &&
> -	test_path_is_missing .git/refs/heads/--help
> +	test_ref_missing refs/heads/--help
>  '
>  
>  test_expect_success 'branch -h in broken repository' '
> @@ -40,7 +40,8 @@ test_expect_success 'branch -h in broken repository' '
>  '
>  
>  test_expect_success 'git branch abc should create a branch' '
> -	git branch abc && test_path_is_file .git/refs/heads/abc
> +	git branch abc &&
> +	test_ref_exists refs/heads/abc
>  '
>  
>  test_expect_success 'git branch abc should fail when abc exists' '
> @@ -61,11 +62,13 @@ test_expect_success 'git branch --force abc should succeed when abc exists' '
>  '
>  
>  test_expect_success 'git branch a/b/c should create a branch' '
> -	git branch a/b/c && test_path_is_file .git/refs/heads/a/b/c
> +	git branch a/b/c &&
> +	test_ref_exists refs/heads/a/b/c
>  '
>  
>  test_expect_success 'git branch mb main... should create a branch' '
> -	git branch mb main... && test_path_is_file .git/refs/heads/mb
> +	git branch mb main... &&
> +	test_ref_exists refs/heads/mb
>  '
>  
>  test_expect_success 'git branch HEAD should fail' '
> @@ -78,14 +81,14 @@ EOF
>  test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
>  	GIT_COMMITTER_DATE="2005-05-26 23:30" \
>  	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
> -	test_path_is_file .git/refs/heads/d/e/f &&
> +	test_ref_exists refs/heads/d/e/f &&
>  	test_path_is_file .git/logs/refs/heads/d/e/f &&
>  	test_cmp expect .git/logs/refs/heads/d/e/f
>  '
>  
>  test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
>  	git branch -d d/e/f &&
> -	test_path_is_missing .git/refs/heads/d/e/f &&
> +	test_ref_missing refs/heads/d/e/f &&
>  	test_must_fail git reflog exists refs/heads/d/e/f
>  '
>  
> @@ -213,7 +216,7 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
>  		test_commit initial &&
>  		git checkout --orphan lonely &&
>  		grep lonely .git/HEAD &&
> -		test_path_is_missing .git/refs/head/lonely &&
> +		test_ref_missing refs/head/lonely &&
>  		git branch -M main mistress &&
>  		grep lonely .git/HEAD
>  	)
> @@ -799,8 +802,8 @@ test_expect_success 'deleting a symref' '
>  	git symbolic-ref refs/heads/symref refs/heads/target &&
>  	echo "Deleted branch symref (was refs/heads/target)." >expect &&
>  	git branch -d symref >actual &&
> -	test_path_is_file .git/refs/heads/target &&
> -	test_path_is_missing .git/refs/heads/symref &&
> +	test_ref_exists refs/heads/target &&
> +	test_ref_missing refs/heads/symref &&
>  	test_cmp expect actual
>  '
>  
> @@ -809,16 +812,16 @@ test_expect_success 'deleting a dangling symref' '
>  	test_path_is_file .git/refs/heads/dangling-symref &&
>  	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
>  	git branch -d dangling-symref >actual &&
> -	test_path_is_missing .git/refs/heads/dangling-symref &&
> +	test_ref_missing refs/heads/dangling-symref &&
>  	test_cmp expect actual
>  '
>  
>  test_expect_success 'deleting a self-referential symref' '
>  	git symbolic-ref refs/heads/self-reference refs/heads/self-reference &&
> -	test_path_is_file .git/refs/heads/self-reference &&
> +	test_ref_exists refs/heads/self-reference &&
>  	echo "Deleted branch self-reference (was refs/heads/self-reference)." >expect &&
>  	git branch -d self-reference >actual &&
> -	test_path_is_missing .git/refs/heads/self-reference &&
> +	test_ref_missing refs/heads/self-reference &&
>  	test_cmp expect actual
>  '
>  
> @@ -826,8 +829,8 @@ test_expect_success 'renaming a symref is not allowed' '
>  	git symbolic-ref refs/heads/topic refs/heads/main &&
>  	test_must_fail git branch -m topic new-topic &&
>  	git symbolic-ref refs/heads/topic &&
> -	test_path_is_file .git/refs/heads/main &&
> -	test_path_is_missing .git/refs/heads/new-topic
> +	test_ref_exists refs/heads/main &&
> +	test_ref_missing refs/heads/new-topic
>  '
>  
>  test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
> @@ -1142,7 +1145,7 @@ EOF
>  test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
>  	GIT_COMMITTER_DATE="2005-05-26 23:30" \
>  	git checkout -b g/h/i -l main &&
> -	test_path_is_file .git/refs/heads/g/h/i &&
> +	test_ref_exists refs/heads/g/h/i &&
>  	test_path_is_file .git/logs/refs/heads/g/h/i &&
>  	test_cmp expect .git/logs/refs/heads/g/h/i
>  '
> diff --git a/t/t5521-pull-options.sh b/t/t5521-pull-options.sh
> index 079b2f2536e..3681859f983 100755
> --- a/t/t5521-pull-options.sh
> +++ b/t/t5521-pull-options.sh
> @@ -143,7 +143,7 @@ test_expect_success 'git pull --dry-run' '
>  		cd clonedry &&
>  		git pull --dry-run ../parent &&
>  		test_path_is_missing .git/FETCH_HEAD &&
> -		test_path_is_missing .git/refs/heads/main &&
> +		test_ref_missing refs/heads/main &&
>  		test_path_is_missing .git/index &&
>  		test_path_is_missing file
>  	)
> @@ -157,7 +157,7 @@ test_expect_success 'git pull --all --dry-run' '
>  		git remote add origin ../parent &&
>  		git pull --all --dry-run &&
>  		test_path_is_missing .git/FETCH_HEAD &&
> -		test_path_is_missing .git/refs/remotes/origin/main &&
> +		test_ref_missing refs/remotes/origin/main &&
>  		test_path_is_missing .git/index &&
>  		test_path_is_missing file
>  	)
> diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
> index 1d7b1abda1a..946c5751885 100755
> --- a/t/t5605-clone-local.sh
> +++ b/t/t5605-clone-local.sh
> @@ -69,7 +69,7 @@ test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
>  	git clone a d &&
>  	(cd d &&
>  	git fetch &&
> -	test ! -e .git/refs/remotes/origin/HEAD)
> +	test_ref_missing refs/remotes/origin/HEAD)
>  '
>  
>  test_expect_success 'bundle clone without .bundle suffix' '

Re: [PATCH 01/11] t: add helpers to test for reference existence

[Eric Sunshine]

On Wed, Oct 18, 2023 at 1:35 AM Patrick Steinhardt <ps@pks.im> wrote:
> Introduce a new subcommand for our ref-store test helper that explicitly
> checks only for the presence or absence of a reference. This addresses
> these limitations:
> [...]
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> @@ -221,6 +221,30 @@ static int cmd_verify_ref(struct ref_store *refs, const char **argv)
> +static int cmd_ref_exists(struct ref_store *refs, const char **argv)
> +{
> +       const char *refname = notnull(*argv++, "refname");
> +       struct strbuf unused_referent = STRBUF_INIT;
> +       struct object_id unused_oid;
> +       unsigned int unused_type;
> +       int failure_errno;
> +
> +       if (refs_read_raw_ref(refs, refname, &unused_oid, &unused_referent,
> +                             &unused_type, &failure_errno)) {
> +               /*
> +                * We handle ENOENT separately here such that it is possible to
> +                * distinguish actually-missing references from any kind of
> +                * generic error.
> +                */
> +               if (failure_errno == ENOENT)
> +                       return 17;
> +               return -1;
> +       }
> +
> +       strbuf_release(&unused_referent);
> +       return 0;
> +}

Unless refs_read_raw_ref() guarantees that `unused_referent` remains
unallocated upon failure[*], then the early returns inside the
conditional leak the strbuf. True, the program is exiting immediately
anyhow, so this (potential) leak isn't significant, but it seems odd
to clean up in one case (return 0) but not in the others (return -1 &
17).

[*] In my (admittedly brief) scan of the code and documentation, I
didn't see any such promise.

Re: [PATCH 04/11] t: convert tests to not write references via the filesystem

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> -test_expect_success "fail to create $n" '
> -	test_when_finished "rm -f .git/$n_dir" &&
> -	touch .git/$n_dir &&
> -	test_must_fail git update-ref $n $A
> +test_expect_success "fail to create $n due to file/directory conflict" '
> +	test_when_finished "git update-ref -d refs/heads/gu" &&
> +	git update-ref refs/heads/gu $A &&
> +	test_must_fail git update-ref refs/heads/gu/fixes $A
>  '

OK, the original checks "if a random garbage file, which may not
necessarily be a ref, exists at $n_dir, we cannot create a ref at
$n_dir/fixes, due to D/F conflict" more directly, but as long as our
intention is to enforce the D/F restriction across different ref
backends [*], creating a ref at $n_dir and making sure $n_dir/fixes
cannot be created is an equivalent check that is better (because it
can be applied for other backends).

    Side note: there is no fundamental need to, though, and there
         are cases where being able to have the 'seen' branch and
         'seen/ps/ref-test-tools' branches at the same time is
         beneficial---packed-refs and ref-table backends would not
         have such an inherent limitation, but they can of course be
         castrated to match what files-backend can(not) do.

> @@ -222,7 +220,7 @@ test_expect_success 'delete symref without dereference when the referred ref is
>  
>  test_expect_success 'update-ref -d is not confused by self-reference' '
>  	git symbolic-ref refs/heads/self refs/heads/self &&
> -	test_when_finished "rm -f .git/refs/heads/self" &&
> +	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
>  	test_path_is_file .git/refs/heads/self &&

I trust that this will be corrected to use some wrapper around "git
symbolic-ref" (or an equivalent for it as a test-tool subcommand) in
some future patch, if not in this series?

>  	test_must_fail git update-ref -d refs/heads/self &&
>  	test_path_is_file .git/refs/heads/self

Likewise.

> @@ -230,7 +228,7 @@ test_expect_success 'update-ref -d is not confused by self-reference' '
>  
>  test_expect_success 'update-ref --no-deref -d can delete self-reference' '
>  	git symbolic-ref refs/heads/self refs/heads/self &&
> -	test_when_finished "rm -f .git/refs/heads/self" &&
> +	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
>  	test_path_is_file .git/refs/heads/self &&
>  	git update-ref --no-deref -d refs/heads/self &&
>  	test_must_fail git show-ref --verify -q refs/heads/self

We already have the "ref is missing" test here.

I'll stop at this point for now; will hopefully continue in a
separate message later.  Thanks.

Re: [PATCH 04/11] t: convert tests to not write references via the filesystem

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> @@ -434,7 +432,7 @@ test_expect_success 'Query "main@{2005-05-28}" (past end of history)' '
>  	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
>  '
>  
> -rm -f .git/$m .git/logs/$m expect
> +git update-ref -d $m

We are not clearing "expect" file.  I do not know if it matters
here, but I am only recording what I noticed.

> diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
> index 10a539158c4..5cce24f1006 100755
> --- a/t/t1450-fsck.sh
> +++ b/t/t1450-fsck.sh
> @@ -115,15 +115,16 @@ test_expect_success 'zlib corrupt loose object output ' '
>  '
>  
>  test_expect_success 'branch pointing to non-commit' '
> -	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
> +	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
> +	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&

I have mixed feelings on this.

In olden days, plumbing commands tended to allow to pass anything
the user told them to use, but in more recent versions of Git, we,
probably by mistake, managed to butcher some of the plumbing
commands to make them unable to deliberately "break" repositories,
one victim being "update-ref", i.e.

    $ git update-ref refs/heads/invalid HEAD^{tree}

is rejected these days (I just checked with v1.3.0 and it allows me
to do this), and that is one of the reasons why we manually broke
the repository in these tests.  We need to have a warning message in
comments near the implementation of "ref-store update-ref" that says
never ever attempt to share code with the production version of
update-ref---otherwise this (or the "safety" given to the plumbing
command, possibly by mistake) will be broken, depending on which
direction such a sharing goes.  On the other hand, forcing us to
keep two separate implementations, one deliberately loose to allow
us corrupt repositories, the other for production and actively used,
would mean the former one that is only used for validation would risk
bitrotting.

>  	test_when_finished "git update-ref -d refs/heads/invalid" &&

Not a problem this patch introduces, but I think it is a better
discipline to have when_finished clean-up routine prepared before we
do actual damage, i.e. if I were writing this test today from scratch,
I would expect it to be before "git rev-parse >.git/refs/heads/invalid"
is done.

>  	test_must_fail git fsck 2>out &&
>  	test_i18ngrep "not a commit" out
>  '

A #leftoverbit that is not relevant to the topic; we should clean
these test_i18ngrep and replace them with a plain "grep".

>  test_expect_success 'HEAD link pointing at a funny object' '
> -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
> -	mv .git/HEAD .git/SAVED_HEAD &&
> +	saved_head=$(git rev-parse --verify HEAD) &&
> +	test_when_finished "git update-ref HEAD ${saved_head}" &&
>  	echo $ZERO_OID >.git/HEAD &&

Are you sure .git/HEAD when this test is entered is a detached HEAD?
The title of the test says "HEAD link", and I take it to mean HEAD
is a symlink, and we save it away, while we create a loose ref that
points at 0{40} in a detached HEAD state.  Actually, the original
would also work if HEAD is detached on entry.  In either case,
moving SAVED_HEAD back to HEAD would restore the original state.

But the updated one only works if HEAD upon entry is already
detached.  Is this intended?

> @@ -131,8 +132,8 @@ test_expect_success 'HEAD link pointing at a funny object' '
>  '
>  
>  test_expect_success 'HEAD link pointing at a funny place' '
> -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
> -	mv .git/HEAD .git/SAVED_HEAD &&
> +	saved_head=$(git rev-parse --verify HEAD) &&
> +	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&

Likewise.  Use of "update-ref" in the previous one vs "update-ref
--no-deref" in this one to recover from the damage the tests make
makes me feel that we may be assuming too much.

>  	echo "ref: refs/funny/place" >.git/HEAD &&

Even though "git symbolic-ref" refuses to point HEAD outside refs/,
as plumbing command should, it allows it to point it outside refs/heads/.
so this line should probably become

	git symbolic-ref HEAD refs/funny/place

in the same spirit as the rest of the series.

> @@ -391,7 +393,7 @@ test_expect_success 'tag pointing to nonexistent' '
>  
>  	tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
>  	test_when_finished "remove_object $tag" &&
> -	echo $tag >.git/refs/tags/invalid &&
> +	git update-ref refs/tags/invalid $tag &&

Good (not just this one, but similar ones throughout this patch).

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> As usual, I forgot to mention that this applies on top of current
> master, which is at a9ecda2788 (The eighteenth batch, 2023-10-13) at the
> time of writing. I look forward to the day where I remember to include
> this information in the cover letter...

I heard rumors that we have a --base option that records such a
piece of information in the format-patch output (I haven't used it
myself).  Would it have helped, or perhaps it still needs some
usability tweak?

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Han-Wen Nienhuys]

On Wed, Oct 18, 2023 at 5:32 PM Junio C Hamano <gitster@pobox.com> wrote:
> > this patch series refactors a bunch of our tests to perform less direct
> > disk access to on-disk data structures. Instead, the tests are converted
> > to use Git tools or our test-tool to access data to the best extent
> > possible. This serves two benefits:
>
> Laudable goal.
>
> >     - We increase test coverage of our own code base.
>
> Meaning the new code added to test-tool for this series will also
> get tested and bugs spotted?
>
> >     - We become less dependent on the actual on-disk format.
>
> Yes, this is very desirable.  Without looking at the implementation,
> I see some issues aiming for this goal may involve. [a] using the
> production code for validation would mean our expectation to be
> compared to the reality to be validated can be affected by the same
> bug, making two wrongs to appear right; [b] using a separate
> implementation used only for validation would at least mean we will
> have to make the same mistake in unique part of both implementations
> that is less likely to miss bugs compared to [a], but bugs in shared
> part of the production code and validation code will be hidden the
> same way as [a].

I think it would be really great if there were separate unittests for
the ref backend API. Some of the reftable work was needlessly
difficult because the contract of the API was underspecified. The API
is well compartmentalized in refs-internal.h, and a lot of the API
behavior can be tested as a black box, eg.

* setup symref HEAD pointing to R1
* setup transaction updating ref R1 from C1 to C2
* commit transaction, check that it succeeds
* read ref R1, check if it is C2
* read reflog for R1, see that it has a C1 => C2 update
* read reflog for HEAD, see that it has a C1 => C2 update

Tests for the loose/packed backend could directly mess with the
on-disk files to test failure scenarios.

With unittests like that, the tests can zoom in on the functionality
of the ref backend, and provide more convenient coverage for
dynamic/static analysis.

-- 
Han-Wen Nienhuys - Google Munich
I work 80%. Don't expect answers from me on Fridays.
--
Google Germany GmbH, Erika-Mann-Strasse 33, 80636 Munich
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Paul Manicle, Liana Sebastian

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Junio C Hamano]

Han-Wen Nienhuys <hanwen@google.com> writes:

> I think it would be really great if there were separate unittests for
> the ref backend API. Some of the reftable work was needlessly
> difficult because the contract of the API was underspecified. The API
> is well compartmentalized in refs-internal.h, and a lot of the API
> behavior can be tested as a black box, eg.
>
> * setup symref HEAD pointing to R1
> * setup transaction updating ref R1 from C1 to C2
> * commit transaction, check that it succeeds
> * read ref R1, check if it is C2
> * read reflog for R1, see that it has a C1 => C2 update
> * read reflog for HEAD, see that it has a C1 => C2 update
>
> Tests for the loose/packed backend could directly mess with the
> on-disk files to test failure scenarios.
>
> With unittests like that, the tests can zoom in on the functionality
> of the ref backend, and provide more convenient coverage for
> dynamic/static analysis.

Yeah, I agree that something like that would really be great.

Re: [PATCH 05/11] t: convert tests to not access symrefs via the filesystem

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> @@ -164,9 +164,9 @@ test_expect_success 'rev-parse skips symref pointing to broken name' '
>  test_expect_success 'for-each-ref emits warnings for broken names' '
>  	test-tool ref-store main update-ref msg "refs/heads/broken...ref" $main_sha1 $ZERO_OID REF_SKIP_REFNAME_VERIFICATION &&
>  	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
> -	printf "ref: refs/heads/broken...ref\n" >.git/refs/heads/badname &&
> +	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref &&

I am of two minds here.  While it certainly smells nicer because we
can test ref backends other than the files backend with this change,
we are forcing all ref backends to support creating a symbolic ref
with invalid name, because otherwise "test-tool" would not be able
to do this.  Newer more database-oriented ref backends should be
allowed to implement their file format in which it is imposssible to
store such a bad name, but this makes it impossible.

I guess it is OK, because we would introduce some new prerequisite
(i.e. REF_BACKEND_ALLOWS_BROKEN_REFS) to skip this test on certain
ref backend where making invalid refs is impossible.

Other kind of changes in this patch, e.g., ...

> @@ -315,7 +325,9 @@ test_expect_success 'defaulted HEAD uses remote branch if available' '
>  	git -c init.defaultBranch=branchwithstuff -c protocol.version=2 \
>  		clone "file://$(pwd)/file_unborn_parent" \
>  		file_unborn_child 2>stderr &&
> -	grep "refs/heads/branchwithstuff" file_unborn_child/.git/HEAD &&
> +	echo "refs/heads/branchwithstuff" >expect &&
> +	git -C file_unborn_child symbolic-ref HEAD >actual &&
> +	test_cmp expect actual &&
>  	test_path_is_file file_unborn_child/stuff.t &&
>  	! grep "warning:" stderr
>  '
> diff --git a/t/t9133-git-svn-nested-git-repo.sh b/t/t9133-git-svn-nested-git-repo.sh
> index d8d536269cf..8ca24670acb 100755
> --- a/t/t9133-git-svn-nested-git-repo.sh
> +++ b/t/t9133-git-svn-nested-git-repo.sh
> @@ -11,7 +11,7 @@ test_expect_success 'setup repo with a git repo inside it' '
>  	(
>  		cd s &&
>  		git init &&
> -		test -f .git/HEAD &&
> +		git symbolic-ref HEAD &&
>  		> .git/a &&
>  		echo a > a &&
>  		svn_cmd add .git a &&

... all looked sensible, though.

Thanks.

Re: [PATCH 06/11] t: convert tests to not access reflog via the filesystem

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index 5f505e2f353..b1d2c014132 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -90,7 +90,8 @@ test_expect_success "deleting current branch adds message to HEAD's log" '
>  	git symbolic-ref HEAD $m &&
>  	git update-ref -m delete-$m -d $m &&
>  	test_must_fail git show-ref --verify -q $m &&
> -	grep "delete-$m$" .git/logs/HEAD
> +	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
> +	grep "delete-$m$" actual
>  '
>  
>  test_expect_success "deleting by HEAD adds message to HEAD's log" '
> @@ -99,7 +100,8 @@ test_expect_success "deleting by HEAD adds message to HEAD's log" '
>  	git symbolic-ref HEAD $m &&
>  	git update-ref -m delete-by-head -d HEAD &&
>  	test_must_fail git show-ref --verify -q $m &&
> -	grep "delete-by-head$" .git/logs/HEAD
> +	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
> +	grep "delete-by-head$" actual
>  '

These are quite straight-forward.

>  test_expect_success "verifying $m's log (logged by config)" '
> -	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
> +	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
>  	test-tool ref-store main for-each-reflog-ent $m >actual &&
>  	test_cmp actual expect
>  '

The approach forces us to assume that "git reflog expire" performs
correctly in order to test reflog, but it probably is OK---we'll
notice breakages in "reflog expire" in other tests, hopefully.

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1721 bytes --]

On Wed, Oct 18, 2023 at 04:40:35PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > As usual, I forgot to mention that this applies on top of current
> > master, which is at a9ecda2788 (The eighteenth batch, 2023-10-13) at the
> > time of writing. I look forward to the day where I remember to include
> > this information in the cover letter...
> 
> I heard rumors that we have a --base option that records such a
> piece of information in the format-patch output (I haven't used it
> myself).  Would it have helped, or perhaps it still needs some
> usability tweak?

I was aware of the option, but somehow never thought to use it. I guess
others are in the same boat as this is a semi-frequent topic to come up
on the mailing list.

What will fix this for me from now on is that I've now discovered the
`format.useAutoBase` config. To fix this for everybody else without
having to go through the same process we could change the default to
`format.useAutoBase=auto`. This can lead to hard-to-understand errors as
pointed out by 7efba5fa39f (format-patch: teach format.useAutoBase
"whenAble" option, 2020-10-01) though:

```
$ git format-patch -1 <an old commit>
fatal: base commit shouldn't be in revision list
```

But this was fixed via the new "whenAble" value for this configuration,
which will cause Git to ignore the case where it's unable to figure out
the base commit. So maybe we should consider to make it the new default?
I cannot think of any real downside. In the best case we simplify the
life of many maintainers of mailing list based projects. In the worst
case where we are unable to figure out the base commit nothing changes.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 00/11] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3406 bytes --]

On Thu, Oct 19, 2023 at 12:13:12PM +0200, Han-Wen Nienhuys wrote:
> On Wed, Oct 18, 2023 at 5:32 PM Junio C Hamano <gitster@pobox.com> wrote:
> > > this patch series refactors a bunch of our tests to perform less direct
> > > disk access to on-disk data structures. Instead, the tests are converted
> > > to use Git tools or our test-tool to access data to the best extent
> > > possible. This serves two benefits:
> >
> > Laudable goal.
> >
> > >     - We increase test coverage of our own code base.
> >
> > Meaning the new code added to test-tool for this series will also
> > get tested and bugs spotted?

For now all the helpers of the test-tool only have implicit test
coverage, but I get your point. If we decide to instead transform this
test tool into production-level code as you suggested (e.g. `git
rev-parse --exists`) then this becomes less of a discussion point as we
would of course have proper test coverage for it.

> > >     - We become less dependent on the actual on-disk format.
> >
> > Yes, this is very desirable.  Without looking at the implementation,
> > I see some issues aiming for this goal may involve. [a] using the
> > production code for validation would mean our expectation to be
> > compared to the reality to be validated can be affected by the same
> > bug, making two wrongs to appear right; [b] using a separate
> > implementation used only for validation would at least mean we will
> > have to make the same mistake in unique part of both implementations
> > that is less likely to miss bugs compared to [a], but bugs in shared
> > part of the production code and validation code will be hidden the
> > same way as [a].
> 
> I think it would be really great if there were separate unittests for
> the ref backend API. Some of the reftable work was needlessly
> difficult because the contract of the API was underspecified. The API
> is well compartmentalized in refs-internal.h, and a lot of the API
> behavior can be tested as a black box, eg.
> 
> * setup symref HEAD pointing to R1
> * setup transaction updating ref R1 from C1 to C2
> * commit transaction, check that it succeeds
> * read ref R1, check if it is C2
> * read reflog for R1, see that it has a C1 => C2 update
> * read reflog for HEAD, see that it has a C1 => C2 update
> 
> Tests for the loose/packed backend could directly mess with the
> on-disk files to test failure scenarios.
> 
> With unittests like that, the tests can zoom in on the functionality
> of the ref backend, and provide more convenient coverage for
> dynamic/static analysis.

Agreed. Ideally, I think our tests should be split up into two layers:

    1. Low-level tests which are backend specific. These allow us to
       assert the on-disk state of the respective backend thoroughly and
       also allow us to explicitly test for edge cases that are only of
       relevance to this specific backend.

       The reftable backend in its current (non-upstream) already has
       such tests, but we don't really have explicit tests for the files
       backend. This is a gap that should ideally be filled at some
       point in time.

    2. Higher-level tests should then be allowed to assume that the
       underlying logic works as expected. These tests are thus free to
       use plumbing tools that tie into the reference backends.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 02/11] t: allow skipping expected object ID in `ref-store update-ref`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2507 bytes --]

On Wed, Oct 18, 2023 at 09:08:08AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > We require the caller to pass both the old and new expected object ID to
> > our `test-tool ref-store update-ref` helper. When trying to update a
> > symbolic reference though it's impossible to specify the expected object
> > ID, which means that the test would instead have to force-update the
> > reference. This is currently impossible though.
> >
> > Update the helper to optionally skip verification of the old object ID
> > in case the test passes in an empty old object ID as input.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  t/helper/test-ref-store.c | 11 +++++++----
> >  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> Good.
> 
> Even better would be to make the old one optional, though.

I was also a bit torn when writing this. We could of course make the
behaviour conditional on whether `argc` is 4 or 5. But I wasn't quite
sure how important it is to provide a nice UI for this test helper, and
we don't have `argc` readily available. It's not hard to count them
manually, but until now I was under the impression that the test helpers
only need to be "good enough".

Patrick

> > diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> > index 7400f560ab6..7dc83137584 100644
> > --- a/t/helper/test-ref-store.c
> > +++ b/t/helper/test-ref-store.c
> > @@ -322,16 +322,19 @@ static int cmd_update_ref(struct ref_store *refs, const char **argv)
> >  	const char *new_sha1_buf = notnull(*argv++, "new-sha1");
> >  	const char *old_sha1_buf = notnull(*argv++, "old-sha1");
> >  	unsigned int flags = arg_flags(*argv++, "flags", transaction_flags);
> > -	struct object_id old_oid;
> > +	struct object_id old_oid, *old_oid_ptr = NULL;
> >  	struct object_id new_oid;
> >  
> > -	if (get_oid_hex(old_sha1_buf, &old_oid))
> > -		die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
> > +	if (*old_sha1_buf) {
> > +		if (get_oid_hex(old_sha1_buf, &old_oid))
> > +			die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
> > +		old_oid_ptr = &old_oid;
> > +	}
> >  	if (get_oid_hex(new_sha1_buf, &new_oid))
> >  		die("cannot parse %s as %s", new_sha1_buf, the_hash_algo->name);
> >  
> >  	return refs_update_ref(refs, msg, refname,
> > -			       &new_oid, &old_oid,
> > +			       &new_oid, old_oid_ptr,
> >  			       flags, UPDATE_REFS_DIE_ON_ERR);
> >  }
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 04/11] t: convert tests to not write references via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 4699 bytes --]

On Wed, Oct 18, 2023 at 11:34:23AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > -test_expect_success "fail to create $n" '
> > -	test_when_finished "rm -f .git/$n_dir" &&
> > -	touch .git/$n_dir &&
> > -	test_must_fail git update-ref $n $A
> > +test_expect_success "fail to create $n due to file/directory conflict" '
> > +	test_when_finished "git update-ref -d refs/heads/gu" &&
> > +	git update-ref refs/heads/gu $A &&
> > +	test_must_fail git update-ref refs/heads/gu/fixes $A
> >  '
> 
> OK, the original checks "if a random garbage file, which may not
> necessarily be a ref, exists at $n_dir, we cannot create a ref at
> $n_dir/fixes, due to D/F conflict" more directly, but as long as our
> intention is to enforce the D/F restriction across different ref
> backends [*], creating a ref at $n_dir and making sure $n_dir/fixes
> cannot be created is an equivalent check that is better (because it
> can be applied for other backends).
> 
>     Side note: there is no fundamental need to, though, and there
>          are cases where being able to have the 'seen' branch and
>          'seen/ps/ref-test-tools' branches at the same time is
>          beneficial---packed-refs and ref-table backends would not
>          have such an inherent limitation, but they can of course be
>          castrated to match what files-backend can(not) do.

I think initially it is beneficial to keep any such restriction and cut
back new backends to match them, even though it's more work. The main
reason why I think this makes sense is that hosting providers are the
most likely to update to the newer backend fast. It would be detrimental
to the users though if hosting providers that converted to the newer
backend were able to store such references that would be conflicting
with the files backend because they wouldn't be able to clone such
repositories anymore. And this isn't only true for hosting providers,
but essentially whenever two repositories that use different backends
try to interact with each other.

So even though I wish for a future where we don't need to care for D/F
conflicts anymore, I think the time isn't ripe for it and we should for
now aim for matching behaviour.

> > @@ -222,7 +220,7 @@ test_expect_success 'delete symref without dereference when the referred ref is
> >  
> >  test_expect_success 'update-ref -d is not confused by self-reference' '
> >  	git symbolic-ref refs/heads/self refs/heads/self &&
> > -	test_when_finished "rm -f .git/refs/heads/self" &&
> > +	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
> >  	test_path_is_file .git/refs/heads/self &&
> 
> I trust that this will be corrected to use some wrapper around "git
> symbolic-ref" (or an equivalent for it as a test-tool subcommand) in
> some future patch, if not in this series?

Yup, this is getting fixed in a subsequent patch. I had two different
options to structure this series:

    - Either by test file so that questions like this don't come up when
      a reviewer sees in the context that there are still tests that
      exercise the filesystem directly. This was my first approach.

    - The alternative is to write it such that we address common
      patterns globally, which I then rewrote my first version to.

There were two reasons why I didn't like the first iteration:

    - Commit messages would basically have to reexplain the exact same
      thing for every single testcase as the motivation is the same
      everywhere.

    - I think it's easier to review when you only see the same kind of
      transformation per patch.

But yes, it does have the downside that the reader is now left wondering
why the other call to `test_path_is_file` still exists.

Patrick

> >  	test_must_fail git update-ref -d refs/heads/self &&
> >  	test_path_is_file .git/refs/heads/self
> 
> Likewise.
> 
> > @@ -230,7 +228,7 @@ test_expect_success 'update-ref -d is not confused by self-reference' '
> >  
> >  test_expect_success 'update-ref --no-deref -d can delete self-reference' '
> >  	git symbolic-ref refs/heads/self refs/heads/self &&
> > -	test_when_finished "rm -f .git/refs/heads/self" &&
> > +	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
> >  	test_path_is_file .git/refs/heads/self &&
> >  	git update-ref --no-deref -d refs/heads/self &&
> >  	test_must_fail git show-ref --verify -q refs/heads/self
> 
> We already have the "ref is missing" test here.
> 
> I'll stop at this point for now; will hopefully continue in a
> separate message later.  Thanks.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 04/11] t: convert tests to not write references via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 5886 bytes --]

On Wed, Oct 18, 2023 at 02:18:27PM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > @@ -434,7 +432,7 @@ test_expect_success 'Query "main@{2005-05-28}" (past end of history)' '
> >  	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
> >  '
> >  
> > -rm -f .git/$m .git/logs/$m expect
> > +git update-ref -d $m
> 
> We are not clearing "expect" file.  I do not know if it matters
> here, but I am only recording what I noticed.

Oops, will fix.

> > diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
> > index 10a539158c4..5cce24f1006 100755
> > --- a/t/t1450-fsck.sh
> > +++ b/t/t1450-fsck.sh
> > @@ -115,15 +115,16 @@ test_expect_success 'zlib corrupt loose object output ' '
> >  '
> >  
> >  test_expect_success 'branch pointing to non-commit' '
> > -	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
> > +	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
> > +	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
> 
> I have mixed feelings on this.
> 
> In olden days, plumbing commands tended to allow to pass anything
> the user told them to use, but in more recent versions of Git, we,
> probably by mistake, managed to butcher some of the plumbing
> commands to make them unable to deliberately "break" repositories,
> one victim being "update-ref", i.e.
> 
>     $ git update-ref refs/heads/invalid HEAD^{tree}
> 
> is rejected these days (I just checked with v1.3.0 and it allows me
> to do this), and that is one of the reasons why we manually broke
> the repository in these tests.

My first try was indeed to use git-update-ref(1) to update the ref to
the tree object, and I was surprised to learn that it would not let me
do so.

> We need to have a warning message in
> comments near the implementation of "ref-store update-ref" that says
> never ever attempt to share code with the production version of
> update-ref---otherwise this (or the "safety" given to the plumbing
> command, possibly by mistake) will be broken, depending on which
> direction such a sharing goes.  On the other hand, forcing us to
> keep two separate implementations, one deliberately loose to allow
> us corrupt repositories, the other for production and actively used,
> would mean the former one that is only used for validation would risk
> bitrotting.

Wouldn't any eventual bitrot be contained by the tests though? As our
use of the test helper grows via this patch series its behaviour will
also be verified more thoroughly.

> >  	test_when_finished "git update-ref -d refs/heads/invalid" &&
> 
> Not a problem this patch introduces, but I think it is a better
> discipline to have when_finished clean-up routine prepared before we
> do actual damage, i.e. if I were writing this test today from scratch,
> I would expect it to be before "git rev-parse >.git/refs/heads/invalid"
> is done.

I'll fix this while at it.

> >  	test_must_fail git fsck 2>out &&
> >  	test_i18ngrep "not a commit" out
> >  '
> 
> A #leftoverbit that is not relevant to the topic; we should clean
> these test_i18ngrep and replace them with a plain "grep".
> 
> >  test_expect_success 'HEAD link pointing at a funny object' '
> > -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
> > -	mv .git/HEAD .git/SAVED_HEAD &&
> > +	saved_head=$(git rev-parse --verify HEAD) &&
> > +	test_when_finished "git update-ref HEAD ${saved_head}" &&
> >  	echo $ZERO_OID >.git/HEAD &&
> 
> Are you sure .git/HEAD when this test is entered is a detached HEAD?
> The title of the test says "HEAD link", and I take it to mean HEAD
> is a symlink, and we save it away, while we create a loose ref that
> points at 0{40} in a detached HEAD state.  Actually, the original
> would also work if HEAD is detached on entry.  In either case,
> moving SAVED_HEAD back to HEAD would restore the original state.
> 
> But the updated one only works if HEAD upon entry is already
> detached.  Is this intended?

Yes and no -- it's a reflection of the state when this test runs. One
problem in this test suite here is that many of the tests' states are
heavily interwoven with each other, which only makes this harder to
refactor without making any assumptions.

Well. Instead of restoring to whatever the state was previous to the
test we could also restore it to something sane-ish like "master". That
of course breaks other tests though... I'll investigate.

> > @@ -131,8 +132,8 @@ test_expect_success 'HEAD link pointing at a funny object' '
> >  '
> >  
> >  test_expect_success 'HEAD link pointing at a funny place' '
> > -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
> > -	mv .git/HEAD .git/SAVED_HEAD &&
> > +	saved_head=$(git rev-parse --verify HEAD) &&
> > +	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&
> 
> Likewise.  Use of "update-ref" in the previous one vs "update-ref
> --no-deref" in this one to recover from the damage the tests make
> makes me feel that we may be assuming too much.
> 
> >  	echo "ref: refs/funny/place" >.git/HEAD &&
> 
> Even though "git symbolic-ref" refuses to point HEAD outside refs/,
> as plumbing command should, it allows it to point it outside refs/heads/.
> so this line should probably become
> 
> 	git symbolic-ref HEAD refs/funny/place
> 
> in the same spirit as the rest of the series.

Yup, this will be adressed in a later patch.

Patrick

> > @@ -391,7 +393,7 @@ test_expect_success 'tag pointing to nonexistent' '
> >  
> >  	tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
> >  	test_when_finished "remove_object $tag" &&
> > -	echo $tag >.git/refs/tags/invalid &&
> > +	git update-ref refs/tags/invalid $tag &&
> 
> Good (not just this one, but similar ones throughout this patch).
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 08/11] t4207: delete replace references via git-update-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1598 bytes --]

On Wed, Oct 18, 2023 at 03:27:11PM +0200, Han-Wen Nienhuys wrote:
> On Wed, Oct 18, 2023 at 7:35 AM Patrick Steinhardt <ps@pks.im> wrote:
> >
> > In t4207 we set up a set of replace objects via git-replace(1). Because
> > these references should not be impacting subsequent tests we also set up
> > some cleanup logic that deletes the replacement references via a call to
> > `rm -rf`. This reaches into the internal implementation details of the
> > reference backend and will thus break when we grow an alternative refdb
> > implementation.
> >
> > Refactor the tests to delete the replacement refs via Git commands so
> > that we become independent of the actual refdb that's in use. As we
> > don't have a nice way to delete all replacements or all references in a
> > certain namespace, we opt for a combination of git-for-each-ref(1) and
> > git-update-ref(1)'s `--stdin` mode.
> 
> There is a test helper that can directly access the ref database,
> t/helper/test-ref-store.c.
> 
> If you use that manipulate refs for testing purposes, you make the
> test independent of behavior git-for-each-ref/git-update-ref, which is
> what you want for testing replace-objects?

Is there any specific reason why we shouldn't be using
git-for-each-ref(1) or git-update-ref(1) here? Neither of those commands
are part of the system under test, as we rather care about git-log(1)
here. So as those commands are already being verified in other tests I
think it should be fine to assume that they work as intended here.

Happy to hear differing viewpoints though.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 01/11] t: add helpers to test for reference existence

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2064 bytes --]

On Wed, Oct 18, 2023 at 01:08:45PM -0400, Eric Sunshine wrote:
> On Wed, Oct 18, 2023 at 1:35 AM Patrick Steinhardt <ps@pks.im> wrote:
> > Introduce a new subcommand for our ref-store test helper that explicitly
> > checks only for the presence or absence of a reference. This addresses
> > these limitations:
> > [...]
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> > diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> > @@ -221,6 +221,30 @@ static int cmd_verify_ref(struct ref_store *refs, const char **argv)
> > +static int cmd_ref_exists(struct ref_store *refs, const char **argv)
> > +{
> > +       const char *refname = notnull(*argv++, "refname");
> > +       struct strbuf unused_referent = STRBUF_INIT;
> > +       struct object_id unused_oid;
> > +       unsigned int unused_type;
> > +       int failure_errno;
> > +
> > +       if (refs_read_raw_ref(refs, refname, &unused_oid, &unused_referent,
> > +                             &unused_type, &failure_errno)) {
> > +               /*
> > +                * We handle ENOENT separately here such that it is possible to
> > +                * distinguish actually-missing references from any kind of
> > +                * generic error.
> > +                */
> > +               if (failure_errno == ENOENT)
> > +                       return 17;
> > +               return -1;
> > +       }
> > +
> > +       strbuf_release(&unused_referent);
> > +       return 0;
> > +}
> 
> Unless refs_read_raw_ref() guarantees that `unused_referent` remains
> unallocated upon failure[*], then the early returns inside the
> conditional leak the strbuf. True, the program is exiting immediately
> anyhow, so this (potential) leak isn't significant, but it seems odd
> to clean up in one case (return 0) but not in the others (return -1 &
> 17).
> 
> [*] In my (admittedly brief) scan of the code and documentation, I
> didn't see any such promise.

Agreed, let's just be thorough and plug any potential memor leak here.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 01/11] t: add helpers to test for reference existence

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 9775 bytes --]

On Wed, Oct 18, 2023 at 09:06:50AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > There are two major ways to check for the existence of a reference in
> > our tests:
> >
> >     - `git rev-parse --verify` can be used to check for existence of a
> >       reference. This only works in the case where the reference is well
> >       formed though and resolves to an actual object ID. This does not
> >       work with malformed reference names or invalid contents.
> >
> >     - `test_path_is_file` can be used to check for existence of a loose
> >       reference if it is known to not resolve to an actual object ID. It
> >       by necessity reaches into implementation details of the reference
> >       backend though.
> 
> True.  It would be ideal if we can limit the use of latter when we
> _care_ how the ref is stored (e.g., "we expect it to be stored as a
> loose ref, not packed").  "The ref R at must be pointing at the
> commit X" is better asserted by using the former (or "git show-ref")
> as we not just only want to see the .git/refs/R file holding the
> object name X, but also want to see the production git tools observe
> the same---if what rev-parse or show-ref observes is different from
> the expected state and they say ref R does not point at commit X, we
> should complain (rev-parse or show-ref may be broken in the version
> of Git being tested, but we can assume that their breakage will be
> caught elsewhere in the test suite as well, so as long as we trust
> them, using them as the validator is better than going into the
> implementation detail and assuming things like "new refs always
> appear as a loose ref" that we might want to change in the future).
> 
> > Similarly, there are two equivalent ways to check for the absence of a
> > reference:
> >
> >     - `test_must_fail git rev-parse` can be used to check for the
> >       absence of a reference. It could fail due to a number of reasons
> >       though, and all of these reasons will be thrown into the same bag
> >       as an absent reference.
> >
> >     - `test_path_is_missing` can be used to check explicitly for the
> >       absence of a loose reference, but again reaches into internal
> >       implementation details of the reference backend.
> >
> > So both our tooling to check for the presence and for the absence of
> > references in tests is lacking as either failure cases are thrown into
> > the same bag or we need to reach into internal implementation details of
> > the respective reference backend.
> 
> > Introduce a new subcommand for our ref-store test helper that explicitly
> > checks only for the presence or absence of a reference. This addresses
> > these limitations:
> >
> >     - We can check for the presence of references with malformed names.
> 
> But for the purpose of tests, we can control the input.  When we
> perform an operation that we expect a ref R to be created, we would
> know R is well formed and we can validate using a tool that we know
> would be broken when fed a malformed name.  So I do not see this as
> a huge "limitation".

This is explicitly about the case where such a ref R is not well-formed
though. This limitation was mostly a problem in t1430-bad-ref-name.sh,
which verifies many such scenarios.

> >     - We can check for the presence of references that don't resolve.
> 
> Do you mean a dangling symbolic ref?  We are using a wrong tool if
> you are using rev-parse for that, aren't we?  Isn't symbolic-ref
> there for us for this exact use case?  That is

Again, t1430-bad-ref-name.sh has been the inspiration for this:

```
$ git symbolic-ref refs/heads/bad...name refs/heads/master
$ git symbolic-ref refs/heads/bad...name
fatal: No such ref: refs/heads/bad...name
```

The mismatch that you can write but not read the reference is kind of
astonishing though. We could fix this limitation, but I think there were
more usecases than only bad reference names. I honestly can't quite
remember right now.

> >     - We can explicitly handle the case where a reference is missing by
> >       special-casing ENOENT errors.
> 
> You probably know the error conditions refs_read_raw_ref() can be
> broken better than I do, but this feels a bit too intimate with how
> the method for the files backend happens to be implemented, which at
> the same time, can risk that [a] other backends can implement their
> "ref does not resolve to an object name---is it because it is
> missing?" report incorrectly and [b] we would eventually want to
> know error conditions other than "the ref requested is missing" and
> at that point we would need more "special casing", which does not
> smell easy to scale.

We actually rely on some of these error codes to be consistent across
backends. E.g. "refs.c" itself has higher-level logic that verifies
specific error codes when resolving symrefs. And as we explicitly made
these error codes part of the API design with `refs_read_raw_ref()` my
assumption is that any other backend needs to match the behaviour here.

I also think that this is a somewhat sane assumption to make. While it
may not be a good idea to tie this to standard error codes, the backend
should indeed be able to signal specific error cases to the caller. We
could refactor this to be more explicit about the expected failure cases
in the form of specialized error codes. But I'm not sure whether that
would be worth it for now, but it's sure something to keep in mind for
future patch series.

> >     - We don't need to reach into implementation details of the backend,
> >       which would allow us to use this helper for the future reftable
> >       backend.
> 
> This is exactly what we want to aim for.
> 
> > Next to this subcommand we also provide two wrappers `test_ref_exists`
> > and `test_ref_missing` that make the helper easier to use.
> 
> Hmmmm.  This may introduce "who watches the watchers" problem, no?
> I briefly wondered if a better approach is to teach the production
> code, e.g., rev-parse, to optionally give more detailed diag.  It
> essentially may be the same (making the code in test-ref-store.c
> added by this patch available from rev-parse, we would easily get
> there), so I do not think the distinction matters.
> 
> > diff --git a/t/README b/t/README
> > index 61080859899..779f7e7dd86 100644
> > --- a/t/README
> > +++ b/t/README
> > @@ -928,6 +928,15 @@ see test-lib-functions.sh for the full list and their options.
> >     committer times to defined state.  Subsequent calls will
> >     advance the times by a fixed amount.
> >  
> > + - test_ref_exists <ref>, test_ref_missing <ref>
> > +
> > +   Check whether a reference exists or is missing. In contrast to
> > +   git-rev-parse(1), these helpers also work with invalid reference
> > +   names and references whose contents are unresolvable. The latter
> > +   function also distinguishes generic errors from the case where a
> > +   reference explicitly doesn't exist and is thus safer to use than
> > +   `test_must_fail git rev-parse`.
> > +
> >   - test_commit <message> [<filename> [<contents>]]
> >  
> >     Creates a commit with the given message, committing the given
> > diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
> > index 48552e6a9e0..7400f560ab6 100644
> > --- a/t/helper/test-ref-store.c
> > +++ b/t/helper/test-ref-store.c
> > @@ -1,6 +1,6 @@
> >  #include "test-tool.h"
> >  #include "hex.h"
> > -#include "refs.h"
> > +#include "refs/refs-internal.h"
> >  #include "setup.h"
> >  #include "worktree.h"
> >  #include "object-store-ll.h"
> > @@ -221,6 +221,30 @@ static int cmd_verify_ref(struct ref_store *refs, const char **argv)
> >  	return ret;
> >  }
> >  
> > +static int cmd_ref_exists(struct ref_store *refs, const char **argv)
> > +{
> > +	const char *refname = notnull(*argv++, "refname");
> > +	struct strbuf unused_referent = STRBUF_INIT;
> > +	struct object_id unused_oid;
> > +	unsigned int unused_type;
> > +	int failure_errno;
> > +
> > +	if (refs_read_raw_ref(refs, refname, &unused_oid, &unused_referent,
> > +			      &unused_type, &failure_errno)) {
> > +		/*
> > +		 * We handle ENOENT separately here such that it is possible to
> > +		 * distinguish actually-missing references from any kind of
> > +		 * generic error.
> > +		 */
> > +		if (failure_errno == ENOENT)
> > +			return 17;
> 
> Can we tell between the cases where the ref itself is missing, and
> the requested ref is symbolic and points at a missing ref?  This
> particular case might be OK, but there may other cases where this
> "special case" may not be narrow enough.

Yes, because `refs_read_raw_ref()` doesn't concern itself with recursive
resolving of the reference, which is done at a higher level. It only
reads and parses the reference without caring whether their target
actually exists.

> As long we are going to spend cycles to refine the classification of
> error conditions, which is a very good thing to aim for the reason
> described in the proposed log message, namely "rev-parse can fail
> for reasons other than the ref being absent", I have to wonder again
> that the fruit of such an effort should become available in the
> production code, instead of being kept only in test-tool.

Fair enough, I'm happy to lift this up into production code. I just
didn't think this would be all that useful in general, but I can see
that somebody might want to use such functionality as part of our
plumbing interfaces.

I wonder what the best spot would be for it. Should we add a new
`--exists` switch to git-rev-parse(1)?

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH 08/11] t4207: delete replace references via git-update-ref(1)

[Taylor Blau]

On Wed, Oct 18, 2023 at 07:35:37AM +0200, Patrick Steinhardt wrote:
> In t4207 we set up a set of replace objects via git-replace(1). Because
> these references should not be impacting subsequent tests we also set up
> some cleanup logic that deletes the replacement references via a call to
> `rm -rf`. This reaches into the internal implementation details of the
> reference backend and will thus break when we grow an alternative refdb
> implementation.
>
> Refactor the tests to delete the replacement refs via Git commands so
> that we become independent of the actual refdb that's in use. As we
> don't have a nice way to delete all replacements or all references in a
> certain namespace, we opt for a combination of git-for-each-ref(1) and
> git-update-ref(1)'s `--stdin` mode.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
>  t/t4207-log-decoration-colors.sh | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
> index 21986a866df..d138e513a04 100755
> --- a/t/t4207-log-decoration-colors.sh
> +++ b/t/t4207-log-decoration-colors.sh
> @@ -71,7 +71,7 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
>  '
>
>  test_expect_success 'test coloring with replace-objects' '
> -	test_when_finished rm -rf .git/refs/replace* &&
> +	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&

Here and below, should we avoid the for-each-ref showing up on the
left-hand side of the pipe? I'd think we want something closer to:

    test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} >in && git update-ref --stdin <in" &&

But having to quote the --format argument with "${SQ}"s makes the whole
thing a little awkward to read and parse.

Do you think that something like the below would be a readability
improvement?

diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
index d138e513a0..de8f6638cb 100755
--- a/t/t4207-log-decoration-colors.sh
+++ b/t/t4207-log-decoration-colors.sh
@@ -70,8 +70,13 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
 	cmp_filtered_decorations
 '

--- >8 ---
+remove_replace_refs () {
+	git for-each-ref 'refs/replace*/**' --format='delete %(refname)' >in &&
+	git update-ref --stdin <in
+}
+
 test_expect_success 'test coloring with replace-objects' '
-	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
+	test_when_finished remove_replace_refs &&
 	test_commit C &&
 	test_commit D &&

@@ -99,7 +104,7 @@ EOF
 '

 test_expect_success 'test coloring with grafted commit' '
-	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
+	test_when_finished remove_replace_refs &&

 	git replace --graft HEAD HEAD~2 &&
--- 8< ---

Thanks,
Taylor

Re: [PATCH 02/11] t: allow skipping expected object ID in `ref-store update-ref`

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

>> Good.
>> 
>> Even better would be to make the old one optional, though.
>
> I was also a bit torn when writing this. We could of course make the
> behaviour conditional on whether `argc` is 4 or 5. But I wasn't quite
> sure how important it is to provide a nice UI for this test helper, and
> we don't have `argc` readily available. It's not hard to count them
> manually, but until now I was under the impression that the test helpers
> only need to be "good enough".

Yup, good enough would probably be good enough in this case, I agree
;-)

Re: [PATCH 04/11] t: convert tests to not write references via the filesystem

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

>> OK, the original checks "if a random garbage file, which may not
>> necessarily be a ref, exists at $n_dir, we cannot create a ref at
>> $n_dir/fixes, due to D/F conflict" more directly, but as long as our
>> intention is to enforce the D/F restriction across different ref
>> backends [*], creating a ref at $n_dir and making sure $n_dir/fixes
>> cannot be created is an equivalent check that is better (because it
>> can be applied for other backends).
>> 
>>     Side note: there is no fundamental need to, though, and there
>>          are cases where being able to have the 'seen' branch and
>>          'seen/ps/ref-test-tools' branches at the same time is
>>          beneficial---packed-refs and ref-table backends would not
>>          have such an inherent limitation, but they can of course be
>>          castrated to match what files-backend can(not) do.
>
> I think initially it is beneficial to keep any such restriction and cut
> back new backends to match them, even though it's more work.

Note that the same thing can be said for "Can I have Main and main
branches?".  Loose refs on systems with case-sensitive filesystem
are not penalized, though.

In any case, I think we are in agreement.

>> I trust that this will be corrected to use some wrapper around "git
>> symbolic-ref" (or an equivalent for it as a test-tool subcommand) in
>> some future patch, if not in this series?
>
> Yup, this is getting fixed in a subsequent patch. I had two different
> options to structure this series:
> ...
> There were two reasons why I didn't like the first iteration:

Yup.  I tend to agree with the choice and criteria you made and used
here.

Re: [PATCH 08/11] t4207: delete replace references via git-update-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 3339 bytes --]

On Mon, Oct 23, 2023 at 12:42:08PM -0400, Taylor Blau wrote:
> On Wed, Oct 18, 2023 at 07:35:37AM +0200, Patrick Steinhardt wrote:
> > In t4207 we set up a set of replace objects via git-replace(1). Because
> > these references should not be impacting subsequent tests we also set up
> > some cleanup logic that deletes the replacement references via a call to
> > `rm -rf`. This reaches into the internal implementation details of the
> > reference backend and will thus break when we grow an alternative refdb
> > implementation.
> >
> > Refactor the tests to delete the replacement refs via Git commands so
> > that we become independent of the actual refdb that's in use. As we
> > don't have a nice way to delete all replacements or all references in a
> > certain namespace, we opt for a combination of git-for-each-ref(1) and
> > git-update-ref(1)'s `--stdin` mode.
> >
> > Signed-off-by: Patrick Steinhardt <ps@pks.im>
> > ---
> >  t/t4207-log-decoration-colors.sh | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
> > index 21986a866df..d138e513a04 100755
> > --- a/t/t4207-log-decoration-colors.sh
> > +++ b/t/t4207-log-decoration-colors.sh
> > @@ -71,7 +71,7 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
> >  '
> >
> >  test_expect_success 'test coloring with replace-objects' '
> > -	test_when_finished rm -rf .git/refs/replace* &&
> > +	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
> 
> Here and below, should we avoid the for-each-ref showing up on the
> left-hand side of the pipe? I'd think we want something closer to:
> 
>     test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} >in && git update-ref --stdin <in" &&
> 
> But having to quote the --format argument with "${SQ}"s makes the whole
> thing a little awkward to read and parse.
> 
> Do you think that something like the below would be a readability
> improvement?

Yes, this certainly looks like a good improvement to me, thanks!

Patrick

> diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
> index d138e513a0..de8f6638cb 100755
> --- a/t/t4207-log-decoration-colors.sh
> +++ b/t/t4207-log-decoration-colors.sh
> @@ -70,8 +70,13 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
>  	cmp_filtered_decorations
>  '
> 
> --- >8 ---
> +remove_replace_refs () {
> +	git for-each-ref 'refs/replace*/**' --format='delete %(refname)' >in &&
> +	git update-ref --stdin <in
> +}
> +
>  test_expect_success 'test coloring with replace-objects' '
> -	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
> +	test_when_finished remove_replace_refs &&
>  	test_commit C &&
>  	test_commit D &&
> 
> @@ -99,7 +104,7 @@ EOF
>  '
> 
>  test_expect_success 'test coloring with grafted commit' '
> -	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
> +	test_when_finished remove_replace_refs &&
> 
>  	git replace --graft HEAD HEAD~2 &&
> --- 8< ---
> 
> Thanks,
> Taylor

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 0/9] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 19236 bytes --]

Hi,

this is the second version of my patch series that aims to reduce access
to on-disk data structures in favor of using plumbing tools where
possible.

Changes compared to v1:

    - Patches 1, 3: I've dropped these patches that introduced and
      started to use the test helper for reference existence. This has
      been split out into a separate patch series now that instead
      implements the logic as part of git-show-ref(1), see [1].

    - Patch 4: I've made it more explicit that tests in t1450 are all
      ran in detached HEAD mode via a new `orig_head` variable that is
      set in the test setup. This variable is later used to reset HEAD
      back to that original state.

    - Patch 4, 5: I've reordered some of the logic such that we schedule
      `test_when_finished` before doing the actual mutation of the repo.

    - Patch 8: I've adopted the proposal of a `remove_replace_refs()` 
      helper function to clean up replace refs.

    - Now comes with a base commit. Unbelievable.

Thanks for all the feedback so far!

Patrick

Patrick Steinhardt (9):
  t: allow skipping expected object ID in `ref-store update-ref`
  t: convert tests to not write references via the filesystem
  t: convert tests to not access symrefs via the filesystem
  t: convert tests to not access reflog via the filesystem
  t1450: convert tests to remove worktrees via git-worktree(1)
  t4207: delete replace references via git-update-ref(1)
  t7300: assert exact states of repo
  t7900: assert the absence of refs via git-for-each-ref(1)
  t: mark several tests that assume the files backend with REFFILES

 t/helper/test-ref-store.c          | 11 ++++---
 t/t1400-update-ref.sh              | 50 ++++++++++++++++--------------
 t/t1430-bad-ref-name.sh            | 12 +++----
 t/t1450-fsck.sh                    | 44 +++++++++++++-------------
 t/t2011-checkout-invalid-head.sh   | 16 +++++-----
 t/t3200-branch.sh                  | 41 ++++++++++++------------
 t/t3400-rebase.sh                  |  2 +-
 t/t3404-rebase-interactive.sh      |  2 +-
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t4207-log-decoration-colors.sh   | 10 ++++--
 t/t5526-fetch-submodules.sh        |  2 +-
 t/t5605-clone-local.sh             |  4 +--
 t/t5702-protocol-v2.sh             | 24 ++++++++++----
 t/t7300-clean.sh                   | 23 ++++++++------
 t/t7900-maintenance.sh             |  3 +-
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 17 files changed, 142 insertions(+), 108 deletions(-)

Range-diff against v1:
 1:  e947feb1c77 <  -:  ----------- t: add helpers to test for reference existence
 2:  1f615d62f99 =  1:  c868198f8c1 t: allow skipping expected object ID in `ref-store update-ref`
 3:  ac6a49c7c84 <  -:  ----------- t: convert tests to use helpers for reference existence
 4:  c79431c0bf1 !  2:  4c0939d868e t: convert tests to not write references via the filesystem
    @@ Commit message
         test tool. The latter is required in some cases where safety checks of
         git-update-ref(1) would otherwise reject a reference update.
     
    +    While at it, refactor some of the tests to schedule the cleanup command
    +    via `test_when_finished` before modifying the repository.
    +
         Signed-off-by: Patrick Steinhardt <ps@pks.im>
     
      ## t/t1400-update-ref.sh ##
    @@ t/t1400-update-ref.sh: test_expect_success "delete $m without oldvalue verificat
      
      test_expect_success "create $m (by HEAD)" '
     @@ t/t1400-update-ref.sh: test_expect_success 'delete symref without dereference when the referred ref is
    + '
      
      test_expect_success 'update-ref -d is not confused by self-reference' '
    ++	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
      	git symbolic-ref refs/heads/self refs/heads/self &&
     -	test_when_finished "rm -f .git/refs/heads/self" &&
    -+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
      	test_path_is_file .git/refs/heads/self &&
      	test_must_fail git update-ref -d refs/heads/self &&
      	test_path_is_file .git/refs/heads/self
    -@@ t/t1400-update-ref.sh: test_expect_success 'update-ref -d is not confused by self-reference' '
    + '
      
      test_expect_success 'update-ref --no-deref -d can delete self-reference' '
    ++	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
      	git symbolic-ref refs/heads/self refs/heads/self &&
     -	test_when_finished "rm -f .git/refs/heads/self" &&
    -+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
      	test_path_is_file .git/refs/heads/self &&
      	git update-ref --no-deref -d refs/heads/self &&
      	test_must_fail git show-ref --verify -q refs/heads/self
    @@ t/t1400-update-ref.sh: test_expect_success 'Query "main@{2005-05-28}" (past end
      '
      
     -rm -f .git/$m .git/logs/$m expect
    ++rm -f expect
     +git update-ref -d $m
      
      test_expect_success 'creating initial files' '
      	test_when_finished rm -f M &&
     
      ## t/t1450-fsck.sh ##
    +@@ t/t1450-fsck.sh: test_expect_success setup '
    + 	git config --unset i18n.commitencoding &&
    + 	git checkout HEAD^0 &&
    + 	test_commit B fileB two &&
    ++	orig_head=$(git rev-parse HEAD) &&
    + 	git tag -d A B &&
    + 	git reflog expire --expire=now --all
    + '
     @@ t/t1450-fsck.sh: test_expect_success 'zlib corrupt loose object output ' '
      '
      
      test_expect_success 'branch pointing to non-commit' '
     -	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
     +	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
    -+	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
      	test_when_finished "git update-ref -d refs/heads/invalid" &&
    ++	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
      	test_must_fail git fsck 2>out &&
      	test_i18ngrep "not a commit" out
      '
    @@ t/t1450-fsck.sh: test_expect_success 'zlib corrupt loose object output ' '
      test_expect_success 'HEAD link pointing at a funny object' '
     -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
     -	mv .git/HEAD .git/SAVED_HEAD &&
    -+	saved_head=$(git rev-parse --verify HEAD) &&
    -+	test_when_finished "git update-ref HEAD ${saved_head}" &&
    ++	test_when_finished "git update-ref HEAD $orig_head" &&
      	echo $ZERO_OID >.git/HEAD &&
      	# avoid corrupt/broken HEAD from interfering with repo discovery
      	test_must_fail env GIT_DIR=.git git fsck 2>out &&
    @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny object' '
      test_expect_success 'HEAD link pointing at a funny place' '
     -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
     -	mv .git/HEAD .git/SAVED_HEAD &&
    -+	saved_head=$(git rev-parse --verify HEAD) &&
    -+	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&
    ++	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
      	echo "ref: refs/funny/place" >.git/HEAD &&
      	# avoid corrupt/broken HEAD from interfering with repo discovery
      	test_must_fail env GIT_DIR=.git git fsck 2>out &&
    @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny place' '
      
      test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
     -	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
    -+	saved_head=$(git rev-parse --verify HEAD) &&
    -+	test_when_finished "git update-ref HEAD $saved_head" &&
    ++	test_when_finished "git update-ref HEAD $orig_head" &&
      	test_when_finished "rm -rf .git/worktrees wt" &&
      	git worktree add wt &&
     -	mv .git/HEAD .git/SAVED_HEAD &&
 5:  1ac120368c6 !  3:  048583ed2c3 t: convert tests to not access symrefs via the filesystem
    @@ Commit message
      ## t/t1400-update-ref.sh ##
     @@ t/t1400-update-ref.sh: test_expect_success 'delete symref without dereference when the referred ref is
      test_expect_success 'update-ref -d is not confused by self-reference' '
    - 	git symbolic-ref refs/heads/self refs/heads/self &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
    + 	git symbolic-ref refs/heads/self refs/heads/self &&
     -	test_path_is_file .git/refs/heads/self &&
     +	git symbolic-ref --no-recurse refs/heads/self &&
      	test_must_fail git update-ref -d refs/heads/self &&
    @@ t/t1400-update-ref.sh: test_expect_success 'delete symref without dereference wh
      '
      
      test_expect_success 'update-ref --no-deref -d can delete self-reference' '
    - 	git symbolic-ref refs/heads/self refs/heads/self &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
    + 	git symbolic-ref refs/heads/self refs/heads/self &&
     -	test_path_is_file .git/refs/heads/self &&
     +	git symbolic-ref --no-recurse refs/heads/self &&
      	git update-ref --no-deref -d refs/heads/self &&
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref -d can delete broken na
     -	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    - 	test_ref_exists refs/heads/broken...symref &&
      	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
    + 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delete symref with broken name
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delet
     -	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    - 	test_ref_exists refs/heads/broken...symref &&
      	git branch -d broken...symref >output 2>error &&
    + 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'branch -d can delete symref with broken name' '
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'branch -d can delete symref with b
     -	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    - 	test_ref_exists refs/heads/broken...symref &&
      	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
    + 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delet
     -	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    - 	test_ref_exists refs/heads/broken...symref &&
      	git branch -d broken...symref >output 2>error &&
    + 	test_path_is_missing .git/refs/heads/broken...symref &&
     
      ## t/t1450-fsck.sh ##
     @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny object' '
    + 
      test_expect_success 'HEAD link pointing at a funny place' '
    - 	saved_head=$(git rev-parse --verify HEAD) &&
    - 	test_when_finished "git update-ref --no-deref HEAD ${saved_head}" &&
    + 	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
     -	echo "ref: refs/funny/place" >.git/HEAD &&
     +	test-tool ref-store main create-symref HEAD refs/funny/place &&
      	# avoid corrupt/broken HEAD from interfering with repo discovery
    @@ t/t3200-branch.sh: test_expect_success 'git branch -M should leave orphaned HEAD
     +		git symbolic-ref HEAD >expect &&
     +		echo refs/heads/lonely >actual &&
     +		test_cmp expect actual &&
    - 		test_ref_missing refs/head/lonely &&
    + 		test_path_is_missing .git/refs/head/lonely &&
      		git branch -M main mistress &&
     -		grep lonely .git/HEAD
     +		git symbolic-ref HEAD >expect &&
    @@ t/t3200-branch.sh: test_expect_success 'deleting a symref' '
     +	git symbolic-ref --no-recurse refs/heads/dangling-symref &&
      	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
      	git branch -d dangling-symref >actual &&
    - 	test_ref_missing refs/heads/dangling-symref &&
    + 	test_path_is_missing .git/refs/heads/dangling-symref &&
     
      ## t/t4013-diff-various.sh ##
     @@ t/t4013-diff-various.sh: test_expect_success 'log -S requires an argument' '
 6:  eaac658bbfd !  4:  5e7937e7904 t: convert tests to not access reflog via the filesystem
    @@ t/t3200-branch.sh: test_expect_success 'git branch HEAD should fail' '
      test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
      	GIT_COMMITTER_DATE="2005-05-26 23:30" \
      	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
    - 	test_ref_exists refs/heads/d/e/f &&
    + 	test_path_is_file .git/refs/heads/d/e/f &&
     -	test_path_is_file .git/logs/refs/heads/d/e/f &&
     -	test_cmp expect .git/logs/refs/heads/d/e/f
     +	git reflog show --no-abbrev-commit refs/heads/d/e/f >actual &&
    @@ t/t3200-branch.sh: test_expect_success '--set-upstream-to notices an error to se
      test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
      	GIT_COMMITTER_DATE="2005-05-26 23:30" \
      	git checkout -b g/h/i -l main &&
    - 	test_ref_exists refs/heads/g/h/i &&
    + 	test_path_is_file .git/refs/heads/g/h/i &&
     -	test_path_is_file .git/logs/refs/heads/g/h/i &&
     -	test_cmp expect .git/logs/refs/heads/g/h/i
     +	git reflog show --no-abbrev-commit refs/heads/g/h/i >actual &&
 7:  3dc65a80074 !  5:  089565a358e t1450: convert tests to remove worktrees via git-worktree(1)
    @@ Commit message
     
      ## t/t1450-fsck.sh ##
     @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny place' '
    + 
      test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
    - 	saved_head=$(git rev-parse --verify HEAD) &&
    - 	test_when_finished "git update-ref HEAD $saved_head" &&
    + 	test_when_finished "git update-ref HEAD $orig_head" &&
     -	test_when_finished "rm -rf .git/worktrees wt" &&
     +	test_when_finished "git worktree remove -f wt" &&
      	git worktree add wt &&
 8:  c4d09e3e5db !  6:  cb738888ed7 t4207: delete replace references via git-update-ref(1)
    @@ Commit message
     
      ## t/t4207-log-decoration-colors.sh ##
     @@ t/t4207-log-decoration-colors.sh: ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
    + 	cmp_filtered_decorations
      '
      
    ++remove_replace_refs () {
    ++	git for-each-ref 'refs/replace*/**' --format='delete %(refname)' >in &&
    ++	git update-ref --stdin <in &&
    ++	rm in
    ++}
    ++
      test_expect_success 'test coloring with replace-objects' '
     -	test_when_finished rm -rf .git/refs/replace* &&
    -+	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
    ++	test_when_finished remove_replace_refs &&
      	test_commit C &&
      	test_commit D &&
      
    @@ t/t4207-log-decoration-colors.sh: EOF
      
      test_expect_success 'test coloring with grafted commit' '
     -	test_when_finished rm -rf .git/refs/replace* &&
    -+	test_when_finished "git for-each-ref refs/replace*/** --format=${SQ}delete %(refname)${SQ} | git update-ref --stdin" &&
    ++	test_when_finished remove_replace_refs &&
      
      	git replace --graft HEAD HEAD~2 &&
      
 9:  153b5b199c8 =  7:  e730e011de4 t7300: assert exact states of repo
10:  b99d98b00a3 =  8:  a1bdea52397 t7900: assert the absence of refs via git-for-each-ref(1)
11:  67cb282a414 !  9:  497e43ae5c3 t: mark several tests that assume the files backend with REFFILES
    @@ t/t1450-fsck.sh: test_expect_success 'branch pointing to non-commit' '
      
     -test_expect_success 'HEAD link pointing at a funny object' '
     +test_expect_success REFFILES 'HEAD link pointing at a funny object' '
    - 	saved_head=$(git rev-parse --verify HEAD) &&
    - 	test_when_finished "git update-ref HEAD ${saved_head}" &&
    + 	test_when_finished "git update-ref HEAD $orig_head" &&
      	echo $ZERO_OID >.git/HEAD &&
    + 	# avoid corrupt/broken HEAD from interfering with repo discovery
     @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny place' '
      	test_i18ngrep "HEAD points to something strange" out
      '
      
     -test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
     +test_expect_success REFFILES 'HEAD link pointing at a funny object (from different wt)' '
    - 	saved_head=$(git rev-parse --verify HEAD) &&
    - 	test_when_finished "git update-ref HEAD $saved_head" &&
    + 	test_when_finished "git update-ref HEAD $orig_head" &&
      	test_when_finished "git worktree remove -f wt" &&
    + 	git worktree add wt &&
     @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
      	test_i18ngrep "main-worktree/HEAD: detached HEAD points" out
      '
    @@ t/t2011-checkout-invalid-head.sh: test_expect_success 'create ref directory/file
     
      ## t/t3200-branch.sh ##
     @@ t/t3200-branch.sh: test_expect_success 'git branch --help should not have created a bogus branch' '
    - 	test_ref_missing refs/heads/--help
    + 	test_path_is_missing .git/refs/heads/--help
      '
      
     -test_expect_success 'branch -h in broken repository' '
    @@ t/t3200-branch.sh: test_expect_success 'git branch -M baz bam should succeed whe
      	git worktree add -f bazdir2 baz &&
      	touch .git/worktrees/bazdir1/HEAD.lock &&
     @@ t/t3200-branch.sh: test_expect_success 'renaming a symref is not allowed' '
    - 	test_ref_missing refs/heads/new-topic
    + 	test_path_is_missing .git/refs/heads/new-topic
      '
      
     -test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '

base-commit: a9ecda2788e229afc9b611acaa26d0d9d4da53ed
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 1/9] t: allow skipping expected object ID in `ref-store update-ref`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1771 bytes --]

We require the caller to pass both the old and new expected object ID to
our `test-tool ref-store update-ref` helper. When trying to update a
symbolic reference though it's impossible to specify the expected object
ID, which means that the test would instead have to force-update the
reference. This is currently impossible though.

Update the helper to optionally skip verification of the old object ID
in case the test passes in an empty old object ID as input.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/helper/test-ref-store.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 48552e6a9e0..702ec1f128a 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -298,16 +298,19 @@ static int cmd_update_ref(struct ref_store *refs, const char **argv)
 	const char *new_sha1_buf = notnull(*argv++, "new-sha1");
 	const char *old_sha1_buf = notnull(*argv++, "old-sha1");
 	unsigned int flags = arg_flags(*argv++, "flags", transaction_flags);
-	struct object_id old_oid;
+	struct object_id old_oid, *old_oid_ptr = NULL;
 	struct object_id new_oid;
 
-	if (get_oid_hex(old_sha1_buf, &old_oid))
-		die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+	if (*old_sha1_buf) {
+		if (get_oid_hex(old_sha1_buf, &old_oid))
+			die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+		old_oid_ptr = &old_oid;
+	}
 	if (get_oid_hex(new_sha1_buf, &new_oid))
 		die("cannot parse %s as %s", new_sha1_buf, the_hash_algo->name);
 
 	return refs_update_ref(refs, msg, refname,
-			       &new_oid, &old_oid,
+			       &new_oid, old_oid_ptr,
 			       flags, UPDATE_REFS_DIE_ON_ERR);
 }
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 2/9] t: convert tests to not write references via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 9376 bytes --]

Some of our tests manually create, update or delete references by
writing the respective new values into the filesystem directly. While
this works with the current files reference backend, this will break
once we have a second reference backend implementation in our codebase.

Refactor these tests to instead use git-update-ref(1) or our `ref-store`
test tool. The latter is required in some cases where safety checks of
git-update-ref(1) would otherwise reject a reference update.

While at it, refactor some of the tests to schedule the cleanup command
via `test_when_finished` before modifying the repository.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh         | 17 ++++++++---------
 t/t1450-fsck.sh               | 26 +++++++++++++-------------
 t/t3404-rebase-interactive.sh |  2 +-
 t/t5526-fetch-submodules.sh   |  2 +-
 4 files changed, 23 insertions(+), 24 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 4d66cd7f4a1..91cc6dff724 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -9,8 +9,6 @@ test_description='Test git update-ref and basic ref logging'
 Z=$ZERO_OID
 
 m=refs/heads/main
-n_dir=refs/heads/gu
-n=$n_dir/fixes
 outside=refs/foo
 bare=bare-repo
 
@@ -62,10 +60,10 @@ test_expect_success "delete $m without oldvalue verification" '
 	test_must_fail git show-ref --verify -q $m
 '
 
-test_expect_success "fail to create $n" '
-	test_when_finished "rm -f .git/$n_dir" &&
-	touch .git/$n_dir &&
-	test_must_fail git update-ref $n $A
+test_expect_success "fail to create $n due to file/directory conflict" '
+	test_when_finished "git update-ref -d refs/heads/gu" &&
+	git update-ref refs/heads/gu $A &&
+	test_must_fail git update-ref refs/heads/gu/fixes $A
 '
 
 test_expect_success "create $m (by HEAD)" '
@@ -221,16 +219,16 @@ test_expect_success 'delete symref without dereference when the referred ref is
 '
 
 test_expect_success 'update-ref -d is not confused by self-reference' '
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
 	test_path_is_file .git/refs/heads/self
 '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
@@ -434,7 +432,8 @@ test_expect_success 'Query "main@{2005-05-28}" (past end of history)' '
 	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
 '
 
-rm -f .git/$m .git/logs/$m expect
+rm -f expect
+git update-ref -d $m
 
 test_expect_success 'creating initial files' '
 	test_when_finished rm -f M &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 10a539158c4..f55b539b083 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -15,6 +15,7 @@ test_expect_success setup '
 	git config --unset i18n.commitencoding &&
 	git checkout HEAD^0 &&
 	test_commit B fileB two &&
+	orig_head=$(git rev-parse HEAD) &&
 	git tag -d A B &&
 	git reflog expire --expire=now --all
 '
@@ -115,15 +116,15 @@ test_expect_success 'zlib corrupt loose object output ' '
 '
 
 test_expect_success 'branch pointing to non-commit' '
-	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
 	test_when_finished "git update-ref -d refs/heads/invalid" &&
+	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "not a commit" out
 '
 
 test_expect_success 'HEAD link pointing at a funny object' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	test_when_finished "git update-ref HEAD $orig_head" &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -131,8 +132,7 @@ test_expect_success 'HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'HEAD link pointing at a funny place' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
 	echo "ref: refs/funny/place" >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -140,10 +140,9 @@ test_expect_success 'HEAD link pointing at a funny place' '
 '
 
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
+	test_when_finished "git update-ref HEAD $orig_head" &&
 	test_when_finished "rm -rf .git/worktrees wt" &&
 	git worktree add wt &&
-	mv .git/HEAD .git/SAVED_HEAD &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail git -C wt fsck 2>out &&
@@ -161,7 +160,8 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 test_expect_success 'other worktree HEAD link pointing at missing object' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD &&
+	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
+	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD: invalid sha1 pointer" out
 '
@@ -391,7 +391,7 @@ test_expect_success 'tag pointing to nonexistent' '
 
 	tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/invalid &&
+	git update-ref refs/tags/invalid $tag &&
 	test_when_finished "git update-ref -d refs/tags/invalid" &&
 	test_must_fail git fsck --tags >out &&
 	test_i18ngrep "broken link" out
@@ -411,7 +411,7 @@ test_expect_success 'tag pointing to something else than its type' '
 
 	tag=$(git hash-object -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags
 '
@@ -428,7 +428,7 @@ test_expect_success 'tag with incorrect tag name & missing tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	git fsck --tags 2>out &&
 
@@ -452,7 +452,7 @@ test_expect_success 'tag with bad tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag .*: invalid author/committer" out
@@ -471,7 +471,7 @@ test_expect_success 'tag with NUL in header' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <tag-NUL-header) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag $tag.*unterminated header: NUL at offset" out
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 8ea2bf13026..d2a7a91f170 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -2160,7 +2160,7 @@ test_expect_success '--update-refs: check failed ref update' '
 	# recorded in the update-refs file. We will force-update the
 	# "second" ref, but "git branch -f" will not work because of
 	# the lock in the update-refs file.
-	git rev-parse third >.git/refs/heads/second &&
+	git update-ref refs/heads/second third &&
 
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 26e933f93ae..7ab220fa313 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -771,7 +771,7 @@ test_expect_success 'fetching submodule into a broken repository' '
 	git -C dst fetch --recurse-submodules &&
 
 	# Break the receiving submodule
-	rm -f dst/sub/.git/HEAD &&
+	test-tool -C dst/sub ref-store main delete-refs REF_NO_DEREF msg HEAD &&
 
 	# NOTE: without the fix the following tests will recurse forever!
 	# They should terminate with an error.
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 3/9] t: convert tests to not access symrefs via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 13728 bytes --]

Some of our tests access symbolic references via the filesystem
directly. While this works with the current files reference backend, it
this will break once we have a second reference backend in our codebase.

Refactor these tests to instead use git-symbolic-ref(1) or our
`ref-store` test tool. The latter is required in some cases where safety
checks of git-symbolic-ref(1) would otherwise reject writing a symbolic
reference.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh              |  8 ++++----
 t/t1430-bad-ref-name.sh            | 12 ++++++------
 t/t1450-fsck.sh                    |  4 ++--
 t/t3200-branch.sh                  |  9 ++++++---
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t5605-clone-local.sh             |  2 +-
 t/t5702-protocol-v2.sh             | 24 ++++++++++++++++++------
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 9 files changed, 40 insertions(+), 25 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 91cc6dff724..51a8d0bba98 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -221,15 +221,15 @@ test_expect_success 'delete symref without dereference when the referred ref is
 test_expect_success 'update-ref -d is not confused by self-reference' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
-	test_path_is_file .git/refs/heads/self
+	git symbolic-ref --no-recurse refs/heads/self
 '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
@@ -239,7 +239,7 @@ test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
 	test_when_finished "git update-ref -d refs/heads/ref-to-bad" &&
-	test_path_is_file .git/refs/heads/ref-to-bad &&
+	git symbolic-ref --no-recurse refs/heads/ref-to-bad &&
 	git update-ref --no-deref -d refs/heads/ref-to-bad &&
 	test_must_fail git show-ref --verify -q refs/heads/ref-to-bad
 '
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index ff1c967d550..7d2dfc97256 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -164,9 +164,9 @@ test_expect_success 'rev-parse skips symref pointing to broken name' '
 test_expect_success 'for-each-ref emits warnings for broken names' '
 	test-tool ref-store main update-ref msg "refs/heads/broken...ref" $main_sha1 $ZERO_OID REF_SKIP_REFNAME_VERIFICATION &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
-	printf "ref: refs/heads/broken...ref\n" >.git/refs/heads/badname &&
+	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git for-each-ref >output 2>error &&
 	! grep -e "broken\.\.\.ref" output &&
@@ -252,7 +252,7 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
 	test_path_is_missing .git/refs/heads/broken...symref &&
@@ -261,7 +261,7 @@ test_expect_success 'update-ref --no-deref -d can delete symref with broken name
 '
 
 test_expect_success 'branch -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git branch -d broken...symref >output 2>error &&
 	test_path_is_missing .git/refs/heads/broken...symref &&
@@ -270,7 +270,7 @@ test_expect_success 'branch -d can delete symref with broken name' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
 	test_path_is_missing .git/refs/heads/broken...symref &&
@@ -279,7 +279,7 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
 '
 
 test_expect_success 'branch -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git branch -d broken...symref >output 2>error &&
 	test_path_is_missing .git/refs/heads/broken...symref &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index f55b539b083..a3c97b9c7fc 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -133,7 +133,7 @@ test_expect_success 'HEAD link pointing at a funny object' '
 
 test_expect_success 'HEAD link pointing at a funny place' '
 	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
-	echo "ref: refs/funny/place" >.git/HEAD &&
+	test-tool ref-store main create-symref HEAD refs/funny/place &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
 	test_i18ngrep "HEAD points to something strange" out
@@ -169,7 +169,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "ref: refs/funny/place" >.git/worktrees/other/HEAD &&
+	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD points to something strange" out
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 080e4f24a6e..7d9393f190d 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -212,10 +212,13 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 		cd orphan &&
 		test_commit initial &&
 		git checkout --orphan lonely &&
-		grep lonely .git/HEAD &&
+		git symbolic-ref HEAD >expect &&
+		echo refs/heads/lonely >actual &&
+		test_cmp expect actual &&
 		test_path_is_missing .git/refs/head/lonely &&
 		git branch -M main mistress &&
-		grep lonely .git/HEAD
+		git symbolic-ref HEAD >expect &&
+		test_cmp expect actual
 	)
 '
 
@@ -806,7 +809,7 @@ test_expect_success 'deleting a symref' '
 
 test_expect_success 'deleting a dangling symref' '
 	git symbolic-ref refs/heads/dangling-symref nowhere &&
-	test_path_is_file .git/refs/heads/dangling-symref &&
+	git symbolic-ref --no-recurse refs/heads/dangling-symref &&
 	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
 	git branch -d dangling-symref >actual &&
 	test_path_is_missing .git/refs/heads/dangling-symref &&
diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
index 5de1d190759..5abbea36b39 100755
--- a/t/t4013-diff-various.sh
+++ b/t/t4013-diff-various.sh
@@ -514,7 +514,7 @@ test_expect_success 'log -S requires an argument' '
 '
 
 test_expect_success 'diff --cached on unborn branch' '
-	echo ref: refs/heads/unborn >.git/HEAD &&
+	git symbolic-ref HEAD refs/heads/unborn &&
 	git diff --cached >result &&
 	process_diffs result >actual &&
 	process_diffs "$TEST_DIRECTORY/t4013/diff.diff_--cached" >expected &&
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index af4a123cd22..57b298a4e22 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2265,7 +2265,7 @@ test_expect_success REFFILES 'log diagnoses bogus HEAD hash' '
 
 test_expect_success REFFILES 'log diagnoses bogus HEAD symref' '
 	git init empty &&
-	echo "ref: refs/heads/invalid.lock" > empty/.git/HEAD &&
+	test-tool -C empty ref-store main create-symref HEAD refs/heads/invalid.lock &&
 	test_must_fail git -C empty log 2>stderr &&
 	test_i18ngrep broken stderr &&
 	test_must_fail git -C empty log --default totally-bogus 2>stderr &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index 1d7b1abda1a..a26689de379 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -65,7 +65,7 @@ test_expect_success 'Even without -l, local will make a hardlink' '
 '
 
 test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
-	echo "ref: refs/heads/nonexistent" > a.git/HEAD &&
+	git -C a.git symbolic-ref HEAD refs/heads/nonexistent &&
 	git clone a d &&
 	(cd d &&
 	git fetch &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 6af5c2062fd..dcc4cd95fe7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -221,7 +221,9 @@ test_expect_success 'clone of empty repo propagates name of default branch' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/mydefaultbranch >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success '...but not if explicitly forbidden by config' '
@@ -234,7 +236,9 @@ test_expect_success '...but not if explicitly forbidden by config' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	! grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/main >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'bare clone propagates empty default branch' '
@@ -247,7 +251,9 @@ test_expect_success 'bare clone propagates empty default branch' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare \
 		"file://$(pwd)/file_empty_parent" file_empty_child.git &&
-	grep "refs/heads/mydefaultbranch" file_empty_child.git/HEAD
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_empty_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
@@ -265,7 +271,9 @@ test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	grep "warning: remote HEAD refers to nonexistent ref" stderr
 '
 
@@ -295,7 +303,9 @@ test_expect_success 'bare clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child.git 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	! grep "warning:" stderr
 '
 
@@ -315,7 +325,9 @@ test_expect_success 'defaulted HEAD uses remote branch if available' '
 	git -c init.defaultBranch=branchwithstuff -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/branchwithstuff" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/branchwithstuff" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	test_path_is_file file_unborn_child/stuff.t &&
 	! grep "warning:" stderr
 '
diff --git a/t/t9133-git-svn-nested-git-repo.sh b/t/t9133-git-svn-nested-git-repo.sh
index d8d536269cf..8ca24670acb 100755
--- a/t/t9133-git-svn-nested-git-repo.sh
+++ b/t/t9133-git-svn-nested-git-repo.sh
@@ -11,7 +11,7 @@ test_expect_success 'setup repo with a git repo inside it' '
 	(
 		cd s &&
 		git init &&
-		test -f .git/HEAD &&
+		git symbolic-ref HEAD &&
 		> .git/a &&
 		echo a > a &&
 		svn_cmd add .git a &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 4/9] t: convert tests to not access reflog via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 7139 bytes --]

Some of our tests reach directly into the filesystem in order to both
read or modify the reflog, which will break once we have a second
reference backend in our codebase that stores reflogs differently.

Refactor these tests to either use git-reflog(1) or the ref-store test
helper. Note that the refactoring to use git-reflog(1) also requires us
to adapt our expectations in some cases where we previously verified the
exact on-disk log entries. This seems like an acceptable tradeoff though
to ensure that different backends have the same user-visible behaviour
as any user would typically use git-reflog(1) anyway to access the logs.
Any backend-specific verification of the written on-disk format should
be implemented in a separate, backend-specific test.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh | 17 +++++++++++------
 t/t3200-branch.sh     | 24 ++++++++++++------------
 2 files changed, 23 insertions(+), 18 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 51a8d0bba98..b7d1e5deede 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -90,7 +90,8 @@ test_expect_success "deleting current branch adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-$m -d $m &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-$m$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-$m$" actual
 '
 
 test_expect_success "deleting by HEAD adds message to HEAD's log" '
@@ -99,7 +100,8 @@ test_expect_success "deleting by HEAD adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-by-head -d HEAD &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-by-head$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-by-head$" actual
 '
 
 test_expect_success 'update-ref does not create reflogs by default' '
@@ -130,7 +132,7 @@ test_expect_success 'creates no reflog in bare repository' '
 
 test_expect_success 'core.logAllRefUpdates=true creates reflog in bare repository' '
 	test_when_finished "git -C $bare config --unset core.logAllRefUpdates && \
-		rm $bare/logs/$m" &&
+		test-tool ref-store main delete-reflog $m" &&
 	git -C $bare config core.logAllRefUpdates true &&
 	git -C $bare update-ref $m $bareB &&
 	git -C $bare rev-parse $bareB >expect &&
@@ -263,7 +265,10 @@ test_expect_success "(not) changed .git/$m" '
 	! test $B = $(git show-ref -s --verify $m)
 '
 
-rm -f .git/logs/refs/heads/main
+test_expect_success "clean up reflog" '
+	test-tool ref-store main delete-reflog $m
+'
+
 test_expect_success "create $m (logged by touch)" '
 	test_config core.logAllRefUpdates false &&
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
@@ -316,7 +321,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150260 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000
 EOF
 test_expect_success "verifying $m's log (logged by touch)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
@@ -346,7 +351,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 +0000
 EOF
 test_expect_success "verifying $m's log (logged by config)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 7d9393f190d..a38601dbbb7 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -73,14 +73,14 @@ test_expect_success 'git branch HEAD should fail' '
 '
 
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/d/e/f@{0}: branch: Created from main
 EOF
 test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
 	test_path_is_file .git/refs/heads/d/e/f &&
-	test_path_is_file .git/logs/refs/heads/d/e/f &&
-	test_cmp expect .git/logs/refs/heads/d/e/f
+	git reflog show --no-abbrev-commit refs/heads/d/e/f >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
@@ -200,10 +200,9 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	test $(git rev-parse --abbrev-ref HEAD) = bam
 '
 
-test_expect_success 'git branch -M baz bam should add entries to .git/logs/HEAD' '
-	msg="Branch: renamed refs/heads/baz to refs/heads/bam" &&
-	grep " $ZERO_OID.*$msg$" .git/logs/HEAD &&
-	grep "^$ZERO_OID.*$msg$" .git/logs/HEAD
+test_expect_success 'git branch -M baz bam should add entries to HEAD reflog' '
+	git reflog show HEAD >actual &&
+	grep "HEAD@{0}: Branch: renamed refs/heads/baz to refs/heads/bam" actual
 '
 
 test_expect_success 'git branch -M should leave orphaned HEAD alone' '
@@ -225,7 +224,7 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 test_expect_success 'resulting reflog can be shown by log -g' '
 	oid=$(git rev-parse HEAD) &&
 	cat >expect <<-EOF &&
-	HEAD@{0} $oid $msg
+	HEAD@{0} $oid Branch: renamed refs/heads/baz to refs/heads/bam
 	HEAD@{2} $oid checkout: moving from foo to baz
 	EOF
 	git log -g --format="%gd %H %gs" -2 HEAD >actual &&
@@ -699,7 +698,8 @@ test_expect_success 'git branch -C c1 c2 should succeed when c1 is checked out'
 
 test_expect_success 'git branch -C c1 c2 should never touch HEAD' '
 	msg="Branch: copied refs/heads/c1 to refs/heads/c2" &&
-	! grep "$msg$" .git/logs/HEAD
+	git reflog HEAD >actual &&
+	! grep "$msg$" actual
 '
 
 test_expect_success 'git branch -C main should work when main is checked out' '
@@ -1140,14 +1140,14 @@ test_expect_success '--set-upstream-to notices an error to set branch as own ups
 
 # Keep this test last, as it changes the current branch
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/g/h/i@{0}: branch: Created from main
 EOF
 test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git checkout -b g/h/i -l main &&
 	test_path_is_file .git/refs/heads/g/h/i &&
-	test_path_is_file .git/logs/refs/heads/g/h/i &&
-	test_cmp expect .git/logs/refs/heads/g/h/i
+	git reflog show --no-abbrev-commit refs/heads/g/h/i >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'checkout -b makes reflog by default' '
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 5/9] t1450: convert tests to remove worktrees via git-worktree(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2516 bytes --]

Some of our tests in t1450 create worktrees and then corrupt them.
As it is impossible to delete such worktrees via a normal call to `git
worktree remove`, we instead opt to remove them manually by calling
rm(1) instead.

This is ultimately unnecessary though as we can use the `-f` switch to
remove the worktree. Let's convert the tests to do so such that we don't
have to reach into internal implementation details of worktrees.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1450-fsck.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index a3c97b9c7fc..a6af550867c 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -141,7 +141,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
-	test_when_finished "rm -rf .git/worktrees wt" &&
+	test_when_finished "git worktree remove -f wt" &&
 	git worktree add wt &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
@@ -150,7 +150,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
 	test_must_fail git fsck 2>out &&
@@ -158,7 +158,7 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at missing object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
 	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
@@ -167,7 +167,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 6/9] t4207: delete replace references via git-update-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1784 bytes --]

In t4207 we set up a set of replace objects via git-replace(1). Because
these references should not be impacting subsequent tests we also set up
some cleanup logic that deletes the replacement references via a call to
`rm -rf`. This reaches into the internal implementation details of the
reference backend and will thus break when we grow an alternative refdb
implementation.

Refactor the tests to delete the replacement refs via Git commands so
that we become independent of the actual refdb that's in use. As we
don't have a nice way to delete all replacements or all references in a
certain namespace, we opt for a combination of git-for-each-ref(1) and
git-update-ref(1)'s `--stdin` mode.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t4207-log-decoration-colors.sh | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
index 21986a866df..73ea9e51550 100755
--- a/t/t4207-log-decoration-colors.sh
+++ b/t/t4207-log-decoration-colors.sh
@@ -70,8 +70,14 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
 	cmp_filtered_decorations
 '
 
+remove_replace_refs () {
+	git for-each-ref 'refs/replace*/**' --format='delete %(refname)' >in &&
+	git update-ref --stdin <in &&
+	rm in
+}
+
 test_expect_success 'test coloring with replace-objects' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished remove_replace_refs &&
 	test_commit C &&
 	test_commit D &&
 
@@ -99,7 +105,7 @@ EOF
 '
 
 test_expect_success 'test coloring with grafted commit' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished remove_replace_refs &&
 
 	git replace --graft HEAD HEAD~2 &&
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 7/9] t7300: assert exact states of repo

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]

Some of the tests in t7300 verify that git-clean(1) doesn't touch
repositories that are embedded into the main repository. This is done by
asserting a small set of substructures that are assumed to always exist,
like the "refs/", "objects/" or "HEAD". This has the downside that we
need to assume a specific repository structure that may be subject to
change when new backends for the refdb land. At the same time, we don't
thoroughly assert that git-clean(1) really didn't end up cleaning any
files in the repository either.

Convert the tests to instead assert that all files continue to exist
after git-clean(1) by comparing a file listing via find(1) before and
after executing clean. This makes our actual assertions stricter while
having to care less about the repository's actual on-disk format.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7300-clean.sh | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/t/t7300-clean.sh b/t/t7300-clean.sh
index 0ef7b784573..d7d9202f37f 100755
--- a/t/t7300-clean.sh
+++ b/t/t7300-clean.sh
@@ -517,8 +517,12 @@ test_expect_success 'nested (empty) git should be kept' '
 	git init empty_repo &&
 	mkdir to_clean &&
 	>to_clean/should_clean.this &&
+	# Note that we put the expect file in the .git directory so that it
+	# does not get cleaned.
+	find empty_repo | sort >.git/expect &&
 	git clean -f -d &&
-	test_path_is_file empty_repo/.git/HEAD &&
+	find empty_repo | sort >actual &&
+	test_cmp .git/expect actual &&
 	test_path_is_missing to_clean
 '
 
@@ -559,10 +563,10 @@ test_expect_success 'giving path in nested git work tree will NOT remove it' '
 		mkdir -p bar/baz &&
 		test_commit msg bar/baz/hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/bar/baz &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/bar/ &&
-	test_path_is_file repo/bar/baz/hello.world
+	find repo | sort >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'giving path to nested .git will not remove it' '
@@ -573,10 +577,10 @@ test_expect_success 'giving path to nested .git will not remove it' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/.git/refs &&
-	test_path_is_dir repo/.git/objects &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
@@ -588,9 +592,10 @@ test_expect_success 'giving path to nested .git/ will NOT remove contents' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git/ &&
-	test_path_is_dir repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 8/9] t7900: assert the absence of refs via git-for-each-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

We're asserting that a prefetch of remotes via git-maintenance(1)
doesn't write any references in refs/remotes by validating that the
directory ".git/refs/remotes" is missing. This is quite roundabout: we
don't care about the directory existing, we care about the references
not existing, and the way these are stored is on the behest of the
reference database.

Convert the test to instead check via git-for-each-ref(1) whether any
remote reference exist.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7900-maintenance.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index e56f5980dc4..cefecee732f 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -157,7 +157,8 @@ test_expect_success 'prefetch multiple remotes' '
 	fetchargs="--prefetch --prune --no-tags --no-write-fetch-head --recurse-submodules=no --quiet" &&
 	test_subcommand git fetch remote1 $fetchargs <run-prefetch.txt &&
 	test_subcommand git fetch remote2 $fetchargs <run-prefetch.txt &&
-	test_path_is_missing .git/refs/remotes &&
+	git for-each-ref refs/remotes >actual &&
+	test_must_be_empty actual &&
 	git log prefetch/remotes/remote1/one &&
 	git log prefetch/remotes/remote2/two &&
 	git fetch --all &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v2 9/9] t: mark several tests that assume the files backend with REFFILES

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 8927 bytes --]

Add the REFFILES prerequisite to several tests that assume we're using
the files backend. There are various reasons why we cannot easily
convert those tests to be backend-independent, where the most common
one is that we have no way to write corrupt references into the refdb
via our tooling. We may at a later point in time grow the tooling to
make this possible, but for now we just mark these tests as requiring
the files backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh            |  8 ++++----
 t/t1450-fsck.sh                  |  6 +++---
 t/t2011-checkout-invalid-head.sh | 16 ++++++++--------
 t/t3200-branch.sh                |  8 ++++----
 t/t3400-rebase.sh                |  2 +-
 t/t5605-clone-local.sh           |  2 +-
 6 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index b7d1e5deede..70a760ba378 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -236,7 +236,7 @@ test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
 
-test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+test_expect_success REFFILES 'update-ref --no-deref -d can delete reference to bad ref' '
 	>.git/refs/heads/bad &&
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
@@ -288,7 +288,7 @@ test_expect_success "set $m (logged by touch)" '
 	test $A = $(git show-ref -s --verify $m)
 '
 
-test_expect_success 'empty directory removal' '
+test_expect_success REFFILES 'empty directory removal' '
 	git branch d1/d2/r1 HEAD &&
 	git branch d1/r2 HEAD &&
 	test_path_is_file .git/refs/heads/d1/d2/r1 &&
@@ -300,7 +300,7 @@ test_expect_success 'empty directory removal' '
 	test_path_is_file .git/logs/refs/heads/d1/r2
 '
 
-test_expect_success 'symref empty directory removal' '
+test_expect_success REFFILES 'symref empty directory removal' '
 	git branch e1/e2/r1 HEAD &&
 	git branch e1/r2 HEAD &&
 	git checkout e1/e2/r1 &&
@@ -1639,7 +1639,7 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
-test_expect_success 'directory not created deleting packed ref' '
+test_expect_success REFFILES 'directory not created deleting packed ref' '
 	git branch d1/d2/r1 HEAD &&
 	git pack-refs --all &&
 	test_path_is_missing .git/refs/heads/d1/d2 &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index a6af550867c..50b15bd7fc0 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -123,7 +123,7 @@ test_expect_success 'branch pointing to non-commit' '
 	test_i18ngrep "not a commit" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
@@ -139,7 +139,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 	test_i18ngrep "HEAD points to something strange" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object (from different wt)' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
 	test_when_finished "git worktree remove -f wt" &&
 	git worktree add wt &&
@@ -149,7 +149,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	test_i18ngrep "main-worktree/HEAD: detached HEAD points" out
 '
 
-test_expect_success 'other worktree HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'other worktree HEAD link pointing at a funny object' '
 	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
diff --git a/t/t2011-checkout-invalid-head.sh b/t/t2011-checkout-invalid-head.sh
index d9997e7b6b4..3c8135831b8 100755
--- a/t/t2011-checkout-invalid-head.sh
+++ b/t/t2011-checkout-invalid-head.sh
@@ -18,18 +18,18 @@ test_expect_success 'checkout should not start branch from a tree' '
 	test_must_fail git checkout -b newbranch main^{tree}
 '
 
-test_expect_success 'checkout main from invalid HEAD' '
+test_expect_success REFFILES 'checkout main from invalid HEAD' '
 	echo $ZERO_OID >.git/HEAD &&
 	git checkout main --
 '
 
-test_expect_success 'checkout notices failure to lock HEAD' '
+test_expect_success REFFILES 'checkout notices failure to lock HEAD' '
 	test_when_finished "rm -f .git/HEAD.lock" &&
 	>.git/HEAD.lock &&
 	test_must_fail git checkout -b other
 '
 
-test_expect_success 'create ref directory/file conflict scenario' '
+test_expect_success REFFILES 'create ref directory/file conflict scenario' '
 	git update-ref refs/heads/outer/inner main &&
 
 	# do not rely on symbolic-ref to get a known state,
@@ -39,26 +39,26 @@ test_expect_success 'create ref directory/file conflict scenario' '
 	}
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
 
-test_expect_success 'pack refs' '
+test_expect_success REFFILES 'pack refs' '
 	git pack-refs --all --prune
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index a38601dbbb7..8d82f9bc52a 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -28,7 +28,7 @@ test_expect_success 'git branch --help should not have created a bogus branch' '
 	test_path_is_missing .git/refs/heads/--help
 '
 
-test_expect_success 'branch -h in broken repository' '
+test_expect_success REFFILES 'branch -h in broken repository' '
 	mkdir broken &&
 	(
 		cd broken &&
@@ -242,7 +242,7 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	git worktree prune
 '
 
-test_expect_success 'git branch -M fails if updating any linked working tree fails' '
+test_expect_success REFFILES 'git branch -M fails if updating any linked working tree fails' '
 	git worktree add -b baz bazdir1 &&
 	git worktree add -f bazdir2 baz &&
 	touch .git/worktrees/bazdir1/HEAD.lock &&
@@ -833,14 +833,14 @@ test_expect_success 'renaming a symref is not allowed' '
 	test_path_is_missing .git/refs/heads/new-topic
 '
 
-test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m u v should fail when the reflog for u is a symlink' '
 	git branch --create-reflog u &&
 	mv .git/logs/refs/heads/u real-u &&
 	ln -s real-u .git/logs/refs/heads/u &&
 	test_must_fail git branch -m u v
 '
 
-test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m with symlinked .git/refs' '
 	test_when_finished "rm -rf subdir" &&
 	git init --bare subdir &&
 
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index d3df19a51f8..435943a0891 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -424,7 +424,7 @@ test_expect_success 'refuse to switch to branch checked out elsewhere' '
 	test_i18ngrep "already used by worktree at" err
 '
 
-test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
+test_expect_success REFFILES,MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
 	git checkout main &&
 	mv .git/logs actual_logs &&
 	cmd //c "mklink /D .git\logs ..\actual_logs" &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index a26689de379..8ad6445eca7 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -157,7 +157,7 @@ test_expect_success 'cloning locally respects "-u" for fetching refs' '
 	test_must_fail git clone --bare -u false a should_not_work.git
 '
 
-test_expect_success 'local clone from repo with corrupt refs fails gracefully' '
+test_expect_success REFFILES 'local clone from repo with corrupt refs fails gracefully' '
 	git init corrupt &&
 	test_commit -C corrupt one &&
 	echo a >corrupt/.git/refs/heads/topic &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 5/9] t1450: convert tests to remove worktrees via git-worktree(1)

[Eric Sunshine]

On Tue, Oct 24, 2023 at 10:05 AM Patrick Steinhardt <ps@pks.im> wrote:
> Some of our tests in t1450 create worktrees and then corrupt them.
> As it is impossible to delete such worktrees via a normal call to `git
> worktree remove`, we instead opt to remove them manually by calling
> rm(1) instead.
>
> This is ultimately unnecessary though as we can use the `-f` switch to
> remove the worktree. Let's convert the tests to do so such that we don't
> have to reach into internal implementation details of worktrees.
>
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
> @@ -141,7 +141,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
>  test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
>         test_when_finished "git update-ref HEAD $orig_head" &&
> -       test_when_finished "rm -rf .git/worktrees wt" &&
> +       test_when_finished "git worktree remove -f wt" &&
>         git worktree add wt &&
>         echo $ZERO_OID >.git/HEAD &&

Technically, this is a change of behavior since the original code
removed the entire .git/worktrees directory, which deleted
administrative metainformation for _all_ worktrees, whereas the new
code only deletes administrative metadata for the mentioned worktree.
However, since there are no other existing worktrees at this point in
any of these tests, the result is functionally the same, so the change
of behavior is immaterial. Moreover, the revised code has a smaller
blast-radius, which may be a desirable property since there has been
some movement toward making tests more self-contained so that they can
be run individually more easily.

[PATCH v3 0/9] t: reduce direct disk access to data structures

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 8559 bytes --]

Hi,

this is the third version of my patch series that aims to reduce access
to on-disk data structures in favor of using plumbing tools where
possible.

There are no changes compared to v2 except for a rebase on top of
0497e6c611 (t: use git-show-ref(1) to check for ref existence,
2023-10-31), which has been merged to "next" now and conflicts with this
patch series.

Patrick

Patrick Steinhardt (9):
  t: allow skipping expected object ID in `ref-store update-ref`
  t: convert tests to not write references via the filesystem
  t: convert tests to not access symrefs via the filesystem
  t: convert tests to not access reflog via the filesystem
  t1450: convert tests to remove worktrees via git-worktree(1)
  t4207: delete replace references via git-update-ref(1)
  t7300: assert exact states of repo
  t7900: assert the absence of refs via git-for-each-ref(1)
  t: mark several tests that assume the files backend with REFFILES

 t/helper/test-ref-store.c          | 11 ++++---
 t/t1400-update-ref.sh              | 50 ++++++++++++++++--------------
 t/t1430-bad-ref-name.sh            | 12 +++----
 t/t1450-fsck.sh                    | 44 +++++++++++++-------------
 t/t2011-checkout-invalid-head.sh   | 16 +++++-----
 t/t3200-branch.sh                  | 41 ++++++++++++------------
 t/t3400-rebase.sh                  |  2 +-
 t/t3404-rebase-interactive.sh      |  2 +-
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t4207-log-decoration-colors.sh   | 10 ++++--
 t/t5526-fetch-submodules.sh        |  2 +-
 t/t5605-clone-local.sh             |  4 +--
 t/t5702-protocol-v2.sh             | 24 ++++++++++----
 t/t7300-clean.sh                   | 23 ++++++++------
 t/t7900-maintenance.sh             |  3 +-
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 17 files changed, 142 insertions(+), 108 deletions(-)

Range-diff against v2:
 1:  c868198f8c1 =  1:  3caf9e3f28f t: allow skipping expected object ID in `ref-store update-ref`
 2:  4c0939d868e =  2:  1884b7006cf t: convert tests to not write references via the filesystem
 3:  048583ed2c3 !  3:  d6307177132 t: convert tests to not access symrefs via the filesystem
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref -d can delete broken na
     -	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    + 	test_ref_exists refs/heads/broken...symref &&
      	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
    - 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delete symref with broken name
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delet
     -	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    + 	test_ref_exists refs/heads/broken...symref &&
      	git branch -d broken...symref >output 2>error &&
    - 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'branch -d can delete symref with broken name' '
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'branch -d can delete symref with b
     -	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    + 	test_ref_exists refs/heads/broken...symref &&
      	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
    - 	test_path_is_missing .git/refs/heads/broken...symref &&
     @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
      '
      
    @@ t/t1430-bad-ref-name.sh: test_expect_success 'update-ref --no-deref -d can delet
     -	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
     +	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
      	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
    + 	test_ref_exists refs/heads/broken...symref &&
      	git branch -d broken...symref >output 2>error &&
    - 	test_path_is_missing .git/refs/heads/broken...symref &&
     
      ## t/t1450-fsck.sh ##
     @@ t/t1450-fsck.sh: test_expect_success 'HEAD link pointing at a funny object' '
    @@ t/t3200-branch.sh: test_expect_success 'git branch -M should leave orphaned HEAD
     +		git symbolic-ref HEAD >expect &&
     +		echo refs/heads/lonely >actual &&
     +		test_cmp expect actual &&
    - 		test_path_is_missing .git/refs/head/lonely &&
    + 		test_ref_missing refs/head/lonely &&
      		git branch -M main mistress &&
     -		grep lonely .git/HEAD
     +		git symbolic-ref HEAD >expect &&
    @@ t/t3200-branch.sh: test_expect_success 'deleting a symref' '
     +	git symbolic-ref --no-recurse refs/heads/dangling-symref &&
      	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
      	git branch -d dangling-symref >actual &&
    - 	test_path_is_missing .git/refs/heads/dangling-symref &&
    + 	test_ref_missing refs/heads/dangling-symref &&
     
      ## t/t4013-diff-various.sh ##
     @@ t/t4013-diff-various.sh: test_expect_success 'log -S requires an argument' '
 4:  5e7937e7904 !  4:  fdf67689227 t: convert tests to not access reflog via the filesystem
    @@ t/t3200-branch.sh: test_expect_success 'git branch HEAD should fail' '
      test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
      	GIT_COMMITTER_DATE="2005-05-26 23:30" \
      	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
    - 	test_path_is_file .git/refs/heads/d/e/f &&
    + 	test_ref_exists refs/heads/d/e/f &&
     -	test_path_is_file .git/logs/refs/heads/d/e/f &&
     -	test_cmp expect .git/logs/refs/heads/d/e/f
     +	git reflog show --no-abbrev-commit refs/heads/d/e/f >actual &&
    @@ t/t3200-branch.sh: test_expect_success '--set-upstream-to notices an error to se
      test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
      	GIT_COMMITTER_DATE="2005-05-26 23:30" \
      	git checkout -b g/h/i -l main &&
    - 	test_path_is_file .git/refs/heads/g/h/i &&
    + 	test_ref_exists refs/heads/g/h/i &&
     -	test_path_is_file .git/logs/refs/heads/g/h/i &&
     -	test_cmp expect .git/logs/refs/heads/g/h/i
     +	git reflog show --no-abbrev-commit refs/heads/g/h/i >actual &&
 5:  089565a358e =  5:  a5d55b68828 t1450: convert tests to remove worktrees via git-worktree(1)
 6:  cb738888ed7 =  6:  3213d45b537 t4207: delete replace references via git-update-ref(1)
 7:  e730e011de4 =  7:  36e79f266c2 t7300: assert exact states of repo
 8:  a1bdea52397 =  8:  4af0dc00165 t7900: assert the absence of refs via git-for-each-ref(1)
 9:  497e43ae5c3 !  9:  9fc039a1764 t: mark several tests that assume the files backend with REFFILES
    @@ t/t2011-checkout-invalid-head.sh: test_expect_success 'create ref directory/file
     
      ## t/t3200-branch.sh ##
     @@ t/t3200-branch.sh: test_expect_success 'git branch --help should not have created a bogus branch' '
    - 	test_path_is_missing .git/refs/heads/--help
    + 	test_ref_missing refs/heads/--help
      '
      
     -test_expect_success 'branch -h in broken repository' '
    @@ t/t3200-branch.sh: test_expect_success 'git branch -M baz bam should succeed whe
      	git worktree add -f bazdir2 baz &&
      	touch .git/worktrees/bazdir1/HEAD.lock &&
     @@ t/t3200-branch.sh: test_expect_success 'renaming a symref is not allowed' '
    - 	test_path_is_missing .git/refs/heads/new-topic
    + 	test_ref_missing refs/heads/new-topic
      '
      
     -test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '

base-commit: 0497e6c61172faabf412e013f2fc30a05054a46d
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 1/9] t: allow skipping expected object ID in `ref-store update-ref`

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1771 bytes --]

We require the caller to pass both the old and new expected object ID to
our `test-tool ref-store update-ref` helper. When trying to update a
symbolic reference though it's impossible to specify the expected object
ID, which means that the test would instead have to force-update the
reference. This is currently impossible though.

Update the helper to optionally skip verification of the old object ID
in case the test passes in an empty old object ID as input.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/helper/test-ref-store.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/t/helper/test-ref-store.c b/t/helper/test-ref-store.c
index 48552e6a9e0..702ec1f128a 100644
--- a/t/helper/test-ref-store.c
+++ b/t/helper/test-ref-store.c
@@ -298,16 +298,19 @@ static int cmd_update_ref(struct ref_store *refs, const char **argv)
 	const char *new_sha1_buf = notnull(*argv++, "new-sha1");
 	const char *old_sha1_buf = notnull(*argv++, "old-sha1");
 	unsigned int flags = arg_flags(*argv++, "flags", transaction_flags);
-	struct object_id old_oid;
+	struct object_id old_oid, *old_oid_ptr = NULL;
 	struct object_id new_oid;
 
-	if (get_oid_hex(old_sha1_buf, &old_oid))
-		die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+	if (*old_sha1_buf) {
+		if (get_oid_hex(old_sha1_buf, &old_oid))
+			die("cannot parse %s as %s", old_sha1_buf, the_hash_algo->name);
+		old_oid_ptr = &old_oid;
+	}
 	if (get_oid_hex(new_sha1_buf, &new_oid))
 		die("cannot parse %s as %s", new_sha1_buf, the_hash_algo->name);
 
 	return refs_update_ref(refs, msg, refname,
-			       &new_oid, &old_oid,
+			       &new_oid, old_oid_ptr,
 			       flags, UPDATE_REFS_DIE_ON_ERR);
 }
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 2/9] t: convert tests to not write references via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 9376 bytes --]

Some of our tests manually create, update or delete references by
writing the respective new values into the filesystem directly. While
this works with the current files reference backend, this will break
once we have a second reference backend implementation in our codebase.

Refactor these tests to instead use git-update-ref(1) or our `ref-store`
test tool. The latter is required in some cases where safety checks of
git-update-ref(1) would otherwise reject a reference update.

While at it, refactor some of the tests to schedule the cleanup command
via `test_when_finished` before modifying the repository.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh         | 17 ++++++++---------
 t/t1450-fsck.sh               | 26 +++++++++++++-------------
 t/t3404-rebase-interactive.sh |  2 +-
 t/t5526-fetch-submodules.sh   |  2 +-
 4 files changed, 23 insertions(+), 24 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 4d66cd7f4a1..91cc6dff724 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -9,8 +9,6 @@ test_description='Test git update-ref and basic ref logging'
 Z=$ZERO_OID
 
 m=refs/heads/main
-n_dir=refs/heads/gu
-n=$n_dir/fixes
 outside=refs/foo
 bare=bare-repo
 
@@ -62,10 +60,10 @@ test_expect_success "delete $m without oldvalue verification" '
 	test_must_fail git show-ref --verify -q $m
 '
 
-test_expect_success "fail to create $n" '
-	test_when_finished "rm -f .git/$n_dir" &&
-	touch .git/$n_dir &&
-	test_must_fail git update-ref $n $A
+test_expect_success "fail to create $n due to file/directory conflict" '
+	test_when_finished "git update-ref -d refs/heads/gu" &&
+	git update-ref refs/heads/gu $A &&
+	test_must_fail git update-ref refs/heads/gu/fixes $A
 '
 
 test_expect_success "create $m (by HEAD)" '
@@ -221,16 +219,16 @@ test_expect_success 'delete symref without dereference when the referred ref is
 '
 
 test_expect_success 'update-ref -d is not confused by self-reference' '
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
 	test_path_is_file .git/refs/heads/self
 '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
+	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_when_finished "rm -f .git/refs/heads/self" &&
 	test_path_is_file .git/refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
@@ -434,7 +432,8 @@ test_expect_success 'Query "main@{2005-05-28}" (past end of history)' '
 	test_i18ngrep -F "warning: log for ref $m unexpectedly ended on $ld" e
 '
 
-rm -f .git/$m .git/logs/$m expect
+rm -f expect
+git update-ref -d $m
 
 test_expect_success 'creating initial files' '
 	test_when_finished rm -f M &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 10a539158c4..f55b539b083 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -15,6 +15,7 @@ test_expect_success setup '
 	git config --unset i18n.commitencoding &&
 	git checkout HEAD^0 &&
 	test_commit B fileB two &&
+	orig_head=$(git rev-parse HEAD) &&
 	git tag -d A B &&
 	git reflog expire --expire=now --all
 '
@@ -115,15 +116,15 @@ test_expect_success 'zlib corrupt loose object output ' '
 '
 
 test_expect_success 'branch pointing to non-commit' '
-	git rev-parse HEAD^{tree} >.git/refs/heads/invalid &&
+	tree_oid=$(git rev-parse --verify HEAD^{tree}) &&
 	test_when_finished "git update-ref -d refs/heads/invalid" &&
+	test-tool ref-store main update-ref msg refs/heads/invalid $tree_oid $ZERO_OID REF_SKIP_OID_VERIFICATION &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "not a commit" out
 '
 
 test_expect_success 'HEAD link pointing at a funny object' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	test_when_finished "git update-ref HEAD $orig_head" &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -131,8 +132,7 @@ test_expect_success 'HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'HEAD link pointing at a funny place' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
-	mv .git/HEAD .git/SAVED_HEAD &&
+	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
 	echo "ref: refs/funny/place" >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
@@ -140,10 +140,9 @@ test_expect_success 'HEAD link pointing at a funny place' '
 '
 
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
-	test_when_finished "mv .git/SAVED_HEAD .git/HEAD" &&
+	test_when_finished "git update-ref HEAD $orig_head" &&
 	test_when_finished "rm -rf .git/worktrees wt" &&
 	git worktree add wt &&
-	mv .git/HEAD .git/SAVED_HEAD &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail git -C wt fsck 2>out &&
@@ -161,7 +160,8 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 test_expect_success 'other worktree HEAD link pointing at missing object' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "Contents missing from repo" | git hash-object --stdin >.git/worktrees/other/HEAD &&
+	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
+	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD: invalid sha1 pointer" out
 '
@@ -391,7 +391,7 @@ test_expect_success 'tag pointing to nonexistent' '
 
 	tag=$(git hash-object -t tag -w --stdin <invalid-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/invalid &&
+	git update-ref refs/tags/invalid $tag &&
 	test_when_finished "git update-ref -d refs/tags/invalid" &&
 	test_must_fail git fsck --tags >out &&
 	test_i18ngrep "broken link" out
@@ -411,7 +411,7 @@ test_expect_success 'tag pointing to something else than its type' '
 
 	tag=$(git hash-object -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags
 '
@@ -428,7 +428,7 @@ test_expect_success 'tag with incorrect tag name & missing tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	git fsck --tags 2>out &&
 
@@ -452,7 +452,7 @@ test_expect_success 'tag with bad tagger' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <wrong-tag) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag .*: invalid author/committer" out
@@ -471,7 +471,7 @@ test_expect_success 'tag with NUL in header' '
 
 	tag=$(git hash-object --literally -t tag -w --stdin <tag-NUL-header) &&
 	test_when_finished "remove_object $tag" &&
-	echo $tag >.git/refs/tags/wrong &&
+	git update-ref refs/tags/wrong $tag &&
 	test_when_finished "git update-ref -d refs/tags/wrong" &&
 	test_must_fail git fsck --tags 2>out &&
 	test_i18ngrep "error in tag $tag.*unterminated header: NUL at offset" out
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 8ea2bf13026..d2a7a91f170 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -2160,7 +2160,7 @@ test_expect_success '--update-refs: check failed ref update' '
 	# recorded in the update-refs file. We will force-update the
 	# "second" ref, but "git branch -f" will not work because of
 	# the lock in the update-refs file.
-	git rev-parse third >.git/refs/heads/second &&
+	git update-ref refs/heads/second third &&
 
 	test_must_fail git rebase --continue 2>err &&
 	grep "update_ref failed for ref '\''refs/heads/second'\''" err &&
diff --git a/t/t5526-fetch-submodules.sh b/t/t5526-fetch-submodules.sh
index 26e933f93ae..7ab220fa313 100755
--- a/t/t5526-fetch-submodules.sh
+++ b/t/t5526-fetch-submodules.sh
@@ -771,7 +771,7 @@ test_expect_success 'fetching submodule into a broken repository' '
 	git -C dst fetch --recurse-submodules &&
 
 	# Break the receiving submodule
-	rm -f dst/sub/.git/HEAD &&
+	test-tool -C dst/sub ref-store main delete-refs REF_NO_DEREF msg HEAD &&
 
 	# NOTE: without the fix the following tests will recurse forever!
 	# They should terminate with an error.
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 3/9] t: convert tests to not access symrefs via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 13670 bytes --]

Some of our tests access symbolic references via the filesystem
directly. While this works with the current files reference backend, it
this will break once we have a second reference backend in our codebase.

Refactor these tests to instead use git-symbolic-ref(1) or our
`ref-store` test tool. The latter is required in some cases where safety
checks of git-symbolic-ref(1) would otherwise reject writing a symbolic
reference.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh              |  8 ++++----
 t/t1430-bad-ref-name.sh            | 12 ++++++------
 t/t1450-fsck.sh                    |  4 ++--
 t/t3200-branch.sh                  |  9 ++++++---
 t/t4013-diff-various.sh            |  2 +-
 t/t4202-log.sh                     |  2 +-
 t/t5605-clone-local.sh             |  2 +-
 t/t5702-protocol-v2.sh             | 24 ++++++++++++++++++------
 t/t9133-git-svn-nested-git-repo.sh |  2 +-
 9 files changed, 40 insertions(+), 25 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 91cc6dff724..51a8d0bba98 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -221,15 +221,15 @@ test_expect_success 'delete symref without dereference when the referred ref is
 test_expect_success 'update-ref -d is not confused by self-reference' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	test_must_fail git update-ref -d refs/heads/self &&
-	test_path_is_file .git/refs/heads/self
+	git symbolic-ref --no-recurse refs/heads/self
 '
 
 test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF refs/heads/self" &&
 	git symbolic-ref refs/heads/self refs/heads/self &&
-	test_path_is_file .git/refs/heads/self &&
+	git symbolic-ref --no-recurse refs/heads/self &&
 	git update-ref --no-deref -d refs/heads/self &&
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
@@ -239,7 +239,7 @@ test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
 	test_when_finished "git update-ref -d refs/heads/ref-to-bad" &&
-	test_path_is_file .git/refs/heads/ref-to-bad &&
+	git symbolic-ref --no-recurse refs/heads/ref-to-bad &&
 	git update-ref --no-deref -d refs/heads/ref-to-bad &&
 	test_must_fail git show-ref --verify -q refs/heads/ref-to-bad
 '
diff --git a/t/t1430-bad-ref-name.sh b/t/t1430-bad-ref-name.sh
index 7b7d6953c62..5debb91f7b7 100755
--- a/t/t1430-bad-ref-name.sh
+++ b/t/t1430-bad-ref-name.sh
@@ -164,9 +164,9 @@ test_expect_success 'rev-parse skips symref pointing to broken name' '
 test_expect_success 'for-each-ref emits warnings for broken names' '
 	test-tool ref-store main update-ref msg "refs/heads/broken...ref" $main_sha1 $ZERO_OID REF_SKIP_REFNAME_VERIFICATION &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...ref" &&
-	printf "ref: refs/heads/broken...ref\n" >.git/refs/heads/badname &&
+	test-tool ref-store main create-symref refs/heads/badname refs/heads/broken...ref &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/badname" &&
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	git for-each-ref >output 2>error &&
 	! grep -e "broken\.\.\.ref" output &&
@@ -257,7 +257,7 @@ test_expect_success 'update-ref -d can delete broken name through symref' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
@@ -267,7 +267,7 @@ test_expect_success 'update-ref --no-deref -d can delete symref with broken name
 '
 
 test_expect_success 'branch -d can delete symref with broken name' '
-	printf "ref: refs/heads/main\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/main &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
@@ -277,7 +277,7 @@ test_expect_success 'branch -d can delete symref with broken name' '
 '
 
 test_expect_success 'update-ref --no-deref -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git update-ref --no-deref -d refs/heads/broken...symref >output 2>error &&
@@ -287,7 +287,7 @@ test_expect_success 'update-ref --no-deref -d can delete dangling symref with br
 '
 
 test_expect_success 'branch -d can delete dangling symref with broken name' '
-	printf "ref: refs/heads/idonotexist\n" >.git/refs/heads/broken...symref &&
+	test-tool ref-store main create-symref refs/heads/broken...symref refs/heads/idonotexist &&
 	test_when_finished "test-tool ref-store main delete-refs REF_NO_DEREF msg refs/heads/broken...symref" &&
 	test_ref_exists refs/heads/broken...symref &&
 	git branch -d broken...symref >output 2>error &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index f55b539b083..a3c97b9c7fc 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -133,7 +133,7 @@ test_expect_success 'HEAD link pointing at a funny object' '
 
 test_expect_success 'HEAD link pointing at a funny place' '
 	test_when_finished "git update-ref --no-deref HEAD $orig_head" &&
-	echo "ref: refs/funny/place" >.git/HEAD &&
+	test-tool ref-store main create-symref HEAD refs/funny/place &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
 	test_must_fail env GIT_DIR=.git git fsck 2>out &&
 	test_i18ngrep "HEAD points to something strange" out
@@ -169,7 +169,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
 	test_when_finished "rm -rf .git/worktrees other" &&
 	git worktree add other &&
-	echo "ref: refs/funny/place" >.git/worktrees/other/HEAD &&
+	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
 	test_i18ngrep "worktrees/other/HEAD points to something strange" out
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index bde4f1485b7..874520e3f10 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -215,10 +215,13 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 		cd orphan &&
 		test_commit initial &&
 		git checkout --orphan lonely &&
-		grep lonely .git/HEAD &&
+		git symbolic-ref HEAD >expect &&
+		echo refs/heads/lonely >actual &&
+		test_cmp expect actual &&
 		test_ref_missing refs/head/lonely &&
 		git branch -M main mistress &&
-		grep lonely .git/HEAD
+		git symbolic-ref HEAD >expect &&
+		test_cmp expect actual
 	)
 '
 
@@ -809,7 +812,7 @@ test_expect_success 'deleting a symref' '
 
 test_expect_success 'deleting a dangling symref' '
 	git symbolic-ref refs/heads/dangling-symref nowhere &&
-	test_path_is_file .git/refs/heads/dangling-symref &&
+	git symbolic-ref --no-recurse refs/heads/dangling-symref &&
 	echo "Deleted branch dangling-symref (was nowhere)." >expect &&
 	git branch -d dangling-symref >actual &&
 	test_ref_missing refs/heads/dangling-symref &&
diff --git a/t/t4013-diff-various.sh b/t/t4013-diff-various.sh
index 4b474808311..fc8a0588acc 100755
--- a/t/t4013-diff-various.sh
+++ b/t/t4013-diff-various.sh
@@ -522,7 +522,7 @@ test_expect_success 'log -S requires an argument' '
 '
 
 test_expect_success 'diff --cached on unborn branch' '
-	echo ref: refs/heads/unborn >.git/HEAD &&
+	git symbolic-ref HEAD refs/heads/unborn &&
 	git diff --cached >result &&
 	process_diffs result >actual &&
 	process_diffs "$TEST_DIRECTORY/t4013/diff.diff_--cached" >expected &&
diff --git a/t/t4202-log.sh b/t/t4202-log.sh
index af4a123cd22..57b298a4e22 100755
--- a/t/t4202-log.sh
+++ b/t/t4202-log.sh
@@ -2265,7 +2265,7 @@ test_expect_success REFFILES 'log diagnoses bogus HEAD hash' '
 
 test_expect_success REFFILES 'log diagnoses bogus HEAD symref' '
 	git init empty &&
-	echo "ref: refs/heads/invalid.lock" > empty/.git/HEAD &&
+	test-tool -C empty ref-store main create-symref HEAD refs/heads/invalid.lock &&
 	test_must_fail git -C empty log 2>stderr &&
 	test_i18ngrep broken stderr &&
 	test_must_fail git -C empty log --default totally-bogus 2>stderr &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index 946c5751885..bedd29d0550 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -65,7 +65,7 @@ test_expect_success 'Even without -l, local will make a hardlink' '
 '
 
 test_expect_success 'local clone of repo with nonexistent ref in HEAD' '
-	echo "ref: refs/heads/nonexistent" > a.git/HEAD &&
+	git -C a.git symbolic-ref HEAD refs/heads/nonexistent &&
 	git clone a d &&
 	(cd d &&
 	git fetch &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 6af5c2062fd..dcc4cd95fe7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -221,7 +221,9 @@ test_expect_success 'clone of empty repo propagates name of default branch' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/mydefaultbranch >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success '...but not if explicitly forbidden by config' '
@@ -234,7 +236,9 @@ test_expect_success '...but not if explicitly forbidden by config' '
 	GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME= \
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_empty_parent" file_empty_child &&
-	! grep "refs/heads/mydefaultbranch" file_empty_child/.git/HEAD
+	echo refs/heads/main >expect &&
+	git -C file_empty_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'bare clone propagates empty default branch' '
@@ -247,7 +251,9 @@ test_expect_success 'bare clone propagates empty default branch' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare \
 		"file://$(pwd)/file_empty_parent" file_empty_child.git &&
-	grep "refs/heads/mydefaultbranch" file_empty_child.git/HEAD
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_empty_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
@@ -265,7 +271,9 @@ test_expect_success 'clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	grep "warning: remote HEAD refers to nonexistent ref" stderr
 '
 
@@ -295,7 +303,9 @@ test_expect_success 'bare clone propagates unborn HEAD from non-empty repo' '
 	git -c init.defaultBranch=main -c protocol.version=2 \
 		clone --bare "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child.git 2>stderr &&
-	grep "refs/heads/mydefaultbranch" file_unborn_child.git/HEAD &&
+	echo "refs/heads/mydefaultbranch" >expect &&
+	git -C file_unborn_child.git symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	! grep "warning:" stderr
 '
 
@@ -315,7 +325,9 @@ test_expect_success 'defaulted HEAD uses remote branch if available' '
 	git -c init.defaultBranch=branchwithstuff -c protocol.version=2 \
 		clone "file://$(pwd)/file_unborn_parent" \
 		file_unborn_child 2>stderr &&
-	grep "refs/heads/branchwithstuff" file_unborn_child/.git/HEAD &&
+	echo "refs/heads/branchwithstuff" >expect &&
+	git -C file_unborn_child symbolic-ref HEAD >actual &&
+	test_cmp expect actual &&
 	test_path_is_file file_unborn_child/stuff.t &&
 	! grep "warning:" stderr
 '
diff --git a/t/t9133-git-svn-nested-git-repo.sh b/t/t9133-git-svn-nested-git-repo.sh
index d8d536269cf..8ca24670acb 100755
--- a/t/t9133-git-svn-nested-git-repo.sh
+++ b/t/t9133-git-svn-nested-git-repo.sh
@@ -11,7 +11,7 @@ test_expect_success 'setup repo with a git repo inside it' '
 	(
 		cd s &&
 		git init &&
-		test -f .git/HEAD &&
+		git symbolic-ref HEAD &&
 		> .git/a &&
 		echo a > a &&
 		svn_cmd add .git a &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 4/9] t: convert tests to not access reflog via the filesystem

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 7125 bytes --]

Some of our tests reach directly into the filesystem in order to both
read or modify the reflog, which will break once we have a second
reference backend in our codebase that stores reflogs differently.

Refactor these tests to either use git-reflog(1) or the ref-store test
helper. Note that the refactoring to use git-reflog(1) also requires us
to adapt our expectations in some cases where we previously verified the
exact on-disk log entries. This seems like an acceptable tradeoff though
to ensure that different backends have the same user-visible behaviour
as any user would typically use git-reflog(1) anyway to access the logs.
Any backend-specific verification of the written on-disk format should
be implemented in a separate, backend-specific test.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh | 17 +++++++++++------
 t/t3200-branch.sh     | 24 ++++++++++++------------
 2 files changed, 23 insertions(+), 18 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index 51a8d0bba98..b7d1e5deede 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -90,7 +90,8 @@ test_expect_success "deleting current branch adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-$m -d $m &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-$m$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-$m$" actual
 '
 
 test_expect_success "deleting by HEAD adds message to HEAD's log" '
@@ -99,7 +100,8 @@ test_expect_success "deleting by HEAD adds message to HEAD's log" '
 	git symbolic-ref HEAD $m &&
 	git update-ref -m delete-by-head -d HEAD &&
 	test_must_fail git show-ref --verify -q $m &&
-	grep "delete-by-head$" .git/logs/HEAD
+	test-tool ref-store main for-each-reflog-ent HEAD >actual &&
+	grep "delete-by-head$" actual
 '
 
 test_expect_success 'update-ref does not create reflogs by default' '
@@ -130,7 +132,7 @@ test_expect_success 'creates no reflog in bare repository' '
 
 test_expect_success 'core.logAllRefUpdates=true creates reflog in bare repository' '
 	test_when_finished "git -C $bare config --unset core.logAllRefUpdates && \
-		rm $bare/logs/$m" &&
+		test-tool ref-store main delete-reflog $m" &&
 	git -C $bare config core.logAllRefUpdates true &&
 	git -C $bare update-ref $m $bareB &&
 	git -C $bare rev-parse $bareB >expect &&
@@ -263,7 +265,10 @@ test_expect_success "(not) changed .git/$m" '
 	! test $B = $(git show-ref -s --verify $m)
 '
 
-rm -f .git/logs/refs/heads/main
+test_expect_success "clean up reflog" '
+	test-tool ref-store main delete-reflog $m
+'
+
 test_expect_success "create $m (logged by touch)" '
 	test_config core.logAllRefUpdates false &&
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
@@ -316,7 +321,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150260 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150860 +0000
 EOF
 test_expect_success "verifying $m's log (logged by touch)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
@@ -346,7 +351,7 @@ $A $B $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150380 +0000	Switch
 $B $A $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150980 +0000
 EOF
 test_expect_success "verifying $m's log (logged by config)" '
-	test_when_finished "git update-ref -d $m && rm -rf .git/logs actual expect" &&
+	test_when_finished "git update-ref -d $m && git reflog expire --expire=all --all && rm -rf actual expect" &&
 	test-tool ref-store main for-each-reflog-ent $m >actual &&
 	test_cmp actual expect
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 874520e3f10..933aa9eebbd 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -76,14 +76,14 @@ test_expect_success 'git branch HEAD should fail' '
 '
 
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/d/e/f@{0}: branch: Created from main
 EOF
 test_expect_success 'git branch --create-reflog d/e/f should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git -c core.logallrefupdates=false branch --create-reflog d/e/f &&
 	test_ref_exists refs/heads/d/e/f &&
-	test_path_is_file .git/logs/refs/heads/d/e/f &&
-	test_cmp expect .git/logs/refs/heads/d/e/f
+	git reflog show --no-abbrev-commit refs/heads/d/e/f >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'git branch -d d/e/f should delete a branch and a log' '
@@ -203,10 +203,9 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	test $(git rev-parse --abbrev-ref HEAD) = bam
 '
 
-test_expect_success 'git branch -M baz bam should add entries to .git/logs/HEAD' '
-	msg="Branch: renamed refs/heads/baz to refs/heads/bam" &&
-	grep " $ZERO_OID.*$msg$" .git/logs/HEAD &&
-	grep "^$ZERO_OID.*$msg$" .git/logs/HEAD
+test_expect_success 'git branch -M baz bam should add entries to HEAD reflog' '
+	git reflog show HEAD >actual &&
+	grep "HEAD@{0}: Branch: renamed refs/heads/baz to refs/heads/bam" actual
 '
 
 test_expect_success 'git branch -M should leave orphaned HEAD alone' '
@@ -228,7 +227,7 @@ test_expect_success 'git branch -M should leave orphaned HEAD alone' '
 test_expect_success 'resulting reflog can be shown by log -g' '
 	oid=$(git rev-parse HEAD) &&
 	cat >expect <<-EOF &&
-	HEAD@{0} $oid $msg
+	HEAD@{0} $oid Branch: renamed refs/heads/baz to refs/heads/bam
 	HEAD@{2} $oid checkout: moving from foo to baz
 	EOF
 	git log -g --format="%gd %H %gs" -2 HEAD >actual &&
@@ -702,7 +701,8 @@ test_expect_success 'git branch -C c1 c2 should succeed when c1 is checked out'
 
 test_expect_success 'git branch -C c1 c2 should never touch HEAD' '
 	msg="Branch: copied refs/heads/c1 to refs/heads/c2" &&
-	! grep "$msg$" .git/logs/HEAD
+	git reflog HEAD >actual &&
+	! grep "$msg$" actual
 '
 
 test_expect_success 'git branch -C main should work when main is checked out' '
@@ -1143,14 +1143,14 @@ test_expect_success '--set-upstream-to notices an error to set branch as own ups
 
 # Keep this test last, as it changes the current branch
 cat >expect <<EOF
-$ZERO_OID $HEAD $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> 1117150200 +0000	branch: Created from main
+$HEAD refs/heads/g/h/i@{0}: branch: Created from main
 EOF
 test_expect_success 'git checkout -b g/h/i -l should create a branch and a log' '
 	GIT_COMMITTER_DATE="2005-05-26 23:30" \
 	git checkout -b g/h/i -l main &&
 	test_ref_exists refs/heads/g/h/i &&
-	test_path_is_file .git/logs/refs/heads/g/h/i &&
-	test_cmp expect .git/logs/refs/heads/g/h/i
+	git reflog show --no-abbrev-commit refs/heads/g/h/i >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'checkout -b makes reflog by default' '
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 5/9] t1450: convert tests to remove worktrees via git-worktree(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2516 bytes --]

Some of our tests in t1450 create worktrees and then corrupt them.
As it is impossible to delete such worktrees via a normal call to `git
worktree remove`, we instead opt to remove them manually by calling
rm(1) instead.

This is ultimately unnecessary though as we can use the `-f` switch to
remove the worktree. Let's convert the tests to do so such that we don't
have to reach into internal implementation details of worktrees.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1450-fsck.sh | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index a3c97b9c7fc..a6af550867c 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -141,7 +141,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 
 test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
-	test_when_finished "rm -rf .git/worktrees wt" &&
+	test_when_finished "git worktree remove -f wt" &&
 	git worktree add wt &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
@@ -150,7 +150,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
 	test_must_fail git fsck 2>out &&
@@ -158,7 +158,7 @@ test_expect_success 'other worktree HEAD link pointing at a funny object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at missing object' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	object_id=$(echo "Contents missing from repo" | git hash-object --stdin) &&
 	test-tool -C other ref-store main update-ref msg HEAD $object_id "" REF_NO_DEREF,REF_SKIP_OID_VERIFICATION &&
@@ -167,7 +167,7 @@ test_expect_success 'other worktree HEAD link pointing at missing object' '
 '
 
 test_expect_success 'other worktree HEAD link pointing at a funny place' '
-	test_when_finished "rm -rf .git/worktrees other" &&
+	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	git -C other symbolic-ref HEAD refs/funny/place &&
 	test_must_fail git fsck 2>out &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 6/9] t4207: delete replace references via git-update-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1784 bytes --]

In t4207 we set up a set of replace objects via git-replace(1). Because
these references should not be impacting subsequent tests we also set up
some cleanup logic that deletes the replacement references via a call to
`rm -rf`. This reaches into the internal implementation details of the
reference backend and will thus break when we grow an alternative refdb
implementation.

Refactor the tests to delete the replacement refs via Git commands so
that we become independent of the actual refdb that's in use. As we
don't have a nice way to delete all replacements or all references in a
certain namespace, we opt for a combination of git-for-each-ref(1) and
git-update-ref(1)'s `--stdin` mode.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t4207-log-decoration-colors.sh | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/t/t4207-log-decoration-colors.sh b/t/t4207-log-decoration-colors.sh
index 21986a866df..73ea9e51550 100755
--- a/t/t4207-log-decoration-colors.sh
+++ b/t/t4207-log-decoration-colors.sh
@@ -70,8 +70,14 @@ ${c_tag}tag: ${c_reset}${c_tag}A${c_reset}${c_commit})${c_reset} A
 	cmp_filtered_decorations
 '
 
+remove_replace_refs () {
+	git for-each-ref 'refs/replace*/**' --format='delete %(refname)' >in &&
+	git update-ref --stdin <in &&
+	rm in
+}
+
 test_expect_success 'test coloring with replace-objects' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished remove_replace_refs &&
 	test_commit C &&
 	test_commit D &&
 
@@ -99,7 +105,7 @@ EOF
 '
 
 test_expect_success 'test coloring with grafted commit' '
-	test_when_finished rm -rf .git/refs/replace* &&
+	test_when_finished remove_replace_refs &&
 
 	git replace --graft HEAD HEAD~2 &&
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 7/9] t7300: assert exact states of repo

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 2913 bytes --]

Some of the tests in t7300 verify that git-clean(1) doesn't touch
repositories that are embedded into the main repository. This is done by
asserting a small set of substructures that are assumed to always exist,
like the "refs/", "objects/" or "HEAD". This has the downside that we
need to assume a specific repository structure that may be subject to
change when new backends for the refdb land. At the same time, we don't
thoroughly assert that git-clean(1) really didn't end up cleaning any
files in the repository either.

Convert the tests to instead assert that all files continue to exist
after git-clean(1) by comparing a file listing via find(1) before and
after executing clean. This makes our actual assertions stricter while
having to care less about the repository's actual on-disk format.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7300-clean.sh | 23 ++++++++++++++---------
 1 file changed, 14 insertions(+), 9 deletions(-)

diff --git a/t/t7300-clean.sh b/t/t7300-clean.sh
index 0ef7b784573..d7d9202f37f 100755
--- a/t/t7300-clean.sh
+++ b/t/t7300-clean.sh
@@ -517,8 +517,12 @@ test_expect_success 'nested (empty) git should be kept' '
 	git init empty_repo &&
 	mkdir to_clean &&
 	>to_clean/should_clean.this &&
+	# Note that we put the expect file in the .git directory so that it
+	# does not get cleaned.
+	find empty_repo | sort >.git/expect &&
 	git clean -f -d &&
-	test_path_is_file empty_repo/.git/HEAD &&
+	find empty_repo | sort >actual &&
+	test_cmp .git/expect actual &&
 	test_path_is_missing to_clean
 '
 
@@ -559,10 +563,10 @@ test_expect_success 'giving path in nested git work tree will NOT remove it' '
 		mkdir -p bar/baz &&
 		test_commit msg bar/baz/hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/bar/baz &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/bar/ &&
-	test_path_is_file repo/bar/baz/hello.world
+	find repo | sort >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'giving path to nested .git will not remove it' '
@@ -573,10 +577,10 @@ test_expect_success 'giving path to nested .git will not remove it' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
-	test_path_is_dir repo/.git/refs &&
-	test_path_is_dir repo/.git/objects &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
@@ -588,9 +592,10 @@ test_expect_success 'giving path to nested .git/ will NOT remove contents' '
 		git init &&
 		test_commit msg hello.world
 	) &&
+	find repo | sort >expect &&
 	git clean -f -d repo/.git/ &&
-	test_path_is_dir repo/.git &&
-	test_path_is_file repo/.git/HEAD &&
+	find repo | sort >actual &&
+	test_cmp expect actual &&
 	test_path_is_dir untracked/
 '
 
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 8/9] t7900: assert the absence of refs via git-for-each-ref(1)

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1319 bytes --]

We're asserting that a prefetch of remotes via git-maintenance(1)
doesn't write any references in refs/remotes by validating that the
directory ".git/refs/remotes" is missing. This is quite roundabout: we
don't care about the directory existing, we care about the references
not existing, and the way these are stored is on the behest of the
reference database.

Convert the test to instead check via git-for-each-ref(1) whether any
remote reference exist.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t7900-maintenance.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t7900-maintenance.sh b/t/t7900-maintenance.sh
index e56f5980dc4..cefecee732f 100755
--- a/t/t7900-maintenance.sh
+++ b/t/t7900-maintenance.sh
@@ -157,7 +157,8 @@ test_expect_success 'prefetch multiple remotes' '
 	fetchargs="--prefetch --prune --no-tags --no-write-fetch-head --recurse-submodules=no --quiet" &&
 	test_subcommand git fetch remote1 $fetchargs <run-prefetch.txt &&
 	test_subcommand git fetch remote2 $fetchargs <run-prefetch.txt &&
-	test_path_is_missing .git/refs/remotes &&
+	git for-each-ref refs/remotes >actual &&
+	test_must_be_empty actual &&
 	git log prefetch/remotes/remote1/one &&
 	git log prefetch/remotes/remote2/two &&
 	git fetch --all &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[PATCH v3 9/9] t: mark several tests that assume the files backend with REFFILES

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 8909 bytes --]

Add the REFFILES prerequisite to several tests that assume we're using
the files backend. There are various reasons why we cannot easily
convert those tests to be backend-independent, where the most common
one is that we have no way to write corrupt references into the refdb
via our tooling. We may at a later point in time grow the tooling to
make this possible, but for now we just mark these tests as requiring
the files backend.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---
 t/t1400-update-ref.sh            |  8 ++++----
 t/t1450-fsck.sh                  |  6 +++---
 t/t2011-checkout-invalid-head.sh | 16 ++++++++--------
 t/t3200-branch.sh                |  8 ++++----
 t/t3400-rebase.sh                |  2 +-
 t/t5605-clone-local.sh           |  2 +-
 6 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index b7d1e5deede..70a760ba378 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -236,7 +236,7 @@ test_expect_success 'update-ref --no-deref -d can delete self-reference' '
 	test_must_fail git show-ref --verify -q refs/heads/self
 '
 
-test_expect_success 'update-ref --no-deref -d can delete reference to bad ref' '
+test_expect_success REFFILES 'update-ref --no-deref -d can delete reference to bad ref' '
 	>.git/refs/heads/bad &&
 	test_when_finished "rm -f .git/refs/heads/bad" &&
 	git symbolic-ref refs/heads/ref-to-bad refs/heads/bad &&
@@ -288,7 +288,7 @@ test_expect_success "set $m (logged by touch)" '
 	test $A = $(git show-ref -s --verify $m)
 '
 
-test_expect_success 'empty directory removal' '
+test_expect_success REFFILES 'empty directory removal' '
 	git branch d1/d2/r1 HEAD &&
 	git branch d1/r2 HEAD &&
 	test_path_is_file .git/refs/heads/d1/d2/r1 &&
@@ -300,7 +300,7 @@ test_expect_success 'empty directory removal' '
 	test_path_is_file .git/logs/refs/heads/d1/r2
 '
 
-test_expect_success 'symref empty directory removal' '
+test_expect_success REFFILES 'symref empty directory removal' '
 	git branch e1/e2/r1 HEAD &&
 	git branch e1/r2 HEAD &&
 	git checkout e1/e2/r1 &&
@@ -1639,7 +1639,7 @@ test_expect_success PIPE 'transaction flushes status updates' '
 	test_cmp expected actual
 '
 
-test_expect_success 'directory not created deleting packed ref' '
+test_expect_success REFFILES 'directory not created deleting packed ref' '
 	git branch d1/d2/r1 HEAD &&
 	git pack-refs --all &&
 	test_path_is_missing .git/refs/heads/d1/d2 &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index a6af550867c..50b15bd7fc0 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -123,7 +123,7 @@ test_expect_success 'branch pointing to non-commit' '
 	test_i18ngrep "not a commit" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
 	echo $ZERO_OID >.git/HEAD &&
 	# avoid corrupt/broken HEAD from interfering with repo discovery
@@ -139,7 +139,7 @@ test_expect_success 'HEAD link pointing at a funny place' '
 	test_i18ngrep "HEAD points to something strange" out
 '
 
-test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
+test_expect_success REFFILES 'HEAD link pointing at a funny object (from different wt)' '
 	test_when_finished "git update-ref HEAD $orig_head" &&
 	test_when_finished "git worktree remove -f wt" &&
 	git worktree add wt &&
@@ -149,7 +149,7 @@ test_expect_success 'HEAD link pointing at a funny object (from different wt)' '
 	test_i18ngrep "main-worktree/HEAD: detached HEAD points" out
 '
 
-test_expect_success 'other worktree HEAD link pointing at a funny object' '
+test_expect_success REFFILES 'other worktree HEAD link pointing at a funny object' '
 	test_when_finished "git worktree remove -f other" &&
 	git worktree add other &&
 	echo $ZERO_OID >.git/worktrees/other/HEAD &&
diff --git a/t/t2011-checkout-invalid-head.sh b/t/t2011-checkout-invalid-head.sh
index d9997e7b6b4..3c8135831b8 100755
--- a/t/t2011-checkout-invalid-head.sh
+++ b/t/t2011-checkout-invalid-head.sh
@@ -18,18 +18,18 @@ test_expect_success 'checkout should not start branch from a tree' '
 	test_must_fail git checkout -b newbranch main^{tree}
 '
 
-test_expect_success 'checkout main from invalid HEAD' '
+test_expect_success REFFILES 'checkout main from invalid HEAD' '
 	echo $ZERO_OID >.git/HEAD &&
 	git checkout main --
 '
 
-test_expect_success 'checkout notices failure to lock HEAD' '
+test_expect_success REFFILES 'checkout notices failure to lock HEAD' '
 	test_when_finished "rm -f .git/HEAD.lock" &&
 	>.git/HEAD.lock &&
 	test_must_fail git checkout -b other
 '
 
-test_expect_success 'create ref directory/file conflict scenario' '
+test_expect_success REFFILES 'create ref directory/file conflict scenario' '
 	git update-ref refs/heads/outer/inner main &&
 
 	# do not rely on symbolic-ref to get a known state,
@@ -39,26 +39,26 @@ test_expect_success 'create ref directory/file conflict scenario' '
 	}
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (unpacked, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (unpacked, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
 
-test_expect_success 'pack refs' '
+test_expect_success REFFILES 'pack refs' '
 	git pack-refs --all --prune
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to branch)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to branch)' '
 	reset_to_df &&
 	git checkout main
 '
 
-test_expect_success 'checkout away from d/f HEAD (packed, to detached)' '
+test_expect_success REFFILES 'checkout away from d/f HEAD (packed, to detached)' '
 	reset_to_df &&
 	git checkout --detach main
 '
diff --git a/t/t3200-branch.sh b/t/t3200-branch.sh
index 933aa9eebbd..606c50fe66c 100755
--- a/t/t3200-branch.sh
+++ b/t/t3200-branch.sh
@@ -28,7 +28,7 @@ test_expect_success 'git branch --help should not have created a bogus branch' '
 	test_ref_missing refs/heads/--help
 '
 
-test_expect_success 'branch -h in broken repository' '
+test_expect_success REFFILES 'branch -h in broken repository' '
 	mkdir broken &&
 	(
 		cd broken &&
@@ -245,7 +245,7 @@ test_expect_success 'git branch -M baz bam should succeed when baz is checked ou
 	git worktree prune
 '
 
-test_expect_success 'git branch -M fails if updating any linked working tree fails' '
+test_expect_success REFFILES 'git branch -M fails if updating any linked working tree fails' '
 	git worktree add -b baz bazdir1 &&
 	git worktree add -f bazdir2 baz &&
 	touch .git/worktrees/bazdir1/HEAD.lock &&
@@ -836,14 +836,14 @@ test_expect_success 'renaming a symref is not allowed' '
 	test_ref_missing refs/heads/new-topic
 '
 
-test_expect_success SYMLINKS 'git branch -m u v should fail when the reflog for u is a symlink' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m u v should fail when the reflog for u is a symlink' '
 	git branch --create-reflog u &&
 	mv .git/logs/refs/heads/u real-u &&
 	ln -s real-u .git/logs/refs/heads/u &&
 	test_must_fail git branch -m u v
 '
 
-test_expect_success SYMLINKS 'git branch -m with symlinked .git/refs' '
+test_expect_success SYMLINKS,REFFILES 'git branch -m with symlinked .git/refs' '
 	test_when_finished "rm -rf subdir" &&
 	git init --bare subdir &&
 
diff --git a/t/t3400-rebase.sh b/t/t3400-rebase.sh
index d3df19a51f8..435943a0891 100755
--- a/t/t3400-rebase.sh
+++ b/t/t3400-rebase.sh
@@ -424,7 +424,7 @@ test_expect_success 'refuse to switch to branch checked out elsewhere' '
 	test_i18ngrep "already used by worktree at" err
 '
 
-test_expect_success MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
+test_expect_success REFFILES,MINGW,SYMLINKS_WINDOWS 'rebase when .git/logs is a symlink' '
 	git checkout main &&
 	mv .git/logs actual_logs &&
 	cmd //c "mklink /D .git\logs ..\actual_logs" &&
diff --git a/t/t5605-clone-local.sh b/t/t5605-clone-local.sh
index bedd29d0550..a3055869bc7 100755
--- a/t/t5605-clone-local.sh
+++ b/t/t5605-clone-local.sh
@@ -157,7 +157,7 @@ test_expect_success 'cloning locally respects "-u" for fetching refs' '
 	test_must_fail git clone --bare -u false a should_not_work.git
 '
 
-test_expect_success 'local clone from repo with corrupt refs fails gracefully' '
+test_expect_success REFFILES 'local clone from repo with corrupt refs fails gracefully' '
 	git init corrupt &&
 	test_commit -C corrupt one &&
 	echo a >corrupt/.git/refs/heads/topic &&
-- 
2.42.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]