git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v5 00/39] SHA-256, part 3/3
@ 2020-07-28 23:34 brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 01/39] t: make test-bloom initialize repository brian m. carlson
                   ` (39 more replies)
  0 siblings, 40 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This is the final part required for the stage 4 implementation of
SHA-256.

Changes from v4:
* Fix several commit message to fix typos and inaccuracies.
* Remove useless touch statement in tests.
* Adjust several tests to introduce less dependence on previous test
  assertions other than the setup ones.
* Loosen the regexp for object IDs in a Perl script in a test.
* Ensure help functionality works for git http-fetch.
* Switch http-fetch.c to use the already existing struct argv_array
  instance.
* Avoid a memory leak, albeit a minor one.
* Improve error message for unknown bundle hash algorithm.
* Hoist a strbuf_rtrim in the bundle parsing code to simplify things.
* Specify a minimum version instead of a default version in the bundle
  code.
* Remove a useless line in tests.
* Avoid invoking git hash-object needlessly for the SHA1 prerequisite.
* Be explicit about which algorithms we're testing in CI.
* Properly capitalize "core.repositoryFormatVersion" in documentation.

Changes from v2:
* Move bundle signature parsing into a function.
* Improve parsing of bundle capabilities.
* Improve t5510 to select only the interesting lines with sed.
* Use cat and a heredoc instead of multiple echos.
* Use test_i18ngrep where appropriate.
* Reject extensions.objectFormat for repo v0.
* Update documentation accordingly.

Changes from v1:
* Add a patch to add a bundle v3 format with capabilities that include
  the hash algorithm.
* Remove bundle detection of hash algorithm using hash algorithm length.
* Fix typo in the extensions.objectFormat documentation.
* Document that extensions.objectFormat is ignored for repo v0.
* Use test instead of brackets.
* Use a command group instead of a subshell for efficiency.
* Drop the bizarre test_might_fail, since it isn't needed (and is
  bizarre and wrong).
* Write "SHA-256" instead of "NewHash" in commit messages.
* Split SHA-256 CI tests to an independent run and avoid running a third
  test run to a single CI job.

Range-diff below.

Johannes Schindelin (1):
  t3404: prepare 'short SHA-1 collision' tests for SHA-256

brian m. carlson (38):
  t: make test-bloom initialize repository
  t1001: use $ZERO_OID
  t3305: make hash agnostic
  t6100: make hash size independent
  t6101: make hash size independent
  t6301: make hash size independent
  t6500: specify test values for SHA-256
  t6501: avoid hard-coded objects
  t7003: compute appropriate length constant
  t7063: make hash size independent
  t7201: abstract away SHA-1-specific constants
  t7102: abstract away SHA-1-specific constants
  t7400: make hash size independent
  t7405: make hash size independent
  t7506: avoid checking for SHA-1-specific constants
  t7508: use $ZERO_OID instead of hard-coded constant
  t8002: make hash size independent
  t8003: make hash size independent
  t8011: make hash size independent
  t9300: abstract away SHA-1-specific constants
  t9300: use $ZERO_OID instead of hard-coded object ID
  t9301: make hash size independent
  t9350: make hash size independent
  t9500: ensure that algorithm info is preserved in config
  t9700: make hash size independent
  t5308: make test work with SHA-256
  t0410: mark test with SHA1 prerequisite
  http-fetch: set up git directory before parsing pack hashes
  builtin/verify-pack: implement an --object-format option
  bundle: add new version for use with SHA-256
  setup: add support for reading extensions.objectformat
  Enable SHA-256 support by default
  t: add test_oid option to select hash algorithm
  t: allow testing different hash algorithms via environment
  t: make SHA1 prerequisite depend on default hash
  ci: run tests with SHA-256
  docs: add documentation for extensions.objectFormat
  t: remove test_oid_init in tests

 Documentation/config.txt                  |   2 +
 Documentation/config/extensions.txt       |   8 ++
 Documentation/git-bundle.txt              |   9 +-
 Documentation/technical/bundle-format.txt |  30 ++++-
 builtin/bundle.c                          |   5 +-
 builtin/init-db.c                         |   5 -
 builtin/verify-pack.c                     |  23 ++--
 bundle.c                                  |  83 +++++++++---
 bundle.h                                  |   4 +-
 ci/run-build-and-tests.sh                 |   6 +
 config.mak.dev                            |   2 -
 http-fetch.c                              |   4 +-
 repository.c                              |   4 -
 setup.c                                   |  16 ++-
 t/helper/test-bloom.c                     |   2 +
 t/lib-pack.sh                             |  11 +-
 t/lib-submodule-update.sh                 |   1 -
 t/t0000-basic.sh                          |  15 ++-
 t/t0001-init.sh                           |  33 +++++
 t/t0410-partial-clone.sh                  |   4 +-
 t/t1006-cat-file.sh                       |   2 -
 t/t1050-large.sh                          |   1 -
 t/t1091-sparse-checkout-builtin.sh        |   4 +-
 t/t1410-reflog.sh                         |   1 -
 t/t1450-fsck.sh                           |   1 -
 t/t1500-rev-parse.sh                      |   1 -
 t/t3305-notes-fanout.sh                   |   2 +-
 t/t3308-notes-merge.sh                    |   1 -
 t/t3404-rebase-interactive.sh             |  49 +++++--
 t/t3600-rm.sh                             |   1 -
 t/t3800-mktag.sh                          |   1 -
 t/t4002-diff-basic.sh                     |   2 -
 t/t4027-diff-submodule.sh                 |   1 -
 t/t4134-apply-submodule.sh                |   1 -
 t/t4200-rerere.sh                         |   1 -
 t/t4211-line-log.sh                       |   1 -
 t/t5300-pack-object.sh                    |   3 +-
 t/t5302-pack-index.sh                     |   1 -
 t/t5308-pack-detect-duplicates.sh         |  20 +--
 t/t5313-pack-bounds-checks.sh             |   1 -
 t/t5318-commit-graph.sh                   |   3 +-
 t/t5319-multi-pack-index.sh               |   1 -
 t/t5324-split-commit-graph.sh             |   1 -
 t/t5504-fetch-receive-strict.sh           |   1 -
 t/t5510-fetch.sh                          |  16 ++-
 t/t5530-upload-pack-error.sh              |   1 -
 t/t5562-http-backend-content-length.sh    |   1 -
 t/t5607-clone-bundle.sh                   |  31 +++++
 t/t5702-protocol-v2.sh                    |   3 +-
 t/t5703-upload-pack-ref-in-want.sh        |   1 -
 t/t6006-rev-list-format.sh                |   1 -
 t/t6100-rev-list-in-order.sh              |   4 +-
 t/t6101-rev-parse-parents.sh              |   2 +-
 t/t6301-for-each-ref-errors.sh            |   2 +-
 t/t6500-gc.sh                             |  27 +++-
 t/t6501-freshen-objects.sh                |  14 +-
 t/t7003-filter-branch.sh                  |   3 +-
 t/t7063-status-untracked-cache.sh         | 150 ++++++++++++----------
 t/t7102-reset.sh                          |  93 ++++++++------
 t/t7201-co.sh                             |   6 +-
 t/t7400-submodule-basic.sh                |  26 ++--
 t/t7405-submodule-merge.sh                |   4 +-
 t/t7506-status-submodule.sh               |  12 +-
 t/t7508-status.sh                         |   2 +-
 t/t8002-blame.sh                          |  18 ++-
 t/t8003-blame-corner-cases.sh             |   3 +-
 t/t8011-blame-split-file.sh               |   2 +-
 t/t9300-fast-import.sh                    | 118 +++++++++--------
 t/t9301-fast-import-notes.sh              |  13 +-
 t/t9350-fast-export.sh                    |  15 +--
 t/t9500-gitweb-standalone-no-errors.sh    |  22 +++-
 t/t9700/test.pl                           |   6 +-
 t/test-lib-functions.sh                   |  16 ++-
 t/test-lib.sh                             |   9 +-
 74 files changed, 635 insertions(+), 354 deletions(-)
 create mode 100644 Documentation/config/extensions.txt

Diff-intervalle contre v4 :
 1:  2d720a6beb =  1:  fb53834bd3 t: make test-bloom initialize repository
 2:  530704736a =  2:  8170f55d65 t1001: use $ZERO_OID
 3:  33e7822d0e =  3:  4b14099f1a t3305: make hash agnostic
 4:  82a6827f45 !  4:  8d3ce7cccd t3404: prepare 'short SHA-1 collision' tests for SHA-256
    @@ Commit message
         commit message "collide2" to find such a collision. To make it easier to
         find such a collision, we reduce the number of digits to 4.
     
    -    The `t/oid-info/oid` file now lists the values found in that way, and
    -    the test cases are modified to use those values.
    -
         As the tests are no longer dependent on SHA-1, we also rename their
         titles to talk about "commit IDs" instead of "SHA-1s".
     
 5:  23cbd25060 =  5:  8329368075 t6100: make hash size independent
 6:  9e2553b1f2 =  6:  63595c30d4 t6101: make hash size independent
 7:  c2c5dd407e =  7:  88c7a501db t6301: make hash size independent
 8:  1ab4ed032a =  8:  4546001417 t6500: specify test values for SHA-256
 9:  54fcc025bb =  9:  4a9304a648 t6501: avoid hard-coded objects
10:  a2f420fcd4 = 10:  7785b7cc1f t7003: compute appropriate length constant
11:  766bba42f6 ! 11:  71db1716f5 t7063: make hash size independent
    @@ Metadata
      ## Commit message ##
         t7063: make hash size independent
     
    -    Use test_oid instead of hard-coding a fixed size all-zeros object ID.
    +    Use test_oid instead of hard-coding algorithm-specific constants and
    +    all-zero values.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
    @@ t/t7063-status-untracked-cache.sh: EOF
      two
      EOF
      	test_cmp ../expect-from-test-dump ../actual
    -@@ t/t7063-status-untracked-cache.sh: test_expect_success 'using --untracked-cache does not fail when core.untrackedCa
    - '
    - 
    - test_expect_success 'setting core.untrackedCache to keep' '
    -+	touch ../expect-from-test-dump &&
    - 	git config core.untrackedCache keep &&
    - 	git update-index --untracked-cache &&
    - 	test-tool dump-untracked-cache >../actual &&
     @@ t/t7063-status-untracked-cache.sh: test_expect_success '"status" after file replacement should be clean with UC=tru
      	test-tool dump-untracked-cache >../actual &&
      	grep -F "recurse valid" ../actual >../actual.grep &&
12:  66d242ccd9 ! 12:  79dcdc3496 t7201: abstract away SHA-1-specific constants
    @@ t/t7201-co.sh: test_expect_success 'switch to another branch while carrying a de
      	test_line_count = 1 messages &&
      	H=$(git rev-parse --verify HEAD) &&
      	M=$(git show-ref -s --verify refs/heads/master) &&
    -@@ t/t7201-co.sh: test_expect_success 'checkout to detach HEAD' '
    +@@ t/t7201-co.sh: test_expect_success 'checkout to detach HEAD (with advice declined)' '
    + 
    + test_expect_success 'checkout to detach HEAD' '
      	git config advice.detachedHead true &&
    ++	rev=$(git rev-parse --short renamer^) &&
      	git checkout -f renamer && git clean -f &&
      	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
     -	grep "HEAD is now at 7329388" messages &&
13:  090041ee54 = 13:  f89560df08 t7102: abstract away SHA-1-specific constants
14:  4a6991d6a1 = 14:  79800ba8a3 t7400: make hash size independent
15:  d39bbeb5a2 = 15:  07d287fe0f t7405: make hash size independent
16:  5b0d52ae0e = 16:  1ee37967f2 t7506: avoid checking for SHA-1-specific constants
17:  54d1706107 = 17:  ba3c997fd7 t7508: use $ZERO_OID instead of hard-coded constant
18:  f3f4239ce9 ! 18:  362e0a9440 t8002: make hash size independent
    @@ Metadata
      ## Commit message ##
         t8002: make hash size independent
     
    -    Compute the length of an object ID instead of of hard-coding 40-based
    +    Compute the length of an object ID instead of hard-coding 40-based
         values.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    @@ t/t8002-blame.sh: test_description='git blame'
      . "$TEST_DIRECTORY"/annotate-tests.sh
      
     +test_expect_success 'setup' '
    -+	test_oid_init
    ++	hexsz=$(test_oid hexsz)
     +'
     +
      test_expect_success 'blame untracked file in empty repo' '
    @@ t/t8002-blame.sh: test_expect_success 'blame --abbrev=<n> works' '
      test_expect_success 'blame -l aligns regular and boundary commits' '
     -	check_abbrev 40 -l HEAD &&
     -	check_abbrev 39 -l ^HEAD
    -+	hexsz=$(test_oid hexsz) &&
     +	check_abbrev $hexsz         -l HEAD &&
     +	check_abbrev $((hexsz - 1)) -l ^HEAD
      '
19:  719db3822e = 19:  eeeba30e75 t8003: make hash size independent
20:  803e5665e9 ! 20:  a71b7fb7f1 t8011: make hash size independent
    @@ t/t8011-blame-split-file.sh: test_expect_success 'setup simulated porcelain' '
      	my $field = shift;
      	while (<>) {
     -		if (/^[0-9a-f]{40} /) {
    -+		if (/^[0-9a-f]{40,64} /) {
    ++		if (/^[0-9a-f]{40,} /) {
      			flush();
      			$hash = $&;
      		} elsif (/^$field (.*)/) {
21:  d1c4351bc9 ! 21:  b9a4ac6547 t9300: abstract away SHA-1-specific constants
    @@ Commit message
         t9300: abstract away SHA-1-specific constants
     
         Adjust the test so that it computes variables for object IDs instead of
    -    using hard-coded hashes.
    +    using hard-coded hashes.  In addition, use cut to filter out the object
    +    IDs and verify only the information that we're really interested in.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9300-fast-import.sh ##
     @@ t/t9300-fast-import.sh: test_expect_success 'A: verify pack' '
    + '
      
      test_expect_success 'A: verify diff' '
    ++	copy=$(git rev-parse --verify master:file2) &&
      	cat >expect <<-EOF &&
     -	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
    -+	:000000 100755 0000000000000000000000000000000000000000 $(git rev-parse --verify master:file2) A	copy-of-file2
    ++	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
      	EOF
      	git diff-tree -M -r master verify--import-marks >actual &&
      	compare_diff_raw expect actual &&
22:  db74bf3c8c ! 22:  8d617381c0 t9300: use $ZERO_OID instead of hard-coded object ID
    @@ t/t9300-fast-import.sh: test_expect_success 'A: create pack from stdin' '
      	tag nested
      	mark :7
     @@ t/t9300-fast-import.sh: test_expect_success 'A: verify pack' '
    - 
      test_expect_success 'A: verify diff' '
    + 	copy=$(git rev-parse --verify master:file2) &&
      	cat >expect <<-EOF &&
    --	:000000 100755 0000000000000000000000000000000000000000 $(git rev-parse --verify master:file2) A	copy-of-file2
    -+	:000000 100755 $ZERO_OID $(git rev-parse --verify master:file2) A	copy-of-file2
    +-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
    ++	:000000 100755 $ZERO_OID $copy A	copy-of-file2
      	EOF
      	git diff-tree -M -r master verify--import-marks >actual &&
      	compare_diff_raw expect actual &&
23:  fa7daed2bf = 23:  d3ddbe3bcf t9301: make hash size independent
24:  84aef86ac0 = 24:  70bfe6c076 t9350: make hash size independent
25:  541108fc64 = 25:  c784e76075 t9500: ensure that algorithm info is preserved in config
26:  2de5d2da58 = 26:  00e82dc23e t9700: make hash size independent
27:  c916058c38 = 27:  2d03d136df t5308: make test work with SHA-256
28:  b0781430ef = 28:  d8941dd81d t0410: mark test with SHA1 prerequisite
29:  230a933e87 ! 29:  39eb2e82b5 http-fetch: set up git directory before parsing pack hashes
    @@ Commit message
         To ensure that we can parse packfile hashes of the right length, let's
         set up the git directory before we start parsing arguments.
     
    +    Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    +
      ## http-fetch.c ##
     @@ http-fetch.c: int cmd_main(int argc, const char **argv)
      	int packfile = 0;
      	struct object_id packfile_hash;
      
    -+	setup_git_directory();
    ++	setup_git_directory_gently(NULL);
     +
      	while (arg < argc && argv[arg][0] == '-') {
      		const char *p;
30:  d3dbe9e760 ! 30:  7c71a12f67 builtin/verify-pack: implement an --object-format option
    @@ builtin/verify-pack.c
      {
      	struct child_process index_pack = CHILD_PROCESS_INIT;
     -	const char *argv[] = {"index-pack", NULL, NULL, NULL };
    --	struct strbuf arg = STRBUF_INIT;
    -+	struct argv_array argv = ARGV_ARRAY_INIT;
    -+	struct strbuf arg = STRBUF_INIT, hash_arg = STRBUF_INIT;
    ++	struct argv_array *argv = &index_pack.args;
    + 	struct strbuf arg = STRBUF_INIT;
      	int verbose = flags & VERIFY_PACK_VERBOSE;
      	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
      	int err;
      
    -+	argv_array_push(&argv, "index-pack");
    ++	argv_array_push(argv, "index-pack");
     +
      	if (stat_only)
     -		argv[1] = "--verify-stat-only";
    -+		argv_array_push(&argv, "--verify-stat-only");
    ++		argv_array_push(argv, "--verify-stat-only");
      	else if (verbose)
     -		argv[1] = "--verify-stat";
    -+		argv_array_push(&argv, "--verify-stat");
    ++		argv_array_push(argv, "--verify-stat");
      	else
     -		argv[1] = "--verify";
    -+		argv_array_push(&argv, "--verify");
    ++		argv_array_push(argv, "--verify");
     +
    -+	if (hash_algo) {
    -+		strbuf_addf(&hash_arg, "--object-format=%s", hash_algo);
    -+		argv_array_push(&argv, hash_arg.buf);
    -+	}
    ++	if (hash_algo)
    ++		argv_array_pushf(argv, "--object-format=%s", hash_algo);
      
      	/*
      	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
    @@ builtin/verify-pack.c: static int verify_one_pack(const char *path, unsigned int
      	    !ends_with(arg.buf, ".pack"))
      		strbuf_addstr(&arg, ".pack");
     -	argv[2] = arg.buf;
    -+	argv_array_push(&argv, arg.buf);
    ++	argv_array_push(argv, arg.buf);
      
     -	index_pack.argv = argv;
    -+	index_pack.argv = argv.argv;
      	index_pack.git_cmd = 1;
      
      	err = run_command(&index_pack);
    -@@ builtin/verify-pack.c: static int verify_one_pack(const char *path, unsigned int flags)
    - 		}
    - 	}
    - 	strbuf_release(&arg);
    -+	argv_array_clear(&argv);
    - 
    - 	return err;
    - }
     @@ builtin/verify-pack.c: int cmd_verify_pack(int argc, const char **argv, const char *prefix)
      {
      	int err = 0;
31:  e3805101ce ! 31:  9a0345a1f6 bundle: add new version for use with SHA-256
    @@ bundle.c: static void add_to_ref_list(const struct object_id *oid, const char *n
     +	if (skip_prefix(capability, "object-format=", &arg)) {
     +		int algo = hash_algo_by_name(arg);
     +		if (algo == GIT_HASH_UNKNOWN)
    -+			return error(_("unable to detect hash algorithm"));
    ++			return error(_("unrecognized bundle hash algorithm: %s"), arg);
     +		header->hash_algo = &hash_algos[algo];
     +		return 0;
     +	}
    @@ bundle.c: static int parse_bundle_header(int fd, struct bundle_header *header,
      		int is_prereq = 0;
      		const char *p;
      
    -+		if (header->version == 3 && *buf.buf == '@') {
    -+			buf.buf[--buf.len] = '\0';
    -+			if (parse_capability(header, buf.buf + 1)) {
    -+				status = -1;
    -+				break;
    -+			}
    -+			continue;
    -+		}
    -+
    - 		if (*buf.buf == '-') {
    - 			is_prereq = 1;
    - 			strbuf_remove(&buf, 0, 1);
    - 		}
    +-		if (*buf.buf == '-') {
    +-			is_prereq = 1;
    +-			strbuf_remove(&buf, 0, 1);
    +-		}
      		strbuf_rtrim(&buf);
      
     -		if (!header->hash_algo) {
     -			header->hash_algo = detect_hash_algo(&buf);
     -			if (!header->hash_algo) {
     -				error(_("unknown hash algorithm length"));
    --				status = -1;
    --				break;
    --			}
    --		}
    --
    ++		if (header->version == 3 && *buf.buf == '@') {
    ++			if (parse_capability(header, buf.buf + 1)) {
    + 				status = -1;
    + 				break;
    + 			}
    ++			continue;
    ++		}
    ++
    ++		if (*buf.buf == '-') {
    ++			is_prereq = 1;
    ++			strbuf_remove(&buf, 0, 1);
    + 		}
    + 
      		/*
    - 		 * Tip lines have object name, SP, and refname.
    - 		 * Prerequisites have object name that is optionally
     @@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
      }
      
    @@ bundle.c: static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
      	int bundle_to_stdout;
      	int ref_count = 0;
      	struct rev_info revs;
    -+	int default_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
    ++	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
      
      	bundle_to_stdout = !strcmp(path, "-");
      	if (bundle_to_stdout)
    @@ bundle.c: int create_bundle(struct repository *r, const char *path,
     -	/* write signature */
     -	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
     +	if (version == -1)
    -+		version = default_version;
    ++		version = min_version;
     +
     +	if (version < 2 || version > 3) {
     +		die(_("unsupported bundle version %d"), version);
    -+	} else if (version < default_version) {
    ++	} else if (version < min_version) {
     +		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
     +	} else if (version == 2) {
     +		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
    @@ t/t5510-fetch.sh: test_expect_success 'create bundle 1' '
      '
      
      test_expect_success 'header of bundle looks right' '
    - 	head -n 4 "$D"/bundle1 &&
    +-	head -n 4 "$D"/bundle1 &&
     -	head -n 1 "$D"/bundle1 | grep "^#" &&
     -	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
     -	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
    @@ t/t5510-fetch.sh: test_expect_success 'create bundle 1' '
     +	cat >expect <<-EOF &&
     +	# v3 git bundle
     +	@object-format=$(test_oid algo)
    -+	-OID message
    -+	OID message
    ++	-OID updated by origin
    ++	OID refs/heads/master
     +
     +	EOF
    -+	sed -e "s/$OID_REGEX .*/OID message/g" -e "5q" "$D"/bundle1 >actual &&
    ++	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
     +	test_cmp expect actual
      '
      
32:  c941fcf63c = 32:  4b72b98ce7 setup: add support for reading extensions.objectformat
33:  7c6264ac79 ! 33:  73ef3ef66a Enable SHA-256 support by default
    @@ t/t0001-init.sh: test_expect_success 're-init from a linked worktree' '
     +'
     +
     +test_expect_success 'init honors --object-format' '
    -+	git init --object-format=sha256 explicit &&
    -+	git -C explicit rev-parse --show-object-format >actual &&
    ++	git init --object-format=sha1 explicit-sha1 &&
    ++	git -C explicit-sha1 rev-parse --show-object-format >actual &&
    ++	echo sha1 >expected &&
    ++	test_cmp expected actual &&
    ++	git init --object-format=sha256 explicit-sha256 &&
    ++	git -C explicit-sha256 rev-parse --show-object-format >actual &&
     +	echo sha256 >expected &&
     +	test_cmp expected actual
     +'
34:  2719ab94e6 = 34:  2512a1669c t: add test_oid option to select hash algorithm
35:  1c8e5041d1 = 35:  015636559b t: allow testing different hash algorithms via environment
36:  7213d59222 <  -:  ---------- t: make SHA1 prerequisite depend on default hash
 -:  ---------- > 36:  6f2dec51cd t: make SHA1 prerequisite depend on default hash
37:  caf6c026ba ! 37:  32536503f3 ci: run tests with SHA-256
    @@ Commit message
         run as fast as possible, so we wouldn't want to add a third run to the
         linux-gcc job.  To make sure we properly exercise the code, let's run
         the tests in the default mode (SHA-1) first and then run a second time
    -    with SHA-256.
    +    with SHA-256.  We explicitly specify SHA-1 for the first run so that if
    +    we change the default in the future, we make sure to test both cases.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
    @@ ci/run-build-and-tests.sh: linux-gcc)
      	make test
      	;;
     +linux-clang)
    ++	export GIT_TEST_DEFAULT_HASH=sha1
     +	make test
     +	export GIT_TEST_DEFAULT_HASH=sha256
     +	make test
38:  ec4e6ae166 ! 38:  cc5679d353 docs: add documentation for extensions.objectFormat
    @@ Documentation/config/extensions.txt (new)
     +extensions.objectFormat::
     +	Specify the hash algorithm to use.  The acceptable values are `sha1` and
     +	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
    -+	this key unless `core.repositoryformatversion` is 1.
    ++	this key unless `core.repositoryFormatVersion` is 1.
     ++
     +Note that this setting should only be set by linkgit:git-init[1] or
     +linkgit:git-clone[1].  Trying to change it after initialization will not
39:  3185c62cfc ! 39:  307b75c88e t: remove test_oid_init in tests
    @@ t/t7003-filter-branch.sh: test_description='git filter-branch'
      	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
      	test_commit --notick B &&
     
    - ## t/t8002-blame.sh ##
    -@@ t/t8002-blame.sh: test_description='git blame'
    - PROG='git blame -c'
    - . "$TEST_DIRECTORY"/annotate-tests.sh
    - 
    --test_expect_success 'setup' '
    --	test_oid_init
    --'
    --
    - test_expect_success 'blame untracked file in empty repo' '
    - 	>untracked &&
    - 	test_must_fail git blame untracked
    -
      ## t/t8003-blame-corner-cases.sh ##
     @@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
      pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v5 01/39] t: make test-bloom initialize repository
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 02/39] t1001: use $ZERO_OID brian m. carlson
                   ` (38 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The bloom filter code relies on reading object IDs using parse_oid_hex.
In order to make that work with an appropriate size, we need to have
initialized the repository's hash algorithm.  Since the values we're
processing depend on the repository in use, let's set up the repository
when we run the test helper.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/helper/test-bloom.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/t/helper/test-bloom.c b/t/helper/test-bloom.c
index f0aa80b98e..5e77d56f59 100644
--- a/t/helper/test-bloom.c
+++ b/t/helper/test-bloom.c
@@ -50,6 +50,8 @@ static const char *bloom_usage = "\n"
 
 int cmd__bloom(int argc, const char **argv)
 {
+	setup_git_directory();
+
 	if (argc < 2)
 		usage(bloom_usage);
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 02/39] t1001: use $ZERO_OID
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 01/39] t: make test-bloom initialize repository brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 03/39] t3305: make hash agnostic brian m. carlson
                   ` (37 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $ZERO_OID to make the test hash independent.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t1091-sparse-checkout-builtin.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 7cd45fc139..84acfc48b6 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -369,7 +369,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
 	git clone repo unmerged &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C unmerged rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C unmerged update-index --index-info <input &&
@@ -396,7 +396,7 @@ test_expect_success 'sparse-checkout reapply' '
 	echo dirty >tweak/deep/deeper2/a &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C tweak rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C tweak update-index --index-info <input &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 03/39] t3305: make hash agnostic
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 01/39] t: make test-bloom initialize repository brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 02/39] t1001: use $ZERO_OID brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
                   ` (36 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

When computing the fanout length, let's use test_oid to look up the
hexadecimal size of the hash in question instead of hard-coding a value.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3305-notes-fanout.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index 3b4753e1b4..d5a94b7174 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -7,7 +7,7 @@ test_description='Test that adding/removing many notes triggers automatic fanout
 path_has_fanout() {
 	path=$1 &&
 	fanout=$2 &&
-	after_last_slash=$((40 - $fanout * 2)) &&
+	after_last_slash=$(($(test_oid hexsz) - $fanout * 2)) &&
 	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
 }
 
@@ -31,6 +31,7 @@ all_notes_have_fanout() {
 }
 
 test_expect_success 'tweak test environment' '
+	test_oid_init &&
 	git checkout -b nondeterminism &&
 	test_commit A &&
 	git checkout --orphan with_notes;

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (2 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 03/39] t3305: make hash agnostic brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 05/39] t6100: make hash size independent brian m. carlson
                   ` (35 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The idea of the magic value "ac4f2ee" in this test is to make the
reworded commit `collide2` have the same shortened ID as the commit
`collide3`.

To port the same idea to the SHA-256 version of Git, we therefore need
another magic value that causes the same collision, but this time with
the SHA-256 version of the commit IDs.

In this patch, we add code guarded by `GIT_TEST_FIND_COLLIDER` to do
exactly that. Essentially, a large number of integers is appended to the
commit message "collide2" to find such a collision. To make it easier to
find such a collision, we reduce the number of digits to 4.

As the tests are no longer dependent on SHA-1, we also rename their
titles to talk about "commit IDs" instead of "SHA-1s".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3404-rebase-interactive.sh | 50 ++++++++++++++++++++++++++++-------
 1 file changed, 41 insertions(+), 9 deletions(-)

diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 4a7d21f898..2fb2c581cb 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -30,6 +30,7 @@ Initial setup:
 . "$TEST_DIRECTORY"/lib-rebase.sh
 
 test_expect_success 'setup' '
+	test_oid_init &&
 	test_commit A file1 &&
 	test_commit B file1 &&
 	test_commit C file2 &&
@@ -1250,7 +1251,7 @@ test_expect_success 'rebase -i error on commits with \ in message' '
 	test_expect_code 1 grep  "	emp" error
 '
 
-test_expect_success SHA1 'short SHA-1 setup' '
+test_expect_success 'short commit ID setup' '
 	test_when_finished "git checkout master" &&
 	git checkout --orphan collide &&
 	git rm -rf . &&
@@ -1262,23 +1263,54 @@ test_expect_success SHA1 'short SHA-1 setup' '
 	)
 '
 
-test_expect_success SHA1 'short SHA-1 collide' '
+if test -n "$GIT_TEST_FIND_COLLIDER"
+then
+	author="$(unset test_tick; test_tick; git var GIT_AUTHOR_IDENT)"
+	committer="$(unset test_tick; test_tick; git var GIT_COMMITTER_IDENT)"
+	blob="$(git rev-parse collide2:collide)"
+	from="$(git rev-parse collide1^0)"
+	repl="commit refs/heads/collider-&\\n"
+	repl="${repl}author $author\\ncommitter $committer\\n"
+	repl="${repl}data <<EOF\\ncollide2 &\\nEOF\\n"
+	repl="${repl}from $from\\nM 100644 $blob collide\\n"
+	test_seq 1 32768 | sed "s|.*|$repl|" >script &&
+	git fast-import <script &&
+	git pack-refs &&
+	git for-each-ref >refs &&
+	grep "^$(test_oid t3404_collision)" <refs >matches &&
+	cat matches &&
+	test_line_count -gt 2 matches || {
+		echo "Could not find a collider" >&2
+		exit 1
+	}
+fi
+
+test_expect_success 'short commit ID collide' '
+	test_oid_cache <<-EOF &&
+	# collision-related constants
+	t3404_collision	sha1:6bcd
+	t3404_collision	sha256:0161
+	t3404_collider	sha1:ac4f2ee
+	t3404_collider	sha256:16697
+	EOF
 	test_when_finished "reset_rebase && git checkout master" &&
 	git checkout collide &&
-	colliding_sha1=6bcda37 &&
-	test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
+	colliding_id=$(test_oid t3404_collision) &&
+	hexsz=$(test_oid hexsz) &&
+	test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+	test_config core.abbrev 4 &&
 	(
 		unset test_tick &&
 		test_tick &&
 		set_fake_editor &&
-		FAKE_COMMIT_MESSAGE="collide2 ac4f2ee" \
+		FAKE_COMMIT_MESSAGE="collide2 $(test_oid t3404_collider)" \
 		FAKE_LINES="reword 1 break 2" git rebase -i HEAD~2 &&
-		test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
-		grep "^pick $colliding_sha1 " \
+		test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+		grep "^pick $colliding_id " \
 			.git/rebase-merge/git-rebase-todo.tmp &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo.backup &&
 		git rebase --continue
 	) &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 05/39] t6100: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (3 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 06/39] t6101: " brian m. carlson
                   ` (34 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of hard-coding a constant 40, split the output of rev-list by
field.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6100-rev-list-in-order.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t6100-rev-list-in-order.sh b/t/t6100-rev-list-in-order.sh
index b2bb0a7f61..e934bc239c 100755
--- a/t/t6100-rev-list-in-order.sh
+++ b/t/t6100-rev-list-in-order.sh
@@ -22,7 +22,7 @@ test_expect_success 'setup a commit history with trees, blobs' '
 
 test_expect_success 'rev-list --in-commit-order' '
 	git rev-list --in-commit-order --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
@@ -49,7 +49,7 @@ test_expect_success 'rev-list --in-commit-order' '
 
 test_expect_success 'rev-list lists blobs and trees after commits' '
 	git rev-list --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 06/39] t6101: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (4 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 05/39] t6100: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 07/39] t6301: " brian m. carlson
                   ` (33 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $OID_REGEX instead of a hard-coded regular expression.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6101-rev-parse-parents.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t6101-rev-parse-parents.sh b/t/t6101-rev-parse-parents.sh
index 7683e4a114..7531262a5e 100755
--- a/t/t6101-rev-parse-parents.sh
+++ b/t/t6101-rev-parse-parents.sh
@@ -34,7 +34,7 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'start is valid' '
-	git rev-parse start | grep "^[0-9a-f]\{40\}$"
+	git rev-parse start | grep "^$OID_REGEX$"
 '
 
 test_expect_success 'start^0' '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 07/39] t6301: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (5 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 06/39] t6101: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 08/39] t6500: specify test values for SHA-256 brian m. carlson
                   ` (32 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of hard-coding a fixed length example object ID in the test,
compute one using the translation tables.  Move a variable into the
setup block so that we can ensure the exit status of test_oid_init is
checked.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6301-for-each-ref-errors.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index 49cc65bb58..d545a725a5 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -5,9 +5,10 @@ test_description='for-each-ref errors for broken refs'
 . ./test-lib.sh
 
 ZEROS=$ZERO_OID
-MISSING=abababababababababababababababababababab
 
 test_expect_success setup '
+	test_oid_init &&
+	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
 	git for-each-ref >full-list &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 08/39] t6500: specify test values for SHA-256
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (6 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 07/39] t6301: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 09/39] t6501: avoid hard-coded objects brian m. carlson
                   ` (31 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In this test, we want to produce several blobs whose first two hex
characters are "17", since we look at this object directory as a proxy
for how many loose objects there are before we need to GC.  Use
test_oid_cache to specify strings that will hash to the right values
when turned into blobs.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6500-gc.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 0a69a67117..4a3b8f48ac 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -10,7 +10,24 @@ test_expect_success 'setup' '
 	# do not let the amount of physical memory affects gc
 	# behavior, make sure we always pack everything to one pack by
 	# default
-	git config gc.bigPackThreshold 2g
+	git config gc.bigPackThreshold 2g &&
+
+	# These are simply values which, when hashed as a blob with a newline,
+	# produce a hash where the first byte is 0x17 in their respective
+	# algorithms.
+	test_oid_cache <<-EOF
+	obj1 sha1:263
+	obj1 sha256:34
+
+	obj2 sha1:410
+	obj2 sha256:174
+
+	obj3 sha1:523
+	obj3 sha256:313
+
+	obj4 sha1:790
+	obj4 sha256:481
+	EOF
 '
 
 test_expect_success 'gc empty repository' '
@@ -85,13 +102,13 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	# We need to create two object whose sha1s start with 17
 	# since this is what git gc counts.  As it happens, these
 	# two blobs will do so.
-	test_commit 263 &&
-	test_commit 410 &&
+	test_commit "$(test_oid obj1)" &&
+	test_commit "$(test_oid obj2)" &&
 	# Our first gc will create a pack; our second will create a second pack
 	git gc --auto &&
 	ls .git/objects/pack | sort >existing_packs &&
-	test_commit 523 &&
-	test_commit 790 &&
+	test_commit "$(test_oid obj3)" &&
+	test_commit "$(test_oid obj4)" &&
 
 	git gc --auto 2>err &&
 	test_i18ngrep ! "^warning:" err &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 09/39] t6501: avoid hard-coded objects
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (7 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 08/39] t6500: specify test values for SHA-256 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 10/39] t7003: compute appropriate length constant brian m. carlson
                   ` (30 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test contains hard-coded invalid object IDs.  Make it hash size
independent by generating invalid object IDs using the translation
tables.  Add a setup target to ensure the output of test_oid_init is
checked properly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6501-freshen-objects.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index f30b4849b6..7416ed5ffe 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -45,6 +45,10 @@ maybe_repack () {
 	fi
 }
 
+test_expect_success 'setup' '
+	test_oid_init
+'
+
 for repack in '' true; do
 	title=${repack:+repack}
 	title=${title:-loose}
@@ -128,9 +132,9 @@ for repack in '' true; do
 done
 
 test_expect_success 'do not complain about existing broken links (commit)' '
-	cat >broken-commit <<-\EOF &&
-	tree 0000000000000000000000000000000000000001
-	parent 0000000000000000000000000000000000000002
+	cat >broken-commit <<-EOF &&
+	tree $(test_oid 001)
+	parent $(test_oid 002)
 	author whatever <whatever@example.com> 1234 -0000
 	committer whatever <whatever@example.com> 1234 -0000
 
@@ -143,8 +147,8 @@ test_expect_success 'do not complain about existing broken links (commit)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tree)' '
-	cat >broken-tree <<-\EOF &&
-	100644 blob 0000000000000000000000000000000000000003	foo
+	cat >broken-tree <<-EOF &&
+	100644 blob $(test_oid 003)	foo
 	EOF
 	tree=$(git mktree --missing <broken-tree) &&
 	git gc -q 2>stderr &&
@@ -153,8 +157,8 @@ test_expect_success 'do not complain about existing broken links (tree)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tag)' '
-	cat >broken-tag <<-\EOF &&
-	object 0000000000000000000000000000000000000004
+	cat >broken-tag <<-EOF &&
+	object $(test_oid 004)
 	type commit
 	tag broken
 	tagger whatever <whatever@example.com> 1234 -0000

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 10/39] t7003: compute appropriate length constant
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (8 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 09/39] t6501: avoid hard-coded objects brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 11/39] t7063: make hash size independent brian m. carlson
                   ` (29 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using a specific invalid hard-coded object ID, look one
up from the translation table.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7003-filter-branch.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index e23de7d0b5..beeb2b855a 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -5,6 +5,7 @@ test_description='git filter-branch'
 . "$TEST_DIRECTORY/lib-gpg.sh"
 
 test_expect_success 'setup' '
+	test_oid_init &&
 	test_commit A &&
 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
 	test_commit --notick B &&
@@ -463,10 +464,11 @@ test_expect_success 'rewrite submodule with another content' '
 '
 
 test_expect_success 'replace submodule revision' '
+	invalid=$(test_oid numeric) &&
 	git reset --hard original &&
 	git filter-branch -f --tree-filter \
 	    "if git ls-files --error-unmatch -- submod > /dev/null 2>&1
-	     then git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 submod
+	     then git update-index --cacheinfo 160000 $invalid submod
 	     fi" HEAD &&
 	test $orig_head != $(git show-ref --hash --head HEAD)
 '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 11/39] t7063: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (9 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 10/39] t7003: compute appropriate length constant brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
                   ` (28 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use test_oid instead of hard-coding algorithm-specific constants and
all-zero values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7063-status-untracked-cache.sh | 150 ++++++++++++++++--------------
 1 file changed, 80 insertions(+), 70 deletions(-)

diff --git a/t/t7063-status-untracked-cache.sh b/t/t7063-status-untracked-cache.sh
index 428cff9cf3..a682a3d826 100755
--- a/t/t7063-status-untracked-cache.sh
+++ b/t/t7063-status-untracked-cache.sh
@@ -75,14 +75,24 @@ test_expect_success 'setup' '
 	touch one two three done/one dtwo/two dthree/three &&
 	git add one two done/one &&
 	: >.git/info/exclude &&
-	git update-index --untracked-cache
+	git update-index --untracked-cache &&
+	test_oid_cache <<-EOF
+	root sha1:e6fcc8f2ee31bae321d66afd183fcb7237afae6e
+	root sha256:b90c672088c015b9c83876e919da311bad4cd39639fb139f988af6a11493b974
+
+	exclude sha1:13263c0978fb9fad16b2d580fb800b6d811c3ff0
+	exclude sha256:fe4aaa1bbbbce4cb8f73426748a14c5ad6026b26f90505a0bf2494b165a5b76c
+
+	done sha1:1946f0437f90c5005533cbe1736a6451ca301714
+	done sha256:7f079501d79f665b3acc50f5e0e9e94509084d5032ac20113a37dd5029b757cc
+	EOF
 '
 
 test_expect_success 'untracked cache is empty' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-empty <<EOF &&
-info/exclude 0000000000000000000000000000000000000000
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $ZERO_OID
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
 EOF
@@ -100,17 +110,17 @@ EOF
 
 cat >../dump.expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 
@@ -190,18 +200,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 four
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -239,18 +249,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -284,16 +294,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -303,14 +313,14 @@ test_expect_success 'move two from tracked to untracked' '
 	git rm --cached two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -342,17 +352,17 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
 two
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -362,14 +372,14 @@ test_expect_success 'move two from untracked to tracked' '
 	git add two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -401,16 +411,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -447,16 +457,16 @@ EOF
 test_expect_success 'untracked cache correct after commit' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -508,17 +518,17 @@ EOF
 test_expect_success 'untracked cache correct after status' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -580,22 +590,22 @@ EOF
 test_expect_success 'verify untracked cache dump (sparse/subdirs)' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-from-test-dump <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
 sub/
-/done/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/ $ZERO_OID recurse check_only valid
 sub/
-/done/sub/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/sub/ $ZERO_OID recurse check_only valid
 file
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect-from-test-dump ../actual
@@ -806,8 +816,8 @@ test_expect_success '"status" after file replacement should be clean with UC=tru
 	test-tool dump-untracked-cache >../actual &&
 	grep -F "recurse valid" ../actual >../actual.grep &&
 	cat >../expect.grep <<EOF &&
-/ 0000000000000000000000000000000000000000 recurse valid
-/two/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
+/two/ $ZERO_OID recurse valid
 EOF
 	status_is_clean &&
 	test_cmp ../expect.grep ../actual.grep

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 12/39] t7201: abstract away SHA-1-specific constants
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (10 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 11/39] t7063: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 13/39] t7102: " brian m. carlson
                   ` (27 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7201-co.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index b696bae5f5..4d62b9b00f 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -230,9 +230,10 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 	git config advice.detachedHead false &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	git checkout renamer^ 2>messages &&
-	test_i18ngrep "HEAD is now at 7329388" messages &&
+	test_i18ngrep "HEAD is now at $rev" messages &&
 	test_line_count = 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
@@ -248,9 +249,10 @@ test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 test_expect_success 'checkout to detach HEAD' '
 	git config advice.detachedHead true &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
-	grep "HEAD is now at 7329388" messages &&
+	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 13/39] t7102: abstract away SHA-1-specific constants
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (11 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 14/39] t7400: make hash size independent brian m. carlson
                   ` (26 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7102-reset.sh | 93 +++++++++++++++++++++++++++---------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh
index 97be0d968d..22161b3b2d 100755
--- a/t/t7102-reset.sh
+++ b/t/t7102-reset.sh
@@ -37,17 +37,23 @@ test_expect_success 'creating initial files and commits' '
 
 	echo "2nd line 1st file" >>first &&
 	git commit -a -m "modify 1st file" &&
+	head5p2=$(git rev-parse --verify HEAD) &&
+	head5p2f=$(git rev-parse --short HEAD:first) &&
 
 	git rm first &&
 	git mv second secondfile &&
 	git commit -a -m "remove 1st and rename 2nd" &&
+	head5p1=$(git rev-parse --verify HEAD) &&
+	head5p1s=$(git rev-parse --short HEAD:secondfile) &&
 
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	# "git commit -m" would break MinGW, as Windows refuse to pass
 	# $test_encoding encoded parameter to git.
 	commit_msg $test_encoding | git -c "i18n.commitEncoding=$test_encoding" commit -a -F - &&
-	head5=$(git rev-parse --verify HEAD)
+	head5=$(git rev-parse --verify HEAD) &&
+	head5s=$(git rev-parse --short HEAD:secondfile) &&
+	head5sl=$(git rev-parse HEAD:secondfile)
 '
 # git log --pretty=oneline # to see those SHA1 involved
 
@@ -94,7 +100,7 @@ test_expect_success 'giving a non existing revision should fail' '
 
 test_expect_success 'reset --soft with unmerged index should fail' '
 	touch .git/MERGE_HEAD &&
-	echo "100644 44c5b5884550c17758737edcced463447b91d42b 1	un" |
+	echo "100644 $head5sl 1	un" |
 		git update-index --index-info &&
 	test_must_fail git reset --soft HEAD &&
 	rm .git/MERGE_HEAD &&
@@ -192,7 +198,7 @@ test_expect_success \
 >.diff_expect
 cat >.cached_expect <<EOF
 diff --git a/secondfile b/secondfile
-index 1bbba79..44c5b58 100644
+index $head5p1s..$head5s 100644
 --- a/secondfile
 +++ b/secondfile
 @@ -1 +1,2 @@
@@ -207,7 +213,7 @@ secondfile:
 EOF
 test_expect_success '--soft reset only should show changes in diff --cached' '
 	git reset --soft HEAD^ &&
-	check_changes d1a4bc3abce4829628ae2dcb0d60ef3d1a78b1c4 &&
+	check_changes $head5p1 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head5
 '
@@ -242,7 +248,7 @@ EOF
 test_expect_success \
 	'--hard reset should change the files and undo commits permanently' '
 	git reset --hard HEAD~2 &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	check_changes $head5p2 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head4
 '
@@ -251,7 +257,7 @@ test_expect_success \
 cat >.cached_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -259,14 +265,14 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
 -2nd file
 diff --git a/secondfile b/secondfile
 new file mode 100644
-index 0000000..44c5b58
+index 0000000..$head5s
 --- /dev/null
 +++ b/secondfile
 @@ -0,0 +1,2 @@
@@ -286,13 +292,13 @@ test_expect_success \
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	git add secondfile &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2
 '
 
 cat >.diff_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -300,7 +306,7 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
@@ -314,9 +320,8 @@ secondfile:
 EOF
 test_expect_success '--mixed reset to HEAD should unadd the files' '
 	git reset &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
-	test "$(git rev-parse ORIG_HEAD)" = \
-			ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2 &&
+	test "$(git rev-parse ORIG_HEAD)" = $head5p2
 '
 
 >.diff_expect
@@ -328,7 +333,7 @@ secondfile:
 EOF
 test_expect_success 'redoing the last two commits should succeed' '
 	git add secondfile &&
-	git reset --hard ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	git reset --hard $head5p2 &&
 
 	git rm first &&
 	git mv second secondfile &&
@@ -389,47 +394,55 @@ test_expect_success \
 	check_changes $head5
 '
 
-cat > expect << EOF
-diff --git a/file1 b/file1
-index d00491f..7ed6ff8 100644
---- a/file1
-+++ b/file1
-@@ -1 +1 @@
--1
-+5
-diff --git a/file2 b/file2
-deleted file mode 100644
-index 0cfbf08..0000000
---- a/file2
-+++ /dev/null
-@@ -1 +0,0 @@
--2
-EOF
-cat > cached_expect << EOF
-diff --git a/file4 b/file4
-new file mode 100644
-index 0000000..b8626c4
---- /dev/null
-+++ b/file4
-@@ -0,0 +1 @@
-+4
-EOF
 test_expect_success 'test --mixed <paths>' '
 	echo 1 > file1 &&
 	echo 2 > file2 &&
 	git add file1 file2 &&
 	test_tick &&
 	git commit -m files &&
+	before1=$(git rev-parse --short HEAD:file1) &&
+	before2=$(git rev-parse --short HEAD:file2) &&
 	git rm file2 &&
 	echo 3 > file3 &&
 	echo 4 > file4 &&
 	echo 5 > file1 &&
+	after1=$(git rev-parse --short $(git hash-object file1)) &&
+	after4=$(git rev-parse --short $(git hash-object file4)) &&
 	git add file1 file3 file4 &&
 	git reset HEAD -- file1 file2 file3 &&
 	test_must_fail git diff --quiet &&
 	git diff > output &&
+
+	cat > expect <<-EOF &&
+	diff --git a/file1 b/file1
+	index $before1..$after1 100644
+	--- a/file1
+	+++ b/file1
+	@@ -1 +1 @@
+	-1
+	+5
+	diff --git a/file2 b/file2
+	deleted file mode 100644
+	index $before2..0000000
+	--- a/file2
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-2
+	EOF
+
 	test_cmp expect output &&
 	git diff --cached > output &&
+
+	cat > cached_expect <<-EOF &&
+	diff --git a/file4 b/file4
+	new file mode 100644
+	index 0000000..$after4
+	--- /dev/null
+	+++ b/file4
+	@@ -0,0 +1 @@
+	+4
+	EOF
+
 	test_cmp cached_expect output
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 14/39] t7400: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (12 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 13/39] t7102: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 15/39] t7405: " brian m. carlson
                   ` (25 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using cut with hard-coded hash sizes, use cut with fields, or
where that's not possible, sed with $OID_REGEX, so that the tests are
independent of hash size.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7400-submodule-basic.sh | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 956e17abb3..fec7e0299d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '
 	git submodule add /dir1/b dir1/b &&
 	git submodule add /dir2/b dir2/b &&
 	git commit -m "first submodule commit" &&
-	git submodule--helper list dir1/b |cut -c51- >actual &&
+	git submodule--helper list dir1/b | cut -f 2 >actual &&
 	echo "dir1/b" >expect &&
 	test_cmp expect actual
 '
@@ -1260,7 +1260,7 @@ test_expect_success 'submodule update --init with a specification' '
 	pwd=$(pwd) &&
 	git clone file://"$pwd"/multisuper multisuper_clone &&
 	git -C multisuper_clone submodule update --init . ":(exclude)sub0" &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1271,7 +1271,7 @@ test_expect_success 'submodule update --init with submodule.active set' '
 	git -C multisuper_clone config submodule.active "." &&
 	git -C multisuper_clone config --add submodule.active ":(exclude)sub0" &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1290,7 +1290,7 @@ test_expect_success 'submodule update and setting submodule.<name>.active' '
 	-sub3
 	EOF
 	git -C multisuper_clone submodule update &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1307,12 +1307,12 @@ test_expect_success 'clone active submodule without submodule url set' '
 		git submodule update &&
 		git submodule status >actual_raw &&
 
-		cut -c 1,43- actual_raw >actual &&
+		cut -d" " -f3- actual_raw >actual &&
 		cat >expect <<-\EOF &&
-		 sub0 (test2)
-		 sub1 (test2)
-		 sub2 (test2)
-		 sub3 (test2)
+		sub0 (test2)
+		sub1 (test2)
+		sub2 (test2)
+		sub3 (test2)
 		EOF
 		test_cmp expect actual
 	)
@@ -1328,7 +1328,7 @@ test_expect_success 'clone --recurse-submodules with a pathspec works' '
 	EOF
 
 	git clone --recurse-submodules="sub0" multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expected actual
 '
 
@@ -1345,7 +1345,7 @@ test_expect_success 'clone with multiple --recurse-submodules options' '
 		  --recurse-submodules=":(exclude)sub0" \
 		  --recurse-submodules=":(exclude)sub2" \
 		  multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1373,7 +1373,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 		  --recurse-submodules=":(exclude)sub4" \
 		  multisuper multisuper_clone &&
 
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual &&
 
 	git -C multisuper submodule add ../sub1 sub4 &&
@@ -1382,7 +1382,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 	# obtain the new superproject
 	git -C multisuper_clone pull &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect2 actual
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 15/39] t7405: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (13 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 14/39] t7400: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
                   ` (24 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7405-submodule-merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh
index aa33978ed2..6a1e5f8232 100755
--- a/t/t7405-submodule-merge.sh
+++ b/t/t7405-submodule-merge.sh
@@ -195,7 +195,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
@@ -214,7 +214,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 16/39] t7506: avoid checking for SHA-1-specific constants
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (14 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 15/39] t7405: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
                   ` (23 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test to sanitize the diffs and strip out object IDs from
them, as it does for other object IDs, since we are not interested in
the particular values used.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7506-status-submodule.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/t/t7506-status-submodule.sh b/t/t7506-status-submodule.sh
index 08629a6e70..3fcb44767f 100755
--- a/t/t7506-status-submodule.sh
+++ b/t/t7506-status-submodule.sh
@@ -22,6 +22,10 @@ sanitize_output () {
 	mv output2 output
 }
 
+sanitize_diff () {
+	sed -e "/^index [0-9a-f,]*\.\.[0-9a-f]*/d" "$1"
+}
+
 
 test_expect_success 'setup' '
 	test_create_repo_with_commit sub &&
@@ -269,7 +273,6 @@ short_sha1_merge_sub1=$(cd sub1 && git rev-parse --short HEAD)
 short_sha1_merge_sub2=$(cd sub2 && git rev-parse --short HEAD)
 cat >diff_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -286,7 +289,6 @@ EOF
 
 cat >diff_submodule_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -306,7 +308,8 @@ test_expect_success 'diff with merge conflict in .gitmodules' '
 		cd super &&
 		git diff >../diff_actual 2>&1
 	) &&
-	test_cmp diff_expect diff_actual
+	sanitize_diff diff_actual >diff_sanitized &&
+	test_cmp diff_expect diff_sanitized
 '
 
 test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
@@ -314,7 +317,8 @@ test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
 		cd super &&
 		git diff --submodule >../diff_submodule_actual 2>&1
 	) &&
-	test_cmp diff_submodule_expect diff_submodule_actual
+	sanitize_diff diff_submodule_actual >diff_sanitized &&
+	test_cmp diff_submodule_expect diff_sanitized
 '
 
 # We'll setup different cases for further testing:

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 17/39] t7508: use $ZERO_OID instead of hard-coded constant
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (15 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 18/39] t8002: make hash size independent brian m. carlson
                   ` (22 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use the ZERO_OID variable to abbreviate the all-zeros object ID for
maintainability and to avoid depending on a specific size for the hash.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7508-status.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t7508-status.sh b/t/t7508-status.sh
index 8e969f3e36..e81759319f 100755
--- a/t/t7508-status.sh
+++ b/t/t7508-status.sh
@@ -837,7 +837,7 @@ EOF
 '
 
 cat >expect <<EOF
-:100644 100644 $EMPTY_BLOB 0000000000000000000000000000000000000000 M	dir1/modified
+:100644 100644 $EMPTY_BLOB $ZERO_OID M	dir1/modified
 EOF
 test_expect_success 'status refreshes the index' '
 	touch dir2/added &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 18/39] t8002: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (16 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29  2:24   ` Eric Sunshine
  2020-07-28 23:34 ` [PATCH v5 19/39] t8003: " brian m. carlson
                   ` (21 subsequent siblings)
  39 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Compute the length of an object ID instead of hard-coding 40-based
values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8002-blame.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
index eea048e52c..c3b70b025e 100755
--- a/t/t8002-blame.sh
+++ b/t/t8002-blame.sh
@@ -6,6 +6,10 @@ test_description='git blame'
 PROG='git blame -c'
 . "$TEST_DIRECTORY"/annotate-tests.sh
 
+test_expect_success 'setup' '
+	hexsz=$(test_oid hexsz)
+'
+
 test_expect_success 'blame untracked file in empty repo' '
 	>untracked &&
 	test_must_fail git blame untracked
@@ -105,17 +109,17 @@ test_expect_success 'blame --abbrev=<n> works' '
 '
 
 test_expect_success 'blame -l aligns regular and boundary commits' '
-	check_abbrev 40 -l HEAD &&
-	check_abbrev 39 -l ^HEAD
+	check_abbrev $hexsz         -l HEAD &&
+	check_abbrev $((hexsz - 1)) -l ^HEAD
 '
 
-test_expect_success 'blame --abbrev=40 behaves like -l' '
-	check_abbrev 40 --abbrev=40 HEAD &&
-	check_abbrev 39 --abbrev=40 ^HEAD
+test_expect_success 'blame --abbrev with full length behaves like -l' '
+	check_abbrev $hexsz         --abbrev=$hexsz HEAD &&
+	check_abbrev $((hexsz - 1)) --abbrev=$hexsz ^HEAD
 '
 
-test_expect_success '--no-abbrev works like --abbrev=40' '
-	check_abbrev 40 --no-abbrev
+test_expect_success '--no-abbrev works like --abbrev with full length' '
+	check_abbrev $hexsz --no-abbrev
 '
 
 test_expect_success '--exclude-promisor-objects does not BUG-crash' '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 19/39] t8003: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (17 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 18/39] t8002: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 20/39] t8011: " brian m. carlson
                   ` (20 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

One assertion in this test invokes git with core.abbrev set to "40".
Since we're expecting the full hash length, use test_oid to look up the
full hash length for the hash in use.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8003-blame-corner-cases.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index 9130b887d2..ebe79aa4ea 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,6 +6,7 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
+	test_oid_init &&
 
 	echo A A A A A >one &&
 	echo B B B B B >two &&
@@ -306,7 +307,7 @@ test_expect_success 'blame coalesce' '
 	$oid 1) ABC
 	$oid 2) DEF
 	EOF
-	git -c core.abbrev=40 blame -s giraffe >actual &&
+	git -c core.abbrev=$(test_oid hexsz) blame -s giraffe >actual &&
 	test_cmp expect actual
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 20/39] t8011: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (18 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 19/39] t8003: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
                   ` (19 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Allow lines which start with either a 40- or 64-character hex object ID,
to allow for both SHA-1 and SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8011-blame-split-file.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t8011-blame-split-file.sh b/t/t8011-blame-split-file.sh
index 831125047b..bdda0c03fe 100755
--- a/t/t8011-blame-split-file.sh
+++ b/t/t8011-blame-split-file.sh
@@ -54,7 +54,7 @@ test_expect_success 'setup simulated porcelain' '
 	cat >read-porcelain.pl <<-\EOF
 	my $field = shift;
 	while (<>) {
-		if (/^[0-9a-f]{40} /) {
+		if (/^[0-9a-f]{40,} /) {
 			flush();
 			$hash = $&;
 		} elsif (/^$field (.*)/) {

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 21/39] t9300: abstract away SHA-1-specific constants
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (19 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 20/39] t8011: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
                   ` (18 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.  In addition, use cut to filter out the object
IDs and verify only the information that we're really interested in.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 108 ++++++++++++++++++++++-------------------
 1 file changed, 59 insertions(+), 49 deletions(-)

diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index e151df81c0..f2d2cb06ad 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -284,8 +284,9 @@ test_expect_success 'A: verify pack' '
 '
 
 test_expect_success 'A: verify diff' '
+	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
+	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -528,6 +529,7 @@ test_expect_success 'B: fail on invalid committer (5)' '
 test_expect_success 'C: incremental import create pack from stdin' '
 	newf=$(echo hi newf | git hash-object -w --stdin) &&
 	oldf=$(git rev-parse --verify master:file2) &&
+	thrf=$(git rev-parse --verify master:file3) &&
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/branch
@@ -570,10 +572,11 @@ test_expect_success 'C: verify commit' '
 '
 
 test_expect_success 'C: validate rename result' '
+	zero=$ZERO_OID &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 f1fb5da718392694d0076d677d6d0e364c79b0bc A	file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2	file2/oldf
-	:100644 000000 0d92e9f3374ae2947c23aa477cbc68ce598135f1 0000000000000000000000000000000000000000 D	file3
+	:000000 100755 $zero $newf A	file2/newf
+	:100644 100644 $oldf $oldf R100	file2	file2/oldf
+	:100644 000000 $thrf $zero D	file3
 	EOF
 	git diff-tree -M -r master branch >actual &&
 	compare_diff_raw expect actual
@@ -614,9 +617,11 @@ test_expect_success 'D: verify pack' '
 '
 
 test_expect_success 'D: validate new files added' '
+	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
+	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 e74b7d465e52746be2b4bae983670711e6e66657 A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 A	newdir/interesting
+	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
+	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -779,12 +784,13 @@ test_expect_success 'H: verify pack' '
 '
 
 test_expect_success 'H: validate old files removed, new files added' '
+	f4id=$(git rev-parse HEAD:file4) &&
 	cat >expect <<-EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file2/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file2/oldf
-	:100755 000000 85df50785d62d3b05ab03d9cbf7e4a0b49449730 0000000000000000000000000000000000000000 D	file4
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	h/e/l/lo
-	:100755 000000 e74b7d465e52746be2b4bae983670711e6e66657 0000000000000000000000000000000000000000 D	newdir/exec.sh
+	:100755 000000 $newf $zero D	file2/newf
+	:100644 000000 $oldf $zero D	file2/oldf
+	:100755 000000 $f4id $zero D	file4
+	:100644 100644 $f5id $f5id R100	newdir/interesting	h/e/l/lo
+	:100755 000000 $f6id $zero D	newdir/exec.sh
 	EOF
 	git diff-tree -M -r H^ H >actual &&
 	compare_diff_raw expect actual
@@ -935,14 +941,15 @@ test_expect_success 'L: verify internal tree sorting' '
 	INPUT_END
 
 	cat >expect <<-EXPECT_END &&
-	:100644 100644 4268632... 55d3a52... M	b.
-	:040000 040000 0ae5cac... 443c768... M	b
-	:100644 100644 4268632... 55d3a52... M	ba
+	:100644 100644 M	b.
+	:040000 040000 M	b
+	:100644 100644 M	ba
 	EXPECT_END
 
 	git fast-import <input &&
 	GIT_PRINT_SHA1_ELLIPSIS="yes" git diff-tree --abbrev --raw L^ L >output &&
-	test_cmp expect output
+	cut -d" " -f1,2,5 output >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'L: nested tree copy does not corrupt deltas' '
@@ -1004,7 +1011,7 @@ test_expect_success 'M: rename file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf R100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M1^ M1 >actual &&
@@ -1025,7 +1032,7 @@ test_expect_success 'M: rename file to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	i/am/new/to/you
+	:100755 100755 $newf $newf R100	file2/newf	i/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M2^ M2 >actual &&
@@ -1046,7 +1053,7 @@ test_expect_success 'M: rename subdirectory to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	other/sub/am/new/to/you
+	:100755 100755 $newf $newf R100	i/am/new/to/you	other/sub/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M3^ M3 >actual &&
@@ -1067,11 +1074,11 @@ test_expect_success 'M: rename root to subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2/oldf	sub/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 R100	file4	sub/file4
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	sub/i/am/new/to/you
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 R100	newdir/exec.sh	sub/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	sub/newdir/interesting
+	:100644 100644 $oldf $oldf R100	file2/oldf	sub/file2/oldf
+	:100755 100755 $f4id $f4id R100	file4	sub/file4
+	:100755 100755 $newf $newf R100	i/am/new/to/you	sub/i/am/new/to/you
+	:100755 100755 $f6id $f6id R100	newdir/exec.sh	sub/newdir/exec.sh
+	:100644 100644 $f5id $f5id R100	newdir/interesting	sub/newdir/interesting
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M4^ M4 >actual &&
@@ -1097,7 +1104,7 @@ test_expect_success 'N: copy file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf C100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N1^ N1 >actual &&
@@ -1129,9 +1136,9 @@ test_expect_success 'N: copy then modify subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N2^^ N2 >actual &&
@@ -1162,9 +1169,9 @@ test_expect_success 'N: copy dirty subdirectory' '
 '
 
 test_expect_success 'N: copy directory by id' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -1183,9 +1190,9 @@ test_expect_success 'N: copy directory by id' '
 '
 
 test_expect_success PIPE 'N: read and copy directory' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git update-ref -d refs/heads/N4 &&
 	rm -f backflow &&
@@ -1254,9 +1261,9 @@ test_expect_success PIPE 'N: empty directory reads as missing' '
 '
 
 test_expect_success 'N: copy root directory by tree hash' '
-	cat >expect <<-\EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file3/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 000000 $newf $zero D	file3/newf
+	:100644 000000 $oldf $zero D	file3/oldf
 	EOF
 	root=$(git rev-parse refs/heads/branch^0^{tree}) &&
 	cat >input <<-INPUT_END &&
@@ -1275,12 +1282,12 @@ test_expect_success 'N: copy root directory by tree hash' '
 '
 
 test_expect_success 'N: copy root by path' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	oldroot/file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	oldroot/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 C100	file4	oldroot/file4
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 C100	newdir/exec.sh	oldroot/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	oldroot/newdir/interesting
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	oldroot/file2/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	oldroot/file2/oldf
+	:100755 100755 $f4id $f4id C100	file4	oldroot/file4
+	:100755 100755 $f6id $f6id C100	newdir/exec.sh	oldroot/newdir/exec.sh
+	:100644 100644 $f5id $f5id C100	newdir/interesting	oldroot/newdir/interesting
 	EOF
 	cat >input <<-INPUT_END &&
 	commit refs/heads/N-copy-root-path
@@ -1340,10 +1347,10 @@ test_expect_success 'N: delete directory by copying' '
 '
 
 test_expect_success 'N: modify copied tree' '
-	cat >expect <<-\EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -3117,6 +3124,9 @@ test_expect_success 'U: initialize for U tests' '
 
 	INPUT_END
 
+	f7id=$(echo "blob 1" | git hash-object --stdin) &&
+	f8id=$(echo "sleep well" | git hash-object --stdin) &&
+	f9id=$(echo "au revoir" | git hash-object --stdin) &&
 	git fast-import <input
 '
 
@@ -3137,7 +3147,7 @@ test_expect_success 'U: filedelete file succeeds' '
 
 test_expect_success 'U: validate file delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 2907ebb4bf85d91bf0716bb3bd8a68ef48d6da76 0000000000000000000000000000000000000000 D	good/night.txt
+	:100644 000000 $f8id $ZERO_OID D	good/night.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3162,7 +3172,7 @@ test_expect_success 'U: filedelete directory succeeds' '
 
 test_expect_success 'U: validate directory delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 69cb75792f55123d8389c156b0b41c2ff00ed507 0000000000000000000000000000000000000000 D	good/bye.txt
+	:100644 000000 $f9id $ZERO_OID D	good/bye.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3187,7 +3197,7 @@ test_expect_success 'U: filedelete root succeeds' '
 
 test_expect_success 'U: validate root delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 c18147dc648481eeb65dc5e66628429a64843327 0000000000000000000000000000000000000000 D	hello.c
+	:100644 000000 $f7id $ZERO_OID D	hello.c
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 22/39] t9300: use $ZERO_OID instead of hard-coded object ID
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (20 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 23/39] t9301: make hash size independent brian m. carlson
                   ` (17 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f2d2cb06ad..308c1ef42c 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -92,7 +92,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/to-be-deleted
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :6
@@ -102,7 +102,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/nested
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :7
@@ -286,7 +286,7 @@ test_expect_success 'A: verify pack' '
 test_expect_success 'A: verify diff' '
 	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
+	:000000 100755 $ZERO_OID $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -365,7 +365,7 @@ test_expect_success 'B: fail on invalid blob sha1' '
 	COMMIT
 
 	from refs/heads/master
-	M 755 0000000000000000000000000000000000000001 zero1
+	M 755 $(echo $ZERO_OID | sed -e "s/0$/1/") zero1
 
 	INPUT_END
 
@@ -620,8 +620,8 @@ test_expect_success 'D: validate new files added' '
 	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
 	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
+	:000000 100755 $ZERO_OID $f6id A	newdir/exec.sh
+	:000000 100644 $ZERO_OID $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -2733,7 +2733,7 @@ test_expect_success 'R: corrupt lines do not mess marks file' '
 	rm -f io.marks &&
 	blob=$(echo hi | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:3 0000000000000000000000000000000000000000
+	:3 $ZERO_OID
 	:1 $blob
 	:2 $blob
 	EOF
@@ -3084,7 +3084,7 @@ test_expect_success 'T: delete branch' '
 	git branch to-delete &&
 	git fast-import <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 	EOF
 	test_must_fail git rev-parse --verify refs/heads/to-delete
 '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 23/39] t9301: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (21 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 24/39] t9350: " brian m. carlson
                   ` (16 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using a hard-coded all-zeros object ID, use $ZERO_OID.
Compute the length of the object IDs in use and use this instead of
hard-coding the constant 40.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9301-fast-import-notes.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index ca223dca98..9d1aaac980 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -71,7 +71,7 @@ EOF
 INPUT_END
 
 test_expect_success 'set up master branch' '
-
+	test_oid_init &&
 	git fast-import <input &&
 	git whatchanged master
 '
@@ -470,12 +470,13 @@ test_expect_success 'add lots of commits and notes' '
 '
 
 test_expect_success 'verify that lots of notes trigger a fanout scheme' '
+	hexsz=$(test_oid hexsz) &&
 
 	# None of the entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -518,7 +519,7 @@ test_expect_success 'verify that importing a notes tree respects the fanout sche
 	git ls-tree --name-only refs/notes/other_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -593,7 +594,7 @@ test_expect_success 'verify that changing notes respect existing fanout' '
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -616,7 +617,7 @@ i=$(($num_commits - $remaining_notes))
 for sha1 in $(git rev-list -n $i refs/heads/many_commits)
 do
 	cat >>input <<INPUT_END
-N 0000000000000000000000000000000000000000 $sha1
+N $ZERO_OID $sha1
 INPUT_END
 done
 
@@ -646,7 +647,6 @@ test_expect_success 'remove lots of notes' '
 '
 
 test_expect_success 'verify that removing notes trigger fanout consolidation' '
-
 	# All entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only -r refs/notes/many_notes |
 	while read path
@@ -656,7 +656,7 @@ test_expect_success 'verify that removing notes trigger fanout consolidation' '
 		test "$path" = "deadbeef" && continue
 		test "$path" = "de/adbeef" && continue
 
-		if test $(expr length "$path") -ne 40
+		if test $(expr length "$path") -ne $hexsz
 		then
 			return 1
 		fi

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 24/39] t9350: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (22 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 23/39] t9301: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
                   ` (15 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test checks for several commit object sizes to verify that objects
are encoded as expected. However, the size of a commit object differs
between SHA-1 and SHA-256, since each contains a hex representation of
the tree's object ID. Since these are root commits, compute the size of
each commit by using a constant plus the size of a single hex object ID.

In addition, use $ZERO_OID instead of a hard-coded object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9350-fast-export.sh | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index 690c90fb82..ba79db6a17 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -7,6 +7,7 @@ test_description='git fast-export'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_init &&
 
 	echo break it > file0 &&
 	git add file0 &&
@@ -132,12 +133,12 @@ test_expect_success 'reencoding iso-8859-7' '
 	sed "s/wer/i18n/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, would be 240 bytes.
+		 # The commit object, if not re-encoded, would be 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header drops 20 bytes.
 		 # Re-encoding the Pi character from \xF0 (\360) in iso-8859-7
 		 # to \xCF\x80 (\317\200) in UTF-8 adds a byte.  Check for
 		 # the expected size.
-		 test 221 -eq "$(git cat-file -s i18n)" &&
+		 test $(($(test_oid hexsz) + 181)) -eq "$(git cat-file -s i18n)" &&
 		 # ...and for the expected translation of bytes.
 		 git cat-file commit i18n >actual &&
 		 grep $(printf "\317\200") actual &&
@@ -164,12 +165,12 @@ test_expect_success 'preserving iso-8859-7' '
 	sed "s/wer/i18n-no-recoding/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, is 240 bytes.
+		 # The commit object, if not re-encoded, is 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header would drops 20
 		 # bytes.  Re-encoding the Pi character from \xF0 (\360) in
 		 # iso-8859-7 to \xCF\x80 (\317\200) in UTF-8 adds a byte.
 		 # Check for the expected size...
-		 test 240 -eq "$(git cat-file -s i18n-no-recoding)" &&
+		 test $(($(test_oid hexsz) + 200)) -eq "$(git cat-file -s i18n-no-recoding)" &&
 		 # ...as well as the expected byte.
 		 git cat-file commit i18n-no-recoding >actual &&
 		 grep $(printf "\360") actual &&
@@ -192,7 +193,7 @@ test_expect_success 'encoding preserved if reencoding fails' '
 		 grep ^encoding actual &&
 		 # Verify that the commit has the expected size; i.e.
 		 # that no bytes were re-encoded to a different encoding.
-		 test 252 -eq "$(git cat-file -s i18n-invalid)" &&
+		 test $(($(test_oid hexsz) + 212)) -eq "$(git cat-file -s i18n-invalid)" &&
 		 # ...and check for the original special bytes
 		 grep $(printf "\360") actual &&
 		 grep $(printf "\377") actual)
@@ -694,7 +695,7 @@ test_expect_success 'delete ref because entire history excluded' '
 	git fast-export to-delete ^to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
@@ -704,7 +705,7 @@ test_expect_success 'delete refspec' '
 	git fast-export --refspec :refs/heads/to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 25/39] t9500: ensure that algorithm info is preserved in config
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (23 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 24/39] t9350: " brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 26/39] t9700: make hash size independent brian m. carlson
                   ` (14 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

When we use a hash algorithm other than SHA-1, it's important to
preserve the hash-related values in the config file, but this test
overwrites the config file with a new one. Ensure we copy these values
properly from the old config to the new one so that the repository can
be read if it's using SHA-256.

Note that if there is no extensions.objectFormat value set, git config
will return unsuccessfully if we try to read it; since this is not an
error for us, use test_might_fail.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9500-gitweb-standalone-no-errors.sh | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 267ddc997d..b484e3e250 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -621,12 +621,22 @@ test_expect_success \
 	 git config gitweb.snapshot "zip,tgz, tbz2" &&
 	 gitweb_run "p=.git;a=tree"'
 
-cat >.git/config <<\EOF
-# testing noval and alternate separator
-[gitweb]
-	blame
-	snapshot = zip tgz
-EOF
+test_expect_success 'setup' '
+	version=$(git config core.repositoryformatversion) &&
+	algo=$(test_might_fail git config extensions.objectformat) &&
+	cat >.git/config <<-\EOF &&
+	# testing noval and alternate separator
+	[gitweb]
+		blame
+		snapshot = zip tgz
+	EOF
+	git config core.repositoryformatversion "$version" &&
+	if test -n "$algo"
+	then
+		git config extensions.objectformat "$algo"
+	fi
+'
+
 test_expect_success \
 	'config override: tree view, features enabled in repo config (2)' \
 	'gitweb_run "p=.git;a=tree"'

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 26/39] t9700: make hash size independent
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (24 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 27/39] t5308: make test work with SHA-256 brian m. carlson
                   ` (13 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The Perl test script for t9700 was matching on exactly 40 hex
characters.  With SHA-256, we'll have 64 hex-character object IDs.
Create a variable with a regex which matches exactly 40 or 64 hex
characters and use that to match the output.  Note that both of the uses
of this can be anchored, which makes the code simpler, so do that as
well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9700/test.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 34cd01366f..071ff6d82a 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -23,6 +23,8 @@ sub adjust_dirsep {
 	return $path;
 }
 
+my $oid_re = qr/^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?$/;
+
 BEGIN { use_ok('Git') }
 
 # set up
@@ -93,7 +95,7 @@ sub adjust_dirsep {
 open TEMPFILE, ">$tmpfile" or die "Can't open $tmpfile: $!";
 print TEMPFILE my $test_text = "test blob, to be inserted\n";
 close TEMPFILE or die "Failed writing to $tmpfile: $!";
-like(our $newhash = $r->hash_and_insert_object($tmpfile), qr/[0-9a-fA-F]{40}/,
+like(our $newhash = $r->hash_and_insert_object($tmpfile), $oid_re,
      "hash_and_insert_object: returns hash");
 open TEMPFILE, "+>$tmpfile" or die "Can't open $tmpfile: $!";
 is($r->cat_blob($newhash, \*TEMPFILE), length $test_text, "cat_blob: roundtrip size");
@@ -119,7 +121,7 @@ sub adjust_dirsep {
 
 # commands in sub directory
 my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
-like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
+like($last_commit, $oid_re, 'rev-parse returned hash');
 my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
 isnt($last_commit, $dir_commit, 'log . does not show last commit');
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 27/39] t5308: make test work with SHA-256
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (25 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 26/39] t9700: make hash size independent brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
                   ` (12 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test needs multiple object IDs that have the same first byte.
Update the pack test code to generate a suitable packed value for
SHA-256.  Update the test to use this value when using SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                     |  8 ++++++++
 t/t5308-pack-detect-duplicates.sh | 20 ++++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index f3463170b3..0c799d53c6 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -93,6 +93,14 @@ pack_obj () {
 			;;
 		esac
 		;;
+	# blob containing "\3\326"
+	471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe)
+		case "$2" in
+		'')
+			printf '\062\170\234\143\276\006\000\000\336\000\332'
+			return
+			;;
+		esac
 	esac
 
 	# If it's not a delta, we can convince pack-objects to generate a pack
diff --git a/t/t5308-pack-detect-duplicates.sh b/t/t5308-pack-detect-duplicates.sh
index 6845c1f3c3..693b2411c8 100755
--- a/t/t5308-pack-detect-duplicates.sh
+++ b/t/t5308-pack-detect-duplicates.sh
@@ -4,23 +4,27 @@ test_description='handling of duplicate objects in incoming packfiles'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-pack.sh
 
-if ! test_have_prereq SHA1
-then
-       skip_all='not using SHA-1 for objects'
-       test_done
-fi
+test_expect_success 'setup' '
+	test_oid_cache <<-EOF
+	lo_oid sha1:e68fe8129b546b101aee9510c5328e7f21ca1d18
+	lo_oid sha256:471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe
+
+	missing_oid sha1:e69d000000000000000000000000000000000000
+	missing_oid sha256:4720000000000000000000000000000000000000000000000000000000000000
+	EOF
+'
 
 # The sha1s we have in our pack. It's important that these have the same
 # starting byte, so that they end up in the same fanout section of the index.
 # That lets us make sure we are exercising the binary search with both sets.
-LO_SHA1=e68fe8129b546b101aee9510c5328e7f21ca1d18
-HI_SHA1=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+LO_SHA1=$(test_oid lo_oid)
+HI_SHA1=$EMPTY_BLOB
 
 # And here's a "missing sha1" which will produce failed lookups. It must also
 # be in the same fanout section, and should be between the two (so that during
 # our binary search, we are sure to end up looking at one or the other of the
 # duplicate runs).
-MISSING_SHA1='e69d000000000000000000000000000000000000'
+MISSING_SHA1=$(test_oid missing_oid)
 
 # git will never intentionally create packfiles with
 # duplicate objects, so we have to construct them by hand.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 28/39] t0410: mark test with SHA1 prerequisite
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (26 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 27/39] t5308: make test work with SHA-256 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
                   ` (11 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

These tests try to check that we behave properly if we encounter a
repository with version 0 but an extension.  This is a laudable goal,
but the test cannot work with SHA-256, since SHA-256 repositories always
have an existing extension and are never version 0.

Add a SHA1 prerequisite to these tests.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0410-partial-clone.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index 6aa0f313bd..a5ebdf9ff3 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -42,7 +42,7 @@ test_expect_success 'convert shallow clone to partial clone' '
 	test_cmp_config -C client 1 core.repositoryformatversion
 '
 
-test_expect_success 'convert to partial clone with noop extension' '
+test_expect_success SHA1 'convert to partial clone with noop extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
@@ -53,7 +53,7 @@ test_expect_success 'convert to partial clone with noop extension' '
 	git -C client fetch --unshallow --filter="blob:none"
 '
 
-test_expect_success 'converting to partial clone fails with unrecognized extension' '
+test_expect_success SHA1 'converting to partial clone fails with unrecognized extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (27 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29  2:36   ` Eric Sunshine
  2020-07-28 23:34 ` [PATCH v5 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
                   ` (10 subsequent siblings)
  39 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In dd4b732df7 ("upload-pack: send part of packfile response as uri",
2020-06-10), the git http-fetch code learned how to take  ac --packfile
option.  This option takes an argument, which is the name of a packfile
hash, and parses it using parse_oid_hex.  It does so before calling
setup_git_directory.

However, in a SHA-256 repository this fails to work, since we have not
set the hash algorithm in use and parse_oid_hex fails as a consequence.
To ensure that we can parse packfile hashes of the right length, let's
set up the git directory before we start parsing arguments.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 http-fetch.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/http-fetch.c b/http-fetch.c
index 1df376e745..65050dee2b 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -86,6 +86,8 @@ int cmd_main(int argc, const char **argv)
 	int packfile = 0;
 	struct object_id packfile_hash;
 
+	setup_git_directory_gently(NULL);
+
 	while (arg < argc && argv[arg][0] == '-') {
 		const char *p;
 
@@ -115,8 +117,6 @@ int cmd_main(int argc, const char **argv)
 	if (argc != arg + 2 - (commits_on_stdin || packfile))
 		usage(http_fetch_usage);
 
-	setup_git_directory();
-
 	git_config(git_default_config, NULL);
 
 	if (packfile) {

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 30/39] builtin/verify-pack: implement an --object-format option
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (28 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 31/39] bundle: add new version for use with SHA-256 brian m. carlson
                   ` (9 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

A recently added test in t5702 started using git verify-pack outside of
a repository.  While this poses no problems with SHA-1, with SHA-256 we
implicitly rely on the setup of the repository to initialize our hash
algorithm settings.

Since we're not in a repository here, we need to provide git verify-pack
help to set things up properly.  git index-pack already knows an
--object-format option, so let's accept one as well and pass it down to
our git index-pack invocation.  Since we're now dynamically adjusting
the elements in argv, let's switch to using struct argv_array to manage
them.  Finally, let's make t5702 pass the proper argument on down to its
git verify-pack caller.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/verify-pack.c  | 23 +++++++++++++++--------
 t/t5702-protocol-v2.sh |  2 +-
 2 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c2a1a5c504..3669a90263 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -7,21 +7,26 @@
 #define VERIFY_PACK_VERBOSE 01
 #define VERIFY_PACK_STAT_ONLY 02
 
-static int verify_one_pack(const char *path, unsigned int flags)
+static int verify_one_pack(const char *path, unsigned int flags, const char *hash_algo)
 {
 	struct child_process index_pack = CHILD_PROCESS_INIT;
-	const char *argv[] = {"index-pack", NULL, NULL, NULL };
+	struct argv_array *argv = &index_pack.args;
 	struct strbuf arg = STRBUF_INIT;
 	int verbose = flags & VERIFY_PACK_VERBOSE;
 	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
 	int err;
 
+	argv_array_push(argv, "index-pack");
+
 	if (stat_only)
-		argv[1] = "--verify-stat-only";
+		argv_array_push(argv, "--verify-stat-only");
 	else if (verbose)
-		argv[1] = "--verify-stat";
+		argv_array_push(argv, "--verify-stat");
 	else
-		argv[1] = "--verify";
+		argv_array_push(argv, "--verify");
+
+	if (hash_algo)
+		argv_array_pushf(argv, "--object-format=%s", hash_algo);
 
 	/*
 	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
@@ -31,9 +36,8 @@ static int verify_one_pack(const char *path, unsigned int flags)
 	if (strbuf_strip_suffix(&arg, ".idx") ||
 	    !ends_with(arg.buf, ".pack"))
 		strbuf_addstr(&arg, ".pack");
-	argv[2] = arg.buf;
+	argv_array_push(argv, arg.buf);
 
-	index_pack.argv = argv;
 	index_pack.git_cmd = 1;
 
 	err = run_command(&index_pack);
@@ -60,12 +64,15 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 {
 	int err = 0;
 	unsigned int flags = 0;
+	const char *object_format = NULL;
 	int i;
 	const struct option verify_pack_options[] = {
 		OPT_BIT('v', "verbose", &flags, N_("verbose"),
 			VERIFY_PACK_VERBOSE),
 		OPT_BIT('s', "stat-only", &flags, N_("show statistics only"),
 			VERIFY_PACK_STAT_ONLY),
+		OPT_STRING(0, "object-format", &object_format, N_("hash"),
+			   N_("specify the hash algorithm to use")),
 		OPT_END()
 	};
 
@@ -75,7 +82,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	if (argc < 1)
 		usage_with_options(verify_pack_usage, verify_pack_options);
 	for (i = 0; i < argc; i++) {
-		if (verify_one_pack(argv[i], flags))
+		if (verify_one_pack(argv[i], flags, object_format))
 			err = 1;
 	}
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 1b54c35b01..7fc22171e7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -829,7 +829,7 @@ test_expect_success 'part of packfile response provided as URI' '
 	# Ensure that my-blob and other-blob are in separate packfiles.
 	for idx in http_child/.git/objects/pack/*.idx
 	do
-		git verify-pack --verbose $idx >out &&
+		git verify-pack --object-format=$(test_oid algo) --verbose $idx >out &&
 		{
 			grep "^[0-9a-f]\{16,\} " out || :
 		} >out.objectlist &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 31/39] bundle: add new version for use with SHA-256
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (29 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29  2:53   ` Eric Sunshine
  2020-07-28 23:34 ` [PATCH v5 32/39] setup: add support for reading extensions.objectformat brian m. carlson
                   ` (8 subsequent siblings)
  39 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Currently we detect the hash algorithm in use by the length of the
object ID.  This is inelegant and prevents us from using a different
hash algorithm that is also 256 bits in length.

Since we cannot extend the v2 format in a backward-compatible way, let's
add a v3 format, which is identical, except for the addition of
capabilities, which are prefixed by an at sign.  We add "object-format"
as the only capability and reject unknown capabilities, since we do not
have a network connection and therefore cannot negotiate with the other
side.

For compatibility, default to the v2 format for SHA-1 and require v3
for SHA-256.

In t5510, always use format v3 so we can be sure we produce consistent
results across hash algorithms.  Since head -n N lists the top N lines
instead of the Nth line, let's run our output through sed to normalize
it and compare it against a fixed value, which will make sure we get
exactly what we're expecting.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-bundle.txt              |  9 ++-
 Documentation/technical/bundle-format.txt | 30 +++++++-
 builtin/bundle.c                          |  5 +-
 bundle.c                                  | 83 +++++++++++++++++------
 bundle.h                                  |  4 +-
 t/t5510-fetch.sh                          | 16 +++--
 t/t5607-clone-bundle.sh                   | 31 +++++++++
 7 files changed, 147 insertions(+), 31 deletions(-)

diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index d34b0964be..53804cad4b 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -9,7 +9,8 @@ git-bundle - Move objects and refs by archive
 SYNOPSIS
 --------
 [verse]
-'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied] <file> <git-rev-list-args>
+'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied]
+		    [--version=<version>] <file> <git-rev-list-args>
 'git bundle' verify [-q | --quiet] <file>
 'git bundle' list-heads <file> [<refname>...]
 'git bundle' unbundle <file> [<refname>...]
@@ -102,6 +103,12 @@ unbundle <file>::
 	is activated.  Unlike --all-progress this flag doesn't actually
 	force any progress display by itself.
 
+--version=<version>::
+	Specify the bundle version.  Version 2 is the older format and can only be
+	used with SHA-1 repositories; the newer version 3 contains capabilities that
+	permit extensions. The default is the oldest supported format, based on the
+	hash algorithm in use.
+
 -q::
 --quiet::
 	This flag makes the command not to report its progress
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index 0e828151a5..bac558d049 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -7,6 +7,8 @@ The Git bundle format is a format that represents both refs and Git objects.
 We will use ABNF notation to define the Git bundle format. See
 protocol-common.txt for the details.
 
+A v2 bundle looks like this:
+
 ----
 bundle    = signature *prerequisite *reference LF pack
 signature = "# v2 git bundle" LF
@@ -18,9 +20,28 @@ reference    = obj-id SP refname LF
 pack         = ... ; packfile
 ----
 
+A v3 bundle looks like this:
+
+----
+bundle    = signature *capability *prerequisite *reference LF pack
+signature = "# v3 git bundle" LF
+
+capability   = "@" key ["=" value] LF
+prerequisite = "-" obj-id SP comment LF
+comment      = *CHAR
+reference    = obj-id SP refname LF
+key          = 1*(ALPHA / DIGIT / "-")
+value        = *(%01-09 / %0b-FF)
+
+pack         = ... ; packfile
+----
+
 == Semantics
 
-A Git bundle consists of three parts.
+A Git bundle consists of several parts.
+
+* "Capabilities", which are only in the v3 format, indicate functionality that
+	the bundle requires to be read properly.
 
 * "Prerequisites" lists the objects that are NOT included in the bundle and the
   reader of the bundle MUST already have, in order to use the data in the
@@ -46,3 +67,10 @@ put any string here. The reader of the bundle MUST ignore the comment.
 Note that the prerequisites does not represent a shallow-clone boundary. The
 semantics of the prerequisites and the shallow-clone boundaries are different,
 and the Git bundle v2 format cannot represent a shallow clone repository.
+
+== Capabilities
+
+Because there is no opportunity for negotiation, unknown capabilities cause 'git
+bundle' to abort.  The only known capability is `object-format`, which specifies
+the hash algorithm in use, and can take the same values as the
+`extensions.objectFormat` configuration value.
diff --git a/builtin/bundle.c b/builtin/bundle.c
index f049d27a14..e1a85e7dcc 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -60,6 +60,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 	int all_progress_implied = 0;
 	int progress = isatty(STDERR_FILENO);
 	struct argv_array pack_opts;
+	int version = -1;
 
 	struct option options[] = {
 		OPT_SET_INT('q', "quiet", &progress,
@@ -71,6 +72,8 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 		OPT_BOOL(0, "all-progress-implied",
 			 &all_progress_implied,
 			 N_("similar to --all-progress when progress meter is shown")),
+		OPT_INTEGER(0, "version", &version,
+			    N_("specify bundle format version")),
 		OPT_END()
 	};
 	const char* bundle_file;
@@ -91,7 +94,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 
 	if (!startup_info->have_repository)
 		die(_("Need a repository to create a bundle."));
-	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts);
+	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts, version);
 }
 
 static int cmd_bundle_verify(int argc, const char **argv, const char *prefix) {
diff --git a/bundle.c b/bundle.c
index 2a0d744d3f..35585f237c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -12,7 +12,16 @@
 #include "refs.h"
 #include "argv-array.h"
 
-static const char bundle_signature[] = "# v2 git bundle\n";
+
+static const char v2_bundle_signature[] = "# v2 git bundle\n";
+static const char v3_bundle_signature[] = "# v3 git bundle\n";
+static struct {
+	int version;
+	const char *signature;
+} bundle_sigs[] = {
+	{ 2, v2_bundle_signature },
+	{ 3, v3_bundle_signature },
+};
 
 static void add_to_ref_list(const struct object_id *oid, const char *name,
 		struct ref_list *list)
@@ -23,15 +32,30 @@ static void add_to_ref_list(const struct object_id *oid, const char *name,
 	list->nr++;
 }
 
-static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf)
+static int parse_capability(struct bundle_header *header, const char *capability)
 {
-	size_t len = strcspn(buf->buf, " \n");
-	int algo;
+	const char *arg;
+	if (skip_prefix(capability, "object-format=", &arg)) {
+		int algo = hash_algo_by_name(arg);
+		if (algo == GIT_HASH_UNKNOWN)
+			return error(_("unrecognized bundle hash algorithm: %s"), arg);
+		header->hash_algo = &hash_algos[algo];
+		return 0;
+	}
+	return error(_("unknown capability '%s'"), capability);
+}
 
-	algo = hash_algo_by_length(len / 2);
-	if (algo == GIT_HASH_UNKNOWN)
-		return NULL;
-	return &hash_algos[algo];
+static int parse_bundle_signature(struct bundle_header *header, const char *line)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(bundle_sigs); i++) {
+		if (!strcmp(line, bundle_sigs[i].signature)) {
+			header->version = bundle_sigs[i].version;
+			return 0;
+		}
+	}
+	return -1;
 }
 
 static int parse_bundle_header(int fd, struct bundle_header *header,
@@ -42,14 +66,16 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 
 	/* The bundle header begins with the signature */
 	if (strbuf_getwholeline_fd(&buf, fd, '\n') ||
-	    strcmp(buf.buf, bundle_signature)) {
+	    parse_bundle_signature(header, buf.buf)) {
 		if (report_path)
-			error(_("'%s' does not look like a v2 bundle file"),
+			error(_("'%s' does not look like a v2 or v3 bundle file"),
 			      report_path);
 		status = -1;
 		goto abort;
 	}
 
+	header->hash_algo = the_hash_algo;
+
 	/* The bundle header ends with an empty line */
 	while (!strbuf_getwholeline_fd(&buf, fd, '\n') &&
 	       buf.len && buf.buf[0] != '\n') {
@@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 		int is_prereq = 0;
 		const char *p;
 
-		if (*buf.buf == '-') {
-			is_prereq = 1;
-			strbuf_remove(&buf, 0, 1);
-		}
 		strbuf_rtrim(&buf);
 
-		if (!header->hash_algo) {
-			header->hash_algo = detect_hash_algo(&buf);
-			if (!header->hash_algo) {
-				error(_("unknown hash algorithm length"));
+		if (header->version == 3 && *buf.buf == '@') {
+			if (parse_capability(header, buf.buf + 1)) {
 				status = -1;
 				break;
 			}
+			continue;
+		}
+
+		if (*buf.buf == '-') {
+			is_prereq = 1;
+			strbuf_remove(&buf, 0, 1);
 		}
 
 		/*
@@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 }
 
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options)
+		  int argc, const char **argv, struct argv_array *pack_options, int version)
 {
 	struct lock_file lock = LOCK_INIT;
 	int bundle_fd = -1;
 	int bundle_to_stdout;
 	int ref_count = 0;
 	struct rev_info revs;
+	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
 
 	bundle_to_stdout = !strcmp(path, "-");
 	if (bundle_to_stdout)
@@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
 		bundle_fd = hold_lock_file_for_update(&lock, path,
 						      LOCK_DIE_ON_ERROR);
 
-	/* write signature */
-	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
+	if (version == -1)
+		version = min_version;
+
+	if (version < 2 || version > 3) {
+		die(_("unsupported bundle version %d"), version);
+	} else if (version < min_version) {
+		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
+	} else if (version == 2) {
+		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
+	} else {
+		const char *capability = "@object-format=";
+		write_or_die(bundle_fd, v3_bundle_signature, strlen(v3_bundle_signature));
+		write_or_die(bundle_fd, capability, strlen(capability));
+		write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name));
+		write_or_die(bundle_fd, "\n", 1);
+	}
 
 	/* init revs to list objects for pack-objects later */
 	save_commit_buffer = 0;
diff --git a/bundle.h b/bundle.h
index 2dc9442024..70c84cab08 100644
--- a/bundle.h
+++ b/bundle.h
@@ -13,6 +13,7 @@ struct ref_list {
 };
 
 struct bundle_header {
+	unsigned version;
 	struct ref_list prerequisites;
 	struct ref_list references;
 	const struct git_hash_algo *hash_algo;
@@ -21,7 +22,8 @@ struct bundle_header {
 int is_bundle(const char *path, int quiet);
 int read_bundle_header(const char *path, struct bundle_header *header);
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options);
+		  int argc, const char **argv, struct argv_array *pack_options,
+		  int version);
 int verify_bundle(struct repository *r, struct bundle_header *header, int verbose);
 #define BUNDLE_VERBOSE 1
 int unbundle(struct repository *r, struct bundle_header *header,
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index a66dbe0bde..9243335ab0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -281,15 +281,19 @@ test_expect_success 'create bundle 1' '
 	cd "$D" &&
 	echo >file updated again by origin &&
 	git commit -a -m "tip" &&
-	git bundle create bundle1 master^..master
+	git bundle create --version=3 bundle1 master^..master
 '
 
 test_expect_success 'header of bundle looks right' '
-	head -n 4 "$D"/bundle1 &&
-	head -n 1 "$D"/bundle1 | grep "^#" &&
-	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
-	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
-	head -n 4 "$D"/bundle1 | grep "^$"
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	-OID updated by origin
+	OID refs/heads/master
+
+	EOF
+	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'create bundle 2' '
diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh
index 6d5a977fcb..26985f4b44 100755
--- a/t/t5607-clone-bundle.sh
+++ b/t/t5607-clone-bundle.sh
@@ -4,6 +4,10 @@ test_description='some bundle related tests'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_cache <<-EOF &&
+	version sha1:2
+	version sha256:3
+	EOF
 	test_commit initial &&
 	test_tick &&
 	git tag -m tag tag &&
@@ -94,4 +98,31 @@ test_expect_success 'fetch SHA-1 from bundle' '
 	git fetch --no-tags foo/tip.bundle "$(cat hash)"
 '
 
+test_expect_success 'git bundle uses expected default format' '
+	git bundle create bundle HEAD^.. &&
+	head -n1 bundle | grep "^# v$(test_oid version) git bundle$"
+'
+
+test_expect_success 'git bundle v3 has expected contents' '
+	git branch side HEAD &&
+	git bundle create --version=3 bundle HEAD^..side &&
+	head -n2 bundle >actual &&
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	EOF
+	test_cmp expect actual &&
+	git bundle verify bundle
+'
+
+test_expect_success 'git bundle v3 rejects unknown capabilities' '
+	cat >new <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	@unknown=silly
+	EOF
+	test_must_fail git bundle verify new 2>output &&
+	test_i18ngrep "unknown capability .unknown=silly." output
+'
+
 test_done

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 32/39] setup: add support for reading extensions.objectformat
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (30 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 31/39] bundle: add new version for use with SHA-256 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 33/39] Enable SHA-256 support by default brian m. carlson
                   ` (7 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The transition plan specifies extensions.objectFormat as the indication
that we're using a given hash in a certain repo.  Read this as one of
the extensions we support.  If the user has specified an invalid value,
fail.

Ensure that we reject the extension if the repository format version is
0.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 setup.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 3a81307602..94e68bb4f4 100644
--- a/setup.c
+++ b/setup.c
@@ -470,7 +470,16 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			data->partial_clone = xstrdup(value);
 		} else if (!strcmp(ext, "worktreeconfig"))
 			data->worktree_config = git_config_bool(var, value);
-		else
+		else if (!strcmp(ext, "objectformat")) {
+			int format;
+
+			if (!value)
+				return config_error_nonbool(var);
+			format = hash_algo_by_name(value);
+			if (format == GIT_HASH_UNKNOWN)
+				return error("invalid value for 'extensions.objectformat'");
+			data->hash_algo = format;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
 	}
 
@@ -613,6 +622,11 @@ int verify_repository_format(const struct repository_format *format,
 		return -1;
 	}
 
+	if (format->version <= 0 && format->hash_algo != GIT_HASH_SHA1) {
+		strbuf_addstr(err, _("extensions.objectFormat is not valid in repo v0"));
+		return -1;
+	}
+
 	return 0;
 }
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 33/39] Enable SHA-256 support by default
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (31 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 32/39] setup: add support for reading extensions.objectformat brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29  2:57   ` Eric Sunshine
  2020-07-28 23:34 ` [PATCH v5 34/39] t: add test_oid option to select hash algorithm brian m. carlson
                   ` (6 subsequent siblings)
  39 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that the we have a complete SHA-256 implementation in Git, let's
enable it so people can use it.  Remove the ENABLE_SHA256 define
constant everywhere it's used.  Add tests for initializing a repository
with SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/init-db.c |  5 -----
 config.mak.dev    |  2 --
 repository.c      |  4 ----
 t/t0001-init.sh   | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 33 insertions(+), 11 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index cee64823cb..f70076d38e 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -183,11 +183,6 @@ void initialize_repository_version(int hash_algo)
 	char repo_version_string[10];
 	int repo_version = GIT_REPO_VERSION;
 
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), hash_algos[hash_algo].name);
-#endif
-
 	if (hash_algo != GIT_HASH_SHA1)
 		repo_version = GIT_REPO_VERSION_READ;
 
diff --git a/config.mak.dev b/config.mak.dev
index cd4a82a9eb..89b218d11a 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -16,8 +16,6 @@ DEVELOPER_CFLAGS += -Wstrict-prototypes
 DEVELOPER_CFLAGS += -Wunused
 DEVELOPER_CFLAGS += -Wvla
 
-DEVELOPER_CFLAGS += -DENABLE_SHA256
-
 ifndef COMPILER_FEATURES
 COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
 endif
diff --git a/repository.c b/repository.c
index 6f7f6f002b..a4174ddb06 100644
--- a/repository.c
+++ b/repository.c
@@ -89,10 +89,6 @@ void repo_set_gitdir(struct repository *repo,
 void repo_set_hash_algo(struct repository *repo, int hash_algo)
 {
 	repo->hash_algo = &hash_algos[hash_algo];
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), repo->hash_algo->name);
-#endif
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 6d2467995e..d71d4c7238 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -441,6 +441,39 @@ test_expect_success 're-init from a linked worktree' '
 	)
 '
 
+test_expect_success 'init honors GIT_DEFAULT_HASH' '
+	GIT_DEFAULT_HASH=sha1 git init sha1 &&
+	git -C sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	GIT_DEFAULT_HASH=sha256 git init sha256 &&
+	git -C sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'init honors --object-format' '
+	git init --object-format=sha1 explicit-sha1 &&
+	git -C explicit-sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	git init --object-format=sha256 explicit-sha256 &&
+	git -C explicit-sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'extensions.objectFormat is not allowed with repo version 0' '
+	git init --object-format=sha256 explicit-v0 &&
+	git -C explicit-v0 config core.repositoryformatversion 0 &&
+	test_must_fail git -C explicit-v0 rev-parse --show-object-format
+'
+
+test_expect_success 'init rejects attempts to initialize with different hash' '
+	test_must_fail git -C sha1 init --object-format=sha256 &&
+	test_must_fail git -C sha256 init --object-format=sha1
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 34/39] t: add test_oid option to select hash algorithm
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (32 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 33/39] Enable SHA-256 support by default brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 35/39] t: allow testing different hash algorithms via environment brian m. carlson
                   ` (5 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In some tests, we have data files which are written with a particular
hash algorithm. Instead of keeping two copies of the test files, we can
keep one, and translate the value on the fly.

In order to do so, we'll need to read both the source algorithm and the
current algorithm, so add an optional flag to the test_oid helper that
lets us look up a value for a specified hash algorithm. This should
not cause any conflicts with existing tests, since key arguments to
test_oid are allowed to contains only shell identifier characters.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0000-basic.sh        | 11 +++++++++++
 t/test-lib-functions.sh | 12 +++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2ff176cd5d..2599d026a2 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -928,6 +928,17 @@ test_expect_success 'test_oid can look up data for SHA-256' '
 	test "$hexsz" -eq 64
 '
 
+test_expect_success 'test_oid can look up data for a specified algorithm' '
+	rawsz="$(test_oid --hash=sha1 rawsz)" &&
+	hexsz="$(test_oid --hash=sha1 hexsz)" &&
+	test "$rawsz" -eq 20 &&
+	test "$hexsz" -eq 40 &&
+	rawsz="$(test_oid --hash=sha256 rawsz)" &&
+	hexsz="$(test_oid --hash=sha256 hexsz)" &&
+	test "$rawsz" -eq 32 &&
+	test "$hexsz" -eq 64
+'
+
 test_expect_success 'test_bool_env' '
 	(
 		sane_unset envvar &&
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 3103be8a32..2608e80f11 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1468,7 +1468,17 @@ test_oid_cache () {
 # Look up a per-hash value based on a key ($1).  The value must have been loaded
 # by test_oid_init or test_oid_cache.
 test_oid () {
-	local var="test_oid_${test_hash_algo}_$1" &&
+	local algo="${test_hash_algo}" &&
+
+	case "$1" in
+	--hash=*)
+		algo="${1#--hash=}" &&
+		shift;;
+	*)
+		;;
+	esac &&
+
+	local var="test_oid_${algo}_$1" &&
 
 	# If the variable is unset, we must be missing an entry for this
 	# key-hash pair, so exit with an error.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 35/39] t: allow testing different hash algorithms via environment
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (33 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 34/39] t: add test_oid option to select hash algorithm brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
                   ` (4 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

To allow developers to run the testsuite with a different algorithm than
the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to
specify the algorithm to use. Compute the fixed constants using
test_oid. Move the constant initialization down below the point where
test-lib-functions.sh is loaded so the functions are defined.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib-functions.sh | 4 +---
 t/test-lib.sh           | 3 +++
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 2608e80f11..2e01bb2c2b 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1417,9 +1417,7 @@ test_set_hash () {
 
 # Detect the hash algorithm in use.
 test_detect_hash () {
-	# Currently we only support SHA-1, but in the future this function will
-	# actually detect the algorithm in use.
-	test_hash_algo='sha1'
+	test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}"
 }
 
 # Load common hash metadata and common placeholder object IDs for use with
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 618a7c8d5b..307bb2207e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -452,6 +452,9 @@ export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
 export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
 export EDITOR
 
+GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-sha1}"
+export GIT_DEFAULT_HASH
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (34 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 35/39] t: allow testing different hash algorithms via environment brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29  3:01   ` Eric Sunshine
  2020-07-28 23:34 ` [PATCH v5 37/39] ci: run tests with SHA-256 brian m. carlson
                   ` (3 subsequent siblings)
  39 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Currently, the SHA1 prerequisite depends on the output of git
hash-object.  However, in order for that to produce sane behavior, we
must be in a repository.  If we are not, the default will remain SHA-1,
and we'll produce wrong results if we're using SHA-256 for the testsuite
but the test assertion starts when we're not in a repository.

Check the environment variable we use for this purpose, leaving it to
default to SHA-1 if none is specified.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 307bb2207e..b278d37d12 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
 # which will not work with other hash algorithms and tests that work but don't
 # test anything meaningful (e.g. special values which cause short collisions).
 test_lazy_prereq SHA1 '
-	test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+	case "$GIT_DEFAULT_HASH" in
+	sha1) true ;;
+	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
+	*) false ::
+	esac
 '
 
 test_lazy_prereq REBASE_P '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 37/39] ci: run tests with SHA-256
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (35 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
                   ` (2 subsequent siblings)
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that we have Git supporting SHA-256, we'd like to make sure that we
don't regress that state.  Unfortunately, it's easy to do so, so to
help, let's add code to run one of our CI jobs with SHA-256 as the
default hash.  This will help us detect any problems that may occur.

We pick the linux-clang job because it's relatively fast and the
linux-gcc job already runs the testsuite twice.  We want our tests to
run as fast as possible, so we wouldn't want to add a third run to the
linux-gcc job.  To make sure we properly exercise the code, let's run
the tests in the default mode (SHA-1) first and then run a second time
with SHA-256.  We explicitly specify SHA-1 for the first run so that if
we change the default in the future, we make sure to test both cases.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 ci/run-build-and-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 17e25aade9..6c27b886b8 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -24,6 +24,12 @@ linux-gcc)
 	export GIT_TEST_ADD_I_USE_BUILTIN=1
 	make test
 	;;
+linux-clang)
+	export GIT_TEST_DEFAULT_HASH=sha1
+	make test
+	export GIT_TEST_DEFAULT_HASH=sha256
+	make test
+	;;
 linux-gcc-4.8)
 	# Don't run the tests; we only care about whether Git can be
 	# built with GCC 4.8, as it errors out on some undesired (C99)

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 38/39] docs: add documentation for extensions.objectFormat
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (36 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 37/39] ci: run tests with SHA-256 brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-28 23:34 ` [PATCH v5 39/39] t: remove test_oid_init in tests brian m. carlson
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Document the extensions.objectFormat config setting.  Warn users not to
modify it themselves.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/config.txt            | 2 ++
 Documentation/config/extensions.txt | 8 ++++++++
 2 files changed, 10 insertions(+)
 create mode 100644 Documentation/config/extensions.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index ef0768b91a..3042d80978 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -348,6 +348,8 @@ include::config/diff.txt[]
 
 include::config/difftool.txt[]
 
+include::config/extensions.txt[]
+
 include::config/fastimport.txt[]
 
 include::config/feature.txt[]
diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt
new file mode 100644
index 0000000000..4e23d73cdc
--- /dev/null
+++ b/Documentation/config/extensions.txt
@@ -0,0 +1,8 @@
+extensions.objectFormat::
+	Specify the hash algorithm to use.  The acceptable values are `sha1` and
+	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
+	this key unless `core.repositoryFormatVersion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1].  Trying to change it after initialization will not
+work and will produce hard-to-diagnose issues.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v5 39/39] t: remove test_oid_init in tests
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (37 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
@ 2020-07-28 23:34 ` brian m. carlson
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
  39 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-28 23:34 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that we call test_oid_init in the setup for all test scripts,
there's no point in calling it individually.  Remove all of the places
where we've done so to help keep tests tidy.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                          | 3 ---
 t/lib-submodule-update.sh              | 1 -
 t/t0000-basic.sh                       | 4 ----
 t/t1006-cat-file.sh                    | 2 --
 t/t1050-large.sh                       | 1 -
 t/t1410-reflog.sh                      | 1 -
 t/t1450-fsck.sh                        | 1 -
 t/t1500-rev-parse.sh                   | 1 -
 t/t3305-notes-fanout.sh                | 1 -
 t/t3308-notes-merge.sh                 | 1 -
 t/t3404-rebase-interactive.sh          | 1 -
 t/t3600-rm.sh                          | 1 -
 t/t3800-mktag.sh                       | 1 -
 t/t4002-diff-basic.sh                  | 2 --
 t/t4027-diff-submodule.sh              | 1 -
 t/t4134-apply-submodule.sh             | 1 -
 t/t4200-rerere.sh                      | 1 -
 t/t4211-line-log.sh                    | 1 -
 t/t5300-pack-object.sh                 | 3 +--
 t/t5302-pack-index.sh                  | 1 -
 t/t5313-pack-bounds-checks.sh          | 1 -
 t/t5318-commit-graph.sh                | 3 +--
 t/t5319-multi-pack-index.sh            | 1 -
 t/t5324-split-commit-graph.sh          | 1 -
 t/t5504-fetch-receive-strict.sh        | 1 -
 t/t5530-upload-pack-error.sh           | 1 -
 t/t5562-http-backend-content-length.sh | 1 -
 t/t5702-protocol-v2.sh                 | 1 -
 t/t5703-upload-pack-ref-in-want.sh     | 1 -
 t/t6006-rev-list-format.sh             | 1 -
 t/t6301-for-each-ref-errors.sh         | 1 -
 t/t6501-freshen-objects.sh             | 4 ----
 t/t7003-filter-branch.sh               | 1 -
 t/t8003-blame-corner-cases.sh          | 2 --
 t/t9301-fast-import-notes.sh           | 1 -
 t/t9350-fast-export.sh                 | 2 --
 36 files changed, 2 insertions(+), 50 deletions(-)

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index 0c799d53c6..bb8938ccbe 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -35,8 +35,6 @@ pack_header () {
 # have hardcoded some well-known objects. See the case statements below for the
 # complete list.
 pack_obj () {
-	test_oid_init
-
 	case "$1" in
 	# empty blob
 	$EMPTY_BLOB)
@@ -121,7 +119,6 @@ pack_obj () {
 
 # Compute and append pack trailer to "$1"
 pack_trailer () {
-	test_oid_init &&
 	test-tool $(test_oid algo) -b <"$1" >trailer.tmp &&
 	cat trailer.tmp >>"$1" &&
 	rm -f trailer.tmp
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 07c822c8ff..87a759149f 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -196,7 +196,6 @@ test_git_directory_exists () {
 # the submodule repo if it doesn't exist and configures the most problematic
 # settings for diff.ignoreSubmodules.
 prolog () {
-	test_oid_init &&
 	(test -d submodule_update_repo || create_lib_submodule_repo) &&
 	test_config_global diff.ignoreSubmodules all &&
 	test_config diff.ignoreSubmodules all
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2599d026a2..362feb43e6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -891,10 +891,6 @@ test_expect_success 'test_atexit is run' "
 	test_path_is_missing also-clean-atexit
 "
 
-test_expect_success 'test_oid setup' '
-	test_oid_init
-'
-
 test_expect_success 'test_oid provides sane info by default' '
 	test_oid zero >actual &&
 	grep "^00*\$" actual &&
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 43c4be1e5e..2f501d2dc9 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -140,8 +140,6 @@ test_expect_success '--batch-check without %(rest) considers whole line' '
 	test_cmp expect actual
 '
 
-test_oid_init
-
 tree_sha1=$(git write-tree)
 tree_size=$(($(test_oid rawsz) + 13))
 tree_pretty_content="100644 blob $hello_sha1	hello"
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 6a56d1ca24..61e89a8071 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -12,7 +12,6 @@ file_size () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	# clone does not allow us to pass core.bigfilethreshold to
 	# new repos, so set core.bigfilethreshold globally
 	git config --global core.bigfilethreshold 200k &&
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index 76d9b744a6..730a43d9dd 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -54,7 +54,6 @@ check_dont_have () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	mkdir -p A/B &&
 	echo rat >C &&
 	echo ox >A/D &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 344a2aad82..e81f1cbf0e 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -9,7 +9,6 @@ test_description='git fsck random collection of tests
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	git config gc.auto 0 &&
 	git config i18n.commitencoding ISO-8859-1 &&
 	test_commit A fileA one &&
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 603019b541..408b97d5af 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -59,7 +59,6 @@ test_rev_parse () {
 ROOT=$(pwd)
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	mkdir -p sub/dir work &&
 	cp -R .git repo.git
 '
diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index d5a94b7174..94c1b02251 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -31,7 +31,6 @@ all_notes_have_fanout() {
 }
 
 test_expect_success 'tweak test environment' '
-	test_oid_init &&
 	git checkout -b nondeterminism &&
 	test_commit A &&
 	git checkout --orphan with_notes;
diff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh
index 790e292966..d69c84c640 100755
--- a/t/t3308-notes-merge.sh
+++ b/t/t3308-notes-merge.sh
@@ -22,7 +22,6 @@ test_expect_success setup '
 	# Copy notes to remote-notes
 	git fetch . refs/notes/*:refs/remote-notes/origin/* &&
 
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
 	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 2fb2c581cb..9744e88760 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -30,7 +30,6 @@ Initial setup:
 . "$TEST_DIRECTORY"/lib-rebase.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_commit A file1 &&
 	test_commit B file1 &&
 	test_commit C file2 &&
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index f2c0168941..efec8d13b6 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -241,7 +241,6 @@ test_expect_success 'refresh index before checking if it is up-to-date' '
 '
 
 test_expect_success 'choking "git rm" should not let it die with cruft' '
-	test_oid_init &&
 	git reset -q --hard &&
 	test_when_finished "rm -f .git/index.lock && git reset -q --hard" &&
 	i=0 &&
diff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh
index 64dcc5ec28..d696aa4e52 100755
--- a/t/t3800-mktag.sh
+++ b/t/t3800-mktag.sh
@@ -23,7 +23,6 @@ check_verify_failure () {
 # first create a commit, so we have a valid object/type
 # for the tag.
 test_expect_success 'setup' '
-	test_oid_init &&
 	echo Hello >A &&
 	git update-index --add A &&
 	git commit -m "Initial commit" &&
diff --git a/t/t4002-diff-basic.sh b/t/t4002-diff-basic.sh
index cbcdd10464..6a9f010197 100755
--- a/t/t4002-diff-basic.sh
+++ b/t/t4002-diff-basic.sh
@@ -10,8 +10,6 @@ test_description='Test diff raw-output.
 
 . "$TEST_DIRECTORY"/lib-read-tree-m-3way.sh
 
-test_oid_init
-
 test_oid_cache <<\EOF
 aa_1 sha1:ccba72ad3888a3520b39efcf780b9ee64167535d
 aa_1 sha256:9febfbf18197819b2735c45291f138525d2476d59470f98239647544586ba403
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index e29deaf4a5..d7145ccca4 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -6,7 +6,6 @@ test_description='difference in submodules'
 . "$TEST_DIRECTORY"/diff-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	test_tick &&
 	test_create_repo sub &&
 	(
diff --git a/t/t4134-apply-submodule.sh b/t/t4134-apply-submodule.sh
index 99ed4cc546..d1c16ba33c 100755
--- a/t/t4134-apply-submodule.sh
+++ b/t/t4134-apply-submodule.sh
@@ -8,7 +8,6 @@ test_description='git apply submodule tests'
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	cat > create-sm.patch <<EOF &&
 diff --git a/dir/sm b/dir/sm
 new file mode 160000
diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh
index 831d424c47..d94d25e4f0 100755
--- a/t/t4200-rerere.sh
+++ b/t/t4200-rerere.sh
@@ -25,7 +25,6 @@ test_description='git rerere
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	cat >a1 <<-\EOF &&
 	Some title
 	==========
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index e186c83250..2d1d7b5d19 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -4,7 +4,6 @@ test_description='test log -L'
 . ./test-lib.sh
 
 test_expect_success 'setup (import history)' '
-	test_oid_init &&
 	git fast-import < "$TEST_DIRECTORY"/t4211/history.export &&
 	git reset --hard
 '
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 746cdb626e..cf9f91c941 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -12,8 +12,7 @@ TRASH=$(pwd)
 
 test_expect_success \
     'setup' \
-    'test_oid_init &&
-     rm -f .git/index* &&
+    'rm -f .git/index* &&
      perl -e "print \"a\" x 4096;" > a &&
      perl -e "print \"b\" x 4096;" > b &&
      perl -e "print \"c\" x 4096;" > c &&
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 8981c9b90e..119494bd64 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -7,7 +7,6 @@ test_description='pack index with 64-bit offsets and object CRC'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	rawsz=$(test_oid rawsz) &&
 	rm -rf .git &&
 	git init &&
diff --git a/t/t5313-pack-bounds-checks.sh b/t/t5313-pack-bounds-checks.sh
index 2a4557efc2..535313e4dc 100755
--- a/t/t5313-pack-bounds-checks.sh
+++ b/t/t5313-pack-bounds-checks.sh
@@ -45,7 +45,6 @@ extended_table () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	oid000 sha1:1485
 	oid000 sha256:4222
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 26f332d6a3..67f2663c7c 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -10,8 +10,7 @@ test_expect_success 'setup full repo' '
 	cd "$TRASH_DIRECTORY/full" &&
 	git init &&
 	git config core.commitGraph true &&
-	objdir=".git/objects" &&
-	test_oid_init
+	objdir=".git/objects"
 '
 
 test_expect_success POSIXPERM 'tweak umask for modebit tests' '
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 7214cab36c..7dfff0f8f4 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -29,7 +29,6 @@ midx_read_expect () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	idxoff sha1:2999
 	idxoff sha256:3739
diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index 269d0964a3..3a63639fbc 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -12,7 +12,6 @@ test_expect_success 'setup repo' '
 	git config gc.writeCommitGraph false &&
 	infodir=".git/objects/info" &&
 	graphdir="$infodir/commit-graphs" &&
-	test_oid_init &&
 	test_oid_cache <<-EOM
 	shallow sha1:1760
 	shallow sha256:2064
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index a32efe2b6c..1a16ac4c0d 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -4,7 +4,6 @@ test_description='fetch/receive strict mode'
 . ./test-lib.sh
 
 test_expect_success 'setup and inject "corrupt or missing" object' '
-	test_oid_init &&
 	echo hello >greetings &&
 	git add greetings &&
 	git commit -m greetings &&
diff --git a/t/t5530-upload-pack-error.sh b/t/t5530-upload-pack-error.sh
index 4ce9a9f704..205a2631e7 100755
--- a/t/t5530-upload-pack-error.sh
+++ b/t/t5530-upload-pack-error.sh
@@ -14,7 +14,6 @@ corrupt_repo () {
 }
 
 test_expect_success 'setup and corrupt repository' '
-	test_oid_init &&
 	echo file >file &&
 	git add file &&
 	git rev-parse :file &&
diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
index c6ec625497..e5d3d15ba8 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -46,7 +46,6 @@ ssize_b100dots() {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	HTTP_CONTENT_ENCODING="identity" &&
 	export HTTP_CONTENT_ENCODING &&
 	git config http.receivepack true &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 7fc22171e7..5a60fbe3ed 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -13,7 +13,6 @@ start_git_daemon --export-all --enable=receive-pack
 daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent
 
 test_expect_success 'create repo to be served by git-daemon' '
-	test_oid_init &&
 	git init "$daemon_parent" &&
 	test_commit -C "$daemon_parent" one
 '
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index 748282f058..d9ecf0f4a9 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -43,7 +43,6 @@ write_command () {
 #             \ | /
 #               a
 test_expect_success 'setup repository' '
-	test_oid_init &&
 	test_commit a &&
 	git checkout -b o/foo &&
 	test_commit b &&
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 7e82e43a63..bc95da8a5f 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -32,7 +32,6 @@ changed_iso88591=$(echo "$changed" | iconv -f utf-8 -t $test_encoding)
 truncate_count=20
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	: >foo &&
 	git add foo &&
 	git config i18n.commitEncoding $test_encoding &&
diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index d545a725a5..809854fc0c 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -7,7 +7,6 @@ test_description='for-each-ref errors for broken refs'
 ZEROS=$ZERO_OID
 
 test_expect_success setup '
-	test_oid_init &&
 	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index 7416ed5ffe..8a3bb4105b 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -45,10 +45,6 @@ maybe_repack () {
 	fi
 }
 
-test_expect_success 'setup' '
-	test_oid_init
-'
-
 for repack in '' true; do
 	title=${repack:+repack}
 	title=${title:-loose}
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index beeb2b855a..36477cb1f4 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -5,7 +5,6 @@ test_description='git filter-branch'
 . "$TEST_DIRECTORY/lib-gpg.sh"
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_commit A &&
 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
 	test_commit --notick B &&
diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index ebe79aa4ea..b871dd4f86 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,8 +6,6 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
-	test_oid_init &&
-
 	echo A A A A A >one &&
 	echo B B B B B >two &&
 	echo C C C C C >tres &&
diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index 9d1aaac980..ff3338abc0 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -71,7 +71,6 @@ EOF
 INPUT_END
 
 test_expect_success 'set up master branch' '
-	test_oid_init &&
 	git fast-import <input &&
 	git whatchanged master
 '
diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index ba79db6a17..210b9bc4b6 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -7,8 +7,6 @@ test_description='git fast-export'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
-
 	echo break it > file0 &&
 	git add file0 &&
 	test_tick &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 18/39] t8002: make hash size independent
  2020-07-28 23:34 ` [PATCH v5 18/39] t8002: make hash size independent brian m. carlson
@ 2020-07-29  2:24   ` Eric Sunshine
  2020-07-29 22:31     ` brian m. carlson
  0 siblings, 1 reply; 116+ messages in thread
From: Eric Sunshine @ 2020-07-29  2:24 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Tue, Jul 28, 2020 at 7:35 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> Compute the length of an object ID instead of hard-coding 40-based
> values.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
> @@ -6,6 +6,10 @@ test_description='git blame'
> +test_expect_success 'setup' '
> +       hexsz=$(test_oid hexsz)
> +'

In the previous version of this series, the "setup" test  invoked
test_oid_init() as its very first step. This version doesn't. As a
reviewer, I was caught off-guard by this unexpected and unexplained
difference between versions. The script works fine without
test_oid_init() anyhow since test-lib.sh invokes test_oid_init(), so
the test_oid_init() call introduced here by the previous version was
redundant.

Some of the patches in this series add test_oid_init() calls to their
"setup" tests, while others don't, which makes for a somewhat
confusing impression as one reads the series. In general, it would be
nice for the patches to paint a consistent picture (i.e either
uniformly employ test_oid_init() or don't), however, I would not want
to see a re-roll just for that. Also, since the final patch of the
series ends up removing all those test_oid_init() calls anyhow, it's
all straightened out in the end.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes
  2020-07-28 23:34 ` [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
@ 2020-07-29  2:36   ` Eric Sunshine
  0 siblings, 0 replies; 116+ messages in thread
From: Eric Sunshine @ 2020-07-29  2:36 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Tue, Jul 28, 2020 at 7:35 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> In dd4b732df7 ("upload-pack: send part of packfile response as uri",
> 2020-06-10), the git http-fetch code learned how to take  ac --packfile
> option.  This option takes an argument, which is the name of a packfile
> hash, and parses it using parse_oid_hex.  It does so before calling
> setup_git_directory.
>
> However, in a SHA-256 repository this fails to work, since we have not
> set the hash algorithm in use and parse_oid_hex fails as a consequence.
> To ensure that we can parse packfile hashes of the right length, let's
> set up the git directory before we start parsing arguments.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/http-fetch.c b/http-fetch.c
> @@ -86,6 +86,8 @@ int cmd_main(int argc, const char **argv)
> +       setup_git_directory_gently(NULL);
> +
>         while (arg < argc && argv[arg][0] == '-') {
>                 const char *p;
> @@ -115,8 +117,6 @@ int cmd_main(int argc, const char **argv)
>         if (argc != arg + 2 - (commits_on_stdin || packfile))
>                 usage(http_fetch_usage);
>
> -       setup_git_directory();

In the previous version of this series the setup_git_directory() call
was moved verbatim above the 'while' loop, and my review pointed out
that doing so made "git-http-fetch -h" fail when not in a Git
repository. It was recommended, therefore, to take advantage of
setup_git_directory_gently() to fix this problem.

But note that setup_git_directory() is defined like this:

    const char *setup_git_directory(void)
    {
        return setup_git_directory_gently(NULL);
    }

So, your use of setup_git_directory_gently(NULL) in this version of
the patch is no better than the plain setup_git_directory() call in
the previous version. It is still impossible to ask git-http-fetch for
help when not in a Git repository.

To fix this, you must pass non-NULL for the 'nongit' argument of
setup_git_directory_gently().

For robustness, in order to retain the original behavior (prior to
this patch), it would be best to check the value of 'nongit' _after_
the 'while' loop and error out if not in a repository. Perhaps
something like this (taking a hint from builtin/diff.c):

    setup_git_directory_gently(&nongit);

    while (...) {
        ...
    }

    if (nongit)
        die(_("not a git repository"));

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 31/39] bundle: add new version for use with SHA-256
  2020-07-28 23:34 ` [PATCH v5 31/39] bundle: add new version for use with SHA-256 brian m. carlson
@ 2020-07-29  2:53   ` Eric Sunshine
  2020-07-29 22:28     ` brian m. carlson
  0 siblings, 1 reply; 116+ messages in thread
From: Eric Sunshine @ 2020-07-29  2:53 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> [...]
> Since we cannot extend the v2 format in a backward-compatible way, let's
> add a v3 format, which is identical, except for the addition of
> capabilities, which are prefixed by an at sign.  We add "object-format"
> as the only capability and reject unknown capabilities, since we do not
> have a network connection and therefore cannot negotiate with the other
> side.
> [...]
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/bundle.c b/bundle.c
> @@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
> -               if (*buf.buf == '-') {
> -                       is_prereq = 1;
> -                       strbuf_remove(&buf, 0, 1);
> -               }
>                 strbuf_rtrim(&buf);
>
> -               if (!header->hash_algo) {
> -                       header->hash_algo = detect_hash_algo(&buf);
> -                       if (!header->hash_algo) {
> -                               error(_("unknown hash algorithm length"));
> +               if (header->version == 3 && *buf.buf == '@') {
> +                       if (parse_capability(header, buf.buf + 1)) {
>                                 status = -1;
>                                 break;
>                         }
> +                       continue;
> +               }
> +
> +               if (*buf.buf == '-') {
> +                       is_prereq = 1;
> +                       strbuf_remove(&buf, 0, 1);
>                 }

Moving the strbuf_rtrim() earlier in the loop, as suggested in my
previous review, made the diff a lot noisier, uglier, and more
difficult to read, however, the code itself ends up being easier to
reason about than in the previous round. Good.

By the way (I didn't think of this in my previous review), but
wouldn't it be better for this:

    if (header->version == 3 && *buf.buf == '@') {

to instead be written as:

    if (header->version >= 3 && *buf.buf == '@') {

to future-proof it since versions beyond 3 will also almost certainly
support "@foo" capabilities?

> @@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
> +       int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
> @@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
> +       if (version == -1)
> +               version = min_version;
> +
> +       if (version < 2 || version > 3) {
> +               die(_("unsupported bundle version %d"), version);
> +       } else if (version < min_version) {
> +               die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
> +       } else if (version == 2) {
> +               write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));

This looks better than the previous attempt; less likely to be fragile
as new bundle versions are introduced. Good.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 33/39] Enable SHA-256 support by default
  2020-07-28 23:34 ` [PATCH v5 33/39] Enable SHA-256 support by default brian m. carlson
@ 2020-07-29  2:57   ` Eric Sunshine
  0 siblings, 0 replies; 116+ messages in thread
From: Eric Sunshine @ 2020-07-29  2:57 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> Subject: Enable SHA-256 support by default

If you happen to re-roll, perhaps: s/Enable/enable/

> Now that the we have a complete SHA-256 implementation in Git, let's

Still wonky grammar here (mentioned previously): "Now that the we have..."

> enable it so people can use it.  Remove the ENABLE_SHA256 define
> constant everywhere it's used.  Add tests for initializing a repository
> with SHA-256.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash
  2020-07-28 23:34 ` [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
@ 2020-07-29  3:01   ` Eric Sunshine
  0 siblings, 0 replies; 116+ messages in thread
From: Eric Sunshine @ 2020-07-29  3:01 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Tue, Jul 28, 2020 at 7:36 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> Currently, the SHA1 prerequisite depends on the output of git
> hash-object.  However, in order for that to produce sane behavior, we
> must be in a repository.  If we are not, the default will remain SHA-1,
> and we'll produce wrong results if we're using SHA-256 for the testsuite
> but the test assertion starts when we're not in a repository.
>
> Check the environment variable we use for this purpose, leaving it to
> default to SHA-1 if none is specified.
>
> Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
> ---
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> @@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
>  test_lazy_prereq SHA1 '
> -       test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
> +       case "$GIT_DEFAULT_HASH" in
> +       sha1) true ;;
> +       "") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
> +       *) false ::
> +       esac
>  '

This deserves a s/::/;;/ in the final case arm. The "::" does work --
in a manner of speaking -- but only "by accident"; merely because it's
being interpreted as an argument to 'false' and it happens to be the
final case arm, but it still ought to be fixed.

(I'm pretty sure I typed ";;", not "::", when I gave this code as an
example in my previous review... {goes and checks...} indeed, I did
type ";;".)

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 31/39] bundle: add new version for use with SHA-256
  2020-07-29  2:53   ` Eric Sunshine
@ 2020-07-29 22:28     ` brian m. carlson
  0 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 22:28 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Git List

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

On 2020-07-29 at 02:53:47, Eric Sunshine wrote:
> Moving the strbuf_rtrim() earlier in the loop, as suggested in my
> previous review, made the diff a lot noisier, uglier, and more
> difficult to read, however, the code itself ends up being easier to
> reason about than in the previous round. Good.

Yeah, I noticed that.  The code is much nicer, so thank you for that
suggestion.

> By the way (I didn't think of this in my previous review), but
> wouldn't it be better for this:
> 
>     if (header->version == 3 && *buf.buf == '@') {
> 
> to instead be written as:
> 
>     if (header->version >= 3 && *buf.buf == '@') {
> 
> to future-proof it since versions beyond 3 will also almost certainly
> support "@foo" capabilities?

I think this will need to be dealt with by the author of bundle v4,
since currently bundle v4 will error out before we get here.  I don't
want to make too many assumptions about a future format.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v5 18/39] t8002: make hash size independent
  2020-07-29  2:24   ` Eric Sunshine
@ 2020-07-29 22:31     ` brian m. carlson
  0 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 22:31 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: Git List

[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]

On 2020-07-29 at 02:24:42, Eric Sunshine wrote:
> In the previous version of this series, the "setup" test  invoked
> test_oid_init() as its very first step. This version doesn't. As a
> reviewer, I was caught off-guard by this unexpected and unexplained
> difference between versions. The script works fine without
> test_oid_init() anyhow since test-lib.sh invokes test_oid_init(), so
> the test_oid_init() call introduced here by the previous version was
> redundant.
> 
> Some of the patches in this series add test_oid_init() calls to their
> "setup" tests, while others don't, which makes for a somewhat
> confusing impression as one reads the series. In general, it would be
> nice for the patches to paint a consistent picture (i.e either
> uniformly employ test_oid_init() or don't), however, I would not want
> to see a re-roll just for that. Also, since the final patch of the
> series ends up removing all those test_oid_init() calls anyhow, it's
> all straightened out in the end.

Good point.  I'll try to remove them from the existing tests which add
them in the rest of the series.
-- 
brian m. carlson: Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v6 00/39] SHA-256, part 3/3
  2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
                   ` (38 preceding siblings ...)
  2020-07-28 23:34 ` [PATCH v5 39/39] t: remove test_oid_init in tests brian m. carlson
@ 2020-07-29 23:13 ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 01/39] t: make test-bloom initialize repository brian m. carlson
                     ` (40 more replies)
  39 siblings, 41 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Changes from v5:
* Remove useless test_oid_init invocations throughout the series.
* Fix a commit message for grammar and style.
* Fix a typo in a shell case statement.
* Fix http-fetch to allow parsing options before failing on a missing
  git directory.

Changes from v4:
* Fix several commit message to fix typos and inaccuracies.
* Remove useless touch statement in tests.
* Adjust several tests to introduce less dependence on previous test
  assertions other than the setup ones.
* Loosen the regexp for object IDs in a Perl script in a test.
* Ensure help functionality works for git http-fetch.
* Switch http-fetch.c to use the already existing struct argv_array
  instance.
* Avoid a memory leak, albeit a minor one.
* Improve error message for unknown bundle hash algorithm.
* Hoist a strbuf_rtrim in the bundle parsing code to simplify things.
* Specify a minimum version instead of a default version in the bundle
  code.
* Remove a useless line in tests.
* Avoid invoking git hash-object needlessly for the SHA1 prerequisite.
* Be explicit about which algorithms we're testing in CI.
* Properly capitalize "core.repositoryFormatVersion" in documentation.

Changes from v2:
* Move bundle signature parsing into a function.
* Improve parsing of bundle capabilities.
* Improve t5510 to select only the interesting lines with sed.
* Use cat and a heredoc instead of multiple echos.
* Use test_i18ngrep where appropriate.
* Reject extensions.objectFormat for repo v0.
* Update documentation accordingly.

Changes from v1:
* Add a patch to add a bundle v3 format with capabilities that include
  the hash algorithm.
* Remove bundle detection of hash algorithm using hash algorithm length.
* Fix typo in the extensions.objectFormat documentation.
* Document that extensions.objectFormat is ignored for repo v0.
* Use test instead of brackets.
* Use a command group instead of a subshell for efficiency.
* Drop the bizarre test_might_fail, since it isn't needed (and is
  bizarre and wrong).
* Write "SHA-256" instead of "NewHash" in commit messages.
* Split SHA-256 CI tests to an independent run and avoid running a third
  test run to a single CI job.

Range-diff below.

Johannes Schindelin (1):
  t3404: prepare 'short SHA-1 collision' tests for SHA-256

brian m. carlson (38):
  t: make test-bloom initialize repository
  t1001: use $ZERO_OID
  t3305: make hash agnostic
  t6100: make hash size independent
  t6101: make hash size independent
  t6301: make hash size independent
  t6500: specify test values for SHA-256
  t6501: avoid hard-coded objects
  t7003: compute appropriate length constant
  t7063: make hash size independent
  t7201: abstract away SHA-1-specific constants
  t7102: abstract away SHA-1-specific constants
  t7400: make hash size independent
  t7405: make hash size independent
  t7506: avoid checking for SHA-1-specific constants
  t7508: use $ZERO_OID instead of hard-coded constant
  t8002: make hash size independent
  t8003: make hash size independent
  t8011: make hash size independent
  t9300: abstract away SHA-1-specific constants
  t9300: use $ZERO_OID instead of hard-coded object ID
  t9301: make hash size independent
  t9350: make hash size independent
  t9500: ensure that algorithm info is preserved in config
  t9700: make hash size independent
  t5308: make test work with SHA-256
  t0410: mark test with SHA1 prerequisite
  http-fetch: set up git directory before parsing pack hashes
  builtin/verify-pack: implement an --object-format option
  bundle: add new version for use with SHA-256
  setup: add support for reading extensions.objectformat
  repository: enable SHA-256 support by default
  t: add test_oid option to select hash algorithm
  t: allow testing different hash algorithms via environment
  t: make SHA1 prerequisite depend on default hash
  ci: run tests with SHA-256
  docs: add documentation for extensions.objectFormat
  t: remove test_oid_init in tests

 Documentation/config.txt                  |   2 +
 Documentation/config/extensions.txt       |   8 ++
 Documentation/git-bundle.txt              |   9 +-
 Documentation/technical/bundle-format.txt |  30 ++++-
 builtin/bundle.c                          |   5 +-
 builtin/init-db.c                         |   5 -
 builtin/verify-pack.c                     |  23 ++--
 bundle.c                                  |  83 +++++++++---
 bundle.h                                  |   4 +-
 ci/run-build-and-tests.sh                 |   6 +
 config.mak.dev                            |   2 -
 http-fetch.c                              |   6 +-
 repository.c                              |   4 -
 setup.c                                   |  16 ++-
 t/helper/test-bloom.c                     |   2 +
 t/lib-pack.sh                             |  11 +-
 t/lib-submodule-update.sh                 |   1 -
 t/t0000-basic.sh                          |  15 ++-
 t/t0001-init.sh                           |  33 +++++
 t/t0410-partial-clone.sh                  |   4 +-
 t/t1006-cat-file.sh                       |   2 -
 t/t1050-large.sh                          |   1 -
 t/t1091-sparse-checkout-builtin.sh        |   4 +-
 t/t1410-reflog.sh                         |   1 -
 t/t1450-fsck.sh                           |   1 -
 t/t1500-rev-parse.sh                      |   1 -
 t/t3305-notes-fanout.sh                   |   2 +-
 t/t3308-notes-merge.sh                    |   1 -
 t/t3404-rebase-interactive.sh             |  49 +++++--
 t/t3600-rm.sh                             |   1 -
 t/t3800-mktag.sh                          |   1 -
 t/t4002-diff-basic.sh                     |   2 -
 t/t4027-diff-submodule.sh                 |   1 -
 t/t4134-apply-submodule.sh                |   1 -
 t/t4200-rerere.sh                         |   1 -
 t/t4211-line-log.sh                       |   1 -
 t/t5300-pack-object.sh                    |   3 +-
 t/t5302-pack-index.sh                     |   1 -
 t/t5308-pack-detect-duplicates.sh         |  20 +--
 t/t5313-pack-bounds-checks.sh             |   1 -
 t/t5318-commit-graph.sh                   |   3 +-
 t/t5319-multi-pack-index.sh               |   1 -
 t/t5324-split-commit-graph.sh             |   1 -
 t/t5504-fetch-receive-strict.sh           |   1 -
 t/t5510-fetch.sh                          |  16 ++-
 t/t5530-upload-pack-error.sh              |   1 -
 t/t5562-http-backend-content-length.sh    |   1 -
 t/t5607-clone-bundle.sh                   |  31 +++++
 t/t5702-protocol-v2.sh                    |   3 +-
 t/t5703-upload-pack-ref-in-want.sh        |   1 -
 t/t6006-rev-list-format.sh                |   1 -
 t/t6100-rev-list-in-order.sh              |   4 +-
 t/t6101-rev-parse-parents.sh              |   2 +-
 t/t6301-for-each-ref-errors.sh            |   2 +-
 t/t6500-gc.sh                             |  27 +++-
 t/t6501-freshen-objects.sh                |  14 +-
 t/t7003-filter-branch.sh                  |   3 +-
 t/t7063-status-untracked-cache.sh         | 150 ++++++++++++----------
 t/t7102-reset.sh                          |  93 ++++++++------
 t/t7201-co.sh                             |   6 +-
 t/t7400-submodule-basic.sh                |  26 ++--
 t/t7405-submodule-merge.sh                |   4 +-
 t/t7506-status-submodule.sh               |  12 +-
 t/t7508-status.sh                         |   2 +-
 t/t8002-blame.sh                          |  18 ++-
 t/t8003-blame-corner-cases.sh             |   3 +-
 t/t8011-blame-split-file.sh               |   2 +-
 t/t9300-fast-import.sh                    | 118 +++++++++--------
 t/t9301-fast-import-notes.sh              |  12 +-
 t/t9350-fast-export.sh                    |  14 +-
 t/t9500-gitweb-standalone-no-errors.sh    |  22 +++-
 t/t9700/test.pl                           |   6 +-
 t/test-lib-functions.sh                   |  16 ++-
 t/test-lib.sh                             |   9 +-
 74 files changed, 638 insertions(+), 351 deletions(-)
 create mode 100644 Documentation/config/extensions.txt

Diff-intervalle contre v5 :
 1:  2d720a6beb =  1:  fb53834bd3 t: make test-bloom initialize repository
 2:  530704736a =  2:  8170f55d65 t1001: use $ZERO_OID
 3:  33e7822d0e !  3:  94b585296d t3305: make hash agnostic
    @@ t/t3305-notes-fanout.sh: test_description='Test that adding/removing many notes
      	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
      }
      
    -@@ t/t3305-notes-fanout.sh: all_notes_have_fanout() {
    - }
    - 
    - test_expect_success 'tweak test environment' '
    -+	test_oid_init &&
    - 	git checkout -b nondeterminism &&
    - 	test_commit A &&
    - 	git checkout --orphan with_notes;
 4:  99081937cf !  4:  82b373b830 t3404: prepare 'short SHA-1 collision' tests for SHA-256
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t3404-rebase-interactive.sh ##
    -@@ t/t3404-rebase-interactive.sh: Initial setup:
    - . "$TEST_DIRECTORY"/lib-rebase.sh
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 	test_commit A file1 &&
    - 	test_commit B file1 &&
    - 	test_commit C file2 &&
     @@ t/t3404-rebase-interactive.sh: test_expect_success 'rebase -i error on commits with \ in message' '
      	test_expect_code 1 grep  "	emp" error
      '
 5:  3cfa8bba75 =  5:  adc32c2903 t6100: make hash size independent
 6:  9a7de5790c =  6:  40db5bb392 t6101: make hash size independent
 7:  76942eec3d !  7:  d4055790f3 t6301: make hash size independent
    @@ Commit message
     
         Instead of hard-coding a fixed length example object ID in the test,
         compute one using the translation tables.  Move a variable into the
    -    setup block so that we can ensure the exit status of test_oid_init is
    +    setup block so that we can ensure the exit status of test_oid is
         checked.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
    @@ t/t6301-for-each-ref-errors.sh: test_description='for-each-ref errors for broken
     -MISSING=abababababababababababababababababababab
      
      test_expect_success setup '
    -+	test_oid_init &&
     +	MISSING=$(test_oid deadbeef) &&
      	git commit --allow-empty -m "Initial" &&
      	git tag testtag &&
 8:  b30a8a6736 =  8:  a9578382ab t6500: specify test values for SHA-256
 9:  c3d68dcf1b !  9:  b022dda379 t6501: avoid hard-coded objects
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t6501-freshen-objects.sh ##
    -@@ t/t6501-freshen-objects.sh: maybe_repack () {
    - 	fi
    - }
    - 
    -+test_expect_success 'setup' '
    -+	test_oid_init
    -+'
    -+
    - for repack in '' true; do
    - 	title=${repack:+repack}
    - 	title=${title:-loose}
     @@ t/t6501-freshen-objects.sh: for repack in '' true; do
      done
      
10:  865662f5c5 ! 10:  d9eda002b9 t7003: compute appropriate length constant
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t7003-filter-branch.sh ##
    -@@ t/t7003-filter-branch.sh: test_description='git filter-branch'
    - . "$TEST_DIRECTORY/lib-gpg.sh"
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 	test_commit A &&
    - 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
    - 	test_commit --notick B &&
     @@ t/t7003-filter-branch.sh: test_expect_success 'rewrite submodule with another content' '
      '
      
11:  1a7bd71fb4 = 11:  02da9e5630 t7063: make hash size independent
12:  55c545cf1f = 12:  4f567ee443 t7201: abstract away SHA-1-specific constants
13:  9875576b08 = 13:  cec9eca4a3 t7102: abstract away SHA-1-specific constants
14:  d1a2f91516 = 14:  0408e53002 t7400: make hash size independent
15:  9acdbf294c = 15:  3f0bb9751d t7405: make hash size independent
16:  2b506517de = 16:  406c9088af t7506: avoid checking for SHA-1-specific constants
17:  976974106c = 17:  bea582c49f t7508: use $ZERO_OID instead of hard-coded constant
18:  459e6d2455 = 18:  bb044354bc t8002: make hash size independent
19:  8c19b1280d ! 19:  888ffd32d3 t8003: make hash size independent
    @@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
      pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
      
      test_expect_success setup '
    -+	test_oid_init &&
    - 
    +-
      	echo A A A A A >one &&
      	echo B B B B B >two &&
    + 	echo C C C C C >tres &&
     @@ t/t8003-blame-corner-cases.sh: test_expect_success 'blame coalesce' '
      	$oid 1) ABC
      	$oid 2) DEF
20:  42e8d962dc = 20:  a6f7345990 t8011: make hash size independent
21:  076249fdea = 21:  25458092e4 t9300: abstract away SHA-1-specific constants
22:  36fed34e25 = 22:  637673d0f4 t9300: use $ZERO_OID instead of hard-coded object ID
23:  fd2b44e138 ! 23:  77a42dbf51 t9301: make hash size independent
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9301-fast-import-notes.sh ##
    -@@ t/t9301-fast-import-notes.sh: EOF
    - INPUT_END
    - 
    - test_expect_success 'set up master branch' '
    --
    -+	test_oid_init &&
    - 	git fast-import <input &&
    - 	git whatchanged master
    - '
     @@ t/t9301-fast-import-notes.sh: test_expect_success 'add lots of commits and notes' '
      '
      
24:  5cf2612131 ! 24:  3c405c54fd t9350: make hash size independent
    @@ Commit message
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## t/t9350-fast-export.sh ##
    -@@ t/t9350-fast-export.sh: test_description='git fast-export'
    - . ./test-lib.sh
    - 
    - test_expect_success 'setup' '
    -+	test_oid_init &&
    - 
    - 	echo break it > file0 &&
    - 	git add file0 &&
     @@ t/t9350-fast-export.sh: test_expect_success 'reencoding iso-8859-7' '
      	sed "s/wer/i18n/" iso-8859-7.fi |
      		(cd new &&
25:  64b56a029a = 25:  c75d879bca t9500: ensure that algorithm info is preserved in config
26:  46800750af = 26:  82f2a6146f t9700: make hash size independent
27:  9c5a7b9c38 = 27:  60d7922842 t5308: make test work with SHA-256
28:  1a98d1e4b6 = 28:  c668823701 t0410: mark test with SHA1 prerequisite
29:  359543b8de <  -:  ---------- http-fetch: set up git directory before parsing pack hashes
 -:  ---------- > 29:  e1e0e04e83 http-fetch: set up git directory before parsing pack hashes
30:  bc5e6f6199 = 30:  96e770dce2 builtin/verify-pack: implement an --object-format option
31:  10f9420219 = 31:  dd79604c8b bundle: add new version for use with SHA-256
32:  59717a9f42 = 32:  425fdcca3d setup: add support for reading extensions.objectformat
33:  8313384a88 ! 33:  e7256281f4 Enable SHA-256 support by default
    @@ Metadata
     Author: brian m. carlson <sandals@crustytoothpaste.net>
     
      ## Commit message ##
    -    Enable SHA-256 support by default
    +    repository: enable SHA-256 support by default
     
    -    Now that the we have a complete SHA-256 implementation in Git, let's
    -    enable it so people can use it.  Remove the ENABLE_SHA256 define
    -    constant everywhere it's used.  Add tests for initializing a repository
    -    with SHA-256.
    +    Now that we have a complete SHA-256 implementation in Git, let's enable
    +    it so people can use it.  Remove the ENABLE_SHA256 define constant
    +    everywhere it's used.  Add tests for initializing a repository with
    +    SHA-256.
     
         Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
     
34:  e8439aa0d0 = 34:  65d43826f6 t: add test_oid option to select hash algorithm
35:  e5c9553da9 = 35:  d32ff6431b t: allow testing different hash algorithms via environment
36:  2a6f2ccd55 ! 36:  8cc8c77f48 t: make SHA1 prerequisite depend on default hash
    @@ t/test-lib.sh: test_lazy_prereq CURL '
     +	case "$GIT_DEFAULT_HASH" in
     +	sha1) true ;;
     +	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
    -+	*) false ::
    ++	*) false ;;
     +	esac
      '
      
37:  2eec901cdf = 37:  01d865b03e ci: run tests with SHA-256
38:  6a7fb3e911 = 38:  83f565d1d2 docs: add documentation for extensions.objectFormat
39:  448faa92fe ! 39:  6c3bf65a7d t: remove test_oid_init in tests
    @@ t/t1500-rev-parse.sh: test_rev_parse () {
      	cp -R .git repo.git
      '
     
    - ## t/t3305-notes-fanout.sh ##
    -@@ t/t3305-notes-fanout.sh: all_notes_have_fanout() {
    - }
    - 
    - test_expect_success 'tweak test environment' '
    --	test_oid_init &&
    - 	git checkout -b nondeterminism &&
    - 	test_commit A &&
    - 	git checkout --orphan with_notes;
    -
      ## t/t3308-notes-merge.sh ##
     @@ t/t3308-notes-merge.sh: test_expect_success setup '
      	# Copy notes to remote-notes
    @@ t/t3308-notes-merge.sh: test_expect_success setup '
      	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
      	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
     
    - ## t/t3404-rebase-interactive.sh ##
    -@@ t/t3404-rebase-interactive.sh: Initial setup:
    - . "$TEST_DIRECTORY"/lib-rebase.sh
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    - 	test_commit A file1 &&
    - 	test_commit B file1 &&
    - 	test_commit C file2 &&
    -
      ## t/t3600-rm.sh ##
     @@ t/t3600-rm.sh: test_expect_success 'refresh index before checking if it is up-to-date' '
      '
    @@ t/t6006-rev-list-format.sh: changed_iso88591=$(echo "$changed" | iconv -f utf-8
      	: >foo &&
      	git add foo &&
      	git config i18n.commitEncoding $test_encoding &&
    -
    - ## t/t6301-for-each-ref-errors.sh ##
    -@@ t/t6301-for-each-ref-errors.sh: test_description='for-each-ref errors for broken refs'
    - ZEROS=$ZERO_OID
    - 
    - test_expect_success setup '
    --	test_oid_init &&
    - 	MISSING=$(test_oid deadbeef) &&
    - 	git commit --allow-empty -m "Initial" &&
    - 	git tag testtag &&
    -
    - ## t/t6501-freshen-objects.sh ##
    -@@ t/t6501-freshen-objects.sh: maybe_repack () {
    - 	fi
    - }
    - 
    --test_expect_success 'setup' '
    --	test_oid_init
    --'
    --
    - for repack in '' true; do
    - 	title=${repack:+repack}
    - 	title=${title:-loose}
    -
    - ## t/t7003-filter-branch.sh ##
    -@@ t/t7003-filter-branch.sh: test_description='git filter-branch'
    - . "$TEST_DIRECTORY/lib-gpg.sh"
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    - 	test_commit A &&
    - 	GIT_COMMITTER_DATE="@0 +0000" GIT_AUTHOR_DATE="@0 +0000" &&
    - 	test_commit --notick B &&
    -
    - ## t/t8003-blame-corner-cases.sh ##
    -@@ t/t8003-blame-corner-cases.sh: test_description='git blame corner cases'
    - pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
    - 
    - test_expect_success setup '
    --	test_oid_init &&
    --
    - 	echo A A A A A >one &&
    - 	echo B B B B B >two &&
    - 	echo C C C C C >tres &&
    -
    - ## t/t9301-fast-import-notes.sh ##
    -@@ t/t9301-fast-import-notes.sh: EOF
    - INPUT_END
    - 
    - test_expect_success 'set up master branch' '
    --	test_oid_init &&
    - 	git fast-import <input &&
    - 	git whatchanged master
    - '
    -
    - ## t/t9350-fast-export.sh ##
    -@@ t/t9350-fast-export.sh: test_description='git fast-export'
    - . ./test-lib.sh
    - 
    - test_expect_success 'setup' '
    --	test_oid_init &&
    --
    - 	echo break it > file0 &&
    - 	git add file0 &&
    - 	test_tick &&

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH v6 01/39] t: make test-bloom initialize repository
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 02/39] t1001: use $ZERO_OID brian m. carlson
                     ` (39 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The bloom filter code relies on reading object IDs using parse_oid_hex.
In order to make that work with an appropriate size, we need to have
initialized the repository's hash algorithm.  Since the values we're
processing depend on the repository in use, let's set up the repository
when we run the test helper.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/helper/test-bloom.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/t/helper/test-bloom.c b/t/helper/test-bloom.c
index f0aa80b98e..5e77d56f59 100644
--- a/t/helper/test-bloom.c
+++ b/t/helper/test-bloom.c
@@ -50,6 +50,8 @@ static const char *bloom_usage = "\n"
 
 int cmd__bloom(int argc, const char **argv)
 {
+	setup_git_directory();
+
 	if (argc < 2)
 		usage(bloom_usage);
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 02/39] t1001: use $ZERO_OID
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 01/39] t: make test-bloom initialize repository brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 03/39] t3305: make hash agnostic brian m. carlson
                     ` (38 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $ZERO_OID to make the test hash independent.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t1091-sparse-checkout-builtin.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh
index 7cd45fc139..84acfc48b6 100755
--- a/t/t1091-sparse-checkout-builtin.sh
+++ b/t/t1091-sparse-checkout-builtin.sh
@@ -369,7 +369,7 @@ test_expect_success 'sparse-checkout (init|set|disable) warns with unmerged stat
 	git clone repo unmerged &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C unmerged rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C unmerged update-index --index-info <input &&
@@ -396,7 +396,7 @@ test_expect_success 'sparse-checkout reapply' '
 	echo dirty >tweak/deep/deeper2/a &&
 
 	cat >input <<-EOF &&
-	0 0000000000000000000000000000000000000000	folder1/a
+	0 $ZERO_OID	folder1/a
 	100644 $(git -C tweak rev-parse HEAD:folder1/a) 1	folder1/a
 	EOF
 	git -C tweak update-index --index-info <input &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 03/39] t3305: make hash agnostic
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 01/39] t: make test-bloom initialize repository brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 02/39] t1001: use $ZERO_OID brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
                     ` (37 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

When computing the fanout length, let's use test_oid to look up the
hexadecimal size of the hash in question instead of hard-coding a value.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3305-notes-fanout.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t3305-notes-fanout.sh b/t/t3305-notes-fanout.sh
index 3b4753e1b4..94c1b02251 100755
--- a/t/t3305-notes-fanout.sh
+++ b/t/t3305-notes-fanout.sh
@@ -7,7 +7,7 @@ test_description='Test that adding/removing many notes triggers automatic fanout
 path_has_fanout() {
 	path=$1 &&
 	fanout=$2 &&
-	after_last_slash=$((40 - $fanout * 2)) &&
+	after_last_slash=$(($(test_oid hexsz) - $fanout * 2)) &&
 	echo $path | grep -q "^\([0-9a-f]\{2\}/\)\{$fanout\}[0-9a-f]\{$after_last_slash\}$"
 }
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (2 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 03/39] t3305: make hash agnostic brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 05/39] t6100: make hash size independent brian m. carlson
                     ` (36 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

From: Johannes Schindelin <johannes.schindelin@gmx.de>

The idea of the magic value "ac4f2ee" in this test is to make the
reworded commit `collide2` have the same shortened ID as the commit
`collide3`.

To port the same idea to the SHA-256 version of Git, we therefore need
another magic value that causes the same collision, but this time with
the SHA-256 version of the commit IDs.

In this patch, we add code guarded by `GIT_TEST_FIND_COLLIDER` to do
exactly that. Essentially, a large number of integers is appended to the
commit message "collide2" to find such a collision. To make it easier to
find such a collision, we reduce the number of digits to 4.

As the tests are no longer dependent on SHA-1, we also rename their
titles to talk about "commit IDs" instead of "SHA-1s".

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t3404-rebase-interactive.sh | 49 ++++++++++++++++++++++++++++-------
 1 file changed, 40 insertions(+), 9 deletions(-)

diff --git a/t/t3404-rebase-interactive.sh b/t/t3404-rebase-interactive.sh
index 4a7d21f898..9744e88760 100755
--- a/t/t3404-rebase-interactive.sh
+++ b/t/t3404-rebase-interactive.sh
@@ -1250,7 +1250,7 @@ test_expect_success 'rebase -i error on commits with \ in message' '
 	test_expect_code 1 grep  "	emp" error
 '
 
-test_expect_success SHA1 'short SHA-1 setup' '
+test_expect_success 'short commit ID setup' '
 	test_when_finished "git checkout master" &&
 	git checkout --orphan collide &&
 	git rm -rf . &&
@@ -1262,23 +1262,54 @@ test_expect_success SHA1 'short SHA-1 setup' '
 	)
 '
 
-test_expect_success SHA1 'short SHA-1 collide' '
+if test -n "$GIT_TEST_FIND_COLLIDER"
+then
+	author="$(unset test_tick; test_tick; git var GIT_AUTHOR_IDENT)"
+	committer="$(unset test_tick; test_tick; git var GIT_COMMITTER_IDENT)"
+	blob="$(git rev-parse collide2:collide)"
+	from="$(git rev-parse collide1^0)"
+	repl="commit refs/heads/collider-&\\n"
+	repl="${repl}author $author\\ncommitter $committer\\n"
+	repl="${repl}data <<EOF\\ncollide2 &\\nEOF\\n"
+	repl="${repl}from $from\\nM 100644 $blob collide\\n"
+	test_seq 1 32768 | sed "s|.*|$repl|" >script &&
+	git fast-import <script &&
+	git pack-refs &&
+	git for-each-ref >refs &&
+	grep "^$(test_oid t3404_collision)" <refs >matches &&
+	cat matches &&
+	test_line_count -gt 2 matches || {
+		echo "Could not find a collider" >&2
+		exit 1
+	}
+fi
+
+test_expect_success 'short commit ID collide' '
+	test_oid_cache <<-EOF &&
+	# collision-related constants
+	t3404_collision	sha1:6bcd
+	t3404_collision	sha256:0161
+	t3404_collider	sha1:ac4f2ee
+	t3404_collider	sha256:16697
+	EOF
 	test_when_finished "reset_rebase && git checkout master" &&
 	git checkout collide &&
-	colliding_sha1=6bcda37 &&
-	test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
+	colliding_id=$(test_oid t3404_collision) &&
+	hexsz=$(test_oid hexsz) &&
+	test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+	test_config core.abbrev 4 &&
 	(
 		unset test_tick &&
 		test_tick &&
 		set_fake_editor &&
-		FAKE_COMMIT_MESSAGE="collide2 ac4f2ee" \
+		FAKE_COMMIT_MESSAGE="collide2 $(test_oid t3404_collider)" \
 		FAKE_LINES="reword 1 break 2" git rebase -i HEAD~2 &&
-		test $colliding_sha1 = "$(git rev-parse HEAD | cut -c 1-7)" &&
-		grep "^pick $colliding_sha1 " \
+		test $colliding_id = "$(git rev-parse HEAD | cut -c 1-4)" &&
+		grep "^pick $colliding_id " \
 			.git/rebase-merge/git-rebase-todo.tmp &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo &&
-		grep "^pick [0-9a-f]\{40\}" \
+		grep "^pick [0-9a-f]\{$hexsz\}" \
 			.git/rebase-merge/git-rebase-todo.backup &&
 		git rebase --continue
 	) &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 05/39] t6100: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (3 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 06/39] t6101: " brian m. carlson
                     ` (35 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of hard-coding a constant 40, split the output of rev-list by
field.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6100-rev-list-in-order.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t6100-rev-list-in-order.sh b/t/t6100-rev-list-in-order.sh
index b2bb0a7f61..e934bc239c 100755
--- a/t/t6100-rev-list-in-order.sh
+++ b/t/t6100-rev-list-in-order.sh
@@ -22,7 +22,7 @@ test_expect_success 'setup a commit history with trees, blobs' '
 
 test_expect_success 'rev-list --in-commit-order' '
 	git rev-list --in-commit-order --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}
@@ -49,7 +49,7 @@ test_expect_success 'rev-list --in-commit-order' '
 
 test_expect_success 'rev-list lists blobs and trees after commits' '
 	git rev-list --objects HEAD >actual.raw &&
-	cut -c 1-40 >actual <actual.raw &&
+	cut -d" " -f1 >actual <actual.raw &&
 
 	git cat-file --batch-check="%(objectname)" >expect.raw <<-\EOF &&
 		HEAD^{commit}

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 06/39] t6101: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (4 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 05/39] t6100: make hash size independent brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 07/39] t6301: " brian m. carlson
                     ` (34 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $OID_REGEX instead of a hard-coded regular expression.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6101-rev-parse-parents.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t6101-rev-parse-parents.sh b/t/t6101-rev-parse-parents.sh
index 7683e4a114..7531262a5e 100755
--- a/t/t6101-rev-parse-parents.sh
+++ b/t/t6101-rev-parse-parents.sh
@@ -34,7 +34,7 @@ test_expect_success 'setup' '
 '
 
 test_expect_success 'start is valid' '
-	git rev-parse start | grep "^[0-9a-f]\{40\}$"
+	git rev-parse start | grep "^$OID_REGEX$"
 '
 
 test_expect_success 'start^0' '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 07/39] t6301: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (5 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 06/39] t6101: " brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 08/39] t6500: specify test values for SHA-256 brian m. carlson
                     ` (33 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of hard-coding a fixed length example object ID in the test,
compute one using the translation tables.  Move a variable into the
setup block so that we can ensure the exit status of test_oid is
checked.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6301-for-each-ref-errors.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t6301-for-each-ref-errors.sh b/t/t6301-for-each-ref-errors.sh
index 49cc65bb58..809854fc0c 100755
--- a/t/t6301-for-each-ref-errors.sh
+++ b/t/t6301-for-each-ref-errors.sh
@@ -5,9 +5,9 @@ test_description='for-each-ref errors for broken refs'
 . ./test-lib.sh
 
 ZEROS=$ZERO_OID
-MISSING=abababababababababababababababababababab
 
 test_expect_success setup '
+	MISSING=$(test_oid deadbeef) &&
 	git commit --allow-empty -m "Initial" &&
 	git tag testtag &&
 	git for-each-ref >full-list &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 08/39] t6500: specify test values for SHA-256
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (6 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 07/39] t6301: " brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 09/39] t6501: avoid hard-coded objects brian m. carlson
                     ` (32 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In this test, we want to produce several blobs whose first two hex
characters are "17", since we look at this object directory as a proxy
for how many loose objects there are before we need to GC.  Use
test_oid_cache to specify strings that will hash to the right values
when turned into blobs.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6500-gc.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)

diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
index 0a69a67117..4a3b8f48ac 100755
--- a/t/t6500-gc.sh
+++ b/t/t6500-gc.sh
@@ -10,7 +10,24 @@ test_expect_success 'setup' '
 	# do not let the amount of physical memory affects gc
 	# behavior, make sure we always pack everything to one pack by
 	# default
-	git config gc.bigPackThreshold 2g
+	git config gc.bigPackThreshold 2g &&
+
+	# These are simply values which, when hashed as a blob with a newline,
+	# produce a hash where the first byte is 0x17 in their respective
+	# algorithms.
+	test_oid_cache <<-EOF
+	obj1 sha1:263
+	obj1 sha256:34
+
+	obj2 sha1:410
+	obj2 sha256:174
+
+	obj3 sha1:523
+	obj3 sha256:313
+
+	obj4 sha1:790
+	obj4 sha256:481
+	EOF
 '
 
 test_expect_success 'gc empty repository' '
@@ -85,13 +102,13 @@ test_expect_success 'auto gc with too many loose objects does not attempt to cre
 	# We need to create two object whose sha1s start with 17
 	# since this is what git gc counts.  As it happens, these
 	# two blobs will do so.
-	test_commit 263 &&
-	test_commit 410 &&
+	test_commit "$(test_oid obj1)" &&
+	test_commit "$(test_oid obj2)" &&
 	# Our first gc will create a pack; our second will create a second pack
 	git gc --auto &&
 	ls .git/objects/pack | sort >existing_packs &&
-	test_commit 523 &&
-	test_commit 790 &&
+	test_commit "$(test_oid obj3)" &&
+	test_commit "$(test_oid obj4)" &&
 
 	git gc --auto 2>err &&
 	test_i18ngrep ! "^warning:" err &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 09/39] t6501: avoid hard-coded objects
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (7 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 08/39] t6500: specify test values for SHA-256 brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:13   ` [PATCH v6 10/39] t7003: compute appropriate length constant brian m. carlson
                     ` (31 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test contains hard-coded invalid object IDs.  Make it hash size
independent by generating invalid object IDs using the translation
tables.  Add a setup target to ensure the output of test_oid_init is
checked properly.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t6501-freshen-objects.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/t/t6501-freshen-objects.sh b/t/t6501-freshen-objects.sh
index f30b4849b6..8a3bb4105b 100755
--- a/t/t6501-freshen-objects.sh
+++ b/t/t6501-freshen-objects.sh
@@ -128,9 +128,9 @@ for repack in '' true; do
 done
 
 test_expect_success 'do not complain about existing broken links (commit)' '
-	cat >broken-commit <<-\EOF &&
-	tree 0000000000000000000000000000000000000001
-	parent 0000000000000000000000000000000000000002
+	cat >broken-commit <<-EOF &&
+	tree $(test_oid 001)
+	parent $(test_oid 002)
 	author whatever <whatever@example.com> 1234 -0000
 	committer whatever <whatever@example.com> 1234 -0000
 
@@ -143,8 +143,8 @@ test_expect_success 'do not complain about existing broken links (commit)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tree)' '
-	cat >broken-tree <<-\EOF &&
-	100644 blob 0000000000000000000000000000000000000003	foo
+	cat >broken-tree <<-EOF &&
+	100644 blob $(test_oid 003)	foo
 	EOF
 	tree=$(git mktree --missing <broken-tree) &&
 	git gc -q 2>stderr &&
@@ -153,8 +153,8 @@ test_expect_success 'do not complain about existing broken links (tree)' '
 '
 
 test_expect_success 'do not complain about existing broken links (tag)' '
-	cat >broken-tag <<-\EOF &&
-	object 0000000000000000000000000000000000000004
+	cat >broken-tag <<-EOF &&
+	object $(test_oid 004)
 	type commit
 	tag broken
 	tagger whatever <whatever@example.com> 1234 -0000

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 10/39] t7003: compute appropriate length constant
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (8 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 09/39] t6501: avoid hard-coded objects brian m. carlson
@ 2020-07-29 23:13   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 11/39] t7063: make hash size independent brian m. carlson
                     ` (30 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:13 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using a specific invalid hard-coded object ID, look one
up from the translation table.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7003-filter-branch.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index e23de7d0b5..36477cb1f4 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -463,10 +463,11 @@ test_expect_success 'rewrite submodule with another content' '
 '
 
 test_expect_success 'replace submodule revision' '
+	invalid=$(test_oid numeric) &&
 	git reset --hard original &&
 	git filter-branch -f --tree-filter \
 	    "if git ls-files --error-unmatch -- submod > /dev/null 2>&1
-	     then git update-index --cacheinfo 160000 0123456789012345678901234567890123456789 submod
+	     then git update-index --cacheinfo 160000 $invalid submod
 	     fi" HEAD &&
 	test $orig_head != $(git show-ref --hash --head HEAD)
 '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 11/39] t7063: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (9 preceding siblings ...)
  2020-07-29 23:13   ` [PATCH v6 10/39] t7003: compute appropriate length constant brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
                     ` (29 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use test_oid instead of hard-coding algorithm-specific constants and
all-zero values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7063-status-untracked-cache.sh | 150 ++++++++++++++++--------------
 1 file changed, 80 insertions(+), 70 deletions(-)

diff --git a/t/t7063-status-untracked-cache.sh b/t/t7063-status-untracked-cache.sh
index 428cff9cf3..a682a3d826 100755
--- a/t/t7063-status-untracked-cache.sh
+++ b/t/t7063-status-untracked-cache.sh
@@ -75,14 +75,24 @@ test_expect_success 'setup' '
 	touch one two three done/one dtwo/two dthree/three &&
 	git add one two done/one &&
 	: >.git/info/exclude &&
-	git update-index --untracked-cache
+	git update-index --untracked-cache &&
+	test_oid_cache <<-EOF
+	root sha1:e6fcc8f2ee31bae321d66afd183fcb7237afae6e
+	root sha256:b90c672088c015b9c83876e919da311bad4cd39639fb139f988af6a11493b974
+
+	exclude sha1:13263c0978fb9fad16b2d580fb800b6d811c3ff0
+	exclude sha256:fe4aaa1bbbbce4cb8f73426748a14c5ad6026b26f90505a0bf2494b165a5b76c
+
+	done sha1:1946f0437f90c5005533cbe1736a6451ca301714
+	done sha256:7f079501d79f665b3acc50f5e0e9e94509084d5032ac20113a37dd5029b757cc
+	EOF
 '
 
 test_expect_success 'untracked cache is empty' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-empty <<EOF &&
-info/exclude 0000000000000000000000000000000000000000
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $ZERO_OID
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
 EOF
@@ -100,17 +110,17 @@ EOF
 
 cat >../dump.expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 
@@ -190,18 +200,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
 dthree/
 dtwo/
 four
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -239,18 +249,18 @@ test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
 info/exclude $EMPTY_BLOB
-core.excludesfile 0000000000000000000000000000000000000000
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dthree/
 dtwo/
 three
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
 three
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -284,16 +294,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -303,14 +313,14 @@ test_expect_success 'move two from tracked to untracked' '
 	git rm --cached two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -342,17 +352,17 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
 two
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -362,14 +372,14 @@ test_expect_success 'move two from untracked to tracked' '
 	git add two &&
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/ $(test_oid root) recurse
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -401,16 +411,16 @@ EOF
 test_expect_success 'verify untracked cache dump' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -447,16 +457,16 @@ EOF
 test_expect_success 'untracked cache correct after commit' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 0000000000000000000000000000000000000000 recurse valid
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/ $ZERO_OID recurse valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -508,17 +518,17 @@ EOF
 test_expect_success 'untracked cache correct after status' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect ../actual
@@ -580,22 +590,22 @@ EOF
 test_expect_success 'verify untracked cache dump (sparse/subdirs)' '
 	test-tool dump-untracked-cache >../actual &&
 	cat >../expect-from-test-dump <<EOF &&
-info/exclude 13263c0978fb9fad16b2d580fb800b6d811c3ff0
-core.excludesfile 0000000000000000000000000000000000000000
+info/exclude $(test_oid exclude)
+core.excludesfile $ZERO_OID
 exclude_per_dir .gitignore
 flags 00000006
-/ e6fcc8f2ee31bae321d66afd183fcb7237afae6e recurse valid
+/ $(test_oid root) recurse valid
 .gitignore
 dtwo/
-/done/ 1946f0437f90c5005533cbe1736a6451ca301714 recurse valid
+/done/ $(test_oid done) recurse valid
 five
 sub/
-/done/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/ $ZERO_OID recurse check_only valid
 sub/
-/done/sub/sub/ 0000000000000000000000000000000000000000 recurse check_only valid
+/done/sub/sub/ $ZERO_OID recurse check_only valid
 file
-/dthree/ 0000000000000000000000000000000000000000 recurse check_only valid
-/dtwo/ 0000000000000000000000000000000000000000 recurse check_only valid
+/dthree/ $ZERO_OID recurse check_only valid
+/dtwo/ $ZERO_OID recurse check_only valid
 two
 EOF
 	test_cmp ../expect-from-test-dump ../actual
@@ -806,8 +816,8 @@ test_expect_success '"status" after file replacement should be clean with UC=tru
 	test-tool dump-untracked-cache >../actual &&
 	grep -F "recurse valid" ../actual >../actual.grep &&
 	cat >../expect.grep <<EOF &&
-/ 0000000000000000000000000000000000000000 recurse valid
-/two/ 0000000000000000000000000000000000000000 recurse valid
+/ $ZERO_OID recurse valid
+/two/ $ZERO_OID recurse valid
 EOF
 	status_is_clean &&
 	test_cmp ../expect.grep ../actual.grep

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 12/39] t7201: abstract away SHA-1-specific constants
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (10 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 11/39] t7063: make hash size independent brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 13/39] t7102: " brian m. carlson
                     ` (28 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7201-co.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t7201-co.sh b/t/t7201-co.sh
index b696bae5f5..4d62b9b00f 100755
--- a/t/t7201-co.sh
+++ b/t/t7201-co.sh
@@ -230,9 +230,10 @@ test_expect_success 'switch to another branch while carrying a deletion' '
 test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 	git config advice.detachedHead false &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	git checkout renamer^ 2>messages &&
-	test_i18ngrep "HEAD is now at 7329388" messages &&
+	test_i18ngrep "HEAD is now at $rev" messages &&
 	test_line_count = 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&
@@ -248,9 +249,10 @@ test_expect_success 'checkout to detach HEAD (with advice declined)' '
 
 test_expect_success 'checkout to detach HEAD' '
 	git config advice.detachedHead true &&
+	rev=$(git rev-parse --short renamer^) &&
 	git checkout -f renamer && git clean -f &&
 	GIT_TEST_GETTEXT_POISON=false git checkout renamer^ 2>messages &&
-	grep "HEAD is now at 7329388" messages &&
+	grep "HEAD is now at $rev" messages &&
 	test_line_count -gt 1 messages &&
 	H=$(git rev-parse --verify HEAD) &&
 	M=$(git show-ref -s --verify refs/heads/master) &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 13/39] t7102: abstract away SHA-1-specific constants
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (11 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 14/39] t7400: make hash size independent brian m. carlson
                     ` (27 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7102-reset.sh | 93 +++++++++++++++++++++++++++---------------------
 1 file changed, 53 insertions(+), 40 deletions(-)

diff --git a/t/t7102-reset.sh b/t/t7102-reset.sh
index 97be0d968d..22161b3b2d 100755
--- a/t/t7102-reset.sh
+++ b/t/t7102-reset.sh
@@ -37,17 +37,23 @@ test_expect_success 'creating initial files and commits' '
 
 	echo "2nd line 1st file" >>first &&
 	git commit -a -m "modify 1st file" &&
+	head5p2=$(git rev-parse --verify HEAD) &&
+	head5p2f=$(git rev-parse --short HEAD:first) &&
 
 	git rm first &&
 	git mv second secondfile &&
 	git commit -a -m "remove 1st and rename 2nd" &&
+	head5p1=$(git rev-parse --verify HEAD) &&
+	head5p1s=$(git rev-parse --short HEAD:secondfile) &&
 
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	# "git commit -m" would break MinGW, as Windows refuse to pass
 	# $test_encoding encoded parameter to git.
 	commit_msg $test_encoding | git -c "i18n.commitEncoding=$test_encoding" commit -a -F - &&
-	head5=$(git rev-parse --verify HEAD)
+	head5=$(git rev-parse --verify HEAD) &&
+	head5s=$(git rev-parse --short HEAD:secondfile) &&
+	head5sl=$(git rev-parse HEAD:secondfile)
 '
 # git log --pretty=oneline # to see those SHA1 involved
 
@@ -94,7 +100,7 @@ test_expect_success 'giving a non existing revision should fail' '
 
 test_expect_success 'reset --soft with unmerged index should fail' '
 	touch .git/MERGE_HEAD &&
-	echo "100644 44c5b5884550c17758737edcced463447b91d42b 1	un" |
+	echo "100644 $head5sl 1	un" |
 		git update-index --index-info &&
 	test_must_fail git reset --soft HEAD &&
 	rm .git/MERGE_HEAD &&
@@ -192,7 +198,7 @@ test_expect_success \
 >.diff_expect
 cat >.cached_expect <<EOF
 diff --git a/secondfile b/secondfile
-index 1bbba79..44c5b58 100644
+index $head5p1s..$head5s 100644
 --- a/secondfile
 +++ b/secondfile
 @@ -1 +1,2 @@
@@ -207,7 +213,7 @@ secondfile:
 EOF
 test_expect_success '--soft reset only should show changes in diff --cached' '
 	git reset --soft HEAD^ &&
-	check_changes d1a4bc3abce4829628ae2dcb0d60ef3d1a78b1c4 &&
+	check_changes $head5p1 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head5
 '
@@ -242,7 +248,7 @@ EOF
 test_expect_success \
 	'--hard reset should change the files and undo commits permanently' '
 	git reset --hard HEAD~2 &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	check_changes $head5p2 &&
 	test "$(git rev-parse ORIG_HEAD)" = \
 			$head4
 '
@@ -251,7 +257,7 @@ test_expect_success \
 cat >.cached_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -259,14 +265,14 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
 -2nd file
 diff --git a/secondfile b/secondfile
 new file mode 100644
-index 0000000..44c5b58
+index 0000000..$head5s
 --- /dev/null
 +++ b/secondfile
 @@ -0,0 +1,2 @@
@@ -286,13 +292,13 @@ test_expect_success \
 	echo "1st line 2nd file" >secondfile &&
 	echo "2nd line 2nd file" >>secondfile &&
 	git add secondfile &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2
 '
 
 cat >.diff_expect <<EOF
 diff --git a/first b/first
 deleted file mode 100644
-index 8206c22..0000000
+index $head5p2f..0000000
 --- a/first
 +++ /dev/null
 @@ -1,2 +0,0 @@
@@ -300,7 +306,7 @@ index 8206c22..0000000
 -2nd line 1st file
 diff --git a/second b/second
 deleted file mode 100644
-index 1bbba79..0000000
+index $head5p1s..0000000
 --- a/second
 +++ /dev/null
 @@ -1 +0,0 @@
@@ -314,9 +320,8 @@ secondfile:
 EOF
 test_expect_success '--mixed reset to HEAD should unadd the files' '
 	git reset &&
-	check_changes ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
-	test "$(git rev-parse ORIG_HEAD)" = \
-			ddaefe00f1da16864591c61fdc7adb5d7cd6b74e
+	check_changes $head5p2 &&
+	test "$(git rev-parse ORIG_HEAD)" = $head5p2
 '
 
 >.diff_expect
@@ -328,7 +333,7 @@ secondfile:
 EOF
 test_expect_success 'redoing the last two commits should succeed' '
 	git add secondfile &&
-	git reset --hard ddaefe00f1da16864591c61fdc7adb5d7cd6b74e &&
+	git reset --hard $head5p2 &&
 
 	git rm first &&
 	git mv second secondfile &&
@@ -389,47 +394,55 @@ test_expect_success \
 	check_changes $head5
 '
 
-cat > expect << EOF
-diff --git a/file1 b/file1
-index d00491f..7ed6ff8 100644
---- a/file1
-+++ b/file1
-@@ -1 +1 @@
--1
-+5
-diff --git a/file2 b/file2
-deleted file mode 100644
-index 0cfbf08..0000000
---- a/file2
-+++ /dev/null
-@@ -1 +0,0 @@
--2
-EOF
-cat > cached_expect << EOF
-diff --git a/file4 b/file4
-new file mode 100644
-index 0000000..b8626c4
---- /dev/null
-+++ b/file4
-@@ -0,0 +1 @@
-+4
-EOF
 test_expect_success 'test --mixed <paths>' '
 	echo 1 > file1 &&
 	echo 2 > file2 &&
 	git add file1 file2 &&
 	test_tick &&
 	git commit -m files &&
+	before1=$(git rev-parse --short HEAD:file1) &&
+	before2=$(git rev-parse --short HEAD:file2) &&
 	git rm file2 &&
 	echo 3 > file3 &&
 	echo 4 > file4 &&
 	echo 5 > file1 &&
+	after1=$(git rev-parse --short $(git hash-object file1)) &&
+	after4=$(git rev-parse --short $(git hash-object file4)) &&
 	git add file1 file3 file4 &&
 	git reset HEAD -- file1 file2 file3 &&
 	test_must_fail git diff --quiet &&
 	git diff > output &&
+
+	cat > expect <<-EOF &&
+	diff --git a/file1 b/file1
+	index $before1..$after1 100644
+	--- a/file1
+	+++ b/file1
+	@@ -1 +1 @@
+	-1
+	+5
+	diff --git a/file2 b/file2
+	deleted file mode 100644
+	index $before2..0000000
+	--- a/file2
+	+++ /dev/null
+	@@ -1 +0,0 @@
+	-2
+	EOF
+
 	test_cmp expect output &&
 	git diff --cached > output &&
+
+	cat > cached_expect <<-EOF &&
+	diff --git a/file4 b/file4
+	new file mode 100644
+	index 0000000..$after4
+	--- /dev/null
+	+++ b/file4
+	@@ -0,0 +1 @@
+	+4
+	EOF
+
 	test_cmp cached_expect output
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 14/39] t7400: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (12 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 13/39] t7102: " brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 15/39] t7405: " brian m. carlson
                     ` (26 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using cut with hard-coded hash sizes, use cut with fields, or
where that's not possible, sed with $OID_REGEX, so that the tests are
independent of hash size.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7400-submodule-basic.sh | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/t/t7400-submodule-basic.sh b/t/t7400-submodule-basic.sh
index 956e17abb3..fec7e0299d 100755
--- a/t/t7400-submodule-basic.sh
+++ b/t/t7400-submodule-basic.sh
@@ -1231,7 +1231,7 @@ test_expect_success 'submodule helper list is not confused by common prefixes' '
 	git submodule add /dir1/b dir1/b &&
 	git submodule add /dir2/b dir2/b &&
 	git commit -m "first submodule commit" &&
-	git submodule--helper list dir1/b |cut -c51- >actual &&
+	git submodule--helper list dir1/b | cut -f 2 >actual &&
 	echo "dir1/b" >expect &&
 	test_cmp expect actual
 '
@@ -1260,7 +1260,7 @@ test_expect_success 'submodule update --init with a specification' '
 	pwd=$(pwd) &&
 	git clone file://"$pwd"/multisuper multisuper_clone &&
 	git -C multisuper_clone submodule update --init . ":(exclude)sub0" &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1271,7 +1271,7 @@ test_expect_success 'submodule update --init with submodule.active set' '
 	git -C multisuper_clone config submodule.active "." &&
 	git -C multisuper_clone config --add submodule.active ":(exclude)sub0" &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1290,7 +1290,7 @@ test_expect_success 'submodule update and setting submodule.<name>.active' '
 	-sub3
 	EOF
 	git -C multisuper_clone submodule update &&
-	git -C multisuper_clone submodule status |cut -c 1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1307,12 +1307,12 @@ test_expect_success 'clone active submodule without submodule url set' '
 		git submodule update &&
 		git submodule status >actual_raw &&
 
-		cut -c 1,43- actual_raw >actual &&
+		cut -d" " -f3- actual_raw >actual &&
 		cat >expect <<-\EOF &&
-		 sub0 (test2)
-		 sub1 (test2)
-		 sub2 (test2)
-		 sub3 (test2)
+		sub0 (test2)
+		sub1 (test2)
+		sub2 (test2)
+		sub3 (test2)
 		EOF
 		test_cmp expect actual
 	)
@@ -1328,7 +1328,7 @@ test_expect_success 'clone --recurse-submodules with a pathspec works' '
 	EOF
 
 	git clone --recurse-submodules="sub0" multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expected actual
 '
 
@@ -1345,7 +1345,7 @@ test_expect_success 'clone with multiple --recurse-submodules options' '
 		  --recurse-submodules=":(exclude)sub0" \
 		  --recurse-submodules=":(exclude)sub2" \
 		  multisuper multisuper_clone &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual
 '
 
@@ -1373,7 +1373,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 		  --recurse-submodules=":(exclude)sub4" \
 		  multisuper multisuper_clone &&
 
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect actual &&
 
 	git -C multisuper submodule add ../sub1 sub4 &&
@@ -1382,7 +1382,7 @@ test_expect_success 'clone and subsequent updates correctly auto-initialize subm
 	# obtain the new superproject
 	git -C multisuper_clone pull &&
 	git -C multisuper_clone submodule update --init &&
-	git -C multisuper_clone submodule status |cut -c1,43- >actual &&
+	git -C multisuper_clone submodule status | sed "s/$OID_REGEX //" >actual &&
 	test_cmp expect2 actual
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 15/39] t7405: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (13 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 14/39] t7400: make hash size independent brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
                     ` (25 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use $ZERO_OID instead of hard-coding a fixed size all-zeros object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7405-submodule-merge.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t7405-submodule-merge.sh b/t/t7405-submodule-merge.sh
index aa33978ed2..6a1e5f8232 100755
--- a/t/t7405-submodule-merge.sh
+++ b/t/t7405-submodule-merge.sh
@@ -195,7 +195,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&
@@ -214,7 +214,7 @@ test_expect_success 'git submodule status should display the merge conflict prop
        url = $TRASH_DIRECTORY/sub
 EOF
        cat >expect <<EOF &&
-U0000000000000000000000000000000000000000 sub
+U$ZERO_OID sub
 EOF
        git submodule status > actual &&
        test_cmp expect actual &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 16/39] t7506: avoid checking for SHA-1-specific constants
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (14 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 15/39] t7405: " brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
                     ` (24 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test to sanitize the diffs and strip out object IDs from
them, as it does for other object IDs, since we are not interested in
the particular values used.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7506-status-submodule.sh | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/t/t7506-status-submodule.sh b/t/t7506-status-submodule.sh
index 08629a6e70..3fcb44767f 100755
--- a/t/t7506-status-submodule.sh
+++ b/t/t7506-status-submodule.sh
@@ -22,6 +22,10 @@ sanitize_output () {
 	mv output2 output
 }
 
+sanitize_diff () {
+	sed -e "/^index [0-9a-f,]*\.\.[0-9a-f]*/d" "$1"
+}
+
 
 test_expect_success 'setup' '
 	test_create_repo_with_commit sub &&
@@ -269,7 +273,6 @@ short_sha1_merge_sub1=$(cd sub1 && git rev-parse --short HEAD)
 short_sha1_merge_sub2=$(cd sub2 && git rev-parse --short HEAD)
 cat >diff_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -286,7 +289,6 @@ EOF
 
 cat >diff_submodule_expect <<\EOF
 diff --cc .gitmodules
-index badaa4c,44f999a..0000000
 --- a/.gitmodules
 +++ b/.gitmodules
 @@@ -1,3 -1,3 +1,9 @@@
@@ -306,7 +308,8 @@ test_expect_success 'diff with merge conflict in .gitmodules' '
 		cd super &&
 		git diff >../diff_actual 2>&1
 	) &&
-	test_cmp diff_expect diff_actual
+	sanitize_diff diff_actual >diff_sanitized &&
+	test_cmp diff_expect diff_sanitized
 '
 
 test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
@@ -314,7 +317,8 @@ test_expect_success 'diff --submodule with merge conflict in .gitmodules' '
 		cd super &&
 		git diff --submodule >../diff_submodule_actual 2>&1
 	) &&
-	test_cmp diff_submodule_expect diff_submodule_actual
+	sanitize_diff diff_submodule_actual >diff_sanitized &&
+	test_cmp diff_submodule_expect diff_sanitized
 '
 
 # We'll setup different cases for further testing:

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 17/39] t7508: use $ZERO_OID instead of hard-coded constant
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (15 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 18/39] t8002: make hash size independent brian m. carlson
                     ` (23 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Use the ZERO_OID variable to abbreviate the all-zeros object ID for
maintainability and to avoid depending on a specific size for the hash.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t7508-status.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t7508-status.sh b/t/t7508-status.sh
index 8e969f3e36..e81759319f 100755
--- a/t/t7508-status.sh
+++ b/t/t7508-status.sh
@@ -837,7 +837,7 @@ EOF
 '
 
 cat >expect <<EOF
-:100644 100644 $EMPTY_BLOB 0000000000000000000000000000000000000000 M	dir1/modified
+:100644 100644 $EMPTY_BLOB $ZERO_OID M	dir1/modified
 EOF
 test_expect_success 'status refreshes the index' '
 	touch dir2/added &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 18/39] t8002: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (16 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 19/39] t8003: " brian m. carlson
                     ` (22 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Compute the length of an object ID instead of hard-coding 40-based
values.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8002-blame.sh | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/t/t8002-blame.sh b/t/t8002-blame.sh
index eea048e52c..c3b70b025e 100755
--- a/t/t8002-blame.sh
+++ b/t/t8002-blame.sh
@@ -6,6 +6,10 @@ test_description='git blame'
 PROG='git blame -c'
 . "$TEST_DIRECTORY"/annotate-tests.sh
 
+test_expect_success 'setup' '
+	hexsz=$(test_oid hexsz)
+'
+
 test_expect_success 'blame untracked file in empty repo' '
 	>untracked &&
 	test_must_fail git blame untracked
@@ -105,17 +109,17 @@ test_expect_success 'blame --abbrev=<n> works' '
 '
 
 test_expect_success 'blame -l aligns regular and boundary commits' '
-	check_abbrev 40 -l HEAD &&
-	check_abbrev 39 -l ^HEAD
+	check_abbrev $hexsz         -l HEAD &&
+	check_abbrev $((hexsz - 1)) -l ^HEAD
 '
 
-test_expect_success 'blame --abbrev=40 behaves like -l' '
-	check_abbrev 40 --abbrev=40 HEAD &&
-	check_abbrev 39 --abbrev=40 ^HEAD
+test_expect_success 'blame --abbrev with full length behaves like -l' '
+	check_abbrev $hexsz         --abbrev=$hexsz HEAD &&
+	check_abbrev $((hexsz - 1)) --abbrev=$hexsz ^HEAD
 '
 
-test_expect_success '--no-abbrev works like --abbrev=40' '
-	check_abbrev 40 --no-abbrev
+test_expect_success '--no-abbrev works like --abbrev with full length' '
+	check_abbrev $hexsz --no-abbrev
 '
 
 test_expect_success '--exclude-promisor-objects does not BUG-crash' '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 19/39] t8003: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (17 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 18/39] t8002: make hash size independent brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 20/39] t8011: " brian m. carlson
                     ` (21 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

One assertion in this test invokes git with core.abbrev set to "40".
Since we're expecting the full hash length, use test_oid to look up the
full hash length for the hash in use.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8003-blame-corner-cases.sh | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/t/t8003-blame-corner-cases.sh b/t/t8003-blame-corner-cases.sh
index 9130b887d2..b871dd4f86 100755
--- a/t/t8003-blame-corner-cases.sh
+++ b/t/t8003-blame-corner-cases.sh
@@ -6,7 +6,6 @@ test_description='git blame corner cases'
 pick_fc='s/^[0-9a-f^]* *\([^ ]*\) *(\([^ ]*\) .*/\1-\2/'
 
 test_expect_success setup '
-
 	echo A A A A A >one &&
 	echo B B B B B >two &&
 	echo C C C C C >tres &&
@@ -306,7 +305,7 @@ test_expect_success 'blame coalesce' '
 	$oid 1) ABC
 	$oid 2) DEF
 	EOF
-	git -c core.abbrev=40 blame -s giraffe >actual &&
+	git -c core.abbrev=$(test_oid hexsz) blame -s giraffe >actual &&
 	test_cmp expect actual
 '
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 20/39] t8011: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (18 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 19/39] t8003: " brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
                     ` (20 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Allow lines which start with either a 40- or 64-character hex object ID,
to allow for both SHA-1 and SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t8011-blame-split-file.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t8011-blame-split-file.sh b/t/t8011-blame-split-file.sh
index 831125047b..bdda0c03fe 100755
--- a/t/t8011-blame-split-file.sh
+++ b/t/t8011-blame-split-file.sh
@@ -54,7 +54,7 @@ test_expect_success 'setup simulated porcelain' '
 	cat >read-porcelain.pl <<-\EOF
 	my $field = shift;
 	while (<>) {
-		if (/^[0-9a-f]{40} /) {
+		if (/^[0-9a-f]{40,} /) {
 			flush();
 			$hash = $&;
 		} elsif (/^$field (.*)/) {

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 21/39] t9300: abstract away SHA-1-specific constants
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (19 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 20/39] t8011: " brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
                     ` (19 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Adjust the test so that it computes variables for object IDs instead of
using hard-coded hashes.  In addition, use cut to filter out the object
IDs and verify only the information that we're really interested in.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 108 ++++++++++++++++++++++-------------------
 1 file changed, 59 insertions(+), 49 deletions(-)

diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index e151df81c0..f2d2cb06ad 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -284,8 +284,9 @@ test_expect_success 'A: verify pack' '
 '
 
 test_expect_success 'A: verify diff' '
+	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 A	copy-of-file2
+	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -528,6 +529,7 @@ test_expect_success 'B: fail on invalid committer (5)' '
 test_expect_success 'C: incremental import create pack from stdin' '
 	newf=$(echo hi newf | git hash-object -w --stdin) &&
 	oldf=$(git rev-parse --verify master:file2) &&
+	thrf=$(git rev-parse --verify master:file3) &&
 	test_tick &&
 	cat >input <<-INPUT_END &&
 	commit refs/heads/branch
@@ -570,10 +572,11 @@ test_expect_success 'C: verify commit' '
 '
 
 test_expect_success 'C: validate rename result' '
+	zero=$ZERO_OID &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 f1fb5da718392694d0076d677d6d0e364c79b0bc A	file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2	file2/oldf
-	:100644 000000 0d92e9f3374ae2947c23aa477cbc68ce598135f1 0000000000000000000000000000000000000000 D	file3
+	:000000 100755 $zero $newf A	file2/newf
+	:100644 100644 $oldf $oldf R100	file2	file2/oldf
+	:100644 000000 $thrf $zero D	file3
 	EOF
 	git diff-tree -M -r master branch >actual &&
 	compare_diff_raw expect actual
@@ -614,9 +617,11 @@ test_expect_success 'D: verify pack' '
 '
 
 test_expect_success 'D: validate new files added' '
+	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
+	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 e74b7d465e52746be2b4bae983670711e6e66657 A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 A	newdir/interesting
+	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
+	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -779,12 +784,13 @@ test_expect_success 'H: verify pack' '
 '
 
 test_expect_success 'H: validate old files removed, new files added' '
+	f4id=$(git rev-parse HEAD:file4) &&
 	cat >expect <<-EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file2/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file2/oldf
-	:100755 000000 85df50785d62d3b05ab03d9cbf7e4a0b49449730 0000000000000000000000000000000000000000 D	file4
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	h/e/l/lo
-	:100755 000000 e74b7d465e52746be2b4bae983670711e6e66657 0000000000000000000000000000000000000000 D	newdir/exec.sh
+	:100755 000000 $newf $zero D	file2/newf
+	:100644 000000 $oldf $zero D	file2/oldf
+	:100755 000000 $f4id $zero D	file4
+	:100644 100644 $f5id $f5id R100	newdir/interesting	h/e/l/lo
+	:100755 000000 $f6id $zero D	newdir/exec.sh
 	EOF
 	git diff-tree -M -r H^ H >actual &&
 	compare_diff_raw expect actual
@@ -935,14 +941,15 @@ test_expect_success 'L: verify internal tree sorting' '
 	INPUT_END
 
 	cat >expect <<-EXPECT_END &&
-	:100644 100644 4268632... 55d3a52... M	b.
-	:040000 040000 0ae5cac... 443c768... M	b
-	:100644 100644 4268632... 55d3a52... M	ba
+	:100644 100644 M	b.
+	:040000 040000 M	b
+	:100644 100644 M	ba
 	EXPECT_END
 
 	git fast-import <input &&
 	GIT_PRINT_SHA1_ELLIPSIS="yes" git diff-tree --abbrev --raw L^ L >output &&
-	test_cmp expect output
+	cut -d" " -f1,2,5 output >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'L: nested tree copy does not corrupt deltas' '
@@ -1004,7 +1011,7 @@ test_expect_success 'M: rename file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf R100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M1^ M1 >actual &&
@@ -1025,7 +1032,7 @@ test_expect_success 'M: rename file to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	file2/newf	i/am/new/to/you
+	:100755 100755 $newf $newf R100	file2/newf	i/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M2^ M2 >actual &&
@@ -1046,7 +1053,7 @@ test_expect_success 'M: rename subdirectory to new subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	other/sub/am/new/to/you
+	:100755 100755 $newf $newf R100	i/am/new/to/you	other/sub/am/new/to/you
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M3^ M3 >actual &&
@@ -1067,11 +1074,11 @@ test_expect_success 'M: rename root to subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 R100	file2/oldf	sub/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 R100	file4	sub/file4
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc R100	i/am/new/to/you	sub/i/am/new/to/you
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 R100	newdir/exec.sh	sub/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 R100	newdir/interesting	sub/newdir/interesting
+	:100644 100644 $oldf $oldf R100	file2/oldf	sub/file2/oldf
+	:100755 100755 $f4id $f4id R100	file4	sub/file4
+	:100755 100755 $newf $newf R100	i/am/new/to/you	sub/i/am/new/to/you
+	:100755 100755 $f6id $f6id R100	newdir/exec.sh	sub/newdir/exec.sh
+	:100644 100644 $f5id $f5id R100	newdir/interesting	sub/newdir/interesting
 	EOF
 	git fast-import <input &&
 	git diff-tree -M -r M4^ M4 >actual &&
@@ -1097,7 +1104,7 @@ test_expect_success 'N: copy file in same subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file2/n.e.w.f
+	:100755 100755 $newf $newf C100	file2/newf	file2/n.e.w.f
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N1^ N1 >actual &&
@@ -1129,9 +1136,9 @@ test_expect_success 'N: copy then modify subdirectory' '
 	INPUT_END
 
 	cat >expect <<-EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git fast-import <input &&
 	git diff-tree -C --find-copies-harder -r N2^^ N2 >actual &&
@@ -1162,9 +1169,9 @@ test_expect_success 'N: copy dirty subdirectory' '
 '
 
 test_expect_success 'N: copy directory by id' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -1183,9 +1190,9 @@ test_expect_success 'N: copy directory by id' '
 '
 
 test_expect_success PIPE 'N: read and copy directory' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	git update-ref -d refs/heads/N4 &&
 	rm -f backflow &&
@@ -1254,9 +1261,9 @@ test_expect_success PIPE 'N: empty directory reads as missing' '
 '
 
 test_expect_success 'N: copy root directory by tree hash' '
-	cat >expect <<-\EOF &&
-	:100755 000000 f1fb5da718392694d0076d677d6d0e364c79b0bc 0000000000000000000000000000000000000000 D	file3/newf
-	:100644 000000 7123f7f44e39be127c5eb701e5968176ee9d78b1 0000000000000000000000000000000000000000 D	file3/oldf
+	cat >expect <<-EOF &&
+	:100755 000000 $newf $zero D	file3/newf
+	:100644 000000 $oldf $zero D	file3/oldf
 	EOF
 	root=$(git rev-parse refs/heads/branch^0^{tree}) &&
 	cat >input <<-INPUT_END &&
@@ -1275,12 +1282,12 @@ test_expect_success 'N: copy root directory by tree hash' '
 '
 
 test_expect_success 'N: copy root by path' '
-	cat >expect <<-\EOF &&
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	oldroot/file2/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	oldroot/file2/oldf
-	:100755 100755 85df50785d62d3b05ab03d9cbf7e4a0b49449730 85df50785d62d3b05ab03d9cbf7e4a0b49449730 C100	file4	oldroot/file4
-	:100755 100755 e74b7d465e52746be2b4bae983670711e6e66657 e74b7d465e52746be2b4bae983670711e6e66657 C100	newdir/exec.sh	oldroot/newdir/exec.sh
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	oldroot/newdir/interesting
+	cat >expect <<-EOF &&
+	:100755 100755 $newf $newf C100	file2/newf	oldroot/file2/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	oldroot/file2/oldf
+	:100755 100755 $f4id $f4id C100	file4	oldroot/file4
+	:100755 100755 $f6id $f6id C100	newdir/exec.sh	oldroot/newdir/exec.sh
+	:100644 100644 $f5id $f5id C100	newdir/interesting	oldroot/newdir/interesting
 	EOF
 	cat >input <<-INPUT_END &&
 	commit refs/heads/N-copy-root-path
@@ -1340,10 +1347,10 @@ test_expect_success 'N: delete directory by copying' '
 '
 
 test_expect_success 'N: modify copied tree' '
-	cat >expect <<-\EOF &&
-	:100644 100644 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 fcf778cda181eaa1cbc9e9ce3a2e15ee9f9fe791 C100	newdir/interesting	file3/file5
-	:100755 100755 f1fb5da718392694d0076d677d6d0e364c79b0bc f1fb5da718392694d0076d677d6d0e364c79b0bc C100	file2/newf	file3/newf
-	:100644 100644 7123f7f44e39be127c5eb701e5968176ee9d78b1 7123f7f44e39be127c5eb701e5968176ee9d78b1 C100	file2/oldf	file3/oldf
+	cat >expect <<-EOF &&
+	:100644 100644 $f5id $f5id C100	newdir/interesting	file3/file5
+	:100755 100755 $newf $newf C100	file2/newf	file3/newf
+	:100644 100644 $oldf $oldf C100	file2/oldf	file3/oldf
 	EOF
 	subdir=$(git rev-parse refs/heads/branch^0:file2) &&
 	cat >input <<-INPUT_END &&
@@ -3117,6 +3124,9 @@ test_expect_success 'U: initialize for U tests' '
 
 	INPUT_END
 
+	f7id=$(echo "blob 1" | git hash-object --stdin) &&
+	f8id=$(echo "sleep well" | git hash-object --stdin) &&
+	f9id=$(echo "au revoir" | git hash-object --stdin) &&
 	git fast-import <input
 '
 
@@ -3137,7 +3147,7 @@ test_expect_success 'U: filedelete file succeeds' '
 
 test_expect_success 'U: validate file delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 2907ebb4bf85d91bf0716bb3bd8a68ef48d6da76 0000000000000000000000000000000000000000 D	good/night.txt
+	:100644 000000 $f8id $ZERO_OID D	good/night.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3162,7 +3172,7 @@ test_expect_success 'U: filedelete directory succeeds' '
 
 test_expect_success 'U: validate directory delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 69cb75792f55123d8389c156b0b41c2ff00ed507 0000000000000000000000000000000000000000 D	good/bye.txt
+	:100644 000000 $f9id $ZERO_OID D	good/bye.txt
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&
@@ -3187,7 +3197,7 @@ test_expect_success 'U: filedelete root succeeds' '
 
 test_expect_success 'U: validate root delete result' '
 	cat >expect <<-EOF &&
-	:100644 000000 c18147dc648481eeb65dc5e66628429a64843327 0000000000000000000000000000000000000000 D	hello.c
+	:100644 000000 $f7id $ZERO_OID D	hello.c
 	EOF
 
 	git diff-tree -M -r U^1 U >actual &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 22/39] t9300: use $ZERO_OID instead of hard-coded object ID
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (20 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 23/39] t9301: make hash size independent brian m. carlson
                     ` (18 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9300-fast-import.sh | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index f2d2cb06ad..308c1ef42c 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -92,7 +92,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/to-be-deleted
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :6
@@ -102,7 +102,7 @@ test_expect_success 'A: create pack from stdin' '
 	EOF
 
 	reset refs/tags/nested
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	tag nested
 	mark :7
@@ -286,7 +286,7 @@ test_expect_success 'A: verify pack' '
 test_expect_success 'A: verify diff' '
 	copy=$(git rev-parse --verify master:file2) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $copy A	copy-of-file2
+	:000000 100755 $ZERO_OID $copy A	copy-of-file2
 	EOF
 	git diff-tree -M -r master verify--import-marks >actual &&
 	compare_diff_raw expect actual &&
@@ -365,7 +365,7 @@ test_expect_success 'B: fail on invalid blob sha1' '
 	COMMIT
 
 	from refs/heads/master
-	M 755 0000000000000000000000000000000000000001 zero1
+	M 755 $(echo $ZERO_OID | sed -e "s/0$/1/") zero1
 
 	INPUT_END
 
@@ -620,8 +620,8 @@ test_expect_success 'D: validate new files added' '
 	f5id=$(echo "$file5_data" | git hash-object --stdin) &&
 	f6id=$(echo "$file6_data" | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:000000 100755 0000000000000000000000000000000000000000 $f6id A	newdir/exec.sh
-	:000000 100644 0000000000000000000000000000000000000000 $f5id A	newdir/interesting
+	:000000 100755 $ZERO_OID $f6id A	newdir/exec.sh
+	:000000 100644 $ZERO_OID $f5id A	newdir/interesting
 	EOF
 	git diff-tree -M -r branch^ branch >actual &&
 	compare_diff_raw expect actual
@@ -2733,7 +2733,7 @@ test_expect_success 'R: corrupt lines do not mess marks file' '
 	rm -f io.marks &&
 	blob=$(echo hi | git hash-object --stdin) &&
 	cat >expect <<-EOF &&
-	:3 0000000000000000000000000000000000000000
+	:3 $ZERO_OID
 	:1 $blob
 	:2 $blob
 	EOF
@@ -3084,7 +3084,7 @@ test_expect_success 'T: delete branch' '
 	git branch to-delete &&
 	git fast-import <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 	EOF
 	test_must_fail git rev-parse --verify refs/heads/to-delete
 '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 23/39] t9301: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (21 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 24/39] t9350: " brian m. carlson
                     ` (17 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Instead of using a hard-coded all-zeros object ID, use $ZERO_OID.
Compute the length of the object IDs in use and use this instead of
hard-coding the constant 40.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9301-fast-import-notes.sh | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/t/t9301-fast-import-notes.sh b/t/t9301-fast-import-notes.sh
index ca223dca98..14c1baa739 100755
--- a/t/t9301-fast-import-notes.sh
+++ b/t/t9301-fast-import-notes.sh
@@ -470,12 +470,13 @@ test_expect_success 'add lots of commits and notes' '
 '
 
 test_expect_success 'verify that lots of notes trigger a fanout scheme' '
+	hexsz=$(test_oid hexsz) &&
 
 	# None of the entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -518,7 +519,7 @@ test_expect_success 'verify that importing a notes tree respects the fanout sche
 	git ls-tree --name-only refs/notes/other_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -593,7 +594,7 @@ test_expect_success 'verify that changing notes respect existing fanout' '
 	git ls-tree --name-only refs/notes/many_notes |
 	while read path
 	do
-		if test $(expr length "$path") -ge 40
+		if test $(expr length "$path") -ge $hexsz
 		then
 			return 1
 		fi
@@ -616,7 +617,7 @@ i=$(($num_commits - $remaining_notes))
 for sha1 in $(git rev-list -n $i refs/heads/many_commits)
 do
 	cat >>input <<INPUT_END
-N 0000000000000000000000000000000000000000 $sha1
+N $ZERO_OID $sha1
 INPUT_END
 done
 
@@ -646,7 +647,6 @@ test_expect_success 'remove lots of notes' '
 '
 
 test_expect_success 'verify that removing notes trigger fanout consolidation' '
-
 	# All entries in the top-level notes tree should be a full SHA1
 	git ls-tree --name-only -r refs/notes/many_notes |
 	while read path
@@ -656,7 +656,7 @@ test_expect_success 'verify that removing notes trigger fanout consolidation' '
 		test "$path" = "deadbeef" && continue
 		test "$path" = "de/adbeef" && continue
 
-		if test $(expr length "$path") -ne 40
+		if test $(expr length "$path") -ne $hexsz
 		then
 			return 1
 		fi

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 24/39] t9350: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (22 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 23/39] t9301: make hash size independent brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
                     ` (16 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test checks for several commit object sizes to verify that objects
are encoded as expected. However, the size of a commit object differs
between SHA-1 and SHA-256, since each contains a hex representation of
the tree's object ID. Since these are root commits, compute the size of
each commit by using a constant plus the size of a single hex object ID.

In addition, use $ZERO_OID instead of a hard-coded object ID.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9350-fast-export.sh | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/t/t9350-fast-export.sh b/t/t9350-fast-export.sh
index 690c90fb82..1372842559 100755
--- a/t/t9350-fast-export.sh
+++ b/t/t9350-fast-export.sh
@@ -132,12 +132,12 @@ test_expect_success 'reencoding iso-8859-7' '
 	sed "s/wer/i18n/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, would be 240 bytes.
+		 # The commit object, if not re-encoded, would be 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header drops 20 bytes.
 		 # Re-encoding the Pi character from \xF0 (\360) in iso-8859-7
 		 # to \xCF\x80 (\317\200) in UTF-8 adds a byte.  Check for
 		 # the expected size.
-		 test 221 -eq "$(git cat-file -s i18n)" &&
+		 test $(($(test_oid hexsz) + 181)) -eq "$(git cat-file -s i18n)" &&
 		 # ...and for the expected translation of bytes.
 		 git cat-file commit i18n >actual &&
 		 grep $(printf "\317\200") actual &&
@@ -164,12 +164,12 @@ test_expect_success 'preserving iso-8859-7' '
 	sed "s/wer/i18n-no-recoding/" iso-8859-7.fi |
 		(cd new &&
 		 git fast-import &&
-		 # The commit object, if not re-encoded, is 240 bytes.
+		 # The commit object, if not re-encoded, is 200 bytes plus hash.
 		 # Removing the "encoding iso-8859-7\n" header would drops 20
 		 # bytes.  Re-encoding the Pi character from \xF0 (\360) in
 		 # iso-8859-7 to \xCF\x80 (\317\200) in UTF-8 adds a byte.
 		 # Check for the expected size...
-		 test 240 -eq "$(git cat-file -s i18n-no-recoding)" &&
+		 test $(($(test_oid hexsz) + 200)) -eq "$(git cat-file -s i18n-no-recoding)" &&
 		 # ...as well as the expected byte.
 		 git cat-file commit i18n-no-recoding >actual &&
 		 grep $(printf "\360") actual &&
@@ -192,7 +192,7 @@ test_expect_success 'encoding preserved if reencoding fails' '
 		 grep ^encoding actual &&
 		 # Verify that the commit has the expected size; i.e.
 		 # that no bytes were re-encoded to a different encoding.
-		 test 252 -eq "$(git cat-file -s i18n-invalid)" &&
+		 test $(($(test_oid hexsz) + 212)) -eq "$(git cat-file -s i18n-invalid)" &&
 		 # ...and check for the original special bytes
 		 grep $(printf "\360") actual &&
 		 grep $(printf "\377") actual)
@@ -694,7 +694,7 @@ test_expect_success 'delete ref because entire history excluded' '
 	git fast-export to-delete ^to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual
@@ -704,7 +704,7 @@ test_expect_success 'delete refspec' '
 	git fast-export --refspec :refs/heads/to-delete >actual &&
 	cat >expected <<-EOF &&
 	reset refs/heads/to-delete
-	from 0000000000000000000000000000000000000000
+	from $ZERO_OID
 
 	EOF
 	test_cmp expected actual

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 25/39] t9500: ensure that algorithm info is preserved in config
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (23 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 24/39] t9350: " brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 26/39] t9700: make hash size independent brian m. carlson
                     ` (15 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

When we use a hash algorithm other than SHA-1, it's important to
preserve the hash-related values in the config file, but this test
overwrites the config file with a new one. Ensure we copy these values
properly from the old config to the new one so that the repository can
be read if it's using SHA-256.

Note that if there is no extensions.objectFormat value set, git config
will return unsuccessfully if we try to read it; since this is not an
error for us, use test_might_fail.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9500-gitweb-standalone-no-errors.sh | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/t/t9500-gitweb-standalone-no-errors.sh b/t/t9500-gitweb-standalone-no-errors.sh
index 267ddc997d..b484e3e250 100755
--- a/t/t9500-gitweb-standalone-no-errors.sh
+++ b/t/t9500-gitweb-standalone-no-errors.sh
@@ -621,12 +621,22 @@ test_expect_success \
 	 git config gitweb.snapshot "zip,tgz, tbz2" &&
 	 gitweb_run "p=.git;a=tree"'
 
-cat >.git/config <<\EOF
-# testing noval and alternate separator
-[gitweb]
-	blame
-	snapshot = zip tgz
-EOF
+test_expect_success 'setup' '
+	version=$(git config core.repositoryformatversion) &&
+	algo=$(test_might_fail git config extensions.objectformat) &&
+	cat >.git/config <<-\EOF &&
+	# testing noval and alternate separator
+	[gitweb]
+		blame
+		snapshot = zip tgz
+	EOF
+	git config core.repositoryformatversion "$version" &&
+	if test -n "$algo"
+	then
+		git config extensions.objectformat "$algo"
+	fi
+'
+
 test_expect_success \
 	'config override: tree view, features enabled in repo config (2)' \
 	'gitweb_run "p=.git;a=tree"'

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 26/39] t9700: make hash size independent
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (24 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 27/39] t5308: make test work with SHA-256 brian m. carlson
                     ` (14 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The Perl test script for t9700 was matching on exactly 40 hex
characters.  With SHA-256, we'll have 64 hex-character object IDs.
Create a variable with a regex which matches exactly 40 or 64 hex
characters and use that to match the output.  Note that both of the uses
of this can be anchored, which makes the code simpler, so do that as
well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t9700/test.pl | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/t9700/test.pl b/t/t9700/test.pl
index 34cd01366f..071ff6d82a 100755
--- a/t/t9700/test.pl
+++ b/t/t9700/test.pl
@@ -23,6 +23,8 @@ sub adjust_dirsep {
 	return $path;
 }
 
+my $oid_re = qr/^[0-9a-fA-F]{40}(?:[0-9a-fA-F]{24})?$/;
+
 BEGIN { use_ok('Git') }
 
 # set up
@@ -93,7 +95,7 @@ sub adjust_dirsep {
 open TEMPFILE, ">$tmpfile" or die "Can't open $tmpfile: $!";
 print TEMPFILE my $test_text = "test blob, to be inserted\n";
 close TEMPFILE or die "Failed writing to $tmpfile: $!";
-like(our $newhash = $r->hash_and_insert_object($tmpfile), qr/[0-9a-fA-F]{40}/,
+like(our $newhash = $r->hash_and_insert_object($tmpfile), $oid_re,
      "hash_and_insert_object: returns hash");
 open TEMPFILE, "+>$tmpfile" or die "Can't open $tmpfile: $!";
 is($r->cat_blob($newhash, \*TEMPFILE), length $test_text, "cat_blob: roundtrip size");
@@ -119,7 +121,7 @@ sub adjust_dirsep {
 
 # commands in sub directory
 my $last_commit = $r2->command_oneline(qw(rev-parse --verify HEAD));
-like($last_commit, qr/^[0-9a-fA-F]{40}$/, 'rev-parse returned hash');
+like($last_commit, $oid_re, 'rev-parse returned hash');
 my $dir_commit = $r2->command_oneline('log', '-n1', '--pretty=format:%H', '.');
 isnt($last_commit, $dir_commit, 'log . does not show last commit');
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 27/39] t5308: make test work with SHA-256
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (25 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 26/39] t9700: make hash size independent brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
                     ` (13 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

This test needs multiple object IDs that have the same first byte.
Update the pack test code to generate a suitable packed value for
SHA-256.  Update the test to use this value when using SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                     |  8 ++++++++
 t/t5308-pack-detect-duplicates.sh | 20 ++++++++++++--------
 2 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index f3463170b3..0c799d53c6 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -93,6 +93,14 @@ pack_obj () {
 			;;
 		esac
 		;;
+	# blob containing "\3\326"
+	471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe)
+		case "$2" in
+		'')
+			printf '\062\170\234\143\276\006\000\000\336\000\332'
+			return
+			;;
+		esac
 	esac
 
 	# If it's not a delta, we can convince pack-objects to generate a pack
diff --git a/t/t5308-pack-detect-duplicates.sh b/t/t5308-pack-detect-duplicates.sh
index 6845c1f3c3..693b2411c8 100755
--- a/t/t5308-pack-detect-duplicates.sh
+++ b/t/t5308-pack-detect-duplicates.sh
@@ -4,23 +4,27 @@ test_description='handling of duplicate objects in incoming packfiles'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/lib-pack.sh
 
-if ! test_have_prereq SHA1
-then
-       skip_all='not using SHA-1 for objects'
-       test_done
-fi
+test_expect_success 'setup' '
+	test_oid_cache <<-EOF
+	lo_oid sha1:e68fe8129b546b101aee9510c5328e7f21ca1d18
+	lo_oid sha256:471819e8c52bf11513f100b2810a8aa0622d5cd3d1c913758a071dd4b3bad8fe
+
+	missing_oid sha1:e69d000000000000000000000000000000000000
+	missing_oid sha256:4720000000000000000000000000000000000000000000000000000000000000
+	EOF
+'
 
 # The sha1s we have in our pack. It's important that these have the same
 # starting byte, so that they end up in the same fanout section of the index.
 # That lets us make sure we are exercising the binary search with both sets.
-LO_SHA1=e68fe8129b546b101aee9510c5328e7f21ca1d18
-HI_SHA1=e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+LO_SHA1=$(test_oid lo_oid)
+HI_SHA1=$EMPTY_BLOB
 
 # And here's a "missing sha1" which will produce failed lookups. It must also
 # be in the same fanout section, and should be between the two (so that during
 # our binary search, we are sure to end up looking at one or the other of the
 # duplicate runs).
-MISSING_SHA1='e69d000000000000000000000000000000000000'
+MISSING_SHA1=$(test_oid missing_oid)
 
 # git will never intentionally create packfiles with
 # duplicate objects, so we have to construct them by hand.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 28/39] t0410: mark test with SHA1 prerequisite
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (26 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 27/39] t5308: make test work with SHA-256 brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
                     ` (12 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

These tests try to check that we behave properly if we encounter a
repository with version 0 but an extension.  This is a laudable goal,
but the test cannot work with SHA-256, since SHA-256 repositories always
have an existing extension and are never version 0.

Add a SHA1 prerequisite to these tests.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0410-partial-clone.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/t0410-partial-clone.sh b/t/t0410-partial-clone.sh
index 6aa0f313bd..a5ebdf9ff3 100755
--- a/t/t0410-partial-clone.sh
+++ b/t/t0410-partial-clone.sh
@@ -42,7 +42,7 @@ test_expect_success 'convert shallow clone to partial clone' '
 	test_cmp_config -C client 1 core.repositoryformatversion
 '
 
-test_expect_success 'convert to partial clone with noop extension' '
+test_expect_success SHA1 'convert to partial clone with noop extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&
@@ -53,7 +53,7 @@ test_expect_success 'convert to partial clone with noop extension' '
 	git -C client fetch --unshallow --filter="blob:none"
 '
 
-test_expect_success 'converting to partial clone fails with unrecognized extension' '
+test_expect_success SHA1 'converting to partial clone fails with unrecognized extension' '
 	rm -fr server client &&
 	test_create_repo server &&
 	test_commit -C server my_commit 1 &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 29/39] http-fetch: set up git directory before parsing pack hashes
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (27 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
                     ` (11 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In dd4b732df7 ("upload-pack: send part of packfile response as uri",
2020-06-10), the git http-fetch code learned how to take  ac --packfile
option.  This option takes an argument, which is the name of a packfile
hash, and parses it using parse_oid_hex.  It does so before calling
setup_git_directory.

However, in a SHA-256 repository this fails to work, since we have not
set the hash algorithm in use and parse_oid_hex fails as a consequence.
To ensure that we can parse packfile hashes of the right length, let's
set up the git directory before we start parsing arguments.

Since we still want to allow the invocation of -h to print the help when
we're not in a repository, gracefully handle us being outside of one and
produce an error after argument parsing has finished.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 http-fetch.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/http-fetch.c b/http-fetch.c
index 1df376e745..c4ccc5fea9 100644
--- a/http-fetch.c
+++ b/http-fetch.c
@@ -84,8 +84,11 @@ int cmd_main(int argc, const char **argv)
 	int get_verbosely = 0;
 	int get_recover = 0;
 	int packfile = 0;
+	int nongit;
 	struct object_id packfile_hash;
 
+	setup_git_directory_gently(&nongit);
+
 	while (arg < argc && argv[arg][0] == '-') {
 		const char *p;
 
@@ -115,7 +118,8 @@ int cmd_main(int argc, const char **argv)
 	if (argc != arg + 2 - (commits_on_stdin || packfile))
 		usage(http_fetch_usage);
 
-	setup_git_directory();
+	if (nongit)
+		die(_("not a git repository"));
 
 	git_config(git_default_config, NULL);
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 30/39] builtin/verify-pack: implement an --object-format option
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (28 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 31/39] bundle: add new version for use with SHA-256 brian m. carlson
                     ` (10 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

A recently added test in t5702 started using git verify-pack outside of
a repository.  While this poses no problems with SHA-1, with SHA-256 we
implicitly rely on the setup of the repository to initialize our hash
algorithm settings.

Since we're not in a repository here, we need to provide git verify-pack
help to set things up properly.  git index-pack already knows an
--object-format option, so let's accept one as well and pass it down to
our git index-pack invocation.  Since we're now dynamically adjusting
the elements in argv, let's switch to using struct argv_array to manage
them.  Finally, let's make t5702 pass the proper argument on down to its
git verify-pack caller.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/verify-pack.c  | 23 +++++++++++++++--------
 t/t5702-protocol-v2.sh |  2 +-
 2 files changed, 16 insertions(+), 9 deletions(-)

diff --git a/builtin/verify-pack.c b/builtin/verify-pack.c
index c2a1a5c504..3669a90263 100644
--- a/builtin/verify-pack.c
+++ b/builtin/verify-pack.c
@@ -7,21 +7,26 @@
 #define VERIFY_PACK_VERBOSE 01
 #define VERIFY_PACK_STAT_ONLY 02
 
-static int verify_one_pack(const char *path, unsigned int flags)
+static int verify_one_pack(const char *path, unsigned int flags, const char *hash_algo)
 {
 	struct child_process index_pack = CHILD_PROCESS_INIT;
-	const char *argv[] = {"index-pack", NULL, NULL, NULL };
+	struct argv_array *argv = &index_pack.args;
 	struct strbuf arg = STRBUF_INIT;
 	int verbose = flags & VERIFY_PACK_VERBOSE;
 	int stat_only = flags & VERIFY_PACK_STAT_ONLY;
 	int err;
 
+	argv_array_push(argv, "index-pack");
+
 	if (stat_only)
-		argv[1] = "--verify-stat-only";
+		argv_array_push(argv, "--verify-stat-only");
 	else if (verbose)
-		argv[1] = "--verify-stat";
+		argv_array_push(argv, "--verify-stat");
 	else
-		argv[1] = "--verify";
+		argv_array_push(argv, "--verify");
+
+	if (hash_algo)
+		argv_array_pushf(argv, "--object-format=%s", hash_algo);
 
 	/*
 	 * In addition to "foo.pack" we accept "foo.idx" and "foo";
@@ -31,9 +36,8 @@ static int verify_one_pack(const char *path, unsigned int flags)
 	if (strbuf_strip_suffix(&arg, ".idx") ||
 	    !ends_with(arg.buf, ".pack"))
 		strbuf_addstr(&arg, ".pack");
-	argv[2] = arg.buf;
+	argv_array_push(argv, arg.buf);
 
-	index_pack.argv = argv;
 	index_pack.git_cmd = 1;
 
 	err = run_command(&index_pack);
@@ -60,12 +64,15 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 {
 	int err = 0;
 	unsigned int flags = 0;
+	const char *object_format = NULL;
 	int i;
 	const struct option verify_pack_options[] = {
 		OPT_BIT('v', "verbose", &flags, N_("verbose"),
 			VERIFY_PACK_VERBOSE),
 		OPT_BIT('s', "stat-only", &flags, N_("show statistics only"),
 			VERIFY_PACK_STAT_ONLY),
+		OPT_STRING(0, "object-format", &object_format, N_("hash"),
+			   N_("specify the hash algorithm to use")),
 		OPT_END()
 	};
 
@@ -75,7 +82,7 @@ int cmd_verify_pack(int argc, const char **argv, const char *prefix)
 	if (argc < 1)
 		usage_with_options(verify_pack_usage, verify_pack_options);
 	for (i = 0; i < argc; i++) {
-		if (verify_one_pack(argv[i], flags))
+		if (verify_one_pack(argv[i], flags, object_format))
 			err = 1;
 	}
 
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 1b54c35b01..7fc22171e7 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -829,7 +829,7 @@ test_expect_success 'part of packfile response provided as URI' '
 	# Ensure that my-blob and other-blob are in separate packfiles.
 	for idx in http_child/.git/objects/pack/*.idx
 	do
-		git verify-pack --verbose $idx >out &&
+		git verify-pack --object-format=$(test_oid algo) --verbose $idx >out &&
 		{
 			grep "^[0-9a-f]\{16,\} " out || :
 		} >out.objectlist &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 31/39] bundle: add new version for use with SHA-256
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (29 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 32/39] setup: add support for reading extensions.objectformat brian m. carlson
                     ` (9 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Currently we detect the hash algorithm in use by the length of the
object ID.  This is inelegant and prevents us from using a different
hash algorithm that is also 256 bits in length.

Since we cannot extend the v2 format in a backward-compatible way, let's
add a v3 format, which is identical, except for the addition of
capabilities, which are prefixed by an at sign.  We add "object-format"
as the only capability and reject unknown capabilities, since we do not
have a network connection and therefore cannot negotiate with the other
side.

For compatibility, default to the v2 format for SHA-1 and require v3
for SHA-256.

In t5510, always use format v3 so we can be sure we produce consistent
results across hash algorithms.  Since head -n N lists the top N lines
instead of the Nth line, let's run our output through sed to normalize
it and compare it against a fixed value, which will make sure we get
exactly what we're expecting.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/git-bundle.txt              |  9 ++-
 Documentation/technical/bundle-format.txt | 30 +++++++-
 builtin/bundle.c                          |  5 +-
 bundle.c                                  | 83 +++++++++++++++++------
 bundle.h                                  |  4 +-
 t/t5510-fetch.sh                          | 16 +++--
 t/t5607-clone-bundle.sh                   | 31 +++++++++
 7 files changed, 147 insertions(+), 31 deletions(-)

diff --git a/Documentation/git-bundle.txt b/Documentation/git-bundle.txt
index d34b0964be..53804cad4b 100644
--- a/Documentation/git-bundle.txt
+++ b/Documentation/git-bundle.txt
@@ -9,7 +9,8 @@ git-bundle - Move objects and refs by archive
 SYNOPSIS
 --------
 [verse]
-'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied] <file> <git-rev-list-args>
+'git bundle' create [-q | --quiet | --progress | --all-progress] [--all-progress-implied]
+		    [--version=<version>] <file> <git-rev-list-args>
 'git bundle' verify [-q | --quiet] <file>
 'git bundle' list-heads <file> [<refname>...]
 'git bundle' unbundle <file> [<refname>...]
@@ -102,6 +103,12 @@ unbundle <file>::
 	is activated.  Unlike --all-progress this flag doesn't actually
 	force any progress display by itself.
 
+--version=<version>::
+	Specify the bundle version.  Version 2 is the older format and can only be
+	used with SHA-1 repositories; the newer version 3 contains capabilities that
+	permit extensions. The default is the oldest supported format, based on the
+	hash algorithm in use.
+
 -q::
 --quiet::
 	This flag makes the command not to report its progress
diff --git a/Documentation/technical/bundle-format.txt b/Documentation/technical/bundle-format.txt
index 0e828151a5..bac558d049 100644
--- a/Documentation/technical/bundle-format.txt
+++ b/Documentation/technical/bundle-format.txt
@@ -7,6 +7,8 @@ The Git bundle format is a format that represents both refs and Git objects.
 We will use ABNF notation to define the Git bundle format. See
 protocol-common.txt for the details.
 
+A v2 bundle looks like this:
+
 ----
 bundle    = signature *prerequisite *reference LF pack
 signature = "# v2 git bundle" LF
@@ -18,9 +20,28 @@ reference    = obj-id SP refname LF
 pack         = ... ; packfile
 ----
 
+A v3 bundle looks like this:
+
+----
+bundle    = signature *capability *prerequisite *reference LF pack
+signature = "# v3 git bundle" LF
+
+capability   = "@" key ["=" value] LF
+prerequisite = "-" obj-id SP comment LF
+comment      = *CHAR
+reference    = obj-id SP refname LF
+key          = 1*(ALPHA / DIGIT / "-")
+value        = *(%01-09 / %0b-FF)
+
+pack         = ... ; packfile
+----
+
 == Semantics
 
-A Git bundle consists of three parts.
+A Git bundle consists of several parts.
+
+* "Capabilities", which are only in the v3 format, indicate functionality that
+	the bundle requires to be read properly.
 
 * "Prerequisites" lists the objects that are NOT included in the bundle and the
   reader of the bundle MUST already have, in order to use the data in the
@@ -46,3 +67,10 @@ put any string here. The reader of the bundle MUST ignore the comment.
 Note that the prerequisites does not represent a shallow-clone boundary. The
 semantics of the prerequisites and the shallow-clone boundaries are different,
 and the Git bundle v2 format cannot represent a shallow clone repository.
+
+== Capabilities
+
+Because there is no opportunity for negotiation, unknown capabilities cause 'git
+bundle' to abort.  The only known capability is `object-format`, which specifies
+the hash algorithm in use, and can take the same values as the
+`extensions.objectFormat` configuration value.
diff --git a/builtin/bundle.c b/builtin/bundle.c
index f049d27a14..e1a85e7dcc 100644
--- a/builtin/bundle.c
+++ b/builtin/bundle.c
@@ -60,6 +60,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 	int all_progress_implied = 0;
 	int progress = isatty(STDERR_FILENO);
 	struct argv_array pack_opts;
+	int version = -1;
 
 	struct option options[] = {
 		OPT_SET_INT('q', "quiet", &progress,
@@ -71,6 +72,8 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 		OPT_BOOL(0, "all-progress-implied",
 			 &all_progress_implied,
 			 N_("similar to --all-progress when progress meter is shown")),
+		OPT_INTEGER(0, "version", &version,
+			    N_("specify bundle format version")),
 		OPT_END()
 	};
 	const char* bundle_file;
@@ -91,7 +94,7 @@ static int cmd_bundle_create(int argc, const char **argv, const char *prefix) {
 
 	if (!startup_info->have_repository)
 		die(_("Need a repository to create a bundle."));
-	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts);
+	return !!create_bundle(the_repository, bundle_file, argc, argv, &pack_opts, version);
 }
 
 static int cmd_bundle_verify(int argc, const char **argv, const char *prefix) {
diff --git a/bundle.c b/bundle.c
index 2a0d744d3f..35585f237c 100644
--- a/bundle.c
+++ b/bundle.c
@@ -12,7 +12,16 @@
 #include "refs.h"
 #include "argv-array.h"
 
-static const char bundle_signature[] = "# v2 git bundle\n";
+
+static const char v2_bundle_signature[] = "# v2 git bundle\n";
+static const char v3_bundle_signature[] = "# v3 git bundle\n";
+static struct {
+	int version;
+	const char *signature;
+} bundle_sigs[] = {
+	{ 2, v2_bundle_signature },
+	{ 3, v3_bundle_signature },
+};
 
 static void add_to_ref_list(const struct object_id *oid, const char *name,
 		struct ref_list *list)
@@ -23,15 +32,30 @@ static void add_to_ref_list(const struct object_id *oid, const char *name,
 	list->nr++;
 }
 
-static const struct git_hash_algo *detect_hash_algo(struct strbuf *buf)
+static int parse_capability(struct bundle_header *header, const char *capability)
 {
-	size_t len = strcspn(buf->buf, " \n");
-	int algo;
+	const char *arg;
+	if (skip_prefix(capability, "object-format=", &arg)) {
+		int algo = hash_algo_by_name(arg);
+		if (algo == GIT_HASH_UNKNOWN)
+			return error(_("unrecognized bundle hash algorithm: %s"), arg);
+		header->hash_algo = &hash_algos[algo];
+		return 0;
+	}
+	return error(_("unknown capability '%s'"), capability);
+}
 
-	algo = hash_algo_by_length(len / 2);
-	if (algo == GIT_HASH_UNKNOWN)
-		return NULL;
-	return &hash_algos[algo];
+static int parse_bundle_signature(struct bundle_header *header, const char *line)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(bundle_sigs); i++) {
+		if (!strcmp(line, bundle_sigs[i].signature)) {
+			header->version = bundle_sigs[i].version;
+			return 0;
+		}
+	}
+	return -1;
 }
 
 static int parse_bundle_header(int fd, struct bundle_header *header,
@@ -42,14 +66,16 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 
 	/* The bundle header begins with the signature */
 	if (strbuf_getwholeline_fd(&buf, fd, '\n') ||
-	    strcmp(buf.buf, bundle_signature)) {
+	    parse_bundle_signature(header, buf.buf)) {
 		if (report_path)
-			error(_("'%s' does not look like a v2 bundle file"),
+			error(_("'%s' does not look like a v2 or v3 bundle file"),
 			      report_path);
 		status = -1;
 		goto abort;
 	}
 
+	header->hash_algo = the_hash_algo;
+
 	/* The bundle header ends with an empty line */
 	while (!strbuf_getwholeline_fd(&buf, fd, '\n') &&
 	       buf.len && buf.buf[0] != '\n') {
@@ -57,19 +83,19 @@ static int parse_bundle_header(int fd, struct bundle_header *header,
 		int is_prereq = 0;
 		const char *p;
 
-		if (*buf.buf == '-') {
-			is_prereq = 1;
-			strbuf_remove(&buf, 0, 1);
-		}
 		strbuf_rtrim(&buf);
 
-		if (!header->hash_algo) {
-			header->hash_algo = detect_hash_algo(&buf);
-			if (!header->hash_algo) {
-				error(_("unknown hash algorithm length"));
+		if (header->version == 3 && *buf.buf == '@') {
+			if (parse_capability(header, buf.buf + 1)) {
 				status = -1;
 				break;
 			}
+			continue;
+		}
+
+		if (*buf.buf == '-') {
+			is_prereq = 1;
+			strbuf_remove(&buf, 0, 1);
 		}
 
 		/*
@@ -449,13 +475,14 @@ static int write_bundle_refs(int bundle_fd, struct rev_info *revs)
 }
 
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options)
+		  int argc, const char **argv, struct argv_array *pack_options, int version)
 {
 	struct lock_file lock = LOCK_INIT;
 	int bundle_fd = -1;
 	int bundle_to_stdout;
 	int ref_count = 0;
 	struct rev_info revs;
+	int min_version = the_hash_algo == &hash_algos[GIT_HASH_SHA1] ? 2 : 3;
 
 	bundle_to_stdout = !strcmp(path, "-");
 	if (bundle_to_stdout)
@@ -464,8 +491,22 @@ int create_bundle(struct repository *r, const char *path,
 		bundle_fd = hold_lock_file_for_update(&lock, path,
 						      LOCK_DIE_ON_ERROR);
 
-	/* write signature */
-	write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
+	if (version == -1)
+		version = min_version;
+
+	if (version < 2 || version > 3) {
+		die(_("unsupported bundle version %d"), version);
+	} else if (version < min_version) {
+		die(_("cannot write bundle version %d with algorithm %s"), version, the_hash_algo->name);
+	} else if (version == 2) {
+		write_or_die(bundle_fd, v2_bundle_signature, strlen(v2_bundle_signature));
+	} else {
+		const char *capability = "@object-format=";
+		write_or_die(bundle_fd, v3_bundle_signature, strlen(v3_bundle_signature));
+		write_or_die(bundle_fd, capability, strlen(capability));
+		write_or_die(bundle_fd, the_hash_algo->name, strlen(the_hash_algo->name));
+		write_or_die(bundle_fd, "\n", 1);
+	}
 
 	/* init revs to list objects for pack-objects later */
 	save_commit_buffer = 0;
diff --git a/bundle.h b/bundle.h
index 2dc9442024..70c84cab08 100644
--- a/bundle.h
+++ b/bundle.h
@@ -13,6 +13,7 @@ struct ref_list {
 };
 
 struct bundle_header {
+	unsigned version;
 	struct ref_list prerequisites;
 	struct ref_list references;
 	const struct git_hash_algo *hash_algo;
@@ -21,7 +22,8 @@ struct bundle_header {
 int is_bundle(const char *path, int quiet);
 int read_bundle_header(const char *path, struct bundle_header *header);
 int create_bundle(struct repository *r, const char *path,
-		  int argc, const char **argv, struct argv_array *pack_options);
+		  int argc, const char **argv, struct argv_array *pack_options,
+		  int version);
 int verify_bundle(struct repository *r, struct bundle_header *header, int verbose);
 #define BUNDLE_VERBOSE 1
 int unbundle(struct repository *r, struct bundle_header *header,
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index a66dbe0bde..9243335ab0 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -281,15 +281,19 @@ test_expect_success 'create bundle 1' '
 	cd "$D" &&
 	echo >file updated again by origin &&
 	git commit -a -m "tip" &&
-	git bundle create bundle1 master^..master
+	git bundle create --version=3 bundle1 master^..master
 '
 
 test_expect_success 'header of bundle looks right' '
-	head -n 4 "$D"/bundle1 &&
-	head -n 1 "$D"/bundle1 | grep "^#" &&
-	head -n 2 "$D"/bundle1 | grep "^-$OID_REGEX " &&
-	head -n 3 "$D"/bundle1 | grep "^$OID_REGEX " &&
-	head -n 4 "$D"/bundle1 | grep "^$"
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	-OID updated by origin
+	OID refs/heads/master
+
+	EOF
+	sed -e "s/$OID_REGEX/OID/g" -e "5q" "$D"/bundle1 >actual &&
+	test_cmp expect actual
 '
 
 test_expect_success 'create bundle 2' '
diff --git a/t/t5607-clone-bundle.sh b/t/t5607-clone-bundle.sh
index 6d5a977fcb..26985f4b44 100755
--- a/t/t5607-clone-bundle.sh
+++ b/t/t5607-clone-bundle.sh
@@ -4,6 +4,10 @@ test_description='some bundle related tests'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
+	test_oid_cache <<-EOF &&
+	version sha1:2
+	version sha256:3
+	EOF
 	test_commit initial &&
 	test_tick &&
 	git tag -m tag tag &&
@@ -94,4 +98,31 @@ test_expect_success 'fetch SHA-1 from bundle' '
 	git fetch --no-tags foo/tip.bundle "$(cat hash)"
 '
 
+test_expect_success 'git bundle uses expected default format' '
+	git bundle create bundle HEAD^.. &&
+	head -n1 bundle | grep "^# v$(test_oid version) git bundle$"
+'
+
+test_expect_success 'git bundle v3 has expected contents' '
+	git branch side HEAD &&
+	git bundle create --version=3 bundle HEAD^..side &&
+	head -n2 bundle >actual &&
+	cat >expect <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	EOF
+	test_cmp expect actual &&
+	git bundle verify bundle
+'
+
+test_expect_success 'git bundle v3 rejects unknown capabilities' '
+	cat >new <<-EOF &&
+	# v3 git bundle
+	@object-format=$(test_oid algo)
+	@unknown=silly
+	EOF
+	test_must_fail git bundle verify new 2>output &&
+	test_i18ngrep "unknown capability .unknown=silly." output
+'
+
 test_done

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 32/39] setup: add support for reading extensions.objectformat
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (30 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 31/39] bundle: add new version for use with SHA-256 brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 33/39] repository: enable SHA-256 support by default brian m. carlson
                     ` (8 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

The transition plan specifies extensions.objectFormat as the indication
that we're using a given hash in a certain repo.  Read this as one of
the extensions we support.  If the user has specified an invalid value,
fail.

Ensure that we reject the extension if the repository format version is
0.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 setup.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/setup.c b/setup.c
index 3a81307602..94e68bb4f4 100644
--- a/setup.c
+++ b/setup.c
@@ -470,7 +470,16 @@ static int check_repo_format(const char *var, const char *value, void *vdata)
 			data->partial_clone = xstrdup(value);
 		} else if (!strcmp(ext, "worktreeconfig"))
 			data->worktree_config = git_config_bool(var, value);
-		else
+		else if (!strcmp(ext, "objectformat")) {
+			int format;
+
+			if (!value)
+				return config_error_nonbool(var);
+			format = hash_algo_by_name(value);
+			if (format == GIT_HASH_UNKNOWN)
+				return error("invalid value for 'extensions.objectformat'");
+			data->hash_algo = format;
+		} else
 			string_list_append(&data->unknown_extensions, ext);
 	}
 
@@ -613,6 +622,11 @@ int verify_repository_format(const struct repository_format *format,
 		return -1;
 	}
 
+	if (format->version <= 0 && format->hash_algo != GIT_HASH_SHA1) {
+		strbuf_addstr(err, _("extensions.objectFormat is not valid in repo v0"));
+		return -1;
+	}
+
 	return 0;
 }
 

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 33/39] repository: enable SHA-256 support by default
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (31 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 32/39] setup: add support for reading extensions.objectformat brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 34/39] t: add test_oid option to select hash algorithm brian m. carlson
                     ` (7 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that we have a complete SHA-256 implementation in Git, let's enable
it so people can use it.  Remove the ENABLE_SHA256 define constant
everywhere it's used.  Add tests for initializing a repository with
SHA-256.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 builtin/init-db.c |  5 -----
 config.mak.dev    |  2 --
 repository.c      |  4 ----
 t/t0001-init.sh   | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 33 insertions(+), 11 deletions(-)

diff --git a/builtin/init-db.c b/builtin/init-db.c
index cee64823cb..f70076d38e 100644
--- a/builtin/init-db.c
+++ b/builtin/init-db.c
@@ -183,11 +183,6 @@ void initialize_repository_version(int hash_algo)
 	char repo_version_string[10];
 	int repo_version = GIT_REPO_VERSION;
 
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), hash_algos[hash_algo].name);
-#endif
-
 	if (hash_algo != GIT_HASH_SHA1)
 		repo_version = GIT_REPO_VERSION_READ;
 
diff --git a/config.mak.dev b/config.mak.dev
index cd4a82a9eb..89b218d11a 100644
--- a/config.mak.dev
+++ b/config.mak.dev
@@ -16,8 +16,6 @@ DEVELOPER_CFLAGS += -Wstrict-prototypes
 DEVELOPER_CFLAGS += -Wunused
 DEVELOPER_CFLAGS += -Wvla
 
-DEVELOPER_CFLAGS += -DENABLE_SHA256
-
 ifndef COMPILER_FEATURES
 COMPILER_FEATURES := $(shell ./detect-compiler $(CC))
 endif
diff --git a/repository.c b/repository.c
index 6f7f6f002b..a4174ddb06 100644
--- a/repository.c
+++ b/repository.c
@@ -89,10 +89,6 @@ void repo_set_gitdir(struct repository *repo,
 void repo_set_hash_algo(struct repository *repo, int hash_algo)
 {
 	repo->hash_algo = &hash_algos[hash_algo];
-#ifndef ENABLE_SHA256
-	if (hash_algo != GIT_HASH_SHA1)
-		die(_("The hash algorithm %s is not supported in this build."), repo->hash_algo->name);
-#endif
 }
 
 /*
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 6d2467995e..d71d4c7238 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -441,6 +441,39 @@ test_expect_success 're-init from a linked worktree' '
 	)
 '
 
+test_expect_success 'init honors GIT_DEFAULT_HASH' '
+	GIT_DEFAULT_HASH=sha1 git init sha1 &&
+	git -C sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	GIT_DEFAULT_HASH=sha256 git init sha256 &&
+	git -C sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'init honors --object-format' '
+	git init --object-format=sha1 explicit-sha1 &&
+	git -C explicit-sha1 rev-parse --show-object-format >actual &&
+	echo sha1 >expected &&
+	test_cmp expected actual &&
+	git init --object-format=sha256 explicit-sha256 &&
+	git -C explicit-sha256 rev-parse --show-object-format >actual &&
+	echo sha256 >expected &&
+	test_cmp expected actual
+'
+
+test_expect_success 'extensions.objectFormat is not allowed with repo version 0' '
+	git init --object-format=sha256 explicit-v0 &&
+	git -C explicit-v0 config core.repositoryformatversion 0 &&
+	test_must_fail git -C explicit-v0 rev-parse --show-object-format
+'
+
+test_expect_success 'init rejects attempts to initialize with different hash' '
+	test_must_fail git -C sha1 init --object-format=sha256 &&
+	test_must_fail git -C sha256 init --object-format=sha1
+'
+
 test_expect_success MINGW 'core.hidedotfiles = false' '
 	git config --global core.hidedotfiles false &&
 	rm -rf newdir &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 34/39] t: add test_oid option to select hash algorithm
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (32 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 33/39] repository: enable SHA-256 support by default brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 35/39] t: allow testing different hash algorithms via environment brian m. carlson
                     ` (6 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

In some tests, we have data files which are written with a particular
hash algorithm. Instead of keeping two copies of the test files, we can
keep one, and translate the value on the fly.

In order to do so, we'll need to read both the source algorithm and the
current algorithm, so add an optional flag to the test_oid helper that
lets us look up a value for a specified hash algorithm. This should
not cause any conflicts with existing tests, since key arguments to
test_oid are allowed to contains only shell identifier characters.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/t0000-basic.sh        | 11 +++++++++++
 t/test-lib-functions.sh | 12 +++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2ff176cd5d..2599d026a2 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -928,6 +928,17 @@ test_expect_success 'test_oid can look up data for SHA-256' '
 	test "$hexsz" -eq 64
 '
 
+test_expect_success 'test_oid can look up data for a specified algorithm' '
+	rawsz="$(test_oid --hash=sha1 rawsz)" &&
+	hexsz="$(test_oid --hash=sha1 hexsz)" &&
+	test "$rawsz" -eq 20 &&
+	test "$hexsz" -eq 40 &&
+	rawsz="$(test_oid --hash=sha256 rawsz)" &&
+	hexsz="$(test_oid --hash=sha256 hexsz)" &&
+	test "$rawsz" -eq 32 &&
+	test "$hexsz" -eq 64
+'
+
 test_expect_success 'test_bool_env' '
 	(
 		sane_unset envvar &&
diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 3103be8a32..2608e80f11 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1468,7 +1468,17 @@ test_oid_cache () {
 # Look up a per-hash value based on a key ($1).  The value must have been loaded
 # by test_oid_init or test_oid_cache.
 test_oid () {
-	local var="test_oid_${test_hash_algo}_$1" &&
+	local algo="${test_hash_algo}" &&
+
+	case "$1" in
+	--hash=*)
+		algo="${1#--hash=}" &&
+		shift;;
+	*)
+		;;
+	esac &&
+
+	local var="test_oid_${algo}_$1" &&
 
 	# If the variable is unset, we must be missing an entry for this
 	# key-hash pair, so exit with an error.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 35/39] t: allow testing different hash algorithms via environment
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (33 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 34/39] t: add test_oid option to select hash algorithm brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
                     ` (5 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

To allow developers to run the testsuite with a different algorithm than
the default, provide an environment variable, GIT_TEST_DEFAULT_HASH, to
specify the algorithm to use. Compute the fixed constants using
test_oid. Move the constant initialization down below the point where
test-lib-functions.sh is loaded so the functions are defined.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib-functions.sh | 4 +---
 t/test-lib.sh           | 3 +++
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/t/test-lib-functions.sh b/t/test-lib-functions.sh
index 2608e80f11..2e01bb2c2b 100644
--- a/t/test-lib-functions.sh
+++ b/t/test-lib-functions.sh
@@ -1417,9 +1417,7 @@ test_set_hash () {
 
 # Detect the hash algorithm in use.
 test_detect_hash () {
-	# Currently we only support SHA-1, but in the future this function will
-	# actually detect the algorithm in use.
-	test_hash_algo='sha1'
+	test_hash_algo="${GIT_TEST_DEFAULT_HASH:-sha1}"
 }
 
 # Load common hash metadata and common placeholder object IDs for use with
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 618a7c8d5b..307bb2207e 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -452,6 +452,9 @@ export GIT_AUTHOR_EMAIL GIT_AUTHOR_NAME
 export GIT_COMMITTER_EMAIL GIT_COMMITTER_NAME
 export EDITOR
 
+GIT_DEFAULT_HASH="${GIT_TEST_DEFAULT_HASH:-sha1}"
+export GIT_DEFAULT_HASH
+
 # Tests using GIT_TRACE typically don't want <timestamp> <file>:<line> output
 GIT_TRACE_BARE=1
 export GIT_TRACE_BARE

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 36/39] t: make SHA1 prerequisite depend on default hash
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (34 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 35/39] t: allow testing different hash algorithms via environment brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 37/39] ci: run tests with SHA-256 brian m. carlson
                     ` (4 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Currently, the SHA1 prerequisite depends on the output of git
hash-object.  However, in order for that to produce sane behavior, we
must be in a repository.  If we are not, the default will remain SHA-1,
and we'll produce wrong results if we're using SHA-256 for the testsuite
but the test assertion starts when we're not in a repository.

Check the environment variable we use for this purpose, leaving it to
default to SHA-1 if none is specified.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/test-lib.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index 307bb2207e..0483ed51e1 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -1689,7 +1689,11 @@ test_lazy_prereq CURL '
 # which will not work with other hash algorithms and tests that work but don't
 # test anything meaningful (e.g. special values which cause short collisions).
 test_lazy_prereq SHA1 '
-	test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391
+	case "$GIT_DEFAULT_HASH" in
+	sha1) true ;;
+	"") test $(git hash-object /dev/null) = e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 ;;
+	*) false ;;
+	esac
 '
 
 test_lazy_prereq REBASE_P '

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 37/39] ci: run tests with SHA-256
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (35 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
                     ` (3 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that we have Git supporting SHA-256, we'd like to make sure that we
don't regress that state.  Unfortunately, it's easy to do so, so to
help, let's add code to run one of our CI jobs with SHA-256 as the
default hash.  This will help us detect any problems that may occur.

We pick the linux-clang job because it's relatively fast and the
linux-gcc job already runs the testsuite twice.  We want our tests to
run as fast as possible, so we wouldn't want to add a third run to the
linux-gcc job.  To make sure we properly exercise the code, let's run
the tests in the default mode (SHA-1) first and then run a second time
with SHA-256.  We explicitly specify SHA-1 for the first run so that if
we change the default in the future, we make sure to test both cases.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 ci/run-build-and-tests.sh | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh
index 17e25aade9..6c27b886b8 100755
--- a/ci/run-build-and-tests.sh
+++ b/ci/run-build-and-tests.sh
@@ -24,6 +24,12 @@ linux-gcc)
 	export GIT_TEST_ADD_I_USE_BUILTIN=1
 	make test
 	;;
+linux-clang)
+	export GIT_TEST_DEFAULT_HASH=sha1
+	make test
+	export GIT_TEST_DEFAULT_HASH=sha256
+	make test
+	;;
 linux-gcc-4.8)
 	# Don't run the tests; we only care about whether Git can be
 	# built with GCC 4.8, as it errors out on some undesired (C99)

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 38/39] docs: add documentation for extensions.objectFormat
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (36 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 37/39] ci: run tests with SHA-256 brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-29 23:14   ` [PATCH v6 39/39] t: remove test_oid_init in tests brian m. carlson
                     ` (2 subsequent siblings)
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Document the extensions.objectFormat config setting.  Warn users not to
modify it themselves.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 Documentation/config.txt            | 2 ++
 Documentation/config/extensions.txt | 8 ++++++++
 2 files changed, 10 insertions(+)
 create mode 100644 Documentation/config/extensions.txt

diff --git a/Documentation/config.txt b/Documentation/config.txt
index ef0768b91a..3042d80978 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -348,6 +348,8 @@ include::config/diff.txt[]
 
 include::config/difftool.txt[]
 
+include::config/extensions.txt[]
+
 include::config/fastimport.txt[]
 
 include::config/feature.txt[]
diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt
new file mode 100644
index 0000000000..4e23d73cdc
--- /dev/null
+++ b/Documentation/config/extensions.txt
@@ -0,0 +1,8 @@
+extensions.objectFormat::
+	Specify the hash algorithm to use.  The acceptable values are `sha1` and
+	`sha256`.  If not specified, `sha1` is assumed.  It is an error to specify
+	this key unless `core.repositoryFormatVersion` is 1.
++
+Note that this setting should only be set by linkgit:git-init[1] or
+linkgit:git-clone[1].  Trying to change it after initialization will not
+work and will produce hard-to-diagnose issues.

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH v6 39/39] t: remove test_oid_init in tests
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (37 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
@ 2020-07-29 23:14   ` brian m. carlson
  2020-07-30  2:48   ` [PATCH v6 00/39] SHA-256, part 3/3 Eric Sunshine
  2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
  40 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2020-07-29 23:14 UTC (permalink / raw)
  To: git; +Cc: Eric Sunshine

Now that we call test_oid_init in the setup for all test scripts,
there's no point in calling it individually.  Remove all of the places
where we've done so to help keep tests tidy.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
---
 t/lib-pack.sh                          | 3 ---
 t/lib-submodule-update.sh              | 1 -
 t/t0000-basic.sh                       | 4 ----
 t/t1006-cat-file.sh                    | 2 --
 t/t1050-large.sh                       | 1 -
 t/t1410-reflog.sh                      | 1 -
 t/t1450-fsck.sh                        | 1 -
 t/t1500-rev-parse.sh                   | 1 -
 t/t3308-notes-merge.sh                 | 1 -
 t/t3600-rm.sh                          | 1 -
 t/t3800-mktag.sh                       | 1 -
 t/t4002-diff-basic.sh                  | 2 --
 t/t4027-diff-submodule.sh              | 1 -
 t/t4134-apply-submodule.sh             | 1 -
 t/t4200-rerere.sh                      | 1 -
 t/t4211-line-log.sh                    | 1 -
 t/t5300-pack-object.sh                 | 3 +--
 t/t5302-pack-index.sh                  | 1 -
 t/t5313-pack-bounds-checks.sh          | 1 -
 t/t5318-commit-graph.sh                | 3 +--
 t/t5319-multi-pack-index.sh            | 1 -
 t/t5324-split-commit-graph.sh          | 1 -
 t/t5504-fetch-receive-strict.sh        | 1 -
 t/t5530-upload-pack-error.sh           | 1 -
 t/t5562-http-backend-content-length.sh | 1 -
 t/t5702-protocol-v2.sh                 | 1 -
 t/t5703-upload-pack-ref-in-want.sh     | 1 -
 t/t6006-rev-list-format.sh             | 1 -
 28 files changed, 2 insertions(+), 37 deletions(-)

diff --git a/t/lib-pack.sh b/t/lib-pack.sh
index 0c799d53c6..bb8938ccbe 100644
--- a/t/lib-pack.sh
+++ b/t/lib-pack.sh
@@ -35,8 +35,6 @@ pack_header () {
 # have hardcoded some well-known objects. See the case statements below for the
 # complete list.
 pack_obj () {
-	test_oid_init
-
 	case "$1" in
 	# empty blob
 	$EMPTY_BLOB)
@@ -121,7 +119,6 @@ pack_obj () {
 
 # Compute and append pack trailer to "$1"
 pack_trailer () {
-	test_oid_init &&
 	test-tool $(test_oid algo) -b <"$1" >trailer.tmp &&
 	cat trailer.tmp >>"$1" &&
 	rm -f trailer.tmp
diff --git a/t/lib-submodule-update.sh b/t/lib-submodule-update.sh
index 07c822c8ff..87a759149f 100644
--- a/t/lib-submodule-update.sh
+++ b/t/lib-submodule-update.sh
@@ -196,7 +196,6 @@ test_git_directory_exists () {
 # the submodule repo if it doesn't exist and configures the most problematic
 # settings for diff.ignoreSubmodules.
 prolog () {
-	test_oid_init &&
 	(test -d submodule_update_repo || create_lib_submodule_repo) &&
 	test_config_global diff.ignoreSubmodules all &&
 	test_config diff.ignoreSubmodules all
diff --git a/t/t0000-basic.sh b/t/t0000-basic.sh
index 2599d026a2..362feb43e6 100755
--- a/t/t0000-basic.sh
+++ b/t/t0000-basic.sh
@@ -891,10 +891,6 @@ test_expect_success 'test_atexit is run' "
 	test_path_is_missing also-clean-atexit
 "
 
-test_expect_success 'test_oid setup' '
-	test_oid_init
-'
-
 test_expect_success 'test_oid provides sane info by default' '
 	test_oid zero >actual &&
 	grep "^00*\$" actual &&
diff --git a/t/t1006-cat-file.sh b/t/t1006-cat-file.sh
index 43c4be1e5e..2f501d2dc9 100755
--- a/t/t1006-cat-file.sh
+++ b/t/t1006-cat-file.sh
@@ -140,8 +140,6 @@ test_expect_success '--batch-check without %(rest) considers whole line' '
 	test_cmp expect actual
 '
 
-test_oid_init
-
 tree_sha1=$(git write-tree)
 tree_size=$(($(test_oid rawsz) + 13))
 tree_pretty_content="100644 blob $hello_sha1	hello"
diff --git a/t/t1050-large.sh b/t/t1050-large.sh
index 6a56d1ca24..61e89a8071 100755
--- a/t/t1050-large.sh
+++ b/t/t1050-large.sh
@@ -12,7 +12,6 @@ file_size () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	# clone does not allow us to pass core.bigfilethreshold to
 	# new repos, so set core.bigfilethreshold globally
 	git config --global core.bigfilethreshold 200k &&
diff --git a/t/t1410-reflog.sh b/t/t1410-reflog.sh
index 76d9b744a6..730a43d9dd 100755
--- a/t/t1410-reflog.sh
+++ b/t/t1410-reflog.sh
@@ -54,7 +54,6 @@ check_dont_have () {
 }
 
 test_expect_success setup '
-	test_oid_init &&
 	mkdir -p A/B &&
 	echo rat >C &&
 	echo ox >A/D &&
diff --git a/t/t1450-fsck.sh b/t/t1450-fsck.sh
index 344a2aad82..e81f1cbf0e 100755
--- a/t/t1450-fsck.sh
+++ b/t/t1450-fsck.sh
@@ -9,7 +9,6 @@ test_description='git fsck random collection of tests
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	git config gc.auto 0 &&
 	git config i18n.commitencoding ISO-8859-1 &&
 	test_commit A fileA one &&
diff --git a/t/t1500-rev-parse.sh b/t/t1500-rev-parse.sh
index 603019b541..408b97d5af 100755
--- a/t/t1500-rev-parse.sh
+++ b/t/t1500-rev-parse.sh
@@ -59,7 +59,6 @@ test_rev_parse () {
 ROOT=$(pwd)
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	mkdir -p sub/dir work &&
 	cp -R .git repo.git
 '
diff --git a/t/t3308-notes-merge.sh b/t/t3308-notes-merge.sh
index 790e292966..d69c84c640 100755
--- a/t/t3308-notes-merge.sh
+++ b/t/t3308-notes-merge.sh
@@ -22,7 +22,6 @@ test_expect_success setup '
 	# Copy notes to remote-notes
 	git fetch . refs/notes/*:refs/remote-notes/origin/* &&
 
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	hash4a sha1:5e93d24084d32e1cb61f7070505b9d2530cca987
 	hash3a sha1:8366731eeee53787d2bdf8fc1eff7d94757e8da0
diff --git a/t/t3600-rm.sh b/t/t3600-rm.sh
index f2c0168941..efec8d13b6 100755
--- a/t/t3600-rm.sh
+++ b/t/t3600-rm.sh
@@ -241,7 +241,6 @@ test_expect_success 'refresh index before checking if it is up-to-date' '
 '
 
 test_expect_success 'choking "git rm" should not let it die with cruft' '
-	test_oid_init &&
 	git reset -q --hard &&
 	test_when_finished "rm -f .git/index.lock && git reset -q --hard" &&
 	i=0 &&
diff --git a/t/t3800-mktag.sh b/t/t3800-mktag.sh
index 64dcc5ec28..d696aa4e52 100755
--- a/t/t3800-mktag.sh
+++ b/t/t3800-mktag.sh
@@ -23,7 +23,6 @@ check_verify_failure () {
 # first create a commit, so we have a valid object/type
 # for the tag.
 test_expect_success 'setup' '
-	test_oid_init &&
 	echo Hello >A &&
 	git update-index --add A &&
 	git commit -m "Initial commit" &&
diff --git a/t/t4002-diff-basic.sh b/t/t4002-diff-basic.sh
index cbcdd10464..6a9f010197 100755
--- a/t/t4002-diff-basic.sh
+++ b/t/t4002-diff-basic.sh
@@ -10,8 +10,6 @@ test_description='Test diff raw-output.
 
 . "$TEST_DIRECTORY"/lib-read-tree-m-3way.sh
 
-test_oid_init
-
 test_oid_cache <<\EOF
 aa_1 sha1:ccba72ad3888a3520b39efcf780b9ee64167535d
 aa_1 sha256:9febfbf18197819b2735c45291f138525d2476d59470f98239647544586ba403
diff --git a/t/t4027-diff-submodule.sh b/t/t4027-diff-submodule.sh
index e29deaf4a5..d7145ccca4 100755
--- a/t/t4027-diff-submodule.sh
+++ b/t/t4027-diff-submodule.sh
@@ -6,7 +6,6 @@ test_description='difference in submodules'
 . "$TEST_DIRECTORY"/diff-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	test_tick &&
 	test_create_repo sub &&
 	(
diff --git a/t/t4134-apply-submodule.sh b/t/t4134-apply-submodule.sh
index 99ed4cc546..d1c16ba33c 100755
--- a/t/t4134-apply-submodule.sh
+++ b/t/t4134-apply-submodule.sh
@@ -8,7 +8,6 @@ test_description='git apply submodule tests'
 . ./test-lib.sh
 
 test_expect_success setup '
-	test_oid_init &&
 	cat > create-sm.patch <<EOF &&
 diff --git a/dir/sm b/dir/sm
 new file mode 160000
diff --git a/t/t4200-rerere.sh b/t/t4200-rerere.sh
index 831d424c47..d94d25e4f0 100755
--- a/t/t4200-rerere.sh
+++ b/t/t4200-rerere.sh
@@ -25,7 +25,6 @@ test_description='git rerere
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	cat >a1 <<-\EOF &&
 	Some title
 	==========
diff --git a/t/t4211-line-log.sh b/t/t4211-line-log.sh
index e186c83250..2d1d7b5d19 100755
--- a/t/t4211-line-log.sh
+++ b/t/t4211-line-log.sh
@@ -4,7 +4,6 @@ test_description='test log -L'
 . ./test-lib.sh
 
 test_expect_success 'setup (import history)' '
-	test_oid_init &&
 	git fast-import < "$TEST_DIRECTORY"/t4211/history.export &&
 	git reset --hard
 '
diff --git a/t/t5300-pack-object.sh b/t/t5300-pack-object.sh
index 746cdb626e..cf9f91c941 100755
--- a/t/t5300-pack-object.sh
+++ b/t/t5300-pack-object.sh
@@ -12,8 +12,7 @@ TRASH=$(pwd)
 
 test_expect_success \
     'setup' \
-    'test_oid_init &&
-     rm -f .git/index* &&
+    'rm -f .git/index* &&
      perl -e "print \"a\" x 4096;" > a &&
      perl -e "print \"b\" x 4096;" > b &&
      perl -e "print \"c\" x 4096;" > c &&
diff --git a/t/t5302-pack-index.sh b/t/t5302-pack-index.sh
index 8981c9b90e..119494bd64 100755
--- a/t/t5302-pack-index.sh
+++ b/t/t5302-pack-index.sh
@@ -7,7 +7,6 @@ test_description='pack index with 64-bit offsets and object CRC'
 . ./test-lib.sh
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	rawsz=$(test_oid rawsz) &&
 	rm -rf .git &&
 	git init &&
diff --git a/t/t5313-pack-bounds-checks.sh b/t/t5313-pack-bounds-checks.sh
index 2a4557efc2..535313e4dc 100755
--- a/t/t5313-pack-bounds-checks.sh
+++ b/t/t5313-pack-bounds-checks.sh
@@ -45,7 +45,6 @@ extended_table () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	oid000 sha1:1485
 	oid000 sha256:4222
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index 26f332d6a3..67f2663c7c 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -10,8 +10,7 @@ test_expect_success 'setup full repo' '
 	cd "$TRASH_DIRECTORY/full" &&
 	git init &&
 	git config core.commitGraph true &&
-	objdir=".git/objects" &&
-	test_oid_init
+	objdir=".git/objects"
 '
 
 test_expect_success POSIXPERM 'tweak umask for modebit tests' '
diff --git a/t/t5319-multi-pack-index.sh b/t/t5319-multi-pack-index.sh
index 7214cab36c..7dfff0f8f4 100755
--- a/t/t5319-multi-pack-index.sh
+++ b/t/t5319-multi-pack-index.sh
@@ -29,7 +29,6 @@ midx_read_expect () {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	test_oid_cache <<-EOF
 	idxoff sha1:2999
 	idxoff sha256:3739
diff --git a/t/t5324-split-commit-graph.sh b/t/t5324-split-commit-graph.sh
index 269d0964a3..3a63639fbc 100755
--- a/t/t5324-split-commit-graph.sh
+++ b/t/t5324-split-commit-graph.sh
@@ -12,7 +12,6 @@ test_expect_success 'setup repo' '
 	git config gc.writeCommitGraph false &&
 	infodir=".git/objects/info" &&
 	graphdir="$infodir/commit-graphs" &&
-	test_oid_init &&
 	test_oid_cache <<-EOM
 	shallow sha1:1760
 	shallow sha256:2064
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index a32efe2b6c..1a16ac4c0d 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -4,7 +4,6 @@ test_description='fetch/receive strict mode'
 . ./test-lib.sh
 
 test_expect_success 'setup and inject "corrupt or missing" object' '
-	test_oid_init &&
 	echo hello >greetings &&
 	git add greetings &&
 	git commit -m greetings &&
diff --git a/t/t5530-upload-pack-error.sh b/t/t5530-upload-pack-error.sh
index 4ce9a9f704..205a2631e7 100755
--- a/t/t5530-upload-pack-error.sh
+++ b/t/t5530-upload-pack-error.sh
@@ -14,7 +14,6 @@ corrupt_repo () {
 }
 
 test_expect_success 'setup and corrupt repository' '
-	test_oid_init &&
 	echo file >file &&
 	git add file &&
 	git rev-parse :file &&
diff --git a/t/t5562-http-backend-content-length.sh b/t/t5562-http-backend-content-length.sh
index c6ec625497..e5d3d15ba8 100755
--- a/t/t5562-http-backend-content-length.sh
+++ b/t/t5562-http-backend-content-length.sh
@@ -46,7 +46,6 @@ ssize_b100dots() {
 }
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	HTTP_CONTENT_ENCODING="identity" &&
 	export HTTP_CONTENT_ENCODING &&
 	git config http.receivepack true &&
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index 7fc22171e7..5a60fbe3ed 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -13,7 +13,6 @@ start_git_daemon --export-all --enable=receive-pack
 daemon_parent=$GIT_DAEMON_DOCUMENT_ROOT_PATH/parent
 
 test_expect_success 'create repo to be served by git-daemon' '
-	test_oid_init &&
 	git init "$daemon_parent" &&
 	test_commit -C "$daemon_parent" one
 '
diff --git a/t/t5703-upload-pack-ref-in-want.sh b/t/t5703-upload-pack-ref-in-want.sh
index 748282f058..d9ecf0f4a9 100755
--- a/t/t5703-upload-pack-ref-in-want.sh
+++ b/t/t5703-upload-pack-ref-in-want.sh
@@ -43,7 +43,6 @@ write_command () {
 #             \ | /
 #               a
 test_expect_success 'setup repository' '
-	test_oid_init &&
 	test_commit a &&
 	git checkout -b o/foo &&
 	test_commit b &&
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 7e82e43a63..bc95da8a5f 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -32,7 +32,6 @@ changed_iso88591=$(echo "$changed" | iconv -f utf-8 -t $test_encoding)
 truncate_count=20
 
 test_expect_success 'setup' '
-	test_oid_init &&
 	: >foo &&
 	git add foo &&
 	git config i18n.commitEncoding $test_encoding &&

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 00/39] SHA-256, part 3/3
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (38 preceding siblings ...)
  2020-07-29 23:14   ` [PATCH v6 39/39] t: remove test_oid_init in tests brian m. carlson
@ 2020-07-30  2:48   ` Eric Sunshine
  2020-07-30 16:18     ` Junio C Hamano
  2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
  40 siblings, 1 reply; 116+ messages in thread
From: Eric Sunshine @ 2020-07-30  2:48 UTC (permalink / raw)
  To: brian m. carlson; +Cc: Git List

On Wed, Jul 29, 2020 at 7:15 PM brian m. carlson
<sandals@crustytoothpaste.net> wrote:
> Changes from v5:
> * Remove useless test_oid_init invocations throughout the series.
> * Fix a commit message for grammar and style.
> * Fix a typo in a shell case statement.
> * Fix http-fetch to allow parsing options before failing on a missing
>   git directory.

Thanks, I think this version addresses all the comments from my
reviews of the last several versions of this patch series.

I've read through the entire series a couple times and paid close
attention to the range-diffs, so FWIW, I'm reasonably comfortable
giving v6 my:

Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 00/39] SHA-256, part 3/3
  2020-07-30  2:48   ` [PATCH v6 00/39] SHA-256, part 3/3 Eric Sunshine
@ 2020-07-30 16:18     ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2020-07-30 16:18 UTC (permalink / raw)
  To: Eric Sunshine; +Cc: brian m. carlson, Git List

Eric Sunshine <sunshine@sunshineco.com> writes:

> On Wed, Jul 29, 2020 at 7:15 PM brian m. carlson
> <sandals@crustytoothpaste.net> wrote:
>> Changes from v5:
>> * Remove useless test_oid_init invocations throughout the series.
>> * Fix a commit message for grammar and style.
>> * Fix a typo in a shell case statement.
>> * Fix http-fetch to allow parsing options before failing on a missing
>>   git directory.
>
> Thanks, I think this version addresses all the comments from my
> reviews of the last several versions of this patch series.
>
> I've read through the entire series a couple times and paid close
> attention to the range-diffs, so FWIW, I'm reasonably comfortable
> giving v6 my:
>
> Reviewed-by: Eric Sunshine <sunshine@sunshineco.com>

Thanks, both.


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 00/39] SHA-256, part 3/3
  2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
                     ` (39 preceding siblings ...)
  2020-07-30  2:48   ` [PATCH v6 00/39] SHA-256, part 3/3 Eric Sunshine
@ 2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
  2021-03-10 16:36     ` Elijah Newren
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
  40 siblings, 2 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-10 12:04 UTC (permalink / raw)
  To: brian m. carlson; +Cc: git, Eric Sunshine, Elijah Newren


[I couldn't find a more relevant thing to reply to]

On Thu, Jul 30 2020, brian m. carlson wrote:

> [...]

B.t.w. thanks again for all your work on SHA-1 -> SHA-256.

I found a missing spot that wasn't trivial to fix, so sending an E-Mail:

In git-filter-branch.sh we have:

_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"

Then later we have a case condition based on matching a SHA-1:

        $_x40)
                echo "Ref '$ref' was rewritten"
                if ! git update-ref -m "filter-branch: rewrite" \

Just deleting that case arm has filter-branch tests passing, so whatever
it's meant to do it has zero coverage, which explains why it hasn't
broken with our tests.

I didn't have time to dig, so sending off this E-Mail instead.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 00/39] SHA-256, part 3/3
  2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
@ 2021-03-10 16:36     ` Elijah Newren
  2021-03-10 17:08       ` Jeff King
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
  1 sibling, 1 reply; 116+ messages in thread
From: Elijah Newren @ 2021-03-10 16:36 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, Git Mailing List, Eric Sunshine

On Wed, Mar 10, 2021 at 4:04 AM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
> [I couldn't find a more relevant thing to reply to]
>
> On Thu, Jul 30 2020, brian m. carlson wrote:
>
> > [...]
>
> B.t.w. thanks again for all your work on SHA-1 -> SHA-256.
>
> I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
>
> In git-filter-branch.sh we have:
>
> _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
>
> Then later we have a case condition based on matching a SHA-1:
>
>         $_x40)
>                 echo "Ref '$ref' was rewritten"
>                 if ! git update-ref -m "filter-branch: rewrite" \
>
> Just deleting that case arm has filter-branch tests passing, so whatever
> it's meant to do it has zero coverage, which explains why it hasn't
> broken with our tests.
>
> I didn't have time to dig, so sending off this E-Mail instead.

Hmm, perhaps we should revive the "separate non-core tools out of
git.git" thread --
https://lore.kernel.org/git/a784a61e-1320-be1e-9dfb-d533a01827ec@gmail.com/.

Honestly not sure it's worth bothering fixing filter-branch at this
point, given the strong WARNINGs we added to it.  If anything, I'd
just add another warning to the pile for this issue.  If folks really
still want/need filter-branch once SHA-256 is the only option, I'm
sure they'll be motivated to make the necessary fixes (or just use/fix
filter-repo's filter-branch-ish).

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH 0/3] sha256 fixes for filter-branch
  2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
  2021-03-10 16:36     ` Elijah Newren
@ 2021-03-10 17:06     ` Jeff King
  2021-03-10 17:07       ` [PATCH 1/3] t7003: test ref rewriting explicitly Jeff King
                         ` (4 more replies)
  1 sibling, 5 replies; 116+ messages in thread
From: Jeff King @ 2021-03-10 17:06 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, git, Eric Sunshine, Elijah Newren

On Wed, Mar 10, 2021 at 01:04:37PM +0100, Ævar Arnfjörð Bjarmason wrote:

> I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
> 
> In git-filter-branch.sh we have:
> 
> _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
> 
> Then later we have a case condition based on matching a SHA-1:
> 
>         $_x40)
>                 echo "Ref '$ref' was rewritten"
>                 if ! git update-ref -m "filter-branch: rewrite" \
> 
> Just deleting that case arm has filter-branch tests passing, so whatever
> it's meant to do it has zero coverage, which explains why it hasn't
> broken with our tests.

It actually does get covered. Dropping that case-arm means we'll fall
through to the one below, which does _roughly_ the same thing with a
bunch of extra warnings. But none of the tests actually check the error
messages, so they don't notice.

Here's a series which fixes it, plus extra tests to notice the
distinction.

  [1/3]: t7003: test ref rewriting explicitly
  [2/3]: filter-branch: drop multiple-ancestor warning
  [3/3]: filter-branch: drop $_x40 glob

 git-filter-branch.sh     | 16 ++--------------
 t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
 2 files changed, 33 insertions(+), 14 deletions(-)

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH 1/3] t7003: test ref rewriting explicitly
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
@ 2021-03-10 17:07       ` Jeff King
  2021-03-11  0:10         ` Ævar Arnfjörð Bjarmason
  2021-03-10 17:07       ` [PATCH 2/3] filter-branch: drop multiple-ancestor warning Jeff King
                         ` (3 subsequent siblings)
  4 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2021-03-10 17:07 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, git, Eric Sunshine, Elijah Newren

After it has rewritten all of the commits, filter-branch will then
rewrite each of the input refs based on the resulting map of old/new
commits. But we don't have any explicit test coverage of this code.
Let's make sure we are covering each of those cases:

  - deleting a ref when all of its commits were pruned

  - rewriting a ref based on the mapping (this happens throughout the
    script, but let's make sure we generate the correct messages)

  - rewriting a ref whose tip was excluded, in which case we rewrite to
    the nearest ancestor. Note in this case that we still insist that no
    "warning" line is present (even though it looks like we'd trigger
    the "... was rewritten into multiple commits" one). See the next
    commit for more details.

Note these all pass currently, but the latter two will fail when run
with GIT_TEST_DEFAULT_HASH=sha256.

Signed-off-by: Jeff King <peff@peff.net>
---
 t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)

diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 1c55695034..1349e5b232 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -506,4 +506,35 @@ test_expect_success 'rewrite repository including refs that point at non-commit
 	! fgrep fatal filter-output
 '
 
+test_expect_success 'filter-branch handles ref deletion' '
+	git switch --orphan empty-commit &&
+	git commit --allow-empty -m "empty commit" &&
+	git tag empty &&
+	git branch to-delete &&
+	git filter-branch -f --prune-empty to-delete >out 2>&1 &&
+	grep "to-delete.*was deleted" out &&
+	test_must_fail git rev-parse --verify to-delete
+'
+
+test_expect_success 'filter-branch handles ref rewrite' '
+	git checkout empty &&
+	test_commit to-drop &&
+	git branch rewrite &&
+	git filter-branch -f \
+		--index-filter "git rm --ignore-unmatch --cached to-drop.t" \
+		 rewrite >out 2>&1 &&
+	grep "rewrite.*was rewritten" out &&
+	! grep -i warning out &&
+	git diff-tree empty rewrite
+'
+
+test_expect_success 'filter-branch handles ancestor rewrite' '
+	test_commit to-exclude &&
+	git branch ancestor &&
+	git filter-branch -f ancestor -- :^to-exclude.t >out 2>&1 &&
+	grep "ancestor.*was rewritten" out &&
+	! grep -i warning out &&
+	git diff-tree HEAD^ ancestor
+'
+
 test_done
-- 
2.31.0.rc2.525.gc2268d2248


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 2/3] filter-branch: drop multiple-ancestor warning
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
  2021-03-10 17:07       ` [PATCH 1/3] t7003: test ref rewriting explicitly Jeff King
@ 2021-03-10 17:07       ` Jeff King
  2021-03-10 22:16         ` Junio C Hamano
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
                         ` (2 subsequent siblings)
  4 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2021-03-10 17:07 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, git, Eric Sunshine, Elijah Newren

When a ref maps to a commit that is neither rewritten nor kept by
filter-branch (e.g., because it was eliminated by rev-list's pathspec
selection), we rewrite it to its nearest ancestor.

Since the initial commit in 6f6826c52b (Add git-filter-branch,
2007-06-03), we have warned when there are multiple such ancestors in
the map file. However, the warning code is impossible to trigger these
days. Since a0e46390d3 (filter-branch: fix ref rewriting with
--subdirectory-filter, 2008-08-12), we find the ancestor using "rev-list
-1", so it can only ever have a single value.

This code is made doubly confusing by the fact that we append to the map
file when mapping ancestors. However, this can never yield multiple
values because:

  - we explicitly check whether the map already exists, and if so, do
    nothing (so our "append" will always be to a file that does not
    exist)

  - even if we were to try mapping twice, the process to do so is
    deterministic. I.e., we'd always end up with the same ancestor for a
    given sha1. So warning about it would be pointless; there is no
    ambiguity.

So swap out the warning code for a BUG (which we'll simplify further in
the next commit). And let's stop using the append operator to make the
ancestor-mapping code less confusing.

Signed-off-by: Jeff King <peff@peff.net>
---
 git-filter-branch.sh | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index fea7964617..a1e80bd552 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -492,7 +492,7 @@ then
 		sha1=$(git rev-parse "$ref"^0)
 		test -f "$workdir"/../map/$sha1 && continue
 		ancestor=$(git rev-list --simplify-merges -1 "$ref" "$@")
-		test "$ancestor" && echo $(map $ancestor) >> "$workdir"/../map/$sha1
+		test "$ancestor" && echo $(map $ancestor) >"$workdir"/../map/$sha1
 	done < "$tempdir"/heads
 fi
 
@@ -534,14 +534,7 @@ do
 		fi
 	;;
 	*)
-		# NEEDSWORK: possibly add -Werror, making this an error
-		warn "WARNING: '$ref' was rewritten into multiple commits:"
-		warn "$rewritten"
-		warn "WARNING: Ref '$ref' points to the first one now."
-		rewritten=$(echo "$rewritten" | head -n 1)
-		git update-ref -m "filter-branch: rewrite to first" \
-				"$ref" $rewritten $sha1 ||
-			die "Could not rewrite $ref"
+		die "BUG: multiple ancestors in map file?"
 	;;
 	esac
 	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
-- 
2.31.0.rc2.525.gc2268d2248


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 3/3] filter-branch: drop $_x40 glob
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
  2021-03-10 17:07       ` [PATCH 1/3] t7003: test ref rewriting explicitly Jeff King
  2021-03-10 17:07       ` [PATCH 2/3] filter-branch: drop multiple-ancestor warning Jeff King
@ 2021-03-10 17:07       ` Jeff King
  2021-03-10 22:23         ` Junio C Hamano
                           ` (5 more replies)
  2021-03-10 17:54       ` [PATCH 0/3] sha256 fixes for filter-branch Elijah Newren
  2021-03-10 23:32       ` brian m. carlson
  4 siblings, 6 replies; 116+ messages in thread
From: Jeff King @ 2021-03-10 17:07 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: brian m. carlson, git, Eric Sunshine, Elijah Newren

When checking whether a commit was rewritten to a single object id, we
use a glob that insists on a 40-hex result. This works for sha1, but
fails t7003 when run with GIT_TEST_DEFAULT_HASH=sha256.

Since the previous commit simplified the case statement here, we only
have two arms: an empty string or a single object id. We can just loosen
our glob to match anything, and still distinguish those cases (we lose
the ability to notice bogus input, but that's not a problem; we are the
one who wrote the map in the first place, and anyway update-ref will
complain loudly if the input isn't a valid hash).

Signed-off-by: Jeff King <peff@peff.net>
---
 git-filter-branch.sh | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index a1e80bd552..cb89372813 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -498,8 +498,6 @@ fi
 
 # Finally update the refs
 
-_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
 echo
 while read ref
 do
@@ -519,7 +517,7 @@ do
 		git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
 			die "Could not delete $ref"
 	;;
-	$_x40)
+	*)
 		echo "Ref '$ref' was rewritten"
 		if ! git update-ref -m "filter-branch: rewrite" \
 					"$ref" $rewritten $sha1 2>/dev/null; then
@@ -533,9 +531,6 @@ do
 			fi
 		fi
 	;;
-	*)
-		die "BUG: multiple ancestors in map file?"
-	;;
 	esac
 	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
 		 exit
-- 
2.31.0.rc2.525.gc2268d2248

^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH v6 00/39] SHA-256, part 3/3
  2021-03-10 16:36     ` Elijah Newren
@ 2021-03-10 17:08       ` Jeff King
  0 siblings, 0 replies; 116+ messages in thread
From: Jeff King @ 2021-03-10 17:08 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson,
	Git Mailing List, Eric Sunshine

On Wed, Mar 10, 2021 at 08:36:15AM -0800, Elijah Newren wrote:

> Honestly not sure it's worth bothering fixing filter-branch at this
> point, given the strong WARNINGs we added to it.  If anything, I'd
> just add another warning to the pile for this issue.  If folks really
> still want/need filter-branch once SHA-256 is the only option, I'm
> sure they'll be motivated to make the necessary fixes (or just use/fix
> filter-repo's filter-branch-ish).

You're probably right that it's not a good use of time, but I just sent
a series fixing it. It was one of those cases where I was like "oh, it's
probably just X". And then "no, wait...", and before I knew it I had
wasted an hour fixing filter-branch. ;)

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 0/3] sha256 fixes for filter-branch
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
                         ` (2 preceding siblings ...)
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
@ 2021-03-10 17:54       ` Elijah Newren
  2021-03-10 22:24         ` Junio C Hamano
  2021-03-10 23:32       ` brian m. carlson
  4 siblings, 1 reply; 116+ messages in thread
From: Elijah Newren @ 2021-03-10 17:54 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson,
	Git Mailing List, Eric Sunshine

On Wed, Mar 10, 2021 at 9:06 AM Jeff King <peff@peff.net> wrote:
>
> On Wed, Mar 10, 2021 at 01:04:37PM +0100, Ævar Arnfjörð Bjarmason wrote:
>
> > I found a missing spot that wasn't trivial to fix, so sending an E-Mail:
> >
> > In git-filter-branch.sh we have:
> >
> > _x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> > _x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
> >
> > Then later we have a case condition based on matching a SHA-1:
> >
> >         $_x40)
> >                 echo "Ref '$ref' was rewritten"
> >                 if ! git update-ref -m "filter-branch: rewrite" \
> >
> > Just deleting that case arm has filter-branch tests passing, so whatever
> > it's meant to do it has zero coverage, which explains why it hasn't
> > broken with our tests.
>
> It actually does get covered. Dropping that case-arm means we'll fall
> through to the one below, which does _roughly_ the same thing with a
> bunch of extra warnings. But none of the tests actually check the error
> messages, so they don't notice.
>
> Here's a series which fixes it, plus extra tests to notice the
> distinction.
>
>   [1/3]: t7003: test ref rewriting explicitly
>   [2/3]: filter-branch: drop multiple-ancestor warning
>   [3/3]: filter-branch: drop $_x40 glob
>
>  git-filter-branch.sh     | 16 ++--------------
>  t/t7003-filter-branch.sh | 31 +++++++++++++++++++++++++++++++
>  2 files changed, 33 insertions(+), 14 deletions(-)
>
> -Peff

Ævar successfully nerd sniped you.

...and sucked me in too, since I went through and read your patches.
:-)  They look good to me.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 2/3] filter-branch: drop multiple-ancestor warning
  2021-03-10 17:07       ` [PATCH 2/3] filter-branch: drop multiple-ancestor warning Jeff King
@ 2021-03-10 22:16         ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-10 22:16 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson, git,
	Eric Sunshine, Elijah Newren

Jeff King <peff@peff.net> writes:

> When a ref maps to a commit that is neither rewritten nor kept by
> filter-branch (e.g., because it was eliminated by rev-list's pathspec
> selection), we rewrite it to its nearest ancestor.
>
> Since the initial commit in 6f6826c52b (Add git-filter-branch,
> 2007-06-03), we have warned when there are multiple such ancestors in
> the map file. However, the warning code is impossible to trigger these
> days. Since a0e46390d3 (filter-branch: fix ref rewriting with
> --subdirectory-filter, 2008-08-12), we find the ancestor using "rev-list
> -1", so it can only ever have a single value.

;-)


^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/3] filter-branch: drop $_x40 glob
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
@ 2021-03-10 22:23         ` Junio C Hamano
  2021-03-11  0:14         ` [PATCH 0/4] bisect + tests: remove old $_x05, $_x35 and $_x40 variables Ævar Arnfjörð Bjarmason
                           ` (4 subsequent siblings)
  5 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-10 22:23 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, brian m. carlson, git,
	Eric Sunshine, Elijah Newren

Jeff King <peff@peff.net> writes:

> When checking whether a commit was rewritten to a single object id, we
> use a glob that insists on a 40-hex result. This works for sha1, but
> fails t7003 when run with GIT_TEST_DEFAULT_HASH=sha256.
>
> Since the previous commit simplified the case statement here, we only
> have two arms: an empty string or a single object id. We can just loosen
> our glob to match anything, and still distinguish those cases (we lose
> the ability to notice bogus input, but that's not a problem; we are the
> one who wrote the map in the first place, and anyway update-ref will
> complain loudly if the input isn't a valid hash).

If the input is any valid extended sha-1 expression, it probably
would not barf, so we technically are losing a bit of internal
safety valve to notice bogus rewrite (if the map recorded output of
"git describe --always", this part would not notice and keep working
just fine, but other parts of the code may not be happy).

But if this were a new code that said "It is either empty of non-empty,
and non-empty ones are object names we write, so we do not have to
insist it is hex object names", it is very likely that we would
accept it without any extra checks, so I think this is perfectly OK.

Thanks.

> Signed-off-by: Jeff King <peff@peff.net>
> ---
>  git-filter-branch.sh | 7 +------
>  1 file changed, 1 insertion(+), 6 deletions(-)
>
> diff --git a/git-filter-branch.sh b/git-filter-branch.sh
> index a1e80bd552..cb89372813 100755
> --- a/git-filter-branch.sh
> +++ b/git-filter-branch.sh
> @@ -498,8 +498,6 @@ fi
>  
>  # Finally update the refs
>  
> -_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
> -_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
>  echo
>  while read ref
>  do
> @@ -519,7 +517,7 @@ do
>  		git update-ref -m "filter-branch: delete" -d "$ref" $sha1 ||
>  			die "Could not delete $ref"
>  	;;
> -	$_x40)
> +	*)
>  		echo "Ref '$ref' was rewritten"
>  		if ! git update-ref -m "filter-branch: rewrite" \
>  					"$ref" $rewritten $sha1 2>/dev/null; then
> @@ -533,9 +531,6 @@ do
>  			fi
>  		fi
>  	;;
> -	*)
> -		die "BUG: multiple ancestors in map file?"
> -	;;
>  	esac
>  	git update-ref -m "filter-branch: backup" "$orig_namespace$ref" $sha1 ||
>  		 exit

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 0/3] sha256 fixes for filter-branch
  2021-03-10 17:54       ` [PATCH 0/3] sha256 fixes for filter-branch Elijah Newren
@ 2021-03-10 22:24         ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-10 22:24 UTC (permalink / raw)
  To: Elijah Newren
  Cc: Jeff King, Ævar Arnfjörð Bjarmason,
	brian m. carlson, Git Mailing List, Eric Sunshine

Elijah Newren <newren@gmail.com> writes:

> Ævar successfully nerd sniped you.
>
> ...and sucked me in too, since I went through and read your patches.
> :-)  They look good to me.

Yup, thanks, both.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 0/3] sha256 fixes for filter-branch
  2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
                         ` (3 preceding siblings ...)
  2021-03-10 17:54       ` [PATCH 0/3] sha256 fixes for filter-branch Elijah Newren
@ 2021-03-10 23:32       ` brian m. carlson
  4 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2021-03-10 23:32 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Eric Sunshine,
	Elijah Newren

[-- Attachment #1: Type: text/plain, Size: 663 bytes --]

On 2021-03-10 at 17:06:33, Jeff King wrote:
> It actually does get covered. Dropping that case-arm means we'll fall
> through to the one below, which does _roughly_ the same thing with a
> bunch of extra warnings. But none of the tests actually check the error
> messages, so they don't notice.
> 
> Here's a series which fixes it, plus extra tests to notice the
> distinction.
> 
>   [1/3]: t7003: test ref rewriting explicitly
>   [2/3]: filter-branch: drop multiple-ancestor warning
>   [3/3]: filter-branch: drop $_x40 glob

This series looked good to me.  Thanks for sending it.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 1/3] t7003: test ref rewriting explicitly
  2021-03-10 17:07       ` [PATCH 1/3] t7003: test ref rewriting explicitly Jeff King
@ 2021-03-11  0:10         ` Ævar Arnfjörð Bjarmason
  2021-03-11  1:55           ` brian m. carlson
  0 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:10 UTC (permalink / raw)
  To: Jeff King; +Cc: brian m. carlson, git, Eric Sunshine, Elijah Newren


On Wed, Mar 10 2021, Jeff King wrote:


I've read it all over, looks good to me.

> Note these all pass currently, but the latter two will fail when run
> with GIT_TEST_DEFAULT_HASH=sha256.

I don't know if anyone relies on that. Probably fine to have this as-is.

But wouldn't fixing that just be a matter of adding these tests as
test_expect_failure, and then flipping them to test_expect_success in
3/3?

^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH 0/4] bisect + tests: remove old $_x05, $_x35 and $_x40 variables
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
  2021-03-10 22:23         ` Junio C Hamano
@ 2021-03-11  0:14         ` Ævar Arnfjörð Bjarmason
  2021-03-11  0:14         ` [PATCH 1/4] git-bisect: remove unused SHA-1 $x40 shell variable Ævar Arnfjörð Bjarmason
                           ` (3 subsequent siblings)
  5 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:14 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, brian m . carlson, Eric Sunshine,
	Elijah Newren, Ævar Arnfjörð Bjarmason

A cleanup in the spirit of Jeff King's just-sent series:
https://lore.kernel.org/git/YEj82fOf+F4xJC8S@coredump.intra.peff.net/

Applies independently of that series though.

Ævar Arnfjörð Bjarmason (4):
  git-bisect: remove unused SHA-1 $x40 shell variable
  test-lib: remove unused $_x40 and $_z40 variables
  shortlog tests: rewrite to get rid of --abbrev=35 hardcoding
  tests: get rid of $_x05 from the test suite

 git-bisect.sh                       |  2 -
 t/t3101-ls-tree-dirname.sh          |  2 +-
 t/t3508-cherry-pick-many-commits.sh |  4 +-
 t/t4201-shortlog.sh                 | 92 ++++++++++++++++-------------
 t/t6006-rev-list-format.sh          | 12 ++--
 t/t7600-merge.sh                    |  4 +-
 t/test-lib.sh                       |  9 +--
 7 files changed, 65 insertions(+), 60 deletions(-)

-- 
2.31.0.rc1.213.gbfdb770ff55


^ permalink raw reply	[flat|nested] 116+ messages in thread

* [PATCH 1/4] git-bisect: remove unused SHA-1 $x40 shell variable
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
  2021-03-10 22:23         ` Junio C Hamano
  2021-03-11  0:14         ` [PATCH 0/4] bisect + tests: remove old $_x05, $_x35 and $_x40 variables Ævar Arnfjörð Bjarmason
@ 2021-03-11  0:14         ` Ævar Arnfjörð Bjarmason
  2021-03-11  0:14         ` [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables Ævar Arnfjörð Bjarmason
                           ` (2 subsequent siblings)
  5 siblings, 0 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:14 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, brian m . carlson, Eric Sunshine,
	Elijah Newren, Ævar Arnfjörð Bjarmason

This variable was last used in code removed in
06f5608c14 (bisect--helper: `bisect_start` shell function partially in
C, 2019-01-02).

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 git-bisect.sh | 2 --
 1 file changed, 2 deletions(-)

diff --git a/git-bisect.sh b/git-bisect.sh
index 6a7afaea8d..b59f3aaad4 100755
--- a/git-bisect.sh
+++ b/git-bisect.sh
@@ -34,8 +34,6 @@ Please use "git help bisect" to get the full man page.'
 OPTIONS_SPEC=
 . git-sh-setup
 
-_x40='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x40="$_x40$_x40$_x40$_x40$_x40$_x40$_x40$_x40"
 TERM_BAD=bad
 TERM_GOOD=good
 
-- 
2.31.0.rc1.213.gbfdb770ff55


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
                           ` (2 preceding siblings ...)
  2021-03-11  0:14         ` [PATCH 1/4] git-bisect: remove unused SHA-1 $x40 shell variable Ævar Arnfjörð Bjarmason
@ 2021-03-11  0:14         ` Ævar Arnfjörð Bjarmason
  2021-03-11  1:02           ` Junio C Hamano
  2021-03-11  0:14         ` [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding Ævar Arnfjörð Bjarmason
  2021-03-11  0:14         ` [PATCH 4/4] tests: get rid of $_x05 from the test suite Ævar Arnfjörð Bjarmason
  5 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:14 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, brian m . carlson, Eric Sunshine,
	Elijah Newren, Ævar Arnfjörð Bjarmason

These two have fallen out of use with the SHA-256 migration.

The last use of $_x40 was removed in fc7e73d7ef (t4013: improve
diff-post-processor logic, 2020-08-21) and

The last use of $_z40 was removed in 7a868c51c2 (t5562: use $ZERO_OID,
2019-12-21), but it was then needlessly refactored to be hash-agnostic
in 192b517589 (t: use hash-specific lookup tables to define test
constants, 2020-02-22). We can just remove it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/test-lib.sh | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/t/test-lib.sh b/t/test-lib.sh
index d3f6af6a65..5f2ad2fd81 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 _x35 _x40 _z40 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1381,10 +1381,9 @@ then
 fi
 
 # Convenience
-# A regexp to match 5, 35 and 40 hexdigits
+# A regexp to match 5 and 35 hexdigits
 _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
 _x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
-_x40="$_x35$_x05"
 
 test_oid_init
 
@@ -1393,7 +1392,6 @@ OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
 OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
 EMPTY_TREE=$(test_oid empty_tree)
 EMPTY_BLOB=$(test_oid empty_blob)
-_z40=$ZERO_OID
 
 # Provide an implementation of the 'yes' utility; the upper bound
 # limit is there to help Windows that cannot stop this loop from
-- 
2.31.0.rc1.213.gbfdb770ff55


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
                           ` (3 preceding siblings ...)
  2021-03-11  0:14         ` [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables Ævar Arnfjörð Bjarmason
@ 2021-03-11  0:14         ` Ævar Arnfjörð Bjarmason
  2021-03-11  1:18           ` Junio C Hamano
  2021-03-11 19:15           ` Jeff King
  2021-03-11  0:14         ` [PATCH 4/4] tests: get rid of $_x05 from the test suite Ævar Arnfjörð Bjarmason
  5 siblings, 2 replies; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:14 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, brian m . carlson, Eric Sunshine,
	Elijah Newren, Ævar Arnfjörð Bjarmason

Rewrite much of the test logic introduced in 600372497c (shortlog:
Document and test --format option, 2010-05-03), this allows us to get
rid of the now-unused $_x35 variable in test-lib.sh

There was a minimal migration of this test to SHA-256 in
2ece6ad281 (t: switch $_x40 to $OID_REGEX, 2018-05-13), but actually
we can just get rid of all the assumptions about hashing here, and
make this easier to understand and maintain while we're at it.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t4201-shortlog.sh | 92 +++++++++++++++++++++++++--------------------
 t/test-lib.sh       |  5 +--
 2 files changed, 54 insertions(+), 43 deletions(-)

diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
index 3095b1b2ff..3ef17c06e4 100755
--- a/t/t4201-shortlog.sh
+++ b/t/t4201-shortlog.sh
@@ -19,82 +19,94 @@ test_expect_success 'setup' '
 	commit=$(printf "%s\n" "Test" "" | git commit-tree "$tree") &&
 	git update-ref HEAD "$commit" &&
 
+	echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" >message &&
+
+	sed "s/i/1234/g" <message >tmp &&
+	tr 1234 "\360\235\204\236" <tmp >message.2 &&
+
+	sed "s/i/1234/g" <message >tmp &&
+	tr 1234 "\370\235\204\236" <tmp >message.3 &&
+
+	echo "a								12	34	56	78" >message.4 &&
+	echo "Commit by someone else" >message.5 &&
+
 	echo 2 >a1 &&
-	git commit --quiet -m "This is a very, very long first line for the commit message to see if it is wrapped correctly" a1 &&
+	git commit --quiet -F message a1 &&
 
 	# test if the wrapping is still valid
 	# when replacing all is by treble clefs.
 	echo 3 >a1 &&
-	git commit --quiet -m "$(
-		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
-		sed "s/i/1234/g" |
-		tr 1234 "\360\235\204\236")" a1 &&
+	git commit --quiet -F message.2 a1 &&
 
 	# now fsck up the utf8
 	git config i18n.commitencoding non-utf-8 &&
 	echo 4 >a1 &&
-	git commit --quiet -m "$(
-		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
-		sed "s/i/1234/g" |
-		tr 1234 "\370\235\204\236")" a1 &&
+	git commit --quiet -F message.3 a1 &&
 
 	echo 5 >a1 &&
-	git commit --quiet -m "a								12	34	56	78" a1 &&
+	git commit --quiet -F message.4 a1 &&
 
 	echo 6 >a1 &&
-	git commit --quiet -m "Commit by someone else" \
+	git commit --quiet -F message.5  \
 		--author="Someone else <not!me>" a1 &&
 
-	cat >expect.template <<-\EOF
+	cat >expect.default <<-EOF
 	A U Thor (5):
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
-	      SUBJECT
+	      Test
+	      $(cat message)
+	      $(cat message.2)
+	      $(cat message.3)
+	      $(cat message.4)
 
 	Someone else (1):
-	      SUBJECT
+	      $(cat message.5)
 
 	EOF
 '
 
-fuzz() {
-	file=$1 &&
-	sed "
-			s/$OID_REGEX/OBJECT_NAME/g
-			s/$_x35/OBJID/g
-			s/^ \{6\}[CTa].*/      SUBJECT/g
-			s/^ \{8\}[^ ].*/        CONTINUATION/g
-		" <"$file" >"$file.fuzzy" &&
-	sed "/CONTINUATION/ d" <"$file.fuzzy"
-}
-
 test_expect_success 'default output format' '
 	git shortlog HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect.template log.predictable
+	test_cmp expect.default log
 '
 
 test_expect_success 'pretty format' '
-	sed s/SUBJECT/OBJECT_NAME/ expect.template >expect &&
+	cat >expect <<-EOF &&
+	A U Thor (5):
+	      $(git rev-parse HEAD~5)
+	      $(git rev-parse HEAD~4)
+	      $(git rev-parse HEAD~3)
+	      $(git rev-parse HEAD~2)
+	      $(git rev-parse HEAD~1)
+
+	Someone else (1):
+	      $(git rev-parse HEAD~0)
+
+	EOF
 	git shortlog --format="%H" HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect log
 '
 
 test_expect_success '--abbrev' '
-	sed s/SUBJECT/OBJID/ expect.template >expect &&
+	cut -c 1-41 <expect >expect.abbrev &&
 	git shortlog --format="%h" --abbrev=35 HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect.abbrev log
 '
 
 test_expect_success 'output from user-defined format is re-wrapped' '
-	sed "s/SUBJECT/two lines/" expect.template >expect &&
+	cat >expect <<-EOF &&
+	A U Thor (5):
+	      two lines
+	      two lines
+	      two lines
+	      two lines
+	      two lines
+
+	Someone else (1):
+	      two lines
+
+	EOF
 	git shortlog --format="two%nlines" HEAD >log &&
-	fuzz log >log.predictable &&
-	test_cmp expect log.predictable
+	test_cmp expect log
 '
 
 test_expect_success !MINGW 'shortlog wrapping' '
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 5f2ad2fd81..4d5ba558d3 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1381,9 +1381,8 @@ then
 fi
 
 # Convenience
-# A regexp to match 5 and 35 hexdigits
+# A regexp to match 5 hexdigits
 _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-_x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
 
 test_oid_init
 
-- 
2.31.0.rc1.213.gbfdb770ff55


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* [PATCH 4/4] tests: get rid of $_x05 from the test suite
  2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
                           ` (4 preceding siblings ...)
  2021-03-11  0:14         ` [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding Ævar Arnfjörð Bjarmason
@ 2021-03-11  0:14         ` Ævar Arnfjörð Bjarmason
  2021-03-11  1:23           ` Junio C Hamano
  5 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11  0:14 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, brian m . carlson, Eric Sunshine,
	Elijah Newren, Ævar Arnfjörð Bjarmason

Remove the last users of the $_x05 variable from the tests. It turns
out that all of these tests can be rewritten unambiguously to simply
use [0-9a-f]* instead.

In the case of the tree matching we're relying on there being a <TAB>
after the SHA (but a space between the modes and type), then in some
of the other tests here that an abbreviated SHA is at the start of the
line, etc.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---
 t/t3101-ls-tree-dirname.sh          |  2 +-
 t/t3508-cherry-pick-many-commits.sh |  4 ++--
 t/t6006-rev-list-format.sh          | 12 +++++++-----
 t/t7600-merge.sh                    |  4 ++--
 t/test-lib.sh                       |  6 +-----
 5 files changed, 13 insertions(+), 15 deletions(-)

diff --git a/t/t3101-ls-tree-dirname.sh b/t/t3101-ls-tree-dirname.sh
index 12bf31022a..110ebf7d52 100755
--- a/t/t3101-ls-tree-dirname.sh
+++ b/t/t3101-ls-tree-dirname.sh
@@ -187,7 +187,7 @@ EOF
 
 test_expect_success 'ls-tree --abbrev=5' '
 	git ls-tree --abbrev=5 $tree >current &&
-	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
+	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
 	cat >expected <<\EOF &&
 100644 blob X	1.txt
 100644 blob X	2.txt
diff --git a/t/t3508-cherry-pick-many-commits.sh b/t/t3508-cherry-pick-many-commits.sh
index e8375d1c97..1bf867d139 100755
--- a/t/t3508-cherry-pick-many-commits.sh
+++ b/t/t3508-cherry-pick-many-commits.sh
@@ -84,7 +84,7 @@ test_expect_success 'output to keep user entertained during multi-pick' '
 	git reset --hard first &&
 	test_tick &&
 	git cherry-pick first..fourth >actual &&
-	sed -e "s/$_x05[0-9a-f][0-9a-f]/OBJID/" <actual >actual.fuzzy &&
+	sed -e "s/ [0-9a-f]*\\]/ OBJID]/" <actual >actual.fuzzy &&
 	test_line_count -ge 3 actual.fuzzy &&
 	test_cmp expected actual.fuzzy
 '
@@ -122,7 +122,7 @@ test_expect_success 'output during multi-pick indicates merge strategy' '
 	git reset --hard first &&
 	test_tick &&
 	git cherry-pick --strategy resolve first..fourth >actual &&
-	sed -e "s/$_x05[0-9a-f][0-9a-f]/OBJID/" <actual >actual.fuzzy &&
+	sed -e "s/ [0-9a-f]*\\]/ OBJID]/" <actual >actual.fuzzy &&
 	test_cmp expected actual.fuzzy
 '
 
diff --git a/t/t6006-rev-list-format.sh b/t/t6006-rev-list-format.sh
index 35a2f62392..ec65081e17 100755
--- a/t/t6006-rev-list-format.sh
+++ b/t/t6006-rev-list-format.sh
@@ -453,16 +453,18 @@ test_expect_success 'add SP before non-empty (2)' '
 '
 
 test_expect_success '--abbrev' '
-	echo SHORT SHORT SHORT >expect2 &&
 	echo LONG LONG LONG >expect3 &&
 	git log -1 --format="%h %h %h" HEAD >actual1 &&
 	git log -1 --abbrev=5 --format="%h %h %h" HEAD >actual2 &&
 	git log -1 --abbrev=5 --format="%H %H %H" HEAD >actual3 &&
-	sed -e "s/$OID_REGEX/LONG/g" -e "s/$_x05/SHORT/g" <actual2 >fuzzy2 &&
-	sed -e "s/$OID_REGEX/LONG/g" -e "s/$_x05/SHORT/g" <actual3 >fuzzy3 &&
-	test_cmp expect2 fuzzy2 &&
+	sed -e "s/$OID_REGEX/LONG/g" <actual3 >fuzzy3 &&
+	test_file_size actual2 >expect &&
+	# 3*5 SHAs + 3 separating spaces
+	echo 18 >actual &&
+	test_cmp expect actual &&
 	test_cmp expect3 fuzzy3 &&
-	! test_cmp actual1 actual2
+	! test_cmp actual1 actual2 &&
+	! test_cmp actual2 actual3
 '
 
 test_expect_success '%H is not affected by --abbrev-commit' '
diff --git a/t/t7600-merge.sh b/t/t7600-merge.sh
index 1cbc9715a8..fec5a7b2dd 100755
--- a/t/t7600-merge.sh
+++ b/t/t7600-merge.sh
@@ -189,7 +189,7 @@ test_expect_success 'merge c0 with c1' '
 	verify_head "$c1" &&
 
 	git reflog -1 >reflog.actual &&
-	sed "s/$_x05[0-9a-f]*/OBJID/g" reflog.actual >reflog.fuzzy &&
+	sed "s/^[0-9a-f]*/OBJID/" <reflog.actual >reflog.fuzzy &&
 	test_cmp reflog.expected reflog.fuzzy
 '
 
@@ -220,7 +220,7 @@ test_expect_success 'merge from unborn branch' '
 	verify_head "$c1" &&
 
 	git reflog -1 >reflog.actual &&
-	sed "s/$_x05[0-9a-f][0-9a-f]/OBJID/g" reflog.actual >reflog.fuzzy &&
+	sed "s/^[0-9a-f]*/OBJID/g" reflog.actual >reflog.fuzzy &&
 	test_cmp reflog.expected reflog.fuzzy
 '
 
diff --git a/t/test-lib.sh b/t/test-lib.sh
index 4d5ba558d3..aeb4b2da1c 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -511,7 +511,7 @@ SQ=\'
 # when case-folding filenames
 u200c=$(printf '\342\200\214')
 
-export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
+export LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
 
 # Each test should start with something like this, after copyright notices:
 #
@@ -1380,10 +1380,6 @@ then
 	fi
 fi
 
-# Convenience
-# A regexp to match 5 hexdigits
-_x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
-
 test_oid_init
 
 ZERO_OID=$(test_oid zero)
-- 
2.31.0.rc1.213.gbfdb770ff55


^ permalink raw reply related	[flat|nested] 116+ messages in thread

* Re: [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables
  2021-03-11  0:14         ` [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables Ævar Arnfjörð Bjarmason
@ 2021-03-11  1:02           ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-11  1:02 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, brian m . carlson, Eric Sunshine, Elijah Newren

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> These two have fallen out of use with the SHA-256 migration.
>
> The last use of $_x40 was removed in fc7e73d7ef (t4013: improve
> diff-post-processor logic, 2020-08-21) and
>
> The last use of $_z40 was removed in 7a868c51c2 (t5562: use $ZERO_OID,
> 2019-12-21), but it was then needlessly refactored to be hash-agnostic
> in 192b517589 (t: use hash-specific lookup tables to define test
> constants, 2020-02-22). We can just remove it.
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> ---
>  t/test-lib.sh | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)

Nice.  Curious why we still want x5 and x35, but we'll learn soon
enough in this short series ;-)


> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index d3f6af6a65..5f2ad2fd81 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -511,7 +511,7 @@ SQ=\'
>  # when case-folding filenames
>  u200c=$(printf '\342\200\214')
>  
> -export _x05 _x35 _x40 _z40 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
> +export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
>  
>  # Each test should start with something like this, after copyright notices:
>  #
> @@ -1381,10 +1381,9 @@ then
>  fi
>  
>  # Convenience
> -# A regexp to match 5, 35 and 40 hexdigits
> +# A regexp to match 5 and 35 hexdigits
>  _x05='[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]'
>  _x35="$_x05$_x05$_x05$_x05$_x05$_x05$_x05"
> -_x40="$_x35$_x05"
>  
>  test_oid_init
>  
> @@ -1393,7 +1392,6 @@ OID_REGEX=$(echo $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
>  OIDPATH_REGEX=$(test_oid_to_path $ZERO_OID | sed -e 's/0/[0-9a-f]/g')
>  EMPTY_TREE=$(test_oid empty_tree)
>  EMPTY_BLOB=$(test_oid empty_blob)
> -_z40=$ZERO_OID
>  
>  # Provide an implementation of the 'yes' utility; the upper bound
>  # limit is there to help Windows that cannot stop this loop from

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding
  2021-03-11  0:14         ` [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding Ævar Arnfjörð Bjarmason
@ 2021-03-11  1:18           ` Junio C Hamano
  2021-03-11 19:15           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-11  1:18 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, brian m . carlson, Eric Sunshine, Elijah Newren

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      $(git rev-parse HEAD~5)
> +	      $(git rev-parse HEAD~4)
> +	      $(git rev-parse HEAD~3)
> +	      $(git rev-parse HEAD~2)
> +	      $(git rev-parse HEAD~1)
> +
> +	Someone else (1):
> +	      $(git rev-parse HEAD~0)
> +
> +	EOF
> ...
>  test_expect_success '--abbrev' '
> -	sed s/SUBJECT/OBJID/ expect.template >expect &&
> +	cut -c 1-41 <expect >expect.abbrev &&
>  	git shortlog --format="%h" --abbrev=35 HEAD >log &&

The hardcoded 35 is a bit curious, and 1-41 even more so.

Is the idea that the "expect" prepared earlier has full hexdigits
and indented by 6 columns, the full hexdigits for an object name is
longer than 35 chars in any hash algorithm we use, any sample name
plus (number) plus colon won't be 35 chars long, and 35+6 happens to
be 41 so cutting at 41 column we'd get the first 35 hexdigits of the
object names without losing anything on the summary line?

That's a bit too subtle to my taste, but 

	test_expect_success '--abbrev' '
		abbrev=35 &&
		cut -c 1-$(( abbrev + 6 )) expect >expect.abbrev &&
		git shortlog --format="%h" --abbrev=$abbrev HEAD >log &&
		...

may not make it much easier to follow (without explaining what that
6 is anyway), so I'll let it pass.

Otherwise the steps so far make quite a lot of sense.

Thanks.







^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite
  2021-03-11  0:14         ` [PATCH 4/4] tests: get rid of $_x05 from the test suite Ævar Arnfjörð Bjarmason
@ 2021-03-11  1:23           ` Junio C Hamano
  2021-03-11 10:29             ` Ævar Arnfjörð Bjarmason
  0 siblings, 1 reply; 116+ messages in thread
From: Junio C Hamano @ 2021-03-11  1:23 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, brian m . carlson, Eric Sunshine, Elijah Newren

Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:

> Remove the last users of the $_x05 variable from the tests. It turns
> out that all of these tests can be rewritten unambiguously to simply
> use [0-9a-f]* instead.

I am unsure about the "unambiguously" part, e.g.

> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&

does't the preimage say "we expect at least 5 hexdigits to be shown
here"?  The postimage lets even an empty string to pass. 

> In the case of the tree matching we're relying on there being a <TAB>
> after the SHA (but a space between the modes and type), then in some
> of the other tests here that an abbreviated SHA is at the start of the
> line, etc.

Sure, but these being tests, I am not sure we should be assuming the
correct input to these transformations.

>  test_expect_success 'ls-tree --abbrev=5' '
>  	git ls-tree --abbrev=5 $tree >current &&
> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>  	cat >expected <<\EOF &&
>  100644 blob X	1.txt
>  100644 blob X	2.txt

This one is particularly iffy.  The --abbrev=5 test is designed to
ensure that the resulting abbreviated object names are at least 5
hexdigits long, even when the repository is so small that only 4
hexdigits are sufficient to avoid ambiguity, while allowing the
output to be longer than specified 5 (when 5 turns out to be
insufficient for disambiguation).

So, I dunno.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 1/3] t7003: test ref rewriting explicitly
  2021-03-11  0:10         ` Ævar Arnfjörð Bjarmason
@ 2021-03-11  1:55           ` brian m. carlson
  2021-03-11  2:24             ` Jeff King
  0 siblings, 1 reply; 116+ messages in thread
From: brian m. carlson @ 2021-03-11  1:55 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: Jeff King, git, Eric Sunshine, Elijah Newren

[-- Attachment #1: Type: text/plain, Size: 584 bytes --]

On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
> 
> On Wed, Mar 10 2021, Jeff King wrote:
> 
> 
> I've read it all over, looks good to me.
> 
> > Note these all pass currently, but the latter two will fail when run
> > with GIT_TEST_DEFAULT_HASH=sha256.
> 
> I don't know if anyone relies on that. Probably fine to have this as-is.

Our CI runs that way, and I always use that mode.  Are you saying that
these will always fail in that mode, or that they will pass at the end
of the series?
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 1/3] t7003: test ref rewriting explicitly
  2021-03-11  1:55           ` brian m. carlson
@ 2021-03-11  2:24             ` Jeff King
  2021-03-11  2:49               ` brian m. carlson
  0 siblings, 1 reply; 116+ messages in thread
From: Jeff King @ 2021-03-11  2:24 UTC (permalink / raw)
  To: brian m. carlson
  Cc: Ævar Arnfjörð Bjarmason, git, Eric Sunshine,
	Elijah Newren

On Thu, Mar 11, 2021 at 01:55:04AM +0000, brian m. carlson wrote:

> On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
> > 
> > On Wed, Mar 10 2021, Jeff King wrote:
> > 
> > 
> > I've read it all over, looks good to me.
> > 
> > > Note these all pass currently, but the latter two will fail when run
> > > with GIT_TEST_DEFAULT_HASH=sha256.
> > 
> > I don't know if anyone relies on that. Probably fine to have this as-is.
> 
> Our CI runs that way, and I always use that mode.  Are you saying that
> these will always fail in that mode, or that they will pass at the end
> of the series?

The latter; everything passes at the end. The only thing that is hurt is
if we care about bisecting the tests with GIT_TEST_DEFAULT_HASH set

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 1/3] t7003: test ref rewriting explicitly
  2021-03-11  2:24             ` Jeff King
@ 2021-03-11  2:49               ` brian m. carlson
  0 siblings, 0 replies; 116+ messages in thread
From: brian m. carlson @ 2021-03-11  2:49 UTC (permalink / raw)
  To: Jeff King
  Cc: Ævar Arnfjörð Bjarmason, git, Eric Sunshine,
	Elijah Newren

[-- Attachment #1: Type: text/plain, Size: 1004 bytes --]

On 2021-03-11 at 02:24:29, Jeff King wrote:
> On Thu, Mar 11, 2021 at 01:55:04AM +0000, brian m. carlson wrote:
> 
> > On 2021-03-11 at 00:10:29, Ævar Arnfjörð Bjarmason wrote:
> > > 
> > > On Wed, Mar 10 2021, Jeff King wrote:
> > > 
> > > > Note these all pass currently, but the latter two will fail when run
> > > > with GIT_TEST_DEFAULT_HASH=sha256.
> > > 
> > > I don't know if anyone relies on that. Probably fine to have this as-is.
> > 
> > Our CI runs that way, and I always use that mode.  Are you saying that
> > these will always fail in that mode, or that they will pass at the end
> > of the series?
> 
> The latter; everything passes at the end. The only thing that is hurt is
> if we care about bisecting the tests with GIT_TEST_DEFAULT_HASH set

I don't have a strong opinion on this.  I'm unlikely to bisect with the
full testsuite in that way; I'd use a git bisect run with a smaller
testcase.
-- 
brian m. carlson (he/him or they/them)
Houston, Texas, US

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite
  2021-03-11  1:23           ` Junio C Hamano
@ 2021-03-11 10:29             ` Ævar Arnfjörð Bjarmason
  2021-03-11 17:40               ` Junio C Hamano
  0 siblings, 1 reply; 116+ messages in thread
From: Ævar Arnfjörð Bjarmason @ 2021-03-11 10:29 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, brian m . carlson, Eric Sunshine, Elijah Newren,
	Eric Wong


On Thu, Mar 11 2021, Junio C Hamano wrote:

> Ævar Arnfjörð Bjarmason  <avarab@gmail.com> writes:
>
>> Remove the last users of the $_x05 variable from the tests. It turns
>> out that all of these tests can be rewritten unambiguously to simply
>> use [0-9a-f]* instead.
>
> I am unsure about the "unambiguously" part, e.g.
>
>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>
> does't the preimage say "we expect at least 5 hexdigits to be shown
> here"?  The postimage lets even an empty string to pass. 
>
>> In the case of the tree matching we're relying on there being a <TAB>
>> after the SHA (but a space between the modes and type), then in some
>> of the other tests here that an abbreviated SHA is at the start of the
>> line, etc.
>
> Sure, but these being tests, I am not sure we should be assuming the
> correct input to these transformations.
>
>>  test_expect_success 'ls-tree --abbrev=5' '
>>  	git ls-tree --abbrev=5 $tree >current &&
>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>>  	cat >expected <<\EOF &&
>>  100644 blob X	1.txt
>>  100644 blob X	2.txt
>
> This one is particularly iffy.  The --abbrev=5 test is designed to
> ensure that the resulting abbreviated object names are at least 5
> hexdigits long, even when the repository is so small that only 4
> hexdigits are sufficient to avoid ambiguity, while allowing the
> output to be longer than specified 5 (when 5 turns out to be
> insufficient for disambiguation).
>
> So, I dunno.

Yes, I think this patch should be dropped. Do you mind just dropping the
4/4 and having it be a 3-patch series? I can also re-submit a v2 like
that if it's easier...

I saw the feedback on 3/4, yeah I also think that "cut" is a bit nasty,
but probably not worth spending more time on it...

My assumption in writing this patch was that it was fine because we test
the details of abbrev behavior somewhere else, surely...

But is it turns out we don't, I was misrecalling that some version of my
testing for abbreviations in [1] had landed.

But alas it didn't. From rebasing it now (it's far from submission
quality yet) I see that the lack of testing on abbreviations in the
interim has silently caused some regressions on master.

E.g. we seemingly unintentionally started accepting "-c core.abbrev="
(empty string) in a9ecaa06a7 (core.abbrev=no disables abbreviations,
2020-09-01) is a side-effect of starting to accept boolean values. Also
"git branch --abbrev=-12345" became a non-error in the interim (haven't
dug, but some of the refactoring to parse_options() I assume..).

So yeah, I think this patch is a bad idea now, our test coverage for
abbreviations is really hanging on by a thread.

1. https://lore.kernel.org/git/20180608224136.20220-1-avarab@gmail.com/

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 4/4] tests: get rid of $_x05 from the test suite
  2021-03-11 10:29             ` Ævar Arnfjörð Bjarmason
@ 2021-03-11 17:40               ` Junio C Hamano
  0 siblings, 0 replies; 116+ messages in thread
From: Junio C Hamano @ 2021-03-11 17:40 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Jeff King, brian m . carlson, Eric Sunshine, Elijah Newren,
	Eric Wong

Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes:

>>>  test_expect_success 'ls-tree --abbrev=5' '
>>>  	git ls-tree --abbrev=5 $tree >current &&
>>> -	sed -e "s/ $_x05[0-9a-f]*	/ X	/" <current >check &&
>>> +	sed -e "s/ [0-9a-f]*	/ X	/" <current >check &&
>>>  	cat >expected <<\EOF &&
>>>  100644 blob X	1.txt
>>>  100644 blob X	2.txt
>>
>> This one is particularly iffy.  The --abbrev=5 test is designed to
>> ensure that the resulting abbreviated object names are at least 5
>> hexdigits long, even when the repository is so small that only 4
>> hexdigits are sufficient to avoid ambiguity, while allowing the
>> output to be longer than specified 5 (when 5 turns out to be
>> insufficient for disambiguation).
>>
>> So, I dunno.
>
> Yes, I think this patch should be dropped. Do you mind just dropping the
> 4/4 and having it be a 3-patch series? I can also re-submit a v2 like
> that if it's easier...

As long as the earlier three are good to go, I can just cut the tip
of the branch, or just drop it all now and send a three-patch series
after the release.  Either is fine.

> My assumption in writing this patch was that it was fine because we test
> the details of abbrev behavior somewhere else, surely...

I expected that the test whose title is "ls-tree --abbrev=5" is
targetted towards testing the details of abbrev behaviour.  Isn't
that the case?

Is the same assumption brought silent breakages to the other two
patches to the tests, by chance?  I only gave cursory look on them
and don't think anybody else looked at them carefully.

^ permalink raw reply	[flat|nested] 116+ messages in thread

* Re: [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding
  2021-03-11  0:14         ` [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding Ævar Arnfjörð Bjarmason
  2021-03-11  1:18           ` Junio C Hamano
@ 2021-03-11 19:15           ` Jeff King
  1 sibling, 0 replies; 116+ messages in thread
From: Jeff King @ 2021-03-11 19:15 UTC (permalink / raw)
  To: Ævar Arnfjörð Bjarmason
  Cc: git, Junio C Hamano, brian m . carlson, Eric Sunshine, Elijah Newren

On Thu, Mar 11, 2021 at 01:14:46AM +0100, Ævar Arnfjörð Bjarmason wrote:

> Rewrite much of the test logic introduced in 600372497c (shortlog:
> Document and test --format option, 2010-05-03), this allows us to get
> rid of the now-unused $_x35 variable in test-lib.sh
> 
> There was a minimal migration of this test to SHA-256 in
> 2ece6ad281 (t: switch $_x40 to $OID_REGEX, 2018-05-13), but actually
> we can just get rid of all the assumptions about hashing here, and
> make this easier to understand and maintain while we're at it.

Sounds like a good goal. I had to spend a few minutes deciphering what
was going on in the diff, so I'll follow along loudly...

> diff --git a/t/t4201-shortlog.sh b/t/t4201-shortlog.sh
> index 3095b1b2ff..3ef17c06e4 100755
> --- a/t/t4201-shortlog.sh
> +++ b/t/t4201-shortlog.sh
> @@ -19,82 +19,94 @@ test_expect_success 'setup' '
>  	commit=$(printf "%s\n" "Test" "" | git commit-tree "$tree") &&
>  	git update-ref HEAD "$commit" &&
>  
> +	echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" >message &&
> +
> +	sed "s/i/1234/g" <message >tmp &&
> +	tr 1234 "\360\235\204\236" <tmp >message.2 &&
> +
> +	sed "s/i/1234/g" <message >tmp &&
> +	tr 1234 "\370\235\204\236" <tmp >message.3 &&
> +
> +	echo "a								12	34	56	78" >message.4 &&
> +	echo "Commit by someone else" >message.5 &&
> +

This is all moving the messages into files so that you can pull them
into the expected output later. OK (feels weird to use sed and tr
together like this, but maybe it's a portability thing for sed which
doesn't understand octal; anyway, it's definitely not new in your
patch).

That might have been easier to understand if each message was generated
next to its comment. E.g...

>  	# test if the wrapping is still valid
>  	# when replacing all is by treble clefs.
>  	echo 3 >a1 &&
> -	git commit --quiet -m "$(
> -		echo "This is a very, very long first line for the commit message to see if it is wrapped correctly" |
> -		sed "s/i/1234/g" |
> -		tr 1234 "\360\235\204\236")" a1 &&
> +	git commit --quiet -F message.2 a1 &&

...if we made message.2 here, then the comment would be explaining what
the opaque bytes are doing (and likewise why they differ in message.3).

Also, the mismatch between the "3" going into the file and "message.2"
is confusing. It's not important, but I wonder if using "commit
--allow-empty" and just skipping a1, etc, would make it easier to read.

Similarly, --quiet seems pointless here. If the test isn't run with
--verbose, the user doesn't see it either way. If it is, then they
probably want to see the output.

> -	cat >expect.template <<-\EOF
> +	cat >expect.default <<-EOF
>  	A U Thor (5):
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> -	      SUBJECT
> +	      Test
> +	      $(cat message)
> +	      $(cat message.2)
> +	      $(cat message.3)
> +	      $(cat message.4)
>  
>  	Someone else (1):
> -	      SUBJECT
> +	      $(cat message.5)
>  
>  	EOF

OK, and here's where we make the actual expected output, rather than a
template. I think this is an improvement in understanding what's going
on (and also is more robust).

I was slightly confused that we are not looking for the messages to have
been wrapped (just based on the surrounding code), but that is not new
to your patch.

> -fuzz() {
> -	file=$1 &&
> -	sed "
> -			s/$OID_REGEX/OBJECT_NAME/g
> -			s/$_x35/OBJID/g
> -			s/^ \{6\}[CTa].*/      SUBJECT/g
> -			s/^ \{8\}[^ ].*/        CONTINUATION/g
> -		" <"$file" >"$file.fuzzy" &&
> -	sed "/CONTINUATION/ d" <"$file.fuzzy"
> -}
> [...]
>  test_expect_success 'pretty format' '
> -	sed s/SUBJECT/OBJECT_NAME/ expect.template >expect &&
> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      $(git rev-parse HEAD~5)
> +	      $(git rev-parse HEAD~4)
> +	      $(git rev-parse HEAD~3)
> +	      $(git rev-parse HEAD~2)
> +	      $(git rev-parse HEAD~1)

And here we expect the real hashes rather than the fuzz() magic, which
makes sense.

Aside: here and in the others, I cringe a little at the cost of all of
the $() calls inside the here-doc. But I don't think there's a better
way to do it. I'd sometimes use environment variables for this, but the
processing we're doing probably makes it better to keep them in real
files. Or maybe not. TBH, just expanding out the messages in the source
like:

  m1=$(printf "Th\370\235\204\236s \370\235\204\236s a very, very long f\370\235\204\236rst l\370\235\204\236ne for the comm\370\235\204\236t message to see \370\235\204\236f \370\235\204\236t \370\235\204\236s wrapped correctly")

is equally unreadable to the original to me. ;) Possibly it would be
even more readable if some more meaningful pattern of replacements was
used. But I am firmly in bikeshedding territory there, so feel free to
ignore.

>  test_expect_success '--abbrev' '
> -	sed s/SUBJECT/OBJID/ expect.template >expect &&
> +	cut -c 1-41 <expect >expect.abbrev &&
>  	git shortlog --format="%h" --abbrev=35 HEAD >log &&
> -	fuzz log >log.predictable &&
> -	test_cmp expect log.predictable
> +	test_cmp expect.abbrev log
>  '

And this is the same thing as %H, but abbreviated.

I agree with Junio that the "1-41" seems magical. His suggestion to show
the math helps with that (or even just a comment). You could also just
build the "expect" file with:

  $(git rev-parse --abbrev=35 HEAD~5)

etc (I also have to wonder if there is much value in this beyond
checking the first commit, but that certainly predates your patch).

>  test_expect_success 'output from user-defined format is re-wrapped' '
> -	sed "s/SUBJECT/two lines/" expect.template >expect &&
> +	cat >expect <<-EOF &&
> +	A U Thor (5):
> +	      two lines
> +	      two lines
> +	      two lines
> +	      two lines
> +	      two lines
> +
> +	Someone else (1):
> +	      two lines
> +
> +	EOF
>  	git shortlog --format="two%nlines" HEAD >log &&
> -	fuzz log >log.predictable &&
> -	test_cmp expect log.predictable
> +	test_cmp expect log
>  '

And this one is IMHO much improved in readability.

>  test_expect_success !MINGW 'shortlog wrapping' '
> diff --git a/t/test-lib.sh b/t/test-lib.sh
> index 5f2ad2fd81..4d5ba558d3 100644
> --- a/t/test-lib.sh
> +++ b/t/test-lib.sh
> @@ -511,7 +511,7 @@ SQ=\'
>  # when case-folding filenames
>  u200c=$(printf '\342\200\214')
>  
> -export _x05 _x35 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX
> +export _x05 LF u200c EMPTY_TREE EMPTY_BLOB ZERO_OID OID_REGEX

And now we can get rid of _x05. Yay.

-Peff

^ permalink raw reply	[flat|nested] 116+ messages in thread

end of thread, other threads:[~2021-03-11 19:16 UTC | newest]

Thread overview: 116+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-28 23:34 [PATCH v5 00/39] SHA-256, part 3/3 brian m. carlson
2020-07-28 23:34 ` [PATCH v5 01/39] t: make test-bloom initialize repository brian m. carlson
2020-07-28 23:34 ` [PATCH v5 02/39] t1001: use $ZERO_OID brian m. carlson
2020-07-28 23:34 ` [PATCH v5 03/39] t3305: make hash agnostic brian m. carlson
2020-07-28 23:34 ` [PATCH v5 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
2020-07-28 23:34 ` [PATCH v5 05/39] t6100: make hash size independent brian m. carlson
2020-07-28 23:34 ` [PATCH v5 06/39] t6101: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 07/39] t6301: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 08/39] t6500: specify test values for SHA-256 brian m. carlson
2020-07-28 23:34 ` [PATCH v5 09/39] t6501: avoid hard-coded objects brian m. carlson
2020-07-28 23:34 ` [PATCH v5 10/39] t7003: compute appropriate length constant brian m. carlson
2020-07-28 23:34 ` [PATCH v5 11/39] t7063: make hash size independent brian m. carlson
2020-07-28 23:34 ` [PATCH v5 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
2020-07-28 23:34 ` [PATCH v5 13/39] t7102: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 14/39] t7400: make hash size independent brian m. carlson
2020-07-28 23:34 ` [PATCH v5 15/39] t7405: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
2020-07-28 23:34 ` [PATCH v5 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
2020-07-28 23:34 ` [PATCH v5 18/39] t8002: make hash size independent brian m. carlson
2020-07-29  2:24   ` Eric Sunshine
2020-07-29 22:31     ` brian m. carlson
2020-07-28 23:34 ` [PATCH v5 19/39] t8003: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 20/39] t8011: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
2020-07-28 23:34 ` [PATCH v5 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
2020-07-28 23:34 ` [PATCH v5 23/39] t9301: make hash size independent brian m. carlson
2020-07-28 23:34 ` [PATCH v5 24/39] t9350: " brian m. carlson
2020-07-28 23:34 ` [PATCH v5 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
2020-07-28 23:34 ` [PATCH v5 26/39] t9700: make hash size independent brian m. carlson
2020-07-28 23:34 ` [PATCH v5 27/39] t5308: make test work with SHA-256 brian m. carlson
2020-07-28 23:34 ` [PATCH v5 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
2020-07-28 23:34 ` [PATCH v5 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
2020-07-29  2:36   ` Eric Sunshine
2020-07-28 23:34 ` [PATCH v5 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
2020-07-28 23:34 ` [PATCH v5 31/39] bundle: add new version for use with SHA-256 brian m. carlson
2020-07-29  2:53   ` Eric Sunshine
2020-07-29 22:28     ` brian m. carlson
2020-07-28 23:34 ` [PATCH v5 32/39] setup: add support for reading extensions.objectformat brian m. carlson
2020-07-28 23:34 ` [PATCH v5 33/39] Enable SHA-256 support by default brian m. carlson
2020-07-29  2:57   ` Eric Sunshine
2020-07-28 23:34 ` [PATCH v5 34/39] t: add test_oid option to select hash algorithm brian m. carlson
2020-07-28 23:34 ` [PATCH v5 35/39] t: allow testing different hash algorithms via environment brian m. carlson
2020-07-28 23:34 ` [PATCH v5 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
2020-07-29  3:01   ` Eric Sunshine
2020-07-28 23:34 ` [PATCH v5 37/39] ci: run tests with SHA-256 brian m. carlson
2020-07-28 23:34 ` [PATCH v5 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
2020-07-28 23:34 ` [PATCH v5 39/39] t: remove test_oid_init in tests brian m. carlson
2020-07-29 23:13 ` [PATCH v6 00/39] SHA-256, part 3/3 brian m. carlson
2020-07-29 23:13   ` [PATCH v6 01/39] t: make test-bloom initialize repository brian m. carlson
2020-07-29 23:13   ` [PATCH v6 02/39] t1001: use $ZERO_OID brian m. carlson
2020-07-29 23:13   ` [PATCH v6 03/39] t3305: make hash agnostic brian m. carlson
2020-07-29 23:13   ` [PATCH v6 04/39] t3404: prepare 'short SHA-1 collision' tests for SHA-256 brian m. carlson
2020-07-29 23:13   ` [PATCH v6 05/39] t6100: make hash size independent brian m. carlson
2020-07-29 23:13   ` [PATCH v6 06/39] t6101: " brian m. carlson
2020-07-29 23:13   ` [PATCH v6 07/39] t6301: " brian m. carlson
2020-07-29 23:13   ` [PATCH v6 08/39] t6500: specify test values for SHA-256 brian m. carlson
2020-07-29 23:13   ` [PATCH v6 09/39] t6501: avoid hard-coded objects brian m. carlson
2020-07-29 23:13   ` [PATCH v6 10/39] t7003: compute appropriate length constant brian m. carlson
2020-07-29 23:14   ` [PATCH v6 11/39] t7063: make hash size independent brian m. carlson
2020-07-29 23:14   ` [PATCH v6 12/39] t7201: abstract away SHA-1-specific constants brian m. carlson
2020-07-29 23:14   ` [PATCH v6 13/39] t7102: " brian m. carlson
2020-07-29 23:14   ` [PATCH v6 14/39] t7400: make hash size independent brian m. carlson
2020-07-29 23:14   ` [PATCH v6 15/39] t7405: " brian m. carlson
2020-07-29 23:14   ` [PATCH v6 16/39] t7506: avoid checking for SHA-1-specific constants brian m. carlson
2020-07-29 23:14   ` [PATCH v6 17/39] t7508: use $ZERO_OID instead of hard-coded constant brian m. carlson
2020-07-29 23:14   ` [PATCH v6 18/39] t8002: make hash size independent brian m. carlson
2020-07-29 23:14   ` [PATCH v6 19/39] t8003: " brian m. carlson
2020-07-29 23:14   ` [PATCH v6 20/39] t8011: " brian m. carlson
2020-07-29 23:14   ` [PATCH v6 21/39] t9300: abstract away SHA-1-specific constants brian m. carlson
2020-07-29 23:14   ` [PATCH v6 22/39] t9300: use $ZERO_OID instead of hard-coded object ID brian m. carlson
2020-07-29 23:14   ` [PATCH v6 23/39] t9301: make hash size independent brian m. carlson
2020-07-29 23:14   ` [PATCH v6 24/39] t9350: " brian m. carlson
2020-07-29 23:14   ` [PATCH v6 25/39] t9500: ensure that algorithm info is preserved in config brian m. carlson
2020-07-29 23:14   ` [PATCH v6 26/39] t9700: make hash size independent brian m. carlson
2020-07-29 23:14   ` [PATCH v6 27/39] t5308: make test work with SHA-256 brian m. carlson
2020-07-29 23:14   ` [PATCH v6 28/39] t0410: mark test with SHA1 prerequisite brian m. carlson
2020-07-29 23:14   ` [PATCH v6 29/39] http-fetch: set up git directory before parsing pack hashes brian m. carlson
2020-07-29 23:14   ` [PATCH v6 30/39] builtin/verify-pack: implement an --object-format option brian m. carlson
2020-07-29 23:14   ` [PATCH v6 31/39] bundle: add new version for use with SHA-256 brian m. carlson
2020-07-29 23:14   ` [PATCH v6 32/39] setup: add support for reading extensions.objectformat brian m. carlson
2020-07-29 23:14   ` [PATCH v6 33/39] repository: enable SHA-256 support by default brian m. carlson
2020-07-29 23:14   ` [PATCH v6 34/39] t: add test_oid option to select hash algorithm brian m. carlson
2020-07-29 23:14   ` [PATCH v6 35/39] t: allow testing different hash algorithms via environment brian m. carlson
2020-07-29 23:14   ` [PATCH v6 36/39] t: make SHA1 prerequisite depend on default hash brian m. carlson
2020-07-29 23:14   ` [PATCH v6 37/39] ci: run tests with SHA-256 brian m. carlson
2020-07-29 23:14   ` [PATCH v6 38/39] docs: add documentation for extensions.objectFormat brian m. carlson
2020-07-29 23:14   ` [PATCH v6 39/39] t: remove test_oid_init in tests brian m. carlson
2020-07-30  2:48   ` [PATCH v6 00/39] SHA-256, part 3/3 Eric Sunshine
2020-07-30 16:18     ` Junio C Hamano
2021-03-10 12:04   ` Ævar Arnfjörð Bjarmason
2021-03-10 16:36     ` Elijah Newren
2021-03-10 17:08       ` Jeff King
2021-03-10 17:06     ` [PATCH 0/3] sha256 fixes for filter-branch Jeff King
2021-03-10 17:07       ` [PATCH 1/3] t7003: test ref rewriting explicitly Jeff King
2021-03-11  0:10         ` Ævar Arnfjörð Bjarmason
2021-03-11  1:55           ` brian m. carlson
2021-03-11  2:24             ` Jeff King
2021-03-11  2:49               ` brian m. carlson
2021-03-10 17:07       ` [PATCH 2/3] filter-branch: drop multiple-ancestor warning Jeff King
2021-03-10 22:16         ` Junio C Hamano
2021-03-10 17:07       ` [PATCH 3/3] filter-branch: drop $_x40 glob Jeff King
2021-03-10 22:23         ` Junio C Hamano
2021-03-11  0:14         ` [PATCH 0/4] bisect + tests: remove old $_x05, $_x35 and $_x40 variables Ævar Arnfjörð Bjarmason
2021-03-11  0:14         ` [PATCH 1/4] git-bisect: remove unused SHA-1 $x40 shell variable Ævar Arnfjörð Bjarmason
2021-03-11  0:14         ` [PATCH 2/4] test-lib: remove unused $_x40 and $_z40 variables Ævar Arnfjörð Bjarmason
2021-03-11  1:02           ` Junio C Hamano
2021-03-11  0:14         ` [PATCH 3/4] shortlog tests: rewrite to get rid of --abbrev=35 hardcoding Ævar Arnfjörð Bjarmason
2021-03-11  1:18           ` Junio C Hamano
2021-03-11 19:15           ` Jeff King
2021-03-11  0:14         ` [PATCH 4/4] tests: get rid of $_x05 from the test suite Ævar Arnfjörð Bjarmason
2021-03-11  1:23           ` Junio C Hamano
2021-03-11 10:29             ` Ævar Arnfjörð Bjarmason
2021-03-11 17:40               ` Junio C Hamano
2021-03-10 17:54       ` [PATCH 0/3] sha256 fixes for filter-branch Elijah Newren
2021-03-10 22:24         ` Junio C Hamano
2021-03-10 23:32       ` brian m. carlson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).