Git Mailing List Archive on lore.kernel.org
 help / color / Atom feed
From: "brian m. carlson" <sandals@crustytoothpaste.net>
To: <git@vger.kernel.org>
Subject: [RFC PATCH 00/22] SHA-256 stage 4 implementation, part 1/3
Date: Mon, 13 Jan 2020 12:47:07 +0000
Message-ID: <20200113124729.3684846-1-sandals@crustytoothpaste.net> (raw)

This is an RFC series for part 1 of 3 of a SHA-256 implementation.  It
contains a few interesting pieces, and there are also some pieces it
does not contain.

First, it contains the pieces necessary to set up repositories and write
_but not read_ extensions.objectFormat.  In other words, you can create
a SHA-256 repository, but will be unable to read it.  We also
intentionally fail any use of SHA-256 in a repository except when
DEVELOPER=1 is set to discourage anyone from wiring it up, since at this
point it is nonfunctional.

It additionally contains code to adjust the setup code as necessary,
handle repository version 1 in worktrees, properly handle signatures in
tags and commits, and import and export submodules using SHA-256
(required for converting the Git repository).

The setup code now learns an environment variable to specify a default
hash.  This is useful for us because otherwise the test suite would need
every call to "git init" (which is a lot) to add a command line
argument, which would be untidy and burdensome.  It is not recommended
for everyday use.

Furthermore, this contains direct calls to test_oid_init in the test
suite setup code, since the alternative is duplicating those values.
I've opted not to remove the calls in the tests because we have other
topics in flight that may conflict with doing that, but I plan to send a
follow-up patch which does that at the end.  The _z40 variable is
persisted there for compatibility with master and will be dropped once
my current test series in next hits master.

There are also things it does not contain.  As mentioned, it lacks
support for reading extensions.objectFormat at all.  It lacks support
for cloning, fetching, and pushing, which while considered non-goals in
the transition plan, are required for the test suite to even come close
to passing.  That code, both for the original protocol and v2, would be
in part 2.  That code does not provide interoperability between SHA-1
and SHA-256 repositories, which will be the subject of the next major
chunk I do.  Part 3 contains the portions making the
extensions.objectFormat option functional and permitting us to run tests
against the new version.

This series, of course, lacks the test suite fixes which will be
required for the test suite to pass.  Those will be coming in two
further series, one of which I plan to send out soon.

Because this series sets up (and documents!) a useless option (which is
a large footgun) and because I'd like feedback about this approach, this
series is RFC.  I'd also like to know if you think anything is missing
outside of the items I've mentioned, because part 3 will result in the
test suite depending on SHA-256 support and therefore any structural
changes will be difficult to make at that point.  And of course, any
other feedback about this series is certainly welcome.

There may be other things that are interesting about this series, but
this cover letter is too small to contain them, so I encourage you to
look at the series for yourself.

If you'd like to see what the entire series looks like when complete,
you're welcome to inspect the transition-stage-4 branch at
https://github.com/bk2204/git.git.

brian m. carlson (22):
  hex: introduce parsing variants taking hash algorithms
  hex: add functions to parse hex object IDs in any algorithm
  repository: require a build flag to use SHA-256
  t: use hash-specific lookup tables to define test constants
  t6300: abstract away SHA-1-specific constants
  t6300: make hash algorithm independent
  t/helper/test-dump-split-index: initialize git repository
  t/helper: initialize repository if necessary
  t/helper: make repository tests hash independent
  setup: allow check_repository_format to read repository format
  builtin/init-db: allow specifying hash algorithm on command line
  builtin/init-db: add environment variable for new repo hash
  init-db: move writing repo version into a function
  worktree: allow repository version 1
  commit: use expected signature header for SHA-256
  gpg-interface: improve interface for parsing tags
  tag: store SHA-256 signatures in a header
  fast-import: permit reading multiple marks files
  fast-import: add helper function for inserting mark object entries
  fast-import: make find_marks work on any mark set
  fast-import: add a generic function to iterate over marks
  fast-import: add options for rewriting submodules

 Documentation/git-fast-import.txt |  20 +++
 Documentation/git-init.txt        |   7 +-
 Documentation/git.txt             |   6 +
 builtin/clone.c                   |   2 +-
 builtin/commit.c                  |   2 +-
 builtin/fmt-merge-msg.c           |  26 +++-
 builtin/init-db.c                 |  70 +++++++--
 builtin/mktag.c                   |  14 ++
 builtin/receive-pack.c            |   4 +-
 builtin/tag.c                     |  20 ++-
 cache.h                           |  25 ++-
 commit.c                          |  58 +++++--
 commit.h                          |   8 +
 config.mak.dev                    |   2 +
 fast-import.c                     | 246 ++++++++++++++++++++++--------
 gpg-interface.c                   |  17 ++-
 gpg-interface.h                   |   9 +-
 hex.c                             |  57 ++++++-
 log-tree.c                        |  14 +-
 path.c                            |   2 +-
 ref-filter.c                      |  23 ++-
 repository.c                      |   4 +
 sequencer.c                       |   2 +-
 setup.c                           |   6 +-
 t/helper/test-dump-split-index.c  |   2 +
 t/helper/test-repository.c        |  14 +-
 t/t1450-fsck.sh                   |  24 +++
 t/t5801-remote-helpers.sh         |   4 +-
 t/t6300-for-each-ref.sh           |  61 +++++---
 t/t7004-tag.sh                    |   8 +-
 t/t7030-verify-tag.sh             |  17 +++
 t/t7510-signed-commit.sh          |  16 +-
 t/t9300-fast-import.sh            | 109 +++++++++++++
 t/test-lib.sh                     |  29 ++--
 tag.c                             |  15 +-
 worktree.c                        |  10 +-
 36 files changed, 758 insertions(+), 195 deletions(-)


             reply index

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-13 12:47 brian m. carlson [this message]
2020-01-13 12:47 ` [RFC PATCH 01/22] hex: introduce parsing variants taking hash algorithms brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 02/22] hex: add functions to parse hex object IDs in any algorithm brian m. carlson
2020-01-15 21:40   ` Junio C Hamano
2020-01-16  0:22     ` brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 03/22] repository: require a build flag to use SHA-256 brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 04/22] t: use hash-specific lookup tables to define test constants brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 05/22] t6300: abstract away SHA-1-specific constants brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 06/22] t6300: make hash algorithm independent brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 07/22] t/helper/test-dump-split-index: initialize git repository brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 08/22] t/helper: initialize repository if necessary brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 09/22] t/helper: make repository tests hash independent brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 10/22] setup: allow check_repository_format to read repository format brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 11/22] builtin/init-db: allow specifying hash algorithm on command line brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 12/22] builtin/init-db: add environment variable for new repo hash brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 13/22] init-db: move writing repo version into a function brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 14/22] worktree: allow repository version 1 brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 15/22] commit: use expected signature header for SHA-256 brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 16/22] gpg-interface: improve interface for parsing tags brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 17/22] tag: store SHA-256 signatures in a header brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 18/22] fast-import: permit reading multiple marks files brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 19/22] fast-import: add helper function for inserting mark object entries brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 20/22] fast-import: make find_marks work on any mark set brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 21/22] fast-import: add a generic function to iterate over marks brian m. carlson
2020-01-13 12:47 ` [RFC PATCH 22/22] fast-import: add options for rewriting submodules brian m. carlson

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200113124729.3684846-1-sandals@crustytoothpaste.net \
    --to=sandals@crustytoothpaste.net \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Git Mailing List Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/git/0 git/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 git git/ https://lore.kernel.org/git \
		git@vger.kernel.org
	public-inbox-index git

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.git


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git