Re: Patches for message-digest support

Re: Patches for message-digest support

[Bill Zaumen]

... sorry for an additional message.  The patches I just sent
were based on commit 876a6f4991abdd72ea707b193b4f2b831096ad3c
(Update draft release notes to 1.7.9).

I should have also added that the function verify_commit was
tested via a compile-time option, but it is currently not used.
Its purpose is to verify that the (new) digest header in commit
messages is consistent with the commit's tree, parents, other
headers, and the commit message.   For authentication, one
would want to sign the commit SHA-1 hash and the message digest
for the commit (which is stored separate from the commit object).
My patch doesn't do that, but there is a single function that
can be called to look up the digest, if one is present (which may
not be the case due to backwards compatibility issues) - I'd prefer
to have someone familiar with the signature code make any changes.

The version of Makefile in the patch turns off the commit message-digest
header because some of the test scripts won't run with it, due to
those encoding specific SHA-1 values and file lengths, but the test does
run far enough to have created and used a number of commits.  I didn't
want to go to the trouble of updating the test scripts unless the patch
is actually going to get used - updating the scripts is a bit tedious
and you'd probably want to decide on the digest hash first.

Bill

Patches for message-digest support.

[Bill Zaumen]

I just sent a series of 6 patches, roughly similar to the ones I sent a
few weeks ago, but allowing a choice of message digests in addition to
a CRC (kept for testing purposes) - SHA-1, SHA-256, and SHA-512 with
more added easily.  The current default is SHA-256.  The use of SHA-1
for git object IDs is unchanged. Unlike the object-ID digest, the
additional digests do not include the Git object-header.  I also changed
a number of function names, using "digest" or "mdigest" in them.
Searching for the string "digest" is a good way of finding things.
Finally, I added a header to commit messages (conditionally compiled so
this can be turned off) that contains a digest of digests plus some
other fields.  I also broke it up into a series of smaller patches.

Just as a summary:

The first patch contains several new files.  It uses a data structure
for message digests that keeps the bytes of a digest aligned on
32 or 64 bit boundaries to allow fast comparisons. The digests are
stored long with a one-byte code indicating the digest type. The code
handles storing and looking up the digests, including support for
alternate object databases.

The second patch modifies some of the existing git files (the major
changes are in sha1_file.c and pack-write.c) for storing message digests
when an object or a pack index file is created.

The third patch modifies the files in the builtin directory that contain
the implementation of git commands for packing and pruning objects, and
for verifying pack files and counting objects.  The code does some
checks for hash collisions by comparing the digests.  At this point,
each git object will have a digest that can be looked up given the
object's ID.  This mapping is maintained as pack files are
created.

The fourth patch adds a digest header to commit messages.  This header
contains a digest of the digests for the commit's parents and for each
object in the commit's tree, and of the other fields in the commit.
The digest header, like the rest of the commit, is used in computing the
commit's object ID and matching digest.  A function verify_commit
checks the digest header by recomputing it and can be used as desired
for authentication or other purposes.

The fifth patch transfers the message digest corresponding to a SHA-1
ID during fetch or push operations to allow early detection of
collisions.  This is a fast test - a simple lookup - and can be turned
off by removing the "mds-check" capability.

The sixth patch contains documentation.

Bill