How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Patrick Schleizer]

Hi!

How safe are signed git tags? Especially because git uses SHA-1. There
is contradictory information around.

So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
the tag, and checks that `git status` reports no untracked/modified
files, without further manually auditing the code, how secure is this
actually? Is it only as safe as SHA-1?

Let's assume an adversary, that is capable of producing SHA-1 collisions.

Linus Torvalds said: [1]

> Git uses SHA-1 not for security

And goes on.

> The security parts are elsewhere

Could you please elaborate on this? Where are the security parts? Can
you please briefly explain how these work? Where can I read more about this?

Wikipedia says. [2]

> Nonetheless, without second preimage resistance [3] of SHA-1 signed
commits and tags would no longer secure the state of the repository as
they only sign the root of a Merkle tree [4].

Which contradicts what Linus Torvalds said. What does that mean for
security? Which statement is true?

> "The source control management system Git uses SHA-1 not for security
but for ensuring that the data has not changed due to accidental
corruption. Linus Torvalds has said, "If you have disk corruption, if
you have DRAM corruption, if you have any kind of problems at all, Git
will notice them. It's not a question of if, it's a guarantee. You can
have people who try to be malicious. They won't succeed. [...] Nobody
has been able to break SHA-1, but the point is the SHA-1, as far as Git
is concerned, isn't even a security feature. It's purely a consistency
check. The security parts are elsewhere, so a lot of people assume that
since Git uses SHA-1 and SHA-1 is used for cryptographically secure
stuff, they think that, OK, it's a huge security feature. It has nothing
at all to do with security, it's just the best hash you can get. [...] I
guarantee you, if you put your data in Git, you can trust the fact that
five years later, after it was converted from your hard disk to DVD to
whatever new technology and you copied it along, five years later you
can verify that the data you get back out is the exact same data you put
in. [...] One of the reasons I care is for the kernel, we had a break in
on one of the BitKeeper sites where people tried to corrupt the kernel
source code repositories." [6]

If (!) I understand Mike Gerwitz ([...] GNU [...]) 's opinion, his
opinion is, that for best security each and every commit should be
signed for best possible git verification security.

See also:

- Mike Gerwitz's "A Git Horror Story: Repository Integrity With Signed
Commits" [7]

- Verbose reply by Mike Gerwitz to my question. [8]

- Similar question on security stackexchange. [9] Quote: "Nevertheless,
If somebody managed to find a way how to find SHA1 collisions easily,
then git would have much bigger problem."

Cheers,
Patrick

[1] https://www.youtube.com/watch?v=4XpnKHJAok8&t=56m20s
[2] https://en.wikipedia.org/wiki/SHA-1#Data_integrity
[3] https://en.wikipedia.org/wiki/Second_preimage_resistance
[4] https://en.wikipedia.org/wiki/Merkle_tree
[5] https://www.youtube.com/watch?v=4XpnKHJAok8&t=56m20s
[6] https://en.wikipedia.org/wiki/SHA-1#Data_integrity
[7] http://mikegerwitz.com/papers/git-horror-story
[8] https://www.whonix.org/forum/index.php/topic,538.msg4278.html#msg4278
[9]
https://security.stackexchange.com/questions/67920/how-safe-are-signed-git-tags-only-as-safe-as-sha-1-or-somehow-safer

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Sun, Nov 16, 2014 at 03:31:10PM +0000, Patrick Schleizer wrote:

> How safe are signed git tags? Especially because git uses SHA-1. There
> is contradictory information around.
> 
> So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
> the tag, and checks that `git status` reports no untracked/modified
> files, without further manually auditing the code, how secure is this
> actually? Is it only as safe as SHA-1?

Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
only a SHA-1 hash. If somebody can find a collision with a hash you have
signed, they can substitute the colliding data for the data you signed.

Of course, "safe as SHA-1" and "find a collision" are vague. If
pre-image attacks are feasible (i.e., given already-published SHA-1, I
can find a different input with the same SHA-1), then attacks are
trivial. But when people talk about attacks on SHA-1, they are usually
referring to finding a collision between two new pieces of data. You can
also use that in an attack, but it's much less straightforward
(basically, you need to get somebody to sign one of the colliding pieces
of data and then replace it with the other).

And of course there is the question of getting the colliding data to the
victim. Git does collision checks whenever a remote (e.g., from a "git
fetch") gives us data that we already have. So you could poison new
cloners with bad data, but you could not convince a repository with the
existing "good" half of the collision to fetch the "evil" half.

> > Git uses SHA-1 not for security
> 
> And goes on.
> 
> > The security parts are elsewhere
> 
> Could you please elaborate on this? Where are the security parts? Can
> you please briefly explain how these work? Where can I read more about this?

I cannot speak for Linus, but I would not agree that SHA-1 is not part
of Git's security model. If we consider the GPG signature as a black box
(and we largely do in Git), then we _never_ sign the tree contents
itself. We always sign the SHA-1 of the tree, along with some metadata
(whether you are signing a tag or a commit). If an attacker can create
SHA-1 collisions (either by pre-image, or if you agree to sign a tree
containing a potential collision from an attacker), then you are
vulnerable to having the tree contents swapped out after the fact (and
the signature still checking out).

I am not sure that is what Linus is saying, though. In the paragraph you
quote:

> > "The source control management system Git uses SHA-1 not for security
> but for ensuring that the data has not changed due to accidental
> corruption. Linus Torvalds has said, "If you have disk corruption, if
> you have DRAM corruption, if you have any kind of problems at all, Git
> will notice them. It's not a question of if, it's a guarantee. You can
> have people who try to be malicious. They won't succeed. [...] Nobody
> has been able to break SHA-1, but the point is the SHA-1, as far as Git
> is concerned, isn't even a security feature. It's purely a consistency
> check. The security parts are elsewhere, so a lot of people assume that
> since Git uses SHA-1 and SHA-1 is used for cryptographically secure
> stuff, they think that, OK, it's a huge security feature. It has nothing
> at all to do with security, it's just the best hash you can get. [...] I
> guarantee you, if you put your data in Git, you can trust the fact that
> five years later, after it was converted from your hard disk to DVD to
> whatever new technology and you copied it along, five years later you
> can verify that the data you get back out is the exact same data you put
> in. [...] One of the reasons I care is for the kernel, we had a break in
> on one of the BitKeeper sites where people tried to corrupt the kernel
> source code repositories." [6]

I think he is saying more "SHA-1 is about data integrity, not about
authenticity; if you want authenticity, that's elsewhere [handled by
gpg]". Unsaid there is that you can't really have authenticity without
the integrity, and that I think he was assuming that SHA-1 works (he
says "Nobody has been able to break SHA-1..").

> If (!) I understand Mike Gerwitz ([...] GNU [...]) 's opinion, his
> opinion is, that for best security each and every commit should be
> signed for best possible git verification security.

I think it would depend on your threat model. You haven't defined "best
security".

Even without a break in SHA-1, there is value to signing every commit
versus signing just tags. If Linus signs a tag, all it says is "I think
this tree state and the history leading up to it is called v3.16". But
Linus can lie all he likes about who made each commit, and what happened
in each one. Signing commits is more about authenticating individual
commits: who made them, what was the state they were based on, and what
was the end state (and between the two, you can calculate the introduced
changes).

Of course, that comes with its own headaches, too. E.g., in mailing-list
development, the patch is picked up and applied by a maintainer, who
does not necessarily commit it on the same state you used to create it.
And even if they did, the committer information is different than what
you would create locally.  You inherently _can't_ sign that ahead of
time.

-Peff

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Patrick Schleizer]

Dear git developers!

Jeff King wrote:
> On Sun, Nov 16, 2014 at 03:31:10PM +0000, Patrick Schleizer wrote:
> 
>> How safe are signed git tags? Especially because git uses SHA-1. There
>> is contradictory information around.
>>
>> So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
>> the tag, and checks that `git status` reports no untracked/modified
>> files, without further manually auditing the code, how secure is this
>> actually? Is it only as safe as SHA-1?
> 
> Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
> only a SHA-1 hash. If somebody can find a collision with a hash you have
> signed, they can substitute the colliding data for the data you signed.
> 
> Of course, "safe as SHA-1" and "find a collision" are vague. If
> pre-image attacks are feasible (i.e., given already-published SHA-1, I
> can find a different input with the same SHA-1), then attacks are
> trivial. But when people talk about attacks on SHA-1, they are usually
> referring to finding a collision between two new pieces of data. You can
> also use that in an attack, but it's much less straightforward
> (basically, you need to get somebody to sign one of the colliding pieces
> of data and then replace it with the other).

Sounds pretty sad. Isn't this a security issue that should be fixed?

Rather than discussing how feasible collisions in SHA-1 are... Attacks
on SHA-1 are only getting worse, no? Since the Snowden revelations we
know that powerful adversaries that are working on such things and would
use such weaknesses to exploit users.

Dear git developers, could you please make a long story short? Change to
some stronger hash algorithm? (sha256, sha512, or so?) Or provide an
option for that?

> And of course there is the question of getting the colliding data to the
> victim. Git does collision checks whenever a remote (e.g., from a "git
> fetch") gives us data that we already have. So you could poison new
> cloners with bad data, but you could not convince a repository with the
> existing "good" half of the collision to fetch the "evil" half.

Poison git cloners with bad data is exactly my point here. Because
sometimes I am a cloner of my own code - cloning it on a separate
machine - then verify it using gpg - but don't check it any further. In
such cases, I'd prefer if security wouldn't depend on SHA-1.

Cheers,
Patrick

RE: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jason Pyeron]

> -----Original Message-----
> From: Patrick Schleizer
> Sent: Friday, November 21, 2014 18:01
> 
> Dear git developers!
> 
> Jeff King wrote:
> > On Sun, Nov 16, 2014 at 03:31:10PM +0000, Patrick Schleizer wrote:
> > 
> >> How safe are signed git tags? Especially because git uses 
> SHA-1. There
> >> is contradictory information around.
> >>
> >> So if one verifies a git tag (`git tag -v tagname`), then 
> `checksout`s
> >> the tag, and checks that `git status` reports no untracked/modified
> >> files, without further manually auditing the code, how 
> secure is this
> >> actually? Is it only as safe as SHA-1?
> > 
> > Yes, it is only as "safe as SHA-1" in the sense that you 
> have GPG-signed
> > only a SHA-1 hash. If somebody can find a collision with a 
> hash you have
> > signed, they can substitute the colliding data for the data 
> you signed.

The whole issue is a lot better than this makes it sound. Yes it is just a SHA1 hash, but it is a hash of a structured data format.

You have very observable parts of that well structured data providede to the hash.

The commit message, the directory contents, and lastly the files themselves.

For a collision to occur, the commit message would have to likely have garbage in the message of a large nature. To generate a colision by commited file contents is unlikely because the file contents is reduced to a hash in the directory structure, which is in turn reduced to a hash in a commit structure.

This would be noticed.


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00. 

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Fri, Nov 21, 2014 at 06:32:46PM -0500, Jason Pyeron wrote:

> The whole issue is a lot better than this makes it sound. Yes it is
> just a SHA1 hash, but it is a hash of a structured data format.
> 
> You have very observable parts of that well structured data providede to the hash.

Yeah, I glossed over that because I don't know enough about the specific
attacks.  In the worst case, you have a binary file format that lets
people stick arbitrary bits of data in the middle (like the MD5 attacks
on Postscript and PDF files), and you do the collision on the blobs.

But even with that, the sha1s are taken over "blob <n>\0<content>" where
<n> is the number of bytes of <content>. Depending on the exact scheme
for generating probable collisions is less than brute force time, even
that amount of structure may prove problematic. I don't know whether
that is the case for the best-known attacks or not (remember that nobody
has _actually_ generated a sha-1 collision at all yet).

-Peff

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Fri, Nov 21, 2014 at 11:01:26PM +0000, Patrick Schleizer wrote:

> > Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
> > only a SHA-1 hash. If somebody can find a collision with a hash you have
> > signed, they can substitute the colliding data for the data you signed.
> [..]
> Sounds pretty sad. Isn't this a security issue that should be fixed?

Sure, for some definition of "should". It's not a problem today. It may
be a problem in the future. If we were designing git from scratch today,
it would probably make sense to use a different hash, or to somehow
parameterize the hash.

But we're not starting from scratch. A change like that needs to
consider a transition plan. What happens to the existing history? Do we
just rewrite it all using the new hash in all object references? If so,
what do we do with non-object references to sha1s (in external systems,
or even partial sha1s mentioned in commit message)? What do we do with
signed tags which are now invalid?  Or do we graft history with the new
hashes onto the old, letting the two coexist in the same repository? How
do we expand the data structures to handle the extra information needed
to specify the hash type?

None of these problems is insurmountable, but it's going to take real
work on the development side, and is going to create incompatibilities
and headaches on the user side. It's probably something we'll need to
deal with in the next 10-15 years, but nobody knows quite when.

If you'd like to start working on it, I'd be happy to review your
patches. :) But in the meantime, I don't know that anybody considers it
an extremely high priority to work on, versus other fixes and features.

> Rather than discussing how feasible collisions in SHA-1 are... Attacks
> on SHA-1 are only getting worse, no?

Actually, not really. I do not keep up terribly well with the academic
literature, but I don't think that attacks have gotten any worse in the
last few years. Computers _are_ getting faster, though, so a number like
2^61 (which is what Wikipedia claims as the best widely accepted value
for producing a collision) gets more and more feasible as time passes.

Of course, we might find worse attacks (or if you want to put on your
tinfoil hat, perhaps certain government organizations already have and
are keeping them secret). 2^61 is a best case.

> > And of course there is the question of getting the colliding data to the
> > victim. Git does collision checks whenever a remote (e.g., from a "git
> > fetch") gives us data that we already have. So you could poison new
> > cloners with bad data, but you could not convince a repository with the
> > existing "good" half of the collision to fetch the "evil" half.
> 
> Poison git cloners with bad data is exactly my point here. Because
> sometimes I am a cloner of my own code - cloning it on a separate
> machine - then verify it using gpg - but don't check it any further. In
> such cases, I'd prefer if security wouldn't depend on SHA-1.

I agree that cloners are an important category of users to clone. But it
also means that a single fetcher can detect tampering quite easily.
Think about it this way: let's say the Walker/Schneier estimate is
right, and in 2021 it will cost ~$43K to find a collision. You spend the
money, find a collision on some binary blob that's in the kernel,
convince Linus to accept your "good" version, he signs, and then you
hack into kernel.org and replace the blob with your evil version. Now
the first time somebody fetches the evil version, their git complains
about the collision, kernel.org admins investigate, and the problem is
fixed. There's some damage, but ultimately you didn't accomplish much.

Or you could spend that $43K hiring somebody to break into Linus's house
and manipulate the local copy of the kernel on his computer that he's
going to sign. Or buy a zero-day exploit for his browser that gives you
remote code execution on his workstation.

Don't get me wrong. I think moving away from SHA-1 is a good idea, and
something we're going to want to do for security reasons eventually. But
we're definitely not at the point of "well, all of our signatures are
worthless now", and I'm not sure we'll be there sooner than a decade
from now.

-Peff

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Fedor Brunner]

On 22.11.2014 00:01, Patrick Schleizer wrote:
> Dear git developers!
> 
> Jeff King wrote:
>> On Sun, Nov 16, 2014 at 03:31:10PM +0000, Patrick Schleizer wrote:
>>
>>> How safe are signed git tags? Especially because git uses SHA-1. There
>>> is contradictory information around.
>>>
>>> So if one verifies a git tag (`git tag -v tagname`), then `checksout`s
>>> the tag, and checks that `git status` reports no untracked/modified
>>> files, without further manually auditing the code, how secure is this
>>> actually? Is it only as safe as SHA-1?
>>
>> Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
>> only a SHA-1 hash. If somebody can find a collision with a hash you have
>> signed, they can substitute the colliding data for the data you signed.
>>
>> Of course, "safe as SHA-1" and "find a collision" are vague. If
>> pre-image attacks are feasible (i.e., given already-published SHA-1, I
>> can find a different input with the same SHA-1), then attacks are
>> trivial. But when people talk about attacks on SHA-1, they are usually
>> referring to finding a collision between two new pieces of data. You can
>> also use that in an attack, but it's much less straightforward
>> (basically, you need to get somebody to sign one of the colliding pieces
>> of data and then replace it with the other).
> 
> Sounds pretty sad. Isn't this a security issue that should be fixed?
> 
> Rather than discussing how feasible collisions in SHA-1 are... Attacks
> on SHA-1 are only getting worse, no? Since the Snowden revelations we
> know that powerful adversaries that are working on such things and would
> use such weaknesses to exploit users.
> 
> Dear git developers, could you please make a long story short? Change to
> some stronger hash algorithm? (sha256, sha512, or so?) Or provide an
> option for that?
> 
>> And of course there is the question of getting the colliding data to the
>> victim. Git does collision checks whenever a remote (e.g., from a "git
>> fetch") gives us data that we already have. So you could poison new
>> cloners with bad data, but you could not convince a repository with the
>> existing "good" half of the collision to fetch the "evil" half.
> 
> Poison git cloners with bad data is exactly my point here. Because
> sometimes I am a cloner of my own code - cloning it on a separate
> machine - then verify it using gpg - but don't check it any further. In
> such cases, I'd prefer if security wouldn't depend on SHA-1.
> 
> Cheers,
> Patrick
> 
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Dear git developers,
how about skipping SHA-2 and moving to SHA-3 finalist BLAKE.

NIST, in the final report of the SHA-3 competition, said this about the
finalists (which included BLAKE, Keccak, Skein, and Grøstl):

    BLAKE had a security margin — the gap between a known-weak reduced
version and the full version — comparable to Keccak and superior to the
other finalists. (§4.3: “BLAKE and Keccak have very large security
margins.”)
    BLAKE had a depth of analysis — the amount of published research
analyzing its security — comparable to Grøstl and Skein and superior to
the other finalists. (§3.1: “Keccak received a significant amount of
cryptanalysis, although not quite the depth of analysis applied to
BLAKE, Grøstl, or Skein”)
    BLAKE had performance (in software) comparable to Skein and superior
to the other finalists. (§5.1.4: “Skein and BLAKE […] have the best
overall software performance.”)

http://nvlpubs.nist.gov/nistpubs/ir/2012/NIST.IR.7896.pdf



Measurements of SHA-3 finalists, indexed by machine

http://bench.cr.yp.to/results-sha3.html


Fedor

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 18, 2014 at 4:26 AM, Jeff King <peff@peff.net> wrote:
> Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
> only a SHA-1 hash. If somebody can find a collision with a hash you have
> signed, they can substitute the colliding data for the data you signed.

I wonder if we can have an option to sign all blob content of the tree
associated to a commit, and the content of parent commit(s). It's more
expensive than signing just commit/tag content. But it's also safer
without completely ditching SHA-1.
-- 
Duy

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Michael J Gruber]

Duy Nguyen schrieb am 24.11.2014 um 02:23:
> On Tue, Nov 18, 2014 at 4:26 AM, Jeff King <peff@peff.net> wrote:
>> Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
>> only a SHA-1 hash. If somebody can find a collision with a hash you have
>> signed, they can substitute the colliding data for the data you signed.
> 
> I wonder if we can have an option to sign all blob content of the tree
> associated to a commit, and the content of parent commit(s). It's more
> expensive than signing just commit/tag content. But it's also safer
> without completely ditching SHA-1.
> 

This amounts to hashing the blob content with whatever hash you told
your gpg to use (hopefully not sha1 ;) ) and signing that.

You're free to do that now and store the signature wherever your
toolchain deems fit, say in a note or an annotated tag. But that
approach won't sign the history, that is: If you are concerned about the
breakability of sha1, then history is "possibly broken" no matter how
you sign a commit object whose "parent" entry is based on the sha1 of
its parent object.

Cheers
Michael

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Mon, Nov 24, 2014 at 5:15 PM, Michael J Gruber
<git@drmicha.warpmail.net> wrote:
> Duy Nguyen schrieb am 24.11.2014 um 02:23:
>> On Tue, Nov 18, 2014 at 4:26 AM, Jeff King <peff@peff.net> wrote:
>>> Yes, it is only as "safe as SHA-1" in the sense that you have GPG-signed
>>> only a SHA-1 hash. If somebody can find a collision with a hash you have
>>> signed, they can substitute the colliding data for the data you signed.
>>
>> I wonder if we can have an option to sign all blob content of the tree
>> associated to a commit, and the content of parent commit(s). It's more
>> expensive than signing just commit/tag content. But it's also safer
>> without completely ditching SHA-1.
>>
>
> This amounts to hashing the blob content with whatever hash you told
> your gpg to use (hopefully not sha1 ;) ) and signing that.
>
> You're free to do that now and store the signature wherever your
> toolchain deems fit, say in a note or an annotated tag. But that
> approach won't sign the history, that is: If you are concerned about the
> breakability of sha1, then history is "possibly broken" no matter how
> you sign a commit object whose "parent" entry is based on the sha1 of
> its parent object.

If you store the singature in commit message, and if you hash the
parent commit messages as well, which are also signed, then you have
the same chaining effect that we have with SHA-1. I think this could
be done with some hooks already, except maybe for the verification
part.
-- 
Duy

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Mon, Nov 24, 2014 at 06:44:10PM +0700, Duy Nguyen wrote:
> >> I wonder if we can have an option to sign all blob content of the tree
> >> associated to a commit, and the content of parent commit(s). It's more
> >> expensive than signing just commit/tag content. But it's also safer
> >> without completely ditching SHA-1.
> >>
> >
> > This amounts to hashing the blob content with whatever hash you told
> > your gpg to use (hopefully not sha1 ;) ) and signing that.
> >
> > You're free to do that now and store the signature wherever your
> > toolchain deems fit, say in a note or an annotated tag. But that
> > approach won't sign the history, that is: If you are concerned about the
> > breakability of sha1, then history is "possibly broken" no matter how
> > you sign a commit object whose "parent" entry is based on the sha1 of
> > its parent object.
> 
> If you store the singature in commit message, and if you hash the
> parent commit messages as well, which are also signed, then you have
> the same chaining effect that we have with SHA-1. I think this could
> be done with some hooks already, except maybe for the verification
> part.

To demonstrate, a hook like this can take commit object from stdin and
produce some hash lines, which are appended at the end of the commit
message before the new commit object is created. So if I commit "foo"
the final commit message would be

  foo

  SHA512: <long hash>

This script uses sha512sum, but you can add as many hashes as you want
(and pay the penalty at commit time, of course). I think it covers
enough content to validate history up to the last signed commit.

-- 8< --
#!/bin/bash
# commit content
cat >$GIT_DIR/tmp
# parent commit content
sed '/^$/q' $GIT_DIR/tmp |
    grep '^parent ' |
    cut -c 8- |
    xargs -n1 git cat-file commit >>$GIT_DIR/tmp
# all blobs
sed '/^$/q' $GIT_DIR/tmp |
    grep '^tree ' |
    cut -c 6- |
    xargs -n1 git ls-tree -r |
    cut -c 13-52 |
    git cat-file --batch >>$GIT_DIR/tmp
echo
echo "SHA512: `sha512sum < $GIT_DIR/tmp`"
-- 8< --

An extra patch is required to hook this in final commit steps.

-- 8< --
diff --git a/commit.c b/commit.c
index 19cf8f9..c447c1d 100644
--- a/commit.c
+++ b/commit.c
@@ -11,6 +11,8 @@
 #include "commit-slab.h"
 #include "prio-queue.h"
 #include "sha1-lookup.h"
+#include "run-command.h"
+#include "sigchain.h"
 
 static struct commit_extra_header *read_commit_extra_header_lines(const char *buf, size_t len, const char **);
 
@@ -1064,6 +1066,36 @@ struct commit_list *reduce_heads(struct commit_list *heads)
 	return result;
 }
 
+static int run_sign_commit_hook(struct strbuf *buf)
+{
+	struct child_process hook = CHILD_PROCESS_INIT;
+	const char *p = find_hook("sign-commit");
+	int len;
+
+	if (!p)
+		return 0;
+
+	argv_array_push(&hook.args, p);
+	hook.in = -1;
+	hook.out = -1;
+	if (start_command(&hook))
+		return error(_("could not run sign-commit hook"));
+	sigchain_push(SIGPIPE, SIG_IGN);
+	if (write_in_full(hook.in, buf->buf, buf->len) != buf->len) {
+		close(hook.in);
+		close(hook.out);
+		finish_command(&hook);
+		return error(_("sign-commit hook did not accept the data"));
+	}
+	close(hook.in);
+	len = strbuf_read(buf, hook.out, 1024);
+	close(hook.out);
+	sigchain_pop(SIGPIPE);
+	if (finish_command(&hook) || len <= 0)
+		return error(_("sign-commit hook failed to sign the data"));
+	return 0;
+}
+
 static const char gpg_sig_header[] = "gpgsig";
 static const int gpg_sig_header_len = sizeof(gpg_sig_header) - 1;
 
@@ -1555,6 +1587,9 @@ int commit_tree_extended(const char *msg, size_t msg_len,
 	if (encoding_is_utf8 && !verify_utf8(&buffer))
 		fprintf(stderr, commit_utf8_warn);
 
+	if (run_sign_commit_hook(&buffer))
+		return -1;
+
 	if (sign_commit && do_sign_commit(&buffer, sign_commit))
 		return -1;
 
-- 8< --

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Mon, Nov 24, 2014 at 11:15:34AM +0100, Michael J Gruber wrote:

> > I wonder if we can have an option to sign all blob content of the tree
> > associated to a commit, and the content of parent commit(s). It's more
> > expensive than signing just commit/tag content. But it's also safer
> > without completely ditching SHA-1.
> > 
> 
> This amounts to hashing the blob content with whatever hash you told
> your gpg to use (hopefully not sha1 ;) ) and signing that.

Right. You could also create a graph of SHA-256 (or whatever) object
hashes and sign that. I.e., create a parallel to git's trees using
SHA-256 and include a single:

  object-256 ....

line in the tag header. That still involves re-hashing all of the data,
but it would at least be possible to cache (i.e., a mapping of SHA-1 to
SHA-256 hashes). Of course one way to keep that caching layer up to date
would be to just calculate the SHA-256 along with the SHA-1 whenever we
create an object. And then you can sprinkle SHA-256 links in other
places, too, like commit objects.

And now you are halfway down the road to a combined SHA-1/SHA-256 git.
:)

The tricky thing is fitting the extra hash into the tree objects. And of
course the rules for actually generating and/or sending extra objects.

-Peff

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Nico Williams]

Is there a plan for upgrading to a better hash function in the future?
 (E.g., should it become an urgent need.)

What are the roadblocks to adoption of a replacement hash function?
Just documenting this would go a long way towards making it possible
to upgrade some day.

Thanks,

Nico
--

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 1:14 AM, Nico Williams <nico@cryptonector.com> wrote:
> Is there a plan for upgrading to a better hash function in the future?
>  (E.g., should it become an urgent need.)
>
> What are the roadblocks to adoption of a replacement hash function?
> Just documenting this would go a long way towards making it possible
> to upgrade some day.

The biggest obstacle is the assumption of SHA-1 everywhere in the
source code (e.g. assume the object name always takes 20 bytes). Brian
started on cleaning that up [1] but I think it's stalled. Then we need
to deal with upgrade path for SHA-1 repos.

[1] http://thread.gmane.org/gmane.comp.version-control.git/248054
-- 
Duy

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jonathan Nieder]

Duy Nguyen wrote:

> The biggest obstacle is the assumption of SHA-1 everywhere in the
> source code (e.g. assume the object name always takes 20 bytes). Brian
> started on cleaning that up [1] but I think it's stalled. Then we need
> to deal with upgrade path for SHA-1 repos.

I think the biggest obstacle is the upgrade path. ;-)

If the upgrade path is taken care of, I won't mind writing and
reviewing a coccinelle-generated patch to replace 20, 40, 21, 41, and
so on with appropriate constants.  Or we can take the first 20 bytes
of a SHA-256, which is already supposed to have better security
properties than SHA-1.

Another obstacle is hard-coded SHA-1s in tests.

Thanks,
Jonathan

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> I think the biggest obstacle is the upgrade path. ;-)

In the worst case we can always treat new repos as a different VCS. So
people will need a migration from SHA-1 to the new format, just like
they migrate from SVN/CVS to Git. Painful but simple.
-- 
Duy

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Stefan Beller]

That's not *as* painful, because you'd have the beautiful
fast-{import/export} tools in your new and old version control system.
But yeah, there might be better ways to do so.


On Mon, Nov 24, 2014 at 5:52 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> I think the biggest obstacle is the upgrade path. ;-)
>
> In the worst case we can always treat new repos as a different VCS. So
> people will need a migration from SHA-1 to the new format, just like
> they migrate from SVN/CVS to Git. Painful but simple.
> --
> Duy
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Jeff King]

On Tue, Nov 25, 2014 at 08:52:58AM +0700, Duy Nguyen wrote:

> On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
> > I think the biggest obstacle is the upgrade path. ;-)
> 
> In the worst case we can always treat new repos as a different VCS. So
> people will need a migration from SHA-1 to the new format, just like
> they migrate from SVN/CVS to Git. Painful but simple.

Maybe we can fix the tree-sorting order while we are at it. :)

More seriously, there may come a day when we are ready to break
compatibility completely with a new "Git v3.0" (2.0 is already taken, of
course). I do not have immediate plans for it, but it's possible that
multiple factors may make such a move desirable sometime in the next 10
years, and that would be a good time to jump hash algorithms, as well.

So it's possible that procrastinating on SHA-1 issues may be the least
painful route. Or it may just be pushing off the day of pain. :)

-Peff

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Duy Nguyen]

On Tue, Nov 25, 2014 at 10:47 AM, Jeff King <peff@peff.net> wrote:
> Maybe we can fix the tree-sorting order while we are at it. :)

At this speed, there a teeny tiny chance that pack v4 will be ready by
Git v3.0 and we can pile that on top of the new tree format. And we
don't have to worry about v3<->v4 online conversion either because
it's backward incompatible ;)

> More seriously, there may come a day when we are ready to break
> compatibility completely with a new "Git v3.0" (2.0 is already taken, of
> course). I do not have immediate plans for it, but it's possible that
> multiple factors may make such a move desirable sometime in the next 10
> years, and that would be a good time to jump hash algorithms, as well.
>
> So it's possible that procrastinating on SHA-1 issues may be the least
> painful route. Or it may just be pushing off the day of pain. :)
-- 
Duy

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Tue, Nov 25, 2014 at 08:52:58AM +0700, Duy Nguyen wrote:
>
>> On Tue, Nov 25, 2014 at 8:23 AM, Jonathan Nieder <jrnieder@gmail.com> wrote:
>> > I think the biggest obstacle is the upgrade path. ;-)
>> 
>> In the worst case we can always treat new repos as a different VCS. So
>> people will need a migration from SHA-1 to the new format, just like
>> they migrate from SVN/CVS to Git. Painful but simple.
>
> Maybe we can fix the tree-sorting order while we are at it. :)
>
> More seriously, there may come a day when we are ready to break
> compatibility completely with a new "Git v3.0" (2.0 is already taken, of
> course). I do not have immediate plans for it, but it's possible that
> multiple factors may make such a move desirable sometime in the next 10
> years, and that would be a good time to jump hash algorithms, as well.

As the fundamental data model of Git is built around "given the
object name, the same data is retrieved, and equally importantly,
given the same data, the same object name is used, so that we can
say two objects with different names record different contents
without looking at the data", it does not mesh with the use of
tagged hash where object name consists of a pair of <what hash is
used> and <what the hash value is> at all.  It is a proper mindset
to treat it as a different VCS to give us a clean break when (I did
not say "if") we need to switch hashes, I think.

Thanks.

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

On Tue, Nov 25, 2014 at 08:16:15AM +0700, Duy Nguyen wrote:
> On Tue, Nov 25, 2014 at 1:14 AM, Nico Williams <nico@cryptonector.com> wrote:
> > Is there a plan for upgrading to a better hash function in the future?
> >  (E.g., should it become an urgent need.)
> >
> > What are the roadblocks to adoption of a replacement hash function?
> > Just documenting this would go a long way towards making it possible
> > to upgrade some day.
> 
> The biggest obstacle is the assumption of SHA-1 everywhere in the
> source code (e.g. assume the object name always takes 20 bytes). Brian
> started on cleaning that up [1] but I think it's stalled. Then we need
> to deal with upgrade path for SHA-1 repos.

Yes, it is stalled.  It ended up being a Herculean task, so when I pick
up the patch series again, I'll probably submit changes in chunks to
avoid the huge amount of code churn required.  I feel the list and Junio
in particular will appreciate that more.
-- 
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: How safe are signed git tags? Only as safe as SHA-1 or somehow safer?

[bancfc]

Hi, I wanted to chime in on the topic of SHA1 weaknesses and breaks. The 
problem is idea that SHA1 breaks are theoretical and will only  be 
relevant in a decade or two.

I think its a telling sign when even companies like Google [1] and 
Microsoft [2] who collaborate with spy agencies are moving away from 
SHA1 in verifying browser certs and the estimates by reputable 
cryptographers already put us in the realm of feasible breaks at this 
time, with the bar going lower with every passing year [3]. In three 
years common cyber criminals will be able to crack it using moderate 
sized computer clusters or by renting some AWS cycles.

Please reconsider the urgency of moving away from SHA1 for security 
functions in Git.


References:

[1] 
http://thenextweb.com/google/2014/09/05/google-will-start-sunsetting-sha-1-cryptographic-hash-algorithm-chrome-month-finish-q1-2015/

[2] https://www.schneier.com/blog/archives/2013/11/microsoft_retir.html 
(Schneier on Security: Microsoft Retiring SHA-1 in 2016)

[3] https://www.schneier.com/blog/archives/2012/10/when_will_we_se.html 
(When Will We See Collisions for SHA-1?)