git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Junio C Hamano <junkio@cox.net>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Alexander Litvinov <litvinov2004@gmail.com>,
	Git Mailing List <git@vger.kernel.org>
Subject: Re: My git repo is broken, how to fix it ?
Date: Tue, 20 Mar 2007 00:42:28 -0700	[thread overview]
Message-ID: <7vd5349x97.fsf@assigned-by-dhcp.cox.net> (raw)
In-Reply-To: <Pine.LNX.4.64.0703192212280.6730@woody.linux-foundation.org> (Linus Torvalds's message of "Mon, 19 Mar 2007 22:34:10 -0700 (PDT)")

Linus Torvalds <torvalds@linux-foundation.org> writes:

> But to really look at this we should probably add a "really_careful" flag 
> that actually re-verifies the SHA1 on read so that we'd catch these kinds 
> of corruptions early. 
> ...
> I'm hoping somebody has any ideas. We really *could* check the SHA1 on 
> each read (and slow down git a lot) and that would catch corruption much 
> faster and hopefully pinpoint it more quickly where exactly it happens. 

At least, we could do something like this to catch the breakage
when we (re)pack, to prevent damage from propagating.


diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 73d448b..5d0692a 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -65,6 +65,7 @@ static int no_reuse_delta;
 static int local;
 static int incremental;
 static int allow_ofs_delta;
+static int revalidate_sha1;
 
 static struct object_entry **sorted_by_sha, **sorted_by_type;
 static struct object_entry *objects;
@@ -974,8 +975,31 @@ static void add_preferred_base(unsigned char *sha1)
 	it->pcache.tree_size = size;
 }
 
-static void check_object(struct object_entry *entry)
+static void check_object(struct object_entry *entry, int ith, unsigned *last)
 {
+	if (revalidate_sha1) {
+		unsigned char sha1[20];
+		enum object_type type;
+		unsigned long size;
+		void *buf;
+
+		buf = read_sha1_file(entry->sha1, &type, &size);
+		hash_sha1_file(buf, size, typename(type), sha1);
+		if (hashcmp(sha1, entry->sha1))
+			die("'%s': hash mismatch", sha1_to_hex(entry->sha1));
+		free(buf);
+
+		if (progress) {
+			unsigned percent = ith * 100 / nr_objects;
+			if (percent != *last || progress_update) {
+				fprintf(stderr, "%4u%% (%u/%u) done\r",
+					percent, ith, nr_objects);
+				progress_update = 0;
+				*last = percent;
+			}
+		}
+	}
+
 	if (entry->in_pack && !entry->preferred_base) {
 		struct packed_git *p = entry->in_pack;
 		struct pack_window *w_curs = NULL;
@@ -1082,10 +1106,16 @@ static void get_object_details(void)
 {
 	uint32_t i;
 	struct object_entry *entry;
+	unsigned last_percent = 999;
+
+	if (progress && revalidate_sha1)
+		fprintf(stderr, "Revalidating %u objects.\n", nr_objects);
 
 	prepare_pack_ix();
 	for (i = 0, entry = objects; i < nr_objects; i++, entry++)
-		check_object(entry);
+		check_object(entry, i+1, &last_percent);
+	if (progress && revalidate_sha1)
+		fputc('\n', stderr);
 
 	if (nr_objects == nr_result) {
 		/*
@@ -1629,6 +1659,10 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 			rp_av[1] = "--objects-edge";
 			continue;
 		}
+		if (!strcmp("--revalidate", arg)) {
+			revalidate_sha1 = 1;
+			continue;
+		}
 		usage(pack_usage);
 	}
 

  parent reply	other threads:[~2007-03-20  7:42 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-02-28  4:36 My git repo is broken, how to fix it ? Alexander Litvinov
2007-02-28  4:57 ` Linus Torvalds
2007-02-28 11:54   ` Alexander Litvinov
2007-02-28 16:19     ` Linus Torvalds
2007-02-28 19:12       ` Alex Riesen
2007-03-19 13:32       ` Alexander Litvinov
2007-03-19 15:20         ` Linus Torvalds
     [not found]           ` <200703201013.39169.litvinov2004@gmail.com>
2007-03-20  5:34             ` Linus Torvalds
2007-03-20  6:55               ` Alexander Litvinov
2007-03-20  7:42               ` Junio C Hamano [this message]
2007-03-20 15:23                 ` Nicolas Pitre
     [not found] ` <Pine.LNX.4.64.0703200832150.6730@woody.linux-foundation.org>
     [not found]   ` <Pine.LNX.4.64.0703200836490.6730@woody.linux-foundation.org>
     [not found]     ` <200703210956.50018.litvinov2004@gmail.com>
2007-03-22 15:58       ` Linus Torvalds
2007-03-22 16:34         ` Nicolas Pitre
     [not found]       ` <200703211024.04740.litvinov2004@gmail.com>
2007-03-22 16:17         ` Linus Torvalds
2007-03-22 16:29           ` Linus Torvalds
2007-03-22 16:48             ` Linus Torvalds
2007-03-22 17:01               ` Nicolas Pitre
2007-03-22 17:10                 ` Linus Torvalds
2007-03-22 17:28                   ` Nicolas Pitre
2007-03-22 22:13                   ` Jeff King
2007-03-23  0:25                     ` Linus Torvalds
2007-03-23  0:42                       ` Bill Lear
2007-03-23  0:51                       ` Jeff King
2007-03-22 20:31               ` [PATCH] git-apply: Do not free the wrong buffer when we convert the data for writeout Junio C Hamano
2007-03-22 20:55                 ` Linus Torvalds
2007-03-23  3:55                   ` Alexander Litvinov
2007-03-23  3:40               ` My git repo is broken, how to fix it ? Alexander Litvinov
2007-03-22 17:12             ` Johannes Sixt
2021-06-06 17:27 B
2021-06-06 17:28 B
2021-12-25  8:30 Joseph Mitchell
2021-12-26  0:48 ` Lemuria
2023-05-29 18:57 ross thomas

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7vd5349x97.fsf@assigned-by-dhcp.cox.net \
    --to=junkio@cox.net \
    --cc=git@vger.kernel.org \
    --cc=litvinov2004@gmail.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).