[PATCH] Dissociating a repository from its alternates

[PATCH] Dissociating a repository from its alternates

[Junio C Hamano]

People often start out with "clone -l -s" (or --reference) and
later wish to dissociate the repository from its alternates.
The new option -A to "git repack" lets you do so.

What it does:

 - git-pack-objects gets a new option --ignore-alternate-pack.
   The command used in git-repack is told to pack only unpacked
   objects in general, but also is told to pretend objects in
   packs in the local repository are unpacked (this is necessary
   to keep historical huge packs marked with .keep files without
   repacking).  This new option tells it to also pretend that
   objects from packs in alternate repositories are unpacked, to
   make them subject to packing.

 - code to parse revision traversal options gets an additional
   bit, ignore_alternate_pack.

 - has_sha1_pack() function gets an additional argument for that
   bit.  Existing callers pass 0 to it (they continue to behave
   as before).

 - git-repack obviously understands the new -A option.  When it
   is given, it passes --ignore-alternate-pack option to
   git-pack-objects to include objects from borrowed packs in
   the result, and $GIT_OBJECT_DIRECTORY/info/alternates file is
   removed at the end.

The patch was tested very lightly, and I haven't verified the
interactions between various options git-repack takes, and
options it passes to git-pack-objects (especially, I think
interaction with --local and --incremental might be dubious).

Because a bug in git-repack has real risk of corrupting
repository in a bad way, somebody else needs to double check
this patch, if we were to do this.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---
 builtin-count-objects.c |    2 +-
 builtin-pack-objects.c  |    1 +
 builtin-prune-packed.c  |    2 +-
 cache.h                 |    2 +-
 diff.c                  |    2 +-
 git-repack.sh           |   30 ++++++++++++++++++++++--------
 revision.c              |   12 +++++++++---
 revision.h              |    1 +
 sha1_file.c             |   16 +++++++++-------
 9 files changed, 46 insertions(+), 22 deletions(-)

diff --git a/builtin-count-objects.c b/builtin-count-objects.c
index f5b22bb..10c8860 100644
--- a/builtin-count-objects.c
+++ b/builtin-count-objects.c
@@ -62,7 +62,7 @@ static void count_objects(DIR *d, char *path, int len, int verbose,
 		hex[40] = 0;
 		if (get_sha1_hex(hex, sha1))
 			die("internal error");
-		if (has_sha1_pack(sha1, NULL))
+		if (has_sha1_pack(sha1, NULL, 0))
 			(*packed_loose)++;
 	}
 }
diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index 3824ee3..0f5d72c 100644
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1623,6 +1623,7 @@ int cmd_pack_objects(int argc, const char **argv, const char *prefix)
 		}
 		if (!strcmp("--unpacked", arg) ||
 		    !strncmp("--unpacked=", arg, 11) ||
+		    !strcmp("--ignore-alternate-pack", arg) ||
 		    !strcmp("--reflog", arg) ||
 		    !strcmp("--all", arg)) {
 			use_internal_rev_list = 1;
diff --git a/builtin-prune-packed.c b/builtin-prune-packed.c
index 9777300..eb55c58 100644
--- a/builtin-prune-packed.c
+++ b/builtin-prune-packed.c
@@ -20,7 +20,7 @@ static void prune_dir(int i, DIR *dir, char *pathname, int len, int opts)
 		memcpy(hex+2, de->d_name, 38);
 		if (get_sha1_hex(hex, sha1))
 			continue;
-		if (!has_sha1_pack(sha1, NULL))
+		if (!has_sha1_pack(sha1, NULL, 0))
 			continue;
 		memcpy(pathname + len, de->d_name, 38);
 		if (opts & DRY_RUN)
diff --git a/cache.h b/cache.h
index c62b0b0..0c24318 100644
--- a/cache.h
+++ b/cache.h
@@ -266,7 +266,7 @@ extern int write_sha1_from_fd(const unsigned char *sha1, int fd, char *buffer,
 extern int write_sha1_to_fd(int fd, const unsigned char *sha1);
 extern int move_temp_to_file(const char *tmpfile, const char *filename);
 
-extern int has_sha1_pack(const unsigned char *sha1, const char **ignore);
+extern int has_sha1_pack(const unsigned char *sha1, const char **ignore, int ignore_alternates);
 extern int has_sha1_file(const unsigned char *sha1);
 extern void *map_sha1_file(const unsigned char *sha1, unsigned long *);
 extern int legacy_loose_object(unsigned char *);
diff --git a/diff.c b/diff.c
index 12c8b2b..ee46960 100644
--- a/diff.c
+++ b/diff.c
@@ -1254,7 +1254,7 @@ static int reuse_worktree_file(const char *name, const unsigned char *sha1, int
 	 * objects however would tend to be slower as they need
 	 * to be individually opened and inflated.
 	 */
-	if (!FAST_WORKING_DIRECTORY && !want_file && has_sha1_pack(sha1, NULL))
+	if (!FAST_WORKING_DIRECTORY && !want_file && has_sha1_pack(sha1, NULL, 0))
 		return 0;
 
 	len = strlen(name);
diff --git a/git-repack.sh b/git-repack.sh
index ddfa8b4..774286e 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -3,17 +3,18 @@
 # Copyright (c) 2005 Linus Torvalds
 #
 
-USAGE='[-a] [-d] [-f] [-l] [-n] [-q] [--window=N] [--depth=N]'
+USAGE='[-a] [-A] [-d] [-f] [-l] [-n] [-q] [--window=N] [--depth=N]'
 SUBDIRECTORY_OK='Yes'
 . git-sh-setup
 
-no_update_info= all_into_one= remove_redundant=
+no_update_info= all_into_one= remove_redundant= nuke_alternates=
 local= quiet= no_reuse_delta= extra=
 while case "$#" in 0) break ;; esac
 do
 	case "$1" in
 	-n)	no_update_info=t ;;
 	-a)	all_into_one=t ;;
+	-A)	nuke_alternates=t ;;
 	-d)	remove_redundant=t ;;
 	-q)	quiet=-q ;;
 	-f)	no_reuse_delta=--no-reuse-delta ;;
@@ -28,8 +29,7 @@ done
 # Later we will default repack.UseDeltaBaseOffset to true
 default_dbo=false
 
-case "`git config --bool repack.usedeltabaseoffset ||
-       echo $default_dbo`" in
+case "`git config --bool repack.usedeltabaseoffset || echo $default_dbo`" in
 true)
 	extra="$extra --delta-base-offset" ;;
 esac
@@ -40,11 +40,14 @@ rm -f "$PACKTMP"-*
 trap 'rm -f "$PACKTMP"-*' 0 1 2 3 15
 
 # There will be more repacking strategies to come...
-case ",$all_into_one," in
-,,)
+case ",$all_into_one,$nuke_alternates," in
+,,,)
 	args='--unpacked --incremental'
 	;;
-,t,)
+,,t,)
+	args='--incremental --ignore-alternate-pack'
+	;;
+,t,*)
 	if [ -d "$PACKDIR" ]; then
 		for e in `cd "$PACKDIR" && find . -type f -name '*.pack' \
 			| sed -e 's/^\.\///' -e 's/\.pack$//'`
@@ -57,7 +60,12 @@ case ",$all_into_one," in
 			fi
 		done
 	fi
-	[ -z "$args" ] && args='--unpacked --incremental'
+	case "$nuke_alternates" in
+	'')
+		[ -z "$args" ] && args='--unpacked --incremental' ;;
+	t)
+		args="$args --ignore-alternate-pack" ;;
+	esac
 	;;
 esac
 
@@ -113,6 +121,12 @@ then
 	git-prune-packed $quiet
 fi
 
+case "$nuke_alternates" in
+t)
+	rm -f "$GIT_OBJECT_DIRECTORY/info/alternates"
+	;;
+esac
+
 case "$no_update_info" in
 t) : ;;
 *) git-update-server-info ;;
diff --git a/revision.c b/revision.c
index 5b1794b..b9e33c3 100644
--- a/revision.c
+++ b/revision.c
@@ -949,6 +949,10 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, const ch
 				add_ignore_packed(revs, arg+11);
 				continue;
 			}
+			if (!strcmp(arg, "--ignore-alternate-pack")) {
+				revs->ignore_alternate_pack = 1;
+				continue;
+			}
 			if (!strcmp(arg, "-r")) {
 				revs->diff = 1;
 				revs->diffopt.recursive = 1;
@@ -1240,9 +1244,11 @@ static struct commit *get_revision_1(struct rev_info *revs)
 		if (commit->object.flags & SHOWN)
 			continue;
 
-		if (revs->unpacked && has_sha1_pack(commit->object.sha1,
-						    revs->ignore_packed))
-		    continue;
+		if ((revs->unpacked || revs->ignore_alternate_pack) &&
+		    has_sha1_pack(commit->object.sha1,
+				  revs->ignore_packed,
+				  revs->ignore_alternate_pack))
+			continue;
 
 		/* We want to show boundary commits only when their
 		 * children are shown.  When path-limiter is in effect,
diff --git a/revision.h b/revision.h
index 5fec184..ca6affd 100644
--- a/revision.h
+++ b/revision.h
@@ -40,6 +40,7 @@ struct rev_info {
 			edge_hint:1,
 			limited:1,
 			unpacked:1, /* see also ignore_packed below */
+			ignore_alternate_pack:1,
 			boundary:1,
 			left_right:1,
 			parents:1,
diff --git a/sha1_file.c b/sha1_file.c
index b83f59f..2d8c0f2 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -1403,7 +1403,7 @@ static int matches_pack_name(struct packed_git *p, const char *ig)
 	return 1;
 }
 
-static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, const char **ignore_packed)
+static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, const char **ignore_packed, int ignore_alternates)
 {
 	struct packed_git *p;
 	unsigned long offset;
@@ -1419,6 +1419,8 @@ static int find_pack_entry(const unsigned char *sha1, struct pack_entry *e, cons
 			if (*ig)
 				continue;
 		}
+		if (ignore_alternates && !p->pack_local)
+			continue;
 		offset = find_pack_entry_one(sha1, p);
 		if (offset) {
 			/*
@@ -1485,9 +1487,9 @@ int sha1_object_info(const unsigned char *sha1, char *type, unsigned long *sizep
 {
 	struct pack_entry e;
 
-	if (!find_pack_entry(sha1, &e, NULL)) {
+	if (!find_pack_entry(sha1, &e, NULL, 0)) {
 		reprepare_packed_git();
-		if (!find_pack_entry(sha1, &e, NULL))
+		if (!find_pack_entry(sha1, &e, NULL, 0))
 			return sha1_loose_object_info(sha1, type, sizep);
 	}
 	return packed_object_info(e.p, e.offset, type, sizep);
@@ -1497,7 +1499,7 @@ static void *read_packed_sha1(const unsigned char *sha1, char *type, unsigned lo
 {
 	struct pack_entry e;
 
-	if (!find_pack_entry(sha1, &e, NULL))
+	if (!find_pack_entry(sha1, &e, NULL, 0))
 		return NULL;
 	else
 		return unpack_entry(e.p, e.offset, type, size);
@@ -2004,10 +2006,10 @@ int has_pack_file(const unsigned char *sha1)
 	return 1;
 }
 
-int has_sha1_pack(const unsigned char *sha1, const char **ignore_packed)
+int has_sha1_pack(const unsigned char *sha1, const char **ignore_packed, int ignore_alternates)
 {
 	struct pack_entry e;
-	return find_pack_entry(sha1, &e, ignore_packed);
+	return find_pack_entry(sha1, &e, ignore_packed, ignore_alternates);
 }
 
 int has_sha1_file(const unsigned char *sha1)
@@ -2015,7 +2017,7 @@ int has_sha1_file(const unsigned char *sha1)
 	struct stat st;
 	struct pack_entry e;
 
-	if (find_pack_entry(sha1, &e, NULL))
+	if (find_pack_entry(sha1, &e, NULL, 0))
 		return 1;
 	return find_sha1_file(sha1, &st) ? 1 : 0;
 }

[PATCH +1] Dissociating a repository from its alternates

[Junio C Hamano]

On top of the previous patch.  This is needed to:

 - reject "repack -A -l", which does not make any sense; you
   want to repack the objects you borrow from the alternates.

 - make "repack -A" without "-a" to work.

These operations corrupted the repository with the previous
patch alone.

Signed-off-by: Junio C Hamano <junkio@cox.net>
---

 git-repack.sh |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/git-repack.sh b/git-repack.sh
index 774286e..66b5039 100755
--- a/git-repack.sh
+++ b/git-repack.sh
@@ -25,6 +25,10 @@ do
 	esac
 	shift
 done
+if test "$nuke_alternates,$local" = "t,--local"
+then
+	die "-A and -l are incompatible"
+fi
 
 # Later we will default repack.UseDeltaBaseOffset to true
 default_dbo=false
@@ -45,7 +49,7 @@ case ",$all_into_one,$nuke_alternates," in
 	args='--unpacked --incremental'
 	;;
 ,,t,)
-	args='--incremental --ignore-alternate-pack'
+	args='--ignore-alternate-pack'
 	;;
 ,t,*)
 	if [ -d "$PACKDIR" ]; then

Re: [PATCH] Dissociating a repository from its alternates

[Johannes Schindelin]

Hi,

On Wed, 14 Feb 2007, Junio C Hamano wrote:

> People often start out with "clone -l -s" (or --reference) and
> later wish to dissociate the repository from its alternates.

Why not make a really dumb script which just _copies_ (or maybe 
optionally hard link) the objects from the alternate repo, and then kills 
the alternatives file? That should be easier. You always can repack after 
that.

Ciao,
Dscho

Re: [PATCH] Dissociating a repository from its alternates

[James Cloos]

[ARGH; hit r rather than f; cc-ing the list. -JimC]

Wierd.

I was /just/ about to inquire on what the best method was for disassociating.

And then I read this thread.

Effective mind reading!  :)

WRT Johannes's followup, I presume it will re-use the alternate packs,
such that the computation is minimal?

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: [PATCH] Dissociating a repository from its alternates

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> On Wed, 14 Feb 2007, Junio C Hamano wrote:
>
>> People often start out with "clone -l -s" (or --reference) and
>> later wish to dissociate the repository from its alternates.
>
> Why not make a really dumb script which just _copies_ (or maybe 
> optionally hard link) the objects from the alternate repo, and then kills 
> the alternatives file? That should be easier. You always can repack after 
> that.

I think that is much safer.

Re: [PATCH] Dissociating a repository from its alternates

[Michael Hendricks]

On Fri, Feb 16, 2007 at 12:13:01PM -0800, Junio C Hamano wrote:
> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Wed, 14 Feb 2007, Junio C Hamano wrote:
> >
> >> People often start out with "clone -l -s" (or --reference) and
> >> later wish to dissociate the repository from its alternates.
> >
> > Why not make a really dumb script which just _copies_ (or maybe 
> > optionally hard link) the objects from the alternate repo, and then kills 
> > the alternatives file? That should be easier. You always can repack after 
> > that.
> 
> I think that is much safer.

Was such a script ever incorporated into Git?

-- 
Michael

Re: [PATCH] Dissociating a repository from its alternates

[Johannes Schindelin]

Hi,

On Mon, 22 Oct 2007, Michael Hendricks wrote:

> On Fri, Feb 16, 2007 at 12:13:01PM -0800, Junio C Hamano wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > 
> > > On Wed, 14 Feb 2007, Junio C Hamano wrote:
> > >
> > >> People often start out with "clone -l -s" (or --reference) and
> > >> later wish to dissociate the repository from its alternates.
> > >
> > > Why not make a really dumb script which just _copies_ (or maybe 
> > > optionally hard link) the objects from the alternate repo, and then kills 
> > > the alternatives file? That should be easier. You always can repack after 
> > > that.
> > 
> > I think that is much safer.
> 
> Was such a script ever incorporated into Git?

Not that I know of, but "git repack -a && rm .git/objects/info/alternates" 
should do what you want.  You can even make a script of it, add some 
documentation and a test case, and earn git fame by posting a patch ;-)

Ciao,
Dscho

Re: [PATCH] Dissociating a repository from its alternates

[Michael Hendricks]

On Mon, Oct 22, 2007 at 08:05:58PM +0100, Johannes Schindelin wrote:
> On Mon, 22 Oct 2007, Michael Hendricks wrote:
> > Was such a script ever incorporated into Git?
> 
> Not that I know of, but "git repack -a && rm
> .git/objects/info/alternates" should do what you want.  You can even
> make a script of it, add some documentation and a test case, and earn
> git fame by posting a patch ;-)

With 1.5.3.4, it doesn't appear to work:

 $ git clone -s git git2
 Initialized empty Git repository in /Users/michael/src/git2/.git/

 $ cd git2

 $ git repack -a && rm .git/objects/info/alternates
 Generating pack...
 Done counting 0 objects.
 Nothing new to pack.

 $ git status
 # On branch master
 fatal: bad object HEAD

-- 
Michael

Re: [PATCH] Dissociating a repository from its alternates

[Johannes Schindelin]

Hi,

On Mon, 22 Oct 2007, Michael Hendricks wrote:

> On Mon, Oct 22, 2007 at 08:05:58PM +0100, Johannes Schindelin wrote:
> > On Mon, 22 Oct 2007, Michael Hendricks wrote:
> > > Was such a script ever incorporated into Git?
> > 
> > Not that I know of, but "git repack -a && rm
> > .git/objects/info/alternates" should do what you want.  You can even
> > make a script of it, add some documentation and a test case, and earn
> > git fame by posting a patch ;-)
> 
> With 1.5.3.4, it doesn't appear to work:
> 
>  $ git clone -s git git2
>  Initialized empty Git repository in /Users/michael/src/git2/.git/
> 
>  $ cd git2
> 
>  $ git repack -a && rm .git/objects/info/alternates
>  Generating pack...
>  Done counting 0 objects.
>  Nothing new to pack.
> 
>  $ git status
>  # On branch master
>  fatal: bad object HEAD

Indeed.  Seems that somewhere along the line, repack learnt to imply "-l" 
by "-a" or something.  Try this instead of "git repack -a":

	git rev-parse --all |
		git pack-objects --revs .git/objects/pack/pack

(And make sure you read the relevant parts of the documentation to 
understand what you're doing ;-)

Hth,
Dscho