combine git repo historically

combine git repo historically

[bill lam]

I have two git repos, no branches.

repo 1.
  emptyrootcommit -- A ... M 

repo 2.
  emptyrootcommit -- N ... Z

N was evolved from M but the time gap is large, how can I combine them
into one repo

emptyrootcommit -- A ... M -- N ... Z

so that snapshots N .. Z will not be changed.

-- 
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

Re: combine git repo historically

[Johannes Sixt]

bill lam schrieb:
> I have two git repos, no branches.
> 
> repo 1.
>   emptyrootcommit -- A ... M 
> 
> repo 2.
>   emptyrootcommit -- N ... Z
> 
> N was evolved from M but the time gap is large, how can I combine them
> into one repo
> 
> emptyrootcommit -- A ... M -- N ... Z
> 
> so that snapshots N .. Z will not be changed.

$ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
$ git filter-branch --tag-name-filter cat -- --all --not M

i.e. you graft the older history right before the younger history, then
you use git filter-branch to rewrite the parentship of the younger commits.

-- Hannes

Re: combine git repo historically

[Christian Couder]

On Fri, Oct 9, 2009 at 8:02 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> bill lam schrieb:
>> I have two git repos, no branches.
>>
>> repo 1.
>>   emptyrootcommit -- A ... M
>>
>> repo 2.
>>   emptyrootcommit -- N ... Z
>>
>> N was evolved from M but the time gap is large, how can I combine them
>> into one repo
>>
>> emptyrootcommit -- A ... M -- N ... Z
>>
>> so that snapshots N .. Z will not be changed.
>
> $ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
> $ git filter-branch --tag-name-filter cat -- --all --not M
>
> i.e. you graft the older history right before the younger history, then
> you use git filter-branch to rewrite the parentship of the younger commits.

If you cannot create a new history, using "git replace" could be
better than using grafts.
("git replace" is in the "master" branch in the git repository. It
will be in git 1.6.5 that should be released soon.)

Regards,
Christian

Re: combine git repo historically

[bill lam]

On Fri, 09 Oct 2009, Johannes Sixt wrote:
> bill lam schrieb:
> > I have two git repos, no branches.
> > 
> > repo 1.
> >   emptyrootcommit -- A ... M 
> > 
> > repo 2.
> >   emptyrootcommit -- N ... Z
> > 
> > N was evolved from M but the time gap is large, how can I combine them
> > into one repo
> > 
> > emptyrootcommit -- A ... M -- N ... Z
> > 
> > so that snapshots N .. Z will not be changed.
> 
> $ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
> $ git filter-branch --tag-name-filter cat -- --all --not M
> 
> i.e. you graft the older history right before the younger history, then
> you use git filter-branch to rewrite the parentship of the younger commits.
 

Thanks,  graft is new to me. I tested it works but it needs first issue
   git fetch /path/to/repo1
within repo2 to fetch tip M from repo1 into repo2, is it correct?

If I also want to fetch also all objects from repo1, what will the
command to do it.

Christian Couder also mentioned the git-replace command, how to stitch
with it?

-- 
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

Re: combine git repo historically

[Christian Couder]

On Saturday 10 October 2009, bill lam wrote:
> On Fri, 09 Oct 2009, Johannes Sixt wrote:
> > bill lam schrieb:
> > > I have two git repos, no branches.
> > >
> > > repo 1.
> > >   emptyrootcommit -- A ... M
> > >
> > > repo 2.
> > >   emptyrootcommit -- N ... Z
> > >
> > > N was evolved from M but the time gap is large, how can I combine
> > > them into one repo
> > >
> > > emptyrootcommit -- A ... M -- N ... Z
> > >
> > > so that snapshots N .. Z will not be changed.
> >
> > $ echo $(git rev-parse N) $(git rev-parse M) >> .git/info/grafts
> > $ git filter-branch --tag-name-filter cat -- --all --not M
> >
> > i.e. you graft the older history right before the younger history, then
> > you use git filter-branch to rewrite the parentship of the younger
> > commits.
>
> Thanks,  graft is new to me. I tested it works but it needs first issue
>    git fetch /path/to/repo1
> within repo2 to fetch tip M from repo1 into repo2, is it correct?
>
> If I also want to fetch also all objects from repo1, what will the
> command to do it.

Maybe you should do something like:

$ git remote add repo1 /path/to/repo1
$ git fetch repo1

The fetch command should output something like:

remote: Counting objects: 423, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 363 (delta 276), reused 298 (delta 213)
Receiving objects: 100% (363/363), 136.65 KiB, done.
Resolving deltas: 100% (276/276), completed with 29 local objects.
From /path/to/repo1
 * [new branch]      branch1       -> repo1/branch1
 * [new branch]      branch2       -> repo1/branch2

So the branches in repo1 will be available in repo2 as remote/repo1/branch1, 
remote/repo1/branch2, ...

> Christian Couder also mentioned the git-replace command, how to stitch
> with it?

First you need to create a commit that will replace commit N, let's call it 
N' and after that you can use "git replace N N'". So the complex part is to 
create N'.

You want N' to have the same content as N but to have M as parent. So you 
could do something like the following:

(We suppose that commits A to M are in branch1 and that you are in the root 
directory of your repo2 working directory.)

$ git checkout -b repo1-branch1 remote/repo1/branch1
$ git checkout N -- .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit -a

And then use the commit message from commit N, and maybe also say in the 
commit message that it is replacement commit made to link repo1 with repo2 
or something like that.

At this step you have created N' and you should make sure that it is exactly 
what you want. It should point to the same tree as N, it should have M as 
parent, ...

If everything is ok then you can use:

$ git replace N HEAD

And you should be done.

Regards,
Christian.

Re: combine git repo historically

[bill lam]

On Sun, 11 Oct 2009, Christian Couder wrote:
> $ git checkout -b repo1-branch1 remote/repo1/branch1
> $ git checkout N -- .
> ...
> If everything is ok then you can use:
> 
> $ git replace N HEAD

Thanks for detail instruction, I tested it ok except that when trying
to checkout it reported an error

  $ git checkout -b repo1-branch1 remote/oldjproject/master
  fatal: git checkout: updating paths is incompatible with switching branches. 
  Did you intend to checkout 'remote/oldjproject/master' which can not
  be resolved as commit?

but it ran ok by omitting that 'remote/'
$ git checkout -b repo1-branch1 oldjproject/master

Does it need to purge the file system tree before
  git checkout N -- .

so that there will be no artifact leaved by M?

I found that it is necessary to do a 
   git reset --hard (original HEAD)

to complete the story. Is it correct?

git version 1.6.5.rc3.35.g3340

-- 
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

Re: combine git repo historically

[Jakub Narebski]

Christian Couder <chriscool@tuxfamily.org> writes:

> On Saturday 10 October 2009, bill lam wrote:

> > Christian Couder also mentioned the git-replace command, how to stitch
> > with it?
> 
> First you need to create a commit that will replace commit N, let's call it 
> N' and after that you can use "git replace N N'". So the complex part is to 
> create N'.
> 
> You want N' to have the same content as N but to have M as parent. So you 
> could do something like the following:
> 
> (We suppose that commits A to M are in branch1 and that you are in the root 
> directory of your repo2 working directory.)
> 
> $ git checkout -b repo1-branch1 remote/repo1/branch1
> $ git checkout N -- .
> $ export GIT_AUTHOR_NAME=<author name of commit N>
> $ export GIT_AUTHOR_EMAIL=<author email of commit N>
> $ export GIT_AUTHOR_DATE=<date of commit N>
> $ git commit -a
> 
> And then use the commit message from commit N, and maybe also say in the 
> commit message that it is replacement commit made to link repo1 with repo2 
> or something like that.

I think simpler solution would be to use plumbing for that.  First
save commit to be replaced in a (text) file:

 $ git cat-file commit N > COMMIT_N

If you want to edit only commit message and perhaps parentage (like I
think in this case), you need to simply edit COMMIT_N file, and modify
(or add) 'parent' header(s), which is between 'tree' and 'author'
header.

Then put modified commit in repository object database

 $ git hash-object -t commit -w COMMIT_N
 
> At this step you have created N' and you should make sure that it is exactly 
> what you want. It should point to the same tree as N, it should have M as 
> parent, ...
> 
> If everything is ok then you can use:
> 
> $ git replace N HEAD

And then do

 $ git replace N NEW_N

where N is _SHA-1_ of original comit ("git rev-parse --verify N^0"),
and NEW_N is SHA-1 of replacement commit, as written by git-hash-object.
 
> And you should be done.

But I haven't tested this. YMMV.

See also my answer here: 
http://stackoverflow.com/questions/1220557/how-do-i-prepend-history-to-a-git-repo/1547490#1547490

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: combine git repo historically

[Andreas Schwab]

Christian Couder <chriscool@tuxfamily.org> writes:

> You want N' to have the same content as N but to have M as parent. So you 
> could do something like the following:
>
> (We suppose that commits A to M are in branch1 and that you are in the root 
> directory of your repo2 working directory.)
>
> $ git checkout -b repo1-branch1 remote/repo1/branch1
> $ git checkout N -- .
> $ export GIT_AUTHOR_NAME=<author name of commit N>
> $ export GIT_AUTHOR_EMAIL=<author email of commit N>
> $ export GIT_AUTHOR_DATE=<date of commit N>
> $ git commit -a

Isn't that what git cherry-pick does?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: combine git repo historically

[Christian Couder]

On Sunday 11 October 2009, bill lam wrote:
> On Sun, 11 Oct 2009, Christian Couder wrote:
> > $ git checkout -b repo1-branch1 remote/repo1/branch1
> > $ git checkout N -- .
> > ...
> > If everything is ok then you can use:
> >
> > $ git replace N HEAD
>
> Thanks for detail instruction, I tested it ok except that when trying
> to checkout it reported an error
>
>   $ git checkout -b repo1-branch1 remote/oldjproject/master
>   fatal: git checkout: updating paths is incompatible with switching
> branches. Did you intend to checkout 'remote/oldjproject/master' which
> can not be resolved as commit?
>
> but it ran ok by omitting that 'remote/'
> $ git checkout -b repo1-branch1 oldjproject/master
>
> Does it need to purge the file system tree before
>   git checkout N -- .
>
> so that there will be no artifact leaved by M?

Yeah, perhaps, I forgot that there could be some files added and other files 
removed between M and N.

If you remove everything (except the .git directory of course), then doing:

$ git checkout N -- .
$ git add .
$ export GIT_AUTHOR_NAME=<author name of commit N>
$ export GIT_AUTHOR_EMAIL=<author email of commit N>
$ export GIT_AUTHOR_DATE=<date of commit N>
$ git commit

should do the trick if I am not missing something which may very well be the 
case.

So perhaps it is easier to use Jakub's suggestion instead.

> I found that it is necessary to do a
>    git reset --hard (original HEAD)
>
> to complete the story. Is it correct?

Yes, you should go back to your orginal branch at the end.

Regards,
Christian.

Re: combine git repo historically

[Johannes Sixt]

On Sonntag, 11. Oktober 2009, Jakub Narebski wrote:
> Christian Couder <chriscool@tuxfamily.org> writes:
> > $ git checkout -b repo1-branch1 remote/repo1/branch1
> > $ git checkout N -- .
> > $ export GIT_AUTHOR_NAME=<author name of commit N>
> > $ export GIT_AUTHOR_EMAIL=<author email of commit N>
> > $ export GIT_AUTHOR_DATE=<date of commit N>
> > $ git commit -a

This would not preserver the committer, but in particular the committer date 
is rather important for history traversal. You really don't want to change 
it. I used a much more elaborate pipleine involving git-commit-tree because I 
did not think about this simple procedure:

>  $ git cat-file commit N > COMMIT_N
>[ $ edit COMMIT_N  ]
>  $ git hash-object -t commit -w COMMIT_N

Thanks for this tip, Jakub.

Christian, one thing that worries me is that 'git cat-file commit foo' returns 
the replacement (bar) instead of the original (foo). Is it intended? There is 
no way to retrieve the original commit using plumbing unless the replacement 
is removed. Am I right?

-- Hannes

Re: combine git repo historically

[Christian Couder]

On Sunday 11 October 2009, Johannes Sixt wrote:
> On Sonntag, 11. Oktober 2009, Jakub Narebski wrote:
> > Christian Couder <chriscool@tuxfamily.org> writes:
> > > $ git checkout -b repo1-branch1 remote/repo1/branch1
> > > $ git checkout N -- .
> > > $ export GIT_AUTHOR_NAME=<author name of commit N>
> > > $ export GIT_AUTHOR_EMAIL=<author email of commit N>
> > > $ export GIT_AUTHOR_DATE=<date of commit N>
> > > $ git commit -a
>
> This would not preserver the committer, but in particular the committer
> date is rather important for history traversal. You really don't want to
> change it. I used a much more elaborate pipleine involving
> git-commit-tree because I
>
> did not think about this simple procedure:
> >  $ git cat-file commit N > COMMIT_N
> >[ $ edit COMMIT_N  ]
> >  $ git hash-object -t commit -w COMMIT_N
>
> Thanks for this tip, Jakub.
>
> Christian, one thing that worries me is that 'git cat-file commit foo'
> returns the replacement (bar) instead of the original (foo). Is it
> intended? 

Yes it is intended. "git replace" was designed so that the original objects 
are not replaced only when reachability traversal (prune, pack transfer and 
fsck) wants to read the commit. See:

http://article.gmane.org/gmane.comp.version-control.git/104776/

> There is no way to retrieve the original commit using plumbing 
> unless the replacement is removed. Am I right?

I think you are right, but a flag could be easily added to some commands to 
retreive original objects. Or you can remove the replacement, retrieve the 
object and then put back the replacement.

Best regards,
Christian.

Re: combine git repo historically

[Jakub Narebski]

Dnia niedziela 11. października 2009 15:43, Christian Couder napisał:
> On Sunday 11 October 2009, Johannes Sixt wrote:

> > There is no way to retrieve the original commit using plumbing 
> > unless the replacement is removed. Am I right?
> 
> I think you are right, but a flag could be easily added to some commands to 
> retreive original objects. Or you can remove the replacement, retrieve the 
> object and then put back the replacement.

Wouldn't it be better for this option to be option for git wrapper,
i.e.

 $ git --no-replace cat-file -p N

and not

 $ git cat-file --no-replace -p N

-- 
Jakub Narebski
Poland