Questions about the new

Questions about the new

[Sergio]

Hi,

I read from the release notes of git 1.6.5 about the new "replace" mechanism.
It is presented as "a replacement of the "grafts" mechanism, with the added
advantage that it can be transferred across repositories."

Since there is still little information about it, I would like to ask the
following:

1) Grafts and replace entries seem to operate on different aspects of the
history graph. Grafts operate on arcs and replace on nodes. 
As such, replace entries seem less general to me. 
Apparently, to simulate a graft with replace entries, you need to introduce
extra commit objects. For instance, if object B has no parents, to pretend that
it derives from some A, one needs to create an object B' equivalent to B but
for the parents and then replace B by B', is this right? Conversely, I guess
you can always simulate a replace entry with the graft mechanism, without the
need to add any extra commit object. Am I overlooking something? 

2) Is it currently possible to use a replace entry to replace a commit object
with nothing? Namely if B has A as its sole parent, is it possible to have a
replace entry such as A-sha1 becomes null, to pretend that B is a hierarchy
root?  

3) If I remember correctly, there was a reason why grafts were not considered
suitable for transferring across repos. Can someone remind me about it? How
does the replace mechanism address this issue?

Thanks,

Sergio

Re: Questions about the new

[David Kågedal]

Sergio <sergio.callegari@gmail.com> writes:

> Hi,
>
> I read from the release notes of git 1.6.5 about the new "replace" mechanism.
> It is presented as "a replacement of the "grafts" mechanism, with the added
> advantage that it can be transferred across repositories."

Grafts allow you to change a little part of commit objects (the parent
relationship). Replacements allow (and require) you to change the whole
commit object. So obviously, anything you can do with grafts can be done
with replacements.

If you have your A-B-x-x-x history and want to remove A from the
history, you replace B with a commit object that is identical, except
for the missing parent link to A.

Replacements allow you do to other kinds of changes as well (comments,
authorshiips) and changes to other object(?), so it is a much more
general mechanism.

But I missed the discussion about it, so I'm not sure why it was felt it
was needed, or why the grafts mechanism wasn't enough.

-- 
David Kågedal

Re: Questions about the new

[Johannes Sixt]

Sergio schrieb:
> 1) Grafts and replace entries seem to operate on different aspects of the
> history graph. Grafts operate on arcs and replace on nodes. 

Correct, but see below about tree and blob objects.

> As such, replace entries seem less general to me. 

With grafts you can only change parenthood; with replace entries you can
change parenthood *and* all other aspects of a commit (message, author,
committer, dates).

Hence, replace entries are more general than grafts.

> Apparently, to simulate a graft with replace entries, you need to introduce
> extra commit objects. For instance, if object B has no parents, to pretend that
> it derives from some A, one needs to create an object B' equivalent to B but
> for the parents and then replace B by B', is this right?

Yes. Use git-cat-file + edit + git-hash-object as explained in this
message just the other day:
http://thread.gmane.org/gmane.comp.version-control.git/129727/focus=129907

> Conversely, I guess
> you can always simulate a replace entry with the graft mechanism, without the
> need to add any extra commit object. Am I overlooking something? 

You cannot; see above. You can even replace tree objects and blob objects
using replace entries, IIUC, but you cannot do that with grafts.

> 2) Is it currently possible to use a replace entry to replace a commit object
> with nothing? Namely if B has A as its sole parent, is it possible to have a
> replace entry such as A-sha1 becomes null, to pretend that B is a hierarchy
> root?  

Sure. Just make a commit object that does not have parents.

> 3) If I remember correctly, there was a reason why grafts were not considered
> suitable for transferring across repos. Can someone remind me about it? How
> does the replace mechanism address this issue?

The problem with grafts was that, for example, git-pack-objects obeyed the
graft, and could create a broken repository by removing grafted-away
objects. And since git-fsck also obeyed the graft, it did not notice the
breakage.

OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck
never obey replace entries in the history. But they do keep track of them
(and the history that they reference) because they are referenced from the
refs/replace namespace.

-- Hannes

Re: Questions about the new

[Sergio Callegari]

Thanks Johannes for all the detailed explanations

Johannes Sixt <j.sixt <at> viscovery.net> writes:

 >
 > Sergio schrieb:
 > > 1) Grafts and replace entries seem to operate on different aspects 
of the
 > > history graph. Grafts operate on arcs and replace on nodes.
 >
 > Correct, but see below about tree and blob objects.

OK, the replace mechanism also can replace a blob object or a tree.
My focus was on commit objects only.

 >
 > > As such, replace entries seem less general to me.
 >
 > With grafts you can only change parenthood; with replace entries you can
 > change parenthood *and* all other aspects of a commit (message, author,
 > committer, dates).
 >
 > Hence, replace entries are more general than grafts.

Limiting the discussion to commit objects, I think there are two 
possible scenarios.

1) You create new commits objects as needed
2) You do not.

If you follow 1), I believe grafts and replace entries have exactly the same
flexibility.

If I happen not to like commit B in A---B---C and I want A---B'---C 
where B' has
completely different aspects from B I can either replace B by B' or 
graft away
B, pretending that the parent of A is B

But there are many things that can be done with grafts merely adding a graft
(e.g. cutting away a part of history, joining history),  that cannot be done
with replace entries without creating new commits objects.

I was asking because I was wandering whether replace entries were first 
or later
meant to make grafts deprecated. I hope not, because for a few things 
working on
arcs seems still nice.

 > > Apparently, to simulate a graft with replace entries, you need to 
introduce
 > > extra commit objects. For instance, if object B has no parents, to 
pretend >
 > Yes. Use git-cat-file + edit + git-hash-object as explained in this
 > message just the other day:
 > 
http://thread.gmane.org/gmane.comp.version-control.git/129727/focus=129907

Thanks, good pointer. I missed this!

 > > Conversely, I guess
 > > you can always simulate a replace entry with the graft mechanism, 
without the
 > > need to add any extra commit object. Am I overlooking something?
 >
 > You cannot; see above.

Well, I meant for what regards commit objects only.

If I want to replace some commit X by some commit X' I merely need to 
modify the
parent information of all the commits that are child of X so that they 
pretend
to be child of X', or am I missing something?

 > You can even replace tree objects and blob objects
 > using replace entries, IIUC, but you cannot do that with grafts.

Definitely right!
 
 > > 2) Is it currently possible to use a replace entry to replace a 
commit object
 > > with nothing? Namely if B has A as its sole parent, is it possible 
to have a
 > > replace entry such as A-sha1 becomes null, to pretend that B is a 
hierarchy
 > > root? 
 >
 > Sure. Just make a commit object that does not have parents.

OK, you need to create a new commit object. At the beginning for some 
reason I
thought that you could replace an object
with "nothing" or 00000000000000000000000000000000000000000000

 > > 3) If I remember correctly, there was a reason why grafts were not 
considered
 > > suitable for transferring across repos. Can someone remind me about 
it? How
 > > does the replace mechanism address this issue?
 >
 > The problem with grafts was that, for example, git-pack-objects 
obeyed the
 > graft, and could create a broken repository by removing grafted-away
 > objects. And since git-fsck also obeyed the graft, it did not notice the
 > breakage.
 >
 > OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck
 > never obey replace entries in the history. But they do keep track of them
 > (and the history that they reference) because they are referenced 
from the
 > refs/replace namespace.
 >

Thanks for the explanation. Can this be made possible for grafts too? 
Wouldn't
it be a matter of having history walkers never obey grafts but keep track of
them (i.e. of the history of the parenthood they reference)?

Like we have "annotated" or heavyweight tags living as objects in the 
database,
would it be possible or make sense to have annotated grafts or replace 
entries,
so that one can express why, by whom and when history was changed?

Re: Questions about the new

[Dmitry Potapov]

On Mon, Oct 12, 2009 at 02:19:11PM +0200, Johannes Sixt wrote:
> 
> With grafts you can only change parenthood; with replace entries you can
> change parenthood *and* all other aspects of a commit (message, author,
> committer, dates).

Actually, you can. I have written a script that did exactly this. It
required to modify parents to point to the new commit. The tricky part
was that modification could be on top of other modifications, but I was
able to handle this case too. Yet, my script was so hackish that I have
never dared to share it with someone (and I used it only couple times
during CVS to Git conversion).

> 
> Hence, replace entries are more general than grafts.

I think both mechanism are theoretically equivalent, but with grafts,
it was rather difficult to replace objects (but not impossible!).

> The problem with grafts was that, for example, git-pack-objects obeyed the
> graft, and could create a broken repository by removing grafted-away
> objects. And since git-fsck also obeyed the graft, it did not notice the
> breakage.

Moreover, grafted-away objects could be removed by the garbage collector...


Dmitry

Re: Questions about the new

[Junio C Hamano]

Sergio Callegari <sergio.callegari@gmail.com> writes:

> If I want to replace some commit X by some commit X' I merely need to
> modify the
> parent information of all the commits that are child of X so that they
> pretend
> to be child of X', or am I missing something?

You need to find all the commits that are child of X in the first place.
What should happen if your colleague has such a commit in his repository
(which you haven't fetched from yet), you enumerated all children of X
known to you in your graft file and then you fetch from him?  You need to
enumerate all children of X again to keep the graft file up to date.

> Thanks for the explanation. Can this be made possible for grafts too?
> Wouldn't it be a matter of having history walkers never obey grafts but
> keep track of them (i.e. of the history of the parenthood they
> reference)?

In the past we discussed the possibility of that for quite a while but
never saw a successful implementation.  The replace mechanism seemed a
cleaner way to do this, and it turned out to be the case.

You are welcome to try doing that for the grafts, of course.

Re: Questions about the new

[Christian Couder]

On Monday 12 October 2009, Sergio Callegari wrote:
> Thanks Johannes for all the detailed explanations
>
> Johannes Sixt <j.sixt <at> viscovery.net> writes:

[...]

>  > With grafts you can only change parenthood; with replace entries you
>  > can change parenthood *and* all other aspects of a commit (message,
>  > author, committer, dates).
>  >
>  > Hence, replace entries are more general than grafts.
>
> Limiting the discussion to commit objects, I think there are two
> possible scenarios.
>
> 1) You create new commits objects as needed
> 2) You do not.
>
> If you follow 1), I believe grafts and replace entries have exactly the
> same flexibility.
>
> If I happen not to like commit B in A---B---C and I want A---B'---C
> where B' has
> completely different aspects from B I can either replace B by B' or
> graft away
> B, pretending that the parent of A is B

You mean "pretending that the parent of C is A", right?

> But there are many things that can be done with grafts merely adding a
> graft (e.g. cutting away a part of history, joining history),  that
> cannot be done with replace entries without creating new commits objects.

Yes, but when you create a graft, you add a new line in the graft file. You 
don't get the grafts for free.

> I was asking because I was wandering whether replace entries were first
> or later
> meant to make grafts deprecated. I hope not, because for a few things
> working on
> arcs seems still nice.

I don't think they will be deprecated soon. And anyway there will probably 
be a warning when a graft is used if it is deprecated.

[...]

>  > > Conversely, I guess
>  > > you can always simulate a replace entry with the graft mechanism,
>
> without the
>
>  > > need to add any extra commit object. Am I overlooking something?
>  >
>  > You cannot; see above.
>
> Well, I meant for what regards commit objects only.
>
> If I want to replace some commit X by some commit X' I merely need to
> modify the
> parent information of all the commits that are child of X so that they
> pretend
> to be child of X', or am I missing something?

If you use git replace you just need to create commit X' and then use "git 
replace X X'". If you use grafts, yes, you have to modify the parent 
information of all the commits that are child of X.

>  > You can even replace tree objects and blob objects
>  > using replace entries, IIUC, but you cannot do that with grafts.
>
> Definitely right!
>
>  > > 2) Is it currently possible to use a replace entry to replace a
>
> commit object
>
>  > > with nothing? Namely if B has A as its sole parent, is it possible
>
> to have a
>
>  > > replace entry such as A-sha1 becomes null, to pretend that B is a
>
> hierarchy
>
>  > > root?
>  >
>  > Sure. Just make a commit object that does not have parents.
>
> OK, you need to create a new commit object. At the beginning for some
> reason I
> thought that you could replace an object
> with "nothing" or 00000000000000000000000000000000000000000000
>
>  > > 3) If I remember correctly, there was a reason why grafts were not
>
> considered
>
>  > > suitable for transferring across repos. Can someone remind me about
>
> it? How
>
>  > > does the replace mechanism address this issue?
>  >
>  > The problem with grafts was that, for example, git-pack-objects
>
> obeyed the
>
>  > graft, and could create a broken repository by removing grafted-away
>  > objects. And since git-fsck also obeyed the graft, it did not notice
>  > the breakage.
>  >
>  > OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck
>  > never obey replace entries in the history. But they do keep track of
>  > them (and the history that they reference) because they are referenced
>
> from the
>
>  > refs/replace namespace.
>
> Thanks for the explanation. Can this be made possible for grafts too?
> Wouldn't
> it be a matter of having history walkers never obey grafts but keep track
> of them (i.e. of the history of the parenthood they reference)?

The problem is that grafts are special, so all the history walking commands 
should be changed to deal with them specially. With the replace mechanism, 
commits and refs are used, and all the commands already know how to deal 
with them.

> Like we have "annotated" or heavyweight tags living as objects in the
> database,
> would it be possible or make sense to have annotated grafts or replace
> entries,
> so that one can express why, by whom and when history was changed?

There is a patch series about "notes" floating around that deals with 
annotating any commit. So it could be used for that.

And anyway when you create the replacement commit, you can state in the 
commit message that it is a replacement commit, who created it, etc.

Regards,
Christian.

Re: Questions about the new

[Sergio Callegari]

Junio C Hamano wrote:
> Sergio Callegari <sergio.callegari@gmail.com> writes:
>
>   
>> If I want to replace some commit X by some commit X' I merely need to
>> modify the
>> parent information of all the commits that are child of X so that they
>> pretend
>> to be child of X', or am I missing something?
>>     
>
> You need to find all the commits that are child of X in the first place.
> What should happen if your colleague has such a commit in his repository
> (which you haven't fetched from yet), you enumerated all children of X
> known to you in your graft file and then you fetch from him?  You need to
> enumerate all children of X again to keep the graft file up to date.
>   
Ok, that is enlightening. When trying to sort out the differences, 
advantages and disadvantages of
operating on arcs (grafts like) or on nodes (replacements like) I was 
thinking local, rather than
distributed. Now the advantage of working on nodes is much clearer to me.

Thanks as usual for the very clear explanation.

Sergio

Re: Questions about the new refs/replace mechanism

[Jakub Narebski]

Johannes Sixt <j.sixt@viscovery.net> writes:

> Sergio schrieb:

> > 3) If I remember correctly, there was a reason why grafts were not
> > considered suitable for transferring across repos. Can someone
> > remind me about it? How does the replace mechanism address this
> > issue?
> 
> The problem with grafts was that, for example, git-pack-objects obeyed the
> graft, and could create a broken repository by removing grafted-away
> objects. And since git-fsck also obeyed the graft, it did not notice the
> breakage.

To be more detailed, the problem is that if git-pack-objects, git-fsck
and git-gc obeys grafts, it can create broken repository by removing
grafted away objects.  If git-pack-objects, git-fsck and git-gc
doesn't obey grafts, it can created broken repository (well, broken if
we include grafts) by removing grafted in objects.

> 
> OTOH, history walkers (upload-pack, send-pack, pack-objects) and fsck
> never obey replace entries in the history. But they do keep track of them
> (and the history that they reference) because they are referenced from the
> refs/replace namespace.

In the case of refs/replace git-pack-objects, git-fcsk and git-gc
doesn't "obey" refs/replace... but replaced objects are protected by
pruning by being referenced from refs/replace ref.

One of problems with grafts file was to come up with rule what do do
if both repository you fetch from and the repository you fetch into
have both grafts; in the case of refs/replace the usual rules about
(conflicting) refs apply.

It is also easy to select whether to follow refs/replace or not: you
fetch them into your refs/replace or not; you would have to add extra
option to git-fetch to select whether to fetch and follow grafts in
remote you fetch from.

-- 
Jakub Narebski
Poland
ShadeHawk on #git