Should notes handle replace commits?

Should notes handle replace commits?

[Mike Hommey]

Hi,

Take the following dummy repo:

  $ git init
  $ git commit --allow-empty -m 'first'
  $ git notes add -m 'first note'
  $ git commit --allow-empty -m 'second'
  $ git notes add -m 'second note'
  $ git commit --allow-empty -m 'third'
  $ git notes add -m 'third note'

  $ git log --format='%s - %N'
  third - third note

  second - second note

  first - first note

Now, let's say you have some other history that you want to graft:

  $ git checkout --orphan old
  $ git commit --allow-empty -m 'real first'
  $ git notes add -m 'real first note'
  $ git commit --allow-empty -m 'real second'
  $ git notes add -m 'real second note'
  $ git commit --allow-empty -m 'real third'
  $ git notes add -m 'real third note'
  $ git log --format='%s - %N'
  real third - real third note

  real second - real second note

  real first - real first note

Assuming that the "first" commit on master is the same as the "real
third" on old, you can graft with:

  $ git rev-parse master~ old | xargs > .git/info/grafts

And then:
  $ git log master --format='%s - %N'
  third - third note

  second - second note

  real third - real third note

  real second - real second note

  real first - real first note

Now, if you try to do the same with replace:

  $ rm .git/info/grafts
  $ git replace master~2 old
  $ git log master --format='%s - %N'
  third - third note

  second - second note

  real third - first note

  real second - real second note

  real first - real first note

Note how "real third" now has "first note", instead of "real third
note".

So the question is, is this the behavior this should have?

Cheers,

Mike

Re: Should notes handle replace commits?

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> Assuming that the "first" commit on master is the same as the "real
> third" on old, you can graft with:
>
>   $ git rev-parse master~ old | xargs > .git/info/grafts

With "graft", you told Git that the parent of master~1 ("second") is
not master~2 ("third") but is old ("real third").

> And then:
>   $ git log master --format='%s - %N'
>   third - third note
>
>   second - second note
>
>   real third - real third note

So it is expected that after "second" you see "real third", as you
are seeing the "real third" with its notes.

> Now, if you try to do the same with replace:
>
>   $ rm .git/info/grafts
>   $ git replace master~2 old

With "replace", you told Git that the contents (i.e. message, tree,
etc.) of master~2 ("first") is not what it really is but is what
appears in old ("real third").

>   $ git log master --format='%s - %N'
>   third - third note
>
>   second - second note
>
>   real third - first note

Hence Git tries to show master~2, i.e. "first", here; its contents
is replaced with that of "real third", but the object name of the
commit shown after "second" is shown, as far as Git is concerned, is
still that of "first", so it is not surprising that the note that is
associated with it is shown here, as the whole point of "notes" is
that it is kept outside the contents recorded _in_ the commit.

>   real second - real second note
>
>   real first - real first note
>
> Note how "real third" now has "first note", instead of "real third
> note".

So it is not a correct observation that '"real third" now has "first
note"'.  You are still seeing "first", but its message, together
with its tree and its parents, are replaced by those of "real
third".

> So the question is, is this the behavior this should have?

The behaviour is a natural consequence of what graft and replace are
about (i.e. "telling Git what the parents of a commit are" vs
"telling Git what the contents of a commit are"), so the answer to
the "should" question is a resounding "yes".

It is a separate issue if the behaviour is useful for the purpose
you wanted to use "replace" for.

Re: Should notes handle replace commits?

[Mike Hommey]

On Fri, Jan 08, 2016 at 12:09:45PM -0800, Junio C Hamano wrote:
> > So the question is, is this the behavior this should have?
> 
> The behaviour is a natural consequence of what graft and replace are
> about (i.e. "telling Git what the parents of a commit are" vs
> "telling Git what the contents of a commit are"), so the answer to
> the "should" question is a resounding "yes".

It's not only about contents, except for a very broad definition of
contents that includes ancestry. From my POV, replace is more about
"telling Git that this commit (and its parents) is really that one (and
its parents)".

Anyways, the dummy example is using different commit messages, and notes
everywhere, to make what happens to commits and notes obvious.  In the
real world, "first" would be "real third", so the commit messages would
be as expected in all cases. The question is whether users expect the
note of the non-replaced commit to show up. Would they expect no note at
all in the case there is a note on the replacee, but not on the
replaced?  Maybe the right thing to do would be for *both* notes to be
concatenated? I don't know.

Mike

Re: Should notes handle replace commits?

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> On Fri, Jan 08, 2016 at 12:09:45PM -0800, Junio C Hamano wrote:
>> > So the question is, is this the behavior this should have?
>> 
>> The behaviour is a natural consequence of what graft and replace are
>> about (i.e. "telling Git what the parents of a commit are" vs
>> "telling Git what the contents of a commit are"), so the answer to
>> the "should" question is a resounding "yes".
>
> It's not only about contents, except for a very broad definition of
> contents that includes ancestry.

That is not broad at all.  A Git commit knows about its parents in
exactly the same way as it knows about its tree and its own log
message.  Hashing all of them together, without considering which
part is "broad", gives us the content-addressible filesystem, which
is "stupid content tracker", aka Git.

Perhaps you would see what is going on more clearly if you replace
your "git log" with "git rev-list".

If your pre-graft/pre-replace histories were

	A (first)  <--- B (second)  <--- C (third)	master
	X (rFirst) <--- Y (rSecond) <--- Z (rThird)	old

then your "graft" tells Git "B's parent is Z, not A.  If you run
"rev-list master", it will give you "C B Z Y X".  The discrepancy
(relative to the true history) brought in by "grafting" is that
nowhere in "cat-file commit B" you would find Z, even though "log"
and "rev-list" pretends as if Z is a (and the) parent of B.

Your "replace" tells Git "A records what Z records".  If you run
"rev-list master", it will give you "C B A Y X".

A fake history made by "replace" does not have the same discrepancy
as "grafting"; "cat-file commit B" names A as its parent, and asking
what A is gives what actually is in Z, i.e. "cat-file commit A"
shows what "cat-file commit Z" would give you.  The discrepancy with
the reality "replacing" gives you is that hashing what you got from
"cat-file commit A" does not hash to A (it obviously has to hash to
Z).

> From my POV, replace is more about
> "telling Git that this commit (and its parents) is really that one (and
> its parents)".

Your "POV" does not match reality; replace is about telling Git to
give contents recorded for object Z when anybody asks the contents
recorded for object A.

Re: Should notes handle replace commits?

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> Perhaps you would see what is going on more clearly if you replace
> your "git log" with "git rev-list".
>
> If your pre-graft/pre-replace histories were
>
> 	A (first)  <--- B (second)  <--- C (third)	master
> 	X (rFirst) <--- Y (rSecond) <--- Z (rThird)	old
>
> then your "graft" tells Git "B's parent is Z, not A.  If you run
> "rev-list master", it will give you "C B Z Y X".  The discrepancy
> (relative to the true history) brought in by "grafting" is that
> nowhere in "cat-file commit B" you would find Z, even though "log"
> and "rev-list" pretends as if Z is a (and the) parent of B.
>
> Your "replace" tells Git "A records what Z records".  If you run
> "rev-list master", it will give you "C B A Y X".
>
> A fake history made by "replace" does not have the same discrepancy
> as "grafting"; "cat-file commit B" names A as its parent, and asking
> what A is gives what actually is in Z, i.e. "cat-file commit A"
> shows what "cat-file commit Z" would give you.  The discrepancy with
> the reality "replacing" gives you is that hashing what you got from
> "cat-file commit A" does not hash to A (it obviously has to hash to
> Z).
>
>> From my POV, replace is more about
>> "telling Git that this commit (and its parents) is really that one (and
>> its parents)".
>
> Your "POV" does not match reality; replace is about telling Git to
> give contents recorded for object Z when anybody asks the contents
> recorded for object A.

To put it differently, what you did in your two examples with graft
and replace are not equivalent.  With graft, you told commit B that
its parent is not commit A but commit Z.  If you wanted to do the
equivalent with replace, you would have replaced commit B with an
otherwise identical commit B' that records Z as its parent.  But you
didn't; instead, you replaced commit A with Z.

And if you did the equivalent with "replace", your "git rev-list"
would have shown "C B Z Y X" (instead of "C B A Y X"), and when "git
log" showed the second commit, it would have shown the contents of B'
_and_ because Git still thinks it is showing the original B, it
would have shown the notes for B.

Something like this (totally untested) would let you replace B with
an otherwise identical B' that has Z instead of A as its parent:

    $ Bprime=$(git cat-file commit master~ |
             sed -e "s/^parent .*/parent $(git rev-parse old)/" |
             git hash-object -w --stdin -t commit)

    $ git update-ref refs/replace/$(git rev-parse master~) $Bprime

Re: Should notes handle replace commits?

[Mike Hommey]

On Fri, Jan 08, 2016 at 03:51:39PM -0800, Junio C Hamano wrote:
> Mike Hommey <mh@glandium.org> writes:
> 
> > On Fri, Jan 08, 2016 at 12:09:45PM -0800, Junio C Hamano wrote:
> >> > So the question is, is this the behavior this should have?
> >> 
> >> The behaviour is a natural consequence of what graft and replace are
> >> about (i.e. "telling Git what the parents of a commit are" vs
> >> "telling Git what the contents of a commit are"), so the answer to
> >> the "should" question is a resounding "yes".
> >
> > It's not only about contents, except for a very broad definition of
> > contents that includes ancestry.
> 
> That is not broad at all.  A Git commit knows about its parents in
> exactly the same way as it knows about its tree and its own log
> message.  Hashing all of them together, without considering which
> part is "broad", gives us the content-addressible filesystem, which
> is "stupid content tracker", aka Git.
> 
> Perhaps you would see what is going on more clearly if you replace
> your "git log" with "git rev-list".
> 
> If your pre-graft/pre-replace histories were
> 
> 	A (first)  <--- B (second)  <--- C (third)	master
> 	X (rFirst) <--- Y (rSecond) <--- Z (rThird)	old
> 
> then your "graft" tells Git "B's parent is Z, not A.  If you run
> "rev-list master", it will give you "C B Z Y X".  The discrepancy
> (relative to the true history) brought in by "grafting" is that
> nowhere in "cat-file commit B" you would find Z, even though "log"
> and "rev-list" pretends as if Z is a (and the) parent of B.
> 
> Your "replace" tells Git "A records what Z records".  If you run
> "rev-list master", it will give you "C B A Y X".
> 
> A fake history made by "replace" does not have the same discrepancy
> as "grafting"; "cat-file commit B" names A as its parent, and asking
> what A is gives what actually is in Z, i.e. "cat-file commit A"
> shows what "cat-file commit Z" would give you.  The discrepancy with
> the reality "replacing" gives you is that hashing what you got from
> "cat-file commit A" does not hash to A (it obviously has to hash to
> Z).
> 
> > From my POV, replace is more about
> > "telling Git that this commit (and its parents) is really that one (and
> > its parents)".
> 
> Your "POV" does not match reality; replace is about telling Git to
> give contents recorded for object Z when anybody asks the contents
> recorded for object A.

It's not that different to me, but my point is that (almost) everything
about A redirects to Z, as you point out, _except_ notes.

So while `cat-file commit A` gives you what `cat-file commit Z` would,
`notes show A` doesn't give you what `notes show Z` would. And that's
this "inconsistency" that bothers me.

Mike

Re: Should notes handle replace commits?

[Mike Hommey]

On Fri, Jan 08, 2016 at 04:13:02PM -0800, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
> > Perhaps you would see what is going on more clearly if you replace
> > your "git log" with "git rev-list".
> >
> > If your pre-graft/pre-replace histories were
> >
> > 	A (first)  <--- B (second)  <--- C (third)	master
> > 	X (rFirst) <--- Y (rSecond) <--- Z (rThird)	old
> >
> > then your "graft" tells Git "B's parent is Z, not A.  If you run
> > "rev-list master", it will give you "C B Z Y X".  The discrepancy
> > (relative to the true history) brought in by "grafting" is that
> > nowhere in "cat-file commit B" you would find Z, even though "log"
> > and "rev-list" pretends as if Z is a (and the) parent of B.
> >
> > Your "replace" tells Git "A records what Z records".  If you run
> > "rev-list master", it will give you "C B A Y X".
> >
> > A fake history made by "replace" does not have the same discrepancy
> > as "grafting"; "cat-file commit B" names A as its parent, and asking
> > what A is gives what actually is in Z, i.e. "cat-file commit A"
> > shows what "cat-file commit Z" would give you.  The discrepancy with
> > the reality "replacing" gives you is that hashing what you got from
> > "cat-file commit A" does not hash to A (it obviously has to hash to
> > Z).
> >
> >> From my POV, replace is more about
> >> "telling Git that this commit (and its parents) is really that one (and
> >> its parents)".
> >
> > Your "POV" does not match reality; replace is about telling Git to
> > give contents recorded for object Z when anybody asks the contents
> > recorded for object A.
> 
> To put it differently, what you did in your two examples with graft
> and replace are not equivalent.  With graft, you told commit B that
> its parent is not commit A but commit Z.  If you wanted to do the
> equivalent with replace, you would have replaced commit B with an
> otherwise identical commit B' that records Z as its parent.  But you
> didn't; instead, you replaced commit A with Z.
> 
> And if you did the equivalent with "replace", your "git rev-list"
> would have shown "C B Z Y X" (instead of "C B A Y X"), and when "git
> log" showed the second commit, it would have shown the contents of B'
> _and_ because Git still thinks it is showing the original B, it
> would have shown the notes for B.
> 
> Something like this (totally untested) would let you replace B with
> an otherwise identical B' that has Z instead of A as its parent:
> 
>     $ Bprime=$(git cat-file commit master~ |
>              sed -e "s/^parent .*/parent $(git rev-parse old)/" |
>              git hash-object -w --stdin -t commit)
> 
>     $ git update-ref refs/replace/$(git rev-parse master~) $Bprime

git replace --graft does that automatically. But my contention is not
really about graft vs. replace. I should just have skipped that part,
it's largely irrelevant.

Mike

Re: Should notes handle replace commits?

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> So while `cat-file commit A` gives you what `cat-file commit Z` would,
> `notes show A` doesn't give you what `notes show Z` would. And that's
> this "inconsistency" that bothers me.

In any case, 'notes' being a way to add extra information to an
existing object means that with your original "replace" that tells
Git to keep A in the history (and give Z's contents when contents of
A was asked), it is absolutely correct that notes for A is shown.
It would make no sense if notes for Z "followed", because as long as
Git is concerned, you told Git to show A when your "git log master"
followed the history down thru B to its parent, which you did not
rewrite to be Z but kept to be A.  With the approach to replace B
with B' that has Z as its parent, when "git log master" follows the
history down thru C to its parent, Git thinks it is showing B but
reads B', and finds out its parent is Z and goes down to Z, and
notes for these two commits B and Z (not B' and Z) would be shown;
there is no need to "follow".

The true source of your confusion, I think, is that there is a
misunderstanding of what "replace A with Z" does.

It is not "whenever somebody refers to A, pretend as if it is
referring to Z".  If that _were_ the case, then I'd agree that
"whenever somebody else asks notes attached to A, pretend as if
notes attached to Z were asked" might make sense, but that does not
match the reality.

It is not graft vs replace.  It is about what you replace with what
other thing: "replace" is a content replacement mechanism, not
identity replacement mechanism.

Re: Should notes handle replace commits?

[Mike Hommey]

On Fri, Jan 08, 2016 at 05:04:03PM -0800, Junio C Hamano wrote:
> It is not graft vs replace.  It is about what you replace with what
> other thing: "replace" is a content replacement mechanism, not
> identity replacement mechanism.

I guess this is where opinions diverge: in what notes are.
I'd argue notes are content just as much as the tree, except they're not
stored inside the commit.

Mike

Re: Should notes handle replace commits?

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
Sent: Saturday, January 09, 2016 1:04 AM
> Mike Hommey <mh@glandium.org> writes:
>
>> So while `cat-file commit A` gives you what `cat-file commit Z` would,
>> `notes show A` doesn't give you what `notes show Z` would. And that's
>> this "inconsistency" that bothers me.
>
> In any case, 'notes' being a way to add extra information to an
> existing object means that with your original "replace" that tells
> Git to keep A in the history (and give Z's contents when contents of
> A was asked),
>     it is absolutely correct that notes for A is shown.

This seems very wrong to me. Surely we have asked that _all_ references to A 
be replaced by a reference to Z and thence onward to Z's contents.

That is, unless 'git --no-replace-objects' was given as an option.

It is as if everything about the old 'A' has been deleted, and new content 
'Z' replaces it.

In this case surely we want the notes about the contents of the new Z. To 
mix up the old and new will cause confusion, as evidenced by Mike's post.

> It would make no sense if notes for Z "followed", because as long as
> Git is concerned, you told Git to show A when your "git log master"
> followed the history down thru B to its parent, which you did not
> rewrite to be Z but kept to be A.  With the approach to replace B
> with B' that has Z as its parent, when "git log master" follows the
> history down thru C to its parent, Git thinks it is showing B but
> reads B', and finds out its parent is Z and goes down to Z, and
> notes for these two commits B and Z (not B' and Z) would be shown;
> there is no need to "follow".
>
> The true source of your confusion, I think, is that there is a
> misunderstanding of what "replace A with Z" does.
>
> It is not "whenever somebody refers to A, pretend as if it is
> referring to Z".  If that _were_ the case, then I'd agree that
> "whenever somebody else asks notes attached to A, pretend as if
> notes attached to Z were asked" might make sense, but that does not
> match the reality.

I think Mike is pointing out that it is only for notes that this alternate 
reality unexpectedly happens.

>
> It is not graft vs replace.  It is about what you replace with what
> other thing: "replace" is a content replacement mechanism, not
> identity replacement mechanism.

Part of the graft vs replace difficulty (historically) was that a graft 
(easy to use) will keep all the commits, and add a traversal link, while 
replace would essentially 'delete' one of the commits (users did not have a 
spare commit to delete if linking histories). The newer --graft option is 
now the appropriate method of creating an extra commit (that has the extra 
traverse linkage) that can then be 'deleted' and replaced by the commit on 
the other traverse chain.

There is the case where we have a string of commits (A-B-C-D-E) and we 
simply want to swap out commit C and replace it with a new commit C' to 
create (A-B-C'-D-E). A new C' would be prepared, and then upon replacement 
of C by C' we have to ask "What should be done about the old notes that were 
attached to C?".

I think that it maybe here that the issue lies. Some will think that the 
notes of the old C should still be attached to that location in the string, 
while others may think that it's for the user to both notice, and check the 
veracity, and only then carry those old notes across to the new C' 
(manually).

--
Philip

> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Should notes handle replace commits?

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> On Fri, Jan 08, 2016 at 05:04:03PM -0800, Junio C Hamano wrote:
>> It is not graft vs replace.  It is about what you replace with what
>> other thing: "replace" is a content replacement mechanism, not
>> identity replacement mechanism.
>
> I guess this is where opinions diverge: in what notes are.
> I'd argue notes are content just as much as the tree, except they're not
> stored inside the commit.

There is no room for any divergence in opinion around what
"contents" are in the context of Git, though.

Iff the name of the object would change if you change something in
it, then that something is "contents" of the object.  So the list of
parents of a commit is part of the contents of the commit object.

Notes are not.

In fact, that is the primary reason notes exist---the whole point of
notes is that with them you can add pieces of information ABOUT an
existing object without changing the contents of the object.

Re: Should notes handle replace commits?

[Junio C Hamano]

"Philip Oakley" <philipoakley@iee.org> writes:

> From: "Junio C Hamano" <gitster@pobox.com>
> Sent: Saturday, January 09, 2016 1:04 AM
>> Mike Hommey <mh@glandium.org> writes:
>>
>>> So while `cat-file commit A` gives you what `cat-file commit Z` would,
>>> `notes show A` doesn't give you what `notes show Z` would. And that's
>>> this "inconsistency" that bothers me.
>>
>> In any case, 'notes' being a way to add extra information to an
>> existing object means that with your original "replace" that tells
>> Git to keep A in the history (and give Z's contents when contents of
>> A was asked),
>>     it is absolutely correct that notes for A is shown.
>
> This seems very wrong to me. Surely we have asked that _all_
> references to A be replaced by a reference to Z and thence onward to
> Z's contents.

You didn't ask any such thing.

I already said:

    The true source of your confusion, I think, is that there is a
    misunderstanding of what "replace A with Z" does.

    It is NOT "whenever somebody refers to A, pretend as if it is
    referring to Z".  If that _were_ the case, then I'd agree that
    "whenever somebody else asks notes attached to A, pretend as if
    notes attached to Z were asked" might make sense, but that does not
    match the reality.

but then didn't say what "the reality" is (because I already said it
elsewhere upthread).

What you asked is that whenever information in A is asked, a tweaked
version of the original information in A is returned.  The tweaked
version happens to match what is in Z, but after the replacement,
the object Z does not directly get into the picture---as you can see
in the result of log (or rev-list) from C that shows "C B A Y X" (in
tip-to-root order), the end user does not even have to be aware of
Z's existence.

Let's think about the example of grafting "old" history that starts
at X, resurrected from archive since the history of the current
project has already started at A and leading to C in real-world
terms.  A would be like Linus's v2.6.12-rc2 in the Linux kernel
project that does not have any parent.  Then you rebuild the "old"
history from the archive that ends at the same v2.6.12-rc2, so your
X Y Z history also ends with v2.6.12-rc2.  And you would "graft" it
behind the true history.

Commits A and Z would both have the same log message, record the
same tree, made by the same author and committer with same
timestamps.

Now, think which one of A and Z do you think people would want to
"survive" in the graft operation?  Everybody has built (and will
continue to build) on the history that started at A whose tip is
currently at C.

If you have a way to tell Git "Earlier A was recorded as having NO
parent, but I wish we had recorded Y as its parent.  So create a
fake reality where A's parent is Y.", then you can keep the
resulting world view consistent with the "recent-only" real history,
i.e. B's parent is still A.  Some might even have notes on A, or
commits made since A may reference the commit object A by name in
its log, or another superproject may bind A as its subproject, but
because we are not changing the identity/name of A with Z, all of
these "external" references survive the grafting/replacing.

Perhaps it would help if you stop thinking that you are somehow
losing "Z" and "Z" is somehow precious.  It is not, at least in the
"history grafting" context, at all precious.

When you have the true history starting from v2.6.12-rc2 (that is "A
B C"), the thing you REALLY want to do is to resurrect history that
ends at one commit before v2.6.12-rc2 (that is "Y" in the example),
and somehow tweak "A" so that its parent is "Y".  And the way you
tweak "A" to make that happen is to replace its contents with the
contents of a commit that records all the same information as "A"
except that it has a parent that is "Y".  We could do "cat-file
commit A" piped to sed to add a "parent Y" line to it and then
"hash-object" to come up with such a replacement commit, and replace
A with it, but "Z" happens to be such a commit already, so that is
why replacing A with Z would graft Y just behind A.  The resulting
history would show Y as the parent of A and Z won't even be in the
consciousness of the end users.  The "old history resurrection"
project does not even have to create "Z" in the first place.

If the contents of "Z" is different from contents of "A" in some
other way, other than their set of parents, or if "Z" has other
external references that are precious (e.g. a note may refer to it,
or a superproject may bind Z to its tree), then either replacing A
with Z or replacing Z with A would not be a good way to connect the
two histories.  As you would want to keep both, and the "external"
references, such as notes and submodule gitlink bindings, want to
keep refering to both.

In such a case, the best you can do is to tweak A to have Z (instead
of Y) as its fake parent, to make the resulting fake history read
like "C B A Z Y X" (in tip-to-root order), having two copies of
v2.6.12-rc2.  That way you would be able to see both notes for A and
Z, and can tell which note was attached to the true history and
which note was attached to the history resurrected from the archive.