clarification of `rev-list --no-walk ^<rev>`?

clarification of `rev-list --no-walk ^<rev>`?

[Philip Oakley]

A question came up on the Git user list regarding cherry-pick that got me
reading the manual (again), in this case regarding --no-walk ranges.

Essentially my question is: If --no-walk is given to rev-list (e.g. via
charry-pick), and the user includes a caret prefixed rev, when does that
range definition take effect on the command line, especially in light of
the --do-walk option?

In rev-list(1) there are only 8 references to  'range', with only
the --no-walk option saying "This has no effect if a range is specified."
but leaving open the decision as to what does (and does not) comprises the
specification of a range on the cli.

The two and three dot notations are fairly obvious ranges from
gitrevisions(7) as they are complete strings, while the caret prefix is an 
implied range (it needs additional parameters to complete the range, and 
there-in lies the issue).

It can be read that

$ git cherry-pick maint next

would pick two single commits, while

$ git cherry-pick maint next ^master

could implicitly be read as

$ git cherry-pick maint next --do-walk ^master

because the ^ caret starts the range that cancels the --no-walk.

Clearly that's not what is intended, which is

$ git cherry-pick --do-walk maint next ^master

but it is open to interpretation as to where in the command line the caret
range prefix's --do-walk (to countermand the --no-walk) should applied.

If the user did want just the single commit at the tip of maint, and then
the range master..next, what would be their command line, and also, how
would the man page warn against false expectations?

--

Philip

Re: clarification of `rev-list --no-walk ^<rev>`?

[Michael J Gruber]

Philip Oakley venit, vidit, dixit 19.09.2016 12:56:
> A question came up on the Git user list regarding cherry-pick that got me
> reading the manual (again), in this case regarding --no-walk ranges.
> 
> Essentially my question is: If --no-walk is given to rev-list (e.g. via
> charry-pick), and the user includes a caret prefixed rev, when does that
> range definition take effect on the command line, especially in light of
> the --do-walk option?
> 
> In rev-list(1) there are only 8 references to  'range', with only
> the --no-walk option saying "This has no effect if a range is specified."
> but leaving open the decision as to what does (and does not) comprises the
> specification of a range on the cli.
> 
> The two and three dot notations are fairly obvious ranges from
> gitrevisions(7) as they are complete strings, while the caret prefix is an 
> implied range (it needs additional parameters to complete the range, and 
> there-in lies the issue).
> 
> It can be read that
> 
> $ git cherry-pick maint next
> 
> would pick two single commits, while
> 
> $ git cherry-pick maint next ^master
> 
> could implicitly be read as
> 
> $ git cherry-pick maint next --do-walk ^master
> 
> because the ^ caret starts the range that cancels the --no-walk.
> 
> Clearly that's not what is intended, which is
> 
> $ git cherry-pick --do-walk maint next ^master
> 
> but it is open to interpretation as to where in the command line the caret
> range prefix's --do-walk (to countermand the --no-walk) should applied.
> 
> If the user did want just the single commit at the tip of maint, and then
> the range master..next, what would be their command line, and also, how
> would the man page warn against false expectations?

Maybe:

Every negative rev (rev prefixed with ^, or a range) implies a
`--do-walk` (right at its position on the command line).

And then curb the misleading range sentence in the `--no-walk` description.

Michael

Re: clarification of `rev-list --no-walk ^<rev>`?

[Philip Oakley]

From: "Michael J Gruber" <git@drmicha.warpmail.net>
> Philip Oakley venit, vidit, dixit 19.09.2016 12:56:
>> A question came up on the Git user list regarding cherry-pick that got me
>> reading the manual (again), in this case regarding --no-walk ranges.
>>
>> Essentially my question is: If --no-walk is given to rev-list (e.g. via
>> charry-pick), and the user includes a caret prefixed rev, when does that
>> range definition take effect on the command line, especially in light of
>> the --do-walk option?
>>
>> In rev-list(1) there are only 8 references to  'range', with only
>> the --no-walk option saying "This has no effect if a range is specified."
>> but leaving open the decision as to what does (and does not) comprises 
>> the
>> specification of a range on the cli.
>>
>> The two and three dot notations are fairly obvious ranges from
>> gitrevisions(7) as they are complete strings, while the caret prefix is 
>> an
>> implied range (it needs additional parameters to complete the range, and
>> there-in lies the issue).
>>
>> It can be read that
>>
>> $ git cherry-pick maint next
>>
>> would pick two single commits, while
>>
>> $ git cherry-pick maint next ^master
>>
>> could implicitly be read as
>>
>> $ git cherry-pick maint next --do-walk ^master
>>
>> because the ^ caret starts the range that cancels the --no-walk.
>>
>> Clearly that's not what is intended, which is
>>
>> $ git cherry-pick --do-walk maint next ^master
>>
>> but it is open to interpretation as to where in the command line the 
>> caret
>> range prefix's --do-walk (to countermand the --no-walk) should applied.
>>
>> If the user did want just the single commit at the tip of maint, and then
>> the range master..next, what would be their command line, and also, how
>> would the man page warn against false expectations?
>
> Maybe:
>
> Every negative rev (rev prefixed with ^, or a range) implies a
> `--do-walk` (right at its position on the command line).
>
> And then curb the misleading range sentence in the `--no-walk` 
> description.

At the moment the cherry-pick man page's example implies that --do-walk is 
applied from the beginning, rather from the point given on the command line.

I had a very quick search of the *.c code for the options but didn't get any 
further. Hopefully the user issue/misunderstanding is elsewhere... I'll add 
this to my little list.

--
Philip 

Re: clarification of `rev-list --no-walk ^<rev>`?

[Junio C Hamano]

"Philip Oakley" <philipoakley@iee.org> writes:

> At the moment the cherry-pick man page's example implies that
> --do-walk is applied from the beginning, rather from the point given
> on the command line.
>
> I had a very quick search of the *.c code for the options but didn't
> get any further. Hopefully the user issue/misunderstanding is
> elsewhere... I'll add this to my little list.

I think the confusion is coming from not understanding that revision
specifiers cannot have position-dependent semantics, because there
is no "union of multiple sets".  You said

	commits in 'master..next' range and the tip of 'maint'

earlier, and that is a prime specimen of that confusion.  That is
asking "things reachable from next excluding things reachable from
master" computed independently from everything else on the command
line (i.e. that is one set), and "the commit at the tip of 'maint'"
(i.e. that is another set, which consists of a singleton element),
and wanting to take a union of it.  But the revision machinery is
not structured to work that way.  It can only do "reachable from one
enumeration of positive tips, excluding ones reachable from another
enumeration of negative tips".  "no-walk" is a cheap hack that tells
the machinery "stop after collecting that 'one enumeration of
positive tips' and do not walk. Make that enumeration the resulting
set".  Having anything negative in the enumeration of starting
points from the command line automatically turns "no-walk" off, even
for commands that default to "no-walk".

We may need further documentation updates to unconfuse readers.

Re: clarification of `rev-list --no-walk ^<rev>`?

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

>> It can be read that
>> 
>> $ git cherry-pick maint next
>> 
>> would pick two single commits, while
>> 
>> $ git cherry-pick maint next ^master
>> 
>> could implicitly be read as
>> 
>> $ git cherry-pick maint next --do-walk ^master

You can read it as "master..next maint" that does force walking.

>> Clearly that's not what is intended, which is
>> 
>> $ git cherry-pick --do-walk maint next ^master

I do not see the distinction betwee the above two you seem to be
trying to make.  Care to explain?

>> but it is open to interpretation as to where in the command line the caret
>> range prefix's --do-walk (to countermand the --no-walk) should applied.

I do not think it can be position dependent.  Philip probably has a
confused notion that "rev-list A..B C..D" is somehow a union of set
A..B and C..D?

>> If the user did want just the single commit at the tip of maint, and then
>> the range master..next, what would be their command line, and also, how
>> would the man page warn against false expectations?

Yeah, this can show us that all of the have is coming from that
exact confusion I suspected Philip has.  We need to clarify in the
documentation that rev-list set operation does *NOT* have union of
multiple sets to unconfuse the readers.

Re: clarification of `rev-list --no-walk ^<rev>`?

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
> Michael J Gruber <git@drmicha.warpmail.net> writes:
>
>>> It can be read that
>>>
>>> $ git cherry-pick maint next
>>>
>>> would pick two single commits, while
>>>
>>> $ git cherry-pick maint next ^master
>>>
>>> could implicitly be read as
>>>
>>> $ git cherry-pick maint next --do-walk ^master
>
> You can read it as "master..next maint" that does force walking.
>
>>> Clearly that's not what is intended, which is
>>>
>>> $ git cherry-pick --do-walk maint next ^master
>
> I do not see the distinction betwee the above two you seem to be
> trying to make.  Care to explain?
>
>>> but it is open to interpretation as to where in the command line the 
>>> caret
>>> range prefix's --do-walk (to countermand the --no-walk) should applied.
>
> I do not think it can be position dependent.

OK. (background, long story) When I first read the man page, and in trying 
to explain a user confusion between cherry pick commits and am'ing the same 
commits via format-patch (where sometimes the patches had coverlapping 
context issues), I was trying to confirm for myself that 'git cherry-pick B 
C' and 'git cherry-pick C B' should get the same end result, and not be 
mistaken (e.g. user misunderstanding) for a range.

I first spotted the 'git help cherry-pick's line:

<commit> - no traversal is done by default, as if the --no-walk option was 
specified, see git-rev-list(1).

So off I goes to rev-list, and see that the options no-walk/do-walk are said 
(implied) to be position dependent.

--do-walk  - Overrides a previous --no-walk.

Meanwhile the --no-walk option says:

 --no-walk - This (option) has no effect if a range is specified.

So at this point I am wondering about the command line ordering and what 
comprises the range if the negative ref is given last, or at least just 
before a --do-walk.

Thus (at this point) it felt like one of those specification rabbit-holes 
that I often see at $dayjob. It was unclear as to the point at which the 
'range' was to be applied in to the command line to get the expected 
examples.


Climbing back out of the rabbit-hole. I now see that after the end of the 
cherry-pick <commit> line I quoted, there is a secondary "Note that 
specifying a range will feed all <commit>. arguments to a single revision 
walk", which at the time did not register (a classic human error.. Three 
Mile Island et al.).

Similarly, in the rev-list --no-walk option, its says (mid-para) "This has 
no effect if a range is specified." so in some ways that confirms the cherry 
pick statement, but again a less obvious corollary.

However, there is still a small step missing, which is to confirm that using 
a negative ^ref anywhere(?) makes the whole list of refs into a range (i.e. 
it will look-back along the command line) to cancel any --no-walk options in 
place.

Given that a walk usually requires a range, I'm now having difficulty seeing 
how the --no-walk <revs> --do-walk can be combined anyway.

The bits I felt was missing (in the docs) was to say explicitly somewhere 
that a negative ref defined that we had a range (to link back to those 
walk-no-walk statements), and the extent of rev paramaters it applied to.

And after re-reading, that some of those corollary statements about ranges 
flipping the walk-no-walk condition should be brought forward to be more 
obvious within the primary rev-list (to avoid the typical reader error).


>     Philip probably has a
> confused notion that "rev-list A..B C..D" is somehow a union of set
> A..B and C..D?

That wasn't the issue. Though it does beg the question that it's the same as 
"rev-list D B ^A ^C" isn't it?

>
>>> If the user did want just the single commit at the tip of maint, and 
>>> then
>>> the range master..next, what would be their command line, and also, how
>>> would the man page warn against false expectations?
>
> Yeah, this can show us that all of the have is coming from that
> exact confusion I suspected Philip has.  We need to clarify in the
> documentation that rev-list set operation does *NOT* have union of
> multiple sets to unconfuse the readers.

I'd say it was the walk - no walk range confusion. Inclusion of any range 
definition of any sort (in particular ^rev) causes the expectation that an 
ordered list of single revs can be included, to be broken.
i.e. cherry-pick B D F G Q..T;  isn't B D F G R S T, is it?

I've also have a quick browse of the test scripts and didn't see any tests 
that actually cover the example of `git cherry-pick maint next ^master` 
where both have multiple commits to pick, so couldn't see what the test 
would expect.

--
Philip

Re: clarification of `rev-list --no-walk ^<rev>`?

[Junio C Hamano]

"Philip Oakley" <philipoakley@iee.org> writes:

>>     Philip probably has a
>> confused notion that "rev-list A..B C..D" is somehow a union of set
>> A..B and C..D?
>
> That wasn't the issue. Though it does beg the question that it's the
> same as "rev-list D B ^A ^C" isn't it?

If you think it begs the question, then you haven't understood what
I meant by all of the explanation.  Let me repeat:

	"A..B C..D" is exactly a short-hand for "^A B ^C D" which is
	the same as ANY permutation like "D B ^A ^C".

This is because rev-list machinery NEVER does "ah the command line
asks for two or more sets to be computed and then a union of the
results to be returned".  As I said, it collects positive starting
points (B and D) and negative starting points (A and C), and does a
single traversal "ones that are reachable from positives, excluding
the ones that are reachable from negatives".  That is all it does.

Once that is understood, it should be obvious that it would not have
any effect on the result to permute the order of positives and
negatives.

> I'd say it was the walk - no walk range confusion.

You may _additionally_ have had that confusion ;-).

> i.e. cherry-pick B D F G Q..T;  isn't B D F G R S T, is it?

Of course it isn't, but that is not about any ordering.  It is
because the machinery does not work this way:

  - we have positives B D F G and ... oh wait, we see Q..T, so we
    need to traverse ONLY that bit and come up with positives in the
    resulting set, which are R S T, and take a union of all of the
    above; and then

  - now we process --no-walk to show only the resulting positives.

The "--no-walk" tells the rev-list machinery "I have only positives;
do not traverse from them AT ALL but just use these positives".
Only under that condition, the order of the positive ends you list
on the command line matters.

Having ^Q in the input "B D F G ^Q T" contradicts that "I have only
positives" precondtion and forces the machinery to traverse, i.e. to
find things reachable from B D F G T excluding things reachable from
Q.  The order of the output would be topological (i.e. a parent is
not shown until all its children are), with or without --date-order,
and may not match the order you gave the positive ones from the
command line.

You may want the "exclude things reachable from Q" part to not have
any effect on things related to B, D, F or G, but the machinery does
not work that way.  That is because there is only one single traversal
and no union operation involved.

It is a different matter Git born in a fictional world that takes
A..B C..D as a union of two sets is better than the current one.
There might be cases where such a variant of "a set of commits that
is a union of multiple sets" is handy to use.  But I think we are
talking about what Git in our world does, so I'd not go there in
this discussion.

Re: clarification of `rev-list --no-walk ^<rev>`?

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
> "Philip Oakley" <philipoakley@iee.org> writes:
>
>>>     Philip probably has a
>>> confused notion

Hi Junio,

Could you clarify a particular point from here..

> The "--no-walk" tells the rev-list machinery "I have only positives;
> do not traverse from them AT ALL but just use these positives".
> Only under that condition, the order of the positive ends you list
> on the command line matters.

What does "--do-walk" do ("Overrides a previous --no-walk"), and when would 
it be applied?

--
Philip 

Re: clarification of `rev-list --no-walk ^<rev>`?

[Junio C Hamano]

"Philip Oakley" <philipoakley@iee.org> writes:

>> The "--no-walk" tells the rev-list machinery "I have only positives;
>> do not traverse from them AT ALL but just use these positives".
>> Only under that condition, the order of the positive ends you list
>> on the command line matters.
>
> What does "--do-walk" do ("Overrides a previous --no-walk"), and when
> would it be applied?

Wouldn't 

    $ git cherry-pick --no-walk --do-walk A

end up walking the history behind A and reproducing the history
since the root commit?

Re: clarification of `rev-list --no-walk ^<rev>`?

[Philip Oakley]

From: "Junio C Hamano" <gitster@pobox.com>
> "Philip Oakley" <philipoakley@iee.org> writes:
>
>>>     Philip probably has a
>>> confused notion that "rev-list A..B C..D" is somehow a union of set
>>> A..B and C..D?
>>
>> That wasn't the issue. Though it does beg the question that it's the
>> same as "rev-list D B ^A ^C" isn't it?
>
> If you think it begs the question, then you haven't understood what
> I meant by all of the explanation.  Let me repeat:

Apologies. We appear to be having an British/American usage 
misunderstanding. Locally, the answer to the begged (rhetorically asked) 
question is, as you say, "Yes, they are the same". It was simply confirming 
our common understanding.

>
> "A..B C..D" is exactly a short-hand for "^A B ^C D" which is
> the same as ANY permutation like "D B ^A ^C".
>

regards,
Philip 

Re: clarification of `rev-list --no-walk ^<rev>`?

[Michael J Gruber]

Junio C Hamano venit, vidit, dixit 19.09.2016 18:12:
> Michael J Gruber <git@drmicha.warpmail.net> writes:
> 
>>> It can be read that
>>>
>>> $ git cherry-pick maint next
>>>
>>> would pick two single commits, while
>>>
>>> $ git cherry-pick maint next ^master
>>>
>>> could implicitly be read as
>>>
>>> $ git cherry-pick maint next --do-walk ^master
> 
> You can read it as "master..next maint" that does force walking.
> 
>>> Clearly that's not what is intended, which is
>>>
>>> $ git cherry-pick --do-walk maint next ^master
> 
> I do not see the distinction betwee the above two you seem to be
> trying to make.  Care to explain?

I think you answered to e-mail (in-reply-to) and to Philip's actual text
(quotes), but just in case:

[git]✓ git rev-list --no-walk ^HEAD~3 HEAD
47d74601f5c6bbef215a887be2ca877e34391c9f
574dece7b651fbae385add51d7aaea1cc414007a
3fbbf6e9e40b151215cce6c6e25cd4db0232d870
[git]✓ git rev-list ^HEAD~3 --no-walk HEAD
47d74601f5c6bbef215a887be2ca877e34391c9f

The order of revision arguments and options does play role (but where I
put my HEAD does not, uhm), i.e. walk-options vs. negative refs.

The reason is that negative revs come with an implicit --do-walk (we
need to walk to mark uninteresting revs), and the last
--do-walk/--no-walk wins. That's what I meant with my comment.

But there is only one walk (or none), and one setting effective for all
revision arguments.

Michael

Re: clarification of `rev-list --no-walk ^<rev>`?

[Michael J Gruber]

[So many typos, sorry]

Michael J Gruber venit, vidit, dixit 21.09.2016 16:46:
> Junio C Hamano venit, vidit, dixit 19.09.2016 18:12:
>> Michael J Gruber <git@drmicha.warpmail.net> writes:
>>
>>>> It can be read that
>>>>
>>>> $ git cherry-pick maint next
>>>>
>>>> would pick two single commits, while
>>>>
>>>> $ git cherry-pick maint next ^master
>>>>
>>>> could implicitly be read as
>>>>
>>>> $ git cherry-pick maint next --do-walk ^master
>>
>> You can read it as "master..next maint" that does force walking.
>>
>>>> Clearly that's not what is intended, which is
>>>>
>>>> $ git cherry-pick --do-walk maint next ^master
>>
>> I do not see the distinction betwee the above two you seem to be
>> trying to make.  Care to explain?
> 
> I think you answered to e-mail (in-reply-to) and to Philip's actual text
> (quotes), but just in case:

"my e-mail"

> 
> [git]✓ git rev-list --no-walk ^HEAD~3 HEAD
> 47d74601f5c6bbef215a887be2ca877e34391c9f
> 574dece7b651fbae385add51d7aaea1cc414007a
> 3fbbf6e9e40b151215cce6c6e25cd4db0232d870
> [git]✓ git rev-list ^HEAD~3 --no-walk HEAD
> 47d74601f5c6bbef215a887be2ca877e34391c9f
> 
> The order of revision arguments and options does play role (but where I
> put my HEAD does not, uhm), i.e. walk-options vs. negative refs.

"play a role"
"negative revs"

> 
> The reason is that negative revs come with an implicit --do-walk (we
> need to walk to mark uninteresting revs), and the last

"in order to mark"

> --do-walk/--no-walk wins. That's what I meant with my comment.
> 
> But there is only one walk (or none), and one setting effective for all
> revision arguments.
> 
> Michael
> 

Re: clarification of `rev-list --no-walk ^<rev>`?

[Junio C Hamano]

Michael J Gruber <git@drmicha.warpmail.net> writes:

> I think you answered to e-mail (in-reply-to) and to Philip's actual text
> (quotes), but just in case:

Yes, my mistake.  Sorry for the noise.