best way to fastforward all tracking branches after a fetch

best way to fastforward all tracking branches after a fetch

[Gelonida N]

Hi,

What is the best way to fastforward all fastforwardable tracking
branches after a git fetch?

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
> Hi,
> 
> What is the best way to fastforward all fastforwardable tracking
> branches after a git fetch?

I dont think there is a single command to do it for *all*
branches, but for any particular branch, this should work:

    git merge --ff-only @{u}

So what you want would boil down to this script (untested):

    #!/bin/bash
    git status --porcelain -uno | grep . && {echo dirty tree, exiting...; exit 1; }

    for b in `git for-each-ref '--format=%(refname:short)' refs/heads`
    do
        git checkout $b
        git merge --ff-only @{u}
    done

Re: best way to fastforward all tracking branches after a fetch

[Gelonida N]

On 12/11/2011 03:22 AM, Sitaram Chamarty wrote:
> On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
>> Hi,
>>
>> What is the best way to fastforward all fastforwardable tracking
>> branches after a git fetch?
> 
> I dont think there is a single command to do it for *all*
> branches, but for any particular branch, this should work:

A tiny script is fine. I didn't really expect the magic command.

Currently I'm using one script per repository, which hard coded the
branches,
that I want to fast-forward (checking them out and doing a git-pull)

> 
>     git merge --ff-only @{u}
> 
Thanks, '--ff=only @{u}'  is already the first improvement for my script.


> So what you want would boil down to this script (untested):
> 
>     #!/bin/bash
>     git status --porcelain -uno | grep . && {echo dirty tree, exiting...; exit 1; }
> 
>     for b in `git for-each-ref '--format=%(refname:short)' refs/heads`
>     do
>         git checkout $b
>         git merge --ff-only @{u}
>     done



Is there no way to distinguish tracking branches from other branches?
without checking them out?

In order to save time I'd like to avoid checking out local branches.

Ideally I would even like to avoid checking out branches, which don't
need to be forwarded.


I also had to remember on which branch I was in order to avoid, that I
am at a random branch after running the script.

I could imagine something like my snippet below , though I guess,
there's something more elegant.

git stash
mybranch=`git branch | sed -n 's/\* *//p'`
# do_script . . .
git checkout $mybranch
git stash apply

Re: best way to fastforward all tracking branches after a fetch

[Martin Langhoff]

On Sat, Dec 10, 2011 at 7:26 AM, Gelonida N <gelonida@gmail.com> wrote:
> What is the best way to fastforward all fastforwardable tracking
> branches after a git fetch?

It'd be a great addition to git fetch ;-)


m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Re: best way to fastforward all tracking branches after a fetch

[Jakub Narebski]

Don't remove people from Cc, please.

Gelonida N <gelonida@gmail.com> writes:
> On 12/11/2011 03:22 AM, Sitaram Chamarty wrote:
> > On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:

> > So what you want would boil down to this script (untested):
> > 
> >     #!/bin/bash
> >     git status --porcelain -uno | grep . && {echo dirty tree, exiting...; exit 1; }
> > 
> >     for b in `git for-each-ref '--format=%(refname:short)' refs/heads`
> >     do
> >         git checkout $b
> >         git merge --ff-only @{u}
> >     done
> 
> Is there no way to distinguish tracking branches from other branches?
> without checking them out?
> 
> In order to save time I'd like to avoid checking out local branches.

You can use 'upstream' field name in git-for-each-ref invocation,
for example

  git for-each-ref '--format=%(refname:short) %(upstream:short)' refs/heads |
  	grep -e ' [^ ]' |
  	sed  -e 's/ .*$//
 
This could probably be done using only sed -- grep is not necessary.

> Ideally I would even like to avoid checking out branches, which don't
> need to be forwarded.
 
You can use git-update-ref plumbing, but you would have to do the
check if it does fast-forward yourself, and provide reflog message
yourself too.
 
Something like

  git for-each-ref '--format=%(refname) %(upstream)' |
  while read refname upstream
  do
  	# there is upstream
  	test -n "$upstream" || break
  	# and if fast-forwards
  	test $(git merge-base $refname $upstream) = $(git rev-parse $refname) || break
  	git update-ref -m "$message" $refname $upstream
  done

> I also had to remember on which branch I was in order to avoid, that I
> am at a random branch after running the script.
> 
> I could imagine something like my snippet below , though I guess,
> there's something more elegant.
> 
> git stash
> mybranch=`git branch | sed -n 's/\* *//p'`
> # do_script . . .
> git checkout $mybranch
> git stash apply

Don't use git-branch in scripting.  See __git_ps1 function in
contrib/completion/git-completion.bash how it can be done:

  b="$(git symbolic-ref HEAD 2>/dev/null)" ||
  b="$(git rev-parse --verify HEAD)"

Nb. the second part is here only if there is possibility that you are
on detached HEAD (unnamed branch).

HTH (hope that helps)
-- 
Jakub Narębski

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

2011/12/11 Jakub Narebski <jnareb@gmail.com>:
> Don't remove people from Cc, please.
>
> Gelonida N <gelonida@gmail.com> writes:
>> On 12/11/2011 03:22 AM, Sitaram Chamarty wrote:
>> > On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
>
>> > So what you want would boil down to this script (untested):
>> >
>> >     #!/bin/bash
>> >     git status --porcelain -uno | grep . && {echo dirty tree, exiting...; exit 1; }
>> >
>> >     for b in `git for-each-ref '--format=%(refname:short)' refs/heads`
>> >     do
>> >         git checkout $b
>> >         git merge --ff-only @{u}
>> >     done
>>
>> Is there no way to distinguish tracking branches from other branches?
>> without checking them out?
>>
>> In order to save time I'd like to avoid checking out local branches.
>
> You can use 'upstream' field name in git-for-each-ref invocation,
> for example
>
>  git for-each-ref '--format=%(refname:short) %(upstream:short)' refs/heads |
>        grep -e ' [^ ]' |
>        sed  -e 's/ .*$//
>
> This could probably be done using only sed -- grep is not necessary.
>
>> Ideally I would even like to avoid checking out branches, which don't
>> need to be forwarded.
>
> You can use git-update-ref plumbing, but you would have to do the
> check if it does fast-forward yourself, and provide reflog message
> yourself too.

if it's not the currently checked-out branch, 'git branch -f foo
origin/foo' seems to work fine.  However, it only updates the branch
reflog, not the HEAD reflog also, naturally.

FWIW...

> Something like
>
>  git for-each-ref '--format=%(refname) %(upstream)' |
>  while read refname upstream
>  do
>        # there is upstream
>        test -n "$upstream" || break
>        # and if fast-forwards
>        test $(git merge-base $refname $upstream) = $(git rev-parse $refname) || break
>        git update-ref -m "$message" $refname $upstream
>  done
>
>> I also had to remember on which branch I was in order to avoid, that I
>> am at a random branch after running the script.
>>
>> I could imagine something like my snippet below , though I guess,
>> there's something more elegant.
>>
>> git stash
>> mybranch=`git branch | sed -n 's/\* *//p'`
>> # do_script . . .
>> git checkout $mybranch
>> git stash apply
>
> Don't use git-branch in scripting.  See __git_ps1 function in
> contrib/completion/git-completion.bash how it can be done:
>
>  b="$(git symbolic-ref HEAD 2>/dev/null)" ||
>  b="$(git rev-parse --verify HEAD)"
>
> Nb. the second part is here only if there is possibility that you are
> on detached HEAD (unnamed branch).
>
> HTH (hope that helps)
> --
> Jakub Narębski



-- 
Sitaram

Re: best way to fastforward all tracking branches after a fetch

[Andreas Schwab]

Jakub Narebski <jnareb@gmail.com> writes:

> Something like
>
>   git for-each-ref '--format=%(refname) %(upstream)' |
>   while read refname upstream
>   do
>   	# there is upstream
>   	test -n "$upstream" || break
>   	# and if fast-forwards
>   	test $(git merge-base $refname $upstream) = $(git rev-parse $refname) || break
>   	git update-ref -m "$message" $refname $upstream
>   done

You probably meant s/break/continue/.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: best way to fastforward all tracking branches after a fetch

[Jakub Narebski]

Andreas Schwab wrote:
> Jakub Narebski <jnareb@gmail.com> writes:
> 
> > Something like
> >
> >   git for-each-ref '--format=%(refname) %(upstream)' |
> >   while read refname upstream
> >   do
> >   	# there is upstream
> >   	test -n "$upstream" || break

Here you should probably also check if we are already up-to-date:

    	test $(git rev-parse $upstream) = $(git rev-parse $refname) || continue

> >   	# and if fast-forwards
> >   	test $(git merge-base $refname $upstream) = $(git rev-parse $refname) || break
> >   	git update-ref -m "$message" $refname $upstream
> >   done
> 
> You probably meant s/break/continue/.

Yes, sorry about that.

BTW. git-update-ref invocation can be replaced by Sitaram's suggestion:

    	git branch -f $refname $upstream

-- 
Jakub Narebski
Poland

Re: best way to fastforward all tracking branches after a fetch

[Gelonida N]

On 12/11/2011 07:22 PM, Jakub Narebski wrote:
> Don't remove people from Cc, please.

OK,

> 
> Gelonida N <gelonida@gmail.com> writes:
>> On 12/11/2011 03:22 AM, Sitaram Chamarty wrote:

>>
>> In order to save time I'd like to avoid checking out local branches.
> 
> You can use 'upstream' field name in git-for-each-ref invocation,
> for example
> 
>   git for-each-ref '--format=%(refname:short) %(upstream:short)' refs/heads |
>   	grep -e ' [^ ]' |
>   	sed  -e 's/ .*$//
>  
Thanks



> This could probably be done using only sed -- grep is not necessary.
I think the equivalent would be:
sed '/ [^ ]/ s/ .*$//'


> 
>> Ideally I would even like to avoid checking out branches, which don't
>> need to be forwarded.
>  
> You can use git-update-ref plumbing, but you would have to do the
> check if it does fast-forward yourself, and provide reflog message
> yourself too.
>  

True this would probably be fastest. Will read the docs a little to
understand exactly what you're doing. I'm not that much used to all the
commands used in the script.

> Something like
> 
>   git for-each-ref '--format=%(refname) %(upstream)' |
>   while read refname upstream
>   do
>   	# there is upstream
>   	test -n "$upstream" || break
>   	# and if fast-forwards
>   	test $(git merge-base $refname $upstream) = $(git rev-parse $refname) || break
>   	git update-ref -m "$message" $refname $upstream
>   done
> 
>> I also had to remember on which branch I was in order to avoid, that I
>> am at a random branch after running the script.
>>

> 
> Don't use git-branch in scripting.  See __git_ps1 function in
> contrib/completion/git-completion.bash how it can be done:
> 
>   b="$(git symbolic-ref HEAD 2>/dev/null)" ||
>   b="$(git rev-parse --verify HEAD)"
> 
> Nb. the second part is here only if there is possibility that you are
> on detached HEAD (unnamed branch).
> 
> HTH (hope that helps)

It definitely helps. Thanks a lot.
It's always good to see how one can do better after some attempts o some
self made clumsy scripts not suing all the features of git.

Re: best way to fastforward all tracking branches after a fetch

[Stefan Haller]

Gelonida N <gelonida@gmail.com> wrote:

> What is the best way to fastforward all fastforwardable tracking
> branches after a git fetch?

Here's a script that does this.  It isn't very well tested, I hope I
didn't miss any edge cases. Use at your own risk.

(It doesn't fastforward the branch you're on, on the assumtion that if
you said git fetch instead of git pull, you probably had a reason.)

========= 8< =========
#!/bin/sh

currentbranch="`git symbolic-ref HEAD 2>/dev/null`"

git for-each-ref --shell --format='ref=%(refname);upstream=%(upstream)' \
    refs/heads | \
while read entry
do
    eval "$entry"

    # skip the current branch
    test "$ref" = "$currentbranch" && continue

    # skip branches that have no upstream
    test -z "$upstream" && continue

    # skip if upstream doesn't have any new commits
    if [ -z "`git rev-list "$ref..$upstream"`" ]; then
        echo "${ref#refs/heads/} is up to date"
        continue
    fi

    # error if there are local commits
    if [ -n "`git rev-list "$upstream..$ref"`" ]; then
        echo "${ref#refs/heads/} has local commits; can't fast-forward"
        continue
    fi

    echo "${ref#refs/heads/} -> ${upstream#refs/remotes/}"
    git update-ref -m ff-all-branches "$ref" "$upstream"
done
========= >8 =========


-- 
Stefan Haller
Berlin, Germany
http://www.haller-berlin.de/

Re: best way to fastforward all tracking branches after a fetch

[Gelonida N]

On 12/11/2011 09:14 PM, Stefan Haller wrote:
> Gelonida N <gelonida@gmail.com> wrote:
> 
>> What is the best way to fastforward all fastforwardable tracking
>> branches after a git fetch?
> 
> Here's a script that does this.  It isn't very well tested, I hope I
> didn't miss any edge cases. Use at your own risk.
> 
> (It doesn't fastforward the branch you're on, on the assumtion that if
> you said git fetch instead of git pull, you probably had a reason.)

Agreed. it might be reasonable to ignore the current branch if it wasn't
pulled.


Thanks a lot for the script. I will play with it.

> 

Re: best way to fastforward all tracking branches after a fetch

[Andreas Schwab]

Gelonida N <gelonida@gmail.com> writes:

> On 12/11/2011 07:22 PM, Jakub Narebski wrote:
>> You can use 'upstream' field name in git-for-each-ref invocation,
>> for example
>> 
>>   git for-each-ref '--format=%(refname:short) %(upstream:short)' refs/heads |
>>   	grep -e ' [^ ]' |
>>   	sed  -e 's/ .*$//
>>  
> Thanks
>
>
>
>> This could probably be done using only sed -- grep is not necessary.
> I think the equivalent would be:
> sed '/ [^ ]/ s/ .*$//'

You need to suppress printing the non-matching lines.

  sed -n '/ [^ ]/ s/ .*$//p'

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: best way to fastforward all tracking branches after a fetch

[Martin Langhoff]

On Sun, Dec 11, 2011 at 3:27 PM, Gelonida N <gelonida@gmail.com> wrote:
> Agreed. it might be reasonable to ignore the current branch if it wasn't
> pulled.

Actually, what this means is that this should be an enhancement to git
pull ("git pull --all") because pull means fetch + merge.

cheers,



m
-- 
 martin.langhoff@gmail.com
 martin@laptop.org -- Software Architect - OLPC
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff

Re: best way to fastforward all tracking branches after a fetch

[Hallvard B Furuseth]

Stefan Haller writes:
>Gelonida N <gelonida@gmail.com> wrote:
> 
>> What is the best way to fastforward all fastforwardable tracking
>> branches after a git fetch?
> 
> Here's a script that does this.  It isn't very well tested, I hope I
> didn't miss any edge cases. Use at your own risk.

Local branches can track each other.  So the script needs to toposort
the branches, or to loop until either nothing was done or an error
happened.  (The latter to prevent an eternal loop on error.)

I've wished for a more limited 'git ff' command than this:
- git update-ref --ff-only, and branch/fetch options based on this.
- Fast-forward only the branches tracking one particular remote/branch,
  and maybe branches tracking the fast-forwarded branches.
- Fast-forward to another remote or branch than the tracked one.
I have a few aliases for it, but never thought much of a good common
design.

-- 
Hallvard

Re: best way to fastforward all tracking branches after a fetch

[Stefan Haller]

Hallvard B Furuseth <h.b.furuseth@usit.uio.no> wrote:

> Stefan Haller writes:
> >Gelonida N <gelonida@gmail.com> wrote:
> > 
> >> What is the best way to fastforward all fastforwardable tracking
> >> branches after a git fetch?
> > 
> > Here's a script that does this.  It isn't very well tested, I hope I
> > didn't miss any edge cases. Use at your own risk.
> 
> Local branches can track each other.  So the script needs to toposort
> the branches, or to loop until either nothing was done or an error
> happened.  (The latter to prevent an eternal loop on error.)

Is this just theoretical, or are there real use cases for this? What
would be a workflow with such a local tracking branch?

For me personally, the script is good enough, because I only ever have
branches that track an 'origin' branch with the same name.


-- 
Stefan Haller
Berlin, Germany
http://www.haller-berlin.de/

Re: best way to fastforward all tracking branches after a fetch

[Junio C Hamano]

Gelonida N <gelonida@gmail.com> writes:

> What is the best way to fastforward all fastforwardable tracking
> branches after a git fetch?

This lacks context and invites too many tangents, so I'll only touch a few
of them.

First of all, why do you want to do this?

You have many local branches that are forked from remote tracking branches
and can be fast-forwarded to their counterparts, iow, these local branches
are often behind their upstream and you do not have your own development
in this repository. Because you can by definition have only one branch
checked out in your working tree, after a fetch from your origin, they
will further fall behind their counterparts.

Coming from that background, I can see you may want these branches that
are not checked out fast-forwarded to their remote counterparts to keep
them stay current, but I first question that background. Why have these
local branches to begin with, if they always are supposed to match their
remote counterpart?

One possible reason (this is one tangent) is that you want to build and
install tips of many branches fetched from the upstream without doing any
local development in this repository (the "upstream" could be your primary
repository and the changes fed to this repository may be your own
development, so this is different from saying that you as a person is only
following other people's work. It is just that nothing is done to the
history in THIS repository). It could be solved by directly checking out
the remote tracking branches into detached head state, e.g.

    $ for branch in maint master next
      do
        git checkout origin/$branch &&
        make prefix=$HOME/git-$branch all test install || break
      done

and the reason why you want local branches instead may be because your
build infrastructure (i.e. instead of "make" you have a custom script,
just like I use 'Make' script in my 'todo' branch) the does customization
depending on the name of the current branch, and might be more cumbersome
to get the same information for a detached head state (i.e. "the tip of
which remote tracking branch is the current commit?") than asking "git
symbolic-ref" the name of the current branch. But then it is easy to find
out which remote branch was checked out from the reflog for the HEAD (and
it is easier for your script that builds the origin/$branch to use that
information internally when the script calls your 'Make' equivalent). In
any case, it is largely your build customization's problem if this is the
case.

Another tangent. Perhaps the reason why you want these local branches but
they can often be fast-forwarded is because your workflow looks like this:

 (1) you fork a topic from origin/master;
 (2) you develop a bit;
 (3) you push the topic back to origin/master;
 (4) time passes, others push to origin/master, while you work on other
     branches of yours;
 (5) from time to time, you fetch from origin;
 (6) you decide to continue working on the topic, so you check it out,
     and before continuing, you wish it is already up-to-date.

But then after fast-forwarding the topic in (6), your topic's history
contains commits other than those you made to work toward the goal of your
topic, namely, other commits made by others during (4) for random purposes
that do not have anything to do with achieving the goal of the topic of
yours. Your branch is no longer about what you wanted to accomplish on
your topic. This invites two tangents.

One is a question. If you knew that the topic is not cooked fully and
needs further work after step (6), why did you push it back to the
origin/master in the first place at step (3), contaminating the history
everybody else bases their further work on with the contents of your
"half-done" topic?

Another tangent. Perhaps the fork is not made from origin/master but you
are collaboratively working on the same topic with others, and you handed
off the work up to what you have done at step (3), and others continued to
further the goal of the shared topic during (4). If that is the case,
wouldn't it make more sense to delete the topic after you push it back,
and forking at the point when you actually decide to get back into action?

Yet another. Even if you keep the (stale) topic branch that you already
have pushed out to the remote, because you can work on one topic at a time
in a single working tree anyway, perhaps it makes more sense to delay this
fast-forwarding until you actually check out the topic branch? After all,
your wishing to fast-forward "all branches" imply you have many of them,
and it wouldn't be far-fetched for me to imagine that you will check one
of them out a lot less often than you run "git fetch".

In other words, wouldn't a post-checkout hook be a better place to do
this kind of thing, perhaps like this (completely untested)? 

    #!/bin/sh
    old=$1 new=$2 kind=$3

    # did we checkout a branch?
    test "$kind" = 1 || exit 0

    # what did we check out?
    branch=$(git symbolic-ref HEAD 2>/dev/null) || exit 0

    # does it track anything? otherwise nothing needs to be done
    upstream=$(git for-each-ref --format='%(upstream)' "$branch")
    test -z "$upstream" || exit 0

    # are we up-to-date? if so no need to do anything
    test 0 = $(git rev-list "..$upstream" | wc -l) && exit 0

    # do we have something we made? if so no point trying to fast-forward
    test 0 = $(git rev-list "$upstream.." | wc -l) || exit 0

    # attempt a fast-forward merge with it
    git merge --ff-only @{upstream}

That is, of course, assuming that it makes sense to keep these local
branches in the first place.

Re: best way to fastforward all tracking branches after a fetch

[Jeff King]

On Mon, Dec 12, 2011 at 08:33:15AM +0100, Stefan Haller wrote:

> > Local branches can track each other.  So the script needs to toposort
> > the branches, or to loop until either nothing was done or an error
> > happened.  (The latter to prevent an eternal loop on error.)
> 
> Is this just theoretical, or are there real use cases for this? What
> would be a workflow with such a local tracking branch?

I use this all the time.

In git.git, we use a topic branch workflow (i.e., every feature gets its
own topic branch, and topics graduate independently to master as they
are deemed stable). And we use a patch-submission workflow, which means
it's OK for me to rebase my topics locally, because the end-product is a
series of patches sent to the list.

Typically I branch off of "origin/master", so the topic is independent
of anything else. For example, the "jk/credentials" branch in my git
repo is branched from "origin/master" (Junio's master).  But sometimes
there is a topic that depends on another topic, but should not be part
of the same series (because the the first topic can graduate to master,
but the second one may still need more time for discussion and cooking).
In that case, I'll set the upstream to the other local topic branch. An
example of this is the "jk/prompt" series, which depends on
"jk/credentials" for infrastructure, but is really a separate issue.

Having the upstream set is convenient, because I can get _just_ the
commits in jk/prompt with "git log @{u}..". Or I can rebase _just_ the
commits in that topic with "git rebase -i". If my upstream were set to
origin, I would accidentally also rebase all of the commits pulled in
from jk/credentials, too.

While my topics are still in development (i.e., before they have even
hit "next"), I tend to rebase them aggressively (so that I keep them up
to date with git development), using a script that is something like[1]:

  for i in `topics`; do
    git rebase $i@{u} $i
  done

And I do topo-sort my topics for exactly the reason mentioned.

-Peff

[1] https://github.com/peff/git/blob/meta/rebase

Re: best way to fastforward all tracking branches after a fetch

[Junio C Hamano]

Hallvard B Furuseth <h.b.furuseth@usit.uio.no> writes:

> Local branches can track each other.  So the script needs to toposort
> the branches, or to loop until either nothing was done or an error
> happened.  (The latter to prevent an eternal loop on error.)

If you have branches A that forked from B that in turn forked from C, and
if after updating C you want to and can successfully update both B and A
by fast-forwarding, that would only mean that neither A nor B had their
own change since they were forked from their upstream, regardless of local
or remote.

What use do these empty branches have in the first place?

Re: best way to fastforward all tracking branches after a fetch

[Stefan Haller]

Jeff King <peff@peff.net> wrote:

> On Mon, Dec 12, 2011 at 08:33:15AM +0100, Stefan Haller wrote:
> 
> > > Local branches can track each other.  So the script needs to toposort
> > > the branches, or to loop until either nothing was done or an error
> > > happened.  (The latter to prevent an eternal loop on error.)
> > 
> > Is this just theoretical, or are there real use cases for this? What
> > would be a workflow with such a local tracking branch?
> 
> I use this all the time.
> 
> In git.git, we use a topic branch workflow (i.e., every feature gets its
> own topic branch, and topics graduate independently to master as they
> are deemed stable). And we use a patch-submission workflow, which means
> it's OK for me to rebase my topics locally, because the end-product is a
> series of patches sent to the list.
> 
> Typically I branch off of "origin/master", so the topic is independent
> of anything else. For example, the "jk/credentials" branch in my git
> repo is branched from "origin/master" (Junio's master).  But sometimes
> there is a topic that depends on another topic, but should not be part
> of the same series (because the the first topic can graduate to master,
> but the second one may still need more time for discussion and cooking).
> In that case, I'll set the upstream to the other local topic branch. An
> example of this is the "jk/prompt" series, which depends on
> "jk/credentials" for infrastructure, but is really a separate issue.
> 
> Having the upstream set is convenient, because I can get _just_ the
> commits in jk/prompt with "git log @{u}..". Or I can rebase _just_ the
> commits in that topic with "git rebase -i". If my upstream were set to
> origin, I would accidentally also rebase all of the commits pulled in
> from jk/credentials, too.

I see, thanks.  For my script, I'm wondering then if the most sensible
thing to do is to just skip any branch whose upstream doesn't start with
refs/remotes/.

For a future "git pull --all" feature, it would probably only work on
those branches whose upstream is on the remote being pulled from,
anyway.


-- 
Stefan Haller
Berlin, Germany
http://www.haller-berlin.de/

Re: best way to fastforward all tracking branches after a fetch

[Gelonida N]

Thanks for this rather long answer,

On 12/12/2011 09:09 AM, Junio C Hamano wrote:
> Gelonida N <gelonida@gmail.com> writes:
> 
>> What is the best way to fastforward all fastforwardable tracking
>> branches after a git fetch?
> 
> This lacks context and invites too many tangents, so I'll only touch a few
> of them.
> 
> First of all, why do you want to do this?
> 

To explain the scenario:
- small project
- every person works on master and multiple topic branches
   and might alternate rather often
- sometimes several persons work on the same topic branch
  but most of the time not in parallel.
- one person is working from several machines (starting work on
  one and continuing on another)
- additionally we do many pushed in order to be sure,
  that our data is backed up in case of disk failures.
- sometimes I just want to 'build' from a branch, that I am not
   working on. but there I create mostly not even a tracking branch

before changing a machine I want to be sure to have pushed everything. I
wanted to get rid of the warning, that some branches cannot be pushed,
because they aren't fastforwarded

when checking out a branch I want to avoid, that I have to pull manually.



> In other words, wouldn't a post-checkout hook be a better place to do
> this kind of thing, perhaps like this (completely untested)? 
> 
>     #!/bin/sh
>     old=$1 new=$2 kind=$3
> 
>     # did we checkout a branch?
>     test "$kind" = 1 || exit 0
> 
>     # what did we check out?
>     branch=$(git symbolic-ref HEAD 2>/dev/null) || exit 0
> 
>     # does it track anything? otherwise nothing needs to be done
>     upstream=$(git for-each-ref --format='%(upstream)' "$branch")
>     test -z "$upstream" || exit 0
> 
>     # are we up-to-date? if so no need to do anything
>     test 0 = $(git rev-list "..$upstream" | wc -l) && exit 0
> 
>     # do we have something we made? if so no point trying to fast-forward
>     test 0 = $(git rev-list "$upstream.." | wc -l) || exit 0
> 
>     # attempt a fast-forward merge with it
>     git merge --ff-only @{upstream}
> 

This is a solution, I wouldn't get rid of the warnings though when
running git push.

Re: best way to fastforward all tracking branches after a fetch

[Gelonida N]

I forgot one other use case:

- wanting to pull from tracking branches without fastforwarding is not
such a smart idea.

of course I can do
git merge from  remotes/origin/branch
but this is more to type and would vary depending on whether 'd like to
pull from an unpushed tracking branch or from a freshly fetched tracking
branch.


On 12/12/2011 11:13 AM, Gelonida N wrote:
> Thanks for this rather long answer,
> 
> On 12/12/2011 09:09 AM, Junio C Hamano wrote:
>> Gelonida N <gelonida@gmail.com> writes:
>>
>>> What is the best way to fastforward all fastforwardable tracking
>>> branches after a git fetch?
>>
>> This lacks context and invites too many tangents, so I'll only touch a few
>> of them.
>>
>> First of all, why do you want to do this?
>>
> 
> To explain the scenario:
> - small project
> - every person works on master and multiple topic branches
>    and might alternate rather often
> - sometimes several persons work on the same topic branch
>   but most of the time not in parallel.
> - one person is working from several machines (starting work on
>   one and continuing on another)
> - additionally we do many pushed in order to be sure,
>   that our data is backed up in case of disk failures.
> - sometimes I just want to 'build' from a branch, that I am not
>    working on. but there I create mostly not even a tracking branch
> 
> before changing a machine I want to be sure to have pushed everything. I
> wanted to get rid of the warning, that some branches cannot be pushed,
> because they aren't fastforwarded
> 
> when checking out a branch I want to avoid, that I have to pull manually.
> 
> 
> 
>> In other words, wouldn't a post-checkout hook be a better place to do
>> this kind of thing, perhaps like this (completely untested)? 
>>
>>     #!/bin/sh
>>     old=$1 new=$2 kind=$3
>>
>>     # did we checkout a branch?
>>     test "$kind" = 1 || exit 0
>>
>>     # what did we check out?
>>     branch=$(git symbolic-ref HEAD 2>/dev/null) || exit 0
>>
>>     # does it track anything? otherwise nothing needs to be done
>>     upstream=$(git for-each-ref --format='%(upstream)' "$branch")
>>     test -z "$upstream" || exit 0
>>
>>     # are we up-to-date? if so no need to do anything
>>     test 0 = $(git rev-list "..$upstream" | wc -l) && exit 0
>>
>>     # do we have something we made? if so no point trying to fast-forward
>>     test 0 = $(git rev-list "$upstream.." | wc -l) || exit 0
>>
>>     # attempt a fast-forward merge with it
>>     git merge --ff-only @{upstream}
>>
> 
> This is a solution, I wouldn't get rid of the warnings though when
> running git push.
> 

Re: best way to fastforward all tracking branches after a fetch

[Hallvard Breien Furuseth]

Stefan Haller writes:
>Hallvard B Furuseth <h.b.furuseth@usit.uio.no> wrote:
>> Local branches can track each other.  So the script needs to toposort
>> the branches, or to loop until either nothing was done or an error
>> happened.  (The latter to prevent an eternal loop on error.)
> 
> Is this just theoretical, or are there real use cases for this? What
> would be a workflow with such a local tracking branch?

Personally I don't care much, I just noted that the script did not
match the question in the subject line.

-- 
Hallvard

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

[-- Attachment #1: Type: text/plain, Size: 2477 bytes --]

On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
> Hi,
> 
> What is the best way to fastforward all fastforwardable tracking
> branches after a git fetch?

I know this is a somewhat closed topic, but I took some time to
clean up a program I have been using for a while, including some
changes based upon ideas elsewhere in this thread.  The program
"git-branch-check" is attached, and requires perl > 5.10.0.

Note that this does a lot more than just fast-forward all
branches, although it can do that as well.

I alias it (in ~/.gitconfig) to 'bc', so I just run "git bc".
Running with "-h" shows usage:

    Usage: /home/sitaram/bin/git-branch-check [options] [branches]

    Check or fast forward branches.  Default: act upon all local branches if no
    arguments supplied, or just the current branch if '-c' is passed.
            -c      act upon current branch only
            -ff     don't just check, try to fast forward also
            -md     max diff (default 100; see below for details)
            -h      help
    'max diff':
        hide output for two branches different by more than so many commits

My usual usage is just "git bc -c", which may give me:

       1        pu...origin/pu
       1        pu...github/pu
      13        pu...master
           5    pu...q
           7    pu...vrs

This quickly tells me my 'pu' is one ahead of both my own
gitolite server as well as github's copy, and that it is 13
commits ahead of master.  The (unreleased and frequently
rebased) feature branches 'q' and 'vrs' are ahead of pu, which
means a rebase is not pending.  Without the "-c" I may see the
status of master versus its own upstream and other remotes,
etc., also.

The purpose of the max diff limit (default 100) is to hide, for
example, the pair 'master' and 'man' from the git.git repo.
Otherwise you'd see something like:

    27249 973    master...man

which is pretty meaningless.  The sum of those two numbers
should be less than the max.

"git bc -ff" will attempt to fast forward all selected branches
that are ancestors of their respective upstreams.  The current
branch will not be ff-ed if the tree is dirty, since you can't
do this by 'git branch -f'; it has to be an actual merge
command.

The output is not (currently) pipable to other programs because
I use colors (obtained from 'git config --get-color') and
currently it is not conditional on STDOUT being a tty.

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

[-- Attachment #1.1: Type: text/plain, Size: 2703 bytes --]

oops; forgot the program...

On Sat, Dec 17, 2011 at 03:40:09PM +0530, Sitaram Chamarty wrote:
> On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
> > Hi,
> > 
> > What is the best way to fastforward all fastforwardable tracking
> > branches after a git fetch?
> 
> I know this is a somewhat closed topic, but I took some time to
> clean up a program I have been using for a while, including some
> changes based upon ideas elsewhere in this thread.  The program
> "git-branch-check" is attached, and requires perl > 5.10.0.
> 
> Note that this does a lot more than just fast-forward all
> branches, although it can do that as well.
> 
> I alias it (in ~/.gitconfig) to 'bc', so I just run "git bc".
> Running with "-h" shows usage:
> 
>     Usage: /home/sitaram/bin/git-branch-check [options] [branches]
> 
>     Check or fast forward branches.  Default: act upon all local branches if no
>     arguments supplied, or just the current branch if '-c' is passed.
>             -c      act upon current branch only
>             -ff     don't just check, try to fast forward also
>             -md     max diff (default 100; see below for details)
>             -h      help
>     'max diff':
>         hide output for two branches different by more than so many commits
> 
> My usual usage is just "git bc -c", which may give me:
> 
>        1        pu...origin/pu
>        1        pu...github/pu
>       13        pu...master
>            5    pu...q
>            7    pu...vrs
> 
> This quickly tells me my 'pu' is one ahead of both my own
> gitolite server as well as github's copy, and that it is 13
> commits ahead of master.  The (unreleased and frequently
> rebased) feature branches 'q' and 'vrs' are ahead of pu, which
> means a rebase is not pending.  Without the "-c" I may see the
> status of master versus its own upstream and other remotes,
> etc., also.
> 
> The purpose of the max diff limit (default 100) is to hide, for
> example, the pair 'master' and 'man' from the git.git repo.
> Otherwise you'd see something like:
> 
>     27249 973    master...man
> 
> which is pretty meaningless.  The sum of those two numbers
> should be less than the max.
> 
> "git bc -ff" will attempt to fast forward all selected branches
> that are ancestors of their respective upstreams.  The current
> branch will not be ff-ed if the tree is dirty, since you can't
> do this by 'git branch -f'; it has to be an actual merge
> command.
> 
> The output is not (currently) pipable to other programs because
> I use colors (obtained from 'git config --get-color') and
> currently it is not conditional on STDOUT being a tty.



[-- Attachment #1.2: git-branch-check --]
[-- Type: text/plain, Size: 5677 bytes --]

#!/usr/bin/perl -s
use 5.10.0;
use strict;
use warnings;

# ----------------------------------------------------------------------

# bare-minimum subset of 'Tsh' (see github.com/sitaramc/tsh)
{
    my($rc, $text);
    sub rc { return $rc || 0; }
    sub text { return $text || ''; }
    sub lines { return split /\n/, $text; }
    sub try {
        my $cmd = shift; die "try: expects only one argument" if @_;
        $text = `( $cmd ) 2>&1; echo -n RC=\$?`;
        if ($text =~ s/RC=(\d+)$//) {
            $rc = $1;
            return (not $rc);
        }
        die "couldnt find RC= in result; this should not happen:\n$text\n\n...\n";
    }
}

# ----------------------------------------------------------------------

# options; the "-s" above sets one or more of these
# (the format of the lines below is special; it is used by usage() to generate
# help text for the options)
# BEGIN OPTIONS
    our $c;     # act upon current branch only
    our $ff;    # don't just check, try to fast forward also
    our $md;    # max diff (default 100; see below for details)
    our $h;     # help
# END OPTIONS
    $md ||= 100;

# get this over with; usage() exits so don't worry
    usage() if $h;

# get current branch
    my $current = '';
    try "git symbolic-ref HEAD" or die "DETACHED HEAD or no repo";
    ($current = (lines)[0]) =~ s(refs/heads/)();

# get branch names
    # first, all local branches as keys of a hash with upstream name if any, as the value
    my %upstream;
    try "git for-each-ref --perl '--format=\$upstream{%(refname:short)} = %(upstream:short);' refs/heads"
        or die "for-each-ref 1 failed";
    eval text;

    # local branches as a list; keep $current at the top, and the rest sorted
    my @local = ($current, grep { $_ ne $current } sort keys %upstream);

    # remote branches as a list
    try "git for-each-ref '--format=%(refname:short)' refs/remotes"
        or die "for-each-ref 2 failed";
    my @remote = lines;

# decide what branches to act upon.  Default: all local branches.  If any
# arguments are given, then those.  If '-c' is passed, only current branch.
    my @branches = @local;
    @branches = @ARGV if @ARGV;
    @branches = ($current) if $c;

# ----------------------------------------------------------------------

# show the tree state if it's dirty
    print "dirty:\n", text if dirty();

# process selected branches
    for my $b (@branches) {
        # attempt a fast-forward if -ff is passed
        ff($b, $upstream{$b}, $current) if ($ff);
        # check against its own upstream
        check($b, $upstream{$b});
        # then against all remote branches of the same name (I typically have
        # my own gitolite server as 'upstream' but also have github and google
        # code as additional remotes that I push my branches to)
        check($b, grep(m(^[^/]+/$b$), @remote));
    }
    # ...then against all local branches.  We do this in a separate loop
    # so their output is kept separate from the remote compares above.
    for my $b (@branches) {
        check($b, @local);
    }

# DONE...

# ----------------------------------------------------------------------
# subroutines
# ----------------------------------------------------------------------

sub ff {
    # b=branch, u=upstream, c=current
    my ($b, $u, $c) = @_;

    unless ($u) {
        say "$b does not have an upstream";
        return;
    }

    if ($b eq $c and dirty()) {
        say "working tree is dirty; skipping ff for (current branch) $b";
        return;
    }

    # $l = number of commits "l"eft side has over the "r"ight (similarly $r...)
    my($l, $r) = compare($b, $u);
    if ($r and not $l) {
        # there is something to update, and ff is possible
        if ($b eq $c) {
            # current branch; needs an actual merge
            try("git merge --ff-only $u") or die "$b: 'git merge --ff-only $u' failed:\n" .  text;
        } else {
            # other branches can be forced
            try("git branch -f $b $u") or die "$b: 'git branch -f $b $u' failed:\n" .  text;
        }
    }
}

sub check {
    my ($b, @list) = @_;
    state %seen;

    for my $u (@list) {
        next unless $u;
        next if $b eq $u or $seen{$b}{$u};
        # seeing a...b is as good as seeing b...a also
        $seen{$b}{$u} = 1;
        $seen{$u}{$b} = 1;

        my ($l, $r) = compare($b, $u);

        my $abs = $l + $r; next unless $abs;    # if they're equal, don't show it
        next if $abs >= $md;                    # if they're too far apart, don't show it

        print spacepad(4, $l) . color('green') . ($l || ' ');
        print spacepad(4, $r) . color('red')   . ($r || ' ');
        say color('reset') . "    $b...$u";
    }
}

sub compare {
    my ($b, $u) = @_;

    try("git rev-list $u..$b") or die "'git rev-list $u..$b' failed:\n" .  text;
    my $l = lines;

    try("git rev-list $b..$u") or die "'git rev-list $b..$u' failed:\n" .  text;
    my $r = lines;

    return($l, $r);
}

sub dirty {
    try "git status -s -uno | cut -c1-2 | sort | uniq -c; /./";
}

sub color {
    my $color = shift;
    return `git config --get-color "" $color`;
}

sub spacepad {
    return " " x ($_[0] - length($_[1]));
}

sub usage {
    print "
Usage: $0 [options] [branches]

Check or fast forward branches.  Default: act upon all local branches if no
arguments supplied, or just the current branch if '-c' is passed.
";
    @ARGV=($0);
    for ( grep { /BEGIN OPTIONS/../END OPTIONS/ and not /OPTION/ } <> ) {
        s/our \$/\t-/;
        s/; *#/\t/;
        print;
    }
    say "\'max diff':\n    hide output for two branches different by more than so many commits";
    exit 1;
}

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: best way to fastforward all tracking branches after a fetch

[Nazri Ramliy]

On Sat, Dec 17, 2011 at 6:11 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:
> oops; forgot the program...

This is nice!

Stick it on github, or somewhere, please, so that I can always get the
latest and greatest?

Thanks.

nazri

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

On Sat, Dec 17, 2011 at 3:40 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:
> On Sat, Dec 10, 2011 at 01:26:32PM +0100, Gelonida N wrote:
>> Hi,
>>
>> What is the best way to fastforward all fastforwardable tracking
>> branches after a git fetch?
>
> I know this is a somewhat closed topic, but I took some time to
> clean up a program I have been using for a while, including some

A month later and I find I have modified this program

Re: best way to fastforward all tracking branches after a fetch

[Sitaram Chamarty]

(sorry about that previous email; hit send in error.  Here's the complete one)

On Mon, Dec 19, 2011 at 12:01 PM, Nazri Ramliy <ayiehere@gmail.com> wrote:
> On Sat, Dec 17, 2011 at 6:11 PM, Sitaram Chamarty <sitaramc@gmail.com> wrote:
>> oops; forgot the program...
>
> This is nice!
>
> Stick it on github, or somewhere, please, so that I can always get the
> latest and greatest?
>
> Thanks.
>
> nazri

ok; it's on github: http://github.com/sitaramc/git-tools (they're all
standalone tools; you don't have to use the others)

The output is now colorised, and it will now also tell you, for each
remote you have, what branches you are hiding from them and what they
have which you have not started tracking locally.

a "screenshot" is at
http://sitaramc.github.com/git-tools/index.html#index_git_branch_check_

I notice I'm using it more and more, often even as a replacement for
'git status'... at least for repos where I'm juggling multiple
remotes.