RE: git pull on Linux/ACPI release tree

RE: git pull on Linux/ACPI release tree

[Brown, Len]

Linus,
I think Tony has articulated the work-flow problem that
originally started this thread, as well as the fix.

>I'll try to update the using-topic-branches document to capture this.
>Some of the problem is that it doesn't quite capture what I'm doing
>with my test/release branches.
>
>My release branch really is just used as a transfer point to Linus.
>I usually[1] don't leave patches sitting in "release" for long enough
>that I'll be tempted to merge in from Linus ... once I decide that
>some patches are ready to go to Linus I'll update "release" from Linus
>(which will be a fast-forward, so no history) merge in the topic
>branches, do one final sanity build, push to kernel.org and send
>the "please pull" e-mail.
>
>The huge majority of my "automatic update from upstream" merges
>go into my test branch ... which never becomes part of the real
>history as I never ask Linus to pull from it.
>
>-Tony
>
>[1] Sometimes I goof on this because I forget that I've applied
>a trivial patch directly to the release branch without going through
>a topic branch.  I think I'll fix my update script to check 
>for this case.

I figured that checking some trivial patches directly into "release"
would be a convenient way to make sure I didn't forget to push them --
as they didn't depend on anything else in my tree.  Okay.

To make sure that my test branch (where I generate my consolidated
plain patch, and what Andrew pulls) includes everything, I then pull
"release" into "test".  Still good.

But then I decide I need to update my test tree from upstream.
I did this by pulling "linus" into "release", and then pulling
"release" into "test".  This creates the book-keeping merge
in "release" that irritates gitk users.

This "flow", BTW, is a habit I picked up from the
"two-phase release strategy" that we used in bk days.
There I'd pull from upstream down into my to-linus tree and then pull
from the to-linus tree into the to-andrew tree.
I expect BK also created a merge cset, but apparently
nobody was looking at the history like they do with gitk today.

So if I simply don't pull from "linus" into a modified
"release" branch then the cluttered history issue goes away.
I should fetch "linus" into "release" right before I merge
the topic branches into "release" and push upstream.
The fetch is a clean fast-forward, and the merges all have
real content.

This will work as long as "release" doesn't get too old
to be pulled upstream without conflicts.  Based on past
experience with low latency pulls upstream, I think this will be rare.

Andrew will still get cluttered history in the test tree,
but as he's focused on the content and not the (throw-away) history,
this is surely a non-issue.

So problem #1 is solved, yes?

Going forward...
I'm hopeful that gitk users will not be irritated also
by the liberal use of topic branches.  I'm starting to like using
them quite a bit.  Yes, it is true that I could cherry-pick
the topics out of their original context to re-manufacture linear
history.  But that is extra work.  Also, as you poined out,
there is real value in the real history because the context is accurate.
Further, I find that sometimes I need to augment a topic branch
with a follow-up patch.  I can checkout the topic branch an plop
the follow-up right on the tip where it logically should live,
and (Tony's) scripts will remind me when the branch is not fully
pulled into test or release -- so it will never get misplaced.

In the case where a topic branch is a single commit, gitk users
will see both the original commit, as well as the merge commit
back into "release".

-Len

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Brown, Len wrote:
> >
> >The huge majority of my "automatic update from upstream" merges
> >go into my test branch ... which never becomes part of the real
> >history as I never ask Linus to pull from it.
> >
> >-Tony
> >
> >[1] Sometimes I goof on this because I forget that I've applied
> >a trivial patch directly to the release branch without going through
> >a topic branch.  I think I'll fix my update script to check 
> >for this case.
> 
> I figured that checking some trivial patches directly into "release"
> would be a convenient way to make sure I didn't forget to push them --
> as they didn't depend on anything else in my tree.  Okay.

One thing we could do is to make it easier to apply a patch to a 
_non_current_ branch.

In other words, let's say that we want to encourage the separation of a 
"development branch" and a "testing and use" branch (which I'd definitely 
personally like to encourage people to do).

And one way to do that might be to teach "git-apply" to apply patches to a 
non-active branch, and then you keep the "testing and use" branch as your 
_checked_out_ branch (and it's going to be really dirty), but when you 
actually apply patches you could do that to the "development" branch with 
something like

	git-apply -b development < patch-file

(Now, of course, that's only if you apply somebody elses patch - if you 
actually do development _yourself_, you'd either have to check out the 
development branch and do it there, or you'd move the patch you have in 
your "ugly" checked-out testing branch into the development branch with

	git diff | git-apply -b development

or something similar..)

Then you could always do "git pull . development" to pull in the 
development stuff into your working branch - keeping the development 
branch clean all the time.

Do you think that kind of workflow would be more palatable to you? It 
shouldn't be /that/ hard to make git-apply branch-aware... (It was part of 
my original plan, but it is more work than just using the working 
directory, so I never finished the thought).

> I'm hopeful that gitk users will not be irritated also
> by the liberal use of topic branches.

"gitk" is actually pretty good at showing multiple branches. Try doing a

	gitk --all -d

and you'll see all the topic branches in date order. The "-d" isn't 
strictly necessary, and to some degree makes the output messier by 
interleaving the commits from different branches, so you may not want to 
do it, but it is sometimes nice to see the "relative dates" of individual 
commits rather than the denser format that gitk defaults to.

> In the case where a topic branch is a single commit, gitk users
> will see both the original commit, as well as the merge commit
> back into "release".

Yes, topic branches will always imply more commits, but I think they are 
of the "nice" kind.

I definitely encourage people to use git as a distributed concurrent 
development system ratehr than the "collection of patches" thing. Quilt is 
much better at the collection of patches. 

So I'd encourage topic branches - even within something like ACPI, you 
might have separate topics ("interpreter" branch vs "x86" branch vs 
"generic-acpi" branch).

And yes, that will make history sometimes messier too, and it will cause 
more merges, but the difference there is that the merges will be 
meaningful (ie merging the "acpi interpreter" branch into the generic ACPI 
branch suddenly has _meaning_, even if there only ends up being a couple 
of commits per merge).

Ok?

		Linus

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Linus Torvalds wrote:
>
> One thing we could do is to make it easier to apply a patch to a 
> _non_current_ branch.
>   [ ... ]
> Do you think that kind of workflow would be more palatable to you? It 
> shouldn't be /that/ hard to make git-apply branch-aware... (It was part of 
> my original plan, but it is more work than just using the working 
> directory, so I never finished the thought).

Btw, this is true in a bigger sense: the things "git" does have largely 
been driven by user needs. Initially mainly mine, but things like 
"git-rebase" were from people who wanted to work as "sub-maintainers" (eg 
Junio before he became the head honcho for git itself).

But if there are workflow problems, let's try to fix them. The "apply 
patches directly to another branch" suggestion may not be sane (maybe it's 
too confusing to apply a patch and not actually see it in the working 
tree), but workflow suggestions in general are appreciated.

We've made switching branches about as efficient as it can be (but if the 
differences are huge, the cost of re-writing the working directory is 
never going to be low). But switching branches has the "confusion factor" 
(ie you forget which branch you're on, and apply a patch to your working 
branch instead of your development branch), so maybe there are other ways 
of doing the same thing that might be sensible..

So send suggestions to the git lists. Maybe they're insane and can't be 
done, but while I designed git to work with _my_ case (ie mostly merging 
tons of different trees and then having occasional big batches of 
patches), it's certainly _supposed_ to support other maintainers too..

		Linus

Re: git pull on Linux/ACPI release tree

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> One thing we could do is to make it easier to apply a patch to a 
> _non_current_ branch.
>...
> And one way to do that might be to teach "git-apply" to apply patches to a 
> non-active branch,...
>
> 	git diff | git-apply -b development
>
> or something similar..)
>
> Then you could always do "git pull . development" to pull in the 
> development stuff into your working branch - keeping the development 
> branch clean all the time.

I had to do something like that last night, when I hacked on
gitweb.  gitweb as shipped does not work for anybody but kay
(e.g. it has /home/kay hardcoded in it).  So I did:

	$ git clone git://git.kernel.org/pub/scm/git/gitweb gitweb
        $ cd gitweb
        $ git checkout -b custom
        $ edit gitweb.cgi ;# adjust /home/kay -> somewhere else etc.
        $ git commit -a -m "customization for junio's home"

Then I started preparing a proposed fix for Kay:

	$ git checkout -b symref master
        $ edit gitweb.cgi
        $ git commit -a -s -m "make it work on symref repository"

Now the thing is that I cannot test symref branch as is.  I
deliberately omitted the change necessary to make the upstream
work on my local machine from that branch, because I want to
keep my home-machine customization separate from what I will
eventually feed Kay.  So I do a throwaway test branch:

	$ git checkout -b test master
        $ git pull . custom symref ;# an octopus ;-)
        # I could have done two separate pulls, custom then symref.

The interesting part starts here.  Inevitably, I find bugs and
bugs and bugs in the test branch, and I fix them in the working
tree, without committing.  Eventually things starts working.
I did not commit here in the test branch, because the symref
branch is where I intend to keep this set of changes.  So
instead, I did this:

	$ git diff HEAD >P.diff
        $ git checkout -f symref
        $ git reset --soft HEAD^
        $ git apply P.diff
        $ git commit -a -C ORIG_HEAD

Usually I strongly discourage people to use "checkout -f"
because it will leave files that are in the current branch but
not in the new branch behind in the working tree.  Here I used
"checkout -f symref" because I knew this is a one-file project.

Instead of fixing the symref commit in place like this, I could
have committed P.diff as a separate "fixup" commit on top of the
symref branch, in which case the above sequence would have been:

	$ git diff HEAD >P.diff
        $ git checkout -f symref
        $ git apply P.diff
        $ git commit -a -m 'fixup bugs in the previous.'

but I did not --- it would have been more disgusting than
honest.

And after that, the usual format-patch:

	$ git format-patch origin..symref

In either case, this *was* cumbersome.  And I did it twice for
two independent topics.  Admittedly, these topic branches were
both single-commit topics, and in real life your subsystem
maintainers must be facing bigger mess than this toy experience
of mine, but the principle is the same.

I think there are a couple of ways to improve what I had to do.
I'll think aloud here.  The fictitious transcripts all start
after I got things working in the test branch working tree, with
a clean index file (i.e. changes are in the working tree only).

1. Make a commit in the "test" branch, and then cherry-pick the
   commit back to the topic branch:

	$ git commit -a -m "Fix symref fix"
        $ git checkout symref
        $ git cherry-pick -r test

2. Fix "git checkout <branch>" so that it does a reasonable thing
   even when a dirty path is different in current HEAD and
   destination branch.  Then I could:

	$ git checkout symref ;# this would not work in the current git
	    # it would die like this:
            # $ git checkout symref
            # fatal: Entry 'gitweb.cgi' not uptodate. Cannot merge.
	$ git diff ;# just to make sure inevitable automated merge
		    # did the right thing
        $ git commit -a -m "Fix symref fix"
	    # I could collapse them into one instead, like this:
	    # $ git reset --soft HEAD^
	    # $ git commit -a -C ORIG_HEAD

To retest (possibly with latest from Kay), we can rebuild the
test branch from scratch since it is by definition a throwaway
branch and never is exposed to public:

        $ git fetch origin
	$ git checkout test
        $ git reset --head origin
        $ git pull . custom symref

Obviously I prefer to have #2 work well, but #1 would work today.

I am not sure if making "git-apply" to take different branch is
a sane approach.  It might make sense to teach git-applymbox and
git-am about branches, though.  So is teaching git-merge about
merging into different branch.

RE: git pull on Linux/ACPI release tree

[Luben Tuikov]

--- Linus Torvalds <torvalds@osdl.org> wrote:
> But if there are workflow problems, let's try to fix them. The "apply 
> patches directly to another branch" suggestion may not be sane (maybe it's 
> too confusing to apply a patch and not actually see it in the working 
> tree), but workflow suggestions in general are appreciated.

This is sensible, thank you.

A very general workflow I've seen people use is more/less as
I outlined in my previous email:

  tree A  (linus' or trunk)
     Project B  (Tree B)
        Project C  (Tree C, depending on stuff in Project B)

Now this could be how the "managers" see things, but development,
could've "cloned" from Tree B and Tree C further, as is often
customary to have a a) per user tree, or b) per bug tree.

So pull/merge/fetch/whatever follows Tree A->B->C.

It is sensible to have another tree say, called something
like "for_linus" or "upstream" or "product" which includes
what has accumulated in C from B and in B from A, (eq diff(C-A)).
I.e. a "push" tree.  So that I can tell you, "hey,
pull/fetch/merge/whatever the current verb en vogue is, from
here to get latest xyz".

What I also wanted to mention is that Tree B undeniably
depends on the _latest_ state of Tree A, since Project B
uses API/behaviour of the code in Tree A, so one cannot just
say they are independent.  Similarly for Tree C/Project C,
is dependent on B, and dependent on A.

Also sometimes a bugfix in C, prompts a bugfix in A,
so that the bugfix in A doesn't apply unless the bugfix in C.
(To get things more complicated.)

I think this is more/less the most easier to see, understand and
follow workflow approach, which is also the case for other SCMs.

What are the commands to follow to make everyone happy when
pulling from such a development process?

FWIW, "git diff A C | send to Linus" would get you the
"no merge messages/ancestors I want to see" idea, if I understand
this thread correctly.

> We've made switching branches about as efficient as it can be (but if the 
> differences are huge, the cost of re-writing the working directory is 
> never going to be low). But switching branches has the "confusion factor" 
> (ie you forget which branch you're on, and apply a patch to your working 
> branch instead of your development branch), so maybe there are other ways 
> of doing the same thing that might be sensible..

Yes.  Ever since I started used git, I never used branch
switching, but I do have git branches and I do use git branching.

I basically have a branch per directory, whereby the object db
is shared as is remotes/refs/etc, HEAD and index are not shared
of course.

This allows me to do a simple and fast "cd" to change/go to a
different branch, since they are in different directories.
So the time I wait to switch branches is the time the filesystem
takes to do a "cd".

This also allows me to build/test/patch/work on branches
simultaneously.

Thank you,
   Luben

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Luben Tuikov wrote:
> 
> Yes.  Ever since I started used git, I never used branch
> switching, but I do have git branches and I do use git branching.
> 
> I basically have a branch per directory, whereby the object db
> is shared as is remotes/refs/etc, HEAD and index are not shared
> of course.
> 
> This allows me to do a simple and fast "cd" to change/go to a
> different branch, since they are in different directories.
> So the time I wait to switch branches is the time the filesystem
> takes to do a "cd".
> 
> This also allows me to build/test/patch/work on branches
> simultaneously.

Yes. It has many advantages, and it's the approach I pushed pretty hard 
originally, but the "many branches in the same tree" approach seems to 
have become the more common one. Using many branches in the same tree is 
definitely the better approach for _distribution_, but that doesn't 
necessarily mean that it's the better one for development.

For example, you can have a git distribution tree with 20 different 
branches on kernel.org, but do development in 20 different trees with just 
one branch active - and when you do a "git push" to push out your branch 
in your development tree, it just updates that one branch on the 
distribution site.

So git certainly supports that kind of behaviour, but nobody I know 
actually does it that way (not even me, but since I tend to just merge 
other peoples code, I don't actually have multiple branches: I create 
temporary branches for one-off things, but don't maintain them that way).

			Linus

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/10/06, Linus Torvalds <torvalds@osdl.org> wrote:
> Using many branches in the same tree is
> definitely the better approach for _distribution_, but that doesn't
> necessarily mean that it's the better one for development.
(...)
> So git certainly supports that kind of behaviour, but nobody I know
> actually does it that way

Hrm! We do. http://locke.catalyst.net.nz/gitweb?p=moodle.git;a=heads
shows a lot of heads that share 99% of the code. The repo is ~90MB --
and we check each head out with cogito, develop and push. It is a
shared team repo, using git+ssh and sticky gid and umask 002.

Works pretty well I have to add. The only odd thing is that the
fastest way to actually start working on a new branch is to ssh on to
the server and cp moodle.git/refs/heads/{foo,bar} and then cg-clone
that bar branch away. Perhaps I should code up an 'cg-branch-add
--in-server' patch.

regards,


martin

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Luben Tuikov wrote:
> 
> A very general workflow I've seen people use is more/less as
> I outlined in my previous email:
> 
>   tree A  (linus' or trunk)
>      Project B  (Tree B)
>         Project C  (Tree C, depending on stuff in Project B)
> 
> Now this could be how the "managers" see things, but development,
> could've "cloned" from Tree B and Tree C further, as is often
> customary to have a a) per user tree, or b) per bug tree.
> 
> So pull/merge/fetch/whatever follows Tree A->B->C.
> 
> It is sensible to have another tree say, called something
> like "for_linus" or "upstream" or "product" which includes
> what has accumulated in C from B and in B from A, (eq diff(C-A)).
> I.e. a "push" tree.  So that I can tell you, "hey,
> pull/fetch/merge/whatever the current verb en vogue is, from
> here to get latest xyz".
> 
> What I also wanted to mention is that Tree B undeniably
> depends on the _latest_ state of Tree A, since Project B
> uses API/behaviour of the code in Tree A, so one cannot just
> say they are independent.  Similarly for Tree C/Project C,
> is dependent on B, and dependent on A.

Note that in the case where the _latest_ state of the tre you are tracking 
really matters, then doing a "git pull" is absolutely and unquestionably 
the right thing to do. 

So if people thought that I don't want to have sub-maintainers pulling 
from my tree _at_all_, then that was a mis-communication. I don't in any 
way require a linear history, and criss-cross merges are supported 
perfectly well by git, and even encouraged in those situations.

After all, if tree B starts using features that are new to tree A, then 
the merge from A->B is required for functionality, and the synchronization 
is a fundamental part of the history of development. In that cases, the 
history complexity of the resulting tree is a result of real development 
complexity.

Now, obviously, for various reasons we want to avoid having those kinds of 
linkages as much as possible. We like to have develpment of different 
subsystems as independent as possible, not because it makes for a "more 
readable history", but because it makes it a lot easier to debug - if we 
have three independent features/development trees, they can be debugged 
independently too, while any linkages inevitably also mean that any bugs 
end up being interlinked..

		Linus

Re: git pull on Linux/ACPI release tree

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> Now, obviously, for various reasons we want to avoid having those kinds of 
> linkages as much as possible. We like to have develpment of different 
> subsystems as independent as possible, not because it makes for a "more 
> readable history", but because it makes it a lot easier to debug - if we 
> have three independent features/development trees, they can be debugged 
> independently too, while any linkages inevitably also mean that any bugs 
> end up being interlinked..
>
> 		Linus

Yes.  If subproject B uses new features from A (either upstream
or sibling subproject), pulling A into B is inevitable.

On the other hand, if such merges becomes too frequent, it may
be a sign that A's feature set and interface is still changing
too rapidly for downstream use, but developers A and B are not
communicating well and B has not noticed that B might be better
off taking a break, addressing other non-overlapping areas while
giving a bit of time for A to settle things down.

An SCM is just _one_ of the ways for developers to communicate,
it will never be a replacement for developer communication.

Re: git pull on Linux/ACPI release tree

[Kyle Moffett]

On Jan 09, 2006, at 21:50, Linus Torvalds wrote:
> if we  have three independent features/development trees, they can  
> be debugged independently too, while any linkages inevitably also  
> mean that any bugs end up being interlinked..

One example:

If I have ACPI, netdev, and swsusp trees change between an older  
version and a newer one, and my net driver starts breaking during  
suspend, I would be happiest debugging with the following set of  
patches/trees (Heavily simplified):

            ^
            |
           [5]
            |
          broken
         ^  ^   ^
       [2] [3]  [4]
       /    |     \
netdev3  acpi3   swsusp3
    ^       ^        ^
    |       |        |
netdev2  acpi2   swsusp2
    ^       ^        ^
    |       |        |
netdev1  acpi1   swsusp1
       ^    ^    ^
        \   |   /
         \  |  /
          \ | /
           \|/
            |
           [1]
            |
          works


If the old version [1] works and the new one [5] doesn't, then I can  
immediately test [2], [3], and [4].  If one of those doesn't work,  
I've identified the problematic patchset and cut the debugging by  
2/3.  If they all work, then we know precisely that it's the  
interactions between them, which also makes debugging a lot easier.

Cheers,
Kyle Moffett

--
There are two ways of constructing a software design. One way is to  
make it so simple that there are obviously no deficiencies. And the  
other way is to make it so complicated that there are no obvious  
deficiencies.  The first method is far more difficult.
   -- C.A.R. Hoare


-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/10/06, Kyle Moffett <mrmacman_g4-ee4meeAH724@public.gmane.org> wrote:
> If they all work, then we know precisely that it's the
> interactions between them, which also makes debugging a lot easier.

The more complex your tree structure is, the more the interactions are
likely to be part of the problem. Is git-bisect not useful in this
scenario?

cheers,


martin
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Alex Riesen]

On 1/9/06, Junio C Hamano <junkio-j9pdmedNgrk@public.gmane.org> wrote:
> 2. Fix "git checkout <branch>" so that it does a reasonable thing
>    even when a dirty path is different in current HEAD and
>    destination branch.  Then I could:
>
>         $ git checkout symref ;# this would not work in the current git
>             # it would die like this:
>             # $ git checkout symref
>             # fatal: Entry 'gitweb.cgi' not uptodate. Cannot merge.

That is actually very interesting. I already wished sometimes to be
able to switch branches with a dirty working directory (and usually
ended up with git diff+checkout+apply).
Even if it results in a merge and conflict markers in files it looks
like a very practical idea!

>         $ git diff ;# just to make sure inevitable automated merge
>                     # did the right thing
>         $ git commit -a -m "Fix symref fix"
>             # I could collapse them into one instead, like this:
>             # $ git reset --soft HEAD^
>             # $ git commit -a -C ORIG_HEAD
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Kyle Moffett]

On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> On 1/10/06, Kyle Moffett <mrmacman_g4@mac.com> wrote:
>> If they all work, then we know precisely that it's the  
>> interactions between them, which also makes debugging a lot easier.
>
> The more complex your tree structure is, the more the interactions  
> are likely to be part of the problem. Is git-bisect not useful in  
> this scenario?

IIRC git-bisect just does an outright linearization of the whole tree  
anyways, which makes git-bisect work everywhere, even in the presence  
of difficult cross-merges.  On the other hand, if you are git- 
bisecting ACPI changes (perhaps due to some ACPI breakage), and ACPI  
has 10 pulls from mainline, you _also_ have to wade through the  
bisection of any other changes that occurred in mainline, even if  
they're totally irrelevant.  This is why it's useful to only pull  
mainline into your tree (EX: ACPI) when you functionally depend on  
changes there (as Linus so eloquently expounded upon).

Cheers,
Kyle Moffett

--
Q: Why do programmers confuse Halloween and Christmas?
A: Because OCT 31 == DEC 25.

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Kyle Moffett wrote:
>
> On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> > 
> > The more complex your tree structure is, the more the interactions are
> > likely to be part of the problem. Is git-bisect not useful in this scenario?
> 
> IIRC git-bisect just does an outright linearization of the whole tree anyways,
> which makes git-bisect work everywhere, even in the presence of difficult
> cross-merges.

It's not really a linearization - at no time does git-bisect _order_ the 
commits. After all, no linear order actually exists. 

Instead, it really cuts the tree up into successively smaller parts. 

Think of it as doing a binary search in a 2-dimensional surface - you 
can't linearize the plane, but you can decide to test first one half of 
the surface, and then depending on whether it was there, you can halve 
that surface etc.. 

> On the other hand, if you are git-bisecting ACPI changes
> (perhaps due to some ACPI breakage), and ACPI has 10 pulls from mainline, you
> _also_ have to wade through the bisection of any other changes that occurred
> in mainline, even if they're totally irrelevant.

Yes. Although if you _know_ that the problem happened in a specific file 
or specific subdirectory, you can actually tell "git bisect" to only 
bother with changes to that file/directory/set-of-directories to speed up 
the search.

IOW, if you absolutely know that it's ACPI-related, you can do something 
like

	git bisect start drivers/acpi arch/i386/kernel/acpi

to tell the bisect code that it should totally ignore anything that 
doesn't touch those two directories.

However, if it turns out that you were wrong (and the ACPI breakage was 
brought on by something that changed something else), "git bisect" will 
just get confused and report the wrong commit, so this is really something 
you should be careful with (and verify the end result by checking that 
undoing that _particular_ commit really fixes things).

And yes, "git bisect" _will_ work with bugs that depend on two branches of 
a merge: it will point to the merge commit itself as being the problem. 
Now, at that point you really are screwed, and you'll have to figure out 
why both branches work, but the combination of them do not.

Maybe it's as simple as just a merge done wrong (bad manual fixups), but 
maybe it's a perfectly executed merge that just happens to have one branch 
changing the assumptions that the other branch depended on.

Happily, that is not very common. I know people are using "git bisect", 
and I don't think anybody has ever reported it so far. It will happen 
eventually, but I'd actually expect it to be much more common that "git 
bisect" will hit other - worse - problems, like bugs that "come and go", 
and that a simple bisection simply cannot find because they aren't 
totally repeatable.

			Linus

Re: git pull on Linux/ACPI release tree

[Johannes Schindelin]

Hi,

On Tue, 10 Jan 2006, Linus Torvalds wrote:

> 
> On Tue, 10 Jan 2006, Kyle Moffett wrote:
> >
> > On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> > > 
> > > The more complex your tree structure is, the more the interactions are
> > > likely to be part of the problem. Is git-bisect not useful in this scenario?
> > 
> > IIRC git-bisect just does an outright linearization of the whole tree anyways,
> > which makes git-bisect work everywhere, even in the presence of difficult
> > cross-merges.
> 
> It's not really a linearization - at no time does git-bisect _order_ the 
> commits. After all, no linear order actually exists. 
> 
> Instead, it really cuts the tree up into successively smaller parts. 
> 
> Think of it as doing a binary search in a 2-dimensional surface - you 
> can't linearize the plane, but you can decide to test first one half of 
> the surface, and then depending on whether it was there, you can halve 
> that surface etc.. 

How?

If you bisect, you test a commit. If the commit is bad, you assume *all* 
commits before that as bad. If it is good, you assume *all* commits after 
that as good.

Now, if you have a 2-dimensional surface, you don't have a *point*, but 
typically a *line* separating good from bad.

Further, the comparison with 2 dimensions is particularly bad. You 
*have* partially linear development lines, it got *nothing* to do with 
an area. The commits still make up a *list*, and it depends how you 
*order* that list for bisect. (And don't tell me they are not ordered: 
they are.)

If you order the commits by date, you don't get anything meaningful point 
before which it is bad, and after which it is good.

So, how is bisect supposed to work if you don't have one straight 
development line from bad to good?

Ciao,
Dscho


-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Johannes Schindelin wrote:
> > 
> > Think of it as doing a binary search in a 2-dimensional surface - you 
> > can't linearize the plane, but you can decide to test first one half of 
> > the surface, and then depending on whether it was there, you can halve 
> > that surface etc.. 
> 
> How?
> 
> If you bisect, you test a commit. If the commit is bad, you assume *all* 
> commits before that as bad. If it is good, you assume *all* commits after 
> that as good.

No, that's not how bisect works at all.

It's true that if a commit is bad, then all the commits _reachable_ from 
that commit are considered bad. 

And it's true that if a commit is good, then all commits that _reach_ that 
commit are considered good.

But that doesn't mean that there is an ordering. The commits that fall 
into the camp of being "neither good nor bad" are _not_ ordered. There are 
commits in there that are not directly reachable from the good commit.

> Now, if you have a 2-dimensional surface, you don't have a *point*, but 
> typically a *line* separating good from bad.

Exactly. 

And a git graph is not really a two-dimensional surface, but exactly was 
with a 2-dimensional surface, it is _not_ enough to have a *point* to 
separate the good from bad.

You need to have a _set of points_ to separate the good from the bad. You 
can think of it as a line that bisects the surface: if you were to print 
out the development graph, the set of points literally _do_ form a virtual 
line across the development surface.

(Actually, you can't in general print out the development graph on a 
2-dimensional paper without having development lines that cross each 
other, but you could actually do it in three dimensions, where the 
"boundary" between good and bad is actually a 2-dimensional surface in 
3-dimensional space).

But to describe the surface of "known good", you actually just need a list 
of known good commits, and the "commits reachable from those commits" 
_becomes_ the surface.

> Further, the comparison with 2 dimensions is particularly bad.

No it is not. It's a very good comparison.

In a linearized model (one-dimensional, fully ordered set), the only thing 
you need for bisection is two points: the beginning and the end.

In the git model, you need _many_ points to describe the area being 
bisected. Exactly the same way as if you were to bisect a 2-dimensional 
surface.

Now, the git history is _not_ really a two-dimensional surface, so it's 
just an analogy, not an exact identity. But from a visualization 
standpoint, it's a good way to think of each "git bisect" as adding a 
_line_ on the surface rather than a point on a linear line.

> So, how is bisect supposed to work if you don't have one straight 
> development line from bad to good?

Read the code.

I'm pretty proud of it. It's simple, and it's obvious once you think about 
it, but it is pretty novel as far as I know. BK certainly had nothing 
similar, not have I heard of anythign else that does it. Git _might_ be 
the first thing that has ever done it, although it's simple enough that I 
wouldn't be surprised if others have too.

			Linus

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Linus Torvalds wrote:
> 
> Now, the git history is _not_ really a two-dimensional surface, so it's 
> just an analogy, not an exact identity. But from a visualization 
> standpoint, it's a good way to think of each "git bisect" as adding a 
> _line_ on the surface rather than a point on a linear line.

Actually, the way I think of it is akin to the "light cones" in physics. A 
point in space-time doesn't define a fully ordered "before and after": but 
it _does_ describe a "light cone" which tells you what is reachable from 
that point, and what that point reaches. Within those cones, that 
particular point ("commit") has a strict ordering.

And exactly as in physics, in git there's a lot of space that is _not_ 
ordered by that commit. And the way to bisect is basically to find the 
right points in "git space" to create the right "light cone" that you 
find the point where the git space that is reachable from that commit has 
the same volume as the git space that isn't reachable.

And maybe that makes more sense to you (if you're into physics), or maybe 
it makes less sense to you.

Now, since we always search the "git space" in the cone that is defined by 
"reachable from the bad commit, but not reachable from any good commit", 
the way we handle "bad" and "good" is actually not a mirror-image. If we 
fine a new _bad_ commit, we know that it was reachable from the old bad 
commit, and thus the old bad commit is now uninteresting: the new bad 
commit forms a "past light cone" that is a strict subset of the old one, 
so we can totally discard the old bad commit from any future 
consideration. It doesn't tell us anything new.

In contrast, if we find a new _good_ commit, the "past light cone" (aka 
"set of commits reachable from it") is -not- necessarily a proper superset 
of the previous set of good commits, so when we find a good commit, we 
still need to carry the _other_ good commits around, and the "known good" 
universe is the _union_ of all the "good commit past lightcones".

Then the "unknown space" is the set difference of the "past lightcone of 
the bad commit" and of this "union of past lightcones of good commits". 
It's the space that is reachable from the known-bad commit, but not 
reachable from any known-good commit.

So this means that when doing bisection, what we want to do is find the 
point in git space that has _new_ "reachability" within that unknown space 
that is as close to half that volume as space as possible. And that's 
exactly what "git-rev-list --bisect" calculates.

So every time, we try to either move the "known bad" light-cone down in 
time in the unknown space, _or_ we add a new "known good" light-cone. In 
either case, the "unknown git space" keeps shrinking by half each time.

("by half" is not exact, because git space is not only quanticized, it 
also has a rather strange "distance function". In other words, we're 
talking about a rather strange space. The good news is that the space is 
small enough that we can just enumerate every quantum and simply 
calculate the volume it defines in that space. IOW, we do a very 
brute-force thing, and it works fine).

			Linus

Re: git pull on Linux/ACPI release tree

[Johannes Schindelin]

Hi,

[cut down the Cc: list, since this is getting special]

On Tue, 10 Jan 2006, Linus Torvalds wrote:

> > If you bisect, you test a commit. If the commit is bad, you assume *all* 
> > commits before that as bad. If it is good, you assume *all* commits after 
> > that as good.
> 
> No, that's not how bisect works at all.

Okay, so I got that wrong. But for a good reason: this is not the meaning 
of bisection in my lectures. Doesn't matter.

> It's true that if a commit is bad, then all the commits _reachable_ from 
> that commit are considered bad. 
> 
> And it's true that if a commit is good, then all commits that _reach_ that 
> commit are considered good.
> 
> But that doesn't mean that there is an ordering. The commits that fall 
> into the camp of being "neither good nor bad" are _not_ ordered. There are 
> commits in there that are not directly reachable from the good commit.

Those commits not reachable from the good commit are of no interest. Let's 
just ignore them.

> > Now, if you have a 2-dimensional surface, you don't have a *point*, but 
> > typically a *line* separating good from bad.
> 
> Exactly. 
> 
> And a git graph is not really a two-dimensional surface, but exactly was 
> with a 2-dimensional surface, it is _not_ enough to have a *point* to 
> separate the good from bad.
> 
> You need to have a _set of points_ to separate the good from the bad. You 
> can think of it as a line that bisects the surface: if you were to print 
> out the development graph, the set of points literally _do_ form a virtual 
> line across the development surface.

Okay, so there is a cut: Every directed path from good to bad has a single 
commit which is the first bad. Let's call the set of all such bad commits 
the cut set.

Is git-bisect capable of identifying all of the cut set, or just a single 
one?

> > Further, the comparison with 2 dimensions is particularly bad.
> 
> No it is not. It's a very good comparison.

>From your explanation I understand now why you like that comparison.

> > So, how is bisect supposed to work if you don't have one straight 
> > development line from bad to good?
> 
> Read the code.
> 
> I'm pretty proud of it.

I bet nobody can tell ;-)

Well, I read the code. And I answer my own question from 18--19 lines ago:

git-bisect is not capable of identifying the cut set, but pretends that 
there really is only one bad commit (see bisect_bad()).

That may be the best choice if all commits in the cut set except one are 
merges. (It is the best if the cut set contains only one element.)

But I see two problems with that:

- a problem can be introduced independently in two different branches, and
  occur in both of them before the merge (in which case bisect only 
  catches one of the commits), and

- AFAICT if the cut set is one merge and one regular commit, bisect could
  identify the merge by error.

Of course, all this makes only a difference if the bisect has to cross a 
merge.

BTW I think there is a thinko in git-rev-list.txt:

> Thus, if 'git-rev-list --bisect foo ^bar ^baz' outputs 'midpoint', the 
> output of 'git-rev-list foo ^midpoint' and 'git-rev-list midpoint ^bar 
             ^ this should be
	'git-rev-list foo ^midpoint ^bar ^baz'
> ^baz' would be of roughly the same length

Ciao,
Dscho

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Johannes Schindelin wrote:
> 
> Those commits not reachable from the good commit are of no interest. Let's 
> just ignore them.

Note that to avoid confusion, start talking about -multiple- good commits 
early.

So we have a list of "known good islands" in the git-space. And yes, we 
want to ignore anything that is reachable from them.

And here the magic part of"git-bisect.sh" is around line 133:

	... --not $(cd "$GIT_DIR" && ls refs/bisect/good-*) ...

It tells git-rev-parse to generate a list of commits that we're _not_ 
interested in, and that list will be one of the most critical parts of the 
stuff we give to "git-rev-list --bisect".

So that part of the script is literally the part that says "ignore all of 
git space that is reachable from the good commits", because we've listed 
all the good commits as refs named "refs/bisect/good-*".

> > You need to have a _set of points_ to separate the good from the bad. You 
> > can think of it as a line that bisects the surface: if you were to print 
> > out the development graph, the set of points literally _do_ form a virtual 
> > line across the development surface.
> 
> Okay, so there is a cut: Every directed path from good to bad has a single 
> commit which is the first bad. Let's call the set of all such bad commits 
> the cut set.

This set is uninteresting for two reasons:
 - it's hard to calculate
 - it's not the answer we want.

We want the _single_ commit that is the one that generates your "cut set".

Your "cut set" is really the "reachability border" from the single bad 
commit we're interested in to all the possible development lines.

In practice, the "cut set" is just the "bad commit" plus all the merges 
that merge that bad commit with somethign that wasn't reacable from it in 
the first place.

So the "cut set" isn't interesting.

> git-bisect is not capable of identifying the cut set, but pretends that 
> there really is only one bad commit (see bisect_bad()).

Not quite.

It could keep track of all bad commits (in fact, it does so in the log 
file), but the fact is, none but the lastest bad commit we have found 
matters.

By definition, "git bisect" is always going to test a commit that is 
reachable from the previously known bad commit. Agreed? Anything else 
would be insane - we know that we had a bad stat, and we're interested in 
finding out how _that_ bad state happened, so we're only ever interested 
in commits that are ancestors to that bad state.

So our search-space is _literally_ defined by two things:

 - the surface of "known good" commits (which defines the commits that 
   aren't interesting). 

   This is the "--not refs/bisect/good-*" part

 - the last "known bad" commit.

We'll always search the git commit space defined by these two knowns, 
agreed?

Now, realize that if we find a new bad commit, since that bad commit was 
by definition reachable from the _old_ bad commit (since we didn't even 
search outside its reachability), then equally by definition the 
reachability from that new bad commit is a strict superset of the 
reachability of the old bad commit.

So when we find a new bad commit, the old bad commit is no longer 
interesting.

So when you say "pretends that there really is only one bad commit", you 
didn't realize that it's not about "pretending". It's very fundamental: 
there is only ever _one_ bad commit that is interesting. It's the last one 
we found.

Even if we started out with two bad commits (ie some person reported two 
different versions as being bad), we're _still_ not interested in using 
them both. We should pick one of them, because the reachability area 
defined by two bad commits is always a superset of the reachability of 
either one.

So having multiple bad commits is _never_ interesting.

> But I see two problems with that:
> 
> - a problem can be introduced independently in two different branches, and
>   occur in both of them before the merge (in which case bisect only 
>   catches one of the commits), and

This is fine. Depending on whatever random factors, we'll test one of them 
first, and eventually find _one_ of the commits that fix it. If the exact 
same bug was introduced somewhere else, and merged, then undoing just the 
"one" bug will obviously undo the other one too.

If a _different_ bug was introduced (even if it had the same effects), 
yes, you now have two separate bugs. And bisecting two bugs is hard. You 
need to separate them out some way.

> - AFAICT if the cut set is one merge and one regular commit, bisect could
>   identify the merge by error.

It will never identify a commit without having done a full bisection, so 
if it ever had the choice of a "merge" and the "commit leading up to the 
merge", it will always have tried the "commit leading up to the merge", 
and decided that it was fundamentally more recent (had "smaller 
reachability") that the merge, and pinpoint it.

> BTW I think there is a thinko in git-rev-list.txt:
> 
> > Thus, if 'git-rev-list --bisect foo ^bar ^baz' outputs 'midpoint', the 
> > output of 'git-rev-list foo ^midpoint' and 'git-rev-list midpoint ^bar 
>              ^ this should be
> 	'git-rev-list foo ^midpoint ^bar ^baz'
> > ^baz' would be of roughly the same length

Yes.

			Linus

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Linus Torvalds wrote:
> 
> If a _different_ bug was introduced (even if it had the same effects), 
> yes, you now have two separate bugs. And bisecting two bugs is hard. You 
> need to separate them out some way.

Side note: this is seldom a problem in practice. If it was effectively the 
same bug, just finding the one case that triggered it is sufficient: you 
then know what to look for, and if undoing that one commit isn't enough to 
fix it in the current tree (because the same bug existed in another form 
on another branch), you wouldn't actually start bisecting again. You'd 
start grepping the tree for other cases of that bug.

So the biggest advantage of "git bisect" is _not_ that you can just undo 
the buggy commit. In fact, usually you don't even want to undo it, because 
it probably had a raison-d'etre to begin with. The huge deal about "git 
bisect" is that it pinpoints what caused the bug, and then the fix is 
often something else.

Often it's a "Duh! I fixed one thing, but my fix didn't take Xyz into 
account, so it now broke for another reason" moment.

Most bugs are stupid, in other words.

The _real_ problem with git bisect is when you have a non-technical user 
(common) and there are silly bugs that you know of and already fixed that 
aren't really a problem, but that are show-stoppers for the user who isn't 
a kernel developer (or is, but doesn't know git). They're show-stoppers 
not because we care about them, but because they make the "purely 
mechanical" thing be one where you have to have some manual input.

Another problem (that I've not seen in practice yet, but that I bet _will_ 
be the worst issue) is non-reproducible bugs. They are the nastiest kind 
to debug in the first place, and sadly, "git bisect" simply doesn't help 
you with them. There, nothing but some luck and a lot of thinking and 
testing will help you.

			Linus

Re: git pull on Linux/ACPI release tree

[Johannes Schindelin]

Hi,

On Tue, 10 Jan 2006, Linus Torvalds wrote:

> So having multiple bad commits is _never_ interesting.

Okay, I got it. A bug is supposed to be inherited by *all* its 
descendants. Good.

I have to keep in mind that a commit is not actually a patch set, but can 
be two or more (in case of a merge). So, a bug can be present in a 
development line for a long, long time, but be visible only after a merge. 
Since that commit can be compared to at least two trees, one of these 
diffs must show the bug.

Thanks,
Dscho

Re: git pull on Linux/ACPI release tree

[Luben Tuikov]

--- Kyle Moffett <mrmacman_g4-ee4meeAH724@public.gmane.org> wrote:
> they're totally irrelevant.  This is why it's useful to only pull  
> mainline into your tree (EX: ACPI) when you functionally depend on  
> changes there (as Linus so eloquently expounded upon).

Sometimes the dependency is _behavioural_.  For example certain
behaviour of other modules of the kernel changed and you want
to test that your module works ok with them under different
behaviour.  In which case you may or may not have to
change your code after the fact.

    Luben

-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] checkout: automerge local changes while switching branches.

[Junio C Hamano]

When switching branches from A to B, if the working tree has a
local modification at paths that are different between A and B,
we refused the operation saying "cannot merge."  This attempts
to do an automerge for such paths.

Signed-off-by: Junio C Hamano <junkio@cox.net>

---

 Alex Riesen <raa.lkml-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

 > On 1/9/06, Junio C Hamano <junkio-j9pdmedNgrk@public.gmane.org> wrote:
 >> 2. Fix "git checkout <branch>" so that it does a reasonable thing
 >>    even when a dirty path is different in current HEAD and
 >>    destination branch.  Then I could:
 >>
 >>         $ git checkout symref ;# this would not work in the current git
 >>             # it would die like this:
 >>             # $ git checkout symref
 >>             # fatal: Entry 'gitweb.cgi' not uptodate. Cannot merge.
 >
 > That is actually very interesting. I already wished sometimes to be
 > able to switch branches with a dirty working directory (and usually
 > ended up with git diff+checkout+apply).
 > Even if it results in a merge and conflict markers in files it looks
 > like a very practical idea!

 This is still experimental and probably has rough edges, but I
 actually tested it once and it worked fine ;-).

 git-checkout.sh |   24 +++++++++++++++++++++++-
 1 files changed, 23 insertions(+), 1 deletions(-)

7929db987a9aac1d0370b64a8a00ffa13e6bab82
diff --git a/git-checkout.sh b/git-checkout.sh
index 3bbd111..1b2db91 100755
--- a/git-checkout.sh
+++ b/git-checkout.sh
@@ -121,7 +121,29 @@ then
 	git-checkout-index -q -f -u -a
 else
     git-update-index --refresh >/dev/null
-    git-read-tree -m -u $old $new
+    git-read-tree -m -u $old $new || (
+	echo >&2 -n "Try automerge [y/N]? "
+	read yesno
+	case "$yesno" in [yY]*) ;; *) exit 1 ;; esac
+
+	# NEEDSWORK: We may want to reset the index from the $new for
+	# these paths after the automerge happens, but it is not done
+	# yet.  Probably we need to leave unmerged ones alone, and
+	# yank the object name & mode from $new for cleanly merged
+	# paths and stuff them in the index.
+
+	names=`git diff-files --name-only`
+	echo "$names" | git update-index --remove --stdin
+
+	work=`git write-tree` &&
+	git read-tree -m -u $old $work $new || exit
+	if result=`git write-tree 2>/dev/null`
+	then
+	    echo >&2 "Trivially automerged." ;# can this even happen?
+	    exit 0
+	fi
+	git merge-index -o git-merge-one-file -a
+    )
 fi
 
 # 
-- 
1.1.1-g8ecb

Re: git pull on Linux/ACPI release tree

[Matthias Urlichs]

Hi, Linus Torvalds wrote:

> I'm pretty proud of it. It's simple, and it's obvious once you think about 
> it, but it is pretty novel as far as I know. BK certainly had nothing 
> similar, not have I heard of anythign else that does it.

Actually, I've written a hackish script that tries to do simple-minded
bisection (read: it searched for the 50% point on the shortest path
between any-of-GOOD and any-of-BAD, instead of considering the whole
graph) on BK trees. I haven't exactly published the thing anyplace though,
because, well, it was ugly. :-/

Besides, actually working with the current bisection point is no problem
at all for git. Doing the same thing in BK's world view is *painful*,
esp. given the size of the kernel tree.

-- 
Matthias Urlichs   |   {M:U} IT Design @ m-u-it.de   |  smurf@smurf.noris.de
Disclaimer: The quote was selected randomly. Really. | http://smurf.noris.de
 - -
Arthur felt at a bit of a loss. There was a whole Galaxy
of stuff out there for him, and he wondered if it was
churlish of him to complain to himself that it lacked just
two things: the world he was born on and the woman he loved.

Re: git pull on Linux/ACPI release tree

[Adrian Bunk]

On Wed, Jan 11, 2006 at 05:37:06PM -0800, Greg KH wrote:
> On Tue, Jan 10, 2006 at 09:19:09PM +0100, Adrian Bunk wrote:
> > 
> > I am using the workaround of carrying the patches in a mail folder, 
> > applying them in a batch, and not pulling from your tree between 
> > applying a batch of patches and you pulling from my tree.
> 
> Ick, I'd strongly recommend using quilt for this.  It works great for
> just this kind of workflow.

It works in my case because I'm only going through the folder with the 
trivial patches in batches and ask Linus to pull from my tree 
immediately after I'm finished.

That would certainly not be a recommended practice for a subsystem 
maintainer, but I'm handling only trivial patches.

> thanks,
> 
> greg k-h

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: git pull on Linux/ACPI release tree

[Catalin Marinas]

Greg KH <greg-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> wrote:
> On Tue, Jan 10, 2006 at 09:19:09PM +0100, Adrian Bunk wrote:
>> 
>> I am using the workaround of carrying the patches in a mail folder, 
>> applying them in a batch, and not pulling from your tree between 
>> applying a batch of patches and you pulling from my tree.
>
> Ick, I'd strongly recommend using quilt for this.  It works great for
> just this kind of workflow.

Or StGIT :-). Similar workflow (and similar commands) but better
integrated with GIT and better at dealing with conflicts since it uses
a three-way merge when pushing patches rather than applying them with
"patch".

-- 
Catalin
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Greg KH]

On Tue, Jan 10, 2006 at 09:19:09PM +0100, Adrian Bunk wrote:
> 
> I am using the workaround of carrying the patches in a mail folder, 
> applying them in a batch, and not pulling from your tree between 
> applying a batch of patches and you pulling from my tree.

Ick, I'd strongly recommend using quilt for this.  It works great for
just this kind of workflow.

thanks,

greg k-h

Re: git pull on Linux/ACPI release tree

[Andreas Ericsson]

Martin Langhoff wrote:
> On 1/11/06, Adrian Bunk <bunk@stusta.de> wrote:
> 
>>I am using the workaround of carrying the patches in a mail folder,
>>applying them in a batch, and not pulling from your tree between
>>applying a batch of patches and you pulling from my tree.
> 
> 
> In that case, there's a mostly automated way of doing that if you read
> the last couple lines of git-rebase, using something along the lines
> of
> 
>       git-format-patch <yours> <linus> | git-am -3 -k
> 

Isn't this rebase in a nutshell ?

> 
>>I'd say the main problem is that git with several other projects like
>>cogito and stg on top of it allow many different workflows. But finding
>>the one that suits one's needs without doing something in a wrong way
>>is non-trivial.
> 
> 
> You are right about that, but much of the space (of what workflows are
> interesting) is still being explored, and git and the porcelains
> reacting to people's interests. So it's still a moving target. A fast
> moving target.
> 

Good thing there are competent people around to snipe those targets in 
mid-stride. :)

I for one was amazed at how much easier git was to work with than any of 
the other scm's I've tried (quite a few, I never really liked any of 
them), and I really like the fact that it's flexible enough to suit 
(almost) all our needs. The only thing I haven't really found it to be 
satisfactory for is our collection of RPM spec-files and their 
respective patches, where we not so much change files as continuously 
replace them completely. Perhaps that's changed now that most 
git-commands can be run from subdirs.

So, kudos to Linus for inventing it, Junio for nursing it, and the other 
129 developers that have so far contributed to the current release.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/11/06, Adrian Bunk <bunk@stusta.de> wrote:
> I am using the workaround of carrying the patches in a mail folder,
> applying them in a batch, and not pulling from your tree between
> applying a batch of patches and you pulling from my tree.

In that case, there's a mostly automated way of doing that if you read
the last couple lines of git-rebase, using something along the lines
of

      git-format-patch <yours> <linus> | git-am -3 -k

> I'd say the main problem is that git with several other projects like
> cogito and stg on top of it allow many different workflows. But finding
> the one that suits one's needs without doing something in a wrong way
> is non-trivial.

You are right about that, but much of the space (of what workflows are
interesting) is still being explored, and git and the porcelains
reacting to people's interests. So it's still a moving target. A fast
moving target.

cheers,


martin

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Tue, 10 Jan 2006, Adrian Bunk wrote:
> 
> > Now, in this model, you're not really using git as a distributed system. 
> > In this model, you're using git to track somebody elses tree, and track a 
> > few patches on top of it, and then "git rebase" is a way to move the base 
> > that you're tracking your patches against forwards..
> 
> I am using the workaround of carrying the patches in a mail folder, 
> applying them in a batch, and not pulling from your tree between 
> applying a batch of patches and you pulling from my tree.

Yes, that also works.

I think "quilt" is really the right thing here, although stg may be even 
easier due to the more direct git integration. But with a smallish number 
of patches, just doing patch management by hand is obviously simply not a 
huge problem either, so extra tools may just end up confusing the issue.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Adrian Bunk]

On Sun, Jan 08, 2006 at 07:26:50PM -0800, Linus Torvalds wrote:
>...
> THIS is what "rebase" is for. It sounds like what you really want to do is 
> not have a development branch at all, but you just want to track my tree 
> and then keep track of a few branches of your own. In other words, you 
> don't really have a "real" branch - you've got an odd collection of 
> patches that you really want to carry around on top of _my_ branch. No?

Yes.

> Now, in this model, you're not really using git as a distributed system. 
> In this model, you're using git to track somebody elses tree, and track a 
> few patches on top of it, and then "git rebase" is a way to move the base 
> that you're tracking your patches against forwards..

I am using the workaround of carrying the patches in a mail folder, 
applying them in a batch, and not pulling from your tree between 
applying a batch of patches and you pulling from my tree.

> It's also entirely possible that you may want to look at "stacked git"
> (stg), which is really more about a "quilt on top of git" approach. Which
> again, may or may not suit your needs better.
>...

After a quick look, stg seems to be an interesting project that might 
suit my needs.

I'd say the main problem is that git with several other projects like 
cogito and stg on top of it allow many different workflows. But finding 
the one that suits one's needs without doing something in a wrong way
is non-trivial.

It might help if someone could write some kind of "Git for dummies" that 
focusses only on the usage and advantages/disadvantages of user 
interfaces like cogito, stg and (H)GCT and restricts the discussion of 
git internals to a short paragraph in the introduction.

> 		Linus

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: git pull on Linux/ACPI release tree

[Johannes Schindelin]

Hi,

On Mon, 9 Jan 2006, Martin Langhoff wrote:

> In a sense we are still exploring possible/desirable workflows and what 
> the missing pieces are. And yes, some thing don't quite make sense from 
> the outside, perhaps because they just don't or because we arent' 
> explaining them very well.

Maybe what is needed here is this:

T1 - T2 .. Tn .. Tp
  \         \      \
   \         M1     M2
    \       /      /
     B1 .. Bm .. Bo

where T1..Tp are the upstream commits, B1..Bo are the local commits, and
M1.. are the test merges just to make sure nothing breaks?

As long as the Mx commits resolve automatically, no need for an explicit 
merge in the Bx commits, since a pull from B into T will just recreate an 
Mx as next commit in T.

Kind of "throw away merge".

Ciao,
Dscho

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/9/06, Brown, Len <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> I appologize for using the phrase "completely insane".
> The rebase proposal caught me somewhat off-guard and
> I was expressing surprise -- hopefully not taken as insult.
>
> Further, I thank you for your thoughful follow-up.

No worries... and no offence taken! In a sense we are still exploring
possible/desirable workflows and what the missing pieces are. And yes,
some thing don't quite make sense from the outside, perhaps because
they just don't or because we arent' explaining them very well.

In a sense, we do have a bit of a challenge explaining what how all
the parts fit together, even to bk old timers it seems.

> While I don't expect it to become a routine occurnece for me,
> I do see that rebase has utility in some situations.

As long as you've got enough tools to use a workflow that fits you,
it's all good.

cheers,


martin
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: git pull on Linux/ACPI release tree

[Brown, Len]

Martin,
I appologize for using the phrase "completely insane".
The rebase proposal caught me somewhat off-guard and
I was expressing surprise -- hopefully not taken as insult.

Further, I thank you for your thoughful follow-up.

While I don't expect it to become a routine occurnece for me,
I do see that rebase has utility in some situations.

-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Junio C Hamano]

Linus Torvalds <torvalds@osdl.org> writes:

> To be fair, backtracking a "git-rebase" isn't obvious. One of the 
> downsides of rebasing.

I thought "git reset --hard ORIG_HEAD" as usual would do.

RE: git pull on Linux/ACPI release tree

[Brown, Len]

>From: David S. Miller

>I think merges with conflicts that need to get resolved by hand create
>a lot of noise and useless information and therefore to me they are
>pointless.  But this is just my opinion.  It simply works easier to me
>to shuffle the patches in by hand and deal with the rejects one by
>one.  It's very much akin to how Andrew's -mm tree works.

I guess in this model you can do all your development with quilt,
and the value of git is a high-bandwidth bransport medium to
replace e-mail.

>I think a clean history is worth an extra few minutes of someone's
>time.  And note that subsystem development is largely linear anyways.

Maybe true in your neck of the woods, but not true here.

I have more than a dozen topic branches in my tree, and they mature
at different rates.

When a topic branch is in the test tree and and a follow-up patch
is needed, I check out that topic branch and put the patch
exactly in non-linear 3D history where it is meant to live.
When the topic seems fully baked, I can pull the top of the
branch into the release tree.

-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Martin Langhoff wrote:
> 
> and if it does anything but a very trivial merge, backtrack and do a merge.

To be fair, backtracking a "git-rebase" isn't obvious. One of the 
downsides of rebasing.

		Linus

RE: git pull on Linux/ACPI release tree

[Brown, Len]

 
>Which just means that a commit that was tested and found to be working 
>might suddenly not work any more, which can be very surprising ("But I 
>didn't change anything!").

This is why I try to follow the top of tree as closely as possible.
While parts of ACPI are well contained, other parts get stomped on constantly.
My two updates in 10 hours were not becaudse of what I did,
it was in response to seeing the flood gates in 2.6.16 open.

Also, it isn't accurate that nothing changed at my end.
I put some trivial patches into my release branch,
udpated the release branch with upstream,
and then pulled the release branch into my test branch
where there are other patches you haven't seen yet.

I pushed the release branch just to 'clear the deck' of
the trivial patches there since there was no reason to delay
them behind all the stuff still on the test branch.

>So trying out git-rebase and git-cherry-pick just in case you 
>decide to want to use them might be worthwhile. Making it part of your 
>daily routine like David has done? Somewhat questionable, but hey,
>it seems to be working for David, and it does make some things much easier, so..

In the past I have use git-cherry-pick to recover from when a "good" patch
was checked in on top of a "bad" patch, and I wanted to push
the good without the bad.

But the linearization model will not work for me in general.
Branches enable parallel lines of development in git.  If that
is thrown out, then we're basically back at quilt.

thanks,
-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/9/06, Brown, Len <len.brown@intel.com> wrote:
> This is completely insane.
> Do you have any idea what "sometimes has problems merging" means
> in practice?  It means the tools are really nifty in the trivial
> case but worse than worthless when you need them the most.

Len,

all I meant was that you will sometimes see conflicts. And in that
case, you are far better off cancelling the rebase and doing a merge,
where you will have to resolve the conflicts by hand.

git-rebase is for when the potential merge is clearly trivial. In any
other case, you do want a proper merge. But in any case, it is easy to
do

    git-fetch <upstream> && git-rebase <upstream>

and if it does anything but a very trivial merge, backtrack and do a merge.

In any case, if I have any suspicion that the merge may not be trivial, I do

   git-fetch <upstream> && gitk --since=" 1 month ago" upstream master

before deciding on a course of action. Of course, you can merge all
the time. It's whether people care about a readable/useful history
afterwards.

cheers,


martin

Re: git pull on Linux/ACPI release tree

[linux]

> And lastly, is there a tool whereby I can "see" changes
> between repos, kind of like git-diff but being able to
> give URLs too?

Write it yourself.  It's git-fetch + git-diff.

Or, put another way, if you think you need a special tool for working
with a remote repository, you don't understand git-fetch.

Since git history is immutable, there is no difference between a remote
copy and a local copy.  And since fetching is harmless to your local
repository, there's no problem.

If you don't want to copy the entire history, just fetch the tree rather
than the commit.  (Does git-fetch do that?  It's a subset of its current
effects, so would be an easy enough extension.)

RE: git pull on Linux/ACPI release tree

[Brown, Len]

>On 1/9/06, Brown, Len <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>> Perhaps the tools should try to support what "a lot of people"
>> expect, rather than making "a lot of people" do extra work
>> because of the tools?
>
>I think it does. All the tricky stuff that David and Junio have been
>discussing is actually done very transparently by
>
>    git-rebase <upstream>
>
>Now, git-rebase uses git-format-patch <options> | git-am <options> so
>it sometimes has problems merging. In that case, you can choose to
>either resolve the problem (see the doco for how to signal to git-am
>that you've resolved a conflict) or to cancel the rebase. If you
>choose to cancel the rebase, do
>
>   cp .git/refs/heads/{<headname>,<headnamebadrebase>}
>   cat .git/HEAD_ORIG > .git/refs/heads/<headname>
>   git-reset --hard
>   rm -fr .dotest
>
>and you'll be back to where you started. Perhaps this could be rolled
>into something like git-rebase --cancel to make it easier, but that's
>about it. The toolchain definitely supports it.

This is completely insane.
Do you have any idea what "sometimes has problems merging" means
in practice?  It means the tools are really nifty in the trivial
case but worse than worthless when you need them the most.

-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/9/06, Linus Torvalds <torvalds-3NddpPZAyC0@public.gmane.org> wrote:
> And then do
>
>         git-rebase linus
>
> to rebase your development branch to mine.
>
> THIS is what "rebase" is for. It sounds like what you really want to do is
> not have a development branch at all, but you just want to track my tree
> and then keep track of a few branches of your own. In other words, you
> don't really have a "real" branch - you've got an odd collection of
> patches that you really want to carry around on top of _my_ branch. No?

FWIW, I determine whether I should rebase or merge based on

 + Whether the branch/head I maintain is public. For public repos, I
*must* merge carefully as rebase "rewinds" the head and that makes a
mess of any repositor tracking me.

 + Whether the changes on my both sides are significant, and it is
semantically meaningful to have a merge. If either side had just a
couple of minor commits, rebase makes life a lot easier down the path.
If both side clearly saw parallel development, it is more sincere to
merge and let that be recorded.

 + If my attempt to rebase leads to any non-trivial conflicts or
co-dependencies, then I definitely cancel the rebase and merge.

> Now, in this model, you're not really using git as a distributed system.

I'd argue that it is not about distributed or not. It's all in what
you want to record in your history. As such, it is a communication
device -- and I want to make effective use of it. I guess the question
I ask myself is: what will communicate what's happened here most
clearly? What will be useful for people to read? In that context, a
white-lie here and there simplifying the history a bit where it's not
interesting counts as a good thing.

cheers,


martin
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Al Viro wrote:
> 
> How do you deal with conflict resolution?  That's a genuine question -
> I'm not talking about deliberate misuse to hide an attack, just a normal
> situation when you have to resolve something like
> 
> A:
> 	if (foo)
> 		bar
> 
> B:
> 	if (foo & baz)
> 		bar
> 
> A':
> #ifdef X
> 	if (foo)
> 		bar
> 	...
> #endif
> 
> merge of A' and B: trivial conflict

Actually, these days git is pretty good at it. Much better than CVS, 
certainly. You can see the "conflict against my old tree", or "conflict 
against the remote tree" by using the "--ours" or "--theirs" flag to "git 
diff" respectively.

(Or "diff conflict against common base": "git diff --base").

So for your particular example with a trivial base file:

	line    1
	        2
	        3
	        if (foo)
	                bar
	        6
	        7
	        8

and then the changes you had as an example in the A' and B branches, if I 
from A' do a "git pull . B", I get:

	Trying really trivial in-index merge...
	fatal: Merge requires file-level merging
	Nope.
	Merging HEAD with ad56343c578785b8d932224a8676615e7a3e191f
	Merging: 
	9d619225e3adecee6432a36d67d140e29b0acf62 A' case 
	ad56343c578785b8d932224a8676615e7a3e191f B: case 
	found 1 common ancestor(s): 
	93765ba3f64e9c73438e52683fffa68e5a493df7 Base commit 
	Auto-merging A 
	CONFLICT (content): Merge conflict in A 

	Automatic merge failed; fix up by hand

and then the file contains the contents

	line    1
	        2
	        3
	<<<<<<< HEAD/A
	#ifdef X
	        if (foo)
	=======
	        if (foo && baz)
	>>>>>>> ad56343c578785b8d932224a8676615e7a3e191f/A
	                bar
	        6
	#endif
	        7
	        8

ie it will have does a CVS-like merge for me, and I need to fix this up. 
However, to _help_ me fix it up, I can now see what the diff is aganst my 
original version (A'), with "git diff --ours" (the "--ours" is default, so 
it's unnecessary, but just to make it explicit):

	* Unmerged path A
	diff --git a/A b/A
	index 06dd3bc..7334364 100644
	--- a/A
	+++ b/A
	@@ -1,8 +1,12 @@
	 line   1
	        2
	        3
	+<<<<<<< HEAD/A
	 #ifdef X
	        if (foo)
	+=======
	+       if (foo && baz)
	+>>>>>>> ad56343c578785b8d932224a8676615e7a3e191f/A
	                bar
	        6
	 #endif

which is very helpful especially once I have resolved it. IOW, I just edit 
the file and do the trivial resolve, and now I can do a "git diff" again 
to make sure that it looks ok:

	* Unmerged path A
	diff --git a/A b/A
	index 06dd3bc..924fc97 100644
	--- a/A
	+++ b/A
	@@ -2,7 +2,7 @@ line    1
	        2
	        3
	 #ifdef X
	-       if (foo)
	+       if (foo && baz)
	                bar
	        6
	 #endif

ahh, looks good, so I just do "git commit A" and that creates the 
resolved merge.

>  The obvious way (edit file in question, update-index, commit) will not 
> only leave zero information about said conflict and actions needed to 
> deal with it, but will lead to situation when git whatchanged will not 
> show anything useful.

Now, this is a real issue. 

The resolve part is pretty easy, but the fact that it's hard to see in 
"git-whatchanged" is a limitation of git-whatchanged. 

You need to use "gitk", which _does_ know how to show merges as a diff 
(and yes, I just checked).

> Is there any SOP I'm missing here?

You're just missing the fact that git-whatchanged (or rather, 
"git-diff-tree") isn't smart enough to show merges nicely. It really 
_should_. It doesn't. You can choose to show merges with the "-m" flag, 
but that will show diffs against each parent, which really isn't what you 
want. 

I should do the same thing gitk does in git-diff-tree.

> Worse (for my use), format-patch on such tree will not give a useful patchset.
> You get a series of patches that won't apply to _any_ tree. 

Now, git-diff-tree _does_ do that. Use the "-m" flag, and choose the tree 
you want.

And btw, that works with "git-whatchanged" too. You _can_ pass the "-m" 
flag to git-whatchanged, and it will show you each side of the merge 
correctly. So it _works_. It's just such a horrible format that by default 
it prefers to shut up about merges entirely. 

(I don't know of a good three-way diff format. "gitk" can do it, because 
gitk can show colors. That's a big deal when you do three-way - or 
more-way - diffs).

> And that's a fundamental problem behind all that rebase activity, AFAICS.

You do _not_ want to rebase a merge. It not only won't work, it's against 
the whole point of rebasing.

Rebasing is really only a valid operation when you have a few patches OF 
YOUR OWN that you want to move up to a new version of somebody elses tree 
that you are tracking. You fundamentally _cannot_ rebase if you've done 
anything but a linear set of patches. And that has nothing to do with the 
patch difficulty - it simply isn't an operation that makes sense.

(Btw, not making sense doesn't mean it might not work. It sometimes might 
actually work and do what you _hoped_ it would do, but it's basically by 
pure luck, and not because it is a sensible operation. Even stupid 
people hit on the right solution every once in a while - not because 
they thought about things right, but just because they happened to try 
something that worked. The same is true of "git rebase" with merges ;^).

The fundamental reason a rebase doesn't make sense is that if you've done 
a merge, it obviously means that some other branch has done development, 
and already has the commits that you're trying to rebase. And you CANNOT 
rebase for them.

So instead of rebasing across a merge, what you can do is to not do the 
merge at all, but instead rebase one of the two branches against the 
other. Then you can rebase the result against the thing that you wanted to 
rebase them both against. Now you've never rebased a merge - you've just
linearised branches that were linear in themselves against each other.

Basically, a merge ties two branches together. Once you've merged, you 
can't make a linear history any more. The merge fundamentally is not 
linear.

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Adrian Bunk wrote:
> 
> Consider I want to do the following:
> 1. update my tree daily from your tree
> 2. include 10 patches per week into my tree
> 3. ask you once a month to pull from my tree
> 
> How should step 1 be done?

I'd do

	git fetch linus

to fetch my tree as a branch (obviously, this assumes you've set up a 
'.git/remotes/linus' file for the shorthand).

Then, keep your checked-out working-tree that you also do you development 
in your 'devel' branch (or whatever).

And then do

	git-rebase linus

to rebase your development branch to mine.

THIS is what "rebase" is for. It sounds like what you really want to do is 
not have a development branch at all, but you just want to track my tree 
and then keep track of a few branches of your own. In other words, you 
don't really have a "real" branch - you've got an odd collection of 
patches that you really want to carry around on top of _my_ branch. No?

Now, in this model, you're not really using git as a distributed system. 
In this model, you're using git to track somebody elses tree, and track a 
few patches on top of it, and then "git rebase" is a way to move the base 
that you're tracking your patches against forwards..

It's also entirely possible that you may want to look at "stacked git" 
(stg), which is really more about a "quilt on top of git" approach. Which 
again, may or may not suit your needs better.

Now, the other alternative is to use git as a "real" distributed system, 
and then you might keep my "linus" branch around perhaps as a reference 
point, but you don't care too much about it. What you care a lot more 
about is your "real development" branch, and you simply don't rebase that, 
or try to track my branch all that closely. You work in your real 
development branch, and that's your bread and butter. You _don't_ merge my 
tree every day, because you simply don't care - that's a separate issue.

You might have a totally different directory that you use _just_ to track 
my tree, and that is my virgin tree checked out and ready to go to test 
what _I_ am doing - totally independently of your tree.

See? Two totally different usage schenarios. In one, you keep a couple of 
"odd-ball" patches around (hey, it could be many, but I say a couple just 
because the patches aren't a huge deal - they are kind of a small side 
project to you, and not the main focus). And you use "git rebase" to move 
those patches forward to match the "real" tree, aka mine.

In the other schenario, your development tree is a full-fledged real 
branch, with a life of its own, and _not_ slaved to what happens to be 
going on in my tree. You may care about my tree for _other_ reasons, but 
the two aren't really joined at the hip.

The third schenario is somewhere in between: you do pull from my tree, but 
you do it occasionally enough that the merges don't get annoying.

Most people I work with are actually in that gray area. EVERYBODY pulls 
some. Nobody tends to do black-and-white either-or schenario. The question 
is just how much. 

My gut rule: if you have almost as many merges as you have "real work" 
commits, you're doing something bad, and you should pull less often 
(perhaps by using "rebase", perhaps by just realizing that you don't need 
to be tailgating _quite_ that closely).

But 30 "real" commits and 5 merges because the 30 real commits happened 
over four weeks time? Sounds fine to me. 

		Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Sun, 8 Jan 2006, Luben Tuikov wrote:
>
> How about this usage (branch == tree):
> 
> Tree A    (your tree)
>   Tree B     (project B, dependent on Tree A)
>      Tree C     (project C, dependent on project B)
> 
> (i.e. diff(C-A) = diff(C-B) + diff(B-A))
> 
> Your tree is pulled into Tree A as often as your tree
> changes and it just fast forwards.
> 
> If I want to run project B with your latest tree, then
> I resolve/merge from tree A to tree B, compile B
> and run it.
> 
> If I want to run project C and project B with your
> latest tree, I resolve/merge from tree A to tree B
> and from tree B to tree C, compile C and run it.

No.

If tree B is based on _some_point_in_ A, then you just test that.

Because development line B is _independent_ of development line A. The 
fact that A changes doesn't change B - unless they have some real 
dependencies (which we should try to avoid).

So when you update ("fetch" in git parlance) branch A from me, that 
shouldn't affect branch B _nor_ branch C in any way. They clearly do not 
depend on the new stuff in A, since they do their own independent 
development. The fact that they _started_ at some random point during the 
development of A doesn't change that fact.

Now, if you want to _test_ the combined "new stuff in branch A and new 
stuff in branch B", feel free to do that. But realize that that is _not_ 
appropriate in either branch A _nor_ branch B.

So you'd be much better off with a separate "test" branch that you test 
stuff out in, and you then resolve ("pull" in git parlance) both branch A 
and branch B into that test branch.

See? Testing the combination of two branches doesn't actually have 
anything to do with either branch.

At some point, you decide that you want to merge what you've done in 
branch B. That's a _different_ and independent thing from deciding that 
you want to test the combination of two development branches. Clearly, 
it's great to test often, but that has nothing to do with releasing a 
branch.

> In such cases, are you saying that you'd prefer to
> pull from Tree B and Tree C (depending on your needs)?

I'm saying that mixing up the "let's test the combination" and "let's 
merge the two branches" are totally different things and should not be 
mixed up.

One is a random event (and then it makes sense to have, for example, a 
"automated test branch" that automatically merges every day and tests the 
results. I don't think you should expose those random merges to others, 
because they actually hinder the readability of the history for _both_ 
sides.

The other is a _directed_ event. It's the event of saying "branch B" is 
now ready to be merged. Usually that's best done by just saying "please 
pull now" - ie not by merging branch A into branch B (because that's not 
what you actually want, is it? What you want is for the development in 
branch B to show up in branch A - so you want branch A to do the pull).

Now, there's a third kind of event, which is again independent of the 
other two. It's more of a "let's try to keep the 'topic branch' 
development up-to-date with the branch we eventually want to merge the 
topic changes into". That's where you can now do two things:

 - David often "rebases" all of the changes in his "topic branch" (ie 
   conceptually "branch B") to the new top-of-head of "branch A". In other 
   words, he re-writes branch B entirely _as_if_ it was based on the newer 
   state "branch A". This is what "git rebase" is all about.

 - You can just pull from branch A into branch B, as a way to keep branch 
   B more up-to-date with the work in the "main trunk" or whatever. This 
   is ok, but it shouldn't be a common event. It should be something that 
   happens when you (for example) notice during testing that the test 
   merge no longer works cleanly. Or it might be "It's now been two weeks 
   since I synchronized, let's just synchronize to be safe".

See? I'm not objecting to topic branches pulling from my tree in general. 
It's just that they should have a _reason_. There's never any reason to 
pull into a development tree that you haven't done any development in, 
just because you also want to use that development tree for testing.

> Another question:
> Sometimes, a fix for project B finds its way into
> tree C (project C) (since C depended on that fix in B).
> Now I'd like to pull that particular fix, identified by
> its SHA, into project B, and nothing else, for this I can
> use git-cherry-pick, right?

That's one way. It's often the best way, especially if it's a really 
obvious bugfix. Or you could just fix it in your tree yourself. It will 
mean that the two branches have the same fix, but especially if it really 
is an identical fix, it won't be a merge problem.

You _can_ just decide to pull branch B into branch C, but that has a real 
problem, namely that it inexorably links the two together, so that nobody 
can then pull branch C without pulling indirectly branch B at the time 
that B->C merge happened. Sometimes that is ok. But it's nice to avoid it 
if you can.

But for example, if somebody fixed something in the trunk, and you 
actually do need that fix from the trunk for your topic branch 
development, then just doing a pull is _fine_. Now we're back to doing a 
merge that actually has a perfectly good reason.

IOW, don't cherry-pick to avoid merges when the merge really does make 
tons of sense. Merges are good, it's just that _too_ much of a good thing 
is bad.

> And lastly, is there a tool whereby I can "see" changes
> between repos, kind of like git-diff but being able to
> give URLs too?

No, all the good tools really are based on fetching (NOT "pulling") the 
other branch into your local tree as a separate branch. At that point, 
there are tons of wonderful tools you can use.

In other words, say that you want to know what has happened in another 
repository, at git://git.kernel.org/xyzzy. You aren't interested in the 
stuff that is already part of the trunk, you're just interested in what is 
only in that "xyzzy" branch, and how it relates to your code. 

What you'd do is

	git fetch git://git.kernel.org/xyzzy master:xyzzy-snapshot

which says "fetch the 'master' branch from that xyzzy repository, and call 
it 'xyzzy-snapshot' locally.

You can then (for example) fetch the code that is in _my_ tree by doign 
the same time (just call that branch 'linus'), and you can now do

	gitk linus..xyzzy-snapshot HEAD

which looks strange (you give "gitk" _both_ a range from the "linus" 
branch to the "xyzzy-snapshot" _and_ your own HEAD at this time), but what 
it basically does is that the "linus.." syntax tells git that you're not 
interested in anything that is already in the 'linus' branch.

So the above command line will actually graphically show _both_ your 
current HEAD branch _and_ the 'xyzzy-snapshot' branch, in parallel. You 
can see how (if at all) they are related to each other, ignoring all the 
commits that have already made it into my tree.

(You can also do "linus..HEAD" instead of just HEAD and effectively repeat 
the "don't show 'linus' branch any more" twice. It's perfectly equivalent, 
of course. You may also want to use the "-d" flag to "gitk" which tells it 
to show things in date order, instead of a simplified history order).

Or just do "what has xyzzy-snapshot that I do not have in my HEAD":

	git log HEAD..xyzzy-snapshot

(or gitk), or the other way around: what do _I_ have in my HEAD that 
hasn't been pushed to xyzzy-snapshot yet:

	git log xyzzy-snapshot..HEAD

(or do diffs, "git whatchanged -p", or whatever).

In other words, using a few different branches (you can make them up 
dynamically) can be very powerful.

		Linus

Re: git pull on Linux/ACPI release tree

[Al Viro]

On Sun, Jan 08, 2006 at 11:10:20AM -0800, Linus Torvalds wrote:
> That has absolutely nothing to do with anything. It's not the comment 
> (which admittedly gives absolutely no information - but why should it, 
> since the _commit_ itself has no information in it?)

How do you deal with conflict resolution?  That's a genuine question -
I'm not talking about deliberate misuse to hide an attack, just a normal
situation when you have to resolve something like

A:
	if (foo)
		bar

B:
	if (foo & baz)
		bar

A':
#ifdef X
	if (foo)
		bar
	...
#endif

merge of A' and B: trivial conflict

and have git pull fail.  The obvious way (edit file in question, update-index,
commit) will not only leave zero information about said conflict and actions
needed to deal with it, but will lead to situation when git whatchanged will
not show anything useful.   I.e. if conflict turns out to be non-trivial and
ends up being resolved wrong, everyone will have very nasty time trying to
figure out where the breakage had come from when looking at history 6 months
down the road.

Is there any SOP I'm missing here?

Worse (for my use), format-patch on such tree will not give a useful patchset.
You get a series of patches that won't apply to _any_ tree.  Even if all
conflicts had been resolved correctly, they still remain there for everyone
trying to apply the patch series, unless you manually rebase it before
format-patch.

And that's a fundamental problem behind all that rebase activity, AFAICS.
It definitely is in my case, and yes, it's fscking inconvenient in a lot
of respects.  E.g. I'm using git for resync between build trees on several
boxen.  There's a repository holding patchset, plus one clone per build
box.  Fixes for build breakage, etc., get done in those clones; after they
are committed there, I pull into master and then pull from other clones
to spread them to other build trees.  Works fine, but...  Any rebase in
master => instant hell for all clones.  I've ended up with the following
layout that kinda-sorta avoids mess:
master:origin: matches upstream
master:topic branch: _not_ rebased until there is a conflict, never get
	a pull from anywhere
master:master: gets pulls from topic branches and origin _and_ _nothing_ _else_
master:work: where interaction with build boxen and any edits done in master
	repository go.  Edits, commits, pulls from master:master, pulls from
	build boxen.
buildN:origin == master:work
buildN:work: where work on buildN goes.
When I want to get new stuff (== difference between master:master and
master:work) into the patchset, I cherry-pick from work to topic branches
and re-pull them into master:master until it matches master:work.  Then
I pull master:master into master:work to create a point in work history
that marks beginning of new portion of pending stuff.  New stuff upstream
is pulled to master:origin -> master:master -> master:work -> build trees.

That works, and gives me merge-free topic branches I can safely format-patch
while keeping master in sync with mainline _and_ also safe for format-patch.
The price is in rather convoluted SOP.  And the following piece of fun:
when cherry-pick work->topic, pull topic->master or pull origin->master
gives a conflict, it's time to rebase.  Which I do by renaming topic branches
(direct mv in .git/refs/heads), then starting new ones at current origin
and applying old ones to them (cherry + cherry-pick if possible, format-patch
+ applymbox if things get hairy).  Then master is recreated as branch from
origin that gets pulls from topic branches, work is branched from it and
build trees get killed and cloned from scratch.  It's tolerable since I'm
using ccache on build boxen, so it's _not_ that much of rebuild.

However, that clearly is a killer if any poor sucker (me included) ever
clones from master for any other purpose.  And that, BTW, is the main
reason that stops me from moving master to kernel.org right now.

> And yes, you can always work around it, but there's just no point. And 
> none of the other developers seem to need to do it. They do their 
> development, and then they say "please pull". At that point the two 
> histories are tied together, but now they are tied together for a 
> _reason_. It was an intentional synchronization point.
> 
> An "automated pull" by definition has no reason. If it works automated, 
> then the merge has zero semantic meaning. 

I'm afraid you are missing a part of picture.  There is a bunch of git
uses that handle a heap of foam rather than a long-term branching.  I.e.
the tree is tied to mainline closely and most of the stuff in it is
supposed to get flushed into mainline soon after it appears.  I.e. the
situation when we have a mergepoint for fixes that _has_ to follow
mainline closely.

I wonder what life would be without merge nodes and with equality nodes
instead.  I.e. to merge
O -> A1 -> ..... -> An (=A)
  -> B1 -> ..... -> Bm (=B)
would be to create a new branch (C) at Bm, have entire A1...An replayed there,
have B1...Bm replayed in A and then create a node certifying that new head
of A and head of C refer to the same tree.  Plus have a way to see which
commits are claimed to be replays of each other.  At least that way rebase
would be simply saying that old history is superceded by new one, with
equality node proving that it's OK to do.  We would have
O -> M1 -> ....  ->Mn for mainline
O -> B1 -> ....  -> B for branch post-pull
Mn -> P1 -> ... -> P for merge branch
and B == P as equality node.   Old branch would have a bunch of changesets
of its own plus ones from mainline that got there by pulls (including the
last one).  And new branch would contain the ports of not-yet-merged ones
to new mainline head, with the same tree as the result and all further
development going on there rather than in the old branch.  Oh, well...

Re: git pull on Linux/ACPI release tree

[Willy Tarreau]

Hi Adrian,

On Mon, Jan 09, 2006 at 12:06:11AM +0100, Adrian Bunk wrote:
> On Sun, Jan 08, 2006 at 11:41:28AM -0800, Linus Torvalds wrote:
> >...
> > What I object to is that there were _also_ two automated merges within ten 
> > hours or each other, with absolutely _zero_ development in your tree in 
> > between. Why did you do that in your development tree? By _definition_ you 
> > had done zero development. You just tracked the development in _my_ tree.
> >...
> 
> My impression is that you and Len are talking at different levels.
> 
> I can't speak for Len, but let me try to describe a problem in this area 
> I don't know the solution for:
> 
> Consider I want to do the following:
> 1. update my tree daily from your tree
> 2. include 10 patches per week into my tree
> 3. ask you once a month to pull from my tree
> 
> How should step 1 be done?

I believe we all have the same problem. The only solution I found for
this was to proceed like David described. I know even have a 'git-patches'
directory next to my git repo to keep the resulting patches after I do a
'git-format-patch --mbox'.

When Linus called it the 'stupid content tracker', he was half right.
In fact, it's more a 'changes tracker' than a 'content tracker'. It
logs everything you and others do, so if you don't want your operations
to appear on others' history, you have to hide them by working on
temporary trees to generate the patches you will use later.

At first I found this very annoying, but finally, it's a way to ensure
that I always have clean and ordered patches. It's not much different
from what I was doing by hand previously, it's just that all git-xxx
operations take much longer time, possibly because of the compression.
On the other hand, its ability to understand mbox saves me some time.

> In CVS, I'd do a "cvs update -dP ."
> In cogito, the equivalent command seems to be "cg-update".
> 
> CVS has no problems if I have changed MAINTAINERS in one place and it 
> changes daily in your tree in other places, but how do I do the same in 
> git/cogito without creating the merges you don't want to see?
> 
> The solution might be described somewhere in TFM, but this is the class 
> of problems people like me run into when the goal is simply a git tree 
> to both track your tree and send changes to you without any interest in 
> advanced SCM knowledge.

What sometimes worries me is that some operations seem so much complicated
that there is a high risk of doing the wrong thing, and I'm not certain
that this will reduce the number of mistakes I do in a month, compared
to manual patching. I became a real addict to 'git-reset --hard' ...

> cu
> Adrian

Regards,
Willy

-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Adrian Bunk]

On Sun, Jan 08, 2006 at 11:41:28AM -0800, Linus Torvalds wrote:
>...
> What I object to is that there were _also_ two automated merges within ten 
> hours or each other, with absolutely _zero_ development in your tree in 
> between. Why did you do that in your development tree? By _definition_ you 
> had done zero development. You just tracked the development in _my_ tree.
>...

My impression is that you and Len are talking at different levels.

I can't speak for Len, but let me try to describe a problem in this area 
I don't know the solution for:

Consider I want to do the following:
1. update my tree daily from your tree
2. include 10 patches per week into my tree
3. ask you once a month to pull from my tree

How should step 1 be done?

In CVS, I'd do a "cvs update -dP ."
In cogito, the equivalent command seems to be "cg-update".

CVS has no problems if I have changed MAINTAINERS in one place and it 
changes daily in your tree in other places, but how do I do the same in 
git/cogito without creating the merges you don't want to see?

The solution might be described somewhere in TFM, but this is the class 
of problems people like me run into when the goal is simply a git tree 
to both track your tree and send changes to you without any interest in 
advanced SCM knowledge.

> 		Linus

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Luben Tuikov]

--- Linus Torvalds <torvalds@osdl.org> wrote:
> So trying out git-rebase and git-cherry-pick just in case you decide to 
> want to use them might be worthwhile. Making it part of your daily routine 
> like David has done? Somewhat questionable, but hey, it seems to be 
> working for David, and it does make some things much easier, so..

How about this usage (branch == tree):

Tree A    (your tree)
  Tree B     (project B, dependent on Tree A)
     Tree C     (project C, dependent on project B)

(i.e. diff(C-A) = diff(C-B) + diff(B-A))

Your tree is pulled into Tree A as often as your tree
changes and it just fast forwards.

If I want to run project B with your latest tree, then
I resolve/merge from tree A to tree B, compile B
and run it.

If I want to run project C and project B with your
latest tree, I resolve/merge from tree A to tree B
and from tree B to tree C, compile C and run it.

In such cases, are you saying that you'd prefer to
pull from Tree B and Tree C (depending on your needs)?

Another question:
Sometimes, a fix for project B finds its way into
tree C (project C) (since C depended on that fix in B).
Now I'd like to pull that particular fix, identified by
its SHA, into project B, and nothing else, for this I can
use git-cherry-pick, right?

And lastly, is there a tool whereby I can "see" changes
between repos, kind of like git-diff but being able to
give URLs too?

    Luben

Re: git pull on Linux/ACPI release tree

[Tony Luck]

I'll try to update the using-topic-branches document to capture this.
Some of the problem is that it doesn't quite capture what I'm doing
with my test/release branches.

My release branch really is just used as a transfer point to Linus.
I usually[1] don't leave patches sitting in "release" for long enough
that I'll be tempted to merge in from Linus ... once I decide that
some patches are ready to go to Linus I'll update "release" from Linus
(which will be a fast-forward, so no history) merge in the topic
branches, do one final sanity build, push to kernel.org and send
the "please pull" e-mail.

The huge majority of my "automatic update from upstream" merges
go into my test branch ... which never becomes part of the real
history as I never ask Linus to pull from it.

-Tony

[1] Sometimes I goof on this because I forget that I've applied
a trivial patch directly to the release branch without going through
a topic branch.  I think I'll fix my update script to check for this case.

Re: git pull on Linux/ACPI release tree

[David S. Miller]

From: Linus Torvalds <torvalds@osdl.org>
Date: Sun, 8 Jan 2006 11:56:21 -0800 (PST)

> So my suggested git usage is to _not_ play games. Neither do too-frequent
> merges _nor_ play games with git-rebase.
> 
> That said, git-rebase (and associated tools like "git-cherry-pick" etc) 
> can be a very powerful tool, especially if you've screwed something up, 
> and want to clean things up. Re-doing history because you realized that a 
> you did something stupid that you don't want to admit to anybody else.
> 
> So trying out git-rebase and git-cherry-pick just in case you decide to 
> want to use them might be worthwhile. Making it part of your daily routine 
> like David has done? Somewhat questionable, but hey, it seems to be 
> working for David, and it does make some things much easier, so..

The time at which I do the by-hand rebasing the most are the weeks
leading up to a major release.  The reason is to integrate bug fixes
that I know conflict with the 80-odd patches I have queued up for the
next development phase, or that I simply want integrated so that no
_future_ development patches create conflicts.

I think merges with conflicts that need to get resolved by hand create
a lot of noise and useless information and therefore to me they are
pointless.  But this is just my opinion.  It simply works easier to me
to shuffle the patches in by hand and deal with the rejects one by
one.  It's very much akin to how Andrew's -mm tree works.

I think a clean history is worth an extra few minutes of someone's
time.  And note that subsystem development is largely linear anyways.

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Sun, 8 Jan 2006, Junio C Hamano wrote:
> 
> Careful.  I do not think rebase works across merges at all.

Right. You have to do one or the other (rebase your changes to another 
tree _or_ merge another tree into your changes), but not mix the two.

		Linus

Re: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Mon, 9 Jan 2006, Martin Langhoff wrote:
> 
> I think it does. All the tricky stuff that David and Junio have been
> discussing is actually done very transparently by
> 
>     git-rebase <upstream>

Yes, it's fairly easy to do. That said, I would actually discourage it. I 
haven't said anything to David, because he is obviously very comfy with 
the git usage, and it _does_ result in cleaner trees, so especially since 
the networking code ends up being the source of a lot of changes, the 
extra cleanup stage that David does might actually be worth it for that 
case.

But git is actually designed to have parallel development, and what David 
does is to basically artificially linearize it. We merge between us often 
enough that it doesn't really end up losing any historical information 
(since David can't linearize the stuff that we already merged), but in 
_theory_ what David does actually does remove the historical context.

So "git-rebase" is a tool that is designed to allow maintainers to (as the 
command says) rebase their own development and re-linearize it, so that 
they don't see the real history. It's basically the reverse of what Len is 
doing - Len mixes up his history with other peoples history in order to 
keep them in sync, while David bassically "re-does" his history to be on 
top of mine (to keep it _separate_).

The "git-rebase" means that David will always see the development he has 
done/merged as being "on top" of whatever my most recent tree is. It's 
actually a bit scary, because if something goes wrong when David re-bases 
things, he'll have to clean things up by hand, and git won't help him 
much, but hey, it works for him because (a) things seldom go wrong and (b) 
he appears so comfortable with the tool that he _can_ fix things up when 
they do go wrong.

And yes, git-rebase can be very convenient. It has some problems too 
(which is the other reason I don't try to convince other maintainers to 
use it): because it re-writes history, a change that _might_ have worked 
in its original place in history might no longer work after a rebase if it 
depended on something subtle that used to be true but no longer is in the 
new place that it has been rebased to.

Which just means that a commit that was tested and found to be working 
might suddenly not work any more, which can be very surprising ("But I 
didn't change anything!").

On the other hand, this is no different from doing a merge of two 
independent streams of development, and getting a new bug that didn't 
exist in either of the two, just because they changed the assumptions of 
each other (ie not a _mismerge_, but simply two developers changing 
something that the other depended on it, and the bug only appears when 
both the working trees are merged and the end result no longer works).

So my suggested git usage is to _not_ play games. Neither do too-frequent
merges _nor_ play games with git-rebase.

That said, git-rebase (and associated tools like "git-cherry-pick" etc) 
can be a very powerful tool, especially if you've screwed something up, 
and want to clean things up. Re-doing history because you realized that a 
you did something stupid that you don't want to admit to anybody else.

So trying out git-rebase and git-cherry-pick just in case you decide to 
want to use them might be worthwhile. Making it part of your daily routine 
like David has done? Somewhat questionable, but hey, it seems to be 
working for David, and it does make some things much easier, so..

			Linus

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Sun, 8 Jan 2006, Brown, Len wrote:
>
> Perhaps the tools should try to support what "a lot of people"
> expect, rather than making "a lot of people" do extra work
> because of the tools?
> 
> Call me old fashioned, but I believe that tools are supposed to
> make work easier, not harder.

They DO.

Len, you're doing EXTRA WORK that is pointless.

Just stop doing the automated merges. Problems solved. It really is that 
easy. Don't do what David suggests - he does it because he's apparently 
_so_ comfortable with things that he prefers to do extra work just to keep 
his trees extra clean (I actually would disagree - but git makes that 
fairly easy to do, so if you prefer to have as linear a history as 
possible, you can do it with git pretty easily).

Now, I'm only complaining about _automated_ merges. If you have a reason 
to worry about my tree having clashes with your tree, do a real merge. For 
example, in your latest pull, you had a 

	"pull linus into release branch"

merge, where you merged my v2.6.15 tree. That makes perfect sense.

What I object to is that there were _also_ two automated merges within ten 
hours or each other, with absolutely _zero_ development in your tree in 
between. Why did you do that in your development tree? By _definition_ you 
had done zero development. You just tracked the development in _my_ tree.

In case you wonder, the two commits I'm talking about are:

	add5b5ee992e40c9cd8697ea94c223628be162a7
	25da0974601fc8096461f3d3f7ca3aab8e79adfb

and neither of them have any reason to be in a development tree. You 
didn't develop them.

They are real merges, because you had a trivial patch in your tree 
(changing the acpi-devel mailing list address) that I didn't have, so when 
you pulled, your end result was thus always different from something I had 
(so you did a real "merge", even though it was totally trivial), but the 
point is that there is a difference between "the ACPI development tree" 
and "the tree that has random ACPI patches and then tracks Linus' tree as 
closely as possible".

See?

That's the most egregious example. There's two unnecessary pulls on 
December 28 and 29th too (commits 0a5296dc and c1a959d8).

You can do

	gitk 0aec63e..f9a204e1 

to see exactly what I see when I pulled from you. 11 commits, 5 of which 
are just trivial merges that are no development, just tracking _my_ tree. 
Of those, one makes sense (tracking a release).

(NOTE NOTE NOTE! It does make sense to track my tree in case you do big 
changes and you worry about clashes. Then you would want to synchronize 
those big changes with my changes, so that you can resolve any clashes 
early. So I'm not saying that tracking trees is always bad: I'm saying 
that doing so _unnecessarily_ is bad, because it adds no value, and it 
just makes the history harder to read).

Now, most people don't read the history. It gets messy enough quickly 
enough that it's hard to read anyway over time. My tree has tons of _real_ 
merges anyway, since it's by definition the one that is used for most 
synchronization, so my tree is always pretty hard to follow.

But my guess is that this probably makes it harder for _you_ to see what 
you've done too. If you didn't merge with me, then "git log" would show 
just your own changes at the top, and that's likely what you care most 
about anyway, no?

Also, if you didn't pull from me, and you decided that you needed to re-do 
your tree (let's say that you notice that one of your commits was bad 
_before_ you ask me to pull from your tree), then you'd also have an 
easier time re-creating your own development without that buggy change, 
exactly because _your_ tree wouldn't have my changed mixed up in it.

So your merges likely make git harder to use for you, not easier.

		Linus

Re: git pull on Linux/ACPI release tree

[Junio C Hamano]

Martin Langhoff <martin.langhoff-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> writes:

> On 1/9/06, Brown, Len <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>> Perhaps the tools should try to support what "a lot of people"
>> expect, rather than making "a lot of people" do extra work
>> because of the tools?
>
> I think it does. All the tricky stuff that David and Junio have been
> discussing is actually done very transparently by
>
>     git-rebase <upstream>
>
> Now, git-rebase uses git-format-patch <options> | git-am <options> so
> it sometimes has problems merging.

Careful.  I do not think rebase works across merges at all.

-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Martin Langhoff]

On 1/9/06, Brown, Len <len.brown-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> Perhaps the tools should try to support what "a lot of people"
> expect, rather than making "a lot of people" do extra work
> because of the tools?

I think it does. All the tricky stuff that David and Junio have been
discussing is actually done very transparently by

    git-rebase <upstream>

Now, git-rebase uses git-format-patch <options> | git-am <options> so
it sometimes has problems merging. In that case, you can choose to
either resolve the problem (see the doco for how to signal to git-am
that you've resolved a conflict) or to cancel the rebase. If you
choose to cancel the rebase, do

   cp .git/refs/heads/{<headname>,<headnamebadrebase>}
   cat .git/HEAD_ORIG > .git/refs/heads/<headname>
   git-reset --hard
   rm -fr .dotest

and you'll be back to where you started. Perhaps this could be rolled
into something like git-rebase --cancel to make it easier, but that's
about it. The toolchain definitely supports it.

cheers,


martin
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: git pull on Linux/ACPI release tree

[Linus Torvalds]

On Sun, 8 Jan 2006, Brown, Len wrote:
> 
> Is it possible for it git, like bk, to simply ignore merge commits in its summary output?

That's not the point. It does: "git log --no-merges" does exactly that.

But fire up "gitk" to watch the history, and see the difference.

> Note that "Auto-update from upstream" is just the place-holder comment
> embedded in the wrapper script in git/Documentation/howto/using-topic-branches.txt

That has absolutely nothing to do with anything. It's not the comment 
(which admittedly gives absolutely no information - but why should it, 
since the _commit_ itself has no information in it?)

It's like you have empty commits that don't do anything at all, except 
that they are worse, because they have two parents.

> I think that Tony's howto above captures two key requirements
> from all kernel maintainers -- which the exception of you --

No. Your commits make it harder for _everybody_ to track the history. 

A merge by definition "couples" the history of two branches. That's what a 
merge very fundamentally is. It ties two things together. But two things 
that don't have any connection to each other _shouldn't_ be tied together.

Just as an example: because of the extra merges, you've made all your 
commits dependent on what happened in my tree, with no real reason. So 
let's say that somebody reports that something broke in ACPI. Now you 
can't just go to the top of the ACPI history and work backwards - you'll 
have tied up the two histories so that they are intertwined.

And yes, you can always work around it, but there's just no point. And 
none of the other developers seem to need to do it. They do their 
development, and then they say "please pull". At that point the two 
histories are tied together, but now they are tied together for a 
_reason_. It was an intentional synchronization point.

An "automated pull" by definition has no reason. If it works automated, 
then the merge has zero semantic meaning. 

			Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: git pull on Linux/ACPI release tree

[Brown, Len]

 
>I know a lot of people react to this kind of usage with "what's the
>point of the source control system if you're just messing with patches
>in and out of the tree all the time" But as a subsystem maintainer,
>you deal with a lot of changes and it's important to get a pristine
>clean history when you push things to Linus.
>
>In fact, I do this so much that Linus's tree HEAD often equals my
>origin when he pulls.
>
>Merges really suck and I also hate it when the tree gets cluttered
>up with them, and Linus is right, ACPI is the worst offender here.
>
>Yes, we can grep the merges out of the shortlog or whatever, but that
>merging crap is still physically in the tree.
>
>Just don't do it.  Merge into a private branch for testing if you
>don't want to rebuild trees like I do, but push the clean tree to
>Linus.

Perhaps the tools should try to support what "a lot of people"
expect, rather than making "a lot of people" do extra work
because of the tools?

Call me old fashioned, but I believe that tools are supposed to
make work easier, not harder.

-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull on Linux/ACPI release tree

[Junio C Hamano]

"David S. Miller" <davem@davemloft.net> writes:

> I know a lot of people react to this kind of usage with "what's the
> point of the source control system if you're just messing with patches
> in and out of the tree all the time" But as a subsystem maintainer,
> you deal with a lot of changes and it's important to get a pristine
> clean history when you push things to Linus.

I suppose another possibility of rebasing the topic branches
every now and then amounts to almost the same thing; I think
your way is safer just in case something goes wrong.  Maybe
Catalin can give us a short tutorial on StGIT here?

> #!/bin/sh
> (cd $1; git format-patch --mbox origin) || exit 1
> for i in $1/*.txt
> do
>    sed 's/\[PATCH\] //' <$i >tmp.patch
>    git-applymbox -k tmp.patch || exit 1
> done

With "git format-patch --mbox -k origin", you would not need the
sed command.

Or doing it inside a single repository:

   #!/bin/sh 
   git branch -f anchor ;# mark the current head
   git reset --hard linus ;# rewind to linus head
   # extract them, and apply them -- I suspect origin and linus
   # are the same
   git format-patch --stdout -k origin anchor | git am -k -3

To check the results, since the patch you fed to "am" as a whole
should be fairly close to the difference between the linus head
and your resulting HEAD, parhaps:

   git diff $(git merge-base origin anchor) anchor |
       git apply --stat --summary >status.1
   git diff linus HEAD | git apply --stat --summary >status.2
   diff -u status.1 status.2

If you do not like the result, you can "git reset --hard anchor"
to come back to where you started.

* format-patch --stdout implies --mbox.

* -3 to "am" is optional and as a matter of taste.  If you want
  to resolve conflicts by hand to be sure, running "am" without
  it may be preferable.  Otherwise when a patch does not cleanly
  apply it would construct an appropriate merge base tree on the
  fly and runs a 3-way merge.

Re: git pull on Linux/ACPI release tree

[David S. Miller]

From: "Brown, Len" <len.brown@intel.com>
Date: Sun, 8 Jan 2006 02:47:30 -0500

> I'm probably a naïve git user -- but I expect I have a lot of company.
> If there is a better way of using the tool to get the job done,
> I'm certainly a willing customer with open ears.

What I do is simply build a new fresh tree if I feel the urge
to sync with the top of Linus's tree.  I use the script below
which I call "git suck".  It just sucks the patches out of
one tree and sticks them into another tree.  You go:

bash$ cd new-2.6
bash$ git suck ../foo-2.6

It preserves everything except the dates, and it's so incredibly
cheap and fast with GIT.

I know a lot of people react to this kind of usage with "what's the
point of the source control system if you're just messing with patches
in and out of the tree all the time" But as a subsystem maintainer,
you deal with a lot of changes and it's important to get a pristine
clean history when you push things to Linus.

In fact, I do this so much that Linus's tree HEAD often equals my
origin when he pulls.

Merges really suck and I also hate it when the tree gets cluttered
up with them, and Linus is right, ACPI is the worst offender here.

Yes, we can grep the merges out of the shortlog or whatever, but that
merging crap is still physically in the tree.

Just don't do it.  Merge into a private branch for testing if you
don't want to rebuild trees like I do, but push the clean tree to
Linus.

#!/bin/sh
#
# Usage: git suck path-to-tree
#
# Pull all patches relative to 'origin' from the tree specified
# and apply them to the current directory tree, keeping all changelog
# and authorship information identical.  It will update the dates
# of the changes of course.
(cd $1; git format-patch --mbox origin) || exit 1
for i in $1/*.txt
do
   sed 's/\[PATCH\] //' <$i >tmp.patch
   git-applymbox -k tmp.patch || exit 1
done

Re: git pull on Linux/ACPI release tree

[Catalin Marinas]

"Brown, Len" wrote:
>>I _really_ wish you wouldn't have those automatic merges.
>>
>>Why do you do them? They add nothing but ugly and unnecessary 
>>history, and in this pull, I think almost exactly half of the
>>commits were just these empty merges.
>
> Is it possible for it git, like bk, to simply ignore merge commits
> in its summary output?

As Junio suggested, you can have a look at StGIT
(http://www.procode.org/stgit/) for a different workflow. There is a
tutorial both on the web and in the doc/ directory but, anyway, it is
pretty similar to Quilt only that the patches are GIT commits.

In principle, you keep all the patches in a stack whose base is the
HEAD of Linus' kernel. You can indefinitely modify/push/pop the
patches and, once you are happy with the state of the stack, ask Linus
to pull using standard GIT commands (or mail them with 'stg
mail'). You can afterwards pull the latest changes from Linus using
'stg pull'. This operation pops the patches you have, advances the
base of the stack (so no "merge" message) and pushes your patches
back. Since pushing is done with a three-way merge, it detects whether
there are any upstream modifications to your patches (if not, all the
patches should become empty and safely removed from the stack).

You can also have a branch for upstream merges only and you can easily
cherry-pick patches or commits from other branches. This is quite
useful if you want to continue the work on your development branch
until Linus merges your patches.

-- 
Catalin

RE: git pull on Linux/ACPI release tree

[Brown, Len]

Hi Linus,

adding git-u79uwXL29TaiAVqoAR/hOK1cXZ9k6wlg@public.gmane.org

>> please pull this batch of trivial patches from: 
>> 
>> 
>git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release
>
>Len,
>
>I _really_ wish you wouldn't have those automatic merges.
>
>Why do you do them? They add nothing but ugly and unnecessary 
>history, and in this pull, I think almost exactly half of the
>commits were just these empty merges.

Is it possible for it git, like bk, to simply ignore merge commits in its summary output?

Note that "Auto-update from upstream" is just the place-holder comment
embedded in the wrapper script in git/Documentation/howto/using-topic-branches.txt
All instances of it here are from me manually updating --
the only "auto" happening here is the automatic insertion of that comment:-)

I think that Tony's howto above captures two key requirements
from all kernel maintainers -- which the exception of you --
who hang out  in the middle of the process rather than
at the top of the tree.

1. It is important that we be able (and encouraged, not discouraged)
to track the top of tree as closely as we have time to handle.
Divergence and conflicts are best handled as soon as they are noticed
and can be a huge pain if left to fester and discovered
only when it is time to push patches upstream.
Plus, tracking the top of tree means we force more folks to
track the top of tree, and so it gets more testing.  This is goodness.

Earlier in your release cycle when changes are appearing faster,
my need/desire to sync is greater than later in the cycle when changes
are smaller and infrequent.  On average, I think that one sync/day
from upstream is an entirely reasonable frequency.

2. It is also important that we be able to cherry pick individual patches
in our trees so that they don't block each other from going upstream.
Tony's using-topic-branches.txt above is the best way I know of doing that.
I think it is a big improvement over the bk model since I can have a simple
branch for each patch or group of patches rather than an entire repository
dedicatd to each.  But for this to work, I need to be able to update
any and all of the topic branches from upstream, and to merge them with
each other -- just like I could with BK.  Otherwise they become "dated"
in the time they were first integrated, and it is not convenient to do
simple apples/apples comparisons that are needed to debug and test.

I'm probably a naïve git user -- but I expect I have a lot of company.
If there is a better way of using the tool to get the job done,
I'm certainly a willing customer with open ears.

thanks,
-Len
-
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html