xen/tip.git and how it should function?

xen/tip.git and how it should function?

[Konrad Rzeszutek Wilk]

Hey Stefano, Boris, David and everybody else.


I figured that having a one Linux tree for all the maintainers to put Xen related
patches would help with:
 - Testing
 - Logistics with patches overlapping
 - More eyes - hence easier to check something.
 - Easier to backport all Xen changes.

In the past the workflow I had was to create at a certain point (rc3-ish) a
'stable' branch where I would put the patches for the next release. For example
the next one would be called 'stable/for-linus-3.12'.

During the rc3->rc7 time patches would accumulate there. Everytime a patch is put
in, the branch is called merged in the #linux-next branch. This means that linux-next
will always contain the latest from Linus tree and the latest for the next release.

Then when we are two weeks before the merge window I tag it. I create a nice signed tag
called 'stable/for-linus-3.12-rc0-tag' where I do a writeup of all patches. That is
what Linus gets in his GIT PULL.

Once the merge window is completed and we find bugs, we pile them up in this branch
(stable/for-linus-3.12). New features are put in 'stable/for-linus-3.13' .. and so
on.


This worked great for me - as I only had two bins to put things in. Oh and of course
you cannot rebase either one of those trees.

And sometimes I created an 'devel/for-XXX-3.13' which could be rebased and deleted. But
it kept the 'unstable' patches that eventually would trickle to the stable/for-XXX-3.13.

With two maintainers, this should be easy enough - each maintainer can put his or her
patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
to the tip tree.

Or just push them in #linux-next first to make sure they don't blow up.

Thoughts?

Re: xen/tip.git and how it should function?

[Stefano Stabellini]

On Tue, 30 Jul 2013, Konrad Rzeszutek Wilk wrote:
> Hey Stefano, Boris, David and everybody else.
> 
> 
> I figured that having a one Linux tree for all the maintainers to put Xen related
> patches would help with:
>  - Testing
>  - Logistics with patches overlapping
>  - More eyes - hence easier to check something.
>  - Easier to backport all Xen changes.
> 
> In the past the workflow I had was to create at a certain point (rc3-ish) a
> 'stable' branch where I would put the patches for the next release. For example
> the next one would be called 'stable/for-linus-3.12'.
> 
> During the rc3->rc7 time patches would accumulate there. Everytime a patch is put
> in, the branch is called merged in the #linux-next branch. This means that linux-next
> will always contain the latest from Linus tree and the latest for the next release.
> 
> Then when we are two weeks before the merge window I tag it. I create a nice signed tag
> called 'stable/for-linus-3.12-rc0-tag' where I do a writeup of all patches. That is
> what Linus gets in his GIT PULL.
> 
> Once the merge window is completed and we find bugs, we pile them up in this branch
> (stable/for-linus-3.12). New features are put in 'stable/for-linus-3.13' .. and so
> on.

It's worth mentioning that it's possible that new bugs have been
introduced by changes in other parts of the kernel, so we might have to
merge RC1 or RC2 back into stable/for-linus-3.12 before adding new patches
to fix the bugs.


> 
> This worked great for me - as I only had two bins to put things in. Oh and of course
> you cannot rebase either one of those trees.
> 
> And sometimes I created an 'devel/for-XXX-3.13' which could be rebased and deleted. But
> it kept the 'unstable' patches that eventually would trickle to the stable/for-XXX-3.13.
> 
> With two maintainers, this should be easy enough - each maintainer can put his or her
> patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> to the tip tree.

It might be worth giving an heads up to each others, at least before
pull requests to Linus.


> Or just push them in #linux-next first to make sure they don't blow up.

I think this should be required in all cases.

Re: xen/tip.git and how it should function?

[Konrad Rzeszutek Wilk]

On Wed, Jul 31, 2013 at 12:25:51PM +0100, Stefano Stabellini wrote:
> On Tue, 30 Jul 2013, Konrad Rzeszutek Wilk wrote:
> > Hey Stefano, Boris, David and everybody else.
> > 
> > 
> > I figured that having a one Linux tree for all the maintainers to put Xen related
> > patches would help with:
> >  - Testing
> >  - Logistics with patches overlapping
> >  - More eyes - hence easier to check something.
> >  - Easier to backport all Xen changes.
> > 
> > In the past the workflow I had was to create at a certain point (rc3-ish) a
> > 'stable' branch where I would put the patches for the next release. For example
> > the next one would be called 'stable/for-linus-3.12'.
> > 
> > During the rc3->rc7 time patches would accumulate there. Everytime a patch is put
> > in, the branch is called merged in the #linux-next branch. This means that linux-next
> > will always contain the latest from Linus tree and the latest for the next release.
> > 
> > Then when we are two weeks before the merge window I tag it. I create a nice signed tag
> > called 'stable/for-linus-3.12-rc0-tag' where I do a writeup of all patches. That is
> > what Linus gets in his GIT PULL.
> > 
> > Once the merge window is completed and we find bugs, we pile them up in this branch
> > (stable/for-linus-3.12). New features are put in 'stable/for-linus-3.13' .. and so
> > on.
> 
> It's worth mentioning that it's possible that new bugs have been
> introduced by changes in other parts of the kernel, so we might have to
> merge RC1 or RC2 back into stable/for-linus-3.12 before adding new patches
> to fix the bugs.

Right. I think it had happend a couple of times and there was no other way
of doing it. For those I told Linus in the writeup the reason why it was done that way.

> 
> 
> > 
> > This worked great for me - as I only had two bins to put things in. Oh and of course
> > you cannot rebase either one of those trees.
> > 
> > And sometimes I created an 'devel/for-XXX-3.13' which could be rebased and deleted. But
> > it kept the 'unstable' patches that eventually would trickle to the stable/for-XXX-3.13.
> > 
> > With two maintainers, this should be easy enough - each maintainer can put his or her
> > patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> > do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> > to the tip tree.
> 
> It might be worth giving an heads up to each others, at least before
> pull requests to Linus.

Correct.
> 
> 
> > Or just push them in #linux-next first to make sure they don't blow up.
> 
> I think this should be required in all cases.

Right.

I realized that at some point I had a disucssion with Linus about this as
I wanted it to be as easiest for him to pull stuff. I had done
this, but used a different naming (stable/for-XXX-version) and
inside that branch did some merges, but not always. Depending on
the patchsets.


On Tue, Nov 8, 2011 at 9:15 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
>
> The x86 folks are using a model where they separate the functionality in many
> branches and push those. For 3.2, I tried a mix of that (#stable/<subsystem or feature>-3.2)
> to make it easier to keep track of those patches but I am not sure if this complicates
> things. Some maintainers provide you with one branch where they have done the merging
> already and could do that too. Do you have any preference ?

I can go either way. I *don't* much like getting ten pull requests
with ten really trivial branches each with one or two commits - so
there's a limit to just how far you should go - but on the whole I
tend to prefer to get 5 clear pull requests that each have some kind
of "topic" to them over getting one big one.

So absolutely feel free to take <n> branches with similar trivial
stuff, and merge them together to then send me one "here's the trivial
stuff" request. HOWEVER, when you do that, don't merge them into
something recent - that has all the normal problems with bringing in
random "tree of the day", and tends to then just cause ugly double
merges that make history really unreadable.

Instead, do something like this (assuming you have branches "fix1",
"fix2" and "fix3"):

    git checkout -b fixes-for-linus v3.1
    git merge fix1 fix2 fix3

which basically starts at one very common ancestor (which probably
exists in all branches you are merging) and just ties the three
branches together. Or you can just start at one of the branches (ie
replace "v3.1" with "fix1")

The nice thing about keeping things in separate branches is that you
don't need to rebase them, and you can still decide that one of the
branches was just a bad idea - or maybe it's simply not ready yet -
and you can send the other branches.

The downside of separate branches is pretty obvious though - it needs
a fair amount of effort to actually keep things cleanly separated. The
effort does goes down a lot over time as you get used to doing things
that way and no longer spend a lot of time worrying about it, but it
never goes away entirely.  I'd suggest starting out only with *very*
well-specified topic branches, like a "fixes" for really *obvious*
fixes, and then special topic branches for very clear and independent
issues where you have no reason to believe that one topic will then
depend on another.

Because if you get topic branches that depend on each other, and you
end up merging in between them, the topic branches now actually end up
having a history that is *harder* to read than having had them in one
branch to begin with. So do avoid the notion that you'd make two topic
branches and then plan on merging them together (for any other reason
than the final merge for a pull request, of course - that's a merge
for the convenience of upstream, not for any code reasons, and is
different).

Examples of good topic branches outside of x86 can be found in the
acpi tree. Len used to do one topic per bugzilla entry, for example.
That really is a very clear setup, and there the topic branch names
actually are good documentation too - even later on, when they get
tied together in the merge commit that lists them all.

So I heartily encourage topic braches, but they do take some thought
and getting used to to be pleasant. And they can get really messy if
you do them wrong.

                  Linus

Re: xen/tip.git and how it should function?

[Konrad Rzeszutek Wilk]

> > With two maintainers, this should be easy enough - each maintainer can put his or her
> > patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> > do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> > to the tip tree.
> 
> It might be worth giving an heads up to each others, at least before
> pull requests to Linus.
> 


The other thing is that we could name them differently. Say:

	stable/for-linus-3.12.generic (for drivers/xen)
	stable/for-linux-3.12.x86     (arch/x86/xen...)
	stable/for-linux-3.12.arm

and then there are just three GIT PULLs during the merge window?

Re: xen/tip.git and how it should function?

[Stefano Stabellini]

On Wed, 31 Jul 2013, Konrad Rzeszutek Wilk wrote:
> > > With two maintainers, this should be easy enough - each maintainer can put his or her
> > > patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> > > do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> > > to the tip tree.
> > 
> > It might be worth giving an heads up to each others, at least before
> > pull requests to Linus.
> > 
> 
> 
> The other thing is that we could name them differently. Say:
> 
> 	stable/for-linus-3.12.generic (for drivers/xen)
> 	stable/for-linux-3.12.x86     (arch/x86/xen...)
> 	stable/for-linux-3.12.arm
> 
> and then there are just three GIT PULLs during the merge window?
>

I think it's best if we only have one pull request. Regardless of the
number of pull requests certainly we need only one linux-next branch.
Therefore if we go for tree branches, eventually they need to be merged
into a single tree anyway. The three branches strategy might work well
for internal coordination between us though:

- we work independently on our own branches as usual

- when we are ready we give an heads up to the others and we merge into
the single tree and linux-next (could the single tree be linux-next?)

- one of us send the pull request when ready, syncing with the others
first

Re: xen/tip.git and how it should function?

[Boris Ostrovsky]

On 07/31/2013 11:48 AM, Stefano Stabellini wrote:
> On Wed, 31 Jul 2013, Konrad Rzeszutek Wilk wrote:
>>>> With two maintainers, this should be easy enough - each maintainer can put his or her
>>>> patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
>>>> do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
>>>> to the tip tree.
>>> It might be worth giving an heads up to each others, at least before
>>> pull requests to Linus.
>>>
>>
>> The other thing is that we could name them differently. Say:
>>
>> 	stable/for-linus-3.12.generic (for drivers/xen)
>> 	stable/for-linux-3.12.x86     (arch/x86/xen...)
>> 	stable/for-linux-3.12.arm
>>
>> and then there are just three GIT PULLs during the merge window?
>>
> I think it's best if we only have one pull request. Regardless of the
> number of pull requests certainly we need only one linux-next branch.
> Therefore if we go for tree branches, eventually they need to be merged
> into a single tree anyway. The three branches strategy might work well
> for internal coordination between us though:
>
> - we work independently on our own branches as usual
>
> - when we are ready we give an heads up to the others and we merge into
> the single tree and linux-next (could the single tree be linux-next?)

I think it should be linux-next, I don't see why yet another staging tree
could be needed.

-boris

>
> - one of us send the pull request when ready, syncing with the others
> first
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: xen/tip.git and how it should function?

[David Vrabel]

On 31/07/13 16:48, Stefano Stabellini wrote:
> On Wed, 31 Jul 2013, Konrad Rzeszutek Wilk wrote:
>>>> With two maintainers, this should be easy enough - each maintainer can put his or her
>>>> patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
>>>> do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
>>>> to the tip tree.
>>>
>>> It might be worth giving an heads up to each others, at least before
>>> pull requests to Linus.
>>>
>>
>>
>> The other thing is that we could name them differently. Say:
>>
>> 	stable/for-linus-3.12.generic (for drivers/xen)
>> 	stable/for-linux-3.12.x86     (arch/x86/xen...)
>> 	stable/for-linux-3.12.arm
>>
>> and then there are just three GIT PULLs during the merge window?
>>
> 
> I think it's best if we only have one pull request. Regardless of the
> number of pull requests certainly we need only one linux-next branch.
> Therefore if we go for tree branches, eventually they need to be merged
> into a single tree anyway. The three branches strategy might work well
> for internal coordination between us though:
> 
> - we work independently on our own branches as usual
> 
> - when we are ready we give an heads up to the others and we merge into
> the single tree and linux-next (could the single tree be linux-next?)

You're not supposed to base anything off linux-next.  I can't see how it
would work either.

> - one of us send the pull request when ready, syncing with the others
> first

Excepting the above comment, this looks ok as a process to me.

David

Re: xen/tip.git and how it should function?

[Stefano Stabellini]

On Wed, 31 Jul 2013, David Vrabel wrote:
> On 31/07/13 16:48, Stefano Stabellini wrote:
> > On Wed, 31 Jul 2013, Konrad Rzeszutek Wilk wrote:
> >>>> With two maintainers, this should be easy enough - each maintainer can put his or her
> >>>> patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> >>>> do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> >>>> to the tip tree.
> >>>
> >>> It might be worth giving an heads up to each others, at least before
> >>> pull requests to Linus.
> >>>
> >>
> >>
> >> The other thing is that we could name them differently. Say:
> >>
> >> 	stable/for-linus-3.12.generic (for drivers/xen)
> >> 	stable/for-linux-3.12.x86     (arch/x86/xen...)
> >> 	stable/for-linux-3.12.arm
> >>
> >> and then there are just three GIT PULLs during the merge window?
> >>
> > 
> > I think it's best if we only have one pull request. Regardless of the
> > number of pull requests certainly we need only one linux-next branch.
> > Therefore if we go for tree branches, eventually they need to be merged
> > into a single tree anyway. The three branches strategy might work well
> > for internal coordination between us though:
> > 
> > - we work independently on our own branches as usual
> > 
> > - when we are ready we give an heads up to the others and we merge into
> > the single tree and linux-next (could the single tree be linux-next?)
> 
> You're not supposed to base anything off linux-next.  I can't see how it
> would work either.

I meant linux-next as the name of the branch we are going to use for
linux-next. Not the linux-next tree.

Re: xen/tip.git and how it should function?

[Konrad Rzeszutek Wilk]

On Wed, Jul 31, 2013 at 05:22:57PM +0100, Stefano Stabellini wrote:
> On Wed, 31 Jul 2013, David Vrabel wrote:
> > On 31/07/13 16:48, Stefano Stabellini wrote:
> > > On Wed, 31 Jul 2013, Konrad Rzeszutek Wilk wrote:
> > >>>> With two maintainers, this should be easy enough - each maintainer can put his or her
> > >>>> patches in their local tree (or some git tree). Create a 'stable/for-linus-3.12' and
> > >>>> do a git merge on his or her local tree, and push said 'stable/for-linus-3.12' up
> > >>>> to the tip tree.
> > >>>
> > >>> It might be worth giving an heads up to each others, at least before
> > >>> pull requests to Linus.
> > >>>
> > >>
> > >>
> > >> The other thing is that we could name them differently. Say:
> > >>
> > >> 	stable/for-linus-3.12.generic (for drivers/xen)
> > >> 	stable/for-linux-3.12.x86     (arch/x86/xen...)
> > >> 	stable/for-linux-3.12.arm
> > >>
> > >> and then there are just three GIT PULLs during the merge window?
> > >>
> > > 
> > > I think it's best if we only have one pull request. Regardless of the
> > > number of pull requests certainly we need only one linux-next branch.
> > > Therefore if we go for tree branches, eventually they need to be merged
> > > into a single tree anyway. The three branches strategy might work well
> > > for internal coordination between us though:
> > > 
> > > - we work independently on our own branches as usual
> > > 
> > > - when we are ready we give an heads up to the others and we merge into
> > > the single tree and linux-next (could the single tree be linux-next?)
> > 
> > You're not supposed to base anything off linux-next.  I can't see how it
> > would work either.
> 
> I meant linux-next as the name of the branch we are going to use for
> linux-next. Not the linux-next tree.

Right, as patches that are going to Linus are suppose to stew in
linux-next. At least to sort out any other tree dependencies.