[Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Anthony Liguori]

Hi,

I've posted an initial proposal for the 1.0 release on the wiki[1].

The release would be targeted for December 1st.  Unlike previous 
releases, I'm proposing that instead of forking master into a stable 
branch and releasing from there, we stop taking features into master and 
all work on stablizing master.

Historically, we forked stable for the releases simply because it was 
the least intrusive way to get us on a somewhat reasonable release 
schedule.  I think especially since we're targeting a 1.0, we're ready 
to get a bit more serious about releases.

I see the following advantages in using master for releases:

1) All maintainers are participating in the process which means releases 
are much less likely to be delayed due to lack of time from a single person.

2) Everyone (contributors) are forced to focus on stability which should 
improve the release's quality.

3) Feature development can still happen in maintainers trees.  I think 
this is an important step to moving to a merge-window based development 
model which I think will be our best way to scale long term.

Obviously, this will only work well if all the maintainers participate 
so I'd really like to hear what everyone thinks about this.

[1] http://wiki.qemu.org/Planning/1.0

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Blue Swirl]

On Thu, Aug 11, 2011 at 1:46 PM, Anthony Liguori <aliguori@us.ibm.com> wrote:
> Hi,
>
> I've posted an initial proposal for the 1.0 release on the wiki[1].
>
> The release would be targeted for December 1st.  Unlike previous releases,
> I'm proposing that instead of forking master into a stable branch and
> releasing from there, we stop taking features into master and all work on
> stablizing master.
>
> Historically, we forked stable for the releases simply because it was the
> least intrusive way to get us on a somewhat reasonable release schedule.  I
> think especially since we're targeting a 1.0, we're ready to get a bit more
> serious about releases.

Even more historically, with CVS, not forking was the only way.

> I see the following advantages in using master for releases:
>
> 1) All maintainers are participating in the process which means releases are
> much less likely to be delayed due to lack of time from a single person.
>
> 2) Everyone (contributors) are forced to focus on stability which should
> improve the release's quality.
>
> 3) Feature development can still happen in maintainers trees.  I think this
> is an important step to moving to a merge-window based development model
> which I think will be our best way to scale long term.
>
> Obviously, this will only work well if all the maintainers participate so
> I'd really like to hear what everyone thinks about this.
>
> [1] http://wiki.qemu.org/Planning/1.0

In general the idea is OK. Especially soft freeze could solve problems
like those qemu-ga inclusion had.

Two weeks for soft freeze would be close to OK but I think a month of
hard freeze is too long. With the previous releases, the problems in
stable were ironed out within a week or two. How about 2 + 2?

I'm not convinced about merge window model at least how it's used with
Linux kernel development. There will be a lot of breakage during the
merge window. Major changes at the same time would need major rebasing
efforts since they would be developed and merged within same time
frame. Also pretty strong coordination would be needed. We don't have
the luxury of infinite army of monkeys lead by genius who has
dedicated his entire life to the project like kernel has. I'd rather
try to keep the code base at (sort of) release quality at all times
and merge changes every now and then.

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Anthony Liguori]

On 08/11/2011 12:30 PM, Blue Swirl wrote:
> On Thu, Aug 11, 2011 at 1:46 PM, Anthony Liguori<aliguori@us.ibm.com>  wrote:
>> Hi,
>>
>> I've posted an initial proposal for the 1.0 release on the wiki[1].
>>
>> The release would be targeted for December 1st.  Unlike previous releases,
>> I'm proposing that instead of forking master into a stable branch and
>> releasing from there, we stop taking features into master and all work on
>> stablizing master.
>>
>> Historically, we forked stable for the releases simply because it was the
>> least intrusive way to get us on a somewhat reasonable release schedule.  I
>> think especially since we're targeting a 1.0, we're ready to get a bit more
>> serious about releases.
>
> Even more historically, with CVS, not forking was the only way.

Indeed :-)

>> 3) Feature development can still happen in maintainers trees.  I think this
>> is an important step to moving to a merge-window based development model
>> which I think will be our best way to scale long term.
>>
>> Obviously, this will only work well if all the maintainers participate so
>> I'd really like to hear what everyone thinks about this.
>>
>> [1] http://wiki.qemu.org/Planning/1.0
>
> In general the idea is OK. Especially soft freeze could solve problems
> like those qemu-ga inclusion had.
>
> Two weeks for soft freeze would be close to OK but I think a month of
> hard freeze is too long. With the previous releases, the problems in
> stable were ironed out within a week or two. How about 2 + 2?

I feel like 2 weeks was too short for this release and releases in the 
past.  Conversely, I feel like more than 2 weeks on a different branch 
is too long because people move on to shinier things.

That's why I'm proposing 4 weeks of hard freeze w/o opening up a new 
development branch.  That gives us more time to iron out any issues and 
makes sure people still keep focusing on stability.

I want to move to 4 weeks primarily because I want to increase the 
amount of release testing we do.  We haven't had a really active set of 
stable releases in a long time so for the most part, the .0 or .1 
release is what people are going to be using.  I think that means that 
we should put a bit more effort into making .0 be as strong as it can be.

> I'm not convinced about merge window model at least how it's used with
> Linux kernel development. There will be a lot of breakage during the
> merge window. Major changes at the same time would need major rebasing
> efforts since they would be developed and merged within same time
> frame. Also pretty strong coordination would be needed. We don't have
> the luxury of infinite army of monkeys lead by genius who has
> dedicated his entire life to the project like kernel has. I'd rather
> try to keep the code base at (sort of) release quality at all times
> and merge changes every now and then.

That's why I started with a 4 week freeze proposal :-)  It gives us a 
chance to try it out.  If it works well, maybe we try 6-8 weeks for 1.1. 
  If it doesn't, we can drop back down to 2.

As long as we're not making massive changes, I think its a good idea to 
experiment a bit.

Regards,

Anthony Liguori

>

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Gerd Hoffmann]

   Hi,

>> In general the idea is OK. Especially soft freeze could solve problems
>> like those qemu-ga inclusion had.
>>
>> Two weeks for soft freeze would be close to OK but I think a month of
>> hard freeze is too long. With the previous releases, the problems in
>> stable were ironed out within a week or two. How about 2 + 2?
>
> I feel like 2 weeks was too short for this release and releases in the
> past.

Well, one reason for the release process not being that smooth is that a 
big pile of stuff was merged just before 0.15-rc0.  First because there 
was a noticable backlog due to maintainers vacation, and second because 
a bunch of people wanted there stuff be merged for 0.15.

I think two weeks soft freeze and two weeks hard freeze should work 
reasonable well.  I think shorter release cycles will help too, so the 
pressure to get stuff in before the freeze is lower as the following 
release isn't that far away.

So how about this:

   - roughly four weeks development phase.
   - two weeks soft freeze (aka no big stuff).
   - two weeks hard freeze (aka bugfixes only).

Don't be too strict about this, if there is a reason to slip (xmas 
holiday season, summer vacation time, kvm forum, $bignewfeature took a 
bit longer to stabilize, $whateverelse) just stretch the development 
phase (or soft freeze) a bit.  We would probably end up with 4-5 
releases/year when following this model.

>> I'm not convinced about merge window model at least how it's used with
>> Linux kernel development.

Agree.  A short two-week merge window would work out nicely only if we 
would run a qemu-next so most issues can be sorted beforehand.  I don't 
think we have the man power to actually do that.

>> I'd rather
>> try to keep the code base at (sort of) release quality at all times
>> and merge changes every now and then.

Agree.  I think we are not that bad here today.  I'm running qemu from 
fresh master checkouts quite alot and I rarely hit bugs.

cheers,
   Gerd

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Blue Swirl]

On Thu, Aug 11, 2011 at 9:54 PM, Gerd Hoffmann <kraxel@redhat.com> wrote:
>  Hi,
>
>>> In general the idea is OK. Especially soft freeze could solve problems
>>> like those qemu-ga inclusion had.
>>>
>>> Two weeks for soft freeze would be close to OK but I think a month of
>>> hard freeze is too long. With the previous releases, the problems in
>>> stable were ironed out within a week or two. How about 2 + 2?
>>
>> I feel like 2 weeks was too short for this release and releases in the
>> past.
>
> Well, one reason for the release process not being that smooth is that a big
> pile of stuff was merged just before 0.15-rc0.  First because there was a
> noticable backlog due to maintainers vacation, and second because a bunch of
> people wanted there stuff be merged for 0.15.
>
> I think two weeks soft freeze and two weeks hard freeze should work
> reasonable well.  I think shorter release cycles will help too, so the
> pressure to get stuff in before the freeze is lower as the following release
> isn't that far away.
>
> So how about this:
>
>  - roughly four weeks development phase.
>  - two weeks soft freeze (aka no big stuff).
>  - two weeks hard freeze (aka bugfixes only).

This 50% duty cycle would mean that for half of the time, people only
work within their forked trees and then try to merge these forks. I
think the rate should be something like 80% merging, 20% freeze, which
(with one month freeze) would be close to current speed of two
releases per year.

I agree releasing more often is better, but for that to work, I'd go
for something like:
- 2 months development
- two weeks soft freeze
- fork stable rc0, release after 2 to 4 weeks

This would still give 80/20 rate at the expense of hard freeze only
affecting stable fork, yielding four releases per year.

> Don't be too strict about this, if there is a reason to slip (xmas holiday
> season, summer vacation time, kvm forum, $bignewfeature took a bit longer to
> stabilize, $whateverelse) just stretch the development phase (or soft
> freeze) a bit.  We would probably end up with 4-5 releases/year when
> following this model.
>
>>> I'm not convinced about merge window model at least how it's used with
>>> Linux kernel development.
>
> Agree.  A short two-week merge window would work out nicely only if we would
> run a qemu-next so most issues can be sorted beforehand.  I don't think we
> have the man power to actually do that.
>
>>> I'd rather
>>> try to keep the code base at (sort of) release quality at all times
>>> and merge changes every now and then.
>
> Agree.  I think we are not that bad here today.  I'm running qemu from fresh
> master checkouts quite alot and I rarely hit bugs.
>
> cheers,
>  Gerd
>

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Anthony Liguori]

On 08/12/2011 03:46 PM, Blue Swirl wrote:
> On Thu, Aug 11, 2011 at 9:54 PM, Gerd Hoffmann<kraxel@redhat.com>  wrote:
>>   Hi,
>>
>>>> In general the idea is OK. Especially soft freeze could solve problems
>>>> like those qemu-ga inclusion had.
>>>>
>>>> Two weeks for soft freeze would be close to OK but I think a month of
>>>> hard freeze is too long. With the previous releases, the problems in
>>>> stable were ironed out within a week or two. How about 2 + 2?
>>>
>>> I feel like 2 weeks was too short for this release and releases in the
>>> past.
>>
>> Well, one reason for the release process not being that smooth is that a big
>> pile of stuff was merged just before 0.15-rc0.  First because there was a
>> noticable backlog due to maintainers vacation, and second because a bunch of
>> people wanted there stuff be merged for 0.15.
>>
>> I think two weeks soft freeze and two weeks hard freeze should work
>> reasonable well.  I think shorter release cycles will help too, so the
>> pressure to get stuff in before the freeze is lower as the following release
>> isn't that far away.
>>
>> So how about this:
>>
>>   - roughly four weeks development phase.
>>   - two weeks soft freeze (aka no big stuff).
>>   - two weeks hard freeze (aka bugfixes only).
>
> This 50% duty cycle would mean that for half of the time, people only
> work within their forked trees and then try to merge these forks. I
> think the rate should be something like 80% merging, 20% freeze, which
> (with one month freeze) would be close to current speed of two
> releases per year.
>
> I agree releasing more often is better, but for that to work, I'd go
> for something like:
> - 2 months development
> - two weeks soft freeze
> - fork stable rc0, release after 2 to 4 weeks

Maybe something more like:

2 months development
-rc0 goes out (master enters soft feature freeze)
2 weeks development in master, stabilization and careful consideration 
of new features
-rc1 goes out (master enters hard feature freeze)
1 week stabilization
-rc2 goes out
1 week stabilization
-rc3 goes out, -rc3 becomes release

I think a shorter cycle could work better long term.  I think it needs 
to be done as part of the master branch though and I'd wait until 1.1 to 
implement it.

Regards,

Anthony Liguori

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Blue Swirl]

On Sun, Aug 14, 2011 at 1:46 PM, Anthony Liguori <anthony@codemonkey.ws> wrote:
> On 08/12/2011 03:46 PM, Blue Swirl wrote:
>>
>> On Thu, Aug 11, 2011 at 9:54 PM, Gerd Hoffmann<kraxel@redhat.com>  wrote:
>>>
>>>  Hi,
>>>
>>>>> In general the idea is OK. Especially soft freeze could solve problems
>>>>> like those qemu-ga inclusion had.
>>>>>
>>>>> Two weeks for soft freeze would be close to OK but I think a month of
>>>>> hard freeze is too long. With the previous releases, the problems in
>>>>> stable were ironed out within a week or two. How about 2 + 2?
>>>>
>>>> I feel like 2 weeks was too short for this release and releases in the
>>>> past.
>>>
>>> Well, one reason for the release process not being that smooth is that a
>>> big
>>> pile of stuff was merged just before 0.15-rc0.  First because there was a
>>> noticable backlog due to maintainers vacation, and second because a bunch
>>> of
>>> people wanted there stuff be merged for 0.15.
>>>
>>> I think two weeks soft freeze and two weeks hard freeze should work
>>> reasonable well.  I think shorter release cycles will help too, so the
>>> pressure to get stuff in before the freeze is lower as the following
>>> release
>>> isn't that far away.
>>>
>>> So how about this:
>>>
>>>  - roughly four weeks development phase.
>>>  - two weeks soft freeze (aka no big stuff).
>>>  - two weeks hard freeze (aka bugfixes only).
>>
>> This 50% duty cycle would mean that for half of the time, people only
>> work within their forked trees and then try to merge these forks. I
>> think the rate should be something like 80% merging, 20% freeze, which
>> (with one month freeze) would be close to current speed of two
>> releases per year.
>>
>> I agree releasing more often is better, but for that to work, I'd go
>> for something like:
>> - 2 months development
>> - two weeks soft freeze
>> - fork stable rc0, release after 2 to 4 weeks
>
> Maybe something more like:
>
> 2 months development
> -rc0 goes out (master enters soft feature freeze)

Why an rc0 at this point? 0.15-rc0 was in a bad shape because it was
forked just after heavy development (ga etc). I'd nominate release
candidates only after soft freeze, then there would not be any major
changes. Though a rc0 could attract testing efforts from outside and
for those, the earlier the better.

> 2 weeks development in master, stabilization and careful consideration of
> new features
> -rc1 goes out (master enters hard feature freeze)
> 1 week stabilization
> -rc2 goes out
> 1 week stabilization
> -rc3 goes out, -rc3 becomes release

So at this point master would be released? What's the difference in
time between rc3 and release?

Overall this would only give a duty cycle of 67%. For 4 weeks total
freeze, the development would need to be 4 months for an 80% duty
cycle. But I think this version could work too.

> I think a shorter cycle could work better long term.  I think it needs to be
> done as part of the master branch though and I'd wait until 1.1 to implement
> it.
>
> Regards,
>
> Anthony Liguori
>

Re: [Qemu-devel] [RFC] Planning for 1.0 (and freezing the master branch)

[Anthony Liguori]

On 08/14/2011 02:30 PM, Blue Swirl wrote:>> Maybe something more like:
>>
>> 2 months development
>> -rc0 goes out (master enters soft feature freeze)
>
> Why an rc0 at this point? 0.15-rc0 was in a bad shape because it was
> forked just after heavy development (ga etc).

It could be called -beta1 instead of -rc0.  We just need to tag it with 
something.

> I'd nominate release
> candidates only after soft freeze, then there would not be any major
> changes. Though a rc0 could attract testing efforts from outside and
> for those, the earlier the better.
>
>> 2 weeks development in master, stabilization and careful consideration of
>> new features
>> -rc1 goes out (master enters hard feature freeze)
>> 1 week stabilization
>> -rc2 goes out
>> 1 week stabilization
>> -rc3 goes out, -rc3 becomes release
>
> So at this point master would be released? What's the difference in
> time between rc3 and release?

Ideally, nothing.  Having an -rc3 is just a conservative mechanism to 
make sure that there is an absolute final call for testing before teh 
release.

>
> Overall this would only give a duty cycle of 67%. For 4 weeks total
> freeze, the development would need to be 4 months for an 80% duty
> cycle. But I think this version could work too.

Yeah, that's more or less what I'm proposing for 1.0 :-)

Regards,

Anthony Liguori

>
>> I think a shorter cycle could work better long term.  I think it needs to be
>> done as part of the master branch though and I'd wait until 1.1 to implement
>> it.
>>
>> Regards,
>>
>> Anthony Liguori
>>
>