[RFC] Tweaking the release process for Xen 4.6

[RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

Hi, all

With Xen 4.5 released on Jan 14 and the open of Xen 4.6 (commit
0082626f35, Jan 6), we are now one month into 4.6 development window.

This is an email to kick off a discussion with regard to our release
process.The goal is to create a smoother workflow for everyone involved.
Emails to track features will be sent separately. 

# Xen 4.5 time frame retrospect

Xen 4.5 time frame is as followed.

  Development start: 21 Feb  2014
  Feature freeze:    24 Sept 2014
  RC 1:              24 Oct  2014
  RC 2:              11 Nov  2014
  RC 3:              3  Dec  2014
  RC 4:              15 Dec  2014
  Release:           14 Jan  2015

Counting from the point that we forked the tree, it took ~11 months to
ship 4.5. The time spent on development was 7 months (Feb 21 to Sept
24), and the time spent on freeze was ~4 months (Sept 24 to Jan 6).

The good thing was that code quality was ensured, the downside was that
such long freeze made upstreaming new features harder for contributors
-- they need to rush to get their features at least posted in
development window, then argue for whether it's OK to get in after soft
freeze; Otherwise  they risk waiting several more months.

# Scrapping soft freeze

I think existing model (development + freeze) works well in general,
but it would be better if we can shorten the time frame a bit, or try
harder to stick to the 9 months promise.

I would like to propose a tweak: scrap the soft freeze. The soft freeze
is used to allow features posted during development window to get merged
in time for a certain release. However this practice leads to longer
freeze period (new features means more testing, hence longer freeze).

If we scrap the soft freeze, there are several pros:

1. We only need to harden existing features in freeze. That would
   probably shorten the time needed for freeze, or at least keep
   everything within 3 months time frame.

2. Contributors with big features to upstream would not need to rebase
   aggressively because they don't need to worry about features that
   go in between soft freeze and hard freeze.

3. Contributors can accelerate their upstreaming effort by benefiting
   from the possible shorter freeze time.

I think this tweaked process can help make the development flow
smoother. We release in a more timely manner and contributors have less
work to do.

The cons are of course we have shorter time in freeze and lead to
possible less harden code. But I don't really see that as a big
issue, given that we a) we still set aside 3 months for it, b) we
only need to harden existing code.

What contributors need to do with this changed process: develop and
upstream features as usual in development window, but no more arguing
whether a feature should go in after freeze.

What maintainers / committers need to do with this changed process:
"business as usual" during development window, no more new feature
after freeze, only bug fixes can go in.

# Proposed Xen 4.6 time frame

With the above proposal in mind, here is my proposed time frame for
Xen 4.6 release:

  Development start: 6  Jan 2015
  Feature freeze:    10 Jul 2015
  Release date:      9  Oct 2015 (could release earlier)

Note that the release date is not a goal. It is more like a deadline
we try to keep up with. We might be well able to release earlier if
everything is in good shape.

Any thought on this tweaked process? Comments are welcome.

Wei.

Re: [RFC] Tweaking the release process for Xen 4.6

[George Dunlap]

On Tue, Feb 10, 2015 at 10:04 AM, Wei Liu <wei.liu2@citrix.com> wrote:
> Hi, all
>
> With Xen 4.5 released on Jan 14 and the open of Xen 4.6 (commit
> 0082626f35, Jan 6), we are now one month into 4.6 development window.
>
> This is an email to kick off a discussion with regard to our release
> process.The goal is to create a smoother workflow for everyone involved.
> Emails to track features will be sent separately.
>
> # Xen 4.5 time frame retrospect
>
> Xen 4.5 time frame is as followed.
>
>   Development start: 21 Feb  2014
>   Feature freeze:    24 Sept 2014
>   RC 1:              24 Oct  2014
>   RC 2:              11 Nov  2014
>   RC 3:              3  Dec  2014
>   RC 4:              15 Dec  2014
>   Release:           14 Jan  2015
>
> Counting from the point that we forked the tree, it took ~11 months to
> ship 4.5. The time spent on development was 7 months (Feb 21 to Sept
> 24), and the time spent on freeze was ~4 months (Sept 24 to Jan 6).
>
> The good thing was that code quality was ensured, the downside was that
> such long freeze made upstreaming new features harder for contributors
> -- they need to rush to get their features at least posted in
> development window, then argue for whether it's OK to get in after soft
> freeze; Otherwise  they risk waiting several more months.
>
> # Scrapping soft freeze
>
> I think existing model (development + freeze) works well in general,
> but it would be better if we can shorten the time frame a bit, or try
> harder to stick to the 9 months promise.
>
> I would like to propose a tweak: scrap the soft freeze. The soft freeze
> is used to allow features posted during development window to get merged
> in time for a certain release. However this practice leads to longer
> freeze period (new features means more testing, hence longer freeze).
>
> If we scrap the soft freeze, there are several pros:
>
> 1. We only need to harden existing features in freeze. That would
>    probably shorten the time needed for freeze, or at least keep
>    everything within 3 months time frame.
>
> 2. Contributors with big features to upstream would not need to rebase
>    aggressively because they don't need to worry about features that
>    go in between soft freeze and hard freeze.
>
> 3. Contributors can accelerate their upstreaming effort by benefiting
>    from the possible shorter freeze time.
>
> I think this tweaked process can help make the development flow
> smoother. We release in a more timely manner and contributors have less
> work to do.
>
> The cons are of course we have shorter time in freeze and lead to
> possible less harden code. But I don't really see that as a big
> issue, given that we a) we still set aside 3 months for it, b) we
> only need to harden existing code.
>
> What contributors need to do with this changed process: develop and
> upstream features as usual in development window, but no more arguing
> whether a feature should go in after freeze.
>
> What maintainers / committers need to do with this changed process:
> "business as usual" during development window, no more new feature
> after freeze, only bug fixes can go in.
>
> # Proposed Xen 4.6 time frame
>
> With the above proposal in mind, here is my proposed time frame for
> Xen 4.6 release:
>
>   Development start: 6  Jan 2015
>   Feature freeze:    10 Jul 2015
>   Release date:      9  Oct 2015 (could release earlier)
>
> Note that the release date is not a goal. It is more like a deadline
> we try to keep up with. We might be well able to release earlier if
> everything is in good shape.
>
> Any thought on this tweaked process? Comments are welcome.

I think it's certainly worth a try.

 -George

Re: [RFC] Tweaking the release process for Xen 4.6

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 3173 bytes --]

On Tue, 2015-02-10 at 11:11 -0500, George Dunlap wrote:
> On Tue, Feb 10, 2015 at 10:04 AM, Wei Liu <wei.liu2@citrix.com> wrote:

> > # Scrapping soft freeze
> >
> > I think existing model (development + freeze) works well in general,
> > but it would be better if we can shorten the time frame a bit, or try
> > harder to stick to the 9 months promise.
> >
> > I would like to propose a tweak: scrap the soft freeze. The soft freeze
> > is used to allow features posted during development window to get merged
> > in time for a certain release. However this practice leads to longer
> > freeze period (new features means more testing, hence longer freeze).
> >
> > If we scrap the soft freeze, there are several pros:
> >
> > 1. We only need to harden existing features in freeze. That would
> >    probably shorten the time needed for freeze, or at least keep
> >    everything within 3 months time frame.
> >
> > 2. Contributors with big features to upstream would not need to rebase
> >    aggressively because they don't need to worry about features that
> >    go in between soft freeze and hard freeze.
> >
> > 3. Contributors can accelerate their upstreaming effort by benefiting
> >    from the possible shorter freeze time.
> >
> > I think this tweaked process can help make the development flow
> > smoother. We release in a more timely manner and contributors have less
> > work to do.
> >
> > The cons are of course we have shorter time in freeze and lead to
> > possible less harden code. But I don't really see that as a big
> > issue, given that we a) we still set aside 3 months for it, b) we
> > only need to harden existing code.
> >
> > What contributors need to do with this changed process: develop and
> > upstream features as usual in development window, but no more arguing
> > whether a feature should go in after freeze.
> >
> > What maintainers / committers need to do with this changed process:
> > "business as usual" during development window, no more new feature
> > after freeze, only bug fixes can go in.
> >
> > # Proposed Xen 4.6 time frame
> >
> > With the above proposal in mind, here is my proposed time frame for
> > Xen 4.6 release:
> >
> >   Development start: 6  Jan 2015
> >   Feature freeze:    10 Jul 2015
> >   Release date:      9  Oct 2015 (could release earlier)
> >
> > Note that the release date is not a goal. It is more like a deadline
> > we try to keep up with. We might be well able to release earlier if
> > everything is in good shape.
> >
> > Any thought on this tweaked process? Comments are welcome.
> 
> I think it's certainly worth a try.
> 
It does indeed, IMHO.

One thing that we did, at least for the past two cycles, was to have two
distinct events:
 - feature freeze, starting from which no more new features where 
   accepted
 - code freeze, starting from which no more code, apart from bug fixes, 
   where accepted

Genuine question: do we still want this distinction? Maybe getting rid
of it, or cutting it as short as possible/practical could help achieve
what you're after (i.e., less time in freeze)?

Regards,
Dario


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [RFC] Tweaking the release process for Xen 4.6

[George Dunlap]

On Tue, Feb 10, 2015 at 11:47 AM, Dario Faggioli
<dario.faggioli@citrix.com> wrote:
> On Tue, 2015-02-10 at 11:11 -0500, George Dunlap wrote:
>> On Tue, Feb 10, 2015 at 10:04 AM, Wei Liu <wei.liu2@citrix.com> wrote:
>
>> > # Scrapping soft freeze
>> >
>> > I think existing model (development + freeze) works well in general,
>> > but it would be better if we can shorten the time frame a bit, or try
>> > harder to stick to the 9 months promise.
>> >
>> > I would like to propose a tweak: scrap the soft freeze. The soft freeze
>> > is used to allow features posted during development window to get merged
>> > in time for a certain release. However this practice leads to longer
>> > freeze period (new features means more testing, hence longer freeze).
>> >
>> > If we scrap the soft freeze, there are several pros:
>> >
>> > 1. We only need to harden existing features in freeze. That would
>> >    probably shorten the time needed for freeze, or at least keep
>> >    everything within 3 months time frame.
>> >
>> > 2. Contributors with big features to upstream would not need to rebase
>> >    aggressively because they don't need to worry about features that
>> >    go in between soft freeze and hard freeze.
>> >
>> > 3. Contributors can accelerate their upstreaming effort by benefiting
>> >    from the possible shorter freeze time.
>> >
>> > I think this tweaked process can help make the development flow
>> > smoother. We release in a more timely manner and contributors have less
>> > work to do.
>> >
>> > The cons are of course we have shorter time in freeze and lead to
>> > possible less harden code. But I don't really see that as a big
>> > issue, given that we a) we still set aside 3 months for it, b) we
>> > only need to harden existing code.
>> >
>> > What contributors need to do with this changed process: develop and
>> > upstream features as usual in development window, but no more arguing
>> > whether a feature should go in after freeze.
>> >
>> > What maintainers / committers need to do with this changed process:
>> > "business as usual" during development window, no more new feature
>> > after freeze, only bug fixes can go in.
>> >
>> > # Proposed Xen 4.6 time frame
>> >
>> > With the above proposal in mind, here is my proposed time frame for
>> > Xen 4.6 release:
>> >
>> >   Development start: 6  Jan 2015
>> >   Feature freeze:    10 Jul 2015
>> >   Release date:      9  Oct 2015 (could release earlier)
>> >
>> > Note that the release date is not a goal. It is more like a deadline
>> > we try to keep up with. We might be well able to release earlier if
>> > everything is in good shape.
>> >
>> > Any thought on this tweaked process? Comments are welcome.
>>
>> I think it's certainly worth a try.
>>
> It does indeed, IMHO.
>
> One thing that we did, at least for the past two cycles, was to have two
> distinct events:
>  - feature freeze, starting from which no more new features where
>    accepted
>  - code freeze, starting from which no more code, apart from bug fixes,
>    where accepted
>
> Genuine question: do we still want this distinction? Maybe getting rid
> of it, or cutting it as short as possible/practical could help achieve
> what you're after (i.e., less time in freeze)?

I think he answers that here:

>> > # Scrapping soft freeze
>> >
>> > I would like to propose a tweak: scrap the soft freeze. The soft freeze
>> > is used to allow features posted during development window to get merged
>> > in time for a certain release. However this practice leads to longer
>> > freeze period (new features means more testing, hence longer freeze).

 -George

Re: [RFC] Tweaking the release process for Xen 4.6

[Lars Kurth]

Wei,
could you explain what you mean by soft freeze and hard freeze.
Regards
Lars
 
> On 10 Feb 2015, at 15:04, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> Hi, all
> 
> With Xen 4.5 released on Jan 14 and the open of Xen 4.6 (commit
> 0082626f35, Jan 6), we are now one month into 4.6 development window.
> 
> This is an email to kick off a discussion with regard to our release
> process.The goal is to create a smoother workflow for everyone involved.
> Emails to track features will be sent separately. 
> 
> # Xen 4.5 time frame retrospect
> 
> Xen 4.5 time frame is as followed.
> 
>  Development start: 21 Feb  2014
>  Feature freeze:    24 Sept 2014
>  RC 1:              24 Oct  2014
>  RC 2:              11 Nov  2014
>  RC 3:              3  Dec  2014
>  RC 4:              15 Dec  2014
>  Release:           14 Jan  2015
> 
> Counting from the point that we forked the tree, it took ~11 months to
> ship 4.5. The time spent on development was 7 months (Feb 21 to Sept
> 24), and the time spent on freeze was ~4 months (Sept 24 to Jan 6).
> 
> The good thing was that code quality was ensured, the downside was that
> such long freeze made upstreaming new features harder for contributors
> -- they need to rush to get their features at least posted in
> development window, then argue for whether it's OK to get in after soft
> freeze; Otherwise  they risk waiting several more months.
> 
> # Scrapping soft freeze
> 
> I think existing model (development + freeze) works well in general,
> but it would be better if we can shorten the time frame a bit, or try
> harder to stick to the 9 months promise.
> 
> I would like to propose a tweak: scrap the soft freeze. The soft freeze
> is used to allow features posted during development window to get merged
> in time for a certain release. However this practice leads to longer
> freeze period (new features means more testing, hence longer freeze).
> 
> If we scrap the soft freeze, there are several pros:
> 
> 1. We only need to harden existing features in freeze. That would
>   probably shorten the time needed for freeze, or at least keep
>   everything within 3 months time frame.
> 
> 2. Contributors with big features to upstream would not need to rebase
>   aggressively because they don't need to worry about features that
>   go in between soft freeze and hard freeze.
> 
> 3. Contributors can accelerate their upstreaming effort by benefiting
>   from the possible shorter freeze time.
> 
> I think this tweaked process can help make the development flow
> smoother. We release in a more timely manner and contributors have less
> work to do.
> 
> The cons are of course we have shorter time in freeze and lead to
> possible less harden code. But I don't really see that as a big
> issue, given that we a) we still set aside 3 months for it, b) we
> only need to harden existing code.
> 
> What contributors need to do with this changed process: develop and
> upstream features as usual in development window, but no more arguing
> whether a feature should go in after freeze.
> 
> What maintainers / committers need to do with this changed process:
> "business as usual" during development window, no more new feature
> after freeze, only bug fixes can go in.
> 
> # Proposed Xen 4.6 time frame
> 
> With the above proposal in mind, here is my proposed time frame for
> Xen 4.6 release:
> 
>  Development start: 6  Jan 2015
>  Feature freeze:    10 Jul 2015
>  Release date:      9  Oct 2015 (could release earlier)
> 
> Note that the release date is not a goal. It is more like a deadline
> we try to keep up with. We might be well able to release earlier if
> everything is in good shape.
> 
> Any thought on this tweaked process? Comments are welcome.
> 
> Wei.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Tue, Feb 10, 2015 at 06:04:10PM +0000, Lars Kurth wrote:
> Wei,
> could you explain what you mean by soft freeze and hard freeze.
> Regards

They correspond to feature freeze and code freeze respectively.

A bit more background information for developers that are new to the
community. In 4.5 and 4.4 there were two freezes, one called feature
freeze (soft freeze), the other called code freeze (hard freeze).

In 4.4 and 4.5, feature posted before feature freeze but didn't get in
before feature freeze still have a chance to get merged, provided that
1) it's in good shape, 2) there is a strong argument that it should be
merged. Code freeze is the strict freezing point, when no more new
features can go in after that.

That model has its pros and cons. I proposed in my previous email to
just scrap the feature freeze and go with only code freeze, with the
hope to create shorter release cycle and smoother workflow.

Wei.

Re: [RFC] Tweaking the release process for Xen 4.6

[Lars Kurth]

Wei,

thanks for clarifying. I don't think you answered all the practical
implications of your proposal though.
A) Do we start making RC's straight after the Feature Freeze?
B) Will there still be exceptions? I am assuming yes
C) Do you expect a gradual decrease of what bug fixes are allowed after
the Feature Freeze?

If the answer is yes to A), we would have more RC's and more test days.
That means slightly more overhead but also - if we get good community
engagement - better quality.

If the answer is yes to C), the Release Manager would in essence have more
flexibility, but also has to make more decisions. The same applies to B).
 

I am not convinced that the hard freeze really would go away. It would
merely not be as cleanly identifiable and we would gradually move from
soft to hard freeze.

If the answer was NO to any of the questions above, I think you would need
to elaborate what the alternatives are.


Cheers
Lars

On 10/02/2015 18:29, "Wei Liu" <wei.liu2@citrix.com> wrote:

>On Tue, Feb 10, 2015 at 06:04:10PM +0000, Lars Kurth wrote:
>> Wei,
>> could you explain what you mean by soft freeze and hard freeze.
>> Regards
>
>They correspond to feature freeze and code freeze respectively.
>
>A bit more background information for developers that are new to the
>community. In 4.5 and 4.4 there were two freezes, one called feature
>freeze (soft freeze), the other called code freeze (hard freeze).
>
>In 4.4 and 4.5, feature posted before feature freeze but didn't get in
>before feature freeze still have a chance to get merged, provided that
>1) it's in good shape, 2) there is a strong argument that it should be
>merged. Code freeze is the strict freezing point, when no more new
>features can go in after that.
>
>That model has its pros and cons. I proposed in my previous email to
>just scrap the feature freeze and go with only code freeze, with the
>hope to create shorter release cycle and smoother workflow.
>
>Wei.

Re: [RFC] Tweaking the release process for Xen 4.6

[Jan Beulich]

>>> On 10.02.15 at 16:04, <wei.liu2@citrix.com> wrote:
> # Scrapping soft freeze
> 
> I think existing model (development + freeze) works well in general,
> but it would be better if we can shorten the time frame a bit, or try
> harder to stick to the 9 months promise.
> 
> I would like to propose a tweak: scrap the soft freeze. The soft freeze
> is used to allow features posted during development window to get merged
> in time for a certain release. However this practice leads to longer
> freeze period (new features means more testing, hence longer freeze).
> 
> If we scrap the soft freeze, there are several pros:
> 
> 1. We only need to harden existing features in freeze. That would
>    probably shorten the time needed for freeze, or at least keep
>    everything within 3 months time frame.
> 
> 2. Contributors with big features to upstream would not need to rebase
>    aggressively because they don't need to worry about features that
>    go in between soft freeze and hard freeze.
> 
> 3. Contributors can accelerate their upstreaming effort by benefiting
>    from the possible shorter freeze time.
> 
> I think this tweaked process can help make the development flow
> smoother. We release in a more timely manner and contributors have less
> work to do.
> 
> The cons are of course we have shorter time in freeze and lead to
> possible less harden code. But I don't really see that as a big
> issue, given that we a) we still set aside 3 months for it, b) we
> only need to harden existing code.

While I agree with others that giving this is try may certainly be
worth it, I'd like to point out another aspect: Consider a series that
has undergone 9 revisions at the time of the freeze, and the 10th
would be the final one that could go in. It would be rather
unfortunate to tell the submitter that now he's got to wait for 3 or
more months until that code can go in. Or, in order to avoid that
situation, it could increase pressure on maintainers to accept code
without all issues addressed, or with only a relaxed review done.
At least as long as we're suffering from limited reviewing capacity
and at least as long as we're having contributors who step away
from their code the moment their base submission got committed,
we should at least take this aspect into consideration.

Jan

Re: [RFC] Tweaking the release process for Xen 4.6

[Andrew Cooper]

On 11/02/15 08:05, Jan Beulich wrote:
>>>> On 10.02.15 at 16:04, <wei.liu2@citrix.com> wrote:
>> # Scrapping soft freeze
>>
>> I think existing model (development + freeze) works well in general,
>> but it would be better if we can shorten the time frame a bit, or try
>> harder to stick to the 9 months promise.
>>
>> I would like to propose a tweak: scrap the soft freeze. The soft freeze
>> is used to allow features posted during development window to get merged
>> in time for a certain release. However this practice leads to longer
>> freeze period (new features means more testing, hence longer freeze).
>>
>> If we scrap the soft freeze, there are several pros:
>>
>> 1. We only need to harden existing features in freeze. That would
>>    probably shorten the time needed for freeze, or at least keep
>>    everything within 3 months time frame.
>>
>> 2. Contributors with big features to upstream would not need to rebase
>>    aggressively because they don't need to worry about features that
>>    go in between soft freeze and hard freeze.
>>
>> 3. Contributors can accelerate their upstreaming effort by benefiting
>>    from the possible shorter freeze time.
>>
>> I think this tweaked process can help make the development flow
>> smoother. We release in a more timely manner and contributors have less
>> work to do.
>>
>> The cons are of course we have shorter time in freeze and lead to
>> possible less harden code. But I don't really see that as a big
>> issue, given that we a) we still set aside 3 months for it, b) we
>> only need to harden existing code.
> While I agree with others that giving this is try may certainly be
> worth it, I'd like to point out another aspect: Consider a series that
> has undergone 9 revisions at the time of the freeze, and the 10th
> would be the final one that could go in. It would be rather
> unfortunate to tell the submitter that now he's got to wait for 3 or
> more months until that code can go in. Or, in order to avoid that
> situation, it could increase pressure on maintainers to accept code
> without all issues addressed, or with only a relaxed review done.
> At least as long as we're suffering from limited reviewing capacity
> and at least as long as we're having contributors who step away
> from their code the moment their base submission got committed,
> we should at least take this aspect into consideration.

At the point of code freeze, the maintainers can take a decision as to
whether a pending series is just a few tweaks away from acceptance or
not.  A v1 rewriting Xen in C++ is unlikely to qualify, whereas a v10
which is mostly acked/reviewed and only has a few comments remaining is
likely to be able to be turned around in a quick time, and can be
permitted to go for one further round and be accepted.

The other advantage with a shorter release cycle and freeze is that, for
items which do miss the freeze, there is less time until staging opens
again for submissions.

One thing which could help is, where appropriate, leading patches on a
series being accepted even if the series as a whole may have issues. 
Leading patches tend to be cleanup, rather than the meat of a series. 
Taking these patches ahead of the series as a whole would reduce traffic
to xen-devel, reduce the cognitive review load of working out whether it
had changed since last time, and reduce the amount of effort required to
maintain the series as changes are made.  Of course, this should fall
strictly under the prerogative of the maintainer as to whether this is
safe to do, but I feel it could safely happen rather more than it
currently does.

~Andrew

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Tue, Feb 10, 2015 at 07:29:30PM +0000, Lars Kurth wrote:
> Wei,
> 
> thanks for clarifying. I don't think you answered all the practical
> implications of your proposal though.
> A) Do we start making RC's straight after the Feature Freeze?

Yes. Wait for our push gate to clear then arrange RC. Basically that's
a fortnight after the freeze.

Previously we stated the freeze time is three months but in practice it
was always longer than that. It looks to me we didn't set aside enough
leeway for various things (holidays, events and  unexpected bugs etc).

> B) Will there still be exceptions? I am assuming yes

Yes. The bar would be high. The principle is only code that is 1)
self-contained, 2) has little possibility of breaking existing system
can be accepted. Maintainers can actively endorse a series if he wants
to.

> C) Do you expect a gradual decrease of what bug fixes are allowed after
> the Feature Freeze?
> 

Yes. At one point we would want declare non-critical, non-blocker bugs as
known issues and release. Just like what we did in last two releases.

> If the answer is yes to A), we would have more RC's and more test days.
> That means slightly more overhead but also - if we get good community
> engagement - better quality.
> 

The three month freeze is not a goal. If there is no critical bug after
several RCs (1~2 months) we just release and reopen for development.

> If the answer is yes to C), the Release Manager would in essence have more
> flexibility, but also has to make more decisions. The same applies to B).
>  

>From what I saw during last two release cycles, maintainers generally
have very good sense on what is risky to accept and what is not. The
occasions that required release manager to make hard decision were
limited. And with the help from relevant maintainers it was possible to
come to a rational resolution in a timely manner.

> 
> I am not convinced that the hard freeze really would go away. It would
> merely not be as cleanly identifiable and we would gradually move from
> soft to hard freeze.
> 

In a way, the hard (you mean soft?) freeze is still there, but the bar
is higher.  As you said, they are not that clearly identifiable.  We
just don't advertise the period which allows patches to get merged in
last minute. Hopefully this can prompt people to submit their patches
earlier and not have a mentality to rush everything in during last
minute, Resulting in a shorter freeze time and release cycle.

Wei.

> If the answer was NO to any of the questions above, I think you would need
> to elaborate what the alternatives are.
> 
> 
> Cheers
> Lars
> 
> On 10/02/2015 18:29, "Wei Liu" <wei.liu2@citrix.com> wrote:
> 
> >On Tue, Feb 10, 2015 at 06:04:10PM +0000, Lars Kurth wrote:
> >> Wei,
> >> could you explain what you mean by soft freeze and hard freeze.
> >> Regards
> >
> >They correspond to feature freeze and code freeze respectively.
> >
> >A bit more background information for developers that are new to the
> >community. In 4.5 and 4.4 there were two freezes, one called feature
> >freeze (soft freeze), the other called code freeze (hard freeze).
> >
> >In 4.4 and 4.5, feature posted before feature freeze but didn't get in
> >before feature freeze still have a chance to get merged, provided that
> >1) it's in good shape, 2) there is a strong argument that it should be
> >merged. Code freeze is the strict freezing point, when no more new
> >features can go in after that.
> >
> >That model has its pros and cons. I proposed in my previous email to
> >just scrap the feature freeze and go with only code freeze, with the
> >hope to create shorter release cycle and smoother workflow.
> >
> >Wei.
> 

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Wed, Feb 11, 2015 at 10:45:44AM +0000, Andrew Cooper wrote:
> On 11/02/15 08:05, Jan Beulich wrote:
> >>>> On 10.02.15 at 16:04, <wei.liu2@citrix.com> wrote:
> >> # Scrapping soft freeze
> >>
> >> I think existing model (development + freeze) works well in general,
> >> but it would be better if we can shorten the time frame a bit, or try
> >> harder to stick to the 9 months promise.
> >>
> >> I would like to propose a tweak: scrap the soft freeze. The soft freeze
> >> is used to allow features posted during development window to get merged
> >> in time for a certain release. However this practice leads to longer
> >> freeze period (new features means more testing, hence longer freeze).
> >>
> >> If we scrap the soft freeze, there are several pros:
> >>
> >> 1. We only need to harden existing features in freeze. That would
> >>    probably shorten the time needed for freeze, or at least keep
> >>    everything within 3 months time frame.
> >>
> >> 2. Contributors with big features to upstream would not need to rebase
> >>    aggressively because they don't need to worry about features that
> >>    go in between soft freeze and hard freeze.
> >>
> >> 3. Contributors can accelerate their upstreaming effort by benefiting
> >>    from the possible shorter freeze time.
> >>
> >> I think this tweaked process can help make the development flow
> >> smoother. We release in a more timely manner and contributors have less
> >> work to do.
> >>
> >> The cons are of course we have shorter time in freeze and lead to
> >> possible less harden code. But I don't really see that as a big
> >> issue, given that we a) we still set aside 3 months for it, b) we
> >> only need to harden existing code.
> > While I agree with others that giving this is try may certainly be
> > worth it, I'd like to point out another aspect: Consider a series that
> > has undergone 9 revisions at the time of the freeze, and the 10th
> > would be the final one that could go in. It would be rather
> > unfortunate to tell the submitter that now he's got to wait for 3 or
> > more months until that code can go in. Or, in order to avoid that
> > situation, it could increase pressure on maintainers to accept code
> > without all issues addressed, or with only a relaxed review done.
> > At least as long as we're suffering from limited reviewing capacity
> > and at least as long as we're having contributors who step away
> > from their code the moment their base submission got committed,
> > we should at least take this aspect into consideration.
> 
> At the point of code freeze, the maintainers can take a decision as to
> whether a pending series is just a few tweaks away from acceptance or
> not.  A v1 rewriting Xen in C++ is unlikely to qualify, whereas a v10
> which is mostly acked/reviewed and only has a few comments remaining is
> likely to be able to be turned around in a quick time, and can be
> permitted to go for one further round and be accepted.
> 

Yes. I think we can trust maintainers' judgement on this. In fact that
is what we've always been doing during last two cycles -- maintainers
actively endorsed a series if he thought it was important.

> The other advantage with a shorter release cycle and freeze is that, for
> items which do miss the freeze, there is less time until staging opens
> again for submissions.
> 

Agreed.

> One thing which could help is, where appropriate, leading patches on a
> series being accepted even if the series as a whole may have issues. 
> Leading patches tend to be cleanup, rather than the meat of a series. 
> Taking these patches ahead of the series as a whole would reduce traffic
> to xen-devel, reduce the cognitive review load of working out whether it
> had changed since last time, and reduce the amount of effort required to
> maintain the series as changes are made.  Of course, this should fall
> strictly under the prerogative of the maintainer as to whether this is
> safe to do, but I feel it could safely happen rather more than it
> currently does.
> 

Agreed.

Wei.

> ~Andrew

Re: [RFC] Tweaking the release process for Xen 4.6

[Lars Kurth]

Wei,

this sounds reasonable. I just wanted to ensure that I understood fully?
Do you want me to update the presentation I sent you yesterday and which
is also at 
http://www.slideshare.net/xen_com_mgr/xen-project-release-and-roadmap-proce
ss with some mods to outline the changes based on what you described?

As an aside: we should also track this thread on bugs.xenproject.org - I
don't think we need a vote, but it is good to track process changes.

Regards
Lars

On 11/02/2015 11:55, "Wei Liu" <wei.liu2@citrix.com> wrote:

>On Wed, Feb 11, 2015 at 10:45:44AM +0000, Andrew Cooper wrote:
>> On 11/02/15 08:05, Jan Beulich wrote:
>> >>>> On 10.02.15 at 16:04, <wei.liu2@citrix.com> wrote:
>> >> # Scrapping soft freeze
>> >>
>> >> I think existing model (development + freeze) works well in general,
>> >> but it would be better if we can shorten the time frame a bit, or try
>> >> harder to stick to the 9 months promise.
>> >>
>> >> I would like to propose a tweak: scrap the soft freeze. The soft
>>freeze
>> >> is used to allow features posted during development window to get
>>merged
>> >> in time for a certain release. However this practice leads to longer
>> >> freeze period (new features means more testing, hence longer freeze).
>> >>
>> >> If we scrap the soft freeze, there are several pros:
>> >>
>> >> 1. We only need to harden existing features in freeze. That would
>> >>    probably shorten the time needed for freeze, or at least keep
>> >>    everything within 3 months time frame.
>> >>
>> >> 2. Contributors with big features to upstream would not need to
>>rebase
>> >>    aggressively because they don't need to worry about features that
>> >>    go in between soft freeze and hard freeze.
>> >>
>> >> 3. Contributors can accelerate their upstreaming effort by benefiting
>> >>    from the possible shorter freeze time.
>> >>
>> >> I think this tweaked process can help make the development flow
>> >> smoother. We release in a more timely manner and contributors have
>>less
>> >> work to do.
>> >>
>> >> The cons are of course we have shorter time in freeze and lead to
>> >> possible less harden code. But I don't really see that as a big
>> >> issue, given that we a) we still set aside 3 months for it, b) we
>> >> only need to harden existing code.
>> > While I agree with others that giving this is try may certainly be
>> > worth it, I'd like to point out another aspect: Consider a series that
>> > has undergone 9 revisions at the time of the freeze, and the 10th
>> > would be the final one that could go in. It would be rather
>> > unfortunate to tell the submitter that now he's got to wait for 3 or
>> > more months until that code can go in. Or, in order to avoid that
>> > situation, it could increase pressure on maintainers to accept code
>> > without all issues addressed, or with only a relaxed review done.
>> > At least as long as we're suffering from limited reviewing capacity
>> > and at least as long as we're having contributors who step away
>> > from their code the moment their base submission got committed,
>> > we should at least take this aspect into consideration.
>> 
>> At the point of code freeze, the maintainers can take a decision as to
>> whether a pending series is just a few tweaks away from acceptance or
>> not.  A v1 rewriting Xen in C++ is unlikely to qualify, whereas a v10
>> which is mostly acked/reviewed and only has a few comments remaining is
>> likely to be able to be turned around in a quick time, and can be
>> permitted to go for one further round and be accepted.
>> 
>
>Yes. I think we can trust maintainers' judgement on this. In fact that
>is what we've always been doing during last two cycles -- maintainers
>actively endorsed a series if he thought it was important.
>
>> The other advantage with a shorter release cycle and freeze is that, for
>> items which do miss the freeze, there is less time until staging opens
>> again for submissions.
>> 
>
>Agreed.
>
>> One thing which could help is, where appropriate, leading patches on a
>> series being accepted even if the series as a whole may have issues.
>> Leading patches tend to be cleanup, rather than the meat of a series.
>> Taking these patches ahead of the series as a whole would reduce traffic
>> to xen-devel, reduce the cognitive review load of working out whether it
>> had changed since last time, and reduce the amount of effort required to
>> maintain the series as changes are made.  Of course, this should fall
>> strictly under the prerogative of the maintainer as to whether this is
>> safe to do, but I feel it could safely happen rather more than it
>> currently does.
>> 
>
>Agreed.
>
>Wei.
>
>> ~Andrew

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Wed, Feb 11, 2015 at 12:20:20PM +0000, Lars Kurth wrote:
> Wei,
> 
> this sounds reasonable. I just wanted to ensure that I understood fully?
> Do you want me to update the presentation I sent you yesterday and which
> is also at 
> http://www.slideshare.net/xen_com_mgr/xen-project-release-and-roadmap-proce
> ss with some mods to outline the changes based on what you described?
> 

Let's wait for one day or two before modifying.

Many people are in different timezone so I would like to leave a bit
more time for them to say if they have anything to add.

> As an aside: we should also track this thread on bugs.xenproject.org - I
> don't think we need a vote, but it is good to track process changes.
> 

Sure. I will do it today.

Wei.

Re: [RFC] Tweaking the release process for Xen 4.6

[Lars Kurth]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]



On 11/02/2015 12:27, "Wei Liu" <wei.liu2@citrix.com> wrote:

>On Wed, Feb 11, 2015 at 12:20:20PM +0000, Lars Kurth wrote:
>> Wei,
>> 
>> this sounds reasonable. I just wanted to ensure that I understood fully?
>> Do you want me to update the presentation I sent you yesterday and which
>> is also at 
>> 
>>http://www.slideshare.net/xen_com_mgr/xen-project-release-and-roadmap-pro
>>ce
>> ss with some mods to outline the changes based on what you described?
>> 
>
>Let's wait for one day or two before modifying.
>
>Many people are in different timezone so I would like to leave a bit
>more time for them to say if they have anything to add.

Wei,
I added two pictures which shows my understanding of the changes
Regards
Lars



[-- Attachment #2: default[2].jpg --]
[-- Type: image/jpeg, Size: 91395 bytes --]

[-- Attachment #3: default[3].jpg --]
[-- Type: image/jpeg, Size: 123323 bytes --]

[-- Attachment #4: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Processed: Re: [RFC] Tweaking the release process for Xen 4.6

[xen]

Processing commands for xen@bugs.xenproject.org:

> create ^
Created new bug #48 rooted at `<20150210150424.GA32743@zion.uk.xensource.com>'
Title: `Re: [RFC] Tweaking the release process for Xen 4.6'
> title it Tweaking the release process for Xen 4.6
Set title for #48 to `Tweaking the release process for Xen 4.6'
> Thanks
Command failed: Unknown command `Thanks'. at /srv/xen-devel-bugs/lib/emesinae/control.pl line 457, <GEN1> line 3.
Stop processing here.

Modified/created Bugs:
 - 48: http://bugs.xenproject.org/xen/bug/48 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Wed, Feb 11, 2015 at 02:54:46PM +0000, Lars Kurth wrote:
> 
> 
> On 11/02/2015 12:27, "Wei Liu" <wei.liu2@citrix.com> wrote:
> 
> >On Wed, Feb 11, 2015 at 12:20:20PM +0000, Lars Kurth wrote:
> >> Wei,
> >> 
> >> this sounds reasonable. I just wanted to ensure that I understood fully?
> >> Do you want me to update the presentation I sent you yesterday and which
> >> is also at 
> >> 
> >>http://www.slideshare.net/xen_com_mgr/xen-project-release-and-roadmap-pro
> >>ce
> >> ss with some mods to outline the changes based on what you described?
> >> 
> >
> >Let's wait for one day or two before modifying.
> >
> >Many people are in different timezone so I would like to leave a bit
> >more time for them to say if they have anything to add.
> 
> Wei,
> I added two pictures which shows my understanding of the changes

Thanks. I think your new pictures are accurate.

Wei.

> Regards
> Lars
> 
> 

Re: [RFC] Tweaking the release process for Xen 4.6

[Lars Kurth]

On 12/02/2015 11:07, "Wei Liu" <wei.liu2@citrix.com> wrote:
>
>Thanks. I think your new pictures are accurate.
>
>Wei.

In that case, I will re-upload that presentation I shared earlier. It will
show the pre 4.5 workflow and new proposal
Regards
Lars

Re: [RFC] Tweaking the release process for Xen 4.6

[Wei Liu]

On Thu, Feb 12, 2015 at 11:09:52AM +0000, Lars Kurth wrote:
> 
> 
> On 12/02/2015 11:07, "Wei Liu" <wei.liu2@citrix.com> wrote:
> >
> >Thanks. I think your new pictures are accurate.
> >
> >Wei.
> 
> In that case, I will re-upload that presentation I shared earlier. It will
> show the pre 4.5 workflow and new proposal

I think you mean pre-4.6. Yes, showing both are good.

Wei.

> Regards
> Lars
> 

Re: [RFC] Tweaking the release process for Xen 4.6

[Ian Campbell]

On Tue, 2015-02-10 at 15:04 +0000, Wei Liu wrote:

Not much to add... a couple of comments.

> Counting from the point that we forked the tree, it took ~11 months to
> ship 4.5. The time spent on development was 7 months (Feb 21 to Sept
> 24), and the time spent on freeze was ~4 months (Sept 24 to Jan 6).
> 
> The good thing was that code quality was ensured, the downside was that
> such long freeze 

FWIW I was really starting to feel like the freeze had gone on forever
by the end of it, and was feeling like there had been stuff which was
pushed on at the end of the summer which had now been sitting in limbo
for far too long.

> With the above proposal in mind, here is my proposed time frame for
> Xen 4.6 release:
> 
>   Development start: 6  Jan 2015
>   Feature freeze:    10 Jul 2015
>   Release date:      9  Oct 2015 (could release earlier)

This is 6 months of dev + 3 of freeze, for 9 months in total.

> Note that the release date is not a goal. It is more like a deadline
> we try to keep up with. We might be well able to release earlier if
> everything is in good shape.

In fact I think we could reasonably aim for a two month freeze (~6 rcs)
and pencil in the third month as potential slippage time which we don't
plan to use.

> Any thought on this tweaked process? Comments are welcome.
> 
> Wei.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel