Re: Request a freeze exception for COLO in v4.6

Re: Request a freeze exception for COLO in v4.6

[Eddie Dong]

[-- Attachment #1.1: Type: text/plain, Size: 2101 bytes --]

> If I can take one example, 11/25 "tools/libxc: support to resume uncooperative

> HVM guests".  Based on my current understanding this is even a bugfix.  Sadly it

> is not quite ready (or at least, wasn't last night).  But with a few more days we

> can probably get much of this cleanup and preparatory work in.

 

I solicited Wei to give an 2-3 weeks exception time for the patch to be in much better shape, however, people simply ignore request not from Citrix without any retrospection to see where is the problem. I already heard many "believes" and "commitments" from people with the hat of maintainers  and managers, saying that he/she should be able to complete the dependency patch couple weeks before the deadline during Xen Hackathon, and agreed to raise the priority to review the patches. But was any of those commitment really made?

 

Be aware this pushing effort has been lasting for 2 years already, it is not this week, this month, this quarter, even not only this year.

 

Resorting this to one or two technical issue simply doesn't work. People with the maintainer's hat on can arbitrary re-version the existing APIs for their hobby, to block all the follow-up patches for entire release cycles, which is one entire year. In the XEN/COLO case, it is 2 years for similar reasons.

 

>

> Assuming nothing goes horribly wrong, I am very optimistic about COLO making

> it upstream soon, just not in 4.6.

>

 

 I heard similar word  one year ago too,  eveything is repeating without any retrospection: everything Citrix likes, they can happen at any time  anywhere ... Everything else, have to resort to the Citrix maintainers to has redundant bandwidth.

 Any developers from PRC, it is time for you guys to move to other hypervisors. Xen is a Citrix's hobby only, and no longer an active community project. If you couldn't prepare for at least 2 years efforts to push a patch series, Eddie personally would like to suggest you to move to other hypervisors.


From a former 11+ years experience Xen community developer, Xen spot light, and Xen active participant.
Eddie Dong

[-- Attachment #1.2: Type: text/html, Size: 3652 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Request a freeze exception for COLO in v4.6

[Lars Kurth]

[-- Attachment #1.1: Type: text/plain, Size: 6062 bytes --]

Hi all,

I have been travelling on Friday and wanted to appeal for calm on this particular issue. Let's try and focus on making as much progress as we can on the patch series which have freeze exceptions (or partial freeze exception) this week. Continuing a debate on what may have gone wrong with Remus/COLO or other series at this stage is going to be distracting and will affect everyones chances to get the remaining code with freeze exceptions into Xen 4.6. So please, let's focus on making as much progress as we can this week. Having said that, it is absolutely true that the first Remus/COLO and COLO RFC patches have received very little or no review time when they were posted first by Wen Congyang in April 2013. And that situation had not changed until a year later, when the issue was first raised with me (if I recall correctly). 

I do sincerely apologise for this. Personally I would like to see COLO in Xen 4.6, but this is not my decision.

I do also believe that there has been tremendous progress on Remus and COLO in the last 12 months. There has also been great collaboration between maintainers and contributors in the last 12 months, in particular around Remus and COLOPre. Let's build on this and move forward.

= After July 24th =

We should have a discussion *after* July 24th, to see what is going wrong and what we can improve as a community. We can also cover some of the accusations that were made then. It is clear that as a community we do have some issues and challenges that we have to address. I have to personally take responsibility for underestimating some of the issues we face: until about 4 weeks ago, it looked as if many of the issues that I knew of are well on the way of being resolved. I honestly believe that many of the changes we made recently, such as focus on designs, had a very positive effect. I do not believe, that what we are seeing is a sign of a dying community. There were also some specific issues related to Remus/COLO: some have been addressed; others have not yet been addressed.

However, it is not clear yet from the data that I can mine, exactly what is going on in the general case. We have good data mining capability when it comes to git, but *no* data mining capability when it comes to the review process. I am meeting Bitergia tomorrow to see whether they (or I) can implement some functionality that allows us to get metrics related to the review process.

= Data we have =

What is interesting is that since 2012, we have seen an average annual increase of 9% of patches that made it into the Xen Hypervisor. We also have seen a slightly higher increase of Reviewed-by and ACKED-by tags during the same time period (around 10% a year). However, this does not tell us much, as the review period leading up to commits is not covered by git data.

The following graph shows the number of e-mails related to patches on xen-devel@ - both patches, comments on patches, submissions of new versions of patches, etc.



What is striking is that the ratio of discussions related to patches (including posted patches) on xen-devel divided by the patches that made it into Xen has increased almost grown exponentially recently: 5.85 (2012), 7.89 (2013), 8.63 (2014) to 11.65 (2015). This clearly shows that we have some issues with code reviews that are getting worse and that there is an underlying issue which we have to address: there are a number of possible reasons. But let's not speculate now.

Best Regards
Lars

> On 17 Jul 2015, at 09:04, Eddie Dong <13341687268@163.com> wrote:
> 
> > If I can take one example, 11/25 "tools/libxc: support to resume uncooperative <>
> > HVM guests".  Based on my current understanding this is even a bugfix.  Sadly it
> 
> > is not quite ready (or at least, wasn't last night).  But with a few more days we
> 
> > can probably get much of this cleanup and preparatory work in.
> 
>  
> 
> I solicited Wei to give an 2-3 weeks exception time for the patch to be in much better shape, however, people simply ignore request not from Citrix without any retrospection to see where is the problem. I already heard many "believes" and "commitments" from people with the hat of maintainers  and managers, saying that he/she should be able to complete the dependency patch couple weeks before the deadline during Xen Hackathon, and agreed to raise the priority to review the patches. But was any of those commitment really made?
> 
>  
> 
> Be aware this pushing effort has been lasting for 2 years already, it is not this week, this month, this quarter, even not only this year.
> 
>  
> 
> Resorting this to one or two technical issue simply doesn't work. People with the maintainer's hat on can arbitrary re-version the existing APIs for their hobby, to block all the follow-up patches for entire release cycles, which is one entire year. In the XEN/COLO case, it is 2 years for similar reasons.
> 
>  
> 
> >
> 
> > Assuming nothing goes horribly wrong, I am very optimistic about COLO making
> 
> > it upstream soon, just not in 4.6.
> 
> >
> 
>  
> 
>  I heard similar word  one year ago too,  eveything is repeating without any retrospection: everything Citrix likes, they can happen at any time  anywhere ... Everything else, have to resort to the Citrix maintainers to has redundant bandwidth.
> 
>  Any developers from PRC, it is time for you guys to move to other hypervisors. Xen is a Citrix's hobby only, and no longer an active community project. If you couldn't prepare for at least 2 years efforts to push a patch series, Eddie personally would like to suggest you to move to other hypervisors.
> 
> From a former 11+ years experience Xen community developer, Xen spot light, and Xen active participant.
> Eddie Dong
> 
> 
> 来自 网易手机号码邮箱 -- 有手机就有网易手机号码邮箱，了解详情> <http://shouji.163.com/>_______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


[-- Attachment #1.2: Type: text/html, Size: 8980 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Request a freeze exception for COLO in v4.6

[Lars Kurth]

[-- Attachment #1.1: Type: text/plain, Size: 3750 bytes --]

Forgot to attach the graph referred to (or it is not showing up in the archives)
Lars


]
> On 20 Jul 2015, at 07:02, Lars Kurth <lars.kurth.xen@gmail.com> wrote:
> 
> Hi all,
> 
> I have been travelling on Friday and wanted to appeal for calm on this particular issue. Let's try and focus on making as much progress as we can on the patch series which have freeze exceptions (or partial freeze exception) this week. Continuing a debate on what may have gone wrong with Remus/COLO or other series at this stage is going to be distracting and will affect everyones chances to get the remaining code with freeze exceptions into Xen 4.6. So please, let's focus on making as much progress as we can this week. Having said that, it is absolutely true that the first Remus/COLO and COLO RFC patches have received very little or no review time when they were posted first by Wen Congyang in April 2013. And that situation had not changed until a year later, when the issue was first raised with me (if I recall correctly). 
> 
> I do sincerely apologise for this. Personally I would like to see COLO in Xen 4.6, but this is not my decision.
> 
> I do also believe that there has been tremendous progress on Remus and COLO in the last 12 months. There has also been great collaboration between maintainers and contributors in the last 12 months, in particular around Remus and COLOPre. Let's build on this and move forward.
> 
> = After July 24th =
> 
> We should have a discussion *after* July 24th, to see what is going wrong and what we can improve as a community. We can also cover some of the accusations that were made then. It is clear that as a community we do have some issues and challenges that we have to address. I have to personally take responsibility for underestimating some of the issues we face: until about 4 weeks ago, it looked as if many of the issues that I knew of are well on the way of being resolved. I honestly believe that many of the changes we made recently, such as focus on designs, had a very positive effect. I do not believe, that what we are seeing is a sign of a dying community. There were also some specific issues related to Remus/COLO: some have been addressed; others have not yet been addressed.
> 
> However, it is not clear yet from the data that I can mine, exactly what is going on in the general case. We have good data mining capability when it comes to git, but *no* data mining capability when it comes to the review process. I am meeting Bitergia tomorrow to see whether they (or I) can implement some functionality that allows us to get metrics related to the review process.
> 
> = Data we have =
> 
> What is interesting is that since 2012, we have seen an average annual increase of 9% of patches that made it into the Xen Hypervisor. We also have seen a slightly higher increase of Reviewed-by and ACKED-by tags during the same time period (around 10% a year). However, this does not tell us much, as the review period leading up to commits is not covered by git data.
> 
> The following graph shows the number of e-mails related to patches on xen-devel@ - both patches, comments on patches, submissions of new versions of patches, etc.
> 
> <default.png>
> 
> What is striking is that the ratio of discussions related to patches (including posted patches) on xen-devel divided by the patches that made it into Xen has increased almost grown exponentially recently: 5.85 (2012), 7.89 (2013), 8.63 (2014) to 11.65 (2015). This clearly shows that we have some issues with code reviews that are getting worse and that there is an underlying issue which we have to address: there are a number of possible reasons. But let's not speculate now.
> 
> Best Regards
> Lars


[-- Attachment #1.2.1: Type: text/html, Size: 5705 bytes --]

[-- Attachment #1.2.2: default.png --]
[-- Type: image/png, Size: 30228 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Request a freeze exception for COLO in v4.6

[Eddie Dong]

[-- Attachment #1.1: Type: text/plain, Size: 2167 bytes --]

> If I can take one example, 11/25 "tools/libxc: support to resume uncooperative

> HVM guests".  Based on my current understanding this is even a bugfix.  Sadly it

> is not quite ready (or at least, wasn't last night).  But with a few more days we

> can probably get much of this cleanup and preparatory work in.

 

I solicited Wei to give an 2-3 weeks exception time for the patch to be in much better shape, however, people simply ignore request not from Citrix without any retrospection to see where is the problem. I already heard many "believes" and "commitments" from people with the hat of maintainers  and managers, saying that he/she should be able to complete the dependency patch couple weeks before the deadline during Xen Hackathon, and agreed to raise the priority to review the patches. But was any of those commitment really made?

 

Be aware this pushing effort has been lasting for 2 years already, it is not this week, this month, this quarter, even not only this year.

 

Resorting this to one or two technical issue simply doesn't work. People with the maintainer's hat on can arbitrary re-version the existing APIs for their hobby, to block all the follow-up patches for entire release cycles, which is one entire year. In the XEN/COLO case, it is 2 years for similar reasons.

 

>

> Assuming nothing goes horribly wrong, I am very optimistic about COLO making

> it upstream soon, just not in 4.6.

>

 

 I heard similar word  one year ago too,  eveything is repeating without any retrospection: everything Citrix likes, they can happen at any time  anywhere ... Everything else, have to resort to the Citrix maintainers to has redundant bandwidth.

 Any developers from PRC, it is time for you guys to move to other hypervisors. Xen is a Citrix's hobby only, and no longer an active community project. If you couldn't prepare for at least 2 years efforts to push a patch series, Eddie personally would like to suggest you to move to other hypervisors.


From a former 11+ years experience Xen community developer, Xen spot light, and Xen active participant.
Eddie Dong



来自 网易手机号码邮箱 -- 有手机就有网易手机号码邮箱，了解详情>

[-- Attachment #1.2: Type: text/html, Size: 4586 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Request a freeze exception for COLO in v4.6

[Yang Hongyang]

Hello,

I would like to request a freeze exception for COLO.

1.The benefits
* COLO is a VM fault tolerance solution. Currently Remus provides this
   function, but it buffers all output packets, the latency is unacceptable.
   COLO addresses the problem, it can greatly improve the VM availability.
* There are 3rd parties interested in this feature, for example Huawei,
   they already released Xen/COLO in their product with offline patches.
* If this could go into Xen4.6, it will greatly speed up the production
   use of this feature as well as the development. Which could be a huge
   benefits to end-users who wants a VM FT solution to provide "non-stop
   services"

2.The risks
   I would say there should be no risk for including this feature into Xen4.6.
   Because this feature is only a bolt-on. It should sits in it's own corner
   with no harm to the existing code. On the contrary, it improves the
   existing code with quite a lot refactoring.

3.Further maintenance
   Intel and Fujitsu will maintain the code in the future, the feature goes
   into upstream does not mean the end of our development, instead it is the
   very start of our development, we will continue to improve the feature,
   fix bugs and so on.

4.Current status
   The Libxl migration v2 which we depend on should be merged soon, maybe today.
   There're 2 series on list,
   One is the prepare patchset, it is mostly refactoring and bugfix, 6/25 been
   acked. I would say most of the patchset are ready to be merged.
   Another is the main COLO series, it still needs to be carefully reviewed,
   but given the fact that this feature is only a bolt-on and we will continue
   to improve the code, it should be no harm to merge in the near future.
   I'm confident that we could get this ready in the following 1~2 weeks.

-- 
Thanks,
Yang.

Re: Request a freeze exception for COLO in v4.6

[Wei Liu]

On Wed, Jul 15, 2015 at 06:17:17PM +0800, Yang Hongyang wrote:
> Hello,
> 
> I would like to request a freeze exception for COLO.
> 
> 1.The benefits
> * COLO is a VM fault tolerance solution. Currently Remus provides this
>   function, but it buffers all output packets, the latency is unacceptable.
>   COLO addresses the problem, it can greatly improve the VM availability.
> * There are 3rd parties interested in this feature, for example Huawei,
>   they already released Xen/COLO in their product with offline patches.
> * If this could go into Xen4.6, it will greatly speed up the production
>   use of this feature as well as the development. Which could be a huge
>   benefits to end-users who wants a VM FT solution to provide "non-stop
>   services"
> 
> 2.The risks
>   I would say there should be no risk for including this feature into Xen4.6.
>   Because this feature is only a bolt-on. It should sits in it's own corner
>   with no harm to the existing code. On the contrary, it improves the
>   existing code with quite a lot refactoring.
> 
> 3.Further maintenance
>   Intel and Fujitsu will maintain the code in the future, the feature goes
>   into upstream does not mean the end of our development, instead it is the
>   very start of our development, we will continue to improve the feature,
>   fix bugs and so on.
> 
> 4.Current status
>   The Libxl migration v2 which we depend on should be merged soon, maybe today.
>   There're 2 series on list,
>   One is the prepare patchset, it is mostly refactoring and bugfix, 6/25 been
>   acked. I would say most of the patchset are ready to be merged.
>   Another is the main COLO series, it still needs to be carefully reviewed,
>   but given the fact that this feature is only a bolt-on and we will continue
>   to improve the code, it should be no harm to merge in the near future.
>   I'm confident that we could get this ready in the following 1~2 weeks.
> 

Thank you for your email. 

There are currently 50 patches related to COLO pending reviews and acks
(25 pre + 25 COLO itself). Some pre patches are acked but there are
still some comments regarding implementation details in the pre patches
and COLO itself is still lacking review.  Further more, migration v2 is
having some issues and has not passed the push gate (this is not really
your fault).

Given the above situation it's hard for me to justify granting a freeze
exception to these 50 patches. However I do understand it is painful for
you to carry so many patches and the burden of constantly rebasing so
I'm thinking about granting freeze exception to those patches that are
pure code motion / low risk refactoring in the pre series, to reduce
your load of work.

As for the rest patches (rest in pre and COLO itself), I talked to Ian
Jackson about this and he's quite optimistic about getting COLO in for
4.7.

Ian and Ian, anything to add? What are your opinions?

Wei.

> -- 
> Thanks,
> Yang.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Request a freeze exception for COLO in v4.6

[Dong, Eddie]

> 
> There are currently 50 patches related to COLO pending reviews and acks
> (25 pre + 25 COLO itself). Some pre patches are acked but there are still some
> comments regarding implementation details in the pre patches and COLO itself is
> still lacking review.  Further more, migration v2 is having some issues and has
> not passed the push gate (this is not really your fault).
> 
> Given the above situation it's hard for me to justify granting a freeze exception
> to these 50 patches. However I do understand it is painful for you to carry so
> many patches and the burden of constantly rebasing so I'm thinking about
> granting freeze exception to those patches that are pure code motion / low risk
> refactoring in the pre series, to reduce your load of work.
> 
> As for the rest patches (rest in pre and COLO itself), I talked to Ian Jackson about
> this and he's quite optimistic about getting COLO in for 4.7.
> 
> Ian and Ian, anything to add? What are your opinions?
> 

I have been working in Xen for 11 years, witness the entire progress Xen made in the past, from incubation to mature and now. I never spoke for myself.
Now I want speak for myself, not for Intel.

A feature like COLO has been pushing for ~2 years with professional effort from developers, but it was still gated. I didn't see any  introspection from the community, while I was kept telling Xen needs COLO, from the time when Lars visit Shanghai 2 years ago.

Yes, it does have 25 pre patches and 25 function patches, but they are not new, they are posted in the mailinglist for months already, and have been waiting for reviews all the time. Why we couldn't do it before? Why it becomes the reason to block it? Is the community revising himself?

Why we couldn't be frank here? Like Linus does to many features he dislike, simply say Xen doesn't need COLO, that is fine. However I felt I was kept telling that COLO is important to Xen, while maintainers doesn't have bandwidth to review, from the time a few months before Xen 4.5. Things didn't change in Xen 4.6, people kept changing the code base that COLO are depending, and again block COLO.  

Lars and Wei and all the maintainers: Life can be much easier if we are all frank!  You save effort and we save effort, and everybody are happy then: We do have plenty of new features to do, why we stick with Xen/COLO?  

I still remembered when I firstly presented COLO in Xen Summit'2011 in San Diego, when Keir and Ian were still in chairing. At that time, Xen are still leading no matter in new feature and/or adoption... How fast the world changes and how fast the community changes after 4 years.

For people who participating in Xen/COLO effort: if you join the effort because of Eddie, please take a rest and try to do something more interesting, you have done very well in the past, and it was time to call it a stage.  OSVs who want Xen/COLO, they can pull the patch series themselves, and if they don't need, they don't need even if you pushed it into upstream. I ever championed the effort to support Xen for COLO, but that is as of today only.

I saw Xen was sinking for a while and now I felt it was dying quickly ...  

God Bless Xen!! From today, after serving in the community for 11 and a half years, I was no longer a Xen community member and I will appeal people to leave Xen too ...

Intel colleagues: Please remove my name from VMX subsystem as a sub-maintainer.

Thx Eddie

Re: Request a freeze exception for COLO in v4.6

[Ian Jackson]

Wei Liu writes ("Re: [Xen-devel] Request a freeze exception for COLO in v4.6"):
> Ian and Ian, anything to add? What are your opinions?

I think COLO is an exciting and important feature.

Unfortunately I agree with Wei that at this stage taking both series
as they currently stand into 4.6 is not tenable.  (Perhaps if I had
been able to help earlier in the release cycle this would have been
different; so for that I'm sorry.)


I would like to see at least the early parts of `COLOPre' in staging
as soon as possible.  There is much preparatory work there which would
be annoying to rebase around, and which has collateral benefits.

If I can take one example, 11/25 "tools/libxc: support to resume
uncooperative HVM guests".  Based on my current understanding this is
even a bugfix.  Sadly it is not quite ready (or at least, wasn't last
night).  But with a few more days we can probably get much of this
cleanup and preparatory work in.


Assuming nothing goes horribly wrong, I am very optimistic about COLO
making it upstream soon, just not in 4.6.

Thanks,
Ian.

Re: Request a freeze exception for COLO in v4.6

[Wei Liu]

On Fri, Jul 17, 2015 at 11:18:21AM +0100, Ian Jackson wrote:
> Wei Liu writes ("Re: [Xen-devel] Request a freeze exception for COLO in v4.6"):
> > Ian and Ian, anything to add? What are your opinions?
> 
> I think COLO is an exciting and important feature.
> 
> Unfortunately I agree with Wei that at this stage taking both series
> as they currently stand into 4.6 is not tenable.  (Perhaps if I had
> been able to help earlier in the release cycle this would have been
> different; so for that I'm sorry.)
> 
> 
> I would like to see at least the early parts of `COLOPre' in staging
> as soon as possible.  There is much preparatory work there which would
> be annoying to rebase around, and which has collateral benefits.
> 

Thanks for your input. Please apply patches that are ready to go in as
soon as possible.

> If I can take one example, 11/25 "tools/libxc: support to resume
> uncooperative HVM guests".  Based on my current understanding this is
> even a bugfix.  Sadly it is not quite ready (or at least, wasn't last
> night).  But with a few more days we can probably get much of this
> cleanup and preparatory work in.
> 
> 
> Assuming nothing goes horribly wrong, I am very optimistic about COLO
> making it upstream soon, just not in 4.6.
> 
> Thanks,
> Ian.

Re: Request a freeze exception for COLO in v4.6

[Yang Hongyang]

On 07/17/2015 06:18 PM, Ian Jackson wrote:
> Wei Liu writes ("Re: [Xen-devel] Request a freeze exception for COLO in v4.6"):
>> Ian and Ian, anything to add? What are your opinions?
>
> I think COLO is an exciting and important feature.
>
> Unfortunately I agree with Wei that at this stage taking both series
> as they currently stand into 4.6 is not tenable.  (Perhaps if I had
> been able to help earlier in the release cycle this would have been
> different; so for that I'm sorry.)
>
>
> I would like to see at least the early parts of `COLOPre' in staging
> as soon as possible.  There is much preparatory work there which would
> be annoying to rebase around, and which has collateral benefits.

Thanks, if you are going to merge some of the patches, there is the
latest version of `COLOPre' patchset:
https://github.com/macrosheep/xen/tree/colo-v9
The latest 26 commits.

I think all patches are ready to be merged except:
7  libxl/remus: init checkpoint_callback in Remus checkpoint callback
11 tools/libxc: support to resume uncooperative HVM guests
20 tools/libx{l,c}: add back channel to libxc

>
> If I can take one example, 11/25 "tools/libxc: support to resume
> uncooperative HVM guests".  Based on my current understanding this is
> even a bugfix.  Sadly it is not quite ready (or at least, wasn't last
> night).  But with a few more days we can probably get much of this
> cleanup and preparatory work in.
>
>
> Assuming nothing goes horribly wrong, I am very optimistic about COLO
> making it upstream soon, just not in 4.6.
>
> Thanks,
> Ian.
> .
>

-- 
Thanks,
Yang.

Re: Request a freeze exception for COLO in v4.6

[Ian Campbell]

On Mon, 2015-07-20 at 15:27 +0800, Yang Hongyang wrote:
> 
> On 07/17/2015 06:18 PM, Ian Jackson wrote:
> > Wei Liu writes ("Re: [Xen-devel] Request a freeze exception for 
> > COLO in v4.6"):
> > > Ian and Ian, anything to add? What are your opinions?
> > 
> > I think COLO is an exciting and important feature.
> > 
> > Unfortunately I agree with Wei that at this stage taking both 
> > series
> > as they currently stand into 4.6 is not tenable.  (Perhaps if I had
> > been able to help earlier in the release cycle this would have been
> > different; so for that I'm sorry.)
> > 
> > 
> > I would like to see at least the early parts of `COLOPre' in 
> > staging
> > as soon as possible.  There is much preparatory work there which 
> > would
> > be annoying to rebase around, and which has collateral benefits.
> 
> Thanks, if you are going to merge some of the patches, there is the
> latest version of `COLOPre' patchset:
> https://github.com/macrosheep/xen/tree/colo-v9
> The latest 26 commits.

> I think all patches are ready to be merged except:

I would have taken this to mean "are acked", but in fact a reasonable
number of these aren't, although you have addressed the comments from
last time.

I acked these on the basis of what was in the branch:
    libxl/remus: introduce libxl__remus_setup
    tools/libxl: Update libxl_domain_unpause() to support qemu-xen
        (this one was acked-by Wei with my suggestion, which you've
        done so I added it)

but many of the rest have changes which are significant enough to
require reposting IMHO, because I don't feel comfortable acking them
while cherry-picking. (as it happens many wouldn't apply without 7
anyway

> 7  libxl/remus: init checkpoint_callback in Remus checkpoint callback

Unfortunately this had severe knock on effects, i.e. patch 8 onwards
didn't cleanly apply.

I did manage to pick up:
15 tools/libxl: check QEMU state before resume dm

I'm afraid that between the need to repost a bunch of this and the lack
of #7 I only managed to get these 8 to apply:

8db1a08 tools/libxl: Update libxl_domain_unpause() to support qemu-xen
4301269 tools/libxl: check QEMU state before resume dm
c3e886a libxl/remus: introduce libxl__remus_teardown
c230261 libxl/remus: introduce libxl__remus_setup
8dd129f tools/libxl: rename remus checkpoint callbacks
bc3e7ff tools/libxl: move domain resume code into libxl_dom_suspend.c
f1fe54e tools/libxl: move domain suspend code into libxl_dom_suspend.c
73db059 tools/libxl: rename libxl__domain_suspend to libxl__domain_save

Sorting out #7 and reposting would probably unblock a pretty big chunk
of the rest, I think.

Ian.