Migration v2 and related work for 4.6

Migration v2 and related work for 4.6

[Andrew Cooper]

Hello,

I believe I have CC'd all interested parties.  If I have missed anyone,
please shout.

There are several large series:
* Cancellable async operations
* COLO

and a wish to start experimenting with migration on ARM which all
interact with my large series for migration v2 support in libxc and libxl.

I am working on the libxl side of things, but realise that I am being a
blockage to the other areas waiting on migration v2.

The current state of the libxc bit is functionally complete.  It is
shipping in two versions of XenServer, has not changed substantially in
9 months, and not changed in a backwards incompatible way since v1.

I propose that the libxc series be accepted independently of the libxl
series.  The libxc series is still implemented as an API-compatible
alternative to legacy migration, and is not enabled by default, meaning
no change to current users.

However, it will unblock development of ARM migration and COLO, and will
allow people in the wider community to experiment with migration v2.  An
existing `xl migrate` can be swapped from legacy to migration v2 simply
by setting an environment variable (for PV guests.  HVM requires the
libxl changed as the handling of the Qemu save record is changing.)

I feel that this is best interest for the community, to get some testing
available earlier in the development cycle, rather than attempting to
splice 3 large series in a related area together towards the code freeze.

Thoughts? (especially from a release management point of view)

~Andrew

Re: Migration v2 and related work for 4.6

[Ian Campbell]

On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
> I propose that the libxc series be accepted independently of the libxl
> series.

That is most likely a good idea IMHO.

What do you estimate the chances of the libxl bit being done for 4.6 to
be?

Is there an option of actually switching to the new libxc without
switching libxl and fixing libxl later, or would that be as much work as
just fixing libxl?

Ian.

Re: Migration v2 and related work for 4.6

[Andrew Cooper]

On 02/04/15 10:03, Ian Campbell wrote:
> On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
>> I propose that the libxc series be accepted independently of the libxl
>> series.
> That is most likely a good idea IMHO.
>
> What do you estimate the chances of the libxl bit being done for 4.6 to
> be?

I hope to have everything complete for 4.6, including removal of the
legacy code.

Given the current timescales, I would say most likely.

>
> Is there an option of actually switching to the new libxc without
> switching libxl and fixing libxl later, or would that be as much work as
> just fixing libxl?

For PV guests, yes.  One can transparently swap the legacy algorithm
from the v2 algorithm and every works.

For HVM guests, no.  The handling of the Qemu save record and toolstack
records needs fixing.

In the upgrade case, libxl also needs to take care of piping the legacy
stream through the conversion script.

~Andrew

Re: Migration v2 and related work for 4.6

[Wei Liu]

On Thu, Apr 02, 2015 at 10:34:34AM +0100, Andrew Cooper wrote:
> On 02/04/15 10:03, Ian Campbell wrote:
> > On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
> >> I propose that the libxc series be accepted independently of the libxl
> >> series.
> > That is most likely a good idea IMHO.
> >
> > What do you estimate the chances of the libxl bit being done for 4.6 to
> > be?
> 
> I hope to have everything complete for 4.6, including removal of the
> legacy code.
> 

Not sure what "legacy code" you're referring to, but we definitely want
to support 4.5 -> 4.6 migration so the "legacy code" might need to stay
for 4.6?

Wei.

> Given the current timescales, I would say most likely.
> 
> >
> > Is there an option of actually switching to the new libxc without
> > switching libxl and fixing libxl later, or would that be as much work as
> > just fixing libxl?
> 
> For PV guests, yes.  One can transparently swap the legacy algorithm
> from the v2 algorithm and every works.
> 
> For HVM guests, no.  The handling of the Qemu save record and toolstack
> records needs fixing.
> 
> In the upgrade case, libxl also needs to take care of piping the legacy
> stream through the conversion script.
> 
> ~Andrew

Re: Migration v2 and related work for 4.6

[Andrew Cooper]

On 07/04/15 13:00, Wei Liu wrote:
> On Thu, Apr 02, 2015 at 10:34:34AM +0100, Andrew Cooper wrote:
>> On 02/04/15 10:03, Ian Campbell wrote:
>>> On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
>>>> I propose that the libxc series be accepted independently of the libxl
>>>> series.
>>> That is most likely a good idea IMHO.
>>>
>>> What do you estimate the chances of the libxl bit being done for 4.6 to
>>> be?
>> I hope to have everything complete for 4.6, including removal of the
>> legacy code.
>>
> Not sure what "legacy code" you're referring to, but we definitely want
> to support 4.5 -> 4.6 migration so the "legacy code" might need to stay
> for 4.6?

"legacy migration" is what currently exists.

Backwards compatibility is provided by a short python script which
converts legacy to v2.

Once the libxl changes are complete, xc_domain_save.c and
xc_domain_restore.c can be deleted, along with other assorted bits and
pieces.

~Andrew

Re: Migration v2 and related work for 4.6

[Lars Kurth]

We could discuss at the Hackathon next week, and reply to this thread. It seems that at least 50% of all the stake-holders are in fact there
Regards
Lars

> On 7 Apr 2015, at 13:00, Wei Liu <wei.liu2@citrix.com> wrote:
> 
> On Thu, Apr 02, 2015 at 10:34:34AM +0100, Andrew Cooper wrote:
>> On 02/04/15 10:03, Ian Campbell wrote:
>>> On Wed, 2015-04-01 at 12:03 +0100, Andrew Cooper wrote:
>>>> I propose that the libxc series be accepted independently of the libxl
>>>> series.
>>> That is most likely a good idea IMHO.
>>> 
>>> What do you estimate the chances of the libxl bit being done for 4.6 to
>>> be?
>> 
>> I hope to have everything complete for 4.6, including removal of the
>> legacy code.
>> 
> 
> Not sure what "legacy code" you're referring to, but we definitely want
> to support 4.5 -> 4.6 migration so the "legacy code" might need to stay
> for 4.6?
> 
> Wei.
> 
>> Given the current timescales, I would say most likely.
>> 
>>> 
>>> Is there an option of actually switching to the new libxc without
>>> switching libxl and fixing libxl later, or would that be as much work as
>>> just fixing libxl?
>> 
>> For PV guests, yes.  One can transparently swap the legacy algorithm
>> from the v2 algorithm and every works.
>> 
>> For HVM guests, no.  The handling of the Qemu save record and toolstack
>> records needs fixing.
>> 
>> In the upgrade case, libxl also needs to take care of piping the legacy
>> stream through the conversion script.
>> 
>> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Migration v2 and related work for 4.6

[Wei Liu]

On Wed, Apr 01, 2015 at 12:03:51PM +0100, Andrew Cooper wrote:
> Hello,
> 
> I believe I have CC'd all interested parties.  If I have missed anyone,
> please shout.
> 
> There are several large series:
> * Cancellable async operations
> * COLO
> 
> and a wish to start experimenting with migration on ARM which all
> interact with my large series for migration v2 support in libxc and libxl.
> 
> I am working on the libxl side of things, but realise that I am being a
> blockage to the other areas waiting on migration v2.
> 
> The current state of the libxc bit is functionally complete.  It is
> shipping in two versions of XenServer, has not changed substantially in
> 9 months, and not changed in a backwards incompatible way since v1.
> 
> I propose that the libxc series be accepted independently of the libxl
> series.  The libxc series is still implemented as an API-compatible
> alternative to legacy migration, and is not enabled by default, meaning
> no change to current users.
> 
> However, it will unblock development of ARM migration and COLO, and will
> allow people in the wider community to experiment with migration v2.  An
> existing `xl migrate` can be swapped from legacy to migration v2 simply
> by setting an environment variable (for PV guests.  HVM requires the
> libxl changed as the handling of the Qemu save record is changing.)
> 
> I feel that this is best interest for the community, to get some testing
> available earlier in the development cycle, rather than attempting to
> splice 3 large series in a related area together towards the code freeze.
> 
> Thoughts? (especially from a release management point of view)
> 

I've looked at your libxc series. It looks complete. I agree it would be
good to get it in as early as possible.

Wei.

> ~Andrew

Re: Migration v2 and related work for 4.6

[Ian Campbell]

On Tue, 2015-04-07 at 12:59 +0100, Wei Liu wrote:
> On Wed, Apr 01, 2015 at 12:03:51PM +0100, Andrew Cooper wrote:
> > Hello,
> > 
> > I believe I have CC'd all interested parties.  If I have missed anyone,
> > please shout.
> > 
> > There are several large series:
> > * Cancellable async operations
> > * COLO
> > 
> > and a wish to start experimenting with migration on ARM which all
> > interact with my large series for migration v2 support in libxc and libxl.
> > 
> > I am working on the libxl side of things, but realise that I am being a
> > blockage to the other areas waiting on migration v2.
> > 
> > The current state of the libxc bit is functionally complete.  It is
> > shipping in two versions of XenServer, has not changed substantially in
> > 9 months, and not changed in a backwards incompatible way since v1.
> > 
> > I propose that the libxc series be accepted independently of the libxl
> > series.  The libxc series is still implemented as an API-compatible
> > alternative to legacy migration, and is not enabled by default, meaning
> > no change to current users.
> > 
> > However, it will unblock development of ARM migration and COLO, and will
> > allow people in the wider community to experiment with migration v2.  An
> > existing `xl migrate` can be swapped from legacy to migration v2 simply
> > by setting an environment variable (for PV guests.  HVM requires the
> > libxl changed as the handling of the Qemu save record is changing.)
> > 
> > I feel that this is best interest for the community, to get some testing
> > available earlier in the development cycle, rather than attempting to
> > splice 3 large series in a related area together towards the code freeze.
> > 
> > Thoughts? (especially from a release management point of view)
> > 
> 
> I've looked at your libxc series. It looks complete. I agree it would be
> good to get it in as early as possible.

I think the question is more at what point should we flip the switch and
make that new code the default without needing to set magic envvars?

Or to take it a step further, when can we delete all the old code.

Ian.

Re: Migration v2 and related work for 4.6

[Andrew Cooper]

On 15/04/15 13:52, Ian Campbell wrote:
> On Tue, 2015-04-07 at 12:59 +0100, Wei Liu wrote:
>> On Wed, Apr 01, 2015 at 12:03:51PM +0100, Andrew Cooper wrote:
>>> Hello,
>>>
>>> I believe I have CC'd all interested parties.  If I have missed anyone,
>>> please shout.
>>>
>>> There are several large series:
>>> * Cancellable async operations
>>> * COLO
>>>
>>> and a wish to start experimenting with migration on ARM which all
>>> interact with my large series for migration v2 support in libxc and libxl.
>>>
>>> I am working on the libxl side of things, but realise that I am being a
>>> blockage to the other areas waiting on migration v2.
>>>
>>> The current state of the libxc bit is functionally complete.  It is
>>> shipping in two versions of XenServer, has not changed substantially in
>>> 9 months, and not changed in a backwards incompatible way since v1.
>>>
>>> I propose that the libxc series be accepted independently of the libxl
>>> series.  The libxc series is still implemented as an API-compatible
>>> alternative to legacy migration, and is not enabled by default, meaning
>>> no change to current users.
>>>
>>> However, it will unblock development of ARM migration and COLO, and will
>>> allow people in the wider community to experiment with migration v2.  An
>>> existing `xl migrate` can be swapped from legacy to migration v2 simply
>>> by setting an environment variable (for PV guests.  HVM requires the
>>> libxl changed as the handling of the Qemu save record is changing.)
>>>
>>> I feel that this is best interest for the community, to get some testing
>>> available earlier in the development cycle, rather than attempting to
>>> splice 3 large series in a related area together towards the code freeze.
>>>
>>> Thoughts? (especially from a release management point of view)
>>>
>> I've looked at your libxc series. It looks complete. I agree it would be
>> good to get it in as early as possible.
> I think the question is more at what point should we flip the switch and
> make that new code the default without needing to set magic envvars?
>
> Or to take it a step further, when can we delete all the old code.

The other bits needed are:

* Remus/Migrationv2 (not too much, judging from the rfc set)
* A decision regarding tmem.  That is one bit not covered at all any of
the new code, or the legacy conversion.
* Libxl/migrationv2

The libxl migration v2 will contain the legacy->v2 conversion scripts,
which will cover the backwards compatibility aspect.

It is my hope that I can submit a very large pruning patch in time for 4.6.

~Andrew

Re: Migration v2 and related work for 4.6

[Ian Campbell]

On Wed, 2015-04-15 at 13:57 +0100, Andrew Cooper wrote:
> On 15/04/15 13:52, Ian Campbell wrote:
> > On Tue, 2015-04-07 at 12:59 +0100, Wei Liu wrote:
> >> On Wed, Apr 01, 2015 at 12:03:51PM +0100, Andrew Cooper wrote:
> >>> Hello,
> >>>
> >>> I believe I have CC'd all interested parties.  If I have missed anyone,
> >>> please shout.
> >>>
> >>> There are several large series:
> >>> * Cancellable async operations
> >>> * COLO
> >>>
> >>> and a wish to start experimenting with migration on ARM which all
> >>> interact with my large series for migration v2 support in libxc and libxl.
> >>>
> >>> I am working on the libxl side of things, but realise that I am being a
> >>> blockage to the other areas waiting on migration v2.
> >>>
> >>> The current state of the libxc bit is functionally complete.  It is
> >>> shipping in two versions of XenServer, has not changed substantially in
> >>> 9 months, and not changed in a backwards incompatible way since v1.
> >>>
> >>> I propose that the libxc series be accepted independently of the libxl
> >>> series.  The libxc series is still implemented as an API-compatible
> >>> alternative to legacy migration, and is not enabled by default, meaning
> >>> no change to current users.
> >>>
> >>> However, it will unblock development of ARM migration and COLO, and will
> >>> allow people in the wider community to experiment with migration v2.  An
> >>> existing `xl migrate` can be swapped from legacy to migration v2 simply
> >>> by setting an environment variable (for PV guests.  HVM requires the
> >>> libxl changed as the handling of the Qemu save record is changing.)
> >>>
> >>> I feel that this is best interest for the community, to get some testing
> >>> available earlier in the development cycle, rather than attempting to
> >>> splice 3 large series in a related area together towards the code freeze.
> >>>
> >>> Thoughts? (especially from a release management point of view)
> >>>
> >> I've looked at your libxc series. It looks complete. I agree it would be
> >> good to get it in as early as possible.
> > I think the question is more at what point should we flip the switch and
> > make that new code the default without needing to set magic envvars?
> >
> > Or to take it a step further, when can we delete all the old code.
> 
> The other bits needed are:
> 
> * Remus/Migrationv2 (not too much, judging from the rfc set)
> * A decision regarding tmem.  That is one bit not covered at all any of
> the new code, or the legacy conversion.

Adding the tmem maintainer.

Konrad, I think you tmem guys need to step up to the plate and make tmem
work with migration v2, AIUI this has been known for a while and I'm not
in favour of stalling migr2 while we wait.

> * Libxl/migrationv2

> The libxl migration v2 will contain the legacy->v2 conversion scripts,
> which will cover the backwards compatibility aspect.

Ah, OK, and this is why we can't just throw the switch on the libxc side
and do libxl later (for 4.6 or not). I think you've explained this to me
repeatedly and I keep forgetting...

> It is my hope that I can submit a very large pruning patch in time for 4.6.

Great!

Ian.