[Qemu-devel] Question with migrate_set_speed

[Qemu-devel] Question with migrate_set_speed

[Peter Xu]

Hi,

I am playing with live migration and got one question about live
migration set_speed.

Now we can use migrate_set_speed to configure the threshold during
migration (it should be only used for precopy, so let's assume the
migration is a precopy case). However I feel like this single
parameter cannot describe the process very clearly.

The problem is, we use this "speed" value as both:

1. transfer threshold, to make sure migration stream is relatively
   constant and under control

2. value to estimate "when we should stop the vm and start counting
   downtime"

For (1), we want a customized value that best suites our network
environment. E.g., if we have other critical network payloads, we can
set this to a very small number, so the migration stream will be very
small.

For (2), it should be nothing else but possibly the network bandwidth
that the system can provide on the migration link.

We can set_speed to a very small value to avoid disturbing other
network services, that's good. However using the same value to
estimate "when to stop" seems odd, since this value can be far away
from the real network speed (when we are transfering the last bits in
precopy, we should be using the max network speed all the time, which
actually is not affected by the threshold value).

IMHO, it'll be clearer if these are two different tunables, e.g., not
sure whether it'll be cool to add another tunable "set_speed_max", to
configure the speed for the (2) case (when vm is stopped on source).
If so, it'll be clearer, and also bring another benefit: users can
have full control of the last migration phase as well, in case they
don't want to suffer from a network fluctuation for each ends of
migration.

Still haven't thought further. Just throw this question out.

Thanks,

-- peterx

Re: [Qemu-devel] Question with migrate_set_speed

[Dr. David Alan Gilbert]

* Peter Xu (peterx@redhat.com) wrote:
> Hi,
> 
> I am playing with live migration and got one question about live
> migration set_speed.
> 
> Now we can use migrate_set_speed to configure the threshold during
> migration (it should be only used for precopy, so let's assume the
> migration is a precopy case). However I feel like this single
> parameter cannot describe the process very clearly.
> 
> The problem is, we use this "speed" value as both:
> 
> 1. transfer threshold, to make sure migration stream is relatively
>    constant and under control
> 
> 2. value to estimate "when we should stop the vm and start counting
>    downtime"
> 
> For (1), we want a customized value that best suites our network
> environment. E.g., if we have other critical network payloads, we can
> set this to a very small number, so the migration stream will be very
> small.
> 
> For (2), it should be nothing else but possibly the network bandwidth
> that the system can provide on the migration link.

Actually, that's what QEMU does; the migrate_set_speed is really only used
for (1) directly.
(2) actually comes from a measured bandwidth multiplied by the 'migrate_set_downtime'
figure. (See around line 1800ish i migration/migration.c inside
the if (current_time >= initial+time + BUFFER_DELAY) if ).


> We can set_speed to a very small value to avoid disturbing other
> network services, that's good. However using the same value to
> estimate "when to stop" seems odd, since this value can be far away
> from the real network speed (when we are transfering the last bits in
> precopy, we should be using the max network speed all the time, which
> actually is not affected by the threshold value).
> 
> IMHO, it'll be clearer if these are two different tunables, e.g., not
> sure whether it'll be cool to add another tunable "set_speed_max", to
> configure the speed for the (2) case (when vm is stopped on source).
> If so, it'll be clearer, and also bring another benefit: users can
> have full control of the last migration phase as well, in case they
> don't want to suffer from a network fluctuation for each ends of
> migration.
> 
> Still haven't thought further. Just throw this question out.

I don't think people are very good at setting the two tunables we already
have!

Dave

> 
> Thanks,
> 
> -- peterx
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] Question with migrate_set_speed

[Peter Xu]

On Fri, Aug 19, 2016 at 09:13:15AM +0100, Dr. David Alan Gilbert wrote:
> Actually, that's what QEMU does; the migrate_set_speed is really only used
> for (1) directly.
> (2) actually comes from a measured bandwidth multiplied by the 'migrate_set_downtime'
> figure. (See around line 1800ish i migration/migration.c inside
> the if (current_time >= initial+time + BUFFER_DELAY) if ).

You are right. Sorry I didn't notice that. :(

> 
> 
> > We can set_speed to a very small value to avoid disturbing other
> > network services, that's good. However using the same value to
> > estimate "when to stop" seems odd, since this value can be far away
> > from the real network speed (when we are transfering the last bits in
> > precopy, we should be using the max network speed all the time, which
> > actually is not affected by the threshold value).
> > 
> > IMHO, it'll be clearer if these are two different tunables, e.g., not
> > sure whether it'll be cool to add another tunable "set_speed_max", to
> > configure the speed for the (2) case (when vm is stopped on source).
> > If so, it'll be clearer, and also bring another benefit: users can
> > have full control of the last migration phase as well, in case they
> > don't want to suffer from a network fluctuation for each ends of
> > migration.
> > 
> > Still haven't thought further. Just throw this question out.
> 
> I don't think people are very good at setting the two tunables we already
> have!

Then I think current tunables are good enough at least to me. And only
to control the last bit transfer seems not essential to introduce
another one. Thanks!

-- peterx