* [Qemu-devel] [RFC] Migration convergence - a suggestion
@ 2011-12-20 7:06 Ronen Hod
2011-12-20 13:39 ` Anthony Liguori
0 siblings, 1 reply; 4+ messages in thread
From: Ronen Hod @ 2011-12-20 7:06 UTC (permalink / raw)
To: qemu-devel
[-- Attachment #1: Type: text/plain, Size: 1899 bytes --]
Well the issue is not new, anyhow, following a conversation with Orit ...
Since we want the migration to finish, I believe that the "migration
speed" parameter alone cannot do the job.
I suggest using two distinct parameters:
1. Migration speed - will be used to limit the network resources utilization
2. aggressionLevel - A number between 0.0 and 1.0, where low values
imply minimal interruption to the guest, and 1.0 mean that the guest
will be completely stalled.
In any case the migration will have to do its work and finish given any
actual migration-speed, so even low aggressionLevel values will
sometimes imply that the guest will be throttled substantially.
The algorithm:
The aggressionLevel should determine the targetGuest%CPU (how much CPU
time we want to allocate to the guest)
With aggressionLevel = 1.0, the guest gets no CPU-resources (stalled).
With aggressionLevel = 0.0, the guest gets minGuest%CPU, such that
migrationRate == dirtyPagesRate. This minGuest%CPU is continuously
updated based on the running average of the recent samples (more below).
Note that the targetGuest%CPU allocation is continuously updated due to
changes guest behavior, network congestion, and alike.
Some more details
- minGuest%CPU (i.e., for dirtyPagesRate == migrationRate) is easy to
calculate as a running average of
(migrationRate / dirtyPagesRate * guest%CPU)
- There are several methods to calculate the running average, my
favorite is IIR, where, roughly speaking,
newVal = 0.99 * oldVal + 0.01 * newSample
- I would use two measures to ensure that there are more migrated pages
than "dirty" pages.
1. The running average (based on recent samples) of the migrated
pages is larger than that of the new dirty pages
2. The total number of migrated pages so far is larger than the total
number of new dirty pages.
And yes, many details are still missing.
Ronen.
[-- Attachment #2: Type: text/html, Size: 2630 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [RFC] Migration convergence - a suggestion
2011-12-20 7:06 [Qemu-devel] [RFC] Migration convergence - a suggestion Ronen Hod
@ 2011-12-20 13:39 ` Anthony Liguori
2011-12-21 9:47 ` Ronen Hod
0 siblings, 1 reply; 4+ messages in thread
From: Anthony Liguori @ 2011-12-20 13:39 UTC (permalink / raw)
To: Ronen Hod; +Cc: qemu-devel
On 12/20/2011 01:06 AM, Ronen Hod wrote:
> Well the issue is not new, anyhow, following a conversation with Orit ...
>
> Since we want the migration to finish, I believe that the "migration speed"
> parameter alone cannot do the job.
> I suggest using two distinct parameters:
> 1. Migration speed - will be used to limit the network resources utilization
> 2. aggressionLevel - A number between 0.0 and 1.0, where low values imply
> minimal interruption to the guest, and 1.0 mean that the guest will be
> completely stalled.
>
> In any case the migration will have to do its work and finish given any actual
> migration-speed, so even low aggressionLevel values will sometimes imply that
> the guest will be throttled substantially.
>
> The algorithm:
> The aggressionLevel should determine the targetGuest%CPU (how much CPU time we
> want to allocate to the guest)
QEMU has no way to limit the guest CPU time.
Regards,
Anthony Liguori
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [RFC] Migration convergence - a suggestion
2011-12-20 13:39 ` Anthony Liguori
@ 2011-12-21 9:47 ` Ronen Hod
2011-12-21 14:11 ` Anthony Liguori
0 siblings, 1 reply; 4+ messages in thread
From: Ronen Hod @ 2011-12-21 9:47 UTC (permalink / raw)
To: Anthony Liguori; +Cc: qemu-devel
On 12/20/2011 03:39 PM, Anthony Liguori wrote:
> On 12/20/2011 01:06 AM, Ronen Hod wrote:
>> Well the issue is not new, anyhow, following a conversation with Orit
>> ...
>>
>> Since we want the migration to finish, I believe that the "migration
>> speed"
>> parameter alone cannot do the job.
>> I suggest using two distinct parameters:
>> 1. Migration speed - will be used to limit the network resources
>> utilization
>> 2. aggressionLevel - A number between 0.0 and 1.0, where low values
>> imply
>> minimal interruption to the guest, and 1.0 mean that the guest will be
>> completely stalled.
>>
>> In any case the migration will have to do its work and finish given
>> any actual
>> migration-speed, so even low aggressionLevel values will sometimes
>> imply that
>> the guest will be throttled substantially.
>>
>> The algorithm:
>> The aggressionLevel should determine the targetGuest%CPU (how much
>> CPU time we
>> want to allocate to the guest)
>
> QEMU has no way to limit the guest CPU time.
Wouldn't any "yield" (sleep / whatever) limit the guest's CPU time, be
it in qemu or in KVM.
My intention is to suggest an algorithm that is based on guest
throttling. Looking at the relevant BZs, I do not see how we can avoid
it. I certainly have no claims regarding the architecture.
Avi and mst, believe that it is better to continuously control the
guest's CPU from the outside (libvirt) using cgroups. Although less
responsive to changes, it should still work.
In the meantime, I also discovered that everybody has a different point
of view regarding the requirements. Regardless, I believe that the same
basic mechanics (once decided), can do the work
Some relevant configuration "requirements" are:
1. Max bandwidth
2. Min CPU per guest
3. Max guest stall time
4. Max migration time
These requirements will often conflict, and may imply changes in
behavior over time.
I would also suggest that the management GUI will let the user select
the aggression-level (or whatever), and display the implication on all
the other parameters (total-time, %CPU) based on the current behavior of
the guest and network.
Regards, Ronen
>
> Regards,
>
> Anthony Liguori
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Qemu-devel] [RFC] Migration convergence - a suggestion
2011-12-21 9:47 ` Ronen Hod
@ 2011-12-21 14:11 ` Anthony Liguori
0 siblings, 0 replies; 4+ messages in thread
From: Anthony Liguori @ 2011-12-21 14:11 UTC (permalink / raw)
To: Ronen Hod; +Cc: qemu-devel
On 12/21/2011 03:47 AM, Ronen Hod wrote:
> On 12/20/2011 03:39 PM, Anthony Liguori wrote:
>> On 12/20/2011 01:06 AM, Ronen Hod wrote:
>>> Well the issue is not new, anyhow, following a conversation with Orit ...
>>>
>>> Since we want the migration to finish, I believe that the "migration speed"
>>> parameter alone cannot do the job.
>>> I suggest using two distinct parameters:
>>> 1. Migration speed - will be used to limit the network resources utilization
>>> 2. aggressionLevel - A number between 0.0 and 1.0, where low values imply
>>> minimal interruption to the guest, and 1.0 mean that the guest will be
>>> completely stalled.
>>>
>>> In any case the migration will have to do its work and finish given any actual
>>> migration-speed, so even low aggressionLevel values will sometimes imply that
>>> the guest will be throttled substantially.
>>>
>>> The algorithm:
>>> The aggressionLevel should determine the targetGuest%CPU (how much CPU time we
>>> want to allocate to the guest)
>>
>> QEMU has no way to limit the guest CPU time.
>
> Wouldn't any "yield" (sleep / whatever) limit the guest's CPU time, be it in
> qemu or in KVM.
Practically speaking, not really.
> My intention is to suggest an algorithm that is based on guest throttling.
> Looking at the relevant BZs, I do not see how we can avoid it. I certainly have
> no claims regarding the architecture.
> Avi and mst, believe that it is better to continuously control the guest's CPU
> from the outside (libvirt) using cgroups. Although less responsive to changes,
> it should still work.
Yes, that's really the only way to do it I think. You'll need to use the CFS
bandwidth controller.
> In the meantime, I also discovered that everybody has a different point of view
> regarding the requirements. Regardless, I believe that the same basic mechanics
> (once decided), can do the work
> Some relevant configuration "requirements" are:
> 1. Max bandwidth
> 2. Min CPU per guest
> 3. Max guest stall time
> 4. Max migration time
> These requirements will often conflict, and may imply changes in behavior over
> time.
>
> I would also suggest that the management GUI will let the user select the
> aggression-level (or whatever), and display the implication on all the other
> parameters (total-time, %CPU) based on the current behavior of the guest and
> network.
It really depends on your management GUI. If you have a task oriented GUI, you
are unlikely to have a "live migration" feature. Rather, you'll have a
mechanism to enable load balancing (which may use live migration) or you'll have
an "evacuate node" feature to allow system maintenance.
Your downtime policy really depends on the feature. The best thing for QEMU to
do is make sure we expose all of the necessary hooks to allow a wide variety of
policies to be implemented.
Regards,
Anthony Liguori
>
> Regards, Ronen
>
>>
>> Regards,
>>
>> Anthony Liguori
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2011-12-21 14:11 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-12-20 7:06 [Qemu-devel] [RFC] Migration convergence - a suggestion Ronen Hod
2011-12-20 13:39 ` Anthony Liguori
2011-12-21 9:47 ` Ronen Hod
2011-12-21 14:11 ` Anthony Liguori
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.