Re: [PATCH 2 of 2 RFC] xl: allow for moving the domain's memory when changing vcpu affinity

From: George Dunlap <george.dunlap@eu.citrix.com>
To: Dario Faggioli <raistlin@linux.it>
Cc: "Zhang, Yang Z" <yang.z.zhang@intel.com>,
	Andre Przywara <andre.przywara@amd.com>,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	Ian Campbell <Ian.Campbell@citrix.com>,
	xen-devel <xen-devel@lists.xen.org>
Subject: Re: [PATCH 2 of 2 RFC] xl: allow for moving the domain's memory when changing vcpu affinity
Date: Fri, 6 Jul 2012 15:04:05 +0100	[thread overview]
Message-ID: <4FF6F055.7040300@eu.citrix.com> (raw)
In-Reply-To: <1341583048.25268.28.camel@Solace>

On 06/07/12 14:57, Dario Faggioli wrote:
> On Fri, 2012-07-06 at 13:53 +0100, George Dunlap wrote:
>> On 06/07/12 10:54, Dario Faggioli wrote:
>>> By introducing a new '-M' option to the `xl vcpu-pin' command. The actual
>>> memory "movement" is achieved suspending the domain to a ttemporary file and
>>> resuming it with the new vcpu-affinity
>> Hmm... this will work and be reliable, but it seems a bit clunky.
>>
> If I can ask, the idea or the implementation? :-)
>
>> Long
>> term we want to be able to do node migration in the background without
>> shutting down a VM, right?
>>
> Definitely, and we also want to do that automatically, according to some
> load/performance/whatever run-time measurement, without any user
> intervention. However ...
>
>> If that can be done in the 4.3 timeframe,
>> then it seems unnecessary to implement something like this.
>>
> ... I think something like this could still be useful.
>
> IOW, I think it would be worthwhile to have both the automatic memory
> migration happening in background and something like this explicit,
> do-it-all-now, as they serve different purposes.
Well, I think you can have an explicit, do-it-now mechanism that doesn't 
require suspending the VM; and in fact, I think such a mechanism would 
be much preferable.

Compare it to vcpu pinning: if you decided you wanted to change the vcpu 
pinning, would you rather just be able to pin the maps while the VM was 
running, and have it take effect immediately (as it does now), or would 
you rather have to suspend and resume the VM, taking it out of 
commission for 10 seconds or so?

I'm not opposed to the -M option meaning, "...and please move the memory 
based on the new nodemask".  But I don't think implementing it as 
suspend/resume is really that useful, except as a stopgap measure until 
we can get "live" memory migration working.  If we can get that before 
4.3, there's no need for the stopgap measure.

  -George