Re: [PATCH 0 of 8] NUMA Awareness for the Credit Scheduler

From: Dario Faggioli <dario.faggioli@citrix.com>
To: Dan Magenheimer <dan.magenheimer@oracle.com>
Cc: Marcus Granado <Marcus.Granado@eu.citrix.com>,
	Andre Przywara <andre.przywara@amd.com>,
	Ian Campbell <Ian.Campbell@citrix.com>,
	Anil Madhavapeddy <anil@recoil.org>,
	George Dunlap <george.dunlap@eu.citrix.com>,
	Andrew Cooper <Andrew.Cooper3@citrix.com>,
	Juergen Gross <juergen.gross@ts.fujitsu.com>,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	xen-devel@lists.xen.org, Jan Beulich <JBeulich@suse.com>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>,
	Matt Wilson <msw@amazon.com>
Subject: Re: [PATCH 0 of 8] NUMA Awareness for the Credit Scheduler
Date: Tue, 9 Oct 2012 11:45:49 +0100	[thread overview]
Message-ID: <1349779549.3610.79.camel@Abyss> (raw)
In-Reply-To: <f4a57de5-4065-4248-a958-040fe9fe3477@default>

[-- Attachment #1.1: Type: text/plain, Size: 2432 bytes --]

On Mon, 2012-10-08 at 12:43 -0700, Dan Magenheimer wrote: 
> Just wondering... is the NUMA information preserved on live migration?
> I'm not saying that it necessarily should, but it may just work
> due to the implementation (since migration is a form of domain creation).
>
What could I say... yes, but "preserved" is not the right word. :-)

In fact, something happens when you migrate a VM. As you said, migration
is a special case of domain creation, so the placement algorithm will
trigger as a part of the process of creating the target VM (unless you
override the relevant options in the config file during migration
itself). That means the target VM will be placed on one (some) node(s)
of the target host, and it's node-affinity will be set accordingly.

_However_, there is right now no guarantee for the final decision from
the placing algorithm on the target machine to be "compatible" with the
one made on the source machine at initial VM creation time. For
instance, if your VM fits in just one node and is placed there on
machine A, it well could end up being split on two or more nodes when
migrated on machine B (and, of course, the vice versa).

Whether, that is acceptable or not, is of course debatable, and we had a
bit of this discussion already (although no real conclusion has been
reached yet).
My take is that, right now, since we do not yet expose any virtual NUMA
topology to the VM itself, the behaviour described above is fine. As
soon as we'll have some guest NUMA awareness, than it might be
worthwhile to try to preserve it, at least to some extent.

Oh, and BTW, I'm of course talking about migration with xl and libxl. If
you use other toolstacks, then the hypervisor will default to his
current (_without_ this series) behaviour, and it all will depend on who
and when calls xc_domain_node_setaffinity() or, perhaps,
XEN_DOMCTL_setnodeaffinity directly.

> In either case, it might be good to comment about live migration
> on your wiki.
> 
That is definitely a good point, I will put something there about
migration and the behaviour described above.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel