Re: [PATCH 1 of 3 v5/leftover] libxl: enable automatic placement of guests on NUMA nodes [and 1 more messages]

From: Dario Faggioli <raistlin@linux.it>
To: Ian Campbell <Ian.Campbell@citrix.com>
Cc: Andre Przywara <andre.przywara@amd.com>,
	Stefano Stabellini <Stefano.Stabellini@eu.citrix.com>,
	George Dunlap <George.Dunlap@eu.citrix.com>,
	Juergen Gross <juergen.gross@ts.fujitsu.com>,
	Ian Jackson <Ian.Jackson@eu.citrix.com>,
	xen-devel <xen-devel@lists.xen.org>
Subject: Re: [PATCH 1 of 3 v5/leftover] libxl: enable automatic placement of guests on NUMA nodes [and 1 more messages]
Date: Wed, 18 Jul 2012 02:31:34 +0200	[thread overview]
Message-ID: <1342571494.11794.96.camel@Abyss> (raw)
In-Reply-To: <1342556600.18848.10.camel@dagon.hellion.org.uk>

[-- Attachment #1.1: Type: text/plain, Size: 2250 bytes --]

On Tue, 2012-07-17 at 21:23 +0100, Ian Campbell wrote: 
> > I've been told that boxes with 16 NUMA nodes are coming out about
> > now.
> > 
> > This algorithm will start to have unreasonable performance with
> > anything bigger than that.  Eg choosing 16 NUMA nodes out of 32 would
> > involve searching 601,080,390 combinations.  32 out of 64 gives 10^18.
> > 
> > So I think we need a different approach :-(.
> 
> If 16 are coming out now then 32 nodes are at least a little in the
> future, and 16 won't be common just yet.
> 
FWIW, I agree. Consider that 16 nodes systems exists right now only
because AMD has an architecture where they're putting 2 nodes on one
socket, which is something pretty special that might require some
special thinking and consideration already. The point being assuming
future 16, 32 and 64 nodes systems to just be like the machines we see
now, only bigger, won't be something letting me sleep at night... :-D

> The 4.2 freeze is starting to drag on and reworking this significantly
> at this stage seems likely to take quite a while -- can we perhaps go
> with the existing patches (with the known scalability limitation and
> workaround of setting the explicit cpu affinity documented in the
> release notes) and revisit for 4.3 and/or 4.2.1?
> 
That would be fine by me.

I already said in the other e-mail how I think we can and should address
the potential scalability issue, at whatever time you think it's best.
I'm fine with putting a limit on the maximum number of nodes a guest
should span on (and that being more a feature than a workaround, and can
definitely go in the release notes) wherever you like.

If it's just a matter of putting a LIBXL__MAX_GUEST_NODES (or whatever)
macro in libxl_internal.h, and enforce that, I'm fine doing it right
now, as I'm reposting these last three patches either way. We just need
to agree on a value to write besides it. :-)

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel