From mboxrd@z Thu Jan  1 00:00:00 1970
From: Ian Campbell <Ian.Campbell@citrix.com>
Subject: Re: [PATCH v4 21/21] xl: vNUMA support
Date: Thu, 29 Jan 2015 11:10:39 +0000
Message-ID: <1422529839.30641.42.camel@citrix.com>
References: <1422011632-22018-1-git-send-email-wei.liu2@citrix.com>
	<1422011632-22018-22-git-send-email-wei.liu2@citrix.com>
	<1422463565.5187.68.camel@citrix.com>
	<20150128225202.GG14935@zion.uk.xensource.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
In-Reply-To: <20150128225202.GG14935@zion.uk.xensource.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Wei Liu <wei.liu2@citrix.com>
Cc: ufimtseva@gmail.com, andrew.cooper3@citrix.com, dario.faggioli@citrix.com, ian.jackson@eu.citrix.com, xen-devel@lists.xen.org, JBeulich@suse.com
List-Id: xen-devel@lists.xenproject.org

On Wed, 2015-01-28 at 22:52 +0000, Wei Liu wrote:
> > guests, is preballooning allowed there too?
> 
> I need to check PV boot sequence to have a definite answer.
> 
> Currently memory allocation in libxc only deals with a chunk of
> contiguous memory. Not sure if I change that will break some assumptions
> that guest kernel makes.

Please do check, and if it doesn't work today we really ought to have
plan on how to integrate in the future, in case (as seems likely) it
requires cooperation between tools and kernel -- so we can think about
steps now to make it easier on ourselves then...

> > > +=item B<vnuma_pnode_map=[ NUMBER, NUMBER, ... ]>
> > > +
> > > +Specifiy which physical NUMA node a specific virtual NUMA node maps to. The
> > 
> > "Specify" again.
> > 
> > > +number of elements in this list should be equal to the number of virtual
> > > +NUMA nodes defined in B<vnuma_memory=>.
> > 
> > Would it make sense to instead have a single array or e.g. "NODE:SIZE"
> > or something?
> > 
> 
> Or "PNODE:SIZE:VCPUS"?

That seems plausible.

One concern would be future expansion, perhaps foo=bar,baz=wibble?

> > > +=item B<vnuma_vdistance=[ [NUMBER, ..., NUMBER], [NUMBER, ..., NUMBER], ... ]>
> > > +
> > > +Two dimensional list to specify distances among nodes.
> > > +
> > > +The number of elements in the first dimension list equals the number of virtual
> > > +nodes. Each element in position B<i> is a list that specifies the distances
> > > +from node B<i> to other nodes.
> > > +
> > > +For example, for a guest with 2 virtual nodes, user can specify:
> > > +
> > > +  vnuma_vdistance = [ [10, 20], [20, 10] ]
> > 
> > Any guidance on how a user should choose these numbers?
> > 
> 
> I think using the number from numactl is good enough.

Worth mentioning in the docs I think.

> > Do we support a mode where something figures this out based on the
> > underlying distances between the pnode to which a vnode is assigned?
> > 
> 
> Dario is working on that.

I thought he was working on automatic numa placement (i.e. figuring out
the best set of pnodes to map the vnodes to), whereas what I was
suggesting was that given the user has specified a vnode->pnode mapping
it should be possible to construct a distances table pretty trivially
from that. Is Dario working on that too?

Ian.