From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mike Latimer Subject: Re: freemem-slack and large memory environments Date: Thu, 26 Feb 2015 10:38:40 -0700 Message-ID: <4455263.R3RfNP62EE@mlatimer1.dnsdhcp.provo.novell.com> References: <4321015.nah3j6dvJq@mlatimer1.dnsdhcp.provo.novell.com> <2100847.1PzneRuFSU@mlatimer1.dnsdhcp.provo.novell.com> <1424966274.14641.89.camel@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1424966274.14641.89.camel@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Ian Campbell Cc: wei.liu2@citrix.com, xen-devel@lists.xen.org, ian.jackson@eu.citrix.com, Stefano Stabellini List-Id: xen-devel@lists.xenproject.org On Thursday, February 26, 2015 03:57:54 PM Ian Campbell wrote: > On Thu, 2015-02-26 at 08:36 -0700, Mike Latimer wrote: > > There is still one aspect of my original patch that is important. As the > > code currently stands, the target for dom0 is set lower during each > > iteration of the loop. Unless only one iteration is required, dom0 will > > end up being set to a much lower target than is actually required. > > Is this because some sort of slack is applied once per iteration rather > than once at the start or is it something else? No - the slack reservation just complicated the request by (potentially) requiring more free memory than domU initially requested. With or without slack, the central loop in tools/libxl/xl_cmdimpl.c:freemem, frees memory for domU by lowering the memory target for dom0. However, this is not a single request (e.g. free 64GB for domX), rather the memory target for dom0 is set lower during every iteration through: rc = libxl_set_memory_target(ctx, 0, free_memkb - need_memkb, 1, 0); This causes dom0's memory target to be lowered by the needed amount during every iteration of the loop. In practice, this causes the first request to lower dom0's target by the full amount (e.g. -64GB), and subsequent iterations further lower dom0's target by however much memory that still appears to be required (e.g. three iterations of the loop might lower dom0's target by -25GB, then -25GB, for a total of dom0 ballooning down 114GB). The issue itself is due to the loop ignoring the fact that the original request set dom0's target to the correct amount, but the ballooning has not completed. The problem itself is easier to see when domU memory sizes are increased. As mentioned before, starting a 512GB domain should guarantee that multiple iterations of the loop are required, and dom0 will balloon down much further than the required 512GB. Does this clarify the situation? -Mike