Re: freemem-slack and large memory environments

From: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
To: Mike Latimer <mlatimer@suse.com>
Cc: wei.liu2@citrix.com,
	Stefano Stabellini <stefano.stabellini@eu.citrix.com>,
	ian.jackson@eu.citrix.com, Ian Campbell <ian.campbell@citrix.com>,
	xen-devel@lists.xen.org
Subject: Re: freemem-slack and large memory environments
Date: Fri, 27 Feb 2015 10:52:17 +0000	[thread overview]
Message-ID: <alpine.DEB.2.02.1502271044480.23507@kaball.uk.xensource.com> (raw)
In-Reply-To: <1614208.FAtanMl2ZW@mlatimer1.dnsdhcp.provo.novell.com>

On Thu, 26 Feb 2015, Mike Latimer wrote:
> On Thursday, February 26, 2015 01:45:16 PM Mike Latimer wrote:
> > On Thursday, February 26, 2015 05:53:06 PM Stefano Stabellini wrote:
> > > What is the return value of libxl_set_memory_target and
> > > libxl_wait_for_free_memory in that case? Isn't it just a matter of
> > > properly handle the return values?
> > 
> > The return from libxl_set_memory_target is 0, as the assignment works just
> > fine. I don't have the return from libxl_wait_for_free_memory in my notes,
> > so I'll spin up another test and track that down.
> 
> I slightly misspoke here... In my testing, the returns are actually:
> 
>    libxl_set_memory_target = 1

The new memory target is set for dom0 successfully.

>    libxl_wait_for_free_memory = -5

Still there isn't enough free memory in the system.

>    libxl_wait_for_memory_target = 0

However dom0 reached the new memory target already.
Who is stealing your memory?

>       Note - libxl_wait_for_memory_target is confusing, as rc can be set
>       to ERROR_FAIL, but the function returns 0 anyway (unless an error
>       is encountered earlier.) I guess this just means we need to continue
>       to wait...

Maybe I am misunderstanding what you meant, but as far as I can tell rc
is set to ERROR_FAIL only right before the out label in
libxl_wait_for_memory_target. In that case the function would return
ERROR_FAIL.

In any case in the context of libxl_wait_for_memory_target, ERROR_FAIL
means that the memory target has not been reached.

> I was testing spinning up a 64GB guest on a 2TB host. After the ballooning had 
> completed, dom0 had ballooned down an extra ~320GB. On this particular 
> machine, each iteration of the loop was showing only 5-7GB of memory being 
> freed at a time. (The loop took 12 iterations.)

I would investigate why dom0 is ballooning down as much as you asked it
to, but the free memory in the system is still not enough.