* Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
@ 2013-10-29 10:26 James Dingwall
2013-10-29 10:59 ` Wei Liu
0 siblings, 1 reply; 9+ messages in thread
From: James Dingwall @ 2013-10-29 10:26 UTC (permalink / raw)
To: xen-devel
Hi,
I am having some memory ballooning problems which only seem to have
appeared since upgrading to Xen 4.3. In short if I set the domU
configuration file as:
memory = 512
maxmem = 1024
Then even under memory pressure the guest domain does not balloon past
the value of the 'memory' parameter.
xl info shows plenty of available memory in dom0:
total_memory : 32767
free_memory : 12445
xl list -l on the domain shows:
"max_memkb": 1048576,
"target_memkb": 524288,
xl top shows:
NAME guestdom
STATE --b---
CPU(sec) 79
CPU(%) 1.2
MEM(k) 343116
MEM(%) 1.0
MAXMEM(k) 525312
MAXMEM(%) 1.6
VCPUS 2
NETS 0
NETTX(k) 0
NETRX(k) 0
VBDS 2
VBD_OO 0
VBD_RD 8639
VBD_WR 129666
VBD_RSECT 529846
VBD_WSECT 1622593
SSID 0
In the domU dmesg it seems as though the guest has some idea that it has
a 'maxmem' size of 1024Mb:
[ 0.000000] Memory: 408932K/1048188K available (4562K kernel code,
541K rwdata, 1740K rodata, 924K init, 644K bss, 639256K reserved)
but /proc/iomap seems to have reduced it to 'memory':
00000000-00000fff : reserved
00001000-0009ffff : System RAM
000a0000-000fffff : reserved
000f0000-000fffff : System ROM
00100000-1fffffff : System RAM
01000000-01474c38 : Kernel code
01474c39-016b167f : Kernel data
017a0000-01840fff : Kernel bss
20000000-d7feffff : Unusable memory
d7ff0000-d7ffdfff : ACPI Tables
d7ffe000-d7ffffff : ACPI Non-volatile Storage
100000000-11fffffff : System RAM
After running
xl mem-max guestdom 1024
xl list -l remains
"max_memkb": 1048576,
"target_memkb": 524288,
xl top has changed:
NAME guestdom
STATE --b---
CPU(sec) 98
CPU(%) 0.9
MEM(k) 343116
MEM(%) 1.0
MAXMEM(k) 1049600
MAXMEM(%) 3.1
VCPUS 2
NETS 0
NETTX(k) 0
NETRX(k) 0
VBDS 2
VBD_OO 0
VBD_RD 9068
VBD_WR 130257
VBD_RSECT 549640
VBD_WSECT 1628666
SSID 0
Although /proc/iomem doesn't change the guest has no problem ballooning
up to the 'maxmem' value. It seems that this is a bug with xl not
setting the appropriate paramter somewhere when the domain is created.
The domU and dom0 kernel version is 3.11.6, Xen is 4.3. Please let me
know if examinging any other settings before/after the xl mem-max
command is run would be helpful.
Thanks,
James
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 10:26 Bug - Xen 4.3 - xl ignores maxmem setting in domU config file James Dingwall
@ 2013-10-29 10:59 ` Wei Liu
2013-10-29 11:06 ` Andrew Cooper
2013-10-29 11:40 ` James Dingwall
0 siblings, 2 replies; 9+ messages in thread
From: Wei Liu @ 2013-10-29 10:59 UTC (permalink / raw)
To: James Dingwall; +Cc: wei.liu2, xen-devel
On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
> Hi,
>
> I am having some memory ballooning problems which only seem to have
> appeared since upgrading to Xen 4.3. In short if I set the domU
> configuration file as:
>
> memory = 512
> maxmem = 1024
>
> Then even under memory pressure the guest domain does not balloon past
> the value of the 'memory' parameter.
>
> xl info shows plenty of available memory in dom0:
> total_memory : 32767
> free_memory : 12445
>
>
> xl list -l on the domain shows:
> "max_memkb": 1048576,
> "target_memkb": 524288,
>
This is parsed from your config file so they should always look OK to
you.
Does the following patch help?
Git blame tells me the change to set maxmem to target_memkb was
introduced 4 years ago so I suspect there's reason to do that. If we
cannot fix it here we need to insert the corresponding call later.
Wei.
----8<---
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 356f920..fb7965d 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap);
libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap);
- xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT);
+ xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT);
xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL);
state->store_domid = xs_domid ? atoi(xs_domid) : 0;
free(xs_domid);
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 10:59 ` Wei Liu
@ 2013-10-29 11:06 ` Andrew Cooper
2013-10-29 12:08 ` Wei Liu
2013-10-29 11:40 ` James Dingwall
1 sibling, 1 reply; 9+ messages in thread
From: Andrew Cooper @ 2013-10-29 11:06 UTC (permalink / raw)
To: Wei Liu; +Cc: James Dingwall, xen-devel
On 29/10/13 10:59, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
>> Hi,
>>
>> I am having some memory ballooning problems which only seem to have
>> appeared since upgrading to Xen 4.3. In short if I set the domU
>> configuration file as:
>>
>> memory = 512
>> maxmem = 1024
>>
>> Then even under memory pressure the guest domain does not balloon past
>> the value of the 'memory' parameter.
>>
>> xl info shows plenty of available memory in dom0:
>> total_memory : 32767
>> free_memory : 12445
>>
>>
>> xl list -l on the domain shows:
>> "max_memkb": 1048576,
>> "target_memkb": 524288,
>>
> This is parsed from your config file so they should always look OK to
> you.
>
> Does the following patch help?
>
> Git blame tells me the change to set maxmem to target_memkb was
> introduced 4 years ago so I suspect there's reason to do that. If we
> cannot fix it here we need to insert the corresponding call later.
When setmaxmem sets a limit lower than current, the domain can strictly
only balloon down until it is equal to or under the new limit.
Performing a setmaxmem hypercall to info->target_memkb will prevent from
domain from ballooning down then back up a bit, when it has been asked
to balloon down a lot; i.e. it must strictly balloon down to the limit
it has been given.
Therefore, I am not sure this change is valid.
~Andrew
>
> Wei.
> ----8<---
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 356f920..fb7965d 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
> libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap);
> libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap);
>
> - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT);
> + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT);
> xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL);
> state->store_domid = xs_domid ? atoi(xs_domid) : 0;
> free(xs_domid);
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 10:59 ` Wei Liu
2013-10-29 11:06 ` Andrew Cooper
@ 2013-10-29 11:40 ` James Dingwall
1 sibling, 0 replies; 9+ messages in thread
From: James Dingwall @ 2013-10-29 11:40 UTC (permalink / raw)
To: Wei Liu; +Cc: James Dingwall, xen-devel
On Tue, Oct 29, 2013 at 10:59:21AM +0000, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
> > Hi,
> >
> > I am having some memory ballooning problems which only seem to have
> > appeared since upgrading to Xen 4.3. In short if I set the domU
> > configuration file as:
> >
> > memory = 512
> > maxmem = 1024
> >
> > Then even under memory pressure the guest domain does not balloon past
> > the value of the 'memory' parameter.
> >
> > xl info shows plenty of available memory in dom0:
> > total_memory : 32767
> > free_memory : 12445
> >
> >
> > xl list -l on the domain shows:
> > "max_memkb": 1048576,
> > "target_memkb": 524288,
> >
>
> This is parsed from your config file so they should always look OK to
> you.
>
> Does the following patch help?
>
> Git blame tells me the change to set maxmem to target_memkb was
> introduced 4 years ago so I suspect there's reason to do that. If we
> cannot fix it here we need to insert the corresponding call later.
>
> Wei.
> ----8<---
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 356f920..fb7965d 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -235,7 +235,7 @@ int libxl__build_pre(libxl__gc *gc, uint32_t domid,
> libxl_domain_set_nodeaffinity(ctx, domid, &info->nodemap);
> libxl_set_vcpuaffinity_all(ctx, domid, info->max_vcpus, &info->cpumap);
>
> - xc_domain_setmaxmem(ctx->xch, domid, info->target_memkb + LIBXL_MAXMEM_CONSTANT);
> + xc_domain_setmaxmem(ctx->xch, domid, info->max_memkb + LIBXL_MAXMEM_CONSTANT);
> xs_domid = xs_read(ctx->xsh, XBT_NULL, "/tool/xenstored/domid", NULL);
> state->store_domid = xs_domid ? atoi(xs_domid) : 0;
> free(xs_domid);
>
This patch solves the problem for me where maxmem in config file is not
honoured during domain creation. With it applied xl top immediately
shows the correct value in the MAXMEM column.
I can see that xl_cmdimpl.c safely sets max_memkb for the case where the
configuration file does not define maxmem.
Thanks,
James
--
------------------------------------------------------------------------
James Dingwall
e: james@dingwall.me.uk
w: http://www.dingwall.me.uk/
------------------------------------------------------------------------
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 11:06 ` Andrew Cooper
@ 2013-10-29 12:08 ` Wei Liu
2013-10-29 14:49 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 9+ messages in thread
From: Wei Liu @ 2013-10-29 12:08 UTC (permalink / raw)
To: Andrew Cooper; +Cc: James Dingwall, Stefano Stabellini, Wei Liu, xen-devel
On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote:
> On 29/10/13 10:59, Wei Liu wrote:
> > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
> >> Hi,
> >>
> >> I am having some memory ballooning problems which only seem to have
> >> appeared since upgrading to Xen 4.3. In short if I set the domU
> >> configuration file as:
> >>
> >> memory = 512
> >> maxmem = 1024
> >>
> >> Then even under memory pressure the guest domain does not balloon past
> >> the value of the 'memory' parameter.
> >>
> >> xl info shows plenty of available memory in dom0:
> >> total_memory : 32767
> >> free_memory : 12445
> >>
> >>
> >> xl list -l on the domain shows:
> >> "max_memkb": 1048576,
> >> "target_memkb": 524288,
> >>
> > This is parsed from your config file so they should always look OK to
> > you.
> >
> > Does the following patch help?
> >
> > Git blame tells me the change to set maxmem to target_memkb was
> > introduced 4 years ago so I suspect there's reason to do that. If we
> > cannot fix it here we need to insert the corresponding call later.
>
> When setmaxmem sets a limit lower than current, the domain can strictly
> only balloon down until it is equal to or under the new limit.
>
> Performing a setmaxmem hypercall to info->target_memkb will prevent from
> domain from ballooning down then back up a bit, when it has been asked
> to balloon down a lot; i.e. it must strictly balloon down to the limit
> it has been given.
>
> Therefore, I am not sure this change is valid.
>
> ~Andrew
>
Looking at the changeset 9905ac that introduced target_memkb,
target_memkb was always set to max_memkb, so my guess is that the
purpose was still to use max_memkb to be max memory limit.
Probably later changes to that portion of code altered that behavior.
The use of target_memkb in xc_domain_setmaxmem looks logically wrong to
me anyway...
Stefano, thoughts?
Wei.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 12:08 ` Wei Liu
@ 2013-10-29 14:49 ` Konrad Rzeszutek Wilk
2013-10-29 18:02 ` Daniel Kiper
0 siblings, 1 reply; 9+ messages in thread
From: Konrad Rzeszutek Wilk @ 2013-10-29 14:49 UTC (permalink / raw)
To: Wei Liu, Daniel Kiper
Cc: Andrew Cooper, xen-devel, James Dingwall, Stefano Stabellini
On Tue, Oct 29, 2013 at 12:08:04PM +0000, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote:
> > On 29/10/13 10:59, Wei Liu wrote:
> > > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
> > >> Hi,
> > >>
> > >> I am having some memory ballooning problems which only seem to have
> > >> appeared since upgrading to Xen 4.3. In short if I set the domU
> > >> configuration file as:
> > >>
> > >> memory = 512
> > >> maxmem = 1024
> > >>
> > >> Then even under memory pressure the guest domain does not balloon past
> > >> the value of the 'memory' parameter.
> > >>
> > >> xl info shows plenty of available memory in dom0:
> > >> total_memory : 32767
> > >> free_memory : 12445
> > >>
> > >>
> > >> xl list -l on the domain shows:
> > >> "max_memkb": 1048576,
> > >> "target_memkb": 524288,
> > >>
> > > This is parsed from your config file so they should always look OK to
> > > you.
> > >
> > > Does the following patch help?
> > >
> > > Git blame tells me the change to set maxmem to target_memkb was
> > > introduced 4 years ago so I suspect there's reason to do that. If we
> > > cannot fix it here we need to insert the corresponding call later.
> >
> > When setmaxmem sets a limit lower than current, the domain can strictly
> > only balloon down until it is equal to or under the new limit.
> >
> > Performing a setmaxmem hypercall to info->target_memkb will prevent from
> > domain from ballooning down then back up a bit, when it has been asked
> > to balloon down a lot; i.e. it must strictly balloon down to the limit
> > it has been given.
> >
> > Therefore, I am not sure this change is valid.
> >
> > ~Andrew
> >
>
> Looking at the changeset 9905ac that introduced target_memkb,
> target_memkb was always set to max_memkb, so my guess is that the
> purpose was still to use max_memkb to be max memory limit.
>
> Probably later changes to that portion of code altered that behavior.
>
> The use of target_memkb in xc_domain_setmaxmem looks logically wrong to
> me anyway...
>
> Stefano, thoughts?
Lets also include Daniel in this - as he was trying to fix this in the past.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 14:49 ` Konrad Rzeszutek Wilk
@ 2013-10-29 18:02 ` Daniel Kiper
2013-10-30 11:39 ` Wei Liu
0 siblings, 1 reply; 9+ messages in thread
From: Daniel Kiper @ 2013-10-29 18:02 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk
Cc: Wei Liu, ian.campbell, Stefano Stabellini, Andrew Cooper,
James Dingwall, xen-devel, ian.jackson
On Tue, Oct 29, 2013 at 10:49:35AM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Oct 29, 2013 at 12:08:04PM +0000, Wei Liu wrote:
> > On Tue, Oct 29, 2013 at 11:06:01AM +0000, Andrew Cooper wrote:
> > > On 29/10/13 10:59, Wei Liu wrote:
> > > > On Tue, Oct 29, 2013 at 10:26:00AM +0000, James Dingwall wrote:
> > > >> Hi,
> > > >>
> > > >> I am having some memory ballooning problems which only seem to have
> > > >> appeared since upgrading to Xen 4.3. In short if I set the domU
> > > >> configuration file as:
> > > >>
> > > >> memory = 512
> > > >> maxmem = 1024
> > > >>
> > > >> Then even under memory pressure the guest domain does not balloon past
> > > >> the value of the 'memory' parameter.
> > > >>
> > > >> xl info shows plenty of available memory in dom0:
> > > >> total_memory : 32767
> > > >> free_memory : 12445
> > > >>
> > > >>
> > > >> xl list -l on the domain shows:
> > > >> "max_memkb": 1048576,
> > > >> "target_memkb": 524288,
> > > >>
> > > > This is parsed from your config file so they should always look OK to
> > > > you.
> > > >
> > > > Does the following patch help?
> > > >
> > > > Git blame tells me the change to set maxmem to target_memkb was
> > > > introduced 4 years ago so I suspect there's reason to do that. If we
> > > > cannot fix it here we need to insert the corresponding call later.
> > >
> > > When setmaxmem sets a limit lower than current, the domain can strictly
> > > only balloon down until it is equal to or under the new limit.
> > >
> > > Performing a setmaxmem hypercall to info->target_memkb will prevent from
> > > domain from ballooning down then back up a bit, when it has been asked
> > > to balloon down a lot; i.e. it must strictly balloon down to the limit
> > > it has been given.
> > >
> > > Therefore, I am not sure this change is valid.
> > >
> > > ~Andrew
> > >
> >
> > Looking at the changeset 9905ac that introduced target_memkb,
> > target_memkb was always set to max_memkb, so my guess is that the
> > purpose was still to use max_memkb to be max memory limit.
> >
> > Probably later changes to that portion of code altered that behavior.
> >
> > The use of target_memkb in xc_domain_setmaxmem looks logically wrong to
> > me anyway...
> >
> > Stefano, thoughts?
>
> Lets also include Daniel in this - as he was trying to fix this in the past.
As I remember this behavior is by design. However, this is not in line with
xm behavior which sometimes makes a problem. I have discussed this with IanJ
and IanC (both CC-ed here) once. I have proposed some patch series to fix that
issue but later I was not able to continue work on it. If you wish I could
repost them once again with fixes requested by IanJ and IanC. You could also
find latest version of patches here:
http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html
Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-29 18:02 ` Daniel Kiper
@ 2013-10-30 11:39 ` Wei Liu
2013-10-31 13:35 ` Daniel Kiper
0 siblings, 1 reply; 9+ messages in thread
From: Wei Liu @ 2013-10-30 11:39 UTC (permalink / raw)
To: Daniel Kiper
Cc: Wei Liu, ian.campbell, Stefano Stabellini, Andrew Cooper,
James Dingwall, xen-devel, ian.jackson
On Tue, Oct 29, 2013 at 07:02:06PM +0100, Daniel Kiper wrote:
[...]
> > >
> > > Stefano, thoughts?
> >
> > Lets also include Daniel in this - as he was trying to fix this in the past.
>
> As I remember this behavior is by design. However, this is not in line with
> xm behavior which sometimes makes a problem. I have discussed this with IanJ
> and IanC (both CC-ed here) once. I have proposed some patch series to fix that
> issue but later I was not able to continue work on it. If you wish I could
> repost them once again with fixes requested by IanJ and IanC. You could also
> find latest version of patches here:
>
> http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html
>
Oh, sure. If you have something new then please do. ;-)
AFAICT from the latest series, the real meat that fixes this problem is
in patch 2, which got no reply. Does that mean no objection was raised
on that approach?
Wei.
> Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Bug - Xen 4.3 - xl ignores maxmem setting in domU config file
2013-10-30 11:39 ` Wei Liu
@ 2013-10-31 13:35 ` Daniel Kiper
0 siblings, 0 replies; 9+ messages in thread
From: Daniel Kiper @ 2013-10-31 13:35 UTC (permalink / raw)
To: Wei Liu
Cc: ian.campbell, Stefano Stabellini, Andrew Cooper, James Dingwall,
xen-devel, ian.jackson
On Wed, Oct 30, 2013 at 11:39:51AM +0000, Wei Liu wrote:
> On Tue, Oct 29, 2013 at 07:02:06PM +0100, Daniel Kiper wrote:
> [...]
> > > >
> > > > Stefano, thoughts?
> > >
> > > Lets also include Daniel in this - as he was trying to fix this in the past.
> >
> > As I remember this behavior is by design. However, this is not in line with
> > xm behavior which sometimes makes a problem. I have discussed this with IanJ
> > and IanC (both CC-ed here) once. I have proposed some patch series to fix that
> > issue but later I was not able to continue work on it. If you wish I could
> > repost them once again with fixes requested by IanJ and IanC. You could also
> > find latest version of patches here:
> >
> > http://lists.xenproject.org/archives/html/xen-devel/2013-04/msg03072.html
> >
>
> Oh, sure. If you have something new then please do. ;-)
>
> AFAICT from the latest series, the real meat that fixes this problem is
> in patch 2, which got no reply. Does that mean no objection was raised
IIRC patch number 1 is much more important in your case. If you can test
it (and others if it is possible) then it will be nice. However, patches
may require some tweaks to be applied on current unstable tree.
> on that approach?
In general there were only small issues left. I will fix them and repost
when I will be sure that they solve your problems.
Daniel
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2013-10-31 13:35 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-10-29 10:26 Bug - Xen 4.3 - xl ignores maxmem setting in domU config file James Dingwall
2013-10-29 10:59 ` Wei Liu
2013-10-29 11:06 ` Andrew Cooper
2013-10-29 12:08 ` Wei Liu
2013-10-29 14:49 ` Konrad Rzeszutek Wilk
2013-10-29 18:02 ` Daniel Kiper
2013-10-30 11:39 ` Wei Liu
2013-10-31 13:35 ` Daniel Kiper
2013-10-29 11:40 ` James Dingwall
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.