RE: Ballooning up

[Dan Magenheimer <dan.magenheimer@oracle.com>]

> From: Jeremy Fitzhardinge [mailto:jeremy@goop.org]
> Cc: Dan Magenheimer; Daniel Kiper; Stefano Stabellini; Konrad Rzeszutek
> Wilk
> Subject: Ballooning up
> 
>  I finally got around to implementing "ballooning up" in the pvops
> kernels.  Now if you start a domain with "memory=X maxmem=Y", the
> domain
> will start with X MB of memory, but you can use "x[ml] mem-set" to
> expand the domain up to Y.

Nice!

> As a side-effect, it also works for dom0.  If you set dom0_mem on the
> Xen command line, then nr_pages is limited to that value, but the
> kernel
> can still see the system's real E820 map, and therefore adds all the
> system's memory to its own balloon driver, potentially allowing dom0 to
> expand up to take all physical memory.
> 
> However, this may caused bad side-effects if your system memory is much
> larger than your dom0_mem, especially if you use a 32-bit dom0.  I may
> need to add a kernel command line option to limit the max initial
> balloon size to mitigate this...

I would call this dom0 functionality a bug.  I think both Citrix
and Oracle use dom0_mem as a normal boot option for every
installation and, while I think both employ heuristics to choose
a larger dom0_mem for larger physical memory, I don't think it
grows large enough for, say, >256GB physical memory, to accommodate
the necessarily large number of page tables.

So, I'd vote for NOT allowing dom0 to balloon up to physical
memory if dom0_mem is specified, and possibly a kernel command
line option that allows it to grow beyond.  Or, possibly, no
option and never allow dom0 memory to grow beyond dom0_mem
unless (possibly) it grows with hot-plug.

Dan