[PATCH V3 0/1] libxl: set stub domain size based on VRAM size

[PATCH V3 0/1] libxl: set stub domain size based on VRAM size

[Eric Shelton]

I ran into crashes with qemu-traditional stub domain when 16 MB was
assigned to the stdvga virtual video adapter.  These were occurring due
to an out of memory condition arising from stub domains having a fixed
size of 32 MB, with half of that being demanded for video.  Assuming the
the original value of 32 MB was based on having a 4 MB video adapter,
this increases the stub domain size by however much the VRAM exceeds 4 MB. 

Revised per Ian Jackson's and Samuel Thibault's suggestions.

Eric Shelton (1):
  libxl: set stub domain size based on VRAM size

 tools/libxl/libxl_dm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

-- 
2.1.0

[PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[Eric Shelton]

Allocate additional memory to the stub domain for qemu-traditional if
more than 4 MB is assigned to the video adapter to avoid out of memory
condition for QEMU.

Signed-off-by: Eric Shelton <eshelton@pobox.com>
---
 tools/libxl/libxl_dm.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 317a8eb..9a5a937 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -1087,6 +1087,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
 
     dm_config->b_info.max_vcpus = 1;
-    dm_config->b_info.max_memkb = 32 * 1024;
+    dm_config->b_info.max_memkb = 28 * 1024 +
+        guest_config->b_info.video_memkb;
     dm_config->b_info.target_memkb = dm_config->b_info.max_memkb;
 
     dm_config->b_info.u.pv.features = "";
-- 
2.1.0

Re: [PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[George Dunlap]

On Sat, Jul 11, 2015 at 10:30 PM, Eric Shelton <eshelton@pobox.com> wrote:
> Allocate additional memory to the stub domain for qemu-traditional if
> more than 4 MB is assigned to the video adapter to avoid out of memory
> condition for QEMU.
>
> Signed-off-by: Eric Shelton <eshelton@pobox.com>

This seems like a good fix for now, thanks.

But (just speaking to people in general) it does make it even harder
to predict exactly how much host memory a VM is going to end up
needing to start, which is something a lot of people complain about.

I'm beginning to think that one feature we should try to look at is a
way for the admin to specify, "Max host memory used", from which
shadow/p2m/device memory/stubdomain can all be "allocated".

 -George

> ---
>  tools/libxl/libxl_dm.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 317a8eb..9a5a937 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -1087,6 +1087,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
>
>      dm_config->b_info.max_vcpus = 1;
> -    dm_config->b_info.max_memkb = 32 * 1024;
> +    dm_config->b_info.max_memkb = 28 * 1024 +
> +        guest_config->b_info.video_memkb;
>      dm_config->b_info.target_memkb = dm_config->b_info.max_memkb;
>
>      dm_config->b_info.u.pv.features = "";
> --
> 2.1.0
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[Eric Shelton]

[-- Attachment #1.1: Type: text/plain, Size: 1072 bytes --]

On Jul 14, 2015 4:51 AM, "George Dunlap" <George.Dunlap@eu.citrix.com>
wrote:
>
> On Sat, Jul 11, 2015 at 10:30 PM, Eric Shelton <eshelton@pobox.com> wrote:
> > Allocate additional memory to the stub domain for qemu-traditional if
> > more than 4 MB is assigned to the video adapter to avoid out of memory
> > condition for QEMU.
> >
> > Signed-off-by: Eric Shelton <eshelton@pobox.com>
>
> This seems like a good fix for now, thanks.
>
> But (just speaking to people in general) it does make it even harder
> to predict exactly how much host memory a VM is going to end up
> needing to start, which is something a lot of people complain about.
>
> I'm beginning to think that one feature we should try to look at is a
> way for the admin to specify, "Max host memory used", from which
> shadow/p2m/device memory/stubdomain can all be "allocated".

Just curious: what happens when all of that memory has been "allocated,"
yet there is still plenty of memory overall?  Would you not be able to spin
up a domain despite there actually being plenty of memory to do so?

Eric

[-- Attachment #1.2: Type: text/html, Size: 1467 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[George Dunlap]

On 07/14/2015 11:02 AM, Eric Shelton wrote:
> On Jul 14, 2015 4:51 AM, "George Dunlap" <George.Dunlap@eu.citrix.com>
> wrote:
>>
>> On Sat, Jul 11, 2015 at 10:30 PM, Eric Shelton <eshelton@pobox.com> wrote:
>>> Allocate additional memory to the stub domain for qemu-traditional if
>>> more than 4 MB is assigned to the video adapter to avoid out of memory
>>> condition for QEMU.
>>>
>>> Signed-off-by: Eric Shelton <eshelton@pobox.com>
>>
>> This seems like a good fix for now, thanks.
>>
>> But (just speaking to people in general) it does make it even harder
>> to predict exactly how much host memory a VM is going to end up
>> needing to start, which is something a lot of people complain about.
>>
>> I'm beginning to think that one feature we should try to look at is a
>> way for the admin to specify, "Max host memory used", from which
>> shadow/p2m/device memory/stubdomain can all be "allocated".
> 
> Just curious: what happens when all of that memory has been "allocated,"
> yet there is still plenty of memory overall?  Would you not be able to spin
> up a domain despite there actually being plenty of memory to do so?

The idea would be that you would set this value to say, "2048MiB", and
then all the little bits and bobs would come out of that 2048MiB.  So if
you have a 48MiB stubdom, and 4MiB of device ROMs, and 4MiB of p2ms
(including alternate p2ms), and 1MiB of other miscellaneous stuff, then
the guest would only see 1991MiB of virtual RAM.  (If I've done the math
correctly.)

Which does mean, if you set this value to 50MiB, but run it with a
stubdom and 16MiB of video RAM and altp2ms and everything, you're going
to have problems. :-)

But in the normal use case, it should never cause a guest not to be created.

 -George

Re: [PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[Ian Campbell]

On Sat, 2015-07-11 at 17:30 -0400, Eric Shelton wrote:
> Allocate additional memory to the stub domain for qemu-traditional if
> more than 4 MB is assigned to the video adapter to avoid out of memory
> condition for QEMU.

In reply to v2 (after you'd sent v3, I think) Ian J said:
> Well, it does also reduce the stubdom memory when the video memory is
> <4Mby.  That sounds plausible to me but at the very least the commit
> message needs changing, and maybe we should wait a bit to see if one
> of the qemu experts objects.

libxl__domain_build_info_setdefault ensures, for qemu-xen-traditional,
that video_memkb is either precisely 4M (TYPE_CIRRUS) or >= 8M
(TYPE_STD) but allows a free choice for TYPE_NONE, defaulting to 0 (I'd
almost expect it to insist on 0, but there you are).

I suggest adding a second paragraph to the commit message:

        For LIBXL_VGA_INTERFACE_TYPE_STD + CIRRUS video_memkb is always
        at least 4MB. However if LIBXL_VGA_INTERFACE_TYPE_STD is
        selected the video_memkb defaults to 0, resulting in a stubdom
        which is 4MB smaller than before. It seems unlikely that VGA
        disable would require more memory then the other options, so
        this should be ok.

I'm happy to add that while I commit and will Ack + do so tomorrow
unless somebody objects during today.

Any Ack's from the mini-os or QEMU maintainers would still be
appreciated (and would cause me to potentially commit sooner).

Ian.


> 
> Signed-off-by: Eric Shelton <eshelton@pobox.com>
> ---
>  tools/libxl/libxl_dm.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 317a8eb..9a5a937 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -1087,6 +1087,7 @@ void libxl__spawn_stub_dm(libxl__egc *egc, libxl__stub_dm_spawn_state *sdss)
>  
>      dm_config->b_info.max_vcpus = 1;
> -    dm_config->b_info.max_memkb = 32 * 1024;
> +    dm_config->b_info.max_memkb = 28 * 1024 +
> +        guest_config->b_info.video_memkb;
>      dm_config->b_info.target_memkb = dm_config->b_info.max_memkb;
>  
>      dm_config->b_info.u.pv.features = "";

Re: [PATCH V3 1/1] libxl: set stub domain size based on VRAM size

[Ian Campbell]

On Wed, 2015-07-15 at 10:45 +0100, Ian Campbell wrote:
> On Sat, 2015-07-11 at 17:30 -0400, Eric Shelton wrote:
> > Allocate additional memory to the stub domain for qemu-traditional if
> > more than 4 MB is assigned to the video adapter to avoid out of memory
> > condition for QEMU.
> 
> In reply to v2 (after you'd sent v3, I think) Ian J said:
> > Well, it does also reduce the stubdom memory when the video memory is
> > <4Mby.  That sounds plausible to me but at the very least the commit
> > message needs changing, and maybe we should wait a bit to see if one
> > of the qemu experts objects.
> 
> libxl__domain_build_info_setdefault ensures, for qemu-xen-traditional,
> that video_memkb is either precisely 4M (TYPE_CIRRUS) or >= 8M
> (TYPE_STD) but allows a free choice for TYPE_NONE, defaulting to 0 (I'd
> almost expect it to insist on 0, but there you are).
> 
> I suggest adding a second paragraph to the commit message:
> 
>         For LIBXL_VGA_INTERFACE_TYPE_STD + CIRRUS video_memkb is always
>         at least 4MB. However if LIBXL_VGA_INTERFACE_TYPE_STD is
>         selected the video_memkb defaults to 0, resulting in a stubdom
>         which is 4MB smaller than before. It seems unlikely that VGA
>         disable would require more memory then the other options, so
>         this should be ok.
> 
> I'm happy to add that while I commit and will Ack + do so tomorrow
> unless somebody objects during today.

Done.