How to request contiguous memory for a domu

How to request contiguous memory for a domu

[Fan, Huaxiang]

[-- Attachment #1.1: Type: text/plain, Size: 1216 bytes --]

Hi gurus,

I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0 occupy all available physical memory (except those used by xen hypervisor), then /etc/init.d/xendomains try to boot domu specified in /etc/xen/auto, then dom0 balloon out to free a portion of memory (of course the resulting memory should be above dom0_min_mem) to meet the memory requirement of incoming domu.

My question is, is there a guarantee that the domu get contiguous physical memory? Or how can I guarantee that? (My memory capacity is big enough for meeting all domus and dom0 memory requirements.)

What about xen 4.1.0?

Thanks in advance.
HUAXIANG FAN
Software Engineer II

WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD.
ph: +8610.5884.4327
fax: +8610.5884.4727
www.websense.cn<http://www.websense.cn>

Websense TRITON(tm)
For Essential Information Protection(tm)
Web Security<http://www.websense.com/content/Regional/SCH/WebSecurityOverview.aspx> | Data Security<http://www.websense.com/content/Regional/SCH/DataSecurity.aspx> | Email Security<http://www.websense.com/content/Regional/SCH/MessagingSecurity.aspx>



 Protected by Websense Hosted Email Security -- www.websense.com 

[-- Attachment #1.2: Type: text/html, Size: 4561 bytes --]

[-- Attachment #2: Type: text/plain, Size: 137 bytes --]

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Re: How to request contiguous memory for a domu

[Ian Campbell]

On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote:
> Hi gurus,
> 
>  
> 
> I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0
> occupy all available physical memory (except those used by xen
> hypervisor), then /etc/init.d/xendomains try to boot domu specified
> in /etc/xen/auto, then dom0 balloon out to free a portion of memory
> (of course the resulting memory should be above dom0_min_mem) to meet
> the memory requirement of incoming domu.
> 
>  
> 
> My question is, is there a guarantee that the domu get contiguous
> physical memory?

No

>  Or how can I guarantee that?

A guest can try to exchange memory for contiguous memory using
XENMEM_exchange but there are limits on how bug a single contiguous
region can be requested that way (I think it's around order 9 or so).

Why do you want domU memory to be contiguous? There is no general
requirement on dom0 or domU for this to be the case and the p2m/m2p
mechanisms exist to allow non-contiguous machine memory. In general
guests only need to arrange for contiguous memory for specific DMA
buffers to specific physical devices which require it.

Please explain what you are actually trying to achieve so we can best
advise you.

> (My memory capacity is big enough for meeting all domus and dom0
> memory requirements.)
> 
>  
> 
> What about xen 4.1.0?
> 
>  
> 
> Thanks in advance.
> 
> HUAXIANG FAN
> Software Engineer II
> 
> WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD.
> ph: +8610.5884.4327
> fax: +8610.5884.4727
> www.websense.cn
> 
> Websense TRITON™
> For Essential Information Protection™
> Web Security | Data Security | Email Security
> 
> 
> 
> 
> 
> 
>     Protected by Websense Hosted Email Security — www.websense.com
> 

RE: How to request contiguous memory for a domu

[Fan, Huaxiang]

[-- Attachment #1: Type: text/plain, Size: 2743 bytes --]

Hi Ian,

Thanks for your quick reply.

We are running cpu/memory intensive application/services on a domu, we are worry about non-contiguous memory allocation might affect the performance. 

We are migrating the application/services from a single CPU socket (4 cores, i.e. 8 ht cores) hardware platform to a dual CPU socket (8 cores, i.e., 16 ht cores) hardware platform. And we saw performance degradation. As you know each physical and a half memory constitute a node. We also worry about the domu memory allocation across nodes boundary.

Thanks,
Huaxiang

-----Original Message-----
From: Ian Campbell [mailto:Ian.Campbell@citrix.com] 
Sent: Thursday, July 21, 2011 4:45 PM
To: Fan, Huaxiang
Cc: xen-users@lists.xensource.com; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] How to request contiguous memory for a domu

On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote:
> Hi gurus,
> 
>  
> 
> I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0
> occupy all available physical memory (except those used by xen
> hypervisor), then /etc/init.d/xendomains try to boot domu specified
> in /etc/xen/auto, then dom0 balloon out to free a portion of memory
> (of course the resulting memory should be above dom0_min_mem) to meet
> the memory requirement of incoming domu.
> 
>  
> 
> My question is, is there a guarantee that the domu get contiguous
> physical memory?

No

>  Or how can I guarantee that?

A guest can try to exchange memory for contiguous memory using
XENMEM_exchange but there are limits on how bug a single contiguous
region can be requested that way (I think it's around order 9 or so).

Why do you want domU memory to be contiguous? There is no general
requirement on dom0 or domU for this to be the case and the p2m/m2p
mechanisms exist to allow non-contiguous machine memory. In general
guests only need to arrange for contiguous memory for specific DMA
buffers to specific physical devices which require it.

Please explain what you are actually trying to achieve so we can best
advise you.

> (My memory capacity is big enough for meeting all domus and dom0
> memory requirements.)
> 
>  
> 
> What about xen 4.1.0?
> 
>  
> 
> Thanks in advance.
> 
> HUAXIANG FAN
> Software Engineer II
> 
> WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD.
> ph: +8610.5884.4327
> fax: +8610.5884.4727
> www.websense.cn
> 
> Websense TRITON™
> For Essential Information Protection™
> Web Security | Data Security | Email Security
> 
> 
> 
> 
> 
> 
>     Protected by Websense Hosted Email Security — www.websense.com
> 




 To report this as spam, please forward to spam@websense.com.  Thank you.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: How to request contiguous memory for a domu

[Ian Campbell]

Please read http://wiki.xensource.com/xenwiki/AskingXenDevelQuestions
and do not top post and do not cross post between xen-users and
xen-devel.

On Thu, 2011-07-21 at 09:57 +0100, Fan, Huaxiang wrote:
> Hi Ian,
> 
> Thanks for your quick reply.
> 
> We are running cpu/memory intensive application/services on a domu, we
> are worry about non-contiguous memory allocation might affect the
> performance. 
> 
> We are migrating the application/services from a single CPU socket (4
> cores, i.e. 8 ht cores) hardware platform to a dual CPU socket (8
> cores, i.e., 16 ht cores) hardware platform. And we saw performance
> degradation. As you know each physical and a half memory constitute a
> node. We also worry about the domu memory allocation across nodes
> boundary.

Does turning on Xen's NUMA support help at all? And then possibly
pinning each guests VCPUs to a particular physical node. IIRC xl has a
cpupool command which sets up a cpupool per node which might make this a
bit easier. I must admit though that I'm not too familiar with Xen NUMA
capabilities, hopefully someone else on the list can advise further.

Ian.

> 
> Thanks,
> Huaxiang
> 
> -----Original Message-----
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com] 
> Sent: Thursday, July 21, 2011 4:45 PM
> To: Fan, Huaxiang
> Cc: xen-users@lists.xensource.com; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] How to request contiguous memory for a domu
> 
> On Thu, 2011-07-21 at 09:28 +0100, Fan, Huaxiang wrote:
> > Hi gurus,
> > 
> >  
> > 
> > I used xen 3.4.2. I noticed that, every time I reboot my box, the dom0
> > occupy all available physical memory (except those used by xen
> > hypervisor), then /etc/init.d/xendomains try to boot domu specified
> > in /etc/xen/auto, then dom0 balloon out to free a portion of memory
> > (of course the resulting memory should be above dom0_min_mem) to meet
> > the memory requirement of incoming domu.
> > 
> >  
> > 
> > My question is, is there a guarantee that the domu get contiguous
> > physical memory?
> 
> No
> 
> >  Or how can I guarantee that?
> 
> A guest can try to exchange memory for contiguous memory using
> XENMEM_exchange but there are limits on how bug a single contiguous
> region can be requested that way (I think it's around order 9 or so).
> 
> Why do you want domU memory to be contiguous? There is no general
> requirement on dom0 or domU for this to be the case and the p2m/m2p
> mechanisms exist to allow non-contiguous machine memory. In general
> guests only need to arrange for contiguous memory for specific DMA
> buffers to specific physical devices which require it.
> 
> Please explain what you are actually trying to achieve so we can best
> advise you.
> 
> > (My memory capacity is big enough for meeting all domus and dom0
> > memory requirements.)
> > 
> >  
> > 
> > What about xen 4.1.0?
> > 
> >  
> > 
> > Thanks in advance.
> > 
> > HUAXIANG FAN
> > Software Engineer II
> > 
> > WEBSENSE NETWORK SECURITY TECHNOLOGY R&D (BEIJING) CO. LTD.
> > ph: +8610.5884.4327
> > fax: +8610.5884.4727
> > www.websense.cn
> > 
> > Websense TRITON™
> > For Essential Information Protection™
> > Web Security | Data Security | Email Security
> > 
> > 
> > 
> > 
> > 
> > 
> >     Protected by Websense Hosted Email Security — www.websense.com
> > 
> 
> 
> 
> 
>  To report this as spam, please forward to spam@websense.com.  Thank you.