From: Sarah Newman <srn@prgmr.com>
To: Andrew Cooper <andrew.cooper3@citrix.com>,
Jan Beulich <JBeulich@suse.com>
Cc: xen-devel@lists.xen.org
Subject: Re: Can't always start 32 bit domains after 64 bit domains
Date: Mon, 21 Nov 2016 13:06:46 -0800 [thread overview]
Message-ID: <dd9a4dba-a4ea-9edf-491f-30f68ca94636@prgmr.com> (raw)
In-Reply-To: <fd1dec0f-cb92-54a6-e7d2-db6051edb187@prgmr.com>
On 11/21/2016 11:37 AM, Sarah Newman wrote:
> On 11/21/2016 05:21 AM, Andrew Cooper wrote:
>> On 21/11/16 10:05, Jan Beulich wrote:
>
>>>>> Back in the xend days someone here had invented a (crude) mechanism
>>>>> to set aside memory for 32-bit PV domains, but I don't think dealing with
>>>>> this situation in xl has ever seen any interest.
>>>> If I wanted to add that, where would it go?
>>> I don't know, that's a question for xl folks I guess.
>>
>> IIRC, given no other constraints, the Xen heap allocation allocates from
>> the top down to help this exact case.
>
> Yes, that's what I thought it did too, though I can't find my source information for that.
>
>>
>> I suspect that libxl's preference towards NUMA allocation of domains
>> interferes with this, by adding a NUMA constraints to memory allocations
>> for 64bit PV guests.
>
> I ran xl info -n (which I didn't know about before) and that shows the problem much more clearly.
>
> If that's the reason not all the higher memory is being used first: is a potential workaround to pin 64 bit domains to the second physical core on
> boot, and 32 bit domains to the first physical core on boot, and then change the allowed cores with 'xl vcpu-pin' after the domain is loaded?
Free memory on a test server with no domains:
node: memsize memfree distances
0: 148480 142983 10,21
1: 147456 144645 21,10
Free memory booting 116 256M 64-bit domains, limited to cpus='all,^0-1' on boot:
node: memsize memfree distances
0: 148480 128416 10,21
1: 147456 129669 21,10
Free memory booting 116 256M 64-bit domains, limited to cpus='12-23' on boot:
node: memsize memfree distances
0: 148480 143397 10,21
1: 147456 114693 21,10
This looks like a viable workaround. Where should I document it?
--Sarah
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-11-21 21:06 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <3423fa13-18bd-ff7b-f44a-af015eda2eb7@prgmr.com>
2016-11-19 21:22 ` Can't always start 32 bit domains after 64 bit domains Sarah Newman
2016-11-21 8:20 ` Jan Beulich
2016-11-21 9:45 ` Sarah Newman
2016-11-21 10:05 ` Jan Beulich
2016-11-21 13:21 ` Andrew Cooper
2016-11-21 19:37 ` Sarah Newman
2016-11-21 21:06 ` Sarah Newman [this message]
2016-11-22 18:46 ` Dario Faggioli
2016-11-22 19:37 ` Sarah Newman
2016-11-22 21:40 ` Dario Faggioli
2016-11-27 1:14 ` Dario Faggioli
2016-11-27 2:46 ` Sarah Newman
2016-11-22 18:37 ` Dario Faggioli
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dd9a4dba-a4ea-9edf-491f-30f68ca94636@prgmr.com \
--to=srn@prgmr.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.