From: Mike Travis <travis@sgi.com>
To: Yinghai Lu <yinghai@kernel.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>,
Greg KH <gregkh@linuxfoundation.org>,
Nathan Zimmer <nzimmer@sgi.com>, Robin Holt <holt@sgi.com>,
Rob Landley <rob@landley.net>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
the arch/x86 maintainers <x86@kernel.org>,
linux-doc@vger.kernel.org,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [RFC 0/2] Delay initializing of large sections of memory
Date: Thu, 27 Jun 2013 08:50:12 -0700 [thread overview]
Message-ID: <51CC5F34.3030705@sgi.com> (raw)
In-Reply-To: <CAE9FiQWzzQeBMRizdL4Ym2AM424yF9+CEZ3vt5Maunb6tfxKyQ@mail.gmail.com>
On 6/26/2013 11:37 PM, Yinghai Lu wrote:
> On Tue, Jun 25, 2013 at 11:58 AM, Mike Travis <travis@sgi.com> wrote:
>> experimenting as soon as I can. Our 32TB system is being
>> brought back to 16TB (we found a number of problems as we
>> get closer and closer to the 64TB limit), but that's still
>> a significant size.
>
> Hi, Mike,
>
> Can you post e820 memory map on system that have 32TiB or more?
>
> Is there one range size more than 16TiB? like [16TiB, 32TiB)...
>
> Thanks
>
> Yinghai
>
Here is (was) the 32T system with 2048 cores with HT disabled:
BIOS-e820: 0000000000000000 - 000000000007f000 (usable)
BIOS-e820: 000000000007f000 - 0000000000080000 (reserved)
BIOS-e820: 0000000000080000 - 00000000000a0000 (usable)
BIOS-e820: 0000000000100000 - 000000007ad54000 (usable)
BIOS-e820: 000000007ad54000 - 000000007ad55000 (reserved)
BIOS-e820: 000000007ad55000 - 000000007ad68000 (usable)
BIOS-e820: 000000007ad68000 - 000000007afa8000 (reserved)
BIOS-e820: 000000007afa8000 - 000000007bcb7000 (usable)
BIOS-e820: 000000007bcb7000 - 000000007bdb7000 (reserved)
BIOS-e820: 000000007bdb7000 - 000000007beb7000 (unusable)
BIOS-e820: 000000007beb7000 - 000000007bfb7000 (reserved)
BIOS-e820: 000000007bfb7000 - 000000007d1b7000 (ACPI NVS)
BIOS-e820: 000000007d1b7000 - 000000007e000000 (ACPI data)
BIOS-e820: 000000007e000000 - 000000007e318000 (usable)
BIOS-e820: 000000007e318000 - 000000007ef49000 (ACPI data)
BIOS-e820: 000000007ef49000 - 000000007f000000 (usable)
BIOS-e820: 0000000100000000 - 0000001e00000000 (usable)
BIOS-e820: 0000002000000000 - 0000003dff000000 (usable)
BIOS-e820: 0000004000000000 - 0000005dff000000 (usable)
BIOS-e820: 0000006000000000 - 0000007dff000000 (usable)
....
BIOS-e820: 00000e0000000000 - 00000e1dff000000 (usable)
BIOS-e820: 00000e2000000000 - 00000e3dff000000 (usable)
BIOS-e820: 00000e4000000000 - 00000e5dff000000 (usable)
BIOS-e820: 00000e6000000000 - 00000e7dff000000 (usable)
Note that on UV, there is some memory reserved at the end
of each node that is used for the directory RAM by the
UV HUB. That is why the ranges are not contiguous. I'd
have to double check with the BIOS guys, but I don't believe
they would have collapsed ranges across nodes even if the
directory RAM did not exist.
-Mike
next prev parent reply other threads:[~2013-06-27 15:50 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-21 16:25 [RFC 0/2] Delay initializing of large sections of memory Nathan Zimmer
2013-06-21 16:25 ` [RFC 1/2] x86_64, mm: Delay initializing large portion " Nathan Zimmer
2013-06-25 4:14 ` Rob Landley
2013-06-21 16:25 ` [RFC 2/2] x86_64, mm: Reinsert the absent memory Nathan Zimmer
2013-06-23 9:28 ` Ingo Molnar
2013-06-23 9:32 ` Ingo Molnar
2013-06-24 17:38 ` H. Peter Anvin
2013-06-24 19:39 ` Ingo Molnar
2013-06-24 20:08 ` H. Peter Anvin
2013-06-25 7:31 ` Ingo Molnar
2013-06-24 20:36 ` Nathan Zimmer
2013-06-25 7:38 ` Ingo Molnar
2013-06-25 15:07 ` H. Peter Anvin
2013-06-25 17:19 ` Mike Travis
2013-06-25 17:22 ` Mike Travis
2013-06-25 18:43 ` H. Peter Anvin
2013-06-25 18:51 ` Mike Travis
2013-06-26 9:22 ` [RFC] Transparent on-demand memory setup initialization embedded in the (GFP) buddy allocator Ingo Molnar
2013-06-26 13:28 ` Andrew Morton
2013-06-26 13:37 ` Ingo Molnar
2013-06-26 15:02 ` Nathan Zimmer
2013-06-26 16:15 ` Mike Travis
2013-06-26 12:14 ` [RFC 2/2] x86_64, mm: Reinsert the absent memory Ingo Molnar
2013-06-26 14:49 ` Nathan Zimmer
2013-06-26 15:12 ` Dave Hansen
2013-06-26 15:20 ` Nathan Zimmer
2013-06-26 15:58 ` Ingo Molnar
2013-06-26 16:11 ` Nathan Zimmer
2013-06-26 16:07 ` Mike Travis
2013-06-21 16:51 ` [RFC 0/2] Delay initializing of large sections of memory Greg KH
2013-06-21 17:03 ` H. Peter Anvin
2013-06-21 17:18 ` Nathan Zimmer
2013-06-21 17:28 ` H. Peter Anvin
2013-06-21 20:05 ` Nathan Zimmer
2013-06-21 20:08 ` H. Peter Anvin
2013-06-21 20:33 ` Nathan Zimmer
2013-06-21 21:36 ` Mike Travis
2013-06-21 21:07 ` Mike Travis
2013-06-21 18:44 ` Yinghai Lu
2013-06-21 18:50 ` Greg KH
2013-06-21 19:10 ` Yinghai Lu
2013-06-21 19:19 ` Nathan Zimmer
2013-06-21 20:28 ` Yinghai Lu
2013-06-21 20:40 ` Nathan Zimmer
2013-06-21 21:30 ` Mike Travis
2013-06-22 0:23 ` Yinghai Lu
2013-06-25 17:35 ` Mike Travis
2013-06-25 18:17 ` H. Peter Anvin
2013-06-25 18:40 ` Mike Travis
2013-06-25 18:40 ` Yinghai Lu
2013-06-25 18:44 ` H. Peter Anvin
2013-06-25 18:58 ` Mike Travis
2013-06-25 19:03 ` Yinghai Lu
2013-06-25 19:09 ` H. Peter Anvin
2013-06-25 19:28 ` Yinghai Lu
2013-06-27 6:37 ` Yinghai Lu
2013-06-27 11:05 ` Robin Holt
2013-06-27 15:50 ` Mike Travis [this message]
2013-06-26 9:23 ` Ingo Molnar
2013-06-25 18:38 ` Yinghai Lu
2013-06-25 18:42 ` Mike Travis
2013-06-21 18:36 ` Yinghai Lu
2013-06-21 18:44 ` Greg Kroah-Hartman
2013-06-21 19:00 ` Yinghai Lu
2013-06-21 21:28 ` Mike Travis
2013-06-21 21:19 ` Mike Travis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=51CC5F34.3030705@sgi.com \
--to=travis@sgi.com \
--cc=akpm@linux-foundation.org \
--cc=gregkh@linuxfoundation.org \
--cc=holt@sgi.com \
--cc=hpa@zytor.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=nzimmer@sgi.com \
--cc=rob@landley.net \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yinghai@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).