Re: [RFC 0/2] Delay initializing of large sections of memory

From: Yinghai Lu <yinghai@kernel.org>
To: Mike Travis <travis@sgi.com>
Cc: Greg KH <gregkh@linuxfoundation.org>,
	"H. Peter Anvin" <hpa@zytor.com>, Nathan Zimmer <nzimmer@sgi.com>,
	Robin Holt <holt@sgi.com>, Rob Landley <rob@landley.net>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	"the arch/x86 maintainers" <x86@kernel.org>,
	linux-doc@vger.kernel.org,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [RFC 0/2] Delay initializing of large sections of memory
Date: Fri, 21 Jun 2013 17:23:06 -0700	[thread overview]
Message-ID: <CAE9FiQXaUNbK9+g-7j6gfvLrSvUJFWWKapDv658LPe4VfbH=tQ@mail.gmail.com> (raw)
In-Reply-To: <51C4C5F3.3050800@sgi.com>

On Fri, Jun 21, 2013 at 2:30 PM, Mike Travis <travis@sgi.com> wrote:
>
>
> On 6/21/2013 11:50 AM, Greg KH wrote:
>> On Fri, Jun 21, 2013 at 11:44:22AM -0700, Yinghai Lu wrote:
>>> On Fri, Jun 21, 2013 at 10:03 AM, H. Peter Anvin <hpa@zytor.com> wrote:
>>>> On 06/21/2013 09:51 AM, Greg KH wrote:
>>>>
>>>> I suspect the cutoff for this should be a lot lower than 8 TB even, more
>>>> like 128 GB or so.  The only concern is to not set the cutoff so low
>>>> that we can end up running out of memory or with suboptimal NUMA
>>>> placement just because of this.
>>>
>>> I would suggest another way:
>>> only boot the system with boot node (include cpu, ram and pci root buses).
>>> then after boot, could add other nodes.
>>
>> What exactly do you mean by "after boot"?  Often, the boot process of
>> userspace needs those additional cpus and ram in order to initialize
>> everything (like the pci devices) properly.
>
> Exactly.  That's why I left both low and high memory on each node.

looks like you assume every node have same ram, and before booting you
you need to know memory layout to append the boot command line.

We have patchset that moving srat table parse early.
git://git.kernel.org/pub/scm/linux/kernel/git/yinghai/linux-yinghai.git
for-x86-mm
https://git.kernel.org/cgit/linux/kernel/git/yinghai/linux-yinghai.git/log/?h=for-x86-mm

on top that, we could make your patch pass more simple command like
1/2^n of every node, and only need to pass n instead.

Thanks

Yinghai