From: Seth Jennings <sjenning@linux.vnet.ibm.com> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dave Hansen <dave@sr71.net>, Nathan Fontenot <nfont@linux.vnet.ibm.com>, Cody P Schafer <cody@linux.vnet.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Lai Jiangshan <laijs@cn.fujitsu.com>, "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC][PATCH] drivers: base: dynamic memory block creation Date: Fri, 16 Aug 2013 14:07:37 -0500 [thread overview] Message-ID: <20130816190737.GC7265@variantweb.net> (raw) In-Reply-To: <20130814194043.GA10469@kroah.com> On Wed, Aug 14, 2013 at 12:40:43PM -0700, Greg Kroah-Hartman wrote: > On Wed, Aug 14, 2013 at 02:31:45PM -0500, Seth Jennings wrote: > > Large memory systems (~1TB or more) experience boot delays on the order > > of minutes due to the initializing the memory configuration part of > > sysfs at /sys/devices/system/memory/. > > > > ppc64 has a normal memory block size of 256M (however sometimes as low > > as 16M depending on the system LMB size), and (I think) x86 is 128M. With > > 1TB of RAM and a 256M block size, that's 4k memory blocks with 20 sysfs > > entries per block that's around 80k items that need be created at boot > > time in sysfs. Some systems go up to 16TB where the issue is even more > > severe. > > > > This patch provides a means by which users can prevent the creation of > > the memory block attributes at boot time, yet still dynamically create > > them if they are needed. > > > > This patch creates a new boot parameter, "largememory" that will prevent > > memory_dev_init() from creating all of the memory block sysfs attributes > > at boot time. Instead, a new root attribute "show" will allow > > the dynamic creation of the memory block devices. > > Another new root attribute "present" shows the memory blocks present in > > the system; the valid inputs for the "show" attribute. > > Ick, no new boot parameters please, that's just a mess for distros and > users. Yes, I agreed it isn't the best. The reason for it is backward compatibility; or rather the user saying "I knowingly forfeit backward compatibility in favor of fast boot time and all my userspace tools are aware of the new requirement to show memory blocks before trying to use them". The only suggestion I heard that would make full backward compatibility possible is one from Dave to create a new filesystem for memory blocks (not sysfs) where the memory block directories would be dynamically created as programs tried to access/open them. But you'd still have the issue of requiring user intervention to mount that "memoryfs" at /sys/devices/system/memory (or whatever your sysfs mount point was). So it's tricky. > > How about tying this into the work that has been happening on lkml with > booting large-memory systems faster? The work there should solve the > problems you are seeing here (i.e. add memory after booting). It looks > like this is the same issue you are having here, just in a different > part of the kernel. I assume you are referring to the "[RFC v3 0/5] Transparent on-demand struct page initialization embedded in the buddy allocator" thread. I think that trying to solve a different problem than I am trying to solve though. IIUC, that patch series is deferring the initialization of the actually memory pages. I'm working on breaking out just the refactoring patches (no functional change) into a reviewable patch series. Thanks for your time looking at this! Seth
WARNING: multiple messages have this Message-ID (diff)
From: Seth Jennings <sjenning@linux.vnet.ibm.com> To: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Dave Hansen <dave@sr71.net>, Nathan Fontenot <nfont@linux.vnet.ibm.com>, Cody P Schafer <cody@linux.vnet.ibm.com>, Andrew Morton <akpm@linux-foundation.org>, Lai Jiangshan <laijs@cn.fujitsu.com>, "Rafael J. Wysocki" <rafael.j.wysocki@intel.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC][PATCH] drivers: base: dynamic memory block creation Date: Fri, 16 Aug 2013 14:07:37 -0500 [thread overview] Message-ID: <20130816190737.GC7265@variantweb.net> (raw) In-Reply-To: <20130814194043.GA10469@kroah.com> On Wed, Aug 14, 2013 at 12:40:43PM -0700, Greg Kroah-Hartman wrote: > On Wed, Aug 14, 2013 at 02:31:45PM -0500, Seth Jennings wrote: > > Large memory systems (~1TB or more) experience boot delays on the order > > of minutes due to the initializing the memory configuration part of > > sysfs at /sys/devices/system/memory/. > > > > ppc64 has a normal memory block size of 256M (however sometimes as low > > as 16M depending on the system LMB size), and (I think) x86 is 128M. With > > 1TB of RAM and a 256M block size, that's 4k memory blocks with 20 sysfs > > entries per block that's around 80k items that need be created at boot > > time in sysfs. Some systems go up to 16TB where the issue is even more > > severe. > > > > This patch provides a means by which users can prevent the creation of > > the memory block attributes at boot time, yet still dynamically create > > them if they are needed. > > > > This patch creates a new boot parameter, "largememory" that will prevent > > memory_dev_init() from creating all of the memory block sysfs attributes > > at boot time. Instead, a new root attribute "show" will allow > > the dynamic creation of the memory block devices. > > Another new root attribute "present" shows the memory blocks present in > > the system; the valid inputs for the "show" attribute. > > Ick, no new boot parameters please, that's just a mess for distros and > users. Yes, I agreed it isn't the best. The reason for it is backward compatibility; or rather the user saying "I knowingly forfeit backward compatibility in favor of fast boot time and all my userspace tools are aware of the new requirement to show memory blocks before trying to use them". The only suggestion I heard that would make full backward compatibility possible is one from Dave to create a new filesystem for memory blocks (not sysfs) where the memory block directories would be dynamically created as programs tried to access/open them. But you'd still have the issue of requiring user intervention to mount that "memoryfs" at /sys/devices/system/memory (or whatever your sysfs mount point was). So it's tricky. > > How about tying this into the work that has been happening on lkml with > booting large-memory systems faster? The work there should solve the > problems you are seeing here (i.e. add memory after booting). It looks > like this is the same issue you are having here, just in a different > part of the kernel. I assume you are referring to the "[RFC v3 0/5] Transparent on-demand struct page initialization embedded in the buddy allocator" thread. I think that trying to solve a different problem than I am trying to solve though. IIUC, that patch series is deferring the initialization of the actually memory pages. I'm working on breaking out just the refactoring patches (no functional change) into a reviewable patch series. Thanks for your time looking at this! Seth -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-08-17 0:18 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2013-08-14 19:31 [RFC][PATCH] drivers: base: dynamic memory block creation Seth Jennings 2013-08-14 19:31 ` Seth Jennings 2013-08-14 19:40 ` Greg Kroah-Hartman 2013-08-14 19:40 ` Greg Kroah-Hartman 2013-08-16 19:07 ` Seth Jennings [this message] 2013-08-16 19:07 ` Seth Jennings 2013-08-14 19:43 ` Greg Kroah-Hartman 2013-08-14 19:43 ` Greg Kroah-Hartman 2013-08-14 20:05 ` Dave Hansen 2013-08-14 20:05 ` Dave Hansen 2013-08-14 20:35 ` Greg Kroah-Hartman 2013-08-14 20:35 ` Greg Kroah-Hartman 2013-08-14 21:16 ` Seth Jennings 2013-08-14 21:16 ` Seth Jennings 2013-08-14 21:37 ` Yinghai Lu 2013-08-14 21:52 ` Seth Jennings 2013-08-14 21:52 ` Seth Jennings 2013-08-14 23:20 ` Yinghai Lu 2013-08-14 23:20 ` Yinghai Lu 2013-08-15 2:12 ` Michael Ellerman 2013-08-15 2:12 ` Michael Ellerman 2013-08-15 2:12 ` Michael Ellerman 2013-08-14 20:40 ` Nathan Fontenot 2013-08-14 20:40 ` Nathan Fontenot 2013-08-14 20:47 ` Dave Hansen 2013-08-14 20:47 ` Dave Hansen 2013-08-14 21:14 ` Seth Jennings 2013-08-14 21:14 ` Seth Jennings 2013-08-14 21:36 ` Dave Hansen 2013-08-14 21:36 ` Dave Hansen 2013-08-14 21:37 ` Cody P Schafer 2013-08-14 21:37 ` Cody P Schafer 2013-08-14 21:49 ` Dave Hansen 2013-08-14 21:49 ` Dave Hansen 2013-08-15 0:01 ` Rafael J. Wysocki 2013-08-15 0:01 ` Rafael J. Wysocki 2013-08-16 18:41 ` Seth Jennings 2013-08-16 18:41 ` Seth Jennings
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20130816190737.GC7265@variantweb.net \ --to=sjenning@linux.vnet.ibm.com \ --cc=akpm@linux-foundation.org \ --cc=cody@linux.vnet.ibm.com \ --cc=dave@sr71.net \ --cc=gregkh@linuxfoundation.org \ --cc=laijs@cn.fujitsu.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=nfont@linux.vnet.ibm.com \ --cc=rafael.j.wysocki@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.