All of lore.kernel.org
 help / color / mirror / Atom feed
From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
To: Seth Jennings <sjenning@linux.vnet.ibm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Dave Hansen <dave@sr71.net>,
	Cody P Schafer <cody@linux.vnet.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Lai Jiangshan <laijs@cn.fujitsu.com>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [RFC][PATCH] drivers: base: dynamic memory block creation
Date: Wed, 14 Aug 2013 15:40:17 -0500	[thread overview]
Message-ID: <520BEB31.6090103@linux.vnet.ibm.com> (raw)
In-Reply-To: <1376508705-3188-1-git-send-email-sjenning@linux.vnet.ibm.com>

On 08/14/2013 02:31 PM, Seth Jennings wrote:
> Large memory systems (~1TB or more) experience boot delays on the order
> of minutes due to the initializing the memory configuration part of
> sysfs at /sys/devices/system/memory/.

With the previous work that has been done in the memory sysfs layout
I think you need machines with 8 or 16+ TB of memory to see boot delays
that are measured in minutes. The boot delay is there, and with larger
memory systems in he future it will only get worse.

> 
> ppc64 has a normal memory block size of 256M (however sometimes as low
> as 16M depending on the system LMB size), and (I think) x86 is 128M.  With
> 1TB of RAM and a 256M block size, that's 4k memory blocks with 20 sysfs
> entries per block that's around 80k items that need be created at boot
> time in sysfs.  Some systems go up to 16TB where the issue is even more
> severe.
> 

It should also be pointed out that the number of sysfs entries created on
16+ TB system is 100k+. At his scale it is not really human readable to
list all of the entries. The amount of resources used to create all of the
uderlying structures for each of the entries starts to add up also.

I think an approach such as this makes the sysfs memory layout more
human readable and saves on resources.

-Nathan


WARNING: multiple messages have this Message-ID (diff)
From: Nathan Fontenot <nfont@linux.vnet.ibm.com>
To: Seth Jennings <sjenning@linux.vnet.ibm.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Dave Hansen <dave@sr71.net>,
	Cody P Schafer <cody@linux.vnet.ibm.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Lai Jiangshan <laijs@cn.fujitsu.com>,
	"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [RFC][PATCH] drivers: base: dynamic memory block creation
Date: Wed, 14 Aug 2013 15:40:17 -0500	[thread overview]
Message-ID: <520BEB31.6090103@linux.vnet.ibm.com> (raw)
In-Reply-To: <1376508705-3188-1-git-send-email-sjenning@linux.vnet.ibm.com>

On 08/14/2013 02:31 PM, Seth Jennings wrote:
> Large memory systems (~1TB or more) experience boot delays on the order
> of minutes due to the initializing the memory configuration part of
> sysfs at /sys/devices/system/memory/.

With the previous work that has been done in the memory sysfs layout
I think you need machines with 8 or 16+ TB of memory to see boot delays
that are measured in minutes. The boot delay is there, and with larger
memory systems in he future it will only get worse.

> 
> ppc64 has a normal memory block size of 256M (however sometimes as low
> as 16M depending on the system LMB size), and (I think) x86 is 128M.  With
> 1TB of RAM and a 256M block size, that's 4k memory blocks with 20 sysfs
> entries per block that's around 80k items that need be created at boot
> time in sysfs.  Some systems go up to 16TB where the issue is even more
> severe.
> 

It should also be pointed out that the number of sysfs entries created on
16+ TB system is 100k+. At his scale it is not really human readable to
list all of the entries. The amount of resources used to create all of the
uderlying structures for each of the entries starts to add up also.

I think an approach such as this makes the sysfs memory layout more
human readable and saves on resources.

-Nathan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2013-08-14 20:40 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-14 19:31 [RFC][PATCH] drivers: base: dynamic memory block creation Seth Jennings
2013-08-14 19:31 ` Seth Jennings
2013-08-14 19:40 ` Greg Kroah-Hartman
2013-08-14 19:40   ` Greg Kroah-Hartman
2013-08-16 19:07   ` Seth Jennings
2013-08-16 19:07     ` Seth Jennings
2013-08-14 19:43 ` Greg Kroah-Hartman
2013-08-14 19:43   ` Greg Kroah-Hartman
2013-08-14 20:05   ` Dave Hansen
2013-08-14 20:05     ` Dave Hansen
2013-08-14 20:35     ` Greg Kroah-Hartman
2013-08-14 20:35       ` Greg Kroah-Hartman
2013-08-14 21:16       ` Seth Jennings
2013-08-14 21:16         ` Seth Jennings
2013-08-14 21:37       ` Yinghai Lu
2013-08-14 21:52         ` Seth Jennings
2013-08-14 21:52           ` Seth Jennings
2013-08-14 23:20           ` Yinghai Lu
2013-08-14 23:20             ` Yinghai Lu
2013-08-15  2:12           ` Michael Ellerman
2013-08-15  2:12             ` Michael Ellerman
2013-08-15  2:12             ` Michael Ellerman
2013-08-14 20:40 ` Nathan Fontenot [this message]
2013-08-14 20:40   ` Nathan Fontenot
2013-08-14 20:47 ` Dave Hansen
2013-08-14 20:47   ` Dave Hansen
2013-08-14 21:14   ` Seth Jennings
2013-08-14 21:14     ` Seth Jennings
2013-08-14 21:36     ` Dave Hansen
2013-08-14 21:36       ` Dave Hansen
2013-08-14 21:37     ` Cody P Schafer
2013-08-14 21:37       ` Cody P Schafer
2013-08-14 21:49       ` Dave Hansen
2013-08-14 21:49         ` Dave Hansen
2013-08-15  0:01 ` Rafael J. Wysocki
2013-08-15  0:01   ` Rafael J. Wysocki
2013-08-16 18:41   ` Seth Jennings
2013-08-16 18:41     ` Seth Jennings

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=520BEB31.6090103@linux.vnet.ibm.com \
    --to=nfont@linux.vnet.ibm.com \
    --cc=akpm@linux-foundation.org \
    --cc=cody@linux.vnet.ibm.com \
    --cc=dave@sr71.net \
    --cc=gregkh@linuxfoundation.org \
    --cc=laijs@cn.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rafael.j.wysocki@intel.com \
    --cc=sjenning@linux.vnet.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.