From: Ian Kent <raven@themaw.net>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Al Viro <viro@zeniv.linux.org.uk>, Tejun Heo <tj@kernel.org>,
Rick Lindsley <ricklind@linux.vnet.ibm.com>,
Stephen Rothwell <sfr@canb.auug.org.au>,
David Howells <dhowells@redhat.com>,
Miklos Szeredi <miklos@szeredi.hu>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 0/4] kernfs: proposed locking and concurrency improvement
Date: Mon, 25 May 2020 15:23:35 +0800 [thread overview]
Message-ID: <b9e8f8171096813c76df3719719bdda87033fd78.camel@themaw.net> (raw)
In-Reply-To: <20200525061616.GA57080@kroah.com>
On Mon, 2020-05-25 at 08:16 +0200, Greg Kroah-Hartman wrote:
> On Mon, May 25, 2020 at 01:46:59PM +0800, Ian Kent wrote:
> > For very large systems with hundreds of CPUs and TBs of RAM booting
> > can
> > take a very long time.
> >
> > Initial reports showed that booting a configuration of several
> > hundred
> > CPUs and 64TB of RAM would take more than 30 minutes and require
> > kernel
> > parameters of udev.children-max=1024
> > systemd.default_timeout_start_sec=3600
> > to prevent dropping into emergency mode.
> >
> > Gathering information about what's happening during the boot is a
> > bit
> > challenging. But two main issues appeared to be, a large number of
> > path
> > lookups for non-existent files, and high lock contention in the VFS
> > during
> > path walks particularly in the dentry allocation code path.
> >
> > The underlying cause of this was believed to be the sheer number of
> > sysfs
> > memory objects, 100,000+ for a 64TB memory configuration.
>
> Independant of your kernfs changes, why do we really need to
> represent
> all of this memory with that many different "memory objects"? What
> is
> that providing to userspace?
>
> I remember Ben Herrenschmidt did a lot of work on some of the kernfs
> and
> other functions to make large-memory systems boot faster to remove
> some
> of the complexity in our functions, but that too did not look into
> why
> we needed to create so many objects in the first place.
>
> Perhaps you might want to look there instead?
I presumed it was a hardware design requirement or IBM VM design
requirement.
Perhaps Rick can find out more on that question.
Ian
next prev parent reply other threads:[~2020-05-25 7:23 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-25 5:46 [PATCH 0/4] kernfs: proposed locking and concurrency improvement Ian Kent
2020-05-25 5:47 ` [PATCH 1/4] kernfs: switch kernfs to use an rwsem Ian Kent
2020-06-06 15:52 ` [kernfs] ea7c5fc39a: stress-ng.stream.ops_per_sec 11827.2% improvement kernel test robot
2020-06-06 15:52 ` kernel test robot
2020-06-06 18:18 ` Greg Kroah-Hartman
2020-06-06 18:18 ` Greg Kroah-Hartman
2020-06-07 1:13 ` Ian Kent
2020-06-11 2:06 ` kernel test robot
2020-06-11 2:06 ` kernel test robot
2020-06-11 2:20 ` Rick Lindsley
2020-06-11 2:20 ` Rick Lindsley
2020-06-11 3:02 ` Ian Kent
2020-06-11 3:02 ` Ian Kent
2020-06-07 8:40 ` [PATCH 1/4] kernfs: switch kernfs to use an rwsem Ian Kent
2020-06-08 9:58 ` Ian Kent
2020-05-25 5:47 ` [PATCH 2/4] kernfs: move revalidate to be near lookup Ian Kent
2020-05-25 5:47 ` [PATCH 3/4] kernfs: improve kernfs path resolution Ian Kent
2020-05-25 5:47 ` [PATCH 4/4] kernfs: use revision to identify directory node changes Ian Kent
2020-05-25 6:16 ` [PATCH 0/4] kernfs: proposed locking and concurrency improvement Greg Kroah-Hartman
2020-05-25 7:23 ` Ian Kent [this message]
2020-05-25 7:31 ` Greg Kroah-Hartman
2020-05-27 12:44 ` Rick Lindsley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b9e8f8171096813c76df3719719bdda87033fd78.camel@themaw.net \
--to=raven@themaw.net \
--cc=akpm@linux-foundation.org \
--cc=dhowells@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=ricklind@linux.vnet.ibm.com \
--cc=sfr@canb.auug.org.au \
--cc=tj@kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.