Re: [REPOST PATCH v4 0/5] kernfs: proposed locking and concurrency improvement

From: Fox Chen <foxhlchen@gmail.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Ian Kent <raven@themaw.net>, Tejun Heo <tj@kernel.org>,
	Eric Sandeen <sandeen@sandeen.net>,
	Brice Goglin <brice.goglin@gmail.com>,
	Al Viro <viro@zeniv.linux.org.uk>,
	Rick Lindsley <ricklind@linux.vnet.ibm.com>,
	David Howells <dhowells@redhat.com>,
	Miklos Szeredi <miklos@szeredi.hu>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [REPOST PATCH v4 0/5] kernfs: proposed locking and concurrency improvement
Date: Fri, 28 May 2021 19:56:10 +0800	[thread overview]
Message-ID: <CAC2o3D+GH31Q2oxDgBVu8BfEUXuDovUM7uKJq=uHb3Ay+WjhSQ@mail.gmail.com> (raw)
In-Reply-To: <YLCwIfYxM7jYKQxe@kroah.com>

On Fri, May 28, 2021 at 4:56 PM Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Fri, May 28, 2021 at 02:33:42PM +0800, Ian Kent wrote:
> > There have been a few instances of contention on the kernfs_mutex during
> > path walks, a case on very large IBM systems seen by myself, a report by
> > Brice Goglin and followed up by Fox Chen, and I've since seen a couple
> > of other reports by CoreOS users.
> >
> > The common thread is a large number of kernfs path walks leading to
> > slowness of path walks due to kernfs_mutex contention.
> >
> > The problem being that changes to the VFS over some time have increased
> > it's concurrency capabilities to an extent that kernfs's use of a mutex
> > is no longer appropriate. There's also an issue of walks for non-existent
> > paths causing contention if there are quite a few of them which is a less
> > common problem.
> >
> > This patch series is relatively straight forward.
> >
> > All it does is add the ability to take advantage of VFS negative dentry
> > caching to avoid needless dentry alloc/free cycles for lookups of paths
> > that don't exit and change the kernfs_mutex to a read/write semaphore.
> >
> > The patch that tried to stay in VFS rcu-walk mode during path walks has
> > been dropped for two reasons. First, it doesn't actually give very much
> > improvement and, second, if there's a place where mistakes could go
> > unnoticed it would be in that path. This makes the patch series simpler
> > to review and reduces the likelihood of problems going unnoticed and
> > popping up later.
> >
> > The patch to use a revision to identify if a directory has changed has
> > also been dropped. If the directory has changed the dentry revision
> > needs to be updated to avoid subsequent rb tree searches and after
> > changing to use a read/write semaphore the update also requires a lock.
> > But the d_lock is the only lock available at this point which might
> > itself be contended.
>
> Fox, can you take some time and test these to verify it all still works
> properly with your benchmarks?

Sure, I will take a look.
Actually, I've tested it before, but I will test it again to confirm it.

> thanks,
>
> greg k-h

thanks,
fox