Re: [PATCH v5 3/6] kernfs: use VFS negative dentry caching

From: Ian Kent <raven@themaw.net>
To: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Tejun Heo <tj@kernel.org>, Eric Sandeen <sandeen@sandeen.net>,
	Fox Chen <foxhlchen@gmail.com>,
	Brice Goglin <brice.goglin@gmail.com>,
	Al Viro <viro@ZenIV.linux.org.uk>,
	Rick Lindsley <ricklind@linux.vnet.ibm.com>,
	David Howells <dhowells@redhat.com>,
	Miklos Szeredi <miklos@szeredi.hu>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Carlos Maiolino <cmaiolino@redhat.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v5 3/6] kernfs: use VFS negative dentry caching
Date: Tue, 08 Jun 2021 09:56:01 +0800	[thread overview]
Message-ID: <5e34e74c7c6d6b58165702824b8b0ad914a6a5b9.camel@themaw.net> (raw)
In-Reply-To: <87lf7lil7y.fsf@disp2133>

On Mon, 2021-06-07 at 13:27 -0500, Eric W. Biederman wrote:
> Ian Kent <raven@themaw.net> writes:
> 
> > If there are many lookups for non-existent paths these negative
> > lookups
> > can lead to a lot of overhead during path walks.
> > 
> > The VFS allows dentries to be created as negative and hashed, and
> > caches
> > them so they can be used to reduce the fairly high overhead
> > alloc/free
> > cycle that occurs during these lookups.
> > 
> > Use the kernfs node parent revision to identify if a change has
> > been
> > made to the containing directory so that the negative dentry can be
> > discarded and the lookup redone.
> > 
> > Signed-off-by: Ian Kent <raven@themaw.net>
> > ---
> >  fs/kernfs/dir.c |   53 +++++++++++++++++++++++++++++++------------
> > ----------
> >  1 file changed, 31 insertions(+), 22 deletions(-)
> > 
> > diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
> > index b88432c48851f..5ae95e8d1aea1 100644
> > --- a/fs/kernfs/dir.c
> > +++ b/fs/kernfs/dir.c
> > @@ -1039,13 +1039,32 @@ static int kernfs_dop_revalidate(struct
> > dentry *dentry, unsigned int flags)
> >         if (flags & LOOKUP_RCU)
> >                 return -ECHILD;
> >  
> > -       /* Always perform fresh lookup for negatives */
> > -       if (d_really_is_negative(dentry))
> > -               goto out_bad_unlocked;
> > -
> >         kn = kernfs_dentry_node(dentry);
> >         mutex_lock(&kernfs_mutex);
> >  
> > +       /* Negative hashed dentry? */
> > +       if (!kn) {
> > +               struct dentry *d_parent = dget_parent(dentry);
> > +               struct kernfs_node *parent;
> > +
> > +               /* If the kernfs parent node has changed discard
> > and
> > +                * proceed to ->lookup.
> > +                */
> > +               parent = kernfs_dentry_node(d_parent);
> > +               if (parent) {
> > +                       if (kernfs_dir_changed(parent, dentry)) {
> > +                               dput(d_parent);
> > +                               goto out_bad;
> > +                       }
> > +               }
> > +               dput(d_parent);
> > +
> > +               /* The kernfs node doesn't exist, leave the dentry
> > +                * negative and return success.
> > +                */
> > +               goto out;
> > +       }
> 
> What part of this new negative hashed dentry check needs the
> kernfs_mutex?
> 
> I guess it is the reading of kn->dir.rev.

I have an irresistible urge to keep the rb tree stable when
accessing it. It was probably not necessary most of the times
I did it, IIUC even a rebalance will leave the node address
unchanged so it should be just removals and moves to worry
about.

> 
> Since all you are doing is comparing if two fields are equal it
> really should not matter.  Maybe somewhere there needs to be a
> sprinkling of primitives like READ_ONCE.

There is one case that looks tricky, rename will call ->rename()
and a bit later do the move. Thinking about it a READ_ONCE might
be needed even now but taking the rwsem is probably enough.

Not sure about that one?

Moving this out from under the rwsem would be good to do.

Ian
> 
> It just seems like such a waste to put all of that under kernfs_mutex
> on the off chance kn->dir.rev will change while it is being read.
> 
> Eric