From: "Schumaker, Anna" <Anna.Schumaker@netapp.com>
To: "dai.ngo@oracle.com" <dai.ngo@oracle.com>,
"linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
"trondmy@hammerspace.com" <trondmy@hammerspace.com>
Subject: Re: [PATCH RFC] nfs: optimise readdir cache page invalidation
Date: Thu, 23 Jan 2020 18:55:50 +0000 [thread overview]
Message-ID: <44325945a594ba9235393ce6344dd94ab4eaf3e7.camel@netapp.com> (raw)
In-Reply-To: <e04a5d2e2ca57d932346d2372ec45ce32df84231.camel@hammerspace.com>
On Thu, 2020-01-23 at 18:37 +0000, Trond Myklebust wrote:
> NetApp Security WARNING: This is an external email. Do not click links or open
> attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> On Thu, 2020-01-23 at 10:35 -0800, Trond Myklebust wrote:
> > On Wed, 2020-01-22 at 18:48 -0500, Dai Ngo wrote:
> > > When the directory is large and it's being modified by one client
> > > while another client is doing the 'ls -l' on the same directory
> > > then
> > > the cache page invalidation from nfs_force_use_readdirplus causes
> > > the reading client to keep restarting READDIRPLUS from cookie 0
> > > which causes the 'ls -l' to take a very long time to complete,
> > > possibly never completing.
> > >
> > > Currently when nfs_force_use_readdirplus is called to switch from
> > > READDIR to READDIRPLUS, it invalidates all the cached pages of the
> > > directory. This cache page invalidation causes the next nfs_readdir
> > > to re-read the directory content from cookie 0.
> > >
> > > This patch is to optimise the cache invalidation in
> > > nfs_force_use_readdirplus by only truncating the cached pages from
> > > last page index accessed to the end the file. It also marks the
> > > inode to delay invalidating all the cached page of the directory
> > > until the next initial nfs_readdir of the next 'ls' instance.
> > >
> > > Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> >
> > Reviewed-by: Trond Myklebust <trond.myklebust@hammerspace.com>
>
> Sorry I intended this for the PATCHv2... I've looked through that
> version and it looks OK to me.
>
> Anna, please apply the later v2 instead of this one...
I applied the v2 about 5 minutes before you sent this. I'll make sure your
Reviewed-by gets added to it, too :)
Anna
>
> > > ---
> > > fs/nfs/dir.c | 14 +++++++++++++-
> > > include/linux/nfs_fs.h | 3 +++
> > > 2 files changed, 16 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
> > > index e180033e35cf..3fbf2e41d523 100644
> > > --- a/fs/nfs/dir.c
> > > +++ b/fs/nfs/dir.c
> > > @@ -437,7 +437,10 @@ void nfs_force_use_readdirplus(struct inode
> > > *dir)
> > > if (nfs_server_capable(dir, NFS_CAP_READDIRPLUS) &&
> > > !list_empty(&nfsi->open_files)) {
> > > set_bit(NFS_INO_ADVISE_RDPLUS, &nfsi->flags);
> > > - invalidate_mapping_pages(dir->i_mapping, 0, -1);
> > > + nfs_zap_mapping(dir, dir->i_mapping);
> > > + if (NFS_PROTO(dir)->version == 3)
> > > + invalidate_mapping_pages(dir->i_mapping,
> > > + nfsi->page_index + 1, -1);
> > > }
> > > }
> > >
> > > @@ -709,6 +712,9 @@ struct page
> > > *get_cache_page(nfs_readdir_descriptor_t *desc)
> > > int find_cache_page(nfs_readdir_descriptor_t *desc)
> > > {
> > > int res;
> > > + struct inode *inode;
> > > + struct nfs_inode *nfsi;
> > > + struct dentry *dentry;
> > >
> > > desc->page = get_cache_page(desc);
> > > if (IS_ERR(desc->page))
> > > @@ -717,6 +723,12 @@ int find_cache_page(nfs_readdir_descriptor_t
> > > *desc)
> > > res = nfs_readdir_search_array(desc);
> > > if (res != 0)
> > > cache_page_release(desc);
> > > +
> > > + dentry = file_dentry(desc->file);
> > > + inode = d_inode(dentry);
> > > + nfsi = NFS_I(inode);
> > > + nfsi->page_index = desc->page_index;
> > > +
> > > return res;
> > > }
> > >
> > > diff --git a/include/linux/nfs_fs.h b/include/linux/nfs_fs.h
> > > index c06b1fd130f3..a5f8f03ecd59 100644
> > > --- a/include/linux/nfs_fs.h
> > > +++ b/include/linux/nfs_fs.h
> > > @@ -168,6 +168,9 @@ struct nfs_inode {
> > > struct rw_semaphore rmdir_sem;
> > > struct mutex commit_mutex;
> > >
> > > + /* track last access to cached pages */
> > > + unsigned long page_index;
> > > +
> > > #if IS_ENABLED(CONFIG_NFS_V4)
> > > struct nfs4_cached_acl *nfs4_acl;
> > > /* NFSv4 state */
> --
> Trond Myklebust
> CTO, Hammerspace Inc
> 4300 El Camino Real, Suite 105
> Los Altos, CA 94022
> www.hammer.space
>
>
prev parent reply other threads:[~2020-01-23 18:55 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-22 23:48 [PATCH RFC] nfs: optimise readdir cache page invalidation Dai Ngo
2020-01-23 0:10 ` Trond Myklebust
2020-01-23 18:35 ` Trond Myklebust
2020-01-23 18:37 ` Trond Myklebust
2020-01-23 18:55 ` Schumaker, Anna [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=44325945a594ba9235393ce6344dd94ab4eaf3e7.camel@netapp.com \
--to=anna.schumaker@netapp.com \
--cc=dai.ngo@oracle.com \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@hammerspace.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).