Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: trondmy@kernel.org
To: linux-nfs@vger.kernel.org
Subject: [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache
Date: Sat,  7 Nov 2020 09:03:25 -0500
Message-ID: <20201107140325.281678-22-trondmy@kernel.org> (raw)
In-Reply-To: <20201107140325.281678-21-trondmy@kernel.org>

From: Trond Myklebust <trond.myklebust@hammerspace.com>

If the directory is changing, causing the page cache to get invalidated
while we are listing the contents, then the NFS client is currently forced
to read in the entire directory contents from scratch, because it needs
to perform a linear search for the readdir cookie. While this is not
an issue for small directories, it does not scale to directories with
millions of entries.
In order to be able to deal with large directories that are changing,
add a heuristic to ensure that if the page cache is empty, and we are
searching for a cookie that is not the zero cookie, we just default to
performing uncached readdir.

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
 fs/nfs/dir.c | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index 238872d116f7..d7a9efd31ecd 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -917,11 +917,28 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc)
 	return res;
 }
 
+static bool nfs_readdir_dont_search_cache(struct nfs_readdir_descriptor *desc)
+{
+	struct address_space *mapping = desc->file->f_mapping;
+	struct inode *dir = file_inode(desc->file);
+	unsigned int dtsize = NFS_SERVER(dir)->dtsize;
+	loff_t size = i_size_read(dir);
+
+	/*
+	 * Default to uncached readdir if the page cache is empty, and
+	 * we're looking for a non-zero cookie in a large directory.
+	 */
+	return desc->dir_cookie != 0 && mapping->nrpages == 0 && size > dtsize;
+}
+
 /* Search for desc->dir_cookie from the beginning of the page cache */
 static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc)
 {
 	int res;
 
+	if (nfs_readdir_dont_search_cache(desc))
+		return -EBADCOOKIE;
+
 	do {
 		if (desc->page_index == 0) {
 			desc->current_index = 0;
-- 
2.28.0


  reply index

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-07 14:03 [PATCH v4 00/21] Readdir enhancements trondmy
2020-11-07 14:03 ` [PATCH v4 01/21] NFS: Remove unnecessary inode locking in nfs_llseek_dir() trondmy
2020-11-07 14:03   ` [PATCH v4 02/21] NFS: Remove unnecessary inode lock in nfs_fsync_dir() trondmy
2020-11-07 14:03     ` [PATCH v4 03/21] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-07 14:03       ` [PATCH v4 04/21] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-07 14:03         ` [PATCH v4 05/21] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-07 14:03           ` [PATCH v4 06/21] NFS: Clean up directory array handling trondmy
2020-11-07 14:03             ` [PATCH v4 07/21] NFS: Don't discard readdir results trondmy
2020-11-07 14:03               ` [PATCH v4 08/21] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-07 14:03                 ` [PATCH v4 09/21] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-07 14:03                   ` [PATCH v4 10/21] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-07 14:03                     ` [PATCH v4 11/21] NFS: Support larger readdir buffers trondmy
2020-11-07 14:03                       ` [PATCH v4 12/21] NFS: More readdir cleanups trondmy
2020-11-07 14:03                         ` [PATCH v4 13/21] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-07 14:03                           ` [PATCH v4 14/21] NFS: Reduce readdir stack usage trondmy
2020-11-07 14:03                             ` [PATCH v4 15/21] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-07 14:03                               ` [PATCH v4 16/21] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-07 14:03                                 ` [PATCH v4 17/21] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-07 14:03                                   ` [PATCH v4 18/21] NFS: Improve handling of directory verifiers trondmy
2020-11-07 14:03                                     ` [PATCH v4 19/21] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-07 14:03                                       ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir trondmy
2020-11-07 14:03                                         ` trondmy [this message]
2020-11-09 21:41                                           ` [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache Benjamin Coddington
2020-11-09 21:46                                             ` Trond Myklebust
2020-11-11 16:43                                               ` Benjamin Coddington
2020-11-11 17:34                                                 ` Trond Myklebust
2020-11-11 19:53                                                   ` Benjamin Coddington
2020-11-11 20:11                                                     ` Trond Myklebust
2020-11-10 14:48                                           ` David Wysochanski
2020-11-10 20:55                                             ` Trond Myklebust
2020-11-09 20:59                                         ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir Benjamin Coddington
2020-11-09 13:15 ` [PATCH v4 00/21] Readdir enhancements David Wysochanski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201107140325.281678-22-trondmy@kernel.org \
    --to=trondmy@kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org
	public-inbox-index linux-nfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git