Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: trondmy@kernel.org
To: linux-nfs@vger.kernel.org
Subject: [PATCH v4 00/21] Readdir enhancements
Date: Sat,  7 Nov 2020 09:03:04 -0500
Message-ID: <20201107140325.281678-1-trondmy@kernel.org> (raw)

From: Trond Myklebust <trond.myklebust@hammerspace.com>

The following patch series performs a number of cleanups on the readdir
code.
It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
the caching code to ensure that we cache the entire contents of that
1MB call (instead of discarding the data that doesn't fit into a single
page).
For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
optimises searching for cookies in the client's page cache by skipping
over pages that contain cookie values that are not in the range we are
searching for.
Finally, it improves scalability when dealing with very large
directories by turning off caching when those directories are changing,
so as to avoid the need for a linear search on the client of the entire
directory when looking for the first entry pointed to by the current
file descriptor offset.

v2: Fix the handling of the NFSv3/v4 directory verifier.
v3: Optimise searching when the readdir cookies are seen to be ordered.
v4: Optimise performance for large directories that are changing.
    Add in llseek dependency patches.

Trond Myklebust (21):
  NFS: Remove unnecessary inode locking in nfs_llseek_dir()
  NFS: Remove unnecessary inode lock in nfs_fsync_dir()
  NFS: Ensure contents of struct nfs_open_dir_context are consistent
  NFS: Clean up readdir struct nfs_cache_array
  NFS: Clean up nfs_readdir_page_filler()
  NFS: Clean up directory array handling
  NFS: Don't discard readdir results
  NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
  NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
  NFS: Simplify struct nfs_cache_array_entry
  NFS: Support larger readdir buffers
  NFS: More readdir cleanups
  NFS: nfs_do_filldir() does not return a value
  NFS: Reduce readdir stack usage
  NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
  NFS: Allow the NFS generic code to pass in a verifier to readdir
  NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
  NFS: Improve handling of directory verifiers
  NFS: Optimisations for monotonically increasing readdir cookies
  NFS: Reduce number of RPC calls when doing uncached readdir
  NFS: Do uncached readdir when we're seeking a cookie in an empty page
    cache

 fs/nfs/client.c         |   4 +-
 fs/nfs/dir.c            | 702 +++++++++++++++++++++++++---------------
 fs/nfs/inode.c          |   7 -
 fs/nfs/internal.h       |   6 -
 fs/nfs/nfs3proc.c       |  35 +-
 fs/nfs/nfs4proc.c       |  40 +--
 fs/nfs/proc.c           |  18 +-
 include/linux/nfs_fs.h  |   9 +-
 include/linux/nfs_xdr.h |  17 +-
 9 files changed, 508 insertions(+), 330 deletions(-)

-- 
2.28.0


             reply index

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-07 14:03 trondmy [this message]
2020-11-07 14:03 ` [PATCH v4 01/21] NFS: Remove unnecessary inode locking in nfs_llseek_dir() trondmy
2020-11-07 14:03   ` [PATCH v4 02/21] NFS: Remove unnecessary inode lock in nfs_fsync_dir() trondmy
2020-11-07 14:03     ` [PATCH v4 03/21] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-07 14:03       ` [PATCH v4 04/21] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-07 14:03         ` [PATCH v4 05/21] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-07 14:03           ` [PATCH v4 06/21] NFS: Clean up directory array handling trondmy
2020-11-07 14:03             ` [PATCH v4 07/21] NFS: Don't discard readdir results trondmy
2020-11-07 14:03               ` [PATCH v4 08/21] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-07 14:03                 ` [PATCH v4 09/21] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-07 14:03                   ` [PATCH v4 10/21] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-07 14:03                     ` [PATCH v4 11/21] NFS: Support larger readdir buffers trondmy
2020-11-07 14:03                       ` [PATCH v4 12/21] NFS: More readdir cleanups trondmy
2020-11-07 14:03                         ` [PATCH v4 13/21] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-07 14:03                           ` [PATCH v4 14/21] NFS: Reduce readdir stack usage trondmy
2020-11-07 14:03                             ` [PATCH v4 15/21] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-07 14:03                               ` [PATCH v4 16/21] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-07 14:03                                 ` [PATCH v4 17/21] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-07 14:03                                   ` [PATCH v4 18/21] NFS: Improve handling of directory verifiers trondmy
2020-11-07 14:03                                     ` [PATCH v4 19/21] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-07 14:03                                       ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir trondmy
2020-11-07 14:03                                         ` [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache trondmy
2020-11-09 21:41                                           ` Benjamin Coddington
2020-11-09 21:46                                             ` Trond Myklebust
2020-11-11 16:43                                               ` Benjamin Coddington
2020-11-11 17:34                                                 ` Trond Myklebust
2020-11-11 19:53                                                   ` Benjamin Coddington
2020-11-11 20:11                                                     ` Trond Myklebust
2020-11-10 14:48                                           ` David Wysochanski
2020-11-10 20:55                                             ` Trond Myklebust
2020-11-09 20:59                                         ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir Benjamin Coddington
2020-11-09 13:15 ` [PATCH v4 00/21] Readdir enhancements David Wysochanski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201107140325.281678-1-trondmy@kernel.org \
    --to=trondmy@kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org
	public-inbox-index linux-nfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git