linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Wysochanski <dwysocha@redhat.com>
To: trondmy@kernel.org
Cc: linux-nfs <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v4 00/21] Readdir enhancements
Date: Mon, 9 Nov 2020 08:15:39 -0500	[thread overview]
Message-ID: <CALF+zOm7Y8ac7QstbdsmyopaKY4z9507Lff1mHAbQyACRZEX0w@mail.gmail.com> (raw)
In-Reply-To: <20201107140325.281678-1-trondmy@kernel.org>

On Sat, Nov 7, 2020 at 9:13 AM <trondmy@kernel.org> wrote:
>
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
>
> The following patch series performs a number of cleanups on the readdir
> code.
> It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
> the caching code to ensure that we cache the entire contents of that
> 1MB call (instead of discarding the data that doesn't fit into a single
> page).
> For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
> optimises searching for cookies in the client's page cache by skipping
> over pages that contain cookie values that are not in the range we are
> searching for.
> Finally, it improves scalability when dealing with very large
> directories by turning off caching when those directories are changing,
> so as to avoid the need for a linear search on the client of the entire
> directory when looking for the first entry pointed to by the current
> file descriptor offset.
>
> v2: Fix the handling of the NFSv3/v4 directory verifier.
> v3: Optimise searching when the readdir cookies are seen to be ordered.
> v4: Optimise performance for large directories that are changing.
>     Add in llseek dependency patches.
>
> Trond Myklebust (21):
>   NFS: Remove unnecessary inode locking in nfs_llseek_dir()
>   NFS: Remove unnecessary inode lock in nfs_fsync_dir()
>   NFS: Ensure contents of struct nfs_open_dir_context are consistent
>   NFS: Clean up readdir struct nfs_cache_array
>   NFS: Clean up nfs_readdir_page_filler()
>   NFS: Clean up directory array handling
>   NFS: Don't discard readdir results
>   NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
>   NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
>   NFS: Simplify struct nfs_cache_array_entry
>   NFS: Support larger readdir buffers
>   NFS: More readdir cleanups
>   NFS: nfs_do_filldir() does not return a value
>   NFS: Reduce readdir stack usage
>   NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
>   NFS: Allow the NFS generic code to pass in a verifier to readdir
>   NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
>   NFS: Improve handling of directory verifiers
>   NFS: Optimisations for monotonically increasing readdir cookies
>   NFS: Reduce number of RPC calls when doing uncached readdir
>   NFS: Do uncached readdir when we're seeking a cookie in an empty page
>     cache
>

Thanks for these last couple patches to handle changing directories /
uncached scenario better.  I'm testing with this set now and should
have results later today.


      parent reply	other threads:[~2020-11-09 13:26 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-07 14:03 [PATCH v4 00/21] Readdir enhancements trondmy
2020-11-07 14:03 ` [PATCH v4 01/21] NFS: Remove unnecessary inode locking in nfs_llseek_dir() trondmy
2020-11-07 14:03   ` [PATCH v4 02/21] NFS: Remove unnecessary inode lock in nfs_fsync_dir() trondmy
2020-11-07 14:03     ` [PATCH v4 03/21] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-07 14:03       ` [PATCH v4 04/21] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-07 14:03         ` [PATCH v4 05/21] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-07 14:03           ` [PATCH v4 06/21] NFS: Clean up directory array handling trondmy
2020-11-07 14:03             ` [PATCH v4 07/21] NFS: Don't discard readdir results trondmy
2020-11-07 14:03               ` [PATCH v4 08/21] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-07 14:03                 ` [PATCH v4 09/21] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-07 14:03                   ` [PATCH v4 10/21] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-07 14:03                     ` [PATCH v4 11/21] NFS: Support larger readdir buffers trondmy
2020-11-07 14:03                       ` [PATCH v4 12/21] NFS: More readdir cleanups trondmy
2020-11-07 14:03                         ` [PATCH v4 13/21] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-07 14:03                           ` [PATCH v4 14/21] NFS: Reduce readdir stack usage trondmy
2020-11-07 14:03                             ` [PATCH v4 15/21] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-07 14:03                               ` [PATCH v4 16/21] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-07 14:03                                 ` [PATCH v4 17/21] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-07 14:03                                   ` [PATCH v4 18/21] NFS: Improve handling of directory verifiers trondmy
2020-11-07 14:03                                     ` [PATCH v4 19/21] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-07 14:03                                       ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir trondmy
2020-11-07 14:03                                         ` [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache trondmy
2020-11-09 21:41                                           ` Benjamin Coddington
2020-11-09 21:46                                             ` Trond Myklebust
2020-11-11 16:43                                               ` Benjamin Coddington
2020-11-11 17:34                                                 ` Trond Myklebust
2020-11-11 19:53                                                   ` Benjamin Coddington
2020-11-11 20:11                                                     ` Trond Myklebust
2020-11-10 14:48                                           ` David Wysochanski
2020-11-10 20:55                                             ` Trond Myklebust
2020-11-09 20:59                                         ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir Benjamin Coddington
2020-11-09 13:15 ` David Wysochanski [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALF+zOm7Y8ac7QstbdsmyopaKY4z9507Lff1mHAbQyACRZEX0w@mail.gmail.com \
    --to=dwysocha@redhat.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trondmy@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).