Re: [PATCH v4 00/21] Readdir enhancements

[David Wysochanski <dwysocha@redhat.com>]

On Sat, Nov 7, 2020 at 9:13 AM <trondmy@kernel.org> wrote:
>
> From: Trond Myklebust <trond.myklebust@hammerspace.com>
>
> The following patch series performs a number of cleanups on the readdir
> code.
> It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
> the caching code to ensure that we cache the entire contents of that
> 1MB call (instead of discarding the data that doesn't fit into a single
> page).
> For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
> optimises searching for cookies in the client's page cache by skipping
> over pages that contain cookie values that are not in the range we are
> searching for.
> Finally, it improves scalability when dealing with very large
> directories by turning off caching when those directories are changing,
> so as to avoid the need for a linear search on the client of the entire
> directory when looking for the first entry pointed to by the current
> file descriptor offset.
>
> v2: Fix the handling of the NFSv3/v4 directory verifier.
> v3: Optimise searching when the readdir cookies are seen to be ordered.
> v4: Optimise performance for large directories that are changing.
>     Add in llseek dependency patches.
>
> Trond Myklebust (21):
>   NFS: Remove unnecessary inode locking in nfs_llseek_dir()
>   NFS: Remove unnecessary inode lock in nfs_fsync_dir()
>   NFS: Ensure contents of struct nfs_open_dir_context are consistent
>   NFS: Clean up readdir struct nfs_cache_array
>   NFS: Clean up nfs_readdir_page_filler()
>   NFS: Clean up directory array handling
>   NFS: Don't discard readdir results
>   NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
>   NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
>   NFS: Simplify struct nfs_cache_array_entry
>   NFS: Support larger readdir buffers
>   NFS: More readdir cleanups
>   NFS: nfs_do_filldir() does not return a value
>   NFS: Reduce readdir stack usage
>   NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
>   NFS: Allow the NFS generic code to pass in a verifier to readdir
>   NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
>   NFS: Improve handling of directory verifiers
>   NFS: Optimisations for monotonically increasing readdir cookies
>   NFS: Reduce number of RPC calls when doing uncached readdir
>   NFS: Do uncached readdir when we're seeking a cookie in an empty page
>     cache
>

Thanks for these last couple patches to handle changing directories /
uncached scenario better.  I'm testing with this set now and should
have results later today.