From: trondmy@kernel.org
To: linux-nfs@vger.kernel.org
Subject: [PATCH v4 11/21] NFS: Support larger readdir buffers
Date: Sat, 7 Nov 2020 09:03:15 -0500 [thread overview]
Message-ID: <20201107140325.281678-12-trondmy@kernel.org> (raw)
In-Reply-To: <20201107140325.281678-11-trondmy@kernel.org>
From: Trond Myklebust <trond.myklebust@hammerspace.com>
Support readdir buffers of up to 1MB in size so that we can read
large directories using few RPC calls.
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
---
fs/nfs/client.c | 4 ++--
fs/nfs/dir.c | 33 +++++++++++++++++++--------------
fs/nfs/internal.h | 6 ------
3 files changed, 21 insertions(+), 22 deletions(-)
diff --git a/fs/nfs/client.c b/fs/nfs/client.c
index 4b8cc93913f7..f6454ba53d05 100644
--- a/fs/nfs/client.c
+++ b/fs/nfs/client.c
@@ -781,8 +781,8 @@ static void nfs_server_set_fsinfo(struct nfs_server *server,
server->wtmult = nfs_block_bits(fsinfo->wtmult, NULL);
server->dtsize = nfs_block_size(fsinfo->dtpref, NULL);
- if (server->dtsize > PAGE_SIZE * NFS_MAX_READDIR_PAGES)
- server->dtsize = PAGE_SIZE * NFS_MAX_READDIR_PAGES;
+ if (server->dtsize > NFS_MAX_FILE_IO_SIZE)
+ server->dtsize = NFS_MAX_FILE_IO_SIZE;
if (server->dtsize > server->rsize)
server->dtsize = server->rsize;
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index be0e2891fecc..438906dae083 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -727,44 +727,47 @@ static int nfs_readdir_page_filler(struct nfs_readdir_descriptor *desc,
return status;
}
-static
-void nfs_readdir_free_pages(struct page **pages, unsigned int npages)
+static void nfs_readdir_free_pages(struct page **pages, size_t npages)
{
- unsigned int i;
- for (i = 0; i < npages; i++)
- put_page(pages[i]);
+ while (npages--)
+ put_page(pages[npages]);
+ kfree(pages);
}
/*
* nfs_readdir_alloc_pages() will allocate pages that must be freed with a call
* to nfs_readdir_free_pages()
*/
-static
-int nfs_readdir_alloc_pages(struct page **pages, unsigned int npages)
+static struct page **nfs_readdir_alloc_pages(size_t npages)
{
- unsigned int i;
+ struct page **pages;
+ size_t i;
+ pages = kmalloc_array(npages, sizeof(*pages), GFP_KERNEL);
+ if (!pages)
+ return NULL;
for (i = 0; i < npages; i++) {
struct page *page = alloc_page(GFP_KERNEL);
if (page == NULL)
goto out_freepages;
pages[i] = page;
}
- return 0;
+ return pages;
out_freepages:
nfs_readdir_free_pages(pages, i);
- return -ENOMEM;
+ return NULL;
}
static
int nfs_readdir_xdr_to_array(nfs_readdir_descriptor_t *desc, struct page *page, struct inode *inode)
{
- struct page *pages[NFS_MAX_READDIR_PAGES];
+ struct page **pages;
struct nfs_entry entry;
struct file *file = desc->file;
+ size_t array_size;
+ size_t dtsize = NFS_SERVER(inode)->dtsize;
int status = -ENOMEM;
- unsigned int array_size = ARRAY_SIZE(pages);
entry.prev_cookie = 0;
entry.cookie = nfs_readdir_page_last_cookie(page);
@@ -781,9 +784,11 @@ int nfs_readdir_xdr_to_array(nfs_readdir_descriptor_t *desc, struct page *page,
goto out;
}
- status = nfs_readdir_alloc_pages(pages, array_size);
- if (status < 0)
+ array_size = (dtsize + PAGE_SIZE - 1) >> PAGE_SHIFT;
+ pages = nfs_readdir_alloc_pages(array_size);
+ if (!pages)
goto out_release_label;
+
do {
unsigned int pglen;
status = nfs_readdir_xdr_filler(pages, desc, &entry, file, inode);
diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h
index 6673a77884d9..b840d0a91c9d 100644
--- a/fs/nfs/internal.h
+++ b/fs/nfs/internal.h
@@ -56,12 +56,6 @@ static inline bool nfs_lookup_is_soft_revalidate(const struct dentry *dentry)
#define NFS_UNSPEC_RETRANS (UINT_MAX)
#define NFS_UNSPEC_TIMEO (UINT_MAX)
-/*
- * Maximum number of pages that readdir can use for creating
- * a vmapped array of pages.
- */
-#define NFS_MAX_READDIR_PAGES 8
-
struct nfs_client_initdata {
unsigned long init_flags;
const char *hostname; /* Hostname of the server */
--
2.28.0
next prev parent reply other threads:[~2020-11-07 14:13 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-07 14:03 [PATCH v4 00/21] Readdir enhancements trondmy
2020-11-07 14:03 ` [PATCH v4 01/21] NFS: Remove unnecessary inode locking in nfs_llseek_dir() trondmy
2020-11-07 14:03 ` [PATCH v4 02/21] NFS: Remove unnecessary inode lock in nfs_fsync_dir() trondmy
2020-11-07 14:03 ` [PATCH v4 03/21] NFS: Ensure contents of struct nfs_open_dir_context are consistent trondmy
2020-11-07 14:03 ` [PATCH v4 04/21] NFS: Clean up readdir struct nfs_cache_array trondmy
2020-11-07 14:03 ` [PATCH v4 05/21] NFS: Clean up nfs_readdir_page_filler() trondmy
2020-11-07 14:03 ` [PATCH v4 06/21] NFS: Clean up directory array handling trondmy
2020-11-07 14:03 ` [PATCH v4 07/21] NFS: Don't discard readdir results trondmy
2020-11-07 14:03 ` [PATCH v4 08/21] NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array() trondmy
2020-11-07 14:03 ` [PATCH v4 09/21] NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array() trondmy
2020-11-07 14:03 ` [PATCH v4 10/21] NFS: Simplify struct nfs_cache_array_entry trondmy
2020-11-07 14:03 ` trondmy [this message]
2020-11-07 14:03 ` [PATCH v4 12/21] NFS: More readdir cleanups trondmy
2020-11-07 14:03 ` [PATCH v4 13/21] NFS: nfs_do_filldir() does not return a value trondmy
2020-11-07 14:03 ` [PATCH v4 14/21] NFS: Reduce readdir stack usage trondmy
2020-11-07 14:03 ` [PATCH v4 15/21] NFS: Cleanup to remove nfs_readdir_descriptor_t typedef trondmy
2020-11-07 14:03 ` [PATCH v4 16/21] NFS: Allow the NFS generic code to pass in a verifier to readdir trondmy
2020-11-07 14:03 ` [PATCH v4 17/21] NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls trondmy
2020-11-07 14:03 ` [PATCH v4 18/21] NFS: Improve handling of directory verifiers trondmy
2020-11-07 14:03 ` [PATCH v4 19/21] NFS: Optimisations for monotonically increasing readdir cookies trondmy
2020-11-07 14:03 ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir trondmy
2020-11-07 14:03 ` [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache trondmy
2020-11-09 21:41 ` Benjamin Coddington
2020-11-09 21:46 ` Trond Myklebust
2020-11-11 16:43 ` Benjamin Coddington
2020-11-11 17:34 ` Trond Myklebust
2020-11-11 19:53 ` Benjamin Coddington
2020-11-11 20:11 ` Trond Myklebust
2020-11-10 14:48 ` David Wysochanski
2020-11-10 20:55 ` Trond Myklebust
2020-11-09 20:59 ` [PATCH v4 20/21] NFS: Reduce number of RPC calls when doing uncached readdir Benjamin Coddington
2020-11-09 13:15 ` [PATCH v4 00/21] Readdir enhancements David Wysochanski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201107140325.281678-12-trondmy@kernel.org \
--to=trondmy@kernel.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).