From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BC20C388F7 for ; Sat, 7 Nov 2020 14:13:52 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1D226206ED for ; Sat, 7 Nov 2020 14:13:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604758432; bh=X2+lGW7G/4LdYGNIaAOy3OIdR/2FceS3tTfu5QZPVLM=; h=From:To:Subject:Date:In-Reply-To:References:List-ID:From; b=m1ITsMQlKIT1a3SIz8kuU/GJD68wNngRHnxnyCVFBOdlwx34mOPYXXMqJe10p5of0 ohCsqJhRAXW0mVCWtdYFTVFGa4j4q78xOIO4L8kD3G81i2xYyUNPyS02qOmco9JcFW amn4DlbIw5V9eNQBhD8FwQ6dD/3FoTYN9NYs9xKQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728181AbgKGONv (ORCPT ); Sat, 7 Nov 2020 09:13:51 -0500 Received: from mail.kernel.org ([198.145.29.99]:58260 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728144AbgKGONq (ORCPT ); Sat, 7 Nov 2020 09:13:46 -0500 Received: from localhost.localdomain (c-68-36-133-222.hsd1.mi.comcast.net [68.36.133.222]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 963F02087E for ; Sat, 7 Nov 2020 14:13:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604758425; bh=X2+lGW7G/4LdYGNIaAOy3OIdR/2FceS3tTfu5QZPVLM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=1k7Md09odS3jjBoIxghAHHriL785EaRresQVZkcD9eF9apuUWH4x++59tJyC0O8KY hi+dAQrmgDdcVUhwqZLcIqPLofaW0CaybQuUffZay4Msh81rSsHP43RIoGpI2FzCfu LBKb8tzLf9MChKwKqS8ozVQEMNCt/vb5a13OllNU= From: trondmy@kernel.org To: linux-nfs@vger.kernel.org Subject: [PATCH v4 21/21] NFS: Do uncached readdir when we're seeking a cookie in an empty page cache Date: Sat, 7 Nov 2020 09:03:25 -0500 Message-Id: <20201107140325.281678-22-trondmy@kernel.org> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20201107140325.281678-21-trondmy@kernel.org> References: <20201107140325.281678-1-trondmy@kernel.org> <20201107140325.281678-2-trondmy@kernel.org> <20201107140325.281678-3-trondmy@kernel.org> <20201107140325.281678-4-trondmy@kernel.org> <20201107140325.281678-5-trondmy@kernel.org> <20201107140325.281678-6-trondmy@kernel.org> <20201107140325.281678-7-trondmy@kernel.org> <20201107140325.281678-8-trondmy@kernel.org> <20201107140325.281678-9-trondmy@kernel.org> <20201107140325.281678-10-trondmy@kernel.org> <20201107140325.281678-11-trondmy@kernel.org> <20201107140325.281678-12-trondmy@kernel.org> <20201107140325.281678-13-trondmy@kernel.org> <20201107140325.281678-14-trondmy@kernel.org> <20201107140325.281678-15-trondmy@kernel.org> <20201107140325.281678-16-trondmy@kernel.org> <20201107140325.281678-17-trondmy@kernel.org> <20201107140325.281678-18-trondmy@kernel.org> <20201107140325.281678-19-trondmy@kernel.org> <20201107140325.281678-20-trondmy@kernel.org> <20201107140325.281678-21-trondmy@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org From: Trond Myklebust If the directory is changing, causing the page cache to get invalidated while we are listing the contents, then the NFS client is currently forced to read in the entire directory contents from scratch, because it needs to perform a linear search for the readdir cookie. While this is not an issue for small directories, it does not scale to directories with millions of entries. In order to be able to deal with large directories that are changing, add a heuristic to ensure that if the page cache is empty, and we are searching for a cookie that is not the zero cookie, we just default to performing uncached readdir. Signed-off-by: Trond Myklebust --- fs/nfs/dir.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index 238872d116f7..d7a9efd31ecd 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -917,11 +917,28 @@ static int find_and_lock_cache_page(struct nfs_readdir_descriptor *desc) return res; } +static bool nfs_readdir_dont_search_cache(struct nfs_readdir_descriptor *desc) +{ + struct address_space *mapping = desc->file->f_mapping; + struct inode *dir = file_inode(desc->file); + unsigned int dtsize = NFS_SERVER(dir)->dtsize; + loff_t size = i_size_read(dir); + + /* + * Default to uncached readdir if the page cache is empty, and + * we're looking for a non-zero cookie in a large directory. + */ + return desc->dir_cookie != 0 && mapping->nrpages == 0 && size > dtsize; +} + /* Search for desc->dir_cookie from the beginning of the page cache */ static int readdir_search_pagecache(struct nfs_readdir_descriptor *desc) { int res; + if (nfs_readdir_dont_search_cache(desc)) + return -EBADCOOKIE; + do { if (desc->page_index == 0) { desc->current_index = 0; -- 2.28.0