From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CC54C35E0E for ; Tue, 25 Feb 2020 21:48:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 40BDA21927 for ; Tue, 25 Feb 2020 21:48:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="lRlEn+PG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40BDA21927 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 91C976B000C; Tue, 25 Feb 2020 16:48:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 804116B000E; Tue, 25 Feb 2020 16:48:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F6BC6B0032; Tue, 25 Feb 2020 16:48:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0169.hostedemail.com [216.40.44.169]) by kanga.kvack.org (Postfix) with ESMTP id DF5C76B0008 for ; Tue, 25 Feb 2020 16:48:44 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7FA85180AD80F for ; Tue, 25 Feb 2020 21:48:44 +0000 (UTC) X-FDA: 76529989368.18.glove86_2f2c03ceaf757 X-HE-Tag: glove86_2f2c03ceaf757 X-Filterd-Recvd-Size: 5738 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Tue, 25 Feb 2020 21:48:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=WxTpQNEO+ocYFUkm5ruMMVsYQEz+J+5OgXLMose9K6M=; b=lRlEn+PGibRDdb9/fV8DsEH9N7 0JXKPHBhvPmrsSGSSwmg6/jspJA1XoKcKihA6iGQTQKTNdD7fOi6rO0PLwi0isVQ44/hy2jso6jT/ TfIQtow6ahGpX65C9En8+HvL0tjV3yv78BMPnyr24QevlgZy4zUkm4hTSrSjdRiNBrK3UvatfGOXo +9sipN8TbzR4nWpG4DwRdkCWGiOPF6PSJdJ8JeFfZS1UOiF9bskPSYbVmholzsWtQw/YigjOhiWfT oRgiAmFj8GUICFhPyhe9RRGGrlOzFRe2h0U+sMAs/jeWd6KjXjsv+rdGhH+D1r6YO4Zcg2bNdJEj3 T6mQ7PmQ==; Received: from willy by bombadil.infradead.org with local (Exim 4.92.3 #3 (Red Hat Linux)) id 1j6i4H-0007pr-8J; Tue, 25 Feb 2020 21:48:41 +0000 From: Matthew Wilcox To: linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, ocfs2-devel@oss.oracle.com, linux-xfs@vger.kernel.org, Christoph Hellwig Subject: [PATCH v8 10/25] mm: Put readahead pages in cache earlier Date: Tue, 25 Feb 2020 13:48:23 -0800 Message-Id: <20200225214838.30017-11-willy@infradead.org> X-Mailer: git-send-email 2.21.1 In-Reply-To: <20200225214838.30017-1-willy@infradead.org> References: <20200225214838.30017-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" When populating the page cache for readahead, mappings that use ->readpages must populate the page cache themselves as the pages are passed on a linked list which would normally be used for the page cache's LRU. For mappings that use ->readpage or the upcoming ->readahead method= , we can put the pages into the page cache as soon as they're allocated, which solves a race between readahead and direct IO. It also lets us remove the gfp argument from read_pages(). Use the new readahead_page() API to implement the repeated calls to ->readpage(), just like most filesystems will. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/readahead.c | 46 ++++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ddc63d3b07b8..e52b3a7b9da5 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -114,14 +114,14 @@ int read_cache_pages(struct address_space *mapping,= struct list_head *pages, EXPORT_SYMBOL(read_cache_pages); =20 static void read_pages(struct readahead_control *rac, struct list_head *= pages, - gfp_t gfp) + bool skip_page) { const struct address_space_operations *aops =3D rac->mapping->a_ops; + struct page *page; struct blk_plug plug; - unsigned page_idx; =20 if (!readahead_count(rac)) - return; + goto out; =20 blk_start_plug(&plug); =20 @@ -130,23 +130,23 @@ static void read_pages(struct readahead_control *ra= c, struct list_head *pages, readahead_count(rac)); /* Clean up the remaining pages */ put_pages_list(pages); - goto out; - } - - for (page_idx =3D 0; page_idx < readahead_count(rac); page_idx++) { - struct page *page =3D lru_to_page(pages); - list_del(&page->lru); - if (!add_to_page_cache_lru(page, rac->mapping, page->index, - gfp)) + rac->_index +=3D rac->_nr_pages; + rac->_nr_pages =3D 0; + } else { + while ((page =3D readahead_page(rac))) { aops->readpage(rac->file, page); - put_page(page); + put_page(page); + } } =20 -out: blk_finish_plug(&plug); =20 BUG_ON(!list_empty(pages)); - rac->_nr_pages =3D 0; + BUG_ON(readahead_count(rac)); + +out: + if (skip_page) + rac->_index++; } =20 /* @@ -168,6 +168,7 @@ void __do_page_cache_readahead(struct address_space *= mapping, struct readahead_control rac =3D { .mapping =3D mapping, .file =3D filp, + ._index =3D index, }; unsigned long i; =20 @@ -183,6 +184,8 @@ void __do_page_cache_readahead(struct address_space *= mapping, if (index + i > end_index) break; =20 + BUG_ON(index + i !=3D rac._index + rac._nr_pages); + page =3D xa_load(&mapping->i_pages, index + i); if (page && !xa_is_value(page)) { /* @@ -190,15 +193,22 @@ void __do_page_cache_readahead(struct address_space= *mapping, * contiguous pages before continuing with the next * batch. */ - read_pages(&rac, &page_pool, gfp_mask); + read_pages(&rac, &page_pool, true); continue; } =20 page =3D __page_cache_alloc(gfp_mask); if (!page) break; - page->index =3D index + i; - list_add(&page->lru, &page_pool); + if (mapping->a_ops->readpages) { + page->index =3D index + i; + list_add(&page->lru, &page_pool); + } else if (add_to_page_cache_lru(page, mapping, index + i, + gfp_mask) < 0) { + put_page(page); + read_pages(&rac, &page_pool, true); + continue; + } if (i =3D=3D nr_to_read - lookahead_size) SetPageReadahead(page); rac._nr_pages++; @@ -209,7 +219,7 @@ void __do_page_cache_readahead(struct address_space *= mapping, * uptodate then the caller will launch readpage again, and * will then handle the error. */ - read_pages(&rac, &page_pool, gfp_mask); + read_pages(&rac, &page_pool, false); } =20 /* --=20 2.25.0