From: Matthew Wilcox <willy@infradead.org> To: Andrew Morton <akpm@linux-foundation.org> Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-btrfs@vger.kernel.org, linux-erofs@lists.ozlabs.org, linux-ext4@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, ocfs2-devel@oss.oracle.com, linux-xfs@vger.kernel.org, Christoph Hellwig <hch@lst.de>, William Kucharski <william.kucharski@oracle.com> Subject: [PATCH v11 10/25] mm: Put readahead pages in cache earlier Date: Tue, 14 Apr 2020 08:02:18 -0700 Message-ID: <20200414150233.24495-11-willy@infradead.org> (raw) In-Reply-To: <20200414150233.24495-1-willy@infradead.org> From: "Matthew Wilcox (Oracle)" <willy@infradead.org> When populating the page cache for readahead, mappings that use ->readpages must populate the page cache themselves as the pages are passed on a linked list which would normally be used for the page cache's LRU. For mappings that use ->readpage or the upcoming ->readahead method, we can put the pages into the page cache as soon as they're allocated, which solves a race between readahead and direct IO. It also lets us remove the gfp argument from read_pages(). Use the new readahead_page() API to implement the repeated calls to ->readpage(), just like most filesystems will. Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: William Kucharski <william.kucharski@oracle.com> --- mm/readahead.c | 46 ++++++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/mm/readahead.c b/mm/readahead.c index ddc63d3b07b8..e52b3a7b9da5 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -114,14 +114,14 @@ int read_cache_pages(struct address_space *mapping, struct list_head *pages, EXPORT_SYMBOL(read_cache_pages); static void read_pages(struct readahead_control *rac, struct list_head *pages, - gfp_t gfp) + bool skip_page) { const struct address_space_operations *aops = rac->mapping->a_ops; + struct page *page; struct blk_plug plug; - unsigned page_idx; if (!readahead_count(rac)) - return; + goto out; blk_start_plug(&plug); @@ -130,23 +130,23 @@ static void read_pages(struct readahead_control *rac, struct list_head *pages, readahead_count(rac)); /* Clean up the remaining pages */ put_pages_list(pages); - goto out; - } - - for (page_idx = 0; page_idx < readahead_count(rac); page_idx++) { - struct page *page = lru_to_page(pages); - list_del(&page->lru); - if (!add_to_page_cache_lru(page, rac->mapping, page->index, - gfp)) + rac->_index += rac->_nr_pages; + rac->_nr_pages = 0; + } else { + while ((page = readahead_page(rac))) { aops->readpage(rac->file, page); - put_page(page); + put_page(page); + } } -out: blk_finish_plug(&plug); BUG_ON(!list_empty(pages)); - rac->_nr_pages = 0; + BUG_ON(readahead_count(rac)); + +out: + if (skip_page) + rac->_index++; } /* @@ -168,6 +168,7 @@ void __do_page_cache_readahead(struct address_space *mapping, struct readahead_control rac = { .mapping = mapping, .file = filp, + ._index = index, }; unsigned long i; @@ -183,6 +184,8 @@ void __do_page_cache_readahead(struct address_space *mapping, if (index + i > end_index) break; + BUG_ON(index + i != rac._index + rac._nr_pages); + page = xa_load(&mapping->i_pages, index + i); if (page && !xa_is_value(page)) { /* @@ -190,15 +193,22 @@ void __do_page_cache_readahead(struct address_space *mapping, * contiguous pages before continuing with the next * batch. */ - read_pages(&rac, &page_pool, gfp_mask); + read_pages(&rac, &page_pool, true); continue; } page = __page_cache_alloc(gfp_mask); if (!page) break; - page->index = index + i; - list_add(&page->lru, &page_pool); + if (mapping->a_ops->readpages) { + page->index = index + i; + list_add(&page->lru, &page_pool); + } else if (add_to_page_cache_lru(page, mapping, index + i, + gfp_mask) < 0) { + put_page(page); + read_pages(&rac, &page_pool, true); + continue; + } if (i == nr_to_read - lookahead_size) SetPageReadahead(page); rac._nr_pages++; @@ -209,7 +219,7 @@ void __do_page_cache_readahead(struct address_space *mapping, * uptodate then the caller will launch readpage again, and * will then handle the error. */ - read_pages(&rac, &page_pool, gfp_mask); + read_pages(&rac, &page_pool, false); } /* -- 2.25.1
next prev parent reply index Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-14 15:02 [PATCH v11 00/25] Change readahead API Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 01/25] mm: Move readahead prototypes from mm.h Matthew Wilcox 2020-04-15 9:10 ` Johannes Thumshirn 2020-04-14 15:02 ` [PATCH v11 02/25] mm: Return void from various readahead functions Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 03/25] mm: Ignore return value of ->readpages Matthew Wilcox 2020-04-15 9:17 ` Johannes Thumshirn 2020-04-14 15:02 ` [PATCH v11 04/25] mm: Move readahead nr_pages check into read_pages Matthew Wilcox 2020-04-15 9:19 ` Johannes Thumshirn 2020-04-14 15:02 ` [PATCH v11 05/25] mm: Add new readahead_control API Matthew Wilcox 2020-04-15 1:17 ` Andrew Morton 2020-04-15 2:18 ` Matthew Wilcox 2020-04-15 4:56 ` Andrew Morton 2020-04-15 11:22 ` Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 06/25] mm: Use readahead_control to pass arguments Matthew Wilcox 2020-04-15 9:30 ` Johannes Thumshirn 2020-04-14 15:02 ` [PATCH v11 07/25] mm: Rename various 'offset' parameters to 'index' Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 08/25] mm: rename readahead loop variable to 'i' Matthew Wilcox 2020-04-15 9:31 ` Johannes Thumshirn 2020-04-14 15:02 ` [PATCH v11 09/25] mm: Remove 'page_offset' from readahead loop Matthew Wilcox 2020-04-14 15:02 ` Matthew Wilcox [this message] 2020-04-14 15:02 ` [PATCH v11 11/25] mm: Add readahead address space operation Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 12/25] mm: Move end_index check out of readahead loop Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 13/25] mm: Add page_cache_readahead_unbounded Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 14/25] mm: Document why we don't set PageReadahead Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 15/25] mm: Use memalloc_nofs_save in readahead path Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 16/25] fs: Convert mpage_readpages to mpage_readahead Matthew Wilcox 2020-06-16 22:36 ` [Cluster-devel] " Andreas Gruenbacher 2020-06-17 0:32 ` Matthew Wilcox 2020-06-17 0:57 ` Andreas Grünbacher 2020-06-17 2:21 ` Matthew Wilcox 2020-06-18 12:46 ` Andreas Gruenbacher 2020-06-18 15:03 ` Matthew Wilcox 2020-06-18 16:40 ` Andreas Gruenbacher 2020-04-14 15:02 ` [PATCH v11 17/25] btrfs: Convert from readpages to readahead Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 18/25] erofs: Convert uncompressed files " Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 19/25] erofs: Convert compressed " Matthew Wilcox 2020-04-21 5:42 ` Andrew Morton 2020-04-21 7:28 ` Gao Xiang 2020-04-14 15:02 ` [PATCH v11 20/25] ext4: Convert " Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 21/25] ext4: Pass the inode to ext4_mpage_readpages Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 22/25] f2fs: Convert from readpages to readahead Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 23/25] f2fs: Pass the inode to f2fs_mpage_readpages Matthew Wilcox 2020-04-14 15:02 ` [PATCH v11 24/25] fuse: Convert from readpages to readahead Matthew Wilcox 2020-04-20 11:14 ` Miklos Szeredi 2020-04-20 11:43 ` Matthew Wilcox 2020-04-20 11:54 ` Miklos Szeredi 2020-04-14 15:02 ` [PATCH v11 25/25] iomap: " Matthew Wilcox
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200414150233.24495-11-willy@infradead.org \ --to=willy@infradead.org \ --cc=akpm@linux-foundation.org \ --cc=cluster-devel@redhat.com \ --cc=hch@lst.de \ --cc=linux-btrfs@vger.kernel.org \ --cc=linux-erofs@lists.ozlabs.org \ --cc=linux-ext4@vger.kernel.org \ --cc=linux-f2fs-devel@lists.sourceforge.net \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=linux-xfs@vger.kernel.org \ --cc=ocfs2-devel@oss.oracle.com \ --cc=william.kucharski@oracle.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-BTRFS Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \ linux-btrfs@vger.kernel.org public-inbox-index linux-btrfs Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs AGPL code for this site: git clone https://public-inbox.org/public-inbox.git