CEPH-Devel Archive on lore.kernel.org
 help / color / Atom feed
From: David Howells <dhowells@redhat.com>
To: Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Steve French <sfrench@samba.org>,
	Dominique Martinet <asmadeus@codewreck.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
	dhowells@redhat.com, Jeff Layton <jlayton@redhat.com>,
	David Wysochanski <dwysocha@redhat.com>,
	Matthew Wilcox <willy@infradead.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	linux-cachefs@redhat.com, linux-afs@lists.infradead.org,
	linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org,
	ceph-devel@vger.kernel.org, v9fs-developer@lists.sourceforge.net,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH 03/32] mm: Implement readahead_control pageset expansion
Date: Mon, 25 Jan 2021 21:31:26 +0000
Message-ID: <161161028670.2537118.13831420617039766044.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <161161025063.2537118.2009249444682241405.stgit@warthog.procyon.org.uk>

Provide a function, readahead_expand(), that expands the set of pages
specified by a readahead_control object to encompass a revised area with a
proposed size and length.

The proposed area must include all of the old area and may be expanded yet
more by this function so that the edges align on (transparent huge) page
boundaries as allocated.

The expansion will be cut short if a page already exists in either of the
areas being expanded into.  Note that any expansion made in such a case is
not rolled back.

This will be used by fscache so that reads can be expanded to cache granule
boundaries, thereby allowing whole granules to be stored in the cache, but
there are other potential users also.

Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: David Howells <dhowells@redhat.com>
---

 include/linux/pagemap.h |    2 +
 mm/readahead.c          |   70 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 1fa160e682fa..4935ad6171c1 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -774,6 +774,8 @@ extern void __delete_from_page_cache(struct page *page, void *shadow);
 int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask);
 void delete_from_page_cache_batch(struct address_space *mapping,
 				  struct pagevec *pvec);
+void readahead_expand(struct readahead_control *ractl,
+		      loff_t new_start, size_t new_len);
 
 /*
  * Like add_to_page_cache_locked, but used to add newly allocated pages:
diff --git a/mm/readahead.c b/mm/readahead.c
index c5b0457415be..4446dada0bc2 100644
--- a/mm/readahead.c
+++ b/mm/readahead.c
@@ -638,3 +638,73 @@ SYSCALL_DEFINE3(readahead, int, fd, loff_t, offset, size_t, count)
 {
 	return ksys_readahead(fd, offset, count);
 }
+
+/**
+ * readahead_expand - Expand a readahead request
+ * @ractl: The request to be expanded
+ * @new_start: The revised start
+ * @new_len: The revised size of the request
+ *
+ * Attempt to expand a readahead request outwards from the current size to the
+ * specified size by inserting locked pages before and after the current window
+ * to increase the size to the new window.  This may involve the insertion of
+ * THPs, in which case the window may get expanded even beyond what was
+ * requested.
+ *
+ * The algorithm will stop if it encounters a conflicting page already in the
+ * pagecache and leave a smaller expansion than requested.
+ *
+ * The caller must check for this by examining the revised @ractl object for a
+ * different expansion than was requested.
+ */
+void readahead_expand(struct readahead_control *ractl,
+		      loff_t new_start, size_t new_len)
+{
+	struct address_space *mapping = ractl->mapping;
+	pgoff_t new_index, new_nr_pages;
+	gfp_t gfp_mask = readahead_gfp_mask(mapping);
+
+	new_index = new_start / PAGE_SIZE;
+
+	/* Expand the leading edge downwards */
+	while (ractl->_index > new_index) {
+		unsigned long index = ractl->_index - 1;
+		struct page *page = xa_load(&mapping->i_pages, index);
+
+		if (page && !xa_is_value(page))
+			return; /* Page apparently present */
+
+		page = __page_cache_alloc(gfp_mask);
+		if (!page)
+			return;
+		if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) {
+			put_page(page);
+			return;
+		}
+
+		ractl->_nr_pages++;
+		ractl->_index = page->index;
+	}
+
+	new_len += new_start - readahead_pos(ractl);
+	new_nr_pages = DIV_ROUND_UP(new_len, PAGE_SIZE);
+
+	/* Expand the trailing edge upwards */
+	while (ractl->_nr_pages < new_nr_pages) {
+		unsigned long index = ractl->_index + ractl->_nr_pages;
+		struct page *page = xa_load(&mapping->i_pages, index);
+
+		if (page && !xa_is_value(page))
+			return; /* Page apparently present */
+
+		page = __page_cache_alloc(gfp_mask);
+		if (!page)
+			return;
+		if (add_to_page_cache_lru(page, mapping, index, gfp_mask) < 0) {
+			put_page(page);
+			return;
+		}
+		ractl->_nr_pages++;
+	}
+}
+EXPORT_SYMBOL(readahead_expand);



  parent reply index

Thread overview: 41+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-25 21:30 [PATCH 00/32] Network fs helper library & fscache kiocb API [ver #2] David Howells
2021-01-25 21:31 ` [PATCH 01/32] iov_iter: Add ITER_XARRAY David Howells
2021-01-25 21:31 ` [PATCH 02/32] vm: Add wait/unlock functions for PG_fscache David Howells
2021-01-25 21:31 ` David Howells [this message]
2021-01-25 21:31 ` [PATCH 04/32] vfs: Export rw_verify_area() for use by cachefiles David Howells
2021-01-25 21:31 ` [PATCH 05/32] netfs: Make a netfs helper module David Howells
2021-01-25 21:32 ` [PATCH 06/32] netfs: Provide readahead and readpage netfs helpers David Howells
2021-01-25 21:32 ` [PATCH 07/32] netfs: Add tracepoints David Howells
2021-01-25 21:32 ` [PATCH 08/32] netfs: Gather stats David Howells
2021-01-25 21:32 ` [PATCH 09/32] netfs: Add write_begin helper David Howells
2021-01-25 21:32 ` [PATCH 10/32] netfs: Define an interface to talk to a cache David Howells
2021-01-25 21:32 ` [PATCH 11/32] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache David Howells
2021-01-25 21:33 ` [PATCH 12/32] afs: Disable use of the fscache I/O routines David Howells
2021-01-25 21:33 ` [PATCH 13/32] afs: Pass page into dirty region helpers to provide THP size David Howells
2021-01-25 21:33 ` [PATCH 14/32] afs: Print the operation debug_id when logging an unexpected data version David Howells
2021-01-25 21:33 ` [PATCH 15/32] afs: Move key to afs_read struct David Howells
2021-01-25 21:34 ` [PATCH 16/32] afs: Don't truncate iter during data fetch David Howells
2021-01-25 21:34 ` [PATCH 17/32] afs: Log remote unmarshalling errors David Howells
2021-01-25 21:34 ` [PATCH 18/32] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2021-01-25 21:34 ` [PATCH 19/32] afs: Use ITER_XARRAY for writing David Howells
2021-01-25 21:34 ` [PATCH 20/32] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2021-01-25 21:35 ` [PATCH 21/32] afs: Extract writeback extension into its own function David Howells
2021-01-25 21:35 ` [PATCH 22/32] afs: Prepare for use of THPs David Howells
2021-01-25 21:35 ` [PATCH 23/32] afs: Use the fs operation ops to handle FetchData completion David Howells
2021-01-25 21:35 ` [PATCH 24/32] afs: Use new fscache read helper API David Howells
2021-01-25 21:35 ` [PATCH 25/32] NFS: Clean up nfs_readpage() and nfs_readpages() David Howells
2021-01-26  3:59   ` Matthew Wilcox
2021-01-26 15:33     ` David Wysochanski
2021-01-25 21:36 ` [PATCH 26/32] NFS: In nfs_readpage() only increment NFSIOS_READPAGES when read succeeds David Howells
2021-01-25 21:36 ` [PATCH 27/32] NFS: Refactor nfs_readpage() and nfs_readpage_async() to use nfs_readdesc David Howells
2021-01-26  4:05   ` Matthew Wilcox
2021-01-26 15:24     ` David Wysochanski
2021-01-25 21:36 ` [PATCH 28/32] NFS: Call readpage_async_filler() from nfs_readpage_async() David Howells
2021-01-25 21:36 ` [PATCH 29/32] NFS: Add nfs_pageio_complete_read() and remove nfs_readpage_async() David Howells
2021-01-25 21:37 ` [PATCH 30/32] NFS: Allow internal use of read structs and functions David Howells
2021-01-25 21:37 ` [PATCH 31/32] NFS: Convert to the netfs API and nfs_readpage to use netfs_readpage David Howells
2021-01-25 21:37 ` [PATCH 32/32] NFS: Convert readpage to readahead and use netfs_readahead for fscache David Howells
2021-01-26  1:36   ` Matthew Wilcox
2021-01-26 15:24     ` Chuck Lever
2021-01-26 19:10       ` David Wysochanski
2021-01-26 18:25     ` David Wysochanski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=161161028670.2537118.13831420617039766044.stgit@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=dwysocha@redhat.com \
    --cc=jlayton@redhat.com \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=sfrench@samba.org \
    --cc=trondmy@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

CEPH-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/ceph-devel/0 ceph-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 ceph-devel ceph-devel/ https://lore.kernel.org/ceph-devel \
		ceph-devel@vger.kernel.org
	public-inbox-index ceph-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.ceph-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git