linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeffle Xu <jefflexu@linux.alibaba.com>
To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org,
	chao@kernel.org, linux-erofs@lists.ozlabs.org
Cc: linux-fsdevel@vger.kernel.org, joseph.qi@linux.alibaba.com,
	bo.liu@linux.alibaba.com, tao.peng@linux.alibaba.com,
	gerry@linux.alibaba.com, eguan@linux.alibaba.com,
	linux-kernel@vger.kernel.org
Subject: [PATCH v1 17/23] netfs: support on demand read
Date: Mon, 27 Dec 2021 20:54:38 +0800	[thread overview]
Message-ID: <20211227125444.21187-18-jefflexu@linux.alibaba.com> (raw)
In-Reply-To: <20211227125444.21187-1-jefflexu@linux.alibaba.com>

Add demand_read() callback to netfs_cache_ops to implement demand
reading.

The precondition for implementing demand reading semantic is that, all
blob files are sparse and have been placed under corresponding directory
on the first beginning. When upper fs starts to access the blob file, it
will "cache miss" (hit the hole) and then .issue_op() callback will be
called to prepare the data.

Then the working flow is as described below. The .issue_op() callback
could be implemented by netfs_demand_read() helper, which will in turn
call .demand_read() callback of corresponding fscache backend to prepare
the data.

The implementation of .demand_read() callback can be backend specific.
The following patch will introduce an implementation of .demand_read()
callback for cachefiles, which will notify user daemon the requested
file range to read. The .demand_read() callback will get blocked until
the user daemon has prepared the corresponding data.

Then once .demand_read() callback returns with 0, it is guaranteed that
the requested data has been ready. In this case, transform this IO
request to NETFS_READ_FROM_CACHE state, initiate an incomplete
completion and then retry to read from backing file.

Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com>
---
 fs/netfs/read_helper.c | 26 ++++++++++++++++++++++++++
 include/linux/netfs.h  |  4 ++++
 2 files changed, 30 insertions(+)

diff --git a/fs/netfs/read_helper.c b/fs/netfs/read_helper.c
index ca84918b6b5d..8aac65132b67 100644
--- a/fs/netfs/read_helper.c
+++ b/fs/netfs/read_helper.c
@@ -1006,6 +1006,32 @@ int netfs_readpage(struct file *file,
 }
 EXPORT_SYMBOL(netfs_readpage);
 
+void netfs_demand_read(struct netfs_read_subrequest *subreq)
+{
+	struct netfs_read_request *rreq = subreq->rreq;
+	struct netfs_cache_resources *cres = &rreq->cache_resources;
+	loff_t start_pos;
+	size_t len;
+	int ret;
+
+	start_pos = subreq->start + subreq->transferred;
+	len = subreq->len - subreq->transferred;
+
+	/*
+	 * In success case (ret == 0), user daemon has downloaded data for us,
+	 * thus transform to NETFS_READ_FROM_CACHE state and advertise that 0
+	 * byte readed, so that the request will enter into INCOMPLETE state and
+	 * retry to read from backing file.
+	 */
+	ret = cres->ops->demand_read(cres, start_pos, len);
+	if (!ret) {
+		subreq->source = NETFS_READ_FROM_CACHE;
+		__clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
+	}
+
+	netfs_subreq_terminated(subreq, ret, false);
+}
+
 /*
  * Prepare a folio for writing without reading first
  * @folio: The folio being prepared
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index b46c39d98bbd..80a738762deb 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -244,6 +244,9 @@ struct netfs_cache_ops {
 	int (*prepare_write)(struct netfs_cache_resources *cres,
 			     loff_t *_start, size_t *_len, loff_t i_size,
 			     bool no_space_allocated_yet);
+
+	int (*demand_read)(struct netfs_cache_resources *cres,
+			   loff_t start_pos, size_t len);
 };
 
 struct readahead_control;
@@ -259,6 +262,7 @@ extern int netfs_write_begin(struct file *, struct address_space *,
 			     void **,
 			     const struct netfs_read_request_ops *,
 			     void *);
+extern void netfs_demand_read(struct netfs_read_subrequest *);
 
 extern void netfs_subreq_terminated(struct netfs_read_subrequest *, ssize_t, bool);
 extern void netfs_stats_show(struct seq_file *);
-- 
2.27.0


  parent reply	other threads:[~2021-12-27 12:55 UTC|newest]

Thread overview: 40+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-27 12:54 [PATCH v1 00/23] fscache,erofs: fscache-based demand-read semantics Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 01/23] cachefiles: add cachefiles_demand devnode Jeffle Xu
2021-12-28  2:47   ` Joseph Qi
2021-12-28 12:34     ` JeffleXu
2021-12-27 12:54 ` [PATCH v1 02/23] cachefiles: add mode command to distinguish modes Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 03/23] cachefiles: detect backing file size in demand-read mode Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 04/23] netfs: make ops->init_rreq() optional Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 05/23] netfs: add inode parameter to netfs_alloc_read_request() Jeffle Xu
2022-01-04 14:00   ` Gao Xiang
2022-01-13  3:10   ` [Linux-cachefs] " JeffleXu
2022-01-13 12:09     ` Gao Xiang
2021-12-27 12:54 ` [PATCH v1 06/23] erofs: export erofs_map_blocks() Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 07/23] erofs: add nodev mode Jeffle Xu
2022-01-04 14:33   ` Gao Xiang
2022-01-04 14:58     ` Gao Xiang
2022-01-05  9:04       ` JeffleXu
2021-12-27 12:54 ` [PATCH v1 08/23] erofs: register global fscache volume Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 09/23] erofs: add cookie context helper functions Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 10/23] erofs: add anonymous inode managing page cache of blob file Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 11/23] erofs: register cookie context for bootstrap Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 12/23] erofs: implement fscache-based metadata read Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 13/23] erofs: implement fscache-based data read Jeffle Xu
2022-01-03  6:32   ` JeffleXu
2022-01-04 14:40   ` Gao Xiang
2022-01-05  2:29     ` JeffleXu
2021-12-27 12:54 ` [PATCH v1 14/23] erofs: register cookie context for data blobs Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 15/23] erofs: implement fscache-based data read " Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 16/23] erofs: add 'uuid' mount option Jeffle Xu
2021-12-27 12:54 ` Jeffle Xu [this message]
2021-12-27 12:54 ` [PATCH v1 18/23] cachefiles: use idr tree managing pending demand read Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 19/23] cachefiles: implement .demand_read() for " Jeffle Xu
2021-12-27 15:36   ` Matthew Wilcox
2021-12-28 12:33     ` JeffleXu
2022-01-12  9:02       ` JeffleXu
2022-01-19 13:20         ` Matthew Wilcox
2022-01-20 12:43           ` JeffleXu
2021-12-27 12:54 ` [PATCH v1 20/23] cachefiles: implement .poll() " Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 21/23] cachefiles: implement .read() " Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 22/23] cachefiles: add done command " Jeffle Xu
2021-12-27 12:54 ` [PATCH v1 23/23] erofs: support on " Jeffle Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211227125444.21187-18-jefflexu@linux.alibaba.com \
    --to=jefflexu@linux.alibaba.com \
    --cc=bo.liu@linux.alibaba.com \
    --cc=chao@kernel.org \
    --cc=dhowells@redhat.com \
    --cc=eguan@linux.alibaba.com \
    --cc=gerry@linux.alibaba.com \
    --cc=joseph.qi@linux.alibaba.com \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=tao.peng@linux.alibaba.com \
    --cc=xiang@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).