From: Jeffle Xu <jefflexu@linux.alibaba.com> To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org, chao@kernel.org, linux-erofs@lists.ozlabs.org Cc: linux-fsdevel@vger.kernel.org, joseph.qi@linux.alibaba.com, bo.liu@linux.alibaba.com, tao.peng@linux.alibaba.com, gerry@linux.alibaba.com, eguan@linux.alibaba.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 03/20] netfs,fscache: support on-demand reading Date: Tue, 18 Jan 2022 21:11:59 +0800 [thread overview] Message-ID: <20220118131216.85338-4-jefflexu@linux.alibaba.com> (raw) In-Reply-To: <20220118131216.85338-1-jefflexu@linux.alibaba.com> Add ondemand_read() callback to netfs_cache_ops to implement on-demand reading. The precondition for implementing on-demand reading semantic is that, all blob files have been placed under corresponding directory with correct file size (sparse files) on the first beginning. When upper fs starts to access the blob file, it will "cache miss" (hit the hole) and then .issue_op() callback will be called to prepare the data. The following working flow is described as below. The .issue_op() callback could be implemented by netfs_ondemand_read() helper, which will in turn call .ondemand_read() callback of corresponding fscache backend to prepare the data. The implementation of .ondemand_read() callback can be backend specific. The following patch will introduce an implementation of .ondemand_read() callback for cachefiles, which will notify user daemon the requested file range to read. The .ondemand_read() callback will get blocked until the user daemon has prepared the corresponding data. Then once .ondemand_read() callback returns with 0, it is guaranteed that the requested data has been ready. In this case, transform this IO request to NETFS_READ_FROM_CACHE state, initiate an incomplete completion and then retry to read from backing file. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- fs/fscache/Kconfig | 8 ++++++++ fs/netfs/Kconfig | 8 ++++++++ fs/netfs/read_helper.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/netfs.h | 8 ++++++++ 4 files changed, 61 insertions(+) diff --git a/fs/fscache/Kconfig b/fs/fscache/Kconfig index 76316c4a3fb7..f6b5396759ee 100644 --- a/fs/fscache/Kconfig +++ b/fs/fscache/Kconfig @@ -41,3 +41,11 @@ config FSCACHE_DEBUG config FSCACHE_OLD_API bool + +config FSCACHE_ONDEMAND + bool "Support for on-demand reading" + depends on FSCACHE + select NETFS_ONDEMAND + help + This permits on-demand reading with fscache. + If unsure, say N. diff --git a/fs/netfs/Kconfig b/fs/netfs/Kconfig index b4db21022cb4..c4bdd0b032dd 100644 --- a/fs/netfs/Kconfig +++ b/fs/netfs/Kconfig @@ -21,3 +21,11 @@ config NETFS_STATS multi-CPU system these may be on cachelines that keep bouncing between CPUs. On the other hand, the stats are very useful for debugging purposes. Saying 'Y' here is recommended. + +config NETFS_ONDEMAND + bool "Support for on-demand reading" + depends on NETFS_SUPPORT + default n + help + This enables on-demand reading with netfs API. + If unsure, say N. diff --git a/fs/netfs/read_helper.c b/fs/netfs/read_helper.c index 077c0ca96612..b84c184c365d 100644 --- a/fs/netfs/read_helper.c +++ b/fs/netfs/read_helper.c @@ -1013,6 +1013,43 @@ int netfs_readpage(struct file *file, } EXPORT_SYMBOL(netfs_readpage); +#ifdef CONFIG_NETFS_ONDEMAND +void netfs_ondemand_read(struct netfs_read_subrequest *subreq) +{ + struct netfs_read_request *rreq = subreq->rreq; + struct netfs_cache_resources *cres = &rreq->cache_resources; + loff_t start_pos; + size_t len; + int ret = -ENOBUFS; + + /* The cache backend may not be accessible at this moment. */ + if (!cres->ops) + goto out; + + if (!cres->ops->ondemand_read) { + ret = -EOPNOTSUPP; + goto out; + } + + start_pos = subreq->p_start + subreq->transferred; + len = subreq->len - subreq->transferred; + + /* + * In success case (ret == 0), user daemon has prepared data for + * us, thus transform to NETFS_READ_FROM_CACHE state and + * advertise that 0 byte readed, so that the request will enter + * into INCOMPLETE state and retry to read from backing file. + */ + ret = cres->ops->ondemand_read(cres, start_pos, len); + if (!ret) { + subreq->source = NETFS_READ_FROM_CACHE; + __clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags); + } +out: + netfs_subreq_terminated(subreq, ret, false); +} +#endif + /* * Prepare a folio for writing without reading first * @folio: The folio being prepared diff --git a/include/linux/netfs.h b/include/linux/netfs.h index a17740b3b9d6..d6e041293dcc 100644 --- a/include/linux/netfs.h +++ b/include/linux/netfs.h @@ -246,6 +246,11 @@ struct netfs_cache_ops { int (*prepare_write)(struct netfs_cache_resources *cres, loff_t *_start, size_t *_len, loff_t i_size, bool no_space_allocated_yet); + +#ifdef CONFIG_NETFS_ONDEMAND + int (*ondemand_read)(struct netfs_cache_resources *cres, + loff_t start_pos, size_t len); +#endif }; struct readahead_control; @@ -261,6 +266,9 @@ extern int netfs_write_begin(struct file *, struct address_space *, void **, const struct netfs_read_request_ops *, void *); +#ifdef CONFIG_NETFS_ONDEMAND +extern void netfs_ondemand_read(struct netfs_read_subrequest *); +#endif extern void netfs_subreq_terminated(struct netfs_read_subrequest *, ssize_t, bool); extern void netfs_stats_show(struct seq_file *); -- 2.27.0
WARNING: multiple messages have this Message-ID (diff)
From: Jeffle Xu <jefflexu@linux.alibaba.com> To: dhowells@redhat.com, linux-cachefs@redhat.com, xiang@kernel.org, chao@kernel.org, linux-erofs@lists.ozlabs.org Cc: linux-kernel@vger.kernel.org, joseph.qi@linux.alibaba.com, linux-fsdevel@vger.kernel.org, gerry@linux.alibaba.com Subject: [PATCH v2 03/20] netfs,fscache: support on-demand reading Date: Tue, 18 Jan 2022 21:11:59 +0800 [thread overview] Message-ID: <20220118131216.85338-4-jefflexu@linux.alibaba.com> (raw) In-Reply-To: <20220118131216.85338-1-jefflexu@linux.alibaba.com> Add ondemand_read() callback to netfs_cache_ops to implement on-demand reading. The precondition for implementing on-demand reading semantic is that, all blob files have been placed under corresponding directory with correct file size (sparse files) on the first beginning. When upper fs starts to access the blob file, it will "cache miss" (hit the hole) and then .issue_op() callback will be called to prepare the data. The following working flow is described as below. The .issue_op() callback could be implemented by netfs_ondemand_read() helper, which will in turn call .ondemand_read() callback of corresponding fscache backend to prepare the data. The implementation of .ondemand_read() callback can be backend specific. The following patch will introduce an implementation of .ondemand_read() callback for cachefiles, which will notify user daemon the requested file range to read. The .ondemand_read() callback will get blocked until the user daemon has prepared the corresponding data. Then once .ondemand_read() callback returns with 0, it is guaranteed that the requested data has been ready. In this case, transform this IO request to NETFS_READ_FROM_CACHE state, initiate an incomplete completion and then retry to read from backing file. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- fs/fscache/Kconfig | 8 ++++++++ fs/netfs/Kconfig | 8 ++++++++ fs/netfs/read_helper.c | 37 +++++++++++++++++++++++++++++++++++++ include/linux/netfs.h | 8 ++++++++ 4 files changed, 61 insertions(+) diff --git a/fs/fscache/Kconfig b/fs/fscache/Kconfig index 76316c4a3fb7..f6b5396759ee 100644 --- a/fs/fscache/Kconfig +++ b/fs/fscache/Kconfig @@ -41,3 +41,11 @@ config FSCACHE_DEBUG config FSCACHE_OLD_API bool + +config FSCACHE_ONDEMAND + bool "Support for on-demand reading" + depends on FSCACHE + select NETFS_ONDEMAND + help + This permits on-demand reading with fscache. + If unsure, say N. diff --git a/fs/netfs/Kconfig b/fs/netfs/Kconfig index b4db21022cb4..c4bdd0b032dd 100644 --- a/fs/netfs/Kconfig +++ b/fs/netfs/Kconfig @@ -21,3 +21,11 @@ config NETFS_STATS multi-CPU system these may be on cachelines that keep bouncing between CPUs. On the other hand, the stats are very useful for debugging purposes. Saying 'Y' here is recommended. + +config NETFS_ONDEMAND + bool "Support for on-demand reading" + depends on NETFS_SUPPORT + default n + help + This enables on-demand reading with netfs API. + If unsure, say N. diff --git a/fs/netfs/read_helper.c b/fs/netfs/read_helper.c index 077c0ca96612..b84c184c365d 100644 --- a/fs/netfs/read_helper.c +++ b/fs/netfs/read_helper.c @@ -1013,6 +1013,43 @@ int netfs_readpage(struct file *file, } EXPORT_SYMBOL(netfs_readpage); +#ifdef CONFIG_NETFS_ONDEMAND +void netfs_ondemand_read(struct netfs_read_subrequest *subreq) +{ + struct netfs_read_request *rreq = subreq->rreq; + struct netfs_cache_resources *cres = &rreq->cache_resources; + loff_t start_pos; + size_t len; + int ret = -ENOBUFS; + + /* The cache backend may not be accessible at this moment. */ + if (!cres->ops) + goto out; + + if (!cres->ops->ondemand_read) { + ret = -EOPNOTSUPP; + goto out; + } + + start_pos = subreq->p_start + subreq->transferred; + len = subreq->len - subreq->transferred; + + /* + * In success case (ret == 0), user daemon has prepared data for + * us, thus transform to NETFS_READ_FROM_CACHE state and + * advertise that 0 byte readed, so that the request will enter + * into INCOMPLETE state and retry to read from backing file. + */ + ret = cres->ops->ondemand_read(cres, start_pos, len); + if (!ret) { + subreq->source = NETFS_READ_FROM_CACHE; + __clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags); + } +out: + netfs_subreq_terminated(subreq, ret, false); +} +#endif + /* * Prepare a folio for writing without reading first * @folio: The folio being prepared diff --git a/include/linux/netfs.h b/include/linux/netfs.h index a17740b3b9d6..d6e041293dcc 100644 --- a/include/linux/netfs.h +++ b/include/linux/netfs.h @@ -246,6 +246,11 @@ struct netfs_cache_ops { int (*prepare_write)(struct netfs_cache_resources *cres, loff_t *_start, size_t *_len, loff_t i_size, bool no_space_allocated_yet); + +#ifdef CONFIG_NETFS_ONDEMAND + int (*ondemand_read)(struct netfs_cache_resources *cres, + loff_t start_pos, size_t len); +#endif }; struct readahead_control; @@ -261,6 +266,9 @@ extern int netfs_write_begin(struct file *, struct address_space *, void **, const struct netfs_read_request_ops *, void *); +#ifdef CONFIG_NETFS_ONDEMAND +extern void netfs_ondemand_read(struct netfs_read_subrequest *); +#endif extern void netfs_subreq_terminated(struct netfs_read_subrequest *, ssize_t, bool); extern void netfs_stats_show(struct seq_file *); -- 2.27.0
next prev parent reply other threads:[~2022-01-18 13:12 UTC|newest] Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-01-18 13:11 [PATCH v2 00/20] fscache,erofs: fscache-based demand-read semantics Jeffle Xu 2022-01-18 13:11 ` Jeffle Xu 2022-01-18 13:11 ` [PATCH v2 01/20] netfs: make @file optional in netfs_alloc_read_request() Jeffle Xu 2022-01-18 13:11 ` Jeffle Xu 2022-01-18 13:11 ` [PATCH v2 02/20] netfs,cachefiles: manage logical/physical offset separately Jeffle Xu 2022-01-18 13:11 ` [PATCH v2 02/20] netfs, cachefiles: " Jeffle Xu 2022-01-18 13:11 ` Jeffle Xu [this message] 2022-01-18 13:11 ` [PATCH v2 03/20] netfs,fscache: support on-demand reading Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 04/20] cachefiles: extract generic daemon write function Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 05/20] cachefiles: detect backing file size in on-demand read mode Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 06/20] cachefiles: introduce new devnode for " Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 07/20] erofs: use meta buffers for erofs_read_superblock() Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 08/20] erofs: export erofs_map_blocks() Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 09/20] erofs: add mode checking helper Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 10/20] erofs: register global fscache volume Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 11/20] erofs: add cookie context helper functions Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 12/20] erofs: add anonymous inode managing page cache of blob file Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 13/20] erofs: register cookie context for bootstrap blob Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 14/20] erofs: implement fscache-based metadata read Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 15/20] erofs: implement fscache-based data read for non-inline layout Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 16/20] erofs: implement fscache-based data read for inline layout Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 17/20] erofs: register cookie context for data blobs Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 18/20] erofs: implement fscache-based data read " Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 19/20] erofs: add 'uuid' mount option Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-18 13:12 ` [PATCH v2 20/20] erofs: support on-demand reading Jeffle Xu 2022-01-18 13:12 ` Jeffle Xu 2022-01-19 6:40 ` [Linux-cachefs] [PATCH v2 00/20] fscache, erofs: fscache-based demand-read semantics Gao Xiang 2022-01-19 6:40 ` Gao Xiang 2022-01-21 10:57 ` JeffleXu 2022-01-21 10:57 ` JeffleXu 2022-01-24 17:23 ` [PATCH v2 00/20] fscache,erofs: " David Howells 2022-01-24 17:23 ` [PATCH v2 00/20] fscache, erofs: " David Howells 2022-01-25 1:53 ` [PATCH v2 00/20] fscache,erofs: " JeffleXu 2022-01-25 1:53 ` JeffleXu 2022-01-25 2:55 ` JeffleXu 2022-01-25 2:55 ` JeffleXu 2022-01-25 15:34 ` [PATCH v2 11/20] erofs: add cookie context helper functions David Howells 2022-01-25 15:34 ` David Howells 2022-01-26 6:45 ` JeffleXu 2022-01-26 6:45 ` JeffleXu 2022-01-25 16:15 ` [PATCH v2 00/20] fscache, erofs: fscache-based demand-read semantics David Howells 2022-01-25 16:15 ` [PATCH v2 00/20] fscache,erofs: " David Howells 2022-01-26 6:10 ` JeffleXu 2022-01-26 6:10 ` JeffleXu 2022-01-26 8:51 ` David Howells 2022-01-26 8:51 ` [PATCH v2 00/20] fscache, erofs: " David Howells 2022-01-27 7:07 ` [PATCH v2 00/20] fscache,erofs: " JeffleXu 2022-01-27 7:07 ` JeffleXu 2022-01-25 20:27 ` David Howells 2022-01-25 20:27 ` [PATCH v2 00/20] fscache, erofs: " David Howells 2022-01-26 5:26 ` [PATCH v2 00/20] fscache,erofs: " JeffleXu 2022-01-26 5:26 ` JeffleXu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220118131216.85338-4-jefflexu@linux.alibaba.com \ --to=jefflexu@linux.alibaba.com \ --cc=bo.liu@linux.alibaba.com \ --cc=chao@kernel.org \ --cc=dhowells@redhat.com \ --cc=eguan@linux.alibaba.com \ --cc=gerry@linux.alibaba.com \ --cc=joseph.qi@linux.alibaba.com \ --cc=linux-cachefs@redhat.com \ --cc=linux-erofs@lists.ozlabs.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=tao.peng@linux.alibaba.com \ --cc=xiang@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.