All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: linux-cachefs@redhat.com
Cc: dhowells@redhat.com, Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Steve French <sfrench@samba.org>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Jeff Layton <jlayton@kernel.org>,
	Matthew Wilcox <willy@infradead.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	Omar Sandoval <osandov@osandov.com>,
	JeffleXu <jefflexu@linux.alibaba.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
	v9fs-developer@lists.sourceforge.net,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v3 25/68] fscache: Implement raw I/O interface
Date: Thu, 16 Dec 2021 16:13:46 +0000	[thread overview]
Message-ID: <163967122632.1823006.7487049517698562172.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <163967073889.1823006.12237147297060239168.stgit@warthog.procyon.org.uk>

Provide a pair of functions to perform raw I/O on the cache.  The first
function allows an arbitrary asynchronous direct-IO read to be made against
a cache object, though the read should be aligned and sized appropriately
for the backing device:

        int fscache_read(struct netfs_cache_resources *cres,
                         loff_t start_pos,
                         struct iov_iter *iter,
                         enum netfs_read_from_hole read_hole,
                         netfs_io_terminated_t term_func,
                         void *term_func_priv);

The cache resources must have been previously initialised by
fscache_begin_read_operation().  A read operation is sent to the backing
filesystem, starting at start_pos within the file.  The size of the read is
specified by the iterator, as is the location of the output buffer.

If there is a hole in the data it can be ignored and left to the backing
filesystem to deal with (NETFS_READ_HOLE_IGNORE), a hole at the beginning
can be skipped over and the buffer padded with zeros
(NETFS_READ_HOLE_CLEAR) or -ENODATA can be given (NETFS_READ_HOLE_FAIL).

If term_func is not NULL, the operation may be performed asynchronously.
Upon completion, successful or otherwise, (*term_func)() will be called and
passed term_func_priv, along with an error or the amount of data
transferred.  If the op is run asynchronously, fscache_read() will return
-EIOCBQUEUED.

The second function allows an arbitrary asynchronous direct-IO write to be
made against a cache object, though the write should be aligned and sized
appropriately for the backing device:

        int fscache_write(struct netfs_cache_resources *cres,
                          loff_t start_pos,
                          struct iov_iter *iter,
                          netfs_io_terminated_t term_func,
                          void *term_func_priv);

This works in very similar way to fscache_read(), except that there's no
need to deal with holes (they're just overwritten).

The caller is responsible for preventing concurrent overlapping writes.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: linux-cachefs@redhat.com
Link: https://lore.kernel.org/r/163819613224.215744.7877577215582621254.stgit@warthog.procyon.org.uk/ # v1
Link: https://lore.kernel.org/r/163906915386.143852.16936177636106480724.stgit@warthog.procyon.org.uk/ # v2
---

 include/linux/fscache.h        |   74 ++++++++++++++++++++++++++++++++++++++++
 include/trace/events/fscache.h |    2 +
 2 files changed, 76 insertions(+)

diff --git a/include/linux/fscache.h b/include/linux/fscache.h
index 9392812f10b7..bfc52de59aa0 100644
--- a/include/linux/fscache.h
+++ b/include/linux/fscache.h
@@ -430,4 +430,78 @@ int fscache_begin_read_operation(struct netfs_cache_resources *cres,
 	return -ENOBUFS;
 }
 
+/**
+ * fscache_read - Start a read from the cache.
+ * @cres: The cache resources to use
+ * @start_pos: The beginning file offset in the cache file
+ * @iter: The buffer to fill - and also the length
+ * @read_hole: How to handle a hole in the data.
+ * @term_func: The function to call upon completion
+ * @term_func_priv: The private data for @term_func
+ *
+ * Start a read from the cache.  @cres indicates the cache object to read from
+ * and must be obtained by a call to fscache_begin_operation() beforehand.
+ *
+ * The data is read into the iterator, @iter, and that also indicates the size
+ * of the operation.  @start_pos is the start position in the file, though if
+ * @seek_data is set appropriately, the cache can use SEEK_DATA to find the
+ * next piece of data, writing zeros for the hole into the iterator.
+ *
+ * Upon termination of the operation, @term_func will be called and supplied
+ * with @term_func_priv plus the amount of data written, if successful, or the
+ * error code otherwise.
+ *
+ * @read_hole indicates how a partially populated region in the cache should be
+ * handled.  It can be one of a number of settings:
+ *
+ *	NETFS_READ_HOLE_IGNORE - Just try to read (may return a short read).
+ *
+ *	NETFS_READ_HOLE_CLEAR - Seek for data, clearing the part of the buffer
+ *				skipped over, then do as for IGNORE.
+ *
+ *	NETFS_READ_HOLE_FAIL - Give ENODATA if we encounter a hole.
+ */
+static inline
+int fscache_read(struct netfs_cache_resources *cres,
+		 loff_t start_pos,
+		 struct iov_iter *iter,
+		 enum netfs_read_from_hole read_hole,
+		 netfs_io_terminated_t term_func,
+		 void *term_func_priv)
+{
+	const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
+	return ops->read(cres, start_pos, iter, read_hole,
+			 term_func, term_func_priv);
+}
+
+/**
+ * fscache_write - Start a write to the cache.
+ * @cres: The cache resources to use
+ * @start_pos: The beginning file offset in the cache file
+ * @iter: The data to write - and also the length
+ * @term_func: The function to call upon completion
+ * @term_func_priv: The private data for @term_func
+ *
+ * Start a write to the cache.  @cres indicates the cache object to write to and
+ * must be obtained by a call to fscache_begin_operation() beforehand.
+ *
+ * The data to be written is obtained from the iterator, @iter, and that also
+ * indicates the size of the operation.  @start_pos is the start position in
+ * the file.
+ *
+ * Upon termination of the operation, @term_func will be called and supplied
+ * with @term_func_priv plus the amount of data written, if successful, or the
+ * error code otherwise.
+ */
+static inline
+int fscache_write(struct netfs_cache_resources *cres,
+		  loff_t start_pos,
+		  struct iov_iter *iter,
+		  netfs_io_terminated_t term_func,
+		  void *term_func_priv)
+{
+	const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
+	return ops->write(cres, start_pos, iter, term_func, term_func_priv);
+}
+
 #endif /* _LINUX_FSCACHE_H */
diff --git a/include/trace/events/fscache.h b/include/trace/events/fscache.h
index 9f78c903b00a..2459d75659cf 100644
--- a/include/trace/events/fscache.h
+++ b/include/trace/events/fscache.h
@@ -79,6 +79,7 @@ enum fscache_access_trace {
 	fscache_access_io_not_live,
 	fscache_access_io_read,
 	fscache_access_io_wait,
+	fscache_access_io_write,
 	fscache_access_lookup_cookie,
 	fscache_access_lookup_cookie_end,
 	fscache_access_lookup_cookie_end_failed,
@@ -149,6 +150,7 @@ enum fscache_access_trace {
 	EM(fscache_access_io_not_live,		"END   io_notl")	\
 	EM(fscache_access_io_read,		"BEGIN io_read")	\
 	EM(fscache_access_io_wait,		"WAIT  io     ")	\
+	EM(fscache_access_io_write,		"BEGIN io_writ")	\
 	EM(fscache_access_lookup_cookie,	"BEGIN lookup ")	\
 	EM(fscache_access_lookup_cookie_end,	"END   lookup ")	\
 	EM(fscache_access_lookup_cookie_end_failed,"END   lookupf")	\



  parent reply	other threads:[~2021-12-16 16:14 UTC|newest]

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-16 16:05 [PATCH v3 00/68] fscache, cachefiles: Rewrite David Howells
2021-12-16 16:05 ` [PATCH v3 01/68] fscache, cachefiles: Disable configuration David Howells
2021-12-16 16:06 ` [PATCH v3 02/68] cachefiles: Delete the cachefiles driver pending rewrite David Howells
2021-12-16 16:06 ` [PATCH v3 03/68] fscache: Remove the contents of the fscache driver, " David Howells
2021-12-16 16:06 ` [PATCH v3 04/68] netfs: Display the netfs inode number in the netfs_read tracepoint David Howells
2021-12-16 16:06 ` [PATCH v3 05/68] netfs: Pass a flag to ->prepare_write() to say if there's no alloc'd space David Howells
2021-12-16 16:06 ` [PATCH v3 06/68] fscache: Introduce new driver David Howells
2021-12-16 16:07 ` [PATCH v3 07/68] fscache: Implement a hash function David Howells
2021-12-16 16:07 ` [PATCH v3 08/68] fscache: Implement cache registration David Howells
2021-12-16 16:07 ` [PATCH v3 09/68] fscache: Implement volume registration David Howells
2021-12-16 16:08 ` [PATCH v3 10/68] fscache: Implement cookie registration David Howells
2021-12-16 16:08 ` [PATCH v3 11/68] fscache: Implement cache-level access helpers David Howells
2021-12-16 16:09 ` [PATCH v3 12/68] fscache: Implement volume-level " David Howells
2021-12-16 16:09 ` [PATCH v3 13/68] fscache: Implement cookie-level " David Howells
2021-12-16 16:09 ` [PATCH v3 14/68] fscache: Implement functions add/remove a cache David Howells
2021-12-16 16:09 ` [PATCH v3 15/68] fscache: Provide and use cache methods to lookup/create/free a volume David Howells
2021-12-16 16:10 ` [PATCH v3 16/68] fscache: Add a function for a cache backend to note an I/O error David Howells
2021-12-16 16:10 ` [PATCH v3 17/68] fscache: Implement simple cookie state machine David Howells
2021-12-17 19:07   ` Jeff Layton
2021-12-17 19:45   ` David Howells
2021-12-16 16:11 ` [PATCH v3 18/68] fscache: Implement cookie user counting and resource pinning David Howells
2021-12-17 19:42   ` Jeff Layton
2021-12-17 20:43   ` David Howells
2021-12-16 16:11 ` [PATCH v3 19/68] fscache: Implement cookie invalidation David Howells
2021-12-16 16:11 ` [PATCH v3 20/68] fscache: Provide a means to begin an operation David Howells
2021-12-16 16:11 ` [PATCH v3 21/68] fscache: Count data storage objects in a cache David Howells
2021-12-16 16:12 ` [PATCH v3 22/68] fscache: Provide read/write stat counters for the cache David Howells
2021-12-16 16:12 ` [PATCH v3 23/68] fscache: Provide a function to let the netfs update its coherency data David Howells
2021-12-16 16:13 ` [PATCH v3 24/68] netfs: Pass more information on how to deal with a hole in the cache David Howells
2021-12-16 16:13 ` David Howells [this message]
2021-12-16 16:13 ` [PATCH v3 26/68] fscache: Implement higher-level write I/O interface David Howells
2021-12-16 16:14 ` [PATCH v3 27/68] vfs, fscache: Implement pinning of cache usage for writeback David Howells
2021-12-16 16:14 ` [PATCH v3 28/68] fscache: Provide a function to note the release of a page David Howells
2021-12-16 16:14 ` [PATCH v3 29/68] fscache: Provide a function to resize a cookie David Howells
2021-12-16 16:15 ` [PATCH v3 30/68] cachefiles: Introduce rewritten driver David Howells
2021-12-16 16:15 ` [PATCH v3 31/68] cachefiles: Define structs David Howells
2021-12-16 16:15 ` [PATCH v3 32/68] cachefiles: Add some error injection support David Howells
2021-12-16 16:15 ` [PATCH v3 33/68] cachefiles: Add a couple of tracepoints for logging errors David Howells
2021-12-16 16:16 ` [PATCH v3 34/68] cachefiles: Add cache error reporting macro David Howells
2021-12-16 16:16 ` [PATCH v3 35/68] cachefiles: Add security derivation David Howells
2021-12-16 16:16 ` [PATCH v3 36/68] cachefiles: Register a miscdev and parse commands over it David Howells
2021-12-16 16:16 ` [PATCH v3 37/68] cachefiles: Provide a function to check how much space there is David Howells
2021-12-16 16:16 ` [PATCH v3 38/68] vfs, cachefiles: Mark a backing file in use with an inode flag David Howells
2021-12-16 16:16 ` [PATCH v3 39/68] cachefiles: Implement a function to get/create a directory in the cache David Howells
2021-12-16 16:17 ` [PATCH v3 40/68] cachefiles: Implement cache registration and withdrawal David Howells
2021-12-16 16:17 ` [PATCH v3 41/68] cachefiles: Implement volume support David Howells
2021-12-16 16:17 ` [PATCH v3 42/68] cachefiles: Add tracepoints for calls to the VFS David Howells
2021-12-16 16:18 ` [PATCH v3 43/68] cachefiles: Implement object lifecycle funcs David Howells
2021-12-16 16:18 ` [PATCH v3 44/68] cachefiles: Implement key to filename encoding David Howells
2021-12-16 16:18 ` [PATCH v3 45/68] cachefiles: Implement metadata/coherency data storage in xattrs David Howells
2021-12-16 16:19 ` [PATCH v3 46/68] cachefiles: Mark a backing file in use with an inode flag David Howells
2021-12-16 16:19 ` [PATCH v3 47/68] cachefiles: Implement culling daemon commands David Howells
2021-12-16 16:19 ` [PATCH v3 48/68] cachefiles: Implement backing file wrangling David Howells
2021-12-16 16:20 ` [PATCH v3 49/68] cachefiles: Implement begin and end I/O operation David Howells
2021-12-16 16:20 ` [PATCH v3 50/68] cachefiles: Implement cookie resize for truncate David Howells
2021-12-16 16:20 ` [PATCH v3 51/68] cachefiles: Implement the I/O routines David Howells
2021-12-16 16:20 ` [PATCH v3 52/68] fscache, cachefiles: Store the volume coherency data David Howells
2021-12-16 16:20 ` [PATCH v3 53/68] cachefiles: Allow cachefiles to actually function David Howells
2021-12-16 16:21 ` [PATCH v3 54/68] fscache, cachefiles: Display stats of no-space events David Howells
2021-12-16 16:21 ` [PATCH v3 55/68] fscache, cachefiles: Display stat of culling events David Howells
2021-12-16 16:21 ` [PATCH v3 56/68] afs: Handle len being extending over page end in write_begin/write_end David Howells
2021-12-16 16:31   ` Linus Torvalds
2021-12-16 19:28     ` Matthew Wilcox
2021-12-16 19:46       ` Linus Torvalds
2021-12-16 20:20         ` Matthew Wilcox
2021-12-16 21:17       ` David Howells
2021-12-16 16:47   ` David Howells
2021-12-16 16:22 ` [PATCH v3 57/68] afs: Fix afs_write_end() to handle len > page size David Howells
2021-12-16 16:27   ` Linus Torvalds
2021-12-16 16:22 ` [PATCH v3 58/68] afs: Convert afs to use the new fscache API David Howells
2021-12-16 16:22 ` [PATCH v3 59/68] afs: Copy local writes to the cache when writing to the server David Howells
2021-12-16 16:22 ` [PATCH v3 60/68] afs: Skip truncation on the server of data we haven't written yet David Howells
2021-12-16 16:23 ` [PATCH v3 61/68] 9p: Use fscache indexing rewrite and reenable caching David Howells
2021-12-16 16:23 ` [PATCH v3 62/68] 9p: Copy local writes to the cache when writing to the server David Howells
2021-12-16 16:23 ` [PATCH v3 63/68] nfs: Convert to new fscache volume/cookie API David Howells
2021-12-17 12:53   ` David Wysochanski
2021-12-17 13:21   ` David Howells
2021-12-17 13:35     ` David Wysochanski
2021-12-16 16:24 ` [PATCH v3 64/68] nfs: Implement cache I/O by accessing the cache directly David Howells
2021-12-16 16:24 ` [PATCH v3 65/68] cifs: Support fscache indexing rewrite (untested) David Howells
2021-12-16 16:24 ` [PATCH v3 66/68] ceph: conversion to new fscache API David Howells
2021-12-16 16:25 ` [PATCH v3 67/68] ceph: add fscache writeback support David Howells
2021-12-16 16:25 ` [PATCH v3 68/68] fscache: Rewrite documentation David Howells
2021-12-17 20:13 ` [PATCH v3 00/68] fscache, cachefiles: Rewrite Jeff Layton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=163967122632.1823006.7487049517698562172.stgit@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=jefflexu@linux.alibaba.com \
    --cc=jlayton@kernel.org \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=osandov@osandov.com \
    --cc=sfrench@samba.org \
    --cc=torvalds@linux-foundation.org \
    --cc=trondmy@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.