linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Steve French <sfrench@samba.org>,
	Jeff Layton <jlayton@redhat.com>
Cc: dhowells@redhat.com, Matthew Wilcox <willy@infradead.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
	v9fs-developer@lists.sourceforge.net,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH 61/61] fscache: Document the rewritten cache backend API
Date: Mon, 04 May 2020 18:16:49 +0100	[thread overview]
Message-ID: <158861260900.340223.17584394318899057798.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <158861203563.340223.7585359869938129395.stgit@warthog.procyon.org.uk>

Document the rewritten cache backend API.

Signed-off-by: David Howells <dhowells@redhat.com>
---

 Documentation/filesystems/caching/backend-api.txt |  172 ++++++++++++++++++---
 1 file changed, 148 insertions(+), 24 deletions(-)

diff --git a/Documentation/filesystems/caching/backend-api.txt b/Documentation/filesystems/caching/backend-api.txt
index 710d10408fcb..9af6f9618576 100644
--- a/Documentation/filesystems/caching/backend-api.txt
+++ b/Documentation/filesystems/caching/backend-api.txt
@@ -42,7 +42,7 @@ previously initialised cache definition to:
 			      struct fscache_object *fsdef,
 			      const char *tagname);
 
-Two extra arguments should also be supplied:
+Two further arguments should also be supplied:
 
  (*) "fsdef" which should point to the object representation for the FS-Cache
      master index in this cache.  Netfs primary index entries will be created
@@ -50,9 +50,7 @@ Two extra arguments should also be supplied:
      successful and will release it upon withdrawal of the cache.
 
  (*) "tagname" which, if given, should be a text string naming this cache.  If
-     this is NULL, the identifier will be used instead.  For CacheFS, the
-     identifier is set to name the underlying block device and the tag can be
-     supplied by mount.
+     this is NULL, the identifier will be used instead.
 
 This function may return -ENOMEM if it ran out of memory or -EEXIST if the tag
 is already in use.  0 will be returned on success.
@@ -156,28 +154,45 @@ performed on the denizens of the cache.  These are held in a structure of type:
  (*) Allocate a new object [mandatory]:
 
 	struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
-					       struct fscache_cookie *cookie)
+					       struct fscache_cookie *cookie,
+					       struct fscache_object *parent)
 
      This method is used to allocate a cache object representation to back a
      cookie in a particular cache.  fscache_object_init() should be called on
      the object to initialise it prior to returning.
 
+ (*) Prepare lookup data [mandatory]:
 
- (*) Look up and create object [mandatory]:
+	void *(*prepare_lookup_data)(struct fscache_object *object);
 
-	void (*lookup_object)(struct fscache_object *object)
+     This method is used to prepare lookup data that can be passed to the
+     lookup_object and create_object methods.
+
+
+ (*) Look up an object [mandatory]:
+
+	void (*lookup_object)(struct fscache_object *object,
+			      void *lookup_data);
 
      This method is used to look up an object, given that the object is already
-     allocated and attached to the cookie.  This should instantiate that object
-     in the cache if it can.
+     allocated and attached to the cookie.
+
+
+ (*) Create an object [mandatory]:
+
+	void (*create_object)(struct fscache_object *object,
+			      void *lookup_data);
+
+     This method is used to create an object that has previously been looked
+     up.
 
 
  (*) Release lookup data [mandatory]:
 
-	void (*lookup_complete)(struct fscache_object *object)
+	void (*free_lookup_data)(struct fscache_object *object,
+				 void *lookup_data);
 
-     This method is called to ask the cache to release any resources it was
-     using to perform a lookup.
+     This method is called to clean up the lookup data.
 
 
  (*) Increment object refcount [mandatory]:
@@ -202,23 +217,27 @@ performed on the denizens of the cache.  These are held in a structure of type:
 
 	int (*update_object)(struct fscache_object *object)
 
-     This is called to update the index entry for the specified object.  The
-     new information should be in object->cookie->aux and ->object_size.
+     This is called to update the coherency for the specified object and to
+     trim the cache object to size.  The cache should use the information
+     stored in object->cookie->aux and ->object_size for this purpose.
 
 
  (*) Invalidate data object [mandatory]:
 
-	int (*invalidate_object)(struct fscache_operation *op)
+	int (*invalidate_object)(struct fscache_operation *op,
+				 unsigned int flags)
 
      This is called to invalidate a data object.
 
 
  (*) Discard object [mandatory]:
 
-	void (*drop_object)(struct fscache_object *object)
+	void (*drop_object)(struct fscache_object *object,
+			    bool invalidate)
 
      This method is called to indicate that an object has been unbound from its
-     cookie.
+     cookie.  If invalidate is true, the object should be removed from the
+     cache.
 
      This method should not attempt to release any references held by the
      caller.  The caller will invoke the put_object() method as appropriate.
@@ -228,8 +247,16 @@ performed on the denizens of the cache.  These are held in a structure of type:
 
 	void (*put_object)(struct fscache_object *object)
 
-     This method is used to discard a reference to an object.  The object may
-     be freed when all the references to it are released.
+     This method is used to discard a reference to an object.  The object
+     should be freeable when all the references to it are released.
+
+
+ (*) Get object reference count [mandatory]:
+
+	unsigned int (*get_object_usage)(const struct fscache_object *object);
+
+     This method is used to read the reference count on an object for display
+     purposes.
 
 
  (*) Synchronise a cache [mandatory]:
@@ -258,12 +285,85 @@ performed on the denizens of the cache.  These are held in a structure of type:
      size if larger than that already.
 
 
+ (*) Shape request extent [mandatory]:
+
+	unsigned int (*shape_extent)(struct fscache_object *object,
+				     struct fscache_extent *extent,
+				     loff_t i_size, bool for_write);
+
+     This method is called to shape a request according to the granularity of
+     the cache's content tracking.  The cache may expand and contract the
+     extent, but it is required to keep the start page within the shaped
+     extent.
+
+ (*) Read data from the cache [mandatory]:
+
+	int (*read)(struct fscache_object *object,
+		    struct fscache_io_request *req,
+		    struct iov_iter *iter);
+
+     This method is called to read data from the cache into the buffer
+     specified by 'iter'.  The location and length of the read are specified in
+     the request descriptor.  The operation must be synchronous if req->io_done
+     is not set.  req->io_done() must be called if provided, though this may be
+     deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+     The netfs is expected to have rounded its request to units of
+     dio_block_size as returned by ->shape_extent().
+
+     req->error, req->transferred and req->data_from_cache should be updated as
+     appropriate and fscache_end_io_operation() and fscache_put_io_request()
+     called.  If the operation didn't read all the data, -ENODATA should be
+     set.
+
+ (*) Write data to the cache [mandatory]:
+
+	int (*write)(struct fscache_object *object,
+		     struct fscache_io_request *req,
+		     struct iov_iter *iter);
+
+     This method is called to write data from the buffer specified by 'iter'
+     into the cache.  The location and length of the write are specified in the
+     request descriptor.  The operation must be synchronous if req->io_done is
+     not set.  req->io_done() must be called if provided, though this may be
+     deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+     The netfs is expected to have rounded its request to units of
+     dio_block_size as returned by ->shape_extent().
+
+     req->error should be updated as appropriate and fscache_end_io_operation()
+     and fscache_put_io_request() called.  If the operation didn't write all
+     the data, -ENOBUFS should be set.
+
+ (*) Prepare to write to a live cache object [mandatory]:
+
+	int (*prepare_to_write)(struct fscache_object *object);
+
+     This method is called when fscache_use_object() is called on an object
+     that already exists and will_modify is true and FSCACHE_OBJECT_LOCAL_WRITE
+     was not yet set on the object.  This allows the cache to mark the object
+     as being dirty prior to writing data to it, or to take a local copy.
+
+ (*) Display object info [mandatory]:
+
+	int (*display_object)(struct seq_file *m, struct fscache_object *object);
+
+     This method is called to summarise an object in /proc/fs/fscache/objects.
+
+
 ==================
 FS-CACHE UTILITIES
 ==================
 
 FS-Cache provides some utilities that a cache backend may make use of:
 
+ (*) The filesystem index cookie:
+
+	struct fscache_cookie fscache_fsdef_index
+
+     This is the cookie to which the object passed to fscache_add_cache() are
+     bound.
+
  (*) Note occurrence of an I/O error in a cache:
 
 	void fscache_io_error(struct fscache_cache *cache)
@@ -283,14 +383,13 @@ FS-Cache provides some utilities that a cache backend may make use of:
      This initialises all the fields in an object representation.
 
 
- (*) Indicate the destruction of an object:
+ (*) Bracket the destruction of an object.
 
+	void fscache_object_destroy(struct fscache_object *object);
 	void fscache_object_destroyed(struct fscache_cache *cache);
 
-     This must be called to inform FS-Cache that an object that belonged to a
-     cache has been destroyed and deallocated.  This will allow continuation
-     of the cache withdrawal process when it is stopped pending destruction of
-     all the objects.
+     These must be called on either side of object destruction to clean up the
+     FS-Cache parts of an object and do the accounting.
 
 
  (*) Indicate that a stale object was found and discarded:
@@ -314,3 +413,28 @@ FS-Cache provides some utilities that a cache backend may make use of:
 	FSCACHE_OBJECT_NO_SPACE - there was insufficient cache space
 	FSCACHE_OBJECT_WAS_RETIRED - the object was retired when relinquished.
 	FSCACHE_OBJECT_WAS_CULLED - the object was culled to make space.
+
+
+ (*) Get pointers to the cookie index key and coherency data buffers:
+
+	void *fscache_get_key(struct fscache_cookie *cookie);
+	void *fscache_get_aux(struct fscache_cookie *cookie);
+
+     These handle the switching between internal buffering for small data and
+     kmalloc'd buffering for larger data.
+
+
+ (*) End an I/O operation:
+
+	void fscache_end_io_operation(struct fscache_cookie *cookie);
+
+     This is call to manage the accounting at the end of an I/O operation.
+
+
+ (*) Get/put references on I/O request descriptors.
+
+	void fscache_get_io_request(struct fscache_io_request *req);
+	void fscache_put_io_request(struct fscache_io_request *req);
+
+     Manage references on I/O request descriptors.  These may be called in
+     softirq context.



  parent reply	other threads:[~2020-05-04 17:17 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-04 17:07 [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter David Howells
2020-05-04 17:07 ` [RFC PATCH 01/61] afs: Make afs_zap_data() static David Howells
2020-05-04 17:07 ` [RFC PATCH 02/61] iov_iter: Add ITER_MAPPING David Howells
2020-05-04 17:07 ` [RFC PATCH 03/61] vm: Add wait/unlock functions for PG_fscache David Howells
2020-05-04 17:08 ` [RFC PATCH 04/61] vfs: Export rw_verify_area() for use by cachefiles David Howells
2020-05-04 17:08 ` [RFC PATCH 05/61] vfs: Provide S_CACHE_FILE inode flag David Howells
2020-05-04 17:08 ` [RFC PATCH 06/61] afs: Disable use of the fscache I/O routines David Howells
2020-05-04 17:08 ` [RFC PATCH 07/61] fscache: Add a cookie debug ID and use that in traces David Howells
2020-05-04 17:08 ` [RFC PATCH 08/61] fscache: Procfile to display cookies David Howells
2020-05-04 17:08 ` [RFC PATCH 09/61] fscache: Temporarily disable network filesystems' use of fscache David Howells
2020-05-04 17:08 ` [RFC PATCH 10/61] fscache: Remove the old I/O API David Howells
2020-05-04 17:09 ` [RFC PATCH 11/61] fscache: Remove the netfs data from the cookie David Howells
2020-05-04 17:09 ` [RFC PATCH 12/61] fscache: Remove struct fscache_cookie_def David Howells
2020-05-04 17:09 ` [RFC PATCH 13/61] fscache: Remove store_limit* from struct fscache_object David Howells
2020-05-04 17:09 ` [RFC PATCH 14/61] fscache: Remove fscache_check_consistency() David Howells
2020-05-04 17:09 ` [RFC PATCH 15/61] fscache: Remove fscache_attr_changed() David Howells
2020-05-04 17:09 ` [RFC PATCH 16/61] fscache: Remove obsolete stats David Howells
2020-05-04 17:10 ` [RFC PATCH 17/61] fscache: Remove old I/O tracepoints David Howells
2020-05-04 17:10 ` [RFC PATCH 18/61] fscache: Temporarily disable fscache_invalidate() David Howells
2020-05-04 17:10 ` [RFC PATCH 19/61] fscache: Remove the I/O operation manager David Howells
2020-05-04 17:10 ` [RFC PATCH 20/61] cachefiles: Remove tree of active files and use S_CACHE_FILE inode flag David Howells
2020-05-04 17:10 ` [RFC PATCH 21/61] fscache: Provide a simple thread pool for running ops asynchronously David Howells
2020-05-04 17:10 ` [RFC PATCH 23/61] fscache: Rewrite the I/O API based on iov_iter David Howells
2020-05-04 17:11 ` [RFC PATCH 24/61] fscache: Remove fscache_wait_on_invalidate() David Howells
2020-05-04 17:11 ` [RFC PATCH 25/61] fscache: Keep track of size of a file last set independently on the server David Howells
2020-05-04 17:11 ` [RFC PATCH 26/61] fscache, cachefiles: Fix disabled histogram warnings David Howells
2020-05-04 17:11 ` [RFC PATCH 27/61] fscache: Recast assertion in terms of cookie not being an index David Howells
2020-05-04 17:11 ` [RFC PATCH 28/61] cachefiles: Remove some redundant checks on unsigned values David Howells
2020-05-04 17:11 ` [RFC PATCH 29/61] cachefiles: trace: Log coherency checks David Howells
2020-05-04 17:12 ` [RFC PATCH 30/61] cachefiles: Split cachefiles_drop_object() up a bit David Howells
2020-05-04 17:12 ` [RFC PATCH 31/61] cachefiles: Implement new fscache I/O backend API David Howells
2020-05-04 17:12 ` [RFC PATCH 32/61] cachefiles: Merge object->backer into object->dentry David Howells
2020-05-04 17:12 ` [RFC PATCH 33/61] cachefiles: Implement a content-present indicator and bitmap David Howells
2020-05-04 17:12 ` [RFC PATCH 34/61] cachefiles: Implement extent shaper David Howells
2020-05-04 17:12 ` [RFC PATCH 35/61] cachefiles: Round the cachefile size up to DIO block size David Howells
2020-05-04 17:12 ` [RFC PATCH 36/61] cachefiles: Implement read and write parts of new I/O API David Howells
2020-05-04 17:13 ` [RFC PATCH 37/61] cachefiles: Add I/O tracepoints David Howells
2020-05-04 17:13 ` [RFC PATCH 38/61] fscache: Add read helper David Howells
2020-05-04 17:13 ` [RFC PATCH 39/61] fscache: Display cache-specific data in /proc/fs/fscache/objects David Howells
2020-05-04 17:13 ` [RFC PATCH 40/61] fscache: Remove more obsolete stats David Howells
2020-05-04 17:13 ` [RFC PATCH 41/61] fscache: New stats David Howells
2020-05-04 17:13 ` [RFC PATCH 42/61] fscache, cachefiles: Rewrite invalidation David Howells
2020-05-04 17:13 ` [RFC PATCH 43/61] fscache: Implement "will_modify" parameter on fscache_use_cookie() David Howells
2020-05-04 17:14 ` [RFC PATCH 44/61] fscache: Provide resize operation David Howells
2020-05-04 17:14 ` [RFC PATCH 45/61] fscache: Remove the update operation David Howells
2020-05-04 17:14 ` [RFC PATCH 46/61] cachefiles: Shape write requests David Howells
2020-05-04 17:14 ` [RFC PATCH 47/61] afs: Remove afs_zero_fid as it's not used David Howells
2020-05-04 17:14 ` [RFC PATCH 48/61] afs: Move key to afs_read struct David Howells
2020-05-04 17:14 ` [RFC PATCH 49/61] afs: Don't truncate iter during data fetch David Howells
2020-05-04 17:15 ` [RFC PATCH 50/61] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2020-05-04 17:15 ` [RFC PATCH 51/61] afs: Use ITER_MAPPING for writing David Howells
2020-05-04 17:15 ` [RFC PATCH 52/61] afs: Interpose struct fscache_io_request into struct afs_read David Howells
2020-05-04 17:15 ` [RFC PATCH 53/61] afs: Note the amount transferred in fetch-data delivery David Howells
2020-05-04 17:15 ` [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2020-05-05 11:59   ` Matthew Wilcox
2020-05-06  7:57   ` David Howells
2020-05-06 11:09     ` Matthew Wilcox
2020-05-06 14:24     ` David Howells
2020-05-08 14:39     ` David Howells
2020-05-04 17:15 ` [RFC PATCH 55/61] afs: Use new fscache I/O API David Howells
2020-05-04 17:15 ` [RFC PATCH 56/61] afs: Copy local writes to the cache when writing to the server David Howells
2020-05-04 17:16 ` [RFC PATCH 57/61] afs: Invoke fscache_resize_cookie() when handling ATTR_SIZE for setattr David Howells
2020-05-04 17:16 ` [RFC PATCH 58/61] fscache: Rewrite the main document David Howells
2020-05-04 17:16 ` [RFC PATCH 59/61] fscache: Remove the obsolete API bits from the documentation David Howells
2020-05-04 17:16 ` [RFC PATCH 60/61] fscache: Document the new netfs API David Howells
2020-05-04 17:16 ` David Howells [this message]
2020-05-04 17:54 ` [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter Jeff Layton
2020-05-05  6:05 ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=158861260900.340223.17584394318899057798.stgit@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=jlayton@redhat.com \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=sfrench@samba.org \
    --cc=trondmy@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).