All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Steve French <sfrench@samba.org>,
	Jeff Layton <jlayton@redhat.com>
Cc: dhowells@redhat.com, Matthew Wilcox <willy@infradead.org>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	linux-afs@lists.infradead.org, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
	v9fs-developer@lists.sourceforge.net,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH 61/61] fscache: Document the rewritten cache backend API
Date: Mon, 04 May 2020 18:16:49 +0100	[thread overview]
Message-ID: <158861260900.340223.17584394318899057798.stgit@warthog.procyon.org.uk> (raw)
In-Reply-To: <158861203563.340223.7585359869938129395.stgit@warthog.procyon.org.uk>

Document the rewritten cache backend API.

Signed-off-by: David Howells <dhowells@redhat.com>
---

 Documentation/filesystems/caching/backend-api.txt |  172 ++++++++++++++++++---
 1 file changed, 148 insertions(+), 24 deletions(-)

diff --git a/Documentation/filesystems/caching/backend-api.txt b/Documentation/filesystems/caching/backend-api.txt
index 710d10408fcb..9af6f9618576 100644
--- a/Documentation/filesystems/caching/backend-api.txt
+++ b/Documentation/filesystems/caching/backend-api.txt
@@ -42,7 +42,7 @@ previously initialised cache definition to:
 			      struct fscache_object *fsdef,
 			      const char *tagname);
 
-Two extra arguments should also be supplied:
+Two further arguments should also be supplied:
 
  (*) "fsdef" which should point to the object representation for the FS-Cache
      master index in this cache.  Netfs primary index entries will be created
@@ -50,9 +50,7 @@ Two extra arguments should also be supplied:
      successful and will release it upon withdrawal of the cache.
 
  (*) "tagname" which, if given, should be a text string naming this cache.  If
-     this is NULL, the identifier will be used instead.  For CacheFS, the
-     identifier is set to name the underlying block device and the tag can be
-     supplied by mount.
+     this is NULL, the identifier will be used instead.
 
 This function may return -ENOMEM if it ran out of memory or -EEXIST if the tag
 is already in use.  0 will be returned on success.
@@ -156,28 +154,45 @@ performed on the denizens of the cache.  These are held in a structure of type:
  (*) Allocate a new object [mandatory]:
 
 	struct fscache_object *(*alloc_object)(struct fscache_cache *cache,
-					       struct fscache_cookie *cookie)
+					       struct fscache_cookie *cookie,
+					       struct fscache_object *parent)
 
      This method is used to allocate a cache object representation to back a
      cookie in a particular cache.  fscache_object_init() should be called on
      the object to initialise it prior to returning.
 
+ (*) Prepare lookup data [mandatory]:
 
- (*) Look up and create object [mandatory]:
+	void *(*prepare_lookup_data)(struct fscache_object *object);
 
-	void (*lookup_object)(struct fscache_object *object)
+     This method is used to prepare lookup data that can be passed to the
+     lookup_object and create_object methods.
+
+
+ (*) Look up an object [mandatory]:
+
+	void (*lookup_object)(struct fscache_object *object,
+			      void *lookup_data);
 
      This method is used to look up an object, given that the object is already
-     allocated and attached to the cookie.  This should instantiate that object
-     in the cache if it can.
+     allocated and attached to the cookie.
+
+
+ (*) Create an object [mandatory]:
+
+	void (*create_object)(struct fscache_object *object,
+			      void *lookup_data);
+
+     This method is used to create an object that has previously been looked
+     up.
 
 
  (*) Release lookup data [mandatory]:
 
-	void (*lookup_complete)(struct fscache_object *object)
+	void (*free_lookup_data)(struct fscache_object *object,
+				 void *lookup_data);
 
-     This method is called to ask the cache to release any resources it was
-     using to perform a lookup.
+     This method is called to clean up the lookup data.
 
 
  (*) Increment object refcount [mandatory]:
@@ -202,23 +217,27 @@ performed on the denizens of the cache.  These are held in a structure of type:
 
 	int (*update_object)(struct fscache_object *object)
 
-     This is called to update the index entry for the specified object.  The
-     new information should be in object->cookie->aux and ->object_size.
+     This is called to update the coherency for the specified object and to
+     trim the cache object to size.  The cache should use the information
+     stored in object->cookie->aux and ->object_size for this purpose.
 
 
  (*) Invalidate data object [mandatory]:
 
-	int (*invalidate_object)(struct fscache_operation *op)
+	int (*invalidate_object)(struct fscache_operation *op,
+				 unsigned int flags)
 
      This is called to invalidate a data object.
 
 
  (*) Discard object [mandatory]:
 
-	void (*drop_object)(struct fscache_object *object)
+	void (*drop_object)(struct fscache_object *object,
+			    bool invalidate)
 
      This method is called to indicate that an object has been unbound from its
-     cookie.
+     cookie.  If invalidate is true, the object should be removed from the
+     cache.
 
      This method should not attempt to release any references held by the
      caller.  The caller will invoke the put_object() method as appropriate.
@@ -228,8 +247,16 @@ performed on the denizens of the cache.  These are held in a structure of type:
 
 	void (*put_object)(struct fscache_object *object)
 
-     This method is used to discard a reference to an object.  The object may
-     be freed when all the references to it are released.
+     This method is used to discard a reference to an object.  The object
+     should be freeable when all the references to it are released.
+
+
+ (*) Get object reference count [mandatory]:
+
+	unsigned int (*get_object_usage)(const struct fscache_object *object);
+
+     This method is used to read the reference count on an object for display
+     purposes.
 
 
  (*) Synchronise a cache [mandatory]:
@@ -258,12 +285,85 @@ performed on the denizens of the cache.  These are held in a structure of type:
      size if larger than that already.
 
 
+ (*) Shape request extent [mandatory]:
+
+	unsigned int (*shape_extent)(struct fscache_object *object,
+				     struct fscache_extent *extent,
+				     loff_t i_size, bool for_write);
+
+     This method is called to shape a request according to the granularity of
+     the cache's content tracking.  The cache may expand and contract the
+     extent, but it is required to keep the start page within the shaped
+     extent.
+
+ (*) Read data from the cache [mandatory]:
+
+	int (*read)(struct fscache_object *object,
+		    struct fscache_io_request *req,
+		    struct iov_iter *iter);
+
+     This method is called to read data from the cache into the buffer
+     specified by 'iter'.  The location and length of the read are specified in
+     the request descriptor.  The operation must be synchronous if req->io_done
+     is not set.  req->io_done() must be called if provided, though this may be
+     deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+     The netfs is expected to have rounded its request to units of
+     dio_block_size as returned by ->shape_extent().
+
+     req->error, req->transferred and req->data_from_cache should be updated as
+     appropriate and fscache_end_io_operation() and fscache_put_io_request()
+     called.  If the operation didn't read all the data, -ENODATA should be
+     set.
+
+ (*) Write data to the cache [mandatory]:
+
+	int (*write)(struct fscache_object *object,
+		     struct fscache_io_request *req,
+		     struct iov_iter *iter);
+
+     This method is called to write data from the buffer specified by 'iter'
+     into the cache.  The location and length of the write are specified in the
+     request descriptor.  The operation must be synchronous if req->io_done is
+     not set.  req->io_done() must be called if provided, though this may be
+     deferred if asynchronous I/O is used and -EIOCBQUEUED returned.
+
+     The netfs is expected to have rounded its request to units of
+     dio_block_size as returned by ->shape_extent().
+
+     req->error should be updated as appropriate and fscache_end_io_operation()
+     and fscache_put_io_request() called.  If the operation didn't write all
+     the data, -ENOBUFS should be set.
+
+ (*) Prepare to write to a live cache object [mandatory]:
+
+	int (*prepare_to_write)(struct fscache_object *object);
+
+     This method is called when fscache_use_object() is called on an object
+     that already exists and will_modify is true and FSCACHE_OBJECT_LOCAL_WRITE
+     was not yet set on the object.  This allows the cache to mark the object
+     as being dirty prior to writing data to it, or to take a local copy.
+
+ (*) Display object info [mandatory]:
+
+	int (*display_object)(struct seq_file *m, struct fscache_object *object);
+
+     This method is called to summarise an object in /proc/fs/fscache/objects.
+
+
 ==================
 FS-CACHE UTILITIES
 ==================
 
 FS-Cache provides some utilities that a cache backend may make use of:
 
+ (*) The filesystem index cookie:
+
+	struct fscache_cookie fscache_fsdef_index
+
+     This is the cookie to which the object passed to fscache_add_cache() are
+     bound.
+
  (*) Note occurrence of an I/O error in a cache:
 
 	void fscache_io_error(struct fscache_cache *cache)
@@ -283,14 +383,13 @@ FS-Cache provides some utilities that a cache backend may make use of:
      This initialises all the fields in an object representation.
 
 
- (*) Indicate the destruction of an object:
+ (*) Bracket the destruction of an object.
 
+	void fscache_object_destroy(struct fscache_object *object);
 	void fscache_object_destroyed(struct fscache_cache *cache);
 
-     This must be called to inform FS-Cache that an object that belonged to a
-     cache has been destroyed and deallocated.  This will allow continuation
-     of the cache withdrawal process when it is stopped pending destruction of
-     all the objects.
+     These must be called on either side of object destruction to clean up the
+     FS-Cache parts of an object and do the accounting.
 
 
  (*) Indicate that a stale object was found and discarded:
@@ -314,3 +413,28 @@ FS-Cache provides some utilities that a cache backend may make use of:
 	FSCACHE_OBJECT_NO_SPACE - there was insufficient cache space
 	FSCACHE_OBJECT_WAS_RETIRED - the object was retired when relinquished.
 	FSCACHE_OBJECT_WAS_CULLED - the object was culled to make space.
+
+
+ (*) Get pointers to the cookie index key and coherency data buffers:
+
+	void *fscache_get_key(struct fscache_cookie *cookie);
+	void *fscache_get_aux(struct fscache_cookie *cookie);
+
+     These handle the switching between internal buffering for small data and
+     kmalloc'd buffering for larger data.
+
+
+ (*) End an I/O operation:
+
+	void fscache_end_io_operation(struct fscache_cookie *cookie);
+
+     This is call to manage the accounting at the end of an I/O operation.
+
+
+ (*) Get/put references on I/O request descriptors.
+
+	void fscache_get_io_request(struct fscache_io_request *req);
+	void fscache_put_io_request(struct fscache_io_request *req);
+
+     Manage references on I/O request descriptors.  These may be called in
+     softirq context.



  parent reply	other threads:[~2020-05-04 17:17 UTC|newest]

Thread overview: 92+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-04 17:07 [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter David Howells
2020-05-04 17:07 ` David Howells
2020-05-04 17:07 ` [RFC PATCH 01/61] afs: Make afs_zap_data() static David Howells
2020-05-04 17:07 ` [RFC PATCH 02/61] iov_iter: Add ITER_MAPPING David Howells
2020-05-04 17:07   ` David Howells
2020-05-04 17:07 ` [RFC PATCH 03/61] vm: Add wait/unlock functions for PG_fscache David Howells
2020-05-04 17:08 ` [RFC PATCH 04/61] vfs: Export rw_verify_area() for use by cachefiles David Howells
2020-05-04 17:08 ` [RFC PATCH 05/61] vfs: Provide S_CACHE_FILE inode flag David Howells
2020-05-04 17:08   ` David Howells
2020-05-04 17:08 ` [RFC PATCH 06/61] afs: Disable use of the fscache I/O routines David Howells
2020-05-04 17:08 ` [RFC PATCH 07/61] fscache: Add a cookie debug ID and use that in traces David Howells
2020-05-04 17:08 ` [RFC PATCH 08/61] fscache: Procfile to display cookies David Howells
2020-05-04 17:08 ` [RFC PATCH 09/61] fscache: Temporarily disable network filesystems' use of fscache David Howells
2020-05-04 17:08 ` [RFC PATCH 10/61] fscache: Remove the old I/O API David Howells
2020-05-04 17:08   ` David Howells
2020-05-04 17:09 ` [RFC PATCH 11/61] fscache: Remove the netfs data from the cookie David Howells
2020-05-04 17:09 ` [RFC PATCH 12/61] fscache: Remove struct fscache_cookie_def David Howells
2020-05-04 17:09   ` David Howells
2020-05-04 17:09 ` [RFC PATCH 13/61] fscache: Remove store_limit* from struct fscache_object David Howells
2020-05-04 17:09 ` [RFC PATCH 14/61] fscache: Remove fscache_check_consistency() David Howells
2020-05-04 17:09   ` David Howells
2020-05-04 17:09 ` [RFC PATCH 15/61] fscache: Remove fscache_attr_changed() David Howells
2020-05-04 17:09 ` [RFC PATCH 16/61] fscache: Remove obsolete stats David Howells
2020-05-04 17:09   ` David Howells
2020-05-04 17:10 ` [RFC PATCH 17/61] fscache: Remove old I/O tracepoints David Howells
2020-05-04 17:10 ` [RFC PATCH 18/61] fscache: Temporarily disable fscache_invalidate() David Howells
2020-05-04 17:10 ` [RFC PATCH 19/61] fscache: Remove the I/O operation manager David Howells
2020-05-04 17:10   ` David Howells
2020-05-04 17:10 ` [RFC PATCH 20/61] cachefiles: Remove tree of active files and use S_CACHE_FILE inode flag David Howells
2020-05-04 17:10   ` David Howells
2020-05-04 17:10 ` [RFC PATCH 21/61] fscache: Provide a simple thread pool for running ops asynchronously David Howells
2020-05-04 17:10   ` David Howells
2020-05-04 17:10 ` [RFC PATCH 22/61] fscache: Replace the object management state machine David Howells
2020-05-04 17:10 ` [RFC PATCH 23/61] fscache: Rewrite the I/O API based on iov_iter David Howells
2020-05-04 17:11 ` [RFC PATCH 24/61] fscache: Remove fscache_wait_on_invalidate() David Howells
2020-05-04 17:11   ` David Howells
2020-05-04 17:11 ` [RFC PATCH 25/61] fscache: Keep track of size of a file last set independently on the server David Howells
2020-05-04 17:11 ` [RFC PATCH 26/61] fscache, cachefiles: Fix disabled histogram warnings David Howells
2020-05-04 17:11 ` [RFC PATCH 27/61] fscache: Recast assertion in terms of cookie not being an index David Howells
2020-05-04 17:11 ` [RFC PATCH 28/61] cachefiles: Remove some redundant checks on unsigned values David Howells
2020-05-04 17:11 ` [RFC PATCH 29/61] cachefiles: trace: Log coherency checks David Howells
2020-05-04 17:12 ` [RFC PATCH 30/61] cachefiles: Split cachefiles_drop_object() up a bit David Howells
2020-05-04 17:12   ` David Howells
2020-05-04 17:12 ` [RFC PATCH 31/61] cachefiles: Implement new fscache I/O backend API David Howells
2020-05-04 17:12 ` [RFC PATCH 32/61] cachefiles: Merge object->backer into object->dentry David Howells
2020-05-04 17:12   ` David Howells
2020-05-04 17:12 ` [RFC PATCH 33/61] cachefiles: Implement a content-present indicator and bitmap David Howells
2020-05-04 17:12 ` [RFC PATCH 34/61] cachefiles: Implement extent shaper David Howells
2020-05-04 17:12 ` [RFC PATCH 35/61] cachefiles: Round the cachefile size up to DIO block size David Howells
2020-05-04 17:12 ` [RFC PATCH 36/61] cachefiles: Implement read and write parts of new I/O API David Howells
2020-05-04 17:13 ` [RFC PATCH 37/61] cachefiles: Add I/O tracepoints David Howells
2020-05-04 17:13 ` [RFC PATCH 38/61] fscache: Add read helper David Howells
2020-05-04 17:13   ` David Howells
2020-05-04 17:13 ` [RFC PATCH 39/61] fscache: Display cache-specific data in /proc/fs/fscache/objects David Howells
2020-05-04 17:13   ` David Howells
2020-05-04 17:13 ` [RFC PATCH 40/61] fscache: Remove more obsolete stats David Howells
2020-05-04 17:13 ` [RFC PATCH 41/61] fscache: New stats David Howells
2020-05-04 17:13 ` [RFC PATCH 42/61] fscache, cachefiles: Rewrite invalidation David Howells
2020-05-04 17:13 ` [RFC PATCH 43/61] fscache: Implement "will_modify" parameter on fscache_use_cookie() David Howells
2020-05-04 17:14 ` [RFC PATCH 44/61] fscache: Provide resize operation David Howells
2020-05-04 17:14   ` David Howells
2020-05-04 17:14 ` [RFC PATCH 45/61] fscache: Remove the update operation David Howells
2020-05-04 17:14   ` David Howells
2020-05-04 17:14 ` [RFC PATCH 46/61] cachefiles: Shape write requests David Howells
2020-05-04 17:14 ` [RFC PATCH 47/61] afs: Remove afs_zero_fid as it's not used David Howells
2020-05-04 17:14 ` [RFC PATCH 48/61] afs: Move key to afs_read struct David Howells
2020-05-04 17:14   ` David Howells
2020-05-04 17:14 ` [RFC PATCH 49/61] afs: Don't truncate iter during data fetch David Howells
2020-05-04 17:15 ` [RFC PATCH 50/61] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2020-05-04 17:15   ` David Howells
2020-05-04 17:15 ` [RFC PATCH 51/61] afs: Use ITER_MAPPING for writing David Howells
2020-05-04 17:15 ` [RFC PATCH 52/61] afs: Interpose struct fscache_io_request into struct afs_read David Howells
2020-05-04 17:15   ` David Howells
2020-05-04 17:15 ` [RFC PATCH 53/61] afs: Note the amount transferred in fetch-data delivery David Howells
2020-05-04 17:15 ` [RFC PATCH 54/61] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2020-05-05 11:59   ` Matthew Wilcox
2020-05-06  7:57   ` David Howells
2020-05-06 11:09     ` Matthew Wilcox
2020-05-06 14:24     ` David Howells
2020-05-08 14:39     ` David Howells
2020-05-08 14:39       ` David Howells
2020-05-04 17:15 ` [RFC PATCH 55/61] afs: Use new fscache I/O API David Howells
2020-05-04 17:15 ` [RFC PATCH 56/61] afs: Copy local writes to the cache when writing to the server David Howells
2020-05-04 17:16 ` [RFC PATCH 57/61] afs: Invoke fscache_resize_cookie() when handling ATTR_SIZE for setattr David Howells
2020-05-04 17:16 ` [RFC PATCH 58/61] fscache: Rewrite the main document David Howells
2020-05-04 17:16 ` [RFC PATCH 59/61] fscache: Remove the obsolete API bits from the documentation David Howells
2020-05-04 17:16 ` [RFC PATCH 60/61] fscache: Document the new netfs API David Howells
2020-05-04 17:16 ` David Howells [this message]
2020-05-04 17:54 ` [RFC PATCH 00/61] fscache, cachefiles: Rewrite the I/O interface in terms of kiocb/iov_iter Jeff Layton
2020-05-04 17:54   ` Jeff Layton
2020-05-05  6:05 ` Christoph Hellwig
2020-05-05  6:05   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=158861260900.340223.17584394318899057798.stgit@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=jlayton@redhat.com \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=sfrench@samba.org \
    --cc=trondmy@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.