linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: linux-fsdevel@vger.kernel.org, linux-cachefs@redhat.com
Cc: dhowells@redhat.com, jlayton@redhat.com, dwysocha@redhat.com,
	Matthew Wilcox <willy@infradead.org>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Steve French <sfrench@samba.org>,
	Trond Myklebust <trond.myklebust@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Christoph Hellwig <hch@lst.de>,
	dchinner@redhat.com,
	Linus Torvalds <torvalds@linux-foundation.org>,
	v9fs-developer@lists.sourceforge.net,
	linux-afs@lists.infradead.org, ceph-devel@vger.kernel.org,
	linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Cut down implementation of fscache new API
Date: Mon, 18 Jan 2021 23:36:39 +0000	[thread overview]
Message-ID: <160655.1611012999@warthog.procyon.org.uk> (raw)
In-Reply-To: <2758811.1610621106@warthog.procyon.org.uk>

Take a look at:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/

I've extracted the netfs helper library from my patch set and built an
alternative cut-down I/O API for the existing fscache code as a bridge to
moving to a new fscache implementation.  With this, a netfs now has two
choices: use the existing API as is or use the netfs lib and the alternative
API.  You can't mix the two APIs - a netfs has to use one or the other.

It works with AFS, at least for reading data through a cache, and without a
cache, xfstests is quite happy.  I was able to take a bunch of the AFS patches
from my fscache-iter branch (the full rewrite) and apply them with minimal
changes.  Since it goes through the new I/O API in both cases, those changes
should be the same.  The main differences are in the cookie wrangling API.

The alternative API is different from the current in the following ways:

 (1) It uses kiocbs to do async DIO rather than using readpage() with page
     wake queue snooping and vfs_write().

 (2) It uses SEEK_HOLE/SEEK_DATA rather than bmap() to determine the location
     of data in the file.  This is still broken because we can't rely on this
     information in the backing filesystem.

 (3) It completely changes how PG_fscache is used.  As for the new API, it's
     used to indicate an in progress write to the cache from a page rather
     than a page the cache knows about.

 (4) It doesn't keep track of the netfs's pages beyond the termination of an
     I/O operation.  The old API added pages that have outstanding writes to
     the cache to a radix three for a background writer; now an async kiocb is
     dispatched.

 (5) The netfs needs to call fscache_begin_read_operation() from its
     ->begin_cache_operation() handler as passed to the netfs helper lib.
     This tells the netfs helpers how to access the cache.

 (6) It relies on the netfs helper lib to reissue a failed cache read to the
     server.

 (7) Handles THPs.

 (8) Implements completely ->readahead() and ->readpage() and implements a
     chunk of ->write_begin().

Things it doesn't address:

 (1) Mapping the content independently of the backing filesystem's metadata.

 (2) Getting rid of the backpointers into the netfs.

 (3) Simplifying the management of cookies and objects and their processing.

 (4) Holding an open file to the cache for any great length of time.  It gets
     a new file struct for each read op it does on the cache and drops it
     again afterwards.

 (5) Pinning the cache context/state required to handle a deferred write to
     the cache from ->write_begin() as performed by, say, ->writepages().

David


      parent reply	other threads:[~2021-01-18 23:38 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-14 10:45 Redesigning and modernising fscache David Howells
2021-01-14 16:19 ` Matthew Wilcox
2021-01-18 23:36 ` David Howells [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=160655.1611012999@warthog.procyon.org.uk \
    --to=dhowells@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=dchinner@redhat.com \
    --cc=dwysocha@redhat.com \
    --cc=hch@lst.de \
    --cc=jlayton@redhat.com \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=sfrench@samba.org \
    --cc=torvalds@linux-foundation.org \
    --cc=trond.myklebust@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=willy@infradead.org \
    --subject='Re: Cut down implementation of fscache new API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).