From: bfields@fieldses.org (J. Bruce Fields)
To: David Howells <dhowells@redhat.com>
Cc: Trond Myklebust <trondmy@hammerspace.com>,
Anna Schumaker <anna.schumaker@netapp.com>,
Steve French <sfrench@samba.org>,
Dominique Martinet <asmadeus@codewreck.org>,
Takashi Iwai <tiwai@suse.de>,
Matthew Wilcox <willy@infradead.org>,
linux-afs@lists.infradead.org, Jeff Layton <jlayton@redhat.com>,
David Wysochanski <dwysocha@redhat.com>,
Alexander Viro <viro@zeniv.linux.org.uk>,
linux-cachefs@redhat.com, linux-nfs@vger.kernel.org,
linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
v9fs-developer@lists.sourceforge.net,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API
Date: Thu, 21 Jan 2021 11:46:45 -0500 [thread overview]
Message-ID: <20210121164645.GA20964@fieldses.org> (raw)
In-Reply-To: <161118128472.1232039.11746799833066425131.stgit@warthog.procyon.org.uk>
On Wed, Jan 20, 2021 at 10:21:24PM +0000, David Howells wrote:
> Note that this uses SEEK_HOLE/SEEK_DATA to locate the data available
> to be read from the cache. Whilst this is an improvement from the
> bmap interface, it still has a problem with regard to a modern
> extent-based filesystem inserting or removing bridging blocks of
> zeros.
What are the consequences from the point of view of a user?
--b.
>
> This is a step towards overhauling the fscache API. The change is opt-in
> on the part of the network filesystem. A netfs should not try to mix the
> old and the new API because of conflicting ways of handling pages and the
> PG_fscache page flag and because it would be mixing DIO with buffered I/O.
> Further, the helper library can't be used with the old API.
>
> This does not change any of the fscache cookie handling APIs or the way
> invalidation is done.
>
> In the near term, I intend to deprecate and remove the old I/O API
> (fscache_allocate_page{,s}(), fscache_read_or_alloc_page{,s}(),
> fscache_write_page() and fscache_uncache_page()) and eventually replace
> most of fscache/cachefiles with something simpler and easier to follow.
>
> The patchset contains four parts:
>
> (1) Some helper patches, including provision of an ITER_XARRAY iov
> iterator and a function to do readahead expansion.
>
> (2) Patches to add the netfs helper library.
>
> (3) A patch to add the fscache/cachefiles kiocb API
>
> (4) Patches to add support in AFS for this.
>
> With this, AFS without a cache passes all expected xfstests; with a cache,
> there's an extra failure, but that's also there before these patches.
> Fixing that probably requires a greater overhaul.
>
> These patches can be found also on:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=fscache-netfs-lib
>
> David
> ---
> David Howells (24):
> iov_iter: Add ITER_XARRAY
> vm: Add wait/unlock functions for PG_fscache
> mm: Implement readahead_control pageset expansion
> vfs: Export rw_verify_area() for use by cachefiles
> netfs: Make a netfs helper module
> netfs: Provide readahead and readpage netfs helpers
> netfs: Add tracepoints
> netfs: Gather stats
> netfs: Add write_begin helper
> netfs: Define an interface to talk to a cache
> fscache, cachefiles: Add alternate API to use kiocb for read/write to cache
> afs: Disable use of the fscache I/O routines
> afs: Pass page into dirty region helpers to provide THP size
> afs: Print the operation debug_id when logging an unexpected data version
> afs: Move key to afs_read struct
> afs: Don't truncate iter during data fetch
> afs: Log remote unmarshalling errors
> afs: Set up the iov_iter before calling afs_extract_data()
> afs: Use ITER_XARRAY for writing
> afs: Wait on PG_fscache before modifying/releasing a page
> afs: Extract writeback extension into its own function
> afs: Prepare for use of THPs
> afs: Use the fs operation ops to handle FetchData completion
> afs: Use new fscache read helper API
>
> Takashi Iwai (1):
> cachefiles: Drop superfluous readpages aops NULL check
>
>
> fs/Kconfig | 1 +
> fs/Makefile | 1 +
> fs/afs/Kconfig | 1 +
> fs/afs/dir.c | 225 ++++---
> fs/afs/file.c | 472 ++++----------
> fs/afs/fs_operation.c | 4 +-
> fs/afs/fsclient.c | 108 ++--
> fs/afs/inode.c | 7 +-
> fs/afs/internal.h | 57 +-
> fs/afs/rxrpc.c | 150 ++---
> fs/afs/write.c | 610 ++++++++++--------
> fs/afs/yfsclient.c | 82 +--
> fs/cachefiles/Makefile | 1 +
> fs/cachefiles/interface.c | 5 +-
> fs/cachefiles/internal.h | 9 +
> fs/cachefiles/rdwr.c | 2 -
> fs/cachefiles/rdwr2.c | 406 ++++++++++++
> fs/fscache/Makefile | 3 +-
> fs/fscache/internal.h | 3 +
> fs/fscache/page.c | 2 +-
> fs/fscache/page2.c | 116 ++++
> fs/fscache/stats.c | 1 +
> fs/internal.h | 5 -
> fs/netfs/Kconfig | 23 +
> fs/netfs/Makefile | 5 +
> fs/netfs/internal.h | 97 +++
> fs/netfs/read_helper.c | 1142 +++++++++++++++++++++++++++++++++
> fs/netfs/stats.c | 57 ++
> fs/read_write.c | 1 +
> include/linux/fs.h | 1 +
> include/linux/fscache-cache.h | 4 +
> include/linux/fscache.h | 28 +-
> include/linux/netfs.h | 167 +++++
> include/linux/pagemap.h | 16 +
> include/net/af_rxrpc.h | 2 +-
> include/trace/events/afs.h | 74 +--
> include/trace/events/netfs.h | 201 ++++++
> mm/filemap.c | 18 +
> mm/readahead.c | 70 ++
> net/rxrpc/recvmsg.c | 9 +-
> 40 files changed, 3171 insertions(+), 1015 deletions(-)
> create mode 100644 fs/cachefiles/rdwr2.c
> create mode 100644 fs/fscache/page2.c
> create mode 100644 fs/netfs/Kconfig
> create mode 100644 fs/netfs/Makefile
> create mode 100644 fs/netfs/internal.h
> create mode 100644 fs/netfs/read_helper.c
> create mode 100644 fs/netfs/stats.c
> create mode 100644 include/linux/netfs.h
> create mode 100644 include/trace/events/netfs.h
>
next prev parent reply other threads:[~2021-01-21 16:47 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-20 22:21 [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API David Howells
2021-01-20 22:21 ` [PATCH 01/25] iov_iter: Add ITER_XARRAY David Howells
2021-01-21 13:34 ` Matthew Wilcox
2021-01-21 14:54 ` David Howells
2021-01-20 22:21 ` [PATCH 02/25] vm: Add wait/unlock functions for PG_fscache David Howells
2021-01-20 22:21 ` [PATCH 03/25] mm: Implement readahead_control pageset expansion David Howells
2021-01-20 22:22 ` [PATCH 04/25] vfs: Export rw_verify_area() for use by cachefiles David Howells
2021-01-20 22:22 ` [PATCH 05/25] cachefiles: Drop superfluous readpages aops NULL check David Howells
2021-01-20 22:22 ` [PATCH 06/25] netfs: Make a netfs helper module David Howells
2021-01-20 22:22 ` [PATCH 07/25] netfs: Provide readahead and readpage netfs helpers David Howells
2021-01-20 22:23 ` [PATCH 08/25] netfs: Add tracepoints David Howells
2021-01-20 22:23 ` [PATCH 09/25] netfs: Gather stats David Howells
2021-01-20 22:23 ` [PATCH 10/25] netfs: Add write_begin helper David Howells
2021-01-20 22:23 ` [PATCH 11/25] netfs: Define an interface to talk to a cache David Howells
2021-01-20 22:23 ` [PATCH 12/25] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache David Howells
2021-01-20 22:23 ` [PATCH 13/25] afs: Disable use of the fscache I/O routines David Howells
2021-01-20 22:24 ` [PATCH 14/25] afs: Pass page into dirty region helpers to provide THP size David Howells
2021-01-20 22:24 ` [PATCH 15/25] afs: Print the operation debug_id when logging an unexpected data version David Howells
2021-01-20 22:24 ` [PATCH 16/25] afs: Move key to afs_read struct David Howells
2021-01-20 22:24 ` [PATCH 17/25] afs: Don't truncate iter during data fetch David Howells
2021-01-20 22:24 ` [PATCH 18/25] afs: Log remote unmarshalling errors David Howells
2021-01-20 22:25 ` [PATCH 19/25] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2021-01-20 22:25 ` [PATCH 20/25] afs: Use ITER_XARRAY for writing David Howells
2021-01-20 22:25 ` [PATCH 21/25] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2021-01-20 22:25 ` [PATCH 22/25] afs: Extract writeback extension into its own function David Howells
2021-01-20 22:25 ` [PATCH 23/25] afs: Prepare for use of THPs David Howells
2021-01-20 22:26 ` [PATCH 24/25] afs: Use the fs operation ops to handle FetchData completion David Howells
2021-01-20 22:26 ` [PATCH 25/25] afs: Use new fscache read helper API David Howells
2021-01-21 16:46 ` J. Bruce Fields [this message]
2021-01-21 17:02 ` [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API David Howells
2021-01-21 17:43 ` J. Bruce Fields
2021-01-21 18:55 ` David Howells
2021-01-21 19:09 ` J. Bruce Fields
2021-01-21 20:08 ` David Howells
2021-01-22 16:01 ` J. Bruce Fields
2021-01-22 16:06 ` David Howells
2021-01-22 8:23 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210121164645.GA20964@fieldses.org \
--to=bfields@fieldses.org \
--cc=anna.schumaker@netapp.com \
--cc=asmadeus@codewreck.org \
--cc=ceph-devel@vger.kernel.org \
--cc=dhowells@redhat.com \
--cc=dwysocha@redhat.com \
--cc=jlayton@redhat.com \
--cc=linux-afs@lists.infradead.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=sfrench@samba.org \
--cc=tiwai@suse.de \
--cc=trondmy@hammerspace.com \
--cc=v9fs-developer@lists.sourceforge.net \
--cc=viro@zeniv.linux.org.uk \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).