All of lore.kernel.org
 help / color / mirror / Atom feed
From: bfields@fieldses.org (J. Bruce Fields)
To: David Howells <dhowells@redhat.com>
Cc: Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Steve French <sfrench@samba.org>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Takashi Iwai <tiwai@suse.de>,
	Matthew Wilcox <willy@infradead.org>,
	linux-afs@lists.infradead.org, Jeff Layton <jlayton@redhat.com>,
	David Wysochanski <dwysocha@redhat.com>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	linux-cachefs@redhat.com, linux-nfs@vger.kernel.org,
	linux-cifs@vger.kernel.org, ceph-devel@vger.kernel.org,
	v9fs-developer@lists.sourceforge.net,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API
Date: Thu, 21 Jan 2021 11:46:45 -0500	[thread overview]
Message-ID: <20210121164645.GA20964@fieldses.org> (raw)
In-Reply-To: <161118128472.1232039.11746799833066425131.stgit@warthog.procyon.org.uk>

On Wed, Jan 20, 2021 at 10:21:24PM +0000, David Howells wrote:
>      Note that this uses SEEK_HOLE/SEEK_DATA to locate the data available
>      to be read from the cache.  Whilst this is an improvement from the
>      bmap interface, it still has a problem with regard to a modern
>      extent-based filesystem inserting or removing bridging blocks of
>      zeros.

What are the consequences from the point of view of a user?

--b.

> 
> This is a step towards overhauling the fscache API.  The change is opt-in
> on the part of the network filesystem.  A netfs should not try to mix the
> old and the new API because of conflicting ways of handling pages and the
> PG_fscache page flag and because it would be mixing DIO with buffered I/O.
> Further, the helper library can't be used with the old API.
> 
> This does not change any of the fscache cookie handling APIs or the way
> invalidation is done.
> 
> In the near term, I intend to deprecate and remove the old I/O API
> (fscache_allocate_page{,s}(), fscache_read_or_alloc_page{,s}(),
> fscache_write_page() and fscache_uncache_page()) and eventually replace
> most of fscache/cachefiles with something simpler and easier to follow.
> 
> The patchset contains four parts:
> 
>  (1) Some helper patches, including provision of an ITER_XARRAY iov
>      iterator and a function to do readahead expansion.
> 
>  (2) Patches to add the netfs helper library.
> 
>  (3) A patch to add the fscache/cachefiles kiocb API
> 
>  (4) Patches to add support in AFS for this.
> 
> With this, AFS without a cache passes all expected xfstests; with a cache,
> there's an extra failure, but that's also there before these patches.
> Fixing that probably requires a greater overhaul.
> 
> These patches can be found also on:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=fscache-netfs-lib
> 
> David
> ---
> David Howells (24):
>       iov_iter: Add ITER_XARRAY
>       vm: Add wait/unlock functions for PG_fscache
>       mm: Implement readahead_control pageset expansion
>       vfs: Export rw_verify_area() for use by cachefiles
>       netfs: Make a netfs helper module
>       netfs: Provide readahead and readpage netfs helpers
>       netfs: Add tracepoints
>       netfs: Gather stats
>       netfs: Add write_begin helper
>       netfs: Define an interface to talk to a cache
>       fscache, cachefiles: Add alternate API to use kiocb for read/write to cache
>       afs: Disable use of the fscache I/O routines
>       afs: Pass page into dirty region helpers to provide THP size
>       afs: Print the operation debug_id when logging an unexpected data version
>       afs: Move key to afs_read struct
>       afs: Don't truncate iter during data fetch
>       afs: Log remote unmarshalling errors
>       afs: Set up the iov_iter before calling afs_extract_data()
>       afs: Use ITER_XARRAY for writing
>       afs: Wait on PG_fscache before modifying/releasing a page
>       afs: Extract writeback extension into its own function
>       afs: Prepare for use of THPs
>       afs: Use the fs operation ops to handle FetchData completion
>       afs: Use new fscache read helper API
> 
> Takashi Iwai (1):
>       cachefiles: Drop superfluous readpages aops NULL check
> 
> 
>  fs/Kconfig                    |    1 +
>  fs/Makefile                   |    1 +
>  fs/afs/Kconfig                |    1 +
>  fs/afs/dir.c                  |  225 ++++---
>  fs/afs/file.c                 |  472 ++++----------
>  fs/afs/fs_operation.c         |    4 +-
>  fs/afs/fsclient.c             |  108 ++--
>  fs/afs/inode.c                |    7 +-
>  fs/afs/internal.h             |   57 +-
>  fs/afs/rxrpc.c                |  150 ++---
>  fs/afs/write.c                |  610 ++++++++++--------
>  fs/afs/yfsclient.c            |   82 +--
>  fs/cachefiles/Makefile        |    1 +
>  fs/cachefiles/interface.c     |    5 +-
>  fs/cachefiles/internal.h      |    9 +
>  fs/cachefiles/rdwr.c          |    2 -
>  fs/cachefiles/rdwr2.c         |  406 ++++++++++++
>  fs/fscache/Makefile           |    3 +-
>  fs/fscache/internal.h         |    3 +
>  fs/fscache/page.c             |    2 +-
>  fs/fscache/page2.c            |  116 ++++
>  fs/fscache/stats.c            |    1 +
>  fs/internal.h                 |    5 -
>  fs/netfs/Kconfig              |   23 +
>  fs/netfs/Makefile             |    5 +
>  fs/netfs/internal.h           |   97 +++
>  fs/netfs/read_helper.c        | 1142 +++++++++++++++++++++++++++++++++
>  fs/netfs/stats.c              |   57 ++
>  fs/read_write.c               |    1 +
>  include/linux/fs.h            |    1 +
>  include/linux/fscache-cache.h |    4 +
>  include/linux/fscache.h       |   28 +-
>  include/linux/netfs.h         |  167 +++++
>  include/linux/pagemap.h       |   16 +
>  include/net/af_rxrpc.h        |    2 +-
>  include/trace/events/afs.h    |   74 +--
>  include/trace/events/netfs.h  |  201 ++++++
>  mm/filemap.c                  |   18 +
>  mm/readahead.c                |   70 ++
>  net/rxrpc/recvmsg.c           |    9 +-
>  40 files changed, 3171 insertions(+), 1015 deletions(-)
>  create mode 100644 fs/cachefiles/rdwr2.c
>  create mode 100644 fs/fscache/page2.c
>  create mode 100644 fs/netfs/Kconfig
>  create mode 100644 fs/netfs/Makefile
>  create mode 100644 fs/netfs/internal.h
>  create mode 100644 fs/netfs/read_helper.c
>  create mode 100644 fs/netfs/stats.c
>  create mode 100644 include/linux/netfs.h
>  create mode 100644 include/trace/events/netfs.h
> 

  parent reply	other threads:[~2021-01-21 16:48 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-20 22:21 David Howells
2021-01-20 22:21 ` [PATCH 01/25] iov_iter: Add ITER_XARRAY David Howells
2021-01-21 13:34   ` Matthew Wilcox
2021-01-21 14:54   ` David Howells
2021-01-20 22:21 ` [PATCH 02/25] vm: Add wait/unlock functions for PG_fscache David Howells
2021-01-20 22:21 ` [PATCH 03/25] mm: Implement readahead_control pageset expansion David Howells
2021-01-20 22:22 ` [PATCH 04/25] vfs: Export rw_verify_area() for use by cachefiles David Howells
2021-01-20 22:22 ` [PATCH 05/25] cachefiles: Drop superfluous readpages aops NULL check David Howells
2021-01-20 22:22 ` [PATCH 06/25] netfs: Make a netfs helper module David Howells
2021-01-20 22:22 ` [PATCH 07/25] netfs: Provide readahead and readpage netfs helpers David Howells
2021-01-20 22:23 ` [PATCH 08/25] netfs: Add tracepoints David Howells
2021-01-20 22:23 ` [PATCH 09/25] netfs: Gather stats David Howells
2021-01-20 22:23 ` [PATCH 10/25] netfs: Add write_begin helper David Howells
2021-01-20 22:23 ` [PATCH 11/25] netfs: Define an interface to talk to a cache David Howells
2021-01-20 22:23 ` [PATCH 12/25] fscache, cachefiles: Add alternate API to use kiocb for read/write to cache David Howells
2021-01-20 22:23 ` [PATCH 13/25] afs: Disable use of the fscache I/O routines David Howells
2021-01-20 22:24 ` [PATCH 14/25] afs: Pass page into dirty region helpers to provide THP size David Howells
2021-01-20 22:24 ` [PATCH 15/25] afs: Print the operation debug_id when logging an unexpected data version David Howells
2021-01-20 22:24 ` [PATCH 16/25] afs: Move key to afs_read struct David Howells
2021-01-20 22:24 ` [PATCH 17/25] afs: Don't truncate iter during data fetch David Howells
2021-01-20 22:24 ` [PATCH 18/25] afs: Log remote unmarshalling errors David Howells
2021-01-20 22:25 ` [PATCH 19/25] afs: Set up the iov_iter before calling afs_extract_data() David Howells
2021-01-20 22:25 ` [PATCH 20/25] afs: Use ITER_XARRAY for writing David Howells
2021-01-20 22:25 ` [PATCH 21/25] afs: Wait on PG_fscache before modifying/releasing a page David Howells
2021-01-20 22:25 ` [PATCH 22/25] afs: Extract writeback extension into its own function David Howells
2021-01-20 22:25 ` [PATCH 23/25] afs: Prepare for use of THPs David Howells
2021-01-20 22:26 ` [PATCH 24/25] afs: Use the fs operation ops to handle FetchData completion David Howells
2021-01-20 22:26 ` [PATCH 25/25] afs: Use new fscache read helper API David Howells
2021-01-21 16:46 ` J. Bruce Fields [this message]
2021-01-21 17:02 ` [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API David Howells
2021-01-21 17:43   ` J. Bruce Fields
2021-01-21 18:55   ` David Howells
2021-01-21 19:09     ` J. Bruce Fields
2021-01-21 20:08     ` David Howells
2021-01-22 16:01       ` J. Bruce Fields
2021-01-22 16:06       ` David Howells
2021-01-22  8:23     ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210121164645.GA20964@fieldses.org \
    --to=bfields@fieldses.org \
    --cc=anna.schumaker@netapp.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=dhowells@redhat.com \
    --cc=dwysocha@redhat.com \
    --cc=jlayton@redhat.com \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=sfrench@samba.org \
    --cc=tiwai@suse.de \
    --cc=trondmy@hammerspace.com \
    --cc=v9fs-developer@lists.sourceforge.net \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    --subject='Re: [RFC][PATCH 00/25] Network fs helper library & fscache kiocb API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.