Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: Dave Wysochanski <dwysocha@redhat.com>
To: Trond Myklebust <trondmy@hammerspace.com>,
	Anna Schumaker <anna.schumaker@netapp.com>
Cc: linux-nfs@vger.kernel.org, linux-cachefs@redhat.com
Subject: [RFC PATCH v2 0/14] Convert NFS to new FS-Cache iter API
Date: Wed, 29 Jul 2020 10:12:15 -0400
Message-ID: <1596031949-26793-1-git-send-email-dwysocha@redhat.com> (raw)

These patches update the nfs client to use the new FS-Cache API and are at:
https://github.com/DaveWysochanskiRH/kernel/tree/fscache-iter-nfs
https://github.com/DaveWysochanskiRH/kernel/commit/467796a0c75d64401c8963e9266f27d87863ed3e

They are based on David Howells fscache-iter tree at 757ac8b16a0edd3befa15c9bdcb2ab3811be945d
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/commit/?h=fscache-iter&id=757ac8b16a0edd3befa15c9bdcb2ab3811be945d

The first 5 patches refactor some of the NFS read code to facilitate
re-use, while the last 9 patches do the conversion to the new FS-Cache
API.

Changes since v1
* Refactor and general cleanup of read code paths
* Fixes a few build errors when NFS_FSCACHE is not configured
* Fixes directIO data corruption (needed fscache_invalidate() on directIO write)

Summary
* Takes a "least invasive to existing code" approach
  * most fscache bits stay fs/nfs/fscache.[ch]
  * only enable fscache for files open for READ (disable for WRITE)
  * may not be the best approach (see future patcheset items below)
* Basically works and passes a series of tests (see below)
  * No kernel oopses or hangs seen with tests run

Future patchset items
* Call fscache_read_helper_* directly rather than isolation into
  fs/nfs/fscache.c, similar to the AFS conversion
* Add write-through support
  * Eliminate on/off switching of fscache based on whether a
  file is open for read or write
  * TODO: Work out any limitations of NFS versions
* Rework dfprintks and/or add ftrace points
  * fscache/cachefiles has 'debug' logging similar to rpcdebug
  * convert IO path to ftrace, leave non-IO path as dfprintk?

Tests run
* A few individual NFS/fscache unit tests: PASS
* cthon04 (fsc/non-fsc, vers=3,4.0,4.1,4.2, sec=sys): PASS
* iozone tests (fsc, vers=3,4.0,4.1,4.2, sec=sys): PASS
* xfstests/generic (fsc,vers=4.2): 17/151 (Failed/Ran) 595/444 (Total/NotRan)
Failures: generic/029 generic/030 generic/240 generic/294 generic/306 generic/356 generic/357 generic/452 generic/472 generic/493 generic/494 generic/495 generic/496 generic/497 generic/554 generic/568 generic/569
Failed 17 of 595 tests
* kernel build: FAIL (linking module fails; truncate / invalidate related?)

Test not run
* error injections (for example, connection disruptions, server errors during IO, etc)
* pNFS
* many process mixed read/write on same file
* sec=krb5

Dave Wysochanski (14):
  NFS: Clean up nfs_readpage() and nfs_readpages()
  NFS: In nfs_readpage() only increment NFSIOS_READPAGES when read
    succeeds
  NFS: Refactor nfs_readpage() and nfs_readpage_async() to use
    nfs_readdesc
  NFS: Call readpage_async_filler() from nfs_readpage_async()
  NFS: Add nfs_pageio_complete_read() and remove nfs_readpage_async()
  NFS: Allow internal use of read structs and functions
  NFS: Convert nfs_readpage() and readpages() to new fscache API
  NFS: Convert fscache_acquire_cookie and fscache_relinquish_cookie
  NFS: Only use and unuse an fscache cookie a single time based on
    NFS_INO_FSCACHE
  NFS: Convert fscache invalidation and update aux_data and i_size
  NFS: Call nfs_fscache_invalidate() when write extends the size of the
    file
  NFS: Invalidate fscache for direct writes
  NFS: Call fscache_resize_cookie() when inode size changes due to
    setattr
  NFS: Allow NFS use of new fscache API in build

 fs/nfs/Kconfig           |   2 +-
 fs/nfs/direct.c          |   2 +
 fs/nfs/file.c            |  20 +--
 fs/nfs/fscache-index.c   |  94 --------------
 fs/nfs/fscache.c         | 309 +++++++++++++++++++++++------------------------
 fs/nfs/fscache.h         |  99 ++++++---------
 fs/nfs/inode.c           |   4 +-
 fs/nfs/internal.h        |   9 ++
 fs/nfs/nfs4proc.c        |   2 +-
 fs/nfs/pagelist.c        |   1 +
 fs/nfs/read.c            | 217 +++++++++++++++------------------
 fs/nfs/write.c           |   3 +-
 include/linux/nfs_fs.h   |   2 -
 include/linux/nfs_page.h |   1 +
 include/linux/nfs_xdr.h  |   1 +
 15 files changed, 316 insertions(+), 450 deletions(-)

-- 
1.8.3.1


             reply index

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-29 14:12 Dave Wysochanski [this message]
2020-07-29 14:12 ` [RFC PATCH v2 01/14] NFS: Clean up nfs_readpage() and nfs_readpages() Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 02/14] NFS: In nfs_readpage() only increment NFSIOS_READPAGES when read succeeds Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 03/14] NFS: Refactor nfs_readpage() and nfs_readpage_async() to use nfs_readdesc Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 04/14] NFS: Call readpage_async_filler() from nfs_readpage_async() Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 05/14] NFS: Add nfs_pageio_complete_read() and remove nfs_readpage_async() Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 06/14] NFS: Allow internal use of read structs and functions Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 07/14] NFS: Convert nfs_readpage() and readpages() to new fscache API Dave Wysochanski
2020-08-04 17:41   ` [Linux-cachefs] " David Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 08/14] NFS: Convert fscache_acquire_cookie and fscache_relinquish_cookie Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 09/14] NFS: Only use and unuse an fscache cookie a single time based on NFS_INO_FSCACHE Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 10/14] NFS: Convert fscache invalidation and update aux_data and i_size Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 11/14] NFS: Call nfs_fscache_invalidate() when write extends the size of the file Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 12/14] NFS: Invalidate fscache for direct writes Dave Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 13/14] NFS: Call fscache_resize_cookie() when inode size changes due to setattr Dave Wysochanski
2020-07-30 18:39   ` [Linux-cachefs] " Jeff Layton
2020-07-30 19:23     ` David Wysochanski
2020-07-30 19:59       ` David Wysochanski
2020-07-30 20:03       ` David Howells
2020-07-30 21:07         ` David Wysochanski
2020-07-29 14:12 ` [RFC PATCH v2 14/14] NFS: Allow NFS use of new fscache API in build Dave Wysochanski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1596031949-26793-1-git-send-email-dwysocha@redhat.com \
    --to=dwysocha@redhat.com \
    --cc=anna.schumaker@netapp.com \
    --cc=linux-cachefs@redhat.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trondmy@hammerspace.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org
	public-inbox-index linux-nfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git