From: Jan Kara <jack@suse.cz>
To: Jeff Layton <jlayton@redhat.com>
Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
linux-mm@kvack.org, jfs-discussion@lists.sourceforge.net,
linux-xfs@vger.kernel.org, cluster-devel@redhat.com,
linux-f2fs-devel@lists.sourceforge.net,
v9fs-developer@lists.sourceforge.net,
linux-nilfs@vger.kernel.org, linux-block@vger.kernel.org,
dhowells@redhat.com, akpm@linux-foundation.org,
hch@infradead.org, ross.zwisler@linux.intel.com,
mawilcox@microsoft.com, jack@suse.com, viro@zeniv.linux.org.uk,
corbet@lwn.net, neilb@suse.de, clm@fb.com, tytso@mit.edu,
axboe@kernel.dk, josef@toxicpanda.com, hubcap@omnibond.com,
rpeterso@redhat.com, bo.li.liu@oracle.com
Subject: Re: [PATCH v4 14/27] fs: new infrastructure for writeback error handling and reporting
Date: Wed, 10 May 2017 13:48:40 +0200 [thread overview]
Message-ID: <20170510114840.GF25137@quack2.suse.cz> (raw)
In-Reply-To: <20170509154930.29524-15-jlayton@redhat.com>
On Tue 09-05-17 11:49:17, Jeff Layton wrote:
> Most filesystems currently use mapping_set_error and
> filemap_check_errors for setting and reporting/clearing writeback errors
> at the mapping level. filemap_check_errors is indirectly called from
> most of the filemap_fdatawait_* functions and from
> filemap_write_and_wait*. These functions are called from all sorts of
> contexts to wait on writeback to finish -- e.g. mostly in fsync, but
> also in truncate calls, getattr, etc.
>
> The non-fsync callers are problematic. We should be reporting writeback
> errors during fsync, but many places spread over the tree clear out
> errors before they can be properly reported, or report errors at
> nonsensical times.
>
> If I get -EIO on a stat() call, there is no reason for me to assume that
> it is because some previous writeback failed. The fact that it also
> clears out the error such that a subsequent fsync returns 0 is a bug,
> and a nasty one since that's potentially silent data corruption.
>
> This patch adds a small bit of new infrastructure for setting and
> reporting errors during address_space writeback. While the above was my
> original impetus for adding this, I think it's also the case that
> current fsync semantics are just problematic for userland. Most
> applications that call fsync do so to ensure that the data they wrote
> has hit the backing store.
>
> In the case where there are multiple writers to the file at the same
> time, this is really hard to determine. The first one to call fsync will
> see any stored error, and the rest get back 0. The processes with open
> fds may not be associated with one another in any way. They could even
> be in different containers, so ensuring coordination between all fsync
> callers is not really an option.
>
> One way to remedy this would be to track what file descriptor was used
> to dirty the file, but that's rather cumbersome and would likely be
> slow. However, there is a simpler way to improve the semantics here
> without incurring too much overhead.
>
> This set adds an errseq_t to struct address_space, and a corresponding
> one is added to struct file. Writeback errors are recorded in the
> mapping's errseq_t, and the one in struct file is used as the "since"
> value.
>
> This changes the semantics of the Linux fsync implementation such that
> applications can now use it to determine whether there were any
> writeback errors since fsync(fd) was last called (or since the file was
> opened in the case of fsync having never been called).
>
> Note that those writeback errors may have occurred when writing data
> that was dirtied via an entirely different fd, but that's the case now
> with the current mapping_set_error/filemap_check_error infrastructure.
> This will at least prevent you from getting a false report of success.
>
> The new behavior is still consistent with the POSIX spec, and is more
> reliable for application developers. This patch just adds some basic
> infrastructure for doing this. Later patches will change the existing
> code to use this new infrastructure.
>
> Signed-off-by: Jeff Layton <jlayton@redhat.com>
Just one nit below. Otherwise the patch looks good to me. You can add:
Reviewed-by: Jan Kara <jack@suse.cz>
> diff --git a/fs/file_table.c b/fs/file_table.c
> index 954d510b765a..d6138b6411ff 100644
> --- a/fs/file_table.c
> +++ b/fs/file_table.c
> @@ -168,6 +168,7 @@ struct file *alloc_file(const struct path *path, fmode_t mode,
> file->f_path = *path;
> file->f_inode = path->dentry->d_inode;
> file->f_mapping = path->dentry->d_inode->i_mapping;
> + file->f_wb_err = filemap_sample_wb_error(file->f_mapping);
Why do you sample here when you also sample in do_dentry_open()? I didn't
find any alloc_file() callers that would possibly care about writeback
errors...
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
next prev parent reply other threads:[~2017-05-10 11:58 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-09 15:49 [PATCH v4 00/27] fs: introduce new writeback error reporting and convert existing API as a wrapper around it Jeff Layton
2017-05-09 15:49 ` [PATCH v4 01/27] fs: remove unneeded forward definition of mm_struct from fs.h Jeff Layton
2017-05-10 11:04 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 02/27] mm: drop "wait" parameter from write_one_page Jeff Layton
2017-05-09 15:49 ` [PATCH v4 03/27] mm: fix mapping_set_error call in me_pagecache_dirty Jeff Layton
2017-05-09 15:49 ` [PATCH v4 04/27] buffer: use mapping_set_error instead of setting the flag Jeff Layton
2017-05-09 15:49 ` [PATCH v4 05/27] btrfs: btrfs_wait_tree_block_writeback can be void return Jeff Layton
2017-05-10 11:09 ` Jan Kara
2017-05-19 4:07 ` Liu Bo
2017-05-09 15:49 ` [PATCH v4 06/27] fs: check for writeback errors after syncing out buffers in generic_file_fsync Jeff Layton
2017-05-10 12:48 ` Matthew Wilcox
2017-05-09 15:49 ` [PATCH v4 07/27] orangefs: don't call filemap_write_and_wait from fsync Jeff Layton
2017-05-09 15:49 ` [PATCH v4 08/27] dax: set errors in mapping when writeback fails Jeff Layton
2017-05-09 15:49 ` [PATCH v4 09/27] nilfs2: set the mapping error when calling SetPageError on writeback Jeff Layton
2017-05-09 15:49 ` [PATCH v4 10/27] 9p: set mapping error when writeback fails in launder_page Jeff Layton
2017-05-09 15:49 ` [PATCH v4 11/27] fuse: set mapping error in writepage_locked when it fails Jeff Layton
2017-05-10 11:13 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 12/27] cifs: set mapping error when page writeback fails in writepage or launder_pages Jeff Layton
2017-05-10 11:14 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 13/27] lib: add errseq_t type and infrastructure for handling it Jeff Layton
2017-05-09 22:03 ` NeilBrown
2017-05-10 11:29 ` Jeff Layton
2017-05-10 11:34 ` Jan Kara
2017-05-10 11:58 ` Jeff Layton
2017-05-10 14:18 ` Matthew Wilcox
2017-05-10 14:56 ` Jeff Layton
2017-05-09 15:49 ` [PATCH v4 14/27] fs: new infrastructure for writeback error handling and reporting Jeff Layton
2017-05-10 11:48 ` Jan Kara [this message]
2017-05-10 12:19 ` Jeff Layton
2017-05-10 13:46 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 15/27] fs: retrofit old error reporting API onto new infrastructure Jeff Layton
2017-05-15 10:42 ` Jan Kara
2017-05-15 17:58 ` Jeff Layton
2017-05-19 19:20 ` Jeff Layton
2017-05-22 13:38 ` Jan Kara
2017-05-22 13:53 ` Jeff Layton
2017-05-22 17:53 ` Jan Kara
2017-05-22 19:09 ` Jeff Layton
2017-05-23 9:05 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 16/27] fs: adapt sync_file_range to new reporting infrastructure Jeff Layton
2017-05-09 15:49 ` [PATCH v4 17/27] mm: remove AS_EIO and AS_ENOSPC flags Jeff Layton
2017-05-09 15:49 ` [PATCH v4 18/27] mm: don't TestClearPageError in __filemap_fdatawait_range Jeff Layton
2017-05-09 15:49 ` [PATCH v4 19/27] buffer: set errors in mapping at the time that the error occurs Jeff Layton
2017-05-15 11:53 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 20/27] cifs: cleanup writeback handling errors and comments Jeff Layton
2017-05-09 15:49 ` [PATCH v4 21/27] mm: clean up error handling in write_one_page Jeff Layton
2017-05-15 12:01 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 22/27] jbd2: don't reset error in journal_finish_inode_data_buffers Jeff Layton
2017-05-15 11:58 ` Jan Kara
2017-05-09 15:49 ` [PATCH v4 23/27] gfs2: clean up some filemap_* calls Jeff Layton
2017-05-10 16:18 ` Bob Peterson
2017-05-09 15:49 ` [PATCH v4 24/27][RFC] nfs: convert to new errseq_t based error tracking for writeback errors Jeff Layton
2017-05-09 15:49 ` [PATCH v4 25/27] Documentation: flesh out the section in vfs.txt on storing and reporting " Jeff Layton
2017-05-09 16:24 ` Jeff Layton
2017-05-09 15:49 ` [PATCH v4 26/27] mm: flesh out comments over mapping_set_error Jeff Layton
2017-05-09 15:49 ` [PATCH v4 27/27] mm: clean up comments in me_pagecache_dirty Jeff Layton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170510114840.GF25137@quack2.suse.cz \
--to=jack@suse.cz \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=bo.li.liu@oracle.com \
--cc=clm@fb.com \
--cc=cluster-devel@redhat.com \
--cc=corbet@lwn.net \
--cc=dhowells@redhat.com \
--cc=hch@infradead.org \
--cc=hubcap@omnibond.com \
--cc=jack@suse.com \
--cc=jfs-discussion@lists.sourceforge.net \
--cc=jlayton@redhat.com \
--cc=josef@toxicpanda.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=linux-nilfs@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=mawilcox@microsoft.com \
--cc=neilb@suse.de \
--cc=ross.zwisler@linux.intel.com \
--cc=rpeterso@redhat.com \
--cc=tytso@mit.edu \
--cc=v9fs-developer@lists.sourceforge.net \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).