From: Amir Goldstein <amir73il@gmail.com>
To: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>,
Vivek Goyal <vgoyal@redhat.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>,
linux-fsdevel <linux-fsdevel@vger.kernel.org>,
linux-kernel <linux-kernel@vger.kernel.org>,
xieyongji@bytedance.com, fam.zheng@bytedance.com,
Miklos Szeredi <mszeredi@redhat.com>
Subject: Re: [PATCH] fuse: writeback_cache consistency enhancement (writeback_cache_v2)
Date: Tue, 28 Jun 2022 13:09:56 +0300 [thread overview]
Message-ID: <CAOQ4uxjbU27gSbhndxA_ABchfv1X7Steoggto_o-Wc=5shfS7Q@mail.gmail.com> (raw)
In-Reply-To: <20220624055825.29183-1-zhangjiachen.jaycee@bytedance.com>
On Fri, Jun 24, 2022 at 9:03 AM Jiachen Zhang
<zhangjiachen.jaycee@bytedance.com> wrote:
>
> Some users may want both the high performance of the writeback_cahe mode and
> a little bit more consistency among FUSE mounts. In the current writeback
> mode implementation, users of one FUSE mount can never see the file
> expansion done by other FUSE mounts.
>
> Based on the suggested writeback V2 patch in the upstream mailing-list [1],
> this commit allows the cmtime and size to be updated from server in
> writeback mode. Compared with the writeback V2 patch in [1], this commit has
> several differences:
>
> 1. Ensure c/mtime are not updated from kernel to server. IOW, the cmtime
> generated by kernel are just temporary values that are never flushed to
> server, and they can also be updated by the official server cmtime when
> the writeback cache is clean.
>
> 2. Skip mtime-based revalidation when fc->auto_inval_data is set with
> fc->writeback_cache_v2. Because the kernel-generated temporary cmtime
> are likely not equal to the offical server cmtime.
>
> 3. If any page is ever flushed to the server during FUSE_GETATTR
> handling on fuse server, even if the cache is clean when
> fuse_change_attributes() checks, we should not update the i_size. This
> is because the FUSE_GETATTR may get a staled size before the FUSE_WRITE
> request changes server inode size. This commit ensures this by
> increasing attr_version after writeback for writeback_cache_v2. In that
> case, we should also ensure the ordering of the attr_version updating
> and the fi->writepages RB-tree updating. So that if a fuse page
> writeback ever happens during fuse_change_attributes(), either the
> fi->writepages is not empty, or the attr_version is increased. So we
> never mistakenly update a stale file size from server to kernel.
>
> With this patch, writeback mode can consider the server c/mtime as the
> official one. When inode attr is timeout or invalidated, kernel has chance
> to see size and c/mtime modified by others.
>
> Together with another patch [2], a FUSE daemon is able to implement
> close-to-open (CTO) consistency like what is done in NFS clients.
>
> [1] https://lore.kernel.org/linux-fsdevel/Ymfu8fGbfYi4FxQ4@miu.piliscsaba.redhat.com
> [2] https://lore.kernel.org/linux-fsdevel/20220608104202.19461-1-zhangjiachen.jaycee@bytedance.com/
>
> Suggested-by: Miklos Szeredi <mszeredi@redhat.com>
> Signed-off-by: Jiachen Zhang <zhangjiachen.jaycee@bytedance.com>
> ---
> fs/fuse/file.c | 17 +++++++++++++++
> fs/fuse/fuse_i.h | 3 +++
> fs/fuse/inode.c | 44 +++++++++++++++++++++++++++++++++++++--
> include/uapi/linux/fuse.h | 5 +++++
> 4 files changed, 67 insertions(+), 2 deletions(-)
>
> diff --git a/fs/fuse/file.c b/fs/fuse/file.c
> index 9b64e2ff1c96..35bdc7af8468 100644
> --- a/fs/fuse/file.c
> +++ b/fs/fuse/file.c
> @@ -1829,6 +1829,15 @@ static void fuse_writepage_end(struct fuse_mount *fm, struct fuse_args *args,
> */
> fuse_send_writepage(fm, next, inarg->offset + inarg->size);
> }
> +
> + if (fc->writeback_cache_v2)
> + fi->attr_version = atomic64_inc_return(&fc->attr_version);
> + /*
> + * Ensure attr_version increases before the page is move out of the
> + * writepages rb-tree.
> + */
> + smp_mb();
> +
> fi->writectr--;
> fuse_writepage_finish(fm, wpa);
> spin_unlock(&fi->lock);
> @@ -1858,10 +1867,18 @@ static struct fuse_file *fuse_write_file_get(struct fuse_inode *fi)
>
> int fuse_write_inode(struct inode *inode, struct writeback_control *wbc)
> {
> + struct fuse_conn *fc = get_fuse_conn(inode);
> struct fuse_inode *fi = get_fuse_inode(inode);
> struct fuse_file *ff;
> int err;
>
> + /*
> + * Kernel c/mtime should not be updated to the server in the
> + * writeback_cache_v2 mode as server c/mtime are official.
> + */
> + if (fc->writeback_cache_v2)
> + return 0;
> +
> /*
> * Inode is always written before the last reference is dropped and
> * hence this should not be reached from reclaim.
> diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h
> index 488b460e046f..47de36146fb8 100644
> --- a/fs/fuse/fuse_i.h
> +++ b/fs/fuse/fuse_i.h
> @@ -654,6 +654,9 @@ struct fuse_conn {
> /* show legacy mount options */
> unsigned int legacy_opts_show:1;
>
> + /* Improved writeback cache policy */
> + unsigned writeback_cache_v2:1;
> +
Seeing that writeback_cache_v2 depends on writeback_cache
I wonder whether that will not be better represented as:
/** write-back cache policy (default is write-through) */
- unsigned writeback_cache:1;
+ unsigned writeback_cache:2;
Looking at the recently added handle_killpriv_v2, I also wonder
if that would not have been better.
Vivek,
is handle_killpriv_v2 really independent of handle_killpriv?
Seeing test like these worry me as they are inviting bugs:
if (!fc->handle_killpriv && !fc->handle_killpriv_v2) {
Thanks,
Amir.
next prev parent reply other threads:[~2022-06-28 10:10 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-24 5:58 [PATCH] fuse: writeback_cache consistency enhancement (writeback_cache_v2) Jiachen Zhang
2022-06-24 18:28 ` Vivek Goyal
2022-06-28 5:18 ` [External] " Jiachen Zhang
2022-06-29 17:24 ` Vivek Goyal
2022-06-28 10:09 ` Amir Goldstein [this message]
2022-06-29 18:37 ` Vivek Goyal
2022-06-29 17:57 ` Vivek Goyal
2022-07-18 5:17 ` Jiachen Zhang
2022-07-25 13:15 ` Vivek Goyal
2022-06-29 18:14 ` Vivek Goyal
2022-07-18 5:48 ` [External] " Jiachen Zhang
2022-07-25 13:30 ` Vivek Goyal
2022-07-15 10:07 ` Miklos Szeredi
2022-07-18 6:01 ` Jiachen Zhang
2022-07-18 8:18 ` Miklos Szeredi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOQ4uxjbU27gSbhndxA_ABchfv1X7Steoggto_o-Wc=5shfS7Q@mail.gmail.com' \
--to=amir73il@gmail.com \
--cc=fam.zheng@bytedance.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=mszeredi@redhat.com \
--cc=vgoyal@redhat.com \
--cc=xieyongji@bytedance.com \
--cc=zhangjiachen.jaycee@bytedance.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).