linux-unionfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Vivek Goyal <vgoyal@redhat.com>
To: Amir Goldstein <amir73il@gmail.com>
Cc: Chengguang Xu <cgxu519@mykernel.net>, Jan Kara <jack@suse.cz>,
	Miklos Szeredi <miklos@szeredi.hu>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	overlayfs <linux-unionfs@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	ronyjin <ronyjin@tencent.com>,
	charliecgxu <charliecgxu@tencent.com>
Subject: Re: ovl_flush() behavior
Date: Thu, 2 Dec 2021 10:14:28 -0500	[thread overview]
Message-ID: <Yaji1C/wK73jAkho@redhat.com> (raw)
In-Reply-To: <CAOQ4uxg6FATciQhzRifOft4gMZj15G=UA6MUiPX2n9-NR5+1Pg@mail.gmail.com>

On Thu, Dec 02, 2021 at 01:23:17AM +0200, Amir Goldstein wrote:
> > >
> > > To be honest I even don't fully understand what's the ->flush() logic in overlayfs.
> > > Why should we open new underlying file when calling ->flush()?
> > > Is it still correct in the case of opening lower layer first then copy-uped case?
> > >
> >
> > The semantics of flush() are far from being uniform across filesystems.
> > most local filesystems do nothing on close.
> > most network fs only flush dirty data when a writer closes a file
> > but not when a reader closes a file.
> > It is hard to imagine that applications rely on flush-on-close of
> > rdonly fd behavior and I agree that flushing only if original fd was upper
> > makes more sense, so I am not sure if it is really essential for
> > overlayfs to open an upper rdonly fd just to do whatever the upper fs
> > would have done on close of rdonly fd, but maybe there is no good
> > reason to change this behavior either.
> >
> 
> On second thought, I think there may be a good reason to change
> ovl_flush() otherwise I wouldn't have submitted commit
> a390ccb316be ("fuse: add FOPEN_NOFLUSH") - I did observe
> applications that frequently open short lived rdonly fds and suffered
> undesired latencies on close().
> 
> As for "changing existing behavior", I think that most fs used as
> upper do not implement flush at all.
> Using fuse/virtiofs as overlayfs upper is quite new, so maybe that
> is not a problem and maybe the new behavior would be preferred
> for those users?

It probably will be nice not to send flush to fuse server when it is not
required.

Right now in virtiofsd, I see that we are depending on flush being sent
as we are dealing with remote posix lock magic. I am supporting remotme
posix locks in virtiofs and virtiofsd is building these on top of open
file description locks on host. (Can't use posix locks on host as these
locks are per process and virtiofsd is single process working on behalf
of all the guest processes, and unexpected things happen).

When an fd is being closed, flush request is sent and along with it we
also send "lock_owner".

inarg.lock_owner = fuse_lock_owner_id(fm->fc, id);

We basically use this to keep track which process is closing the fd and
release associated OFD locks on host. /me needs to dive into details
to explain it better. Will do that if need be.

Bottom line is that as of now virtiofsd seems to be relying on receiving
FLUSH requests when remote posix locks are enabled. Maybe we can set
FOPEN_NOFLUSH when remote posix locks are not enabled.

Thanks
Vivek


  parent reply	other threads:[~2021-12-02 15:14 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-23 13:08 [RFC PATCH v5 00/10] implement containerized syncfs for overlayfs Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 01/10] ovl: setup overlayfs' private bdi Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 02/10] ovl: implement ->writepages operation Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 03/10] ovl: implement overlayfs' ->evict_inode operation Chengguang Xu
2021-10-06 15:33   ` Miklos Szeredi
2021-10-07  6:08     ` Chengguang Xu
2021-10-07  7:43       ` Miklos Szeredi
2021-09-23 13:08 ` [RFC PATCH v5 04/10] ovl: mark overlayfs' inode dirty on modification Chengguang Xu
2021-10-07 18:43   ` Miklos Szeredi
2021-09-23 13:08 ` [RFC PATCH v5 05/10] ovl: mark overlayfs' inode dirty on shared mmap Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 06/10] ovl: implement overlayfs' ->write_inode operation Chengguang Xu
2021-10-07  9:01   ` Jan Kara
2021-10-07 12:26     ` Chengguang Xu
2021-10-07 14:41       ` Jan Kara
2021-10-07 14:54         ` Chengguang Xu
2021-10-07  9:23   ` Miklos Szeredi
2021-10-07 12:28     ` Chengguang Xu
2021-10-07 12:45       ` Miklos Szeredi
2021-10-07 13:09         ` Chengguang Xu
2021-10-07 13:34           ` Miklos Szeredi
2021-10-07 14:46             ` Jan Kara
2021-10-07 14:53               ` Chengguang Xu
2021-10-07 18:51                 ` Miklos Szeredi
2021-10-08 13:13                   ` Jan Kara
2021-11-16  2:20             ` Chengguang Xu
2021-11-16 12:35               ` Miklos Szeredi
2021-11-17  6:11                 ` Chengguang Xu
2021-11-18  6:32                   ` Chengguang Xu
2021-11-18 11:23                     ` Jan Kara
2021-11-18 12:02                       ` Chengguang Xu
2021-11-18 16:43                         ` Jan Kara
2021-11-19  6:12                           ` Chengguang Xu
2021-11-30 11:22                             ` Jan Kara
2021-11-30 16:09                               ` Chengguang Xu
2021-11-30 19:04                                 ` Amir Goldstein
2021-12-01  2:37                                   ` Chengguang Xu
2021-12-01  6:31                                     ` Chengguang Xu
2021-12-01  7:19                                       ` Amir Goldstein
2021-12-01 13:46                                         ` Jan Kara
2021-12-01 14:59                                           ` Chengguang Xu
2021-12-01 16:24                                           ` Chengguang Xu
2021-12-01 22:47                                             ` Amir Goldstein
2021-12-01 23:23                                               ` ovl_flush() behavior Amir Goldstein
2021-12-02  2:11                                                 ` Chengguang Xu
2021-12-02 15:20                                                   ` Vivek Goyal
2021-12-02 15:59                                                     ` Amir Goldstein
2021-12-02 22:00                                                       ` Vivek Goyal
2021-12-02 15:14                                                 ` Vivek Goyal [this message]
2021-12-05 14:06                                               ` [RFC PATCH v5 06/10] ovl: implement overlayfs' ->write_inode operation Chengguang Xu
2021-12-07  5:33                                                 ` Amir Goldstein
2022-02-05 16:09                                                   ` Chengguang Xu
2022-02-05 16:23                                                     ` Amir Goldstein
2021-09-23 13:08 ` [RFC PATCH v5 07/10] ovl: cache dirty overlayfs' inode Chengguang Xu
2021-10-07 11:09   ` Miklos Szeredi
2021-10-07 12:04     ` Chengguang Xu
2021-10-07 12:27       ` Miklos Szeredi
2021-09-23 13:08 ` [RFC PATCH v5 08/10] fs: export wait_sb_inodes() Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 09/10] fs: introduce new helper sync_fs_and_blockdev() Chengguang Xu
2021-10-19  7:15   ` Amir Goldstein
2021-11-15 11:39     ` Chengguang Xu
2021-09-23 13:08 ` [RFC PATCH v5 10/10] ovl: implement containerized syncfs for overlayfs Chengguang Xu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Yaji1C/wK73jAkho@redhat.com \
    --to=vgoyal@redhat.com \
    --cc=amir73il@gmail.com \
    --cc=cgxu519@mykernel.net \
    --cc=charliecgxu@tencent.com \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-unionfs@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=ronyjin@tencent.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).