From: JeffleXu <jefflexu@linux.alibaba.com>
To: vgoyal@redhat.com, stefanha@redhat.com, miklos@szeredi.hub
Cc: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com,
bo.liu@linux.alibaba.com, joseph.qi@linux.alibaba.com
Subject: Re: [PATCH v6 0/7] fuse,virtiofs: support per-file DAX
Date: Fri, 15 Oct 2021 11:33:12 +0800 [thread overview]
Message-ID: <7b87b4c9-374c-f1db-37ac-d34002fc5a05@linux.alibaba.com> (raw)
In-Reply-To: <20211011030052.98923-1-jefflexu@linux.alibaba.com>
Hi, any comment?
On 10/11/21 11:00 AM, Jeffle Xu wrote:
> changes since v5:
> Overall Design Changes:
> 1. virtiofsd now supports ioctl (only FS_IOC_SETFLAGS and
> FS_IOC_FSSETXATTR), so that users inside guest could set/clear
> persistent inode flags now. (FUSE kernel module has already supported
> .ioctl(), virtiofsd need to suuport it.)
> 2. When FUSE client is mounted with '-o dax=inode', it indicates that
> whether DAX shall be enabled or not for one specific file is
> completely determined by FUSE server while FUSE client has no say on
> it, and the decision whether DAX shall be enabled or not for specific
> file is communicated through FUSE_ATTR_DAX flag of FUSE protocol. The
> algorithm used by virtiofsd to determine whether DAX shall be enabled
> or not is totally implementation specific, and thus the following
> scenario may exist: users inside guest has already set related persistent
> inode flag (i.e. FS_XFLAG_DAX) on corresponding file but FUSE server finnaly
> decides not to enable DAX for this file. This slight semantic difference
> is documented in patch 7. Also because of this, d_mark_dontcache() is
> not called when FS_IOC_SETFLAGS/FS_IOC_FSSETXATTR ioctl is done inside
> guest. It's delayed to be done if the FUSE_ATTR_DAX flag **indeed**
> changes (as showed in patch 6).
> 3. patch 1: slightly modify logic of fuse_should_enable_dax()
> 4. patch 4: add back negotiation during FUSE_INIT. FUSE client shall
> advertise to FUSE server that it's in per-file DAX mode, so that FUSE
> server may omit querying persistent inode flags on host if FUSE client
> is not mounted in per-file DAX mode, giving querying persistent inode
> flags could be quite expensive.
>
>
> chanegs since v4:
> - drop support for setting/clearing FS_DAX inside guest
> - and thus drop the negotiation phase during FUSE_INIT
>
> This patchset adds support of per-file DAX for virtiofs, which is
> inspired by Ira Weiny's work on ext4[1] and xfs[2].
>
> Any comment is welcome.
>
> [1] commit 9cb20f94afcd ("fs/ext4: Make DAX mount option a tri-state")
> [2] commit 02beb2686ff9 ("fs/xfs: Make DAX mount option a tri-state")
>
> [Purpose]
> DAX may be limited in some specific situation. When the number of usable
> DAX windows is under watermark, the recalim routine will be triggered to
> reclaim some DAX windows. It may have a negative impact on the
> performance, since some processes may need to wait for DAX windows to be
> recalimed and reused then. To mitigate the performance degradation, the
> overall DAX window need to be expanded larger.
>
> However, simply expanding the DAX window may not be a good deal in some
> scenario. To maintain one DAX window chunk (i.e., 2MB in size), 32KB
> (512 * 64 bytes) memory footprint will be consumed for page descriptors
> inside guest, which is greater than the memory footprint if it uses
> guest page cache when DAX disabled. Thus it'd better disable DAX for
> those files smaller than 32KB, to reduce the demand for DAX window and
> thus avoid the unworthy memory overhead.
>
> Per-file DAX feature is introduced to address this issue, by offering a
> finer grained control for dax to users, trying to achieve a balance
> between performance and memory overhead.
>
>
> [Note]
> When the per-file DAX hint changes while the file is still *opened*, it
> is quite complicated and maybe fragile to dynamically change the DAX
> state, since dynamic switching needs to switch a_ops atomiclly. Ira
> Weiny had ever implemented a so called i_aops_sem lock [3] but
> eventually gave up since the complexity of the implementation
> [4][5][6][7].
>
> Hence mark the inode and corresponding dentries as DONE_CACHE once the
> per-file DAX hint changes, so that the inode instance will be evicted
> and freed as soon as possible once the file is closed and the last
> reference to the inode is put. And then when the file gets reopened next
> time, the new instantiated inode will reflect the new DAX state.
>
> In summary, when the per-file DAX hint changes for an *opened* file, the
> DAX state of the file won't be updated until this file is closed and
> reopened later. This is also how ext4/xfs per-file DAX works.
>
> [3] https://lore.kernel.org/lkml/20200227052442.22524-7-ira.weiny@intel.com/
> [4] https://patchwork.kernel.org/project/xfs/cover/20200407182958.568475-1-ira.weiny@intel.com/
> [5] https://lore.kernel.org/lkml/20200305155144.GA5598@lst.de/
> [6] https://lore.kernel.org/lkml/20200401040021.GC56958@magnolia/
> [7] https://lore.kernel.org/lkml/20200403182904.GP80283@magnolia/
>
> changes since v3:
> - bug fix (patch 6): s/"IS_DAX(inode) != newdax"/"!!IS_DAX(inode) !=
> newdax"
> - during FUSE_INIT, advertise capability for per-file DAX only when
> mounted as "-o dax=inode" (patch 4)
>
> changes since v2:
> - modify fuse_show_options() accordingly to make it compatible with
> new tri-state mount option (patch 2)
> - extract FUSE protocol changes into one separate patch (patch 3)
> - FUSE server/client need to negotiate if they support per-file DAX
> (patch 4)
> - extract DONT_CACHE logic into patch 6/7
>
> v5: https://lore.kernel.org/all/20210923092526.72341-1-jefflexu@linux.alibaba.com/
> v4: https://lore.kernel.org/linux-fsdevel/20210817022220.17574-1-jefflexu@linux.alibaba.com/
> v3: https://www.spinics.net/lists/linux-fsdevel/msg200852.html
> v2: https://www.spinics.net/lists/linux-fsdevel/msg199584.html
> v1: https://www.spinics.net/lists/linux-virtualization/msg51008.html
>
>
> Jeffle Xu (7):
> fuse: add fuse_should_enable_dax() helper
> fuse: make DAX mount option a tri-state
> fuse: support per-file DAX in fuse protocol
> fuse: negotiate per-file DAX in FUSE_INIT
> fuse: enable per-file DAX
> fuse: mark inode DONT_CACHE when per-file DAX hint changes
> Documentation/filesystem/dax: record DAX on virtiofs
>
> Documentation/filesystems/dax.rst | 20 +++++++++++++++--
> fs/fuse/dax.c | 36 ++++++++++++++++++++++++++++---
> fs/fuse/file.c | 4 ++--
> fs/fuse/fuse_i.h | 19 ++++++++++++----
> fs/fuse/inode.c | 17 +++++++++++----
> fs/fuse/virtio_fs.c | 16 ++++++++++++--
> include/uapi/linux/fuse.h | 9 +++++++-
> 7 files changed, 103 insertions(+), 18 deletions(-)
>
--
Thanks,
Jeffle
next prev parent reply other threads:[~2021-10-15 3:33 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-11 3:00 [PATCH v6 0/7] fuse,virtiofs: support per-file DAX Jeffle Xu
2021-10-11 3:00 ` [PATCH v6 1/7] fuse: add fuse_should_enable_dax() helper Jeffle Xu
2021-10-11 3:00 ` [PATCH v6 2/7] fuse: make DAX mount option a tri-state Jeffle Xu
2021-10-18 14:10 ` Vivek Goyal
2021-10-20 2:52 ` JeffleXu
2021-10-20 14:48 ` Vivek Goyal
2021-10-29 8:33 ` JeffleXu
2021-10-29 13:03 ` Vivek Goyal
2021-11-01 8:21 ` JeffleXu
2021-10-20 15:17 ` Vivek Goyal
2021-10-22 6:54 ` JeffleXu
2021-10-25 17:52 ` Ira Weiny
2021-10-25 18:12 ` Vivek Goyal
2021-10-25 19:02 ` Ira Weiny
2021-10-25 19:33 ` Vivek Goyal
2021-10-25 20:41 ` Ira Weiny
2021-10-26 13:45 ` Vivek Goyal
2021-10-27 6:00 ` JeffleXu
2021-10-11 3:00 ` [PATCH v6 3/7] fuse: support per-file DAX in fuse protocol Jeffle Xu
2021-10-18 14:14 ` Vivek Goyal
2021-10-18 14:20 ` Vivek Goyal
2021-10-20 3:04 ` JeffleXu
2021-10-20 14:54 ` Vivek Goyal
2021-10-11 3:00 ` [PATCH v6 4/7] fuse: negotiate per-file DAX in FUSE_INIT Jeffle Xu
2021-10-18 14:30 ` Vivek Goyal
2021-10-20 3:10 ` JeffleXu
2021-10-20 15:44 ` Vivek Goyal
2021-10-11 3:00 ` [PATCH v6 5/7] fuse: enable per-file DAX Jeffle Xu
2021-10-18 15:11 ` Vivek Goyal
2021-10-11 3:00 ` [PATCH v6 6/7] fuse: mark inode DONT_CACHE when per-file DAX hint changes Jeffle Xu
2021-10-18 15:19 ` Vivek Goyal
2021-10-27 5:05 ` JeffleXu
2021-10-11 3:00 ` [PATCH v6 7/7] Documentation/filesystem/dax: record DAX on virtiofs Jeffle Xu
2021-10-15 3:33 ` JeffleXu [this message]
2021-10-18 15:21 ` [PATCH v6 0/7] fuse,virtiofs: support per-file DAX Vivek Goyal
2021-10-20 5:22 ` JeffleXu
2021-10-20 16:06 ` Vivek Goyal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7b87b4c9-374c-f1db-37ac-d34002fc5a05@linux.alibaba.com \
--to=jefflexu@linux.alibaba.com \
--cc=bo.liu@linux.alibaba.com \
--cc=joseph.qi@linux.alibaba.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=miklos@szeredi.hub \
--cc=stefanha@redhat.com \
--cc=vgoyal@redhat.com \
--cc=virtio-fs@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).