From: Jeffle Xu <jefflexu@linux.alibaba.com> To: vgoyal@redhat.com, stefanha@redhat.com, miklos@szeredi.hu Cc: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, bo.liu@linux.alibaba.com, joseph.qi@linux.alibaba.com Subject: [virtiofsd PATCH v5 2/2] virtiofsd: support per-file DAX Date: Thu, 23 Sep 2021 17:35:45 +0800 [thread overview] Message-ID: <20210923093545.81512-3-jefflexu@linux.alibaba.com> (raw) In-Reply-To: <20210923093545.81512-1-jefflexu@linux.alibaba.com> When DAX window is fully utilized and needs to be exapnded to avoid the performance fluctuation triggered by DAX window recaliming, it may not be wise to allocate DAX window for files with size smaller than some specific point, considering from the perspective of reducing memory overhead. To maintain one DAX window chunk (e.g., 2MB in size), 32KB (512 * 64 bytes) memory footprint will be consumed for page descriptors inside guest. Thus it'd better disable DAX for those files smaller than 32KB, to reduce the demand for DAX window and thus avoid the unworthy memory overhead. Thus only flag the file with FUSE_ATTR_DAX when the file size is greater than 32 KB. The guest will enable DAX only for those files flagged with FUSE_ATTR_DAX, when virtiofs is mounted with '-o dax=inode'. To be noted that both FUSE_LOOKUP and FUSE_READDIRPLUS are affected, and will convey FUSE_ATTR_DAX flag to the guest. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- tools/virtiofsd/passthrough_ll.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index b76d878509..6893180c6b 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -53,12 +53,26 @@ #include <sys/syscall.h> #include <sys/wait.h> #include <sys/xattr.h> +#include <sys/user.h> #include <syslog.h> #include "qemu/cutils.h" #include "passthrough_helpers.h" #include "passthrough_seccomp.h" +/* + * One page descriptor (64 bytes in size) needs to be maintained for every page + * in the DAX window chunk, i.e., there is certain guest memory overhead when + * DAX is enabled. Thus disable DAX for those files smaller than this certain + * memory overhead if virtiofs is mounted in per-file DAX mode, in which case + * the guest page cache will consume less memory when DAX is disabled. + */ +#define FUSE_DAX_SHIFT 21 +#define PAGE_DESC_SHIFT 6 +#define FUSE_PERFILE_DAX_SHIFT \ + (FUSE_DAX_SHIFT - PAGE_SHIFT + PAGE_DESC_SHIFT) +#define FUSE_PERFILE_DAX_POINT (1 << FUSE_PERFILE_DAX_SHIFT) + /* Keep track of inode posix locks for each owner. */ struct lo_inode_plock { uint64_t lock_owner; @@ -1023,6 +1037,11 @@ static int lo_do_lookup(fuse_req_t req, fuse_ino_t parent, const char *name, e->attr_flags |= FUSE_ATTR_SUBMOUNT; } + if (S_ISREG(e->attr.st_mode) && + (e->attr.st_size > FUSE_PERFILE_DAX_POINT)) { + e->attr_flags |= FUSE_ATTR_DAX; + } + inode = lo_find(lo, &e->attr, mnt_id); if (inode) { close(newfd); -- 2.27.0
WARNING: multiple messages have this Message-ID (diff)
From: Jeffle Xu <jefflexu@linux.alibaba.com> To: vgoyal@redhat.com, stefanha@redhat.com, miklos@szeredi.hu Cc: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, joseph.qi@linux.alibaba.com Subject: [Virtio-fs] [virtiofsd PATCH v5 2/2] virtiofsd: support per-file DAX Date: Thu, 23 Sep 2021 17:35:45 +0800 [thread overview] Message-ID: <20210923093545.81512-3-jefflexu@linux.alibaba.com> (raw) In-Reply-To: <20210923093545.81512-1-jefflexu@linux.alibaba.com> When DAX window is fully utilized and needs to be exapnded to avoid the performance fluctuation triggered by DAX window recaliming, it may not be wise to allocate DAX window for files with size smaller than some specific point, considering from the perspective of reducing memory overhead. To maintain one DAX window chunk (e.g., 2MB in size), 32KB (512 * 64 bytes) memory footprint will be consumed for page descriptors inside guest. Thus it'd better disable DAX for those files smaller than 32KB, to reduce the demand for DAX window and thus avoid the unworthy memory overhead. Thus only flag the file with FUSE_ATTR_DAX when the file size is greater than 32 KB. The guest will enable DAX only for those files flagged with FUSE_ATTR_DAX, when virtiofs is mounted with '-o dax=inode'. To be noted that both FUSE_LOOKUP and FUSE_READDIRPLUS are affected, and will convey FUSE_ATTR_DAX flag to the guest. Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> --- tools/virtiofsd/passthrough_ll.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tools/virtiofsd/passthrough_ll.c b/tools/virtiofsd/passthrough_ll.c index b76d878509..6893180c6b 100644 --- a/tools/virtiofsd/passthrough_ll.c +++ b/tools/virtiofsd/passthrough_ll.c @@ -53,12 +53,26 @@ #include <sys/syscall.h> #include <sys/wait.h> #include <sys/xattr.h> +#include <sys/user.h> #include <syslog.h> #include "qemu/cutils.h" #include "passthrough_helpers.h" #include "passthrough_seccomp.h" +/* + * One page descriptor (64 bytes in size) needs to be maintained for every page + * in the DAX window chunk, i.e., there is certain guest memory overhead when + * DAX is enabled. Thus disable DAX for those files smaller than this certain + * memory overhead if virtiofs is mounted in per-file DAX mode, in which case + * the guest page cache will consume less memory when DAX is disabled. + */ +#define FUSE_DAX_SHIFT 21 +#define PAGE_DESC_SHIFT 6 +#define FUSE_PERFILE_DAX_SHIFT \ + (FUSE_DAX_SHIFT - PAGE_SHIFT + PAGE_DESC_SHIFT) +#define FUSE_PERFILE_DAX_POINT (1 << FUSE_PERFILE_DAX_SHIFT) + /* Keep track of inode posix locks for each owner. */ struct lo_inode_plock { uint64_t lock_owner; @@ -1023,6 +1037,11 @@ static int lo_do_lookup(fuse_req_t req, fuse_ino_t parent, const char *name, e->attr_flags |= FUSE_ATTR_SUBMOUNT; } + if (S_ISREG(e->attr.st_mode) && + (e->attr.st_size > FUSE_PERFILE_DAX_POINT)) { + e->attr_flags |= FUSE_ATTR_DAX; + } + inode = lo_find(lo, &e->attr, mnt_id); if (inode) { close(newfd); -- 2.27.0
next prev parent reply other threads:[~2021-09-23 9:35 UTC|newest] Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-09-23 9:25 [PATCH v5 0/5] fuse,virtiofs: support per-file DAX Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:25 ` [PATCH v5 1/5] fuse: add fuse_should_enable_dax() helper Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 18:58 ` Vivek Goyal 2021-09-23 18:58 ` [Virtio-fs] " Vivek Goyal 2021-09-23 9:25 ` [PATCH v5 2/5] fuse: make DAX mount option a tri-state Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 19:02 ` Vivek Goyal 2021-09-23 19:02 ` [Virtio-fs] " Vivek Goyal 2021-09-23 22:26 ` Dave Chinner 2021-09-23 22:26 ` [Virtio-fs] " Dave Chinner 2021-09-24 1:02 ` Vivek Goyal 2021-09-24 1:02 ` [Virtio-fs] " Vivek Goyal 2021-09-24 3:06 ` JeffleXu 2021-09-24 3:06 ` [Virtio-fs] " JeffleXu 2021-09-24 12:43 ` Vivek Goyal 2021-09-24 12:43 ` [Virtio-fs] " Vivek Goyal 2021-09-26 2:06 ` JeffleXu 2021-09-26 2:06 ` [Virtio-fs] " JeffleXu 2021-09-27 0:21 ` Dave Chinner 2021-09-27 0:21 ` [Virtio-fs] " Dave Chinner 2021-09-27 2:28 ` JeffleXu 2021-09-27 2:28 ` [Virtio-fs] " JeffleXu 2021-09-28 3:44 ` Dave Chinner 2021-09-28 3:44 ` [Virtio-fs] " Dave Chinner 2021-09-28 5:17 ` JeffleXu 2021-09-28 5:17 ` [Virtio-fs] " JeffleXu 2021-09-28 14:34 ` Vivek Goyal 2021-09-28 14:34 ` [Virtio-fs] " Vivek Goyal 2021-09-27 15:32 ` Vivek Goyal 2021-09-27 15:32 ` [Virtio-fs] " Vivek Goyal 2021-09-28 3:23 ` Dave Chinner 2021-09-28 3:23 ` [Virtio-fs] " Dave Chinner 2021-09-23 9:25 ` [PATCH v5 3/5] fuse: support per-file DAX Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:25 ` [PATCH v5 4/5] fuse: enable " Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:25 ` [PATCH v5 5/5] fuse: mark inode DONT_CACHE when per-file DAX hint changes Jeffle Xu 2021-09-23 9:25 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:35 ` [virtiofsd PATCH v5 0/2] virtiofsd: support per-file DAX Jeffle Xu 2021-09-23 9:35 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:35 ` [virtiofsd PATCH v5 1/2] virtiofsd: add FUSE_ATTR_DAX to fuse protocol Jeffle Xu 2021-09-23 9:35 ` [Virtio-fs] " Jeffle Xu 2021-09-23 9:35 ` Jeffle Xu [this message] 2021-09-23 9:35 ` [Virtio-fs] [virtiofsd PATCH v5 2/2] virtiofsd: support per-file DAX Jeffle Xu 2021-09-23 18:57 ` [PATCH v5 0/5] fuse,virtiofs: " Vivek Goyal 2021-09-23 18:57 ` [Virtio-fs] " Vivek Goyal 2021-10-27 3:42 ` JeffleXu 2021-10-27 3:42 ` [Virtio-fs] " JeffleXu 2021-10-27 14:45 ` Vivek Goyal 2021-10-27 14:45 ` [Virtio-fs] " Vivek Goyal
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210923093545.81512-3-jefflexu@linux.alibaba.com \ --to=jefflexu@linux.alibaba.com \ --cc=bo.liu@linux.alibaba.com \ --cc=joseph.qi@linux.alibaba.com \ --cc=linux-fsdevel@vger.kernel.org \ --cc=miklos@szeredi.hu \ --cc=stefanha@redhat.com \ --cc=vgoyal@redhat.com \ --cc=virtio-fs@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.