From: "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> To: qemu-devel@nongnu.org, stefanha@redhat.com, vgoyal@redhat.com, virtio-fs@redhat.com Subject: [PATCH v2 00/25] virtiofs dax patches Date: Wed, 14 Apr 2021 16:51:12 +0100 [thread overview] Message-ID: <20210414155137.46522-1-dgilbert@redhat.com> (raw) From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> This series adds support for acceleration of virtiofs via DAX mapping, using features added in the 5.11 Linux kernel. DAX originally existed in the kernel for mapping real storage devices directly into memory, so that reads/writes turn into reads/writes directly mapped into the storage device. virtiofs's DAX support is similar; a PCI BAR is exposed on the virtiofs device corresponding to a DAX 'cache' of a user defined size. The guest daemon then requests files to be mapped into that cache; when that happens the virtiofsd sends filedescriptors and commands back to the QEMU that mmap's those files directly into the memory slot exposed to kvm. The guest can then directly read/write to the files exposed by virtiofs by reading/writing into the BAR. A typical invocation would be: -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=4G and then the guest must mount with -o dax Note that the cache doesn't really take VM up on the host, because everything placed there is just an mmap of a file, so you can afford to use quite a large cache size. Unlike a real DAX device, the cache is a finite size that's potentially smaller than the underlying filesystem (especially when mapping granuality is taken into account). Mapping, unmapping and remapping must take place to juggle files into the cache if it's too small. Some workloads benefit more than others. Gotchas: a) If something else on the host truncates an mmap'd file, kvm gets rather upset; for this reason it's advised that DAX is currently only suitable for use on non-shared filesystems. Dave v2 Cleanups from first review, rebase on current head Dr. David Alan Gilbert (20): DAX: vhost-user: Rework slave return values virtiofsd: Don't assume header layout DAX: libvhost-user: Route slave message payload DAX: libvhost-user: Allow popping a queue element with bad pointers DAX subprojects/libvhost-user: Add virtio-fs slave types DAX: virtio: Add shared memory capability DAX: virtio-fs: Add cache BAR DAX: virtio-fs: Add vhost-user slave commands for mapping DAX: virtio-fs: Fill in slave commands for mapping DAX: virtiofsd Add cache accessor functions DAX: virtiofsd: Add setup/remove mappings fuse commands DAX: virtiofsd: Add setup/remove mapping handlers to passthrough_ll DAX: virtiofsd: Wire up passthrough_ll's lo_setupmapping DAX: virtiofsd: route se down to destroy method DAX: virtiofsd: Perform an unmap on destroy DAX/unmap: virtiofsd: Add VHOST_USER_SLAVE_FS_IO DAX/unmap virtiofsd: Add wrappers for VHOST_USER_SLAVE_FS_IO DAX/unmap virtiofsd: Parse unmappable elements DAX/unmap virtiofsd: Route unmappable reads DAX/unmap virtiofsd: route unmappable write to slave command Stefan Hajnoczi (1): DAX:virtiofsd: implement FUSE_INIT map_alignment field Vivek Goyal (4): DAX: virtiofsd: Make lo_removemapping() work vhost-user-fs: Extend VhostUserFSSlaveMsg to pass additional info vhost-user-fs: Implement drop CAP_FSETID functionality virtiofsd: Ask qemu to drop CAP_FSETID if client asked for it block/export/vhost-user-blk-server.c | 2 +- contrib/vhost-user-blk/vhost-user-blk.c | 3 +- contrib/vhost-user-gpu/vhost-user-gpu.c | 5 +- contrib/vhost-user-input/main.c | 4 +- contrib/vhost-user-scsi/vhost-user-scsi.c | 2 +- docs/interop/vhost-user.rst | 37 ++ hw/virtio/meson.build | 1 + hw/virtio/trace-events | 6 + hw/virtio/vhost-backend.c | 6 +- hw/virtio/vhost-user-fs-pci.c | 32 ++ hw/virtio/vhost-user-fs.c | 395 ++++++++++++++++++++++ hw/virtio/vhost-user.c | 60 +++- hw/virtio/virtio-pci.c | 20 ++ hw/virtio/virtio-pci.h | 4 + include/hw/virtio/vhost-backend.h | 2 +- include/hw/virtio/vhost-user-fs.h | 43 +++ meson.build | 6 + subprojects/libvhost-user/libvhost-user.c | 113 ++++++- subprojects/libvhost-user/libvhost-user.h | 57 +++- tests/vhost-user-bridge.c | 4 +- tools/virtiofsd/buffer.c | 22 +- tools/virtiofsd/fuse_common.h | 17 +- tools/virtiofsd/fuse_lowlevel.c | 92 ++++- tools/virtiofsd/fuse_lowlevel.h | 78 ++++- tools/virtiofsd/fuse_virtio.c | 372 ++++++++++++++++---- tools/virtiofsd/passthrough_ll.c | 117 ++++++- 26 files changed, 1380 insertions(+), 120 deletions(-) -- 2.31.1
WARNING: multiple messages have this Message-ID (diff)
From: "Dr. David Alan Gilbert (git)" <dgilbert@redhat.com> To: qemu-devel@nongnu.org, stefanha@redhat.com, vgoyal@redhat.com, virtio-fs@redhat.com Subject: [Virtio-fs] [PATCH v2 00/25] virtiofs dax patches Date: Wed, 14 Apr 2021 16:51:12 +0100 [thread overview] Message-ID: <20210414155137.46522-1-dgilbert@redhat.com> (raw) From: "Dr. David Alan Gilbert" <dgilbert@redhat.com> This series adds support for acceleration of virtiofs via DAX mapping, using features added in the 5.11 Linux kernel. DAX originally existed in the kernel for mapping real storage devices directly into memory, so that reads/writes turn into reads/writes directly mapped into the storage device. virtiofs's DAX support is similar; a PCI BAR is exposed on the virtiofs device corresponding to a DAX 'cache' of a user defined size. The guest daemon then requests files to be mapped into that cache; when that happens the virtiofsd sends filedescriptors and commands back to the QEMU that mmap's those files directly into the memory slot exposed to kvm. The guest can then directly read/write to the files exposed by virtiofs by reading/writing into the BAR. A typical invocation would be: -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag=myfs,cache-size=4G and then the guest must mount with -o dax Note that the cache doesn't really take VM up on the host, because everything placed there is just an mmap of a file, so you can afford to use quite a large cache size. Unlike a real DAX device, the cache is a finite size that's potentially smaller than the underlying filesystem (especially when mapping granuality is taken into account). Mapping, unmapping and remapping must take place to juggle files into the cache if it's too small. Some workloads benefit more than others. Gotchas: a) If something else on the host truncates an mmap'd file, kvm gets rather upset; for this reason it's advised that DAX is currently only suitable for use on non-shared filesystems. Dave v2 Cleanups from first review, rebase on current head Dr. David Alan Gilbert (20): DAX: vhost-user: Rework slave return values virtiofsd: Don't assume header layout DAX: libvhost-user: Route slave message payload DAX: libvhost-user: Allow popping a queue element with bad pointers DAX subprojects/libvhost-user: Add virtio-fs slave types DAX: virtio: Add shared memory capability DAX: virtio-fs: Add cache BAR DAX: virtio-fs: Add vhost-user slave commands for mapping DAX: virtio-fs: Fill in slave commands for mapping DAX: virtiofsd Add cache accessor functions DAX: virtiofsd: Add setup/remove mappings fuse commands DAX: virtiofsd: Add setup/remove mapping handlers to passthrough_ll DAX: virtiofsd: Wire up passthrough_ll's lo_setupmapping DAX: virtiofsd: route se down to destroy method DAX: virtiofsd: Perform an unmap on destroy DAX/unmap: virtiofsd: Add VHOST_USER_SLAVE_FS_IO DAX/unmap virtiofsd: Add wrappers for VHOST_USER_SLAVE_FS_IO DAX/unmap virtiofsd: Parse unmappable elements DAX/unmap virtiofsd: Route unmappable reads DAX/unmap virtiofsd: route unmappable write to slave command Stefan Hajnoczi (1): DAX:virtiofsd: implement FUSE_INIT map_alignment field Vivek Goyal (4): DAX: virtiofsd: Make lo_removemapping() work vhost-user-fs: Extend VhostUserFSSlaveMsg to pass additional info vhost-user-fs: Implement drop CAP_FSETID functionality virtiofsd: Ask qemu to drop CAP_FSETID if client asked for it block/export/vhost-user-blk-server.c | 2 +- contrib/vhost-user-blk/vhost-user-blk.c | 3 +- contrib/vhost-user-gpu/vhost-user-gpu.c | 5 +- contrib/vhost-user-input/main.c | 4 +- contrib/vhost-user-scsi/vhost-user-scsi.c | 2 +- docs/interop/vhost-user.rst | 37 ++ hw/virtio/meson.build | 1 + hw/virtio/trace-events | 6 + hw/virtio/vhost-backend.c | 6 +- hw/virtio/vhost-user-fs-pci.c | 32 ++ hw/virtio/vhost-user-fs.c | 395 ++++++++++++++++++++++ hw/virtio/vhost-user.c | 60 +++- hw/virtio/virtio-pci.c | 20 ++ hw/virtio/virtio-pci.h | 4 + include/hw/virtio/vhost-backend.h | 2 +- include/hw/virtio/vhost-user-fs.h | 43 +++ meson.build | 6 + subprojects/libvhost-user/libvhost-user.c | 113 ++++++- subprojects/libvhost-user/libvhost-user.h | 57 +++- tests/vhost-user-bridge.c | 4 +- tools/virtiofsd/buffer.c | 22 +- tools/virtiofsd/fuse_common.h | 17 +- tools/virtiofsd/fuse_lowlevel.c | 92 ++++- tools/virtiofsd/fuse_lowlevel.h | 78 ++++- tools/virtiofsd/fuse_virtio.c | 372 ++++++++++++++++---- tools/virtiofsd/passthrough_ll.c | 117 ++++++- 26 files changed, 1380 insertions(+), 120 deletions(-) -- 2.31.1
next reply other threads:[~2021-04-14 15:54 UTC|newest] Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-04-14 15:51 Dr. David Alan Gilbert (git) [this message] 2021-04-14 15:51 ` [Virtio-fs] [PATCH v2 00/25] virtiofs dax patches Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 01/25] DAX: vhost-user: Rework slave return values Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-16 10:59 ` Greg Kurz 2021-04-16 10:59 ` Greg Kurz 2021-04-21 17:31 ` Dr. David Alan Gilbert 2021-04-21 17:31 ` Dr. David Alan Gilbert 2021-04-14 15:51 ` [PATCH v2 02/25] virtiofsd: Don't assume header layout Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 03/25] DAX: libvhost-user: Route slave message payload Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 04/25] DAX: libvhost-user: Allow popping a queue element with bad pointers Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 05/25] DAX subprojects/libvhost-user: Add virtio-fs slave types Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 06/25] DAX: virtio: Add shared memory capability Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 07/25] DAX: virtio-fs: Add cache BAR Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 08/25] DAX: virtio-fs: Add vhost-user slave commands for mapping Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 16:35 ` Greg Kurz 2021-04-14 16:35 ` Greg Kurz 2021-04-21 17:49 ` Dr. David Alan Gilbert 2021-04-21 17:49 ` Dr. David Alan Gilbert 2021-04-14 15:51 ` [PATCH v2 09/25] DAX: virtio-fs: Fill in " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 10/25] DAX: virtiofsd Add cache accessor functions Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 11/25] DAX: virtiofsd: Add setup/remove mappings fuse commands Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 12/25] DAX: virtiofsd: Add setup/remove mapping handlers to passthrough_ll Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 13/25] DAX: virtiofsd: Wire up passthrough_ll's lo_setupmapping Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 14/25] DAX: virtiofsd: Make lo_removemapping() work Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 15/25] DAX: virtiofsd: route se down to destroy method Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 16/25] DAX: virtiofsd: Perform an unmap on destroy Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 17/25] DAX/unmap: virtiofsd: Add VHOST_USER_SLAVE_FS_IO Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-21 20:07 ` Vivek Goyal 2021-04-21 20:07 ` Vivek Goyal 2021-04-22 9:29 ` Dr. David Alan Gilbert 2021-04-22 9:29 ` Dr. David Alan Gilbert 2021-04-22 15:40 ` Vivek Goyal 2021-04-22 15:40 ` Vivek Goyal 2021-04-22 15:48 ` Dr. David Alan Gilbert 2021-04-22 15:48 ` Dr. David Alan Gilbert 2021-04-14 15:51 ` [PATCH v2 18/25] DAX/unmap virtiofsd: Add wrappers for VHOST_USER_SLAVE_FS_IO Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 19/25] DAX/unmap virtiofsd: Parse unmappable elements Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 20/25] DAX/unmap virtiofsd: Route unmappable reads Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 21/25] DAX/unmap virtiofsd: route unmappable write to slave command Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 22/25] DAX:virtiofsd: implement FUSE_INIT map_alignment field Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 23/25] vhost-user-fs: Extend VhostUserFSSlaveMsg to pass additional info Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 24/25] vhost-user-fs: Implement drop CAP_FSETID functionality Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [PATCH v2 25/25] virtiofsd: Ask qemu to drop CAP_FSETID if client asked for it Dr. David Alan Gilbert (git) 2021-04-14 15:51 ` [Virtio-fs] " Dr. David Alan Gilbert (git)
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210414155137.46522-1-dgilbert@redhat.com \ --to=dgilbert@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=stefanha@redhat.com \ --cc=vgoyal@redhat.com \ --cc=virtio-fs@redhat.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.