Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Peng Tao <bergwolf@gmail.com>
To: Alessio Balsini <balsini@android.com>
Cc: Miklos Szeredi <miklos@szeredi.hu>,
	Akilesh Kailash <akailash@google.com>,
	Amir Goldstein <amir73il@gmail.com>,
	Antonio SJ Musumeci <trapexit@spawn.link>,
	David Anderson <dvander@google.com>,
	Giuseppe Scrivano <gscrivan@redhat.com>,
	Jann Horn <jannh@google.com>, Jens Axboe <axboe@kernel.dk>,
	Martijn Coenen <maco@android.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Lawrence <paullawrence@google.com>,
	Stefano Duo <duostefano93@gmail.com>,
	Zimuzo Ezeozue <zezeozue@google.com>,
	fuse-devel@lists.sourceforge.net, kernel-team@android.com,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH V10 0/5] fuse: Add support for passthrough read/write
Date: Sat, 28 Nov 2020 10:10:37 +0800
Message-ID: <CA+a=Yy76W4Xob6UVXsPLA_FKF_8+QFQSEF98yALjRmuOhnVOdw@mail.gmail.com> (raw)
In-Reply-To: <20201026125016.1905945-1-balsini@android.com>

On Tue, Oct 27, 2020 at 1:00 AM Alessio Balsini <balsini@android.com> wrote:
>
> This is the 10th version of the series. Please find the changelog at the
> bottom of this cover letter.
>
> Add support for file system passthrough read/write of files when enabled in
> userspace through the option FUSE_PASSTHROUGH.
>
> There are file systems based on FUSE that are intended to enforce special
> policies or trigger complicated decision makings at the file operations
> level. Android, for example, uses FUSE to enforce fine-grained access
> policies that also depend on the file contents.
> Sometimes it happens that at open or create time a file is identified as
> not requiring additional checks for consequent reads/writes, thus FUSE
> would simply act as a passive bridge between the process accessing the FUSE
> file system and the lower file system. Splicing and caching help reduce the
> FUSE overhead, but there are still read/write operations forwarded to the
> userspace FUSE daemon that could be avoided.
>
> This series has been inspired by the original patches from Nikhilesh Reddy,
> the idea and code of which has been elaborated and improved thanks to the
> community support.
>
> When the FUSE_PASSTHROUGH capability is enabled, the FUSE daemon may decide
> while handling the open/create operations, if the given file can be
> accessed in passthrough mode. This means that all the further read and
> write operations would be forwarded by the kernel directly to the lower
> file system using the VFS layer rather than to the FUSE daemon.
> All the requests other than reads or writes are still handled by the
> userspace FUSE daemon.
> This allows for improved performance on reads and writes, especially in the
> case of reads at random offsets, for which no (readahead) caching mechanism
> would help.
> Benchmarks show improved performance that is close to native file system
> access when doing massive manipulations on a single opened file, especially
> in the case of random reads, for which the bandwidth increased by almost 2X
> or sequential writes for which the improvement is close to 3X.
>
> The creation of this direct connection (passthrough) between FUSE file
> objects and file objects in the lower file system happens in a way that
> reminds of passing file descriptors via sockets:
> - a process requests the opening of a file handled by FUSE, so the kernel
>   forwards the request to the FUSE daemon;
> - the FUSE daemon opens the target file in the lower file system, getting
>   its file descriptor;
> - the FUSE daemon also decides according to its internal policies if
>   passthrough can be enabled for that file, and, if so, can perform a
>   FUSE_DEV_IOC_PASSTHROUGH_OPEN ioctl() on /dev/fuse, passing the file
>   descriptor obtained at the previous step and the fuse_req unique
>   identifier;
> - the kernel translates the file descriptor to the file pointer navigating
>   through the opened files of the "current" process and temporarily stores
>   it in the associated open/create fuse_req's passthrough_filp;
> - when the FUSE daemon has done with the request and it's time for the
>   kernel to close it, it checks if the passthrough_filp is available and in
>   case updates the additional field in the fuse_file owned by the process
>   accessing the FUSE file system.
> From now on, all the read/write operations performed by that process will
> be redirected to the corresponding lower file system file by creating new
> VFS requests.
> Since the read/write operation to the lower file system is executed with
> the current process's credentials, it might happen that it does not have
> enough privileges to succeed. For this reason, the process temporarily
> receives the same credentials as the FUSE daemon, that are reverted as soon
> as the read/write operation completes, emulating the behavior of the
> request to be performed by the FUSE daemon itself. This solution has been
> inspired by the way overlayfs handles read/write operations.
> Asynchronous IO is supported as well, handled by creating separate AIO
> requests for the lower file system that will be internally tracked by FUSE,
> that intercepts and propagates their completion through an internal
> ki_completed callback similar to the current implementation of overlayfs.
> The ioctl() has been designed taking as a reference and trying to converge
> to the fuse2 implementation. For example, the fuse_passthrough_out data
> structure has extra fields that will allow for further extensions of the
> feature.
>
>
>     Performance on SSD
>
> What follows has been performed with this change [V6] rebased on top of
> vanilla v5.8 Linux kernel, using a custom passthrough_hp FUSE daemon that
> enables pass-through for each file that is opened during both "open" and
> "create". Tests were run on an Intel Xeon E5-2678V3, 32GiB of RAM, with an
> ext4-formatted SSD as the lower file system, with no special tuning, e.g.,
> all the involved processes are SCHED_OTHER, ondemand is the frequency
> governor with no frequency restrictions, and turbo-boost, as well as
> p-state, are active. This is because I noticed that, for such high-level
> benchmarks, results consistency was minimally affected by these features.
> The source code of the updated libfuse library and passthrough_hp is shared
> at the following repository:
>
>     https://github.com/balsini/libfuse/tree/fuse-passthrough-stable-v.3.9.4
The libfuse changes are not updated with the latest ioctl UAPI change yet.

> * UAPI updated: ioctl() now returns an ID that will be used at
>   open/create response time to reference the passthrough file

Cheers,
Tao
-- 
Into Sth. Rich & Strange

  parent reply index

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-26 12:50 Alessio Balsini
2020-10-26 12:50 ` [PATCH V10 1/5] fuse: Definitions and ioctl() for passthrough Alessio Balsini
2020-10-26 12:50 ` [PATCH V10 2/5] fuse: Passthrough initialization and release Alessio Balsini
2020-11-26 13:33   ` Peng Tao
2020-11-27 13:41     ` Alessio Balsini
2020-11-28  1:57       ` Peng Tao
2020-12-16 16:46         ` Alessio Balsini
     [not found]   ` <3bf58b6f-c7eb-7baa-384d-ae0830d8bceb@tcl.com>
2020-12-16 16:55     ` Alessio Balsini
2020-10-26 12:50 ` [PATCH V10 3/5] fuse: Introduce synchronous read and write for passthrough Alessio Balsini
2020-10-26 12:50 ` [PATCH V10 4/5] fuse: Handle asynchronous read and write in passthrough Alessio Balsini
2020-10-26 12:50 ` [PATCH V10 5/5] fuse: Use daemon creds in passthrough mode Alessio Balsini
2020-11-28  2:10 ` Peng Tao [this message]
2020-11-30 11:08   ` [PATCH V10 0/5] fuse: Add support for passthrough read/write Alessio Balsini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CA+a=Yy76W4Xob6UVXsPLA_FKF_8+QFQSEF98yALjRmuOhnVOdw@mail.gmail.com' \
    --to=bergwolf@gmail.com \
    --cc=akailash@google.com \
    --cc=amir73il@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=balsini@android.com \
    --cc=duostefano93@gmail.com \
    --cc=dvander@google.com \
    --cc=fuse-devel@lists.sourceforge.net \
    --cc=gscrivan@redhat.com \
    --cc=jannh@google.com \
    --cc=kernel-team@android.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maco@android.com \
    --cc=miklos@szeredi.hu \
    --cc=palmer@dabbelt.com \
    --cc=paullawrence@google.com \
    --cc=trapexit@spawn.link \
    --cc=zezeozue@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git