Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
From: Alessio Balsini <balsini@android.com>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: Alessio Balsini <balsini@android.com>,
	Akilesh Kailash <akailash@google.com>,
	Amir Goldstein <amir73il@gmail.com>,
	Antonio SJ Musumeci <trapexit@spawn.link>,
	David Anderson <dvander@google.com>,
	Giuseppe Scrivano <gscrivan@redhat.com>,
	Jann Horn <jannh@google.com>, Jens Axboe <axboe@kernel.dk>,
	Martijn Coenen <maco@android.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Lawrence <paullawrence@google.com>,
	Stefano Duo <stefanoduo@google.com>,
	Zimuzo Ezeozue <zezeozue@google.com>,
	fuse-devel <fuse-devel@lists.sourceforge.net>,
	kernel-team <kernel-team@android.com>,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH V9 0/4] fuse: Add support for passthrough read/write
Date: Wed, 21 Oct 2020 16:39:36 +0100
Message-ID: <20201021153936.GA24818@google.com> (raw)
In-Reply-To: <20201002133802.GA3595556@google.com>

Hi Miklos, all,
 
After being stuck with some strange and hard to reproduce results from my SSD,
I finally decided to overcome the biggest chunk of inconsistencies by
forgetting about the SSD and switching to a RAM block device to host my lower
file system.
Getting rid of the discrete storage device removes a huge component of
slowness, highlighting the performance difference of the software parts (and
probably goodness of CPU cache and its coherence/invalidation mechanisms).
 
More specifically, out of my system's 32 GiB of RAM, I reserved 24 for
/dev/ram0, which has been formatted as ext4.
That file system has been completely filled and then cleaned up before running
the benchmarks to make sure all the memory addresses were marked as used and
removed from the page cache.
 
As for the last time, I've been using a slightly modified libfuse
passthrough_hp.cc example, that simply enables the passthrough mode at every
open/create operation:

  git@github.com:balsini/libfuse fuse-passthrough-stable-v.3.9.4
 
The following tests were ran using fio-3.23 with the following configuration:
- bs=4Ki
- size=20Gi
- ioengine=sync
- fsync_on_close=1
- randseed=0
- create_only=0 (set to 1 during a first dry run to create the test file)

As for the tool configuration, the following benchmarks would perform a single
open operation each, focusing on just the read/write perfromance.

The file size of 20 GiB has been chosen to not completely fit the page cache.
 
As mentioned in my previous email, all the caches were dropped before running
every benchmark with
 
  echo 3 > /proc/sys/vm/drop_caches
 
All the benchmarks were run 10 times, with 1 minute cool down between each run.
 
Here the updated results for this patch set:
 
+-----------+-------------+-------------+-------------+
|           |             | FUSE        |             |
| MiB/s     | FUSE        | passthrough | native      |
+-----------+-------------+-------------+-------------+
| read      | 1341(±4.2%) | 1485(±1.1%) |  1634(±.5%) |
+-----------+-------------+-------------+-------------+
| write     |   49(±2.1%) | 1304(±2.6%) | 1363(±3.0%) |
+-----------+-------------+-------------+-------------+
| randread  |   43(±1.3%) | 643(±11.1%) |  715(±1.1%) |
+-----------+-------------+-------------+-------------+
| randwrite |  27(±39.9%) |  763(±1.1%) |  790(±1.0%) |
+-----------+-------------+-------------+-------------+
 
This table shows that FUSE, except for the sequential reads, is left behind
FUSE passthrough and native performance. The extremely good FUSE performance
for sequential reads is the result of a great read-ahead mechanism, that has
been easy to prove by showing that performance dropped after setting
read_ahead_kb to 0.
Except for FUSE randwrite and passthrough randread with respectively ~40% and
~11% standard deviations, all the other results are relatively stable.
Nevertheless, these two standard deviation exceptions are not sufficient to
invalidate the results, that are still showing clear performance benefits.
I'm also kind of happy to see that passthrough, that for each read/write
operation traverses the VFS layer twice, now maintains consistent slightly
lower performance than native.
 
I wanted to make sure the results were consistent before jumping back to your
feedback on the series.
 
Thanks,
Alessio

      reply index

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-24 13:13 Alessio Balsini
2020-09-24 13:13 ` [PATCH V9 1/4] fuse: Definitions and ioctl() for passthrough Alessio Balsini
2020-09-29 14:37   ` Alessio Balsini
2020-09-30 15:44   ` Miklos Szeredi
2020-10-22 16:12     ` Alessio Balsini
2020-09-24 13:13 ` [PATCH V9 2/4] fuse: Trace daemon creds Alessio Balsini
2020-09-30 18:45   ` Miklos Szeredi
2020-09-30 19:16     ` Antonio SJ Musumeci
2020-10-22 16:14       ` Alessio Balsini
2020-09-24 13:13 ` [PATCH V9 3/4] fuse: Introduce synchronous read and write for passthrough Alessio Balsini
2020-09-30 18:50   ` Miklos Szeredi
2020-10-22 16:17     ` Alessio Balsini
2020-09-24 13:13 ` [PATCH V9 4/4] fuse: Handle asynchronous read and write in passthrough Alessio Balsini
2020-09-30 18:54   ` Miklos Szeredi
2020-10-22 16:38     ` Alessio Balsini
2020-09-30 15:33 ` [PATCH V9 0/4] fuse: Add support for passthrough read/write Miklos Szeredi
2020-10-02 13:38   ` Alessio Balsini
2020-10-21 15:39     ` Alessio Balsini [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201021153936.GA24818@google.com \
    --to=balsini@android.com \
    --cc=akailash@google.com \
    --cc=amir73il@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=dvander@google.com \
    --cc=fuse-devel@lists.sourceforge.net \
    --cc=gscrivan@redhat.com \
    --cc=jannh@google.com \
    --cc=kernel-team@android.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maco@android.com \
    --cc=miklos@szeredi.hu \
    --cc=palmer@dabbelt.com \
    --cc=paullawrence@google.com \
    --cc=stefanoduo@google.com \
    --cc=trapexit@spawn.link \
    --cc=zezeozue@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git