From: Feng Shuo <steve.shuo.feng@gmail.com>
To: Maxim Patlasov <mpatlasov@parallels.com>
Cc: miklos@szeredi.hu, dev@parallels.com,
"fuse-devel@lists.sourceforge.net"
<fuse-devel@lists.sourceforge.net>,
linux-kernel@vger.kernel.org, jbottomley@parallels.com,
viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
xemul@openvz.org
Subject: Re: [PATCH v2 00/14] fuse: An attempt to implement a write-back cache policy
Date: Tue, 27 Nov 2012 09:04:18 +0800 [thread overview]
Message-ID: <CALmpHyb5wpn-dABuEwpypUggRY=1aULvJW0z17HpH50pr1=HKg@mail.gmail.com> (raw)
In-Reply-To: <20121116170123.3196.93431.stgit@maximpc.sw.ru>
Hi Maxim,
I'm new to fuse but have some experience with NFS. From my
understanding after reviewing your patchset, it seems only work with
local file system or a distributed file system whose file is never
modified (could be grown but no or very few modified) because it
doesn't exam the pre/post status of the writing object (e.g. a file).
So if a file is modified outside, fuse might not get any chance to
handle it...... Correct me if I got wrong since I'm really new to
fuse. :-)
On Sat, Nov 17, 2012 at 1:04 AM, Maxim Patlasov <mpatlasov@parallels.com> wrote:
> Hi,
>
> This is the second iteration of Pavel Emelyanov's patch-set implementing
> write-back policy for FUSE page cache. Initial patch-set description was
> the following:
>
> One of the problems with the existing FUSE implementation is that it uses the
> write-through cache policy which results in performance problems on certain
> workloads. E.g. when copying a big file into a FUSE file the cp pushes every
> 128k to the userspace synchronously. This becomes a problem when the userspace
> back-end uses networking for storing the data.
>
> A good solution of this is switching the FUSE page cache into a write-back policy.
> With this file data are pushed to the userspace with big chunks (depending on the
> dirty memory limits, but this is much more than 128k) which lets the FUSE daemons
> handle the size updates in a more efficient manner.
>
> The writeback feature is per-connection and is explicitly configurable at the
> init stage (is it worth making it CAP_SOMETHING protected?) When the writeback is
> turned ON:
>
> * still copy writeback pages to temporary buffer when sending a writeback request
> and finish the page writeback immediately
>
> * make kernel maintain the inode's i_size to avoid frequent i_size synchronization
> with the user space
>
> * take NR_WRITEBACK_TEMP into account when makeing balance_dirty_pages decision.
> This protects us from having too many dirty pages on FUSE
>
> The provided patchset survives the fsx test. Performance measurements are not yet
> all finished, but the mentioned copying of a huge file becomes noticeably faster
> even on machines with few RAM and doesn't make the system stuck (the dirty pages
> balancer does its work OK). Applies on top of v3.5-rc4.
>
> We are currently exploring this with our own distributed storage implementation
> which is heavily oriented on storing big blobs of data with extremely rare meta-data
> updates (virtual machines' and containers' disk images). With the existing cache
> policy a typical usage scenario -- copying a big VM disk into a cloud -- takes way
> too much time to proceed, much longer than if it was simply scp-ed over the same
> network. The write-back policy (as I mentioned) noticeably improves this scenario.
> Kirill (in Cc) can share more details about the performance and the storage concepts
> details if required.
>
> Changed in v2:
> - numerous bugfixes:
> - fuse_write_begin and fuse_writepages_fill and fuse_writepage_locked must wait
> on page writeback because page writeback can extend beyond the lifetime of
> the page-cache page
> - fuse_send_writepages can end_page_writeback on original page only after adding
> request to fi->writepages list; otherwise another writeback may happen inside
> the gap between end_page_writeback and adding to the list
> - fuse_direct_io must wait on page writeback; otherwise data corruption is possible
> due to reordering requests
> - fuse_flush must flush dirty memory and wait for all writeback on given inode
> before sending FUSE_FLUSH to userspace; otherwise FUSE_FLUSH is not reliable
> - fuse_file_fallocate must hold i_mutex around FUSE_FALLOCATE and i_size update;
> otherwise a race with a writer extending i_size is possible
> - fix handling errors in fuse_writepages and fuse_send_writepages
> - handle i_mtime intelligently if writeback cache is on (see patch #7 (update i_mtime
> on buffered writes) for details.
> - put enabling writeback cache under fusermount control; (see mount option
> 'allow_wbcache' introduced by patch #13 (turn writeback cache on))
> - rebased on v3.7-rc5
>
> Thanks,
> Maxim
>
> ---
>
> Maxim Patlasov (14):
> fuse: Linking file to inode helper
> fuse: Getting file for writeback helper
> fuse: Prepare to handle short reads
> fuse: Prepare to handle multiple pages in writeback
> fuse: Connection bit for enabling writeback
> fuse: Trust kernel i_size only
> fuse: Update i_mtime on buffered writes
> fuse: Flush files on wb close
> fuse: Implement writepages and write_begin/write_end callbacks
> fuse: fuse_writepage_locked() should wait on writeback
> fuse: fuse_flush() should wait on writeback
> fuse: Fix O_DIRECT operations vs cached writeback misorder
> fuse: Turn writeback cache on
> mm: Account for WRITEBACK_TEMP in balance_dirty_pages
>
>
> fs/fuse/dir.c | 51 ++++
> fs/fuse/file.c | 523 +++++++++++++++++++++++++++++++++++++++++----
> fs/fuse/fuse_i.h | 20 ++
> fs/fuse/inode.c | 98 ++++++++
> include/uapi/linux/fuse.h | 1
> mm/page-writeback.c | 3
> 6 files changed, 638 insertions(+), 58 deletions(-)
>
> --
> Signature
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Feng Shuo
Tel: (86)10-59851155-2116
Fax: (86)10-59851155-2008
Tianjin Zhongke Blue Whale Information Technologies Co., Ltd
10th Floor, Tower A, The GATE building, No. 19 Zhong-guan-cun Avenue
Haidian District, Beijing, China
Postcode 100080
next prev parent reply other threads:[~2012-11-27 1:04 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-16 17:04 [PATCH v2 00/14] fuse: An attempt to implement a write-back cache policy Maxim Patlasov
2012-11-16 17:05 ` [PATCH 01/14] fuse: Linking file to inode helper Maxim Patlasov
2012-11-16 17:05 ` [PATCH 02/14] fuse: Getting file for writeback helper Maxim Patlasov
2012-11-16 17:06 ` [PATCH 03/14] fuse: Prepare to handle short reads Maxim Patlasov
2012-11-16 17:07 ` [PATCH 04/14] fuse: Prepare to handle multiple pages in writeback Maxim Patlasov
2012-11-16 17:07 ` [PATCH 05/14] fuse: Connection bit for enabling writeback Maxim Patlasov
2012-11-16 17:07 ` [PATCH 06/14] fuse: Trust kernel i_size only Maxim Patlasov
2012-12-05 16:39 ` [PATCH] fuse: Trust kernel i_size only - v2 Maxim Patlasov
2012-12-05 16:40 ` [PATCH] fuse: Implement writepages and write_begin/write_end callbacks " Maxim Patlasov
2012-11-16 17:09 ` [PATCH 07/14] fuse: Update i_mtime on buffered writes Maxim Patlasov
2012-11-16 17:09 ` [PATCH 08/14] fuse: Flush files on wb close Maxim Patlasov
2012-11-16 17:09 ` [PATCH 09/14] fuse: Implement writepages and write_begin/write_end callbacks Maxim Patlasov
2012-11-16 17:09 ` [PATCH 10/14] fuse: fuse_writepage_locked() should wait on writeback Maxim Patlasov
2012-11-16 17:10 ` [PATCH 11/14] fuse: fuse_flush() " Maxim Patlasov
2012-11-16 17:10 ` [PATCH 12/14] fuse: Fix O_DIRECT operations vs cached writeback misorder Maxim Patlasov
2012-12-05 16:43 ` [PATCH] fuse: Fix O_DIRECT operations vs cached writeback misorder - v2 Maxim Patlasov
2012-11-16 17:10 ` [PATCH 13/14] fuse: Turn writeback cache on Maxim Patlasov
2012-11-16 17:10 ` [PATCH 14/14] mm: Account for WRITEBACK_TEMP in balance_dirty_pages Maxim Patlasov
2012-11-21 12:01 ` Maxim Patlasov
2012-11-22 13:27 ` Jaegeuk Hanse
2012-11-22 13:56 ` Maxim V. Patlasov
2012-11-27 1:04 ` Feng Shuo [this message]
2012-11-27 7:56 ` [PATCH v2 00/14] fuse: An attempt to implement a write-back cache policy Maxim V. Patlasov
2012-12-12 14:53 ` Maxim V. Patlasov
2013-01-15 15:20 ` Maxim V. Patlasov
2013-01-25 10:21 ` Miklos Szeredi
2013-01-25 12:50 ` Maxim V. Patlasov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CALmpHyb5wpn-dABuEwpypUggRY=1aULvJW0z17HpH50pr1=HKg@mail.gmail.com' \
--to=steve.shuo.feng@gmail.com \
--cc=dev@parallels.com \
--cc=fuse-devel@lists.sourceforge.net \
--cc=jbottomley@parallels.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=miklos@szeredi.hu \
--cc=mpatlasov@parallels.com \
--cc=viro@zeniv.linux.org.uk \
--cc=xemul@openvz.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).