From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: kernel-team@fb.com
Subject: Re: [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly
Date: Thu, 15 Aug 2019 14:14:54 -0700 [thread overview]
Message-ID: <20190815211454.GC27472@vader> (raw)
In-Reply-To: <cover.1565900769.git.osandov@fb.com>
On Thu, Aug 15, 2019 at 02:04:01PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
>
> Hello,
>
> This series adds a way to write compressed data directly to Btrfs. The
> intended use case is making send/receive on compressed file systems more
> efficient; however, the interface is general enough that it could be
> used in other scenarios. Patch 5 is the main change; see that for more
> details.
>
> Patches 1-3 are small fixes/cleanups that I ran into while implementing
> this; they should go in regardless of the remainder of the series. Patch
> 4 exports a required VFS interface.
>
> An example program and test case are available at [1].
>
> To preemptively address a few concerns:
>
> - Writing arbitrary, untrusted data which we feed to the decompression
> algorithm can be a security risk. For that reason, the ioctl is
> restricted to CAP_SYS_ADMIN. The Btrfs code is properly hardened
> against invalid compressed data/incorrect lengths, and the compression
> libraries are mature, but better safe than sorry for now.
> - If the user is writing their own compressed data rather than just
> blindly feeding in something from btrfs send, they need to know some
> implementation details about the compression format. For zlib, there
> are no special requirements. For zstd, a non-default compression
> parameter must be used. For lzo, we have our own wrapper format since
> lzo doesn't have a standard wrapper format. It feels a little wrong to
> expose these details, but they are part of the on-disk format, so they
> must be stable regardless.
> - The permissions checks duplicated from the VFS code are fairly
> minimal.
>
> This series is based on misc-next.
>
> This is an RFC, so please, comment away.
>
> Thanks!
>
> 1: https://github.com/osandov/xfstests/tree/btrfs-compressed-write
>
> Omar Sandoval (5):
> Btrfs: use correct count in btrfs_file_write_iter()
> Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs
> Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree
> fs: export rw_verify_area()
> Btrfs: add ioctl for directly writing compressed data
>
> fs/btrfs/compression.c | 6 +-
> fs/btrfs/compression.h | 14 +--
> fs/btrfs/ctree.h | 12 ++
> fs/btrfs/extent_io.c | 6 +-
> fs/btrfs/file.c | 22 ++--
> fs/btrfs/free-space-cache.c | 9 +-
> fs/btrfs/inode.c | 232 +++++++++++++++++++++++++++++++----
> fs/btrfs/ioctl.c | 101 ++++++++++++++-
> fs/btrfs/tests/inode-tests.c | 12 +-
> fs/internal.h | 5 -
> fs/read_write.c | 1 +
> include/linux/fs.h | 1 +
> include/uapi/linux/btrfs.h | 63 ++++++++++
> 13 files changed, 415 insertions(+), 69 deletions(-)
I forgot to CC fsdevel. I'll do that for v2.
next prev parent reply other threads:[~2019-08-15 21:14 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-15 21:04 [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly Omar Sandoval
2019-08-15 21:04 ` [PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter() Omar Sandoval
2019-08-16 16:56 ` Josef Bacik
2019-08-15 21:04 ` [PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs Omar Sandoval
2019-08-16 16:59 ` Josef Bacik
2019-08-27 12:35 ` David Sterba
2019-08-27 17:44 ` Omar Sandoval
2019-08-27 18:16 ` David Sterba
2019-08-15 21:04 ` [PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree Omar Sandoval
2019-08-16 16:59 ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 4/5] fs: export rw_verify_area() Omar Sandoval
2019-08-16 17:02 ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data Omar Sandoval
2019-08-26 21:36 ` Josef Bacik
2019-08-27 6:26 ` Nikolay Borisov
2019-08-27 11:57 ` Josef Bacik
2019-08-27 18:06 ` Omar Sandoval
2019-08-27 18:22 ` Omar Sandoval
2019-08-27 18:28 ` Josef Bacik
2019-08-28 12:06 ` David Sterba
2019-09-03 17:14 ` Omar Sandoval
2019-08-15 21:14 ` Omar Sandoval [this message]
2019-08-27 18:31 ` [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190815211454.GC27472@vader \
--to=osandov@osandov.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).