From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: kernel-team@fb.com
Subject: [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly
Date: Thu, 15 Aug 2019 14:04:01 -0700 [thread overview]
Message-ID: <cover.1565900769.git.osandov@fb.com> (raw)
From: Omar Sandoval <osandov@fb.com>
Hello,
This series adds a way to write compressed data directly to Btrfs. The
intended use case is making send/receive on compressed file systems more
efficient; however, the interface is general enough that it could be
used in other scenarios. Patch 5 is the main change; see that for more
details.
Patches 1-3 are small fixes/cleanups that I ran into while implementing
this; they should go in regardless of the remainder of the series. Patch
4 exports a required VFS interface.
An example program and test case are available at [1].
To preemptively address a few concerns:
- Writing arbitrary, untrusted data which we feed to the decompression
algorithm can be a security risk. For that reason, the ioctl is
restricted to CAP_SYS_ADMIN. The Btrfs code is properly hardened
against invalid compressed data/incorrect lengths, and the compression
libraries are mature, but better safe than sorry for now.
- If the user is writing their own compressed data rather than just
blindly feeding in something from btrfs send, they need to know some
implementation details about the compression format. For zlib, there
are no special requirements. For zstd, a non-default compression
parameter must be used. For lzo, we have our own wrapper format since
lzo doesn't have a standard wrapper format. It feels a little wrong to
expose these details, but they are part of the on-disk format, so they
must be stable regardless.
- The permissions checks duplicated from the VFS code are fairly
minimal.
This series is based on misc-next.
This is an RFC, so please, comment away.
Thanks!
1: https://github.com/osandov/xfstests/tree/btrfs-compressed-write
Omar Sandoval (5):
Btrfs: use correct count in btrfs_file_write_iter()
Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs
Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree
fs: export rw_verify_area()
Btrfs: add ioctl for directly writing compressed data
fs/btrfs/compression.c | 6 +-
fs/btrfs/compression.h | 14 +--
fs/btrfs/ctree.h | 12 ++
fs/btrfs/extent_io.c | 6 +-
fs/btrfs/file.c | 22 ++--
fs/btrfs/free-space-cache.c | 9 +-
fs/btrfs/inode.c | 232 +++++++++++++++++++++++++++++++----
fs/btrfs/ioctl.c | 101 ++++++++++++++-
fs/btrfs/tests/inode-tests.c | 12 +-
fs/internal.h | 5 -
fs/read_write.c | 1 +
include/linux/fs.h | 1 +
include/uapi/linux/btrfs.h | 63 ++++++++++
13 files changed, 415 insertions(+), 69 deletions(-)
--
2.17.1
next reply other threads:[~2019-08-15 21:04 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-15 21:04 Omar Sandoval [this message]
2019-08-15 21:04 ` [PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter() Omar Sandoval
2019-08-16 16:56 ` Josef Bacik
2019-08-15 21:04 ` [PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs Omar Sandoval
2019-08-16 16:59 ` Josef Bacik
2019-08-27 12:35 ` David Sterba
2019-08-27 17:44 ` Omar Sandoval
2019-08-27 18:16 ` David Sterba
2019-08-15 21:04 ` [PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree Omar Sandoval
2019-08-16 16:59 ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 4/5] fs: export rw_verify_area() Omar Sandoval
2019-08-16 17:02 ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data Omar Sandoval
2019-08-26 21:36 ` Josef Bacik
2019-08-27 6:26 ` Nikolay Borisov
2019-08-27 11:57 ` Josef Bacik
2019-08-27 18:06 ` Omar Sandoval
2019-08-27 18:22 ` Omar Sandoval
2019-08-27 18:28 ` Josef Bacik
2019-08-28 12:06 ` David Sterba
2019-09-03 17:14 ` Omar Sandoval
2019-08-15 21:14 ` [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly Omar Sandoval
2019-08-27 18:31 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1565900769.git.osandov@fb.com \
--to=osandov@osandov.com \
--cc=kernel-team@fb.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).