Linux-BTRFS Archive on lore.kernel.org
 help / color / Atom feed
From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: kernel-team@fb.com
Subject: Re: [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly
Date: Thu, 15 Aug 2019 14:14:54 -0700
Message-ID: <20190815211454.GC27472@vader> (raw)
In-Reply-To: <cover.1565900769.git.osandov@fb.com>

On Thu, Aug 15, 2019 at 02:04:01PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
> 
> Hello,
> 
> This series adds a way to write compressed data directly to Btrfs. The
> intended use case is making send/receive on compressed file systems more
> efficient; however, the interface is general enough that it could be
> used in other scenarios. Patch 5 is the main change; see that for more
> details.
> 
> Patches 1-3 are small fixes/cleanups that I ran into while implementing
> this; they should go in regardless of the remainder of the series. Patch
> 4 exports a required VFS interface.
> 
> An example program and test case are available at [1].
> 
> To preemptively address a few concerns:
> 
> - Writing arbitrary, untrusted data which we feed to the decompression
>   algorithm can be a security risk. For that reason, the ioctl is
>   restricted to CAP_SYS_ADMIN. The Btrfs code is properly hardened
>   against invalid compressed data/incorrect lengths, and the compression
>   libraries are mature, but better safe than sorry for now.
> - If the user is writing their own compressed data rather than just
>   blindly feeding in something from btrfs send, they need to know some
>   implementation details about the compression format. For zlib, there
>   are no special requirements. For zstd, a non-default compression
>   parameter must be used. For lzo, we have our own wrapper format since
>   lzo doesn't have a standard wrapper format. It feels a little wrong to
>   expose these details, but they are part of the on-disk format, so they
>   must be stable regardless.
> - The permissions checks duplicated from the VFS code are fairly
>   minimal.
> 
> This series is based on misc-next.
> 
> This is an RFC, so please, comment away.
> 
> Thanks!
> 
> 1: https://github.com/osandov/xfstests/tree/btrfs-compressed-write
> 
> Omar Sandoval (5):
>   Btrfs: use correct count in btrfs_file_write_iter()
>   Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs
>   Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree
>   fs: export rw_verify_area()
>   Btrfs: add ioctl for directly writing compressed data
> 
>  fs/btrfs/compression.c       |   6 +-
>  fs/btrfs/compression.h       |  14 +--
>  fs/btrfs/ctree.h             |  12 ++
>  fs/btrfs/extent_io.c         |   6 +-
>  fs/btrfs/file.c              |  22 ++--
>  fs/btrfs/free-space-cache.c  |   9 +-
>  fs/btrfs/inode.c             | 232 +++++++++++++++++++++++++++++++----
>  fs/btrfs/ioctl.c             | 101 ++++++++++++++-
>  fs/btrfs/tests/inode-tests.c |  12 +-
>  fs/internal.h                |   5 -
>  fs/read_write.c              |   1 +
>  include/linux/fs.h           |   1 +
>  include/uapi/linux/btrfs.h   |  63 ++++++++++
>  13 files changed, 415 insertions(+), 69 deletions(-)

I forgot to CC fsdevel. I'll do that for v2.

      parent reply index

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-15 21:04 Omar Sandoval
2019-08-15 21:04 ` [PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter() Omar Sandoval
2019-08-16 16:56   ` Josef Bacik
2019-08-15 21:04 ` [PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs Omar Sandoval
2019-08-16 16:59   ` Josef Bacik
2019-08-15 21:04 ` [PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree Omar Sandoval
2019-08-16 16:59   ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 4/5] fs: export rw_verify_area() Omar Sandoval
2019-08-16 17:02   ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data Omar Sandoval
2019-08-15 21:14 ` Omar Sandoval [this message]

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190815211454.GC27472@vader \
    --to=osandov@osandov.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox