linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Josef Bacik <josef@toxicpanda.com>
To: Omar Sandoval <osandov@osandov.com>
Cc: linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: Re: [RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data
Date: Mon, 26 Aug 2019 17:36:19 -0400	[thread overview]
Message-ID: <20190826213618.qdsivmmwwlxkqtxc@macbook-pro-91.dhcp.thefacebook.com> (raw)
In-Reply-To: <78747c3028ce91db9856e7fbd98ccbb2609acdc6.1565900769.git.osandov@fb.com>

On Thu, Aug 15, 2019 at 02:04:06PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
> 
> This adds an API for writing compressed data directly to the filesystem.
> The use case that I have in mind is send/receive: currently, when
> sending data from one compressed filesystem to another, the sending side
> decompresses the data and the receiving side recompresses it before
> writing it out. This is wasteful and can be avoided if we can just send
> and write compressed extents. The send part will be implemented in a
> separate series, as this ioctl can stand alone.
> 
> The interface is essentially pwrite(2) with some extra information:
> 
> - The input buffer contains the compressed data.
> - Both the compressed and decompressed sizes of the data are given.
> - The compression type (zlib, lzo, or zstd) is given.
> 
> A more detailed description of the interface, including restrictions and
> edge cases, is included in include/uapi/linux/btrfs.h.
> 
> The implementation is similar to direct I/O: we have to flush any
> ordered extents, invalidate the page cache, and do the io
> tree/delalloc/extent map/ordered extent dance. From there, we can reuse
> the compression code with a minor modification to distinguish the new
> ioctl from writeback.
>

I've looked at this a few times, the locking and space reservation stuff look
right.  What about encrypted send/recieve?  Are we going to want to use this to
just blind copy encrypted data without having to decrypt/re-encrypt?  Should
this be taken into consideration for this interface?  I'll think more about it,
but I can't really see any better option than this.  Thanks,

Josef 

  reply	other threads:[~2019-08-26 21:36 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-15 21:04 [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly Omar Sandoval
2019-08-15 21:04 ` [PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter() Omar Sandoval
2019-08-16 16:56   ` Josef Bacik
2019-08-15 21:04 ` [PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs Omar Sandoval
2019-08-16 16:59   ` Josef Bacik
2019-08-27 12:35   ` David Sterba
2019-08-27 17:44     ` Omar Sandoval
2019-08-27 18:16       ` David Sterba
2019-08-15 21:04 ` [PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree Omar Sandoval
2019-08-16 16:59   ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 4/5] fs: export rw_verify_area() Omar Sandoval
2019-08-16 17:02   ` Josef Bacik
2019-08-15 21:04 ` [RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data Omar Sandoval
2019-08-26 21:36   ` Josef Bacik [this message]
2019-08-27  6:26     ` Nikolay Borisov
2019-08-27 11:57       ` Josef Bacik
2019-08-27 18:06         ` Omar Sandoval
2019-08-27 18:22           ` Omar Sandoval
2019-08-27 18:28             ` Josef Bacik
2019-08-28 12:06   ` David Sterba
2019-09-03 17:14     ` Omar Sandoval
2019-08-15 21:14 ` [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly Omar Sandoval
2019-08-27 18:31 ` David Sterba

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190826213618.qdsivmmwwlxkqtxc@macbook-pro-91.dhcp.thefacebook.com \
    --to=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=osandov@osandov.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).