Re: [PATCH RESEND v9 0/9] fs: interface for directly reading/writing compressed data

From: Omar Sandoval <osandov@osandov.com>
To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
	Al Viro <viro@zeniv.linux.org.uk>,
	Christoph Hellwig <hch@infradead.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
	Dave Chinner <david@fromorbit.com>, Jann Horn <jannh@google.com>,
	Amir Goldstein <amir73il@gmail.com>,
	Aleksa Sarai <cyphar@cyphar.com>,
	linux-api@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH RESEND v9 0/9] fs: interface for directly reading/writing compressed data
Date: Tue, 4 May 2021 10:15:51 -0700	[thread overview]
Message-ID: <YJGBR5SnnQeJdIb1@relinquished.localdomain> (raw)
In-Reply-To: <cover.1619463858.git.osandov@fb.com>

On Mon, Apr 26, 2021 at 12:06:03PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
> 
> This series adds an API for reading compressed data on a filesystem
> without decompressing it as well as support for writing compressed data
> directly to the filesystem. I have test cases (including fsstress
> support) and example programs which I'll send up once the dust settles
> [1].
> 
> The main use-case is Btrfs send/receive: currently, when sending data
> from one compressed filesystem to another, the sending side decompresses
> the data and the receiving side recompresses it before writing it out.
> This is wasteful and can be avoided if we can just send and write
> compressed extents. The patches implementing the send/receive support
> were sent with the last submission of this series [2].
> 
> Patches 1-3 add the VFS support, UAPI, and documentation. Patches 4-7
> are Btrfs prep patches. Patch 8 adds Btrfs encoded read support and
> patch 9 adds Btrfs encoded write support.
> 
> These patches are based on Dave Sterba's Btrfs misc-next branch [3],
> which is in turn currently based on v5.12-rc8.
> 
> This is a resend of v9 [4], rebased on the latest kdave/misc-next
> branch.
> 
> Omar Sandoval (9):
>   iov_iter: add copy_struct_from_iter()
>   fs: add O_ALLOW_ENCODED open flag
>   fs: add RWF_ENCODED for reading/writing compressed data
>   btrfs: don't advance offset for compressed bios in
>     btrfs_csum_one_bio()
>   btrfs: add ram_bytes and offset to btrfs_ordered_extent
>   btrfs: support different disk extent size for delalloc
>   btrfs: optionally extend i_size in cow_file_range_inline()
>   btrfs: implement RWF_ENCODED reads
>   btrfs: implement RWF_ENCODED writes
> 
> 1: https://github.com/osandov/xfstests/tree/rwf-encoded
> 2: https://lore.kernel.org/linux-btrfs/cover.1615922753.git.osandov@fb.com/
> 3: https://github.com/kdave/btrfs-devel/tree/misc-next
> 4: https://lore.kernel.org/linux-btrfs/cover.1617258892.git.osandov@fb.com/
> 
> Omar Sandoval (9):
>   iov_iter: add copy_struct_from_iter()
>   fs: add O_ALLOW_ENCODED open flag
>   fs: add RWF_ENCODED for reading/writing compressed data
>   btrfs: don't advance offset for compressed bios in
>     btrfs_csum_one_bio()
>   btrfs: add ram_bytes and offset to btrfs_ordered_extent
>   btrfs: support different disk extent size for delalloc
>   btrfs: optionally extend i_size in cow_file_range_inline()
>   btrfs: implement RWF_ENCODED reads
>   btrfs: implement RWF_ENCODED writes
> 
>  Documentation/filesystems/encoded_io.rst | 240 ++++++
>  Documentation/filesystems/index.rst      |   1 +
>  arch/alpha/include/uapi/asm/fcntl.h      |   1 +
>  arch/parisc/include/uapi/asm/fcntl.h     |   1 +
>  arch/sparc/include/uapi/asm/fcntl.h      |   1 +
>  fs/btrfs/compression.c                   |  12 +-
>  fs/btrfs/compression.h                   |   6 +-
>  fs/btrfs/ctree.h                         |   9 +-
>  fs/btrfs/delalloc-space.c                |  18 +-
>  fs/btrfs/file-item.c                     |  35 +-
>  fs/btrfs/file.c                          |  46 +-
>  fs/btrfs/inode.c                         | 929 +++++++++++++++++++++--
>  fs/btrfs/ordered-data.c                  | 124 +--
>  fs/btrfs/ordered-data.h                  |  25 +-
>  fs/btrfs/relocation.c                    |   4 +-
>  fs/fcntl.c                               |  10 +-
>  fs/namei.c                               |   4 +
>  fs/read_write.c                          | 168 +++-
>  include/linux/encoded_io.h               |  17 +
>  include/linux/fcntl.h                    |   2 +-
>  include/linux/fs.h                       |  13 +
>  include/linux/uio.h                      |   1 +
>  include/uapi/asm-generic/fcntl.h         |   4 +
>  include/uapi/linux/encoded_io.h          |  30 +
>  include/uapi/linux/fs.h                  |   5 +-
>  lib/iov_iter.c                           |  91 +++
>  26 files changed, 1563 insertions(+), 234 deletions(-)
>  create mode 100644 Documentation/filesystems/encoded_io.rst
>  create mode 100644 include/linux/encoded_io.h
>  create mode 100644 include/uapi/linux/encoded_io.h

Ping.