From: Omar Sandoval <osandov@osandov.com>
To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
Al Viro <viro@zeniv.linux.org.uk>
Cc: Dave Chinner <david@fromorbit.com>, Jann Horn <jannh@google.com>,
Amir Goldstein <amir73il@gmail.com>,
Aleksa Sarai <cyphar@cyphar.com>,
linux-api@vger.kernel.org, kernel-team@fb.com
Subject: Re: [RFC PATCH v3 00/12] fs: interface for directly reading/writing compressed data
Date: Thu, 5 Dec 2019 10:58:23 -0800 [thread overview]
Message-ID: <20191205185823.GB18377@vader> (raw)
In-Reply-To: <cover.1574273658.git.osandov@fb.com>
On Wed, Nov 20, 2019 at 10:24:20AM -0800, Omar Sandoval wrote:
> From: Omar Sandoval <osandov@fb.com>
>
> Hello,
>
> This series adds an API for reading compressed data on a filesystem
> without decompressing it as well as support for writing compressed data
> directly to the filesystem. As with the previous submissions, I've
> included a man page patch describing the API, and test cases and example
> programs are available [1].
>
> This version reworks the VFS interface to be backward and forward
> compatible and support for writing inline and bookend extents to the
> Btrfs implementation.
>
> Patches 1-3 add the VFS support. Patches 4-7 are Btrfs cleanups
> necessary for the encoded I/O support that can go in independently of
> this series. Patches 8-10 are Btrfs prep patches. Patch 11 adds Btrfs
> encoded read support and patch 12 adds Btrfs encoded write support.
>
> A few TODOs remain:
>
> - Once we've settled on the interface, I'll add RWF_ENCODED support to
> fsstress and friends and send up the xfstests patches in [1].
> - btrfs_encoded_read() still doesn't implement repair.
>
> Changes from v2 [2]:
>
> - Rebase on v5.4-rc8
> - Add patch 1 introducing copy_struct_from_iter() as suggested by Aleksa
> - Rename O_ENCODED to O_ALLOW_ENCODED as suggested by Amir
> - Add arch-specific definitions of O_ALLOW_ENCODED for alpha, parisc,
> and sparc
> - Rework the VFS interface to be backward and forward compatible
> - Document the VFS interface as requested by Dave
> - Use __aligned_u64 for struct encoded_iov as noted by Aleksa
> - Fix len/unencoded_len mixup in mm/filemap.c as noted by Nikolay
> - Add support for writing inline and bookend extents to Btrfs
> - Use ENOBUFS for "buffers not big enough for encoded extent" case and
> E2BIG for "encoded_iov has unsupported fields" case
>
> Please share any comments on the API or implementation. Thanks!
>
> 1: https://github.com/osandov/xfstests/tree/rwf-encoded
> 2: https://lore.kernel.org/linux-btrfs/cover.1571164762.git.osandov@fb.com/
>
> Omar Sandoval (12):
> iov_iter: add copy_struct_from_iter()
> fs: add O_ALLOW_ENCODED open flag
> fs: add RWF_ENCODED for reading/writing compressed data
> btrfs: get rid of trivial __btrfs_lookup_bio_sums() wrappers
> btrfs: don't advance offset for compressed bios in
> btrfs_csum_one_bio()
> btrfs: remove dead snapshot-aware defrag code
> btrfs: make btrfs_ordered_extent naming consistent with
> btrfs_file_extent_item
> btrfs: add ram_bytes and offset to btrfs_ordered_extent
> btrfs: support different disk extent size for delalloc
> btrfs: optionally extend i_size in cow_file_range_inline()
> btrfs: implement RWF_ENCODED reads
> btrfs: implement RWF_ENCODED writes
>
> Documentation/filesystems/encoded_io.rst | 79 +
> Documentation/filesystems/index.rst | 1 +
> arch/alpha/include/uapi/asm/fcntl.h | 1 +
> arch/parisc/include/uapi/asm/fcntl.h | 1 +
> arch/sparc/include/uapi/asm/fcntl.h | 1 +
> fs/btrfs/compression.c | 15 +-
> fs/btrfs/compression.h | 5 +-
> fs/btrfs/ctree.h | 13 +-
> fs/btrfs/delalloc-space.c | 38 +-
> fs/btrfs/delalloc-space.h | 4 +-
> fs/btrfs/file-item.c | 54 +-
> fs/btrfs/file.c | 61 +-
> fs/btrfs/inode.c | 2463 +++++++++++-----------
> fs/btrfs/ordered-data.c | 106 +-
> fs/btrfs/ordered-data.h | 28 +-
> fs/btrfs/relocation.c | 9 +-
> fs/fcntl.c | 10 +-
> fs/namei.c | 4 +
> include/linux/fcntl.h | 2 +-
> include/linux/fs.h | 16 +
> include/linux/uio.h | 2 +
> include/trace/events/btrfs.h | 6 +-
> include/uapi/asm-generic/fcntl.h | 4 +
> include/uapi/linux/fs.h | 33 +-
> lib/iov_iter.c | 82 +
> mm/filemap.c | 165 +-
> 26 files changed, 1807 insertions(+), 1396 deletions(-)
> create mode 100644 Documentation/filesystems/encoded_io.rst
Ping. Al, would you mind taking a look at the generic bits/interface?
next prev parent reply other threads:[~2019-12-05 18:58 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-20 18:24 [RFC PATCH v3 00/12] fs: interface for directly reading/writing compressed data Omar Sandoval
2019-11-20 18:24 ` [PATCH man-pages v2] Document encoded I/O Omar Sandoval
2019-12-05 18:58 ` Omar Sandoval [this message]
2019-11-20 18:24 ` [RFC PATCH v3 01/12] iov_iter: add copy_struct_from_iter() Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 02/12] fs: add O_ALLOW_ENCODED open flag Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 03/12] fs: add RWF_ENCODED for reading/writing compressed data Omar Sandoval
2019-11-26 13:53 ` Nikolay Borisov
2019-11-26 17:36 ` Omar Sandoval
2019-11-27 9:00 ` Nikolay Borisov
2019-11-27 9:10 ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 04/12] btrfs: get rid of trivial __btrfs_lookup_bio_sums() wrappers Omar Sandoval
2019-11-26 13:56 ` Nikolay Borisov
2019-11-26 17:42 ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 05/12] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio() Omar Sandoval
2019-11-26 14:18 ` Nikolay Borisov
2019-11-26 17:50 ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 06/12] btrfs: remove dead snapshot-aware defrag code Omar Sandoval
2019-11-26 15:13 ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 07/12] btrfs: make btrfs_ordered_extent naming consistent with btrfs_file_extent_item Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 08/12] btrfs: add ram_bytes and offset to btrfs_ordered_extent Omar Sandoval
2019-11-27 10:13 ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 09/12] btrfs: support different disk extent size for delalloc Omar Sandoval
2019-11-27 10:33 ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 10/12] btrfs: optionally extend i_size in cow_file_range_inline() Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 11/12] btrfs: implement RWF_ENCODED reads Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 12/12] btrfs: implement RWF_ENCODED writes Omar Sandoval
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191205185823.GB18377@vader \
--to=osandov@osandov.com \
--cc=amir73il@gmail.com \
--cc=cyphar@cyphar.com \
--cc=david@fromorbit.com \
--cc=jannh@google.com \
--cc=kernel-team@fb.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).