linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC PATCH v3 00/12] fs: interface for directly reading/writing compressed data
  2019-11-20 18:24 ` [PATCH man-pages v2] Document encoded I/O Omar Sandoval
@ 2019-11-20 18:24 Omar Sandoval
  2019-11-20 18:24 ` [PATCH man-pages v2] Document encoded I/O Omar Sandoval
                   ` (12 more replies)
  0 siblings, 13 replies; 26+ messages in thread
From: Omar Sandoval @ 2019-11-20 18:24 UTC (permalink / raw)
  To: linux-fsdevel, linux-btrfs
  Cc: Dave Chinner, Jann Horn, Amir Goldstein, Aleksa Sarai, linux-api,
	kernel-team

From: Omar Sandoval <osandov@fb.com>

Hello,

This series adds an API for reading compressed data on a filesystem
without decompressing it as well as support for writing compressed data
directly to the filesystem. As with the previous submissions, I've
included a man page patch describing the API, and test cases and example
programs are available [1].

This version reworks the VFS interface to be backward and forward
compatible and support for writing inline and bookend extents to the
Btrfs implementation.

Patches 1-3 add the VFS support. Patches 4-7 are Btrfs cleanups
necessary for the encoded I/O support that can go in independently of
this series. Patches 8-10 are Btrfs prep patches. Patch 11 adds Btrfs
encoded read support and patch 12 adds Btrfs encoded write support.

A few TODOs remain:

- Once we've settled on the interface, I'll add RWF_ENCODED support to
  fsstress and friends and send up the xfstests patches in [1].
- btrfs_encoded_read() still doesn't implement repair.

Changes from v2 [2]:

- Rebase on v5.4-rc8
- Add patch 1 introducing copy_struct_from_iter() as suggested by Aleksa
- Rename O_ENCODED to O_ALLOW_ENCODED as suggested by Amir
- Add arch-specific definitions of O_ALLOW_ENCODED for alpha, parisc,
  and sparc
- Rework the VFS interface to be backward and forward compatible
- Document the VFS interface as requested by Dave
- Use __aligned_u64 for struct encoded_iov as noted by Aleksa
- Fix len/unencoded_len mixup in mm/filemap.c as noted by Nikolay
- Add support for writing inline and bookend extents to Btrfs
- Use ENOBUFS for "buffers not big enough for encoded extent" case and
  E2BIG for "encoded_iov has unsupported fields" case

Please share any comments on the API or implementation. Thanks!

1: https://github.com/osandov/xfstests/tree/rwf-encoded
2: https://lore.kernel.org/linux-btrfs/cover.1571164762.git.osandov@fb.com/

Omar Sandoval (12):
  iov_iter: add copy_struct_from_iter()
  fs: add O_ALLOW_ENCODED open flag
  fs: add RWF_ENCODED for reading/writing compressed data
  btrfs: get rid of trivial __btrfs_lookup_bio_sums() wrappers
  btrfs: don't advance offset for compressed bios in
    btrfs_csum_one_bio()
  btrfs: remove dead snapshot-aware defrag code
  btrfs: make btrfs_ordered_extent naming consistent with
    btrfs_file_extent_item
  btrfs: add ram_bytes and offset to btrfs_ordered_extent
  btrfs: support different disk extent size for delalloc
  btrfs: optionally extend i_size in cow_file_range_inline()
  btrfs: implement RWF_ENCODED reads
  btrfs: implement RWF_ENCODED writes

 Documentation/filesystems/encoded_io.rst |   79 +
 Documentation/filesystems/index.rst      |    1 +
 arch/alpha/include/uapi/asm/fcntl.h      |    1 +
 arch/parisc/include/uapi/asm/fcntl.h     |    1 +
 arch/sparc/include/uapi/asm/fcntl.h      |    1 +
 fs/btrfs/compression.c                   |   15 +-
 fs/btrfs/compression.h                   |    5 +-
 fs/btrfs/ctree.h                         |   13 +-
 fs/btrfs/delalloc-space.c                |   38 +-
 fs/btrfs/delalloc-space.h                |    4 +-
 fs/btrfs/file-item.c                     |   54 +-
 fs/btrfs/file.c                          |   61 +-
 fs/btrfs/inode.c                         | 2463 +++++++++++-----------
 fs/btrfs/ordered-data.c                  |  106 +-
 fs/btrfs/ordered-data.h                  |   28 +-
 fs/btrfs/relocation.c                    |    9 +-
 fs/fcntl.c                               |   10 +-
 fs/namei.c                               |    4 +
 include/linux/fcntl.h                    |    2 +-
 include/linux/fs.h                       |   16 +
 include/linux/uio.h                      |    2 +
 include/trace/events/btrfs.h             |    6 +-
 include/uapi/asm-generic/fcntl.h         |    4 +
 include/uapi/linux/fs.h                  |   33 +-
 lib/iov_iter.c                           |   82 +
 mm/filemap.c                             |  165 +-
 26 files changed, 1807 insertions(+), 1396 deletions(-)
 create mode 100644 Documentation/filesystems/encoded_io.rst

-- 
2.24.0


^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2019-12-05 18:58 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-20 18:24 [RFC PATCH v3 00/12] fs: interface for directly reading/writing compressed data Omar Sandoval
2019-11-20 18:24 ` [PATCH man-pages v2] Document encoded I/O Omar Sandoval
2019-12-05 18:58   ` [RFC PATCH v3 00/12] fs: interface for directly reading/writing compressed data Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 01/12] iov_iter: add copy_struct_from_iter() Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 02/12] fs: add O_ALLOW_ENCODED open flag Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 03/12] fs: add RWF_ENCODED for reading/writing compressed data Omar Sandoval
2019-11-26 13:53   ` Nikolay Borisov
2019-11-26 17:36     ` Omar Sandoval
2019-11-27  9:00       ` Nikolay Borisov
2019-11-27  9:10         ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 04/12] btrfs: get rid of trivial __btrfs_lookup_bio_sums() wrappers Omar Sandoval
2019-11-26 13:56   ` Nikolay Borisov
2019-11-26 17:42     ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 05/12] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio() Omar Sandoval
2019-11-26 14:18   ` Nikolay Borisov
2019-11-26 17:50     ` Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 06/12] btrfs: remove dead snapshot-aware defrag code Omar Sandoval
2019-11-26 15:13   ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 07/12] btrfs: make btrfs_ordered_extent naming consistent with btrfs_file_extent_item Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 08/12] btrfs: add ram_bytes and offset to btrfs_ordered_extent Omar Sandoval
2019-11-27 10:13   ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 09/12] btrfs: support different disk extent size for delalloc Omar Sandoval
2019-11-27 10:33   ` Nikolay Borisov
2019-11-20 18:24 ` [RFC PATCH v3 10/12] btrfs: optionally extend i_size in cow_file_range_inline() Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 11/12] btrfs: implement RWF_ENCODED reads Omar Sandoval
2019-11-20 18:24 ` [RFC PATCH v3 12/12] btrfs: implement RWF_ENCODED writes Omar Sandoval

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).