From: Omar Sandoval <osandov@osandov.com>
To: linux-fsdevel@vger.kernel.org, linux-btrfs@vger.kernel.org,
Al Viro <viro@zeniv.linux.org.uk>,
Christoph Hellwig <hch@infradead.org>
Cc: Dave Chinner <david@fromorbit.com>, Jann Horn <jannh@google.com>,
Amir Goldstein <amir73il@gmail.com>,
Aleksa Sarai <cyphar@cyphar.com>,
linux-api@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH v7 00/10] fs: interface for directly reading/writing compressed data
Date: Fri, 22 Jan 2021 12:46:46 -0800 [thread overview]
Message-ID: <cover.1611346706.git.osandov@fb.com> (raw)
From: Omar Sandoval <osandov@fb.com>
This series adds an API for reading compressed data on a filesystem
without decompressing it as well as support for writing compressed data
directly to the filesystem. As with the previous submissions, I've
included a man page patch describing the API. I have test cases
(including fsstress support) and example programs which I'll send up
[1].
The main use-case is Btrfs send/receive: currently, when sending data
from one compressed filesystem to another, the sending side decompresses
the data and the receiving side recompresses it before writing it out.
This is wasteful and can be avoided if we can just send and write
compressed extents. The patches implementing the send/receive support
will be sent shortly.
Patches 1-3 add the VFS support and UAPI. Patch 4 is a fix that this
series depends on; it can be merged independently. Patches 5-8 are Btrfs
prep patches. Patch 9 adds Btrfs encoded read support and patch 10 adds
Btrfs encoded write support.
These patches are based on Dave Sterba's Btrfs misc-next branch [2],
which is in turn currently based on v5.11-rc4.
Changes since v6 [3]:
- Dropped O_CLOEXEC requirement for O_ALLOW_ENCODED.
- Readded lost FMODE_ENCODED_IO check.
- Used macros instead of enum for ENCODED_IOV_COMPRESSION_* and
ENCODED_IOV_ENCRYPTION_*.
- Moved encoded I/O definitions to their own UAPI header file.
- Fixed style nits.
- Added reviewed-bys.
- Addressed man page nits.
1: https://github.com/osandov/xfstests/tree/rwf-encoded
2: https://github.com/kdave/btrfs-devel/tree/misc-next
3: https://lore.kernel.org/linux-btrfs/cover.1605723568.git.osandov@fb.com/
Omar Sandoval (10):
iov_iter: add copy_struct_from_iter()
fs: add O_ALLOW_ENCODED open flag
fs: add RWF_ENCODED for reading/writing compressed data
btrfs: fix check_data_csum() error message for direct I/O
btrfs: don't advance offset for compressed bios in
btrfs_csum_one_bio()
btrfs: add ram_bytes and offset to btrfs_ordered_extent
btrfs: support different disk extent size for delalloc
btrfs: optionally extend i_size in cow_file_range_inline()
btrfs: implement RWF_ENCODED reads
btrfs: implement RWF_ENCODED writes
Documentation/filesystems/encoded_io.rst | 74 ++
Documentation/filesystems/index.rst | 1 +
arch/alpha/include/uapi/asm/fcntl.h | 1 +
arch/parisc/include/uapi/asm/fcntl.h | 1 +
arch/sparc/include/uapi/asm/fcntl.h | 1 +
fs/btrfs/compression.c | 12 +-
fs/btrfs/compression.h | 6 +-
fs/btrfs/ctree.h | 9 +-
fs/btrfs/delalloc-space.c | 18 +-
fs/btrfs/file-item.c | 35 +-
fs/btrfs/file.c | 46 +-
fs/btrfs/inode.c | 936 ++++++++++++++++++++---
fs/btrfs/ordered-data.c | 80 +-
fs/btrfs/ordered-data.h | 18 +-
fs/btrfs/relocation.c | 4 +-
fs/fcntl.c | 10 +-
fs/namei.c | 4 +
fs/read_write.c | 168 +++-
include/linux/encoded_io.h | 17 +
include/linux/fcntl.h | 2 +-
include/linux/fs.h | 13 +
include/linux/uio.h | 2 +
include/uapi/asm-generic/fcntl.h | 4 +
include/uapi/linux/encoded_io.h | 30 +
include/uapi/linux/fs.h | 5 +-
lib/iov_iter.c | 82 ++
26 files changed, 1378 insertions(+), 201 deletions(-)
create mode 100644 Documentation/filesystems/encoded_io.rst
create mode 100644 include/linux/encoded_io.h
create mode 100644 include/uapi/linux/encoded_io.h
--
2.30.0
next reply other threads:[~2021-01-22 20:50 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-22 20:46 Omar Sandoval [this message]
2021-01-22 20:46 ` [PATCH man-pages v7] Document encoded I/O Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 01/10] iov_iter: add copy_struct_from_iter() Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 02/10] fs: add O_ALLOW_ENCODED open flag Omar Sandoval
2021-01-25 20:57 ` Josef Bacik
2021-01-22 20:46 ` [PATCH v7 03/10] fs: add RWF_ENCODED for reading/writing compressed data Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 04/10] btrfs: fix check_data_csum() error message for direct I/O Omar Sandoval
2021-01-25 20:59 ` Josef Bacik
2021-01-22 20:46 ` [PATCH v7 05/10] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio() Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 06/10] btrfs: add ram_bytes and offset to btrfs_ordered_extent Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 07/10] btrfs: support different disk extent size for delalloc Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 08/10] btrfs: optionally extend i_size in cow_file_range_inline() Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 09/10] btrfs: implement RWF_ENCODED reads Omar Sandoval
2021-01-22 20:46 ` [PATCH v7 10/10] btrfs: implement RWF_ENCODED writes Omar Sandoval
2021-01-29 16:32 ` [PATCH v7 00/10] fs: interface for directly reading/writing compressed data Josef Bacik
2021-02-03 16:03 ` Omar Sandoval
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1611346706.git.osandov@fb.com \
--to=osandov@osandov.com \
--cc=amir73il@gmail.com \
--cc=cyphar@cyphar.com \
--cc=david@fromorbit.com \
--cc=hch@infradead.org \
--cc=jannh@google.com \
--cc=kernel-team@fb.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.