linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: kernel-team@fb.com, linux-fsdevel@vger.kernel.org,
	Al Viro <viro@zeniv.linux.org.uk>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	linux-api@vger.kernel.org
Subject: [PATCH v10 10/14] btrfs: add send stream v2 definitions
Date: Tue, 17 Aug 2021 14:06:42 -0700	[thread overview]
Message-ID: <d60216724a767a7dc4063f546cd7abac984012fd.1629234193.git.osandov@fb.com> (raw)
In-Reply-To: <cover.1629234193.git.osandov@fb.com>

From: Omar Sandoval <osandov@fb.com>

This adds the definitions of the new commands for send stream version 2
and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a.
chattr), and encoded writes. It also documents two changes to the send
stream format in v2: the receiver shouldn't assume a maximum command
size, and the DATA attribute is encoded differently to allow for writes
larger than 64k. These will be implemented in subsequent changes, and
then the ioctl will accept the new flags.

Reviewed-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
---
 fs/btrfs/send.c            |  2 +-
 fs/btrfs/send.h            | 30 +++++++++++++++++++++++++++++-
 include/uapi/linux/btrfs.h | 13 +++++++++++++
 3 files changed, 43 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 72f9b865e847..02ac158837ce 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -7294,7 +7294,7 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg)
 
 	sctx->clone_roots_cnt = arg->clone_sources_count;
 
-	sctx->send_max_size = BTRFS_SEND_BUF_SIZE;
+	sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1;
 	sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL);
 	if (!sctx->send_buf) {
 		ret = -ENOMEM;
diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h
index de91488b7cd0..9f4f7b96b1eb 100644
--- a/fs/btrfs/send.h
+++ b/fs/btrfs/send.h
@@ -12,7 +12,11 @@
 #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream"
 #define BTRFS_SEND_STREAM_VERSION 1
 
-#define BTRFS_SEND_BUF_SIZE SZ_64K
+/*
+ * In send stream v1, no command is larger than 64k. In send stream v2, no limit
+ * should be assumed.
+ */
+#define BTRFS_SEND_BUF_SIZE_V1 SZ_64K
 
 enum btrfs_tlv_type {
 	BTRFS_TLV_U8,
@@ -76,6 +80,13 @@ enum btrfs_send_cmd {
 
 	BTRFS_SEND_C_END,
 	BTRFS_SEND_C_UPDATE_EXTENT,
+
+	/* The following commands were added in send stream v2. */
+
+	BTRFS_SEND_C_FALLOCATE,
+	BTRFS_SEND_C_SETFLAGS,
+	BTRFS_SEND_C_ENCODED_WRITE,
+
 	__BTRFS_SEND_C_MAX,
 };
 #define BTRFS_SEND_C_MAX (__BTRFS_SEND_C_MAX - 1)
@@ -106,6 +117,11 @@ enum {
 	BTRFS_SEND_A_PATH_LINK,
 
 	BTRFS_SEND_A_FILE_OFFSET,
+	/*
+	 * In send stream v2, this attribute is special: it must be the last
+	 * attribute in a command, its header contains only the type, and its
+	 * length is implicitly the remaining length of the command.
+	 */
 	BTRFS_SEND_A_DATA,
 
 	BTRFS_SEND_A_CLONE_UUID,
@@ -114,6 +130,18 @@ enum {
 	BTRFS_SEND_A_CLONE_OFFSET,
 	BTRFS_SEND_A_CLONE_LEN,
 
+	/* The following attributes were added in send stream v2. */
+
+	BTRFS_SEND_A_FALLOCATE_MODE,
+
+	BTRFS_SEND_A_SETFLAGS_FLAGS,
+
+	BTRFS_SEND_A_UNENCODED_FILE_LEN,
+	BTRFS_SEND_A_UNENCODED_LEN,
+	BTRFS_SEND_A_UNENCODED_OFFSET,
+	BTRFS_SEND_A_COMPRESSION,
+	BTRFS_SEND_A_ENCRYPTION,
+
 	__BTRFS_SEND_A_MAX,
 };
 #define BTRFS_SEND_A_MAX (__BTRFS_SEND_A_MAX - 1)
diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h
index 95da52955894..4f875f355e83 100644
--- a/include/uapi/linux/btrfs.h
+++ b/include/uapi/linux/btrfs.h
@@ -771,6 +771,19 @@ struct btrfs_ioctl_received_subvol_args {
  */
 #define BTRFS_SEND_FLAG_OMIT_END_CMD		0x4
 
+/*
+ * Use version 2 of the send stream, which adds new commands and supports larger
+ * writes.
+ */
+#define BTRFS_SEND_FLAG_STREAM_V2		0x8
+
+/*
+ * Send compressed data using the ENCODED_WRITE command instead of decompressing
+ * the data and sending it with the WRITE command. This requires
+ * BTRFS_SEND_FLAG_STREAM_V2.
+ */
+#define BTRFS_SEND_FLAG_COMPRESSED		0x10
+
 #define BTRFS_SEND_FLAG_MASK \
 	(BTRFS_SEND_FLAG_NO_FILE_DATA | \
 	 BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \
-- 
2.32.0


  parent reply	other threads:[~2021-08-17 21:07 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-17 21:06 [PATCH v10 00/14] btrfs: add ioctls and send/receive support for reading/writing compressed data Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 01/14] fs: export rw_verify_area() Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 02/14] fs: export variant of generic_write_checks without iov_iter Omar Sandoval
2021-08-20  7:59   ` Nikolay Borisov
2021-08-20 17:31     ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 03/14] btrfs: don't advance offset for compressed bios in btrfs_csum_one_bio() Omar Sandoval
2021-08-20  8:08   ` Nikolay Borisov
2021-08-20 17:37     ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 04/14] btrfs: add ram_bytes and offset to btrfs_ordered_extent Omar Sandoval
2021-08-20  8:34   ` Nikolay Borisov
2021-08-20 17:43     ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 05/14] btrfs: support different disk extent size for delalloc Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 06/14] btrfs: optionally extend i_size in cow_file_range_inline() Omar Sandoval
2021-08-20  8:51   ` Nikolay Borisov
2021-08-20  9:13     ` Qu Wenruo
2021-08-20 18:11       ` Omar Sandoval
2021-08-21  1:11         ` Qu Wenruo
2021-08-23 18:16           ` Omar Sandoval
2021-08-23 23:32             ` Qu Wenruo
2021-08-23 23:46               ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 07/14] btrfs: add definitions + documentation for encoded I/O ioctls Omar Sandoval
2021-08-20  8:56   ` Nikolay Borisov
2021-08-20 17:48     ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 08/14] btrfs: add BTRFS_IOC_ENCODED_READ Omar Sandoval
2021-08-20 12:30   ` Nikolay Borisov
2021-08-20 17:58     ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 09/14] btrfs: add BTRFS_IOC_ENCODED_WRITE Omar Sandoval
2021-08-20 13:44   ` Nikolay Borisov
2021-08-20 17:59     ` Omar Sandoval
2021-08-17 21:06 ` Omar Sandoval [this message]
2021-08-17 21:06 ` [PATCH v10 11/14] btrfs: send: write larger chunks when using stream v2 Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 12/14] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2 Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 13/14] btrfs: send: send compressed extents with encoded writes Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 14/14] btrfs: send: enable support for stream v2 and compressed writes Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 01/10] btrfs-progs: receive: support v2 send stream larger tlv_len Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 02/10] btrfs-progs: receive: dynamically allocate sctx->read_buf Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 03/10] btrfs-progs: receive: support v2 send stream DATA tlv format Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 04/10] btrfs-progs: receive: add send stream v2 cmds and attrs to send.h Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 05/10] btrfs-progs: receive: process encoded_write commands Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 06/10] btrfs-progs: receive: encoded_write fallback to explicit decode and write Omar Sandoval
2021-08-18 18:07   ` Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 07/10] btrfs-progs: receive: process fallocate commands Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 08/10] btrfs-progs: receive: process setflags ioctl commands Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 09/10] btrfs-progs: send: stream v2 ioctl flags Omar Sandoval
2021-08-17 21:06 ` [PATCH v10 10/10] btrfs-progs: receive: add tests for basic encoded_write send/receive Omar Sandoval

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d60216724a767a7dc4063f546cd7abac984012fd.1629234193.git.osandov@fb.com \
    --to=osandov@osandov.com \
    --cc=kernel-team@fb.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=torvalds@linux-foundation.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).