linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH v2 2/5] btrfs: send: write larger chunks when using stream v2
Date: Wed, 18 Nov 2020 11:18:46 -0800	[thread overview]
Message-ID: <ea4b519738e34068c3558000555b5e8f2d40c3b5.1605723600.git.osandov@fb.com> (raw)
In-Reply-To: <cover.1605723600.git.osandov@fb.com>

From: Omar Sandoval <osandov@fb.com>

The length field of the send stream TLV header is 16 bits. This means
that the maximum amount of data that can be sent for one write is 64k
minus one. However, encoded writes must be able to send the maximum
compressed extent (128k) in one command. To support this, send stream
version 2 encodes the DATA attribute differently: it has no length
field, and the length is implicitly up to the end of containing command
(which has a 32-bit length field). Although this is necessary for
encoded writes, normal writes can benefit from it, too.

For v2, let's bump up the send buffer to the maximum compressed extent
size plus 16k for the other metadata (144k total). Since this will most
likely be vmalloc'd (and always will be after the next commit), we round
it up to the next page since we might as well use the rest of the page
on systems with >16k pages.

Signed-off-by: Omar Sandoval <osandov@fb.com>
---
 fs/btrfs/send.c | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 3cd090c3ffda..822458e6ffae 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -4918,14 +4918,27 @@ static inline u64 max_send_read_size(const struct send_ctx *sctx)
 
 static int put_data_header(struct send_ctx *sctx, u32 len)
 {
-	struct btrfs_tlv_header *hdr;
+	if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) {
+		/*
+		 * In v2, the data attribute header doesn't include a length; it
+		 * is implicitly to the end of the command.
+		 */
+		if (sctx->send_max_size - sctx->send_size < 2 + len)
+			return -EOVERFLOW;
+		put_unaligned_le16(BTRFS_SEND_A_DATA,
+				   sctx->send_buf + sctx->send_size);
+		sctx->send_size += 2;
+	} else {
+		struct btrfs_tlv_header *hdr;
 
-	if (sctx->send_max_size - sctx->send_size < sizeof(*hdr) + len)
-		return -EOVERFLOW;
-	hdr = (struct btrfs_tlv_header *)(sctx->send_buf + sctx->send_size);
-	put_unaligned_le16(BTRFS_SEND_A_DATA, &hdr->tlv_type);
-	put_unaligned_le16(len, &hdr->tlv_len);
-	sctx->send_size += sizeof(*hdr);
+		if (sctx->send_max_size - sctx->send_size < sizeof(*hdr) + len)
+			return -EOVERFLOW;
+		hdr = (struct btrfs_tlv_header *)(sctx->send_buf +
+						  sctx->send_size);
+		put_unaligned_le16(BTRFS_SEND_A_DATA, &hdr->tlv_type);
+		put_unaligned_le16(len, &hdr->tlv_len);
+		sctx->send_size += sizeof(*hdr);
+	}
 	return 0;
 }
 
@@ -7255,7 +7268,12 @@ long btrfs_ioctl_send(struct file *mnt_file, struct btrfs_ioctl_send_args *arg)
 
 	sctx->clone_roots_cnt = arg->clone_sources_count;
 
-	sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1;
+	if (sctx->flags & BTRFS_SEND_FLAG_STREAM_V2) {
+		sctx->send_max_size = ALIGN(SZ_16K + BTRFS_MAX_COMPRESSED,
+					    PAGE_SIZE);
+	} else {
+		sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1;
+	}
 	sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL);
 	if (!sctx->send_buf) {
 		ret = -ENOMEM;
-- 
2.29.2


  parent reply	other threads:[~2020-11-18 19:19 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-18 19:18 [PATCH v2 0/5] btrfs: implement send/receive of compressed extents without decompressing Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 1/5] btrfs: add send stream v2 definitions Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 01/13] btrfs-progs: send: fix crash on unknown option Omar Sandoval
2020-12-16 16:20   ` David Sterba
2020-11-18 19:18 ` [PATCH v2 02/13] btrfs-progs: receive: support v2 send stream larger tlv_len Omar Sandoval
2020-11-18 19:18 ` Omar Sandoval [this message]
2020-11-18 19:18 ` [PATCH v2 03/13] btrfs-progs: receive: dynamically allocate sctx->read_buf Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 3/5] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2 Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 04/13] btrfs-progs: receive: support v2 send stream DATA tlv format Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 4/5] btrfs: send: send compressed extents with encoded writes Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 05/13] btrfs-progs: receive: add send stream v2 cmds and attrs to send.h Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 5/5] btrfs: send: enable support for stream v2 and compressed writes Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 06/13] btrfs-progs: receive: add stub implementation for pwritev2 Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 07/13] btrfs-progs: receive: open files with O_CLOEXEC Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 08/13] btrfs-progs: receive: process encoded_write commands Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 09/13] btrfs-progs: receive: encoded_write fallback to explicit decode and write Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 10/13] btrfs-progs: receive: process fallocate commands Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 11/13] btrfs-progs: receive: process setflags ioctl commands Omar Sandoval
2020-11-18 19:18 ` [PATCH v2 12/13] btrfs-progs: send: stream v2 ioctl flags Omar Sandoval
2020-11-18 19:19 ` [PATCH v2 13/13] btrfs-progs: receive: add tests for basic encoded_write send/receive Omar Sandoval

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ea4b519738e34068c3558000555b5e8f2d40c3b5.1605723600.git.osandov@fb.com \
    --to=osandov@osandov.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).