All of lore.kernel.org
 help / color / mirror / Atom feed
From: robbieko <robbieko@synology.com>
To: linux-btrfs@vger.kernel.org
Cc: Robbie Ko <robbieko@synology.com>
Subject: [PATCH] Btrfs : send, truncate first to enhance many small files
Date: Mon,  4 Dec 2017 15:02:39 +0800	[thread overview]
Message-ID: <1512370959-17182-1-git-send-email-robbieko@synology.com> (raw)

From: Robbie Ko <robbieko@synology.com>

The commands generated by send contain the following step:
1. mkfile o1851-19-0
2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c
3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c - name=user.xattr data_len=4 data=test
4. write alsa-driver/alsa-kernel/isa/es1688/es1688.c - offset=0, len=10458
5. truncate alsa-driver/alsa-kernel/isa/es1688/es1688.c size=10458
6. chown alsa-driver/alsa-kernel/isa/es1688/es1688.c - uid=1024, gid=100
7. chmod alsa-driver/alsa-kernel/isa/es1688/es1688.c - mode=0644
8. utimes alsa-driver/alsa-kernel/isa/es1688/es1688.c

After writing file content, it will truncate file to the correct size.
Btrfs truncate will flush last page if size does not align to sectorsize,
and this will cause receive process to wait until flush finishes.
In order to avoid waiting flushing data.This patch changes the order so
that truncate command is sent before write command.

Overall performance improves by 102 percent when sending 790000 small files.
original: 32m45.311s
patch: 16m8.387s

Signed-off-by: Robbie Ko <robbieko@synology.com>
---
 fs/btrfs/send.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 20d3300..7ae2347 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -5857,10 +5857,6 @@ static int finish_inode_if_needed(struct send_ctx *sctx, int at_end)
 					goto out;
 			}
 		}
-		ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen,
-				sctx->cur_inode_size);
-		if (ret < 0)
-			goto out;
 	}
 
 	if (need_chown) {
@@ -6044,6 +6040,15 @@ static int changed_inode(struct send_ctx *sctx,
 					sctx->left_path->nodes[0], left_ii);
 		}
 	}
+	if (result == BTRFS_COMPARE_TREE_NEW ||
+	    result == BTRFS_COMPARE_TREE_CHANGED) {
+		if (S_ISREG(sctx->cur_inode_mode)) {
+			ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen,
+						sctx->cur_inode_size);
+			if (ret < 0)
+				goto out;
+		}
+	}
 
 out:
 	return ret;
-- 
1.9.1


             reply	other threads:[~2017-12-04  7:03 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-12-04  7:02 robbieko [this message]
2017-12-04  7:19 ` [PATCH] Btrfs : send, truncate first to enhance many small files Qu Wenruo
2017-12-04 11:13   ` Filipe Manana
2017-12-04 11:28     ` Qu Wenruo
2017-12-04 12:08       ` Filipe Manana
2017-12-08  6:29         ` robbieko
2017-12-08  6:37           ` Qu Wenruo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1512370959-17182-1-git-send-email-robbieko@synology.com \
    --to=robbieko@synology.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.