Re: [PATCH] Btrfs : send, truncate first to enhance many small files

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2560 bytes --]



On 2017年12月04日 15:02, robbieko wrote:
> From: Robbie Ko <robbieko@synology.com>
> 
> The commands generated by send contain the following step:
> 1. mkfile o1851-19-0
> 2. rename o1851-19-0 -> alsa-driver/alsa-kernel/isa/es1688/es1688.c
> 3. set_xattr alsa-driver/alsa-kernel/isa/es1688/es1688.c - name=user.xattr data_len=4 data=test
> 4. write alsa-driver/alsa-kernel/isa/es1688/es1688.c - offset=0, len=10458
> 5. truncate alsa-driver/alsa-kernel/isa/es1688/es1688.c size=10458
> 6. chown alsa-driver/alsa-kernel/isa/es1688/es1688.c - uid=1024, gid=100
> 7. chmod alsa-driver/alsa-kernel/isa/es1688/es1688.c - mode=0644
> 8. utimes alsa-driver/alsa-kernel/isa/es1688/es1688.c
> 
> After writing file content, it will truncate file to the correct size.
> Btrfs truncate will flush last page if size does not align to sectorsize,
> and this will cause receive process to wait until flush finishes.
> In order to avoid waiting flushing data.This patch changes the order so
> that truncate command is sent before write command.

Personally speaking, it's better to optimize the receive side.

For example, at receive side, if we already know that the file size is
not changed at all, then just skip the truncate command.

In your send dump, step 5 is not needed at all, and can be skipped to
speed up the receive procedure.

Thanks,
Qu

> 
> Overall performance improves by 102 percent when sending 790000 small files.
> original: 32m45.311s
> patch: 16m8.387s
> 
> Signed-off-by: Robbie Ko <robbieko@synology.com>
> ---
>  fs/btrfs/send.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
> index 20d3300..7ae2347 100644
> --- a/fs/btrfs/send.c
> +++ b/fs/btrfs/send.c
> @@ -5857,10 +5857,6 @@ static int finish_inode_if_needed(struct send_ctx *sctx, int at_end)
>  					goto out;
>  			}
>  		}
> -		ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen,
> -				sctx->cur_inode_size);
> -		if (ret < 0)
> -			goto out;
>  	}
>  
>  	if (need_chown) {
> @@ -6044,6 +6040,15 @@ static int changed_inode(struct send_ctx *sctx,
>  					sctx->left_path->nodes[0], left_ii);
>  		}
>  	}
> +	if (result == BTRFS_COMPARE_TREE_NEW ||
> +	    result == BTRFS_COMPARE_TREE_CHANGED) {
> +		if (S_ISREG(sctx->cur_inode_mode)) {
> +			ret = send_truncate(sctx, sctx->cur_ino, sctx->cur_inode_gen,
> +						sctx->cur_inode_size);
> +			if (ret < 0)
> +				goto out;
> +		}
> +	}
>  
>  out:
>  	return ret;
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 520 bytes --]