Re: [PATCH v15 3/7] btrfs: add send stream v2 definitions

[Omar Sandoval <osandov@osandov.com>]

On Wed, May 18, 2022 at 11:00:03PM +0200, David Sterba wrote:
> On Mon, Apr 04, 2022 at 10:29:05AM -0700, Omar Sandoval wrote:
> > From: Omar Sandoval <osandov@fb.com>
> > 
> > This adds the definitions of the new commands for send stream version 2
> > and their respective attributes: fallocate, FS_IOC_SETFLAGS (a.k.a.
> > chattr), and encoded writes. It also documents two changes to the send
> > stream format in v2: the receiver shouldn't assume a maximum command
> > size, and the DATA attribute is encoded differently to allow for writes
> > larger than 64k. These will be implemented in subsequent changes, and
> > then the ioctl will accept the new version and flag.
> > 
> > Reviewed-by: Josef Bacik <josef@toxicpanda.com>
> > Reviewed-by: Sweet Tea Dorminy <sweettea-kernel@dorminy.me>
> > Signed-off-by: Omar Sandoval <osandov@fb.com>
> > ---
> >  fs/btrfs/send.c            |  2 +-
> >  fs/btrfs/send.h            | 40 ++++++++++++++++++++++++++++++++++----
> >  include/uapi/linux/btrfs.h |  7 +++++++
> >  3 files changed, 44 insertions(+), 5 deletions(-)
> > 
> > diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
> > index 9363f625fa17..1f141de3a7d6 100644
> > --- a/fs/btrfs/send.c
> > +++ b/fs/btrfs/send.c
> > @@ -7459,7 +7459,7 @@ long btrfs_ioctl_send(struct inode *inode, struct btrfs_ioctl_send_args *arg)
> >  
> >  	sctx->clone_roots_cnt = arg->clone_sources_count;
> >  
> > -	sctx->send_max_size = BTRFS_SEND_BUF_SIZE;
> > +	sctx->send_max_size = BTRFS_SEND_BUF_SIZE_V1;
> >  	sctx->send_buf = kvmalloc(sctx->send_max_size, GFP_KERNEL);
> >  	if (!sctx->send_buf) {
> >  		ret = -ENOMEM;
> > diff --git a/fs/btrfs/send.h b/fs/btrfs/send.h
> > index 67721e0281ba..805d8095209a 100644
> > --- a/fs/btrfs/send.h
> > +++ b/fs/btrfs/send.h
> > @@ -12,7 +12,11 @@
> >  #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream"
> >  #define BTRFS_SEND_STREAM_VERSION 1
> >  
> > -#define BTRFS_SEND_BUF_SIZE SZ_64K
> > +/*
> > + * In send stream v1, no command is larger than 64k. In send stream v2, no limit
> > + * should be assumed.
> > + */
> > +#define BTRFS_SEND_BUF_SIZE_V1 SZ_64K
> >  
> >  enum btrfs_tlv_type {
> >  	BTRFS_TLV_U8,
> > @@ -80,16 +84,20 @@ enum btrfs_send_cmd {
> >  	BTRFS_SEND_C_MAX_V1 = 22,
> >  
> >  	/* Version 2 */
> > -	BTRFS_SEND_C_MAX_V2 = 22,
> > +	BTRFS_SEND_C_FALLOCATE = 23,
> > +	BTRFS_SEND_C_SETFLAGS = 24,
> 
> Do you have patches that implement the fallocate modes and setflags? I
> don't see it in this patchset.

Nope, as discussed before, in order to keep the patch series managable,
this series adds the definitions and receive support for fallocate and
setflags, but leaves the send side to be implemented at a later time.

I implemented fallocate for send back in 2019:
https://github.com/osandov/linux/commits/btrfs-send-v2. It passed some
basic testing back then, but it'd need a big rebase and more testing.

> The setflags should be switched to
> something closer to the recent refactoring that unifies all the
> flags/attrs to fileattr. I have a prototype patch for that, comparing
> the inode flags in the same way as file mode, the tricky part is on the
> receive side how to apply them correctly. On the sending side it's
> simple though.

The way this series documents (and implements in receive)
BTRFS_SEND_C_SETFLAGS is that it's a simple call to FS_IOC_SETFLAGS with
given flags. I don't think this is affected by the change to fileattr,
unless I'm misunderstanding.

This is in line with the other commands being straightforward system
calls, but it does mean that the sending side has to deal with the
complexities of an immutable or append-only file being modified between
incremental sends (by temporarily clearing the flag), and of inherited
flags (e.g., a COW file inside of a NOCOW directory). I suppose it'd
also be possible to have SETFLAGS define the final flags and leave it up
to receive to make that happen by temporarily setting/clearing flags as
necessary, but that is a bit inconsistent with how we've handled other
commands.